The White Hat's Dilemma

https://docs.google.com/presentation/d/1UfOxCIIlcU-iRcUeA6p6fyEE4qUbSuFMqmSuWjRsL_4/pub?start=false&loop=false&delayms=3000

by secalex • 13 years ago

255 101 13 years ago

docs.google.com

Selfcommit• 13 years ago

Who is the finnish guy?

interknot• 13 years ago

Mikko Hypponen, from F-Secure.

http://mikko.hypponen.com/ http://www.f-secure.com/weblog/

dajusu• 13 years ago

What letter was asked to be signed at the end?

secalex5• 13 years ago

The EFF's CFAA reform letter. They had folks in the room to gather paper signatures from attendees, and remote folks will eventually get a chance to sign electronically.

https://www.eff.org/deeplinks/2013/08/letter?utm_source=twit...

gnosis• 13 years ago

Does anyone have a link to a plain-text version of this that doesn't require access to the Google spyware site?

casca2• 13 years ago

Are you asking for this? https://docs.google.com/presentation/d/1UfOxCIIlcU-iRcUeA6p6...

delinka• 13 years ago

I think you missed the whole "...doesn't require access to Google..."

glomph2• 13 years ago

https://mega.co.nz/#!VAkR2bzR!cn7P3VIsAR2ub8O2T6acJCCGgK1vaN...

casca2• 13 years ago

I love the Ultima 4 reference

secalex5• 13 years ago

Thanks. Still my favorite game ever.

neilk2• 13 years ago

What class did you get?

throwaway3902• 13 years ago

I used to work for Blizzard. The Chinese government requested that we modify the WoW client so that they could intercept all chat. As far as I know, no-one said anything, including me - and Blizzard, of course, was more than happy to comply, given the size of the market and the risk of being forbidden to do business there. There were plenty of other MMOGs happy to play ball and eat that cake.

I didn't say anything. It was happening to "them", Chinese nationals. Not only that, but "they" should know better than to say sensitive things online, because even if we didn't install the back door, I reasoned, it wouldn't be too hard to get that data through various other means.

I really regret not only my participation, but not making a big stink about it. No-one did. I strongly suspect that that same system is being being used domestically, now. Clearly it was the wrong thing to do. I've regretted my role in that implementation for several years. I shouldn't have participated, and I should have protested. Even if it didn't stop it, at least the company leadership might have felt the heat. But I was a coward and I didn't want to lose my job, didn't want to fight a legal battle, and, like I said, it was just China spying on it's people, which everyone knew they do anyway.

And who knows? The news probably would have been ignored, or, if it wasn't, I might have been branded as a coward and a disloyal employee, betraying the people who put food on my table. And I being under 30, overpaid, over-priviledged, etc. I can hear the Fox News commentators even now. That, to me, has been the most difficult thing about Snowden, is that here's someone who did the right thing, who revealed wrong-doing on the part of our government, and there are a lot of people who say he's the wrongdoer, who attack him as disloyal and worse. A back door in a game used by China? Who would even care about that? And if they did, I'd just be torn to shreds, unemployable and with heaven-knows-what kind of future.

The reaction to Manning and Snowden, particularly the lack of strong public support, sends a strong signal that people don't want to know. They don't want to upset the apple cart. They don't want to challenge the government, they don't want to question it, not even when it's clearly violating it's own most important rules - the rules that, presumably, we've been fighting to promote these last 200 years. It seems hopeless.

ihsw• 13 years ago

That's what it means to be an ideologue - willingness to die for your values. Don't be ashamed about your decisions, a strong sense of self-preservation is perfectly natural.

kalkin3• 13 years ago

What s/he did can be perfectly natural and not imply that s/he's a worse-than-average person, but still be worthy of shame. I don't mean to be an asshole here, but it's important for all of us to recognize that normal humans (ie. we) have a capacity to do awful things if circumstances encourage it - the Stanford Prison Experiment would be a famous illustration. That's why these slides are helpful.

marquis• 13 years ago

I might have been branded as a coward

If you have an ethical choice to make, consider the reaction of your esteemed colleagues (get counsel if you feel you can), not that of the mainstream news. The general support for those such as Snowden and Manning among my peers would prove that indeed they are held in high regard among those to whom it matters. The decision you made was your decision and you should not curse yourself for it but learn from it and support those who did make decisions to release materials and information.

javajosh2• 13 years ago

I think it would be great if someone with money in Silicon Valley were to make public statements along the lines of, "If you blow the whistle, and it's legit, you won't have to worry about your career. We want people who are driven to do the right thing. We want to hire heroes."

If Google really wants to make a statement about being not-evil, make Snowden a generous offer for employment while he's in Russia.

mr_spothawk4• 13 years ago

Fuck it, I'll go ahead and say it:

First, thanks for coming clean. You're a human being who's made a really bad decision.

You should feel terrible for this. "They" are not merely your possible friends, or your relative's relatives, or your neighbors' cousins... "They" are other human beings who might someday decide to stand up for their human rights. "They" are millions of people you sold out for easy "dinner" & "rent".

And now the tool you helped build to invade their privacy is likely being turned on all your real friends, and your real relatives, and your real neighbors. And you're not even getting paid anymore.

You should feel terrible for selling out for so little. You weren't as afraid of being unemployed as you were of having to do something. You were mentally lazy and your moral compass was clearly defunct.

Too bad your behavior is not undoable and you'll have to live with it for the rest of your life. Maybe you can do something to recover for it in the future.

Hopefully your ethical failure can serve as an example of what not to do.

icebraining• 13 years ago

What did you hope to accomplish with this post? "Here's someone who did something bad, who can't undo it, and who feels bad enough to come clean - let's rub it in so as to discourage others from doing so"?

jwatte• 13 years ago

Selling out for "so little"? I bet he sold out for over hundred thousand dollars, or more (depending on how long he wirked tgere.) The need to know where food and rent are coming from is strong in humans, and the system takes full advantage of it. If you want to go against that, it will take a lot more than some ultimately futile work stoppage and sacrificing your future career. The system is what it is, the golden rule applies, and to change the system, it would take prolonged civil war. I'd that what you want?

() he who has the gold, makes the rule.

ansdkfus13• 13 years ago

Don't we all make terrible choices both explicitly as well as implicitly? When faced with a choice between the moral rectitude and the livelihood of your family, it's very difficult to choose the former. At least this guy has courage to come clean and admit his wrongdoing.

elbear• 13 years ago

I consider your comment very unproductive. You basically berated someone who already came out and said they were sorry for what they did. He realizes his mistake and repents. The fact that you want to make him feel even more guilty doesn't help at all and I find it very sad.

ian00• 13 years ago

How about a program that would encrypt/decrypt game chat for users, since the wow interface has programmability. The act of commenting here and anonymously disclosing this, is a good thing.

lawnchair_larry2• 13 years ago

But much like with the US government spying, China's laws permit them to spy. It was all legal. So what's the big deal? ;)

tluyben22• 13 years ago

I know/hope you are kidding, but I still feel the need to respond.

That whatever is legal or not can still go against your own feel of right or wrong. Who cares if something is legal somewhere and you don't agree with it. There are probably many examples of things legal in countries you visit which your moral compass would prevent you from taking part in. My personal annoyance being bullfighting in Spain (where I live 4 months of the year).

alan_cx• 13 years ago

"Legal" does not equal "ethical" or "moral", in any country.

wslh• 13 years ago

I was asked by government agency to protect (even from other government agencies) specific individuals information that were probable involved in corruption cases. I was vocal and fought against it and was ready to publicize it once finished. At the end the project was cancelled.

So... I don't believe in neutral ethical engineering decisions when the outcomes are crystal clear. I am not talking about ethical decisions on working at Zynga, at the end people can decide if they play their stupid games or not.

peterkelly• 13 years ago

Just wanted to say I really admire you for having the courage to come out and say this publicly (even though it was anonymously).

The world needs more people like you. We all do things we regret later in life. I have great respect for the fact that you've reflected on your involvement in this, come to regret it, and have now chosen to speak out about it.

I think this particular case actually deserves a HN story in itself.

cheapsteak• 13 years ago

If it makes you feel any better, you would not have made a difference.

Having lived there for nearly a decade, including middle and high school, I can tell you that people here just don't care anymore, and news papers and websites would know better than to publish a story like that anyway; you would have self-immolated for nothing.

anonymous• 13 years ago

If it were me, I'd have looked for a way to sneak it into a popup that shows the first time you run the updated client. Something like

    In the interest of your safety and
    the prosperity of the Chinese, all
    your chat messages are henceforth
    constantly monitored by government
    officials. Have a nice day.
    [OK]

It may ultimately turn out to be impossible to do, but I would like to at least try.

freewizard• 13 years ago

Don't feel too bad about what you did to Chinese people. At least game chat is far less a privacy concern for me, compared to forum, email or instant message which have been fully censored in China already.

But you might want to feel sorry about your government who clearly "improved" a lot in the last few years after it learned Yahoo's leak person email to Chinese government.

monotrophic• 13 years ago

This just in: People Viewing Post On Internet Believe Everything They Read. Common Sense Responds, "I Give Up!"... HINT: People who work for Blizzard don't work for every Blizzard office, as Blizzard operates offices in multiple countries... In other news: Apple Inc. Still Operates Sweatshop-Like Conditions As Internet Cares About WoW Instead.

cyanbane• 13 years ago

Great presentation and something that programmers in general (not just infosec) need to have a personal decision model for. Everyone should be able to make their own decision to these questions as they see fit, but the more we talk about issues like this the more we see where other people like us (who maybe were put into this position in the context of "work") have decided on a stance (and the repercussions of said stance) the better off we all are. We who work on machines and not man don't have an oath that we are taught to follow and/or live by, and I don't necessarily think we should. That being said, the Jr. programmer working for a small firm can encounter decisions of ethical importance as much as a black/white/grey/green/mauve hat infosec can. To me, this is the core value of what a site like HN provides and probably the main reason I read the comments on HN more than I do the articles.

scotty79• 13 years ago

I'd say correct answer almost always is to leave quietly. Let's leave doing immoral things to immoral people and let's hope their employers starve due to elevated fees.

Also if you live in US you should always put your own safety in the first place. US justice system becomes most significant threat to capable citizens.

glomph2• 13 years ago

I think it is worth thinking about the idea that whatever your particular moral framework is it should not be about 'making a difference' but making the most effective difference you can. Actually if you hold something to be important you should want to do the most that you can. Exceedingly often what this means is doing something different to the majority of people. Often this goes against conventional wisdom.

tptacek5• 13 years ago

Here's an alternative vantage point, my vantage point, one I think makes these kinds of ethical quandaries easier to navigate:

* I'm not a "white hat" or a "black hat"

* I'm not deliberately involved in any kind of "cyber" conflict

* I don't do what I do because I'm battling the forces of evil, or organized crime, or anything else

Instead: I do engineering. The same way a contract driver developer does, or a Rails dev. I happen to work in a particularly challenging problem domain. My work happens to have some interesting implications. But those implications are not the reason I work in the field; I work here because it allows me to grapple with compilers, number theory, low-level networking, hardware, OS kernels, and every imaginable development platform. It's about the craft.

I find this vantage point, which appears amoral, makes the ethical dilemmas easier to resolve. If a company like Narus asks me to help them make a network monitoring system harder to evade, I don't have to put that request into some ethical framework that considers the good that application might do. I just turn the work down. Same goes for the US Government; no, sorry, not interested.

Total respect for Alex (the "white hat consulting company" he founded is iSec Partners, our sister company and former archrival). I get the sense that Alex engages intentionally with these dilemmas, that he wants to be a part of something larger than himself and, I think, larger than the craft. As a result, sure, he has to live a carefully examined life, and make sure the projects he's working on aren't skewing his compass. I admire him for picking his way through those problems. But I'm every bit as engaged with the field as Alex is, and I'm here to tell you that you don't have to get tangled up in these kinds of ethical problems if you don't want to.

cgag• 13 years ago

What about this vantage point leads you to say no when Narus asks you to help with their spy software or whatever?

dylangs10304• 13 years ago

It's categorical rejection. Instead of opening up that venue and having to navigate the murky waters of cognitive dissonance and moral obligations against lucrative opportunity, you simply don't open the door. You stick to a client base that is meaningful (relatively so) but not highly impactful and potentially dangerous down the road.

clicks4• 13 years ago

Reading what you just wrote reminded me of the famous Edmund Burke quote: "All that is necessary for evil to triumph is for good men to do nothing".

If it had not been for the acquiescence of engineers who took part in the creation of PRISM, XKeyscore, etc. we... well, we would not have PRISM, XKeyscore, etc. Increasingly there is no such thing as an "amoral" position when it comes to a lot of these things -- you're either an entity who willingly chooses profit over principles, or you do something to defeat the evil as you see it (or, at least refuse to take part in it). In this day and age the conscientiousness of man is one of the last remaining defenses to fight the many evils, new or old, mercurial or familiar. It falls on all of us to think of the moral ramifications of our actions, in the workplace and off, and choose carefully and to the extent we comfortably can to see humanity continue prosperously.

I don't mean this to be a thoughtless, idealistic anti-NSA tirade, I'm frankly very okay with folks working on hip new technology that catches the bad guys, I just think your decision framework which is devoid of any ethical considerations is highly, highly dangerous and I wish for the good of us all that it doesn't catch on.

black4eternity• 13 years ago

Probably they were told it was for the greater good, misled into creating a surveillance tool.

mdt• 13 years ago

If it had not been for the acquiescence of engineers who took part in the creation of PRISM, XKeyscore, etc. we... well, we would not have PRISM, XKeyscore, etc.

Oh, come on, man. If you think this is true in the fullest sense, you are not a thinking individual. Mere technological feasibility is 99% of the battle; implementation is the last, most inevitable step.

the famous Edmund Burke quote: "All that is necessary for evil to triumph is for good men to do nothing".

Googler. The quote may be famous, but Burke is not. Like most men with noble fighting words, he had more noble words in him than he did noble fighting.

badman_ting• 13 years ago

I just don't see what this has to do with what tptacek posted. Seems to me he is indeed advocating refusing to take part.

__--__2• 13 years ago

I think your philosophy is the more dangerous one. "Evil" and "Good" are highly subjective and not nearly as straightforward as you're imagining. You see yourself as battling evil. So do the creators of PRISM. Nobody does anything because they think it's bad for the country, or humanity. We are all the heroes of our own story. The greater good has been used all through history to justify horrific acts, and those people genuinely thought they were doing the right thing.

or you do something to defeat the evil as you see it (or, at least refuse to take part in it)

Refusing to take part doesn't absolve you of anything, according to your philosophy. That's exactly what Thomas says he does - refuses to take part. And by refusing to take part, you're still doing nothing to thwart what you see as evil.

qxf2• 13 years ago

I am understanding it differently, may be because after much deliberation I fell into the same position. I interpret the parent comment to mean that the questions posed are very, very relevant and worth grappling with, but knowing the answers is not a pre-requisite to work in the chosen field.

I don't know about the original commenter, but I fell into this position as being the best for me based on, "I am overwhelmed by the number of things that I need to know to make a judgement of good/bad over here, but don't or cannot know. There is too much random chance in my life to figure out how my actions play out. Until I grow wiser, let me do what that chance has laid my way, knowing fully well that I am operating in the dark."

The big, muddying parameters for me to answer 'is what I am doing right?' were:

1. In what context?

2. Over what timeframe?

The larger the context, the longer the timeframe, the more the number of competing principles I had to prioritize, often in inconsistent ways over different aspects of life. In the end, I defaulted to the original commenter's position, with the blind optimism that I would somehow, somewhere in the future get more clarity and wisdom through experience.

tptacek5• 13 years ago

Did you read every other sentence of what I wrote? None of this has anything to do with my comment.

kalkin3• 13 years ago

he wants to be a part of something larger than himself

In fact, you too are part of something larger than yourself and as you work you make decisions that affect it (us) whether or not you think about it. Ignoring an ethical quandary isn't the same as escaping it.

So what ethical stance are you saying you take? You identify as "amoral" and seem to use that to mean "simply self-interested", where self-interest involves doing a craft you enjoy. But then you say there are jobs you wouldn't take for reasons external to the technology. cgag's question is a good one, and I think the contradiction there points to a flaw in the approach of starting out by thinking you can avoid ethical choices. Since you can't really, the only result will be that you make them without thinking them through.

secalex5• 13 years ago

Tom,

I missed you at Defcon for multiple reasons, not the least being the opportunity to get your feedback on the talk as delivered. Maybe we can run a pan-NCC internal conference this fall and see what everybody else is working on. Chicago is nice and central between SF and Manchester.

A big part of the talk was my theory that our industry can no longer claim neutrality; like medicine or law, our actions have become innately entwined with ethical dilemmas that I feel to be better dealt with explicitly and ahead of the moment of decision. I don't think you necessarily disagree, since you lay out two lines you are not willing to cross even if you do not specify your reasoning.

I expect somebody as seasoned and experienced as you can make these decisions subconsciously without violating your basic principles. Younger, less experienced individuals may find this to be a greater challenge and they were the real target of my talk.

In my eyes your actions definitely make you a white hat, even if you avoid the label.

walshemj• 13 years ago

well professional engineers and technicians already have this

in the UK (engineering council)

"All professional engineers and technicians are bound by the Codes of Conduct of their professional engineering institutions. "

and in the US from the NSPE

Engineers, in the fulfillment of their professional duties, shall:

Hold paramount the safety, health, and welfare of the public. Perform services only in areas of their competence. Issue public statements only in an objective and truthful manner. Act for each employer or client as faithful agents or trustees. Avoid deceptive acts. Conduct themselves honorably, responsibly, ethically, and lawfully so as to enhance the honor, reputation, and usefulness of the profession.

blake80862• 13 years ago

What answers would you give to the questions he posed?

How would you hope other people would answer them?

Sven72• 13 years ago

Engineers need to learn the art of building social and political capital. Also patience. Just the finances, scientific mindset and a moral conscience doesn't get you real change. I have seen too many bright people "rant and run" when confronted with morally ambiguous/uncomfortable situations. The only way to deal with this, if you really believe in something, is to stick around and convince others in the group, one by one, over the long term. There are no shortcuts or hacks to this process.

dasil003• 13 years ago

Correct me if I'm wrong, but you're saying the ethically questionable work will not be the most serious from an engineering perspective and thus less interesting for a hard-core security engineer. If that's your position then I think it needs to be fleshed out more than just saying you can amorally and categorically reject contracts from Narus or the US Govt. What is fundamentally uninteresting about their systems? I'm sure a lot of engineers working on PRISM et al found it to be very technically challenging and rewarding work.

dlitz2• 13 years ago

Slide 28. What does "IR" mean?

agnokapathetic• 13 years ago

Incident Response

dlitz2• 13 years ago

Thanks!

Qantourisc• 13 years ago

Why do people even sign NDA's ?

lmartel• 13 years ago

Have you ever worked for a company that didn't ask you to sign one?

That's not rhetorical, I'm curious--it seems that virtually every company has some amount of proprietary information, isn't that what drives competitive advantage in software?

xnxn• 13 years ago

Sure, I've never had to sign one (for any employer).

osivertsson• 13 years ago

I signed them when I was younger, but have been able to avoid them since a couple of years. In my opinion, the law should be enough in most cases.

Recently I was offered a good and well-paying job. Late in the recruitment phase there was this far-reaching NDA I would have to sign. I asked to be given two days to read and evaluate it, which apparently labeled me as someone very strange, I should just flip through it in 10 seconds and sign like everybody else.

Two days later when I questioned their HR/Legal about some of the clauses they got very uncomfortable, and tried to avoid making a big deal out of them, saying it was "standard" and "at this site, they had never taken anyone to court based on it".

I didn't sign, and hence they could no longer offer me the job. Oh well, their loss, they would have to do without the value I could provide to their business.

lmm• 13 years ago

That's not rhetorical, I'm curious--it seems that virtually every company has some amount of proprietary information, isn't that what drives competitive advantage in software?

Very rarely in the software side IME. Software is basically a commodity at this point - sure, there are better and worse ways to write it, but given the requirements and UI, any vaguely competent company could implement it in comparable timeframes. Competitive advantage tends to lie either in a faster turnaround time on making changes (but there's no secret to how you do that - hire good programmers and keep your codebase clean - taking a competitor's code is not going to help with this), or in the non-software side of the business (client relationships etc.)

Phlarp2• 13 years ago

Most freelancers sign an NDA before even being considered for a contract or project. It's a pretty standard part of any employment arrangement in our industry.

Ask the opposite question, as an entrepreneur; why would you even ask someone to sign an NDA.

nonchalance• 13 years ago

I question the slide regarding trade secrets:

* the names are misspelled: first person is Sergey Aleynikov (not alinikov) and second person is Samarth Agarwal (not agrawal)

* in each circumstance, there was actual trade secret theft. That part is clear. The slide itself seems to suggest something beyond that, but they essentially took code that they wrote for their employer (and they signed contracts clearly saying that it belongs to the employers)

secalex5• 13 years ago

Thank you for catching the spelling mistakes.

The point of that slide is that trade secret theft is a very old problem, and that there is a long history of criminal and civil case law to look to when punishing that kind of action. Those individuals were all charged under the Economic Espionage Act and face extreme penalties. I see this as another version of overcharging under the CFAA; the Federal Government has one standard for doing something on paper and a much harsher one for the same activities while using an SVN repo.

I am not defending the actions of those men, I just feel that the civil remedies that have been used for decades are more appropriate than having the soul-destroying power of the US DOJ turned against them on behalf of their employers.

The ethical dilemma exists for the technologist who performs the investigation and testifies against her former co-worker. What responsibility does she have to see justice done? I don't have an answer, but that was the question posed by the slides on justice.

chipsy• 13 years ago

My favored moral framework for most situations is the noblesse oblige: If, by chance or by choice, you have the privilege of affecting a lot of people, you now have the responsibility of supporting the most marginalized members of that group, regardless of whatever prejudice against them you may have had.

This is, in a lot of cases, a nearly impossible obligation to completely fulfill, but in application, it leads to both a closer examination of privilege and to moral decisions and outcomes that are progressive.

Rickasaurus• 13 years ago

Was this talk recorded?

treenyc• 13 years ago

Thank you ALex for bring up these issues. I just would like to point out that ethics and morality are both normative propositions (in the sense that they are different cross culture and society). Basically what is consider desirable vs. undesirable behavior. As we all must have found out by now, what is consider desirable and undesirable that very different from place to place.

It would perhaps to be more constructive to consider a positive model of integrity (Positive as in positive theory in economics). In many ways we have confused morality and ethics with integrity. Integrity when distinguished in the positive model it can be apply consistently across culture, societies, groups or organization (kind like the law of gravity).

For those who are interested, you download the short paper by Dr. Mike Jensen on social science research network related to positive model of integrity:

http://ssrn.com/abstract=1511274

The White Hat's Dilemma

Related Stories