Lavabit founder turned over the private SSL keys as an 11 page printout

http://thenextweb.com/us/2013/10/03/you-wont-believe-what-email-provider-lavabit-did-to-avoid-giving-the-us-government-its-data/

by lukashed • 13 years ago

240 179 13 years ago

thenextweb.com

stedaniels2• 13 years ago

Should have put a typo in there. Can you imagine the double/tripple/etc checking going on!?

devx2• 13 years ago

Going 7 pages into it: "Wait...which line was I at again?!"

chris_wot• 13 years ago

They would use OCR.

velik_m• 13 years ago

Or use one of those Xerox copiers that randomly change numbers...

fnordfnordfnord5• 13 years ago

and a funky font that is difficult to OCR.

mseebach• 13 years ago

This is what happens when you try to "hack" the legal system as if it was a simple, deterministic computer program (it comes up here frequently as the "FBI wasn't here"-sign-in-library hack).

When a judge orders you to do something, you have to comply with the substance of the order, not evade it by sort-of technically complying while actually avoiding it.

I have all the respect in the world for Levison, he did the right thing and it takes guts. But trying to "hack" a judge's order is just silly.

devx2• 13 years ago

I don't know. It might've been worth a shot to send it in PDF format the second time, after he asked to deliver it digitally, just as a sign of protest. Didn't Twitter refuse to comply with a judge's order once, too?

contextual• 13 years ago

Levison's response was a courageous form of protest. Your response is smug and naive. You have no clue what he went through.

It's easy to call the right game plays from your armchair.

rjknight2• 13 years ago

I disagree. If the judge wanted an electronic copy then he or she could ask for that, which is indeed what the judge later did. Levison would have known that his initial 'compliance' wouldn't make the problem go away, but it did buy him some time while the judge figured out what to do next (or the FBI figured out that they needed to be more precise in what they were asking for). He didn't refuse to comply at any point - he complied with the original request, at which point the judge gave him fresh instructions and ordered him to comply with those or face a fine.

Consider it from another perspective - people often request data from governments or large corporations and, when ordered or pressured into providing it they often do so in the least helpful format available. They are rarely punished for this because the law doesn't really have a position on whether CSV is a better document format than PDFs of scanned printouts, or whether a 50-page printout is more useful than a CD-ROM or USB stick. Why should the rules be any different for someone like Levison?

gedrap• 13 years ago

>He didn't refuse to comply at any point - he complied with the original request

Just like the parent said, while he kind of technically complied. But, again, it's more than obvious to everyone that it's not what was expected/demanded from him.

>Why should the rules be any different for someone like Levison?

Because it's related to Snowden. Again, it's quite obvious to everyone, that it's a hot issue and the government might go far to solve it.

Like it was said above, while I really support and respect Levison, it's not something you should be trying to fool - it's not a computer program...

cdvonstinkpot• 13 years ago

He should have delivered the digital versions in punchcards.

shabble2• 13 years ago

There are specific terms in the GPL[1], and probably elsewhere, that address this issue:

[3(a)] Accompany it with the complete corresponding machine-readable source code, which must be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange;

It's still open to some interpretation, such as exactly what constitutes "customary media", but probably does rule out the printed on the side of a cow[2] approach taken here.

[1] from v2: https://www.gnu.org/licenses/old-licenses/gpl-2.0.html

[2] http://www.snopes.com/business/bank/cowcheck.asp

specialist2• 13 years ago

he complied with the original request

Malicious compliance.

Having tried to get useful answers via FOIA requests, I know first hand that playing stupid is standard procedure.

Tit for tat.

If the judge wanted an electronic copy...

Yup. The persons making the policies and rulings often don't understand the technology.

Like Levison, I'm more likely than not to do something spiteful, just to rub their noses in their ignorance.

DanBC• 13 years ago

I have all the respect in the world for Levison, he did the right thing and it takes guts. But trying to "hack" a judge's order is just silly.

Perhaps he knew it was silly but just wanted something to show he's trying to resist the order. It's been a very popular move with people, and probably caused people to remember who he is, increasing customers for his next business.

It's a shame he's still got so little in the legal defence fund. $20,000 is nothing for a legal case.

aric• 13 years ago

Conscientious objections aren't silly. He's risking further abuse.

It takes guts, just as you say. It takes guts specifically because he's unwilling to gracefully comply. "Monkey-wrenching" usually appears silly to people who are defeatist or subservient to a particular order/agenda. I'm not saying you are. I appreciate your comment otherwise. I'm only suggesting that you reconsider your view on his tactic, if only a delaying tactic, as an act of courage rather than silliness.

squidi• 13 years ago

Civil disobedience can be a wonderful thing. If the Civil Rights movement followed every judge's instruction they would not have achieved what they did.

msy• 13 years ago

There's plenty of examples of governments forced to comply with open data/FOIA doing precisely this. You can definitely hack the legal system, you just need to understand it fully and properly first, which tends to be where people, particularly hackers with high opinions of their own abilities fall down.

fnordfnordfnord5• 13 years ago

When a judge orders you to do something, you have to comply with the substance of the order, not evade it by sort-of technically complying while actually avoiding it.

Attorneys do this kind of crap all of the time. Often, when ordered to provide docs for discovery, they will print pdfs and re-scan them in (sometimes over and over again, to make them barely legible) to make opposing counsel's task that much harder. The judge in this case may not have had much appreciation for Levison's wit, but it may not be as silly as you think. Perhaps Levison knew he was at the end-game and merely needed a brief delay to tidy up some loose end, in which case his tactic worked well.

kabdib• 13 years ago

Tit for tat -- you ask the government for information, and it comes back all redacted.

MagicWishMonkey2• 13 years ago

He didn't hack anything, this was a simple act of civil disobedience. Nothing more.

ajays• 13 years ago

I disagree. "Hacking" the system by creatively interpreting the laws is what lawyers do! It's their job to hack the legal code to their clients' benefit.

mathattack• 13 years ago

It's kind of like paying a fine in pennies.

Couldn't they just scan the document?

darkarmani2• 13 years ago

This is what happens when you try to "hack" the legal system

What happens? The prosecution has to go back to the judge and demand a different format? What a terrible consequence.

ihsw• 13 years ago

You're implying malicious intent, but he may have simply wanted to buy some time. The number of options to defer punitive measures will have been very limited and this one seems pretty straight-forward.

njharman• 13 years ago

I view it as form of protest / civil disobedience. And of course PR.

noonespecial• 13 years ago

Hacking the legal system in exactly this spirit is fully 1/3 of a common lawyer's daily job.

neves• 13 years ago

Google is doing it in Brazil. They say it isn't necessary to comply to Brazilian laws because their servers aren't located in Brazil. BTW, it is a lie.

See: ping google.com PING google.com (201.17.31.88) 56(84) bytes of data. 64 bytes from c9111f58.virtua.com.br (201.17.31.88): icmp_req=1 ttl=60 time=11.8 ms 64 bytes from c9111f58.virtua.com.br (201.17.31.88): icmp_req=2 ttl=60 time=12.9 ms 64 bytes from c9111f58.virtua.com.br (201.17.31.88): icmp_req=3 ttl=60 time=11.5 ms 64 bytes from c9111f58.virtua.com.br (201.17.31.88): icmp_req=4 ttl=60 time=13.1 ms

I'm in Rio de Janeiro. In 6ms travelling at the speed of ligth, I'd travel 1800km. Florida (the nearest American state) is more than 7000km far from my city.

xuhu• 13 years ago

offtopic: scrolling through the printout by dragging the scrollbar handle slowly (~3 pages per sec) crashes the chrome renderer (tried on 2 win7-32 boxes)

stedaniels2• 13 years ago

Yeah, I got that too. Win 7 - 64bit. Chrome 29

kevin-brown• 13 years ago

On Ubuntu x64, it's sluggish (as it's loading) but doesn't crash. It took up a solid 1GB of memory though, according to Chrome.

rdl10• 13 years ago

The irony would be using the same software/format for this as was used to export the PGP Source Code legally in the 1900s.

troels• 13 years ago

Upvoted for 1900s. That really makes me feel old.

Osmium• 13 years ago

What format was that? Edit: Thanks!

rdl10• 13 years ago

http://www.amazon.com/PGP-Internals-Philip-R-Zimmermann/dp/0...

http://www.pgpi.org/pgpi/project/scanning/

keeperofdakeys• 13 years ago

http://www.pgpi.org/pgpi/project/scanning/

Daniel_Newby• 13 years ago

Perfect forward secrecy, bitches.

lucian1900• 13 years ago

That does nothing for future intercepts.

jlgreco2• 13 years ago

That is what killing your business is for.

ig13• 13 years ago

Don't do this. If it's a clean print-out it's trivial to OCR at a five nines accuracy rate.

dalore2• 13 years ago

You would 100% accuracy though. One char difference would render the key useless.

bcoates• 13 years ago

Not really, you've leaked so much of the key that doing exhaustive search of possible OCR errors becomes trivial.

tomaac• 13 years ago

Depending on font he used, I believe for OCR some characters might be difficult:

o01|IO

And even one wrong character makes SSL key wrong.

ig13• 13 years ago

There's OCR systems which work on the basis of internal font consistency. They break the page into a series of single character images, and because the same character repeated is close to identical it's trivial to match them up, so you can easily build a map of characters.

You then just need a human to label each character once. With a pixel image comparison 0 looks completely different from o.

If they're using a standard font then a regular OCR (you'd only need four nines accuracy to get it 100% correct) would be fine, even with a weird font it still be easy to get that level of accuracy.

lambda• 13 years ago

Doesn't matter. A few wrong characters can easily be brute-forced. Once you have enough of the characters, you can just write a program to try modifying a few of them until you get a key that works.

option_greek• 13 years ago

Even at 4 point font (as mentioned else where) ?

ig13• 13 years ago

4pt on a 600dpi printer is about 35 dots per character, should be more than enough to get a clean read.

aridiculous• 13 years ago

Somehow I think the founder of an ultra-secure email provider knows this.

speeq• 13 years ago

"At approximately 1:30p.m. eDT on August 2, 2013, Mr. Levison gave the FBI a printout of what he represented to be the encryption keys needed to operate the penregister. This printout, in what appears to be 4-point type, consists of 11 pages of largely illegible characters.

Moreover, each of the five encryption keys contains 512 individual characters - or a total of 2560 characters. To make use of these keys, the FBI would have to manually input all 2560 characters, and one incorrect keystroke in this laborious process would render the FBI collection system incapable of collecting decrypted data."

drdaeman• 13 years ago

To make use of these keys, the FBI would have to manually input all 2560 characters

They would probably have to OCR the thing.

johnchristopher• 13 years ago

What are interns for ?

Or are they "non-essential" employees ?

Colliwinks• 13 years ago

Should have printed it out as images via ReCaptcha

archangel_one2• 13 years ago

2560 characters isn't that many. You'd have to proofread the OCR'd version anyway, I'd say it'd be easier to just type it in.

MagicWishMonkey2• 13 years ago

Nope. Ladar intentionally used an obscure font (some variant of Helvetica with extremely narrow kerning) to make OCR difficult.

Obviously it would be possible to apply OCR, but it would take a few days/weeks to train the algorithm. An out of the box solution would not work.

deanclatworthy• 13 years ago

Presumably they could quite easily OCR it.

StavrosK2• 13 years ago

Well, with a 4pt font, accuracy would probably suffer greatly.

tiatia2• 13 years ago

OCR? Nearly impossible.

OCR only works decent in combination with a dictionary.

desas4• 13 years ago

It's in the court documents. They did OCR it, to present as an exhibit. It was illegible, they also stated that the original was nearly as illegible.

dalore2• 13 years ago

Should have given the next lot with 1 char per page in a huge font (so it's legible), but then "accidentally" drop it and change the order of pages.

archangel_one2• 13 years ago

As funny as that might be, there is such a thing as contempt of court which I guess he'd find less so.

rmc• 13 years ago

Sure, it'd be hilarious. Until the court says "You didn't do what you were ordered to. Either do it, or be fined".

RBerenguel• 13 years ago

I would have kept sending printouts with ever so slightly larger fonts (i.e. 10 steps from 4 point to 5 points.)

Terretta2• 13 years ago

To make use of these keys, the FBI would have to manually input all 2560 characters...

My reaction to that was "oh boo hoo".

When people my age were kids (get off my lawn) we used to type pages of raw hexadecimal from the back of magazines into a machine prompt. We didn't cry about it, we were just careful.

I can't find a hexadecimal example at the moment, but look at some of these TRS-80 programs (pp. 110-111, 143) which have multiple pages of data/digits to transcribe:

http://archive.org/stream/80-programs-for-the-trs-80-1979jim...

Btw, the originals were often bad photocopies. So maybe not 4 point font, but certainly ambiguity in some bits.

16s• 13 years ago

And also... C64 programs. They were manually typed in by 10 year old kids. The programs were printed in the computer magazines (early 80s) that dealt specifically with C64's. These programs were far more than 2560 characters (in some cases).

If young, interested kids can do it, I'd hope that a nation state could figure it out.

Link: http://upload.wikimedia.org/wikipedia/en/a/a5/ComputesGazett...

mpyne4• 13 years ago

My reaction to that was "oh boo hoo".

Yeah, in the grand scheme of things it's easier to get a few people to type that in (it's parallelizable, after all) than to wait for another court order. Though I'd still have brought it to the judge's attention as this is like the dictionary definition of "contempt of court". If someone tried to be GPL-compliant in this fashion they'd be laughed off the mailing list.

ctdonath• 13 years ago

Creative Computing magazine once published the entire source code for the game Adventure [1] ... on one page. (I still have that issue.)

[1] Great stuff at the time; precursor to Zork.

ameoba• 13 years ago

I remember them having checksums to ensure line integrity.

tlarkworthy• 13 years ago

You get three people to independently input the codes and take a vote on the characters (or inspect the disagreements)

Voilà, information is transported reliably over a noisey medium! error correcting codes in practice 101

EDIT: voilà typo fixed :p

kerneis3• 13 years ago

It's "voilà", not "viola". Both of them are French words, but the latter means "raped" (probably not what you meant).

kerneis3• 13 years ago

I don't get it. 2560 characters in 4pt should fit in far less than 11 pages…

drharris2• 13 years ago

They were to turn over the keys and all information required to decrypt everything. Presumably, they wrote a LOT of 4pt boilerplate on how to use the keys for decryption.

vacri• 13 years ago

Single sided, double-spaced, abstract, appendices, title page, dedication page, headers, footers... still doesn't seem to be enough. Maybe the numbers were in Roman numerals?

r0h1n• 13 years ago

Even cooler would have been Lavabit scanning and photocopying the 2650-character printouts on Xerox machines :)

https://news.ycombinator.com/item?id=6156238

csmatt• 13 years ago

hahaha. Thanks for making me smile this morning :)

sigkill2• 13 years ago

It's not a big deal though. They subsequently specifically asked him to put it on a CD in PEM format and deliver it by 5 that day or be faced $5000 fine per day or something like that.

jessaustin9• 13 years ago

...put it on a CD...

There should be a standard virus-laden ISO image specifically designed for "compliance" with court orders.

chaz• 13 years ago

4 people can get this data entry done in an hour. One person reads out loud while the other person types and confirms each character. The second team does exactly the same work enabling a diff on the completed work to quickly find errors once everything is done. One character per second isn't unreasonable, so that's less than an hour. Or the same team does it twice in 2 hours.

samstave• 13 years ago

Speak it out loud to Siri!

kbenson• 13 years ago

I suspect the real reason it was done was so a few "typos" could be slipped in and blamed on the poor copy. A few 1's and 7's swapped would do it.

brownbat• 13 years ago

To make use of these keys, the FBI would have to manually input...

Has the FBI never heard of OCR, or were these printed as CAPTCHAs?

mbq2• 13 years ago

So he did surrender the keys -- this makes his business shutdown move substantially less heroic that everyone thought.

faceplanter• 13 years ago

It was ordered by a judge. What do you suggest he would do otherwise?

markeganfuller• 13 years ago

Delete them before he was ordered to hand them over?

mbq2• 13 years ago

I'm only pointing out that this _may_ mean that the archives were compromised regardless of the fact he heroically shut down the whole thing.

throwwit• 13 years ago

lol... remember this?: http://www.theguardian.com/technology/blog/2012/aug/29/apple...

jlebrech• 13 years ago

shoulda put it on floppy disks, or minidisks

rdl10• 13 years ago

I really don't get why dude didn't respond to the requests for information with a totally straightfaced, and factual, "we do not have the technical capability to recover messages for a user using the encryption software". Not "we could possibly defeat our own system, but we don't want to." Just the "we do not have the technical capability to do that, but we're happy to turn over any other information which we can provide, pursuant to lawful order or warrant." Be polite, totally responsive, completely honest, but don't go into the "yes, we could possibly build something which would possibly do what you want, but we don't want to."

Terretta2• 13 years ago

Some of us don't like to lie.

rdl10• 13 years ago

It would be absolutely factual. There's a difference between "we could build x, given months of engineering effort, possibly including outside experts" and "we can do x in response to a court order in our current system, on demand."

There is no prior requirement for an email provider to be CALEA compliant, so there doesn't need to be a particular LI function built into the system.

jasonkeene• 13 years ago

You don't get it. Emails were encrypted at rest with the user's password. This was publicly disclosed by lavabit on their site. With the SSL key material the state could decrypt the user's password from network traffic. The encrypted emails and SSL key material are obtained through the courts, SSL encrypted passwords via surveillance, bob's your uncle they can read a user's email. Lavabit wouldn't need to "build" anything and couldn't argue they didn't have the technical capability to turn over SSL keys and encrypted email data. This is why he shut down, so that user's wouldn't continue to submit their passwords over the wire using a compromised SSL key.

rdl10• 13 years ago

The only reason they got the SSL key search warrant was his aggressive noncompliance with the pen register/pen trap. I didn't realize pen register applied to non-CALEA entities (this is a PATRIOT 216 thing); if it didn't, there would have been no justification for 1) forcing his cooperation and 2) getting SSL keys from him.

With Patriot 216 pen trap, they can compel full cooperation to the same standard as a CALEA covered entity, which they knew he couldn't provide as well as their own pen trap device (at least without work; they found $2k unreasonable to implement it!), so they can get a warrant for SSL keys for their pen trap. With that they can do whatever.

There may be a solution in SSL keys which can't be exported (HSMs) into the pen trap; you'd potentially be able to require a secret compatriot offshore (or via a cutout) to assist in adding a new load balancer or front end, so you'd be technically unable to comply. They could require you generate and use new keys, but users could detect that, and you could warn of this when you first set up the system.

zenocon• 13 years ago

He should have printed one character per page, and put it in a 2,560 page 3-ring binder.

aw3c2• 13 years ago

* unnumbered

ikusalic• 13 years ago

Even if futile, beautiful example of civil disobedience. If only there were more people that at least tried as he did. I'm supporting his rally[1] and hope more people will.

[1] https://rally.org/lavabit

ballard2• 13 years ago

Passive aggressive ftw. I would have used an excessively decorative script font and not number the pages or identify which keys. Perhaps even a typo or two, that'd buy some time.

On the size, 4 pt is generous. I remember printing an allowed note index card for high school final exams in 1 pt Times New Roman on a laser printer @ 600 dpi. At arm's length, it looked like a series of lines, but up-close the text was quite readable. Under magnification, the print quality was amazing.

Ygg2• 13 years ago

Write everything in 4pt Frustro font. Headaches for everybody!!

jlgreco2• 13 years ago

He should have used "wingdings".

ballard2• 13 years ago

I like your thinking.

wglb• 13 years ago

Sounds like an act of malicious obedience.

jonahx• 13 years ago

"In an attempt at dry humor"

I think that is a little better than an "attempt"...

eli3• 13 years ago

Sorry, but I'm not a fan. Either comply with the order or don't, but don't play games.

This is the same nonsense the government pulls when they want to "technically" comply with a FOIA request, but want to make life as hard as possible for the requester.

mpyne4• 13 years ago

Apparently double standards are wrong when the government does it, but are OK when "we" do it.

What I'm not a fan of, however, is that Levison had claimed that he's be willing to comply with (indeed, has complied with) specific warrants. He left the impression that the FBI was asking him to ruin his security in general.

However the court filings show that the request for the SSL priv keys came only after Levison failed to comply with... a specific warrant relating to Snowden and Snowden alone. So if Levison wasn't lying, he was definitely leaving something major out of his story.

eli3• 13 years ago

Agreed. Though considering there are gag orders in play here, I'm willing to give people the benefit of the doubt.

anaphor2• 13 years ago

He was probably still working on a way to compromise Snowden while leaving all of the users still secure, and the FBI was probably just impatient. But that's pure speculation on my part.

debacle• 13 years ago

This man is a hero. I don't know if he's a saint (he's probably not - no one is), but he's definitely a hero.

bhauer• 13 years ago

FYI, I would guess that Attachment A starting on page 144 is a reproduction of the keys in a very small font. If that is correct, they are indeed illegible.

drucken• 13 years ago

Isn't the key this sentence: "Wired reported the government as complaining that a Lavabit representative indicated that Lavabit had the technical capability to decrypt the information, but that Lavabit did not want to ‘defeat [its] own system."?

If they used something like Perfect Forward Secrecy, could they not legitimately have claimed to be unable to defeat their own system (as it is currently configured)?

jloughry2• 13 years ago

THE POLICE HAVE REQUESTED ALL FILES UNDER DIRECTORY TITLE RAND.

"Dump it for them at 300 baud."

"Art! Are you sure?"

"We cleaned Rand's directory first thing. Took out everything not routine, then we added a few files. Old engineering catalogs. Maintenance schedules. Ratings of TV shows. Makes a pretty big file---" MILLIE, what is the total stored in Rand's directory?

23,567,892 BYTES

"Good Lord. Art, that will take hours to print out---"

"Yeah, that gives the cops a hobby." [1]

[1] Larry Niven and Jerry Pournelle. Oath of Fealty. New York: Pocket Books, 1981. ISBN 0-671-82802-9, pp. 287--8.

Fuxy• 13 years ago

Technically speaking he did comply with the order. They never specified what format it should be in.

Plus companies shouldn't be required to beautify data for the government. They got what they asked for it-s up to them to figure out how to use it.

I know this is a bit sneaky this time but if they allow this to stand then soon companies will have to employ full time staff to beatify data for the government with pretty graphs and such because a database dump is incomprehensible.

It's like a kid asking you how to make a website and then complaining that HTML, CSS, JavaScript,etc is incomprehensible.

wellboy• 13 years ago

But you can simply scan 11 pages in and run a software over it that reads all characters. There´s lots of software like that.

This would have taken the FBI 20 mins...

lectrick• 13 years ago

Smarter would have been to copyright the keys so that giving them a copy would violate DMCA.

Take THAT!

joelrunyon• 13 years ago

The worst part about this is that it's not even that hard to fix. If you wanted to - you could hire a VA for $5/hour to digitize it (and hire another to QA). Total cost probably less than $100.

So intrusive AND lazy :).

Lavabit founder turned over the private SSL keys as an 11 page printout

Related Stories