How I compiled TrueCrypt 7.1a for Win32 and matched the official binaries

https://madiba.encs.concordia.ca/~x_decarn/truecrypt-binaries-analysis/

by maqr • 13 years ago

346 105 13 years ago

madiba.encs.concordia.ca

generalpf• 13 years ago

That's amazing work. Well done to the author.

junto• 13 years ago

Came here to say this too. It is this kind of in depth investigative work, that has genuinely taken time and smarts to achieve, is what I come to HN to enjoy reading. All credit to the author.

yeukhon2• 13 years ago

"TrueCrypt is a project that doesn't provide deterministic builds."

Why? What is the benefit of doing so when everyone wants a deterministic build?

dublinben2• 13 years ago

They also use a unique license that is incompatible with most other open source projects. They seem to have strong reasons for doing things the way they do, and aren't fond of explaining them.

im3w1l2• 13 years ago

and aren't fond of explaining them.

Who tried to poke them how, and what happened?

aroch• 13 years ago

Deterministic builds are hard...really hard. The combined power of the Debian community has trouble getting deterministic builds, as does the Tor project.

yeukhon2• 13 years ago

Oh! I thought that was the norm. Author made it sounded like TrueCrypt doesn't and others do the way he said it.

sliverstorm5• 13 years ago

When I think of deterministic builds, I think of the ROMs on the old GameBoy. Apparently some of the ROMs had whatever empty space was left in the fixed-size image padded with random data pulled straight out of the build host's memory. As if consistent tool versions weren't already enough of a challenge!

gngeal• 13 years ago

Deterministic builds are hard...really hard.

I'd have thought that deterministic builds are really simple unless your toolkit ecosystem is FUBAR. After all, a compiler is a simple function from input to output (unless the FUBAR ecosystem syndrome arises, as I said).

fidz• 13 years ago

I don't understand, what is the problem if we have different compiled binary?

Lagged2Death• 13 years ago

I wish the term "reproducible builds" were the more popular one. I think that's more accurate.

3825• 13 years ago

For what it is worth, that is what Debian[0] seems to call it as well.

[0] https://wiki.debian.org/ReproducibleBuilds

eterm• 13 years ago

Tldr: Binaries didn't match, here's some handwaving at the differences.

e40• 13 years ago

I think that's inaccurate. The disassembly of them matched perfectly.

PeterisP3• 13 years ago

Well then, what and why are the differences? I mean, if there's an arbitrary data block somewhere, then the "matching disassembly" can have wildly different behavior by simply copying & executing parts of that block.

zokier• 13 years ago

Just a slightly off-topic question, but WTF does TC require VC 1.52 for?

sliverstorm5• 13 years ago

Considering they document their build process very carefully, I can only imagine they wound up standardizing on that library a long time ago and keep using it to this day.

acqq5• 13 years ago

Yes, even if there is any recent open source alternative to build 16-bit code, selecting the 20 years old compiler binaries (http://en.wikipedia.org/wiki/Visual_C++) for which it can be easiest to be sure they haven't changed (there's still enough CD's around produced at that time, I believe I still have one too) is a damn good decision.

twistedpair2• 13 years ago

Hmm... 20 year older binaries? Anyone ever read "Reflections on Trusting Trust"? Perhaps they don't trust newer code from MS? Think about it, MS can't backdoor an old binary.

bri3d• 13 years ago

It's the easiest way to build C or C++ targeting 16-bit real mode with a Windows host/toolchain, and they use it to compile a boot loader.

trentmb• 13 years ago

It's the last version that can produce 16-bit code?

CUViper3• 13 years ago

TrueCrypt is not backdoored in a way that is not visible from the sources

... as long as you also trust the compiler not to introduce any backdoor... (cf. Reflections on Trusting Trust)

dmix• 13 years ago

Oh god, not this conversation again.

CUViper3• 13 years ago

Why not? If you're being so paranoid about the origin of a binary, you have to at least acknowledge the fact that you're trusting the compiler in making this comparison.

So let me throw out an idea that might help justify this trust too. Compile the same TrueCrypt sources with a totally different compiler, then use both binaries in a deterministic way and compare the raw encrypted result. (I'm assuming here that the same encryption keys and data will give the same result, but don't know for sure if that's true.)

antocv2• 13 years ago

Im with you.

Lets all just stop right here. No, dont mention that cross-compile with different compilers and compare. Just everyone stop.

twistedpair2• 13 years ago

You know that at a minimum, some manager at the NSA/CIA read that seminal paper and salivated at the prospect of a compromised compiler. Whether they are other there or not, I'm certain millions have been spent attempting it.

sown• 13 years ago

It's not crazy talk. :)

I seem to remember retrieving from a BBS way back when, an MS-DOS shareware pascal or C compiler of some kind that would leave behind a serial number foot print in executables that the author said he could use to prove that an unregistered version of his product was used. I wish I could remember it now, though

tedunangst4• 13 years ago

Dropping a 4-16 byte identifier string into an unused portion of a binary is world's away from reading source code, solving the halting problem to determine exactly which part you want to backdoor, and then outputting a backdoor binary. Shit, gcc embeds its version string in every object file, that doesn't mean it's a trap.

schoen• 13 years ago

Oh yeah! I remember this too.

I think we're thinking of the A86 shareware assembler:

https://en.wikipedia.org/wiki/A86_%28software%29

wai1234• 13 years ago

This is a great first step but we're not done yet. It proves the binaries are built from the published code, but only when the published code has been thoroughly vetted can we conclude there is no backdoor.

edwintorok4• 13 years ago

Also the build is not deterministic yet, even on his own machine he got some differences in truecrypt.sys between successive builds. By definition the build is not (yet) fully deterministic then:

"Using the same source and same project directory results in the same pattern of difference in the block starting at 0002CBAC, as the pattern shown between my build from the correct project directory and the original file. This means that this difference is a normal result of the compilation process, and can be considered harmless from our point of view"

tripzilch2• 13 years ago

Indeed, that part itches me a little as well.

I share the author's belief it's probably benign, and on the assumption that the source code contains no crazy obfuscated magic, it'd be pretty hard to actually hide malicious code or behaviour there.

Still if they want to be thorough (and they should), I'd have liked a bit more explanation than waving it away with a "well these bytes seem to change all the time, so that's okay then". That's not really a justification, it's just an explanation. And it's not okay.

0x02• 13 years ago

The disassembly for both versions of the file was a 100% match, though, which is a pretty good indicator that the binary difference must be something unimportant and not related to the actual code.

lallysingh• 13 years ago

It seems like if you can have the tools alter the embedded timestamp (even as a post-processing step), you can match the binaries pretty closely.

The last bit is the signature, which you can't duplicate, but you can also just take out, as it's not code. But, if you're paranoid about that, zero-out or remove the signature after verifying it.

nilved4• 13 years ago

Why are people so adamant about auditing TrueCrypt instead of just migrating to an alternative?

josephlord• 13 years ago

TrueCrypt is highly trusted, they are verifying that trust is well placed. What alternative were you proposing?

dublinben2• 13 years ago

It's important for Windows users to have access to trusted encryption programs.

bliker• 13 years ago

I am just shooting into darkness, but would not it be easier to compile it twice and diff outcomes to find found out what parts are being changed so those can be ruled out?

pionar• 13 years ago

He did that further down in the article, when he noticed that file path was encoded in the TrueCrypt.sys file.

pamparosendo• 13 years ago

I entered just to say it's an incredible work done by this guy... it's been years since I analized a file on hex mode (from Norton Commander, jeje).

proctor2• 13 years ago

it seems to me that the relaxed gpg key verification that the author uses doesn't give us any more assurances regarding the authenticity of the source than a simple hash offered on the website would. i think in this situation, if the author did not intend to attempt more rigorous verification of the truecrypt pgp key, at least cross-checking that the key offered on the site matches the key offered on a public key server pgp.mit.edu for example would be prudent before signing the truecrypt key with your own.

  Import the .asc file in the keyring (File > Import certificates).
  Now you should mark the key as trusted: right click on the TrueCrypt Foundation public key 
  in the list under Imported Certificate tab > Change Owner Trust, and set it as I believe checks are casual.
  You should also generate your own key pair to sign this key in order to show you really trust 
  it and get a nice confirmation when verifying the binary.

acqq5• 13 years ago

You can do yours checks and compare with the author's! It's a very fast thing to do for the checks you mention. The more people repeat it, the better.

proctor2• 13 years ago

i think there are two concerns here. one is that the source is not tainted by a third party during or before the download. the second (arguably much more important in this case) concern is that the compiled binary matches the source. the second concern is addressed well by the author as far as i can tell, but i think that there is room for improvement in their concerns about the former. i assume they have thought about this and do have at least some concerns because it is mentioned that

  The PGP signature of the binary can be downloaded 
  through the button PGP Signature, which makes you 
  download TrueCrypt Setup 7.1a.exe.sig over HTTPS 
  (*although with the NSA in the middle, it might not 
  mean much*).

[emphasis mine]

cross-referencing the pgp signature with at least one other (public) source would go a long way toward allaying those concerns (that the HTTPS might not mean much).

this criticism is in no way meant to detract from the rest of the work, and i mean only to refer to pgp sig verification best practices here.

smegel• 13 years ago

Kudos for effort.

xbeta• 13 years ago

Coolest post I've read today! Good work!

TheRealWatson• 13 years ago

Please God, don't let the author be working for the NSA. These days I get suspicious at every "it's all good" piece of news.

pilif• 13 years ago

The good thing about his analysis: It has all the information you need to reproduce it and form your own conclusion. So even if he was working for the NSA, by following his steps, you will either come to the same result or not.

That's how science works.

pointernil• 13 years ago

I get the point reg. verifying the Windows-Compiling-Build, but wouldn't the same verification on an open source platform allow for even easier (maybe even automatic) verification?

How about an vmware/vbox image setup explicitly for that purpose? Not feasible for windows due to licencing issues, i guess.

Also, huge kudos for the effort going into this work. Thanks!

How I compiled TrueCrypt 7.1a for Win32 and matched the official binaries

Related Stories