Debian reproducibility statistics

https://tests.reproducible-builds.org/debian/reproducible.html

by lamby • 9 years ago

188 70 9 years ago

bialpio• 9 years ago

Can someone shed some light on what exactly "reproducible" means in this context?

dozzie2• 9 years ago

It's a term for binaries (usually ELF) being byte-to-byte equal in two different runs of a compiler. This way you can build a binary package from source package and if its content is the same, you know what source code was used to build the package, and then you can e.g. inspect the code for backdoors or build debug symbols without planning for that beforehand.

https://en.wikipedia.org/wiki/Reproducible_build

gnasr• 9 years ago

And why Debian is not reproducible?

0xcde4c3db2• 9 years ago

Debian seems to only considers a build reproducible if the entire .deb file is byte-for-byte identical, not just the ELF files. A bunch of the packages they're tracking are flagged for documentation files such as PDFs not coming out identical.

penpapersw2• 9 years ago

More details at https://reproducible-builds.org/ which is linked to at the bottom of OP's page.

agumonkey• 9 years ago

What a nice progression https://tests.reproducible-builds.org/debian/stats_issues.pn...

Project started in 2014 it seems.

Mchl• 9 years ago

I'd invest in that

tome• 9 years ago

But would you have been able to hold your nerve through May?

hmmm___food• 9 years ago

Maybe im missing something but it seems the justification behind this is based on a situation in which source wasnt open. Debian is open so why is reproducibility a priority?

kalmi10• 9 years ago

So that anyone can check whether a binary (on the mirrors) has been tampered with in ways that are not present in the source.

yakubin• 9 years ago

On the contrary. Reproducibility means that given the source you are able to produce the exact same binary, i.e. you can verify that someone didn't modify it before building. When software isn't open, then reproducibility is meaningless, because you don't have the source, so: 1. you can't verify anything (you can't compile anything); 2. assurance of the binary not being modified gives you nothing, since you don't know if there is no malicious code in the original source code.

thechao• 9 years ago

I know Debian has lofty goals with respect to reproducibility, but on a purely "I hate waiting for builds", getting deterministic object files can prevent spurious linking. For example: change a comment, watch your code link for 10 minutes.

lamby4• 9 years ago

Mmm. Think of the CO2/power savings at a big coding shop like, I dunno, Google...

wiz21c2• 9 years ago

Just to give an example, think about some software you put in a voting machine. Even if the code is open source and given to voters, they need to be able to compile it themselves and compare their compiled version to the one in the voting machine... (well, to be honest, this only makes sense if the voting machine runs the compiled code only and doesn't sneak other stuff in at runtime, but that's another story)

AdmiralAsshat• 9 years ago

Not always. TrueCrypt was open source, and yet a huge number of people were worried that the distributed binaries could be backdoor-ed in some way. See this article about someone trying to create a reproducible binary of TrueCrypt.[0]

[0]https://madiba.encs.concordia.ca/~x_decarn/truecrypt-binarie...

bluGill2• 9 years ago

Reflections on trusting trust. http://dl.acm.org/citation.cfm?id=358210 The paper is a classic in computer science so you should know it.

stordoff• 9 years ago

Debian being open only gives you any sort of assurance if you can prove the binary you are using is compiled from that source. Without reproducible builds, you can only (easily) do that if you are building the source yourself (which obviously most people don't).

Reproducibility doesn't really make much sense when the code _isn't_ open - knowing that unknown source code can reliably produce the same output isn't that valuable.

vegbrasil• 9 years ago

Is Docker (or any other container platform) a facilitator to reproducible builds? Making the environment standard between builds is probably easier in a container.

georgyo• 9 years ago

Builds have been happening in single use chroots for a very long time.

The problem is if code includes the current date, or generates some randomness at build time. These issues must be identified and patched.

0xcde4c3db2• 9 years ago

Docker is part of a broader "reproducible build environment" strategy, but doesn't really help with some of the things that cause problems (timestamps, kernel version, random IDs).

wtallis2• 9 years ago

Docker seeks to reproduce a functionally equivalent software environment, motivated by version management concerns. Debian is trying to reproduce bitwise identical build products, motivated by security concerns.

samueloph• 9 years ago

Debian builders already run on sbuild[1], which uses chroot, so it's already a contained environment.

I'm not sure how the reproducible builds machines works for forcing two different build results, but i'd bet they're using sbuild too.

Here's a diffoscope of one of my packages which is not reproducible right now[2].

I also highly recommend the reproducible builds talk at this year's debconf with the repro build folks[3].

[1]https://wiki.debian.org/sbuild

[2]https://tests.reproducible-builds.org/debian/rb-pkg/unstable...

[3]https://debconf17.debconf.org/talks/14/

zdw• 9 years ago

Docker images built with the first-party toolchain aren't reproducible - if you run `docker build ...` on a Dockerfile, then delete the image and rerun it, you'll get a different set of image hashes. This is likely due to timestamp embedding.

There are other toolsets that supposedly create byte-identical Docker images generation (Bazel, some others), but I haven't tried them.

pavlov• 9 years ago

The lack of a .yet TLD is a real missed opportunity.

penpapersw2• 9 years ago

Meh, joke sites like this aren't nearly as prominent as sites for apps, I'd really like a .app TLD instead. That said, the distinction between app and service is blurring a lot, so Spotify and VS Code probably both qualify for .app but one also has a web interface. Everything is confusing, let's just stick to .com

thinkMOAR• 9 years ago

Why do you consider this a joke site?

And just because you are easily confused, everybody should stick to .com only?

entelechy3• 9 years ago

I wonder if switching from cmake to buck[1] and buckaroo[2] would simplify and increase reproducibility.

[1] https://buckbuild.com

[2] https://buckaroo.pm

philippnagel• 9 years ago

Also interesting: https://bazel.build/

yorwba2• 9 years ago

Funny that you should mention bazel on the topic of reproducible builds. When I last used it to build TensorFlow, the build was very much not reproducible. I took me three rebuilds with same flags to finally get GPU support compiled in.

moosingin3space• 9 years ago

Based on the experiences of systemd, gtk, and X11 in switching to Meson, I'd think Meson might be the best choice here. While Buck, Bazel, Pants, etc. are designed for large projects, Meson is designed for small-to-medium-size projects and integrates with pkg-config, which in turn should provide simpler integration with distribution package managers. My experiences with Bazel demonstrated that integration with distribution packages can be quite difficult.

entelechy3• 9 years ago

Although never mentioned, you can use pkg-config with buck by embedding your pkg-config command into the compiler flags eg.:

cxxflags = ${pkg-config opencv python --cflags --libs}

Furthermore I don't see the reason why you woudn't use buck for small projects. What makes you think buck is less suitable for small projects ?

entelechy3• 9 years ago

why I'm consider that using pkg-config in buck may break reproducibility: https://github.com/facebook/buck/issues/1430

sfrigon• 9 years ago

Is there any known problems / pitfalls regarding cmake and reproducible builds? Just curious to know.

jwilk• 9 years ago

cmake uses absolute paths almost everywhere.

This is a big problem if you want reproducible builds with varying build directory.

https://lists.alioth.debian.org/pipermail/reproducible-build...

guipsp• 9 years ago

^CEO of LoopPerfect, which maintains Buckaroo

Might want to disclose that.

geofft3• 9 years ago

Well, this is Debian - they generally don't maintain the software, they just package it. Small changes can be pushed upstream, but "Here's a brand new build system that we promise is super cool" is generally not a patch that people like taking :)

rstuart4133• 9 years ago

Well, this is Debian - they generally don't maintain the software

They definitely maintain software. There is a whole pile of packages that aren't maintained upstream any more that are still maintained in Debian, they write things like manual pages for commands that don't have then, they bash the build scripts into something that works on a whole pile of architectures the author never considers, they manage what is probably the worlds biggest issue track system, and then there is the security team producing patches for versions upstream no longer supports.

But probably it's just your wording. While they do maintain software, they are mostly concerned with packaging existing software, not writing new stuff.

lamby4• 9 years ago

Mods, how come my post was completely edited? :)

rahiel• 9 years ago

This was the originally submitted link: http://isdebianreproducibleyet.com/

lamby4• 9 years ago

Indeed, which is a completely different "view" on things...

dang• 9 years ago

The is-foo-bar-yet.com -> NO/YES trope has been off topic on HN for years, mainly because such pages are unsubstantive but also because it's long been a cliché.

If we can find a more substantive page to link to on the same topic, we'll sometimes change the URL. Usually we post a comment explaining that we did so, but it depends who's on duty at the time.

Sir_Cmpwn• 9 years ago

I personally don't think is-foo-bar-yet.com has been played out, nor do I agree with editing cliche links like this. The link is more substantive than you let on.

YouKnowBetter• 9 years ago

On a side note: A point that many a company ignores when they demand some assurance so their software vendor a) can be held liable b) will hand over the source in case of drama, is to give the source code in Escrow. The number one error these companies make is to think that source code without a reproducible build environment means anything at all. Reproducibility is not easy. I applaud Debian (and in the older days TrueCrypt) for giving this more exposure.

bluGill2• 9 years ago

Someplace my company has a closet with a computer containing Windows XP (not sure which service pack, but not the latest), and and old version of visual studio, just in case we need to fix a bug. I know we still have it because the project got resurrected recently and we had to clone the harddrive a few times because nobody could find a copy of that version of visual studio that can be installed. (Microsoft dropped support for whatever version of WinCE with no upgrade path)

monocasa• 9 years ago

We have a bunch of VMs cloned from ancient machines for the same reason.

mmagin• 9 years ago

See also this page of tools: https://reproducible-builds.org/tools/

disorderfs is nice for ferreting out readdir()-ordering effects.

Debian reproducibility statistics

Related Stories