OpenBSD has started a massive strip-down and cleanup of OpenSSL

https://lobste.rs/s/3utipo/openbsd_has_started_a_massive_strip-down_and_cleanup_of_openssl

by Serow225 • 12 years ago

304 99 12 years ago

LeoNatan252• 12 years ago

Good. Another look at any code after a long period often yields various degrees of improvements due to a different perspective at the later date.

LukeShu2• 12 years ago

This isn't "another look"; OpenSSL is not affiliated with OpenBSD/OpenSSH, despite the similar name. This is OpenBSD forking OpenSSL.

Someone at the linked URL humorously commented "I hope the new version is called OpenOpenSSL."

LeoNatan252• 12 years ago

? Did you think I assumed "similarity" due to "Open" in the title? It is "another look". Another look can come from original writers, yes, but often a second pair of eyes brings a different perspective that is almost always beneficial.

iSnow• 12 years ago

Or just call it "WideOpenSSL" until they are done fixing the code and writing tests.

I like how the OpenBSD team is picking up the ball here instead of throwing up their hands in despair, even if it means forking the code.

ekianjo• 12 years ago

Linking a lobster thread on HN, instead of the original link? Why would you want to do that?

Serow225• 12 years ago

I know it's unusual, but in this case I thought it was justified because the original BSD CVS link offers little context of what's going on unless you parse the changelog, whereas the lobster thread has a good summary by jcs.

gopalv• 12 years ago

Removal of MacOS/Windows junk?

Is this going to be an openbsd only fork (I guess linux will get a build of some sort as well).

astrodust3• 12 years ago

I'm taking this to mean MacOS as in MacOS 9 and prior, not OS X which is unrelated. Building for MacOS was always an ugly ordeal.

duskwuff• 12 years ago

Most likely, yes.

I wouldn't be surprised if a lot of the "Windows" support code is of similar age, and thus was actually only needed to support pre-NT Windows.

Sanddancer• 12 years ago

It's almost certainly referring to pre-OSX versions. However, if it ends up being handled like OpenSSH, the core will be OpenBSD only, with porting handled by another team.

plorkyeran• 12 years ago

If the cleanup turns out well I assume it'll get ported back to other platforms. It's very hard to change code with a bunch of stuff ifdeffed out (since you're basically guaranteed to break that stuff regardless of how hard you try not to), so even if they cared about other platforms this would probably be a reasonable approach if they're going to make major changes all over the place.

asveikau• 12 years ago

I was looking at this recently. Most of the Windows ifdefs in OpenSSL are junk. Even on Windows.

A lot of OpenSSL is portable C and IMO they should just let portable C be portable C.

rakoo• 12 years ago

It says "build junk", not only "junk". I haven't dug through, but maybe it was all the junk that was used to compile OpenSSL on said platforms.

If the code can be compiled without this, then they might as well remove it.

astrodust3• 12 years ago

"Removal of MacOS, Netware, OS/2, VMS and Windows build junk"

There's surely code in this project older than some of the developers working on it.

hkphooey• 12 years ago

Or maybe only gray-haired curmudgeons are working on this... CVS is quite a high entry barrier for young trendy Githubbers!

mahmud• 12 years ago

Is this an attempt at sarcasm, or just good ole ageism?

mrweasel• 12 years ago

I'm actually really impressed what so many of the OpenBSD developers can delete/reformat/edit the OpenSSL code at the speed they seem to be working right now and not run into issues with CVS. One awesome thing about CVS is cvsweb ( which is a separate thing, I know), I much prefer browsing around, diffing and viewing changes in cvsweb compared to say GitHub or Bitbucket.

Sort a side note: OpenBSD really seems to have picked up speed after they fundraising. It wonderful to see the money has been so well spent.

ParadisoShlee• 12 years ago

"Removal of all heartbeat functionality which resulted in Heartbleed"

Instead of simply fixing the bug and moving on, we're decided to assume the whole thing is tainted and burn it alive like a depressed teenager at the salem witch trials?

or is the heartbeat and IETF RFC considered suspicious and the heartbeat extension offers us nothing?

kansface• 12 years ago

It probably isn't needed by 99.999% of users. It should not have been added.

yuhong• 12 years ago

I think it can be added back if needed after the code is properly reviewed.

jerf2• 12 years ago

In addition to kansface's point, I'd observe it's a very recent addition. We made it a long way without it.

There's a quote referenced so often it has become a cliche: "Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away." by Antoine de Saint-Exupery. I think it's sometimes over-applied, but in the case of crypto software, it really fits. In a sense, the heartbeat code got unlucky and hit the mega-anti-jackpot, disproportional to its "actual" risk... but there was risk in adding it, and even before we know the outcome it would have, somebody really should have been asking hard questions about whether the reward justified the risk. I imagine there's a lot of other bits of the code where the risk/reward tradeoff suggests removal. You can look at much of what has already been trashed from that point of view. For instance, if you have no real need to support a particular defunct architecture, why carry around its code? It has no benefit, but non-zero risk; toast it.

colechristensen2• 12 years ago

An good engineer's time should be spent more on taking away than on adding.

pipeep• 12 years ago

It's the wrong abstraction layer for a heartbeat signal. That can easily be done by the application if it's needed. TLS and DTLS should provide encryption wrappers for TCP and UDP. That's it.

Sure, it's convenient to have it already available for you to use as an application developer, but critical security software needs to be kept as small and simple as possible.

AnthonyMouse• 12 years ago

or is the heartbeat and IETF RFC considered suspicious and the heartbeat extension offers us nothing?

The heartbeat extension is classic protocol bloat. In theory it can be useful for DTLS as a keep-alive for NAT port mappings or to do PMTU discovery, but you can do either one without it fairly easily. And then they went and added it to traditional non-datagram TLS where it serves the traditional screen door:submarine relationship.

lugg• 12 years ago

The beauty of open source is that you dont need to worry about silly things like productivity and return on time invested if you dont want to.

The code is a mess from what I hear. As long as they keep the functionality intact I dont really think anyone can complain.

I was quietly just thinking... Man I've gone to some lengths to force a rewrite in the past but this just takes it to a whole new level.

thrownaway24243• 12 years ago

Unfortunately it's going to be difficult to have much confidence in a huge edit of OpenSSL since OpenSSL is almost devoid of meaningful tests. It would be very easy for one of these "resolve merge conflicts" changelists to contain an error of the goto fail; goto fail; variety.

clarry5• 12 years ago

Obviously there's always a risk that something slips through. However it's not just a huge edit, it's also a huge review. And that is made so much easier when a ton of junk is removed from the code base, and the style is changed towards readable.

thrownaway24243• 12 years ago

When I do code reviews I read the tests first. How can you review code with no tests?

All the bugs in OpenSSL to this date have been reviewed by someone. Unfortunately those people haven't been veteran industry programmers but rather academics and "hero" types who think the kinds of things in OpenSSL are good code. We can only hope that OpenBSD folks can clean it up a bit but I'm not hopeful, because OpenBSD is just a different tribe of "hero" programmers.

guelo• 12 years ago

Is this fork a vote of no confidence in the OpenSSL team? Is it guaranteed that this fork will be used in OpenBSD? Could this fork be used in Linux? Will they be cooperating with OpenSSL by sending them bug reports etc?

zdw• 12 years ago

Traditionally, OpenBSD has made "Portable" versions of their code available for integration on other systems - for example:

http://www.openssh.com/portable.html

Basically, that's the version that gets those sorts of patches.

I'd imagine if this reaches the point where people want to put the OpenBSD fork of OpenSSL on other platforms, a similar approach would be used.

It's been pretty successful for OpenSSH:

http://www.openssh.com/usage/graphs.html

colechristensen2• 12 years ago

Nearly every operating system distribution maintains it's own fork of most system utilities and libraries with varying levels of divergence from upstream. A piece of recent news around here was Python 2.7 being maintained by RedHat for several years beyond upstream support. Some of the differences make it upstream, some of them don't. Doubtful OpenBSD is trying to usurp the current OpenSSL team, but if they do good quality work, it might be very influential a few years down the road.

copergi3• 12 years ago

Is it guaranteed that this fork will be used in OpenBSD?

It already is used in OpenBSD. The changes are being made in the OpenBSD source tree.

imthatguyama• 12 years ago

As a Linux fan who has been hearing good things about FreeBSD, will this work out of the box on FreeBSD?

ary3• 12 years ago

In the past I was part of an attempt to make asymmetric crypto easier to use for a commercial product. My role was to help design the product (simple, collaborative file sharing; you've heard it before) and to let the experts ensure the crypto was used properly. What drove me absolutely nuts was the level of paranoia and uncertainty around the safety of the libraries and algorithms used. The available experts were confident in the concepts, but feared every implementation. "Par for the course" you might say, but the status quo around OpenSSL and other libraries is unacceptable.

From an application developer's perspective crypto libraries look like a lightning rod even for the experts. No one wants to get too involved or make recommendations lest they back the wrong horse (admittedly an easy thing to do). So kudos to the OpenBSD team for rolling up their sleeves and attempting to build a solid foundation for the future.

I'd be happy if OpenSSL is simply fixed. I'd be overjoyed if there was a solid crypto system underneath an OpenSSL compatible API that gives us a path towards an open source, reusable crypto platform.

erichurkman• 12 years ago

It's easy to back the wrong horse when the whole race track is full of horses with broken legs.

stingraycharles• 12 years ago

This might sound totally irrelevant, but it kind of reminds me of the situation with NASA. Once a certain technology has been proven to work, people do NOT like to deviate from that change, since hey, it might fail and no one wants to take the blame.

The end result is that NASA is still using computers from the 60s, and it took someone like Elon Musk to change the status quo which resulted in reduced costs (which is why NASA is currently contracting SpaceX, which can do things cheaper).

akiselev2• 12 years ago

Bullshit. SpaceX dropped the launch cost, which is far away from the overall mission costs that NASA has to deal with. Dropping the launch cost means decreasing the cost of red tape and infrastructure (such as using fuel stored at higher temperatures), and increasing modularity and benefits of scale but it doesn't help decrease the cost of R&D and manufacturing of satellites and mars rovers.

NASA uses the RAD750 [1] as well as a few other still in production CPUs. Remember Voyager? The satellite is still flying and sending back priceless data after many decades? To launch missions (and critical infrastructure like GPS) you need years, decades even, for usage & reliability data to accumulate and industry to perfect the manufacturing process of parts that can handle environments that are, by any definition, at the extremes. But, please, show me a mission launched in the last 5 years with a 50 year old CPU design.

ericcumbee• 12 years ago

or in enterprise IT "No one ever got fired for buying IBM"

dingaling• 12 years ago

(which is why NASA is currently contracting SpaceX, which can do things cheaper).

And they're also contracting Orbital Sciences and Sierra Nevada and Boeing and probably a bunch of others of whom I have no knowledge.

SpaceX is following in the footsteps of many other private companies that have developed spaceflight hardware on their own initiative. They might be 'younger' and more agile but it's not a new innovation.

adventured• 12 years ago

I'd be surprised if NASA is using, on a wide-spread basis for missions, computer hardware older than 20 or 25 years at this point. I'm extremely skeptical they are still using much (any?) hardware from the 1960s or 1970s. Your exaggeration goes way too far.

chubot• 12 years ago

Do you know about NaCl (by DJB)? It doesn't implement SSL, but it is a very easy to use library for asymmetric crypto.

http://nacl.cr.yp.to/box.html

Can any crypto experts comment on whether it is feasible / how much work it is to implement SSL on NaCl? Maybe the issue is that NaCl doesn't support all the ciphers you need.

ary3• 12 years ago

I do, and I hope to give it a try in the latest product I'm building. Like everyone else I'd prefer to see some expert validation of NaCl before I put a ton of trust into it. That said DJB's track record is pretty good in my eyes (I liked the design of qmail, a lot).

gnoway• 12 years ago

I have been wondering the same thing. This link suggests that there are problems with NaCl preventing adoption, and puts forth a repackaged alternative called Sodium:

http://labs.opendns.com/2013/03/06/announcing-sodium-a-new-c...

Tinned_Tuna• 12 years ago

The SSL/TLS protocol unfortunately uses some known-bad constructions, which lead to intractable issues (see: BEAST, Lucky13 for examples)

NaCl's goals are vastly different to those of SSL/TLS. SSL/TLS aims to provide a simple, clean interface with sane defaults for the majority of simple use-cases, whereas SSL/TLS aims to provide an interface with near-infinite flexibility for the case of providing an encrypted, authenticated tunnel.

NaCl also deliberately does not support lots of ciphers, as that makes it easy for developers to choose poorly, for example, (Alleged) RC4, as is supported in OpenSSL.

lucian1900• 12 years ago

NaCl supports only one cipher for each purpose by design, it could never be used to implement TLS. It's only useful for making your own protocol.

drewcrawford• 12 years ago

Part of the problem here is that it is very hard to audit something OpenSSL's size. It has 450,000 lines of code. Are you going to skim, let alone audit, a mathematics-heavy codebase of that size? No.

Some of that is OpenSSL's fault, but a lot of it is inherent in implementing a 200-page (before extensions!) standard. GnuTLS is 170 KLOC, CyaSSL is 205 KLOC, and even PolarSSL, commonly thought of as the gold standard in trimming the fat, is 55 KLOC.

If you want to reduce the level of paranoia, you would have to pair down the size and scope of TLS by an order of magnitude. For the record, I support that kind of thing, but you would have to cut a lot of things people actually use, not to mention breaking backwards compatibility.

ary3• 12 years ago

While I understand where you've coming from I think your attitude is dangerous. Implementing the SSL/TLS spec(s) is always going to be difficult while they are in widespread use. That said refactoring the existing libraries (to make them more readable and understandable) and then auditing them (to gain confidence in their efficacy) is especially important given the nature of their applications.

Doing the hard work of getting this right, or accepting the status quo and getting it wrong, will have a very real impact on the future of many, many people. Should the time come that changes have to be made to the standards themselves then we can debate the merits of breaking backwards compatibility. I value pragmatism more than most (I think) but there comes a point where the cost of not changing is higher than the cost of changing. I think we're there.

splicer4• 12 years ago

How about adding "completion of our manpages" or "making our website less shitty" to that list? See https://www.openssl.org/docs/

na85• 12 years ago

Hmm, no I'd rather them make a quality product than work on their website.

The last thing OpenBSD or OpenSSL needs is a whack of totally-useless js on their website.

thrownaway24243• 12 years ago

Documentation would be a HUGE help to the code quality of OpenSSL because someone would have to actually sit down and describe its ghastly interface. Whoever it was would get about half way through it and then stop and say "Wait, what? Seriously?"

splicer4• 12 years ago

I didn't mention javascript. Try running their site through a validator: http://validator.w3.org/check?uri=https%3A%2F%2Fwww.openssl....

Duplicate IDs and font tags? Come on...

parennoob• 12 years ago

For what it's worth, OpenSSL and OpenBSD/OpenSSH are not maintained by the same team, even though the names are similar. This was what I was given to understand last week after answers to a slew of questions asking about this.

splicer4• 12 years ago

Ah, now all my downvotes make sense ;) The article gave me the impression that OpenBSD was the maintainer of OpenSSL.

copergi3• 12 years ago

How about adding "completion of our manpages"

That is pretty much a given. OpenBSD considers deficiencies in documentation the same as deficiencies in the software.

or "making our website less shitty"

OpenBSD developers have nothing to do with openssl's website.

splicer4• 12 years ago

If it's a given, then why do all 3 manpages on https://www.openssl.org/docs/ say "[STILL INCOMPLETE]" and have for years?

Edit: I just read parennoob's comment about how OpenBSD isn't the maintainer of OpenSSL. I didn't know this was the case.

orblivion• 12 years ago

Wow, this is what the Internet runs on?

midas007• 12 years ago

Appending to the PHK on the OP link, it would be nice to be able to randomly choose amongst several crypto implementations.

PythonicAlpha• 12 years ago

Understandable reaction, when it can be seen how people that wanted to have all the newest features in their product (newest TSL features in this case) shoot themselves into their foot while others that stayed longer with the old (proven) version where save ...

That is one reason for the popularity of Debian: (at least in the past) they did not jump on any new bandwagon immediately. Other distributions have much newer software, but also the newest bugs.

eikenberry• 12 years ago

Excellent PR move on the part of OpenBSD. This should definitely improve their donations and work.

Qantourisc• 12 years ago

TLDR; but in my opinion the bug is causes by using C, use well test container structures/functions, like those who test the bound of the container during runtime. Those brave enough could disable the bound-checking at compilation for performance gains.

tiquorsj• 12 years ago

Yay! Now OpenBSD can introduce a massive security bug. /s

estebank• 12 years ago

They are long overdue for one =)

jordanthoms• 12 years ago

OpenBSD has a great security record, definitely excited to see what they can do if they take over OpenSSL maintenance.

gioele• 12 years ago

That list misses a an important bullet point:

* Added dozens of tests.

nnq• 12 years ago

...still waiting for an article about a team of super-heroes starting to rewrite OpenSSL in Haskell

nicolast• 12 years ago

You mean http://hackage.haskell.org/package/tls ?

wiz21• 12 years ago

I don't know much about OpenSSL but what I know for sure is that refactoring the code itself might not be enough. There must be some process involved to keep track of what was refactored, what was not, enforce rigorous testing, systematically review the code, and make sure all of that is publicly available (when I look at the website, that information is nowhere to be found)

Corrado• 12 years ago

"OpenSSL must die, for it will never get any better." - PHK http://queue.acm.org/detail.cfm?id=2602816 A pretty good, recent look at OpenSSL code and how it can be improved.

protomyth2• 12 years ago

OpenBSD Journal[1] points out "To clarify, not all of the cryptographic engines were removed; the padlock and aesni engines are still in place."

1) http://undeadly.org/cgi?action=article&sid=20140415093252

bazzargh• 12 years ago

Finding it easier to read through the commit log here than in cvsweb:

http://freshbsd.org/search?q=libssl

protomyth2• 12 years ago

That is a pretty nice way to track the changes. I see some of the comments are getting ranty http://freshbsd.org/commit/openbsd/58777eed1cff7c5b34cbc0262...

uuid_to_string• 12 years ago

Who knows, maybe one day we'll be able to install OpenSSL without having to first install Perl?

IMO, this a symbol of OpenSSL's unreasonable complexity.

OpenSSL is a default system library yet the user cannot compile it with only default system libraries.

(OpenBSD has added perl to their userland but this is not universal across other UNIX-like OS.)

Where are the compile-time options to turn things off?

The compile process has apparently become so "challenging" that the developers could not figure it how to do it easily without using perl.

geerlingguy• 12 years ago

This will be interesting to watch; I hope OpenBSD can do for OpenSSL what they did for OpenSSH—clean it, refine it, repackage it, and make it even better, ready for a new decade of use. I published a blog post which mentions how OpenBSD transformed SSH into the de-facto remote administration standard it is today just this morning[1].

I think this will be good news!

[1] https://servercheck.in/blog/brief-history-ssh-and-remote-acc...

clarry5• 12 years ago

Even though passwords are sent in the clear, ...

SixSigma• 12 years ago

Ssh has set back the computing landscape 20 years. Why oh why the tty?

http://harmful.cat-v.org/software/ssh

JimmaDaRustla• 12 years ago

We take for granted the "modern" security tools and libraries.

But at the same time, one would typically assume that something as large as OpenSSL has been reviewed and tested...

OpenBSD has started a massive strip-down and cleanup of OpenSSL

Related Stories