Go crypto: bridging the performance gap

https://blog.cloudflare.com/go-crypto-bridging-the-performance-gap/

by jgrahamc • 11 years ago

211 87 11 years ago

blog.cloudflare.com

4ad4• 11 years ago

I will happily look over these once they are part of upstream Go, or I will happily review them once they are proposed upstream, but I will take a pass on their "special fork of Go".

CyberShadow• 11 years ago

I don't understand, why reimplement this if it is already implemented with adequate performance/quality in OpenSSL? I thought Go didn't have issues calling C code?

larssorenson• 11 years ago

I think it's the overhead of the call or maybe their distaste for the state of repair of OpenSSL, e.g. Heartbleed, that drove them to it. Alternatively it could just be the chance to write some crypto in Go assembly and have it be included in their code base.

vitriol833• 11 years ago

I agree, ECDH and AES-GCM are sufficiently complex to implement it makes sense to call into OpenSSL. OpenSSL has had its problems, but these tend to be with TLS protocol handling. The underlying cryptographic constructions have had a lot of attention.

giovannibajo1• 11 years ago

Well, to tell the truth, the underlying crypto functions are also the ones harder to get wrong, using test vectors. The hardest part is probably trying to make them constant-time, but AGL is also the author and maintainer of Go crypto library, and he did extensive research on constant functions, and contributed code to OpenSSL as well. He even hacked valgrind to check for constant-time at runtime, which blows my mind (https://www.imperialviolet.org/2010/04/01/ctgrind.html).

So I think the general advise not to rewrite a TLS library doesn't fully apply to the Go team like it would apply to us.

jgrahamc4• 11 years ago

Sure, you could do that but imagine the nightmare of modifying the Go standard libraries for stuff like TLS to hook into the OpenSSL C interface. We actually did that for a while and it was horrible: http://blog.jgc.org/2013/01/integrating-openssl-crypto-funct...

4ad4• 11 years ago

Calling C code from Go is something you do as a last resort. It breaks some desirable properties of Go programming, like fast compilation, cross compiling (at least without effort), and static binaries (without serious effort). Code called through cgo will never be idiomatic Go code, unless perhaps if hidden under an abstraction layer which takes time and effort to write and adds overhead.

On top of that, the cgo overhead is significant, not only in time, but in threads, so you'd want it only for big chunks of data and program accounting for it.

Another advantage of Go crypto is having been written in a memory safe language, and with runtime bounds checking. The assembly fast paths break some of this, but not most of it.

andrewchambers2• 11 years ago

Go has moving memory so it generally means calling C requires a bunch of copying. This is slow and annoying.

1_player• 11 years ago

Go has moving memory so it generally means calling C SHOULD/WILL require a bunch of copying.

I've been working on a few Go frontend to C libraries and researching what's the exact memory model when calling C: Yes, Go has moving memory and you _shouldn't_ pass a Go pointer to C, and it's actively discouraged [1], but since many libraries just ignore that advice [2] for performance reason Go isn't currently AFAIK not moving any pointer passed through cgo.

That will probably be changed in Go 1.5+, I reckon an official post from the Go developer about the current and future state of C/Go memory interaction would help clarify this.

1: https://github.com/golang/go/issues/8310

2: comment #31 of https://github.com/golang/go/issues/8310

robmccoll3• 11 years ago

Are you sure that's true? Perhaps they have reserved the ability to do that in the future (can you point to doc on that? ), but right now I'm fairly certain that go does no such thing. The only copies necessary when calling C code are for strings since they aren't guaranteed / required to be null terminated byte arrays in Go. And cgo has C ABI compatibility so function call overhead is non existent.

ominous_prime2• 11 years ago

No, the slow part is switching stacks, and coordinating with the scheduler. Go's GC doesn't yet move memory, and even so there will probably be allowances for passing pointers from Go to C when it is implemented.

pjc502• 11 years ago

OpenSSL no longer really qualifies as "adequate safety".

pjmlp3• 11 years ago

The more C code that gets replaced out there increases the system security.

eloff3• 11 years ago

Because assembly is safer than C? Did you read the article?

istvan__2• 11 years ago

I don't think that OpenSSL is a great example of OSS, there are several areas where they don't follow bad practices for no reason and you can just look at their history of serious security flaws. Why would you want to integrate with that? In this case it is better and cleaner to implement the functionality in your code (in this case Go).

0xdeadbeefbabe• 11 years ago

Having a history of security flaws is better than having a future of security flaws.

baby• 11 years ago

OSS?

pcwalton• 11 years ago

cgo is slow due to Go's stack growth and userspace threading model. (The limitations are basically fundamental to the way Go handles concurrency.)

aikah• 11 years ago

I'm curious whether the Go team will merge Go crypto into the core or not. It will show how open they are to third party contributions.

4ad4• 11 years ago

Your comment is significantly out of place. The Go team is extremely open to quality external contributors. 463 people have contributed to Go so far, (obviously) most of them outside the core Go team. The Windows port was exclusively done and maintained by contributors. I have done the arm64 Go compiler, which is upstream now, the Solaris port, and I am now doing the sparc64 compiler. Many 3rd party contributors do many things every day.

The Go project is an extremely open project. More than half of the people who have direct commit access are external contributors.

coldtea• 11 years ago

Your comment is significantly out of place. The Go team is extremely open to quality external contributors.

I think his comment is still valid. Adapting something major like this is not the same as accepting bugfixes from hundrends of people, or ports to a different architecture.

Even more different would be accepting some code for the standard library whose API wasn't designed by the core team.

From what I've seen the core team is quite opinionated and micro-managing things.

travjones• 11 years ago

It's great to see that companies can turn a profit AND contribute to open source projects that benefit the web as a whole. Thanks, Cloudflare.

michaelt• 11 years ago

Cloudflare's Universal SSL is pretty great - I used to host my static website on S3, but that means no SSL or ipv6. Putting everything through cloudflare sorts both those out.

The original announcement [1] mentioned they were planning support for adding in the HSTS header - as jgrahamc is here responding to comments, I'd be interested to hear how far they've got with that :)

[1] https://blog.cloudflare.com/introducing-universal-ssl/

jgrahamc4• 11 years ago

We added it during "Week of SSL": https://blog.cloudflare.com/enforce-web-policy-with-hypertex...

thomasahle2• 11 years ago

"Let's just rewrite all the crypto ourselves! In assembly!"

Sorry to be a buzzkill, but that sounds like a recipe for disaster.

imaginenore2• 11 years ago

If it's tested extensively, it should be fine. Somebody has to write the crypto.

vitriol833• 11 years ago

without formal methods it's impossible to fully test crypto implementations, such as ECDH, because the number of possible inputs are enormous. bugs in small proportion of inputs can lead to fault attacks. and furthermore side channel attacks are very common.

pjc502• 11 years ago

Yeah, the point of moving from C to Go would be to get away from all the disasters. But if the core is small enough using short bursts of inline assembly to actually calculate the cipher, while all the complex protocol handling and buffer manipulation is done in the safe layer, it would be a good solution.

Edit: that does indeed appear to be what they've done. In particular using the AESENC instruction. https://github.com/cloudflare/go/blob/master/src/crypto/aes/...

4ad4• 11 years ago

All the low-level crypto is written in assembly. And not only for speed, but for ensuring properties like constant-time execution, etc. It's the same for OpenSSL, and the same for commercial crypto libraries. Go is not at all different here.

The difference is that the higher-level crypto is written in Go, not in C; Go is memory safe, much more strongly-typed in general, and with run-time bounds checking which eliminate buffer overflows.

The bugs are almost never in the low-level algorithms, they are in the higher-level components.

alfiedotwtf• 11 years ago

There was a post on HN a few weeks ago talking about "ensuring properties like constant-time execution" isn't possible as instruction timings doesn't take in account things like caching, pipelining, task switching, microcode optimisations etc.

zurn• 11 years ago

OpenSSL does optional assembly implementations for many primitive+platform combinations, but also many of the algorithms under crypto/ have zero or one architectures covered. And many asm implementations predate widespread concerns about timing attacks.

jussij• 11 years ago

And only a year or two ago OpenSSL was found to have a major hole (i.e. remember Heartbleed), caused by a buffer overrun bug that had been around for years. If I remember correctly I also remember reading on the Go forums Go didn't have that issue, only because it had been fully re-written.

grittygrease• 11 years ago

The OpenSSL AES-GCM and P256 assembly code was also written by Vlad. There's no better person to write the Golang version.

thomasahle2• 11 years ago

That's a good argument. However (unless the changes get merged upstream) it is still one more crypto library for 'bad people' to find weaknesses in.

sdevlin5• 11 years ago

Neat, I didn't even notice this.

Vlad Krasnov is also a co-author of this paper on state-of-the-art P256 implementation: https://eprint.iacr.org/2013/816.pdf.

sdevlin5• 11 years ago

A couple points on this.

First, as some have noted, serious crypto primitive implementations are written in assembly. This is both to achieve state-of-the-art performance as well as data-independent execution times. The latter is important to prevent timing attacks.

Second: I'm not sure if this was your point, but some have invoked Heartbleed and other native code disasters. But the kind of problems that lead to Heartbleed aren't likely to be a problem in low-level crypto implementations. This is because they tend to operate on fixed-size buffers using algorithms with little or no conditional logic. While there could certainly be mathematical flaws (i.e. producing the wrong output), something like a buffer overrun is not likely here.

If you look in basically any crypto library, you will find important primitives implemented in assembly. This is even true in the main Go repository, where AES is implemented in assembly.

ufo• 11 years ago

Given the many vulnerabilities related to the use of AES-CBC with HMAC

What are they talking about here? Are there any important ones if you MAC after encryption? The only vulnerabilities I know of are when you MAC before you encrypt.

tptacek3• 11 years ago

They're talking about the TLS CBC constructions, not AES-CBC and HMAC in the abstract.

atlbeer• 11 years ago

Off-topic but, they probably shouldn't be using a Southeast train image in this blog post as they are notoriously bad for delays and slow service in the UK[1] :)

[1]http://www.which.co.uk/home-and-garden/leisure/reviews-ns/be...

jgrahamc4• 11 years ago

I added that. As someone who lives in London I'm well aware of how bad their service is. It was a deliberate ploy to make you spend more time on the blog post by getting your to shake you head about the image and then read the post.

ashearer• 11 years ago

Nice work! The article benchmarks a > 20X speedup for AES-128-GCM, for performance described in the text as "on par" with OpenSSL. It would be helpful for reference to have an OpenSSL column added to the benchmark table.

Intermernet• 11 years ago

I didn't know the name Vlad Krasnow (I'm not that familiar with the crypto world), but from looking at these [1], [2], it seems that he would know what he's talking about from both a crypto and performance viewpoint.

attempting to make them part of the official Go build for the good of the community

Would be interesting to hear what Adam Langley thinks about this, but I couldn't find anything recent on the golang-dev list.

[1] http://dblp.uni-trier.de/pers/hd/k/Krasnov:Vlad

[2] http://stackoverflow.com/users/1516766/vlad-krasnov

stock_toaster• 11 years ago

I found this: https://github.com/cloudflare/go/issues/5

tav2• 11 years ago

Nice work! Do you guys also have a similar implementation of ChaCha20-Poly1305 for Go? If so, any chance you could share that too?

jgrahamc4• 11 years ago

That work has been done for OpenSSL, I guess we could add for Go: https://blog.cloudflare.com/do-the-chacha-better-mobile-perf...

tav2• 11 years ago

That would be pretty awesome! Also, do you happen to know if your current patch is safe to apply on top of OpenSSL 1.0.2a?

biftek• 11 years ago

I'm not sure what's involved with getting assembly working in Go (my experience has only been with servers/clients and CLI's), but could this not have been implemented as a stand alone package? What's the benefit of a fork in this case?

jzelinskie• 11 years ago

I'm also wondering the answer to this question. To my understanding, go-crypto is actually maintained separately from the Go stdlib by the Go developers[0]. So why isn't this just a fork of crypto? Why fork the entire language? Why can't this be upstreamed?

[0] https://github.com/golang/crypto

sdevlin5• 11 years ago

This is a library of supplemental crypto algorithms. You won't see things like AES, SHA-2, or the NIST curves in here. Those things are part of the standard library in Go.

sdevlin5• 11 years ago

I can't speak for Cloudflare, but I would guess that they want these changes to be merged upstream. If that happens, the benefit is that consumers of the stdlib crypto API will get increased security and performance for free in a future update.

elithrar• 11 years ago

I can't speak for Cloudflare, but I would guess that they want these changes to be merged upstream. If that happens, the benefit is that consumers of the stdlib crypto API will get increased security and performance for free in a future update.

Yes, they do: https://go-review.googlesource.com/#/c/8968/

amelius• 11 years ago

I'm also wondering, why fork the compiler? What if another company forks Go to implement, say, a faster spam filter. How would one ever combine these two forks?

jvermillard• 11 years ago

Strange they dont mention AES-CCM at all. I'm using it a lot in IoT applications (dtls). It's not frequent in the "web" world?

donpark2• 11 years ago

Wish Cloudflare spread some of that Go asm love to Ed25519.

higherpurpose• 11 years ago

Indeed. Couldn't they have supported it for a small part of users like they're doing with ChaCha20-Poly1305?

donpark2• 11 years ago

ChaCha20-Poly1305's 3-4x performance gain over AES-GCM made it worthwhile for them where Ed25519's benefits are not as clear cut to non-proponents. Too bad.

Go crypto: bridging the performance gap

Related Stories