gRPC in Production

https://about.sourcegraph.com/go/grpc-in-production-alan-shreve

by loppers92 • 9 years ago

186 56 9 years ago

about.sourcegraph.com

nemothekid• 9 years ago

Thrift is Facebook's version of gRPC right? If so, I don't quite understand the comparison, and how gRPC succeeds where thrift "fails". Wouldn't all language implementations of gRPC have to be well documented, reliable, highly performant and easy to install?

azurezyq• 9 years ago

on the contrary, I always found thrift lack good documentation.

vikiomega9• 9 years ago

That's the sense I got too but beyond the initial hurdles it's good enough.

atombender• 9 years ago

I guess Google wanted to start with a clean slate based on the design principles already established internally (a system called Stubby).

gRPC was designed from the start for HTTP/2, which comes with some benefits: It's able to work wherever HTTP works (load balancers and proxies), can multiplex calls over a single stream (Thrift on the JVM, where it's most popular, uses a thread per socket), supports cancelation and streaming and so on. gRPC is arguably more opinionated than Thrift here; Thrift is both a serialization format and an RPC mechanism, and for Thrift RPC you can choose between different transports and framings, of which HTTP is just one. gRPC is HTTP-only, and is (at least nominally) serialization-format-agnostic; you could, in principle, use Thrift over gRPC instead of Protocol Buffers, for example. So in this sense, gRPC is more pure and generic than Thrift. (I'm sure Thrift fans might disagree here.)

I haven't actually used Thrift, so maybe someone else can chime in about other reasons gRPC is preferable.

wrsh07• 9 years ago

My understanding is that Thrift is Facebook's analog to Google's Stubby. Or gRPC [which is the next generation of Stubby].

It's not implementing the gRPC interface. I don't think gRPC was open-sourced [2015] early enough for Thrift [2007] to be built to its API.

Disclosure: Google employee who uses Stubby. [but is not on the team]

puzzle• 9 years ago

Thrift was written at Facebook by an ex Google intern trying to recreate something close to protobuf+Stubby.

fizx• 9 years ago

Thrift is comparable to GRPC, and more user-friendly in practice than GRPC. GRPC is more flexible, rigorous (anal).

I disagree that Thrift failed, but GRPC has more momentum right now IMO.

Source (GRPC at current startup, multiple years of thrift)

cookiecaper• 9 years ago

Thrift is popular and it "supports" more targets, but I found a lot of bugs in some of its lesser-known implementations. This may not be an issue if you're using a widely-used Thrift platform, but if you're choosing Thrift because it supports the target you want while another RPC doesn't, do watch out for weird bugs and issues.

I haven't tried to use Protobufs or gRPC in a serious way so I can't say if it's better or worse, but I would hope it supports fewer targets because it takes stability and QC more seriously.

brango3• 9 years ago

The only thing holding back gRPC is JS web support. If it had that it'd be time to drop swagger completely. As it is you need to go protobuf -> swagger -> js lib, but it's cumbersome and doesn't work 100% (e.g adding auth keys, etc.).

Will there be any progress on JS web, or can it just not be done at all with HTTP1? Even a subset of features for basic GET/POST would be fine...

buckhx• 9 years ago

We've been using grpc-web from Improbable successfully for internal services. I'm working on a more production scale application soon that I'm planning on leveraging grpc-web for. The biggest missing feature is client side streaming support which is hamstrung the new wgfetch/streams API, but I feel confident in finding solutions for situations that generally require client side streaming.

https://github.com/improbable-eng/grpc-web

brango3• 9 years ago

I hadn't seen that. Thanks!

simonhorlick• 9 years ago

We've just gone live with a grpc-web based application. It's still early days and there are some rough edges, but it's an absolute joy to use and the amount of time we've saved is crazy.

cshenton2• 9 years ago

There's also the fact that, for high level languages like python, the gRPC server has lower throughput than available python http servers. Even with the overhead of parsing http and whatever message format you're sending over it.

weberc22• 9 years ago

This is surprising given that there Python implementation is apparently C. Anyway I can't imagine this is the bottleneck in any nontrivial Python app.

phamilton• 9 years ago

I'd say load balancing is still a big gRPC hurdle. See https://github.com/grpc/grpc/blob/master/doc/load-balancing.... for the current official stance on load balancing. It's pretty convoluted IMO.

brango3• 9 years ago

IIRC on K8s there are some gRPC-aware load balancers, possibly Envoy, I forget.

YZF• 9 years ago

I agree. You kind of have to roll your own for every language you use which is odd. And retries. Hopefully some more standard way of handling this will emerge.

EDIT: I enjoyed the GopherCon talk... I don't think the video is up yet but when it is I recommend watching...

leastangle• 9 years ago

I like the approach of custom name resolver though. Pretty straight forward to build one e.g. backed by Consul.

morecoffee5• 9 years ago

Surprisingly, the reason why gRPC JS web support doesn't exist is because of Chrome. There is currently no way to access the HTTP trailers from Javascript. gRPC depends on trailers to tell when an RPC is complete when doing bidirectional streaming.

Please tell browser devs to provide access to them! They have been part of the HTTP spec for years. Only recently have browsers entertained the idea:

https://bugzilla.mozilla.org/show_bug.cgi?id=1339096

https://bugs.webkit.org/show_bug.cgi?id=168232

https://bugs.chromium.org/p/chromium/issues/detail?id=691599

https://developer.microsoft.com/en-us/microsoft-edge/platfor...

_ydfu3• 9 years ago

Another downside in gRPC is that (de)serialization can be relatively slow. One of my pet projects is a tagging server that takes a list of tags and returns a list of objects that match, retrieved from RocksDB. I tested gRPC (proto and FlatBuffers) and Cap'n'Proto. For all of them, the process looked like:

Insert: Object comes in (built in RPC system for both), object is assigned ID (which is added before serialization), object is written to RocksDB, object is serialized and indexed.

Query: List of strings come in, strings are looked up in tag index, intersection of results is done, objects are retrieved from RocksDB, objects are deserialized, objects are added to list, objects are returned to client.

FlatBuffers was unfortunately a no-go since it doesn't permit modification of fields that haven't yet been set and it didn't have a sane way of making a copy (which would also be slow).

Protocol Buffers worked but the pure Python driver is incredibly slow and the C++ driver for Python was non-default and marked "experimental". I eventually adapted to it but it was still quite slow even on the server side. From memory a response with C++ client and server took ~9ms, 4-5ms of which were spent on serialization/deserialization.

Cap'n'Proto eventually won for me. The Python driver is unstable and memory usage soars like an eagle until Linux shoots it down with an OOM but the server was much faster. Typical response times were closer to 3ms, most of which was spent in RocksDB or the actual index lookup. A downside though was that Cap'n'Proto has its own built in and odd library for async stuff and doesn't really support threading.

shaklee32• 9 years ago

I looked at capn proto as well, but ultimately decided against it because the common view online was grpc was much better at rpc. Maybe it's gotten better since then, but capnproto at the time was really comparable to protobufs and not grpc.

maccard3• 9 years ago

Last time I looked, capn porto was a no-go on Windows for RPC.

kentonv3• 9 years ago

Cap'n Proto includes both serialization (comparable to protobuf) and RPC (comparable to gRPC).

The main argument I've heard in favor of gRPC is that it is supported in more languages. Otherwise, I'm not sure what would make gRPC "much better at RPC" -- Cap'n Proto is actually much more expressive, and has features like capabilities and promise pipelining which gRPC lacks.

Do you remember the arguments you saw? I'd be curious to know what they were.

(Disclosure: I'm the author of Cap'n Proto and also Protobuf v2.)

trimbo• 9 years ago

From memory a response with C++ client and server took ~9ms, 4-5ms of which were spent on serialization/deserialization

What's the message size?

Veratyr2• 9 years ago

It was about a megabyte from memory. Lots of text.

morecoffee5• 9 years ago

gRPC is slow with Python, but most people don't pick Python if they are serious about performance. gRPC has continuous benchmarks running[1] to track and improve perf. I agree the Python one has some serious problems, but don't let it be the whole story. The C++ implementation is 100x faster.

[1] https://performance-dot-grpc-testing.appspot.com/explore?das...

dyu-• 9 years ago

For the non-pingpong benchmark [1], c++ (300k rps) looks to be around 2x faster than java (150k rps), while the latter is around 2x faster than go (75k rps).

It looks like go's non-compacting/non-generational gc is at play here. Even csharp is faster than go on that benchmark.

The one you posted has no deserialization overhead since its just ping/pong.

[1] https://performance-dot-grpc-testing.appspot.com/explore?das...

edit: fix typo

Veratyr2• 9 years ago

I'm aware and I did try out a C++ client but there was still a definite upper bound to its performance and latency was still ~2x Cap'n'Proto.

therealwardo• 9 years ago

It is important to note that message complexity has a lot to do with proto performance. Anyone with a basic understanding of algorithms could tell you that, but I prefer to be shown things so here's a repo with some examples of Python proto serialization performance so you can see how it does as message complexity increases:

https://github.com/theRealWardo/proto-bench

So yes, if your application has to send a lot of structured data around, you will end up paying a non-trivial serialization cost. On the other hand, if your application is sending really simple messages then I highly doubt proto serialization is the performance bottleneck you should focus on.

stevvooe• 9 years ago

Decent overview, but, remember that when evaluating something, projects change over time. Even better, you can be the change you want to see! Thus far, the grpc project has been fairly responsive in making solid changes, either through PRs or filing issues. ;)

Regarding the complaint about errors, there is already protocol support for structured error handling: https://godoc.org/google.golang.org/grpc/status. https://github.com/grpc/grpc-go/pull/1358 should make this easier to use. In practice, the provided error code set is very good, so give them a try before making things more complex.

Worst case, you can just stuff things in a header or trailer.

misterbowfinger• 9 years ago

Inefficient (textual representations aren’t optimal for networks)

REST APIs don't _have_ to be text-based, AFAIK. Why not just send/receive binary?

ehsankia• 9 years ago

Sure, but then you'd have to write a serializer/deserializer for your messages. You could also use an existing one like Bson but then why not just use Protobuf?

cshenton2• 9 years ago

Because bson, msgpack etc are self describing? It means a way lower barrier to entry.

morecoffee5• 9 years ago

For the same reason people use JSON. It's simpler, and there is a fast parser built into the browser. Even non browser clients have optimized JSON coders which is good enough.

That said, JSON is still slower and more bloated than Protobuf, and you get Protobuf for free when using gRPC.

tnolet• 9 years ago

The point about operations is a very, very valid constraint of REST. Easy and common stuff like, "run this thing in the background", "send of this one, ephemeral, message" are very unnatural. Maybe a hybrid / bastard child of REST and gRPC would be a good marriage of resource and operations modelling.

QuercusMax• 9 years ago

The Google Cloud standard for async operations via the google.longrunning.Operation service: https://github.com/googleapis/googleapis/blob/master/google/.... This should be usable either via REST/JSON or gRPC.

It's designed to be generic for any kind of async operation, and to be "mixed in" with your APIs. There are utilities for waiting for an operation to finish (via polling); in theory it should be possible to use some type of server-push to avoid polling, but I'm not sure if anybody's doing this.

(I'm an Alphabet employee who's working on APIs delivered via gRPC / REST/JSON.)

joneholland2• 9 years ago

You can add Expedia to the list of production grpc users. Our entire hotel pricing backend is grpc microservices written in Scala. Some of these services take > 100k TPS. We have found grpc to be extremely scaleable.

If that sounds interesting, I'm hiring engineers, shoot me a note at joholland at Expedia.com

morecoffee5• 9 years ago

Do you use the Java generated stubs or generate your own? There are several people using gRPC with Scala that could benefit from more idiomatic stubs.

joneholland2• 9 years ago

We use the java stubs. We briefly looked at some scala proto generators, but most times you dump the proto class into a rich scala type right away anyways.

gRPC in Production

Related Stories