HN user
I think you can go a long way just with example code. A simple (ideally single file, <300 lines) RPC server that does I/O in a thread, an example of building, reading and mutating a complex data structure with lots of lists and nesting and such, some kind of demo of kj and another server that has multiple event loops could be quite useful.
In my experience, RPC in gRPC was nicer to work with but Cap'n'Proto's RPC was faster (due to the format I believe).
Why was gRPC nicer to work with in my opinion?
- Comprehensive and readable documentation. Cap'n'Proto has a high level overview on the main page but the documentation is far from complete. For example regarding I/O, a core part of many RPC systems, "Function calls that do I/O must do so asynchronously, and must return a “promise” for the result." is said, then no explanation is given on how to implement a function that does it (all the examples assume an existing library returns kj promises and since that's custom, no such libraries exist). For serialization, how do I build a list of integers? The "Lists" section on the serialization page doesn't explain this at all, it only mentions the init method for lists of lists or lists of blobs. The gRPC docs on the other hand are quite good, with numerous easy to read tutorials and examples.
- Built in support for threading. An event loop is limited to the throughput of a single core and there'll always be a point at which you need either threading or multiple processes. Since I don't want to have a bunch of different copies of a particular thing in memory, it works better for me to have a bunch of threads with access to shared memory. Again, the docs say "While multiple threads are allowed, each thread must have its own event loop." but don't document how to do it (and particularly, do it while sharing a port). Plus, there's no suggestions on how to return a promise that's fulfilled by another thread (this was and still is a big problem for me, I still haven't figured it out). gRPC makes this dead simple by just using threads (or a queue system I haven't looked into much yet).
- No kj. I might be more okay with it if the documentation was better but at the moment it's near-nonexistent and wandering around the code base for ~30 mins just to figure out how to construct and return a promise was pretty painful.
Really, you can sum up most of my problems in one word: Documentation.
Oh and as you said, language support. The Python driver is really important to me yet seems unfinished.
Another downside in gRPC is that (de)serialization can be relatively slow. One of my pet projects is a tagging server that takes a list of tags and returns a list of objects that match, retrieved from RocksDB. I tested gRPC (proto and FlatBuffers) and Cap'n'Proto. For all of them, the process looked like:
Insert: Object comes in (built in RPC system for both), object is assigned ID (which is added before serialization), object is written to RocksDB, object is serialized and indexed.
Query: List of strings come in, strings are looked up in tag index, intersection of results is done, objects are retrieved from RocksDB, objects are deserialized, objects are added to list, objects are returned to client.
FlatBuffers was unfortunately a no-go since it doesn't permit modification of fields that haven't yet been set and it didn't have a sane way of making a copy (which would also be slow).
Protocol Buffers worked but the pure Python driver is incredibly slow and the C++ driver for Python was non-default and marked "experimental". I eventually adapted to it but it was still quite slow even on the server side. From memory a response with C++ client and server took ~9ms, 4-5ms of which were spent on serialization/deserialization.
Cap'n'Proto eventually won for me. The Python driver is unstable and memory usage soars like an eagle until Linux shoots it down with an OOM but the server was much faster. Typical response times were closer to 3ms, most of which was spent in RocksDB or the actual index lookup. A downside though was that Cap'n'Proto has its own built in and odd library for async stuff and doesn't really support threading.