Ask HN: GRPC vs. 0mq/nanomsg?

https://news.ycombinator.com/item?id=16617748

by baybal2 • 8 years ago

15 8 8 years ago

news.ycombinator.com

My intention is to have few load balanced NGINX servers with DPDK patches to do further request routing within the infrastructure.

With recent addition of gRPC to nginx, I thought of foregoing having an additional service running on the same machine as nginx to translate public API requests to our internal mq based services.

The bad thing is that gRPC is HTTP/TCP based, while our internal services were all zero copy/IP-less/RDMA based and our engineering has grown used to do synchronous processes across the rack.

Is it a worthy undertaking to write an own gRPC transport that can work on top of some RDMA protocol like SDP?

Or am I better staying with double layer routing: nginx --UWSGI--> script translating http requests to mq formats --SDP--> internal services

gdamore3• 8 years ago

Latest NNG (nanomsg-next-gen) has the ability to support HTTP and SP protocol in the same application. You have to write your own code to map the two and add business logic, but it should be pretty straight-forward.

NNG doesn't have an RDMA style transport today, although that can be developed. (Staysail Systems offers commercial support of NNG and nanomsg, including custom development.)

Ultimately, there are lots of ways to solve this, but you need to have a better understanding of your particular needs -- the fact that you're using DPDK and RDMA suggests to me that latency and performance are very important to you, and some more assessment would be required to make a good recommendation.

If you want, you can reach out to me -- I'm the author of NNG and now maintainer of nanomsg. I'm also the CTO for Staysail, so if you need commercial help, I can offer that. (See www.staystail.tech or info@staysail.tech for contact info.)

gdamore3• 8 years ago

Oh, btw, NNG has pretty good docs now. See the reference manual on the site ( nanomsg.github.io/nng/man ) or leanpub (purchase - https://leanpub.com/nngmanual )

baybal23• 8 years ago

the fact that you're using DPDK and RDMA suggests to me that latency and performance are very important to you, and some more assessment would be required to make a good recommendation.

A company preparing to handle wire speed services for millions of Internet toasters, fridges and such.

Stuff like high frequency logging, handling millions of active TCP connections on a single endpoint, faster than realtime media recoding/recognition, wire speed object storage, distributed search and indexing.

Effectively, the engineering task is to squeeze Amazon DC into a single rack or two that can be shipped to a colo.

We are a Chinese company, so I doubt we will ever be hiring engineering cadres or external contractors at anywhere near American wages.

gdamore3• 8 years ago

With the tasks you're trying to achieve, you should have both the budget and the ability to hire competent engineers anywhere in the world. Maybe not to hire hundreds of them, or keep them employed indefinitely, but a few dollars (or yen or whatever) invested wisely at the beginning can spell the difference between success and failure, or between having a model that works efficiently and profitably, and one that has to be re-engineered and costs many many times what would have been invested up front.

maerF0x0• 8 years ago

Because you mentioned grpc + nginx ... https://www.nginx.com/blog/nginx-1-13-10-grpc/

Ask HN: GRPC vs. 0mq/nanomsg?

Related Stories