Rb: A Redis parallelization toolkit for Python

http://blog.getsentry.com/2015/08/20/rb-redis-parallelization-toolkit.html

by zeeg • 11 years ago

54 7 11 years ago

blog.getsentry.com

the_mitsuhiko3• 11 years ago

Armin from Sentry here.

If you have any questions about the library or the whole problem in general feel free to shoot questions my way.

kevinastone• 11 years ago

Could you expand on why you didn't implement this using Futures?

the_mitsuhiko3• 11 years ago

Futures as a concept compared to Promises as a concept? I think the difference is largely naming though i assume one implies being read only from the outside? The promise API just mirrors ES6 so that it's familiar.

btown• 11 years ago

So why not just use:

    gevent.joinall([gevent.spawn(call_get, get_client_for_key(key), key) for key in keys])

gevent uses epoll/kqueue in the background as well; as far as I know, no threads are spawned or locked. Is there some kind of fancy scheduling going on in rb?

zeeg• 11 years ago

Forcing a dependency on people isn't always the friendliest thing, especially when it comes to version, especially in Python.

We don't use gevent anywhere else, and while I'm certain it has a better implementation, it's about the right tradeoffs.

the_mitsuhiko3• 11 years ago

We are not using gevent anywhere and we can't really because of it not working well for the majority of our work. The redis pyrhon library only works with gevent if you monkey patch everything and that would be very problematic for us.

Lastly that does not support pipelining still, something that rb does for you n

itamarhaber• 11 years ago

no questions, only kudos and one of my favorite items in the latest edition of Redis Watch (COOLEST. LOGO. EVER.)

Rb: A Redis parallelization toolkit for Python

Related Stories