Is Redis a noSQL db, a cache, or a messaging server?

http://antirez.com/post/redis-weekly-update-3-publish-submit.html

by akkartik • 16 years ago

59 22 16 years ago

sketerpot• 16 years ago

This needs repeating:

> This is how it is possible to write modular, scalable applications today: you need a very fast "global state" (the DB side), caching, ways to create queues of jobs as soon or later you need to process things in the background, and sometimes you need the ability to pass messages around in a pub/sub many-to-many scenario. And this things are converging, they are just different sides of the same things, at least when the DB part is designed as it is in Redis.

mcs• 16 years ago

With Redis, memcached finally has a contender for most awesome cache... thing. However I'm surprised I don't see more benchmarks. Performance should be roughly the same unless there's huge I/O contention for persistent storing.

dlsspy4• 16 years ago

I'm a bit biased, but I'm not as optimistic as some.

memcached had a design similar to redis when I first got involved with it a few years ago. It was fine for a lot of systems, but didn't scale to the larger ones. It took a lot of work for us to get it to the point where it could saturate all of the cores you could give it.

If anyone can ever produce a benchmark where memcached does not perform better than redis, please file a bug in memcached, or at the very least, start a conversation.

The closest I've seen were tests that degraded to client tests. In one case, someone took a very poorly performing memcached client and compared it against a very well-tuned redis client and a couple of relational databases and showed that memcached was the slowest of all. sigh

In most of these cases, the test environments perform at the very least an order of magnitude slower than what I get on my development laptop (I have no trouble sustaining 90k writes per second with either of a simple c++ thing I threw together yesterday or my high level java client).

On real production-quality machines with good networks, 300-400k ops per second isn't hard to achieve. Even between two old "large" (2 core, 7GB) EC2 nodes, I've been doing 90k ops per second all day (which is somewhere around the point where they start to throttle my bandwidth).

wanderr2• 16 years ago

Redis isn't a direct competitor to memcached. The great thing about Redis (disk-backed cache) is actually a liability if you're using it like an L2 cache which is how memcached is intended to be used.

To be clear: in memcached you can effectively set-and-forget keys and never worry about running out of memory. It uses an LRU to figure out what to throw away when memory gets full. In Redis if you want to achieve that behavior you need to expire your keys, which is another round trip every time. That means expire alone is also not enough, because you could set the key successfully but fail to set the expire. That means you need to also keep track of all your keys and periodically do cleanup.

Redis is awesome, but I would definitely not use it to replace memcached except in the cases where I absolutely need the data to survive a reboot and am willing to go through the hassle of manually managing the cleanup of every key.

mmaunder• 16 years ago

It's not a cache because memcache is already all the cache we need. Not sure why you mention messaging server unless you're expecting listeners to poll the server instead of get told about events. So I think it's a noSQL DB.

For my practical uses I'm most interested in Redis as a backing store for distributed persistent redundant key-value stores like:

http://kumofs.sourceforge.net/ http://opensource.plurk.com/LightCloud/

And my requirements are: 1. Speed. Gobs of it. Ridiculous amounts of speed. 2. Data must get written to disk periodically and I don't care if you lose a few minutes worth when I lose a disk. I'd like some control over how much data I'm prepared to lose vs speed. 3. O(log N) scalability. 4. More interesting ways to sort, extract, modify and delete data stored in a key or multiple keys without impacting performance.

Thanks Salvatore for your hard work!!

pashields• 16 years ago

Just to clarify, there is no polling in redis blocking list ops. I believe the pub/sub doesn't require polling either. Given that, I can think of a lot cases where Redis will work fine as a messaging server. If you need a very sophisticated routing mechanism, you will need to look at a more dedicated solution. But if you can express the message routing in either the rpush/blpop style or in redis pub/sub, why not?

moe2• 16 years ago

We've found the primitives that redis gave us to be sufficient even for a fairly complex routing design.

Rabbitmq (which we used previously) might still have an edge for very complex scenarios, but AMQP is generally a royal pain and we've had serious issues with rabbitmq reliability under load (and debugging the problems thereafter!).

Consequently we switched our hairy mix of bindings to a straightforward "LPUSH to everyone whom it may concern" and wouldn't look back.

That may be less efficient in theory, but the machine-load (i/o, CPU) went from "uh oh" to "barely measurable" for us. Even more importantly we now feel to have a much better grip on the system because if something goes wrong then we'll know where to look. In contrast rabbit was effectively a blackbox, which turned into a huge problem once it ceased to "just work".

jasonwatkinspdx2• 16 years ago

"It's not a cache because memcache is already all the cache we need."

That's quite the display of logical thought there. Rottweilers aren't dogs because black labs are all the dog we need.

Is Redis a noSQL db, a cache, or a messaging server?

Related Stories