NoSQL East 2009 day 2: Pig/Twitter, Cascading, Neo4j, Redis, Sherpa/Yahoo

http://journal.uggedal.com/nosql-east-2009---summary-of-day-2

by uggedal • 17 years ago

30 6 17 years ago

journal.uggedal.com

antirez3• 17 years ago

There is some problem with the numbers for Redis:

"Can do 19,600 gets and 13,900 sets a second on a MacBook Pro"

In my macbook (not PRO) Redis performs like this:

    % ./redis-benchmark -q
    SET: 34705.88 requests per second
    GET: 31055.90 requests per second
    INCR: 28739.25 requests per second
    LPUSH: 35013.98 requests per second
    LPOP: 30496.95 requests per second
    ^C

But redis sucks on Mac OS X compared to how it performs on Linux. The same macbook running Linux reaches almost 100k query/sec. An entry level server running Linux is in the 150k/sec zone.

Sorry but I spent some time in order to make Redis so fast, so to see numbers an order of magnitude less does not make me happy ;)

About the replication: Redis supports master-slave replication with very very fast first synchronization. The replication is non-blocking, this means that if you attach N slaves to the master it continues to reply to clients without troubles when synching with the slaves.

If the link between master and slaves goes down the two will resynchronize again automatically. It's possible to use a replica in order to enhance data durability.

Replication can be controlled at runtime. For instance if you want an instance to become a replica of another instance all you need to do is something like this:

    echo -e "slaveof 1.2.3.4 6379\r\n" | nc 1.1.1.1 6379

Final note about the snapshotting persistence mode, in Redis edge on git there is already support for append-only journal, that makes Redis an option even when data is very important.

lucifer2• 17 years ago

The benchmark is a C program. Do any of the clients come close to matching the benchmark?

antirez3• 17 years ago

Yes, it's just about parallelization.

If you meter the performance, even of a C client, in a busy loop, you are really measuring the round time trip, because it's a request-reply protocol, and most clients block until the reply is not ready.

Even using a Ruby / Python / ... client, if you run N of this clients, you'll see that Redis can handle this number of queries every second.

seancribbs• 17 years ago

Kevin did those benchmarks (as I watched) on a single MBP using the Ruby client, specifically Ezra Zygmuntowicz's benchmark script. I would definitely consider his numbers circumstantial, but also the bottom end of what's possible considering the fact that it's a laptop and the performance characteristics of Ruby.

NoSQL East 2009 day 2: Pig/Twitter, Cascading, Neo4j, Redis, Sherpa/Yahoo

Related Stories