695k TPS with Node.js and VoltDB

http://voltdb.com/company/blog/695k-tps-nodejs-and-voltdb

by dscape • 14 years ago

53 40 14 years ago

So voltdb is an in memory SQL Db with some durability and replication functionality. Is there any good documents on how it compares with some of its competitors/ who are its competitors?

huggyface• 14 years ago

Someone please correct me if I'm wrong, however I believe that VoltDB is highly tuned for a specific usage that benefits lots of small operations on large amounts of data. It is less beneficial, if not detrimental, for large operations on medium or large amounts of data.

AlisdairO• 14 years ago

You're entirely correct - VoltDB is very OLTP focused. Its predecessor, H-Store (decent paper here: http://www.vldb.org/pvldb/1/1454211.pdf ) ran on a single-thread-per-processor model that was very performant for short operations, but could obviously cause high latencies in the presence of long running operations. Presumably Volt runs on a similar model.

edit: ah, MichaelGG has explained it much better in another part of the comment thread!

ams6110• 14 years ago

Indeed, I'd never heard about it and found myself intrigued. After a bit of reading, it appears that VoltDB supports a very limited number of DDL and DML operations (essentially, you can create tables an views, and select, insert, update, and delete). There are a few basic aggregate functions like SUM and COUNT and no single-row functions. If you need to do anything at all complicated you're doing it in Java, or returning the results to the client program so it can do it.

VoltDB sound like a potentially great thing for an application that's really (and only) OLTP. But it's definitely not ready to replace Oracle, PostgreSQL, etc. for more general-purpose needs.

fredholahan4• 14 years ago

As an in-memory RDBMS, VoltDB competes directly against products like Oracle TimesTen and IBM's SolidDB. Comparatively, VoltDB's architecture is designed specifically to scale out on clusters of commodity servers (TimesTem and SolidDB are single-node datastores), giving Volt access to larger amounts of inexpensive main memory.

VoltDB also competes indirectly with NoSQL products like Cassandra, which achieve performance and scale primarily by sacrificing SQL and ACID transactions (Volt preserves both).

As an in-memory database, VoltDB offers durability through command logging and disaster recovery via a recently-introduced feature called database (WAN) replication.

Hope this helps.

MichaelGG7• 14 years ago

I think durability should not be defined in terms of actual backing hardware. VoltDBs transactions commit and stay committed. If you restore from a backup, just like any other database, you'll have data loss until the backup point. What if my hard drive crashes when it loses power? At that point I'm in the same position as when I lose RAM.

VoltDB is often compared to other in-memory NoSQL systems, although I'm not aware of any that actually offer cross-server transactions. VoltDB is best when you have a high volume OLTP workload and need to really scale beyond what traditional RDBMS can do.

Edit: Here's a benchmark using VoltDB as a key-value store: http://voltdb.com/company/blog/key-value-benchmarking

maratd• 14 years ago

> What if my hard drive crashes when it loses power? At that point I'm in the same position as when I lose RAM.

Oh sure, except that the probability of losing your hard drive when you lose power is about 1 in a very big number. Whereas, the probability of losing the contents of your RAM when you lose power is 100%.

Now, MySQL (and in fact, most RDBMS outfits) provide tools for fixing corrupt data. But you actually need corrupt data to fix. That's the entire point of a hard drive.

I'm sure VoltDB would be perfect for a chat application or something of that nature. It would be terrible for anything that involved the transfer of $$$ (banking, ecommerce, etc).

aaronblohowiak• 14 years ago

VoltDB maintains a total perfect ordering to transactions as well.

tferris2• 14 years ago

Sounds nice. Some questions:

- I assume you did the tests with direct SQL statements without any ORM? How would an ORM affect the test's performance (with an Node.JS-ORM of your choice, e.g. sequelize)?

- How does Node+VoltDB compare to i.e. Node+PostgreSQL or Node+MySQL

- What does in memory SQL exactly mean? How is data made persistent?

tferris2• 14 years ago

Just read this on VoltDB's site:

"Third is NewSQL, which obtains better performance and scale in the only way possible – with a totally new architecture. NewSQL delivers high performance and scalability while preserving the traditional ACID functions needed for OLTP. NewSQL systems like VoltDB deliver better throughput than NoSQL without requiring transactions to be supported in user code. NewSQL systems also preserve the high-level language capabilities of SQL."

How does this work?

MichaelGG7• 14 years ago

Read the H-Store publications[1], which served as the starting point for Volt.

Basically, by staying in memory and eliminating locking, you get awesome performance. However, why should you have to give up SQL, tables, and transactions? Those things make developing apps easy. So why not get "NoSQL-like" performance, but keeping the good parts of "SQL"?

1: http://hstore.cs.brown.edu/publications/

MichaelGG7• 14 years ago

VoltDB doesn't really support ad-hoc SQL. It can execute ad-hoc by flipping a flag, but performance is achieved by using stored procedures. You write the sprocs in Java, and their compiler looks at the SQL statements and optimizes.

VoltDB smokes other RDBMS (Postgres, MySQL, MSSQL, etc.) for certain OLTP workloads because it stays purely in-memory, but removes all locking. Normal databases spend a huge amount of time dealing with locks, even if the memory is in RAM. VoltDB was based on the H-Store project; you can read more about the ideas at [1].

Making transactions execute serially[2] adds latency to an individual transaction, but the point with VoltDB is to send tons of transactions so that there's always work queued up. You should be able to get average latency of, say, 10ms for the type of app mentioned here. I'm surprised it's not mentioned in the blog post. (Maybe on EC2, the latency numbers were less-than-ideal, but throughput was fine.)

SQL and ACID have nothing to do with the backing store. Using RAM is just as valid as using disk. Practically, with VoltDB you're going to set the "k factor" - number of replicas you want. This allows you to lose k servers without suffering data loss (put batteries on the servers; that should be pretty good). There's a snapshot backup feature, and the enterprise edition has a disk logger to further minimize the loss window in case of catastrophic failure.

1: http://hstore.cs.brown.edu/publications/ 2: More or less. Servers are sync'd via NTP, allowing one to know when it's safe to run a specific transaction. There's also some ways Volt might know when one tx is independent and safe to run before another.

jhugg4• 14 years ago

To update what you've said with insider info, ad-hoc SQL performance is currently a big focus at VoltDB. We fully support ad-hoc access in auto-commit mode through our native clients or through JDBC. Currently, we can do hundreds or thousands of statements/sec, and we expect performance to increase dramatically in the next few months.

Here's the Epic in JIRA: https://issues.voltdb.com/secure/IssueNavigator.jspa?reset=t...

Typically, applications are not 100% ad-hoc on VoltDB, but we're starting to see more hybrid use cases. Stored procedures can be used for extreme throughput or for transactional requirements, but other parts of the app can be more flexible.

pron2• 14 years ago

Well, it removes all locking and achieves scalability only if you partition (shard) the data and don't have cross-partition transactions.

fredholahan4• 14 years ago

A few answers:

Yes, the tests were run without an ORM. Although VoltDB is accessible via JDBC, best throughput is achieved with parameterized SQL embedded in Java stored procedures. That's how the Node.js benchmark app (Voter) was implemented.

Node+Volt will be much faster than Node+Postgres or Node+MySQL for many OLTP-style workloads. So it's a great fit for apps like financial trading, digital advertising, online gaming and network monitoring - each of which requires super-high throughput writes (at single ms latencies) and fast, simple analytics. Node+Postgres or Node+MySQL would be a better fit for general purpose database workloads (e.g., with large-grained reporting).

In-memory means that Volt stores all data in main memory. Durability is achieved through the use of a highly innovative command log, which logs all transaction invocations to persistent storage. You can tune this feature to log synchronously (100% guaranteed durability at reduced transaction latency) or asynchronously (slightly more "lossy", but less of a latency impact). VoltDB also recently released a full database replication feature that supports WAN replication for disaster recovery.

I hope these responses are in some way helpful.

MichaelGG7• 14 years ago

What were the latency statistics? How good of time sync were you able to get the EC2 instances?

MichaelGG7• 14 years ago

I'd like to add an independent voice too. I tried VoltDB out around a year ago. Ran it on 3 servers, with k=1 (all data replicated on 2 nodes). Servers were Dell PowerEdge 860 - only 8GB RAM, and a Q6600 quad-core processor. They cost maybe $500 or so. Even with a low-end buildout, I was able to hit around 110k TPS. This wasn't one of VoltDB's sample apps; it was a small proof-of-concept for a telecom application. They don't seem to be riding the hype train.

As far as open source: I was using the .NET client, and fixed a few issues with it. The folks at VoltDB were really responsive about working with me and accepting my patches. They also took time to explain the inner workings, sorta give me a launching point to investigating the core code. Pretty good experience for a commercial open source project.

signa11• 14 years ago

> it was a small proof-of-concept for a telecom application.

would you describe what telecom application this was ? was it billing, ocs, pcrf or something else ? thanks !

MichaelGG7• 14 years ago

Yes, very basic realtime billing/CDR storage. With traditional DBs, I have to resort to batching balance: Read N messages from queue, compute overall balance updates, then apply at once. With VoltDB, I can just fire off records as they come in.

amouat• 14 years ago

Is EC2 really the right way to do these tests? I would have thought it would introduce unwanted variables related to network congestion and server load caused by other EC2 users.

jhugg4• 14 years ago

You might get a better number on bare metal, and you certainly won't get a worse one, so in a sense it's not ideal.

On the other hand, a big chunk of the web is run from AWS. Demonstrating performance there implies performance in many places. It's also an easily reproducible environment, which is helpful for benchmarking.

fredholahan4• 14 years ago

Fair question. We had some reservations about using EC2 for the reasons you mention. But we wanted to avoid any perceptions that we'd somehow cooked the books, so decided to run the tests on "neutral" gear.

If you look at the report details, you'll see some of the lumpiness your question suggests, particularly related to the network. The Node instances were beginning to starve when database server CPU utilization was still pretty low.

We would certainly have gotten better numbers if we ran these benchmarks on bare metal or a closely-tended cloud infrastructure. But we're a pretty low-BS group, so we'll stand behind what EC2 gave us.

salimmadjd• 14 years ago

looks very interesting. If I architect a tracking system again, I'd look at this setup.

olalonde• 14 years ago

> that’s beneficial to the scaling architectures of both Node.js

That's not very accurate (unless you run multiple Node.js processes). See http://stackoverflow.com/questions/2387724/node-js-on-multi-...

jqueryin• 14 years ago

    TITLE DISCLAIMER: 695k using 20 fairly large machines

I was, at first glance, pretty impressed with the 695k TPS title claim. Upon reading the article, it became apparent that this was based on running 8 8-core EC2 m2.4xlarge instances for node.js in addition to 12 8-core EC2 m2.4xlarge instances for VoltDB. In terms of server architecture, this is quite a large setup.

Just for an idea of how big of a machine we're talking for each of these instances:

    High-Memory Quadruple Extra Large Instance
    68.4 GB of memory
    26 EC2 Compute Units (8 virtual cores with 3.25 EC2 Compute Units each)
    1690 GB of instance storage
    64-bit platform
    I/O Performance: High
    API name: m2.4xlarge
    $1.800 an hour running Linux on demand
    $864.00 a day baseline for 20 of these

This isn't meant to discredit the author or work, just to make sure it's apparent that this solution is for serious businesses only with deep pockets and boatloads of traffic. You'd be spending $26,784 for the servers alone over a 31 day period if using them on demand.

The results are definitely impressive for the right target audience.

stock_toaster• 14 years ago

Also near the bottom.

  > On the database side, I configured VoltDB to run at K=0,
  > which means the database was running with HA switched off.
  > Most applications would not run that way in a production
  > environment, but my goal was to test client-side throughput
  > and scaling; I wasn’t really focused the database side of
  > the tests.

Testament to the author for mentioning it. It makes me interested in voltdb, whereas before it was unknown to me. But the numbers should indeed be taken with a grain of salt I think.

spullara• 14 years ago

I'm not sure if I am supposed to be impressed or not. On my macbook pro with 1 redis master and 1 redis slave I can do about 300k updates / second to the master replicated to the slave. Comparable? Who knows?

These benchmarks don't mean much to me -- what matters is performance for your application.

arielweisberg• 14 years ago

I'll start by saying I dig Redis. I wish we had a storage engine that offered some of the kinds of flexibility that Redis does. If I were making Volt from scratch I would embed Redis or something with flexible schema and secondary indexes and spend less time working on SQL and more time working on distributed functionality.

>These benchmarks don't mean much to me -- what matters is performance for your application.

Very true. Think about how you would implement an application like Voter in Redis. Voter is an event counting app that applies business logic server side to do what is basically fraud detection.

Let's take a look at what each of these transactions is. https://github.com/VoltDB/voltdb/blob/master/examples/voter/... 3 statements and an entire round trip to validate data is kosher. 1 insert/update statement that is actually updating two different summary views

You would need to implement your own locking to get the isolation necessary to implement the business logic and there would be at least two round trips for each vote. You would also have to write your own code to maintain the materialized view used to print the summary. Sure you could implement the validation logic client side, but then you lose the ability to transactionally update said logic and the state it depends on.

With the new scripting functionality in Redis you would be able to do it in one round trip but you would still end up writing your own code for the view and rollback. How you will attach a debugger to debug your server side logic is also worth considering. I mistakenly wrote that Redis doesn't cache Lua scripts, but it actually does.

Now that you have server side logic you will also probably want to start maintaining replicated state to make some of your transactions single shard transactions. With Redis you are on your own for updating replicated state in a way that stays consistent across the cluster even when restoring different shards.

Also, no matter how fast Redis is you may reach a point where you have run out of disk/network IO, or even CPU. Now you are looking at sharding Redis. You are probably going to want to maintain replicated state to make more of your transactions single shard transactions. With Redis you are on your own for updating replicated state in a way that stays consistent across the cluster even when restoring different shards.

Don't forget re-sharding and backups which are now your problem. You definitely want to be able to roll back to before that last application update that corrupted your data. Don't forget that you will need to bring the replicas up in the correct state as well. Preferably in parallel with bringing up the master since you are down right now, and it would be nice if it resharded at the same time since you took this opportunity to add capacity. Redis cluster is coming, but it isn't here today and depending on the application you may not want to risk it not being ready when the time comes.

You should at least double the number of ops (where an op is roundtrip to the DB) done by Volt if not more. That said, doing half as much work inside the procedure doesn't make it twice as fast and doing twice as much doesn't make it twice as slow. The dominant cost for such a trivial operation is not the operation itself. Redis is much better at this because it is single threaded and written in C. My bet is that the number of instructions and cache misses to accept and execute a command in Redis is much smaller because the execution engine isn't dealing with stored procedures, schema, undo logging, SQL, and code expecting to deal with distributed transactions even if none are in play.

>I'm not sure if I am supposed to be impressed or not.

If you haven't tl;dr-ed already I'd call it good.

brn52• 14 years ago

The benchmark was very much about whether you CAN scale if you need it. You'd probably not rent this size hardware at Amazon if you needed it on a daily basis. So $26,784 is probably not realistic.

At any rate, as I mention in my longer report on the benchmark [1] the high total translates into very LOW hardware requirements for lesser needs. The number to go by may rather be "10k+ TPS per core for linear scaling ACID transactions." I think that's the number that matters. Linear 10k+/core ACID TPS.

[1] http://community.voltdb.com/sites/default/files/NodejsBenchm...

purplefox• 14 years ago

If you think node.js is fast try http://vertx.io.

It's like node but isn't single threaded so scales over multiple cores without having to fork. Also it's polyglot so you can don't have to use JS if you don't want.

I'm hoping to publish some performance results vs node.js before our 1.0.final release in the next few weeks.

kolev• 14 years ago

How does vert.x compare to to Apache Deft?

695k TPS with Node.js and VoltDB

Related Stories