VoltDB - in memory, ACID compliant, partitioned, SQL database

http://www.voltdb.com/product

by bensummers • 16 years ago

60 41 16 years ago

I guess in memory means that it never writes data to the disks. So what happens when the machine reboots?

cx015• 16 years ago

From the FAQ: "Durability: VoltDB provides both replication of partitions (known as K-safety) and periodic database snapshots to ensure the availability of the data."

mmagin• 16 years ago

Their FAQ states: Durability: VoltDB provides both replication of partitions (known as K-safety) and periodic database snapshots to ensure the availability of the data.

I suppose it depends on your definition of "D". Seems like this might be acceptable if you had different replicas on different power sources, etc.

jws• 16 years ago

I expect there is a persistent store of some sort, you'll have to register to find out. I'm also unable to find a license without registering.

It does appear from the FAQ that you build your transactions as Java stored procedures. Constraining a generic SQL database that way will make it much easier to distribute and scale.

tlack• 16 years ago

In-memory DBs can take a lot of excellent shortcuts but I have to wonder if their usability is limited. After all, I'd think most interesting data sets end up quickly eclipsing the available RAM in most off the shelf/affordable servers.

cx015• 16 years ago

Keep in mind that VoltDB targets the OLTP market, where datasets tend to be not that large. If you build a small cluster of 50 nodes with 256GB RAM each, you have about 4.3TB of storage (with a replication factor of 3).

st3fan• 16 years ago

Have to fill in a form to see a whitepaper. Yeah whatever.

jbellis• 16 years ago

It's based on the work discussed in http://cs-www.cs.yale.edu/homes/dna/papers/hstore-cc.pdf

Pahalial• 16 years ago

They're just gauging interest and making sure to get emails they can push adoption to later, it's a classic sales technique. Don't hate.

Regardless, the links they send are open:

VoltDB product overview: http://www.voltdb.com/_pdf/VoltDBOverview.pdf

VoltDB Technical Architecture white paper: http://www.voltdb.com/_pdf/VoltDBTechnicalOverviewWhitePaper...

xtacy2• 16 years ago

I don't understand how they could avoid locks. They say that each replica runs independently and hence that avoids locking; what about write updates to the same object from different clients? That would need locking of some form at some level.

They should also explicitly discuss the failure models for which they're guaranteeing Durability. If you're scaling to "web-scale", and running this db in a data-centre, then, a single failure would wipe out a rack of machines. Coordinated failures are not uncommon in data-centres. What about those?

itistoday• 16 years ago

From their whitepaper:

"Conventional databases experience disk and user stalls within transactions. Rather than let the CPU be idle during the stalls, those DBMSs interleave SQL execution from multiple transactions during the waits so the CPU is always busy. This is what requires much of the complex latching and locking overhead.

VoltDB doesn’t experience user stalls (since transactions happen within stored procedures) or disk stalls (because VoltDB processes data in main memory). Therefore, it is able to eliminate the overhead associated with multi-threading (latching) and locking. Each VoltDB execution engine is single-threaded and contains a queue of transaction requests, which it executes sequentially—and exclusively—against its data. Elimination of stalls and the need for locking and latching overhead and allows typical VoltDB SQL operations to complete in microseconds.

For single-partition transactions, each VoltDB engine operates autonomously. For multi-partition transactions, one engine distributes and coordinates work plans for the other engines. VoltDB assumes that an application designer can construct a partitioning/cloning scheme and a transaction design that makes a large majority of the transactions local to a single virtual node. Many common applications such as order-fulfillment, software as a service (SaaS), Web 2.0 and trading systems have this property."

http://www.voltdb.com/_pdf/VoltDBTechnicalOverviewWhitePaper...

stingraycharles• 16 years ago

That still doesn't completely answer the question how it avoids race conditions and the likes without locking. Assume I'm executing multiple long-running transactions, each hitting and modifying a lot (millions) of different objects. How are race conditions avoided in this case ? One engine that distributes and coordinates work plans for the other engines sounds nice, but does this essentially mean that only one of these transactions is able to run at the same time, even when they could be running concurrently when using locking?

stcredzero4• 16 years ago

It's Prevayler all over again!

http://www.prevayler.org/

(Not really. Prevayler uses a log.)

cx015• 16 years ago

It seems that it's single-threaded, so all writes will be serialized, hence no locking.

marcua• 16 years ago

You avoid locks by serializing transactions at any site. Since you're not waiting on disk (in memory DB) and each partition runs on its own block of memory and has its own cpu/thread, you simply don't let two transactions on the same partition run concurrently.

See http://cs-www.cs.yale.edu/homes/dna/papers/hstore-cc.pdf for cases where you want to run two transactions in the same location concurrently in h-store (the academic precursor to VoltDB).

xtacy2• 16 years ago

I see; what about cases where transactions involve objects at multiple servers? I guess I am confusing server specific locks vs client specific locks.

adamilardi• 16 years ago

SQL doesn't scale didn't they get the memo...lol

superjared• 16 years ago

Early adopter signup gives access to the source, which is licensed under GPLv3. However, there is a binding Beta agreement which states that I may not redistribute the code/application.

The VoltDB software is being published as open source software under the Gnu Public License V3. In other words, as Early Release program members, you now have access to the source files for VoltDB and the development tools for browsing the source code and the current issues lists.

Please note, however, that the Beta agreement is still in effect until VoltDB is officially available. So we ask that you not redistribute the source or binaries beyond your own use at this time.

Is this legally enforceable?

lsb• 16 years ago

Section 7:

All other non-permissive additional terms are considered “further restrictions” within the meaning of section 10. If the Program as you received it, or any part of it, contains a notice stating that it is governed by this License along with a term that is a further restriction, you may remove that term. If a license document contains a further restriction but permits relicensing or conveying under this License, you may add to a covered work material governed by the terms of that license document, provided that the further restriction does not survive such relicensing or conveying.

I don't think it's enforceable, but IANAL.

Scriptor• 16 years ago

What prevents them from just using the parts of the GPL they want?

silas• 16 years ago

http://www.gnu.org/licenses/gpl-faq.html#DoesTheGPLAllowNDA

carbocation• 16 years ago

From that FAQ: "If someone asks you to sign an NDA for receiving GPL-covered software copyrighted by the FSF, please inform us immediately by writing to license-violation@fsf.org."

khafra• 16 years ago

Looks to me like it's carefully worded to be a non-binding NDA--they "ask" that you not redistribute the source, but they don't provide the code solely upon receipt of that promise, or threaten any penalties for it.

As far as I know, it's both legal and meaningless to ask for anything you like.

sad• 16 years ago

Legally, probably not. They aren't demanding that you not redistribute, just asking. And asking politely. They are banking on the good nature of people. Good for them! And us.

pinko2• 16 years ago

Exactly. The Condor Project did this for many years and, miraculously, every one of the hundreds of users to whom we gave the source respected the request and the source never leaked. (It is now fully open.)

cabalamat• 16 years ago

If they don't want people to redistribute the code, why on earth are they GPL'ing it?

jfager• 16 years ago

Probably not, but if people start disrespecting their wishes, and the code is all their own (that is, does not include other parties' GPL'd code), then they can simply change the license and get the same effect.

ck2• 16 years ago

So, how fast does it run ALTER ?

sovande• 16 years ago

GPLv3, wonder what their business model is.

VoltDB - in memory, ACID compliant, partitioned, SQL database

Related Stories