Friendly: NoSQL With MySQL in Ruby

http://jamesgolick.com/2009/12/16/introducing-friendly-nosql-with-mysql-in-ruby.html

by edwardog • 17 years ago

83 17 17 years ago

antirez2• 17 years ago

Even if NoSQL solutions are new, there are no data durability concerns in my opinion.

I can talk for Redis but my guess is that's the same for the other datastores. In Redis to have full durability enable append-only-file: it's a text-only file, you can check by hand what's written inside your database.

In addition even while running in append only file mode, you can still ask Redis to "BGSAVE" (crete a copy of the dataset into an .rdb file in background). It's a single file and you can save it everywhere.

With the append only file, it's trivial to write a script to export ALL YOUR DATA into MySQL every 24 hours for instance. So you have your data in:

a) compact .rdb file you can backup easily. b) the append only file c) MySQL, where you can perform query to check if the data is sane.

In addition in the next months Redis will get a JSON export feature built-in, so you can also d) save data into a JSON file.

This new products are still younger than MySQL, and there are more bugs probably, but it's just a matter of being a bit more careful to use this DBs in production without concerns, experiencing a huge performance gain.

p.s. since the release of Redis 1.0 no critical bugs were found.

jamesgolick2• 17 years ago

Actually, we do run redis in production, and it has been awesome for us. But, for this particular aspect of our application, we see about 20GB of data growth per week. So, an all-in-RAM db isn't really practical.

Either way, durability wasn't the issue for us. Mongodb has no write concurrency. So, when it got so overloaded with data that queries were taking forever, we had no way to purge older data without blocking all read & write operations for hours.

antirez2• 17 years ago

Ok sorry for assuming that the problem was durability / reliability.

If MongoDB has no write concurrency MySQL is simply better from this point of view for your use case as far as I understand, but I'm not very good at MongoDB.

Btw Redis will get Virtual Memory in a few months, I hope you can reconsider it later ;)

eliot_10gen2• 17 years ago

Did you try having multiple collections and then just dropping them?

eliot_10gen2• 17 years ago

Or if you want to only keep x amount of data around - thats what capped collections were designed for.

mdirolf• 17 years ago

This is fixed now - large deletes won't block the world forever: http://jira.mongodb.org/browse/SERVER-494

astine• 17 years ago

This reminds me of Elephant, a CL OODBMS that can use Postgres as a key/value store backend. OODBs, document stores, et al. are useful but using a complete RDBMS as a backend is both overkill and inefficient in my opinion.

smanek• 17 years ago

Yeah, there is a reason Elephant is nearly an order of magnitude faster when run on Berkeley DB.

aarongough• 17 years ago

Looks great! I remember seeing the article about FriendFeed's datbase setup and thinking how neat it was. Very nice to see a simple implementation of that idea become easily available.

Now I just have to find a project where I'll need to use it!

Tompy• 17 years ago

If only this was written in python...

A port anyone?

leej2• 17 years ago

PHP port too.

dpritchett• 17 years ago

FriendFeed's Tornado has a database.py on github, maybe it will get you started: http://github.com/facebook/tornado/tree/master/tornado/

apgwoz2• 17 years ago

database.py isn't really in the same realm, but it could certainly be modified and enhanced to support this abstraction.

apgwoz2• 17 years ago

Actually, friendly definitely "beat me to market", but I did start building this on top of sqlite3 a few weeks ago. My purpose was just for play--I was going to naively compare its performance to something like Tokyo Cabinet to see if matched at all. I can throw the code up somewhere, but it's neither complete, nor well thought out, merely a hack.

leej2• 17 years ago

how is the cpu trade-off like in this type of setting? one more point is that i dont know ruby and i'm curious how can he add an index from that column?

riffraff• 17 years ago

just to point it out, postgres now has a builtin hash data type. This sees like it could be a better solution thatn serialized json http://www.postgresql.org/docs/8.4/static/hstore.html

Friendly: NoSQL With MySQL in Ruby

Related Stories