Building Better Node.js Applications with RethinkDB

https://nodecraft.com/blog/dev/building-better-node-js-apps-with-rethinkdb

by CherryJimbo • 11 years ago

100 40 11 years ago

woah2• 11 years ago

So is rethinkdb what mongo was always meant to be?

girvo• 11 years ago

In my experience, yes, pretty much! It's a really amazing database, and it's baked-in web UI for management and dead-simple clustering is just perfect for the use cases I've slotted RethinkDB into.

netcraft2• 11 years ago

so are we saying that

    r.table('users').filter(function(user){return user('signup_date').gt(signupFilter);}).run(...

is running this filter in the database, not in node?

andrewmunsell• 11 years ago

It does run on the database server-- see the documentation for more information: http://rethinkdb.com/api/javascript/filter/

They key line(s) is/are:

Predicates to filter are evaluated on the server, and must use ReQL expressions. You cannot use standard JavaScript comparison operators such as ==, </> and ||/&&.

It's essentially equivalent to:

r.table('users').filter(r.row("signup_date").gt(signupFilter)).run(conn, callback);

netcraft2• 11 years ago

ah, that makes a bit more sense. Still impressive though. Thanks for the information.

korijn• 11 years ago

Am I correct in assuming that the implementation is similar to that of LINQ and ORMs like NHibernate or Entity Framework? Example: https://msdn.microsoft.com/en-us/data/jj573936.aspx

coffeemug3• 11 years ago

A small note -- you can use arbitrary javascript expressions with the `r.js` command (rethinkdb.com/api/javascript/js/).

falcolas• 11 years ago

over 70 million API requests a week.

Congratulations. However, if you're going to imply that traditional SQL databases are non-performant in comparison, you may want to do a bit more looking at the performance numbers both PostgreSQL and MySQL are capable of.

100+ million transactions per day is pretty easy to achieve with a traditional SQL DB. Also, you can do a lot of filtering in software specialized for those use cases, reducing the need to consume processing time in the Node.js V8 thread.

I don't want to knock RethinkDB here, just point out that there are stronger performance cases to be made.

nulltype2• 11 years ago

The filtering done in this article is actually run on the server, not the client.

CherryJimbo• 11 years ago

You're absolutely right - there's nothing stopping a standard MySQL setup pulling off the same stats, but it just wasn't what we were looking for, coupled with the various other decision points made in the post. RethinkDB's native ability to shard and scale is the key performance case to be made, in my opinion.

As mentioned by another user though, the filtering here is done entirely server-side (at RethinkDB's level), so the V8 thread performance isn't really a concern. https://news.ycombinator.com/item?id=9411738 for more info.

727374• 11 years ago

Yes, seconded. I read the article really interested in learning what level of capacity Rethink is capable of in the wild, but didn't get a great feel for that. It's not very helpful to frame database performance in operations per week. Peak load would be a better starting point.

shin_lao• 11 years ago

You are correct, our customers (we sell a key/value store) manage 1B entries / day and do north of 500 million requests per day.

70 millions requests / week is actually very weak for a NoSQL database and should be left out of the promotion material.

blazedd• 11 years ago

The filter you see is actually compiled and sent to the server. The client isn't doing any filtering. This is actually one of the more powerful features in RethinkDB, IMO. The query language mimics local javascript.

btrombley• 11 years ago

All of the benefits sound the same as MongoDB, which also does sharding and JavaScript queries, including arbitrary code in "callbacks" in map-reduce or $where clauses (though it's slow).

Can someone explain the benefit of RethinkDB over MongoDB?

nulltype2• 11 years ago

http://rethinkdb.com/docs/rethinkdb-vs-mongodb/

neumino• 11 years ago

Just one benefit is that it has joins, server-side joins.

coffeemug3• 11 years ago

Slava @ Rethink here.

RethinkDB is based on a fundamentally different architecture from MongoDB. Instead of polling for changes, the developer can tell RethinkDB to continuously push updated query results in realtime -- check out http://rethinkdb.com/faq/ for more details on this.

Just FYI, Rethink does sharding and JavaScript queries (including callbacks) -- http://rethinkdb.com/api/javascript/js/. The `js` command interops with every other command in RethinkDB and works really well.

atarian• 11 years ago

Seems like a great option for real-time on the backend. But what would you use to push real-time data to a client-side JS app?

jkarneges• 11 years ago

Lots of choices there: Faye, Socket.io, Pushpin, PubNub, etc.

I'm curious to see someone try integrating RethinkDB with Meteor.

Matthias247• 11 years ago

WAMP and SignalR also come to my mind

tehbeard• 11 years ago

Don't forget vanilla tech like WebSocket, EventSource or good old fashioned ajax.

hoggle• 11 years ago

I'm probably forever spoiled by Rails' ActiveRecord - has anybody here used http://thinky.io with Node/RethinkDB? I really like to define my objects in an OOP fashion with class/instance methods included and would prefer a convenience "ORM" library for that purpose.

habitue3• 11 years ago

Thinky is really top notch. Michel keeps it up to date and constantly adds new features to it. I highly recommend checking it out

__Joker2• 11 years ago

Just a shout out, I am not sure when they evaluated the different db's. But Postgres ships with hstore and json store for people who want to go schema-less.

woah2• 11 years ago

I'd rather use a database purpose built for what I'm using it for. I.e., if I'm using Postgres, I'll suck it up and plan a schema. Their json stuff strikes me as a second class citizen.

__Joker2• 11 years ago

I am curious, why you think postgres json stuff is second class citizen ?

ttty• 11 years ago

I can easily switch "RethinkDB" with "MongoDB" and the article would still be correct.

bovermyer• 11 years ago

Except for the part about real time notifications, which is RethinkDB's killer feature.

tfb• 11 years ago

How would RethinkDB's real-time capabilities compare to doing something like this with Mongo? (I'm genuinely curious about any shortcomings with the following implementation, besides scaling.)

    // plugins/replicate.js

    var replicator = require('replicator');

    module.exports = exports = function replicatePlugin (schema, options) {
      schema.pre('save', function (next) {
        // notify subscribers of save
        replicator.notify('save', this);
        next();
      });
    }



    // schemas/game.js

    var replicatePlugin = require('../plugins/replicate.js');
    var GameSchema = new Schema({ ... });
    GameSchema.plugin(replicatePlugin);

ebbv2• 11 years ago

Maybe it's just me but:

    var signupFilter = new Date();
    signupFilter.setDate(signupFilter.getDate()-30); // get a timestamp from 30 days ago
    r.table('users').filter(function(user){
         return user('signup_date').gt(signupFilter);
    }).run(function(err, user){
         // ... a cursor to stream through the data which can also be converted
         //  to an array using the toArray method
    });

Is a lot nastier than:

    db.query('SELECT * FROM users WHERE signup_date > CURDATE() - INTERVAL 30 DAY;', {}, function (err, users) {
        // Do stuff
    });

No?

filearts2• 11 years ago

I would agree that the latter is less 'nasty' in that it is a query written in a DSL designed to make querying tabular data more understandable. It would be a major failing of SQL if that were not the case.

However, what the former does that the latter does not is provide a mechanism to build up and modify queries. It can be very helpful to pass around and affect partial queries when you are looking to eliminate repetitive code for building multiple, similar queries.

This is probably why many abstraction layers have query builders that look a lot like the syntax that you call nasty to generate the 'less nasty' raw SQL.

ebbv2• 11 years ago

So, basically you're saying it's OK that the abstraction layer for RethinkDB creates nasty code because it's native JS? I don't agree. I think nasty code is never OK.

Why can't the RethinkDB abstraction layer just be better? Why isn't:

    var dateFilter = new Date();
    dateFilter = dateFilter.setDate(dateFilter.getDate() - 30);
    rdb.table('users').get([ { column: signup_date, gt: dateFilter } ]).each(function (err, user) {
        // Do stuff
    });

Possible? That's just off the top of my head how I would do it in a way that's native JS and far better than what's presented in the article.

mercurial2• 11 years ago

It's more verbose, but if you have had the chance of building SQL queries from raw strings, it's obvious that the first solution composes a lot better (eg, you can add a filter in a nice way).

mrits• 11 years ago

One of the best optimizations I have done was switch my caching from using a key-value serialization to an array form. I'm caching fairly large data reports about computer performance. It brought the average cache size from 100 megs to around 4 megs.

[{'first_name':'mike'},{'first_name':bill'},...]

{meta: {'header':['first_name']]}, data: [['mike'],['bill],...]};

This is why I can't consider databases like RethinkDB or Mongo. They are taking such a big hit already for using something like BSON as a storage that I don't even want to be involved in that train of thought anymore.

Jean-Philipe• 11 years ago

r.table('users').filter(function(user){ return user('signup_date').gt(signupFilter); }).run(function(err, user){

Whenever I see one of those "awesome" no-SQL queries, I can't help but think about how ugly and bulky they are compared to SQL.

Or compared knex.js for node:

select().from('users').where('signup_date', '>', signupFilter)

It supports promises. What's wrong with that?

Building Better Node.js Applications with RethinkDB

Related Stories