Does Meteor Scale?

http://meteorhacks.com/does-meteor-scale.html

by arunoda • 13 years ago

48 35 13 years ago

meteorhacks.com

judah• 13 years ago

That was a difficult read. Poor English.

rglover• 13 years ago

Give the guy a break. He does a lot of great work and does his best with English (I believe he's based in Sri Lanka).

Tarang• 13 years ago

I think this is one of the best articles out there and its very understandable on the issues when it comes to scaling meteor.

brentjanderson• 13 years ago

Arunoda does make a lot of great contributions to the Meteor community. MongoDB oplog tailing was actually introduced by him earlier (http://meteorhacks.com/lets-scale-meteor.html). Although the core Meteor team probably had this on their radar, his implementation was the first publicly distributed, and he's one of the leading Meteor users pushing its use into production and reporting on how it goes.

ale55andro• 13 years ago

Well written piece actually. Perhaps you read something else.

cenhyperion• 13 years ago

Really? I had no issues reading it and completely understood the content.

jjamz• 13 years ago

A new version was pushed part of the way through the day. Editing wasn't finished when it first went out.

arunoda7• 13 years ago

That's what exactly happened and @jjamz was helping me on the editing.

(anyway, I really welcome the english errors you raised, but I've done a lot of articles with my bad english. I hope most of Meteor user's got something out from them. That's the the point I think we should consider)

sheetjs• 13 years ago

Is there any large scale production deployment of meteor?

arunoda7• 13 years ago

AFAIK there is no deployments with millions of users yet. But there are plenty of apps with few thousand users.

Start Here: http://www.meteor.com/gallery

bachback• 13 years ago

Thanks for that link. I very much agree. Meteor is going to be massive.

jggonz• 13 years ago

I've used meteor for several projects, and I have found it to be great for developing quickly, but it does have all these scaling issues that are usually fixed via 'unofficial hacks'. I can't wait until V 1.0 is out. I hope it comes with all the promised scaling tools.

Here's a 'very beta' analytics tool I've been working on that uses Meteor and MongoDB:

http://www.quikdash.com

Feel free to sign up and play with it and if you can, please give me some feedback on how I could make it better! Thanks!

JulianMorrison2• 13 years ago

Has anybody made a Meteor back end for rethinkdb yet? Because MongoDB is an immediate show-stopper.

arunoda7• 13 years ago

Check this: https://github.com/tuhinc/rethink-livedata

KaoruAoiShiho2• 13 years ago

What would the performance of this be since the oplog stuff is mongo specific?

reustle• 13 years ago

Is it an actual show stopper, or just some mental block you have personally?

JulianMorrison2• 13 years ago

Obviously I can only speak for myself, but, in my opinion MongoDB is broken at a deep fundamental design level because the clever technological metaphor it's built upon (mmap is cache, we can overwrite in RAM and let the OS handle writes!) is just mistaken. All its other problems - trashing data, hogging resources, excessive locking, poor performance - are derived from that fundamental error, and all the fixes consist of beating a gradual retreat from it, a process rather like incrementally evolving a brick into a F1 race car. One has to ask, wouldn't it be better to start off without the brick? And rethinkdb is basically what MongoDB is, from a user's perspective, but it never had the brick.

duaneb• 13 years ago

It's a major indicator of poor system design.

EDIT: To be clear, I'm not necessarily knocking MongoDB—people who know more about it than I would do a better job of evaluating that. But realistically a swappable DB backend isn't that difficult, and if it is, they should seriously reconsider how they are using the database.

saraid2162• 13 years ago

In before someone quotes Betteridge's Law.

quarterto• 13 years ago

https://news.ycombinator.com/item?id=5742893

saraid2162• 13 years ago

You ought to model your law after Godwin's in order to make it more true.

edem• 13 years ago

When I saw "Scale" I almost thought this is another Scala post.

kungfooguru• 13 years ago

Horizontal scaling is completely wrong.

fomb• 13 years ago

Wait. They aren't getting rid of sticky sessions?

alanning• 13 years ago

Meteor (and its DDP comm protocol) is stateful. Not quite sure what you mean?

arunoda7• 13 years ago

Sticky Sessions is not something bad. It's bit difficult to handle. This suites very well for Meteor's usecase.

KaoruAoiShiho2• 13 years ago

Any clue when Meteor is going to finish their implementation of smartcollections? It seems kind of odd that you have it done for months and months but they still haven't managed it...

arunoda7• 13 years ago

It will come with Meteor 1.0 (early next year). See a preview of meteor's oplog tailing https://groups.google.com/forum/#!topic/meteor-core/6GR1HKrN...

glasser• 13 years ago

Yep, as Arunoda says I'm actively working on it right now, currently hoping for an initial release this month.

I really appreciate the work Arunoda has done on smart collections; my implementation and his package have both learned from each other. That said, there are a number of subtle race conditions involved in synchronizing queries between the oplog and the main database, and working out all the little details as carefully as possible has taken time!

nemothekid• 13 years ago

Interesting, I didn't know the oplog was a documented feature available to users of mongo. Certainly an interesting way to be notified of changes.

arunoda7• 13 years ago

Oplog is a capped collection. We can tail capped collection and get new document as we write into it.

nickzoic• 13 years ago

It's a very useful feature, which can feed a cache layer, act as a kind of trigger mechanism or give you a (very limited) kind of transaction.

Does Meteor Scale?

Related Stories