Getting Meteor to 1.0

http://www.meteor.com/blog/2013/10/01/geoff-schmidt-at-devshop-8-getting-meteor-to-10

by yaliceme • 13 years ago

93 53 13 years ago

pdfcollect• 13 years ago

Any hope of writing/supporting meteor with python backend? :)

Sure! The Meteor client and the Meteor server speak a protocol called DDP, which runs over websockets or HTTP long polling. Meteor clients can connect to any DDP server and vice versa. So you just need to implement a DDP server for Python.

DDP is pretty simple (just a few messages) and the spec for the 'pre1' version of the protocol is in Git: https://github.com/meteor/meteor/blob/devel/packages/livedat...

This is pretty close to version '1' of the protocol that will be locked down in Meteor 1.0: https://trello.com/c/kMkw800Z/53-official-ddp-specification

1qaz2wsx3edc• 13 years ago

I love the DDP as a concept. It's akin to REST-on-steroids, over a websocket with pubsubhubbub sprinkled on top. I really wish more people knew about it, I hope drivers are adapted quickly (some have been). Interoperability & extensibility will make it a killer web feature.

sonnym3• 13 years ago

After seeing the first, extremely slick demo of meteor, I was, as I believe a lot of people were, extremely excited to see where the project would go. Then came the funding, and some twelve or sixteen months of development, and I finally decided to dive in.

I really wanted to like it. I was predisposed to do so.

But from the very beginning, I was confronted with something I consider a deal breaker. While I am wary and generally disinclined toward the increasingly popular pattern of curl an installation script and pipe into sh, in this case it is not only skeptical, it is downright ludicrous. Why would I, nay anyone, want to install an npm package this way? Should not the installation instructions simply be `npm install -g meteor`? And what if I need to work on multiple meteor projects with different versions? Surely there is support for adding it to your package.json file, but why is this not the primary means of installation and well documented?

Maybe I am being overly sensitive to these issues, but it really baffled me that the very foundation upon which a meteor project is predicated would be contradictory to the typical node workflow. I will probably try meteor to spike out a project at some point in time, but I do not foresee myself using it extensively in the near future.

misuba• 13 years ago

npm install -g meteorite

Then create a site with it and specify whatever local version of meteor you want.

The deets: https://github.com/oortcloud/meteorite/

geoffschmidt8• 13 years ago

Meteor version locking is in core now (as of 0.6.0), and the rest of Meteorite (fetching packages from Atmosphere) will be folded into core as part of 1.0 :)

flylib• 13 years ago

it's because your getting a full mongo database installed and configured with it

geoffschmidt8• 13 years ago

Meteor is a lot more than an npm package. For example, if you type these four commands..

$ curl https://install.meteor.com | /bin/sh; meteor create myapp; cd myapp; meteor

you're up and running with a complete stack including MongoDB, node, the dozen-ish core packages that make up the Meteor pubsub and realtime templating stack, and the 'meteor' build tool (which can do things like compile coffeescript and less, generate source maps, minify your production code, and provide a realtime development environment where whenever you save a file your app updates in your browser.)

As for versioning, it's actually got a great way of doing that. Each of your Meteor projects is locked to a specific Meteor release version (similar to an Ubuntu release, it's a release-engineered snapshot of the Meteor core packages), which you can set with the `meteor update` command. And this has automatic `npm shrinkwrap` integration so that if you use npm packages alongside Meteor packages, everyone on your team is always running exactly the same versions of all of the code in your app. The Meteor tools automatically handle all the work of downloading any needed Meteor versions and keeping them installed side-by-side on your laptop, using the correct one for a given project, and notifying you of available updates.

sonnym3• 13 years ago

I want to like meteor.

It sounds like the installation script, aside from handling the MongoDB and node setup, could be treated like a typical node package. Based on the fact that meteor has no package.json, seemingly none of those core packages are installed via npm either.

I may be alone in this, but I guess I would just prefer if my web framework expected me to have the interpreter and database installed, and used the package manager already provided for this environment.

sgdesign6• 13 years ago

Seems to me like you're throwing the baby out with the bath water. I personally think that's a mistake, but in any case if you don't want to run that command on your own machine, you can always try out Meteor with something like https://www.nitrous.io/

sonnym3• 13 years ago

It is not about the running of the command - I read the installation script, and it looks fine.

It is about my desire to work on a node application in what I consider to be a standard way. I want to be able to treat my entire application as a node package, leveraging npm for dependency resolution.

Think of it like a rails application. For local development, you probably use rbenv (or rvm), bundler, and have the version of rails you need listed in your Gemfile. You allow the package manager provided with the language and maybe a layer to keep environments separated (already provided by npm) to take care of dependency resolution and updates.

I want a node framework where I can use the package.json file as one does the Gemfile for ruby applications.

akbar5014• 13 years ago

Don't get too hung up on the installation. I think their point was to make setup/installation/getting excessively simple.

With regard to why not "npm install -g meteor", I think the short answer is that the Meteor installation setups up/installs a lot more than just one node package.

mr_luc2• 13 years ago

Thank you.

Saying that Node has a package manager is a huge understatement. Node has one hell of a package manager, but you have to hack[1] around Meteor to use npm modules, even in the latest version.

My impression of Meteor, and the impression a number of node people seem to share, and which I've seen reinforced on HN, is:

Meteor is in an almost adversarial relationship with the node.js open-source community, because while they're good hackers building something cool, they took millions of dollars in funding[2] and want to keep their options open for monetizing that codebase.

This makes every decision they make differently than other open-source real-time frameworks (to build their own package manager, require a contributor agreement on an MIT licensed project, and use a nonstandard install process[3,4]) considerably more worrisome.

And it's frustrating, because there's nothing going on in Meteor that necessitates going outside of the "node.js ecosystem" -- aka NPM [4,5,6].

That impression might be unfair in some way, and obviously the meteor people spend a lot of time responding to just these sort of concerns from the node.js community.

But "Meteor" means "worry" to me.

---

[1] http://meteorhacks.com/complete-npm-integration-for-meteor.h... [2] http://venturebeat.com/2012/07/25/meteor-funding/ [3] https://atmosphere.meteor.com/ [4] https://github.com/meteor/meteor/pull/516#issuecomment-12919... [5] http://derbyjs.com/ [6] https://github.com/substack/node-browserify

glasser• 13 years ago

I'm not sure what your problem with contributor agreements are; basically every sizable open source project has one. You know, like... Node: http://nodejs.org/cla.html

geoffschmidt8• 13 years ago

Well, all of Meteor is under a MIT license, so if we think we're going to "monetize that codebase" we're pretty stupid :) I'm not a lawyer but I think that under the MIT license there's basically nothing that we can do with the code that you can't do too. Our actual plan is to sell operational tools for larger companies that have mission-critical Meteor apps in production.

The incredible amount of work that we've put into JavaScript build tools over the last two years has all been with the aim of creating a radically easier, faster developer experience, because we know that that's incredibly powerful marketing for the rest of our crazy ideas. In other words, we did it because we thought that the UX of the existing JavaScript tooling was just too janky. Seriously, try Meteor for yourself and see what you think -- maybe you think our work sucks and that we wasted our time, but if so, I wish you would just tell me that (I would genuinely love to know) instead of reading ulterior motives into what was a labor of love and a ton of hard work.

Of course there are many rough edges and it's not done (that's why it's not 1.0 yet) but from people that have actually used it for a while, we actually get the opposite feedback, which is that they want us to go much further down this path, and that's why we continue to slave away at what is by far the least fun part of building a framework.

mr_luc2• 13 years ago

Hmm. I've actually changed my mind, (though not about their rejection of npm) -- after digging around, I see that there's been more communication about how Meteor plans to make money, and hearing that they have feasible plans that don't sound like a conflict of interest with open source is better than any technical argument for assuaging concerns.

But about me 'imputing bad motives', or whatever -- I said "worry", and "almost adversarial", and I stand by that.

When a company pops up and says "Hi, we're from Silicon Valley, and we have a plan to monetize a framework made from your language of choice; we have VC funding and we leap-frogged similar community frameworks -- now, if you want to write apps with the new hotness, please use our new package manager instead of the excellent community one, and don't install our framework with the community standard method either" ...

You worry. You just do. I think that meteor/derby/etc style app dev is the future, so I'm worried by anything that might threaten to lock it down.

I'm glad to see that upon investigation that worry is diminished, though. And obviously I applaud your efforts in advancing that future (I just wish that it were done in an 'npm install meteor' way).

rglover• 13 years ago

Thanks for posting this. Great overview of what to look forward to. As someone writing an app in production based on 0.6.5, should I plan on any big "rewrites" as the framework approaches 1.0? A better question might be, what will be backwards compatible and what will I have to scrap? It sounds like packages are the one area I should be the most careful with...

akbar5014• 13 years ago

There will be breaking changes as Meteor moves from 0.6.5.1 to 0.7 to 0.8 up until it reaches 1.0. This is part of the bargain of working with pre 1.0 software (APIs etc. will be stabilized once 1.0 is reached).

I have found these "breaks" are minor and usually require a few minutes to fix. That said, I also tend to lag behind by a few weeks as other in the community work through the details of the breaking changes.

Meteor Hacks (http://meteorhacks.com) and EventedMind (https://www.eventedmind.com) are two great resources.

barkingcat• 13 years ago

I'd love to see freebsd support for the install scripts. Last I looked support was in its infancy. Maybe I should revisit.

sgdesign6• 13 years ago

If you're curious about Meteor, I recently wrote a short introduction to it from the point of view of front-end JS developers: https://www.discovermeteor.com/2013/09/17/meteor-for-front-e...

akbar5014• 13 years ago

Congrats to the MDG team on having 1.0 within reach, and thanks for releasing an amazing platform.

Meteor is one of those platforms that you should try. I could give you a menu of features, but the productivity boost from Meteor is one of those things you really should experience. The productivity boost is very real and its quantifiable.

An experienced developer can get a very good feel for Meteor over a single weekend: 1. Buy the book: http://www.discovermeteor.com (the book is $39...I did not purchase the other packages) 2. Spend a weekend doing nothing but coding

Come Monday morning you'll know what Meteor is about.

PS: I am not part of MDG, and I have nothing to do with the book (other than having bought a copy).

puller• 13 years ago

When you say it is quantifiable - can you actually quantify it for us, or is this just hyperbole?

parasj3• 13 years ago

TL;DR: It's great for realtime apps, but not so much for the traditional web app as compared to something like RoR.

My perspective as someone who has been using it since its release. It depends on what kind of app you are building.

Coming from RoR, Meteor actually is more of a pain given that it still has edge cases that can eat developer time, especially once you dive past the initial levels of tutorials and documentation. RoR also has well established best practices and norms, unlike Meteor and perhaps Node.js as a whole. This means you will be writing a lot more boilerplate code. Meteor has some memory issues as well at times. You have to be aware of memory leaks and will frequent the profiler often, especially with large client-side applications. I've had some projects that I've worked on in Meteor hit 1GB of RAM client-side. Recent releases have been working on this, though. Live reload and hot code pushes are kind of a moot point with something like the Live Reload plugin for Sublime Text or equivalent.

For realtime applications, Meteor is a whole different beast. Realtime applications are much easier to code. I'm comparing this to making a realtime Node.js application. Meteor takes care of the boilerplate for you, pushing changes automatically to the client. It's not difficult to do something like socket.io and node.js but it is nice to have that all handled for you.

It's great for getting an application off the floor - it has a lot of that magic RoR had when it was released. Once applications get more advanced, the structure and maturity of RoR make it easier to work with. These issues are things Meteor is working on, especially in terms of stabilizing the spec/API and working on fixing the edge cases. With 1.0, it seems they are bringing some more stability to it by making it production ready. Its strengths are much more apparent in realtime applications as compared to a traditional web framework.

Edit: As compared to other node.js frameworks like Express, Meteor is much easier to work with - I probably would not go back to raw Express after Meteor. It's been moving fast to 1.0 where many of the issues I've had with it should be resolved with stability and as edge cases are resolved.

akbar5014• 13 years ago

Yes, we measured it. I am rounding these numbers for simplicity, but essentially, our split of developer time and KLOC follows:

# Labor and spend allocation

- 75% of engineers were on our server-side (Java) (1)

- 25% of engineers were on front-end (JavaScript/HTML/CSS)

# KLOC (2)

- 14% Java

- 54% JavaScript

- Everything else is in other (Scala, CSS, HTML, shell scripts, SQL, etc.)

From speaking with other companies, we found these ratios are within the normal range. We've seen 60% server-side to 80% server-side, but the directional finding appear to be actionable. As mentioned in Note 1 below, we excluded backend work from server-side work.

Also, and this is important, these metrics hold for our single page applications (SPA) projects. I do not have metrics for projects with a more traditional server-side only rendering...but I would guess that these finding do not hold true in such a situation.

If anyone else is willing to share their metrics I would be very interested. I'm always open to seeing the bigger picture in case there are some important facts that I have not factored into my decision making process.

# Notes

(1) This 75% is after segmenting server-side work (i.e. Java application server) from back-end processing (i.e. Kafka, Storm, Hadoop). Obviously, data processing will remain in Java, so we excluded it from our comparison.

(2) KLOC is a poor metric, but it was quick and dirty for us and provided good directional information. Also, all developers (front and back end) are good. So we did not need to adjust for quality of code (i.e. good developers have lower KLOC on average).

sgdesign6• 13 years ago

Co-author of Discover Meteor here, glad you liked the book!

Kequc• 13 years ago

Curious to learn about meteor but probably will end up needing an up to date book when I actually get around to programming with it. So if I purchase the $40 package, will that include updates up to maybe a year from now?

jclos• 13 years ago

I insta-bought the book after hearing the Changelog episode on it. Thank you for writing it.

parasj3• 13 years ago

I've been using Meteor since its release and its been great at making apps at maximum velocity.

You mentioned that some people are already scaling Meteor across multiple servers. Looking at it, round robin load balancing won't work given requests to a server seem to be stateful. How is that being done as of now? Do you guys have any guidance on how to do that?

geoffschmidt8• 13 years ago

Yes, if you're using DDP over sockjs instead of just over websockets (which Meteor currently does out of the box to support older browsers), you'll need to use a HTTP proxy with sticky session support.

This will be in the official docs at some point, but for now, Arunoda's blog post about this is a great starting point:

http://meteorhacks.com/load-balancing-your-meteor-app.html

joeblau• 13 years ago

I made a meteor app and it was one of the best experiences I've ever had making a web application. Even though what I made was fairly basic, the ease of creating a real-time application was amazing.

prottmann• 13 years ago

Hope you finish soon. I am excited than for an review of meteor and derby, i think that this are the two most exciting projects at the moment.

aphelion• 13 years ago

I'm very impressed by the technology behind Meteor, I just wish it weren't quite so "opinionated". It relies on standard parts of the node/js web stack, it would be great to have a stand-alone library that supported DDP for front-ends and servers of the developer's choosing.

mark_l_watson• 13 years ago

Earlier this year, I did an experiment: I implemented the design for a reasonably simple rich web app using two different stacks: Clojure on the server Clojurescript for the client, and the other being Meteor.

I have lots of Clojure experience but little Javascript experience. Still, the Meteor implementation took me half as long and had more features. That was a surprising result for me.

MrBra2• 13 years ago

Metheor got about something like 16M $ in funding, if I remember good, and they can't afford a proper microphone for the speaker? :)

MrBra2• 13 years ago

comeon, downvoters, don't you have a sense of humor? ... man...

geoffschmidt8• 13 years ago

Doing AV right is hard! You can devote your whole career to learning how to do AV well, and I have a ton of respect for the skills of the people that choose this path. It's a lot more than plugging in a microphone.

We do try to improve AV with every Devshop, and we have recently started to bring in AV professionals to help. But honestly, pre-1.0, I think it's way more appropriate for us to be spending our time and money on advancing the framework than on super slick video presentations. In many ways it's more like the Meteor community's time and money.

driverdan• 13 years ago

I've read through the documentation twice (once when it was released and again a week ago) and reviewed the basic demos. Meteor's front end structure seems like it's missing something. It has a data layer and a view / template layer but nothing else. No routing, controllers, etc.

Is there a good demo / tutorial that integrates these missing components, either from meteorite libs or well known projects like Angular or Backbone?

jacquesc• 13 years ago

I agree, it's missing a good routing system. iron-router is a 3rd party solution but I didn't have good luck with it. Seems this is functionality that is core to the framework (it's hard to build anything more than a tech demo without routing).

jumpman222• 13 years ago

Once you understand iron-router, which takes an hour or two of going through the documentation and trying examples, it really does everything you could ask a router to do. I think its worth a shot of trying it.

Getting Meteor to 1.0

Related Stories