Partychat — migrating from Google App Engine to EC2

http://www.vijayp.ca/blog/?p=162

by BarkMore • 15 years ago

57 12 15 years ago

vijayp.ca

hello_moto• 15 years ago

Seems like the world is full of super-talented people these days...

mike-cardwell• 15 years ago

Can someone TLDR the TLDR for me please

jrockway• 15 years ago

Why do you think that Google's costs are less when sending messages inside their "intranet" versus sending messages to and from the outside world? Their network is huge and it's unlikely that your users XMPP accounts are managed by a server in the same data centers as your App Engine instances. So it's not costing Google more, it probably costs them the same.

Also keep in mind that if your time is not free, App Engine may be cheaper than EC2. At some point, renting out a colo and buying your own hardware is cheaper than EC2. And then at another point, it's cheaper to build and manage your own data center. It's all a continuum of effort and upfront costs vs. monthly costs.

vinhboy• 15 years ago

I build an image hosting site, similar to imgur, on GAE. Some people in brazil are using it.

Last month, the bill went from $4 a month, to $4 a week.

As far as I can tell, the site did not get any more popular than it was a month ago.

That is a ridiculous increase!

Needless to say, I took down the site.

latchkey4• 15 years ago

Google hasn't made 'random' pricing changes. Sure, they didn't do the best job of handling the switch from beta to a real live service, but they did listen to the community and they learned a lot of lessons. I really don't think they will make the same mistakes twice. Maybe you missed the meetup at Thirsty Bear in SF where they bought us all drinks and apologized? Even Guido was there.

I think you are wrong about the costs of EC2. First off, you have to run your own servers on EC2. When one of those servers goes down in the middle of the night for some random unexplained reason, who are you going to call? Yourself? If you don't think EC2 servers go down, you are kidding yourself.

With GAE, I'll never have to hire IT staff to manage servers or networks.

The high replication datastore on GAE is amazing. I'll never have to worry about running out of capacity, backups, or downtime for that matter. I've been told that entire datacenters have gone offline and nobody even noticed. Now, that is reliability.

It sounds like a lot of your issues surrounded using XMPP on GAE. I think that is a perfectly valid thing to complain about. That said, just cause XMPP is expensive doesn't mean that all of GAE is expensive (or problematic). As you found out by writing your own C++ servers and what not, XMPP is a difficult service to provide reliably. Maybe that is worth that cost?

Note: I'm building my sporting event ticketing website entirely on GAE. People will be able to register for marathons, bike races, etc. It may cost a bit more for their service, but given that I won't have to hire any staff to manage servers or worry about adding capacity when thousands of people all want register for an event at the same time, I think that is worth it.

Note2: I'm a user of PartyChatApp. Thanks for providing it.

cstejerean2• 15 years ago

Careful with the "or worry about adding capacity" part. I've talked to several people that built products on top of GAE because it would just scale, and all were surprised to discover the opposite. Certain queries would start to take inexplicably long and there was no way to figure out why or how to fix it, etc. Things might have gotten better recently, but about a year ago everyone I talked to that tried to scale on GAE gave up and moved elsewhere.

latchkey4• 15 years ago

The datastore is complicated and not what you'd normally expect from a SQL type system. It is a real mental shift when designing your schema to be efficient.

That said, using it is well understood at this point and my co-founder is Jeff Schnitzer who wrote Objectify [1] which is the best way to tie Java to the datastore. Outside of Google, I think he understands the datastore more than anyone else on this planet. Thanks for the warning, but I think we will be ok. =)

Also note, I'm speaking from a Java perspective. Using Python has definitely had MT issues which are being resolved with the new version of Python 2.7 available there now. I'd still recommend using either Go or Java for your app.

[1] http://objectify-appengine.googlecode.com/

vijayp• 15 years ago

I never really meant to imply that App Engine doesn't make it a lot easier to build and administer your code.

It's more than just a price/stability tradeoff. The problem is, as an App Engine user, one is totally at the mercy of any future price changes on App Engine because it is nearly impossible to seamlessly migrate away.

The DNS aliases (appspotchat.com, for instance) can only point to Google, and the API does not really work well on any other platform such as AppScale. So, as we discovered, at any time in the future, one's service could suddenly cost 10x as much, and one won't really have the option to move quickly. If one intends to scale, it's better to never get into that state in the first place, and develop on AWS instead. If EC2 raises its prices (highly unlikely since computing power is increasing and costs are decreasing), one can always move to rackspace or just get a private server.

It's of course true that writing stuff on App Engine can sometimes require a lot less engineering work. But the difference is not really that substantial when compared to the possibility of being stuck on a platform that all of a sudden makes your company unprofitable in unpredictable ways. Changing a running service is very hard. Avoiding the problem by not getting stuck on App Engine is not trivial, but in my opinion the right call.

latchkey4• 15 years ago

What I am hearing is a few things:

a) You are upset about the fact that they increased their prices when they transitioned from a well documented beta service to a fully supported commercial offering.

b) You are upset that there is some features missing, such as the way DNS is handled.

c) You are upset that there is a proprietary nature about the service.

I think those are all pretty much valid concerns, but let me address them:

Yes, the costs did go up for some people, including you, it seems. But remember again, it wasn't just some random increase. They switched out of beta. There was never a hint at a promise that they would keep the prices at the same level.

As I mentioned before, they admitted they messed up in their messaging. They heard people screaming at them loud and clear and went back and actually adjusted a lot of their prices. You say that AWS won't increase prices because of costs decreasing, I can only hope that GAE, with a similar model will do the same.

That said, you really can't compare GAE with AWS because they are entirely different offerings. You absolutely need an IT staff (be it yourself or someone else) to run, monitor and manage those servers on AWS. You don't need that with GAE and I find that to be a massive selling point. I've seen first hand what happens when you have to hire IT staff or stay up at 3am figuring out what server problems you are having. It sucks because a) it is nearly impossible to hire good sysadmins right now, b) who wants to be on call 24/7? There is also the lack of the high replication datastore on AWS. There is nothing even close. Personally, I'm willing to have a bit of vendor lockin for those features.

Yes, features are missing. Google knows about it and has been doing regular releases of the SDK to address missing features. I'm pretty confident that as the service becomes more profitable for them, they will dedicate more resources to it. If not, then I have 3 years to migrate and find something else. For now, the feature set is good enough for me. That might not have been the case for you, but that isn't entirely the fault of anyone. It is still a maturing product.

randall• 15 years ago

PS- Partychat is amazing. If you use Google Apps for your biz, Partychat is like the missing chatroom feature.

Partychat — migrating from Google App Engine to EC2

Related Stories