Google App Engine to support SQL

http://groups.google.com/group/google-appengine/browse_thread/thread/9740dd2a8bdf4206

by scorpion032 • 15 years ago

120 32 15 years ago

groups.google.com

redstripe2• 15 years ago

Nice feature, but not super important for new projects. I suppose it will make porting some existing projects over.

I didn't remember seeing SQL on the roadmap however so I had another look ( http://code.google.com/appengine/docs/roadmap.html ) and I'm very excited to see it as been revised with a few awesome enhancements. I've had several projects I wanted to try on app engine which were impractical because of GAE limitations like no full text search, no programmatic access to blobstore, no bulk export/backup, no mapreduce. The future is looking a lot brighter for GAE.

ibejoeb• 15 years ago

Why isn't it important for new projects? I have a ton of projects that I can't do on GAE precisely because of this. GQL is extremely limited.

URSpider94• 15 years ago

At the risk of re-hashing a number of previous discussions, GQL appears limited when compared to SQL because Google's database is designed to scale to extremely large datasets and massive numbers of concurrent users. The features that are missing were removed intentionally because they don't scale. If you can re-think your database use to be consistent with this new paradigm, then you'll have a much more scaleable app when you're done.

redstripe2• 15 years ago

It is extremely limited. I just accepted this as a result of the limitations of a distributed datastore and disqualified GAE as a candidate for any query intensive apps I want to do.

The other limitations I listed didn't seem to have architectural causes so I find them more frustrating.

dlib• 15 years ago

I was halfway a weekend project to find out GAE can't do LIKE-statements or an equivalent thereof, only with some hacking this is sort of possible. Moved the project to RoR and got it working in no time. Another project I had couldn't be done because the datastore was to slow. I love GAE, and use it for quite some little projects but you have to know its limitations. SQL (and hopefully the speed that comes with it) are certainly a step in the right direction.

nir• 15 years ago

It's a very big deal for new projects, if it means Django will run out of the box. GAE is a pretty awesome platform for small projects, a clear migration path in/out will make it much more attractive for bigger projects.

jjm2• 15 years ago

This could help out those who are building projects using Django &/or Django-nonrel.

zoomzoom• 15 years ago

Kind of kills one of the main reasons for the nonrel branch to exist, no? Wasn't django-nonrel an outgrowth of efforts to put django on appengine?

I think that the nonrel project has utility on other places (redis, mongo, etc...) but ABP (all buttons pressed, creators of nonrel) was pushing django on GAE alot, IIRC.

jjm2• 15 years ago

I agree. Don't forget that work has begun for a Cassandra driver as well.

Truth be told however, Django and a SQL based system aren't all that bad.

No matter what, it will be a long time before Django gets full 1.0 support for NoSql in any shape or form. That means, 100% Django coverage.

wkornewald• 15 years ago

It depends on who you ask. :) We've created Django-nonrel and djangoappengine for people who need GAE's scalability or at least don't want to deal with scalability as an afterthought. Of course using the GAE datastore for everything can be really painful (and dbindexer is supposed to help in this regard - at least a little bit), but now you can have the best of both worlds and pick the right tool for the task.

charlemagne• 15 years ago

I'm looking forward to testing out porting one of my Django apps to GAE, particularly since the Heroku clones for Django are not yet ready for public consumption.

netmau5• 15 years ago

Hopefully this is the real deal and not an abstraction on the current datastore. When using the datastore in a relational way, you get some pretty horrid performance.

Strom• 15 years ago

It is the real deal and it will scale, and I quote, "as well as SQL scales".

scorpion0323• 15 years ago

The "free scaling" using appengine datastore is nothing more than theoretical at this point. The service has severe datastore latency problems and non trivial downtime. Looks like using RDBMS, solves both these issues for now, while Google can ponder around long term solution for the bigtable access.

sean_lynch• 15 years ago

Member of the App Engine team here. Yes, to confirm, this is not an abstraction on the current Datastore but a separate "real-deal" service.

hpaavola• 15 years ago

We are living weird times. Multitasking, copy/paste, SQL support, these things are old, but still newsworthy when added to new platforms/services.

scorpion0323• 15 years ago

There is a difference between humans speaking and monkeys speaking. For hundreds of thousands (if not millions) of people that use the datastore API, this is a big thing.

r00fus• 15 years ago

There is still no reasonable competitor to SQL. It's an entrenched standard, that NoSQL doesn't completely replace.

Luckily, it's an open standard (like email) and adding in support for an open, strongly-desired, useful stamdard as a feature should be applauded.

charlesju• 15 years ago

SQL isn't special.

Scalable SQL is.

callahad• 15 years ago

So long as the performance is reasonable, this will go a long way towards removing lock-in barriers with AppEngine.

tantalor• 15 years ago

"The datastore is high-latency and extremely unreliable" - http://www-cs-students.stanford.edu/~silver/gae.html

pjscott• 15 years ago

I've found that, if you want to make responsive apps on GAE, you must use memcache. At any scale. Personally, I gave up and left for better options. The generous free quotas aren't worth the trouble.

I hear they've improved latency since I left, so take this with a grain of salt. I'm still not going back any time in the next few years, though.

scorpion0323• 15 years ago

Which is where this could help in the first place, by reducing the load on the datastore.

magicalist• 15 years ago

and the HN response: http://news.ycombinator.com/item?id=2264517

ultrasaurus• 15 years ago

Reading between the lines, this shows that App Engine is a growing & responsive project, and makes me less nervous about building on it.

grandalf• 15 years ago

I'm trying to feel that way too... but the fact is there are people on the app engine team who could write SQL support for app engine in a few days in their spare time.

The problems with app engine and the datastore have shown that there was a high level misunderstanding underlying the core design of the datastore... and how it would function in the context of lots of simultaneous users.

So I'd feel a lot more confident if Google addressed this concern other than with the high availability datastore...

Kilimanjaro• 15 years ago

Google should buy a SQL engine like Postgres now that mySQL is in oracle's hands. People will need SQL databases till the end of times, and coupled with BigTable that can be a killer in the database space.

ShabbyDoo• 15 years ago

Look at this from the perspective of Google Apps corporate users. Imagine that the IT intern has just completed the corporate cafeteria's lunch menu system, and it's time to deploy it. While it's then least significant application the IT department manages, there's still a high fixed labor cost for infrastructure roll-out and management. The DBA must provision instances across dev, QA, and production environments. Onto which servers will the application be deployed? Must they be procured? Even if the lunch menu app is deployed on a VM instance, that OS image still requires security patches and other upgrades. The cost of deploying the intern's work likely exceeds his/her salary for the summer.

With SQL on AppEngine, corporate developers can build apps like this without learning how to use NoSQL for persistence. Even if GAE's SQL doesn't allow scale-out sufficient for SaaS vendors and such, it will fill an important niche in Google's apparent goal of moving corps onto their cloud. As another commenter here suggested, the bar for perf/scale in many cases may be no higher than that provided by a MSAccess DB hosted on a Windows fileshare.

Stormbringer• 15 years ago

I rather got the feeling that the datastore was much better for certain kinds of things than SQL. However, unless you are an expert in both SQL and the use of BigTable, how would you know when to use the one over the other? Okay, so obviously SQL is 'designed' to be good at transactions (whether it is or not is an exercise for the readers), and BigTable is really good for fuzzy search.

I'm guessing that SQL is actually going to be more useful for anyone doing e-commerce (hence transactions) or anyone with familiarity with traditional SQL based persistence (even - shock horror - JDBC).

SQL is bland, BigTable is sexy. But another word for bland is mature...

Google App Engine to support SQL

Related Stories