My [Google] Compute Engine Pipe Dream

http://bjk5.com/post/26346827037/my-compute-engine-pipe-dream

by tghw • 14 years ago

11 9 14 years ago

kanwisher• 14 years ago

7 terabytes seems like a lot of data perhaps the author is counting blob storage also. In which case surely s3 is the largest blob storage database already.

kamens4• 14 years ago

No, but I'm counting our indexes which store tons and tons of denormalized data (Google counts these when assessing datastore size). If you completely ignore all indexes, we're at 1TB (but even that includes some denormalized data).

meanguy3• 14 years ago

It seems like AppEngine is saving your ass at the moment but aren't you worried about scale? This is sort of a classic "storage not data" problem where you mapreduce raw data to a structured store for reporting. Are you really still querying everything live? When do you expect this to break down?

ChuckMcM• 14 years ago

I read this, and I wonder "What happens to your company when Google goes away?"

It seems like Google has everyone gunning for it, from anti-trust, to privacy, to patents, to governments. Every tech company that has been in a war of attrition with the world has paid a heavy price. IBM and Microsoft were relatively recent examples. The battles wear on the company and while shields are at maximum now, they erode over time. People get burned out from fighting, and governments change.

I have no idea what the Google of 2020 will look like compared to the Google of 2010. But having been at Sun as a 'startup' and watched it rise, flare up, and then fade away, I realize that tech companies have a horrible track record of permanence. The list is long, Compaq (the unkillable titan), DEC (the company that 'invented computing for the masses'), 3COM, Tandem, Cabletron, Ungerman-Bass, Telebit, Etc.

Google is on a path to join them. Google is intensely protective about how they do what they do. A common complaint on the ex-Google email lists is "Where can I find feature <x> in the FOSS or even enterprise software world?" when they slip below the waves, there is a very real possibility that stuff running on their infrastructure not only won't be extractable (due to lack of access) but even if it was could not be recreated elsewhere.

I see Google's CE/AE and Amazon's EC2/AWS as a quick way to demonstrate you're product has 'legs' and if you can afford to run it on their stuff and still make a profit then, once you get above about 500 'instances' you can afford to run it yourself as well. The downside is that once you get that validation, its probably a good idea to start your migration plan off of them.

My [Google] Compute Engine Pipe Dream

Related Stories