Parallel Universe (YC S12) Developing Spatial Databases For Matrix-Style Games

http://techcrunch.com/2012/07/09/parallel-universe-spacebase-will-make-for-new-matrix-style-mmo-games/

by pron • 14 years ago

95 42 14 years ago

makmanalp• 14 years ago

Hmm, I wonder how many potential customers they have.

CyrusL• 14 years ago

Having talked to Ron's cofounder, it sounds like there are a surprising number of applications for SpaceBase outside of gaming. The TC article seems to focus on gaming but I have a hunch it's mostly that they're going after one vertical at a time.

taybenlor• 14 years ago

The company I work for is a potential customer. We store thousands of locations per minute in one of our Postgres databases.

johnrob• 14 years ago

Location based services could be big. Asking "what's nearby meeting criteria A,B,C" requires a non traditional backend.

fleitz2• 14 years ago

Not really, Postgres works fine for most of this stuff, and as of 9.1 without PostGIS

'nearby' means ordering your query using the <-> operator.

'meeting criteria' goes in the where clause.

For example to find interesting things nearby in postgres:

  SELECT * from interesting_things as it where (it.A ='A' AND it.B ='B' AND it.C = 'C') 
  order by it.location <-> point '(current_lat,current_lon)' LIMIT 10

http://wiki.postgresql.org/wiki/Whats_new_in_PostgreSQL_9.1#...

gcoins• 14 years ago

It doesn't necessarily require a non-traditional backend, as you could design your backend using plain old rdbms. What does require a non-traditional design would be your app, however.

jandrewrogers6• 14 years ago

Far more than you might imagine, at least if they develop a more sophisticated geometry model. The real limitation may be the pure in-memory model. Many of the high-value applications have enormous working sets, much larger than what will fit in a small RAM-based cluster. Hundreds of terabytes is where the applications just start to become interesting. In this sense, focusing on game worlds is probably a good idea because it is one of the use cases that will fit within their immediate scaling targets.

They are correct that traditional database spatial indexes are slow and scale terribly, being designed for relatively small and static data sets. It does not sound like they are pushing the state-of-the-art, just meeting an under-served need in the gaming space, a market which I can validate as existing but with somewhat limited revenue potential even if you sign the major game companies. It is a good "base hit" startup opportunity if they can execute it well. (I have designed massive-scale real-time spatial database engines for a number of years; we passed up the gaming market because the size of the market was too small relative to other markets for this technology.)

akozak• 14 years ago

Forgive the basic question: Is cost the only limiting factor for pure in-memory with large datasets?

pron6• 14 years ago

Actually, we're very much pushing state-of-the-art :)

It's just that we're tackling a different problem - low latency applications - and when I say low latency I mean in the microseconds range. Big spatial data is a very interesting problem, and tracking a large number (though less than billions) of moving objects is another - though very different - interesting problem.

For working sets that don't fit in one machine's RAM we offer a cluster.

kelleyk• 14 years ago

I'd be curious to hear more about what they're actually doing (technically speaking). Is this a wrapper around kd-trees, R-trees, and their friends and relatives? Is it something fancier?

pron6• 14 years ago

The spatial index is an R-tree variant that doesn't degrade and allows concurrent writes (multiples writers at a time, let alone readers). Readers don't block writers and don't use locks, while writers lock a small subset of the database for atomic transactions (one of the most common operations we need to support is a "move" of an object from one location to another - that has to be atomic).

For parallelization of queries and transactions we use fork-join.

tintor• 14 years ago

How do you do lock-free updates to the R-tree structure?

hendzen2• 14 years ago

Is this limited to 2d and 3d space or can it handle n-dimensions?

pron6• 14 years ago

2d and 3d only. Data of higher dimensions is usually not as dynamic (the data points are created and deleted but rarely moved), and SpaceBase is optimized for lots of updates. Also, higher dimensions require a different data-structure (see: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.53.6...)

hendzen2• 14 years ago

n-dimensional would be useful for realtime machine learning applications.

sown• 14 years ago

So, say hypothetically, if I wanted to try my hand at implementing a toy version of something similar for fun, what would I need to at least have a grasp at?

reedlaw• 14 years ago

What a terrible article.

> Oracle and Redis are examples of traditional, spatial database environments.

Redis is traditional? Redis is spatial? What are they talking about?

> Spaccebase is an in-memory technology, which is how it can be so fast. Thats’ what truly separates it from the rest. It tracks not when or what happened but where it occurred. And that all occurs in-memory. It’s like any application that runs this way. It’s very fast in getting data. In this case, we are talking about spatial data.

This entire paragraph is ridiculous. Just a bunch of buzz words repeated ad nauseum. What database does not use "in-memory" technology? I guess that's why "it's like any application that runs this way". Yeah, and French cuisine is like any food that's cooked that way.

dmix• 14 years ago

Yet we still link to TC articles...

YC startups always launch on Techcrunch and top HN within an hour.

Is there not a better place?

Ogre2• 14 years ago

In this case, a direct link to the site (http://paralleluniverse.co/) would be better. There is actual information there. To be fair, the first two words of the TC article were a link to the site. Signal to Noise over the first couple words of that article is quite high!

JPKab2• 14 years ago

I remember years ago when a commenter on Techcrunch bashed an article and called Techcrunch "the special olympics of tech journalism." At the time, I thought it was an insult to Techcrunch. Now I realize it's an insult to the fine athletes of the Special Olympics.

etherealG• 14 years ago

couldn't agree more, anyone know what the cutoff is for getting a down vote button on posts!

samstokes• 14 years ago

I also enjoyed:

> It’s the NOSQL capabilities that makes this so interesting. A NOSQL database is designed for big data.

Still, glad to see the HN comments here provide the perspective and analysis that TC couldn't.

JPKab2• 14 years ago

Lol. I shudder whenever a Techcrunch writer starts throwing around the term "NoSQL", as if NoSQL automatically makes something fast.......

Attention buzzword slinging pseudo-technologist journalists: NoSQL is a term which describes what something ISN'T, not what it is.

ricardobeat• 14 years ago

Maybe they meant MongoDB, which has spatial queries and doesn't live in memory?

TimJRobinson• 14 years ago

Is this going to be targeted towards big companies or startups (in pricing) ? I'm working on a game at the moment that could definitely benefit from this tech and I'd love to try it out but I'm bootstrapping so not sure if it would be affordable.

pron6• 14 years ago

Shoot us an e-mail.

jblow• 14 years ago

As a longtime profesional game developer, I kind of don't get it. Is this company out of business as soon as Oracle implements a loose octree that is ACID? Is there something wrong with spatial hashing?

etc, etc.

jandrewrogers6• 14 years ago

It depends on the requirements of your data model. Anybody with a modicum of competence can scale a point cloud but real-world non-point geometry models is where systems like Oracle have difficulty. Polygons, lines, vectors, etc are a real problem. So-called "spatial hashing" (it had a different name in the 1970s and again in the early 1990s -- the wheel of computer science) has a number of real limitations which is why it was never really used (Oracle has patents on it that have already expired!).

Also, traditional transactional database engines are not designed for the kinds of insert/update rates that are common for many spatial applications. This is a problem for machine-generated data sources generally. It requires an architecture designed specifically for that kind of (ab)use case.

hastur• 14 years ago

How would you do it then?

I mean, if you wanted to implement a 3d spatial db like that yourself?

Ogre2• 14 years ago

It's being marketed as a database technology, but from a game development perspective it sounds more like "Engine" technology. The lines are admittedly blurry sometimes, but this doesn't seem to offer durability of data, which is what I think of when someone says "database" in the context of MMO servers. That said, the problems this is solving are important problems for any MMO server engine.

Requiring a JVM be embedded in your server is probably not going to go over well with many game developers. It's not initially going over well with me, but if I were currently trying to solve this problem, I would at least want to run some benchmarks against it.

I guess the big question I have is, how would this make existing MMOs better? If we take WoW as an example, dumping everyone into a single realm (shard) would solve the problem of wanting to play with your friends, who are on different realms. But 200x as many players competing for kills or standing around the auction house aren't going to make the game better. Most of your time would still be spent in instances, with a maximum of 25 players (40 if you go back in time - a case whre less massive was deemed more fun). And that's ignoring the problem of actually rendering that much stuff on the client.

I don't mean to sound overly critical, I could find ways to use this, but I'm not sure they're marketing it very well to MMO developers. Maybe the Eve guys would like it, but other than that game, I'm not sure the scaling problems MMOs have are the same scaling problems that this solves.

chii• 14 years ago

"Most of your time would still be spent in instances, with a maximum of 25 players (40 if you go back in time - a case whre less massive was deemed more fun). And that's ignoring the problem of actually rendering that much stuff on the client."

you haven't played Eve Online have you? Its a game that allows you to play, on the same "shard", with hundreds of players on grid at the same time, fighting it out.

Any tech that puts the massive back into MMO is good - lately, its all been instancing and sharding. That sort of game isn't MMO enough to make it MMO. In fact, i've been hearing lately, people calling Diablo 3 an MMO. What a farce.

Edit: oh, didn't you mentioned Eve already.

debacle• 14 years ago

I always thought instancing and sharding were solutions to the social aspects of server overpopulation - it's not fun to do a quest when you have to wait for spawns, etc.

Qworg• 14 years ago

I don't think it will make existing MMOs better. It will free them from the terrible restrictions that MMOs had to take on (like shards and instances) based on technology, rather than gameplay choice.

Parallel Universe (YC S12) Developing Spatial Databases For Matrix-Style Games

Related Stories