Querying Riak Just Got Easier: Introducing Secondary Indices

http://www.slideshare.net/rklophaus/querying-riak-just-got-easier-introducing-secondary-indices

by rbranson • 15 years ago

76 15 15 years ago

rbranson2• 15 years ago

This presentation is particularly interesting because I think it really touches on why secondary indices should be implemented at the datastore level and are impractical to simply graft onto a distributed K/V store.

arielweisberg3• 15 years ago

I don't follow how these don't always end up being distributed queries since the index key doesn't include the partition key. Where is the locality coming from in the index scanning and value retrieval?

If the key is individual armor type, there could (and usually will) be values in any price range at every partition.

Is the index partitioned separately on its own key? Is a copy of the value stored in the index or is it then retrieved separately from the index scan?

rbranson2• 15 years ago

It leverages merge_index from Riak Search, which stores the index data on the same partition as the object. Queries are only performed against a subset of partitions that the "query planner" thinks will contain the object.

arielweisberg3• 15 years ago

If the index key does not start with the partition key, then won't that end up being everything in the majority of cases?

In the examples given (price, license plate) there is no locality between the partition keys (armor id, person) and the index key. A query for all armor priced between 200-400 would end up touching every partition that contains armor priced between 200-400. Unless the set of armor is small you will end up needing to scan every partition.

psadauskas• 15 years ago

Awesome feature, but why these terrible URLs?

  # Query for category_bin = "armor"
  curl http://127.0.0.1:8098/buckets/loot/index/category_bin/eq/armor
  {"keys":["gauntlet24"]}

  # Query for price_int between 300 and 500
  curl http://127.0.0.1:8098/buckets/loot/index/price_int/range/300/500
  {"keys":["gauntlet24"]}

Why not use URI query parameters for the query parameters?

  /buckets/loot/index?category_bin=armor
  /buckets/loot/index?price_int=[300..500]

seancribbs• 15 years ago

That is still under debate AFAIK. The latter form would allow you to compose multiple index lookups as well, which is why I'm for it (although the query planner might not support it yet).

KevBurnsJr2• 15 years ago

What about

  /buckets/loot/index/category_bin,armor/price_int,300,500

Looks a lot like link walk syntax, don't it?

KevBurnsJr2• 15 years ago

Here are two (minor) considerations

1) If you want to be pedantic about HTTP, RFC 2616 states with respect to responses from URIs containing query strings that "caches MUST NOT treat responses to such URIs as fresh unless the server provides an explicit expiration time." Although this clause is broadly ignored by the vast majority of middleware, it could be argued that slashes are more correct. http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec13...

2) Middleware like Squid comes with strip_query_terms on by default so if you put Squid in front of a Riak cluster and you wanted to see what was actually being run you'd have to make changes to the config file. Otherwise the request uri in the logs just reads "/buckets/loot/index?"

alnayyir• 15 years ago

The candor about the trade-offs involved in picking one database vs. another is definitely appreciated and is a tone the community needs to take on the subject in general.

tolitius• 15 years ago

http://www.dotkam.com/2011/07/06/noram-db-if-it-does-not-fit...

I agree PR and lots of BUZZ makes it really difficult to choose the right DB.

My latest obsession is Riak => Basho is VERY honest, and quick in helping you out and telling where and how Riak can help you, and most importantly what would NOT be a good fit for Riak. ( I am not working for them :)

/Anatoly

Querying Riak Just Got Easier: Introducing Secondary Indices

Related Stories