Sirius: A distributed system library from Comcast

http://comcast.github.io/sirius/overview.html?attempt=2

by akkartik • 12 years ago

50 23 12 years ago

comcast.github.io

whadar• 12 years ago

How does it compare Redis?

bri3d3• 12 years ago

Redis implements operations against simple in-memory data structures with a network frontend, not a distributed storage/replication system.

Sirius lives at a lower level: you could, for example, use Sirius to build the storage system backing a distributed Redis clone.

ihsw• 12 years ago

The network front-end is key here, especially since the goal of Redis is -- in spirit and in actuality -- to act as a network front-end to in-memory data structures.

It's damn good at it too, but disassociating that network front-end may make sense in the future.

agibsonccc2• 12 years ago

Hazelcast is a favorite of mine: http://hazelcast.org

bri3d3• 12 years ago

On cursory first glance Sirius looks more advanced than Hazelcast: Sirius seems to provide substantially more flexible cluster topologies along with "real" consistency behavior (via Paxos). Plus, Sirius seems to come with a built-in persistence system with at least some thought applied to it, while last I checked Hazelcast left persistence entirely up to the library consumer.

As a disclaimer I haven't looked at the implementations of either, only the documentation (and I used Hazelcast in a project a while ago).

larsmak• 12 years ago

Having used HazelCast extensively, and considering making some of my code Sirius-depentant, I can tell you that it is not more advanced. Sirius is basically just a distributed key-value store w/no partitioning, whereas HazelCast has a lot of abstractions built on top of the simpler principle[1]. In fact, it might be a bit overkill in some situations, and that's one of the reasons I will consider Sirius.

[1] http://www.hazelcast.org/docs/3.1/manual/html-single/

fizwhiz• 12 years ago

LSM tree, anyone?

ojilles• 12 years ago

I'm not an american citizen, but I can't have been the only one with a reaction of: "Comcast? ... Comcast?!". But this is really awesome -- hope more tech is going to get open sourced from Comcast and similar companies.

opendais3• 12 years ago

Comcast is terrible to its consumers because they know you can't run away from them and get better service at that price point [generally].

Open sourcing stuff that costs them [very little] and may result in a cost savings in the future and doesn't allow you to disrupt their core business? Why not?

Maybe I'm just a bit cynical. ;)

ars3• 12 years ago

You are cynical.

People complain about comcast because they are the only choice - when you don't get a choice you hate what you get.

Their actual service is perfectly fine. I'm sure you can dig up some bad stories, but think about it from a percentage point of view: Comcast is huge, there are bound to be some problems, but as a percentage of total customer the number of problems is perfectly normal.

PS. No connection with Comcast, I have a sore spot about herd mentality. People just believe things without checking for themself (or based on anecdotes).

Said a different way:

If you have tons of choices you will highly review your choice as a form of choice validation. If you have only 1 or few choices you will only review the company if there are problems.

The result of that is poor reviews without actual poor service.

anon_coward932• 12 years ago

Comcast does lots of really good work on the back end. They've been spending a ton of money developing all of their new platforms using open source tools -- which inevitably means they'll create new stuff like this and contribute it back. They have a literal army of open source nerds hacking in ruby, scala, erlang, etc.

Comcast realized a long time ago that if they didn't want Google to eat their lunch, they needed to approach product development more like Google does. It's starting to pay off. I would expect to see more open source projects coming from them in the near future.

Bjoern3• 12 years ago

Reminds me of Infinispan [1] or other similar Data Grid software.

[1] http://infinispan.org/

Bjoern3• 12 years ago

Is it possible to do write through / write back to some other data store?

comcast-jonm3• 12 years ago

This isn't currently supported, although Sirius has some options that might be helpful. Sirius isn't really a data store per se; it's really a persistent, replicated update stream (transaction log) aimed at letting you build and maintain your own in-process representation of the dataset.

As such, you could set up a separate cluster running in "follower" mode, subscribing to the stream of updates and them writing them out to some other datastore.

Bjoern3• 12 years ago

Thank you for clarifying that.

Sirius: A distributed system library from Comcast

Related Stories