The Secrets of Building Realtime Big Data Systems

http://www.slideshare.net/nathanmarz/the-secrets-of-building-realtime-big-data-systems

by nathanmarz • 15 years ago

61 6 15 years ago

jp• 15 years ago

How does Python hook into all this ? Seems very J2EE. Jython or API? Or maybe they just use it instead of SH to set up env ?

BackType looks cool btw, got an API key a while back and started playing with it.

omakase• 15 years ago

We use Python for all our data collecting, general scripting, and some of our stream processing (though stream processing is moving to a more generalized framework that is Java/Clojure based).

Also, our underlying API (that serves results to all our products/API/etc) is a collection of Python servers written with Twisted and Thrift.

devinfoley• 15 years ago

I love that the slide after "[Data systems must be] 1. Robust to machine failure" is an error "Oops! This slide did not convert properly hence cannot be displayed." Oh the irony.

n_are_q• 15 years ago

Clever. Does the system also support non-aggregate queries? Such as if you wanted to display to a user individual pieces of data for their site for example.

flog• 15 years ago

I thought the stated 200 machine cluster for 300 reqs per sec seemed quite high... Anyone else with RT DM experience care to comment?

gumbo• 15 years ago

Thanks for the great slideshow. I've found another slideshow that is realy great: "The Secrets of Building Realtime Big Data Systems"

The Secrets of Building Realtime Big Data Systems

Related Stories