News for hackers. Stuff that matters.
Re:The Secrets of Building Realtime Big Data Systems by devinfoley on Saturday March 26, 2011 @02:52AM (#2371533)
I love that the slide after "[Data systems must be] 1. Robust to machine failure" is an error "Oops! This slide did not convert properly hence cannot be displayed." Oh the irony.
Re:The Secrets of Building Realtime Big Data Systems by jp on Saturday March 26, 2011 @01:17AM (#2371420)
How does Python hook into all this ? Seems very J2EE. Jython or API? Or maybe they just use it instead of SH to set up env ?

BackType looks cool btw, got an API key a while back and started playing with it.

Re:The Secrets of Building Realtime Big Data Systems by omakase on Saturday March 26, 2011 @01:30AM (#2371436)
We use Python for all our data collecting, general scripting, and some of our stream processing (though stream processing is moving to a more generalized framework that is Java/Clojure based).

Also, our underlying API (that serves results to all our products/API/etc) is a collection of Python servers written with Twisted and Thrift.

Re:The Secrets of Building Realtime Big Data Systems by n_are_q on Saturday March 26, 2011 @05:36AM (#2371732)
Clever. Does the system also support non-aggregate queries? Such as if you wanted to display to a user individual pieces of data for their site for example.
Re:The Secrets of Building Realtime Big Data Systems by gumbo on Monday March 28, 2011 @03:31PM (#2379702)
Thanks for the great slideshow. I've found another slideshow that is realy great: "The Secrets of Building Realtime Big Data Systems"
Re:The Secrets of Building Realtime Big Data Systems by flog on Saturday March 26, 2011 @07:49AM (#2371876)
I thought the stated 200 machine cluster for 300 reqs per sec seemed quite high... Anyone else with RT DM experience care to comment?