Re:How FlightCaster Squeezes Predictions from Flight Data
by pskomoroch on Monday August 24, 2009 @06:13PM
(#783536)
The real world Lisp/Clojure + Hadoop approach is definitely fun to hear about. Also interesting to see a YCombinator team with 8+ people including a domain expert. Mashing up 4+ messy data sources is tough to do. Very unconventional on many fronts.
Re:How FlightCaster Squeezes Predictions from Flight Data
by jaf12duke on Monday August 24, 2009 @06:57PM
(#783606)
It's totally fun--we love it. Everyone has a role that they own and we trust each other to execute. Stay tuned--this is just a small slice of what's coming...
Jason (@FlightCaster)
Re:How FlightCaster Squeezes Predictions from Flight Data
by technomancy on Monday August 24, 2009 @11:22PM
(#784054)
> The idea of iterating with Clojure in a REPL on a small dataset to develop or refine an algorithm, then pressing a button and see how it does running on some large dataset on EC2, sounds sublime.
Unfortunately it usually works out to spending a few hours iterating on the functions in the REPL till it works great and then spending a week battling with Hadoop to make it actually run the way you intended it to. (no exaggeration here.)
Though we've been working with raw Hadoop; it sounds like Cascading makes it much less painful. We'll see how that goes.
Re:How FlightCaster Squeezes Predictions from Flight Data
by bradfordcross on Wednesday August 26, 2009 @12:50AM
(#786593)
True dat. Spoken like a true clown zen master. It wouldn't be clowncomputing if we didn't have red rubber noses and funny rainbow hair.
Even if they never release any of the glue code that makes all this happen, just knowing it is possible is very encouraging.