Ballista: Distributed Compute with Rust, Apache Arrow, and Kubernetes

Re:Ballista: Distributed Compute with Rust, Apache Arrow, an... by andygrove on Wednesday July 17, 2019 @04:24AM (#20457544)

Some great points there.

I have multiple goals here.

Goal #1 is to demonstrate how Rust can be effective at data science / analytics and evangelize a bit.

Goal #2 is to have fun trying to build some of this myself and improve my skills as a software engineer.

Goal #3 (getting a little more vague now) is to ensure that in some hypothetical future I can get a job working with Rust instead of JVM/Apache Spark.

[ Parent | on HN ]

Re:Ballista: Distributed Compute with Rust, Apache Arrow, an... by m0zg on Wednesday July 17, 2019 @04:34AM (#20457591)

Got it. Fun side project then! Good luck. Rust does seem like a good fit for something like this.

I think the hard-ass responses in the thread (all unwarranted) simply misunderstand the goals.

[ Parent | on HN ]

Re:Ballista: Distributed Compute with Rust, Apache Arrow, an... by cauterize on Wednesday July 17, 2019 @12:32PM (#20461285)

FWIW, I'm keeping your name in the back of my head for when my current company inevitably fails at their big data revolution. Biggest product produces 10-15 PB/day that must be processed. Our existing solution is 20 years old and relies on navigating C code with function line counts on the orders of 10k. Rust would've saved me a lot of time from living in gdb.

Really enjoyed the article. Keep it up!

[ Parent | on HN ]