Re:Ballista: Distributed Compute with Rust, Apache Arrow, an...
by diroussel on Wednesday July 17, 2019 @03:08PM
(#20462944)
It’s not possible to do a direct port. Since Spark relies on java serialisation to send executable code and data (closures) over the wire to workers. This relies on byte code. Rust is compiled ahead of time.
So you can sort of see how the essence of spark can be ported. It’s essentially a monad. But a direct port to Rust is not possible because Rust uses AOT and the JVM was built to load code dynamically.
So you can sort of see how the essence of spark can be ported. It’s essentially a monad. But a direct port to Rust is not possible because Rust uses AOT and the JVM was built to load code dynamically.