Re:RDDs are the new bytecode of Apache Spark
by pacala on Sunday May 31, 2015 @12:11AM
(#9633035)
It's software, everything is possible :) It is fairly straightforward to add compile time information via a simple case class definition. There is still a dynamic type check that json matches the case class, but the spec of the structure of the data is done in one place, not spread out arbitrarily over the query code base.
[ Parent
| on HN ]
That part is actually coming: https://github.com/apache/spark/pull/5713