Re:AthenaX – Uber Engineering’s Open Source Streaming Analyt...
by nevi-me on Monday October 09, 2017 @09:59PM
(#15438756)
My experience teaching some graduates in a BI shop. SQL is more common, and tools that support SQL tend to be used better.
I've "taught" them how to use Spark, but being a team of varying prior experience, the Scala API meant them learning Scala, the Python one was a bit better, but they did much better with the SQL DSL.
Regarding your concern re maintainability: UDF's tend to be the problem, I'm also curious to know re their support issues, and also: can anyone write their own UDF (the code requires registering a .jar), or is there a team that helps business users in that regard?
I've "taught" them how to use Spark, but being a team of varying prior experience, the Scala API meant them learning Scala, the Python one was a bit better, but they did much better with the SQL DSL.
Regarding your concern re maintainability: UDF's tend to be the problem, I'm also curious to know re their support issues, and also: can anyone write their own UDF (the code requires registering a .jar), or is there a team that helps business users in that regard?