Re:Scala projects are difficult to maintain
by thinkharderdev on Monday March 22, 2021 @10:41AM
(#26541485)
Fair enough, the whole SI-2712 debacle is indeed a source of issues well beyond Spark but in general it's just a matter of adding a compiler flag to sort it out. The other example is of course the whole mess of macros...
On the other side though, strict backwards compat creates it's own set of costs (that are often harder to quantify. Java backwards compatibility is great but it seems like it is also at least prat of the reason Java moves at such a glacial pace.
1. Yea, but languages that are backwards compatible have libs that are useful for way longer. I can still use Java JAR files that were built with Java 8, and published many years ago. The open source libs get stale so much quicker in Scala.
2. I'd actually argue that Spark is less weird that some other dependencies. If you look at the cats README (https://github.com/typelevel/cats) there is this caveat: Cats relies on improved type inference via the fix for SI-2712, which is not enabled by default. For Scala 2.11.9+ or 2.12 you should add the following to your build.sbt: scalacOptions += "-Ypartial-unification"
More cats specific stuff here: https://github.com/sbt/sbt/releases/tag/v1.5.0-RC2
I use Spark a lot so used that for the examples, but think other libs cause even weirder maintenance challenges.