> In my previous company we used Scala + Spark and I hated it initially. I had no idea how syntax worked, no clue about functional programming or the ecosystem.

It's funny that if you tell Cat-purist that Spark is functional-style, they would laugh at your face :)

That's an unhelpful thing to say. Most people agree that the Spark scala DSL is functional style. It's not as advanced in the style as other scala ecosystems but it has been a very good on ramp for people discovering functional style and is widely celebrated for that

Scala + Spark is absolutely the sane face of Scala. Try using stuff like cats (basically Haskel ported to Scala) to experience the deep end of the pool.

To discard Cats as 'basically Haskell ported to Scala' is doing it a bit of a disservice. Yes, it explores pure functional programming, but that design space is huge.