Spark is not just written in Scala, they shared the same philosophy of underthought design.

Luckily, there's a little reason of using Spark after Yandex released their Clickhouse: it better compresses data, it is much faster, it doesn't need crazy infrastructure of low quality Apache's code, it can be used in a realtime which is really huge.