Scala actually became MUCH more popular with Spark. I think the original point stands - the platform pulls the language.
Java is popular due to the jvm - it was the first jvm language after all. It got popular before others like scala & clojure managed to get off the ground, at all.
Python was hugely popular long before these were a thing. It was already seen as one of if not the most beginner friendly language at the beginning of the 2000s. It became the language of choice for ML because it was already popular with beginners not the other way round.
That's not how history played out at all. Java spent much of its life being pitched as a way to make C++ developers more productive.
The jvm was actively hated by many for years - slow startup times, excessive memory usage, it didn't used to enjoy the rich tooling ecosystem it has now and it was seen as opaque and hard to tweak for performance.
To this day it's not that hard to find Java devs who want off the hotspot jvm - whether that's to go onto other JVMs with different tradeoffs (Azul or whatever) or whether to go native compilation (GraalVM) instead.
However all the Java shops I hear about are also looking into Kotlin at some level of adoption. From all the new JVM languages Kotlin integrates the best with the existing Java environment and for displacing a language in an existing niche an easy upgrade path is the most important. So I think Kotlin will become an important language in the JVM ecosystem, it will just take a long time because these types of businesses are conservative in their tech choices.
I fail to think of any other platform that could run these monstrosity CRUD enterprise apps as fast as the JVM can. Sure, C++ can be written to utilize hardware better, but with all the classes and interfaces around with everything being virtual, a good JIT compiler can skip method lookups over AOT compiled languages.
While both of these things are true, they not connected in the way you imply, since Java is a pretty low level language by today's standards.
In the Java school of business app engineering, writing the code is rarely a big part of the effort, so it doesn't matter if the langugae is not very good or expressive. Java wins at having a big commodity-like labour pool of programmers, and there's a lot of inertia and stability in the platform.
There are of course a lot of people who use more expressive and creative tools in making business apps, like eg the many companies using Clojure, Scala, Ruby, Python etc for them, so it's not the only way to skin the cat.
On the JVM it is like trying to replace C on UNIX.
Sure, this is anecdotal. But I'd say the same of Java's dominance in the JVM space. Java's continued dominance is not a sure thing from my vantage point.
Groovy was all the adoption rage across German JUGs back in 2010, then everyone was going to rewrite the JVM in Scala, or was it Clojure?
Now a couple of places are adopting Kotlin outside Android, nice, eventually will migrate back in about 5 years time.
https://trends.google.com/trends/explore?q=%2Fm%2F07sbkfb,%2...
This is less relevant today. The host blessed languages do have an advantage, but I would not say it is insurmountable. It might have been the case in the past, but the modern JVM is a platform, it is no longer a glorified Java language interpreter.
> Now a couple of places are adopting Kotlin outside Android, nice, eventually will migrate back in about 5 years time.
Maybe. Maybe not. Most developers I talked to that have experienced the transition do not want to go back to Java.
This isn't to say Java will die. It will continue to thrive. But Java dominance (on the JVM or as a whole) isn't a sure thing anymore.
Python - which took perls lunch over a decade or so and now in more recent history became the obvious language when data scientists and ai/ml practitioners needed to adopt a platform. Its popularity has even led to it also expanding into education in a huge way after Python became the most popular scripting language.
Java - the jvm is an utterly amazing piece of engineering and yet still to this day most people still write java instead of the other better languages (kotlin, scala, clojure) hosted on that platform, and they deploy only to Linux environments. The platform with its write once run anywhere and industry leading garbage collector that gets better than 50% memory utilisation, isn’t the thing that attracts people - its the language.
Functional languages used to be the cool thing before object oriented design came along and unlocked the ability to build bigger systems. The ultimate programming language (lisp) has always encourages functional since what, the 60s?