> [...] move quickly/rapidly adopt new language and runtime features, don't generally care much for backwards compatibility in the library ecosystem, have many competing frameworks and libraries for everything. etc.
None of this applies to Go and in fact, Java is much worse when it comes to churn than Go is. Just look at how many companies are stuck on Java 8. Go updates are fearless - there's no reason not to run the newest version available.
> [...] move quickly/rapidly adopt new language and runtime features,
Go 1.0 was released almost exactly 10 years ago ([1]). Since, there have been *zero* major language changes. The changes that did happen were small, typically to address inconsistencies or small inconveniences. Though, there is one big change coming in this month's release - generics.
Also about design of the ecosystem in general.
In JVM land almost all code is implemented in managed code. This means the likelihood of severe (runtime degrading) bugs is greatly reduced. It also means in practice you can just step into any arbitrary library code using an interactive debugger to fully follow the flow of execution.
Contrast that to Python where every library worth using is actually written in C against the CPython interpreter itself. This means a few things: Broadly speaking you have just extended the exposure of the core runtime by the amount of code you have loaded in native extensions, meaning more bugs, more severe vulnerabilities and generally a bad day.
It also means that code is "invisible" to the runtime. Python has pretty poor tooling anyway but what little tooling is has becomes useless once your problems venture into C extensions and you end up having to drop down into GDB to work out what is going on. This is made harder by the fact that Python doesn't really play that nicely vith the Valgrind toolkit in my experience making use after free/leaks/etc and other nasty bugs harder to debug than straight C code.
Finally it also means you can't just pick up and move your Python code to a different runtime despite them being available. Unlike Java which will mostly run just fine on Hotspot or J9/other proprietary JVM your Python code is actually now CPython code. In practice I guess this isn't a big deal but it's worth mentioning.
TLDR: Python is old, Java is stable. There is a difference.
pandas? numpy? tensorflow? pytorch? All native extensions. All the fast encoders/decoders for various file formats etc are all native extensions. Talk to message queue or anything else that requires fast I/O? Probably also C. Python itself is so slow that it's only really used as glue for these very fast libraries.
So it's not about control it's about actually being able to understand the tools you are using. If you are using this stack but don't know C then you don't really understand the software you are using.
Again, contrast to Java where as long as you know Java you can probably most libraries in common use with a few exceptions. The main exceptions are crypto, compression libs and stuff like Netty.
When you say a Java programmer knows the entire system, where do you draw the line? The stack system of the JVM? The assembly I assume it later runs, the machine language? Honestly, unless you're running Forth on a very simple chip, I don't think you can possibly hold the entire system in your head at one time.
You don't even need to understand JVM bytecode (though that can really help for performance optimisation).
You can't use almost any meaningful library without running into a C extension in Python-land and that is what I have a serious issue with. (same goes for Ruby and JS though I guess)
The former is old, boring and sticks to what works. They are both slow moving, have large established frameworks for each applicable domain, prioritize backwards compatibility, have highly evolved IDEs that support both the language but also the dominant frameworks etc.
The latter are easy to learn, emphasize the easy creation of new applications and libraries, move quickly/rapidly adopt new language and runtime features, don't generally care much for backwards compatibility in the library ecosystem, have many competing frameworks and libraries for everything. etc.
They are just very very different and as a result produce very different codebases that have very different tradeoffs in hiring, maintainence costs, development velocity, pivotability etc.
Sometimes those languages are the right tool for the job but IMO mostly in cases where you aren't sure what that job is. If you already know what you want to build and how to build it then the "boring" tech stack is generally the way to go in my experience.