So RL is usually too cumbersome for relatively little gain.
Source: I'm a roboticist working with RL.
1) One of the key things in control systems is to deal with uncertainty. If you have a hundred trains, none of them should crash. You want a control system which you can understand to work even in corner cases. ML is bad at that.
2) You need to train an ML. Unless you have a very good model of the thing you're controlling (generating which is usually harder than designing an optimal control system), training an RL means crashing airplanes, burning out amplifiers, or destroying self-driving cars as the system "learns." Even so, when moving from model to reality, it's likely you didn't capture some oddball part of the system.
3) In many cases, there is some give-and-take between control system design and system design. In an extreme case, the PID can be purely mechanical or electrical (e.g. an op-amp). In a less extreme case, if you can design the mechanical system to avoid having e.g. a right-half-plane singularity, controls becomes easier. RL isn't at the level where you can do that kind of codesign.
4) It's a lot more complex.
So "in general," no. In some specific cases, yes.
RL is mostly beneficial for rather complex systems, such as walking robots or anytime a word like "nonholonomic" comes up. For relatively linear, relatively low-dimensional systems, classical methods will give better results for less effort.
An RL algorithm running on a general-purpose computer, you'll be lucky to hit tens of milliseconds.
That makes a big difference.
For those unfamiliar with control systems, delay matters a lot. Something like an audio amplifier (where you can't hear anything <20kHz) will generally have a bandwidth in the tens of megahertz, and more is better. A rule-of-thumb is:
https://en.wikipedia.org/wiki/Gain%E2%80%93bandwidth_product
In other words, if you'd like to bring down error by a factor of 10,000, your loop will generally like to be around 10,000x faster than whatever it's controlling. If something has a time constant of 1 second, you still might want to be able to react at the speed of microseconds. That's not a hard-and-fast rule per se (I can give all sorts of ways to work around it), but what is true is the more general principle that minimizing delays can be very, very helpful (even for things which otherwise seem pretty slow).
The whole concept never really ‘clicked’ until I watched some demonstrations, like this [1] video. After seeing that one in particular, I realised that it’s everywhere… not everything is just a robotic on/off, or a directly proportional response based on feedback from some input.
Determinism is often valued in safety critical systems. The system can't work _sometimes_. The system can't even work correctly _eventually_. It must produce the desired result in the time allotted. Actuating the thing 20ms too late could be as bad doing it incorrectly depending on the application.
I'm no ML expert, but that seems to be a problem for those types of solutions in some spaces.