For example the AWS services integrate metrics to define behaviors (scale up, scale down, etc), you can implement pretty convenient autotuning algorithms, even those that learn from previous metrics.
The Linux kernel networking and I/O layers are plenty of autotuning code. Powertop is just another example maybe. Even a perl interpreter makes some dark magic.
If I think on my own systems implementation, I think I have do noob autotuning too (alert control, alert frequency, recovery steps, recovery notification... systems increasing their swap before a cron runs, dynamic filtering tables based on app/network input, sysctl autotuning daemons.... etc...)
There are too much to improve regarding autonomous systems, but autotuning is just about programing programmable components, for react on events. Depending on those components, you have to coordinate some kind of events or very different ones. From IRQ interrupts, to a cache hit in the other end of the connection, there are a plenty of parts to implement autotuning, from many vendors, and maybe, this is the root of the issue.
But special case JITs that essentially build a version of a function out of predefined blocks, are not that hard to implement. That's enough to remove current (constant) state dependant branches and do constant folding.
Useful for parsing, some generic data structures and graphics. Anything that is executed very many times, has configurable (but constant) state that controls conditional branching or a part of computation that could be constant folded.
Lazy human's can't be bothered specifying what they really want and dealing with another level of abstraction when twiddling familiar knobs will do.
* Lack of information -- in order to pick the right parameters, the system needs to know what you're trying to achieve (e.g., do you want to minimize latency or maximize throughput?). Communicating that more effectively than just specifying parameters is challenging.
* Instead of telling the system what you want, it can try to figure it out -- to some extent -- by observing program behavior and dynamically fiddling with value. The problem with this approach is that it creates feedback loops that muddle things beyond repair.
* Optimization over a high dimensional space is hard. As an example (even with just one dimension), see this bit from Doug Lea's talk about why dynamic tuning of spinning vs. blocking on concurrency constructs doesn't work: https://youtu.be/sq0MX3fHkro?t=39m48s
* Writing systems that never reboot[1] and are efficient is very hard. Dynamic variables usually require inserting checks at runtime that can't be optimized away by an AOT compiler. This is where a speculatively-optimizing JIT comes in handy, but JITs aren't appropriate for all uses, and even where they are, good optimizing JITs are notoriously hard to write.
--
> Why does the JVM still need messing around GC settings to get acceptable performance/memory usage?
OpenJDK's HotSpot does fairly reasonable auto-tuning now (known as "GC ergonomics"). You can pick either the throughput collector and maybe set your young-gen ratio, or G1 and set a maximum-pause goal. Along with the maximum heap size, those are just three values, one of which is binary, another is often unnecessary and the third can be a very rough estimation. This should be more than enough for the vast majority of systems, certainly with Java 8. Much of the GC tuning parameters you see in the wild are old remnants from before GC ergonomics, that people are afraid to pull out.
[1]: http://steve-yegge.blogspot.co.il/2007/01/pinocchio-problem....