I would highly recommend https://www.youtube.com/watch?v=PYylPRX6z4Q as a way of framing the problem.

It seems like our options for ML are currently: self-play, dataset fitting, or human feedback. It's possible to solve any problem that can fit into one of those molds.

(Unfortunately the opposite is true: if a problem doesn't fit into those categories, it seems very hard to solve with any form of ML.)

But, now that I think about it a bit more, the sentence "use a trained net as a tool for building a controller with known continuity properties" seems neither obvious nor straightforward to do. I'm decently well-versed in AI techniques at this point, and I can't really think how I'd sit down and write a program that would generate such a system. Anyone have any leads?

Basically, the goal is to make a neural net that helps you design a system that can be predictable. One way would be to write a loss function that penalizes unpredictability, and then let it run until it converges on a predictable control system. But that's a bit like saying "Just draw an owl." Not so easy to think of the actual code to do that.

One approach is that you have a semi-traditional control system, which has integrators, filters, adders, etc. All these components have tuning parameters, and tuning complex controllers is hard. But once you get them tuned, they behave in a reasonably predictable way.

The machine learning part is used to get a predictor. You operate the thing and train a model of what will happen for various inputs. The machine learning system is just an observer in this training phase; it doesn't control anything. You train from recorded data.

Then you use the trained model to tune the controller. The model lets you get an output for any set of inputs, so you can now choose input test sets which are suitable for tuning the model, like changing one input at a time and noting the output change.

Automatically generating the structure of the controller (how those blocks are connected) is a separate problem. That's called "system identification".

See [1].

[1] http://www.mpc.berkeley.edu/research/adaptive-and-learning-p...

So what is the difference of using this approach with exploring the set of inputs to find the optimal one using a search algorithm like A* or the many different ones?

I’ve been struggling to find where machine learning starts and what is just good old statistics (used in data mining for ages) and search algorithms like A*.

They serve the same purpose (optimisation). The advantage of ML is that you don't need to be able to define an objective function instead you just have a data set.

Neural Networks are good at curve fitting which makes them useful for inference +- noise.

In control systems you need to do inference to predict feedback signals +- noise.

So you can use a neural network for that. Train it on input and feedback data from your samples.

Your control system still needs to account for predictability, smooth behaviour etc so you have the rest of your machinery to handle that.

This is not an end-to-end system though.

Anyone know if there are end-to-end deep learning controllers with guarantees on the output space?

There's https://journals.sagepub.com/doi/abs/10.1177/027836491985942.... I think Aaron Ames' group is working on theoretical guarantees of neural net policies as well.

Perhaps it could just try and learn the parameters to some kind of pre existing control theory algorithm? Perhaps with a simulator it could fall into the first category you mentioned.