This sentence and comment is a revelation to me so thanks!
I was very into control theory from 2001-2006 before jumping into Bayes nets and eventually ML. Having not really returned to CT since, it's always been a question in my mind how to use ML for SCADA & PLCs without running into the problems that you describe. As with most problems like this, it seems obvious now that it's described as such - though not actually obvious to those like me outside of it!
It seems like our options for ML are currently: self-play, dataset fitting, or human feedback. It's possible to solve any problem that can fit into one of those molds.
(Unfortunately the opposite is true: if a problem doesn't fit into those categories, it seems very hard to solve with any form of ML.)
But, now that I think about it a bit more, the sentence "use a trained net as a tool for building a controller with known continuity properties" seems neither obvious nor straightforward to do. I'm decently well-versed in AI techniques at this point, and I can't really think how I'd sit down and write a program that would generate such a system. Anyone have any leads?
Basically, the goal is to make a neural net that helps you design a system that can be predictable. One way would be to write a loss function that penalizes unpredictability, and then let it run until it converges on a predictable control system. But that's a bit like saying "Just draw an owl." Not so easy to think of the actual code to do that.
The machine learning part is used to get a predictor. You operate the thing and train a model of what will happen for various inputs. The machine learning system is just an observer in this training phase; it doesn't control anything. You train from recorded data.
Then you use the trained model to tune the controller. The model lets you get an output for any set of inputs, so you can now choose input test sets which are suitable for tuning the model, like changing one input at a time and noting the output change.
Automatically generating the structure of the controller (how those blocks are connected) is a separate problem. That's called "system identification".
See [1].
[1] http://www.mpc.berkeley.edu/research/adaptive-and-learning-p...
I’ve been struggling to find where machine learning starts and what is just good old statistics (used in data mining for ages) and search algorithms like A*.
In control systems you need to do inference to predict feedback signals +- noise.
So you can use a neural network for that. Train it on input and feedback data from your samples.
Your control system still needs to account for predictability, smooth behaviour etc so you have the rest of your machinery to handle that.
This is not an end-to-end system though.
Anyone know if there are end-to-end deep learning controllers with guarantees on the output space?
A friend's theory on Google Maps is that it sometimes intentionally sends you down a non-optimal path in order to gather data on alternate routes. This friend isn't in the ML space, so this is just their anecdotal observation. Are they right, then?
It sometimes suggests a route I don't normally take, then when I continue on the route I normally take, the ETA updates to say I will now arrive sooner.
Google Maps intentionally suggests to take longer routes.
However the interesting thing about braess paradox is that even if you keep the number of cars constant, introducing a new road can make travel times worse.
Google's implementation is impressive, but I see no reason why it would benefit from people traveling suboptimal paths.
Don't take that as a dismissal of Google maps: it's the most impressive path finder I know, working with high performance over long distances, taking into account real time information, and calculating meaningful alternative routes to choose from. It's an impressive piece of technology that actually makes the world a better place by saving humanity a lot of travel time that can now be spent on better activities.
Google Maps would certainly be high on my list of best inventions of the last few decades. At the same time the routing is just a good path finder with a good travel time estimator informed by real time data.
I'm not a bloody ant or part of their hyper-organism.
Is that actually true? Everything we do at work with neural nets seems to indicate continuity. We do stuff with mathematical simulations and imagelike processing. Latent space for auto encoders tend to be smoothly varying throughout the range of outputs.
I thought the problem is just that it isn't proven that neural nets are sufficiently smooth.
If you had all possible images of all possible roads in all possible (lighting, weather,..) conditions, etc. then the ideal behaviour is just a function from all inputs to all desired actions. In the absence of this infinity of data you need a model. Any model which has no explicit understanding of the causal behaviour of an environment is going to misbehave for one of those unseen data points (eg. an unseen road in a weather/lighting/etc. condition).
"Correct"(/Safe) behaviour is not a function of pixels. No statistical model which associates pixel patterns with action can be correct.
I disagree. That sounds like an older interpretation of behavior of simple deep nets like the inception family. My experience with auto encoders and GANs is such that the nets can and unquestionably do interpolate between training data points. What's more, the latent spaces display order reflecting logic - much in the way that you can perform arithmetic on BERT encodings in what amounts to a form of logic. This is the where SOTA is now and I think it's a strong step towards AI, though we're still far from it.
The trick is in understanding the boundaries in high dimensional space that your training data represents. So there is some degree of covering all your bases, so to speak, and that requires a new kind of intuition. These are very exciting times in tech/ML.
The problem is that the right behaviour is not a model fitted to that data. It isn't "fitted" at all.
When I turn the thermostat the temperature in the room increases. If the thermostat is broken, the temperature does not. The predicting the effect of the thermostat on the room requires intervening in the room to find out if it is broken; it requires having a model of the room, of the thermostat, etc.
No system that is not in direct causal contact with its environment can adapt to it. The system -- as it is in contact --- needs to be explicitly modelling the causally relevant features of that environment.
This isn't a statistical problem. You cannot learn a function over images to actions because the environment is absent from those images.
An infinite number of 2D images contain no 3D information. An infinite number of 3D images contain no skeletons (ie., inner-structure of objects). An infinite number of images of object pieces contain no information on object behaviour. An infinite number of videos of an object behaving contains no information of its behaviour when broken. An infinite number of videos of all possible breakages in all existing environments contains no information about behaviour in new environments.
Animals solve this problem by playing with objects, building models of those objects (their causal properties, ie., how they interact with other objects). That requires being-in an environment and explicitly modelling it.
There cannot be a "non-bogus" system arrived at via ML. Statistics itself is deficient in providing tools to design systems that "do what they should".
Any paradigm which makes the cartesian assumption that "behaviour is a function of data" is necessarily incapable of intelligent adaptation. Environments "as data" are infinities.
ie., For statistics to work you need all the relevant variables of an environment to make the "correct decision", and all the data needed to train against those. That's infinite.
The "child on road" column isn't going to be fed into the machine. The machine does not, and cannot, even model what a "child" is. Patterns among fractions of infinites is not a basis for saftey (, nor for intelligence).
I don't mean to dismiss your points, though. I wholely agree that if we really want to be moving towards an AI that is anything like animal intelligence, things like play are crucial. I'm a big fan of the attention schema theory. And if that is right, then building systems that can generate such models for their own mental state and others is the way to "consciousness".
https://en.wikipedia.org/wiki/Attention_schema_theory
It's all well and good to build systems that can look at faces and statistically say, "oh they're mad". And I think people have this idea that maybe if we build enough of these systems and jam them together, we'll make an SI. But until we can build systems that can model why that person is mad, based on the system's own experience and models, we're just making parlor tricks.
A model of that kind has to be able to generate predictions given the agents interaction with the object. Ie., how will the thermostat behave if I turn on the air conditioning.
The problem is that we acquire these models during an entire lifetime of being embedded in an environment, esp. a social one.
A self-driving car is never going to predict human behaviour given its own behaviour without having lived a human life. The question of what it should do given how another driver/pedestrian/etc. behaves isn't something it can solve with "mere statistics".
It might be that human behaviour around roads ends up being extremely predictable, I'm doubtful that's the case. I think a car is going to need a Theory-of-Mind to navigate complex social driving environments, ie., a causal model of the behaviour of other animals.
As someone with a lot of experience in self-driving cars, my opinion has changed over the course of the last decade from "we can create smart enough models of these separate problems to create a statistically safer product" to "first you need to invent general AI."
It becomes immediately obvious as you encounter more and more edge cases, but you would never even begin to think of these edge cases and you have no idea how to handle them (even when hard coding them on individual cases - and you couldn't anyway as there are too many) so you realize the car actually has to be able to think. What's worse, it has to be able to think far enough into the future to anticipate anything that could end badly.
The most interesting part of self-driving cars is definitely on the prediction teams - their job is to predict the world however many seconds into the future and incorporate that into the path planning. As you can guess, the car often predicts the future incorrectly. It's just a ridiculously hard problem. I think the current toolbox of ML is just woefully, completely, entirely inadequate to tackle this monster.
But translated to the problem at hand, your argument is correct.
start here: https://arxiv.org/pdf/1312.6199.pdf
I think this is exactly what the GP was trying to say (at least, this is what I understand from it). The continuity bit is about the (non ML) controller, not about the neural network.
Nearly all neural networks generate continuous output. They do interpolate between their learned points. The problem is that it is very hard to verify (and impossible to predict) the interpolation function, so it may cross outside of the safety area on any point.
It's interesting what's happening as the control theory people get into machine learning. The controls people don't typically run a neural net as a controller. They use the trained net as a tool for building a controller with known continuity properties. The trouble with pure neural net controllers is that they sometimes do something totally bogus for some data point within the normal input space. That's not OK in control systems.
This has all the math of machine learning plus the math of control theory and I don't understand it, although I sometimes look at the papers.
[1] https://news.stanford.edu/2015/10/20/marty-autonomous-delore...