> How does a computer know it is working properly?
That is the crux of the issue. Any command that potential malware may give, may also be given legitimately. How one tell those two apart is context. And context is a hard subject even for humans.
Even biology can't get it straight. After all, some of our most resilient diseases exploit the normal signals of cells for their own purposes.
My biggest fear is about failure modes. It seems that there is a great deal of trust being placed in ML output but what happens when it is wrong?
Remember it's not infallible and should not be treated as such.
Training a machine implies some sort of evolutionary model (a training set describes a fitness landscape). Maybe this will work (doubtful, across such a large and variable surface), but how about thinking about this on a fundamental design level? How does a computer know it is working properly?