Re:Unauthenticated flag flipping yields remote code execution
by jerf on Wednesday April 11, 2018 @02:41PM
(#16814116)
There is another mathematically-correct layer of security you can get, though, which is the IO interface to the real world that you provide the input. Barring (huge!) bugs in the (very simple!) interpreter, it doesn't matter how much basic brainfuck [1] from a user you execute, they aren't going to get a shell. Worst they can do is burn CPU until you shut them down. (Brainfuck interpreters are already generally pretty tightly RAM limited.)
Sandboxing is of course easier said than done, but it's still mathematically valid. The problem is that our real hardware has too many metaphorical sharp edges from a mathematical point of view, not that it doesn't work.
To do that input needs to be decidable. Input from possibly-malicious sources needs to be limited to a grammar no more complex than deterministic context-free[1]. Accepting Turing complete input introduces the possibility that recognizing anything about the input might be provably undecidable.
[1] https://archive.org/details/The_Science_of_Insecurity_