I don't understand how you'd do hard real-time when one of the major inputs, the neural-networks, aren't deterministic? In a hard real-time system, what happens if your neural-networks are late telling you where the lanes are?

In reality the comma-AI isn't controlling the car, a number of peripherals on the canbus are controlling the car. All those peripherals are hard-real time, and the comma-AI is bridging the non-deterministic neural networks and those hard-real-time peripherals.

At some point you need to deal with the non-deterministic neural networks, and bridge them into a hard real-time control system. People complaining that part of the system isn't real-time just seems like they haven't thought about it very much. Of course the UI and neural networks aren't going to be hard real-time.

Neural networks are very much real-time. They generally consist of a fixed series of math operations of fixed size. Only the weights and inputs change.

The neural net will never be late. It just might be inaccurate.

Huh, I suppose they aren't inherently incompatible with real-time constraints. It looks like it's pretty challenging to do in practice though.

I'm still somewhat confused as to how multi-object recognition works in a real-time way, naively I'd presume that recognizing n+1 objects in a frame would take longer than n objects.

You might be able to more practically get real-time neural networks using something like YOLO (you only look once)?

Either way it seems to me like hard real-time neural networks are a very challenging problem.