This is the same issue as with self-driving cars. The vendor will never take responsibility so who will?

This is why they forcibly keep the human in the loop in those cars. The same with AI. Every AI has "always double check your answers" stamped over it. Yet the unreliability is downplayed. The messages are just there so they have a finger to point when it goes wrong.

This is exactly the type of thing that's worrying about AI and I'm glad the EU is trying to hold this back a bit.