> without much practical thought as to increasing controllability

That's not fair, Waymo and others are using redundant backup systems and obviously try to think of every failure case they can. Besides, aren't the remote-control parts as likely to fail as any other? I sure as hell wouldn't want to drive a car over facetime...