Yeah, that kind of agent prediction is a biggie. Agent prediction is something journalists don't talk about much, but is a huge research topic for AV practitioners.

And since you make note of expected behavior of objects... my all-time favorite bug report out of Waymo (that made the public news) is a classic classifier/predictor/planner corner case leading to deadlock. Classifier identified a cyclist and labeled as "stopped" if cyclist has foot on the ground, and "moving" if both feet on pedals. Predictor would plot a trajectory for moving cyclist, planner decides action based on predicted agent behavior. So... Waymo vehicle and cyclist on two corners of 4-way stop. Cyclist is stopped, doing a perfectly balanced track stand with both feet clipped in. Feet on pedals so cyclist is labeled as moving agent... planner yields right-of-way to cyclist according to rules-of-the-road. Deadlock -- nobody moves.