You are correct. There is a concept in more traditional engineering called “human factors”, meaning you have to include humans (and their limitations) in the design.

Expecting a human to remain attentive all the time is like using a sensor outside it’s linearity range to detect and mitigate a hazard. Good initiative but bad implementation and possibly a gap in the design culture

https://sma.nasa.gov/sma-disciplines/human-factors