I can tell pretty quickly if something is pedestrian or a bicycle and plan accordingly. In addition I can tell where the pedestrian is looking. Sometimes I slow down, sometimes I don't depending on how I assess the situation.

I think it comes down to the fact that the classification algorithms are not ready for primetime.