You are right. There are several reasons for why you want to fuse (different) sensors:
- reduce uncertainty
- estimate biases and systematic errors.
As you said, different sensors are affected by environmental conditions differently, so it's only reasonable to use all the information instead of only rely on one measurement method
I can almost see the objection that multiple cameras are as good as one camera+lidar, but I think it's a mistake to trust any system that can't check itself for consistency across multiple bands. It doesn't take a very good radar to keep you from ramming a fire truck. In fact, whatever runs the cruise control's distance sensor should have been enough to prevent a bunch of the Tesla oopsies reported in the press. When tackling one of the hardest engineering problems faced by humankind, it seems stupid not to take advantage of all the data you can get.