IMHO not using lidars sounds like a premature optimisation and a complication, with a level of hubris.

This is a difficult problem to solve and perhaps a pragmatic approach was/is to make your life as simple as possible to help get to a fully working solution, even if more expensive, then you can improve cost and optimise.