There's one place it could really shine and that's manufacturing space habitats if we could successfully put manufacturing in space for those it might tip the scales since it'd remove the launch cost from a large portion of the structure.

That’s a real chicken and egg problem. Why put that many people into space in the first place?

Tourism seems like an option, but if you can drop launch costs enough that it’s ‘cheap’ then sending stuff up is also ‘cheap’.

IMO, the trick to getting space mining to take off is not robots it’s the need for billions of tons worth of low complexity stuff in orbit.

Off world cryptocurrency mining and AI computations. A few hundred years from now this may not sound absurd.

Two hundred years from now, GPU's will still be scarce and expensive thanks to all the solar powered crypto mining satellites in orbit.

When we have robots and automated tech to send a small factory mining robot and convert volatiles on the asteroid into fuel to move it to near the earth, then it will become much cheaper. The cost to send a small surveyor to an asteroid today is only a billion or so. Eventually it will be a bit less, and then it will be a matter of time between landing on one and moving it slowly into an orbit we want.

I suppose the exception could be rare earth metals, asteroids which contain them in sufficient quantities might be worth going after.

'Rare earth' metals are not actually rare. The cost (both capital costs and environmental costs) to refine them are high, but there is plenty of ore available if you have the refining capacity. There are some elements that are actually rare (e.g. tantalum) but there is no reason to believe they will be any easier to find in asteroids and the cost to refine will be even greater.

Gold’s price collapsing when a meaningfully rich asteroid source can be mined seems likely.

This would be a good thing. It's a wonderful metal, just useless when it's worth so much. Silver would be awesome to crash too, since it's more conductive than copper.

Gold isn't a hugely valuable material as a target for asteroid mining. Platinum is a much better option.