Thought experiment: SpaceX is able to lasso up a 100% diamond asteroid and bring it to earth. Suddenly supply of diamonds is 1000x what it was expected to be that year. Diamonds will be worth marginally less as a result, however, it does not mean all that diamond asteroid is worthless. I am sure diamond mines would close for a bit as that asteroid is likely cheaper per unit than some other methods. Suddenly because diamonds because relatively cheaper they can begin to be used for all kinds of currently uneconomical uses. Maybe diamond fiber becomes the next typed of networking cable, or diamond asphalt makes our roads last forever, or diamond knives become all the rage for top chefs -- I don't know, but we'll find a way to use cheaper diamonds, and that asteroid is going to be worth a boatload no matter what.
Wish I had more time to think about this, but work calls.
But I see your argument.
Personally though I would want to keep the minerals up in space, figure out how to refine them there, and then use them up there. Since the cost of getting large quantities of materials up there is still rather expensive.
Going beyond our planet for resources constitutes a huge step in human development - a step we're currently opting out of, but one day a mining program such as this might get the ball rolling again.
I've thought people would mine asteroids in the future because they're a great source for my favorite metal, iridium. I just didn't think it'd happen so soon.