I'm not so sure that works.
With gold, it was not the case that everyone just "agreed to agree" that it was valuable.
Rather, gold was valued independently of its "investment value" due to its use as a status symbol/decoration/jewelry.
Likewise, because of that, there's no risk of everyone suddenly "disagreeing" that it's valuable, causing a huge collapse.
With bitcoin, it's unclear to me that there is a way to either bootstrap this "agreement" in the first place, or to sustain it.
So what? That's not the relevant risk. The relevant risk is that whatever you are using as a currency will DECREASE in value. Say, from the level of something highly valued as a medium of exchange accepted everywhere, to the level of a mere decoration. Or from a peak value where you bought in big during a speculative bubble, to a lower value where (yes) it's still worth something, but worth less than if you had kept it in treasury bonds or an index fund or a farm.
The lack of underlying utility is the criticism that Warren Buffet makes of gold. Fundamentally, there's no reason why we couldn't all wake up tomorrow, think "this is stupid," and stop spending so much money on gold. Bitcoin is 100% speculative. People will need food and electricity tomorrow, but they won't starve for lack of gold nor bitcoins. Yet somehow, gold's value has remained and in fact increased over time.
For my part, I find both gold and bitcoins to be irrational investments and I am annoyed that they cost as much as they do. Nonetheless, other people's actions may make them rational investments.
[1] Technically, there is a physical asset underlying gold with some utility, but I believe its market value is primarily determined by its investment value and not the underlying asset.