That is only true if the loss is actually permanent. If the system comes to assume that there are only 10,000,000 coins in circulation, what happens if a substantial portion of the "lost" 11,000,000 shows back up? Now you just had a 50% inflation overnight...
Of course, the chance that huge amounts are lost together is quite low.
I don't think you could double the amount of bitcoins overnight. And even if you could I doubt that it would lead to rapid inflation. Rather gradual as additional bitcoins spread throughout the economy.
No actual worth is lost. You just lost the proof that you have a right to something useful or pleasant. It will be claimed by other people. Market will discover how much bitcoins were lost and bitcoin price will raise accordingly so other people holding bitcoin will be entitled to tiny bit more of the valuable things that society provides.
If you break your glass bowl it's a loss for the humanity. If you loose bitcoin wallet it's just a loss for you.