It varies per country. In the Netherlands you can request to keep the old document; it will be returned to you with a couple of ~12mm holes punched through it.

My girlfriend needed her old passport for travelling with an Interrail train ticket because the passport expired between the time of ordering the tickets (and submitting the document number) and travelling.

Well, your experience would be limited to the one or two countries you're a citizen of :)

It's definitely a thing across various ex-Soviet countries, and likely others as well.