What could be niche about this? It's the most overwhelmingly common case.
Most people change jobs every couple years and your interviewees ex-bosses are also in the industry doing the same job hopping. It is a near certainty that nearly all (if not all) their ex-bosses have moved to different companies already.
This is the most common case, by far.
Are you suggesting going further back, to the candidate's previous previous employer, and hoping that their direct manager from years ago is able to provide reliable and accurate testimony on the current state of the candidate ?
I'm surprised this is not a universal experience based on these responses. Every candidate I've interviewed has given at least one ex-boss as a reference. Every reference I've ever given has been ex-bosses.
You're right in that HR of their previous (likely current) employes won't say or allow saying anything.
But of course they're likely not giving as references their current boss, for obvious reasons.
Their previous bosses though, who have also left that company, will speak to you freely.