We (UK) have use by dates, and best before dates.
They mean exactly what they say.
i.e. Raw chicken will have a use by date, eat it beyond this and you risk getting the shits. On the other hand all fruit, veg, tins and most dairy simply have best before dates.... meaning use your eyes and nose to see if it is OK.
So given the myriad of different factors*, where along the bell curve do you decide to put the cut off? The longer the duration, the more people will eventually ingest expired food and risk illness, but also less food is wasted.
From a purely economic perspective, it might come down to the cost of food waste vs hospitalizations and lost productivity.
From a user experience perspective, I’d rather only eat food before it’s expired and throw away some good food rather than throw away less food while occasionally eating unsafe food.
It’s a trade off, really, and probably not some massive conspiracy, although I wouldn’t be surprised if the agencies responsible had been “overly cautious” with the expiration dates. Perhaps to increase profits or promote higher than necessary food production to increase our food security.
* - There’s lots of factors that would change that date. For example, is the yogurt stored at the front or back of a refrigerator? It’s usually colder in the back so it might last longer there. How long did it take to get it home or to stock it? Where in the industrial chiller at the supermarket did it sit? Closer to the aisle where it’s warmer? Was the cold chain** disturbed in general? What’s the specific fat content of that style/brand/cow? There’s really a lot at play here.
If it had started as "Best by" then we'd have substantially less stigma around it.
The silliest though are people who do so, but happily leave a pack of opened UHT milk in the fridge for months because it wouldn't expire for another year or so.