Secondly, you should be extremely skeptical about the 40 year claim. After all, people move, and the popularity of allotments waxes and wanes.
Thirdly, the people who allocate allotments are mostly looking for someone who is going to show up and take care of their plot so it doesn't get overgrown. Expecting a lot of manual labour and then pricing out anyone with a background in manual labour might not be the smartest move.
Couldn't the owners address that by including a clause like "if you let your plot get overgrown, you have to pay a penalty (maybe 3x what it costs to pay someone to come take care of it), and if you do it twice, we're not renewing your lease"?
So you're buying a kinda-5-year-lease. But the exact length of your lease depends how well you look after it.
We live in a fucking society. Not everything needs to be minmax rent seeking.
Which is stack-ranking - fire the lowest x% of performers every year. Yeah nahhh, no problems with that idea, nope.
It effectively becomes land you own permanently. And the price would probably go from 'a few hundred bucks for something that will entertain me for a year' to 'an investment that will rise in value and I can give my children, so $400k'.
Knowing that a fixed percentage of owners will be stripped of their holding each year makes it a very unattractive investment, while not preventing most of the other uses of an allotment.
Naturally, people with the most affluence would prefer the free market approach. Everyone else would prefer a non-zero chance of getting a plot eventually rather than a zero chance of being able to afford one at all.
That is an analysis by choice. There are other ways of looking at the world, and at economics, than "measure everything".
The "metrics" methods of analysis are useful, but insufficient. Human dignity/happiness cannot be measured, and from the evidence of this New Yorker piece, it is both that the writer is getting from the allotment.
I'm pretty sure I've spent more in equipment, consumables, seeds, etc. than I've got out of it so far but it's very enjoyable.