His proposed change at the end is that we must always guess and check (emphasis on check) for now, and we may eventually get into a "Psychohistory" level of understanding in the future, but we are far from it now.
Only if the value of whatever smidgen of knowledge you gain (assuming you do actually gain any) outweighs the much-more-than-smidgen cost of the intervention to all the people who weren't helped, and in a significant number of cases were harmed, by it.
As you can see, I don't think whatever benefits are gained actually do outweigh the costs. Which means that no, continuing to try such interventions is not a win.
The lesson we should be learning from all this is to stop trying to do so many things top down. That whenever you see some kind of large scale statistic that you think is not what it should be, the last thing you should do is concoct a large scale intervention to try to change it. Still less should you sacrifice a poor goat who did nothing to deserve it, just to underscore the point that your intervention won't work.
The goal of science can only be to learn whether it helps people, who it helps, who it hurts, etc. Then it can be implemented, or not, depending on the results.
Longitudinal studies tracking populations over time does not imply that sociological intervention experiments, especially political ones, will be reliably recreated over time.
If my hunch were to price right, anything we'd learn there would have a short half live and may not hold out past, say, a generation or two.
I can say anecdotally that my grandparents would respond differently to many situations and interventions than I would. They grew up during the Great Depression and he flew in the war, those two events alone very much impacted how they viewed things for the rest of their life.
Looking at those only a generation behind me and even they seem to respond very differently than I'd expect those in my generation would.
The question I was raising was whether said results would be useful beyond a decade or two into the future, not across different cities today.
No, paying $3B for information of dubious value and which was pretty obvious ahead of time is not a win.
The reason the article resonated with me is because policies and interventions are often sold as "this thing is self-evidently great, and if you oppose it you're an idiot", rather than "this seems promising, so let's give it a try and scale it up if it works well". When I look at proposed measures at my local government level (for example) almost all of the supporting literature focuses on why the problem is important, rather than why the proposed interventions are likely to succeed. Being "effective" as a legislator means enacting policy, so if you want to be effective you try to keep the conversation on the first part.
It's quite interesting to appreciate that these issues were observed, and written about, some 2400 years ago. Times when ostensibly everything would have been practically alien. Yet it emphasizes that the thing that's really changing is not humanity, so much as it is technology. We're still trying, and failing, to deal with the exact same issues as we were thousands of years ago.
And mind you that's just the first time it was written down. I wouldn't doubt that these observations and issues long predate even Plato. Because I think one thing that's become clear is that he wasn't necessarily unimaginably brilliant so much as he was simply capable of objectively seeing the patterns in society and recording them. Though perhaps that is some form of brilliance in its own right.
There aren't too many people comfortable with intentionally letting others suffer for the sake of the scientific method, and those who do have a hard time getting past ethics review boards.
Instead we have this reality where we throw money at problems and claim credit whenever something good happens, and claim we did good work when nothign changes.
These interventions target very complex systems. It's not unexpected that we need to try a bunch of things that don't work before we find something that does, to the extent that most of the things we try are ineffective, right?