We already see this with people arguing with near 100% confidence of X happening because of Y and Z…in a vacuum.
I want to see something similar to what you’re asking, don’t get me wrong. But the underbelly of that is a broken clock being right twice a day and far too much weight and decision making being put into some model that doesn’t actually reflect reality
- Statistically, more models tends to be better (in the same sense that more information is better) under the assumption that people can tell better from worse.
- I like how you talk about _applying_ models instead of _using_ models. It is an easy one word change that (to me at least) emphasizes the choice points of: (a) selecting the model; (b) selecting parameters; (c) interpretation.
- All models (by definition) are unrealistic in some way. If a model operates at a useful level of abstraction, inaccuracies at lower levels may be acceptable. In fact, such inaccuracies may be the key to making the model tractable and efficient.
[1]: https://hubbardresearch.com/about/applied-information-econom...
Maybe they even allow clients to play around with the model directly, but I wouldn’t bet on it. I’m guessing they gate access to it via consulting fees. One wouldn’t want a client to “misinterpret” the model, would we? :P
Furthermore, the idea of open sourcing their base model isn’t probably something that they would consider; it is likely perceived as incompatible with their business model.
[1]: https://www.amazon.com/How-Measure-Anything-Intangibles-Busi...
Do we have any economic models which managed to predict any non-obvious, relevant things?
- GDPNow if you want shorter term: https://www.atlantafed.org/cqer/research/gdpnow
- DSGE if you want longer term: https://www.newyorkfed.org/research/policy/dsge#/overview
It’s an aggregate measure and doesn’t tell the distribution of wealth amongst nation.
Also, it’s a metric that was sort of useful in the 1950s and during wartime when this country manufactured goods. Now it’s all unclear in this modern economy. It doesn’t take into proper account of services, quality of care, quality of life.
It doesn’t even take into account _losses_ that may occur. Such as a natural disaster. Sure, in a post disaster GDP goes up because region is rebuilding. But the losses due to disaster are not figured into GDP.
It’s a broken metric. Stop using this as an indicator of economic health and human welfare.
edit; never heard of DSGE but yet again looks like yet another flawed “model” that assumes a level of “equilibrium” and seems to have been birthed out of the pseudoscience known as neoclassical (and orthodox) economics.
This touches on a key point: just because we can predict something doesn’t mean we should optimize for it. Still, being aware of consequences is useful: perhaps we want to satisfice under various constraints.
But a bigger problem is the lack of a variety of models to choose from and compare.
My suggestion? If we want to encourage more models to exist in the world, we should use a mixture of (1) criticizing models (like above); (2) pointing to existing alternative models; (3) building new models; (4) writing up specific comparisons of models; (5) showing how to make models; (6) writing up successful case studies. And lots more.
Not possible with this neoclassical economy we live in. It’s an ongoing grift that has been ongoing for _decades_ which has allowed for irrational thinking to prevail, for wealth and success to be focused into a select group, and for multibillion dollar corporations to be bailed out when their own decisions drive the company to the ground. Privatize the profits, socialize the losses.
There are ways to track ownership of entities and relationships. There are ways to quantify how contracts are awarded, the results, and the impacts. These are complex models, but they can give insight into the dynamics.
I was hopeful ~5-10 years ago with the rise of data science, but I don’t know how much this growth corresponded with decision makers using the models both broadly and in intellectually honest ways.
There are areas where the results seem clear; e.g. advertising and product-specific decision points, such as recommendation engines.
Beyond these short-term ML applications, there are other domains where modeling is essential: weather forecasting, politics, markets, economics. But this is just the tip of the iceberg.
Here is a metric I’m interested in: what percentage of decisions are informed by quantitative models?
Aside: Before you complain that e.g. “some decisions aren’t impactful enough to warrant a model-backed approach”, let me say that I sort of agree, but… There are indeed lightweight quantitative models (or at least processes) that can track / predict when using a heavier model is worth it!
I don’t have systematic data on this; it is just a hunch, inspired by what I have heard and seen.
Why? Effective modeling provides differential advantages.
-intra-organizational diffusion: technically savvy people will rise into management.
- inter-organizational dispersion: techniques used by winners will spread across organizations and industries.
I know... I'm asking for too much... this is usually proprietary stuff, so people are reluctant to share. But I want to live in a world where sensible people can compare models of what might happen. We don't have to necessarily agree on the assumptions or results. But we could at least be open about why we predict various outcomes. That would be better foundation that how we do things now, which is really embarrassing for a world with our capabilities.
Eventually, I want to live in a world where we have sensible discussions about our predictions and about which outcomes are more and less favorable.
P.S. If you are building a product or startup doing this, let me know. Or let everyone know. I built a product in this general area many years ago, but I didn't find product-market fit. Anyhow, I just want modeling tools like this to exist; I don't care who makes them.
P.P.S. I'm very intentionally not bringing in my political philosophy nor level of personal concern here. To be open, I am personally affected, as are many of my friends, but I want to find some kind of common ground that doesn't make this about my particular take on the overall situation.