I have strong doubts, that the wisdom of the crowd here is competent in judging whether a revolutionary new superconductor is real, or not.
Yeah we saw how wise was the crowd's wisdom with crypto.
This market fetishism is out of control.
If the rational actors are actually effective at making such adjustments I don't know, I bet there's statistics out there on how well prediction markets correlate with reality.
In any case, even if the market was perfect, it wouldn't tell us if LK-99 would reproduce, which I guess is the meat of your question. It would just tell us how likely it is that an experimental result made under those specific circumstances would reproduce. And what you could do with that information depends on what your answer to the question: "How would I be affected if LK-99 would reproduce?" would be.
If you're a big energy business leader, and you want to filter what topics to spend your valuable time on maybe you could set a rule that you only want spend time reading scientific papers that have >10% odds of being legit.
More realistically though, I think things like prediction markets are mostly useful to traders who are trying to arbitrage things like resource markets. What's the price of copper going to do when this turns out to be true? You could adjust your futures based on that.
https://imaginatorium.org/stuff/nose.htm
Basically, making an average of a large number of estimates of an unknown value will (of course) fail if most/none of the estimators have any idea of the actual value being estimated.
It's the same with this topic. You won't get an answer to the question "Is this particular paper true or not?" but you'll get an answer to the question "Are papers submitted under these circumstances making claims like this likely to be true?". The crowd will only answer the question they can answer. I think that's from "Thinking fast and slow".
The point of the story when I originally heard it is that no one has SEEN the Emperor's nose. So any statistical function (like averaging) of estimates is totally useless as they are all guesses.
No one has seen 'papers submitted under these circumstances' so no amount of 'crowd wisdom' will make any difference.
Also, as an aside the idea that 'the crowd will only answer the question they can answer' is ... bizarre. People will answer anything you ask them, and you have no way to know if they are just making it up.
A subtle distinction is who is allowed to participate in a prediction market.
"Everyone with $1" is a terrible answer, and produces the bad results people are pointing to.
Financial markets avoid this because of their scale, where there's enough smart money to (usually) punish stupid money.
Absent that scale, it's just stupid money muddling the decisions of smart money.
Prediction markets with a knowledge barrier to entry would produce better results.
Also, suggesting that there is such a thing as 'smart' money - presumably due to having more of it? - is amusing. As pointed out elsewhere in this discussion, there has been a lot of smart money acting particularly dumb over the last few years.
And the relevant question isn't whether 'smart' money does dumb things: it's whether 'smart' money does dumb things less frequently than a random sample of money.
No one is an oracle, and there are absolutely outlier events that specifically confound experts, but I can't believe that increased expertise is uncorrelated with increased accuracy.
Unless the group of people is not a crowd but rather a mob.
And society doesn't spend money. People do.
They are producing predictions of future value. It's not clear when you're only considering a single case, but what if you only have enough money to fund two projects and you have 15 applicants? You could pay a panel of experts to evaluate them and now you can only fund one project, or you can exploit the prediction market and fund the projects that seem to have the best chance of success according to the crowd. So in effect, the crowd is funding projects by freeing up funds that would otherwise go towards bureaucracy.
The wisdom of the crowds works given a large and diverse sample of independent predictors. People who don't know anything will vote randomly so their votes effectively cancel each other out, but people who know more about a particular topic will be biased towards correct answers.
The way resources are allocated is very important, but getting it optimal is very hard. Stock market is one of structures which helps to create long-term incentives to optimize resource allocation.
You can directly see how it works if you compare market-based economies to e.g. a planned economy Soviet Union: a lot of goods produced by Soviet industry were not in demand, especially consumer goods. When Soviet Union was no more a lot of factories were closed because they were producing some utterly irrelevant shit.
> Is any of this liquidity going to actual (replication) research, because if it is not, again: what are these markets tangibly producing?
Many economic concepts work in practice only at scale. E.g. if there's a one-off $1000 incentive, it might not attract people capable of doing that. But if there's an opportunity to make $1000 every second, people might put an effort into taking that opportunity.
Prediction markets create incentives to do particular stuff, as all markets do.
If there was enough money at stake, it could definitely incentivize replication research.
There are two scenarios.
Scenario 1: Suppose you have a lab with all necessary equipment and materials. Normally you would use it for your own research (i.e. research new materials). But if there's e.g. $100M prediction market on replication of a particular result, you might consider redirecting it to replicating that research instead.
If you do it before others, you can sell your replication proof to a hedge fund which will then get a position on prediction market before revealing the proof.
Scenario 2: If there's enough money in replication markets, hedge funds might specifically fund laboratories which replicate stuff.
Vigorish.
> The latter is a perfectly reasonable thing to do - it's not easy for an average person to "pay off some lab", but if they provide liquidity to the prediction market they are giving an explicit subsidy for anyone that can answer the question.
Is any of this liquidity going to actual (replication) research, because if it is not, again: what are these markets tangibly producing? Moving a bunch of numbers around a ledger does not seem very useful.