convexly

That claim is testable. The 2026 microstructure work on the Kalshi tape (72M trades, Becker) documented a systematic +1.12% excess return for liquidity providing makers and a symmetric -1.12% for takers, plus a longshot bias where 1-cent yes shares pay -41% EV and 1-cent no shares pay +23% EV. That edge is in the market microstructure layer. A patient maker who never trades the frontpage market makes money on the marginal bidder asymmetry alone.

Akey, Gregoire, Harvie, Martineau (2026) on Polymarket find that the top 1% of wallets capture about 84% of all profits, and that the largest whale wallets are NOT the most sophisticated. Large capital systematically bleeds expected value to small order traders. Reichenbach and Walther (2025) document within-Polymarket skill persistence at the 124M-trade scale, so skill differentials are measurable across users distinct from the insider trading question.

I ran this analysis using Convexly, a calibration-tracking tool I'm working on. The Brier scoring math in the post is the same math that runs in the product. Disclosure out of the way.

What I did: pulled trade data for the top 100 Polymarket profit wallets from the public API, resolved 3,651 unique positions, and computed per-wallet Brier scores. Then tested the correlation between calibration and profit using Spearman rank correlation (not Pearson, because the profit distribution is fat-tailed, Hill alpha ~1.6).

The headline finding: Spearman r = +0.608 between Brier score and realized profit. Worse calibration predicts bigger profits. The correlation gets stronger when you drop the top 10 by profit (+0.72), so it isn't outlier-driven. The worst-calibrated whales earn 4.66x the median profit of the better-calibrated whales.

The leaderboard is a concentration ranking. Median single-event concentration: 69.8%. Twenty of the top 100 made their biggest money on the 2024 election.

After separating the four wallets Chainalysis publicly attributed to "Theo" (the French trader who commissioned private YouGov polls), 8 wallets remain in a narrow cluster on the popular-vote markets in a 3-week window around election day. Chainalysis did not link BetTom42 or alexmulti to Theo.

Full 47-column CSV is downloadable from the post. Happy to answer methodology questions.

Quick update: 1,934 quiz completions, 44.5% scored overconfident. Most interesting finding was that the quiz itself got more engagement than the product behind it. Added educational tooltips, a public roadmap with voting, and UTM tracking based on feedback here and from users that reached out directly!

Both really good points. The research does suggest that the core skill does transfer. The quiz can help with long horizon predictions. The mechanism itself seems to be the actual awareness of overconfidence rather than just domain-specific knowledge. With that being said, the gap between the quiz and real-world application is real, and tracking both over time is part of why I built the decision logging side. For your question about teams, that's a built-in feature already! Submissions are "sealed" so you submit before seeing others. The team feature also has a believability-weighted aggregation based on each submitter's track record, and I also built an IC mode for investment committees. The problem you describe about one calibrated person in a room with two uncalibrated ones is what the sealed model prevents. Everyone makes draws their own conclusion, then they compare!

Thank you! Happy to hear to how it compares!

That's all true. I'm a solo founder and have been using Claude heavily to build this. It definitely shows in many places, and I'll make sure to clean those up. I did not expect to get this many visits from a show HN (almost at 1600 quiz takers from the last few hours alone). The core math is sound, but I agree the presentation needs more care. Appreciate the honest feedback!

The change to buttons was based on feedback I got today. The slider disappearing is a bug. Pushing a fix now!

Made a few changes based on feedback from this thread: full results now shown immediately with no email gate, changed the UX to include true/false/uncertain buttons + a confidence slider, I cleaned up the quiz result page, and fixed the die probability question. Thanks for all the honest feedback!

Can't fault that, actually a great workflow. Calibration helps you know which ones to research!

Update at 2 hours: 1350+ quiz takers! 50% overconfident, 40% well-calibrated, and 10% underconfident. The average score is around 0.228, with the best score still at 0.007 (nearly perfect). The pattern so far is people are most overconfident in the 70-90% range, but are right closer to ~55% of the time.

Great recommendation. That was one of the biggest influences for starting to write my decisions down and then building this.

That's fair, I'll flag those or maybe even add regional context. Nice score, well above average!

That's awesome, hadn't seen this one! I like the confidence interval approach.

That's honest feedback, I appreciate it. The post quiz shouldn't feel like a sales funnel. I'll clean that up. Working on it!

Your point on scoring is correct, if you're 100% confident and right on everything you would score a perfect 0. The calibration insight is in how you handle the questions where you don't know the answer. Say you're highly knowledgeable and 95% confident on everything, but get 2 wrong scores compared to someone that says they are 70% confident on those same two questions. That would indicate that you are overconfident compared to the other person!

Actually means you have really strong knowledge. That 17% gap though is what gets people in trouble in high-stakes settings!

That's definitely better framing for this question. Much cleaner way to illustrate that point!

I just removed that, full results should be fully visible without email! A hard refresh should show the update.

That's a really good UX idea. I can see how it's not the most intuitive now. Separating the direction from the confidence level would make it much clearer. Adding that to my list.

Fair point! Bad question on my end. The overconfidence was based on all 10 questions though, not just that one!

You're right, that's a mistake in how I phrased the question. It should say "six times in a row" not "twice in a row". Fixing it now! Thanks for pointing that out!

For sure. Zscalar can block certain requests. Glad it works on your phone!

Great question. Calibration specifically is about whether your confidence in an answer matches your accuracy, not whether you know the answer. Someone who knows a lot but is always 90% confident would score poorly even if they're wrong 20% of the time, as an example.

In terms of research, Tetlock's Expert Political Judgement and Superforecasting were the foundation. He did a 20 year study that showed domain experts were barely better than chance at long-range predictions. The Brier score was the standard metric for that research.

That's the second best score I've seen today out of 700+ quiz takers! Exceptional calibration. The confidence angle is the whole point, people don't know how far off they actually are until they see the hard data!

Just checked and everything is up. That might just be a console warning, but shouldn't affect the quiz. Can you try a hard refresh (ctrl+shift+R)? If that still doesn't work, what browser are you on?

The variance is normal, the questions pull from a pool of 138 questions so far. 0.177 is strong. Setting everything to 50% would just get you 0.25, so you did way better on the first attempt. The goal isn't 50/50 on everything, only on the occasions where you are not confident that you are right.

Just pushed a fix for that! You should be able to see everything without inputting your email now. I've made a note about font size, thank you for the feedback.

The point I was going for was more so how people handle questions they don't know the answer to. Someone that is "well-calibrated" would set things they are uncertain about at closer to 50% instead of guessing one way or the other (overconfident). That score is excellent, so it suggests you did exactly that!