avionical

Annual $10k prediction contest on a variety of topics. $3k prize pool for the best question submissions.

$15,000 for accurate forecasts about the Trump admin.

There's calibration, but you can also just see contests where you pit the community aggregate against individual forecasters and see who wins. The Metaculus aggregate is really dominant in this contest of predicting outcomes in 2023, for example. See this: https://www.astralcodexten.com/p/who-predicted-2023

Open to all in the US, but $12.5k is earmarked for undergrad forecasters

Specifically, what autonomous replication and adaptation (ARA) abilities will they possess and how will they fare against industry benchmarks.

He's never lost a bet (with academics, researchers, pundits, etc.) for 17 years of public betting. You can see his current open bets here and predict whether he'll win.

Unclear, but the predictions appear to be better on things like US elections than prediction markets or 538: https://firstsigma.substack.com/p/midterm-elections-forecast...

No, it's also 'AI and Machine Learning,' 'Artificial Intelligence, and 'Forecasting AI Progress.' The list of resolved questions is roughly this: https://www.metaculus.com/questions/?status=resolved&has_gro... (though that will include a few questions that have resolved since the analysis.)

And you may want to check the weighted Metaculus Prediction if you haven't already.

I find it useful for a few different non-coding work tasks, and I don't think I'm unusual there.

I wouldn't say AGI in 10 years is predicting an obvious outcome. And the same goes for many other questions on the site.

What would be good evidence of a high-quality user base with the relevant skills? A transparent, well-calibrated track record?

There are several AI categories on the track record page; make sure you're not just selecting the one or you'll miss a lot. There's a careful analysis of the overall track record on AI questions here: https://www.metaculus.com/notebooks/16708/exploring-metaculu...

The short version is that the Brier score is much better than .25 for AI questions, and the weighted Metaculus Prediction is more accurate still.

Anyone can forecast, sure. But there's a large body of research on the accuracy of aggregated forecasts and on the ability of forecasters to become more accurate with practice. (Thinking here in particular of work by Mellers & Tetlock.)

Metaculus provides a transparent track record of community forecasts here: https://www.metaculus.com/questions/track-record/ It's very difficult for any one person to consistently beat the community.

True, though clearing up some of the murkiness is part of what forecast questions like these are for, with their explicitly spelled out resolution criteria.

It sure seems like people are getting value from the current crop of tools: https://fortune.com/2023/06/15/nine-of-ten-developers-using-...

What browser are you using? Not seeing any issues myself.

Professional forecasters and researchers share analyses and predictions on topic areas like AI alignment, longevity research, and geopolitics.

One of the very first questions the site ran. It's a totally different platform and community in 2022.

The platform indicated an invasion before other prediction markets and before the vast majority of commentators. It's not just now coming around.