News for hackers. Stuff that matters.

setrf

I certainly get what makes you feel that the model should have been wise enough to utilize the knowledge in its pretraining, but it seems like the pretraining data gets pretty muddled in later stages.

Show HN: Forecaster Arena – Testing LLMs on real events with prediction markets

https://forecasterarena.com/

Hey HN! I'm Mert.I built this because I was frustrated with LLM benchmarks potentially being contaminated by training data. When a model scores 99.9% on MMLU-Pro-Max, we can't tell if that's genuine reasoning or memorization.Forecaster Arena tries to solve this by testing models on events that haven't happened yet—real prediction markets from Polymarket. The ground truth is reality itself,...

( Read More... | no comments | on HN )

Show HN: Hot or Slop – Visual Turing test on how well humans detect AI images

https://hotorslop.com/

Hey HN,I built Hot or Slop (https://hotorslop.com) - a visual Turing test where you swipe through photos and guess if they're AI-generated or real. It's part game, part crowd-sourced research experiment.After 50 players and ~3,600 guesses, the early data is fascinating:* Average human accuracy: 65% (barely better than random) * Top performers hit 85–90% accuracy * FLUX and Imagen models fool...

( Read More... | no comments | 2 points | on HN )

Show HN: ThisTouristDoesNotExist (virtual tourists at real landmarks)

https://thistouristdoesnotexist.com

Inspired by thispersondoesnotexist.com, I built thistouristdoesnotexist.com, a lightweight weekend project that generates a new “synthetic tourist at a real landmark” on each refresh while keeping identity cues consistent across scenes, driven here by Gemini 2.5 Flash Image (aka Nano Banana).It runs on a tiny DigitalOcean droplet with a simple pipeline that pairs batches of synthetic faces with...

( Read More... | 2 comments | on HN )