News for hackers. Stuff that matters.
Re:The Claude Delusion by bihla on Saturday November 14, 2026 @10:06AM (#50497607)
He also tries to dunk on Gates by saying he claimed he produced novel problems in science to test AI.

I don’t think the author realizes that “AP” is an exam for American high schoolers. Gates is only claiming he made new test questions at a high school+ level that wouldn’t have been in the training set for AI. I don’t see how it’s so unbelievable he could do that.

Re:The Claude Delusion by lapcat on Saturday November 14, 2026 @10:37AM (#50497860)
> Gates is only claiming he made new test questions at a high school+ level that wouldn’t have been in the training set for AI. I don’t see how it’s so unbelievable he could do that.

It’s unbelievable because Gates is neither a biology exam author nor a biology textbook author, so how would he even know what‘s in the training set for AI?

Re:The Claude Delusion by lapcat on Saturday November 14, 2026 @10:57AM (#50498041)
> Gates is only claiming he made new test questions at a high school+ level that wouldn’t have been in the training set for AI. I don’t see how it’s so unbelievable he could do that.

It’s unbelievable because Gates is neither a biology exam author nor a biology textbook author, so how would he even know what’s in the training set for AI? To know, you would need to be an experienced biology teacher and survey the pedagogical literature.

Re:The Claude Delusion by ianm218 on Saturday November 14, 2026 @11:20AM (#50498229)
If you did well on AP Bio in the distant past and have kept up with the field of biology like Gates it is normal you could ask some high school exam level questions.

When I was studying for AP Calculus or Literature etc I’m sure I could’ve made decent sample questions.

Re:The Claude Delusion by lapcat on Saturday November 14, 2026 @11:29AM (#50498298)
Nobody is disputing whether Gates could come up with some exam questions. They're disputing whether Gates could come up with novel exam questions that weren't in the AI training data.
Re:The Claude Delusion by Systemerror7A69 on Saturday November 14, 2026 @11:44AM (#50498413)
Especially because good questions are more than simply asking about factoid X in Book Y. Good questions test if you really understood something and can transfer knowledge.

Coming up with such questions is not easy, and just because you did well on a test or "kept up with the field a bit" does not mean you can ASK such questions very well.

Re:The Claude Delusion by TeMPOraL on Saturday November 14, 2026 @01:38PM (#50499550)
Maybe he was into biology for a hobby. Would fit his character, much like half of HNers. And would fit the fact that he funded biotech R&D too.
Re:The Claude Delusion by lapcat on Saturday November 14, 2026 @01:45PM (#50499619)
> much like half of HNers are probably near-experts in some random field they haven't had formal training in

More like half of HNs believe they're near-experts in some random field.

Re:The Claude Delusion by arw0n on Saturday November 14, 2026 @02:24PM (#50499952)
This is the crux though, isn't it? Are people as overconfident as the article claims, or are there many 'hobbyist' experts milling around? I did a Bachelor's in philosophy before switching to CS, and have been fascinated by IR/political theory since high school. While I could not hope to learn it all, I can hold a good conversation with a researcher in either philosophy or IR, especially if it intersects with a topic of my core expertise.

To flip it around, I have friends who I would consider (hobbyist) experts in CS. The geneticist is much better at C++ and OOP than me, the geologist knows a lot about 3d modelling and the whole GIS world, and the economist knows way more about DS, AI and statistics. Who am I to say they aren't experts? This kind of rigid division of field and expertise made no sense in university, and it only breaks apart more as we get older and more experienced.

Re:The Claude Delusion by lapcat on Saturday November 14, 2026 @02:45PM (#50500123)
> Are people as overconfident as the article claims, or are there many 'hobbyist' experts milling around? I did a Bachelor's in philosophy before switching to CS, and have been fascinated by IR/political theory since high school.

Heh, well, by coincidence, before I became a programmer I spent (too many) years as a PhD candidate in philosophy, and I'm skeptical about the extent to which undergraduates can be considered experts. ;-)

That was a long time ago, though, and despite my experience, I've undoubtedly fallen far behind the current work in the field.

Re:The Claude Delusion by TeMPOraL on Saturday November 14, 2026 @07:17PM (#50502519)
Surely even undergrads know something, and knowing more than high-school curriculum - which is the bar we're discussing - isn't some particularly difficult achievement.
Re:The Claude Delusion by lapcat on Saturday November 14, 2026 @07:40PM (#50502703)
> Surely even undergrads know something

I should hope so, after paying so much tuition!

> knowing more than high-school curriculum - which is the bar we're discussing

No, that's not the bar you were discussing. Originally you said "near-experts", but now you've lowered the bar way down.

Re:The Claude Delusion by keeda on Saturday November 14, 2026 @02:17PM (#50499900)
What if we flip this question around. Given what we’ve seen LLMs achieve today, if Bill Gates actually got an accurate read on their capabilities back then, what does that say about his expertise in biology? Maybe he got Dunning-Kreuger’d and made a lucky guess… or his assessment was accurate.

Given his track record and the state of AI today, I know which one I find more likely.