News for hackers. Stuff that matters.
Re:Citations on the Anthropic API by simonw on Thursday January 23, 2025 @10:02PM (#42810105)
It's important to understand what's going on here. Claude isn't providing references because it has a copy of the papers and knows their URLs.

It's instead providing references because enough of the OTHER stuff in the training data (which is entirely undocumented but we can assume includes a scrape of the web, blog posts, Reddit, other papers etc) talked about those papers in a way that provided context on the paper and enough of a title/URL that Claude could usefully return that.

By its nature then you'll only ever find references to VERY popular papers that way. That might be what you want! But you shouldn't expect Claude to be able to dig into the lesser known research around a topic based exclusively on its model weights.

For that you'd be much better off with a system like Gemini Deep Research: https://blog.google/products/gemini/google-gemini-deep-resea...

Re:Citations on the Anthropic API by robwwilliams on Saturday January 25, 2025 @12:50PM (#42823236)
Not consistent with my empirical tests. I asked Claude for references on the demographics of wild rodent populations—-not exactly a hot topic, and all four references were spot on, including two not even listed in PubMed.