News for hackers. Stuff that matters.

A comment on Trained LLMs exclusively on pre-1913 texts. Parent comment

Re:Trained LLMs exclusively on pre-1913 texts by vintermann on Friday December 19, 2025 @02:28AM (#46323158)
One thing I haven't seen anyone bring up yet in this thread, is that there's a big risk of leakage. If even big image models had CSAM sneak into their training material, how can we trust data from our time hasn't snuck into these historical models?

I've used Google books a lot in the past, and Google's time-filtering feature in searches too. Not to mention Spotify's search features targeting date of production. All had huge temporal mislabeling problems.

[ Parent | on HN ]
Re:Trained LLMs exclusively on pre-1913 texts by DGoettlich on Friday December 19, 2025 @10:16AM (#46326748)
Also one of our fears. What we've done so far is to drop docs where the datasource was doubtful about the date of publication, if there are multiple possible dates we take the latest to be conservative. During training, we validate that the model learns pre- but not post-cutoff facts. https://github.com/DGoettlich/history-llms/blob/main/ranke-4...

If you have other ideas or think thats not enough, I'd be curious to know! ([email protected])

[ Parent | on HN ]