TimeCapsuleLLM: LLM trained only on data from 1800-1875

Re:TimeCapsuleLLM: LLM trained only on data from 1800-1875 by water-data-dude on Monday January 12, 2026 @12:31PM (#46591578)

It'd be difficult to prove that you hadn't leaked information to the model. The big gotcha of LLMs is that you train them on BIG corpuses of data, which means it's hard to say "X isn't in this corpus", or "this corpus only contains Y". You could TRY to assemble a set of training data that only contains text from before a certain date, but it'd be tricky as heck to be SURE about it.

Ways data might leak to the model that come to mind: misfiled/mislabled documents, footnotes, annotations, document metadata.

[ Parent | on HN ]

Re:TimeCapsuleLLM: LLM trained only on data from 1800-1875 by gwern on Monday January 12, 2026 @01:31PM (#46592311)

There's also severe selection effects: what documents have been preserved, printed, and scanned because they turned out to be on the right track towards relativity?

[ Parent | on HN ]

Re:TimeCapsuleLLM: LLM trained only on data from 1800-1875 by mxfh on Monday January 12, 2026 @02:51PM (#46593313)

This.

Especially for London there is a huge chunk of recorded parliament debates.

More interesting for dialoge seems training on recorded correspondence in form of letters anyway.

And that corpus script just looks odd to say the least, just oversample by X?

[ Parent | on HN ]

Re:TimeCapsuleLLM: LLM trained only on data from 1800-1875 by water-data-dude on Tuesday January 13, 2026 @01:22PM (#46605339)

Oh! I honestly didn't think about that, but that's a very good point!

[ Parent | on HN ]

Re:TimeCapsuleLLM: LLM trained only on data from 1800-1875 by reassess_blind on Tuesday January 13, 2026 @01:48AM (#46597953)

Just Ctrl+F the data. /s

[ Parent | on HN ]