News for hackers. Stuff that matters.
Re:Britannica11.org – a structured edition of the 1911 Encyc... by peterldowns on Tuesday April 21, 2026 @03:10PM (#47853126)
I've been meaning to build ~exactly this experience, but for the 1952 Encyclopedia Brittanica Great Books of the World collection and its experimental index Syntopicon [0]. Would love to know more about how you OCR'd or otherwise ingested and parsed the raw material. I have a physical copy of the books, and I found some samizdat raw-image scans and started working on a custom OCR pipeline, but wondering if maybe I could learn from your approach...

[0] https://en.wikipedia.org/wiki/A_Syntopicon

Re:Britannica11.org – a structured edition of the 1911 Encyc... by ahaspel on Tuesday April 21, 2026 @03:16PM (#47853213)
I'm familiar with the Syntopicon, which would be fun to structure.

I didn’t do OCR myself, except for the topic index and to fill in a few gaps. I started from existing Wikisource text and then built a pipeline around that: cleaning (headers, hyphenation, etc.), detecting article boundaries, reconstructing sections, and linking things back to the original page images. Most of the effort went into rendering the complex layouts, and handling the cross-linking, not the initial ingestion.

Glad to go into more detail if you’re interested, but that’s the gist of it.

Re:Britannica11.org – a structured edition of the 1911 Encyc... by peterldowns on Tuesday April 21, 2026 @04:01PM (#47853791)
Ah ok thanks very much!
Re:Britannica11.org – a structured edition of the 1911 Encyc... by xnobodyx on Wednesday April 22, 2026 @01:48PM (#47866874)
would love to hear more details. are you familiar with the semantic lab at pratt's work - https://semlab.io/projects (also see https://tools.semlab.io )?
Re:Britannica11.org – a structured edition of the 1911 Encyc... by zozbot234 on Tuesday April 21, 2026 @03:21PM (#47853280)
That collection is not in the public domain, AIUI? You might be able to do it for the Harvard Classics, which has a nice collection-wide index of terms. https://en.wikisource.org/wiki/The_Harvard_Classics has links to the scans.
Re:Britannica11.org – a structured edition of the 1911 Encyc... by peterldowns on Tuesday April 21, 2026 @04:01PM (#47853784)
Oh no not in the public domain, I better not build something cool!