News for hackers. Stuff that matters.
Re:Inside the OED: can the world’s biggest dictionary surviv... by kragen on Sunday February 25, 2018 @06:24PM (#16461828)
The OED can not only survive the internet; it will flourish on the internet. We can incorporate the entire thing into Wiktionary, which, as https://news.ycombinator.com/item?id=16461656 points out, is already in some ways more comprehensive and reliable. Unfortunately, for copyright reasons, we have to go back to its first edition, and even the last volume of the first edition might still be in copyright.

I announced the project at https://www.mail-archive.com/[email protected]/msg001..., bought a copy of the dictionary, and spent a bunch of nights at the archive scanning its volumes. https://old.datahub.io/dataset/oed talks a bit about the available data halfway through the project. I did eventually scan the whole thing, and since then other people have contributed other scans. So far nobody has OCRed it and imported it wholesale into Wiktionary.

At some point I hacked together a web service that would show you the page image for a given word (perhaps after a few tries), but I don't have it running now. https://archive.org/details/oed11_201407 seems to be a downloadable program that does something similar.

It's really unfortunate that the past century of work at the Oxford University Press will be lost and have to be redone. Such is the price of copyright.

Re:Inside the OED: can the world’s biggest dictionary surviv... by ComputerGuru on Sunday February 25, 2018 @11:30PM (#16463138)
OCR has come a _long_ way since 2005. I was an early adopter around 2000, and the results scared me away from OCR for thE better part of two decades. I recently revisted it with no special approach, just trying Adobe Acrobat Pro’s OCR module and was very pleasantly surprised with the progress that has been made.

Adobe (and I’m sure many others) now preserve the image but place text behind it in a hidden layer or else use a pseudo ML process to create a font from the scan and fit it to the text with high accuracy (and low binary sizes).

Might be worth revisiting.

Edit: just saw the full mailing list url (on phone, hard to see anything) and realized who I am replying to. Hi! I have missed your writings! I recommend “only a constant factor worse than optimal” all the time to people.

Re:Inside the OED: can the world’s biggest dictionary surviv... by kragen on Monday February 26, 2018 @08:36AM (#16465116)
I'm flattered! I'll come back to publishing soon, possibly pseudonymously.
Re:Inside the OED: can the world’s biggest dictionary surviv... by _emacsomancer_ on Sunday February 25, 2018 @09:40PM (#16462659)
I'm assuming the 2nd edn. OED I downloaded from https://web.archive.org/web/20170805001549/http://download.h... for my ereader is of dubious copyright status?
Re:Inside the OED: can the world’s biggest dictionary surviv... by walterbell on Sunday February 25, 2018 @06:28PM (#16461842)
Are you familiar with this scan+index? https://archive.org/details/oed11_201407
Re:Inside the OED: can the world’s biggest dictionary surviv... by kragen on Sunday February 25, 2018 @06:28PM (#16461845)
I wasn't, no! Which scan did he use? I've added that to my comment.
Re:Inside the OED: can the world’s biggest dictionary surviv... by walterbell on Sunday February 25, 2018 @07:15PM (#16462050)
There is contact info on his page: http://www.theabsolute.net/dictionaries/
Re:Inside the OED: can the world’s biggest dictionary surviv... by donquichotte on Monday February 26, 2018 @05:43AM (#16464415)
So it's from 1888? I'm anispeptic, phrasmotic, and even compunctuous to see if "contrafibularities" is already in there.
Re:Inside the OED: can the world’s biggest dictionary surviv... by kragen on Monday February 26, 2018 @08:37AM (#16465123)
Only the first volume is from 1888. It took almost 40 years to finish the rest of the work, publishing it in fascicles for beta testing.