Sure on rare occasions you'd look up something that was left out, but that already happens in the other direction (towards rare words) because that and the are taking up space where more difficult words could have been included. Smaller dictionaries often don't have the word I wanted, but do have a ton of words I'd never need to look up.
[1] https://english.stackexchange.com/questions/42480/words-with...
And I don't think a rare use of a common word is any less likely to be looked up than a rare word.
And its one of the reasons I'd like to know about the toolset used to construct dictionaries - I'd give anything to be able to throw a word set at these tools, have a custom dictionary constructed from it, and repeat until unity...
[1] https://www.amazon.com/Little-Oxford-English-Dictionary/dp/B...
I understand that unity is a difficult thing to attain in dictionaries, but perhaps there are tools and methods used by lexicographers that I need to learn about, as an interested party.
https://www.amazon.com/Thinkers-Thesaurus-Sophisticated-Alte...
What else do you need? A hypergraph of word-vertexes and relation-edges animated in webgl, layered by categories and streamed from an elastic back end? That's your brain.
In printed form, or is web-based okay?
I don't. I do have a background in dictionaries, though: I created The Online Slang Dictionary (http://onlineslangdictionary.com/) which is the eldest slang dictionary and thesaurus on the web. I've given some minutes of thought to turning the underlying code into a product...
What sort of data would you like to capture in the DB, and what would apps do with it?
The reason is that I'm fascinated with dictionaries - the physical kind - in general, and would like to incorporate some sort of dictionary/glossary system in most of my apps as a means of documenting the technical terms for the subject those apps address.
Another thing is that I'm a complete newbie to the subject of lexicography, and I'd love to learn more from like-minded individuals, on what kind of tooling and methodology is out there. I suppose I could go and look at the sources for such things as the OpenDict, sdcv (https://wiki.archlinux.org/index.php/Sdcv) and so on .. but surely there must be standards for the ways Websters and so on construct their lexicography. I'm just not familiar with these tools, so I'd hoped to have some insight from other like-minded individuals in this community, before I launch into a google-trail of my own investigation.
With your site (wonderful, by the way) - did you invent your own schema for the database, or are you using some standard tooling on the backend?
My ideal tooling would give me the ability to pipe a text file into it, construct a list of all un-defined terms/words/symbols, give the means of importing definitions from known sources for each word, and so on. Ultimately I'd love to be able to construct custom dictionaries for any given corpus, such that no term in that corpus is undefined; and to be even more meta about it, no term in the dictionary itself would be undefined. Obviously this is a highly iterative thing, as the undefined terms list grows with every definition added - which is why I think I have to understand the tooling better before I start constructing my own processes for this task.