News for hackers. Stuff that matters.
Re:Minority voices ‘filtered’ out of Google Natural Language... by AlbertCory on Friday September 24, 2021 @11:43AM (#28643953)
Just a note on something I do have personal experience with:

The authors also observed that the text of many patents are initially obtained via imperfect examples of Optical Character Recognition (OCR), with their accompanying errors in English possibly passed through to the C4 data with little or no annotation that would distinguish it from acceptable English.

While patent offices do release PDFs of all their patent docs (and it isn't just patents; it's all the back-and-forth between the examiners & the applicant, too), a huge percentage are images of paper documents. You can always tell just by double-clicking on a word -- if the word doesn't highlight, it's an image.

OCR output from these things is generally terrible. There is no way it should ever be input to any ML model.

There are no comments on this comment yet.