News for hackers. Stuff that matters.
Re:Donut: OCR-Free Document Understanding Transformer by AmazingTurtle on Monday May 29, 2023 @06:05AM (#36112422)
I tested it out with a bunch of personal documents. Results were disappointing. Did not match up with the promised scores, not even slightly.

I think the traditional approach to scanning and classifying without AI/ML is the way to go, for the next 5 years at very least.

Re:Donut: OCR-Free Document Understanding Transformer by jstummbillig on Monday May 29, 2023 @06:39AM (#36112601)
https://cloud.google.com/use-cases/ocr

By my estimation, this already beats all "traditional approaches" for at least a few month now (purely inferring from when I stumbled across it, no clue how long it has been true before that).

Re:Donut: OCR-Free Document Understanding Transformer by j16sdiz on Monday May 29, 2023 @07:29AM (#36112927)
Do they feed your data to recaptcha ?
Re:Donut: OCR-Free Document Understanding Transformer by threeseed on Monday May 29, 2023 @07:35AM (#36112965)
Google has scanned 40 million+ physical books and magazines

Which it used OCR to produce digital text.

So one source of training data at least.

Re:Donut: OCR-Free Document Understanding Transformer by jstummbillig on Monday May 29, 2023 @08:04AM (#36113120)
I did not check. I also never checked if they share my mails on google search with you -- but I trust their ambition to not be sued into the ground for doing something immensely stupid.

Leaking sensitive data of enterprise customers as training material for public recaptchas falls in that category.

Re:Donut: OCR-Free Document Understanding Transformer by subbu on Monday May 29, 2023 @12:33PM (#36115460)
have you tried Azure's OCR? https://learn.microsoft.com/en-us/azure/cognitive-services/c.... Is it comparable to Google's?
Re:Donut: OCR-Free Document Understanding Transformer by driscoll42 on Monday May 29, 2023 @01:30PM (#36116032)
I did some OCR tests on some 1960s era documents (all in English). Mix of typed and handwritten. I had as results:

Google Vision: 95.62% HW - 99.4% Typed

Amazon Texttract: 95.63% HW - 99.3% Typed

Azure: 95.9% HW - 98.1% Typed

Then if curious, TrOCR was the best FOSS solution at 79.5% HW and 97.4% Typed. (However it took roughly 200x longer than Tesseract which was 43% HW and 97.0% Typed)

Re:Donut: OCR-Free Document Understanding Transformer by dimatura on Monday May 29, 2023 @05:07PM (#36118120)
When did you do this test? I don't have any numbers handy, but a couple years ago I compared google's OCR vs AWS's on "text in the wild" pictures. AWS' wasn't bad, but it was definitely outperformed by the google one. The open-source solutions I tried (tesseract and some academic deep-learning code) were far behind.
Re:Donut: OCR-Free Document Understanding Transformer by driscoll42 on Monday May 29, 2023 @05:34PM (#36118364)
This was a couple months ago now, so not that long ago. For OCR I have found that it highly depends on the type of image you are looking at. In my case these were all scanned documents of good but not great scan quality, all in English. I expect if you were talking about random photos with text in them, you'd see the FOSS solutions do much worse, and much more variance in the Google vs Amazon vs Azure. I would be curious about the academic deep learning one you tried.
Re:Donut: OCR-Free Document Understanding Transformer by dimatura on Monday May 29, 2023 @06:57PM (#36119091)
The main one was https://github.com/JaidedAI/EasyOCR, mostly because, as promised, it was pretty easy to use, and uses pytorch (which I preferred in case I wanted to tweak it). It has been updated since, but at the time it was using CRNN, which is a solid model, especially for the time - it wasn't (academic) SOTA but not far behind that. I'm sure I could've coaxed better performance than I got out of it with some retraining and hyperparameter tuning.
Re:Donut: OCR-Free Document Understanding Transformer by driscoll42 on Monday May 29, 2023 @07:15PM (#36119242)
Interesting. I tried easyOCR, I found on handwriting it was about 35%, on typed it was 95.7%, so not bad at all with typed, but for handwriting pretty bad. I focused on Tesseract and TrOCR since it wasn't working out that well, still could easily have just been my particular use case.

I also tested paddleocr and keras ocr to round them all out.

At some point I really need to finish my project enough to write up some blog articles and post a bunch of code repos for others to use.

Re:Donut: OCR-Free Document Understanding Transformer by mhitza on Monday May 29, 2023 @06:40AM (#36112605)
Do you (or anyone else) know which would be a good open source OCR for PDFs and images?
Re:Donut: OCR-Free Document Understanding Transformer by lelandfe on Monday May 29, 2023 @06:56AM (#36112714)
OCRmyPDF is the typical answer: https://github.com/ocrmypdf/OCRmyPDF

It uses Tesseract under the hood. Results tend to just be OK in my experience.

Re:Donut: OCR-Free Document Understanding Transformer by xrd on Monday May 29, 2023 @09:25AM (#36113710)
Tesseract is amazing. It is simple and generally good results.
Re:Donut: OCR-Free Document Understanding Transformer by version_five on Monday May 29, 2023 @10:18AM (#36114237)
I was just playing with tesseract last week (I'd used it years ago) and wasn't too happy. I had a pretty simple pdf that was in what you could think of as an old typewritten font, but easily legible, and I got all kinds of word fragments and nonsense characters in the output. I know that high quality ocr systems include a language model to coerce the read text into the most probable words. Is tesseract just supposed to be the first stage of such a system?

I'll note that when I put the tesseract output into chatgpt and prompted it saying it was ocr'd text and asking to clean it up, it worked very well.

Re:Donut: OCR-Free Document Understanding Transformer by lelandfe on Monday May 29, 2023 @10:36AM (#36114423)
I tried to figure out how to train Tesseract on a specific font and just gave up after a while. Coercion from a dictionary or language model also sounds like it would be a good idea.
Re:Donut: OCR-Free Document Understanding Transformer by flaviut on Monday May 29, 2023 @01:40PM (#36116136)
I was just processing a document with tesseract & ocrmypdf, and two things:

My first time processing it, I used `ocrmypdf --redo-ocr` because it looked like there was some existing OCR. After processing, the OCR was crap because ocrmypdf didn't realize it was OCR but thought it was real text in the document that should be kept. This was fixable using `ocrmypdf --force-ocr`.

Before realizing this, I discovered that Tesseract 4 & 5 use a neural network-based recognition. I then came across this step-by-step guide on fine-tuning Tesseract for a specific document set: https://www.statworx.com/en/content-hub/blog/fine-tuning-tes...

I didn't end up following the fine-tuning process because at this point `ocrmypdf --force-ocr` worked excellently, but I thought the draw_box_file_data.py script from their example was particularly useful: https://gist.github.com/flaviut/d901be509425098645e4ae527a9e...

Re:Donut: OCR-Free Document Understanding Transformer by denysvitali on Monday May 29, 2023 @06:10PM (#36118711)
FWIW, I'm using Google's ML Kit which runs completely on-device and doesn't send the documents to Google. It works better than tesseract for my use case.

I did a presentation on the topic recently: https://clis-everywhere.k8s.best/16

I'll soon make the stack open source, but it shouldn't be hard to recreate given the inputs I've already provided.

Re:Donut: OCR-Free Document Understanding Transformer by dataflow on Monday May 29, 2023 @01:08PM (#36115796)
I could never get tesseract to give good output. It baffles me when people say it's good. Do I need to tune it somehow or something?
Re:Donut: OCR-Free Document Understanding Transformer by bobosha on Monday May 29, 2023 @09:19AM (#36113667)
easyocr was the best of the bunch - however it still leaves quite a bit to be desired.
Re:Donut: OCR-Free Document Understanding Transformer by wahnfrieden on Monday May 29, 2023 @10:00AM (#36114043)
Not open but free - Apple’s
Re:Donut: OCR-Free Document Understanding Transformer by occamrazor on Monday May 29, 2023 @01:04PM (#36115765)
Which Apple product?
Re:Donut: OCR-Free Document Understanding Transformer by imaurer on Monday May 29, 2023 @02:25PM (#36116526)
Two places I use it: Preview on my Mac, photos on my phone. Haven’t seen an api yet.
Re:Donut: OCR-Free Document Understanding Transformer by momo93 on Monday May 29, 2023 @02:47PM (#36116694)
Re:Donut: OCR-Free Document Understanding Transformer by vosper on Monday May 29, 2023 @03:27PM (#36117118)
If you want to OCR specific text you can use Textsniper on Mac and draw a box on whatever part of the screen you want to capture. I'm guessing under the hood it's just using Apple's OCR tech, which does work very well (at least if you're on Apple Silicon, it's not quite so fast on my 2015 Intel Macbook Pro)
Re:Donut: OCR-Free Document Understanding Transformer by wahnfrieden on Monday May 29, 2023 @03:59PM (#36117461)
VNRecognizeTextRequest and DataScannerViewController
Re:Donut: OCR-Free Document Understanding Transformer by kcorbitt on Monday May 29, 2023 @10:06AM (#36114105)
When I was evaluating options a few months ago I found https://github.com/PaddlePaddle/PaddleOCR to be a very strong contender for my use case (reading product labels), but you'll definitely want to put together some representative docs/images and test a bunch of solutions to see what works for you.
Re:Donut: OCR-Free Document Understanding Transformer by joaquincabezas on Monday May 29, 2023 @07:33PM (#36119373)
Did you finally select it? Our team is using PaddleOCR with good results! We wrote an article (https://medium.com/adevinta-tech-blog/text-in-image-2-0-impr...) about it.
Re:Donut: OCR-Free Document Understanding Transformer by driscoll42 on Monday May 29, 2023 @01:32PM (#36116060)
Tessearct is generally the overall best for typed documents, though it struggles with handwriting. TrOCR is better than Tesseract, especially with handwriting, but requires a GPU to have any speed. Tesseract from my tests was roughly 200X faster than TrOCR (not an exaggeration)
Re:Donut: OCR-Free Document Understanding Transformer by iamflimflam1 on Monday May 29, 2023 @06:59AM (#36112735)
I think the online demos have been fine tunes to work on receipts.
Re:Donut: OCR-Free Document Understanding Transformer by loudmax on Monday May 29, 2023 @08:54AM (#36113470)
Developments in this space are coming really fast, and reading words are squarely within the capabilities of neural engines. 5 years is a very long time in AI years.
Re:Donut: OCR-Free Document Understanding Transformer by paddw on Monday May 29, 2023 @09:22AM (#36113683)
For documents which are mostly pretty clean you are probably right. The ceiling for AI/ML is definitely higher though, and very useful right now if you know specifically what type of document you expect to look at, but expect it to be messy.
Re:Donut: OCR-Free Document Understanding Transformer by jrpt on Monday May 29, 2023 @10:47AM (#36114523)
How does it compare to something like https://docalysis.com?
Re:Donut: OCR-Free Document Understanding Transformer by nogridbag on Monday May 29, 2023 @11:16AM (#36114787)
What would you recommend for classifying documents? Most of the companies I've evaluated market their product as using fancy AI/ML, but instead they have hundreds of people, usually in India, manually classifying the documents.
Re:Donut: OCR-Free Document Understanding Transformer by refulgentis on Monday May 29, 2023 @11:28AM (#36114887)
I strongly believe everything just has to go through OpenAI or Anthropic, for now. These models are significantly better than any NLP models I try swapping in.

But this isn’t much help if you must classify images.

Re:Donut: OCR-Free Document Understanding Transformer by jameshart on Monday May 29, 2023 @12:28PM (#36115421)
Five years? Where’s that number coming from?

This seems like exactly the kind of problem that will see rapid improvements as people point more LLMs at multimodal input.

Right now making predictions for ML capabilities on a five year timeframe seems foolhardy.