Show HN: LLM-aided OCR – Correcting Tesseract OCR errors with LLMs

Re:Show HN: LLM-aided OCR – Correcting Tesseract OCR errors ... by troysk on Saturday August 10, 2024 @03:16AM (#41207884)

How to segment the document without LLM?

I prefer to do all of this in 1 step with an LLM with a good prompt and few shots.

With so many passes with images, the costs/time will be high with ViT being slower.

[ Parent | on HN ]

Re:Show HN: LLM-aided OCR – Correcting Tesseract OCR errors ... by jszymborski on Saturday August 10, 2024 @06:38AM (#41208588)

Segmenting can likely be done on a really small resolution and with a CNN, making it real short.

There are some heuristic ways of doing it but i doubt you'll be able to distinguish equations from text.

[ Parent | on HN ]

Re:Show HN: LLM-aided OCR – Correcting Tesseract OCR errors ... by wahnfrieden on Saturday August 10, 2024 @12:58PM (#41210737)

Apple APIs such as Live Text, subject identification, Vision. Run them on a server, too

[ Parent | on HN ]

Re:Show HN: LLM-aided OCR – Correcting Tesseract OCR errors ... by troysk on Monday August 12, 2024 @05:35AM (#41222715)

Segmenting at lower resolution and then using them at higher resolution using resolution multipliers don't work as other items bleed in. FastSAM paper has some interesting ideas on doing this with CNNs which I guess SAM2 have superseded. However, the complication in the pipeline is not worth the result as I find vision LLMs are able to do almost the same task within the same OCR prompt.

[ Parent | on HN ]