Replace OCR with Vision Language Models

Re:Replace OCR with Vision Language Models by constantinum on Wednesday February 26, 2025 @10:01PM (#43190868)

The primary issue with LLMs is hallucination, which can lead to incorrect data and flawed business decisions.

For example, Llamaparse(https://docs.llamaindex.ai/en/stable/llama_cloud/llama_parse...) uses LLMs for PDF text extraction but faces hallucination problems. See this issue for more details: https://github.com/run-llama/llama_parse/issues/420.

For those interested, try LLMWhisperer(https://unstract.com/llmwhisperer/) for OCR. It avoids LLMs, eliminates hallucination issues, and preserves the input document layout for better context.

Examples of extracting complex layout:

https://imgur.com/a/YQMkLpA

https://imgur.com/a/NlZOrtX

https://imgur.com/a/htIm6cf

[ Parent | on HN ]

Re:Replace OCR with Vision Language Models by Hackbraten on Wednesday February 26, 2025 @10:16PM (#43190945)

> try LLMWhisperer(https://unstract.com/llmwhisperer/) for OCR. It avoids LLMs

The website you linked says it uses LLMs?

[ Parent | on HN ]

Re:Replace OCR with Vision Language Models by constantinum on Thursday February 27, 2025 @04:46AM (#43192782)

The tool doesn't use any LLMs for processing/parsing the data. It parses and converts into raw text.

The final output(raw text) of the parsing is then fed to LLMs for data extraction. e.g. Extracting data from insurance, banking, and invoice documents.

[ Parent | on HN ]

Re:Replace OCR with Vision Language Models by ungerik on Thursday February 27, 2025 @09:08AM (#43194507)

Those images look exactly like what you get from every OCR tool out there if you use the XY information.

[ Parent | on HN ]