News for hackers. Stuff that matters.
Re:Mistral OCR by bambax on Thursday March 06, 2025 @04:42PM (#43285350)
It's not bad! But it still hallucinates. Here's an example of an (admittedly difficult) image:

https://i.imgur.com/jcwW5AG.jpeg

For the blocks in the center, it outputs:

> Claude, duc de Saint-Simon, pair et chevalier des ordres, gouverneur de Blaye, Senlis, etc., né le 16 août 1607 , 3 mai 1693 ; ép. 1○, le 26 septembre 1644, Diane - Henriette de Budos de Portes, morte le 2 décembre 1670; 2○, le 17 octobre 1672, Charlotte de l'Aubespine, morte le 6 octobre 1725.

This is perfect! But then the next one:

> Louis, commandeur de Malte, Louis de Fay Laurent bre 1644, Diane - Henriette de Budos de Portes, de Cressonsac. du Chastelet, mortilhomme aux gardes, 2 juin 1679.

This is really bad because

1/ a portion of the text of the previous bloc is repeated

2/ a portion of the next bloc is imported here where it shouldn't be ("Cressonsac"), and of the right most bloc ("Chastelet")

3/ but worst of all, a whole word is invented, "mortilhomme" that appears nowhere in the original. (The word doesn't exist in French so in that case it would be easier to spot; but the risk is when words are invented, that do exist and "feel right" in the context.)

(Correct text for the second bloc should be:

> Louis, commandeur de Malte, capitaine aux gardes, 2 juin 1679.)

Re:Mistral OCR by layer8 on Thursday March 06, 2025 @05:34PM (#43285723)
> This is perfect!

Just a nit, but I wouldn’t call it perfect when using U+25CB ○ WHITE CIRCLE instead of what should be U+00BA º MASCULINE ORDINAL INDICATOR, or alternatively a superscript “o”. These are https://fr.wikipedia.org/wiki/Adverbe_ordinal#Premiers_adver....

There’s also extra spaces after the “1607” and around the hyphen in “Diane-Henriette”.

Lastly, U+2019 instead of U+0027 would be more appropriate for the apostrophe, all the more since in the image it looks like the former and not like the latter.

Re:Mistral OCR by jorvi on Thursday March 06, 2025 @06:22PM (#43286032)
> Just a nit, but I wouldn’t call it perfect when using U+25CB ○ WHITE CIRCLE instead of what should be U+00BA º MASCULINE ORDINAL INDICATOR, or alternatively a superscript “o”

Or degree symbol. Although it should be able to figure out which to use according to the context.

Re:Mistral OCR by TeMPOraL on Thursday March 06, 2025 @05:45PM (#43285799)
This is "reasoning model" stuff even for humans :).
Re:Mistral OCR by layer8 on Thursday March 06, 2025 @05:50PM (#43285837)
There is OCR software that analyses which language is used, and then applies heuristics for the recognized language to steer the character recognition in terms of character sequence likelihoods and punctuation rules.

I don’t think you need a reasoning model for that, just better training; although conversely a reasoning model should hopefully notice the errors — though LLM tokenization might still throw a wrench into that.

Re:Mistral OCR by MatthiasPortzel on Thursday March 06, 2025 @08:55PM (#43286844)
Slightly unrelated, but I once used Apple’s built-in OCR feature LiveText to copy a short string out of an image. It appeared to work, but I later realized it had copied “M” as U+041C (Cyrillic Capital Letter Em), causing a regex to fail to match. OCR giving identical characters is only good enough until it’s not.
Re:Mistral OCR by raffraffraff on Friday March 07, 2025 @03:33AM (#43288305)
It feels like, after the OCR step there should be language and subject matter detection, with a final sweep with a spelling / grammar checker that has the right "dictionary" selected. (That, right there, is my naivety on the subject, but I would have thought that the type of problem you're describing isn't OCR but classical spelling and grammar checking?)
Re:Mistral OCR by layer8 on Friday March 07, 2025 @08:02AM (#43289822)
It’s OCR in that the wrong characters are being recognized. This is not about fixing spelling or punctuation mistakes present in the source image, it’s that errors are being introduced due to a lack of accuracy of this OCR with regard to punctuation and typography. The punctuation errors are not different in principle from the case of producing a misspelled word that wasn’t misspelled in the original.

A subsequent cleanup pass that fixes grammar/spelling errors wouldn’t be appropriate when the goal is to faithfully reproduce the original text.

Re:Mistral OCR by raffraffraff on Friday March 07, 2025 @05:02PM (#43295245)
> A subsequent cleanup pass that fixes grammar/spelling errors, as you propose, wouldn’t be appropriate when the goal is to faithfully reproduce the original text

I suppose that depends on why it's wrong. Did the model accurately read a real typo in the image or did it incorrectly decipher a character? If a spelling & grammar pass fixes the latter, isn't it valid?

Re:Mistral OCR by pbhjpbhj on Friday March 07, 2025 @10:39AM (#43291028)
Not unrelated - OneNote 'copy text from image' has started producing lots of incorrect OCR results, but they're all non-words.

For example, from a clear image of a printed page (in a standard font), it will give me 'cornprising' instead of 'comprising'; 'niatter' instead of 'matter'. Excepting the spell-check underline they'd be hard to spot as with relatively tight kerning all the errors look like the originals.

I'm surprised as 1) I've not had these sorts of errors before, 2) they're not words, and words must be heavily weighted for in the OCR engine (I'd have thought).

Re:Mistral OCR by bambax on Thursday March 06, 2025 @04:55PM (#43285451)
Another test with a text in English, which is maybe more fair (although Mistral is a French company ;-). This image is from Parliamentary debates of the parliament of New Zealand in 1854-55:

https://i.imgur.com/1uVAWx9.png

Here's the output of the first paragraph, with mistakes in brackets:

> drafts would be laid on the table, and a long discussion would ensue; whereas a Committee would be able to frame a document which, with perhaps a few verbal emundations [emendations], would be adopted; the time of the House would thus be saved, and its business expected [expedited]. With regard to the question of the comparative advantages of The-day [Tuesday]* and Friday, he should vote for the amendment, on the principle that the wishes of members from a distance should be considered on all sensations [occasions] where a principle would not be compromised or the convenience of the House interfered with. He hoped the honourable member for the Town of Christchurch would adopt the suggestion he (Mr. Forssith [Forsaith]) had thrown out and said [add] to his motion the names of a Committee.*

Some mistakes are minor (emnundations/emendations or Forssith/Forsaith), but others are very bad, because they are unpredictable and don't correspond to any pattern, and therefore can be very hard to spot: sensations instead of occasions, or expected in lieu of expedited... That last one really changes the meaning of the sentence.

Re:Mistral OCR by spudlyo on Thursday March 06, 2025 @05:02PM (#43285501)
I want to rejoice that OCR is now a "solved" problem, but I feel like hallucinations are just as problematic as the kind of stuff I have to put up with tesseract -- both require careful manual proofreading for an acceptable degree of confidence. I guess I'll have to try it and see for myself just how much better these solutions are for my public domain archive.org Latin language reader & textbook projects.
Re:Mistral OCR by qingcharles on Thursday March 06, 2025 @07:49PM (#43286537)
It depends on your use-case. For mine, I'm mining millions of scanned PDF pages to get approximate short summaries of long documents. The occasional hallucination won't damage the project. I realize I'm an outlier, and I would obviously prefer a solution that was as accurate as possible.
Re:Mistral OCR by eMPee584 on Thursday March 06, 2025 @07:44PM (#43286519)
possibly doing both & diffing the output to spot contested bits?
Re:Mistral OCR by spudlyo on Thursday March 06, 2025 @08:17PM (#43286665)
that’s my current idea, use two different ocr models and diff the results to spot check for errors. at these prices why not?
Re:Mistral OCR by Kokichi on Thursday March 06, 2025 @07:57PM (#43286573)
All it ever does is hallucinate
Re:Mistral OCR by samstave on Thursday March 06, 2025 @08:02PM (#43286599)
Wrong test.

I want a feed of ALL THE LEGAL DOCUMENTS publically available for clear and flagrant malfeasance. FULL START.

--

I do not give a shit about shakespear.

I want the fire-hose of all the legal documents in library of congress.

---

Furthor EVERY single muniple court should be shaking in their boots and every single legal entity should be terrified that the truth behind legalese lies can be revealed.

Re:Mistral OCR by Biganon on Thursday March 06, 2025 @09:27PM (#43286958)
...are you okay?
Re:Mistral OCR by thomasfromcdnjs on Thursday March 06, 2025 @08:48PM (#43286818)
Does anyone know the correlation between our abilities to parse PDF's and the quality of our LLM's training datasets?

If a lot of scientific papers have been pdf's and hitherto had bad conversions to text/tokens, can we expect to see major gains in our training and therefore better outputs?

Re:Mistral OCR by rossant on Friday March 07, 2025 @02:17AM (#43287918)
Your example doesn't seem that difficult to me.