News for hackers. Stuff that matters.
Re:Llama 3.1 Omni Model by langcss on Thursday September 19, 2024 @12:53AM (#41588624)
You need a lot more power. I found gpt4o struggles doing basic OCR of printed text by hallucinating alot, while tesseract engine (old skool) gets it perfect. You need the model to be powerful enough to do everything.

You can work around this by the way by sending the output through a checking stage.

So picture -> gpt4o -> out1, picture -> tesseract -> out2, out1,out2 -> llm.

Might work for sound too.

Re:Llama 3.1 Omni Model by killerstorm on Thursday September 19, 2024 @03:11AM (#41589356)
Speech is inherently easier to represent as a sequence of tokens than a high-resolution image.

Best speech to text is already NN transformer based anyway, so in theory it's only better to use a combined model

Re:Llama 3.1 Omni Model by falcor84 on Thursday September 19, 2024 @05:50AM (#41590182)
Interesting, I've actually been using gpt4o extensively for OCR and didn't encounter any significant issues - could I ask you to please give an example of an image of (otherwise legible) text that it hallucinates on?
Re:Llama 3.1 Omni Model by schrodinger on Thursday September 19, 2024 @11:50AM (#41593130)
Same, it's perfect at OCR. Generating an image with text in it however… nope!
Re:Llama 3.1 Omni Model by langcss on Friday September 20, 2024 @01:07AM (#41598970)
I'll send you an email.