Ask HN: Is anyone doing anything cool with tiny language models?

Re:Ask HN: Is anyone doing anything cool with tiny language ... by JLCarveth on Tuesday January 21, 2025 @08:22PM (#42787549)

I used a small (3b, I think) model plus tesseract.js to perform OCR on an image of a nutritional facts table and output structured JSON.

[ Parent | on HN ]

Re:Ask HN: Is anyone doing anything cool with tiny language ... by deivid on Wednesday January 22, 2025 @12:11AM (#42789249)

What was the model? What kind of performance did you get out of it?

Could you share a link to your project, if it is public?

[ Parent | on HN ]

Re:Ask HN: Is anyone doing anything cool with tiny language ... by JLCarveth on Wednesday January 22, 2025 @07:54AM (#42792274)

https://github.com/JLCarveth/nutrition-llama

I've had good speed / reliability with TheBloke/rocket-3B-GGUF on Huggingface, the Q2_K model. I'm sure there are better models out there now, though.

It takes ~8-10 seconds to process an image on my M2 Macbook, so not quite quick enough to run on phones yet, but the accuracy of the output has been quite good.

[ Parent | on HN ]

Re:Ask HN: Is anyone doing anything cool with tiny language ... by tigrank on Wednesday January 22, 2025 @01:34AM (#42789735)

All that server side or client?

[ Parent | on HN ]

Re:Ask HN: Is anyone doing anything cool with tiny language ... by ian_zcy on Wednesday January 22, 2025 @04:27AM (#42790829)

what are you feed into the model? Image (like product packaging) or Image of Structured Table? I found out that model performs good in general with sturctured table, but fails a lot over images.

[ Parent | on HN ]