News for hackers. Stuff that matters.

A comment on Ask HN: Is anyone doing anything cool with tiny language models?. Parent comment

Re:Ask HN: Is anyone doing anything cool with tiny language ... by deivid on Wednesday January 22, 2025 @12:11AM (#42789249)
What was the model? What kind of performance did you get out of it?

Could you share a link to your project, if it is public?

[ Parent | on HN ]
Re:Ask HN: Is anyone doing anything cool with tiny language ... by JLCarveth on Wednesday January 22, 2025 @07:54AM (#42792274)
https://github.com/JLCarveth/nutrition-llama

I've had good speed / reliability with TheBloke/rocket-3B-GGUF on Huggingface, the Q2_K model. I'm sure there are better models out there now, though.

It takes ~8-10 seconds to process an image on my M2 Macbook, so not quite quick enough to run on phones yet, but the accuracy of the output has been quite good.

[ Parent | on HN ]