News for hackers. Stuff that matters.

A comment on Ask HN: Is anyone doing anything cool with tiny language models?. Parent comment

Re:Ask HN: Is anyone doing anything cool with tiny language ... by mettamage on Tuesday January 21, 2025 @03:40PM (#42784953)
Llama 3.2 has about 3.2b parameters. I have to admit, I use bigger ones like phi-4 (14.7b) and Llama 3.3 (70.6b) but I think Llama 3.2 could do de-anonimization and anonimization of code
[ Parent | on HN ]
Re:Ask HN: Is anyone doing anything cool with tiny language ... by OxfordOutlander on Tuesday January 21, 2025 @03:52PM (#42785057)
+1 this idea. I do the same. Just do it locally using ollama, also using 3.2 3b
[ Parent | on HN ]
Re:Ask HN: Is anyone doing anything cool with tiny language ... by RicoElectrico on Tuesday January 21, 2025 @04:21PM (#42785333)
Llama 3.2 punches way above its weight. For general "language manipulation" tasks it's good enough - and it can be used on a CPU with acceptable speed.
[ Parent | on HN ]
Re:Ask HN: Is anyone doing anything cool with tiny language ... by seunosewa on Tuesday January 21, 2025 @05:17PM (#42785773)
How many tokens/s?
[ Parent | on HN ]
Re:Ask HN: Is anyone doing anything cool with tiny language ... by iamnotagenius on Wednesday January 22, 2025 @07:58AM (#42792310)
10-15t/s on 12400 with ddr5
[ Parent | on HN ]