Ask HN: Is anyone doing anything cool with tiny language models?

Re:Ask HN: Is anyone doing anything cool with tiny language ... by politelemon on Tuesday January 21, 2025 @03:24PM (#42784789)

Could you recommend a tiny language model I could try out locally?

[ Parent | on HN ]

Re:Ask HN: Is anyone doing anything cool with tiny language ... by mettamage on Tuesday January 21, 2025 @03:40PM (#42784953)

Llama 3.2 has about 3.2b parameters. I have to admit, I use bigger ones like phi-4 (14.7b) and Llama 3.3 (70.6b) but I think Llama 3.2 could do de-anonimization and anonimization of code

[ Parent | on HN ]

Re:Ask HN: Is anyone doing anything cool with tiny language ... by OxfordOutlander on Tuesday January 21, 2025 @03:52PM (#42785057)

+1 this idea. I do the same. Just do it locally using ollama, also using 3.2 3b

[ Parent | on HN ]

Re:Ask HN: Is anyone doing anything cool with tiny language ... by RicoElectrico on Tuesday January 21, 2025 @04:21PM (#42785333)

Llama 3.2 punches way above its weight. For general "language manipulation" tasks it's good enough - and it can be used on a CPU with acceptable speed.

[ Parent | on HN ]

Re:Ask HN: Is anyone doing anything cool with tiny language ... by seunosewa on Tuesday January 21, 2025 @05:17PM (#42785773)

How many tokens/s?

[ Parent | on HN ]

Re:Ask HN: Is anyone doing anything cool with tiny language ... by iamnotagenius on Wednesday January 22, 2025 @07:58AM (#42792310)

10-15t/s on 12400 with ddr5

[ Parent | on HN ]