News for hackers. Stuff that matters.

A comment on Quantized Llama models with increased speed and a reduced memory footprint. Parent comment

Re:Quantized Llama models with increased speed and a reduced... by simonw on Thursday October 24, 2024 @05:10PM (#41939828)
MLC Chat is a great iPhone app for running models (it's on Android too) and currently ships with Llama 3.2 3B Instruct - not the version Meta released today, its a quantized version of their previous release.

I wouldn't be surprised to see it add the new ones shortly, it's quite actively maintained.

https://apps.apple.com/us/app/mlc-chat/id6448482937

[ Parent | on HN ]
Re:Quantized Llama models with increased speed and a reduced... by Havoc on Thursday October 24, 2024 @08:43PM (#41941255)
Seems much more stable than the last time I tried it too
[ Parent | on HN ]