Re:Apple Releases Open Source AI Models That Run On-Device
by simonw on Thursday April 25, 2024 @02:14AM
(#40154044)
I tried and failed to follow that. I'm looking for a report from someone who has got it (or something like it) to work.
[ Parent
| on HN ]
Re:Apple Releases Open Source AI Models That Run On-Device
by ein0p on Thursday April 25, 2024 @03:27AM
(#40154535)
I’ve tried their 1.1B model. The only hiccup was that it seems to require mlx 0.10.0 which is what’s in requirements.txt. You also have to place the llama tokenizer file into the model dir - they do not distribute it. The models published for MLX do not seem to be instruction tuned, so with their default prompt they get repetitive. But I suppose you could convert the instruction tuned checkpoints with the script in the repo.