Re:A comprehensive guide to running Llama 2 locally
by npsomaratna on Tuesday July 25, 2023 @10:26PM
(#36872404)
I think you'd need to offload the model into CoreML to do so, right? My understanding is that none of the current popular inference frameworks do this (not yet, at least).
[ Parent
| on HN ]
Re:A comprehensive guide to running Llama 2 locally
by DennisP on Wednesday July 26, 2023 @10:14AM
(#36878071)
I did find some support from Apple: https://machinelearning.apple.com/research/neural-engine-tra...