News for hackers. Stuff that matters.
Re:A comprehensive guide to running Llama 2 locally by DennisP on Tuesday July 25, 2023 @05:32PM (#36869870)
I've been wondering, is the M2's neural engine usable for this?
Re:A comprehensive guide to running Llama 2 locally by npsomaratna on Tuesday July 25, 2023 @10:26PM (#36872404)
I think you'd need to offload the model into CoreML to do so, right? My understanding is that none of the current popular inference frameworks do this (not yet, at least).
Re:A comprehensive guide to running Llama 2 locally by DennisP on Wednesday July 26, 2023 @10:14AM (#36878071)