Re:A comprehensive guide to running Llama 2 locally
by mschuster91 on Tuesday July 25, 2023 @04:23PM
(#36868811)
> DISCLAIMER: I'm one of the developers behind Ollama.
I got a feature suggestion - would it be possible to have the ollama CLI automatically start up the GUI/daemon if it's not running? There's only so much stuff one can keep in a Macbook Air's auto start.
Re:A comprehensive guide to running Llama 2 locally
by jmorgan on Tuesday July 25, 2023 @04:38PM
(#36869062)
Good suggestion! This is definitely on the radar, so that running `ollama` will start the server when its needed (instead of erroring!): https://github.com/jmorganca/ollama/issues/47
Re:A comprehensive guide to running Llama 2 locally
by DennisP on Tuesday July 25, 2023 @05:32PM
(#36869870)
I've been wondering, is the M2's neural engine usable for this?
Re:A comprehensive guide to running Llama 2 locally
by npsomaratna on Tuesday July 25, 2023 @10:26PM
(#36872404)
I think you'd need to offload the model into CoreML to do so, right? My understanding is that none of the current popular inference frameworks do this (not yet, at least).
Re:A comprehensive guide to running Llama 2 locally
by DennisP on Wednesday July 26, 2023 @10:14AM
(#36878071)
I did find some support from Apple: https://machinelearning.apple.com/research/neural-engine-tra...
DISCLAIMER: I'm one of the developers behind Ollama.