Re:What's new in Llama 2 and how to run it locally
by bravura on Monday August 07, 2023 @07:20AM
(#37032971)
Does it support Metal / MPS acceleration?
[ Parent
| on HN ]
Re:What's new in Llama 2 and how to run it locally
by SOLAR_FIELDS on Monday August 07, 2023 @08:51AM
(#37033716)
I’ve gotten mine running with FastChat - they have a Metal/MPS option.
Sadly 7b is not very good for SQL tasks. I think even with RAG it would struggle.