If nothing else, I continue to be amazed and how uninteroperable certain technologies are.
I had to remove glibc and gcc to get llama to compile on my intel macbook. Masking/hiding them from my environment didn’t work, as it went out and found them and their header files instead of clang.
Which eventually worked fine.
In a forum like this. i’m confused why someone would hate my report of how I had to solve a problem in my circustances. I hope to learn someday.
The reason I considered easy was because I have very little knowledge in this area, in fact this is the first time I ever ran a machine learning model on my computer.
I could not do it with the unmodified pytorch model (my GPU is not powerful enough to run even the 7B model), but I was surprised on how easy it was running with llama.cpp. I literally just followed the steps in the github page.
But I was biased in saying it was easy, since I do have other knowledge (such as C development on Linux) which helped me.
Llama is a very high-quality foundation LLM, you can already run it very easily using llama.cpp and will get the raw output you need. https://github.com/ggerganov/llama.cpp
There's already instructions on how anyone can fine-tune it to behave similarly to ChatGPT for as little as $100: https://crfm.stanford.edu/2023/03/13/alpaca.html