News for hackers. Stuff that matters.
Re:Stanford Alpaca web demo suspended “until further notice” by tarruda on Friday March 17, 2023 @05:05PM (#35202895)
> if these models are trained on essentially raw WWW sourced data, I would personally want raw output.

Llama is a very high-quality foundation LLM, you can already run it very easily using llama.cpp and will get the raw output you need. https://github.com/ggerganov/llama.cpp

There's already instructions on how anyone can fine-tune it to behave similarly to ChatGPT for as little as $100: https://crfm.stanford.edu/2023/03/13/alpaca.html

Re:Stanford Alpaca web demo suspended “until further notice” by b33j0r on Friday March 17, 2023 @05:30PM (#35203181)
“Easily” was a minor canard, or at least… it took me a couple of efforts over a couple of days to get the dependencies to play as nicely as “someone with a brand new M2 arm laptop.”

If nothing else, I continue to be amazed and how uninteroperable certain technologies are.

I had to remove glibc and gcc to get llama to compile on my intel macbook. Masking/hiding them from my environment didn’t work, as it went out and found them and their header files instead of clang.

Which eventually worked fine.

Re:Stanford Alpaca web demo suspended “until further notice” by b33j0r on Saturday March 18, 2023 @04:57AM (#35207433)
Well. I guess I’ll never understand why someone would dislike a comment about an experience different from theirs

In a forum like this. i’m confused why someone would hate my report of how I had to solve a problem in my circustances. I hope to learn someday.

Re:Stanford Alpaca web demo suspended “until further notice” by tarruda on Saturday March 18, 2023 @09:13AM (#35208828)
I did not dislike your comment.

The reason I considered easy was because I have very little knowledge in this area, in fact this is the first time I ever ran a machine learning model on my computer.

I could not do it with the unmodified pytorch model (my GPU is not powerful enough to run even the 7B model), but I was surprised on how easy it was running with llama.cpp. I literally just followed the steps in the github page.

But I was biased in saying it was easy, since I do have other knowledge (such as C development on Linux) which helped me.