Re:Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style...
by tarruda on Thursday March 16, 2023 @07:11PM
(#35190255)
Thanks, I've saw this leaked chat prompt and that gave me some ideas.
So under the hoods, ChatGPT is just a model like Llama where they prepend every user input with a context that makes it behave like a chatbot?
Re:Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style...
by thot_experiment on Thursday March 16, 2023 @07:35PM
(#35190501)
(I don't work at OpenAI so take it with a grain of salt) Yes and No they are similar. It is basically just a fancy autocomplete like llama, but I believe it's specifically been trained on chat content, or at least finetuned on such, and it probably uses a more chat focused labeling scheme on the training data as well to help it perform well on that specific task and be conversational.
If you're not trying to get it to be a chatbot it's much easier, here's a prompt that worked for me on the first try in oneshot predict mode with 13Bq4 on a 1080Ti:
and when given that it finished it with: