News for hackers. Stuff that matters.

A comment on Show HN: AI companions stack – create and host your own AI companions. Parent comment

Re:Show HN: AI companions stack – create and host your own A... by crooked-v on Wednesday July 12, 2023 @07:31PM (#36703113)
The real limiter is having enough GPU RAM to train and run a usefully large model. Everything else is just fiddly details.
[ Parent | on HN ]
Re:Show HN: AI companions stack – create and host your own A... by brucethemoose2 on Wednesday July 12, 2023 @07:43PM (#36703223)
Not anymore. Llama.cpp and kobold.cpp run well with low vram and non Nvidia GPUs.

Huggingface is full of finetunes now, and I believe a 33b model can be finetuned on a single 3090.

Llama.cpp is developing some kind of training, but I have no idea what the requirements will be.

[ Parent | on HN ]
Re:Show HN: AI companions stack – create and host your own A... by causality0 on Wednesday July 12, 2023 @10:15PM (#36704284)
To me, GPT's real "secret sauce" is how it primes the model for interactive prompting instead of just text completion.
[ Parent | on HN ]
Re:Show HN: AI companions stack – create and host your own A... by ynniv on Wednesday July 12, 2023 @10:42PM (#36704439)
There are "chat" and "instruct" variants of most models now.
[ Parent | on HN ]