News for hackers. Stuff that matters.

A comment on Running Dolly 2.0 on Paperspace. Parent comment

Re:Running Dolly 2.0 on Paperspace by summarity on Thursday April 13, 2023 @09:12AM (#35554888)
llama.cpp is just a frontend for the GGML tensor library, which required models be converted (optionally quantized) to GGML format.

Of course people are working on that for Dolly as well: https://huggingface.co/snphs/dolly-v2-12b-q4

[ Parent | on HN ]
Re:Running Dolly 2.0 on Paperspace by rodrigobellusci on Thursday April 13, 2023 @11:20AM (#35556827)
Besides converting Dolly's weights to ggml it's also necessary to implement the model using ggml operators, right? Or does the ggml format also carry with it the architecture of the model?
[ Parent | on HN ]