Llama.cpp is an inference engine. The author of llama.cpp designed gguf. Funcionary is a model that does function calling. You can download functionary weights in the gguf format and then run it using llama.cpp on low-end machines using CPU or GPU or a mix of both.

ok so the answer is no, thanks

The GP educated you and you still said the answer was "no"? It's clearly "yes".

I read the original question as "is the 'Functionary' thing something that works with any LLM you can run in llama.cpp, not just only with the special 'Functionary' finetuned models?", to which the answer now is clearly no.

Very interesting question, as I also read top-level post as saying it's a llama-cpp-python feature, and therefore I could shove some random dolphin-mixtral or whatever I have on my hard drive at it, and it will work.

This question is confusing to me.

Functionary is a series of function calling llama architecture finetuned models, not a new base model, if thats what you mean.

Hence, it can be turned into a gguf like any other llama finetune.

It just has a very esoteric prompting format, but llama-cpp-python is specifically set up to leverage it.

from the parent:

> TBH I would just recommend using that and all its under-the-radar features (like grammar, and built in function calling support with Functionary).

I never heard about Functionary, this comment seems to say “I would use llama.cpp plus Functionary”. Now, since llama.cpp allows to plug in several models, I was wondering if Functionary works with several models too, ie is Functionary a library that allows to add function calling to models?

The following answer to my comment helped clarifying that Functionary is not a lib, but a specific model, or that is what I understand from the answer. So the answer is “no, you can’t use functionary with many models (eg it’s not like llama.cpp that works with multiple models) because Functionary is a model itself”

:-)