> TBH I would just recommend using that and all its under-the-radar features (like grammar, and built in function calling support with Functionary).
I never heard about Functionary, this comment seems to say “I would use llama.cpp plus Functionary”. Now, since llama.cpp allows to plug in several models, I was wondering if Functionary works with several models too, ie is Functionary a library that allows to add function calling to models?
The following answer to my comment helped clarifying that Functionary is not a lib, but a specific model, or that is what I understand from the answer. So the answer is “no, you can’t use functionary with many models (eg it’s not like llama.cpp that works with multiple models) because Functionary is a model itself”
:-)
Functionary is a series of function calling llama architecture finetuned models, not a new base model, if thats what you mean.
Hence, it can be turned into a gguf like any other llama finetune.
It just has a very esoteric prompting format, but llama-cpp-python is specifically set up to leverage it.