News for hackers. Stuff that matters.
Re:Ask HN: What's the best self hosted/local alternative to ... by coolspot on Wednesday May 31, 2023 @01:16PM (#36140868)
Guanaco is indeed very capable and can replace GPT 3.5 in almost all scenarios, based on my tests.

Easy way to self-host it is to use text-generation-webui[1] and 33B 4-bit quantized GGML model from TheBloke[2].

[1] https://github.com/oobabooga/text-generation-webui

[2] https://huggingface.co/TheBloke/guanaco-33B-GGML

There are no comments on this comment yet.