This is the easiest to setup: https://faraday.dev/

I think Wizard is the “meta” for technical questions now.

Is this open source? I don't see a Linux download link and I want to try building it for Linux myself

Here is an open source project that supports voice as well: https://github.com/cogentapps/chat-with-gpt I think it is meant to be used with ElevenLabs and OpenAI API, but might be easy to configure for use with local Whisper.cpp + llama?

ollama is dead easy to use. It's a single binary that downloads models on demand, like docker downloads images.

  pacman -S ollama
  ollama serve
  ollama run llama2:13b 'insert prompt'

https://ollama.ai/

ollama wraps llama.cpp into a docker container, correct? Besides that it seeks like a go server for chat?

The ollama shell is really nice.

It's not open source but it is still (for now) free: lmstudio.ai. Chat histories, a good config UI, easy prompt management, model management, model discovery, easy set up, cross platform, able to serve an API for connectivity to other tools.

They're hiring and have no currently disclosed monetization strategy so I expect a rugpull soon where some now-free feature gets paywalled or purposefully crippled, but it's not like porcelain apps for free LLMs that rely entirely on llama.cpp to function can do vendor lock-in. I'd second ollama if OSS is a higher priority than features though.

Depends on what you mean by "best"? Absolute bleeding edge fastest possible inference? ExLlama or ExLlamaV2 on a 4090.