News for hackers. Stuff that matters.
Re:LocalPilot: Open-source GitHub Copilot on your MacBook by SushiHippie on Thursday October 19, 2023 @11:14AM (#37944013)
FWIW: you can use any other proxy server for this to any openai compatible api server.

e.g. with mitmproxy and llama-cpp-python server

  python -m llama_cpp.server --n_ctx 4096 --n_gpu_layers 1 --model ./path/to/..gguf
and then with mitmproxy in another terminal

  mitmproxy -p 5001 --mode reverse:http://127.0.0.1:8000
and then set this in your vscode settings.json (the same as for localpilot):

  "github.copilot.advanced": {
      "debug.testOverrideProxyUrl": "http://localhost:5001",
      "debug.overrideProxyUrl": "http://localhost:5001"
  }
works way better for me than localpilot
Re:LocalPilot: Open-source GitHub Copilot on your MacBook by cgrs on Thursday October 19, 2023 @12:18PM (#37944893)
Just out of curiosity, why do you need a reverse proxy? Couldn't you connect to that openai-compatible server directly?
Re:LocalPilot: Open-source GitHub Copilot on your MacBook by SushiHippie on Thursday October 19, 2023 @12:26PM (#37944974)
Thanks, I don't know why I didn't think of that. Just tried it and it works at least with the llama-cpp-python server, but I suppose some other servers and very likely hosted services (SNI, certificate subjects, ...) would have problems with that.
Re:LocalPilot: Open-source GitHub Copilot on your MacBook by newman314 on Thursday October 19, 2023 @02:07PM (#37946332)
I'm assuming that this means that I could also potentially connect to a local more powerful box?
Re:LocalPilot: Open-source GitHub Copilot on your MacBook by SushiHippie on Thursday October 19, 2023 @03:09PM (#37947263)
Yup
Re:LocalPilot: Open-source GitHub Copilot on your MacBook by hoschicz on Thursday October 19, 2023 @04:27PM (#37948260)
How is context-from-following-tokens implemented in the pure OAI API? I assumed there must be specialized models that have two separate context windows.