News for hackers. Stuff that matters.

Show HN: TurboPilot: Copilot clone runs code completion LLM on your CPU

https://github.com/ravenscroftj/turbopilot

Hi HN,

I spent my easter weekend stuck in the house with COVID and I decided to play with llama.cpp [1] and fauxpilot [2] to see if I could get LLM code assist working on pure CPU.

As a proof of concept I'd say I've proven that it's possible. However there's still a lot to do. The auto complete is quite slow at the moment. PRs welcome.

[1] https://github.com/ggerganov/llama.cpp

[2] https://github.com/fauxpilot/fauxpilot

( 4 comments | 3 points | on HN )
Re:Show HN: TurboPilot: Copilot clone runs code completion L... by jslakro on Monday April 10, 2023 @04:01PM (#35516739)
Re:Show HN: TurboPilot: Copilot clone runs code completion L... by danduma on Monday April 10, 2023 @07:04PM (#35518480)
Nice, but "Tabby requires Pascal or newer NVIDIA GPU."

As I understand the whole point of TurboPilot is it runs on CPU :D

Re:Show HN: TurboPilot: Copilot clone runs code completion L... by ranguna on Tuesday April 11, 2023 @03:32AM (#35521974)
Tabby infers on the gpu and is slow, I can only imagine how slow truepilot is on the cpu.

If people want auto complete, it needs to be super fast. For slow inference, a better application would be a chatbot that reads your code and answers questions, like cody from sourcegraph.

Re:Show HN: TurboPilot: Copilot clone runs code completion L... by wsxiaoys on Monday April 10, 2023 @10:42PM (#35520104)