Show HN: I made a self-hosted ChatGPT UI

Re:Show HN: I made a self-hosted ChatGPT UI by noduerme on Tuesday March 14, 2023 @10:18AM (#35151921)

Does the llama code that dropped leverage the GPU at all? On an M1 it appears to just run on as many CPU cores as you want to throw at it. The 65B heats up 8 cores real nicely, and it's slow, but I imagine it would be a lot faster on the GPU.

[ Parent | on HN ]

Re:Show HN: I made a self-hosted ChatGPT UI by Tostino on Tuesday March 14, 2023 @10:27AM (#35152051)

I've seen people saying that limiting it to 4 cores out of the 8 total can actually lead to improved performance. Have you seen that?

[ Parent | on HN ]

Re:Show HN: I made a self-hosted ChatGPT UI by noduerme on Tuesday March 14, 2023 @10:30AM (#35152109)

8 starts and runs a bit faster for me if plugged in and before the fan kicks on and the CPU starts throttling. Once that happens it's probably better to stick with 4.

[ Parent | on HN ]

Re:Show HN: I made a self-hosted ChatGPT UI by brianjking on Tuesday March 14, 2023 @11:25PM (#35163425)

All of the llama implementations for Apple are CPU only afaik.

[ Parent | on HN ]