News for hackers. Stuff that matters.
Re:Show HN: I made a self-hosted ChatGPT UI by wongarsu on Tuesday March 14, 2023 @12:58PM (#35154570)
If you run it with 4-bit quantization completely on the CPU (similar to llama.cpp), ChatGPT should run in about 90 GB of RAM. Which is easy to get your hands on for a desktop, but it's out of reach for notebooks.

Also expect performance of couple seconds per token in that setup, for now you need something involving GPUs

There are no comments on this comment yet.