Re:Show HN: I've built a locally running Perplexity clone
by noisy_boy on Thursday April 04, 2024 @04:13AM
(#39927744)
Would be good if the readme mentions minimum hardware specs to get a reasonably decent performance. E.g. I have a ThinkPad X1 extreme i7 with MaxQ graphics, any hopes of running this on it without completely ruining the performance?
[ Parent
| on HN ]
Re:Show HN: I've built a locally running Perplexity clone
by nilsherzig on Thursday April 04, 2024 @04:41AM
(#39927915)
You could run the LLM using your CPU and normal (non video) ram. But that's a lot slower. There are people working on making it a lot faster tho. The bottleneck is the transfer speed between the ram Sticks and the CPU.
Just taking a guess, but I wouldn't expect more than a couple tokens (more or less like syllables) per second. Which is probably to slow, since it has to read a couple thousand per search result.
It's hard to provide minimum requirements, since there are so many edge cases.