Re:It's Time to Stop Taking Sam Altman at His Word
by wkat4242 on Saturday October 05, 2024 @05:17PM
(#41752978)
An old Ryzen CPU (2600 IIRC) and Radeon Pro VII 16GB. Got it new at a really good price.
It works ok but with a large context it can still run out of memory and also gets a lot slower. With small context it's super snappy and surprisingly good. What it is bad at are facts/knowledge but this is not something a LLM is meant to do anyway. OpenWebUI has really good search engine integration which makes it work like perplexity does. That's a better option for knowledge usecases.
It works ok but with a large context it can still run out of memory and also gets a lot slower. With small context it's super snappy and surprisingly good. What it is bad at are facts/knowledge but this is not something a LLM is meant to do anyway. OpenWebUI has really good search engine integration which makes it work like perplexity does. That's a better option for knowledge usecases.