> By contrast, text generation has not improved significantly, in my opinion, for more than a year now
With less than $800 worth of hardware including everything but the monitor, you can run an open weight model more powerful than GPT 3.5 locally, at around 6 - 7T/s[0]. I would say that is a huge improvement.
With less than $800 worth of hardware including everything but the monitor, you can run an open weight model more powerful than GPT 3.5 locally, at around 6 - 7T/s[0]. I would say that is a huge improvement.
[0] https://www.reddit.com/r/LocalLLaMA/comments/1cmmob0/p40_bui...