News for hackers. Stuff that matters.
Re:The RWKV language model: An RNN with the advantages of a ... by azeirah on Thursday March 30, 2023 @04:49PM (#35378643)
Not entirely certain about this, but I believe that because Apple M-series GPUs use system memory you could possible run the larger models on accessible (albeit expensive) consumer hardware.

Need 64GB for 30B

And 128GB for the 65B

I'm not sure about the performance, but I think it should be ok? Especially given how much Apple has been investing in what -- I believe they call -- neural cores?

Here's some more context: https://news.ycombinator.com/item?id=35105364

There are no comments on this comment yet.