News for hackers. Stuff that matters.
Re:Run a 1T parameter model on a 32gb Mac by streaming tenso... by frikk on Tuesday March 24, 2026 @12:46PM (#47505525)
This is interesting work, thank you for sharing. What hardware would you buy today for experimenting? Seems like the new gen of macbook pros are pretty powerful?
Re:Run a 1T parameter model on a 32gb Mac by streaming tenso... by tatef on Tuesday March 24, 2026 @02:38PM (#47507207)
Yes definitely. I use a M1 Max with 32gb of RAM daily and it's about on par from a performance standpoint with the new base M5 Pro 24gb. You can check the benchmarks in the repo if you're interested in seeing specific performance metrics, but investing in Apple hardware with as much memory as possible will generally get you furthest in this game.