Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on New iMac with M4. Parent comment
Larger models that don’t fit are only moderately better, while much slower. I’d want higher memory bandwidth over more memory.
The next five years will push for better models that are smaller. Smaller is faster and more useful.
I feel like we are in the brute force phase of model development and that it will pass.
Running 405B param model in 16bit on my laptop would be neat, but I’d stop after the novelty wore off.
There are no comments on this comment yet.
Larger models that don’t fit are only moderately better, while much slower. I’d want higher memory bandwidth over more memory.
The next five years will push for better models that are smaller. Smaller is faster and more useful.
I feel like we are in the brute force phase of model development and that it will pass.
Running 405B param model in 16bit on my laptop would be neat, but I’d stop after the novelty wore off.