News for hackers. Stuff that matters.

A comment on Apple M3 Ultra. Parent comment

Re:Apple M3 Ultra by rfoo on Wednesday March 05, 2025 @01:08PM (#43270100)
For decode, MoE is nice for either bs=1 (decoding for a single user), or bs=<very large> (do EP to efficiently serve a large amount of users).

Anything in between suffers.

[ Parent | on HN ]

There are no comments on this comment yet.