Re:Better and Faster Large Language Models via Multi-Token P...
by mike_hearn on Wednesday May 01, 2024 @07:59AM
(#40222046)
Is it possible that anti-scaling effect occurs because you are removing some middle layers to free up space for the extra output heads? I only scanned the paper quickly but what happens if you treat the technique as strictly additive and don't keep parameter sizes fixed?
[ Parent
| on HN ]
There are no comments on this comment yet.