Re:Llama 32K Context Released by Together AI
by npsomaratna on Saturday July 29, 2023 @01:41AM
(#36917284)
My understanding is that in NTK aware RoPE scaling, the model does pay uniform attention. With older methods, not as much.
[ Parent
| on HN ]
There are no comments on this comment yet.