News for hackers. Stuff that matters.

A comment on Ask HN: How do you scale transformer context lengths over multiple machines?. Parent comment

Re:Ask HN: How do you scale transformer context lengths over... by brucethemoose2 on Friday November 24, 2023 @03:04AM (#38401609)
Indeed :P

Honestly I'm not sure how context "sharding" works on multiple GPUs atm. Decent, really long context OSS models like Yi 200K and YARN finetunes are very new.

[ Parent | on HN ]

There are no comments on this comment yet.