News for hackers. Stuff that matters.

A comment on Ask HN: How do you scale transformer context lengths over multiple machines?. Parent comment

Re:Ask HN: How do you scale transformer context lengths over... by mlstudies on Friday November 24, 2023 @02:51AM (#38401548)
wouldn't that mean trying to fit it on one machine?
[ Parent | on HN ]
Re:Ask HN: How do you scale transformer context lengths over... by brucethemoose2 on Friday November 24, 2023 @03:04AM (#38401609)
Indeed :P

Honestly I'm not sure how context "sharding" works on multiple GPUs atm. Decent, really long context OSS models like Yi 200K and YARN finetunes are very new.

[ Parent | on HN ]