Re:Google Cloud TPU Multislice Training
by DavidSJ on Saturday November 11, 2023 @12:03PM
(#38232077)
Oops, I dropped the factor of 6, so it’s actually 626M tokens (306k sequences), which is about 150x what’s typical.
[ Parent
| on HN ]
There are no comments on this comment yet.