Re:Transformer Attention is off by one
by PoignardAzur on Monday July 24, 2023 @06:18PM
(#36855021)
6.7B isn't "needs a datacenter" scale.
[ Parent
| on HN ]
Re:Transformer Attention is off by one
by WithinReason on Monday July 24, 2023 @06:28PM
(#36855130)
It's in the million dollar range. XLnet which is a 1.3B model cost $245,000 to train for example.