Re:Direct initialization of transformers using larger pretra...
by gwern on Friday December 22, 2023 @05:59PM
(#38739605)
Amazingly enough, that pretty much works, but you'll never hear about it because the name of it is impossible to search for: https://www.wpeebles.com/Gpt
[ Parent
| on HN ]
There are no comments on this comment yet.