News for hackers. Stuff that matters.
Re:Direct initialization of transformers using larger pretra... by gwern on Friday December 22, 2023 @05:59PM (#38739605)
Amazingly enough, that pretty much works, but you'll never hear about it because the name of it is impossible to search for: https://www.wpeebles.com/Gpt

There are no comments on this comment yet.