News for hackers. Stuff that matters.

A comment on Differential Transformer. Parent comment

Re:Differential Transformer by karmasimida on Tuesday October 08, 2024 @12:08PM (#41778916)
I mean it doesn’t necessarily needs 2x QK to match that performance, in terms of accuracy, of a regular transformer right?
[ Parent | on HN ]

There are no comments on this comment yet.