News for hackers. Stuff that matters.
Re:ChatGPT for Teams by benreesman on Thursday January 11, 2024 @03:07AM (#38949209)
No, they could be using any of the variants of pointwise scalar trig-style embedding, one imagines it's at least a little custom to their particular training setup.

It was just an example of a modern positional encoding. I regret that I implied inside knowledge about that level of detail. They're doing something clever on scalar pointwise positional encoding but as for what who knows.

There are no comments on this comment yet.