News for hackers. Stuff that matters.

A comment on 3Blue1Brown: Visualizing Attention, a Transformer's Heart. Parent comment

Re:3Blue1Brown: Visualizing Attention, a Transformer's Heart by causal on Monday April 15, 2024 @11:26PM (#40048088)
But if I understand correctly, GPT-4 reduces that to a 1536-dimensional vector. Roughly 1/8th. It's counterintuitive to me.
[ Parent | on HN ]

There are no comments on this comment yet.