News for hackers. Stuff that matters.

A comment on Visual ChatGPT. Parent comment

Re:Visual ChatGPT by lysecret on Friday March 10, 2023 @05:03AM (#35092554)
All decent replies but, the truth is that it is mostly a quirk of transformer architecture which scales quadratically in the length of the sequence because transformers look at all pairwise combinations of the input tokens. So you can get the memory to whatever level you want to just increase the sequence length (also known as context):
[ Parent | on HN ]
Re:Visual ChatGPT by Der_Einzige on Friday March 10, 2023 @10:14AM (#35095328)
Flash attention gets this down to linear time complexity with no loss of quality.
[ Parent | on HN ]
Re:Visual ChatGPT by sebzim4500 on Friday March 10, 2023 @12:09PM (#35097359)
No it doesn't, it just improves the constant factor by a lot. Unless you are thinking of block sparse attention, which can chance complexity (assuming you scale sparsity with size) but decreases quality.
[ Parent | on HN ]