News for hackers. Stuff that matters.

A comment on Bytes Are All You Need: Transformers Operating Directly on File Bytes. Parent comment

Re:Bytes Are All You Need: Transformers Operating Directly o... by jerpint on Saturday June 03, 2023 @08:38PM (#36181900)
Models never « see » anything to begin with, it’s all matrices. And since we ditched convents, locality doesn’t even matter anymore (almost). RGB is just convenient for humans, nothing says it’s optimal for deep learning
[ Parent | on HN ]
Re:Bytes Are All You Need: Transformers Operating Directly o... by IIAOPSW on Saturday June 03, 2023 @10:12PM (#36182404)
Yeah, real humans see with a fourier transform in a highly optimized basis for projecting 3d down to the 2d retina. Not cold soulless math like those machines!

http://www.math.utah.edu/~bresslof/publications/01-3.pdf

[ Parent | on HN ]
Re:Bytes Are All You Need: Transformers Operating Directly o... by goldenkey on Sunday June 04, 2023 @07:50AM (#36184876)
Much hardware and software for codecs use wave basis as well.

https://en.wikipedia.org/wiki/Discrete_cosine_transform

[ Parent | on HN ]
Re:Bytes Are All You Need: Transformers Operating Directly o... by astrange on Sunday June 04, 2023 @04:13AM (#36183927)
Humans see more than 8-bit RGB. Humans can see light polarization and stereo disparity, but more importantly we can interact with things we look at.

XYZ is the "most optimal" 3-channel colorspace but it's a simple transformation from RGB, so it doesn't matter - the model can learn it if it wants to.

[ Parent | on HN ]