Bytes Are All You Need: Transformers Operating Directly on File Bytes

Re:Bytes Are All You Need: Transformers Operating Directly o... by raverbashing on Sunday June 04, 2023 @02:59AM (#36183638)

Yeah but this is an ML 101 view of the thing, so let me put it this way

The topology of the (real) input data is part of the dimensionality reduction needed for the identification to work, especially at a given number of parameters

Understanding how the data is connected in the input (there are image lines and they form a "shape") helps a lot and CNNs used this to their advantage

Maybe a very powerful RNN could deduce this by itself, but it is wasted processing.

GPT also uses the input topology to its favour when it uses tokens, process text sequentially, etc

[ Parent | on HN ]

Re:Bytes Are All You Need: Transformers Operating Directly o... by quickthrower2 on Sunday June 04, 2023 @07:40AM (#36184821)

If this is the case, and I believe you, then this implies: "bytes are NOT all you need". You need a good representation of your image for the network to work well.

This is probably what you would classically call "feature engineering"? And this is all hyperparameter search stuff in a way.

[ Parent | on HN ]