XYZ is the "most optimal" 3-channel colorspace but it's a simple transformation from RGB, so it doesn't matter - the model can learn it if it wants to.
I am beginning this journey, and it takes a lot of the magic out of it.
Really what is happening is just mathematical operations: a lot of matrix multiplications, some simple non-linear functions (because linear algebra alone cannot represent all logic), some statistical stuff to stop the numbers getting out of control.
Importantly there are millions or billions of "magic numbers" that get updated as the model learns called parameters.
Whether you train the network on representation A of the image, or representation B of the image, doesn't change much philosophically. It is just a function f(x) applied before the function g(x, parameters), so you get g(f(X), parameters).
Now you could argue, well it might mean "something". After all you can reduce our brains to being a big mathematical functions.
Possibly.
But I think in this case it is too simple. The AI is just looking at a slightly different representation. Similar to changing VS code from/to dark mode for humans, or something like that. And the modellers might be changing representations all the time anyway as they tinker with things.
The topology of the (real) input data is part of the dimensionality reduction needed for the identification to work, especially at a given number of parameters
Understanding how the data is connected in the input (there are image lines and they form a "shape") helps a lot and CNNs used this to their advantage
Maybe a very powerful RNN could deduce this by itself, but it is wasted processing.
GPT also uses the input topology to its favour when it uses tokens, process text sequentially, etc
This is probably what you would classically call "feature engineering"? And this is all hyperparameter search stuff in a way.
I agree that a higher degree of understanding lessens the emotional impact but often you just need to pause for a second, look back and appreciate what we’ve accomplished as a species.
Sometimes I get on a plane and mid flight I realize “wait a second, I’m in a metal box flying in the sky at 800 km/h and I can breathe, eat, drink and watch tv” and I get goosebumps, even though I’m kind of familiar with the physics of it.
Maybe some things just have an inherent “awe factor” and no matter how well you understand them you still get those butterflies. Or maybe I’m just blabbering :)
Really what is happening is just mathematical operations: a lot of matrix multiplications, some simple non-linear functions (because linear algebra alone cannot represent all logic), some statistical stuff to stop the numbers getting out of control."
And all that happens in the human brain is "just" chemical reactions.
All that happens on the Earth is "just" chemistry.
Possibly."
> We also demonstrate ByteFormer's ability to perform inference with a hypothetical privacy-preserving camera which avoids forming full images by consistently masking 90% of pixel channels, while still achieving 71.35% accuracy on ImageNet.
So the way this algorithm smooshes the matrices is "privacy-preserving" because it doesn't actually take in all the bytes. Just some of them. So it is not an invasion of your privacy, if your phone checks all your stored photos for child pornography, because it never actually _looks_ at the images. It just _peeks_ at a bit of them.
No one cares about software touching their bytes: they should care a lot about software which is built with the null hypothesis of "this person has child pornography, notify the authorities".
Inference from partial images is super cool research for example to deal with corrupted image files, but not for privacy imo.
They raise the philosophical question of what does it mean to see
According to brain scan of Daniel Kish, it seems that despite being blind, he can actually see through echolocation
Reference: https://www.npr.org/2015/11/20/455906507/how-can-you-see-wit...
They should have named the system "Blindsight"[1].
Apple has a serious corporate concern about proposed legal requirements to detect and report child porno[2], while other legal requirements try to protect user privacy. This appears to be an attempt to say, "We can detect your CP image storage without ever actually looking at your stored images."
[1] https://en.wikipedia.org/wiki/Blindsight
[2] https://en.wikipedia.org/wiki/Regulation_to_Prevent_and_Comb...