Like you said, there's already a significant connection between convolutional networks and the Fourier domain (the convolution theorem).
Tangentially, I've recently worked on a project that focused on implementing convolution in the Fourier domain, and how that allows one to control useful properties of convolutions (like "smoothness" and orthogonality).
I made a demonstration of convolution in the Fourier domain in PyTorch, which you might find interesting: https://nbviewer.jupyter.org/github/locuslab/orthogonal-conv...
More generally, you could look here for more code and the corresponding paper: https://github.com/locuslab/orthogonal-convolutions
In particular I'm interested in the efficiency gains from avoiding convolutions and the possibility of running a compressed frequency-domain NN on CPUs.
Yeah it kinda works when you feed JPEG coefficients into a typical time-domain CNN, but mathematically it seems that if you're using frequencies as inputs, your convolutions should become simple multiplications. Am I wrong?
- a ton of data (JPEGs, MPEGs, MFCCs) exists in the compressed form of FFT-ed features
- FFTs are hardware encoded/decoded & optimised
- FFTs are great at separating signal from noise, leading to a huge reduction in input features (eg JPEG encoded) as compared to raw pixel values
- convolutions in the time domain are just multiplications in the frequency domain
- converting to log polar coordinates turns them into additions
- distance between two log polar coordinates could be the loss function
However, I've not had much luck getting a frequency domain network like this to converge. Anyone tried something similar with success?