News for hackers. Stuff that matters.
Re:The Fourier transform is a neural network by ashertrockman on Thursday April 29, 2021 @12:30PM (#26983212)
The choice of activation function isn't entirely clear to me, but I think it's definitely possible to make a network that operates entirely in the frequency domain. It would probably be pretty easy to start experimenting with such a thing with the nice complex number and FFT support in PyTorch 1.8. :)

Like you said, there's already a significant connection between convolutional networks and the Fourier domain (the convolution theorem).

Tangentially, I've recently worked on a project that focused on implementing convolution in the Fourier domain, and how that allows one to control useful properties of convolutions (like "smoothness" and orthogonality).

I made a demonstration of convolution in the Fourier domain in PyTorch, which you might find interesting: https://nbviewer.jupyter.org/github/locuslab/orthogonal-conv...

More generally, you could look here for more code and the corresponding paper: https://github.com/locuslab/orthogonal-convolutions

There are no comments on this comment yet.