I'm surprised to see almost no discussion of fourier series in that paper, considering fourier series is all about representing signals as linear combinations of sinusoidal functions.

You may be interested in [1] where they go to a great extend to show that the convolution operation that we consider in DL is the dual of fourier series [2].

[1] https://geometricdeeplearning.com

[2] https://arxiv.org/pdf/2104.13478.pdf page 27 (23 if you count book pages).

Is convolution in DL not implemented with the FFT as the underlying workhorse?

Probably no since FFT is slower and less parallelizable than products.

I like your article and went to your home page to find more good articles and I like what I saw. Thank you for sharing. The only thing is that I am reading from my phone and the site is not very mobile friendly