News for hackers. Stuff that matters.

A comment on The Fourier transform is a neural network. Parent comment

Re:The Fourier transform is a neural network by amcoastal on Thursday April 29, 2021 @08:46AM (#26980459)
I dont think its surprising but its definitely a cool little exercise. Somewhat aside of the content of the post, I could see some use cases where specifically implementing DFT layers in your architectures could lead to improved performance over using just the raw inputs or activations. Noisy datasets, or synthetic data based transfer learning come to mind as potential uses for the DFT as a step in your ML pipeline.
[ Parent | on HN ]
Re:The Fourier transform is a neural network by threatripper on Thursday April 29, 2021 @08:55AM (#26980551)
There is the FFT in TensorFlow already: https://www.tensorflow.org/api_docs/python/tf/signal/fft
[ Parent | on HN ]
Re:The Fourier transform is a neural network by amelius on Thursday April 29, 2021 @09:13AM (#26980710)
And FFT (Fast Fourier Transform) is more efficient than matrix multiplication (EDIT: by a matrix with Fourier elements).
[ Parent | on HN ]
Re:The Fourier transform is a neural network by bigbillheck on Thursday April 29, 2021 @10:12AM (#26981409)
You know, I'll go out on the limb and say (unhelpfully) that a FFT is more efficient than any general dense matvec.
[ Parent | on HN ]
Re:The Fourier transform is a neural network by touisteur on Thursday April 29, 2021 @04:33PM (#26986344)
Bah, when you've maxxed out your cuda cores, who's to prevent you from reaching for those sweet tensor core 100+ tflops mat-mul now?
[ Parent | on HN ]
Re:The Fourier transform is a neural network by mochomocha on Thursday April 29, 2021 @09:58AM (#26981233)
This is analogous to saying "an umbrella is better than a toaster". They are two different things.
[ Parent | on HN ]
Re:The Fourier transform is a neural network by Joker_vD on Thursday April 29, 2021 @10:22AM (#26981523)
Why do people insist on comparing only things that are same? The same things are same, no point whatsoever comparing them: they're same! No, you should only compare things that are different, that way you learn more about their differences.

And what's better, an umbrella or a toaster, depends on the one's goals: if one'd like to fry some bread, toaster is more useful; if one'd like to cover from rain, umbrella is far superior, although toaster could used be too; if one'd like to drive a nail into the wall, toaster again is better; etc.

[ Parent | on HN ]
Re:The Fourier transform is a neural network by mochomocha on Thursday April 29, 2021 @11:35AM (#26982508)
My statement was as approximate as the one I was replying to. If the original statement had been "you can use FFT to implement some operations involving matrix multiplication faster" I wouldn't have taken issue with it. But saying "FFT is faster than matrix multiplication" has little meaning.
[ Parent | on HN ]
Re:The Fourier transform is a neural network by vladTheInhaler on Thursday April 29, 2021 @12:08PM (#26982961)
The naive implementation of the DFT in terms of a matrix-vector multiplication has time complexity O(n^2) [1]. The point is that the FFT approach is much faster than that implementation.

[1] https://en.wikipedia.org/wiki/DFT_matrix

[ Parent | on HN ]
Re:The Fourier transform is a neural network by cdavid on Thursday April 29, 2021 @10:08AM (#26981348)
Fourier transform, more exactly DFT, is a special case of matrix multiplication, when seen as a linear operator. The fact that you can do operations faster than the usual N*2 matrix / vector operation is due to the specific form of the underlying matrix (Van Der Mont matrix).

So they are not that different: the specific structure of the Fourier basis allows for more efficient matrix * vector operation.

[ Parent | on HN ]
Re:The Fourier transform is a neural network by layoutIfNeeded on Thursday April 29, 2021 @11:12AM (#26982207)
Vandermonde. He was not Dutch, he was French. https://en.wikipedia.org/wiki/Alexandre-Théophile_Vandermond...
[ Parent | on HN ]
Re:The Fourier transform is a neural network by ok123456 on Thursday April 29, 2021 @10:27AM (#26981590)
You can implement a DFT with matrix multiplication using the vandermonde matrix with roots of the unitary polynomial as the basis.
[ Parent | on HN ]
Re:The Fourier transform is a neural network by hamiltonkibbe on Thursday April 29, 2021 @10:49AM (#26981879)
They are different things, but you can use both to protect you from the rain, and an umbrella does that more efficiently than a toaster. The drawback is that the umbrella doesn't generalize to the broader field of bagel toasting
[ Parent | on HN ]
Re:The Fourier transform is a neural network by amelius on Thursday April 29, 2021 @10:02AM (#26981280)
Not if you perform these operations in order to obtain a Fourier transform.
[ Parent | on HN ]
Re:The Fourier transform is a neural network by amcoastal on Thursday April 29, 2021 @07:51PM (#26988306)
That's really cool. I didn't know they had this, maybe they have a 2 dimensional version as well.
[ Parent | on HN ]