News for hackers. Stuff that matters.

A comment on Sharpened Cosine Distance as an Alternative for Convolutions. Parent comment

Re:Sharpened Cosine Distance as an Alternative for Convolutions by krapht on Friday July 22, 2022 @12:45PM (#32194093)
optimal among linear functions, and the noise is gaussian...

of course noise is not gaussian in many interesting applications and nothing is stopping a nonlinear function from getting better results than a matched filter, we just have no theory to discover what such a function would be

so yes, like you, i'd need to see results on actual data before spending time to learn more about it...

[ Parent | on HN ]
Re:Sharpened Cosine Distance as an Alternative for Convolutions by fxtentacle on Friday July 22, 2022 @01:19PM (#32194537)
In my opinion, this looks promising:

https://github.com/stevenwalton/SCS-CCT

4% improvement over Compact Transformers, which is considered SOTA

EDIT: As meragrin_ pointed out, I read the graph the wrong way. It's actually 4% worse. Sorry.

[ Parent | on HN ]
Re:Sharpened Cosine Distance as an Alternative for Convolutions by meragrin_ on Friday July 22, 2022 @02:08PM (#32195223)
Where do you see improvement? According to the graph and text, there is a 4% decrease in performance. It is slightly worse.
[ Parent | on HN ]