A Trick for Backpropagation of Linear Transformations

Posted by tripplyons on Thursday September 11, 2025 @11:05AM
from the tripplyons.com dept.

https://tripplyons.com/blog/backprop-trick

( 6 comments | 3 points | on HN )

Re:A Trick for Backpropagation of Linear Transformations by greatgib on Sunday September 14, 2025 @07:44AM (#45239114)

Nice and easy article for a normally complicated topic.

[ Parent | on HN ]

Re:A Trick for Backpropagation of Linear Transformations by hansvm on Sunday September 14, 2025 @09:36AM (#45239668)

That's quite a neat trick. You do need to be careful if the same matrix appears more than once in your einsum call though.

[ Parent | on HN ]

Re:A Trick for Backpropagation of Linear Transformations by brandonb on Sunday September 14, 2025 @10:39AM (#45240086)

Very nicely explained. Your article on rendering fractals was also well-done!

[ Parent | on HN ]

Re:A Trick for Backpropagation of Linear Transformations by tripplyons on Sunday September 14, 2025 @07:45PM (#45244498)

Thank you!

[ Parent | on HN ]

Re:A Trick for Backpropagation of Linear Transformations by fzimmermann89 on Sunday September 14, 2025 @05:38PM (#45243468)

..and for complex valued tensors, you need to conjugate.

[ Parent | on HN ]

Re:A Trick for Backpropagation of Linear Transformations by tripplyons on Sunday September 14, 2025 @07:51PM (#45244539)

I've only accounted for real numbers. I'm not sure how to cleanly account for conjugates when some of the einsums would need them and others wouldn't.

For example, a matrix product would need a complex conjugate, but a Hadamard product wouldn't.

If there is an elegant way to extend this to complex numbers, let me know!

[ Parent | on HN ]