News for hackers. Stuff that matters.

A comment on The Matrix Calculus You Need for Deep Learning. Parent comment

Re:The Matrix Calculus You Need for Deep Learning by PartiallyTyped on Monday July 31, 2023 @01:22PM (#36945894)
Adding, these operators are also "polymorphic"; for matrix multiplication the only operations you need are (non commutative) multiplication and addition; thus you can use elements of any non-commutative ring, i.e. a set of elements with those two operations :D

Matrices themselves form non-commutative rings too; and based on this, you can think of a 4N x 4N matrix as a 4x4 matrix whose elements are NxN matrices [1] :D

[1] https://youtu.be/FX4C-JpTFgY?list=PL49CF3715CB9EF31D&t=1107

You already know whose lecture it is :D

I love math.. I should have become a mathematician ...

[ Parent | on HN ]
Re:The Matrix Calculus You Need for Deep Learning by mrfox321 on Monday July 31, 2023 @01:37PM (#36946113)
Re [1]: it's fairly concrete to simply say that matrix multiplication can be performed block-wise.
[ Parent | on HN ]
Re:The Matrix Calculus You Need for Deep Learning by PartiallyTyped on Monday July 31, 2023 @02:25PM (#36946777)
I don’t disagree; but that is just an example of MM. The gist is not that you can do block multiplication; but that you can define matrices over any valid ring, which includes other matrices - ie blocks.
[ Parent | on HN ]
Re:The Matrix Calculus You Need for Deep Learning by mrfox321 on Monday July 31, 2023 @03:20PM (#36947501)
Yeah matrices are more abstract. I guess I am just pointing out that your concrete example of non-commutative rings (matrices of matrices) still needs a proof to demonstrate bijection between 4N x 4N (scalar) and 4 x 4 (N x N(scalar)).

Block MM demonstrates the equivalence.

[ Parent | on HN ]
Re:The Matrix Calculus You Need for Deep Learning by tikhonj on Monday July 31, 2023 @08:13PM (#36950671)
You can even generalize linear algebra algorithms to closed semirings and have some really cool algorithms pop out, like finding the shortest path in graphs. There's a great paper called "Fun with Semirings" that goes into more details; unfortunately looks like the PDF isn't easily available online any more, but I found some slides[1] that seem to cover the same ideas well enough.

[1]: https://pdfs.semanticscholar.org/2e43/477e26a54b2d1a046c2140...

[ Parent | on HN ]
Re:The Matrix Calculus You Need for Deep Learning by PartiallyTyped on Tuesday August 01, 2023 @02:39AM (#36953053)
This deserves its own HN post imho.
[ Parent | on HN ]
Re:The Matrix Calculus You Need for Deep Learning by PartiallyTyped on Tuesday August 01, 2023 @06:47AM (#36954495)
Okay I went over the slides and good lord this would have made my life easier not too long ago.
[ Parent | on HN ]