I read paper"Matrix calculus you need for deep learning" and if you go through the concepts in blog, it is similar to how pytorch autograd works. Thank you for yor reply, it makes me to think in different angle. May be I should look into Einstein summation convention and try writing another blog. Thanks