> the training process can be seen as a giant matrix multiplication in one step.

It can't be seen that way.

Deep learning becomes deep when you alter linear matrix multiplications with non-continuous functions. After that you can't reduce the problem into one step. You also lose associativity so you can't even do matrix chain multiplication optimization.