There seems to be a misconception among non machine learning programmers that ML is all about multiplying matrices. In reality, there is a lot of architecture involved and having a programming languages that facilitates that would be enormously helpful.

Yes, there is a lot of multiplying matrices. But you need to think of a matrix as like a <div> or something and multiplying matrices and placing two <div>s next to each other. Then, think of how to create a dynamic website from that. There is a lot of architecturing involved other than multiplying matrices.