My own point of view is that linear algebra is by far the most successful part of mathematics: 'Most' questions you can come up with have satisfactory answers. This is in contrast to, say, number theory, where there's a bunch of nice elementary results and a lot of interesting questions that seem nigh impossible to solve.

As a result, it's a pretty common game in mathematics to start with something new or difficult that you want to describe, and then do your level best to turn your questions into linear algebra problems so that you can actually get answers. The extent to which this doesn't work is the extent to which you need to develop new ideas. (One example of such an approach is algebraic graph theory. Turn a graph into an interesting matrix, and then use the linear algebraic properties of that matrix to describe interesting properties of your graph.)

The link to graph theory is beautiful, too. Entries in an matrix can represent edge weights, and taking a random walk on a graph can be represented as a matrix-vector multiplication, and the stationary distribution is the singular vector. How cool is that? When you start to link together abstractions from different fields of mathematics and science, you get these fantastic insights that are just mind bogglingly awesome. This is what makes all the pain of wading through an ocean of symbols and equations worthwhile, imho.

I view that as "if you can find a way to express your problem in terms of linear equations, there are well-known techniques for finding solutions."