It's explicitly about shifting your thinking to a more abstract level. It's great if already have a working knowledge of matrices.
If you're ready to get very serious with mathematics, I like Algebra by Mac Lane and Birkhoff. (Note that this is a different book from A Survey of Modern Algebra by the same authors, which I'm sure is okay, but... a different book.)
If you're not ready to get very serious with mathematics... well, then why do you want to learn algebra?
PS I agree about Mac Lane, he's great
Well, I agree, but life is short. Why learn algebra rather than complex analysis or probability theory or functional analysis or differential geometry or mathematical logic or general relativity or quantum mechanics or any one of dozens of other topics from physics or mathematics that will almost surely seem more interesting and fun to a non-mathematician? (Not to mention non-mathematical topics...)
I agree there are tons of other things that people should study, like basic linguistics, relativity, quantum physics, optics etc. Geometry should be on top of the list, for as Plato said “let no one ignorant of geometry enter”.
But I don’t think time is an issue really, if you just spend an hour or so each evening reading you csn quickly get a basic understanding of lots of topics in not thst many years.
http://eu.wiley.com/WileyCDA/WileyTitle/productCd-0471433349...
In Ancient Egypt for example when they needed to explain things like the Pythagorean theorem they would give examples, because there was no concept of abstract variables yet. I think this is similar.
In the same way, many people are comfortable working with vectors, transforming them with matrices etc but they have never been taught to think of transforms in an abstract way. So if they want to check if a certain matrix actually performs the transformation they want, they apply it to a bunch of random vectors to see if it's reasonable instead of just checking what it does to the base vectors, because they have no real concept of linear transforms, and only vague ideas about dimensionality and degrees of freedom.
To some extent it IS phobia, because if you just want to rotate a vector 90 degrees you don't really need concepts of orthonormal transformations so you might be reluctant to look at it that way, but if on the other hand abstract algebra is very natural then the abstraction is trivial to do, it just makes things easier.