While you are right, a related issue is that most maths textbooks are atrocious at most aspects that aren't writing pages of equations. There is almost nobody sitting at the 3-way intersection of great mathematician, great writer and great educator who can then write great textbooks.
I've found books with terms like "History" and "Philosophy" in the title are much better places to learn about mathmatics; combined with wikipedia for formulas and details. Any book with excercises but no solutions or historical context has turned out to be basically useless to me, even as a reference (wikipedia is usually better for simple stuff).
I've got something like 4 books on statistics on my shelf at the moment. The only one that I've actually manged to read and learn something from has been Chatterjee's Philosophy of Statistics, because it talks about what techniques were developed in context of which problem, failed alternative approaches, explains what was confusing to some of the greatest minds in the history of statistics, etc. This has been vastly more useful in setting up a framework for what the world of statistics looks like that I can attach a whole bunch of proofs and suchlike too. It has been enlightening in a way that textbooks can't really manage.
A person that doesn't understand the distributive property, going into Calculus 1 is not the person that is going to understand Rudin or Spivak on their own without guidance. Yet, people in mathematics communities would recommend Spivak or Rudin anyway and look down on them if they used Stewart or Thomas.
To them if you aren't learning a subject in the fullest rigor possible, then it isn't worth learning. It doesn't matter if the person doesn't even know the basics, they blindly recommend these type of books regardless.
These are excellent books once you already seen the material before, not so much for the average person that doesn't know what the distributive property is.
There's a difference between a person starting off in mathematics, versus someone who has done several undergrad/grad level courses and knows the "system". If you've gone through math formally earlier in your life, you should be able to pick up a math textbook and learn the subject. Unfortunately, many math textbook do not make this easy.
That's why Khan Academy is a so fantastical resource. They really build a very clear explanation of the concepts. Every teacher should watch it before classes.
It's not free but it's an amazing comic book textboom and fun instersting challenging problems for kids.
Buy the books and share them with your neighbors, and share an online account if you can't afford your own.
Here's a simple example. The distributive property says:
a (b + c) = ab + ac
I can teach students to expand 3(x + 2). With some practice almost all of them will get this. They'll say
3(x+2) = 3x + 3(2) = 3x + 6
Do you know how hard it is to convince people that due to the nature of equality you can reverse the steps? Some never understand that
3x + 6 = 3(x + 2)
Some will never understand that 3x + 2x = 5x because of the distributive property and that combining like terms for the expression 3x + ax is the same process.
I don't think there is any desire to be elitist in terms of having good books that are appropriate. It's just hard to do. We've gone through various cycles of reforming calculus, introducing old concepts in a new way. Through all of the changes one thing has remained constant in my experience. The percentage of people who can get it remains the same.
I do agree with your point on providing solutions.