I worked for a company who specifically made it clear that "rising up in the ranks" was not the only way to progress. People could stay out of the managerial ranks and theoretically still advance. However, this is not how it worked out in practice. The sense that managers were somehow better than the non-managers still pervaded.
I think it's a result of it being the managers' job to tell non-managers what to do, schedule their time, etc. Managers, by definition, exert control over the time and lives of non-managers. I think this affects the interpersonal relationships between the groups.
It might also have been affected by the perception that people outside the organization had. Managers have titles that confer on them a leadership position, so outsiders automatically gravitate to them as persons in charge. Perhaps it was a failure in execution by our organization to control expectations of our clients and customers, but I don't think so.
You may get asked, "so you never thought about management?". But you can rehearse a response to that. It's like the one about when you've been in a difficult situation. Example, "In a way I was managing" - you worked on this project, planned it this way, made these decisions. Or you could describe aspects you liked about the pace or type of work you were doing. Talk about the problems that would have been easier if you had been leading strategy, and about the mechanisms you instead had to use to influence strategy, and the reason why the tradeoff worked for you better than if you'd been managing. Use the question as an opportunity to highlight particular technical skills that you brought to the team and mention the rule of comparative advantage.
A great thing about IT is that the tools are so cheap now that anyone who wants to prove themselves can run a home project at virtually no cost to demonstrate their strengths.
Ageism in IT is very real. This is just one of the mechanisms by which it operates. The other side is the (youngest) programmers who haven't yet grown out of language fanboyism and dismiss your experience of last year's technology as irrelevant. Fortunately they're rarely in positions of hiring authority.
The IT industry takes about 10 years to cycle through fads. Someone I consider experienced has been through at least one cycle. I want people who can say XML? Oh yes, well I've never used XML but in the 90s I was doing EDI... (insert tech of choice)
At my previous company, where I began in the startup stage, we had lots of people older than 40 and 50 in IT. We had a QA guy who had to be at least 60. I suppose it all depends on the company (and probably the geographic subculture) but this "ageism" stuff is out of my experience -- I'll hire someone who knows what he's doing and is motivated to do it, period. Stupid shit like "but he has a wrinkle" is moot.
Same in banking, you might be able to stay a trader, if you're good. You couldn't stay an analyst, it's up or out. Or in engineering, you don't get to actually design machinery or chips or whatever for long, again, up or out, gotta make room for the fresh intake of graduates.
This might not be the best way to work, but it's the norm, and a programmer who really can stay hands-on their whole career is fortunate indeed. My own strategy in this area is to stay hands-on but do less work directly, spend my time automating common tasks and developing tools to make junior members of my team more productive. We'll have to see how that plays out...
It is hard to become a system architect without many years of experience under your belt.
I know a lot of very good developers who are in their 30s and 40s (not too many who are in their 50s, although I'm sure they exist and there will be more in the near future!) and have no desire to stop coding and become management. In some cases it's a very conscious lifestyle choice: you can maintain much more flexible hours and even do telework if you are writing code, while someone in a management role has to be physically present more often.
The idea that everyone should move into management is poisonous, both to developers who want to remain developers and feel pushed, and to management, which gets filled with people who don't really want to be there.
Some of the better places I've worked have recognized that some people want to get into project or staff management, and other people would prefer to perfect their craft and work independently. They link salary not to progress along a "ladder," but to actual value to the organization -- which often leads to developers who are making more than their project managers. This strikes me as totally appropriate, and it surprises me sometimes that not all companies are like this.
A company that pushes its best developers into management positions they're not thrilled about (either through 'career planning' or by making that the only way to get raises) is shooting itself in the foot in the long run.
"Traditionally, programmers are offered two career paths. They can become managers (a CTO track) or senior programmers (an architect track). The problem is both tracks terminate in roles that many programmers don't want. One path leads to pure management, the other to systems design work. Where do programmers who actually want to program go?
"Elsewhere."