You can't get around needing to strap things to your head when you're operating a jackhammer. Nerd at a desk? At the least we should fight for a little dignity.
There is a huge overlap in my experience between people who like open offices because it's easy for them to socialize and ask questions, and people who like to pester their busy coworkers with questions they could have figured out themselves. Also people who get bored easily and like to pester coworkers because they need a break from whatever they've been working on for the last 15 minutes.
Basically whenever I hear someone say, "I love open offices because when I don't know how to do something, my coworkers are right there to help me", I can't help but think, "yeah, I bet you do, and I bet your coworkers are ready to claw their eyes out when you come around asking something because you couldn't be bothered to read a man page". Every time someone touts coworker accessibility for questions as a critical reason to have open offices, I can't help but wonder what the ratio is between the time they spend asking other people questions and the time they spend answering other people's questions, because it always sounds like it's pretty huge.
"We should all be on the Slack/HipChat channel so we can get instant answers to all of our questions!" - the people who have lots of questions
"Great, all I need, another source of interruptions!" - the people who get asked lots of questions
(Guess which one I consider myself to be?)
Active noise cancelling systems generally only suppress continuous steady sounds, e.g. fan noise or motor hum. It actually makes conversations and other transient activities _more_ audible by reducing the background sound.
That does lead the the annoying problem that I have to have something stuck in my ear (vs on, which isn't quite as annoying) for 6ish hours a day. Also, they're expensive af, and require frequent charging since I never remember to turn off the noise cancelling when I'm done with them.
I once worked in an office where the founder would stroll out of his office (natch), and have a conversation with one person, extending to a second person, into a decision-type meeting with more people. Call it an evolving standup.
Even if you're locked into headphones this is can be a big distraction equivalent to not wearing headphones at all.
tl;dr: sometimes "the noise" is not noise.
Besides I really enjoy just silence sometimes, being forced to listen to music does not necessarily improve my focus.
And I work in my own private corner office.
There also exist on-ear headphones that don't apply so much pressure. So it rather seems to me that the model that you tested simply does not fit your requirements.
I also prefer over-ear over on-ear, but did not have the problem with on-ear headphones that you described.
They cut the interview short, and I never heard from them again.
They know we don't like sitting next to someone yapping on the phone all day. They just don't care.
Perhaps I was too harsh, there probably are some businesses that actually don't know that developers are much different from customers support, accountants or typists when it comes to the need for sustained concentration and focus.
(I mean no disrespect to customer support, accountants or typists.)
Ironically, speaking of accountants, I at least have found that in most businesses, the accounting staff do in fact have their own isolated working space. They justify it because of the "sensitivity" of their data, which is true. But the cynic in me thinks it's because they know developers tend to make more money (on average) and so they influence the situation and pitch to the CxO's to "reduce costs" by moving everyone (except themselves) into public spaces.
Your first point is correct -- it's because of the cost savings. But cost savings only applies to those critters who dwell at the bottom of the org chart.
And last but not least, significantly fewer accounts are necessary in a software company compared to the sheer amount of developers. (monkey ones or otherwise)
As long as the company can survive, they just don't want to give the bottom-rungs much at all.
In the context of my own work as a developer, focus for me entails trying to come up with the math to do something new. This entails hours of thinking about it in bed but also trying to juggle the ideas in my head. (I'll be the first to admit that thinking is probably not my strong suit.)
BTW: I'm looking for a New Hampshire based CPA for my sole proprietorship if you know of anyone. :-)
Or the cost savings are reaped by different people than those who bear the productivity hit.
Maybe you're a programmer and you want 100% silence, that should be available.
Maybe you're like me, who gets a lot more done in a small, focused group. You should be able to squat in a collaboration room all day and get shit done.
Maybe you don't mind ambient noise, and like to sit on a couch to work so people can stop by and visit and interrupt you for a chat and to share an idea. That should be available too.
Like most things in life, it comes down to having options available to people, and letting them make choices versus trying to predict or control behavior for "performance".