You'd have trouble designing a more depressing environment than an open office in a building interior with fluorescent troffers if you tried. Good riddance.
I think what happened is that as the work done in offices gained more value people who worked in offices were convinced that having your own space was more prestigious (by furniture companies, office space companies, etc.) so it became normal to move away from shared cattle pens.
Now shared space is being sold as collaborative so as to insinuate anyone not on board with the concept must be an uncollaborative sort of fellow.
I recall working in a high priced tower a place which made money, some people got proper offices, but most people got three foot beautiful mahogany desks side by side. Expensive desks but floor space was even more expensive.
People hated cublicals because they imagined the alternative was a private office. Instead the alternative is no real private space at all. People started believing cublicals are the worst of both worlds. Not as private as an office, but still limits community and collaboration.
In the media, cublical farms were considered at best a joke and at worst the tyranny of the modern workplace.
What is pushing the trend further is insane real estate prices. Now even executives and high value employees (lawyers, management, etc) are getting pressured to give up offices.
Feels like cargo-cult management.
What changed is that IT operations became a bigger thing, and we started needing more programmers. The traditional cubicle created the illusion of a private space, but made IT ops hard because you couldn't see anybody.
Now open offices are back. Wifi is displacing many of the managed cabling requirements, and people are into the feeling of "energy" that comes with a newsroom like environment.
If you look around you and see a big open room of people futzing around with headphones -- take the hint. You're the clerk of the 2010's.
Good point.
Whenever this topic comes up I wonder why would a business risk de-valuing one of its biggest investments (payroll) by creating a potentially productivity killing environment. But if programming is now considered a cost centre (like clerks were in their day), I wonder what would be considered the profit centre of a software company?
Why should they care if it means that the software devs in a completely different part of the business are now less productive? Most large organisations are so dysfunctional that the negatively impacted parts of the business can't fight back against bean counters from elsewhere.
This is playing out at my work right now with virtual desktops being imposed on all developers, even though it means that projects that used to take 1 min to compile now often take 15 mins to compile. Crazy.
For example, a company I used to work at sold its building and rented (leased?) only three floors and the basement from the new owners. This injected cash into the company. I don't know what the slightly smaller cubicles cost the company in terms of employee productivity. How do we quantify such things?
The next place I worked (steel industry) at had my cubicle which was at least four times as big as that place before but I'm confident my productivity didn't go up four times because of it.
I'll agree with gp though. It is reprehensible to defend open floor plans as anything other than a cost cutting measure.
The thing is it doesn't have to quadruple in order to justify the extra real-estate cost because your monthly salary is (probably) much higher than the monthly rent on your desk space.
If you're paid $120k (10k/month) and rent is $5/sq-ft per month [1]; going from 36 sq-ft of space for your desk (small) to 144 sq-ft of space (4x as much) will cost $540/month. Meanwhile even a 10% productivity decrease would cost ~$1k per month.
Edit: Thinking more about it, the productivity loss is even worse than above. You might be paid $120k/year, but the company expects to extract some multiple of this number in value from the work you do (I've heard target numbers like $500k/year) - so a 10% decrease in productivity is more like $4k/month in lost value.
[1] https://www.quora.com/How-much-is-typical-office-space-per-s...
Additionally, high overhead increases risk. Maximum productivity is only important as long as you have more work than there are employees to do it. What if there's a downturn (as in 2008)? Now you're locked into an expensive lease and no longer have the revenue to support it. The company with lower overhead can hold out longer until things turn around. Employees can be laid off. Getting out of a lease or selling a building are harder.