When I started programming years ago I had to wear a tie to work, and shared an office with one other person. I've since worked in large open plans, at home, in cubes, and in what would be called team rooms with everyone from the intern to the boss sitting within earshot.
IHMO, team rooms really do work. They help facilitate communication while not bothering those working with undue noise. If the team is under a crunch then naturally non-work conversations are kept to a minimum because the whole team knows it is crunch time. When there is more free time then the team adjusts accordingly.
Ad-hoc team rooms sound like a very good idea though.
This is an important point. For many of us who have worked in startups, "open office plan" really just means "team room". I enjoyed working in open office plans in smaller companies since it basically just meant 5-10 engineers sitting in the same space.
Last year I worked for a mid-size company of about ~200 employees, most of whom were crammed into a gigantic factory like space with endless arrays of fluorescent lights lining the ceiling.
The experience was among the most distracting I can remember. This was in no small part due to having to overhear loud conversations between people in different departments, such as marketing and sales, which require much more verbal communication than engineering. if I can help it I will never work in a similar environment again.
the reason startups do open plan offices is because no one is concentrating on managing the culture of the startup as it grows. Open plan is then I suspect both cause and symptom.
A campaign for team rooms is needed
That's great for a single team all working on one thing - but I do not see the open plan office as a series of nicely appointed open plan offices for 4-8 people with sound-proofing.
I have worked in open plan offices with hundreds of people in the same open-plan - hearing birthdays, laughter, horror, lunchtimes and socialising that have nothing to do with me personally or professionally.
Cornell may have found that it is useful to know what the rest if your physically co-located team is working on - yes - but not hundreds of others that I need to filter out too.
If serendipity is valuable to you as a company, throw a Wednesday afternoon mixer and expect groups to form and make something new to try out. But give them team rooms for that wednesday.
I am a big fan of "team rooms", and a big big fan of remote working and always on video. But living in the middle of hundreds of others just teaches us to ignore everyone.