Lots of people consider namespacing and packaging up your source well as a way to do this. I think that works up to a point, but once you start hitting really big projects, I think the way that Java enforces and cleans up some of this optional stuff and makes these things as part of the language makes it a little more natural. Instead of having to be careful about your work (meaning if you aren't something will go wrong), you simply have to do the best practice to make progress. On smaller projects, where the manager can review every commit, every line of code, every way that work was done, and can enforce a cohesive and singular idea about how it should be done, it can work. But once the scope expands beyond what one management drone can handle it starts to become something that falls into the next layer of Management Science abstraction.
tl;dr Making things that are optional in some languages into requirements (or the code won't work) seems to have been a good optimization.
Now imagine writing a complex application like Excel, or Photoshop procedurally. It's not impossible. During the Windows 3 series that was actually done.
Here's how it looks :
http://www.transmissionzero.co.uk/computing/win16-apps-in-c/
Read through it. Ideally install Borland C++ in a windows 3.1 in Dosbox and make it work. Then you will fully understand why people switched to OO.
But the more general reason is this : procedural programming doesn't allow you to raise the abstraction level of your code more than a bit. OO allows for more "distant" abstractions. Algorithms that are far further removed from the actual datatype being operated on. Since we've seen a regression in program complexity due to web browsers being the main application delivery method, OO is falling out of favor.
On the other hand, maybe mobile development is simply the pendulum already swinging back. Both android and ios apps are OO apps, by necessity. And as everybody (hopefully) agrees, they beat the pants of websites.
But yeah, I wouldn't want to write Excel in procedural C.
One of the things that's very hard to understand is how much large efforts rely on our ability and understanding to manage and coordinate the effort of all these people providing work input -- yet management science is still remarkably poorly understood.
OO is nice because objects define work-units which are easy to measure and track through a management structure and the development of which kind of map to a work breakdown structure in much the same way assemblies and sub-assemblies go into a bridge or a car in traditional engineering:
Phase 1:
Part A: develop all the classes and methods on each class.
Part B: test and certify components.
Phase 2:
Part A: integrate 1A components.
Part B: test and certify integration.
Phase 3: repeat phases 1 and 2 until you have finished software
"oh team X is working on component (class) A2? when does A2 move to testing? okay, I'll schedule that. You know component B6? Yeah, it failed testing and needs to be reworked, but it shouldn't hold up phase 2: integration."
Lots of people give Java and Java shops lots of flak because the work is unglamorous, but they've semi-succeeded in mapping software engineering to the management science practices that were formulated during the Industrial Revolution...meaning they've turned software development into assembly line work.
This is why good software developers hate working in these places. It would be like a fine watch craftsman going to work on the Timex digital watch assembly line at Foxconn.
The Management Practices that informed OO and drive it are the dominant way to "make stuff on schedule" that humans have managed to come up with. Like it or not, it's why cars get made, t-shirts end up in stores for $10, and your state DMV has an on-line renewal site.
It's interesting to study large software projects that failed as well. SAIC has a couple of well known ones: Virtual Case File [1] and CityTime [2], Healthcare.gov [3] was another good example. It's even more enlightening to compare these failures to big traditional engineering failures: Seattle Tunnel [4], Space Shuttle failures, etc.
One thing to keep in mind. Large software projects are often more complex on raw unit count than even the most complex engineering efforts in history. For example, the Space Shuttle is considered one of the most complex pieces of engineering ever, and it only had 2.5 million parts. [5] OS X 10.4 by comparison has almost 100 million lines of code. [6] It's not a perfect comparison of course, but it does help cement in the magnitude of modern software efforts.
If OO has a failing it's that humans haven't managed to come up with a better way to break complex work down or to define software methodologies that map well to how we know how to manage and coordinate complex tasks. OO works very well when considered in this context, it does more or less what it's supposed to. It's not intended to provide a methodology for elegant or creative software crafted by a single person or small team of experts.
We've learned a lot in the decades since OO has become the dominant force in software organization. And Management Science has improved a couple tick marks as well. The next paradigm shift is likely close. But make no mistake it'll probably be a streamlined OO-like system of trackable, schedulable work components. But it won't happen quickly, literally trillions of dollars in work effort have been put into OO from the software to the management practices.
1 - https://en.wikipedia.org/wiki/Virtual_Case_File
2 - http://washingtontechnology.com/articles/2011/10/25/saic-sac...
3 - http://www.cio.com/article/2380827/developer/6-software-deve...
4 - http://www.popularmechanics.com/technology/engineering/infra...
5 - http://spaceflight.nasa.gov/shuttle/upgrades/upgrades5.html
6 - http://www.informationisbeautiful.net/visualizations/million...