The greybeards from the 70s weren't much smarter than today's programmers. They were the same curious hackers from today, with the advantage of being born in the right place at the right time, when the technology was still developing, so they were forced to build their own tools and operating systems.
> We need more abstractions, and yes, they will run slower and have their issues.
I disagree, and side with Jon Blow on this: abstractions (if done well) create the illusion of simplicity and more often that not hide the apparent complexity of lower levels. Sometimes this complexity is indeed too difficult to work with, but often it's the problem itself that needs to be simplified instead of creating an abstraction layer on top.
I think as an industry we've failed to make meaningful abstractions while educating new programmers on the lower level functionality. A lot of today's programmers learned on Python, PHP, Ruby, JavaScript, etc., which are incredibly complex tools by themselves. And only a minority of those will end up going back and really learning the fundamentals in the same way hackers in the 60s and 70s did.
> IMO the way out is open source. Open platforms, open standards, open policies.
Agreed. But education and simplification are also crucial.
Regarding abstractions, I feel you are talking about leaky abstractions - systems that offer a simplified interface but still manage to burden you with all the implementation details. It's hard to identify such poorly engineered building blocks until it's too late. Thus layers get built on top of them and it becomes too costly to go back and rework the stack. This is a problem with development processes favoring new features over paying the accumulated tech debt.
Still, good building blocks can exist. You can (and often must) have complexity if it's properly isolated and does not leak. I'd say JVM is great example of abstraction that is slower and internally more complex than SW done native, but brings much to the table as a platform to build on. Other examples: BeamVM, ZeroMQ, Lua (leaky, but at least very simple). Browsers unfortunately are too burdened with legacy and security issues.
I feel my formal education has failed to teach me how to design proper interfaces between systems. Instead we are taught pointer arithmetic and mainstream OOP ("cat IS an animal").
I pick on Chrome because the data is available. And because I regretfully have about 8 copies that code on my computer. But I bet we'd see the same curve with lots of modern software. The LOC numbers for microsoft windows have become so large that I can't really comprehend how so many programatic structures can do so little.
I once heard this architecture pattern referred to as a pile of rocks. Piles of rocks are really simple and elegant - you can always add features to your pile of rocks. Just add rocks on the top until its tall enough! Piles of rocks are really easy to debug too. Just shake the pile (unit tests), and when anything collapses, add rocks until the hole is filled in (= patch that specific issue). Then rinse and repeat. You don't need to bother with modelling or proofs or any of that stuff when working on a pile of rocks.
Look at those Haskell programmers over there building aqueducts using archways. Peh. they should get jobs writing real programs.
Ahem.
[1] https://www.openhub.net/p/chrome/analyses/latest/languages_s...
The day after I watched the talk I was compelled to watch the mentioned Apollo 11 documentary. For me this was a great talk, but feel free to disagree.
Was the documentary "When We Left Earth"?
I don't agree on his conclusions. The real source of problem is that now we have maybe x100000 more software than we had in 70s. It's that many more programmers, so not just the 1% smartest greybeards as before. We need more abstractions, and yes, they will run slower and have their issues.
Also, not everybody is sitting at the top of hierarchy of abstractions. Some roll up their sleeves and work on JIT runtimes and breakthrough DB algorithms.
All those blocks of software need to communicate with the platform and between them. IMO the way out is open source. Open platforms, open standards, open policies. Every time I found a good piece of code in company's huge codebase, it was open source library. Every time. You have to open up to external world to produce well engineered piece of software. The lack of financial models for open source is the obstacle. We should work on making simple and robust software profitable.