Unfortunately my stance is that fundamentally things won't change until we get hit with some actual hardware limitations again. Most devs and people in general prefer a semblance of a working solution quickly for short-term gains rather than spending the actual time that's needed to create something of high quality that performs well and will work for the next 30 years. It's quite a sad state of affairs.
With that said I'm generally optimistic. There is a small niche community of people that does actually care about these things. Probably won't take over the world, but the light of wisdom won't be lost!
It's a self-perpetuating issue: people build stuff saying "it won't/can't last 30 years" for various reasons (money, time, skill, resources, expectations, company culture, business landscape etc). So then software doesn't last 30 years for those same various reasons.
The idea that we think systems did used to last longer is probably some survivor bias. However, software that has survived decades was probably created with a completely different set of methodologies, resources and incentives to modern software.
We swap JS frameworks constantly, but when we’ll reach a good paradigm, we’ll stick with it. At one point, React might be the final framework, or rather, one of its descendants.
Develop a good vm that can either be part of the browser or can be easily launched from it and get all of the browser/OS makers on the same page.
We only have what we have because of a lack of real leadership.
When the c++11 abi break happened it was a big pain in the ass, but once MSVC decided in 2015 that they were going to stop breaking ABI I think it was the stability that c++ needed to fossilize…
Spoken as a C++ fan.
Writing bad code to just get past the next sprint or release is madness.
I'm writing software with the assumption that it'll be used for at least 30 years there, with a lot of guard rails and transparency/observability mechanisms, because I know the next person working there will thank me.
I am aware that capitalism essentially dictates externalising cost as much as possible, but with software- much for the same reason capitalism loves it (a copy is cheap and can be sold at full price despite being constructed just once) means that these externalities can scale exponentially.
Teams in particular is an outlier as in most cases it is essentially forced on people.
It doesn't dictate externalising cost as much as possible unless you have a very short-term view.
Short-term view businesses get eaten pretty quickly in a free capitalist system.
People forget that half of capitalism's advantage is the "creative destruction" part - if businesses are allowed to fail, capitalism works well and creates net value.
Premature optimisation is bad, but there’s now so many devs who don’t do _any _ at all. They don’t improve any existing code, they’re not writing software that is amenable to later optimisation, inefficient architectures and unnecessary busywork abounds.
Are we surprised that years of “product first, bug fixes later, performance almost never” has left us with an ecosystem that is a disaster?
The fact that people don't stay long enough in companies or work on a long project themselves to see the fruits of their labour down the line is a point that is discussed in some of the other comments here in this thread. I agree with it as well. In general if you job hop a lot you won't see the after effects of your actions. And the industry is such that if you want to get paid, you need to move. To reiterate - it's a sad state of affairs.
Users have now been taught that $10 is a lot to pay for an app and the result is a lot of buggy, slow software.
Those big software packages are sold to admins anyway.
Realistically only about 5% or so of my former colleagues could take on performance as a priority even if you said to them that they shouldn't do outright wasteful things and just try to minimize slowness instead of optimizing, because their entire careers have been spent optimizing only for programmer satisfaction (and no, this does not intrinsically mean "simplicity", they are orthogonal).
Disclaimer: Take everything below with a grain of salt. I think you're right that if this was an easy road to take, people would already be doing it in droves... But, I also think that most people lack the skill and wisdom to execute the below, which is perhaps a cynical view of things, but it's the one I have nonetheless.
The reason I think most software can be faster, better and cheaper is this:
1. Most software is developed with too many people, this is a massive drag on productivity and costs.
2. Developers are generally overpaid and US developers especially so, this compounds for terrible results with #1. This is particularly bad since most developers are really only gluing libraries together and are unable to write those libraries themselves, because they've never had to actually write their own things.
3. Most software is developed as if dependencies have no cost, when they present some of the highest cost-over-time vectors. Dependencies are technical debt more than anything else; you're borrowing against the future understanding of your system which impacts development speed, maintenance and understanding the characteristics of your final product. Not only that; many dependencies are so cumbersome that the work associated with integrating them even in the beginning is actually more costly than simply making the thing you needed.
4. Most software is built with ideas that are detrimental to understanding, development speed and maintenance: Both OOP and FP are overused and treated as guiding lights in development, which leads to poor results over time. I say this as someone who has worked with "functional" languages and FP as a style for over 10 years. Just about the only useful part of the FP playbook is to consider making functions pure because that's nice. FP as a style is not as bad for understanding as classic OOP is, mind you, but it's generally terrible for performance and even the best of the best environments for it are awful in terms of performance. FP code of the more "extreme" kind (Haskell, my dearest) is also (unfortunately) sometimes very detrimental to understanding.
Outside of really edge case stuff like real time low level systems software optimizing performance is not that hard and I've worked with many engineers over my long career that can do it. They just rarely have the incentive. In a few particular areas where it's critical and users are willing to pay software can still command a premium price. Ableton Live is a good example.
> Outside of really edge case stuff like real time low level systems software optimizing performance is not that hard and I've worked with many engineers over my long career that can do it. They just rarely have the incentive. In a few particular areas where it's critical and users are willing to pay software can still command a premium price. Ableton Live is a good example.
This seems like a generous take to me, but I suppose it's usually better for the soul to assume the best when it comes to people. Companies with explicit performance requirements will of course self-select for people capable of actually considering performance (or die), but I don't take that to mean that the rest of the software development workforce is actually able to, because I've seen so, so many examples of the exact opposite.
Aside, he's treated like a celebrity in the game developer niche and I can't understand why.
Depending on what tech you use it can be easier or harder to do as well. I'm making a game with Love2D now, which has made supporting Mac and Linux rather trivial so far, although I've run into challenges with mobile platforms and the web even though it support them (it does work, but it takes more low-level glue code to support native phone features, and the web doesn't seem to be well maintained and my game is throwing webassembly errors currently when I try to run it).
And my previous game (which is on the backburner for now) was made with Monogame, and while that technically has support for Mac and Linux (as well as mobile), I've had quite a few issues even just getting the Mac version working well, like issues with resolution, rendering 3D properly, getting shaders not to break, etc. And they haven't kept up with the latest Mac updates the past few years and have had to make a special push to try to get it caught back up. I've probably sunk a good 20+ hours trying to get it working well before putting that aside to work on the actual game again and I still might have to rearchitect things a bunch in order to get that working.
Meanwhile Unity would probably be pretty dirt simple to port, for the most part, but it comes with other tradeoffs, like not being open source, and trying to pull a stunt a couple years ago where they pulled the rug out from under developers with changing licensing to something aggressive (that convinced other developers to port their games away from the platform), etc.
And there's Godot, which seems to be getting more support again (which is great, I even considered it or my current game, I just like coding in Love2D a bit better), but if you ever want your game on consoles you have to pay a third party to port your games to consoles for you.
The guy you linked makes their own engine (and to be fair, so does Jonathan Blow, who you're critiquing), which is great, but not everyone wants to get that low level. I would rather spend more time focusing on building the games themselves, which is already hard enough, rather than spending all that time building an engine.
It was for that reason that I spent several years focused on board game design instead (as I can make a working game with drawing on index cards and some colored plastic cubes in less than an hour), although that has its own frustrations with significant hurdles to get your game signed by publishers as an unknown designer (I did get one signed four years ago, and it's still not released yet), and large financial risks being made for manufacturing and distribution.
Edit: Also the person you linked to isn't even sure it was financially worth it to support all of those platforms, they just do it for other reasons:
"Do you make your money back? It’s hard to say definitely which could mean no. Linux and macOS sales are low so the direct rewards are also low. ...For Smith and Winston [their current game] right now I don’t think it’s been worth it financially (but it will probably make it’s money back on Launch)"
Imagine if a company had been able to systemize the whims of a paranoid regime, allowing them to track and spy on their citizens with impunity in secret, and the population became so inured to the idea that it became an accepted facet of their culture.
Or what if a political party dominated a state and systematized a way to identify and detect oppositional thought, stamping it out before a counterculture could ever arise. Maybe that thought was tied to a particular religious, ethnic, and/or cultural group.
What if these companies are here today, selling those products to the highest (nation-state) bidders, and the methods they're employing to keep the conceptual wheels turning at scale rely on filtering that selects for candidates who will gladly jump through abstract hoops set before them, concerned only with the how and never the why of what they're doing.
I haven't watched that talk by Blow yet so maybe he covers my concern.
I think you have to be mindful of incentives structures and constraints. There's a reason the industry went down the path that it did and if you don't address that directly you are doomed to failure. Consumers want more features, the business demands more stuff to increase its customer base, and the software developers are stuck attempting to meet demand.
On one hand you can invent everything yourself and do away with abstractions. Since I'm in the embedded space I know what this looks like. It is very "productive" in the sense that developers are slinging a lot of code. It isn't maintainable though and eventually it becomes a huge problem. First no one has enough knowledge to really implement everything to the point of it being robust and bug free. This goes against specialization. How many mechanical engineers are designing their own custom molding machine in order to make parts? Basically none, they all use mold houses. How many electrical engineers are designing their own custom PCB(A) processing machines or ICs/components? Again, basically none. It is financially impossible. Only in software I regularly see this sentiment. Granted these aren't perfect 1-to-1 analogies but hopefully it gets the idea across. On the other hand you can go down the route of abstractions. This is really what market forces have incentivized. This also has plenty of issues which are being discussed here.
One thought that I've had, admittedly not fully considered, is that perhaps F/OSS is acting negatively on software in general. When it comes to other engineering disciplines there is a cost associated with what they do. You pay someone to make the molds, the parts from the molds, etc... It's also generally quite expensive. With software the upfront cost to adopting yet another open source library is zero to the business. That is there is no effective feedback mechanism of if we adopt X we need to pay $Y. Like I said, I haven't fully thought through this but if the cost of software is artificially low that would seem to indicate the business and by extensions customers don't see the true cost and are themselves incentivized to ask for more at an artificially low price thus leading to issues we are currently seeing. Now don't misread me, I love open source software and have nothing but respect for their developers; I've even committed to my fair share of open source projects. As I've learned more about economics I've been trying to view this through the lens of resource allocation though and it has lead me to this thought.
I no longer think that's true. Instead, I think consumers want reliability, but more features is a way to justify subscription pricing segregation and increases.
I play games with known bugs, and on imperfect hardware, because I unwilling to pay more. Some experiences are rare, so I tolerate some jank because there aren't enough competitors.
I’m not even sure if this is true anymore. We got new features foisted on us most of the time.
In game development, whenever we go with highly abstract middleware, it always ends up limiting us in what we can do, at what level of performance, how much we can steer it towards our hardware targets, and similar. Moreover, when teams become so lean that they can only do high level programming and something breaks close to the metal, I’ve seen even “senior” programmers in the AAA industry flail around with no idea how to debug it, and no skills to understand the low level code.
I’ve seen gameplay programmers who don’t understand RAII and graphics programmers who don’t know how to draw a shape with OpenGL. Those are examples of core discipline knowledge lost in the games industry. Aka what we have now, we might not know anymore how to build from scratch. Or at least most software engineers in the industry wouldn’t. It cannot end well.
Building your own in my exp is a better idea — then you can at least always steer it, improve and evolve it, and fix it. And you don’t accidentally build companies with knowledge a mile wide and an inch deep, which genuinely cannot ship innovative products (technically it is impossible).
mechanical shit was local, electronic shit is global, not comparable at all
The first half of the 20th century excelled in mechanical destruction. Thus far, the electronic age has been much less bloody.
Since they mentioned nukes it seemed like an obvious example where local things can be catastrophic.
The theoretical risk of electronic things malfunctioning in some global way that they mentioned has never resulted in any nuclear weapons being deployed, but we've actually seen the local mechanical approach they disregard be devastating.
You're not looking at the right signs.
In terms of pollution and environmental damage, including with direct human impact such as microplastics, things have been getting worse pretty much exponentially quicker every year _since_ the end of WW2.
We are fixing it up by refactoring - many through adding abstractions.
I’m sure code with bad abstractions can scale poorly, but I’m not clear how code without abstractions can scale at all.
That's quite unrelated to abstractions. It's just poorly written code, for whatever reasons may have led there.
And I’d rather work in a codebase where the owners are constantly doing such refactors than not.
edit: if you feel the need to downvote, feel free to share why you think my question is problematic - I think that "poorly written" to describe excessive code file sizes is such a wooly phrase as to be basically useless in a discussion like this, and when examined more closely usually comes down to "poorly abstracted". But I stand to be corrected.
That’s really the entirety of the issue, right there.
People aren’t just accepting bad software, they’re paying for it, which incentivizes a race to the bottom.
It’s like “The War on Drugs,” that looks only at the suppliers, without ever considering the “pull,” created by the consumers.
As long as there are people willing to pay for crap, there will be companies that will make and sell crap.
Not only that, companies that try to make “not-crap,” will be driven out of business.
That's easy to stop. Disallow warranty disclaimers.
The EU has something like this, the Cyber Resilience Act, and it has an exception for FOSS.
It works out okay to require compliance from paid vendors - if it costs more to comply with security regulations, that will be included in the price. But it's not reasonable to impose requirements on volunteers that way. After the modification, the EU regulation puts the burden on paid vendors who use free software (who can afford to pay for it), not on free software volunteers.
Don't be silly.
Paid software == a requirement to be of merchantable quality: to do what it claims, do it reliably, and keep doing it, so long as clearly stated requirements are met.
If this means all paid apps become tiny simplistic things in weird restrictive type-safe languages like Ada -- win!
Free software == you break it, you get to keep both halves. Our liability is what you paid, nothing more.
Think about what happens when two people video call each other from opposite sides of the world. How many layers of hardware and software abstraction are they passing through, to have a reliable encrypted video conversation on a handheld computer where the majority of the lag is due to the speed of light? How much of that would you like to remove?
I would venture an alternative bogeyman - "move fast and break things" AKA the drive for profits. It's perfectly possible (as illustrated above) to reliably extract great performance from a tower of abstractions, while building those abstractions in a way that empowers developers; what's usually missing is the will to spend time and money doing it.
And then the video player crashes trying to play an ad
Preventing the Collapse of Civilization / Jonathan Blow (Thekla, Inc)
https://www.youtube.com/watch?v=ZSRHeXYDLko
Software technology is in decline despite appearances of progress. While hardware improvements and machine learning create an illusion of advancement, software's fundamental robustness and reliability are deteriorating. Modern software development has become unnecessarily complex, with excessive abstraction layers making simple tasks difficult. This complexity reduces programmer productivity and hinders knowledge transfer between generations. Society has grown to accept buggy, unreliable software as normal. Unless active steps are taken to simplify software systems across all levels, from operating systems to development tools, civilization faces the risk of significant technological regression similar to historical collapses.