More importantly, the absolute number of these inexperienced programmers (and architects, and admins, and ops people, etc...) is growing, so the "total ignorance" is increasing.
I've just gotten involved in a project to "modernise" a "legacy" ASP.NET project. By legacy, they mean ASP.NET 4 running on .NET Framework 4.8 with SQL Server 2017. Not exactly ancient! This apparently is no longer the hot new thing and "has to be" rewritten using cloud microservices or something.
The app actually runs just fine in its current state. But none of the programmers involved know the pre-core version of ASP.NET or anything but Azure. So it's time to revamp it.
I see huge software frameworks sort of like political parties. The community around each of them promotes the benefits of using their framework over someone else’s, and we “vote” by hitching our products to these tools.
If you didn’t choose wisely, or the framework didn’t develop in a way that benefited you or your org, you’re probably keeping an eye out for what’s next.
The other component to all of this is figuring out how to stay relevant in the industry, and I see more and more frameworks making design decisions around this fact.
Take Gatsby, for example. I still can’t figure out why they use graphql to populate templates other than developers wanting graphql experience, ergo let’s use gatsby. The whole point of graphql was to minimize data transfer from server to client, but somebody thought this was somehow more beneficial to generating a static page than a vanilla JS object.
> Every time Slack redesigns its interface, users need to relearn where all the buttons are. Nothing can possibly justify rolling out ten million redesigns;
I think this partly falls on engineering orgs with development teams that need to justify their existence by constantly rolling out these kinds of changes. Because of Slack decides to freeze the UI, how many front end devs are going to need to be reallocated?
This rant was somewhat aimless, but I think ultimately that many of your rightful criticisms are the product of the politics of engineering companies and communities.
I don't doubt that. I would, however, like to see some people in those organizations try to enact changes, particularly those nearer the top. The business benefits of doing less unnecessary work are relatively obvious.
Separately, a lot of the attitudes towards legacy software use ought to change. Most techies regard organizations still running Windows 7 in 2020 with a mix of amusement and scorn. "Windows 7 came out X years ago, why haven't they upgraded by now?"
Well, why haven't they? Maybe because they already paid for the development of Custom Software X, and aren't particularly keen in redoing all of that work just because Microsoft decided to change things. I don't feel great about Microsoft holding security patches hostage just because more Windows 10 customers would improve their brand or some such. I understand needing to charge for said patches past a certain period, but why not continue providing it for a reasonable price, as long as their are paying customers?
It's very young. Thousands of years passed between the earliest mathematics and the first calculus. Software has had, at most, a hundred years.
Things like microservices, SOLID, dependency hell, npm are all examples of the software organism trying to figure out what works in reality and what doesn't. None of the individual cells -- us developers -- can get a big enough view of the whole thing to understand ahead of time exactly what will work and what won't in large scales and over long timelines. We might argue from some ideology or personal experience, but we don't really know.
Software wants to be inexpensive, both in initial development and in ongoing maintenance. But what does inexpensive really mean? Does it mean that it starts with a formal proof that takes weeks to develop and then is cast into code and never modified again, or does it mean a function that's banged out in minutes and then has an entire suite of test cases written against it and is maintained in minutes and hours here and there for years?
The software organism has learned some things it shouldn't do, but that required lots of developers trying it out, and other developers wailing and gnashing their teeth about it and altogether saying, "this is bad and it doesn't work and we must do it another way!"
We aren't going to see software development reach maturity in our lifetimes. I don't think it will happen for another hundred years at least. I think, eventually, software might settle down to a handful of purpose-built languages, and somehow all the problems of code reuse and distribution and documentation will gradually get solved, probably over some massive and absurdly complex network. Developers might become more like immune system cells, wandering around the network and fixing up the little bits here and there that they specialize in. I dunno. In any case, before any fo that happens, we first have to find out why every other approach to software development is worse.
30 years ago, if you worked in software, then you were almost certainly building shrink-wrapped products, that had to "go gold" before being printed onto 1000s of CDs at a factory. How quickly/easily could you change anything? Not for years, typically. Hence, you were very careful to try and build things right the first time, but the ironic thing is, with such long feedback cycles (i.e. you never really got feedback), that all this waterfall-style effort to plan everything and do everything right the first, rarely ever materialized as perfect/seamless UX for the end user anyways. It turns out, you need to iterate to improve.
Nowadays, the pendulum may have swung too far in the opposite direction... it could legitimately be said that, some things are too easy, tinkering with changes is too easy, and thus nobody expects to have to maintain something for a long period of time. If the cost of rewriting everything is low enough, people will be tempted to just do that whenever they change their minds about something, rather than forcing themselves to predict the future upfront and make durable decisions that will survive for 5-10 years out in the wild with no changes.
IMO, there's no single right answer for all scenarios, all products, all forms of engineering, etc. But it's explainable why such trends may emerge over time.
This is certainly the case in modern game development. It's now routine for a game to receive a major patch within 2 weeks of release.
> there's no single right answer for all scenarios, all products, all forms of engineering, etc.
Sure. A search engine for cat photos can go ahead with a 'minimum viable product' release, and improve incrementally. Not so for avionics.
Avionics cannot be developed iteratively, for domain-specific reasons. The costs of re-testing, re-certifying, and deploying, would be crushing. Besides that, how can an aircraft be delivered if its avionics are unfinished? What's the gap between 'viable' and 'finished' here? Games are of course a different domain.
Not all of it is. C is remarkably stable, for instance. C++ is much quicker to add language features, but has a real emphasis on backward-compatibility. I think Java and C# have done a pretty good job there too. Web front-end frameworks have a higher churn rate, yes, but for now, Angular and React are king, and have been for a while. Maybe they'll be toppled by some other near-identical framework in time.
> Is there an end point where we all actually figure out how to develop software properly?
Depends on the domain I suppose. Embedded code, operating systems, and drivers, have been written mostly in C for decades now. Perhaps languages like Rust will make some inroads, but the 'normal way' has been pretty stable. I imagine other non-software fields have seen more radical changes than that over the same interval.
> Every time Slack redesigns its interface, users need to relearn where all the buttons are. Nothing can possibly justify rolling out ten million redesigns
I once read a rather cynical take on this same question: these companies hire a UI team, and that team isn't about to admit that they have no meaningful work to do.
Is it though? Or is it barely trying to catch up with development of hardware and new fields of application?
Or should we just do AI, web and realtime distributed databsesa in Fortran and Cobol?
Maybe people who feel the excessive flux just lack the capacity to notice how fast environment changes? And that's why they feel flux excessive not borderline inadequate?
Software is much easier to change than other things that humans build. It's easier to replace, sometimes with success, than what other disciplines have to work with. A lot of software isn't used in safety critical areas either so failing is acceptable. Lots of companies on the web have lots of downtime and still make money.
Even poorly written software can absolutely print money with huge margins. And old software that's still around and kicking is often the most successful!! A form of survivorship bias.
Sure, _products_ often matter a lot. But within them and as long as they still solve their problems, it doesn't really matter what's inside. If a product I use as an end user has a lot of internal churn, but still solves my problem adequately, I don't really care for the most part.
Sure, if it's medical software, or aviation software, or software to replace an elevator panel, I might care a lot. But your average web app or desktop program?
However, the vast majority of programming is done to build nearly indistinguishable CRUD SaaS apps. If we could just spend half our time doing real, interesting exploration of programming-space to satisfy our curiosity, maybe we could all agree to build apps with a more boring tech stack. However, that would require a large scale change in incentives that is hard to imagine shy of a Space Race style event.
... or Jonathan Blow was right and C was more or less the global max, and everything else was a mistake. I really hope that isn’t the case.
For instance, look at programming languages: to write provably correct code you need dependent types, but none of the dependently typed languages produce optimal code or have a viable ecosystem. This problem has been unsolved for more than 30 years. We also didn't even have a barely decent production language until Rust in 2015.
Or databases: the best database is PostgreSQL and it doesn't even have things as basic as unsigned integers in core (!!!) and doesn't support multi-master in core (and of course, written in an horrible language, which is C). Again, problem unsolved for more than 30 years.
Or operating systems: all OSes are fundamentally insecure, because they are not written in a proper programming language, and at least the desktop ones don't generally use a hierarchical architecture, but rather have a single gigantic kernel-user boundary (plus they have horrible APIs, written in horrible languages, etc.).
Most other programs have no chance of even being provably correct to a given spec since the foundations are missing, and thus much less of provably correctly implementing the provably optimal solution to a problem. They are not even engineered properly with reasonable expectation of correctness, because there is no liability if you release an incorrect program.
Of course it's not clear if this is fixable without developing a general purpose AI to write the software or massive societal changes.
This is true of both language design and software practices. It's painfully obvious. There are no standards for what constitutes "better" or even "good". Is strict typing better? To what degree? What's it worth as a tradeoff? There are almost no quantitative analyses (https://quorumlanguage.com/ - an attempt was made), which is lamented https://vimeo.com/9270320 by some.
Even ROLES are not well defined (https://www.youtube.com/watch?v=GhfVK_ubk8U) because roles are dependent on known concerns, which are also fuzzy in this industry. Saying development is immature is so understated, it's laughable.
> As for "Machiavellian Python maintainers", that's real. There have been explicit attempts to apply pain to Python 2.7 users. This has decreased since von Rossum got stuck maintaining Python 2.7 code at Dropbox as his day job.
(https://news.ycombinator.com/item?id=13020680)
For java 8/9, it was because Brian Goetz still has a boss. We can just compare https://www.youtube.com/watch?v=2y5Pv4yN0b0 to https://www.youtube.com/watch?v=wHoRBvt3U6o. Hopefully some lessons were learnt as well.
Laziness. Rather than work from the lowest level denominator (what can vanilla javascript do for you?), do a quick google, see some random comment somewhere on the internet that uses library X and just add it to the lumbering Frankenstein's monster of dependencies without a care for the future.
React + Redux + Typescript is nice in terms of frontend development, but the shear number of packages required for a small list of direct dependencies is staggering. Really it is absurd. It was created by people who have the above two tendencies I would say.
Stop blaming this all on junior programmers and javascript. That's just saying those damn kids and their lack of respect for their elders is what's causing it. Truth is this is a relatively young industry, that's being used in just about every single aspect of life, where the cost to make a change is minimal, where the ability to learn is one of the most accessible skills, where business demands change on a dime.
Stop your own arrogance and laziness in understanding the landscape and the people in it.
Framework Z feels legitimized, it's a boon to a lot of careers. Lots more people use it. Recruiters start to ask for it as a requirement.
I don't want this to sound cynical, because I think it can sound cynical. I suspect the industry starts from good first principles of wanting to learn new things and try new things and make improvements, and grow themselves. But there is also serious capitalist pressure on the whole system. And a lot of real money on the line, and careers and Twitter followers and conference speaking engagements.
So between the good reasons and the bad reasons, in web development in particular, things move way way way too fast for little gain.
As I see it, truly mature software should introduce breaking changes twice a decade at most, and ideally less often than that. I don't doubt that most software updates provide a net benefit, but what of the inherent cost of change? Changes require every single user to put in extra work, to adapt to the new version's functionality. Why are software projects so cavalier with their users's time?
And this applies to end-user software too, by the way. Every time Slack redesigns its interface, users need to relearn where all the buttons are. Nothing can possibly justify rolling out ten million redesigns; if a facelift is in order, leave it in the oven for long enough to get it right, and then be done.