Craigslist and the Quartz app are just a few examples of what I'm talking about. Nobody except maybe some stakeholders ever asked for those things to be more fancy than they are. They are simple and they work. The end.
Unfortunately, we are in an era where everyone wants to be like Google, or at least thinks they need to. I do not believe that the rest of the world would be worse off if they stopped spending time making "production ready" software that visually competes with the Big 5.
Software becomes complex because we allow it to become complex. Just because you can add another feature doesn't mean that you now have more value for a finite amount of pay for X developer time. Companies seem often blind to continual costs for support that go up every time each time the solution is to add a new feature or software package for a small segment of the customer base or the company itself.
Whomever is in charge should acquiesce less often. But everyone wants to be a wizard.
If they don't care about those things, they aren't doing a very good job of voting with their wallets. It's much easier to sell a product that is pretty.
> "elegant" code
That isn't for users' benefit. The people paying to maintain the software do care how much it costs to maintain the software. Elegant code is an attempt to make it cheaper to maintain.
> a ton of niche features
People buying software do care about that, and they are often a distinct group from users.
> mobile apps, your annoying push notifications, your fancy menu bars that you also fix to the top of the screen for whatever reason, your autoplay videos, your little "delightful" asides that interrupt the content
None of those are for users' benefit. Those are for the benefit of customers (advertisers).
So perhaps saying that users expect more isn't telling the whole story. However stakeholders certainly expect more.
I'm tired of repeating this, but here it is:
Voting with your wallet. Is. Bullshit. Consumers don't pick from the space of all possible products, they choose from what's available on the market. Especially in software, it's producers who collectively determine trends. If they all fall into a fad, customers have no way of saying no.
IMO they are exactly like military-industrial complex where increasingly complicated software will need newer and complicated hardware because "customers demanding it".
This is very far from current state of affairs, where for almost every possible kind of software there are a lot of different developers with vastly different products, different designs and ideologies. They may not be as well-known, but they exist in their niche, and you can definitely vote with your wallet for one that looks to your liking.
Extreme example - mobile todo app. I think that one in ten professional developers released one to app store just as part of his portfolio. And users still choose apps with mainstream design, look and feel. If that's not voting with their wallets, what is?
To shorten my last post, it's poor development practices and a lack of foresight from upper management that makes software too complicated. You could make an interface that looks like it's from the mid 2000's(garbage by today's standards) and people would still use it if it works well and is reliable.
Look at HN. Do we need more from it than it already provides? If it was up to the average upper management, the backend would need constant retooling for every feature request from different departments, 12 forms of analytics, newsletter signup forms in between posts, job alerts, a mobile app with a nag message on mobile pages, single sign on from Facebook and Google, share buttons, a waffle iron, etc. None of which the actual users and even the developers ever wanted.
EDIT: Some people might point to Slack as an exception to my argument. Indeed, Slack looked very nice compared to the competition when it first came out. But what made it a killer app was that it took the best of the IRC experience and made it a universal experience between devices and browsers. Plus it can use a company's SSO. When it first came out, it was also not very complicated. I remember when it didn't have reactions or threads, but it was a breath of fresh air after having come from using things like HipChat and Gchat, both of which were pretty dreadful.
I agree with your point about nagging, popups, and analytics, but I don't think the existence of those annoyances are an argument against the OP.
https://news.ycombinator.com/item?id=16261510
"Software is really about codifying process -- pun completely intended. What happens is some slick new software comes along that implements that happy 80% path on the process, and does it quickly and efficiently. Then business complains about the 20% falling into the gap, and starts adding features to reduce that, thereby slowing down the processing. This keeps going, until someone has an idea and implements slick new software that implements that happy 80% path on the process, and does it quickly and efficiently..."
To a limited extent, yes, but if we're being honest here, the main "additional demands" that most users have for most new pieces of software are "simpler" and "prettier", neither of which automatically requires additional complexity under the hood.
It means that the "simpler" software has to be aware of all kinds of edge cases and solve them automagically, because the user expects a larger degree of automation. Software seems "simpler" to the user because it's working on a higher level of abstraction (and levels of abstraction tend to be "leaky") and by reducing the domain of details that the user needs to be aware of. In order to do that properly, the "simpler" software now needs code to take care of all those details as they are effectively removed from the user's awareness, while the earlier software could leave some of those edge cases up to the user.
No, it doesn't, or at least it doesn't if you write the application in a language that is expressive enough to cater for the edge cases without bloating the code base.
Consider a language like Red, which allows you to write GUI applications in a few lines of code. Must be a large runtime behind the scenes enabling this, right? It's clear that this is not the case as the runtime, compiler and libraries fit within 2MB. How is it possible for Red to achieve so much with so little? I'll leave that for you to discover for yourself.
EDIT: As an aside, the Java 1.1 JVM was only about 15MB.
Two reasons:
1. Computing is a relatively young field, and not all of the best practices have made it to the mainstream.
2. The quickest way to develop is to bolt additional code onto the side of a project rather than considering how the overall requirements for the software application may be better served by designing reusable helper functions and refactoring.
Also, I think you've dismissed Red a little too quickly. The GUI DSL is not the only factor that keeps the codebase compact. I'm using the word "compact" rather than "small" as I recognise that Red is still in its infancy, but with the word "compact" I'm trying to emphasise that Red does a lot with not a lot of code.
> "EDIT: As an aside, the Java 1.1 JVM was only about 15MB."
How big was the JCL (Java Class Library) at the same time? Also, 15MB for a VM on its own is not that impressive. Even Oracle have managed to make small (feature complete) VMs, for example:
https://blog.plan99.net/graal-truffle-134d8f28fb69
"The resulting runtimes have peak performance competitive with the best hand-tuned language-specific compilers on the market. For example, the TruffleJS engine which implements JavaScript is competitive with V8 in benchmarks."
"To give a feel for how easy it is to write these engines, TruffleJS is only about 80,000 lines of code compared to about 1.7 million for V8."
http://nirvdrum.com/2017/02/15/truffleruby-on-the-substrate-...
"The Substrate VM addresses those deficiencies by producing a static binary of a Truffle language runtime. It performs an extensive static analysis of the runtime, noting which classes and methods are used and stripping away the ones that are not. The AOT compiler then performs some up-front optimizations such as trivial method inlining and constructs the metadata necessary for Graal to perform its runtime optimizations. The final output is a version of the Truffle language interpreter fully compiled to native machine code; i.e., there is no Java bytecode anywhere. As an added benefit, the binary size is considerably smaller than the JVM’s because all the unused classes and methods are excluded."
The quoted number was for the entire JRE installer:
http://www.oracle.com/technetwork/java/javasebusiness/downlo...
Users also, for the most part, demand more correct. Not always, and not in all applications, but the tolerance for jank is going down over time. And this is a big reason why I use React and Redux: because React maps better to my understanding of how code works (namely, that extracting state to the imperative outer shell of the functional core leads to generally more correct and maintainable code) and Redux maps better to my understanding of how data works (namely, that specific and replayable transforms create manageable state). It's also a reason why I use containers: to turn the runtime environment into an "API" that is evaluated and tested in isolation from the "API consumer" that lives in the container.
We are progressively moving towards software with these demands, too, and tools are evolving to respond to them. The panic around complexity is, IME, hyperbolic; a failure to choose tools with wisdom is endemic, it's not new.
I run Linux as my desktop, and I've been using Linux on a daily basis since the mid 90s. The amount of work that it does for its user now compared to then isn't even comparable.
How fast did this page download? Pretty fast probably. That speed happens, in no small part, because of enormously complex software running many devices between your computer and the source. Yes, hardware has gotten a lot faster, but software has gotten enormously more complex to not only keep up with the hardware, but also add and facilitate huge optimizations on top of that.
I agree that there the amount of 'useless' or low quality software is also staggering. That doesn't really surprise me, it is what it is.
And I also agree that there is a huge amount of synthetic, un-needed and artificial complexity around these days, with more being made all the time.
But I think those categories come pretty naturally, by default, from the total mass of software being created.
I guess there's a good chance that the percentage of 'good' vs. 'bad' software has always been roughly the same.
And here i sit, yelling at the screen any time any of those automagical daemons and signal buses break.
Because they make fixing the problem once and for all a royal pain!
And i moved away from Windows to get away from that crap.
It used to be we had software A read or write a file (optionally one in /dev), and output some results. Maybe those results were piped to another program that drew something within X11.
But now there is daemon A talking to daemon B over signal bus C, that again talk to program D over either bus C or E, or maybe even Z, that then many go on to draw something (likely just an eternally spinning widget because something got lost in the game of telephone).
This is the dilemma of The Linux Desktop (pick Ubuntu, GNOME, systemd, or all of them). Things like internationalization, plug and play, network configuration, are all important to provide. Trying to provide them to the demographic of people that use Microsoft Windows ends up with solutions that resemble those of Microsoft Windows. Meanwhile power users and developers get alienated and end up implementing those things in a far more efficient and efficient to use way. Running Emacs as my X11 window manager gives me far better internationalization capabilities than GNOME. xmodmap is a better way to configure the keyboard than the GUI tools in GNOME and LXDE (no more worries about losing configuration on updates, for one). Linux web browser video playback and font antialiasing has been such a great feature that I now do half of my browsing with w3m (through emacs-w3m) with the X11 misc-fixed fonts. It is faster to connect to wireless networks using OpenBSD's ifconfig than it is to use nm-applet. The list just goes on - who even needs desktop notifications? Do all projects that try to be "user friendly," like GNOME and Ubuntu and systemd, end up being power user unfriendly?
Interesting question. Can you think of any examples of projects that started (roughly speaking) simple, power-user friendly but not end-user friendly that grew to become complex, power AND end-user friendly?
A post ago you were saying users didn't care about mobile apps.
> Plus it can use a company's SSO
So can IRC, so can HipChat, that wasn't the issue.
> When it first came out, it was also not very complicated. I remember when it didn't have reactions or threads, but it was a breath of fresh air after having come from using things like HipChat and Gchat, both of which were pretty dreadful.
They were dreadful in the UX department. Slack and HipChat in particular are indistinguishable at the feature level.
Majority of those are not complex technically. While they require a lot of graphic design time, they are not that difficult to code. Software is not complex because of top bar or material design or autoplay video.
That's why at many companies, whatever tools and languages allow you to get an MVP out the door before a competitor are not merely accepted but are often celebrated and lauded.
When risk really matters, such as the commercial flight industry, the pace moves much slower. It takes comparatively forever for new features to reach the market because the cost of mistakes is so significant (human life). If business behaved the way regulated commercial flight behaved, we wouldn't be having this conversation.
Pointing to open source or expressive languages as a source of complexity is, obviously to me, a red herring. The failure is our lack of rigorous engineering practices - those practices we gave up in the interest of beating our competitors to market.
You don't even have to look very far back in time to recall when software was expected to be correct upon delivery. Nowadays, particularly with online patching, we are totally accustomed to bugs and frequent fixes. Only a decade ago this was unheard of. The cost of reprinting manuals, burning new CDs (or gasp stacks of floppies) and shipping those out to all customers was so prohibitive that it was worth delaying release until all important features were in place and a full series of tests had been performed.
It's our behavior and expectations that got us where we are, not the tools. And probably there's no going back. This is the age of instant, so we'll do our best to adapt. Ironically, the human who used to have to BE the integration between his/her software tools is now the monitor making sure that the automated integrations are managed properly when an unexpected situation arises. And now we've built new layers of logging tools to help us spot these situations. But we're still stuck managing the processes...
The linked article has some points I agree strongly with, but I disagree in its approach to tools. An MVP that's out the door as soon as possible is unlikely to have the sort of incidental complexity tech debt that kills projects. It can't, because the focus was on shipping.
A quick prototype for an MVP is great, get stuff out the door as soon as possible. If your app needs some javascript, write it however lets you get it out the door the quickest - if that means importing someone else's big framework, fine.
I think the secret is more akin to keep _your own code_ as minimal as possible. Your infrastructure and stack should respect the M part of MVP just as much as the product should.
Cause I rarely see true MVPs. I see "ok in order to scale to a million users in the first year, we're going to need x, y, and z on top of...". And then you're not just importing someone else's JS code and writing your business logic on top. Now you're building several systems that have to work together correctly even as they get modified independently in the future, and so in the process you're encoding a bunch of assumptions about your business that the MVP hasn't actually proved out yet! And that's tech debt of the deadly sort.
We're far away from having tools that will prevent a dev from building something overly complicated if they really really want to. Instead, we have to prevent it by always asking the hard questions "do we really need this right now" and not being afraid to look like the fun police.
If the MVP is successful, that success almost guarantees a continuation of the risky practices. If it's not successful, it probably means the jig is up and the code goes into /dev/null.
I don't have a lot of big team experience. But in my limited experience working with other devs, I've seen a mixture of the builders and the doers. But what I've seen in almost every case (except for the too-much-cash-flow .com cases, which are largely a thing of the past... perhaps replaced with VC funded frenzy behavior) is management that doesn't want to spend any more time on a feature than is "absolutely" necessary. And they have comparisons to offer - other companies that did the thing within a certain timeframe (unrealistic or MVPish).
It's true though, the tools we use aren't really project development tools; they're still mostly get-it-done tools. We probably would be well served with commercial (funded, supported) components and frameworks that devs work with. At least some of the obvious mistakes wouldn't be repeated.
But then we return to the realities of the modern business world. Project budgets are compared to competitors, and there's always some competitor that has outsourced a project for 1/5 of what we suggest it should take, or they've bought a COTS kit and "tailored" it for their needs. Basically, the management expectations have become quite unrealistic. Ultimately, it's easier to change jobs than to effect a change of understanding and behavior in a company. In a techie way, it's a race to the bottom.
The alternative is to build a side project that isn't total garbage. If enough of us do that, we reshape the expectations of users.
While I generally agree with you - do remember that Rich talks about "information-driven situated programs" - there are plenty of occasions when you don't do that. Like, when you're building a business. It could be a fatal flaw to think that your early prototype is a "situated program" - it is, in fact, just a quick hack meant to validate a concept. You start by not knowing enough about the market & your customers - so you shouldn't optimize for simplicity, you should optimize for time-to-value. New shiny tools might be good for this purpose (they're often shallow, but also they often deliver shiny&meaningful value fast).
Software is really about codifying process -- pun completely intended. What happens is some slick new software comes along that implements that happy 80% path on the process, and does it quickly and efficiently. Then business complains about the 20% falling into the gap, and starts adding features to reduce that, thereby slowing down the processing. This keeps going, until someone has an idea and implements slick new software that implements that happy 80% path on the process, and does it quickly and efficiently...
You're completely correct. Software needs are easy and hard, and times change.
Eh, maybe I've just turned into an old curmudgeon, but I very rarely fire up a "new" application and think "wow, this is so much better than the old days".
At least in terms of design, I find that most applications held up as some "gold standard" - iOS, Slack, Discord - are incredibly intrusive and obnoxious (not to mention unstable).
Applications used to be a lot simpler and less opinionated. They did one thing, and they did it very well.
I suspect it's a case of too many cooks in the kitchen, too much money and teams that are too large.
Computers entertainment has changed: the internet works (no more unsupported browsers), gaming works (rarely hard system requirements like specific video cards). Business works too (no more emailing spreadsheets, accounting software works, shared calendars)
Applications did one thing, they didn't always do it well.
Things that are now routine for apps that used to be considered unacceptable:
- Don't start if you're offline
- Limited or crap drag/drop.
- No app icon or start menu integration.
- File type associations, forget about it.
- Generic object linking/embedding, gone.
- Scriptability, gone.
A well written Windows 95 app would work offline, integrate with the Explorer, have an icon, might be able to embed itself into Word documents, and probably exported an API via DCOM. Modern apps? Uh, well.... they do similar things but at the same speed at best, quite possibly slower. They routinely barf if they can't reach the internet (e.g. any web app). Every resource dimension is 10x larger. Quite likely, I can't actually save stuff to a file. They crash less, but experience other kinds of flakyness more. And they probably find scrolling through any kind of non-trivial dataset a monumental challenge. Slack in particular is terribly flaky.
Simple example, company I work for sells both physical and digital products. They used to have separate systems to handle each case and everything worked great. We are now building a new system to handle both at the same time. Issue is that due to US tax codes, there is a different tax rate used for a physically shipped product VS a digitally delivered one. The system has to account for this and no one ever considered this in the design or planning because they never had to deal with this concept before because each system just had to deal with one tax rate per order before.
If in the rare case a customer purchases both types of products in a single order, both tax rates have to be accounted for. Simple work around to save a bunch of work is just have the system limited to only digital or physical per order. This is not a optimal solution so complexity increases by a good deal, especially when refunds, reporting etc. is factored in.
This is not necessarily a bad thing but it is the reason we have such large code bases now, because we need to handle these edge cases without manual human intervention.
Our systems have to be both a jack and master of all trades.
If you're using something like Docker or Kubernetes, good justifications are like "Docker replaces a bunch of scripts we used to have to use to deploy our compiled binaries over a fleet of servers" and "Kubernetes really helped us measure and manage the shared hosting environment that runs our lightweight services".
How often do you hear these justifications, or others that refer to actual improvements in the technical processes? Usually it's just "containers are really cool d00d" or "We spend way too much on AWS, and since the concept of not using AWS is unfathomable to us, we are collapsing everything down to 8 instances that run Kubernetes and spending tens of thousands of dollars on the engineering overhaul to support that instead of just provisioning cheaper servers."
The 40-80% try to find some balance between "now" and "the space of potential futures".
Point being, most developers aren't maniacs nor idiots.
Most developers, myself included, are ignorant and do things without quite understanding the reasons for them. Given how much easier it is socially to make the case for "we should follow this best practice" than "we should do the cowboy thing here", overengineering is the natural way of things. All we can do is fight to keep it down to a manageable level.
I.e. overengineering.
It's not that other developers are "maniacs or idiots". The issue is much finer-grained than that. Without strong technical leadership and responsible system architecture, it is easy for any project to go afield, no matter how talented the individual developers are in isolation.
Independently, most professional developers who've had a job for a while and jumped across a few different companies will be able to write acceptably reasonable code for the type of problems they're used to seeing. Even if they're using the cutting-edgiest, most fragile tooling, it will basically be fine within the context of a small demo and/or side project. Like you said, most of them are not maniacs, nor are they idiots, and they can easily manage something that is entirely under their sole control.
Without strong technical authorities and coordination in a team setting, however, you get these independent units trying to plug together, only to find that the couplers are fundamentally incompatible. Lots of individual developers are doing things that are reasonable in isolation and within the constraints of their specific timelines and concerns (some of these concerns are personal but still work-related, e.g., looking smart to peers), but which require huge effort to fix and make cross-compatible.
This is not just a matter of matching an API, though that may be part of it. The processes, incentives, and overall mindsets have to fit together well. Dev X needs to be able to read Dev Y's code from last week and understand what they were trying to do.
It's harder to read code than to write it, since writing is just an attempt to brain dump a structure you already hold, whereas reading is attempting to load a structure of someone else's conception -- a totally foreign construct -- based only on the slow, imperfect dump they were able to get on the day they wrote the code. You'll sometimes hear talk of catharsis in writing code, because it's a pleasure to express this mental model in a convenient reproducible context. You basically never hear the same of reading code.
Now, if you get Dev Y and Dev X on the same wavelength, then this isn't too big of an obstacle, and you have a sane, productive, happy group that is working under strong technical leadership to make cohesive, reasonable choices in the context of the broader team and company needs.
But if you can't synchronize the wavelengths, Dev Y moans about how stupid Dev X is the whole time, how it's better to use Pet Thing Z instead of Pet Thing A, so anything he touches gets its PTA ripped out for a PTZ, and how dev W should be moved over to replace dev X, since they're much more cooperative.
This continues for months and years. Stack enough of this together and it all just starts to blur. By the time you get to the wide scale view of the company's whole ecosystem, you're ready to trade in your keyboard for a mop and become a janitor.
tl;dr "a person is smart; people are dumb, panicky, dangerous animals and you know it".
The world is chock full of developers who want to reinvent the scroll bar, badly, in JavaScript and seriously, genuinely believe this is both a good idea and a wise use of their company's precious development time and money.
It often looks akin to a bunch of civil engineers building a large rail bridge with no up-front design by making short-sighted decisions every two weeks as to which bit to add next, eventually sticking enough random bits of steel on to get a bicycle safely over it, before declaring it "finished".
So we have parts of the system that try to abstract away core business concerns and some big in-house framework that is poorly written and that you can’t put on your resume.
If you have a better answer than single-page application tooling for this that is not "well, print a huge document, people will totally scroll to find things and wait for postbacks instead of expecting a navigable drill-down interface that responds to changes as they are made," I would genuinely be interested in hearing about it.
And that's not even taking into account perceptive effects; if you sit in a UX lab, having a page flash during rendering makes people think it feels slower. (I have done this test--adding a hook to the end of a web request to hide spinners and draw a white "flash" over a SPA--and the subjective "snappiness" score dropped from an average of 8 to an average of 5.) Nor is it taking into account slow connections and degraded functionality during interruptions, but I'll cut you some slack on that one just because so few people pay attention to it in the first place.
I'd love for you to explain more, but I'm gonna register ahead of time that I kind of expect a "doctor, it hurts when I do this" explanation.
If the process of loading a new HTML page takes less than 300 ms or even 500 ms, it's negligible in terms of UX. It's easy to achieve this on a traditional site, especially if you use caching.
I may be cynical about it due to my own experiences, but I do not like SPAs anymore. You pay for a highly contextually specific increase in perceived performance with a lot of problems in unexpected places. Happy to go on and on about it if you like.
And, FWIW, I have been party to metrics and user experience that shows that 200ms of latency can double your bounce rate. That 200ms of latency is not going to be coming from downloading react.min.js.
You may be right in some niches, but overall what you describe does not square for applications I build nor applications that my clients build, and I've been doing this pretty broadly for awhile.
SSR does help though. But then there's this whole rehydration...
> React and Redux are under 100k minified.
And if initial payload is truly that much of a concern, Preact + Redux is under 5KB minified/gzipped.
Concrete example: Photo organizing program
Old style simple, non-complicated: Photos exist as files on a single computer that are manually placed into albums by people, manually shared in curated batches by printing or emailing.
New style complicated, replicates human judgement: Photos exist in the cloud shared across devices. AI software takes date, location, face & object detection to create automated albums, reject the photos that people aren't looking, etc.
Also: quality and dev cost are always factored in and complexity wins. As a developer I always argue for the simple, clean, maintainable, performant solution - but the business need of the customer is such that they’d nearly always rather have the slow, expensive, buggy and bloated solution (that is - given the choice at fixed cost). Even after pressuring customers on this - the answer in my experience is always “yes we accept poor performance, poor reliability so long as the features are there. Make it complex, we’ll pay for it with money and we’ll pay for it with accepting poor quality. Go for 10 features working 80% rather than 3 features working perfectly”.
And the reason they want this? It’s because their existing apps (from the ”good old days”) might be fast and so simple it’s superficially bug free but it just doesn’t solve their problem. They have to do manual work instead and it’s costing them.
So in the end we are making apps with complexity so high that making them properly (I.e with the kind of thought and care that goes into e.g a compiler) would take at least ten times as long. But customers would reject that product either on cost or lack of features or both.
Not only we have old businesses being entirely automated, automation itself is also opening up completely new markets.
Software complexity is exploding probably at the same rate it's absorbing the complexity that used to be scattered around companies in a less formalized, less cost-efficient way, across people and processes, w/ all the challenges that this implies (communication, correctness, monitoring, maintenance, etc).
The reason software is so complex now is that our expectations as users is orders of magnitude higher than it was "in the good old days". I'm old enough to remember the good old days, so I can speak with a little authority here.
The Pareto principle may not be exactly accurate, but it describes many situations quite well. And in software, it fits very well. 80% of the requirements take about 20% of the code (and complexity). It's the edge and special cases that add the bulk of the work.
When a program just did one useful thing, we humans would do our human thing and integrate several different programs with our manual effort. If an edge case appeared, we didn't even identify it as an edge case - we're built for handling edge cases! We just made the minor _human_ judgement and fixed the data and pushed it into the next program.
Software is complex because we're trying to make it replace more human activity. And beyond the core functionality, human activity is all about applying human judgement. If anyone is familiar with people training people on a particular task, particularly the quaint concept of apprenticeship, it involves teaching and showing and mentoring until a student has enough knowledge AND wisdom to get the job done.
So to boil it down, modern software is primarily complex because it attempts to replicate human judgement and wisdom.
That software complexity is not killing us. It may be a pain in the ass, but if anyone can remember what it was like before we had these tools, I would dare to say that it's still a whole lot better than before computers.