The other issues are easily addressed by stopping to cater to lazy users so much. Do we need yet another bloaty app to do basic things? Look at itunes. Its a monster. I'm not even sure how to get a CSV of contacts off an iphone. In the past this was trivial with any contact manager because the feature was baked in. Now that's a feature for "enterprise admins."
The remaining things on his list are the 'appification of everything.' Everything is now a shitty touch app that's incredibly dumbed down and buggy. Why try hard, especially when you userbase is locked down by 2 year carrier contracts? Or when all their music is trapped in an itunes database that can't be moved anywhere or given to anyone.
Apple won. This is "easy" and "just works." Big buggy apps with dumbed downed features designed to lock user data as deeply as possible. We've entered a new era of vendor lock-in. It the 1980s all over again. We need open protocols to work just enough for web and email. Everything else is up for grabs.
Get used to it, this is what the market has chosen. Meanwhile, I have to somehow train my users on how to use Windows 8's new tablet-based GUI on their standard PCs. This ain't progress. This is fashion.
It's actually gotten so bad now that I've found myself increasingly hacking together my own small apps and scripts to do some simple things that I want because it's often actually easier to build than figuring out how to use some of the software out there. It's actually an even better investment because I know my own stuff will never magically become shittier because of some update that gets pushed out automatically.
Basically it seems to be sour grapes that consumers are finally getting what they want after 20 years of geek rule... It's not quite the 80s again.
Apple uses a lot of sqllite and xml for your user data. I'd be curious as to what part of your data you are unable to get access to?
Perhaps things have changed now, but previously MS had a setup with 2-4 testers per one developer whereby latter would churn out code as fast as he can (of a respective quality) and testers were expected to identify critical bugs so that the dev can plug them, in the same speedy manner as he'd code the original. I had few close friends work for Microsoft in the 90s and the impression I formed after chatting with them was that the MS products routinely were something that was hacked and hammered into a state of not crashing too frequently. So while Microsoft might have a strong focus on testing, it was used to be completely misplaced and misguided.
Reading through Scott's article, he points out that it's not uncommon for an issue to be raised by many users and ignored by the product owner (my personal pet issue at the moment is Google Calendar no longer allowing me to accept invites sent to non-GMail addresses).
It might not be cost-practical to fix issues that affect a small section of your audience, but I'd like to think there's a certain qualitative aspect that comes into play, and if the issue affects 'n' people it should get a banner regardless of whether 'n' is only a minute percentage of your audience.
The bug gets punted because it's on a non-critical path, there's no data lost, it doesn't crash the entire application, and there's a workaround, even though the workaround sucks. Next time around, the bug is weighed against new feature work, and judged based on how it aligns with the product's core pillars. Not important enough this cycle, maybe next cycle.
Next cycle comes around, and the team no longer knows how to fix the bug, because it's no longer fresh in anyone's mind, or the person who wrote it doesn't work there anymore. The broken module is now essentially a black box, and the cost of fixing the bug has increased drastically. The bug gets punted again until the next release, because hey, we've got a workaround.
The project will be cancelled before the bug is fixed.
Not all big companies work this way, but enough of them do, or have subgroups within them that do, that it creates usability problems like the ones in the article. It's not that the problems don't get found by QA, it's just that nobody can be bothered to address them.
it most certainly won't be cancelled, despite multiple farked edge cases, if the critical path is providing business value. the customer increases revenues, the software company turns a profit, the managers and developers get nice bonuses to give their children a nice education, our 401k's get stable returns, the world gets buggy software but not THAT buggy, and everyone is happy except for a few cranky developers who get paid a lot of money to maintain shitty software from 9-5 inbetween disney world trips and cruises.
We have many new shiny FM tools now, though, that are far closer to regular programming languages. In particular, Agda, Coq, Isabelle are fairly easy to pick up for people with an FP background (rather than a formal methods background).
The breakthroughs in SMT solver performance have been astounding to watch, and their use for "brute force" FM is certainly growing.
Of course, verified code is still many times the cost of unverified code, but the gap is coming down.
I studied in Eindhoven under a pupil of Dijkstra and my father studied under Dijkstra; formal methods are kind of a natural thing for me since I was introduced to computers 30 years ago, but no clients are paying for the kind of time needed to write these proofs. Well, rephrasing; no client of us.
I think there needs to be a breakthrough in the way proofs and software are written; now it's not very practical in most cases.
Making complicated stuff work is hard. There is a reason some people design tools to do one thing and do it well - it's easy to be sure it does that one thing correctly all the time and easier to check it keeps doing it.
Maybe we're betting our simplicity on the stack we build upon (I certainly build a lot of simple stuff on top of very tall stacks of mostly simple stuff), I'm not sure there are no surprises hidden in it. However, we should strive for making things (machines, software) that are simple inside. When we do that, it's easier to make the simplicity transpire and become an elegant form-follows-function kind of design. It will not be possible to avoid all bugs, but by making conscious decisions against complicated designs, we may make some lives less miserable.
There are plenty of cases when developers do avoidable things that will almost certainly cause issues. And then those things do cause issues. And then developers change nothing in their behavior, and continue doing those same kinds of things - because they don't care. I see results of this almost every day.
The most important things the shuttle group does -- carefully planning the software in advance, writing no code until the design is complete, making no changes without supporting blueprints, keeping a completely accurate record of the code -- are not expensive. The process isn't even rocket science. Its standard practice in almost every engineering discipline except software engineering.
The Space Shuttle went up the first time with a few K of core memory. The software effort was really pretty trivial compared with systems today.
"Take the upgrade of the software to permit the shuttle to navigate with Global Positioning Satellites, a change that involves just 1.5% of the program, or 6,366 lines of code. The specs for that one change run 2,500 pages, a volume thicker than a phone book. The specs for the current program fill 30 volumes and run 40,000 pages."
-- http://www.popsci.com/node/31716I don't know how applicable the NASA way is to commercial firms. They have different goals. National Mission Success vs. shareholder returns. The commercial firms are still making money and happy shareholders with a lot of bugs. While a single unit conversion bug on the Mars Climate Orbiter resulted in the loss of a multi-billion dollar spacecraft.
In my view, one should preferably have several independent proof verifiers that have been manually inspected for there to be sufficient trust. It is ok if such verifiers are slow, simplistic, and use lots of memory, given that they should preferably be used as a final check that the verification process worked correctly.
I work for a hardware company. Bugs are really, really, bad. If we find a hardware bug in real silicon, at best, we catch it the moment we get the first chip back, and it means that we have a multiple month delay as we fix it and tape out a new chip, not to mention the cost of throwing away all of the partially fabbed chips we've got, plus the multiple million dollar cost of a new mask set. At worst, we have a recall [1]. We take testing very seriously, and we do a lot more formal verification than most software companies.
The only things you can be sure about are things that have been formally verified [2], and the list of things that you can formally verify is tiny. We formally verified our adder. It took months. Then we did the multiplier, which was much harder. It took about the same amount of time because of the experience we gained doing the adder, but it wasn't easy. Division took a lot longer, even with the experience of doing addition and multiplication. To think that we can advance the state of the art of "things that work" from something like a multiplier to a complex piece of software with "care" and our "collective will" seems overly optimistic.
Everything's going to be broken for the foreseeable future. Putting more effort into testing and less into features is a difference in degree, not in kind. It won't even prevent articles like this from being written because, if all you want to do is find ten bugs in all the software you use, that's still going to be trivial. Considering how much progress has been made in formal methods since 1970, I expect that finding 10 annoying bugs in all of the software I use will be trivial for my entire lifetime.
[1] Well, you don't have to do a recall. AMD had a hardware bug that could be fixed by a patch that degraded performance by 10%. Sun famously didn't include ECC in their L2 cache, which resulted in transient failures for a number of customers, and they made customers sign an NDA before replacing their parts. Guess how much people trusted AMD and Sun afterwards?
[2] Even then, you're never really sure. How do you know the formal verification process itself isn't buggy? It's turtles all the way down. I know some folks who were trying to build a formally verified OS, and they stopped using ACL2 after discovering a few bugs in it. After all, how can you trust your proof if the proof system itself has bugs? ACL2 is old and crusty, but that's precisely why it's used for more hardware FV than everything else combined, outside of Intel and IBM (both of whom have their own, excellent, internal tools). It's old enough to have great libraries. There are newer systems that have better architectures, but they don't have anything approaching the same level of library support for hardware. Yet another tradeoff of time to market vs. correctness. It can't be avoided.
Say you're an engineer who's worried that ACL2 is too buggy for your company to use. You tell your manager. She points out that maybe five ACL2 bugs are discovered every year, and they get more minor each year, as the system gets cleaned up. Moreover, none of the bugs discovered in the past three years have affected any of your proofs, and you wouldn't expect them to have an effect on any proof techniques you're going to use. So you stick with ACL2. And, because you do, there's a tiny risk of a bug. What does this example have to do with the original post? Bugs come from making little decisions like this. No single decision is sure to cause a problem, or (in a company that's serious about testing) even likely to cause a problem, but multiply that tiny probability by the number of times you have to make a tradeoff and the number of lines of code, and it's a statistical certainty that you'll have bugs.