and yet you don't see very many buildings collapse, bridges fail, and damns flood down valley. The idea is that there should be liability assigned to important systems, for which this liability makes the onus on the creator/owner to build in safe guards, checks or other protections to prevent disasters.
Why this isn't applied to software engineering is a whole nother story, but i think it probably should. Move fast and break things is not something i wanted to hear tbh.
Meanwhile, software's inner workings are encapsulated so that they're not self-evident as to consequences if things go awry. Furthermore, software is inconsistent in how it works, and this trains people to think that software is just naturally glitchy, but the glitches aren't a big deal. See the joke about different types of engineers also: https://www.reddit.com/r/Jokes/comments/pqr8t3/four_engineer...
The mechanical engineer says: “It’s a broken starter”
The electrical engineer says: “Dead battery”
The chemical engineer says: “Impurities in the gasoline”
The IT engineer says: “Hey guys, I have an idea, how about we all get out of the car and get back in”
If that's what the IT guy recommends, how do you get users, never mind corporate managers and executives, to take software system quality seriously? Obviously, this isn't what a proper software guy will think, the proper software guy knows that things are a bit more complex than that. But this is what the software guy communicates to non-technical people. And only a few rare non-technical people will tell the software guy, "OK, look, just tell me what's really going on here, how deep the problem is, and what we need to do to fix it, no matter what it takes." Most people won't have the time for that because fact of the matter is that the system satisfices needs until it doesn't, and then it's too late.
Software can, and should, also be helpfully smarter. Where there's a chance for invalid inputs, they should be validated. That would help with errors like:
* Authentication failed, check your account credentials.
* Could not connect to remote server.
* Invalid data input, failed to parse around byte/octet 1,234,666, near line 56789 (printable filtered) text: 'Invalid String Example Here'
Yes, those are my take on 'software' versions of the starter, battery, and impurity.
Lots of well written software will do this, mostly good compilers for software.
Poorly written software, often hides these errors if it even collects them at all. PHBs and other less professional people seem allergic to thinking, for them it's easier to call in a specialist or just not do it with the broken tool.
I have worked with ECU programming. When test drivers got stuck and called me it was the first thing I told them. "Restart the vehicle". If that didn't work, "disconnect the battery and wait 5 minutes".
They quickly learned.
2. Early implementations of Britain's TPWS on railways are exactly like this. Suppose you're pulling into a terminus station and, for whatever reason, you happen to stop such that your train's sensors are right on top of the TPWS "toast rack" (basically think radio transmitter). When the next driver turns the train on, it can see a TPWS transmission. Now, it knows perfectly well it isn't moving, so we're not in some terrifying near-death scenario, and it could let you, a human train driver, sort this out by, you know, not crashing the train into the station buffers. Nope, in v1.0 the firmware just considers starting in this situation to be a fatal error and won't let you move at all, the only authorised solution is to get rid of any passengers who've boarded your train, switch the train into it's (unsafe for passenger service) maintenance mode, drive it away from the beacons a few yards, stop, turn off the power, and then reboot the computer where it now can't see a troubling TPWS transmitter and now it's a working train again.
"The IT person says". IT is not an engineering discipline (nor is software development in most cases), although they really should be.
And "turn it off and on again" is the canonical "joke".
This sort of thing is inevitable once a company is controlled by people who don't understand what the company does.
Management replaced know-how with software, and it mostly worked. Then they probably fired the few remaining people that had the know-how. Then, when it became painfully clear something was wrong, management chose to blame anyone, including to the point of pursuing prison sentences, rather than admitting they didn't know their business and someone made a mistake. They chose to destroy people's lives rather than admit they couldn't do the ONE job they claim they can.
I think we should examine why this doesn't happen with structural engineering firms.
For the third time on this thread I'm going to say it is because you have to have proper qualifications to do that for a living.
And the enormous Millennium Tower in San Francisco is sinking and tilting.
So it's not unique to software.
At 10pm, a janitor came in, unplugged the computer, and plugged in his floor clearer, cleaned the floor, and then when done, unplugged the floor cleaner, and plugged the computer back in.
The consultant then suggested using the outlet on the other wall that had a free outlet (and got a 'difficult to unplug' cover for the outlet with the computer).
The next night he stayed late again and the janitor used the outlet on the other wall.
The consultant then told management that the problem had been solved and it was a buffer problem.
Apparently there are tons of these stories [2] [3], many of them probably urban legends.
But there was one last year that was definitely real [4], when a cleaner removed power to a freezer holding decades of samples, apparently because they were annoyed by an alarm sound it was making.
[1] https://www.goodreads.com/book/show/3227607-the-devouring-fu...
[2] https://old.reddit.com/r/talesfromtechsupport/comments/5yrs1...
[3] https://www.logikalsolutions.com/wordpress/information-techn...
[4] https://www.theguardian.com/us-news/2023/jun/27/cleaner-coll...
https://archive.org/details/devouringfungust00jenn/page/96/m... (pages 97 to 98) is the proper telling of the story.
There is legal accountability though, and a much better understanding of who can be held to blame for what. Software suppliers are pretty much immune to consequences of carelessness.
However, I think the real problem is that software is far more complex than a bridge or a building. IT system have complex hardware (far more complex than any mechanical device) running even more complex software (counting all the layers from OS up). On top of that people (meaning users/buyers) have no idea how to evaluate safety/reliability/security and mostly seem to regard it as a nice to have to be traded off against other nice to haves, not an essential baseline.
Once its implement everyone assumes the computer must be right, and acts accordingly. The presumption that computers are right is even enshrined in UK law (the intent was to stop people getting out of things like speeding tickets by claiming speed cameras where faulty) but I think everyone has come across situations where the final word on a dispute was "the computer says so".
It does not get done everywhere because it is not a high enough priority.
The English term "Engineer" can be used by anybody though.
You're doing super well if you're successful enough that some of your values need to be re-written.
So the US didn't fall far from the tree. It's just like 'daddy'. <wipes away imperial tear>
They kinda have - check out the cladding scandal [1]. It's pretty major, a lot of properties are essentially unmortgageable until they get inspected to make sure they're safe.
[1] https://en.wikipedia.org/wiki/United_Kingdom_cladding_crisis
There is still hope for Grenfell prosecutions after the inquest, but justice repeatedly delayed for no good reason is justice denied, obviously.
You used to have to get your design cleared by a Local Authority Buildings Inspector. Now you can opt out and get a private company to do it!
Some were deregulation, but some were outright deliberate fraud, like when Kingspan and Celotex scammed the test process and produced misleading safety documents to get their flammable insulation used where it wasn't legal.
And local authorities don't come out of it well either - the building was owned and maintained by a local authority, they hired the architects, the design-and-build contractor, and the guy whose job it was to carry out fire risk assessments. And they chose a bidding process that drove the price so low the bidders who wanted to use non-flammable cladding all dropped out.
Definitely need to stricter regulation though - the idea we could trust the construction industry to use flammable cladding safely has proven false. Flammable cladding should be banned on tall buildings all together.
Again, how is it you are allowed to do your own testing, rather than submit samples regularly to a test house? Or perhaps products on higher risk buildings should be spot checked. A sample from the jobsite itself could be tested.
Self- regulation is a joke.
Just you know, it turns out while the test rigs were being built, the BRE might have looked the other way while someone put fireproof magnesium oxide boards over the temperature sensors.
And when the manufacturer sent 'all the test results' to LABC for the certification, they didn't include the tests that ended in "a raging inferno". You see, that test was terminated early, so it wasn't a completed test. And when the manufacturer got the certificate granted, which included a bunch of caveats about how the material specifically had to be used, they immediately quoted it in their marketing materials without any of those caveats.
Of course all this fraud was done by some 18 year old junior employee, who was told it was 'industry standard behaviour' by a mid-level employee (who has conveniently since died) and the CEO had 'absolutely no idea' this was going on right under his nose.
Cant stop laughing. For some strange reason this sentence reminded me of Yes Minister.
When software breaks you restart and it's back to "working" order. It would be a different scenario if the computer would set itself on fire on failure.
These people have been called 'software developers' or just 'programmers' back in the day. In fact, I'd argue that most commercial software development is more like plumbing than engineering.
Here's a hint: if you're writing control software for airplanes, medical devices or industrial plants, you're an engineer; if you're developing a UI frontend for a website, you're probably not.
I've worked on enough projects with "real engineers" to see that rigor varies significantly with the person. I've seen a dropout with more engineering rigor than a Waterloo grad (granted, this was at a company with a very selective interview process...)
In practice, you do whatever and then pay a licensed PE with an absolutely massive liability insurance policy to stamp your design. If something goes wrong, they take the fall and go sip drinks on a beach somewhere.
As usual, incentives rule everything around us.
Hi! Let me tell you about my wonderful country. Norway! I'm sure you've heard about it. The socialist utopia in Scandinavia! ;-)
The Braskereidfoss dam failed during a flood last fall. Luckily just a 'river dam' and not a reservoir dam. Still rather bad.
Oh, and during the same freak whether event, but different place in the country. The Randklev railway bridge failed over the river Lågen, just by Ringebu.
And during the last 8 years, we've had two road bridges collapse. One by Sjoa in 2016 (Perkolo bridge) and then 16 months ago Tretten bridge. We had the fantastic idea of building wooden bridge sfor road traffic. Here's a nice article on Tretten bridge: https://en.wikipedia.org/wiki/Tretten_Bridge
But yeah, not too many building collapses . Just a bunch of infrastructure collapse.
Computers, at its best existed for only 200 years. That's counting from the difference engine that Babbage never finished building, and if we are talking about the first modern computer that does contribute significantly in a way, that's around the time ENVAC and Z3 appeared, just around 80 years.
Structural Engineering existed for over 2000 years. With that comes with a lot of people died in natural disasters and advancing material science to improve structural integrity. But even the Japanese to this day still can't solve the earthquake and the problems the fallout makes.
> Why this isn't applied to software engineering is a whole nother story
I actually think it is. That's why real-time operating system and mission-critical hardware exists for human to fly in the air and explore the space, not counting a lot of time-sensitive industry robotics software that controls the actuators in real time as well.
That said, the strict requirements of real-time programming requires a lot of expertise such as CPU cycle counting, CPU slack reduction, timing requirements and choice of algorithm (such as EDF scheduling and real time memory allocators like TLSF, and you won't see jemalloc on embedded devices, right?), stack or heap memory allocation which also complicates the programming stuff, because removing malloc might free you from OOM but this means a lot of functions needs to accept extra parameters to state the output location, and that means a lot of normal software can't be used. (You can go for a hybrid approach by using arena allocation, but that still isn't a perfect solution)
As you see, even for soft real time engineering (which I actually mentioned so far, I don't know much about the real hardcore "hard real time engineering" though), the sheer complexity here already, means there's a lot of design decisions which simply just makes people stay away and just go for normal software engineering (but in the end, if everything is predicable its fine).
> Move fast and break things is not something i wanted to hear tbh.
When Zuckerberg said that, he's referring to his startup mindset when Facebook was really just a startup in his day. In a highly competitive market environment, startups has to fight desperately for their own survival, even with lots of fundings and VC rounds.
Startup can move very fast and their agility is the only weapon against the old dogs. Now you've become one of the old dogs, you don't break things.
Show me a similar story about stuff this bad happening at Google and I'll believe that there's something wrong with the field as practiced at its best.
This is why the CEO was rewarded with a CBE and a cushy NHS post in 2019 when the scandal was in full swing. She was acting like a good girl for her political masters in managing the crisis, stringing it out long enough in the hope that everyone would forget
the problem is people asking idiots who cant program even a simple button, to program serious enterprise, government, and military systems.
As time goes on, I feel more and more strongly that we shouldn't compare software engineering with other types of engineering at all. My younger brother works in the structural engineering space and the types of issues he's faced across different roles is very different to the types of issues I face in software.
Building design and construction have to handle the physical world and dealing with the realities of physically building the structure. Once it's built, however, the occupants don't make drastic changes to the building itself. In comparison, something like Facebook or Twitter needed to change drastically just to remain usable as they grew. "Just stop adding users" isn't a sensible solution to the problem.
Just to be clear that I do not excuse the, frankly, shit software design of Horizon and the fucking appalling behaviour of the Post Office throughout this scandal. I do think, however, that comparing software engineering the other types of engineering does a disservice to all and doesn't take us closer to actually improving the craft.
The structural engineers take a piece of steel out into the desert and they dump sand on it until it collapses. Then they divide whatever weight the sand was by 3 and write it down into a book which is then published for all the other structural engineers. This type of steel with this type of configuration with this length can hold up to this much weight.
Meanwhile, in software engineering the limiting factor that prevents collapse is often "these sets of constraints and requirements are too complicated for the team to keep track of in their heads in this messy codebase". Not only do we not have a way to test this and publish the results, it's not even obvious what exactly you're even trying to measure.
[Cyclomatic complexity isn't very well correlated to defects (iirc lines of code is better correlated). Design patterns, best practices, and code smells are all just poetry that we use to story tell our ways out of blame. And Weyuker's 9 Properties are at best a waste of time. There is currently no way to measure bad code that fails under load until it does so in production.]
FWIW, techniques to perform such measures have existed for ~30 years in PL research (some of Rust's type system comes from this kind of research), but I don't know of any industrial code that makes use of it.
So far the only[] thing I've found was Weyuker's 9 Properties from measurement theory, and that did not seem particularly compelling.
Now, I get that linear/affine types and dependent types (and theorem provers, etc) can be used to prove that you're following the spec. But this doesn't prove that the code is easily comprehended by the people interacting with it. Code that is provably correct (even in the event where the spec is actually exactly what you want) can still fail because it's too complicated for anyone to modify it anymore.
For example, the function composition operator type in agda is kind of intimidating:
_∘_ : ∀ {a b c}
{A : Set a} {B : A → Set b} {C : {x : A} → B x → Set c} →
(∀ {x} (y : B x) → C y) → (g : (x : A) → B x) →
((x : A) → C (g x))
Weyuker and cyclomatic complexity seemed to be the only things from academia.
Can you mention the features of these languages that you think are worth looking into?
Granted, this doesn't fulfill all the requirements for being resilient, but if we manage to make it usable (let's say as usable as Rust), it would be a pretty large step forward.
Moving safely means moving slow, and that costs more - it's as simple as that. Plus it would also automatically disqualify like 80% of the workforce, due to the need for a proper formal education and certification, which would literally paralyze the industry.
And the current solution is not that different from what we have e.g. in civil engineering, the building codes for high-rise buildings are way more strict and complex than the regulations for a garden shed - in many places no permit is needed at all for it. And sometimes that means that that shad will fall in a catastrophic way, too.
Would that be so bad, in the grand scheme of things?
But you're right, it generally requires you have a 4-year degree and have worked under a licensed engineer for some number of years.
Them hiring only people with degrees is almost certainly orthogonal to legal requirements in such cases.
Any sort of professional interaction with patients, sure, whole different ballgame.
Components at every stage of the design and manufacturing of a commercial airplane are signed off by licensed engineers who are legally liable for negligence, and yet Boeing could outsource the 737 Max's software to $9/hr programmers in India. Obviously physical engineering failures still happen, but it's ridiculous that there's no liable professional responsible for software that keeps a plane full of hundreds of people from falling out of the sky.
If a licensed accountant was caught lying about hundreds of people stealing from their employer to the point of getting some sent to jail he could easily do jail time for negligence or perjury. Instead some seemingly anonymous dev wrote shoddy software that ruined hundreds of lives and the Post Office gets to just say that the computer did an oopsie.
Because bits don‘t rot and while you need to build a new bridge now and then which will include your learnings from the old one, you never have to replace software because it can be copied ad infinitum.
It feels like there is no ground truth in computing, each layer has to assume the layer below is generating random errors and try to account for that, because humans are not perfect.
The laws of physics stay the same, technology is constantly changing.
Because those engineers are required to have rigorous qualifications. Please see my comment linked below
Once a physical thing is built or being manufactured, the cost of change is exponentially higher than with software.
So software engineers can experiment more frequently and can get away with low quality, as it can always be fixed later.
Of course, this same mentality can carry into industries where the stakes are life and death, which is where accountability is very much lacking.
It’s that way because software is much much much more complex than that.
Building is a good example. One architect after university has enough knowledge and mental capability to design a building that won’t topple: There are discrete number of parameters to work on and fallible elements are identified and understood very well. There’s manageable amount of degrees of freedom that architect is working with. It’s also not very common for physics to get an annual update.
But when you go to funny-cat-pictures.com there are layers upon layers of complexity. There’s TCP, UDP, traffic encryption, indefinite number of hoops in routing, indefinite number of electronic devices with various quality, etc. Request comes happens from millions lines of code operating system and generation happens on a different millions lines of code of operating system with thousands of lines of codes of software code that interacts with databases, proxies, CDNs, load balancers and is templating, serving, translating, transpiling, compiling and serving while defending against adverse attacks, managing the cache, optimizing for network topology and requester rendering software. That changes every single day slightly.
In 99% the worst case is - your picture won’t load. But when you get to the more serious software development (in a way that it’s critical domain, not that “serve cat pictures” isn’t serious job, mind you) all of it is very very visible.
Every element is so unimaginably complex that it has literal tomes of knowledge written and published about it. Describing every element of transaction between end user and cat serving website would take hundreds of pages - and it would be different than other cat website.
Most of the building, bridges etc. was made either by one person or small group of people working on it.
Rarely software is written seldom, and many well known products have tens of thousands of inconceivable smart engineers and we still can make fun of their - not so rare - failures.
So yeah. Systems suck, organizations suck, people suck. But they suck in relatively safe but also complex environment.
For years software engineering was made fun of because it’s not real engineering. But when you look close enough the environment is orders of magnitude more difficult. Civil engineers might be offended but they can’t hold a candle to structures software is keeping straight.
Software is a bridge built on a raft, floating on an ocean, which tries to get shot down by armed pirates and semi-controlled by a cost-saving manager trying to look good in annual progress report.
> A: Because I am very, very smrt and engineering is for babies.
Lol. Sounds like someone was very upset about being called "not a real engineer".
Maybe that's survivor bias? Crappy organizations fail at infrastructure projects before they even get off the ground.
Conversly softer endeavors involving just money, people and services can operate for a long time on finger crossing.
Money, for one thing. Actually building the thing IRL is the expensive bit, but it's effectively free for software. So engineers are incentivised to get things right the first time, and software developers are not. It's much cheaper to skimp on QA, let users find your fuck-ups, and push an update.
This case was brought to public attention and repairs were attempted only because it's huge and involved hundreds or thousands of people.
How many disparate cases there are, where people's lives are destroyed and innocents are rotting in jails, we have to ask?
Judges, salesman, and managers don't understand that.
The skillfull programmer may accept liability when you give him a verification team with a few PhDs, the ability to withhold signoffs, flexible deadlines etc. etc. Few are willing or required to pay for that. So they get a mystery box with a 90% chance of crap.
"We need some software, ok let's get a big reputable company in to do it for us, we shouldn't get bogged down with all those horrible technical details"
They do understand that. They do not care.
I think for the courts the issue is a bit more subtle. The question is, who's job is it to prove that the other person is wrong ("burden of proof")? Should it be the job of the prosecutor to prove that Intel's processor produces the right answer when an ADD instruction is executed? Or should it be the job of the defendant to show that Intel's processor doesn't produce the right answer? What about proving that the compiler produced binaries which faithfully represent the algorithm? What about Excel?
In our normal life, if a computer is doing the wrong thing, we don't start by assuming a broken compiler; we start by assuming that the new, not-well-tested code is probably broken.
It seems that in the UK before the 90's, the burden of proof was always on the prosecutor to prove almost everything about the system, which is kind of ridiculous. So they passed a law trying to fix it, but messed it up the other way, putting the entire burden of proof on the defendant, without giving them any real way to disprove it. (I mean, shouldn't "discovery" at least mean I can inspect the source code?)
A more balanced law would say that widely-used software with extensive test suites can generally be assumed to be working properly; but that custom-purpose software needs at least some level of evidence that it's correct, and that defendants have a right to inspect any software that's used against them in court for defects.
Laypeople generally understand that software may crap the bed in the sense of "the system is down, please wait, then try again". But few people have experienced subtle changes in stored data.
A judge looking into his document cloud may be ready to see a "sorry, not available right now" notice, but doesn't expect that some sinister program is, in the background, silently editing texts of his judgments and pronouncing people guilty when he intended to free them etc.
The problem with the Horizon scandal is in this sinister manipulation of data. It may also have been done by Fujitsu people themselves, in order to cover some tracks and tamper with evidence. This is a very untypical failure mode.
The legal system failed these people horribly. And the people who pursued these cases with no direct evidence whatsoever should suffer jail time.
However, the role of software here must not be minimized. Software makes it easier than ever to diffuse responsibility and create opaque processes that leave the least powerful people at the bottom of the hierarchy holding the bag. By rigidly encoding flawed assumptions and executing them without question, software is the ultimate realizer of our Kafkaesque nightmares.
No argument that the software bears fault, too.
However, when you accuse someone of stealing money, you should have to prove that they stole the money. This isn't some invisible crime. There should need to be evidence that the stolen money went into their account, got spent to buy something, got pulled from the till on camera, got transferred to Bitcoin--something.
The fact that all these people got convicted with no evidence that the money was ever in their possession is a gigantic legal problem.
I disagree, the software glitch was the problem here.
We are supposed to be able to rely on computers to store and add numbers or report a system failure. This accounting software showed in black and white that some funds that the sub-postmasters were responsible for had gone missing.
What else was the legal system supposed to do? The broken software was simulating crime perfectly.
Except it wasn't; the main problem was how the PO was handling it. ICL/Fujitsu were aware of near-identical bugs in an earlier project[1], and PO employees omitted parts of an audit from 2004 that described similar issues as well[2]
It all goes back to ICL/Fujitsu and the PO being aware of the issue and withholding the information from anyone not already "in the know"; lawyers, judges, changing witness statements to hide incriminating evidence, etc.
Because we’re not professionals. We don’t profess anything and do not have standards. There is no regulation for our industry and no IT association that can strike you off from practicing this craft. There is no accountability, and when there is no accountability, people naturally regress to either lazy or exciting behaviours.
In the early 2000s there was a TV ad campaign for “The People’s Post Office” where the sub-postmaster role was played by John Henshaw, a character actor known for playing hard bastards and, in his most recent role on The Cops, an exploitative bent copper from Bradford. A strange but apt piece of casting.
Imagine you are talking everyday to weird, bitter, arrogant, rude customers for years - even with high salary you will be not so positive.
The entire field of cryptography wouldn’t even exist if the boundary of research ended at “good actors”.
I am skeptical that there are software development practices that will allow me to hire a team of feckless incompetents and have them develop quality software. If you know of any I'm interested to hear about them.
But you are entitled to your opinion and that's fine. We can agree to disagree, nothing wrong with that.
That’s the point. Cryptography is significantly better precisely because the research effort has gone into systems that are hard for people to fuck up.
Just look at the fight was to get everyone to agree that the model should be, “everything including the algorithm should be public, except for the key”. That’s a socioeconomic argument.
> Even when the technical ideas are perfect, organizations still need to implement the ideas. That implementation tends to not follow allegedly perfect specifications for many reasons.
And that’s why making safe systems where mistakes are protected against is a critical area of research.
Rust is popular because it protects against whole classes of bugs, despite it being no faster than C/C++.
You can talk about best practices all you want. You're not going to get most organizations to afford or convince the best people capable of following best practices to come work for them. And even if they did, those best people will leave before they can even change the technical culture. No effective person would put up with the insanity that exists in subpar organizations, many of which continue to exist in spite of their incompetence for many other reasons.
You have no concept of working in the real world where people who suck exist. You talk like you've only ever worked with all-star Linus Torvalds types. Of course it's easy to do what you are recommending when you're working on the Linux kernel, for FAANG, startups with competent founders, etc. All those organizations are able to do what you recommend for reasons that many other organizations can't.
You're a Xoogler working with startups. I get it. You're in that world. You have no idea how to fix an organization like the British Postal Office so that they will do IT competently.
They didn’t have to do anything more than sign in to their account and they got better security than the military leadership of WW2.
It’s easy to lose focus and take “dumb users” as an undefeatable entity, but that’s the lazy way out. The most significant advancement of cryptography was asymmetric crypto that explicitly meant a moron leaking the public key didn’t mean shit.
You can't just drop an Instagram-like app into the British Postal Office and then everything's great. Instagram works as a standalone app that doesn't need to comply with any exogenous processes or standards. It can set the standard process for itself, and then all of its users need to adapt to it. There is no way to design an app outside of the British Postal Office that will fulfill their needs and then drop it inside of the British Postal Office and expect to work. Even if you forced the organization to reorganize itself in order to adapt to the app (which happens a lot), the problems will be inevitable.
Even Office 365 or Google Apps, stars in the SaaS space, require internal administration and customization when being used inside organizations and they can be misused. Something as simple as this person should be part of this security group but not part of that security group. Such misconfigurations are inevitable because organizations are messy.
People smarter than you and I have been trying to solve the problem of good IT governance for decades and have so far failed. And the problem has nothing to do with the quality of the software engineers who make the product, nor their technical decisions. It is orthogonal to the real issues. The fact that dumb users can use encryption today without realizing it has zero implications on solving the issues that organizations actually face. It has zero implications on how they use their technology. The only thing it's done is made the technology more trustworthy for transactions of information, but it did nothing to change work habits, decisions, or perceptions about technology. We know because there are studies on how people interact with technology.
These are not technological problems. They are human problems. Things as simple as "I am petty and don't like that employee" or "I'm gonna make sure that my friend gets to have sole responsibility for that app's strategic focus, even though he knows nothing about how to do that department's work, but he's my friend" or "I need this political win and that's more important than hiring the right technology experts or implementing the right feature the right way." They're simple problems to express but intractable to solve. They're intractable because they're emotional and irrational, spawned by people who need therapy. And most often, the people with these problems aren't stupid or dumb. They're actually often smart, which is why they're also often in the position to make the wrong decisions for the wrong reasons. And then it trickles down throughout the organizational culture.
Technology cannot solve this simply because technology can always be discarded or misconfigured, despite the technology's design. Nobody can force an organization to use a technology, especially when it doesn't have the necessary experts to implement it properly. The biggest problem was that nobody at the British Post Office cared for quality control of the system. Bugs continue to be found in cryptography. They're rare, but they are found. Then they are patched. A lot of organizations don't care and then they have security holes simply because nobody cares about patching. Such lack of care extends beyond just cryptography. Automated updated certificates like LetsEncrypt does not solve this problem because the problem runs deeper than keeping a certificate up to date or running automated security patching. Certainly, nobody can force an organization to use LetsEncrypt. Nobody can force an organization to keep the right people in the right security groups. Nobody can force an organization to disable network accounts for employees fired for embezzlement. Nobody can force an organization to care about documenting, reporting, and fixing bugs.
Being right on a technology level has no bearing on whether one can ensure that an organization makes the right decisions overall. Most of the most important decisions aren't even directly related to technology. Even if the easy solution is as simple as use a SaaS that is as simple as Instagram. The dysfunctional ones will say, "Screw that, I want my bonus or I want my job security or whatever, I'll make sure we never use that Instagram-like app, or anything like it." Or worse, they'll try to use it with the best of intentions and then still screw it up massively when they deploy it for employees.
If you can't accept that possibility, we have to agree to disagree.
I do, people are fucking hard to deal with an the entire field of cryptography is a technical solution to that. If you were at the helm it wouldn’t exist.
>There is no way to design an app outside of the British Postal Office that will fulfill their needs and then drop it inside of the British Postal Office and expect to work.
Yet it does every day and you just take it for granted. What voltage electricity do they use? Do they use phones? Do they use email? Do they use TLS? Do they use standard plumbing?
>These are not technological problems. They are human problems.
Human problems are for technology to solve. Keyword exchanges for ensuring Diffie Hellman secrecy are completely to deal with humans and the problem space it’s meant to address.
>Technology cannot solve this simply because technology can always be discarded or misconfigured, despite the technology's design.
Again, this is wrong. If the organization interacts with the public they are forced to interact with interfaces defined by browser manufacturers. It’s getting extremely difficult to run an insecure website that takes passwords without being flagged as such by systems outside of the organization’s control (e.g. chrome).
You can’t just refuse to implement smtp authentication, DKIM, and SPF anymore and expect to be able to email the rest of the world.
Your view is of the world as it existed during the 2000s. A giant buffet of technological choices, some good and some bad. That world is gone.
We now (for better or worse) have minimum bars set by a cabal of large internet companies if you intend to interact with the general public. Safari will not lower the bar for accessing the postal service, no matter how bristled the mustache of the politician who requests it.
The entire field of research you purport to not exist and not matter is what drives this cabal forward. They remove various footguns every day and take the reins out of incompetent operators hands further with every release.
Your argument seems to be “I’ve thought of a case where humans can screw up so therefore the entire field of taking the ability for them to screw up away is invalid”. It’s completely ridiculous and I’m pointing out that it is both active and making major differences.
If you don’t want to participate that’s fine. But telling yourself it’s because it’s an unsolvable problem is just a lie.
None of your solution philosophy would have prevented what the UK Postal Office experienced. Their issues had nothing to do with interacting with the public. Their use case was an internal black box where things went to hell and they refused to see that it went to hell because it looked like it was working.
You have no idea that we're talking about different things. You can just refuse to implement smtp authentication, DKIM, and SPF and not email the rest of the world. It is possible when you don't care about communicating with the rest of the world and you only use it for something it wasn't meant to do. And then weird phenomena emerge from that. Or you are emailing the rest of the world but you have no clue that you're having one-sided conversations because you don't care about replies. There are so many ways your assumptions can fall apart but it doesn't matter because they're not even trying to use the darn thing for e-mail.
You can design perfect technology in your perfect world but you can't force people to use it the way you're planning. In all your examples, you have the assumption that people will use technology for what you're designing it to do. Those assumptions mean nothing when they use your technology for something you didn't expect and it seems to work for them anyway. And there's no counterpart that they actually care about that tells them otherwise. You can try to explain to them that it's not working. You can even show evidence that their outputs aren't matching what they claim they want. But they won't listen because to them, it looks like it is working.
You need to stop bringing up examples and use cases that have nothing to do with the problems that the UK Postal Office was actually experiencing, and that all subpar organizations experience. Their problems are not the problems you are trying to solve in your logic. And even good organizations experience the same problems too, just to a lesser degree.
> The entire field of research you purport to not exist and not matter is what drives this cabal forward. They remove various footguns every day and take the reins out of incompetent operators hands further with every release.
I never said it didn't exist. I am saying it's not related. Again, the entire field of research that you champion solves a problem that is completely different from the problem that the UK Postal Office experienced. If you understand what happened with the UK Postal Office completely and then offer viable explanations of how your ideas would have prevented their problems and achieved their organizational goals, I will say I'm wrong. Hey, I'm not so arrogant to say it's impossible. But I will say that everything you've said so far is so unrelated to what their problems actually were at a root cause level. To say otherwise is a lie.
To be fair, I used to think like you. It was because I believed that we could create technological solutions to human problems that I didn't understand why organizations didn't just do technology properly. After diving into the research literature, I have realized that's naive. I no longer think like you. It is more complicated than what code alone can resolve. If you can't provide a solid analysis of how your ideas would have prevented the UK Post Office's problems, we really need to agree to disagree.
What is "IS" in this context? I did some Operations Research modules at uni and thoroughly enjoyed it, but it had nothing to say about why projects didn't work.
It's enough work to do one's own job well already. The go getters can of course do it as a natural course of action, but they are outliers. There are a limited number of job opportunities that require developing this experience on the job, so there are limited opportunities to become a good boundary spanner in the first place. Furthermore, people aren't naturally interested in multiple disparate subjects. True renaissance folks like Leonardo da Vinci who are interested in becoming experts in both art and engineering are rare. Elon Musk types that will try to dive deep into multiple unrelated areas are rare. All of this adds up to boundary spanners being rare. As such, leaders who can develop cultures that handle multiple areas simultaneously (see the founders of Flexport who understand both tech and shipping logistics) are rare.
In short, expertise is hard to develop, expertise in multiple areas is rare, and coordination between two areas that understand different worlds is difficult without boundary spanners. As a result, you get failures. See any software engineer who creates a startup to try to revolutionize some old-school industry and then fails dramatically because they don't understand the problems that actually need to be solved. The outliers will figure it out, but not everyone can become an outlier due to reasons discussed, among other reasons.
Just a few basic things that wasn't included, no audit/transaction logs, transactions modified by tech support to keep the system running.
Operators couldn't prove they didn't steal funds, and the british law that computers systems are to be trusted as fact, pretty much convicted them all.
There are two problems here. First, the branch manager is responsible for calculated shortfalls, even if the software is broken. Second, there is no way to overturn broken software. Third, the prosecutors are overzealous in trying to shut these people up and convict them straight away.
The software itself was just a convenient medium for abuse of authority.
Our current system of the world quite strongly disincentivises honesty and integrity - rather, being a bombastic charlatan with a flexible relationship with the truth will get you anywhere.
So everything checks out.
Our relatively small skunkworks team developed apps that changed the end-to-end solution delivery processes for major business units, both consumer and business sectors, saved the company 8 digits in opex and capex each year, and won an international award for "Best Support Team" (the Stevies, sort of known as the Oscars of the business world). Our greatest feat that year that enabled us to win the award was keeping the company afloat during a four-month union labour dispute by improvising solutions that automated everything in sight. At the end of the labour dispute, the CEO send a company-wide email about how important we were, awarded us this made-up award "Holding the Fort". When the union came back to work, we trained them how to use the new tools, but we unfortunately were also enablers of heavy downsizing, which I always disliked. Some of these people were hardworking people who did nothing wrong and followed the rules. Many of them were elderly and had little chance to go back to school to get new skills (we're talking 50-year-old clerical workers, etc). It drastically changed how I thought about corporate software work. That being said, we were all young cowboys, and it was possibly the best team I've ever experienced in my life.
I experienced the absolute opposite in many ways when I worked overseas for IBM, managing projects that spanned the Asia Pacific. I was the go-to PM many of their mission-critical infrastructure projects, including helping with datacenter migration from Japan to Australia, necessitated by the 2011 Fukushima earthquake and tsunami. I also experienced a middle ground as a venue technology manager for the Vancouver 2010 Olympics. Lots of pressure and set processes, but a lot of extremely competent people too.
Look, I'm not doubting your experience, but I've had mine too, which shaped my views, just as I'm sure that your experiences have shaped yours. We can agree to disagree, nothing wrong with that.
I can think of two things that I believe would make a difference in any LargeCorp: First, a standarized way to visualise and execute business logic that allows developers and management to reason together. (The no-code movement is on the right track in fostering a common way to interface with code). And second, a responsible editor for each piece of code.
I think a key factor is that software historically hasn't enjoyed industrialisation to the degree of hardware (or construction for that matter). I can buy a standardized CPU of millions of transistors and integrate it into a standardized motherboard with just a snap. We have managed to standardize software up to the OS level, but after that it's up to the developer and her shortcomings.
https://www.codevalley.com/ does some interesting work.
You will find that people just suck no matter what field you work in.
>Of course, I quickly found out that IS research had already figured most of this out, and that perhaps, people were just people and crappy organizations were just crappy organizations, and perhaps that's something that will never change because bell curve distributions exist for almost everything.
Hence why we need to keep things simple. The human part will never change, or at least change at rate that will take many generations to improve if you are an optimist. I actually prefer things to be Hybrid rather than all-in digital.
If all people could be selfless, humble, intelligent, competent at everything they ever try to do, AND be people of integrity, I'm sure many of the world's problems would disappear, in addition to just IT problems. People suck, organizations suck, societies suck, and we all suck in our own way. I think organizational research like I'm trying to do still tries to point us towards a better path nevertheless, but sometimes it's a throw your hands in the air thing. Some people will never care.
Needless to say, I'm no longer focused on researching this topic, as it seems really well-researched already. But it's still interesting to see that this particular example still pops up in a news report now and then. There are still plenty of other big examples that pop up every year, but this one seems to have staying power to stay in the news media.