https://www.amazon.com/Field-Guide-Understanding-Human-Error...
My view is that expecting humans to stop making mistakes is much less effective than fixing the systems that amplify those mistakes into large, irreversible impacts.
It's easy to be an armchair engineer and say "well, obviously, don't make this mistake in the future".
As a species, we will always take shortcuts. If a mental pathway doesn't need to be exercised to do something, it won't be. If we see the same popup a hundred times, we're going to ignore the contents by the 100th time because we're used to it.
But it shouldn't be possible to make this mistake if this was designed properly and in a way that made it clear that what you're about to do is actually dangerous and not "dangerous" like the other dozen times you've seen the same message.
A common knee-jerk reaction to "users aren't taking our warnings seriously" is making the warning look more scary and involving more (mechanical) steps – such as two confirmations instead of one. Well, that's pattern matchable and subject to desensitization.
Like the article states, it's more efficient to instead show what you're about to remove, i.e. a summary of the content as opposed to an identifier, then you get both novelty and proportional scariness, depending on how "big" it is.
That said, superlatives like "impossible" aren't realistic here. As long as an action is possible, it is also possible to fuck it up. The challenge screen does require typing in the repo name which is a pretty solid attention-getter. It would be nice to list the important side effects in big angry letters but I'll bet the list of side-effects that are important to some people would be pointlessly long. Maybe a sanity check requiring repos with extremes in certain metrics to get a sanity check from support first— but I have had some preeeeettty protracted response times from GitHub support.
There's many good ways to design this that wouldn't result in pointlessly long lists (which is another problem). You don't need to expose everything that will happen when you do an action, just the "most important".
With unlimited budget, you could compare a bunch of metrics to a baseline to figure out what might be important. Have a lot of stars, way more than the average? That would be nice to highlight. Only have a normal number of forks? Probably less important to highlight that. At the end of the day, Github repos aren't extremely unique. Sure there's probably extremely rare edge cases, but for most repos, comparing against a relatively small number of metrics should get enough to paint a picture of what might get broken. Everything else can be hidden behind a "... and X more" if people are curious.
This both highlights the most important things that might happen, and also provide people with an action to do before confirming, hopefully helping to break the autopilot at the same time.
Also, if you must have I_WANT_CANCER type prompts, try to make them involve expressing what it is that the human is agreeing to, because sometimes it's only at that point the human realises their plan was very stupid. GitHub does that here (you have to type the name of the repo, to delete main_project you'll need to type main_project) but many things do not.
This is why I like Git's force-with-lease. When I try to forcibly overwrite the state, because I know Jim's push was wrong and must be undone, force-with-lease makes me express that, "--force-with-lease=jim-daft-change". And when in fact what I'm going to overwrite is Lisa's urgent bug fix, not Jim's erroneous change, the force-with-lease fails and to get it to apply this way I'd need to say "--force-with-lease=lisa-bug-fix" and that's one last opportunity to say "Wait, what? That's not what I want to do".
- (not intrinsic) many interfaces only give you a few seconds to undo your action, which wouldn't help in this case
- (intrinsic) some events can be layered on top of your change, meaning that you have to either block the undo or reverse all the subsequent events too. This wouldn't be the case for this HTTPie incident, but does prevent UNDO functionality from being a silver bullet.
> "most important"
Yes— the problem is determining the most important metrics for the human who's reading the dialog. Sorting the second part is a lot more difficult than the first.
> With unlimited budget
This framing is only useful to determine what behaviors would be great, not feasibility. We seem to both agree on the ideal behavior but not on the feasibility.
> you could compare a bunch of metrics to a baseline to figure out what might be important.
Important to whom?
> Have a lot of stars, way more than the average? That would be nice to highlight. Only have a normal number of forks? Probably less important to highlight that.
Average for what? Just average for everybody? For the deprecated SlackTextViewController which has vastly higher numbers overall than the unmeasurably more important net-snmp? What about for repos that serve as core functionality to frameworks that only care about forks? XML Schemas that people refer to but never interact with? What about for a company that doesn't care about stars because they're incredibly popular outside of that repo but it's critical for them to monitor the small number of people who've forked their repo?
> Sure there's probably extremely rare edge cases, but for most repos, comparing against a relatively small number of metrics should get enough to paint a picture of what might get broken.
The repo about which you said this mistake should not have been possible was an extremely rare edge case. The article went to great lengths to explore just how fantastically different this repo was. And if you're talking about painting pictures you're using the wrong medium. More information can make the message more pointed but only if it's brief and extremely relevant. If it's not, it's going to reduce the focus on the danger message.
> Everything else can be hidden behind a "... and X more" if people are curious.
Aaaand we come full circle. The whole point of this is to put the metrics most important to that user right in front of their face in bold type because that's going to stop autopilot mistakes. If it's hidden, it failed. Beyond that, the order in which you list things connotes importance— closer to the beginning means more important. Being hidden connotes far lesser importance. If you put the wrong things at the top and the right things behind a click then the autopilot user subconsciously assumes they'd care about the hidden things even less. You're increasing the risk of failure rather than decreasing it. If you're going to highlight anything, you better highlight the right thing.
The existing dialog is pretty clear that you should investigate what it does before you actually execute the command without giving people false clues, and that may be the best compromise. You don't convey false impressions but still convey danger.
> At the end of the day, Github repos aren't extremely unique.
From a quantitative perspective they're not incredibly diverse, even if they're probably more diverse than you imagine. What is immeasurably diverse is the roles those software packages serve, the people who interact with them, and what they need to accomplish their goals. Assuming you can boil that down to a heuristic using available data is something that needs to be concretely proven— not assumed.
It is obviously possible to design a much better, contextual warning with (more) relevant details highlighted. How much time is usually set aside for those?
And yet, it's never going to stop mistakes.
And how many other places like these are there in a large service like GitHub?
A much better way to fix this is to expect mistakes and make it possible to revert instead. Soft deletes can make that trivial to implement for the most part too (except in GDPR like cases where you've got to really remove stuff).
This is not quite correct. In requires inputting the repo name. It's attention-getting the first time. But recently with some colleagues as part of a handoff we went through 50+ repos and archived the irrelevant ones. Very quickly the person doing the actual archiving went from typing to just copy-pasting the repo names. For programmers, copy-pasting from one spot to another can become very mechanical.
So I think this UI choice works well for something that is done very occasionally. But it's no better if people are in the habit. Whereas showing novel information about the cost of the choice would be helpful in breaking the routine. Then could then also make it so that they had to type something only when the operation has a high cost, and what they typed could be something not directly copy-pasteable.
If someone deliberately doing the same thing over-and-over can avoid the manual effort, that's fine. Their frustration would likely make them think about it less. You don't, and shouldn't get prompted to do the same thing using the API, either.
Of course that’s a lot harder to do, since it’s a lot more than a UX change. So I’m not really slagging GitHub here. But it’s the right way to attack this problem.
In this case, I don't quite understand why stars and forks need to be deleted when switching visibility, since it seems like the two shouldn't be linked. I can star and fork my private repos just fine, so it seems like it was just easier to clear it out than to deal with RBAC at that level.
Many people won't notice some of their stars having disappeared, but many would be surprised to see their forks deleted without their consent, especially those that are not merely a mirror. People can have diverged forks after adding substantial changes without feeding them back to the original for a variety of reasons.
Pretend to do it, actually do it 24 hours later, and if they "undo" the delete within that time frame, just remove it from the delete queue process.
I still think you're right, but it's not even as simple as UX and small backend changes. Allowing the repo owner to re-play other peoples' actions comes with complicated policy decisions. GitHub could make it so only support can restore those, but now they're in the awkward position of having to say when it's okay to restore and when it isn't. Still, it may be the best compromise.
It's sort of equally easy to say Github should do X or Y to improve the UX in this (rare?) case without actually knowing the effort or opportunity cost.
Like maybe I'd rather Github figure out how to not have outages so frequently and how to not accidentally give people million+ dollar bills and such.
I doubt the engineers working on the stability issues or billing have anything to do with UX and vice versa. It's not like there's a singular focus that everyone at the company must prioritize above all else. Multiple people can work on multiple projects and not impact one another at all.
On the other, you have a few Microsoft hacks who could easily do the right thing on behalf of those millions of users.
Considering this project was out of ordinary in having 54k stars (article mentions being in top 80), they should not be surprised that their case is not top of mind.
Sure, it would be nice to highlight the most destructive of actions, but they already had to type out full project path as confirmation.
I do believe it would be better if Github allowed restoring data in the next few days (soft deletes ftw) once they hit this issue themselves in the past.
And Github support should have recognized that this project is a special, out of ordinary project, and afforded it some engineering time to restore everything.
Lots and lots of websites and companies use the number of stargazers as a strangely important metric for success, so losing them can be really bad.
- click "change visibility" in the "Danger Zone" area of the settings.
- select "make private" with an additional warning shown that you will lose all stars and watchers. Ok, maybe it should mention how many.
- type the name of the repo into a box and then click "I understand, change repo visibility".
And yes, sometimes it is important to expect humans to not make mistakes. For example, at railway crossings. Even if you drove over it 100 times and no train came, the 101th time you may still die if you don't check for a train before crossing.
For somebody going through a railway crossing, yes, they shouldn't allow themselves to be prone any mistakes there. However, for the people involved in the construction of the railway crossing, they should certainly expect everyone to screw up and scrutinize the safety. We don't want to leave anything affecting livelihoods to chance.
"On average, each year around 400 people in the European Union and over 300 in the United States are killed in level crossing accidents."
If no one in the world drank, smoked pot, or did drugs, how much better off would we all be? 100,000 deaths come from alcohol use every year in the United States alone[1]. That doesn't even account for countless cases of abuse, broken families, crime, and other negative effects of alcohol and drugs.
So many people say "oh well it's fine if I do it, I'm responsible" but then at some point someone isn't fine and isn't as responsible as they think they are.
[1] https://www.cdc.gov/alcohol/features/excessive-alcohol-death....
Perhaps that's true for your definition of "better off", which is perfectly fine, but it isn't universal. Even though I don't do drugs, I don't think it should be anyone's business to police what other people do with their minds and their bodies on their own time, as long as they don't pose a threat to anyone else.
One, those substances are fine for a lot of people.
Two, some of those substances are fine for most anybody. I've never even heard of a deadly marijuana overdose, and the evidence shows no increase in mortality for marijuana users.
Three, we already tried alcohol prohibition, and we are currently trying drug prohibition. It does not solve he problem you care about, while creating other large problems.
So really, you sound like somebody who has a personal hobbyhorse and uses pretty much anything (like say, losing stars on GitHub or mentioning a book on airplane safety investigations) to argue for it. And that sort of motivated reasoning around argument for societal change strikes me as way more dangerous than somebody eating a THC edible.
Perhaps if we think more like adults instead of infants we can try to understand complex issues better instead of reinforcing black and white stereotypes of the world or rehashing whatever our favourite source of ignorance tells us.
It may be an even worse drug. People on drugs mostly have some awareness that their ideas aren't very good. But for those drunk on power, the fact that their same great idea has already been tried and led to total disaster is no cause for concern at all. You see, they're obviously smarter and better than the last batch of power-addicts who tried that, so they'll do it right this time. Heaven help us if we ever discovered a chemical intoxicant that was capable of making people that deluded.
Anecdotal: You have to enter the full name for a couple destructive actions on GitHub, and every time I had to do it was so jarring to me that I stopped everything, rechecked what I'm doing twice, started wondering if I might be dreaming, am on drugs or a voice in my head is telling me to do something dumb. Like, yes we should improve nonetheless, and I usually try to put myself into other people's shoes before judging them, but this is just one of the few times where I just can't help but, you know, think it was somewhat dumb to make this mistake with the current system in place. Like, next time we have this stars and watchers warning in place and still someone will manage to proceed on autopilot, what do we do next? Have a siren sound go off in addition? Have the user enable their mic and spell out the repo name? Send a written letter to GitHub? You'll never get the error rate to 0, at which point would you rather accept people making mistakes and calling them out on it than adding more inconvenience on top that just bothers everybody else?
Nothing else on GitHub is like this: orgs and users are treated as the same class of entity pretty much all the time. I could easily see myself making the same mistake on autopilot.
This is applicable to almost any activity: "Sure he was driving drunk, but the car's manufacturer should have prevented that from causing any damage!" I agree that GitHub should improve the design here - privating a '10 star / 1 week old' repo shouldn't be treated the same as privating a '50k star / 10 year old' repo. I don't mean to diminish the fact that GitHub's UI should be improved here.
But the author needs to take some responsibility and realize they were ""driving distracted"", and not act like GitHub is 100% at fault here. Just because GitHub didn't act perfectly doesn't mean the author didn't make any mistakes.
But what you are saying is that marginal effort to prevent mistakes is not worthwhile.
Cars, to take your example, would not be as safe as they are today if they followed the principles you've shown in this post and the grandparent. And yes, even when driving drunk, which is illegal, is safer too - stay in lane, adaptive cruise control, automatic braking, etc. have all incrementally made even extremely ill-advised behaviours safer.
With industrial machines you have to assume people will do the dumbest thing possible. Because someone will find a way to get crushed in a moving part if they can.
You have safety fence after safety fence and regularly test that your lockouts work.
If you don't do all of this and someone dies you can face very harsh legal penalties.
We don't do this with software when the cost of failure is so much lower but we should still understand smart humans will make mistakes.
I mean, it would be disappointing to read it to find it summarizes exactly to what you just said.
At least that is the idea.
The latter needs some kind of consequence in place to offset the benefits from purposefully making said mistakes.
From an individual user's perspective, you still need own your shit and avoid making these mistakes. You can't rely on everything you use having smart safeguards.
There will always be dumb/careless users that a product should consider. There will always be suboptimal UI (we could've just as easily been talking about CLI database tools) that a user should watch out for
If every system we're interacting daily should be fool-proof, it will requires enormous time and effort. Naturally, we invests more time and effort when it's about life and death situation. If we only lose 2 cents by our mistake, we just don't care.
Of course, I'd like to have more fool-proof design in all the destructive interactions from all websites. But, the question is, does it worth it?
https://www.amazon.com/Field-Guide-Understanding-Human-Error...
The wages of sin is death.
I don't know. Github employees themselves have made this mistake as outlined in the post, and they were easily able to recover from it, which probably lowered the priority on changing any UX.
Essentially, it's a complete non-issue depending on whether Github cares about you.
Would you like the police in your area to behave in this manner? I think not.
> Sadly we can't easily do that without triggering lots of issue. We tried with a couple of projects in the past and found it caused a bunch of bad things to happen.
https://twitter.com/martinwoodward/status/149333028949351629...
On the other hand, you should really, really, really hope that GitHub has internal guardrails to prevent their engineers making cowboy edits to the production databases without manager approval.
Obviously it would be nice to see A similar response from GitHub regardless of whether the issue at hand is affecting themselves versus a member of their community. But, as long as humans are making the decisions, that’s probably a pipe dream. And honestly, policing could very well be the standard example of this.
First of all, this is self-evident. But second of all and more importantly: it doesn’t scale.
The reason we have blame-free postmortems is because people will always make mistakes, so we find ways to change process or design to minimize those mistakes.
If you think you’ll never do something on autopilot, you’re fooling yourself.
The author also neglects to mention he had to type the full project name into a dialog box, which would be critical to any real postmortem. If typing the repo name into GitHub didn’t wake him up, why should we believe listing the number of stars would? The author said he thought he was in a different repo, but that makes no sense.
The real lesson, imo, is for people obsessed with stars to set up permissions, use dedicated accounts, and not toy around with settings you know are dangerous before you’ve had your coffee.
That’s a lot more reasonable than GitHub spending time to implement yet another warning box feature that the author has proved they can ignore regardless of how obvious GitHub makes it.
Please, quote from the article where you read this. I don't see it.
In the article I read, the author admits fault:
> I accidentally made the project’s repository private for a moment.
The author goes on to explain how they made the mistake.
> What put me on the wrong path was an otherwise completely unrelated action: I had just done the same (i.e., hidden an empty README) on my personal profile by making jakubroztocil/jakubroztocil private.
> GitHub’s conceptual model treats users and organizations as very similar entities when it comes to profiles and repos. In this context, and since I just wanted to repeat the same benign action on our organization’s profile, my brain switched to auto-pilot mode.
> I didn’t realize at the moment there’s an inconsistency in the naming of this special repo containing profile READMEs and that it differs for users and organizations: name/name vs. name/.github.
> That’s why I proceeded to make httpie/httpie private instead of httpie/.github without realizing my mistake.
The author does not attempt to hide or misrepresent the GitHub UI that requires confirmation. They show it for both "httpie/httpie" and "httpie/.github".
It does not matter that you have to type the repo name, because once you are on auto-pilot, you proceed past this step. In fact, the author doesn't admit it, but if he's like me, I'll bet they copy/pasted the name. Because the name alone is not sufficient.
The author then goes on to suggest what might have kicked him out of auto-pilot. Providing more information about the repo other than just its name. The name alone is NOT sufficient. And they author shows some example UIs from their own software.
Your suggestion: "The real lesson, imo, is for people obsessed with stars to set up permissions, use dedicated accounts, and not toy around with settings you know are dangerous before you’ve had your coffee."
This simply does not scale. It would require everyone to learn the same lesson for themselves.
I've been a sys admin and developer for over 25 years. People make mistakes. They make them all the time. Putting context-free (or minimal context in the case of the GitHub UI) "Are you sure?" dialogs in front of them does nothing to prevent mistakes.
All the author is asking for is a bit more context from the GitHub UI to help the user realize the significance of their action. This would help future GitHub users to avoid the same error.
It's quite amusing to read this so soon after.
To make a better system for every human, you can either try changing all the humans, or you can change the system that is supposed to serve them. One is easier than the other.
I would also say that post does not come across to me as he is only blaming GitHub.
A prompt that pops up to warn you that you're doing something potentially dangerous, when you know that what you're doing is benign, is a prompt that's crying wolf. You'll learn to tune it out.
A well-intentioned PM then thought the fix was to alias rm and add prompts to confirm. They had a whole project to add this alias everywhere.
First it broke existing scripts all over the place. So they fixed them to auto apply the confirmation.
Cut to a few months later and someone rm-rf’d something again, ignoring the prompt (or maybe auto applying it).
So the PM wanted a better message and a second prompt.
People are well intentioned but I’ve seen this path of stupidity that doesn’t solve the problem but is an activity in response to a problem.
When it comes to repetitive or familiar actions, this is physically impossible. If a repeated action is dangerous you cannot use a human interrupting themselves as your only control. It just won't happen.
Overall this is a classic UI design failure. In addition to the "people physically can't take themselves off autopilot" problem, the interface destroys the user's data without providing a way to get it back. It would take relatively little engineering effort to make this reversible.
When I administered many VAX / UNIX machines waaay back, the terminals were all configured to be 'special' for the super user. It would have been HORRIFICALLY BAD if any command were executed unknowingly.
GH does -not- distinguish for project sizes, so there is NO SUPPORT for its users in this way. Every feature is to be considered with the same "shotgun severity".
I would say that makes things even worse. Because it's just training you to rely on -i/I as a sensible default. One day, you'll encounter that one distro that doesn't have that, and it'll be the time where you really wish it were.
An option that shows a y/n action. That's what GitHub does when you're about to make private/delete a repo (even more asking you to type out its entire name) and yet here we're.
Tbh, when I've deleted repo's, I've mostly just copied the repo name and pasted it in the box. Careless of me? Definitely. Can GitHub change that carelessness? Probably not. Will a box that says "Stars: 54k" help? I think it'd help me, at the very least.
[ ] Delete 54,000 stars?
[ ] Delete 8,000 watchers?
[ ] etcThere’s also an issue of taking a repo private, making changes when no one gets alerted, and making it public to make it harder for people to know what’s going on.
They could always notify watchers about the repo being made private.
"Just don't make any mistakes" is the worst stance one can have in this context, it denies reality and only assures you're unprepared when (not if) something important goes wrong.
If they're in autopilot, how are they going to "make sure"?
Everyone has been in this spot several times. Those who didn't are just lucky.
There's got to be external contingencies to protect people when they're in autopilot.
Mistakes happen. It's better to have a really good Undo than a lot of confirmation dialogs.
What’s the star threshold? 1000? 5?
I’d hate to now have to type in the number of watchers and stars as part of the private/delete/move/etc prompt for all the times I know what I’m doing.
It’s a balance between punishing the non-stupid users vs protecting the stupid users. And people switch between these categories. I might be non-stupid today and stupid tomorrow.
That... isn't how "autopilot" (or humans in general) work(s).
I mean this isn't exactly how it works. But it's a bit scary.
Thing is, we type all day, and can type pretty much any string at a decent speed without putting much thought into it.
Any of us must have mistakenly typed a full file path or API URL, just to realize it’s not the right machine or project after having typed it all. It just take 2 or 3s max after all.
I think it becomes an arms race to get attention on important things, we can do better by focusing more on every single destructive operation. We can also avoid having too similar interfaces for completely different contexts.
In this particular case the stars would have been a useful clue, but in the majority of similar cases the wouldn't be any stars. It would be a useless and redundant signal.
Where does it say that they deleted any repo?
Humans will be humans, we can't help it.
Structures or systems that handle large amounts of user interactions should predict such use cases and design accordingly. a.k.a make them "idiot proof".
I copy/paste the repo name every time I delete a repo on GitHub. Yeah, it’s still a minor impediment, but it’s definitely not “TYPE the account and repo name.”
Humans err.
Have the advertisements/sign-up/GDPR pop-ups trained us to 'engage autopilot to dismiss the pop-up' as soon as possible irrespective of its importance?
“I take responsibility, but…” is not “I take responsibility.”
Yes, GitHub probably should show the stars in the warning UI, and hopefully that will prevent some of these mistakes. But GitHub makes it pretty hard to make this mistake already - the author had to _type out_ the name of the repo they wanted deleted into the warning box. At that point, it's hard to believe the author when they claim that this one addition to the warning UI would have definitely stopped them when they weren't paying enough attention to notice they had typed the entirely wrong repo into the confirmation box.