You don't have to know what would be a mistake. E.g. if the tool is used most of the time to operate on a small set of servers, you have some extra confirmation or command-line option for removing a large set.

That's good UI design in tools with powerful destructive capabilities. You make the UI to do lots of things v.s. the few things you do routinely different enough that there's no mistaking them.

You can also have the program tell the user what's going to happen (if it can be computed beforehand), e.g. "This will affect 138 server(s)."

Yes, but be careful. UIs like that tend to accumulate "--yes" options, because you don't feel like being asked every time for 1 server. Then one day you screw up the wildcard and it's 1000 servers, but you used the --yes template.

Which is why I'm pointing out that to design UIs like these you should fall back on slightly different UIs depending on the severity of the operation.

This is a good pattern to use. The more pre-feedback I get, the less likely I am to make a horrible mistake. However one problem I often see with this pattern is the numbers are not formatted for humans to read. Suppose it prompts:

  "1382345166 agents will be affected. Proceed? (y/n)"

Was that ~100k or ~1M agents? I can't tell unless I count the number of digits, which itself is slow and error-prone. It's worse if I'm in the middle of some high-pressure operation, because this verification detour will break my concentration and maybe I'll forget some important detail.

Now if the number is formatted for a human to consume, I don't have to break flow and am much less likely to make an "order-of-magnitude error":

  "1,382,345,166 (1.4M) agents will be affected. Proceed? (y/n)"

I always attempt to build tooling & automation and use it during a project, rather than running lots of one-off commands. I find this usually saves me & my team a lot of time over the course of a project, and helps reduce the number of magical incantations I need to keep stored in my limited mental rolodex. I seem to have better outcomes than when I build automation as an afterthought.

This doesn't work. Users learn to ignore the message.

I think it depends on the quality of the feedback. Most tooling sucks, so the messages are very literal trace statements peppered through the code. , vs what the user-facing impact will be. When the thing is just spitting raw information at me, I'm probably going to train myself to ignore it. But if it can tell me what is going to happen, in terms that I care about, then I'll pay attention.

Imagine I just entered a command to remove too many servers that will cause an outage:

  "Finished removing servers" 
  (better than no message, I suppose)

vs

  "Finished removing 8 servers"
  (better, it's still too late to prevent my mistake 
    but at least I can figure out the scale of my mistake)

vs

  "8 servers will be removed. Press `y` to continue"
  (better, no indication of impact but if I'm paying
     attention I might catch the mistake)

vs

  "40% capacity (8 servers) will be removed. 
    Load will increase by 66% on the remaining 12 servers. 
    This is above the safety threshold of a 20% increase. 
    You can override by entering `live dangerously`."
  (preemptive safety check--imagine the text is also red so it stands out)

Obviously some UIs make some errors less likely. You don't have the "launch the nukes" button right next to the "make coffee" button, because humans are clumsy and don't pay attention.

Fat-finger implies you made your mistake once. A UI can't stop you from setting out to do the wrong thing, but it can make it astronomically unlikely to do a different action than the one you intended.

Simple example: I have a git hook which complains at me if I push to master. If I decide "screw you, I want to push to master", it can't assess my decision, but it easily fixes "oops, I thought I was on my branch".

A good UI should be able to help, especially in critical situations. I imagine Amazon will consider something like this:

> The dosage you ordered is an order of magnitude greater than the dosage most commonly ordered for this medicine. Continue? y/n

But that still allows you to make a mistake - by pressing y when that's the wrong thing to do.

There's a balance to be struck. I'd say number of hoops you have to jump through to do something should scale with the potential impact of an operation.

That said, the only way to completely prevent mistakes is to make the tool unable to do anything at all.

(Or to encode every possible meaning of the word "mistake" in your software. If you could do that, you would probably get a Nobel prize for it.)

In a program I wrote I make the user manually type "I AGREE" (case-sensitive) in a prompt before continuing, just to avoid situations where people just tap "y" a bunch of times.

Habituation is a powerful thing: a safety-critical program used in the 90s had a similar, hard-coded safety prompt (<10 uppercase ASCII characters). Within a few weeks, all elevated permission users had the combination committed to muscle memory and would bang it out without hesitation, just by reflex: "Warning: please confirm these potentially unsaf-" "IAGREE!"

It's indeed a real problem. Hell, I myself am habituated to logins and passwords for frequently used dialog boxes, and so just two days ago I tried to log in on my work's JIRA account using test credentials for an app we're developing...

For securing very dangerous commands, I'd recommend asking the user to retype a phrase composed of random words, or maybe a random 8-character hexadecimal number - something that's different every time, so can't be memorized.

I think that even if someone can't memorize the exact characters, they'll memorize the task of having to type over the characters. Better would be to never ask for confirmation except in the worst of worst cases.

That's what I meant in my original comment when I wrote that "number of hoops you have to jump through to do something should scale with the potential impact of an operation". Harmless operations - no confirmation. Something that could mess up your work - y-or-n-p confirmation. Something that could fuck up the whole infrastructure - you'd better get ready to retype a mix of "I DO UNDERSTAND WHAT I'M JUST ABOUT TO DO" and some random hashes.

Not sure if even that would work.

I've almost deleted my heroku production server even though you need to type (or copy paste....ahem...) the full server name (e.g. thawing-temple-23345).

I think the reason was that because in my mind I was 100% sure this was the right server, when the confirmation came up I didn't stopped to look if indeed this was the correct one so I mechanically started to type the name of the server and just a second before I clicked ok, I had this genius idea to double check.... Oh boy... My heart dropped to the floor when I realized what was I about to do.

You could say that indeed Heroku's system of avoiding errors worked correctly....

However the confirmation dialog wasn't what made me stop... Instead it was my past-self's experience screaming at me and remembering me that ONE time where I did fucked up a production server years ago (it cost the company a full day of customers' bids... Imagine the shame of calling all the winning bidders and asking them what price did they end up bidding to win....)

My point is, maybe no number of confirmation dialogs however complex they are, will stop mistakes if the operator is fixed on doing X. If you are working in a semi-autopilot mode because you obviously are very smart and careful (ahem..) you will just do whatever the dialog asks you to do without actually thinking what you are doing.

What, then, will make you stop and verify? My only guess is that experience is the only way. I.e. only when you seriously fuck up you learn that no matter how many safety systems or complex confirmation dialogs there are you still need to double and triple check each character you typed, lest you want to go through that bad experience again....

A well-designed confirmation doesn't give you the same prompt for deleting some random test server as it does for deleting a production server. That helps with the "autopilot mode" issue.

I agree that it should help reduce the amount of mistakes.

But I still believe auto-pilot mode is a real thing (and a danger!) .

My point is that I'm not sure if it's even possible to design one that actually cuts errors to 0.

And if that's indeed the case, even if it's close to 0, it's still non-zero, thus at the scale Amazon operates at, it's very probable that it will happen at least one time.

Maybe sometime in the future AI systems will help here?

I totally agree that it's a real issue, a danger, and that it's impossible to cut errors to zero.

I've also built complex systems that have been run in production for years with relatively few typo-related problems. The way I do it is with the design patterns like the one I just mentioned, which is also what TeMPOraL was talking about (and I guess you missed it.)

If you have the same kind of confirmation whenever you delete a thing, whether it's an important thing or not, you're designing a system which encourages bad auto-pilot habits.

You'll also note that Amazon's description of the way that they plan on changing their system is intended to fire extra confirmation only when it looks like the operator is about to make a massive mistake. That follows the design pattern I'm suggesting.

> My point is that I'm not sure if it's even possible to design one that actually cuts errors to 0.

Personally, I don't believe it is without making the tool impotent. But you can try and push down the error probability down to arbitrarily low value.

Still no help against "whoops, took down a different production instance than intended."

We're assuming that the software in question is even aware of the potential impact. It might not have that information.

This prevents fat-finger mistakes.

You could go further and try to prevent cat-on-the-keyboard mistakes, which is maybe what you're describing (solve this math equation to prove you are a human who is sufficiently not inebriated). Or even further and prevent malicious, trench-coat wearing, pointy-nosed trouble-makers.

The point is, yes, it is possible. That's what good design does.

It's not possible to be perfect, but you can certainly do better than taking down S3 because of a single command gone wrong.

One thing I have been doing for my own command line tools is making a preview feature for what a command will do and make the preview state be default. It's simple, but if the S3 engineer first saw a readout of the huge list of servers that were going to be taken offline instead of the small expected list we probably would not be talking about this. There's obviously a ton more you can do here (have the tool throw up "are you sure" messages for unusual inputs, etc).

If the computer knows exactly what actions would be a mistake - how? The difference between correct and incorrect (not to mention legal and illegal) is usually inferred from a much wider context than what is accessible to a script. Mind you, in this specific case, Amazon even implies that such a command could have been correct under other circumstances.

So, this means a) strong superhuman AI (good luck), b) deciding from an ambiguous input to one of possibly mistaken actions (good luck mapping all possible correct states), or c) drool-proof interface ("It looks you're trying to shut down S3, would you like some help with that?").

TL;DR: yes, but it's a cure worse than the disease.

> If the computer knows exactly what actions would be a mistake - how?

I don't know. I was suggesting it wasn't realistic to do that, and therefore it wasn't realistic to implement a UI that prevents you making mistakes.

That's what they claim they will do to ameliorate this. They will build limits into their tools.

Why weren't they there already?

Maybe they were, but they missed this one thing?

Hindsight is 20/20.

Because human error isn't foreseeable? Or a disgruntled employee?

Because there are limits to engineering resources even at Amazon.

It can validate inputs and not let you enter out of range data. You can know that an answer is wrong without knowing what the right answer is.

Possibly the values are all with range. It was just that this operation only worked on elements that were a subset. No amount of validation will catch that error.

You could feedback a clarification, but if that happens too often nobody will double check it after they have seen it over and over.

While you can't prevent user error without preventing user capability, you can (as others have observed) follow some common heuristics to avoid common failure modes.

A confirm step in something as sensitive as this operation is important. It won't stop all user error, but it gives a user about to accidentally turn off the lights on US-EAST-1 an opportunity to realize that's what their command will do.

No. But a good UI can help you see the mistake you're about to make.

> Is it possible to build a UI that will not allow you to make a mistake?

no, because developers still produce code with bugs.