> but the item will remain on the checklist in perpetuity,

Why do you say this?

Everytime you see a system flaw because of a missed step, you add to checklist. Everytime you have a process flaw because of a uselss step, you remove from the checklist. There's no problem here, checklist maintenance can be a step in the checklist.

> ("Skins, skis, boots, .

"Spectacles, testicles, wallet, and watch!"

A release checklist (for us) is inherently very risk-averse, as its raison d'être is to prevent a very expensive & dire catastrophe, and every item on the list is (metaphorically) "written in blood" [0]. As a result it's politically intractable to remove anything.

[0]: https://medium.com/war-is-boring/u-s-navy-supercarriers-domi...

I would say that a system too entrenched to consider removing anything, is fundamentally at risk of not being able to correctly analyze the system when adding anything to the checklist, and is itself at an increased risk to fail.

The problem is it is easy to look an item and say "we no longer need this, remove it". It is much harder to be sure you are fixed all possible ways the problem could have happened. Maybe the system auto-signs the release - but does it in all cases? Is there a rare special case where the automatic process doesn't happen? Do you really know what tricks they have to pull in that branch office in India?

That's why the same consideration to adding an item needs to be put into removing an item, need to be put into reviewing if the checklist is relevant still at all.

Broken down a different way, you need two systems: an automatic system so you're not reconsidering decisions that have good outcome (a checklist), and a considerate system that decides what goes into or out of the automatic system.

If the considerate system is broken or just frozen, then you have a problem. How long it will take to manifest is a different question.

I think there's an asymmetry that you're not quite capturing here: when you add an item to the checklist, you have a specific concrete failure at hand (the process just failed, you had everyone on hand to do your root cause analysis, and you know exactly why). Later when you want to remove it, you (a) lose context because time has passed and memories are fallible and/or a different person is dealing with it, and (b) you have to have some reasonable assurance that you've covered other ways the same failure can occur. No amount of process can completely eliminate this asymmetry so at some point you're forced to make a tradeoff based on how risk averse you want to be.

The problem is not lost context. The problem is that adding an item to a checklist protects you from a category of outcomes. If you have a checklist item to make sure a release is signed, you will never push an unsigned release.

Removing an item from a checklist is done in response to a change in inputs. Sure, you may have automated release signing - but unless you are 100% confident that you are aware, and have mitigated 100% of the ways in which this can fail, you cannot, and should not remove the 'check that the release is signed' step.

Lost context has nothing to do with this. Unless you are an omniscient god, you probably cannot reason, with 100% certainty, that you have mitigated every possible input that produced a bad output.

So, check your outputs.

So it could be an automated test to check if the release is signed as well instead of a checklist item.

Having long checklists for comparatively simple tasks really hurts productivity, plus they're often used as an excuse to put automation in place, because 'the process is already defined' and 'people are already used to it'.

When designing a process, it is of utmost importance to keep checklist lengths minimal.

I can definitely concede there can be some asymmetry, but I think it's system dependent.

As the Boeing 737 Max 8 shows, adding new items (in this case a new design element) is also fraught with risks. You have to get the root analysis correct when adding or removing elements. Adding has risks of unknown (not accounted for) like removing has risks of unknown (accounting forgotten).

In the end, I guess I still believe the best strength is in good analysis at the consideration stage.

Adding a design element is completely different from adding an item to a checklist.

The asymmetry with checklists is that it's completely risk-free to add a new check, but it's risky to remove one. For example, someone might say "we're not totally sure that our system won't fail when we do X, so let's check that in QA, or at runtime, or at takeoff, or whatever." Now that the check is there, it protects you from failures when you do X. And now you're in a situation where you can't safely remove that from the checklist unless you can prove that your system won't fail when you do X. Adding requires only a suspicion, removing requires rigorous proof.

The case you describe with the 737 Max isn't the same at all. There's an actual risk when adding a new component to the system, but no risk when adding additional verification. That's not to say that there aren't other costs, but it can't directly make your system less reliable.

Just one example: If I add a step of, If patient enters with heart attack symptom X, was patient injected with 100cc of <drug>?

It's not a risk free check at all. It will likely increase the rate at which the drug is administered, with all types of plusses and minuses associated with adding a component to a system.

I think the case of 737 max is they game the checklist approach - use the guy who should be checked to be the guy to check it. Hence the argument to remove an item is so dangerous. (But then it is also may be bad blood. Life is more complex than a checklist. )

Chesterton's Fence: http://www.chesterton.org/taking-a-fence-down/

But the point of Chesterton's fence is not that you should never knock down a fence :P

But what it does do is tell us that the difficulty of removing a fence is irrelevant to its utility. :)

I feel like it is the reason why disruption works so well: After an organisation exists for a while, 100s of fences are clogging up the streets. Some of them useful, some of them used to be useful, some of them never thought out well to begin with.

So disruption is taking a bulldozer, and driving right over them. The clearly usefull fences get erected quickly. The forgotten usefull ones are rediscovered after a while, maybe causing some minor damage in the process. But the grand majority turned out to be useless, and is now gone for good.

Of course, if there is a pack of nuclear civilization killing wolves out there, they better stay fenced off or else. The trick is identifying them. There might be a role for a regulator in there.

So the issue isn’t with checklists, it’s with policy.

policy is just a way to record a decision-making process. Yet he issue is that getting the right information to be reasonably sure it’s safe to remove a checklist item is hard and made harder the longer ago the item was added

Agreed. But items can move to other checklists. In this case the list item should become a unit test for the signing code since a unit test is essentially a check list for automated systems.

Where do you work, so that I know to avoid it? Documentation needs updating as things change.

Long time ago when I was living with my parents, our family checklist was "Gas, light, water, cellphone, cat".

Check that the gas stove is off (important). Lights off. No running water (after a few, um, incidents). You didn't leave your phone, did you, and make sure the cat didn't run out (he'd always try).

I always wondered why "lights" were on the list, since nobody gave a damn about electricity bills.

Guess for the same reason that "testicles" are in the parent comment's list :)

Except "spectacles, testicles, wallet, and watch" is the Catholic sign of the cross (face, lower torso, left chest, right chest). Nothing to do with checklists.

I meant that "light" was on the list because it made the phrase roll off the tongue easier.

I've found cellphones don't need to be on the list but the charging cords do!

Leaving the lights on with the curtains drawn - and possibly the radio playing - is a hack to discourage break-ins. Someone seems to be home, but burglars can't actually look inside to check.

Too many people use that trick now. Even the 'put a light in an automatic timer' trick is common.

Instead just put a sign up that says "Any unauthorized testicles entering the building will be donated to a good cause".

Electric company propaganda!

I think a more modern version of that last checklist would include keys and phone.

> I think they make sense for infrequently exercised routines of moderate complexity that are 100% execution

Yes. There have been several formal studies of this effect, e.g. [0]. Checklists offset skill fade, which is in effect the 'forgetting curve'. Some skills (the classic riding a bike) last for ages, others (a good example being field fault diagnosis) are lost very quickly even with good initial training.

Incidentally, I once made a good camping packing checklist by making an inventory of the contents of my car as I unpacked it at the end of a holiday and the things I'd forgotten were still fresh in my mind.

[0] https://www.gmc-uk.org/-/media/about/skills-fade-literature-...

Sounds like you need to add "Periodically review & maintain checklist" to your checklist.

And don't forget to make a Checklist review checklist so you don't mess up your checklist review.

But add a counter to detect recursion and break..

> I suppose I checklist myself when I rappel or go skiing ("Skins, skis, boots, ... working up the body")

This is a perfect example. I have a snowboarding checklist and it's reduced the cognitive load of packing tremendously.

Things to do the night before, stuff to wear for the drive, stuff to pack, what to include if I'm expecting extra cold weather, and a final checklist for things to do in the morning (e.g. get lunch out of the fridge).

For travel I have a checklist of things that I always need to take (toiletries, spare phone battery, Nalgene, pajamas) and then the stuff that varies is all I have to think about. Almost every time I travel there's something on my list of "stuff that always gets packed" which I would have forgotten.

I even take it a step further and I have a dedicated storage bin for all of my ski gear. When I get home, I religiously wash everything, then put it all 100% back in the bin. Next time I go skiing, I just grab everything out of the bin. Rinse and repeat.

Is religious washing like baptising?

Hell, I do the same thing when traveling. I 'get dressed' in my head to check that I'm entirely clothed. If I weren't as good at visualizing things I'd probably just write it down.

I never forget socks but I've had to buy a few toothbrushes and deodorant...

I always have to remind my wife that there are stores along the way - she is the pack the kitchen sink type.

Another situation in which it helps I think : cooking. At least for me a checklist (written or mental) helps keeping the kitchen clean a makes better meal. For a few years now I've started to plan my cooking like I plan more abstract algorithms.

Recipes are checklists too.

Also: Mise en place

I trust that someone in the near future will mess with the release build process and the signage will be dropped. Or there might be cases when the signature is suddenly invalid for reasons you cannot forsee yet. Keep the item on the list, especially if it checks the outcome of a longer process.

Maybe an item on the checklist to refactor the checklist (as needed)?

Or maybe a checklist for refactoring all the checklists?

/this could get out of hand...

You don't need that in the checklist. But what you do need is a periodic review of checklists and other procedure and process documents.

Suppose you have training material for new hires ("How we use git 101"). Do you hand them the same document from 10 years ago? Does it have accurate setup instructions (now you use Bitbucket and not Gitlab or vice versa, and it presents internally on a different domain).

All of these kinds of documents should be reviewed with some frequency (and ideally you should have a way to submit changes outside of the explicit review times, the explicit reviews are to catch things that weren't found earlier; wikis can help here). If you don't, then you do end up with the problem being discussed. You'll have a lot of checklists or procedures being done "because it says so" and not because they're needed.

At some point you have to run through the process with enough disengaged brain cells that you can think about what you're doing while doing it. With processes that only get used in emergencies, this doesn't happen often enough.

I suspect that's part of the secret behind Netflix Chaos Monkey. Every single process gets run multiple times a month, and it's normalized enough that it's not a constant panic mode thing.

I would argue that you want an automated check for "verify release has been signed" then if it fails you are alerted. This is better than manually checking it every time or assuming it will always be signed. A trust but verify approach. Then one day you may find that a trusted cert has expired and signing is broken or a key is not available b/c it got accidentally deleted etc... Lots of things can go and will go wrong.

I confess to not usually using a checklist when I travel because most times are a bit different. What I do though--I travel a lot--is I have a couple containers of "travel stuff" down in my kitchen including some pre-made kits that I just throw in my bag. I pick out what I need, add clothes as needed, and that works pretty well.

> I confess to not usually using a checklist when I travel because most times are a bit different.

Don't you find that most trips have a ton of stuff in common though? Pajamas, contact case/solution, loofah, spare phone battery, ear plugs, phone charger & battery charger, reusable water bottle, etc. always go with me on every trip.

IMO the stuff that always goes is the stuff that I tend to forget, exactly because there's nothing unique about this trip that would make me remember it. If I'm going to the beach I know I need my swimsuit and beach shirts because I can walk through my activities and think about what I need for each one. But I'm likely to forget pajamas because I'm not thinking about "go to bed every night" as an activity to plan for.

A lot of the things you describe (or my versions thereof) are in one of the kits I always bring and the others are elsewhere in the container I rifle through when packing. Sure, I could have a checklist (and probably should) but forgetting the basic stuff isn't something I have a regular problem with. I do travel every few weeks so I have the routine pretty well down.

Apart from the pajamas, you could easily just buy extras of those things and keep them together for travelling. It's so much easier just to grab a set and check afterwards what needs to be replaced or refilled.

This is why I use OmniFocus for my travel checklist: nested lists. If I’m not going on a ‘work trip’ I can just tick that whole sub-list off with one tap.

Checklists FTW. Nobody above me has mentioned Atul Gawande’s book ‘The Checklist Manifesto’ so, just on the off-chance you’re not aware of it, there you go. I loved it. (Gawande is the doctor who implemented the checklist in this article.)

I've been using packing crates, which helps.

I have a packing crate for socks, one for underwear, one for causalwear and pants.

Then I just need to think how many nights I'll be gone to pick how much goes into each.

(Also reduces space in my carry on so I have room for conference schwag on the return leg)

The bottom line, if you travel a lot, is to essentially leave your bags packed for a typical trip. I might need to top some medications off, pack the right amount of socks and underwear, decide on the clothes for a particular trip given weather/activities, electronics, etc. It sounds like a lot of stuff but I can basically grab what I need for a typical 3-7 day trip in less than 30 minutes.

The other thing is just bring less stuff. A lot of people are amazed by how little I carry and I don't consider myself extreme. Consider what you really need as opposed to want.

I started using a hanging basket in my bathroom I can just throw in, but it's not feasible to leave my bags backed if for no other reason than the clothes would wrinkle.

(Then again maybe I'm not travelling enough for your advice to apply...)

I agree on the bringing less stuff front. I've slowly pared down a lot of what I bring, and just stop in a drugstore to pick up if I forgot some minor thing.

A good reusable checklist system can help with this. I've been using an app called SplashShopper since the Palm Pilot days that lets me choose what I need for any particular trip from a list of common items. This lets the list adapt to the particular needs of each trip. It's the only app I've ever found that hits this sweet spot of reusability. Unfortunately the company that makes it does a poor job of updating it for iOS, so I keep an ancient iPad around just so I can keep using it.

> Good checklist design is not trivial, however.

I understand you're saying, but this statement exemplifies so well something I think I have gotten worse at since becoming a software engineer.

In my previous life, if I tried to apply checklists to my life, I would have taken a naive approach, applied my best effort, and that would be the end of it.

Now I would be much more likely to consider edge cases and counter-examples, compare my checklists to the "optimal checklist", and as a result I might fall into a checklist-design rabbit hole, or else constantly question the efficacy of my checklists in knowledge of the true complexity of the problem.

I don't think it's always beneficial to think in those terms.

I feel like some people have vastly different sensitivities to doing pointless things. There are some people who would see items like that on the checklist, and go, okay, it's a minute to check that, no big deal, I'll do it and get on with my life. There are people who look at it and are almost offended that their time is being wasted by process. I don't really know what to DO about this, but I do find it's one thing that makes collaborating difficult when a group has members with vastly different tendencies in this regard.

>"Verify release has been signed" is added to the checklist. However a bright soul also integrates signing into the release build.

If that bright soul also added a 'verify the signature' step to the build process, and the ability for the build system to know that signing failed, you've automated the entire thing and now your checklist is documentation for the build process.

Curating a list isn't about building it as much as it is about maintaining it.

Then remove it from the check list? They're not immutable.

Create a checklist to review your other checklists on a schedule?