I think it's a consequence of taking away individual judgement and becoming, it seems, terrified of letting a person make a decision, because sometimes they're bad at it, or have biases, or whatever. Organizations are afraid of individuals deciding things, and individuals are eager to push their decisions onto a "process" or "policy" because if they actually make a decision and anything goes wrong—whether or not the decision was a good one given the information they had—they're toast, fired if they're lucky, publicly demonized and blackballed and fined and jailed if they're not. The individual cost of a decision that is bad—even if it's only bad in hindsight—is too often way out of proportion to the harm it caused, so why risk it?

I don't know why it's happening, but it sure explains a lot of things across much of US society. Politics, business, education, parenting. I wouldn't be surprised if it's part of what's behind the frightening US "cost disease" phenomenon from the Slate Star Codex post that was discussed here a few months back.

I'm only speculating, but I think social media has only exacerbated this problem. Now every screw-up an employee makes (or a teacher, or an administrator, or whatever) is magnified times ten to everyone on the aggrieved party's Facebook feed. Have a teacher that said something a parent took offense to? An employee who misplaced an item you had ordered? An employee can always fall back on the policy and push the blame to the institution and the institution can say "we'll review our policies".

Funny thing is that as a bureaucrat, coming up with policies is my job. It tends to improve overall outcomes and satisfaction, but the same outliers described above are also magnified ten-fold (this policy is stupid because ...).

My best guess is that it's the result of 20-30 years of never accepting "shit happens, we looked and everything seems basically fine, we're not changing anything" as an answer when something bad happens and someone asks "so what are you doing to make sure this doesn't happen again?"—even when that answer's actually appropriate. The consequences are so bad if you make the do-nothing call and get it wrong (or are just unlucky) that everyone does a whole bunch of stuff to ensure they can say "look at all the things I did!" if lightning strikes twice (or if they were simply wrong and something did need to be done, but it wasn't clear until a pattern emerged)

Instead we always freak out and add a bunch of new policies/guidelines, some more mandatory educational videos or classes, and maybe bring down the hammer on some people as well.

[EDIT] in fact it may be yet another symptom of post-Nixon-era cynicism. We expect institutions to lie to us about everything (which, to be fair, they often do and often have), so we (rather, the media) dig into any problem expecting criminality and neglect and generally bad behavior (and a great story!), so institutions have to go nuts to make sure it never looks like there's anything there worth reporting as maybe-sort-of-kind-of any of those bad things.

The FAA, NTSB, and airlines have made air travel incredibly safe over the years by operating the way you described.

I am by no means an expert on this subject so correct me if I'm wrong but....

The exception that I've noticed from the airline-industry, is that they actually want to solve problems systemically and don't care who has to take the blame.

For something to go wrong, multiple failures have to happen, and there's not a lot of blame put on individuals, as much as "the process".

In most other organizations, IME, fixing "the process" is expensive. It takes time, and effort, and it's quicker and easier to blame "those idiots in IT/accounting/HR/etc". Most of those types of organizations don't actually want to solve problems, they just want the problem to go away. So they end up choosing the easier option, rather than dealing with the problem systematically.

/bitter rant

I am not an expert but I know many pilots, watch only documentaries, read research papers almost exclusively, and have stayed in a Best Western.

NTSB is, as far as I understand, actually very careful about not blaming anyone. They look for causes and solutions, not blame. They, as far as I know, pride themselves on this.

They try, again inasmuch as I know, to not blame a person - but to blame a process. This is not a distinction without difference. It fosters an attitude of cooperation and openness.

Again, I'm not an expert but I know a bunch of pilots and listen when they speak. If the NTSB blamed people, I suspect they'd have a lesser willingness to speak positively about NTSB.

Edit: Someone beat me to it. I will leave this as I think it offers a bit more of a comprehensive view.

> They try, again inasmuch as I know, to not blame a person - but to blame a process. This is not a distinction without difference. It fosters an attitude of cooperation and openness.

Because even when the cause was blatant human error, it can still be a process problem -- how did this type of human error slice through all the protections against it? What process can be put into place to prevent disaster even when some bonehead does that same thing again?

But air travel safety is atypical.

If the crash rate for planes was the same as it is for cars then air travel would have to be prohibited as a necessary measure to prevent the extinction of the human race.

Most endeavors are not of that sort.

And so in most other contexts we could use their methods to produce a process that will prevent a particular category of trouble, but suffering the trouble costs $2M (instead of $2B) and implementing the process costs $20M.

And then people will want to implement it anyway, even though they shouldn't, because "it solves the problem".

Or, seeing the obvious fallacy in spending $20M to prevent $2M in harm, a "compromise" is proposed to spend $1M to prevent 5% of the $2M harm, still with no one doing the math. And then, problem still 95% unsolved, more half measures are kludged on over time until the surrounding bureaucracy becomes politically powerful enough to be self-sustaining.

Because people don't want to admit that some diseases aren't worse than their cures.

> If the crash rate for planes was the same as it is for cars then air travel would have to be prohibited as a necessary measure to prevent the extinction of the human race.

Well, no, this is obviously false. Much more car travel is done than air travel; bringing air fatalities up to the level of car fatalities would have a negligible effect on the human population.

Bringing crashes per vehicle mile up to the level of highway crashes per vehicle mile according to https://www.rita.dot.gov/bts/sites/rita.dot.gov.bts/files/pu..., assuming that an average airplane carries 140 people, and assuming (very wrongly) that crashing it kills all 140, you could get deaths to airplane crashes up to 843,000 per year for the US. This would be significant -- it would be an increase of over 30% in the annual death rate -- but it is less than the existing surplus of annual births over annual deaths. A modest reduction in population growth just isn't going to drive the human race extinct.

You're assuming the existing expense of making air travel safe is not reducing the quantity of air travel done.

Imagine it was as easy to become a pilot as to get a driver's license, there were no flight plans or restrictions on where people could take off or land and a 500MPH jet could be purchased for $150,000 because there would be less regulatory overhead and more competition in aircraft production.

Affluent people would be commuting by jet. There could easily be ten times as many air miles traveled or more.

If you want to "reason" that way, take into account that (1) if people started dying from air travel at high rates, they'd do less of it; and (2) societies with much higher death rates have no problem growing anyway, as they compensate with higher birth rates.

> if people started dying from air travel at high rates, they'd do less of it

That doesn't seem to stop people from driving cars.

> societies with much higher death rates have no problem growing anyway, as they compensate with higher birth rates.

That's just survivorship bias. Why would one cause the other? There were also societies with high death rates and low birth rates which for the obvious reason no longer exist.

> There were also societies with high death rates and low birth rates which for the obvious reason no longer exist.

If you claim to be worried about extinction of the human race, you'll have to consider all societies, not just the ones with pathologically low birth rates. A tiny group of people committing suicide has zero effect on the overall human population.

>They look for causes and solutions, not blame. They, as far as I know, pride themselves on this.

If the cause is a person's behavior, that is the same thing as blame. Look at the report of the Pinnacle 3701. Probable causes from [1]:

1. the pilots' unprofessional behavior, deviation from standard operating procedures, and poor airmanship;

2. the pilots' failure to prepare for an emergency landing in a timely manner, including communicating with air traffic controllers immediately after the emergency about the loss of both engines and the availability of landing sites;

3. the pilots' improper management of the double engine failure checklist

It's hard to look at that and say that the pilots weren't being blamed.

1. https://en.wikipedia.org/wiki/Pinnacle_Airlines_Flight_3701

It does happen, but is the exception. Look at the aftermath section. You can also look at the aftermath sections of other flights.

Sometimes, a human is to blame and there's no getting around that. This doesn't change their goal.

The difference is the approach — they look for the cause and sometimes the cause is the blame.

When you only look for the blame, you tend to be satisfied once you reach that goal and might ignore the cause.

NTSB, i think, no one takes the blame. For example, if the error is something like "Mechanic didn't tighten bolt to appropriate torque."

So, what's wrong? maybe they forgot and need a checklist item? Maybe the reading on the wrench should be recorded in the log? should there be a second person that verifies?

They know they ask a lot of people, and it has to be right, every time. People get sick, tired, distracted, whatever. we're people. People are the least reliable part of the system.

I think blame only falls to a person when they've lied. Falsified a log, claimed to do something they didn't etc.

That's the thing. In the Air Force (at least)- there is no thought of cost/benefit analysis. A single machine fails once because a bolt wasn't torqued and causes $5000 of damage ... So they add the checklist item, and the second person, and an extra training item, and a log record that has to be maintained. The list of special rules and required documentation only ever gets longer. Eventually, you spend all your time on the checklists and maintaining the documentation, and none on actually doing whatever procedure you were originally messing with that bolt for.

As someone who was a mechanic in the Army, I feel that checklists more often helped then harmed. Many mechanical problems we saw in the shop were easily attributable to an operator skipping a step on their PMCS. Doing something relatively dangerous or confusing could be error prone, especially for people who haven't done it much, and a checklist can help ensure the safety of everyone in the shop. A good example is running an engine indoors. Before you did it, there were a serious of steps you had to walk through before you turned it on, that reduced people getting sick from carbon monoxide quite a bit.

"The Checklist Manifesto" is an -excellent- read on the helpfulness of checklists in preventing problems and mistakes (primarily from a surgeon and medical viewpoint but he covers airlines as well.)

http://atulgawande.com/book/the-checklist-manifesto/

As a submarine fire control tech, I always thought it was cumbersome to have three full size binders open in my lap during weapon firing drills, casualty drills, everything except calm open water transit. But flipping those pages and grease pencilling things even while in the hot of prepping weapons to leave the tube avoided a LOT of "hurry up and whoops!" mistakes. Sometimes crufty stuff needs a review and rewrite but, n the whole, I'd rather know what happen wrong the previous times and avoid it myself.

This kinda contradicts the idea further up that too much attention is paid to process, at the detriment of, for lack of a better term, "skill" (or "personal responsibility")

I think the focus on process can be terribly annoying, because it's so much easier to say "Bob fell asleep at the machine" than diving into all the events that led up to Bob falling asleep. Note that both can be true or, more accurately, the process problems actually need the statement "bob fell asleep" to be true to make sense.

The idea to prioritise processes is that it's repeatable. Individual behaviour isn't within your control, except for the processes that hired somebody, educated them, defined their tasks, and created the environment in which they perform these tasks.

In a small team, or family, or group of friends, it's perfectly fine to dock Bob's allowance for falling asleep. And the instinct to exact revenge, or punish people is strong, because those instincts are the result of evolution and norms of society, both of which until recently only knew small groups where everyone knew everybody. That's why it takes a lot of discipline to act against these instincts, and obviously, as you said, because focussing on the process may also uncover others who failed, and it is more tedious.

People get frustrated with bureaucracy. Sometimes they're obviously right. But the idea that all bureaucracy is wasteful, or (equally common) that governments have any particular talent for wasteful bureaucracy is somewhat misguided. Because there aren't many examples for organising teams of sometimes hundreds of thousands of people to work on shared goals without a wasteful bureaucracy, and governments and the private sector tend not to differ that much.

I've seen one interesting example of a different structure for organising a large number of people: capitalism. It arguably works, being excellent at delegating authority to exactly the right person for any possible decision. But I believe even that example isn't convincing, if you start considering all the "waste" that capitalism hides in plain sight: advertisement, competing efforts doubling work, the complete financial sector, non-monetary losses due to financial hardship, the losses from unemployment being essentially wasted potential etc.

But unlike the NTSB, the military still routinely blames "pilot error" to cover up problems with terrible aircraft designs.

http://www.military.com/daily-news/2016/01/29/billows-of-dus...

Not accepting "shit happens" makes sense in aviation, and you're right about that

But taking out decision control and relying on automation has had its problems.

I think it's less about outside regulators than organizations/individuals optimizing to cover their own asses. I assure you, I'm not in the "gov'ment regulators ruin everything" camp.

> I think it's less about outside regulators than organizations/individuals optimizing to cover their own asses. I assure you, I'm not in the "gov'ment regulators ruin everything" camp.

The problem being that the ass-covering is a predictable response to known stimuli.

What do you propose to do to get them to stop doing that?

I believe you're being slightly unfair to those institutions. As one example: the US Government, as a policy, never lies to the American press (and, by extension, the people). They may sometimes do so to the foreign press, or they'll deny comment, or they'll give you an answer that doesn't actually answer your question. But there used to be a line they dared not to cross.

There are many reasons for this. Some will insist it's the morally right thing to do. But a better explanation is usually that it's in no one's interest: Career bureaucrats are simply never in a position where lying is a good option unless they're trying to cover up a crime they committed. They're risking their job, and possibly criminal prosecution, without any real potential upside. For elected officials, the same is usually true–with the potential exception of a lame-duck President who has nowhere to go but retirement. But at that point, they've probably gotten used to managing their office without lies.

Of course that's a description of the past, and the lower reaches of elected executive office (but not employees) have a worse track record, where many a town mayor with a coke habit has lied to cover his creative accounting. Nixon also comes to mind. But before you say "Iraqi WMDs", I'd say that was arguably a case of a government convincing themselves through wishful thinking.

Maybe they're just really good at lying? It's the old problem: is the revelation of a scandal proof of spreading corruption, or does it show that the system is working, and the bad apples are caught? It really seems to come down to personal judgement. I can only say that having interacted with governments on multiple levels, and seen a lot of, for example, the vast amounts of data they produce and decisions they make that are never the subject of the news, I have come away with the impression that the Government is usually trustworthy, and that decisions are often made with a surprising lack of emotions. For example, I once won a lawsuit against a local government. Yet a few months later, we won a tender that we deserved to win, even though the decision was made by the very same people we had previously fought in court.

If all else fails: even if government were corrupt to its core, saying so would be less effective than acting as if it weren't, and focussing on specific incidents. Because faced with a public that believes only the worst of every politician removes all incentives for them not to become exactly that sort of thoroughly corrupt politician. And if it's part of society's lore that all powerful people lie, that becomes a self-fulfilling prophecy when people new to government just follow the clues they got from popular culture.

> the US Government, as a policy, never lies to the American press

I'm sorry.. what?

If you speak any foreign languages try watching government-controlled media in 3rd world countries.

They just... make stuff up. Stuff that can be easily checked via a quick Google search.

Look at Russia's state-controlled TV, the current leader in this.

I agree other countries are worse but the US government makes things up and lies constantly. Obama wasn't perfect, but I don't think it's partisan to point out how Trump's admin is utterly blatant in this, comically so.

> As one example: the US Government, as a policy, never lies to the American press (and, by extension, the people).

Is this some kind of advanced sarcasm?

If you accept that the US Government didn't lie just told the least untrue thing it could then it fits. It just turn outs the former is identical to the latter in most cases.

I think you meant to say "US Government, as a policy, never tells the truth". If they did, we the people would have thrown them out on their asses before their terms are up.

Yeah, I think the big problem is trying to come up with the one big issue and change that, as if that will not have a whole bunch of issues.

People want one answer on what the problem is, because then the most amount of people can understand it in a tweet. They don't want complicated solutions. You say it yourself, it is one problem.

I think you're observing Pournelle's Iron Law of Bureaucracy, which I guess could be stated pithily as "The goal of every bureaucracy is to become self-sustaining. Those that don't are replaced by those that do":

https://en.wikipedia.org/wiki/Jerry_Pournelle#Iron_Law_of_Bu...

It's interesting that another active comment thread on the HN front page today is about evolutionary game theory & the selfish gene. I'd posit that there's a whole class of "evolutionary systems", subject to the axioms of 1.) large numbers 2.) random variation 3.) selection bias and 4.) competitive pressure. And any such system will always tend toward survival and maintenance of the status quo as its primary imperative, for the simple, tautological reason that those organizations that don't are replaced by those that do. This operates on multiple levels, from the molecule to the gene to the cell to the individual to the firm to the bureaucracy to the society as a whole. Oftentimes, behavior that makes no sense on one level seems obvious if you consider the "actor" to be the level above or below what you're looking at: the gene instead of the individual, or the firm instead of the individual, or the society instead of the firm.

In your example, for instance, the primary actor whose survival that the bureaucracy optimizes for isn't the individual: it's the bureaucracy itself. Individuals who serve their own interests instead of the bureaucracy are rejected and fired, as are individuals who serve the other individuals that the bureaucracy supposedly benefits. Why? Because bureaucracies without this incentive structure are replaced by bureaucracies with it.

And there actually are plenty of areas within U.S. society where bureaucracies are not the primary actors. The individual is still valued over the firm or bureaucracy within therapy culture, artist communities, fandom, Internet forums, dinner parties, and many other social contexts. But because there is no central bureaucracy or organization to point to, these niches fall off our radar screen; niches where individual humans remain dominant are by definition areas without enough resources to support a bureaucracy.

Baumol's cost disease is different, and IMHO a more short-term ailment that's afflicting developed economies as they transition to the information age. This phenomena of growing ever more complex systems on top of individuals is pervasive, and will continue to happen as long as the population keeps growing.

The trouble with that analysis is that "self-sustaining" doesn't inherently imply large or bureaucratic or inefficient. It's possible for a large inefficient bureaucracy to be replaced by a more efficient self-sustaining system with less centralization.

The bureaucracy can fight but it can also lose. Blockbuster Video is no longer a thing that exists because of market pressure. Standard Oil was broken up because of political pressure. There have existed countries that, following the conclusion of a major war, actually disbanded the majority of their military forces, both voluntarily and due to external force. Countries have revolutions where the revolutionaries win and then exile or execute the previous ruling bureaucrats.

In theory the same thing can happen through the democratic process, although this is obviously less common because bureaucracies have a solid understanding of how to subvert electoral opposition.

Your examples are of bureaucracies destroyed from the outside, by stronger competitors for the same resources. It's quite impossible to curb a bureaucracy from the inside, e.g. through "democratic process". The system will not fix the problems with the system.

Perhaps, but so don't do that.

If the wasteful bureaucracy will not commit suicide then help some less wasteful one to commit fratricide.

Just need a competing Navy to conquer the US. Easy as pie.

If they have container ships, I wouldn't bet against them...

I suppose it's like cows that evolve ever larger horns to compete for mates, then one day a new predator comes in and kills all the cows because their horns are too heavy and they can't run fast enough.

I think that's slightly different, an "outside of context" problem, as Banks would say. The cows didn't leave enough flexibility to adapt to the unexpected, rather than just being inefficient.

> I think you're observing Pournelle's Iron Law of Bureaucracy

This sounds like a special case of the more general law of survival of the fittest (where 'fittest' means 'most self-sustaining').

A less cynical take on the taking away of individual judgement is that it arises from the need for individuals to be replaceable rather than solely as a means of deflecting blame ...

Coastguard ships for example cycle the entire crew every two years. I imagine it must be similar in the navy. There is very little institutional memory retention -- and so the "process" emerges to try to direct all the human cogs as they pass through ...

UK experience here. All military units (including ships) experience staffing ‘churn’ as individuals join and leave for career and other reasons. Some military units are periodically completely reset (e.g. a new ship or a ship that has been extensive refitted), and they then have extensive training regimes to become ready for operations again. The churn then starts again. Some units persist for long periods of time (e.g. a formed air squadron) and do not get ‘reset’ under usual circumstances, although they do have churn. Others (e.g. army formations) have repeating multi-year cycles of training, being ready for ops, and other tasks.

The institutional memory is notionally preserved by the instructors in the schools and training centres (as well as documented Standard Operating Procedures, standing orders, etc). New joiners will have undergone standardized training, and although they still have to establish their role (storming, forming, norming, etc) when they join an existing team, they do in principle understand what they are supposed to be doing. In some cases, new joiners bring valuable new knowledge, skills or attitudes, so churn isn’t always bad. Individuals who have joined up for the long term will themselves have cyclic periods of training, with initial trade training followed by time in a unit, then additional specialised training at a school, then more operational tours, etc. Some highly-trained individuals may be embedded in the units themselves (e.g. at a barracks) to provide standardized, decentralised on-the-job training / supervision to newbies who have joined from a military trade school.

More importantly, ALL military needs to be ready for massive war. If another world war breaks out they need to quickly scale to 100x as many humans. That cannot be done unless everything is easy to teach. Follow a checklist exactly is easy to teach.

One extra consideration here is that as power centralizes, the costs of individual mistakes increases.

It seems interesting to contrast this with the explanation of the financial crises that decision makers had 'privatized benefits and externalized costs' (I remember hearing it from Nassim Taleb, but there were probably others making the same point)

aligning incentives between organizations and the people who make them up is hard

> Organizations are afraid of individuals deciding things, and individuals are eager to push their decisions onto a "process" or "policy"

But! but we're in IT and the Agile Manifesto is the exact reaction to that! We have a living proof that "another way" is possible. Hire champions and trust them on the task, it makes them more enterprising and crafters and makes them deliver systematic higher quality. Remember how IBM's IT used to be: Pathetic chain of specification-driven documents that "decision-makers" couldn't possibly read.

Let's just say for the moment that the rest of the industries haven't found an "Agile metgodology" that befits them. Let's hope it's just a matter of time before government-sized organizations have an epiphany about how to apply Agile to death-risking processes.

Looking at this issue more broadly, I definitely agree that non-IT industries need to take into account Agile methodology to improve their results.

I've been looking for the reason why IT has come up with Agile before anyone else (I don't consider Lean Industrial Management as Agile whatsoever) and I simply think it's because the first industry with so much information in it (it's in the title even...).

Now that other industries have to orchestrate huge processes with huge amounts of information, I hope they'll take into account what IT brought up to solve this issue instead of taking the "IBM route" further (more policies, more guidelines, more bureaucracy).

> Looking at this issue more broadly, I definitely agree that non-IT industries need to take into account Agile methodology to improve their results.

Non-IT industries are often ahead of the “Agile” movement in IT, having adopted Lean methodology and related approaches that actually applying engineering approaches to solve the problem identified, but not actionably addressed, in the Agile Manifesto.

Most so-called Agile methodologies (the exceptions mostly being adaptations from Lean methodologies, either in other industries or from the Lean Software Development movement) fail to do this, and Agile has largely become exactly the kind of cargo cult, situation-blind consultant-driven, value-blind nightmare that the Agile Manifesto was a response against.

I believe in Dan Carlin's recent Hardcore History, The Destroyer of Worlds* he talks about the changes in military commands and micromanagement that have occurred with advances in telecommunications.

He compared the discretion given to commanders in the American West vs. Civil War battlefields. The more realtime the communications, the more that higher commands interjected in lower level decisions.

With advances in telecommunications, we shift decision-making up the chain of command, leaving subordinates to manage inspections and admin-weenieness.

*I can't find a transcript to verify, and as much as I'd like to re-listen to the 6 hour episode, I just can't at the moment.

It also could have a Harvard Business Review episode, after reading this: http://www.leadingageoregon.org/assets/docs/Other-resources/...

In the Battle of France in WW2 some German commanders (pretty sure it was either Rommel or Guderian) would regularly have radio "problems" isolating them from their HQs when they found tactical opportunities they wanted to exploit.

Supposedly the British Commandos were in part formed so that the generals could get Churchill off their back.

This while over at the German side Hitler would increasingly micromanage as the war dragged on.

You could make the argument that WW2 would have turned out very differently if Hitler hadn't been allowed to meddle with operations on the eastern front (e.g. calling a position "fortified" and forbidding retreat, which led to horrific casualties and usually ended up with the generals disobeying orders and retreating anyway, so the result was often the worst of both outcomes). Germany was broken by Stalin and Hitler before the Allies even hit the beaches.

This is a problem which is commen in the US-Military. As I know a dozen Soldiers - mainly Army - of the Bundeswehr, they all recall their trainings against US-Army units stationed in Germany. They all said: "Take out the higher ranking officers, then they are done." Even though the German Units were completely inferior in the main strength of the US, their superior quality material-based approach, they mostly managed to rip them to shreds with a high degree of flexibility. Practice in the Bundeswehr is to tell Officers, Nomcoms and Enlisted only "What to Achieve" and "Why to Achieve it", not "How to achieve it", allowing for higher initiative, less micromanagement and greater flexibility. What every US-Officer most feared were the Hit-and-Run tactics of German "Wiesel"-Vehicles, something the armored platoons were completely unable to counter. Or, as one US-Officer at a beer-fueled post-post-debriefing said: "We may have the better Material, but you have the better Soldiers"

I don't remember this from the "Destroyer of worlds", but hey, 6 hours, it might have been there somewhere and I just missed it.

BTW - it's not the recent one anymore, there's a new one. Yay!

The new one ("Celtic Holocaust") is about the Roman conquest of the Celtic people. Definitely worth a listen if you're into history/Carlin podcasts.

Personally I'm hoping for a "Common Sense" podcast by him sometime soon !

I saw that when I looked for a transcript (a recent, not the recent I guess). I'm two months behind in my podcast feed, so I'll catch up to the next one eventually!

Goodheart's Law [1]: "When a measure becomes a target, it ceases to be a good measure."

[1] https://en.wikipedia.org/wiki/Goodhart%27s_law

Let's popularize this amongst the ranks of management, senior administrators and every other supervisor.... and especially the groups that help create the tests and stats and 'merit' systems.

> The tests that start as attempts to evaluate and quantify effectiveness become the goals themselves, rather than measures of the true goals.

I think you need a second factor to make flawed metrics a big problem: increased competition.

More competition strongly incentivizes all actors in a complex system to favor optimizing over the currently-measured metrics rather than the true goal.

You see this playing out across all parts of society today: academics that chase impact score rather than groundbreaking research, bankers that chase short-term profits to the detriment of actual sound banking, and now we have a Navy that can't do basic seamanship.

I think we really need to recognize as a society that if the thing you care about is inherently hard to measure (e.g. science, the overall stability of the financial sector, the ability to fight the next war while still at peacetime), there are no shortcuts, you just need to suck it up and keep trying to measure the real thing rather than only relying on proxies.

Throwing more and more effort at optimizing over bad metrics will only lead to long term disaster.

This is so common one of Murphy's Laws of Combat Operations is: "No combat ready unit has passed inspection. No inspection ready unit has ever passed combat."

A favorite saying for USMC infantry, and much truer than people realize. There is a required balance though, and the best units have officers that know how to walk that tightrope, and listen to and respect their NCO's. A great example of this would be, much fewer uniform inspections, much more combat testing. Less formations 15 mins early by BN CO + 15 mins early for company CO, +15 mins for platoon, +15 mins for squad leader... It's the type of thing that really affects morale which will in turn affects effectiveness.

Off-topic: Let me get an Oorah.

On-topic: I noticed the tendency towards formality when dealing with many, many municipalities. This is, in and of itself, inefficient in many areas.

Example: A hole opened up in my schedule and I can fit your meeting in sooner. By the time I got a response, that hole was long since gone. Worse, I left that hole open in hopes that they'd respond sooner. They will say that you should have let them know, but you left a half-dozen messages with their secretary.

Lesson learned: Get to know the secretarial staff. I'm not going to suggest you bribe them, but I understand if you do.

Compromise: You get an 'errr', cause I'm salty.

Always makes friends with the doc, supply, and admin. Usually through booze and not being an ass.

My default reply would get voted down, a lot.

I don't care. My verbiage is intentional.

We all salty, Marine. We all salty. It took me years before I could leave my rack without ensuring it was squared away. It took years before I could invite the missus to touch me while I slept.

Eight years is what I served. Worse, it was in two pieces and separated by time at MIT. Yeah, I was a smart Marine. Don't laugh, I can drive a broom like it is nobodies business.

Anyhow, I won't tell you I understand. I will tell you that I can relate. As for the karma, I ain't never scared.

But, you're right. Always be nice to the folks who decide your life. Strangely, that's frequently not the person who projects power. However, I speak to the choir. Hopefully someone else will pick up on this. Otherwise, I ain't never scared. Karma was meant for burning.

<we all salty

Methinks us that are out tend to be salty, those still in, seems like drawdown has bled a lot of salty knowledge out.

I would claim the core problem is an environment than incentivizes people to care more for their careers than for the actual work they are doing.

Sounds pretty much like Soviet Union, actually [0]. The chickens have come to roost, indeed. Managerial science driven process oriented organizations and what happened in the soviet countries share the same basic tenent: individuals don't matter. This is a cancer for performance and quality of life. The former should concern all economically minded people and the latter just everyone. It's not a lose-money, gain happiness proposition, it's a gain-money, gain happiness one.

The problem is that local optimization (career, prestige) causes the global system to reach an unoptimal state. In computer science we say - greedy algorithms seldom lead to globally optimal solutions[1].

Of course, no one is stopping from developing key-performance indicators for a person being a good person - but this seems really arse-backwards way to approach the problem as being a good person is what most people would prefer to do naturally given a healthy environment to begin with.

To slightly misquote Suzuki on his basic idea on how to maintain good character in a group: "Watch, but don't judge. But, do watch". [2]. Ie. make it understood everyone is responsible for their results but don't be an anal retentive accountant while you're at it.

[0] http://militera.lib.ru/research/suvorov12/index.html This is a damn fascinating read. A bit like autobiographic Catch-22.

[1] https://en.wikipedia.org/wiki/Greedy_algorithm

[2] https://www.amazon.com/Zen-Mind-Beginners-Informal-Meditatio...

In my view, hand-in-hand with measurement is the proliferation of stuff to measure.

It occurs to me that at some point in the history of navigation, the skipper was probably the only literate person on the ship, if that. His log, if he kept one, was the only source of data. And you couldn't get his account until he got all the way back to port, assuming he survived.

Today, the ship (or business) is a floating, real time data acquisition server. Folks all over the world are analyzing the performance of the ship from every conceivable angle. They are literally gathering and processing more data than the skipper herself can even comprehend.

Your culture is what you actually reward and punish, not the abstract ideals that you talk about.

There's a great aphorism with similar meaning:

People do what you inspect, not what you expect.

edit: Wikiquote attributes to Louis V. Gerstner, Jr. (IBM) but earlier instances can be found (see comments below).

Lovely bit of language.

That's an old nuclear-Navy saying from at least as early as the 1970s: You get what you INspect, not what you EXpect.

Earliest citation I could find is a management book from 1962:

http://www.barrypopik.com/index.php/new_york_city/entry/you_...

I've thought about not saying this, I intend no disrespect to anyone. Are there two militaries now? There is JSOC, with a budget, toys and gear, their pick of everything and they are a smallish, nimble ultra trained force designed to engage these terrorist groups and do special tactical missions, fighting dirty if needed. Then there is the greater navy and army and, respectfully, it is sort of a conservative voter friendly rural jobs program. I'm not saying this about the volunteers that sign up, but national policy. It only makes sense that they'd squeeze every dime from it like everything else. To top it off, there are the private contracting outfits that do stuff too, where does that fit in?

The idea that they're supposed to learn on the job is insane, you don't get a lot of second chances on aflight deck, you don't get to make a lot of mistakes on a boat twice.

There sort of are two militaries:

- One we maintain to prove to our potential adversaries that we maintain it, in order to preserve stalemate.

- One we maintain to actually use.

I'm going to venture a guess that a well-functioning defensive military is inherently unstable, as anyone who understands how powerful it is will realize that it never actually has to function.

> well-functioning defensive military is inherently unstable

This is why it makes sense to have a purely defensive military work on a reserve and draft basis. You don't really need the defensive military - until you do - in which case you want to magic a really big and reasonably competent one into existence quickly.

It is not as simple as that e.g. JSOC doesn't have its own ships, limited aviation, etc. It is totally reliant on the "regular" military to support its operations. And of course it gets most of its recruits from there too - AFAIK only the SEALs do direct entry for civilians.

I think the "second military" you describe is more of a "minimal readiness for SHTF situation" group.

Because no politician wants to go down in history books as the person who let the military shrink and lose readiness for a WW3. Sure, maybe the American military has indeed lost readiness, but it's more nuanced and can't be pinned on one person. So just like other parts of this thread suggest, it's a giant CYA operation.

I hate to be so grand but this seems to be a problem across many sectors of the entire western world.

Over and over, in many different guises, I hear stories like this one. What characterizes them all is excessive faith in reductionist measurement-driven processes, and then the attendant army of humans-of-stock-photography consultants that inevitably lurks in the vicinity of such processes.

Goodhart's Law, "When a measure becomes a target it ceases to be a good measure."

Juking the stats ala The Wire, gaming statistics about crime becomes the objective over good police policy or reporting good, valid data on crime. https://www.themarshallproject.org/2015/04/29/david-simon-on... http://www.motherjones.com/crime-justice/2012/06/fbi-crime-s...

When I worked at DoD, it was explained to me with an analogy of monkeys and ladders. You have a ladder, and at the top is a bunch of bananas. You rope the ladder with electrical tape, so that any monkey who attempts to climb is shocked. Eventually, the monkeys will socialize and prevent each other from climbing the ladder.

What's interesting is that this will continue even after you remove the electrical wire.

Sounds like this mythical experiment - http://www.throwcase.com/2014/12/21/that-five-monkeys-and-a-...

Partially mythical, there were no bananas and if I remember well there were only two monkeys. [1]

[1] https://www.scribd.com/doc/73492989/Stephenson-1966-Cultural...

What was that quote? If you measure success by testing, then you will get results that improve performance against the test at the detriment of all else?

Sometimes (often) involving cheating where existential considerations for organizations or leadership is involved.

"When a measure becomes a target, it ceases to be a good measure."

https://en.wikipedia.org/wiki/Goodhart%27s_law

not in military, but the joke among colleagues has been that we are very in following our processes, not so good in getting anything done. sadly this is excused because the process was followed and the auditors are happy

Is the degradation of US public schools really due to extensive testing?

I believe the idea is that the entire curriculum is then centered around teaching the test, versus what kids actually need. As to whether that's the primary issue, no idea.

Do you really think eliminating standardised tests would improve the education system?

I'd be far more okay with standardized tests if the requirement also banned homework until at least Junior High. Otherwise it's a race to the bottom to see which schools can suck up as much of the children's and families home time to help improve the schools scores.

How would you propose that students practice learning and studying on their own, outside of a classroom setting, without homework?

I don't propose that children do any of that. That's what high school is for, and 4 years of "practice" is more than enough.

Some of the reason schools start homework so early is that parents demand it. I've seen parents complain that a first grade class isn't academic enough because the teacher didn't assign enough homework.

There's an excluded middle, in that statement. De-emphasizing the importance may be an option.

Fair point. Let me rephrase the question:

Do you really think reducing standardised tests would improve the education system?

If so, what objective evidence can you provide for this belief?

If you measure it less, you'll have a weaker argument for how the education system is doing!

Your comment is precisely the problem: objective metrics will always support their proponents' arguments for more measurement, while any upsides to a lessened objective grading will be difficult to show because they'll involve subjective assessment or less assessment.

Do not confuse the investigator's convenience in accessing a truth value for the truth value itself.

That argument has been made many times and in great detail. See for instance the work of Diane Ravitch, former US Assistant Secretary of Education (https://en.m.wikipedia.org/wiki/Diane_Ravitch)

Edit: she was Assistant Secretary, not Secretary

School-funding and even whether a school closes or not is tied to these tests, so there would be some work involved in replacing that aspect of standardized testing in US schools as well.

No, the opposite. We didn't understand the decline of US public schools until we started trying to systematically measure their performance.

Standardized testing has been around for decades, but was a tool for sorting students. It is only within the last decade or so that we have tried to measure the performance of schools, districts, and states, as systems.

It always blows my mind when I see people on HN complaining about school testing and measurement. If you tried to run a web application or a business with zero ways to measure its performance, how do you think that would go?

The thing is, what do you gain by knowing that school X scores 13% lower on standardized tests than school Y? That school X is worse than school Y? How does that help you provide a better educational experience for the students?

My mom's been a teacher for ~40 years, and one thing to realize about standardized tests it that they don't provide any new information to the teachers---the teachers already know which students are struggling and which students aren't. In fact, standardized testing reduces teachers' ability to respond to variation in student ability, because all students need to be prepped for the same test. Preparing lesson plans is one of the most time-consuming aspects of being a teacher, and standardized tests make that task more difficult by adding more requirements. IMO, it would be better to focus on ways to reduce teachers' administrative burdens---perhaps by making it easier to collaborate and share lesson plans with other teachers[1].

[1] And I mean genuinely make it easier to collaborate, not forcing teachers to spend 90 minutes a day trying to enter their plans into the latest and greatest ed-tech website. If you want the information standardized, hire someone specifically to transcribe paper plans into whatever system the school administrators pick.

It is possible (even likely) that our current standardized tests need some adjustment to fit the needs. However the purpose of standardized tests is there are standardized needs. As a member of society I want everybody to know some things.

If a kid passes first grade without getting the correct answer to 2+2 there is a problem, it isn't acceptable to say your school doesn't teach math until second grade. In a single school that is just fine, but people move and so on a national scale that does not work.

Now I understand that not all kids have the same abilities or backgrounds, and both make a big difference in how far those kids can go. That needs to be managed, but it needs to be managed in the standardized process.

It does seem to me that teachers should have a few common lesson plans to choose from. There is no reason a teacher should have to create their own plan. Being a good teacher within a plan should be easier because they can prepare to use a working plan.

  Now I understand that not all kids have the same abilities
  or backgrounds, and both make a big difference in how far
  those kids can go. That needs to be managed, but it needs to
  be managed in the standardized process.

If our society were prepared to manage the fundamental issues that plague our educational system, then your rationale makes perfect sense, assuming that the standardized lesson plans were designed to facilitate the assessment of meaningful programs. But we're not prepared. Rather, the standardization efforts were an excuse to avoid addressing the fundamental issues, and have become a distraction that causes authorities to focus on the wrong aspects of educational reform.

The overwhelming factor in educational achievement of a population is the quality of home life, and in particular the participation of adult family members. Poor families have neither the time, money, nor enculturation to facilitate meaningful improvements in home life. Without a social commitment targeting the root causes, such as by providing free day care, the standardization efforts achieve nothing, and unnecessarily constrain local control.

Since the Bush-era standardization reforms, the achievement gap has only _grown_. And we're still matriculating kids without the requisite skills as the alternatives, like holding kids back year-after-year, are neither desirable nor practical.

"How does that help you provide a better educational experience for the students?"

It helps you identify which schools are in need of special attention and require structural changes.

With some competitive pressure from school choice at the margins, coupled with per-student funding, it would also provide incentives for schools to improve.

In practice, it seems to create a black hole where a school that serves a population that isn't good at tests (e.g disabled kids, non English speakers) gets low scores so everyone who could get a good score leaves and then the school does even worse and gets less funding and then the school is failing and the "fix" is to fire the administration, spend money bringing in new staff who don't know the area or the students, and expect that to solve the systemic issues revealed by poor test scores. To avoid this, schools are indeed incentivized to improve - for instance, by directing low performing students to stay home on the day of the test, or by discouraging students with learning difficulties from attending at all.

This effect is visible in urban centers as well.

Public education is supposed to be a leg up into the world for everyone, not the start of the rich getting richer and the poor getting poorer.

That means we are not using the tests results correctly though. Knowing there is a problem is the first step in fixing it - but it doesn't tell you what the next step is.

The idea that competitive pressure improves school performance is one of those simple, obvious, and wrong Econ 101 ideas.

Even if the bad schools can't respond to incentives to improve, allowing students to choose good schools over bad ones naturally increases utility.

No it doesn't. It could easily result in less social utility by promoting self-segregation and the concentration of funding into high achievement schools where the marginal utility of each additional dollar is substantially diminished at the outset.

The reason why kids from poor families do poorly in school is also the reason why those families are _least_ likely to be selective in their choice of schools. These are not independent phenomena.

So unless the only utility function that matters is freedom to choose your school, approaches like vouchers are unlikely to improve things and could easily not only perpetuate but exacerbate achievement gaps and derivative social issues, like crime.

The evidence indicates that voucher systems improve test scores, graduation rates, and other aspects of both public and private schools, not just the latter:

http://educationnext.org/rising-tide/

  Scaled-up voucher programs like those previously advocated
  by Secretary DeVos show the worst effects. There have
  recently been four statewide voucher programs: Florida,
  Indiana, Louisiana, and Ohio. The Florida study is
  inconclusive, and the others show large negative effects. In
  some respects, the Louisiana results are more convincing
  because the results have been corroborated by two different
  sets of researchers and students were assigned to vouchers
  by lottery—the most rigorous way to evaluate vouchers. In
  terms of providing convincing results, the Indiana and
  Ohio programs, are somewhere in between, but these show
  negative results as well.

  Voucher supporters argue that the results have been worse in
  the recent statewide programs because they have been
  “heavily regulated,” by which they mean the requirements
  that students be tested, that these results be made publicly
  available, and that schools must let in any student who is
  eligible for the voucher. The fact that this fairly minimal
  oversight is considered controversial or heavy-handed,
  however, only reinforces that private schools are designed
  to be exclusive and have little interest in external
  accountability.

https://www.brookings.edu/opinions/why-managed-competition-i...

15 years later, http://news.stanford.edu/2017/02/28/vouchers-not-improve-stu...

I wouldn't expect charter schools to actually change much because the evidence is very clear that the most important factor in achievement, overwhelmingly, is the child's home environment.

Do you have evidence to back up your claim that it is "wrong"?

If your metrics for your web application aren't a good proxy for user experience, those metrics can drive you to make your application worse.

Focusing schools on testable performance leads to unintended consequences. My kids had heavy homework loads starting in first grade. They were still kids! and got robbed of much of the joy of childhood that should been afternoons playing at the playground and running around the neighborhood. All so the schools could pump up their test scores, by forcing their "customers" to carry the load.

There's some relevant business wisdom: "what gets measured gets done".

By aligning our educational institutions towards testing we created educational institutions that teach the test and not the rest.

Standardized testing is super important. I agree, though, it's a pretty poor proxy for how well schools are performing and who needs what... By way of example: schools with low-scoring test takers likely have environmental issues that impact school performance. The conclusion should likely be beefing up the budget and expanding the schools role to being more "community centers" full of social workers.

Poverty is a cycle, individual disruptive elements in school hurt all students, and that little 'war on drugs' has removed a lot of the support structure we expect in those families... Reducing the budget of schools with students in bad situations who can't test properly because their students are in bad situations and therefore can't test properly seems like a self-fulfilling prophecy. A prophecy that happens to align very nicely with the ideology and stated goals of the group that pushed "no child left behind".

> By way of example: schools with low-scoring test takers likely have environmental issues that impact school performance. The conclusion should likely be beefing up the budget and expanding the schools role to being more "community centers" full of social workers.

This is a great idea I would be fully in favor of. But we can't start having these types of conversations unless we know how schools are actually doing.

Reminds me of something I read on HN the other day, I'm not sure if it's true but one commenter said he got a driver's license without ever actually demonstrating he can drive.

If it's true then it's outrageous that this can happen in the US.

A good admin weenie would have laid down the law and forced the ships in their fleet to step up operational awareness and training. Astonishingly whatever attempt was made to address the problem was useless.

Emphasis is put on didactics rather than common sense.

> The tests that start as attempts to evaluate and quantify effectiveness become the goals themselves, rather than measures of the true goals.

Is this a dank meme?

sounds like the tests are bad.

sounds like US Army in 1975.

>“We put a premium on being good inspection takers and admin weenies.”

such a shift is natural and necessary in the context of the future warfare being completely robotic/autonomous. People role would be exactly inspection and admin. And as preview of the future - we've just had 2 people driven destroyers lose to the most probably autopiloted a cargo ship and a tanker.