For context, Ofsted is the much-maligned government inspectorate of schools in England. They are currently under increased scrutiny following an inquest into the suicide of a headteacher following an inspection that concluded at the end of last year: https://www.bbc.co.uk/news/education-67639942 and https://www.bbc.com/news/education-67639943

I hope people think more about cases like this and the Post Office when they go to say “it’s not like we’re curing cancer” or “my bad code isn’t a matter of life or death”.

You might be far removed from the implications of doing a poor job but that doesn’t mean everyone else is.

While I do agree in principle, I think it's also the case that "the software" has been used as a convenient scapegoat and cover for what were incredibly poor practices and dishonest, irresponsible conduct.

The fact that software is involved is in some ways a side issue - and while I do agree we as techies do need to consider the responsibility that we have, I'm equally frustrated how the narrative in the Post Office case has been deliberately shifted towards "it was because the software had bugs" as opposed to "multiple people in charge were deliberately deceitful and behaved completely reprehensibly".

This is borne out by the UK government's attempted shift of all the responsibility on to Fujitsu in the Post Office case (who are absolutely not blameless) while trying to push attention away from the Post Office's - and by extension the government's - own considerable culpability.

I'm sure they'll seek to do the same in this Ofsted situation.

I get it, but I have to disagree on two ground.

Firstly, in UK law there is an implicit assumption that computers are "reliable". This means that the burden of proof is on the accused proving the software has bugs, rather than them being presumed innocent until proven guilty.

That means in the Post Office example, people were convicted because they couldn't prove the software had bugs, they were just saying "you have to prove I'm guilty, because I'm not", and the court said "no, we don't, computers are reliable and that proves you are lying and therefore guilty".

In the Ofsted case, given that schools can be put into special measures, teachers and headteachers fired, and in one case a headteacher killed herself because of the data collected and decision made on it, if those systems are not reliable they are not fit for purpose, because it turns the innocent into victims in terms of criminal, and civil employment law.

There have been calls for this assumption to be changed [1], but right now: if you are innocent but a piece of software says you are guilty, in the UK the burden of proof falls to you, regardless of how awful the people, their processes and conduct are on the other side.

Secondly, there's a deeper moral issue here. Engineers are arguably aiding and abetting. It is almost certainly the case that engineers working on Horizon and Ofsted systems did not tell all the truth all the time at the bequest of their managers and other colleagues throughout all this.

"Just doing my job", is the excuse of the scoundrel. I do not buy it. If you're complicit with despicable people, you're complicit. End of.

Yes, UK HMG has answers to give, culpability and responsibility, but let's not pretend that engineers built terrible systems, didn't have to face any consequences, and were enabled by arse covering exercises further up the chain.

We can - and must - do better than this.

[1] https://www.theguardian.com/uk-news/2024/jan/12/update-law-o...

'That means in the Post Office example, people were convicted because they couldn't prove the software had bugs, they were just saying "you have to prove I'm guilty, because I'm not", and the court said "no, we don't, computers are reliable and that proves you are lying and therefore guilty".'

Not sure this is true, people were mainly convicted because they pleaded guilty to avoid going to jail.

There may be instances where you are correct but I'm not aware of any? The general rule of being innocent unless proven guilty does, generally, hold. The issue with these cases is that many people convicted early on had no proper legal representation.

Alan Bates, for instance, refused to accept the charges and was never convicted because no evidence of his guilt was ever presented. He stood his ground and was an outlier case.

> "Just doing my job", is the excuse of the scoundrel. I do not buy it. If you're complicit with despicable people, you're complicit. End of.

There's a world of difference between human fallibility, and knowingly doing evil as part of one's job. I won't call the bricklayer evil if someone builds a 10-story skyscraper [1] atop his humble garden shed foundation, that ends up collapsing and killing everyone inside.

[1] To make the analogy perfectly clear - the skyscraper is not the software, but the persecution based on assuming the software is infallible.

In this case, one of the main issues seems to be that assigning a grade of "unknown/uncertain" wasn't an option, regardless of what issues could have happened during the field work ?

I'm also annoyed at how the Guardian is parroting the "made up" term, which is a terrible misrepresentation of "relying on your memory of the events".

Speaking of decision-helping tools, there has been a French law for some years now : for any administrative use of them, the target citizen can request the algorithm to be explained to them in clear language.

(I already have my popcorn ready for the inevitable clash with the fiscal inspection, which not only tends to operate as if laws didn't apply to them, but also I hear started using neural networks, which have basically no way to conform to this requirement.)

In uk they weren't allowed to inspect the software for a looooong time. And you can't prove anything without inspection.

> Firstly, in UK law there is an implicit assumption that computers are "reliable". This means that the burden of proof is on the accused proving the software has bugs, rather than them being presumed innocent until proven guilty.

This really is an incredibly stupid law. I could see assuming that computers are correctly operating as _instructed_ (e.g. the cpu doesn’t have bugs), but assuming that software is implemented correctly is just crazy. Everyone knows that software is buggy and updates are released all the time. Declaring that software is infallible is ridiculous.

The scapegoating happens because the software is the one "tangible" thing in the chain of blame. It's the one thing that can't be queried and reply with "if I had known...".

I do agree with you that Ofsted will likely use the same deflection tactics.

It is a failure of all of the humans involved. Whether it is inadequate acceptance testing, prematurely closed bug reports, optimizing for [unreasonable] timelines, or lack of oversight (and to be clear, I am not advocating for micromanagement).

I feel both of these statements are true: 1) certain people bear more fault than others and 2) developers aren't blameless.

I would like us to reduce our "blame surface area" with a reasonable amount of effort over what we _can_ control.

and the government were told the system was likely to be unreliable before they even bought it:

https://www.telegraph.co.uk/politics/2024/01/26/sir-geoff-mu...

The whole thing was a shambles and there was no scrutiny from the start.

Postmasters were literally calling Fujitsu support and telling them that they can see the software is literally wrong(giving explicit examples of depositing £100 and the software showing that there should be £200 in the till) and literally nothing was done about it.

Absolutely. As far as I know both Fujitsu and the Post Office were aware of the bugs in the system, and deliberately covered them up.

Your comment brings up an old memory. Lady mentioned once when she was a regional manager at a department store chain. They started having the till go short every so often at one store. Usually it's one person stealing. But looking at the video's they couldn't tell who. And there wasn't a pattern. Then it started happening at two other stores. And then another. The head of security had a hunch, took her to one of the stores after hours and pulled the register apart and they found two hundred dollars inside the machine.

The first store had been the first to get a new model register. And the new model tended to slurp bills out of the till if you closed it too hard.

So this isn't a new software thing. It's a problem with a management culture that shifts blame rather than get to the root of the problem.

That's a great case, and well worth studying for anybody involved in forensics or adjacent fields. Way too often the first target for 'the blame' is stuck with it without someone taking the trouble to really find out what is happening.

While it's obviously better for bugs to not happen, the issue here is that both Ofsted and the Post Office covered up the existence of the bugs, lied about it and didn't fix them.

If they'd been like "oh, a bug. We'll fix it" then there would be no issue.

In this case, Ofsted are playing the role of the Post Office prosecutors making career-limiting decisions based on inadequate or forged evidence.

The code is usually a reflection of the organisation, not the individual. So while we should all try to write as few bugs as possible a poorly run team can cause at least as many software defects as poor individuals.

budgets are a value statement.

just cause you're an ant here doesn't make you ethically culpable.

this isn't individual devs fault

this is bad practice by service integrators

What is that first link saying? That teachers need LESS oversight? This must be a joke.

Also what does software have to do with that case?

It's probably suggesting that oversight which is made public as a one word rating like "inadequate" is not a good way to fix problems in a school, since the exact same experienced teachers and principals you'd need to turn a school around are the most likely to avoid an "inadequate" school.

Presumably this feels even more unfair if the computer eats the inspectors' homework.

> What is that first link saying? That teachers need LESS oversight? This must be a joke.

Here's what it's saying, literally:

> An Ofsted inspection "contributed" to the death of head teacher Ruth Perry, an inquest has ruled.

> The inspection "lacked fairness, respect and sensitivity" and was at times "rude and intimidating", senior coroner Heidi Connor said.

> Mrs Perry, 53, took her own life in January while waiting for an Ofsted report to be published.

Seems entirely uncontroversial for them to review their practices and nowhere in there is it saying that teachers need less oversight.

Could well be that the kind of oversight that teachers are getting in the current system is counterproductive to the stated goals of said oversight.

> What is that first link saying? That teachers need LESS oversight? This must be a joke.

Are you suggesting this is a binary decision, where the only options is choosing "more oversight" or "less oversight"?

The status quo is "the same amount of oversight". And this article seems to say that since a teacher killed herself for getting "bad grades" then we should have less oversight.

How about "better oversight", "constructive oversight" or "beneficial oversight"? Let's not fall into a trap by restricting the space to a single dimension of quantity.

It's not saying that. It's saying that the current oversight is highly flawed and urgently needs changed. You're falsely equating qualitative change to quantitive change.

I believe the point the parent was making is that this isn't a single axes. There are a variety of different types of oversight that can be done on different aspects of schooling.

This is such a simplistic conclusion to draw that I can’t help but see it as intentionally dishonest.

Oversight is not a knob that you turn up or down. I speak as someone with experience in this field when I say that education like a lot of other things is complex enough that “overnight”, especially with output suitable for public consumption, is far from trivial to deliver.

Again, I don’t think that you’re arguing in good faith.

There are many problems with the current oversight system. Neither “less oversight” nor “more oversight” correctly describes the changes one can wish for.

The fundamental goal is not to punish teachers and headmasters/headmistreses. The goal is to ensure good quality and safe education for the students.

If the school is inadequate or even close to it that should never be a surprise to the people working in it. The expectations should have been clear enough that anyone can self-asses themselves. There should be no big surprises during the inspection. That is neither “less” nor “more” oversight, but more predictable oversight.

Furthermore the headteacher was in her position for 13 years. If the problems found were indeed so outrageous to rate the whole school inadequate then shouldn’t she have been inspected earlier and more often? During those 13 years multiple cohorts of students entered the school, studied and then graduated from it. The oversight report alleges during the whole time they attended a school with serious safeguarding issues. Isn’t that really bad? How did the oversight system let that happen? This is in fact asking for not less oversight but for more frequent one.

On the other hand the school managed to turn around in a few short months and become rated “good” from “inadequate”. It is alleged the problems actually were not deep and they were rectified within weeks after the inspection. There were some discrepancies with record keeping and a few protocols needed tightening up. The common understanding is that Ruth would have lost her job when the “inadequate” inspection report is published. Is that the best way to achieve our goal of high quality and safe education system? Wouldn’t it be better if the oversight system gave them an advance warning and time to improve? Absolutely no feedback for 13 years and then suddenly feedback with the most severe consequences is not a way to ensure good outcomes.

> teachers need LESS oversight?

They absolutely do. In fact they're going on strike in the UK over intolerable working conditions, and while the MSM report this as ostensibly about "pay" the deeper reason is that they are micro-managed to hell.

Yes. They need less oversight, more training (particularly in tech and tech-adjacent issues) and more power over curriculum.

> more training (particularly in tech and tech-adjacent issues) and more power over curriculum.

Yes yes!

They are crying out for this too. I was involved in an international conference a few weeks ago on attachment theory and education and all the UK and US teachers said that they want more training to become more self-sufficient in IT. They want more autonomy over elements of the curriculum that use tech.

They are also uncomfortable with using Big-Tech products in schools and concerned for the children over privacy.

Unfortunately I can't post the video b/c rights/confidentiality but if I am ever able I will do so.

They're state employees. That's not how things work in the public sector.

I mean we all get OKRs, objectives etc but government employees are a special case.

Most UK teachers aren't state employees any more: they're employed by academy trusts.

Semantics. All the funding comes from the state.

It is really not semantics, because of the amount of control that academies have over the work the teachers do, and because of their freedom to hire and fire teachers.

Do they have freedom over how much they pay the teachers?

Yes, academies are not bound to the “Blue Book”, and can pay teachers whatever they like under whatever conditions they like (with the exceptions of teachers TUPEd in).

Both the Post-Office/Horizon and OFSTED scandals involve blind deference to unfit technology that have led to deaths. We have been covering these on cybershow.uk with three hour long episodes in the pipeline for release soon. If any HN readers are in the UK and worked for Post Office, OFSTED or at a school and want to contribute insights do please get in touch.

>>much maligned

Yes, by teachers and their unions. Parents generally like it and choose schools based on its ratings which are generally credible.

Parents on their first kid have no idea what dealing with a school system really is like. They do on the second kid, but who goes there these days. And even on the second, you may want to use the same school as the first's because you can't afford to drive kids to 2 different schools every morning.

Edit: to make it more clear, parents may seem to like this ofsted thing at a point when they don't have any other source and they don't know better.

Liked by parents before their kids get into the school, often detested after publishing a report that doesn’t match their own experience. I’ve personally seen them make weird recommendations that damaged the school.

Until it hits close to home and people know better.

For instance, in the case of that headteacher, I personally know that her school was one of the most highly regarded, with the best results, in town. The headteacher even acted as a consultant to other schools and had been at that school for about 15 years.

Then one day, and one day only, Ofsted comes and decrees that the school is OK but that the "leadership" is poor, with tragic consequences. The outpouring of support from parents was huge and anger towards Ofsted very real.

Do they? As a parent I consider OFSTED a low quality signal. If you’re reducing a school to a single word, I don’t think you’re taking the job very seriously. I wonder how others feel.

You'd chose an "inadequate" school over an "outstanding" one?

Do carry on putting your politics over your children's future.

When choosing a school we talked to the headmaster, pupils, teachers, and current parents. That’s far more rich than an inspector’s rushed visit.

Not sure what you mean by politics.

I am not a parent.

If I were a parent, I would assume Ofsted scores to be useless. Why? Because I know teachers and what they experience about the Ofsted process, which is Kafkaesque and officious.

It is much easier to look at the exam results, visit the school, ask people in an area what the kids are like at the bus stop —- are they well-behaved or are they scary? —- and ask young adults who recently left school what the teachers are like.

Haha. "Hi kids. Are your teachers nice to you?" That's a good way to get put on a register.

But assuming the kids were nice, yet the school was rated "inadequate" - you would choose that over a school rated "outstanding"? I can tell you're not a parent.

Apart from your (judging by your comment history apparently quite typical) insulting snide bullshit reading of my comment, for which go fuck yourself:

Yes. I know of a school locally to me that is rated inadequate that the parents adore, that I would choose over the next nearest independent day school which turns out weird aloof little Stepford children.

As for the snarky intentional bullshit misreading: “recently left school” in the UK means “completed school” as well, as in “recently left school for university” or “recently left school for college”. Engage your brain when I am talking about young adults and it should be clear I was talking about school leavers. They are the right people to ask if a school is good or not, because they are the ones most recently impacted by its actual performance.

By the way, well done for being able to tell I am not a parent. Perhaps you deduced that from the opening sentence where I said “I am not a parent”? That was a tricky one, so clever you :-)

I think it is easy to claim you would make an position in line with your ideology when you don't have skin in the game.

> The coroner said even when she attended hospital feeling suicidal, Mrs Perry felt she couldn't discuss the report with mental health professionals.

Holy shit, how did THAT happen ?!?

> Two key sources of stress for Ruth Perry, according to the evidence, were the long wait for the report to be published, and the strict confidentiality warning that came with the draft report.

https://www.bbc.co.uk/news/education-67639943

Yes, I did read that, but what happened to patient-doctor confidentiality, it's not like her name was Edward Snowden ?!?

I really wonder what was in that handbook...

> Holy shit, how did THAT happen ?!?

Because OFSTED have been hounding critics, compiling blacklists, sabotaging careers and attempting to have conference speakers critical of them deplatformed.

I'll have a dig and see if I can come back with some reference on that.

best I can find for now [0]

EDIT: added link

[0] https://www.theguardian.com/education/2023/sep/30/revealed-u...

If getting grades is too much for adults, imagine what it does to children.

Suicide among young people over grades and exams isn't exactly rare[1], but only when a teacher collapses under that same pressure we need to have a talk? About grading the adults?

What a strange world we live in.

[1] https://www.theguardian.com/society/2017/jul/13/suicides-by-...

How often does their IT fail? Outsourced to Crapita or Fooljitsu by any chance?

With Google Docs/Office 365 the last time I lost a full document is years ago now.

I am old enough to have stored my laptop in the fridge when writing my PhD thesis, to keep the fault rate down. However, that is when hard drives where still mechanical, and online backups not a common thing.

I don't know, but offstead things might contain "incidents" with child names and whatnot. Google Docs may not comply with the standards needed. I think the bigger clouds have the security levels, but there may be costs involved. Don't know basically but it might be a deal breaker.

Yes, it is very likely Fooljitsu and Crapita consultants manage to convince them that existing systems do not comply with the relevant laws, and propose an in-house system. I have worked in software consultancy myself. The problem is that the project often starts with senior Engineers, who then gradually get moved to impress new clients, with the final product being delivered by overworked junior Engineers.

Until the government gets better at overseeing such large projects, this will keep on happening. However, as governments not spending their own money, but tax payers money instead, there is little incentive.

This is where the law needs to confront with actual reality.

The reality is that data leakage, incorrect access, etc. are all much more likely with a bespoke in-house product than with google docs, even if they haven’t bothered getting the cert needed.

Yes, but I think ownership is easier to prove when the product is inhouse, regardless of what reality is, if you can point at the thing that has the data in the room next door, that is easier to show than somewhere in cyber space.

For that reason I think these bespoke solutions are not going anywhere for the foreseeable.

In regards to the Post Office scandal, that same ballsup could happen in any cloud.

I do not know about education, but there are rules for medical data. For example, with the NHS you have to use approved cloud providers and the medical data has to be stored in the UK.

There is also might be functionality that is not part of Google Docs in terms of metadata on notes, who information is searchable etc.

'Replacing notes from memory' is not ideal but it's far removed from 'make up evidence' used in the headline.

Depends probably on how extensive and detailed the notes are. And the big problem seems to be that they are hiding the fact that those notes were from memory.

Indeed, but memory is fine, then why take notes at all?

I think the point is that if the inspector only finds out by chance that the notes were not committed then they might have to make it up from memory in a day or week, it just isn't reliable.

Why not double account, why not have a separate (software?) dictaphone and write notes? If one is lost, there's the other.

Primary notes or journals should always be hand-written on paper or perhaps recorded, then entered into a more structured database system. This is why lab notebooks are still used in science and engineering. This is why doctors dictate their chart entries. You can always reconstruct from those.

Lab notebooks are often digital today. Of course they shouldn't get lost and all changes need to be logged. The software described here would be entirely unacceptable as a lab notebook.

Take a look at this figure[0] from "The Neuroscience of Memory: Implications for the Courtroom"

[0] https://www.ncbi.nlm.nih.gov/core/lw/2.0/html/tileshop_pmc/t...

> sometimes crashing unexpectedly and losing all notes from interviews, or even whole days of evidence, so that inspectors have to replace those notes from memory without telling the school

Let me dig out those comments said software bug is OK something like that.

I think what most comments try to point out, is that the problem is not the bug, but the institutions taking action knowing the software is buggy

For example the Post Office, both them and Fujitsu knew the bug for 20 years, they did not request it to be fixed, and in court denied the software could have any problem, while prosecuting people and closing branches. Looks more like a feature than a bug

Here is how I would inspect schools:

* Collect together a bunch of metrics of each school. Eg. student test scores, parent satisfaction scores, number of police callouts to the school, number of leaks in the school roof. Also include metrics that aren't obviously good/bad: Number of acres of playgrounds, average tenure of staff, etc.

* Gather data of the success of past students, 30-50 years on. For example, employment rate, total earnings, percentage convicted, percentage in good health.

* Build a model to predict success metrics from the school metrics.

* To rate a school, go collect the school metrics, then run through the model to predict future success metrics. That is your rating.

Sure, such an approach has the correlation/causation problem. But this is self-correcting if schools try to optimize their scores as the models are rebuilt each year.

You’ve just made a map of rich and poor areas with extra steps.

Which is effective. If you’re a parent trying to decide which schools you want your kids in, maps of where the money is and maps of school rankings are damn near interchangeable (mostly not for funding-related reasons, though). You could use either and come to similar conclusions.

All these metrics are easily manipulated. Test scores can be manipulated by giving easier tests. Measuring number of police call-outs just encourages not calling police on emergencies.

Sure, but assuming everyone puts similar amounts of efforts into manipulating the metrics, the model will come to reflect the fact that those metrics are no longer predictive, and there will no longer be a benefit to manipulating them.

Thats the benefit of this scheme - it isn't statistically sound, but all the statistical shortcomings eliminate themselves as people try to exploit them.

The OFSTED electronic evidence gathering system seems to have been developed by (or in collaboration with) an outfit called Rainmaker Solutions. They seem to target government agencies.

I wanted to see if they are in any way related to Fujitsu; so I visited their "website". But it's not a proper website; instead, it's a demo page for their gee-whizz designers. The navigation is at best excentric. For no good reason, they've replaced the mouse cursor with a blob and a caret. They've tinkered with the scrollbars. There's no About page, which might have told me who owns them, or who their partners are. It's pure puff, an information-free zone.

https://rainmaker.solutions/

If I wanted a useable, practical solution, I'd run a mile from the designers of that website.

Correction - there is an About page. It’s completely useless and tells you nothing about the company other than they would like to call their team ‘kick-ass multi-disciplinary problem solvers’. Highly trustworthy stuff.

Also

> Creative Disruptors.

Really?

Yeah, you don't generally want "Creative Disruptors" developing your government IT systems.

Pretty sure "England making up evidence" is behind half of America's constitution.