Ofsted inspectors 'make up evidence' about a school's performance when IT fails

https://www.theguardian.com/education/2024/feb/03/ofsted-inspectors-make-up-evidence-about-a-schools-performance-when-it-fails

by YeGoblynQueenne • 2 years ago

130 80 2 years ago

theguardian.com

DicIfTEx• 2 years ago

For context, Ofsted is the much-maligned government inspectorate of schools in England. They are currently under increased scrutiny following an inquest into the suicide of a headteacher following an inspection that concluded at the end of last year: https://www.bbc.co.uk/news/education-67639942 and https://www.bbc.com/news/education-67639943

oogali3• 2 years ago

I hope people think more about cases like this and the Post Office when they go to say “it’s not like we’re curing cancer” or “my bad code isn’t a matter of life or death”.

You might be far removed from the implications of doing a poor job but that doesn’t mean everyone else is.

tailspin20192• 2 years ago

While I do agree in principle, I think it's also the case that "the software" has been used as a convenient scapegoat and cover for what were incredibly poor practices and dishonest, irresponsible conduct.

The fact that software is involved is in some ways a side issue - and while I do agree we as techies do need to consider the responsibility that we have, I'm equally frustrated how the narrative in the Post Office case has been deliberately shifted towards "it was because the software had bugs" as opposed to "multiple people in charge were deliberately deceitful and behaved completely reprehensibly".

This is borne out by the UK government's attempted shift of all the responsibility on to Fujitsu in the Post Office case (who are absolutely not blameless) while trying to push attention away from the Post Office's - and by extension the government's - own considerable culpability.

I'm sure they'll seek to do the same in this Ofsted situation.

IshKebab• 2 years ago

While it's obviously better for bugs to not happen, the issue here is that both Ofsted and the Post Office covered up the existence of the bugs, lied about it and didn't fix them.

If they'd been like "oh, a bug. We'll fix it" then there would be no issue.

pjc50• 2 years ago

In this case, Ofsted are playing the role of the Post Office prosecutors making career-limiting decisions based on inadequate or forged evidence.

andy_ppp• 2 years ago

The code is usually a reflection of the organisation, not the individual. So while we should all try to write as few bugs as possible a poorly run team can cause at least as many software defects as poor individuals.

cyanydeez• 2 years ago

budgets are a value statement.

just cause you're an ant here doesn't make you ethically culpable.

iovrthoughtthis• 2 years ago

this isn't individual devs fault

this is bad practice by service integrators

vdaea2• 2 years ago

What is that first link saying? That teachers need LESS oversight? This must be a joke.

Also what does software have to do with that case?

lkdfjlkdfjlg• 2 years ago

What is that first link saying? That teachers need LESS oversight? This must be a joke.

Are you suggesting this is a binary decision, where the only options is choosing "more oversight" or "less oversight"?

strken• 2 years ago

It's probably suggesting that oversight which is made public as a one word rating like "inadequate" is not a good way to fix problems in a school, since the exact same experienced teachers and principals you'd need to turn a school around are the most likely to avoid an "inadequate" school.

Presumably this feels even more unfair if the computer eats the inspectors' homework.

rcxdude• 2 years ago

Could well be that the kind of oversight that teachers are getting in the current system is counterproductive to the stated goals of said oversight.

tailspin20192• 2 years ago

What is that first link saying? That teachers need LESS oversight? This must be a joke.

Here's what it's saying, literally:

An Ofsted inspection "contributed" to the death of head teacher Ruth Perry, an inquest has ruled.
The inspection "lacked fairness, respect and sensitivity" and was at times "rude and intimidating", senior coroner Heidi Connor said.
Mrs Perry, 53, took her own life in January while waiting for an Ofsted report to be published.

Seems entirely uncontroversial for them to review their practices and nowhere in there is it saying that teachers need less oversight.

nonrandomstring4• 2 years ago

teachers need LESS oversight?

They absolutely do. In fact they're going on strike in the UK over intolerable working conditions, and while the MSM report this as ostensibly about "pay" the deeper reason is that they are micro-managed to hell.

gadders7• 2 years ago

They're state employees. That's not how things work in the public sector.

I mean we all get OKRs, objectives etc but government employees are a special case.

nonrandomstring4• 2 years ago

Both the Post-Office/Horizon and OFSTED scandals involve blind deference to unfit technology that have led to deaths. We have been covering these on cybershow.uk with three hour long episodes in the pipeline for release soon. If any HN readers are in the UK and worked for Post Office, OFSTED or at a school and want to contribute insights do please get in touch.

gadders7• 2 years ago

>much maligned

Yes, by teachers and their unions. Parents generally like it and choose schools based on its ratings which are generally credible.

blowski• 2 years ago

Liked by parents before their kids get into the school, often detested after publishing a report that doesn’t match their own experience. I’ve personally seen them make weird recommendations that damaged the school.

mytailorisrich• 2 years ago

Until it hits close to home and people know better.

For instance, in the case of that headteacher, I personally know that her school was one of the most highly regarded, with the best results, in town. The headteacher even acted as a consultant to other schools and had been at that school for about 15 years.

Then one day, and one day only, Ofsted comes and decrees that the school is OK but that the "leadership" is poor, with tragic consequences. The outpouring of support from parents was huge and anger towards Ofsted very real.

nottorp• 2 years ago

Parents on their first kid have no idea what dealing with a school system really is like. They do on the second kid, but who goes there these days. And even on the second, you may want to use the same school as the first's because you can't afford to drive kids to 2 different schools every morning.

Edit: to make it more clear, parents may seem to like this ofsted thing at a point when they don't have any other source and they don't know better.

afandian3• 2 years ago

Do they? As a parent I consider OFSTED a low quality signal. If you’re reducing a school to a single word, I don’t think you’re taking the job very seriously. I wonder how others feel.

bemusedthrow753• 2 years ago

I am not a parent.

If I were a parent, I would assume Ofsted scores to be useless. Why? Because I know teachers and what they experience about the Ofsted process, which is Kafkaesque and officious.

It is much easier to look at the exam results, visit the school, ask people in an area what the kids are like at the bus stop —- are they well-behaved or are they scary? —- and ask young adults who recently left school what the teachers are like.

BlueTemplar3• 2 years ago

The coroner said even when she attended hospital feeling suicidal, Mrs Perry felt she couldn't discuss the report with mental health professionals.

Holy shit, how did THAT happen ?!?

afandian3• 2 years ago

Two key sources of stress for Ruth Perry, according to the evidence, were the long wait for the report to be published, and the strict confidentiality warning that came with the draft report.

https://www.bbc.co.uk/news/education-67639943

nonrandomstring4• 2 years ago

Holy shit, how did THAT happen ?!?

Because OFSTED have been hounding critics, compiling blacklists, sabotaging careers and attempting to have conference speakers critical of them deplatformed.

I'll have a dig and see if I can come back with some reference on that.

best I can find for now [0]

EDIT: added link

[0] https://www.theguardian.com/education/2023/sep/30/revealed-u...

chmod775• 2 years ago

If getting grades is too much for adults, imagine what it does to children.

Suicide among young people over grades and exams isn't exactly rare[1], but only when a teacher collapses under that same pressure we need to have a talk? About grading the adults?

What a strange world we live in.

[1] https://www.theguardian.com/society/2017/jul/13/suicides-by-...

akasakahakada• 2 years ago

sometimes crashing unexpectedly and losing all notes from interviews, or even whole days of evidence, so that inspectors have to replace those notes from memory without telling the school

Let me dig out those comments said software bug is OK something like that.

guilhas• 2 years ago

I think what most comments try to point out, is that the problem is not the bug, but the institutions taking action knowing the software is buggy

For example the Post Office, both them and Fujitsu knew the bug for 20 years, they did not request it to be fixed, and in court denied the software could have any problem, while prosecuting people and closing branches. Looks more like a feature than a bug

dazc2• 2 years ago

'Replacing notes from memory' is not ideal but it's far removed from 'make up evidence' used in the headline.

fabian2k2• 2 years ago

Depends probably on how extensive and detailed the notes are. And the big problem seems to be that they are hiding the fact that those notes were from memory.

cowpig• 2 years ago

Take a look at this figure[0] from "The Neuroscience of Memory: Implications for the Courtroom"

[0] https://www.ncbi.nlm.nih.gov/core/lw/2.0/html/tileshop_pmc/t...

6c696e75783• 2 years ago

Indeed, but memory is fine, then why take notes at all?

I think the point is that if the inspector only finds out by chance that the notes were not committed then they might have to make it up from memory in a day or week, it just isn't reliable.

Why not double account, why not have a separate (software?) dictaphone and write notes? If one is lost, there's the other.

SoftTalker• 2 years ago

Primary notes or journals should always be hand-written on paper or perhaps recorded, then entered into a more structured database system. This is why lab notebooks are still used in science and engineering. This is why doctors dictate their chart entries. You can always reconstruct from those.

oxfordmale2• 2 years ago

How often does their IT fail? Outsourced to Crapita or Fooljitsu by any chance?

With Google Docs/Office 365 the last time I lost a full document is years ago now.

I am old enough to have stored my laptop in the fridge when writing my PhD thesis, to keep the fault rate down. However, that is when hard drives where still mechanical, and online backups not a common thing.

6c696e75783• 2 years ago

I don't know, but offstead things might contain "incidents" with child names and whatnot. Google Docs may not comply with the standards needed. I think the bigger clouds have the security levels, but there may be costs involved. Don't know basically but it might be a deal breaker.

oxfordmale2• 2 years ago

Yes, it is very likely Fooljitsu and Crapita consultants manage to convince them that existing systems do not comply with the relevant laws, and propose an in-house system. I have worked in software consultancy myself. The problem is that the project often starts with senior Engineers, who then gradually get moved to impress new clients, with the final product being delivered by overworked junior Engineers.

Until the government gets better at overseeing such large projects, this will keep on happening. However, as governments not spending their own money, but tax payers money instead, there is little incentive.

whimsicalism• 2 years ago

This is where the law needs to confront with actual reality.

The reality is that data leakage, incorrect access, etc. are all much more likely with a bespoke in-house product than with google docs, even if they haven’t bothered getting the cert needed.

graemep2• 2 years ago

I do not know about education, but there are rules for medical data. For example, with the NHS you have to use approved cloud providers and the medical data has to be stored in the UK.

There is also might be functionality that is not part of Google Docs in terms of metadata on notes, who information is searchable etc.

londons_explore2• 2 years ago

Here is how I would inspect schools:

* Collect together a bunch of metrics of each school. Eg. student test scores, parent satisfaction scores, number of police callouts to the school, number of leaks in the school roof. Also include metrics that aren't obviously good/bad: Number of acres of playgrounds, average tenure of staff, etc.

* Gather data of the success of past students, 30-50 years on. For example, employment rate, total earnings, percentage convicted, percentage in good health.

* Build a model to predict success metrics from the school metrics.

* To rate a school, go collect the school metrics, then run through the model to predict future success metrics. That is your rating.

Sure, such an approach has the correlation/causation problem. But this is self-correcting if schools try to optimize their scores as the models are rebuilt each year.

charlieyu1• 2 years ago

All these metrics are easily manipulated. Test scores can be manipulated by giving easier tests. Measuring number of police call-outs just encourages not calling police on emergencies.

oogali3• 2 years ago

https://en.wikipedia.org/wiki/Goodhart%27s_law#Generalizatio...

londons_explore2• 2 years ago

Sure, but assuming everyone puts similar amounts of efforts into manipulating the metrics, the model will come to reflect the fact that those metrics are no longer predictive, and there will no longer be a benefit to manipulating them.

Thats the benefit of this scheme - it isn't statistically sound, but all the statistical shortcomings eliminate themselves as people try to exploit them.

wharvle• 2 years ago

You’ve just made a map of rich and poor areas with extra steps.

Which is effective. If you’re a parent trying to decide which schools you want your kids in, maps of where the money is and maps of school rankings are damn near interchangeable (mostly not for funding-related reasons, though). You could use either and come to similar conclusions.

denton-scratch2• 2 years ago

The OFSTED electronic evidence gathering system seems to have been developed by (or in collaboration with) an outfit called Rainmaker Solutions. They seem to target government agencies.

I wanted to see if they are in any way related to Fujitsu; so I visited their "website". But it's not a proper website; instead, it's a demo page for their gee-whizz designers. The navigation is at best excentric. For no good reason, they've replaced the mouse cursor with a blob and a caret. They've tinkered with the scrollbars. There's no About page, which might have told me who owns them, or who their partners are. It's pure puff, an information-free zone.

https://rainmaker.solutions/

If I wanted a useable, practical solution, I'd run a mile from the designers of that website.

jpsouth• 2 years ago

Correction - there is an About page. It’s completely useless and tells you nothing about the company other than they would like to call their team ‘kick-ass multi-disciplinary problem solvers’. Highly trustworthy stuff.

Also

Creative Disruptors.

Really?

denton-scratch2• 2 years ago

Yeah, you don't generally want "Creative Disruptors" developing your government IT systems.

u32480932048• 2 years ago

Pretty sure "England making up evidence" is behind half of America's constitution.

Ofsted inspectors 'make up evidence' about a school's performance when IT fails

Related Stories