How a software glitch at the UK Post Office ruined lives

https://www.cnn.com/2024/01/13/business/uk-post-office-fujitsu-horizon-scandal/index.html

by IronWolve • 3 years ago

385 331 3 years ago

IronWolve4• 3 years ago

Wouldn't say it was just a software glitch, bureaucracy tried to cover it for bad press at all levels in the government ran agency.

calpaterson• 3 years ago

Indeed. And it was a number of different bugs, not just one.

iforgotpassword3• 3 years ago

That's why I don't really (knowingly) want to work on something that could ruin lives if there's a bug. medical devices? Hell no. (But you never know where your code ends up running.)

As you said this didn't just involve software issues to get that big, but I still wonder how all the devs that worked on this over the years could just shrug it off all these years. You can't tell me not a single dev got wind of these issues. Did they just immediately jump to "impossible I'm the most awesome person on this planet no way there's a problem these people all just stole money". I'd have sleepless nights going over the whole system in my mind trying to figure out where things went wrong. Is this some sort of dunning -kruger going on? I don't really consider myself a coding wizard or anything, but I think I know about enough to know I'll keep producing bugs til the day I die. Rust won't stop logic bugs from happening.

bigfudge• 3 years ago

Perhaps your humility qualifies you for it! It strikes me that the problems aren’t caused by lack of technical chops per se - it’s almost always because people don’t consider the possible downsides of mistakes and act accordingly.

guappa4• 3 years ago

Well it was basically some banking software… I'm following a BBC podcast about the story, and basically the managers and investigators at the post office knew there were issues.

They told every single victim "you're the only one who's claiming there are issues".

ivix• 3 years ago

It's not an government agency. The post office is a completely private company.

switch0072• 3 years ago

Not any old private company. The government just happens to own all the shares

"The Post Office is owned by the government, through the Department for Business, Energy and Industrial Strategy (BEIS) and UK Government Investments (UKGI), however, the Post Office Ltd Board has responsibility for the operations of the Post Office" [0]

And "The Department of Business Energy and Industrial Strategy holds government responsibility for postal affairs, including the Post Office" ... "the Post Office Ltd remains accountable to the government" [0]

[0] https://researchbriefings.files.parliament.uk/documents/CBP-...

PakG113• 3 years ago

This is a new summary of the real-life case that inspired me to go do PhD studies in information systems. I wanted to understand why IT incompetence still could exist to this degree in this day and age after all the knowledge the world had developed about good IT and software practices over decades of experience. Of course, I quickly found out that IS research had already figured most of this out, and that perhaps, people were just people and crappy organizations were just crappy organizations, and perhaps that's something that will never change because bell curve distributions exist for almost everything.

If all people could be selfless, humble, intelligent, competent at everything they ever try to do, AND be people of integrity, I'm sure many of the world's problems would disappear, in addition to just IT problems. People suck, organizations suck, societies suck, and we all suck in our own way. I think organizational research like I'm trying to do still tries to point us towards a better path nevertheless, but sometimes it's a throw your hands in the air thing. Some people will never care.

Needless to say, I'm no longer focused on researching this topic, as it seems really well-researched already. But it's still interesting to see that this particular example still pops up in a news report now and then. There are still plenty of other big examples that pop up every year, but this one seems to have staying power to stay in the news media.

IronWolve4• 3 years ago

I did a quick look around for some blogs about this early paper to digital transaction register migration, I didn't see much for such a major case.

Just a few basic things that wasn't included, no audit/transaction logs, transactions modified by tech support to keep the system running.

Operators couldn't prove they didn't steal funds, and the british law that computers systems are to be trusted as fact, pretty much convicted them all.

imtringued3• 3 years ago

This isn't a software error and it is pretty clear.

There are two problems here. First, the branch manager is responsible for calculated shortfalls, even if the software is broken. Second, there is no way to overturn broken software. Third, the prosecutors are overzealous in trying to shut these people up and convict them straight away.

The software itself was just a convenient medium for abuse of authority.

porker• 3 years ago

I quickly found out that IS research had already figured most of this out

What is "IS" in this context? I did some Operations Research modules at uni and thoroughly enjoyed it, but it had nothing to say about why projects didn't work.

onei3• 3 years ago

Information systems at a guess

onei3• 3 years ago

Were there any particular themes that stood out as you when understanding the causes of IT incompetence? I'm hoping there's a less depressing answer than "some people will never care".

PakG113• 3 years ago

I'd say that the biggest thing that stood out to me was what people call communities of practice. Each professional community has their own knowledge and best practices, but these don't generalize well to other professions. So you have boundary spanners who can bridge two professional communities by being good at both, but those types of experts are rare. Also, the amount of experience and learning required to become an expert in two professional communities, rather than just one, requires such a large amount of time that most people can't be bothered to put in the effort.

It's enough work to do one's own job well already. The go getters can of course do it as a natural course of action, but they are outliers. There are a limited number of job opportunities that require developing this experience on the job, so there are limited opportunities to become a good boundary spanner in the first place. Furthermore, people aren't naturally interested in multiple disparate subjects. True renaissance folks like Leonardo da Vinci who are interested in becoming experts in both art and engineering are rare. Elon Musk types that will try to dive deep into multiple unrelated areas are rare. All of this adds up to boundary spanners being rare. As such, leaders who can develop cultures that handle multiple areas simultaneously (see the founders of Flexport who understand both tech and shipping logistics) are rare.

In short, expertise is hard to develop, expertise in multiple areas is rare, and coordination between two areas that understand different worlds is difficult without boundary spanners. As a result, you get failures. See any software engineer who creates a startup to try to revolutionize some old-school industry and then fails dramatically because they don't understand the problems that actually need to be solved. The outliers will figure it out, but not everyone can become an outlier due to reasons discussed, among other reasons.

kortilla7• 3 years ago

This is not well researched at all. Anything that depends on “if everyone acted intelligently and good will” is broken. If you’re interested in academia I strongly recommend you go back and look at designing systems that function in the face of incompetence and even adversaries.

PakG113• 3 years ago

You can have your opinion, but I have mine after reading lots of research papers. Obviously, there are ways that things can still be improved. But even the design of systems that function in the face of incompetence and even adversaries will face intractable problems of incompetent leadership and governance. It's one thing to say "this is how it should be done" and it's another to say that we've managed to get people to do it the way it should be done. No matter how much improvement we see on ideas for how it should be done, we're still no closer to solving the second part, despite much effort. In the end, there will always be incompetent leaders in charge at some point for some project somewhere, and more often than we'd prefer. You can lead a horse to water, you can't make it drink.

Joeboy2• 3 years ago

Are there software dev processes that operate well in the face of incompetent or malicious people? I can think of ways to mitigate the damage, but at the end of the day surely you need some competent and conscientious people on your project.

chii• 3 years ago

I quickly found out that IS research had already figured most of this out People suck, organizations suck, societies suck

and yet you don't see very many buildings collapse, bridges fail, and damns flood down valley. The idea is that there should be liability assigned to important systems, for which this liability makes the onus on the creator/owner to build in safe guards, checks or other protections to prevent disasters.

Why this isn't applied to software engineering is a whole nother story, but i think it probably should. Move fast and break things is not something i wanted to hear tbh.

saurik• 3 years ago

Even Facebook realized it was dumb and Mark Zuckerberg apologized to us all on stage at f8 a decade or so ago and said they were now going to say "move fast with stable infra", yet I still see tons of people fetishize and emulate the original.

stevefan1999• 3 years ago

and yet you don't see very many buildings collapse, bridges fail, and damns flood down valley

Computers, at its best existed for only 200 years. That's counting from the difference engine that Babbage never finished building, and if we are talking about the first modern computer that does contribute significantly in a way, that's around the time ENVAC and Z3 appeared, just around 80 years.

Structural Engineering existed for over 2000 years. With that comes with a lot of people died in natural disasters and advancing material science to improve structural integrity. But even the Japanese to this day still can't solve the earthquake and the problems the fallout makes.

Why this isn't applied to software engineering is a whole nother story

I actually think it is. That's why real-time operating system and mission-critical hardware exists for human to fly in the air and explore the space, not counting a lot of time-sensitive industry robotics software that controls the actuators in real time as well.

That said, the strict requirements of real-time programming requires a lot of expertise such as CPU cycle counting, CPU slack reduction, timing requirements and choice of algorithm (such as EDF scheduling and real time memory allocators like TLSF, and you won't see jemalloc on embedded devices, right?), stack or heap memory allocation which also complicates the programming stuff, because removing malloc might free you from OOM but this means a lot of functions needs to accept extra parameters to state the output location, and that means a lot of normal software can't be used. (You can go for a hybrid approach by using arena allocation, but that still isn't a perfect solution)

As you see, even for soft real time engineering (which I actually mentioned so far, I don't know much about the real hardcore "hard real time engineering" though), the sheer complexity here already, means there's a lot of design decisions which simply just makes people stay away and just go for normal software engineering (but in the end, if everything is predicable its fine).

Move fast and break things is not something i wanted to hear tbh.

When Zuckerberg said that, he's referring to his startup mindset when Facebook was really just a startup in his day. In a highly competitive market environment, startups has to fight desperately for their own survival, even with lots of fundings and VC rounds.

Startup can move very fast and their agility is the only weapon against the old dogs. Now you've become one of the old dogs, you don't break things.

PakG113• 3 years ago

There is something about how buildings, bridges, and damns are tangible and therefore easier for people to conceptualize and appreciate. There is a natural literacy there in that you don't have to understand what goes into making a bridge to understand the consequences of having a bad bridge.

Meanwhile, software's inner workings are encapsulated so that they're not self-evident as to consequences if things go awry. Furthermore, software is inconsistent in how it works, and this trains people to think that software is just naturally glitchy, but the glitches aren't a big deal. See the joke about different types of engineers also: https://www.reddit.com/r/Jokes/comments/pqr8t3/four_engineer...

The mechanical engineer says: “It’s a broken starter”

The electrical engineer says: “Dead battery”

The chemical engineer says: “Impurities in the gasoline”

The IT engineer says: “Hey guys, I have an idea, how about we all get out of the car and get back in”

If that's what the IT guy recommends, how do you get users, never mind corporate managers and executives, to take software system quality seriously? Obviously, this isn't what a proper software guy will think, the proper software guy knows that things are a bit more complex than that. But this is what the software guy communicates to non-technical people. And only a few rare non-technical people will tell the software guy, "OK, look, just tell me what's really going on here, how deep the problem is, and what we need to do to fix it, no matter what it takes." Most people won't have the time for that because fact of the matter is that the system satisfices needs until it doesn't, and then it's too late.

charcircuit• 3 years ago

It's easier for software to have a catastrophic failure than a building, bridge, or dam. It's not really a fair comparison when those are like easy mode compared to software.

spacechild13• 3 years ago

For me the biggest joke is that these people call themselves "software engineers" in the first place.

thom3• 3 years ago

This is a British scandal, and we’re quite capable of avoiding consequences for malfeasance in every sector of our economy, not just IT thank you very much.

cornel_io• 3 years ago

This isn't a story about software engineering, it's a story about a shitty governmental department in a mediocre country not fixing something that everyone knows is broken, and threatening people with jail time instead.

Show me a similar story about stuff this bad happening at Google and I'll believe that there's something wrong with the field as practiced at its best.

arcade79• 3 years ago

and yet you don't see very many buildings collapse, bridges fail, and damns flood down valley.

Hi! Let me tell you about my wonderful country. Norway! I'm sure you've heard about it. The socialist utopia in Scandinavia! ;-)

The Braskereidfoss dam failed during a flood last fall. Luckily just a 'river dam' and not a reservoir dam. Still rather bad.

Oh, and during the same freak whether event, but different place in the country. The Randklev railway bridge failed over the river Lågen, just by Ringebu.

And during the last 8 years, we've had two road bridges collapse. One by Sjoa in 2016 (Perkolo bridge) and then 16 months ago Tretten bridge. We had the fantastic idea of building wooden bridge sfor road traffic. Here's a nice article on Tretten bridge: https://en.wikipedia.org/wiki/Tretten_Bridge

But yeah, not too many building collapses . Just a bunch of infrastructure collapse.

where-group-by2• 3 years ago

When a dam or a bridge breaks it is game over.

When software breaks you restart and it's back to "working" order. It would be a different scenario if the computer would set itself on fire on failure.

ivanhoe2• 3 years ago

Move fast and break things is not something i wanted to hear tbh.

Moving safely means moving slow, and that costs more - it's as simple as that. Plus it would also automatically disqualify like 80% of the workforce, due to the need for a proper formal education and certification, which would literally paralyze the industry.

And the current solution is not that different from what we have e.g. in civil engineering, the building codes for high-rise buildings are way more strict and complex than the regulations for a garden shed - in many places no permit is needed at all for it. And sometimes that means that that shad will fall in a catastrophic way, too.

fzeindl3• 3 years ago

Why this isn't applied to software engineering is a whole nother story, but i think it probably should.

Because bits don‘t rot and while you need to build a new bridge now and then which will include your learnings from the old one, you never have to replace software because it can be copied ad infinitum.

phatfish• 3 years ago

I think one of the issues is that software engineers are building on sand. Civil engineers have Newton and Einstein to fall back on.

It feels like there is no ground truth in computing, each layer has to assume the layer below is generating random errors and try to account for that, because humans are not perfect.

The laws of physics stay the same, technology is constantly changing.

jimnotgym6• 3 years ago

and yet you don't see very many buildings collapse, bridges fail, and damns flood down valley.

Because those engineers are required to have rigorous qualifications. Please see my comment linked below

https://news.ycombinator.com/item?id=39014625

berniedurfee• 3 years ago

I always figure the major difference between software engineering and engineering physical things, is that software is virtual and fluid.

Once a physical thing is built or being manufactured, the cost of change is exponentially higher than with software.

So software engineers can experiment more frequently and can get away with low quality, as it can always be fixed later.

Of course, this same mentality can carry into industries where the stakes are life and death, which is where accountability is very much lacking.

xlii• 3 years ago

I’ll bite.

It’s that way because software is much much much more complex than that.

Building is a good example. One architect after university has enough knowledge and mental capability to design a building that won’t topple: There are discrete number of parameters to work on and fallible elements are identified and understood very well. There’s manageable amount of degrees of freedom that architect is working with. It’s also not very common for physics to get an annual update.

But when you go to funny-cat-pictures.com there are layers upon layers of complexity. There’s TCP, UDP, traffic encryption, indefinite number of hoops in routing, indefinite number of electronic devices with various quality, etc. Request comes happens from millions lines of code operating system and generation happens on a different millions lines of code of operating system with thousands of lines of codes of software code that interacts with databases, proxies, CDNs, load balancers and is templating, serving, translating, transpiling, compiling and serving while defending against adverse attacks, managing the cache, optimizing for network topology and requester rendering software. That changes every single day slightly.

In 99% the worst case is - your picture won’t load. But when you get to the more serious software development (in a way that it’s critical domain, not that “serve cat pictures” isn’t serious job, mind you) all of it is very very visible.

Every element is so unimaginably complex that it has literal tomes of knowledge written and published about it. Describing every element of transaction between end user and cat serving website would take hundreds of pages - and it would be different than other cat website.

Most of the building, bridges etc. was made either by one person or small group of people working on it.

Rarely software is written seldom, and many well known products have tens of thousands of inconceivable smart engineers and we still can make fun of their - not so rare - failures.

So yeah. Systems suck, organizations suck, people suck. But they suck in relatively safe but also complex environment.

For years software engineering was made fun of because it’s not real engineering. But when you look close enough the environment is orders of magnitude more difficult. Civil engineers might be offended but they can’t hold a candle to structures software is keeping straight.

Software is a bridge built on a raft, floating on an ocean, which tries to get shot down by armed pirates and semi-controlled by a cost-saving manager trying to look good in annual progress report.

scotty79• 3 years ago

and yet you don't see very many buildings collapse, bridges fail, and damns flood down valley

Maybe that's survivor bias? Crappy organizations fail at infrastructure projects before they even get off the ground.

Conversly softer endeavors involving just money, people and services can operate for a long time on finger crossing.

deanishe• 3 years ago

Why this isn't applied to software engineering is a whole nother story, but i think it probably should.

Money, for one thing. Actually building the thing IRL is the expensive bit, but it's effectively free for software. So engineers are incentivised to get things right the first time, and software developers are not. It's much cheaper to skimp on QA, let users find your fuck-ups, and push an update.

DeathArrow• 3 years ago

The biggest issue isn't the software glitch system. It's the legal system that threatened innocent people with prison for theft unless they admit they are guilty for crimes they didn't do and pay for damages they didn't do.

This case was brought to public attention and repairs were attempted only because it's huge and involved hundreds or thousands of people.

How many disparate cases there are, where people's lives are destroyed and innocents are rotting in jails, we have to ask?

PakG113• 3 years ago

There are not as many that involve jail, but there are a variety that involve ruined lives and even bankruptcy. A recent example is the Phoenix pay system that was used to pay Canadian federal government employees and contractors. https://en.wikipedia.org/wiki/Phoenix_pay_system And I agree that the problem wasn't the glitches. I personally think it was the corporate governance that failed, not the software development and debugging process. The legal system was complicit and enlarged the overall consequences, but the but for test tells me that it was the poor corporate governance that was at fault for a root cause.

usr1106• 3 years ago

Every skillful programmer knows that all software is crap.

Judges, salesman, and managers don't understand that.

bsder2• 3 years ago

Agreed. The bug is a footnote.

The legal system failed these people horribly. And the people who pursued these cases with no direct evidence whatsoever should suffer jail time.

jrumbut• 3 years ago

The biggest issue isn't the software glitch system.

I disagree, the software glitch was the problem here.

We are supposed to be able to rely on computers to store and add numbers or report a system failure. This accounting software showed in black and white that some funds that the sub-postmasters were responsible for had gone missing.

What else was the legal system supposed to do? The broken software was simulating crime perfectly.

ivanhoe2• 3 years ago

Exactly, and by the number of errors among the death-row/lifelong convictions - which one would presume are the most sensitive and carefully reviewed ones - these numbers are HUGE, especially in the US.

scott_w3• 3 years ago

It also doesn't help that the lawyers in the case fabricated evidence and covered up the issues with Horizon to secure the convictions. The people doing this need to spend some time in one of the overcrowded prison cells they sent the sub-postmasters to.

madaxe_again2• 3 years ago

people of integrity

Our current system of the world quite strongly disincentivises honesty and integrity - rather, being a bombastic charlatan with a flexible relationship with the truth will get you anywhere.

heads• 3 years ago

It would be really interesting to study The Post Office in particular. Something about this organisation attracts some very sour people. Or perhaps they weren’t always like this but have become so in my adult lifetime over the last few decades?

In the early 2000s there was a TV ad campaign for “The People’s Post Office” where the sub-postmaster role was played by John Henshaw, a character actor known for playing hard bastards and, in his most recent role on The Cops, an exploitative bent copper from Bradford. A strange but apt piece of casting.

ponector• 3 years ago

Low salary coupled with customer facing job creates sour people.

Imagine you are talking everyday to weird, bitter, arrogant, rude customers for years - even with high salary you will be not so positive.

risyachka• 3 years ago

Most of the time issues like these are from companies that pay like $50k salary for senior positions.

So everything checks out.

lijok• 3 years ago

why IT incompetence still could exist to this degree in this day and age after all the knowledge the world had developed about good IT and software practices over decades of experience

Because we’re not professionals. We don’t profess anything and do not have standards. There is no regulation for our industry and no IT association that can strike you off from practicing this craft. There is no accountability, and when there is no accountability, people naturally regress to either lazy or exciting behaviours.

andy_ppp4• 3 years ago

I could give you a huge list but mostly it is computer programmers having the specification constantly changed by management and stakeholders. Even bad software developers can eventually make the software functional, even good software developers can write bad software if the organisation is going out of its way to break everything they do.

PakG113• 3 years ago

There is definitely a sensemaking process where organizations have to figure out what it is that they really need. But I wouldn't fault organizations for that. Most startups go through the same process trying to figure out product-market fit and you don't see those startups blaming their customers for not knowing what they want.

o-o-• 3 years ago

Some people suck at management, some people suck at coding, and some people suck at self-awareness.

I can think of two things that I believe would make a difference in any LargeCorp: First, a standarized way to visualise and execute business logic that allows developers and management to reason together. (The no-code movement is on the right track in fostering a common way to interface with code). And second, a responsible editor for each piece of code.

I think a key factor is that software historically hasn't enjoyed industrialisation to the degree of hardware (or construction for that matter). I can buy a standardized CPU of millions of transistors and integrate it into a standardized motherboard with just a snap. We have managed to standardize software up to the OS level, but after that it's up to the developer and her shortcomings.

https://www.codevalley.com/ does some interesting work.

fzeindl3• 3 years ago

Don‘t think like that. Medicine is a complicated field as well, but after evidence based medicine was invented death rates were going down. It is certainly possible to find and enforce lists of best practices.

epgui• 3 years ago

The flip side of that is that despite how awesome modern medicine is, we still can't get people to get vaccinated (for free!) and put a mask on their face during an infectious disease emergency.

You will find that people just suck no matter what field you work in.

ksec2• 3 years ago

People who are selfless, humble, intelligent, competent AND be people of integrity, are never the people who win the contract for any information system though.

Of course, I quickly found out that IS research had already figured most of this out, and that perhaps, people were just people and crappy organizations were just crappy organizations, and perhaps that's something that will never change because bell curve distributions exist for almost everything.

Hence why we need to keep things simple. The human part will never change, or at least change at rate that will take many generations to improve if you are an optimist. I actually prefer things to be Hybrid rather than all-in digital.

st-keller• 3 years ago

I found the caption misleading. This article describes the dreadful ramifications of error(s) in a complex IT system. What it falls short of are details on the hows.

IronWolve4• 3 years ago

It was from the article, not sure how its misleading.

Its pretty much the biggest tech scandal in the news right now.

https://en.wikipedia.org/wiki/British_Post_Office_scandal

varispeed2• 3 years ago

Because it wasn't a software glitch. Probably such misleading headline has been chosen to help Fujitsu regain some of its share value.

omoikane• 3 years ago

Discussions on this wiki article from few days ago:

https://news.ycombinator.com/item?id=38937705

Actually the article has been posted since 2 years ago, but didn't get much discussion until recently.

https://hn.algolia.com/?q=https%3A%2F%2Fen.wikipedia.org%2Fw...

a-dub• 3 years ago

didn't terry gilliam make a movie about literal bugs in government computers ruining lives back in the early 80s?

IronWolve4• 3 years ago

Mr Bates vs The Post Office just came out on the ITV, is about the incident in the article.

trailer @ https://www.youtube.com/watch?v=zPkvYXufpAY

vidarh2• 3 years ago

Brazil. With a very unfortunate typo causing mistaken identity.

HideousKojima• 3 years ago

The most hilarious part to me was when the main character's friend (the government torturer guy) suggested that the mixup was intentionally caused by the wrongfully accused (or something along those lines?) as a way to make the government look bad.

Arn_Thor2• 3 years ago

Brazil

momentoftop• 3 years ago

"Tuttle? His name's Buttle. There must be some mistake."

"Mistake? Ha! We don't make mistakes."

Proceeds to drop ceiling plug through ceiling.

"That's bloody typical. They've gone back to metric without telling us."

onetimeuse923043• 3 years ago

Here some more information about what really went down, technically, behind the scenes: https://www.theguardian.com/uk-news/2024/jan/09/how-the-post...

"One member of the development team, David McDonnell, who had worked on the Epos system side of the project, told the inquiry that “of eight [people] in the development team, two were very good, another two were mediocre but we could work with them, and then there were probably three or four who just weren’t up to it and weren’t capable of producing professional code”."

(Just in case somebody says I am putting blame on developers) Obviously, the responsibility is firmly on management. People making code bugs should not be held responsible for other people going to prison for it.

onei3• 3 years ago

From that article (and a few others)

In fact, staff at Fujitsu, which made and operated the Horizon system, were capable of remotely accessing branch accounts, and had “unrestricted and unaudited” access to those systems, the inquiry heard.

This has always bothered me. Sure, it's possible to build APIs that audit access completely. But I can easily write code that circumvents those APIs. Code isn't like a building where the walls are impenetrable and the doors the only possible access points - we can redecorate without ever touching the door. Building in an unaudited backdoor for operators seems bad, but if you can edit the source code the backdoors are infinite.

fbdab103• 3 years ago

There should be application level auditing and database level. The people with access to managing the database level auditing should be extremely limited.

josephg9• 3 years ago

Obviously, the responsibility is firmly on management. People making code bugs should not be held responsible for other people going to prison for it.

This is a controversial opinion but I disagree, at least to a point. Managers don’t really know what we do. The only people who really understand the engineering trade offs involved are engineers. When lives are on the line as a result of our work, we shouldn’t be insulated from the consequences of our choices. That’s not good for society and ultimately not good for us. We change the world with our work. It’s healthy to understand and own the consequences of that.

The law agrees in parts. The principle of tort law is that everyone is responsible for foreseeable harm caused to your “neighbours”. Your degree of responsibility - and in turn liability - scales with how much expertise you have in the domain. An expert should have been able to foresee the harm more than a novice. The senior engineers on the team should have done better. I believe they are at fault.

(IANAL, this is not legal advice, yadda yadda)

LunaSea2• 3 years ago

Managers don’t really know what we do

That looks a lot like incompetent management.

onetimeuse923043• 3 years ago

An expert should have been able to foresee the harm more than a novice. The senior engineers on the team should have done better. I believe they are at fault.

Well, the reality is usually at these large software development agencies that senior engineers are prevented from doing what they think is right.

For example, they might have been pressed to deliver new features in an extremely inefficient system. They might have been inundated by low quality code from less experienced devs. They might have been busy with communication with stakeholders and unable to do much about it.

So singling out these developers would be like singling out couple cops for being racist when the entire Police department is known for racism. You know, technically you are right but that still does not seem to be right thing to do in case of a systemic problem.

The question would be if these developers had any way of knowing the consequences of what they were doing.

Also, "good developers" is a relative term. In an organisation like that a "good developer" might simply describe a person that is at all capable of writing working code. It does not mean they were experienced or aware what is happening around them.

It is the responsibility of managers to recognise the issues with the environment their people are working in.

sitharus• 3 years ago

I agree, but only if we have the same standing and legal protection as professional engineers.

If I’m going to be legally responsible for software bugs I must have the legal right to tell management that their timelines are not possible, and that they can’t deploy software I won’t sign off on.

That and for outsourced software the executives become personally responsible.

imtringued3• 3 years ago

So, you're telling me your manager doesn't care if the software is broken and has legal consequences for it's users?

That strategy only works if the software itself is inconsequential.

jbjohns• 3 years ago

The only one who can have responsibility for anything is the one who has the authority. So long as developers are in a position of "just get it done or you're fired" as well as being outsourced to save costs, they have no authority in this and therefor zero responsibility. If management "doesn't know what we do" and doesn't want to have the responsibility then they have to give us the authority to say "no, this is not going to be done tomorrow and we're not cutting any corners".

Kinrany• 3 years ago

I agree, except that it's important to remember that code by itself is as meaningless as programs with no side effects. A bug becomes a problem when it's a part of a running program.

paulddraper• 3 years ago

Managers don’t really know what we do.

Uh.....

The only people who really understand the engineering trade offs involved are engineers.

Maybe you should have engineering managers who understand the subject?

stavros2• 3 years ago

People making code bugs should not be held responsible for other people going to prison for it.

Why not? If I'm the single developer and seller of this app, should I not be held responsible? What if there's also a QA person? Two of each?

Should the person selling or marketing the app be held responsible instead, even if they aren't technical? Why are the developers who didn't care enough to double-check their code free of responsibility?

tpm• 3 years ago

If I'm the single developer and seller of this app

In that case you are also the responsible manager or product owner.

Should the person selling or marketing the app be held responsible instead, even if they aren't technical?

Of course. The person who takes the customers money is responsible for delivering the result and any warranty.

Why are the developers who didn't care enough to double-check their code free of responsibility?

They are not the product owners. They don't decide what is the correct way the product works. Maybe they created the bugs because they implemented the specification exactly as written?

watwut2• 3 years ago

In your hypothetical situation, I would blame the justice and banking system. It should not be so vulnerable or eager to believe an app made by one person, a self made "expert" on something, new theory etc.

Like, you as a single seller are also responsible for making false claims. But, the justice itself should be more robust then that.

mike_hearn• 3 years ago

there were probably three or four who just weren’t up to it and weren’t capable of producing professional code

See the other discussion on HN front page about coding tests in interviewing.

Horizon was a child of the 90s. The software industry has changed a lot since then. Back in those days only Microsoft was routinely requiring programmers to code during the interview, so hiring was nearly random. Software teams often looked like that, with a tiny number of people who could write working code covering for many more who just couldn't at all.

The Daily WTF dates from this time. It's full of stories like the Brillant Paula Bean:

https://thedailywtf.com/articles/The_Brillant_Paula_Bean

You don't hear stories like that much anymore. The industry settled on testing concrete skills before hiring, and that washed out a lot of the people who previously managed to get hired into projects despite not being able to code properly. Whether Fujitsu does it or not now, no clue. But that situation wasn't unusual back then.

_fat_santa• 3 years ago

I would say another child of the 90's is management / users being too trusting of technology which IMO is the real reason for this mess. These days, everyone knows that software can make mistakes and one shouldn't rely on a single system / data point when it comes to critical decisions (ie. sending someone to jail).

I don't think that was so much the case in the 90's, attitudes seemed to have been that these sorts of systems don't make mistakes, and therefore can always be trusted. I look at this as the 90's version of "The Titanic is an unsinkable ship"

EDIT: I would add that you can rely on systems / single data points for some decisions. How I see it is trust in a single point of data goes down as the criticality of the decision goes up. If someone sends you a message to meet them for lunch downstairs, that decision has low criticality and therefor you can rely on a single data point / application to make the decision. However for more critical decisions (ie. should we send this person to jail for stealing money), trust in any system should be low by default and a consensus from multiple data sources is required.

commandlinefan• 3 years ago

that situation wasn't unusual

Um, I'm working as a professional computer programmer today, in 2024, and I can assure that programming continues to be replete with incompetent programmers who are not capable of producing professional code.

surfingdino2• 3 years ago

The worst thing about it was the assumption that the computer system was right and the people were crooks. This should give people wanting to stick AI into every orifice of our lives a pause, but I have a feeling it will not.

Gee8r92• 3 years ago

It was not a software glitch! Such thing would get normally discovered and fixed in a few months

It was corrupt managers, police and goverment! Literal conspiracy to destroy lives of innocent people!

iforgotpassword3• 3 years ago

So the corruption made it that somehow that bug-free software produced inconsistencies in accounting? Interesting.

vidarh2• 3 years ago

The point is that the bugs on their own would have been merely a nuisance leading to unnecessary audits if not for the severe mismanagement around them.

mathieuh• 3 years ago

There were hundreds of these cases, and the affect post-masters were calling the IT helpline repeatedly saying sums of money were going missing. Personally I think that if you have hundreds of people of good reputation and standing swearing up and down that they didn't steal money, and there's a record of their having called repeatedly to report erroneous data in the software, then there should be some sort of investigation to figure out what was going on. Instead they were shouted down and threatened.

imtringued3• 3 years ago

Nobody said the software is bug free. That is your interpretation. The corruption lies in the fact that the bug wasn't fixed.

passerby1• 3 years ago

Would someone suggest how would hundreds of investigations (and courts? Or there were no courts at all?) were not able to shed any light on the bug? How is it even possible to continue for more than 20 years?

arkh• 3 years ago

Imagine you are head of some administration. Times are getting hard, profits are decreasing so you can not embezzle money as much or as easily. You know people under you must be stealing, and some new software just happen to be able to prove it. Here is the solution to help pay your third mansion in the south of France.

g_p2• 3 years ago

It's worth noting that courts (in the common law system) don't generally play any kind of inquisitive role in cases - there's a presumption that both sides bring legal representation for an adversarial debate where the court is impartial and independent.

Given many of the sub post masters struggled to fund their legal cases (and legal aid is not likely going to stretch to funding the intensity of expert witness work required to discredit a whole system), this also seems to be one of the non technical failings not often talked about.

There's certainly questions about why courts didn't spot issues, but ultimately the common law adversarial system assumes both parties can get equal quality of representation. Unless you have very strong technical knowledge (this site is a non representative sample!) you'd struggle to really shed any light on the bug and get a court to agree with your line of defence.

Ultimately that's what did happen in the group litigation by Bates, about which the TV drama was ultimately created.

jdietrich• 3 years ago

The Post Office knew that the Horizon system was deeply flawed and that the prosecutions they were pursuing were unsound. They intentionally covered up this fact. The courts adjudicated based on evidence provided by the Post Office, but that evidence intentionally omitted things like error logs that would have undermined their case. The defendants had a right to see that evidence, but the Post Office simply denied that it existed. The Post Office were helped by the fact that the rules of evidence presume that computer records are accurate unless shown otherwise - the burden was on the defence to show that Horizon was flawed, not on the Post Office to show that it was reliable.

Lots of things had to go wrong for this miscarriage of justice to happen, but at the core of it all is the fact that the Post Office lied for years - to the defendants, to the courts, to the media, to government. Things should have been handled differently, the case has revealed a number of systematic failings that need to be urgently addressed, but there's only so much the legal system can do to defend against very sophisticated people who engage in a determined campaign of deceit. We can reduce the risk of something like this happening again, but sadly we can't eliminate that risk entirely.

https://davidallengreen.com/2024/01/how-the-legal-system-mad...

onion2k2• 3 years ago

In my last team I banned use of the word 'glitch'. It was a catch-all phrase for "bug I don't want to take ownership of" used by developers and product owners alike when they were talking to customers. It has no place in a modern tech team.

m463• 3 years ago

Can you own everything?

mjh2539• 3 years ago

You can't own everything, but referring to something as a "glitch" is essentially ITs thought-terminating cliche.

onion2k2• 3 years ago

You can own all the bugs.

alphalima• 3 years ago

I have also banned "random", replacing with "intermittent", in the past.

varispeed2• 3 years ago

These days there are many work places where people don't have enough skill finding and fixing "bugs". People used to copy and paste Stack Overflow answers into the company code base until it "worked". Then if there was a problem it could sit for weeks or months with someone and then moved to another person. It's interesting because in most cases doing simple prints just to see what's going on could give a hint what to fix, but even that was too complex. It's getting worse because if software is making money then there is further incentive to cut costs of development (hiring less experienced and cheaper engineers) and corners (let's not do TDD it's a waste of time). UK government is addicted to big consultancies that are famous for this and they have been exploiting public sector by delivering half arsed projects (if at all) at extremely high cost while tasking cheap and inexperience labour to do the work and pocketing the rest of the money. There is no organisation interested in investigating all those contracts.

freetanga3• 3 years ago

I have read about this somewhere else. The key issue seemed to be in the Post Office internal “Justice” system, which is opaque, biased and refused to consider evidence. It’s pretty stupid that in an age where we strife for universal rights, your job could strip you from defending yourself on the public justice system, with all its defects and warranties…

The IT bug was an issue, sure, but the political mismanagement of an institution stuck in the past is what caused all the ruin for so many people. And it flew under the radar until Netflix made a movie. Actually, the lady running the PO was awarded a Goverment recognition.

IT and code generation is full of pitfalls, but this one lays somewhere else.

freetanga3• 3 years ago

From the FT article:

__The state-owned Post Office acted as investigator and prosecutor in the cases, using the general right in English law for any individuals and organisations to pursue private prosecutions without involving the CPS.

A public inquiry into the scandal has heard that the Post Office, among other aggressive legal tactics, accused sub-postmasters of theft to pressure them into pleading guilty to lesser charges.

The CPS has identified 11 cases it brought against sub-postmasters that involved “notable evidence” from the Horizon system.

Legal experts said the government had been warned several years ago that private prosecutions carried a higher risk because those pursuing them were more likely to have motivations other than securing justice.

Lord Ken Macdonald KC, a former director of public prosecutions, said: “If you’ve got a body with skin in the game [such as the Post Office] acting as a prosecutor, that creates obvious risks and dangers.”__

lmm3• 3 years ago

IMO the private prosecution is a necessary safeguard. In other countries you get the opposite problem: a gross injustice has been done, but the public prosecutor won't prosecute (often because the perpetrator was connected with the government in one way or another), so nothing gets done.

owisd2• 3 years ago

It wasn’t actually a private prosecution, the Post Office has the right to prosecute on behalf of The Crown. The CPS can intervene on private prosecutions so it might not have flown under the radar for so long if it had actually been a private prosecution.

jack_riminton9• 3 years ago

Management but no leadership

The CEO managed the crisis, therefore she was rewarded. The government should have shown leadership and demanded answers after the first few years of warning signs. Yet it took 20 years and a TV drama to force them to show any leadership

freetanga3• 3 years ago

Well, they only took 60+ years in admitting they went over they line by castrating Turing for being gay - after his huge relevance in keeping the UK safe in WW2 and using his name and face as poster for the role of UK in Computing History.

The UK Goverment really needs to shake off all this archaic nonsense and modernize themlselves. It’s not in line with the fantastic culture of their country.

esskay• 3 years ago

Slight correction, ITV made the series Mr Bates vs the Post Office, not Netfix (no idea if its being shown internationally on Netflix under license however).

gklitz3• 3 years ago

The key issue seemed to be in the Post Office internal “Justice” system, which is opaque, biased and refused to consider evidence.

No doubt pushed forwards by the racists believing in their heart that some of these sub postmasters HAD to be guilty because of their ethnicity.

rideontime• 3 years ago

Something to take into account when we consider replacing our legal system with a blockchain.

deathanatos• 3 years ago

It’s pretty stupid that in an age where we strife for universal rights, your job could strip you from defending yourself on the public justice system, with all its defects and warranties…

I agree it's pretty stupid, but it is becoming the case for more and more people; while slightly different than the situation in the UK, forced arbitration clauses strip the right of the employee to seek justice, and they're getting more and more common.

madsbuch3• 3 years ago

Based on the article, this also appears to be miscarriage of justice. A lawyer recommending an innocent person to plea guilty sounds like a lawyer not doing their work properly.

guappa4• 3 years ago

They didn't have access to the full documentation, let alone be able to review the software itself.

madsbuch3• 3 years ago

They have access to the suspect. If the suspect don't understand whether they did it, it should call for a full-scale investigation of the software so at least somebody understands exactly why they are guilty.

In Denmark the police thoroughly investigates cases also when the accused plead guilty.

In a country that has a rule of law, punishment is not negotiation. I do realize that it is up for interpretation for the Anglo system due to its over-commercialized legal system.

gmac2• 3 years ago

Agreed. Although, as I understand it, this also happens to be a staple of the US legal system: https://stfrancislaw.com/blog/whats-a-plea-bargain-and-how-d...

quickthrower2• 3 years ago

Or they know they are up against a kangaroo court.

madsbuch3• 3 years ago

whixh would be an even more serious matter for something trying to resemble a first world country.

derriz• 3 years ago

Plenty of obvious villains in this story but where was the legal system in all this? 900 prosecutions without any real evidence or just because “the computer says so”? As one of the 3 pillars, isn’t the legal system and judiciary supposed to act as a backstop against this sort of capricious mass-persecution of completely innocent people?

madaxe_again2• 3 years ago

The post office can draw its own prosecutions - no need for the prosecution service etc., and in general, a magistrate confronted with His Majesty’s Postal Service’s Honourable Legal Team (OBE, CBE) and This Dirty Bloke With a Regional Accent What We Reckon Stole From Us will chose the government every time.

samtho• 3 years ago

This is the real failure. The software is neither smart nor dumb, it’s a machine. In this case, it was broken, but people who actually have the ability to critically analyze and judge situations deferred to the output of a machine that they have no real visibility into the internals of, and took its output as gospel.

laurieg2• 3 years ago

Here are the judge's instructions to the jury for one of the trials:

"There is no direct evidence of her taking any money [...] She adamantly denies stealing. There is no CCTV evidence. There are no fingerprints or marked bank notes or anything of that kind. There is no evidence of her accumulating cash anywhere else or spending large sums of money or paying off debts, no evidence about her bank accounts at all. Nothing incriminating was found when her home was searched." (The only evidence was a shortfall of cash compared to what the Post Office’s Horizon computer system said should have been in the branch.) "Do you accept the prosecution case that there is ample evidence before you to establish that Horizon is a tried and tested system in use at thousands of post offices for several years, fundamentally robust and reliable?"

My word against yours wouldn't be enough to meet the standard of "beyond a reasonable doubt", but the Post Office's word backed up by a computer system? It seems that was convincing enough for the jury. They gave a guilty verdict in the above case.

elthran• 3 years ago

This sort of thing is why I've always hated the concept of a random jury from the general populace - In this example, I don't want Joe Bloggs the butcher's understanding of complex computer systems being the determination of whether I am criminally convicted, I want 12 people who have at least some experience programming.

veltas2• 3 years ago

Which trial?

forinti2• 3 years ago

The judge should have asked for an accountant to review the books.

Surely the system must be able to spit out what services and products were sold and how much they would be worth.

ChocMontePy• 3 years ago

One of the problems is that the UK legal system has a presumption that computers are reliable. They are assumed to be working properly unless proved otherwise, which shifts the burden of proof on the person trying to claim that they are not working properly.

Many commentators are saying that this presumption should be changed:

https://www.theguardian.com/uk-news/2024/jan/12/update-law-o...

https://www.forbes.com/sites/emmawoollacott/2024/01/15/law-o...

thom3• 3 years ago

This was coupled by the victims being lied to that they were the only ones. Had they known this was systematic they could have mounted a more effective defence.

127361• 3 years ago

So does that mean guilty until proven innocent? If I'm correct here, then that's utterly sickening.

dukeyukey3• 3 years ago

You have to assume that at some point though, right? There are so many layers of software it's never gonna be possible to inspect it all the way down.

veltas2• 3 years ago

If you want an example read all of https://www.bailii.org/ew/cases/EWHC/QB/2007/5.html

LightHugger• 3 years ago

in the UK the legal system has devolved beyond all reason

stephen_g• 3 years ago

Incredible that literally hundreds of people were apparently in the same situation - none of them had any paper trail of money being moved into their accounts, no unexplained wealth, no extravagant spending… All of them just that the computer said so!

I guess part of then problem is that the justice system takes every case in isolation, but the legal system really needs some mechanism where there’s a “hang on, something is wrong here” after the first few…

First change in this case specifically is probably stopping the archaic convention of the post office making their own prosecutions in the UK…

asmnzxklopqw• 3 years ago

Wow, if something like this happened in the mainland Europe, probably many heads would fall, up to the ministerial level. For the establishment in good ol’ England it’s just another day.

irdc• 3 years ago

Did you hear about the Dutch childcare benefits scandal (https://en.wikipedia.org/wiki/Dutch_childcare_benefits_scand...) yet? That one affected about 28’000 parents with 70’000 children and, due to the illegal separation of children from their parents this resulted in, is at the crimes-against-humanity level.

InCityDreams• 3 years ago

if something like this happened in mainland Europe, probably many heads would fall....

Absolutely no they wouldn't.

laurieg2• 3 years ago

This story has finally been getting the air it deserves.

If you want a summary and insight into the staggering scale of the injustice, this article from Private Eye magazine is worth reading:

https://www.private-eye.co.uk/pictures/special_reports/justi...

This BBC radio programme, started in 2020, also gives a lot of good information including details of how suspected sub-postmasters were questioned by the Post Office.

https://www.bbc.co.uk/sounds/brand/m000jf7j

time4tea• 3 years ago

In UK law, it is presumed that computers systems work perfectly, and it is up to the "other side" to prove they didn't.

In short, if the computer said so, it is a fact, in court.

Lots more information here

https://evidencecritical.systems/2022/06/30/briefing-presump...

hnfong2• 3 years ago

In UK law, it is presumed that computers systems work perfectly, and it is up to the "other side" to prove they didn't.

This is kind of true, but there's a bit of nuance -- presumptions can be rebutted by the other side adducing evidence to the contrary. The evidence does not have to be strong enough to "prove" anything, it's generally sufficient that it suggests the presumption might be invalid. So in colloquial terms you don't have to prove the computer system was flawed, just a real possibility.

That said, the article you linked to does go on to explain why even this scheme didn't work out in the post office cases.

jbverschoor• 3 years ago

Right how, software is seen as truth. This is simply not always the case. I’m happy ai is getting a lot of attention, as people seem to be more aware that its judgement can be wrong.

What is needed is the requirement that software decisions must disclose their data and decisions path/ “algorithm” in court.

Another thing we need laws for is banning a person from using a system. It’s insane that you can be banned for life without recourse or explanation. It’s basically you being thrown in jail for life without reason.

interestica• 3 years ago

The BBC aired this story as a documentary in 2015 as part of the Panorama series. They received threats from the Post Office over the content.

The Post Office threatened and lied to the BBC in a failed effort to suppress key evidence that helped clear postmasters in the Horizon scandal.
The Post Office's false claims did not stop the programme, but they did cause the BBC to delay the broadcast by several weeks.

https://www.bbc.com/news/uk-67884743

It wasn't just a "glitch" -- it was also a PR campaign (quite successful up to now) that supporessed the voices of those affected.

shaoonb• 3 years ago

I find it shocking that this has been known about for so long, but only now that there was a TV drama about it is it actually being addressed. It came up in the news and I thought "what, I remember hearing about this 5 years ago, surely it's been sorted out by now?"

bspammer• 3 years ago

My parents both work for the BBC and are pretty disheartened about this (though they're happy people finally care). It really feels like presenting the results of journalism as a drama, rather than a documentary, is orders of magnitude more effective at enacting change.

SLJ7• 3 years ago

I would like to understand enough about law, governments and the postal service to assign blame here, but I'll just say that it's really fucking sad that this is still in any way ongoing after being known about for so many years. This shouldn't have needed to be publicised in such a way before the wheels of justice started turning. I remember hearing about this several years ago and thinking "Jesus, that's rough. Technology sure ruins lives." And now I'm hearing about the (still ongoing) situation in 2024 and I no longer blame the tech. It's all people. People are awful.

abrookewood• 3 years ago

This is very similar to the robodebt scheme [0] that happened in Australia with similarly devastating consequences: - Welfare recipients' suicide after receiving automated debt recovery notices - Debt notices were issued to deceased people. - Issuing debt notices to disability pensioners.

Looks like Wikipedia has termed this Algocracy (government by algorithm).

[0] https://en.wikipedia.org/wiki/Robodebt_scheme

partomniscient• 3 years ago

It's kind of similar, but in Robodebt it was more a case of covertly implementing a wrong algorithm for debt calculation (because it inferred good results for the government of the time), placing the responsibility of disproving the debt on the individual with ridiculous requirements and no support or process to easily challenge them.

It was really a result of the Liberal & National Party (LNP) who were in government at the time wanting to:

- look tough on welfare cheats for image/election purposes

- claim a potential massive source of income (that wasn't really there) for the next budget

Most of the wrong calculations were due to a result of income averaging fortnight or multiple fortnights worth of income and extrapolating them to an annual figure, which is the absolute wrong thing to do when a lot of people on welfare worked as casuals and/or had short term jobs and the fortnight(/s) were non-representative of their yearly income.

In some cases the first time the alleged debtee found out about the debt was via a demand from a debt collection agency.

When individual action was taken against them, the government was reducing debts to $0.00 and then claiming there was no longer a reason for a case against them and had it dropped. The only reason the debt recovery program was stopped was because lawyers acting upon someones behalf also claimed interest on a paid false debt, which meant they couldn't succeed using the same tactic at which point they rolled over and admitted the unlawfulness of the scheme, which had been going for many years at this point.

I recommend this youtube 3 part series to get a good understanding of what was involved:

[0] https://www.youtube.com/watch?v=OfsL9GAbl3M

It starts out mentioning that it resulted in the largest class action lawsuit in Australian history...

zimpenfish3• 3 years ago

- look tough on welfare cheats for image/election purposes

Ironically the same reason for the Horizon system which ballooned into the PO Scandal.

smdyc13• 3 years ago

And of course, the public servants and the politicians involved largely got off scot free. In the Australian Robodebt case as well as the British Post Office, the key people involved should be facing criminal proceedings to deter this sort of thing!

vijaybritto• 3 years ago

P1: I brought down prod with a faulty migration script

P2: oh Im responsible for the deaths of many people and court cases spanning 2 decades. There's even a tv show based on the story

kunley• 3 years ago

I would guess it's the corruption of tenders in the public sector.

The organisation in the public sector must choose the right vendor and then any sh*t sold by this vendor is correct, otherwise it would prove the wrong choice.

londons_explore• 3 years ago

I don't fully understand how people end up in prison over this... If the software says there should be X money in the cash drawer, but there is actually Y money, then clearly something is wrong, so you investigate.

You get the software to print off a list of every sale made that day with timestamps. You check the stores CCTV so see when every customer came into and out of the store. You cross reference. Most are small stores that probably only see 300 customers all day anyway.

If you are a store manager wrongly accused of fraud, of course you can be bothered to do this.

It would only take one store to do this to find out that the software was wrong.

ghusbands• 3 years ago

There were more than 900 convictions, and you think that not one of those accused would have tried to prove their innocence?

Typically, when it gets to the point of being an accusation, employees will lose all access to all systems. It's also widely reported that evidence was routinely suppressed and emails conveniently lost.

And the software was found wrong in November 2019, when a judge ruled that the software contained bugs, errors and defects, which caused a settlement in that case and a cessation of further cases.

The oddest part is that with all these Horizon problems being pointed out for decades and then a judge ruling that it was faulty, there still hadn't been an inquiry until very recently.

gmerc• 3 years ago

it’s not the software glitch that ruined life.

It was people making decisions about suppressing evidence, lying, not correcting the mistake.

It’s like saying faulty breaks killed thousand people and omitting that the company could have recalled the car after the first car but chose not to.

Is suspect this kind of reporting is intentional to minimize pushback. “How Fujitsu and the Post Office drove people into suicide” gets you angry lawyers. Blaming the glitch not.

oliwarner• 3 years ago

The glitch wasn't the reason people went to prison. It was incompetence, perverse outcome-based executive incentives, plain corruption and baseless hubris.

The problem that this unearthed was that evidence of crimes committed through information systems can be obscenely complex and therefore obscenely expensive to defend against.

"Computer says guilty" shouldn't be enough, but a defence would take months of debugging. Not something somebody on a £20k salary could ever afford.

But that's what happend. Hubris that Fujitsu's system was infallible. Targets and bonuses that stopped management asking uncomfortable questions. Layers of incompetence meaning people weren't asking the right questions, missing the correct burden of proof in the legal process.

And all this over an accounting system that can be forensically picked apart. Just imagine how bad it'll be when it's a black-box AI.

alexey-salmin• 3 years ago

The title should be "how post office executives knowingly ruined lives and obstructed justice"

forinti2• 3 years ago

I wonder if you could provoke the doubling bug until you got to a ridiculous amount or an overflow that would be just too ridiculous to not be considered a bug.

If a post office owed a billion pounds then that would be impossible to blame on the postmaster.

hnfong2• 3 years ago

Yes but ironically if you intentionally triggered the bug you'll definitely be found guilty of fraud.

I don't think the "I was just trying to show how ridiculous the bug was" defense will go too well in court.

zubairq• 3 years ago

The UK Postcode scandal is interesting in that it was exposed at all. Many people in countless industries have had their lives ruined by similar software bugs and bad industry practices, but those people have not been able to fight or to get justice.

BonoboIO• 3 years ago

I think the higher ups of the post office should got to prison for a loooong time, strip their wealth.

It was a software glitch, but the coverup was massive, even years ago. It was intentional to hide their incompetence.

arp2422• 3 years ago

I really hate how this is being framed as "a software glitch caused [...]".

It wasn't a "software glitch" that caused anything. Yes the software sucked, but lots of software sucks. It was human decisions that caused this, by using software they knew (or could and should have known) was not fit for purpose. And by ignoring people who told them it was not fit for purpose after it was deployed. And the government not stepping in. And National Federation of Sub-postmasters. &c &c &c. All of this is well documented.

That the initial mistakes were made was kind of ridiculous for lots of reasons, but okay, mistakes happen. But that it took 20 fucking years to correct while the broken software continued to be used is just indescribable, and absolutely not the fault of any software but the result of human choices. That's the real problem; in an alternative universe people realized Horizon was a piece of crap shortly after being deployed, the mistakes were corrected, and they fixed it or stopped using it.

Humans caused the misery. Not software. The software bit is almost a minor detail IMO. By framing it as "a software glitch" people will focus on software to "fix" things, but that's not where things need to be fixed.

vb-8448• 3 years ago

But that it took 20 fucking years to correct while the broken software continued to be used is just indescribable, and absolutely not the fault of any software but the result of human choices

100% agree. Managers who were in charge of these choices should be convicted.

ChocolateGod• 3 years ago

If the ITV show is factually accurate, looks like the managers knowingly tried to send innocent people to prison to cover their own careers and reputation.

They should have the book thrown at them.

snarfy• 3 years ago

With nearly everything in the world run by software these days, it's an easy scapegoat, and an even easier clickbait title. Engineers should be furious about this framing. Even most legit software issues can be trace to poor management, poor processes, rushed deliveries, and inadequate funding. But you would never know that from reading mainstream media. To them we are all bumbling overpaid idiots.

alentred• 3 years ago

Exactly, framing the software removes the accountability from where it belongs - people. Software cannot be responsible for anything, people can, and should assume it.

_heimdall• 3 years ago

Can we really say it wasn't the software that ultimately caused this when the software absolutely had bugs?

If nothing else this puts into focus plenty of bureaucratic problems, but we wouldn't be here without the underlying buggy software that opened the door for bureaucracy to fail so badly.

maccard• 3 years ago

Can we really say it wasn't the software that ultimately caused this when the software absolutely had bugs?

Yes, because as the OP clearly stated, the problem wasn't the bugs.

The problem was the bugs were noticed, flagged, discussed, _and systemically ignored_.

richeyryan• 3 years ago

Substitute software for standard accountancy practices or a filing system that led to lost files, giving the incorrect impression that postmasters were stealing, and the result would have been the same. The core issue is an organisiation that values avoiding blame at all costs over transparency and accountability.

brianmcc• 3 years ago

Yes we can. The levels of corrupt behaviour involved in this story, going right to the very top of Government, literally, absolutely transcend any "bugs happen" narrative.

arp2422• 3 years ago

In any organisation things will go wrong at times, for all sort of reasons: software bugs, human error, organisational dysfunction, malicious actors, whatever.

Mistakes get made. And even if the original mistake was hugely egregious, it's usually not a huge problem if you actually deal with it well (barring things like medical errors, but even there, trying to cover up a mistake can cause more harm than the initial mistake, at least in some cases).

For these kind of institutional problems most of the time it's really the response to the mistake that turns a mistake in to a huge catastrofuck.

PH95VuimJjqBqy• 3 years ago

glad to see this as the top comment, this is basically what I stepped into the thread to point out.

describing this as a software glitch is kind of heinous in its own right.

Your game crashing is a software glitch, what happened in the UK is far beyond that. It'd be like describing the issues Boeing is experiencing as a "hardware glitch". Decisions were made that should not have been made, full-stop.

mensetmanusman• 3 years ago

It's only human to blame inanimate objects for their own failings...

gmac2• 3 years ago

Definitely true. But the software does also seem to have sucked extremely and surprisingly hard for something that was mission-critical and handling millions of pounds of transactions a day. From what I've read (which is admittedly so far less than I'd like), the whole architecture was deeply inappropriate: it wasn't just subject to a 'bug' or 'glitch'.

johndunne2• 3 years ago

A key detail is that the Post Office specifically had a method for secretly and remotely editing accounts of post masters and lied about its existence. Along with experts lying under oath that the software had no known issues that could result in the account errors they were seeing, when in fact the ‘expert’ was well aware at the point that horizon was rife with such bugs.

acyou• 3 years ago

Maybe the post title is inaccurate. The article title itself continues with "and a centuries-old British company ruined lives".

It wasn't a "bridge collapse" that caused anything. Yes the bridge sucked, but lots of bridges suck. It was human decisions that caused this, by using a bridge they knew (or could and should have known) was not fit for purpose. And by ignoring people who told them it was not fit for purpose after it was deployed. And the government not stepping in. And Association of Professional Engineers. &c &c &c. All of this is well documented.

That the initial mistakes were made was kind of ridiculous for lots of reasons, but okay, mistakes happen. But that it took 20 fucking years to correct while the faulty bridge continued to be used is just indescribable, and absolutely not the fault of any bridge but the result of human choices. That's the real problem; in an alternative universe people realized the bridge was a piece of crap shortly after being deployed, the mistakes were corrected, and they fixed it or stopped using it.

Humans caused the misery. Not the bridge. The bridge bit is almost a minor detail IMO. By framing it as "a bridge collapse" people will focus on bridge design to "fix" things, but that's not where things need to be fixed.

When will the software sector be regulated like other sectors?

FirmwareBurner2• 3 years ago

When will the software sector be regulated like other sectors?

But you just said the SW sector didn't cause this, it was those who ordered it and were responsible for managing the public service using it.

dspillett2• 3 years ago

But that it took 20 fucking years

20 years and a high-profile TV serialisation heightening public perception of the problem a little too close to an election for comfort. The government is running around saying they care and how much of a tragedy they think it is not because they genuinely care but because the voting public currently care and there is a general election within the next 12 months.

One of the massively empathy lacking comments from a couple of weeks ago (prior to the proposed law change specifically to speed up exonerations for this scandal, announced last week) was that they couldn't just quash all the convictions without retrial because that might let some genuine cases of fraud through – as if letting one or two criminals off more lightly (they have had 20 year of whatever punishment was originally handed down and its side effects) is such a problem compared to continuing to punish hundreds of innocents. Let them all off, then retry those few if you find new compelling evidence. But before doing that: get the people who tried to cover it all up, who bullied many into silence despite knowing (at least some of) the truth, and those who just stood by and let it all keep happening, etc., both in the post office and the government, into court first – what they have done is far more reprehensible than what those post workers were (wrongly) accused of. Handing back one CBE just doesn't cut it.

yurytom• 3 years ago

I remember meeting two guys in a pub. They worked for a bank. When I told them I work in IT, they said, 'Oh, it's because of IT systems failure that we had the 2008 financial crisis.' They truly believed in what they were saying.

jimnotgym6• 3 years ago

It still intrigues me that if I want a steel beam in my house I need a 'Chartered Engineer' to calculate the size. If I want to know what it will cost I use a 'Chartered Quantity Surveyor', and to work out the financial implications I might use a Chartered Accountant. All of these have rigorous multi-year training, with experience requirements...and a code of ethics. This is all for a £1K contract.

Meanwhile, if I want a multi-billion dollar critical infrastructure software project, I can use people who learnt to code at summer camp. Isn't it time we had proper qualifications for software 'engineers'?

jwmoz• 3 years ago

You were never going to get the best of the best working on this system in a random soviet looking tower block in Bracknell.

jongjong• 3 years ago

As a software engineer who has built distributed systems, I can attest that building reliable software is really difficult, bordering on impossible.

The hardest part is probably handling and recovering from all possible failure scenarios. You need to make sure that the system could crash while in the middle of processing any line of logic in your system and it should be able to recover elegantly; without skipping anything and without re-processing what has already been processed (which can cause duplication of records).

The challenge with distributed/partitioned systems specifically is that atomicity is much harder to achieve and strategies for achieving a similar result are complex and error-prone (e.g. two phase commits, using idempotency to avoid double-insertion)... For complex database transactions involving several tables with a custom two-phase commit mechanism, you have to be careful to process records of different types in a specific order. Also, you need to set up your database indexes carefully for fast lookup and sorting...

switch0072• 3 years ago

For anyone thinking the Post Office being a private company absolves the Government of any responsibility, not so fast.

It's wholly owned by the Government and there is a minister responsible for postal affairs, which includes the Post Office. However it has its own board for day-to-day operations.

The government has a _lot_ of blood on their hands

Source: https://researchbriefings.files.parliament.uk/documents/CBP-...

vlovich1232• 3 years ago

This story just gets worse and worse. Interesting that Sunak only promises to try to overturn convictions and get some kind of financial remuneration for victims, but no discussion about having a investigation into how this structurally happened in the first place and figuring out who needs to be punished for coming up with these incentives in the first place?

g_p2• 3 years ago

I believe this is coming, but (rightly) from the police investigating credible allegations of criminal wrongdoing, rather than from Government intervention.

2x fairly prominent Fujitsu people have been interviewed under caution on suspicion of perjury, and some of the recent revelations from recent testimony in both the parliamentary committee hearing and the public inquiry seem likely to open the door to further such investigations.

In terms of internal conduct, the Post Office is now (as of first week of January) apparently under investigation for potential fraud offences. There is some speculation this includes individuals, as well as the entity itself (given many fraudulent actions would be carried out by individual people).

I imagine such prosecutions (properly carried out, independent of government actions) may help focus people's memories and also begin to start the process of punishing those individuals concerned, and implicate others in the process.

vlovich1232• 3 years ago

I just feel for these families. I recall reading about the software bug so long ago and they’re just now starting to talk about getting them retired.

tlivolsi• 3 years ago

Is anyone being held accountable for this?

thebruce87m• 3 years ago

The people responsible are probably still deciding on the scapegoat. Give it time.

MikeDelta• 3 years ago

The glitch caused it, but it was the way the UK Post Office handled it that ruined lives.

tremon• 3 years ago

I absolutely object to this phrasing. No, the software glitch itself did not ruin lives. It was management choosing to conduct a witch hunt instead of a thorough investigation that ruined lives.

HacklesRaised• 3 years ago

A software glitch didn't ruin lives, a duplicitous organization with dishonest people at the helm ruined lives. The problems with Fujitsu's system were widespread and understood but the Post Office pursued these people regardless. It should come as no surprise that the money they fleeced from these helpless victims likely went to executive pay. We all know that capitalism is a terrible system, it's just better than any other system that has been proferred, but profit motive brings out the very worst in people.

testemailfordg2• 3 years ago

When the elite have a stake in the deals or receive kickbacks, you shouldn't expect the system to run fairly. Any issues found, would be quickly sweeped under the rug with all might. Even judiciary isn't immune from the elite, can't even expect a fair investigation when elites are involved.

How a software glitch at the UK Post Office ruined lives

Related Stories