Why It Is Important That Software Projects Fail (2008)

http://www.berglas.org/Articles/ImportantThatSoftwareFails/ImportantThatSoftwareFails.html

by perfunctory • 7 years ago

139 18 7 years ago

Quite an interesting article which at least verges on parody.

The thing about the automation of bureaucracy is that what it being produced doesn't a value but rather has a "redistribution effect", so more tax codes produce at best a more fine-tuned rearrangement of wealth according to the ideas of legislators. So increased productivity is meaningless.

One thing I'd add is that there areas that produce useful items but where the insertion of automation has helped much either - health care is an example I find interesting. And certainly there are places where computer-based large-scale automation has resulted in serious increases in productivity (I believe rail roads had this in the 1990s).

So question of where automation tends to fail and where it tends to succeed is a bit open - but it seems like "organizing complex, interlocked human interactions" is one place where failure is common.

dang• 7 years ago

A thread from 2009: https://news.ycombinator.com/item?id=932956

mushufasa• 7 years ago

I think this is hilarious, but probably not going to continue to be true. Yes, a lot of software products fail, but automation is increasingly driving capital growth and replacing labor. The p2p gig economy does cut out a lot of bureaucracy, and AI advances... both of which were hard to predict in '08.

The article he cited, Parkinson 1955 (originally in the Economist), on the relationship of british naval bureaucracy to work, is even more interesting imo! http://www.berglas.org/Articles/parkinsons_law.pdf . It hypothesizes a growth equation for bureaucracy where the only exogenous factor is the underlying population. I'd be curious whether more rigorous research says anything about bureaucratic growth, as we're stretching further towards an era where minimal % of the population has to do any 'blue collar' labor. Maybe in the field of institutional economics?

Supermancho• 7 years ago

It hypothesizes a growth equation for bureaucracy where the only exogenous factor is the underlying population

Having observed additional layers of bureaucracy develop over time, I have noticed the additional factor of attempting to get blood from a stone. When a the critical metric (eg revenue growth) falls, a perverse incentive of increasing control is pursued after some short gains. Following the pereto curve, this too is less effective over time.

marmaduke• 7 years ago

attempting to get blood from a stone

This is an excellent expression. I have observed this sort of thing as well, especially with regards to deadlines that get pushed back.

yosefk• 7 years ago

Berglas actually wrote about AI as well (http://www.berglas.org/Articles/AIKillGrandchildren/AIKillGr...), and suggested to legally limit chip frequency to 1 MHz to avoid AI proliferation, on the theory that from an evolutionary standpoint, rogue AI is inevitable. More recently (during the current deep learning bubble) he wrote a whole book about AI.

gridlockd2• 7 years ago

Yes, a lot of software products fail, but automation is increasingly driving capital growth and replacing labor. The p2p gig economy does cut out a lot of bureaucracy, and AI advances... both of which were hard to predict in '08.

The gig economy may cut out "a lot of bureaucracy", but it's replacing it with other forms of pointless work.

I don't think this is more true anywhere than in software. How many AngularJS developers does it take to change a light bulb? Answer: Never mind, we're rewriting the frontend in React.

mattmanser• 7 years ago

I'm interested how you and the GP think it reduces bureaucracy?

Surely it's more, now a bunch of "contractors" have to do loads of company admin off the clock.

It's just a dodge, shifting bureaucracy and costs from a company onto an individual who's being forced to masquerade as a company. And possibly a flash in the pan, it is certainly possible the gig economy will be an illegal working practice in 20 years time in socially progressive countries.

twic• 7 years ago

driving capital growth and replacing labor. The p2p gig economy

The p2p gig economy does not drive capital growth or replace labour.

erikpukinskis• 7 years ago

As the primary net effect of software is to facilitate bureaucratic complexity it is therefor essential that software projects fail

I love this conceptualization of what software is for. But I have the complete opposite moral interpretation.

Unstated is the presumption that bureaucratic complexity is bad. I think increasing it is the goal of society, and ethics constantly demand us to increase it.

For example, it’s easy for banks to simply reject loan applicants who were convicted of felonies. There’s no easy upside: given existing loan approval and repayment processes there’s no clear way to give those loans profitably.

There is a huge social cost to that group lacking access to credit though. It increases crime and violence.

And there is probably a long term financial cost too: some felons will reliably make their payments. And still more COULD reliably make payments with the right support.

So how does a bank separate out those groups and provide that support?

Increase bureaucratic complexity!

I am not necessarily in favor of locating that complexity in a state government. As an anarchist I would want to locate it as close to the affected community as possible....

But to me the dream of software is nearly infinite bureaucratic complexity that affords all of us greater freedom, and health.

notduncansmith• 7 years ago

I would love to know (not sarcastically) your definition of anarchy that is compatible with bureaucracy - the two seem at odds.

splittingTimes• 7 years ago

From my understanding, Anarchists look at an power structure and ask how does it legitimize it's existence. If it was put in place by the people and not some unaccountable body, then it is ok to have it, otherwise it needs to be dismantled and or replaced. (This is a very, very simplified look at a very narrow aspect of anarchism of course.)

For example a lot of decisions in Anarchist communities will be driven by arriving at consensus of all concerned by the decision, instead of a "mob rule of the majority" like in Democratic voting process.

The way you arrive at the decision is a bureaucratic process if you will, that the community has agreed upon.

That's my take on it

LogicWolfe• 7 years ago

A good discussion of the different ways in which this term is applied can be found here: http://theconversation.com/what-is-anarchism-all-about-50373.

To quote a relevant passage: "Anarchism is a process whereby authority and domination is being replaced with non-hierarchical, horizontal structures, with voluntary associations between human beings."

If bureaucratic complexity is the complexity of the structures that organize our society, its existence is entirely consistent with an anarchist society. The structures would no longer be hierarchal, but anarchism makes no claim that organization (or the complexity associated with it) shouldn't exist.

raxxorrax• 7 years ago

I believe that complex rules leave more room for exploitation and additionally induces legal uncertainty.

Comprehensive bureaucracy can be good but also often fails to efficiently handle problems and gives advantages to specialists.

The result can be something like this:

https://en.wikipedia.org/wiki/CumEx-Files

So how does a bank separate out those groups and provide that support?

Isn't it a bureaucracy that provided the information about felons in the first place? Not saying that interest in information is invalid, I am just doubtful about the solution.

inflatableDodo• 7 years ago

Interesting take. Another good reason for increasing bureaucratic complexity, especially in power structures, is so that it will serve as a tarpit for megalomaniacs. The EU is probably one of the best examples of this.

gridlockd2• 7 years ago

"When engineers set out to build a bridge a bridge gets built"

Not necessarily. The main difference here is that most bridge projects are not private investments and so cost overruns are less likely to cause complete failure, but the cost overruns can still be massive:

https://www.mercurynews.com/2018/05/03/letter-bay-bridge-cos...

danohuiginn• 7 years ago

Some beautiful synchronicity going on with the headlines here: https://ohuiginn.net/tmp/hn_inventors.png

tempodox• 7 years ago

If an engineer were to deliberately under specify a load bearing beam in order to cut costs, they would be sent to jail. But when a programmer cuts a corner they get promoted.

In the absence of other hard limits, project failure is the only reality check on software development. Without that, we'd all live in Cockaigne, fried chickens flying into our mouths.

Why It Is Important That Software Projects Fail (2008)

Related Stories