Presumption of stupidity

http://www.aaronkharris.com/presumption-of-stupitidy

by garry • 11 years ago

510 178 11 years ago

swanson6• 11 years ago

The same presumption happens for people, too. Developers tend to assume that the people that wrote the terribly messy code that you inherited were incompetent. I think a much more productive and healthy attitude is to assume that everyone was doing the best they could, given their resources, knowledge, and deadlines at the time.

That might be a false assumption (look, some people just don't care) but you gain very little by complaining and getting mad at things that already happened.

We love to complain about things our predecessors did wrong, but often, we don't do those things either :)

networked• 11 years ago

I believe that the error the article describes comes from the family of "inside view" errors (see https://en.wikipedia.org/wiki/Reference_class_forecasting). Taking an outside view instead is a very widely applicable principle.

hn_in_cl2• 11 years ago

There's also the fact that decisions can be very hard to un-make (even in something as malleable as code). This can lead to a bunch of slightly bad/messy decisions all coming together into something that is unimaginably messy/obtuse.

huhtenberg• 11 years ago

Developers tend to assume that the people that wrote the terribly messy code that you inherited were incompetent.

Don't over-generalize.

There is a messy code and there is a messy code. Changing shared data without synchronization is incompetence, but having a 3-page long for-loop is not. In reality though, once you've cleaned stables a dozen times spaghetti code is the red flag of incompetence and the correlation is there.

zzalpha• 11 years ago

once you've cleaned stables a dozen times spaghetti code is the red flag of incompetence and the correlation is there.

Question is, where's the incompetence?

Yup, like any profession, there are developers out there who just suck.

But the same is true of middle and senior company management. Can you honestly say you've never cut a corner because some incompetent manager mismanaged the schedule or customer expectations and then forced you to compromise quality in order to meet an absurd date?

And in any organization with that type of management, those issues are rarely isolated. Any one manager can do a little damage, but when an organization is dysfunctional, well-meaning coders on the ground may be forced to compromise code quality time and time again in order to deal with unrealistic schedules.

Think of it like a professional sports team. Sometimes, yeah, the players just suck. But a bad coach or general manager can have an outsized effect on an organization.

Now, I would never claim that all bad code is a product of bad management. Again, some folks just suck at their job. But speaking as a guy in middle management, I'd be willing to believe at least half of the bad code lying around in the real world is a product of an incompetent management structure.

Getting back to the article, that would represent the kind of hidden incentive that would cause an outsider to assume developer incompetence, even though the reality is very much different.

swanson6• 11 years ago

I absolutely agree that there exists terrible code and that there are incompetent individuals or folks that simply don't care. But if one assumes that as the default response, I don't think that is very helpful.

Even in your example of shared data/synchronization - what if the original developers were told that the code would never need to be thread-safe? Or what if there was a constraint that required this trade-off to be made? It's not so black and white.

zorpner• 11 years ago

Over-generalizer, know thyself.

s_kilk• 11 years ago

Agreed.

One example I see all the time is my more liberal/left-leaning friends assuming that their conservative adversaries (especially on the internet) are motivated solely by stupidity.

This trips them up when they are confronted with evidence of intelligence and solid reasoning from the people they had presumed to be capable of neither.

The reverse also happens of course, with the conservative person characterising their adversary as the stereotypical "Dumb Hippy".

lotharbot8• 11 years ago

Also happens with religion.

pc862• 11 years ago

Some of the worst code I've had to work with in my nearly 10-year long career so far was written by one of the smartest programmer's I've met. His problem was that everything because an exercise in designing this all-encompassing abstracted-to-hell-and-back web of interfaces, services, aggregates, domains, etc etc.

Indicative of this was a request to take a table of data that already existed (on a website) and add a button to export a CSV of that data. For anyone familiar with .NET (and I assume most other languages) if you've already got the data in the format you want to export this is literally a 10-minute task, including a unit test or two for the functionality. His quote was something on order of 18 hours which included time to write a set of TableDataExportService methods that would support a whole host of file formats in the future.

tomjen32• 11 years ago

YAGNI.

Software should be implemented as simple as possible, and then refactored as necessary when new functionality is required (the only exception is things you are very sure is going to be needed, e.g a password reset functionality on a password dialog).

Anyhow that is my development philosophy.

hinkley• 11 years ago

I started on a project filled entirely with 'senior people' once, and was really pumped about the prospect of doing Serious Programming with serious people, instead of doing a bunch of mickey mouse crap.

Six months later everything was going horribly wrong because there was not a single one of us who was willing to solve a simple problem with simple code. Everybody was engineering the hell out of every single 'solution' and the code was impossible to read.

From this I learned a couple of things. One was that I had not learned as much from my Second System Syndrome as I thought I had. The second was that every project benefits from having people who are entertained by solving 'mundane' problems, to whom you can assign all concerns that are not part of the information architecture.

But the most important is that the best solution is -never- the one that is dazzlingly brilliant. It's often the one that's subtly clever (everyone agrees "that works", but some can wax poetic about how great it is at satisfying the concern), but sometimes it's the one that's dead simple.

Few solutions are easier to replace than the dead simple one.

jameshart• 11 years ago

I've consumed enough CSV data to be wary of CSV data that was generated in ten minutes - including the unit tests. Are you sure you already have the data 'in the right format', for example? Maybe the data that's presented on the page has already been formatted for localization, so when you use that in your CSV export you put out dates in US or european format depending on the user's preferences, creating some hard-to track-down integration bugs later. Or maybe it only includes the display name for the status code, not the status code itself, so when three months later you change the display code from 'cancelled' to 'removed', all your clients' Excel macros break.

And once you've done that ten minute job on this page, how long will it take you to add it to the other 25 pages which also have tables of data that need CSV export? And when the table format changes to add another column, does the next developer also have to adjust your CSV output code?

Sure, YAGNI, but... there's no excuse to just throw a bunch of CSV-export logic inline into a page that previously had shown no interest in knowing how to format CSV files. Take a little longer, think about where to put the logic. There's a middle ground here.

swanson6• 11 years ago

I don't mean to advocate against YAGNI or writing simple code. I can't authoritatively to your specific example, but what I am suggesting is that you default your assumptions to a view that this developer had some reason for the choices that were made.

Maybe this client was notorious for asking for CSV export but really meant CSV, XLS, XLSX, PDF? Maybe the build and release infrastructure is so slow that any change - no matter how small - needs 3 days to be built, tested, and deployed? Maybe the complexity makes sense in other areas of the system and they decided to adopt patterns across the codebase to aid in teaching/onboarding new developers?

Just to be clear, maybe you will find that the reasons are totally invalid and this is gold-plated-abstracted-to-hell code. (It sounds like you probably will)

But if you assume from the start that everything is an over-complicated pile of junk and you could rewrite it in a day, I think you will find yourself jaded and unhappy with your environment.

salmonfamine• 11 years ago

"A great tailor does little cutting" ;)

angersock2• 11 years ago

The issue is that, even allowing for good intentions, the fact tends to remain that the code is messy, undertested, and underspecified.

I've worked on a legacy codebase where the developer both admitted that the code was a hack job, got guilty-defensive about it, and then refused to schedule any time to help make progress.

You just want to shake them by the shoulders and be like "Look, it's fine that you did the best you could, but you need to either help clean up the mess or get out of the way."

Chesterton's Fence is wise in moderation--otherwise, it ends up as technological hoarding.

swanson6• 11 years ago

Sure - at the end of the day even the best intentions can result in a poor quality output. I've just found that my own personal outlook is much better if I stop complaining about how some other person messed up and focused on where the code is currently and how we can solve the problems at hand given our new knowledge/resources/constraints.

ombudsperson2• 11 years ago

you make a very valid point!, Thx, :-)

artumi-richard• 11 years ago

I think you can generalise one more step up. Markets that look horribly inefficient may well not be.

codyguy2• 11 years ago

Or maybe they are ;) How to find out? For some people, the only way to confirm is to try and be tested.

suhail• 11 years ago

Great advice.

I do think that you should try to think about how you might try to solve something before looking at what your competitors do. The reason being that it's easy to trap our minds into thinking that there are no other solutions unless they fit into a similar box of what's already working. Naïveté combined with thinking for yourself can often be a powerful reason why many startups succeed.

If your solution ends up looking similar, at least, it was likely derived from first principals vs the path of least resistance: blind copying.

logicallee3• 11 years ago

I couldn't disagree more. When Mark Zuckerberg turned down $1 billion from Yahoo[1] when he was 22, and FB was two years old at the time, because they were "stupid and didn't get it, so they obviously were't valuing the company" properly he was right.

The direct quote is:

Thiel described the argument Zuckerberg finally came down on like this: "[Yahoo] had no definitive idea about the future. They did not properly value things that did not yet exist so they were therefore undervaluing the business."

Yahoo's market capitalization in July 2006 was $42.51 billion. A 22 year-old presumed they were stupid, and he was right. [2]

Today FB has a market cap of $264.91B and Yahoo? Down to $35 billion after 9 years of growth.

[1] http://www.inc.com/allison-fass/peter-thiel-mark-zuckerberg-...

[2] by the way to get the market valuation at the time, I did this search: http://www.wolframalpha.com/input/?i=what+was+yahoo%27s+mark... I can't believe it worked! I used wolframalpha because this is the kind of search they promise they can answer - and they were right, they actually delivered. Nobody else on the face of the planet does this, and it shouldn't even be possible. But it is. If you think something is possible, JUST DO IT. If you think your competitors are stupid (compared to what you think you can do), you're probably right. (or you wouldn't have that thought.)

ikeboy• 11 years ago

FWIW, https://www.google.com/search?q=what+was+yahoo%27s+market+ca... has an answer in the third link.

logicallee3• 11 years ago

yes, but what's amazing is that it's obvious how to calculate it. You look at a chart, mouse over the price at the time, figure out the number of shares at the time (mostly affected by any splits or new shares) and multiply. Instead of actually calculating it, though, I asked a robot to, using human language.

Finding someone who happened to ask the exact same question by doing an Internet search is a different problem entirely. Wolfram Alpha can instantly answer questions that require certain levels of deduction or calculation, and which have never been asked in the history of the world. Google can do that too: for mathematical equations like 453442433 * 234523432 + 3485478347434879. Applying this to questions of a general type that require real-world data is miraculous.

yaur• 11 years ago

If you think your competitors are stupid (compared to what you think you can do), you're probably right. (or you wouldn't have that thought.) No, you are probably not the next Zuckerberg, or Jobs, or Page. If you don't understand why your behaving a certain way its much more likely that you are the one who doesn't understand the market.

logicallee3• 11 years ago

yaur, I'll simply have to disagree with you. If you ask most people "why is Nike stupid, why are their running shoes completely wrong" nobody would give you an answer, nor do I have an answer. Anyone (including me) would say: "They're not stupid, they put billions of dollars of research into engineering and have a great understanding, their shoes are very comfortable and that's why they can charge $100+ for them."

... except for the 50 people in the world who know something that Nike doesn't. Those 50 people who do have an answer to that question are the people who can topple them by doing something better.

Nobody delusional can explain something clearly, and answer all your questions about it - if someone clearly explains something and you think they're delusional for it, likely you're just not qualified to judge. You may be the kind of person who, if you got back sent 1,000 years into the past, would learn to ride a horse and learn some trade, because it's obvious that recreating anything from the future is "impossible", or someone would do it. Nothing wrong with that. Just doesn't apply to startups.

That advice in the context of startups is simply completely wrong, irrelevant, and inappropriate. Couldn't disagree with it more.

minikites• 11 years ago

Chesterton's Fence:

In the matter of reforming things, as distinct from deforming them, there is one plain and simple principle; a principle which will probably be called a paradox. There exists in such a case a certain institution or law; let us say, for the sake of simplicity, a fence or gate erected across a road. The more modern type of reformer goes gaily up to it and says, “I don’t see the use of this; let us clear it away.” To which the more intelligent type of reformer will do well to answer: “If you don’t see the use of it, I certainly won’t let you clear it away. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you to destroy it.

danr4• 11 years ago

Thanks for this.

HCIdivision175• 11 years ago

This philosophy is one of the cornerstones of my engineering practice. IFF you can describe why something is there - without using shortcuts/crutches like 'their dumb' - then we may be able to consider changing it. Otherwise, it's dangerous and risky to fool with it. Too many times did I see something absurd and paint myself into a corner trying to 'fix' it, when there really was a bizarre edge case that this covered. (And in plants, you usually only find these edge cases after enormously expensive production loss events!)

And it's a fantastic principle in general, since it's like the technical equivalent of the Principle of Charity.

( http://philosophy.lander.edu/oriental/charity.html )

JoshTriplett3• 11 years ago

While that's entirely true, it's also true that any code covering an obscure corner case ought to document that corner case and why the code is needed. And similarly, any policy or law should be completely clear about what problem it wants to solve. That would make it far easier to maintain.

swanson6• 11 years ago

This is an awesome comment, thank you for sharing. Do you know of any other resources/materials like this? (Specifically would love to learn more about these kind of principles as related to mechanical engineering or a factory/process setting)

Bartweiss6• 11 years ago

Chesterton's Fence should be one of the core principles of software development, and it calls to mind the famous "Never rewrite software" article.

If you're refactoring, you can use unit tests and proofs of equivalence to demonstrate that your changes don't pose a threat. If you're rewriting a whole codebase, you're likely to wander down the same paths of "wait, the simple way doesn't work" that the person before you took.

Even bafflingly small changes like removing always-false tests have been known to break code, which ought to inspire real hesitation any time you see "stupid" code written by a smart person.

Too• 11 years ago

I worked in a similar setting, writing safety critical code. Any time we found code that was so dumb that it couldn't possibly be correct, without documenting why, we still did a blame to see when it was written, if the blame was older than the current source control system we did a blame in the previous one (this could be 10 year old code). Once we found when it was written we looked at any commit messages, if nothing obvious was written there we looked at general code movement around the time of the commit, we looked at any documents changed around the time of the commit, we looked at issue tracker tickets filed and closed around the time of the commit, we looked at test cases written around the time of the commit, test reports that had different results before/after. In essence we did everything in order to understand how the person thought when writing the code. This had always been a code base with very rigorous processes around it so explaining things by "he probably was lazy and let it slip by" was not a valid excuse before all other reasons had been investigated.

Obvious "errors" like if(false) is a stupid example of this, why would someone commit if(false)? Is it supposed to be if(true)? Is it for debugging only or is it code that should actually be removed? Maybe it's used for other reasons by a preprocessing-tool that you are not aware of?

wyager• 11 years ago

The principle of charity sounds extremely questionable; what is the purported benefit of suspending logical thought for the purposes of understanding something that may be wrong? I want to understand the idea logically, so I don't want to substitute in some second-rate folk understanding while I'm considering it.

Nacraile• 11 years ago

I'm going to dispute the "iff". Chesterton's fence is a useful idea to keep in mind, but it is flawed in that it is excessively conservative. It demands that you prove a negative, which is not reasonable in general. (How do you prove that that apparently useless doohicky isn't staving off Cthulu's wrath?).

It's important to try to understand why something exists, but it is also important to understand that the reason often is that it was either unintentional or silly, but you won't be able to prove it because it isn't documented and the person responsible is gone/unknown.

akavel• 11 years ago

A corollary is that actual bugs tend to become the sturdiest, longest lasting fragments of code. Because they're by definition incorrect, so no one can really fully and correctly "describe why they're there", and thus people tend to feel afraid to fix the code ("dangerous and risky to fool with it"), because maybe that's not a bug? (and thus "someone might punish me later for touching this code")

ctdonath4• 11 years ago

"I don't understand it" is an admission of ignorance, not wise authority justifying condemnation. Few seem to understand this.

If you can't concoct a strong* argument favoring the opposing view, you don't understand the issue well enough.

*Edit: as suggested, it's "strong" argument, not necessarily "convincing". If you can make a convincing argument for the other side but not for your own, perhaps you need reconsider your stance.

mwfunk• 11 years ago

This is the most concise expression I've heard of something I've felt was a fundamental truth for a long time, thank you!

lotharbot8• 11 years ago

slight caveat: I think you can understand an issue very well and be able to concoct a strong argument in favor of the opposing view, which is nonetheless not very convincing. (Sometimes you may also be able to concoct a strong but unconvincing argument in favor of your own view. Some issues are like that -- the best arguments on both sides still aren't really convincing.)

Domenic_S• 11 years ago

admission of ignorance

And although "ignorance" has a negative connotation, I argue it's a neutral thing unless otherwise modified, ie. as opposed to "willful ignorance".

The healthcare field uses the gentler term "knowledge deficit" regarding patients; I've taken to using that term in engineering contexts as well.

tomjen32• 11 years ago

I can't make a strong argument for the Earth being flat, does that mean I don't understand the question well enough?

I can't come up with a single coherent good argument against gay marriage, does that mean I don't understand the question?

I think your rule is very useful as a guide line, but please don't make it a rule. There really are policies and ideas that are one sided.

legulere• 11 years ago

Understanding is also important when instead of getting rid of something/not copying some tactic you want to copy it. I once was in some cheap cinema where instead of the heavy red curtains they had some structures they painted red on the wall. My guess is that you have the curtains on the wall for a better sound quality, the sound in this cinema was really bad because of reverberation.

The only problem is that it's sometimes not possible anymore to find out why something is the way it is.

mechadock• 11 years ago

There's a very similar notion in negotiation; that you should understand why your opponent is willing to grant a specific position before you accept it.

mfringel• 11 years ago

The primary issue with Chesterton's long-revered fence is that puts the entire burden of proof on the person who desires to take it down.

What if the fence was put there for purposes of adverse posession? Those who would defend the fence, absent any documentation, would ascribe the most noble uses to it, as their desire is to maintain the status quo, or at least the status pro se. They would, to use Chesterton's term, 'go gaily up to it' and defend it with their lives, saying "clearly this fence always existed, and thus should always be."

Chesterton's 'modern reformer' is a strawman as bald-faced as any, and it's all too easy to use "Chesterton's Fence" as a defense of mindless conservatism.

maxerickson2• 11 years ago

Isn't it just a nice little parable intended to clearly transmit the idea?

fecklessyouth2• 11 years ago

What if the fence was put there for purposes of adverse posession?

Answering that question is the very reason for pausing to investigate its purpose. Questioning past purposes before reforming is not equivalent to "defending the fence absent any documentation." Your own characterization of this "mindless conservatism" is itself a straw-man of the behavior Chesterton's fence recommends.

The alternative is an endless cycle of deconstruction that impedes any progress, as every possible step or rational for it brings a possible mask to power. The Nietzschean project, though fun, has produced nothing besides criticism, and would never succeed in taking any kind of action about said fence.

run4yourlives23• 11 years ago

the entire burden of proof on the person who desires to take it down.

That's exactly the point though - If a person can successfully defend the existence of a fence against your insistence for change, they've not demonstrated that the fence is required beyond all doubt, they've demonstrated that YOU are not capable enough to be the one that changes it, since your knowledge of the domain can't even defeat a "silly" insistence of maintaining the status quo.

fecklessyouth2• 11 years ago

Anyone interested in the necessity of understanding history for philosophical and moral thinking ought to read MacIntyre's Whose Justice, Which Rationality. It argues that a "history-constituted tradition" is the inevitable system in which we make intellectual progress, and analyses the major movements in philosophy as support.

http://www.amazon.com/Whose-Justice-Rationality-Alasdair-Mac...

_petronius2• 11 years ago

I ran across a slightly more polemical phrasing of this principle once, while googling for something unrelated. I've long kept it in a file on my work desktop as a sort of reminder to myself:

"There are often very valid (and even necessary) reasons for why certain things are done in certain ways; these reasons often become clear to us only after we have more deeply investigated into the full details of how things work, whereas our first, less than fully informed reaction may be to regard them as silly, or to attribute them to the wrong cause or agent, or to entirely misunderstand them." (John H Meyers)

TheOtherHobbes2• 11 years ago

This statement worries me. It's equivalent to small-c conservatism and sounds suspiciously like "You don't know enough to have an opinion" - which is perfectly fine when it's reality-based, but can also be an easy excuse for enforcing groupthink when it's not.

There's also the implication that alternatives have already been tried. But what if they haven't? How are you supposed to tell?

Clearly you can't, unless the alternatives have been documented historically. If there's no documentation you're firmly in the land of convention, tradition, and opinion - not effectiveness.

I'd prefer a model that assumes running a start-up is like running an experiment on a market. You probe the market with a complete package of technology, marketing, networking, and funding. Then you assess the results. And then - the hard part - you try to work out which parts of the package aren't working, and design a new experiment.

You absolute should examine competing models for validity, and you absolutely shouldn't assume they're wrong because you're just that awesome.

But assuming they're a better market fit without reality-testing and evidence makes as little sense as assuming they're bad at what they're doing.

If that were true, there would be no room to innovate at all.

ombudsperson2• 11 years ago

Might depend a lot upon who erected the fence in the first place: is it a fence in your mind, after your considered opinion has decided the path it wants to take? By all means, tear the fence down in your mind first, you might get an inkling as to why the fence exists in the first place. Of course, if the law or institution emanates from the people who think others' thoughts, you may be able to figure out whether you need the fence there or not faster

wfo• 11 years ago

I've seen this very often here when the topic of regulation comes up. Some think that any regulation of any kind is literally evil and exists for no reason other than to get in the way of "innovation". They demand to repeal them (or just ignore them, breaking the law) without understanding what they are for, just because they don't fit a particular business model.

Confusion• 11 years ago

Problem: sometimes (often?) the reasons for the fence are so old and so far from relevant today that a cautious reformer cannot reasonably figure out why it's there (unless he first becomes an historian). If we insist he first understands the original reason, the result is that the fence remains, at much cost.

If you've ever worked on legacy software, you know what I mean.

jasode• 11 years ago

When I read the essay, I thought of P Thiel's question of self-reflection that analyzes in the reverse direction:

"What important truth do very few people agree with you on?"[1]

I interpret "truth" to really be a highly-opinionated belief rather than something like "2+2=4". In other words, what factors do you believe in that would make the business model successful that outsiders would dismiss as insane or stupid?

(On trivia related note: I notice the blog as the title of "stupitidy" instead of "stupidity" so I'm not sure if there's an inside joke I missed.)

[1]http://fortune.com/2014/09/04/peter-thiels-the-challenge-of-...

mnw21cam• 11 years ago

Love the (deliberate? ironic?) mis-spelling of stupidity in the title/url.

bsdpython• 11 years ago

The title is correct but the URL is misspelled. I assume they fixed the title after publishing so in that case the URL won't change. Clearly a typo in the original.

akharris• 11 years ago

Wish I could claim to have been either deliberate or ironic...but I actually got careless. Thanks for pointing it out!

dceddia• 11 years ago

This is a great thing to consider, and I think this presumption of stupidity bleeds over into other areas of life too.

Developers: inherited code is considered guilty until proven innocent. Or maybe more accurately, guilty until you've rewritten it. Surely the old developer had no idea what they were doing.

The "other faction": Democrats/Republicans, different religions, rich vs. poor people... most generalizations about the faction you don't belong to start off with thinking "they're so stupid". "Look at those Republicans/Democrats. Can't they see that Trump/Obama is just lying through his teeth?"

Bad actors: The presumption of stupidity carries over into the way people think about computer hackers and terrorists and the like. You'll see stories about how "those terrorists are learning how to use cell phones to detonate bombs!" or how "criminals are migrating online to prey on people with phishing attacks!" The underlying assumption is that they're stupid, but getting (dangerously) smarter.

I think we'd make a lot more headway in most areas by assuming our competitors, detractors, and wrong-doers are probably already pretty smart.

pacala• 11 years ago

Inherited code is considered guilty until proven innocent. Or maybe more accurately, guilty until you've rewritten it.

I'm facing this dilemma right now, and I find it a whole lot more nuanced. The previous dev definitely had the right ideas, but he also produced XXXXX lines of code. I would take me years to wrap my head around the reasons for all these lines. Is this a bug or a feature? It would take me weeks to rewrite it, drop 75% of the corner cases for which I have no test case / clue why they have been coded. Then become the grumpy old dev that says "no" to 95% of feature requests and keep things simple. At the risk of my job.

swanson6• 11 years ago

You might find value in exploring the idea of Characterization Tests or Software Seams.

arnorhs• 11 years ago

Often the reason can also be evolution. Where the code started off doing something specific - but then as it evolved, more features were added, approaches changed, the code evolved to do something totally different.

Also "starters" - people who are good with getting something shippable fast and iterating on features and not the same people as the ones that write truly great code - simply b/c they're driven by different things.

But it's very situational.

brandonb2• 11 years ago

To generalize: competitors in a market usually behave rationally, and what looks like "stupid" behavior from afar may actually be unseen incentives.

Which suggests a test for your understanding of a market: can you map out the incentives and explain why what looks like apparently-irrational behavior is happening?

For example, in healthcare, we waste 30%+ of the $3T we spend each year. Much of that waste is due to hospital readmissions for an ongoing condition like heart failure. Startups sometimes try to fix this by developing a special machine learning algorithm to predict readmissions and apply an intervention. But even when the technology succeeds, the business fails: hospitals charge for readmissions, so there's an active disincentive for the hospital to buy the product. (That is now changing with ACOs, and a change in incentives is an opportunity for new companies.)

StavrosK3• 11 years ago

I don't understand your comment. You say "we waste 30%", but then say "hospitals charge". Who's the "we" who's wasting 30%, and why are startups trying to sell something to prevent X to the people who make money from X, rather than to the people who lose money from X?

brandonb2• 11 years ago

"We" referred to US healthcare spending [1], and the party with the right incentive for this problem is the payers: Medicare, Medicaid, and private insurance like Aetna, Blue Shield, United Healthcare, etc.

While the payers have the incentive to reduce readmissions (saves them money, leading to lower insurance premiums for you), they don't usually have the access to do so -- they're not the one seeing the patient or prescribing medicine.

The payers could, of course, try to change the way that THEY reimburse the hospital to align the incentives. For Medicare and Medicaid, that requires a law--which is why the Affordable Care Act is creating opportunities for new startups as it rolls out. For private insurance, I think they'd like to change reimbursement, but they have relatively little market power compared to healthcare providers: http://rockhealth.com/wp-content/uploads/2012/12/Kocher-et-a...

[1] Completely different set of incentives (and thus problems) in other healthcare systems.

glibgil• 11 years ago

A readmission is basically sending someone home early or failing to care properly for them on an out patient basis after they leave the hospital. Sending someone to the hospital twice when you could send them once is wasteful.

nradov• 11 years ago

It's not just ACOs. CMS is also changing the reimbursement structure to penalize hospitals that have unnecessary readmissions.

https://www.cms.gov/Medicare/medicare-fee-for-service-paymen...

ropman76• 11 years ago

The hospital admission/re-admission policy (or lack thereof) is exactly the type of thinking that this article is addressing. ER/Urgent Care doctors have a list of criteria for hospital admissions that are not always present at the time but may comeback later. If they are use more caution then required a hundred dollar medical bill may turn into a several thousand dollar hospital stay over a minor issue. Doctors are dealing with imperfect information when making a diagnosis so to say that they need an "algorithm" is trivializing a complex problem. There is room for a lot of improvement in terms of helping medical staff make better decisions with more complete information. I am uneasy that some of the new regulations in this area will end up handcuffing doctors in ways that will not always be in the best interests of the patients Note: not medical staff. Many years ago I processed claims for an insurance company and had some long and heated discussions about this topic with hospital staff. If some of these new regs are like what my oversized insurance company had, it will not end well.

kesor• 11 years ago

An excellent keynote presentation showing this in the public-home-maintenance business in the UK is was presented by John Seddon at https://www.youtube.com/watch?v=hbNsQFd8DQw

RRiccio• 11 years ago

Agree. Same can be said about countries. Laws/incentives vary drastically and change what the "correct" strategy is from country to country.

QuantumRoar2• 11 years ago

Really? Is that the way some people reason about their competitors? Something like: "Oh, I guess they're just stupid and we're so smart."

Who in their right mind would do such a thing. Even if the competitors really are stupid (who knows?), it doesn't give you any advantage to assume that.

lewisjoe• 11 years ago

No, it isn't that way. It mostly ends up as a consequence of initially over-estimating the competitor. So your mind looks for their tiniest shortcomings to make sure that they do things wrong too.

Once it finds such a thing, subconsciously the mind registers the competitor as weaker than they actually are, to redeem your mind from the uncomfortable prick of a all-powerful competitor. It doesn't happen for everyone though. It takes a bit of rationality to overcome such conclusions.

I personally know people who'd read and stress negative reviews of their competitors' products, and eventually conclude that their competitors are not really thoughtful in their decision making.

TL;DR - Yes.

rwallace• 11 years ago

Ah, metaness!

So can we assume people who assume other people are stupid, are stupid? Or is there some benefit from doing things that way that might not be apparent at first sight?

One possibility that comes to mind is the substitution of parallelism for serial processing, or put another way, letting the world be its own model. Instead of one startup spending a lot of time thinking and researching (and maybe missing the market window if there was one), let ten startups just assume and go for it. Maybe nine will be wrong and fail and one will be right and succeed.

QuantumRoar2• 11 years ago

Instead of one startup spending a lot of time thinking and researching (and maybe missing the market window if there was one), let ten startups just assume and go for it. Maybe nine will be wrong and fail and one will be right and succeed.

But that'd be something like evolution or anthropic computing. In order to find a solution to a hard problem, write down something. If it is not a solution, kill yourself. Conditioned on looking at anything at all, you look at the correct solution.

Or we could use the distinct advantage of humans and think about it properly.

thenomad• 11 years ago

An important line in the piece:

"Of course, just because you presume intelligence doesn't mean that every decision made was smart."

I'd rephrase as follows: it's unwise to assume stupidity on the part of your competition, but it's very wise to allow the possibility of stupidity.

With the corollary that if there's an inexpensive way to capitalise on that stupidity if it exists, it's probably worth trying, just in case the thing that's walking like a duck and quacking like a duck is in fact a duck.

As a tangent to that - the chances that assumptions of stupidity are correct go up in direct proportion to your level of domain knowledge.

I see a lot of non-film people say "the movie industry does $FOO and that's really stupid", for example, and 95% of the time, they're wrong and there are good reasons for doing $FOO.

However, I also see people who know the film world (including me) say "a lot of / most filmmakers do $BAR and it's dumb" - and $BAR has a considerably higher chance of actually being a dumb, common mistake.

mcphage• 11 years ago

A little more succinct might be: "It's unwise to assume stupidity on the part of your competition, but don't rule out stupidity."

danielweber• 11 years ago

I've experienced this myself.

Sometimes what our competitors were doing was stupid, and we ate their lunch.

Sometimes what our competitors were doing was the only way to really run things, and we had to adapt to follow them.

PaulHoule• 11 years ago

One thing I learned the hard way is that if you are on the right track, your competitors are probably barking up the same tree and are further along than you would think baes on what is public.

For instance there was a period of many years where both Google and Bing image search were embarrassingly bad and I was able to build something far better for a certain range of queries.

It took me a year to build out my system but in that year, Bing and Google both improved dramatically, so my demo comparing results with them was no longer impressive at all.

marcosdumay• 11 years ago

Technology has some phase changes, where it suddenly changes from "doing X is extremely hard" to "hey, X is actually very easy to do!" without any ado, and no obvious reasons.

There are too many histories where after a long time of nothing happening, everybody suddenly starts working on the same problem, without any kind of coordination.

codyguy2• 11 years ago

I am confident of the value I deliver but I don't call my competitors stupid. One person being correct or a winner doesn't mean others are stupid. Maybe some people can deliver value where the competition doesn't. Could be due to some leverage or insight or creativity.

jacquesm• 11 years ago

I tend to err on the side of caution with stuff like this, for instance when inheriting a code-base I assume the previous author actually knew what he/she was doing. But sometimes (not often) that can work against you as well. For instance when after spending sufficient time with said codebase you realize the original writer was entirely out of their depth and this was likely the first time they'd attempted to write something this complex.

But more often than not it is the presumption of intelligence that pays off.

jleader• 11 years ago

A lot of people seem to be unaware that companies survive by satisficing. That is, you don't have to do everything "right" to succeed in business. You just have to do most things well enough to not fail (don't break the law, don't forget to file your paperwork, pay your bills, etc.), and a few (one?) things outstandingly enough to win customers.

We've been brought up in school to think we have to get nearly every answer right on the test in order to get a good grade (and get more than half of the answers right just to not flunk out). In the real world, getting one right answer, and not screwing the rest up too badly is often enough (and sometimes only barely achievable!).

So maybe your competitor did something "stupid" because they're stupid, or maybe it's because that thing doesn't actually matter that much, and they're focused on doing something else incredibly well instead.

csense• 11 years ago

This sounds like an instance of the fundamental attribution error [1]. It's a known human cognitive bias to blame others' failings on internal characteristics while seeing your own situation as more of a product of external influences.

[1] https://en.wikipedia.org/wiki/Fundamental_attribution_error

griffinmahon• 11 years ago

Also relevant is the reversed phrase, "Never attribute to stupidity that which can be adequately explained by X (some other reasonable cause)."

k__• 11 years ago

I had the experience, that things people identify as "stupid decisions" are often just "economical decisions"

For example, a company I worked for had the best technology, but bad UI and the competitors had good UI, but their tech was old and inaccurate.

For years we thought they were imbeciles, because they didn't update their tech and we would smash them in the future, because they cannot catch up with us.

But in the end the customers bought the software with the better UI and didn't look behind the scenes.

So their decision was logical. Why pour money and time in parts of the software when noone wants to pay for this.

jhonovich• 11 years ago

It is valuable to determine what technology, not available in the past, would cause a reasonable insider to change their decisions if they could implement that technology.

normloman2• 11 years ago

Founders presume the stupidity of the competition because they're arrogant. Silicon valley, with it's notion of creative destruction and disrupting the establishment, encourages arrogance. We're blinded by the notion that new always trumps old, so we never consider that the established industry has reasons behind how it runs.

Patrick_Devine• 11 years ago

I agree, but I think this is what makes SV wonderful. You have to be somewhat naive to think your approach is going to be better than the status quo, particularly when your competition is well established, and probably well capitalized.

The reason why big companies can't turn on a dime is that they have a lot of people who work for them, and an existing customer base. It's difficult to align all of those people on a new, and potentially better way of solving a problem. It's not that they are collectively "idiots", it's that they have an established business with an established track record. This is quintessentially the Innovators Dilemma. Technology markets almost always move downmarket to the cheaper solution which at first looks like a "toy".

normloman2• 11 years ago

Yes! I wish more people would actually read that book instead of throwing the word "disruption" around vaguely. After reading it, you'll realize the established business almost always has the advantage, and start ups only succeed in certain environments.

Bartweiss6• 11 years ago

This is an excellent observation. When you encounter a suboptimal system, there's a substantial chance that it either produces some unnoticed benefit or results from some coordination problem that can't be overcome by "just not doing that".

In either case, successful solutions have to work around the gap in the system rather than simply charging into it.

simonswords82• 11 years ago

Great article, taking the thinking further...

Everybody watches their competitors, it's entirely natural. It's solid advice to study them, and try to stay/get ahead of them where possible. This doesn't just apply to app features, but every facet of the business across many disciplines (sales/marketing/development/back office etc).

On the other hand, building a business based solely on a competitor's business decisions and not doing your own homework is the path to madness. We might take inspiration from our competitors, but we always check in with our customers next to make sure they actually want the feature. It's also our job to get feedback on not just what we're doing but also how we're planning to do it, as our users might have unique business requirements that our competitor's users do not.

riemannzeta• 11 years ago

Despite the flaws in the rational model of economics and the efficient market hypothesis more generally, I have always been fond of the more humble, observant posture it gives us in considering others' behavior. The flaws in the rational model are well-known. But as a presumption, it certainly works better than its opposite.

btbuildem• 11 years ago

Those folks are just being introspective, that's all..

jusben1369• 11 years ago

I think this is the classic problem of advice giving that's so prevalent in the startup community today. It won't be too long until we're all praising a tweet or article that talks about only those who brashly challenge the status quo and assume the entrenched players are vulnerable, bogged down with legacy issues and fat and lazy on an existing revenue stream are ripe for the disrupting. Those who sit back and say "Well maybe there's a reason they do things this way I'm not sure" aren't bold enough and won't be the recipients of the spoils of disruption.

Not that this article is bad. It's just datapoint 107 that a founder has to reconcile with all the other competing advice.

amsilprotag• 11 years ago

Many times throughout the Entrepreneurial Thought Leaders podcast[1], a founder expresses the sentiment, "If I had known how hard it was going to be, I might not have started the company." The difficulty discounting has also been cited as the reason why many successful founders are young and inexperienced. So while hubris can lead to disastrous consequences for engineers, it could be viewed as an asset for inciting action.

[1] http://ecorner.stanford.edu/podcasts.html

brianmcconnell• 11 years ago

The important thing in analyzing a competitor's behavior is to understand the incentives motivating that behavior.

A common example in startupland is a company whose senior management has short term incentives that reward a fast exit over long term growth. That company may very well behave in ways that appear dumb to competitors with a long term focus. But if the "seasoned" CEO and his cronies get their compensation even in a mediocre deal, why bother trying to build a company for the ages when they can cash out, rest a bit and land in a similar situation at the next gig?

joseraul• 11 years ago

People may have a sense of superiority, especially smart ones. Chess world champion (and genius) Bobby Fischer once said: "My opponents make good moves too. Sometimes I don't take these things into consideration".

noobplusplus• 11 years ago

A nice article to manipulate your competitors from doing a market survey!

DrNuke• 11 years ago

Uh, I was thinking this article is more business-side than operational? Easily put: a business exists if it stays afloat and fences are often the way not to go under, even if they appear stupid from outside. Many times, fences are the only common ground between sellers and buyers. Removing fences is, a lot of times, pretty stupid = you geniuses operate at a loss and survive from artificial money or VCs until you are allowed to.

tomcam• 11 years ago

I have never disparaged my competitors, but I'm a small fry. I can say that during four years at Microsoft (1996-2000, Development Tools group) I never heard the products of competitors disparaged that way. In fact, there was a weekly presentation of competing products and invariably the interest was in where we were lacking, not what was bad about them.

Likewise when customers came to visit us at trade shows my boss would sit politely through their compliments, then immediately jump to the question "So what don't you like about our product?"

Fast forward to today. I'm friends with top people at a Really Big Guitar Company and a Huge Amplifier company. Even in private, these C-level execs show nothing but respect for products of their competitors. They are not ashamed to own and even personally use said products (especially vintage ones).

It seems to me that dissing your competitors even privately can make you dangerously blind to the challenges they pose to you, set a bad example for your employees, and also restrict your job prospects should you decide to work for a competitor one day.

stuxnet79• 11 years ago

Interesting - thank you for sharing this.

colordrops• 11 years ago

This mindset can be seen in other systems of thought. There are a large number of species known to man, yet we somehow think that we are the pinnacle of the tree of life, despite the fact that this is statistically highly unlikely when only taking into account pure numbers. Considering the dimensions now accessible to us that were completely unknown even 500 years ago, it doesn't seem a large leap at all to posit that there are other dimensions we are currentlt unaware of that contain life forms in the same tree as ours that are far more advanced and perhaps even invisible to us.

lpolovets• 11 years ago

This is so true. As a investor, I hear a lot of pitches where the founders say their competitive advantage is that they execute better (the flip side of believing everyone else is dumb is believing you're especially smart). Do you know who else claims they "execute better"? Everyone. That kind of attitude usually reveals that a founder doesn't have a real sense of what makes their company special and defensible, and is a bit of a yellow flag for investors -- well, at least for me.

donarb• 11 years ago

Once again, xkcd explains all.

https://xkcd.com/277/

ZoeZoeBee• 11 years ago

This works well when considering people in your own field, you know what it took for you to get there and you can assume they've had similar experiences.

However when you are considering the general public it is best to presume stupidity and design with that in mind.

shurcooL• 11 years ago

Agree with the first 3 paragraphs so much, well put.

bro-stick• 11 years ago

Not much else to add. Presume a larger, smarter, better funded team is working stealthily in another office somewhere to kick our ass... anything else is complacency. Worse, following that path, leads to hubris at some juncture: excuses rationalizing cutting the wrong corners or shortchanging the customer that could prove fatal in a game of inches in the marketplace. There are at least a quadrillion ways to fail, and 99.997% of them will be my doing. Rational paranoia is healthy, because your product/service needs to be so well-regarded by people other than the team or supporters that it demoralizes potential adversaries that they don't want to compete. Even then, it still may not be currently as good in other key areas of focus as a competitors.

(Btw, the Thiel view of not picking fights you can't dominate and Buffett's sticking to defensible business models is a good mindset to calibrate a venture's success per risk gut perception. And with timing, team and execution you might just make something that hits.)

andyidsinga• 11 years ago

see also: Sarah Silverman's bit on scientology and things that sound weird.

nitwit005• 11 years ago

This is probably not a correctable problem. People don't start businesses if they think the competition is highly competitive and intelligent. They start a business of they perceive a weakness in the market, or believe they have a unique capacity to succeed.

tr352• 11 years ago

Working in academia, I've experienced this too. I've read work in my field of research that I dismissed as bad work or not worthwhile, simply because I didn't fully understand them. Too complicated, strange background assumptions, not well motivated, etcetera.

Then later, while developing my own work, I find that I end up with the same complications, that I'm forced to make the same background assumptions, and I have the same difficulty in motivating my choices.

Presumption of stupidity

Related Stories