Innovation heroes are a sign of a dysfunctional organization

https://steveblank.substack.com/p/why-innovation-heroes-are-a-sign

by sblank • 2 years ago

348 166 2 years ago

steveblank.substack.com

God yes please give us an "innovation doctrine."

This is certainly an interesting article to read about but I'm not sure his suggestions or analysis are that substantive. Complex institutions develop rules to simplify decision making and streamline information flows. They choke otherwise. "Develop an innovation doctrine" isn't really effective advice.

Mathnerd3143• 2 years ago

There are various articles on developing a culture of innovation, e.g. https://hbr.org/2019/01/the-hard-truth-about-innovative-cult.... Probably some books too. Even ChatGPT probably has decent advice. Management is not technically complex, it just requires putting in the work. But of course it is not easy, e.g. the first advice in the HBR article is to fire incompetent people, whereas the example here was government where firing incompetent people is notoriously hard.

nathan_compton2• 2 years ago

chatGPT probably has decent advice

If chatGPT has anything other than banal platitudes, I'll eat my hat.

OldGuyInTheClub2• 2 years ago

I agree. I am in aerospace. When times are good, there's money to bring in speakers like this every couple of months to talk to our management. We get the slides afterward and they are refreshingly free of content. All of them essentially repeat the cliche, "Think outside the box!"

We even got one of those Innovation Forums where people could propose ideas, get them upvoted, with the promise of tchotchkes at the end for success. Not a single one got funded. Every efficiency improvement, every streamlining suggestion affects someone's budget and headcount. When that person has to approve, nothing will happen. And, in this industry, a lot of rules are actually law. They have to be followed.

But, the consultants and professional keynote speakers seem to be making good money off of it.

setgree2• 2 years ago

I favor the general point here, but the leading anecdote doesn’t really fit with the lessons learned. A government organization generally does not need an innovation doctrine to avoid being outcompeted because they are a monopoly provider. They maintain that monopoly through force.

If you want to get a government agency to perform better, fix the incentives.

MeetingsBrowser2• 2 years ago

A government organization generally does not need an innovation doctrine to avoid being outcompeted because they face no meaningful competition

Maybe true for some government organizations, but not true on the whole.

Governments spend billions (and sometimes trillions) on innovation to compete with each other. Collectively, government programs probably account for 99.99% of all "innovation" spending.

And they have the highest stakes when it comes to being "outcompeted".

tredigi2• 2 years ago

Doesn't the leading anecdote give an example of a (dis)incentive that needs fixing? That it takes 10 months and lots of head-banging and then you get a $100 bonus, that would certainly disincentivise me to do any type of innovation.

renewiltord• 2 years ago

POSIWID implies that unchange is the desire of the government. People desire unchange and this is a tool they use to try to force it on everyone.

setgree2• 2 years ago

It does, but the last lesson learned is:

All large organizations – both government and corporate—need an innovation doctrine or else risk being outpaced by competitors.

I don’t think the anecdote fits conclusion. The reasons to innovate are many, but being outcompeted is not that salient to, say, an IT person in some agency.

toomuchtodo2• 2 years ago

Incentives matter, certainly, but you must suss out what those incentives are and what they need to be to arrive at the desired outcome. Look no further than the US Digital Service and 18F (within GSA). You are not paid top dollar, but you are put in front of meaningful work and enabled to deliver (although that in itself is an incentive for the practitioners recruited). From the bottom of the impact report I cite: "We need you. Let’s help millions of people together." right above the Call to Action to apply.

The USDS is enabled to succeed in this mission through the support of the Executive Office of the President, the equivalent of corporate executive sponsorship. Culture comes from the top.

https://www.usds.gov/impact-report/2024/

https://www.usds.gov/impact-report/2024/by-the-numbers/

(full disclosure: went through a USDS interview cycle and was extended an offer, no other affiliation)

merely-unlikely• 2 years ago

The USDS is enabled to succeed in this mission through the support of the Executive Office of the President, the equivalent of corporate executive sponsorship. Culture comes from the top.

That sentence is the essence of generic corpo-speak. Doesn't really mean anything without specifics.

betenoire• 2 years ago

I immediately thought of the government. I interned for the transportation department. I was one of these "innovation heros". It was a budget thing as explained to me. If I didn't have work to do, they told me to do homework, I wasn't even allowed to help people on things outside of my department, since they'd have to report on that in budgeting details and be accountable of it to the taxpayer.

kurthr• 2 years ago

Well, except when they do.

Can you really imagine a military that didn't have and celebrate heroes?

If a large organization can't have those that sacrifice themselves and break rules to achieve the greater goal, then they're unlikely to succeed against a significant foe. At the same time communication within large organizations is challenging and leadership is unlikely to know what the challenges at the front line are. Fostering all innovation is as likely to lead to regularly scheduled mediocre "improvements" and "features" that nobody really wants in order to meet whatever metric is in place (to the detriment of what is not explicitly measured).

The argument of the article seems to be, just be so good and well directed by both senior and middle managers that exactly the "right" innovations occur within the process. That likely means those in the trenches aren't getting what they need.

The most effective rapid innovation method I've seen (though painful and challenging to implement) is having separate teams competing to several performance milestones (which allow more generic goals and targeted metrics). At the milestones they share their results and innovations, which competing teams can then use/combine to compete against them. Management needs real goals with known tradeoffs, 2-3x more people than a single team, and it's stressful hitting deadlines knowing that failure is an option. The failure mode is putting all the "best" people on one team which is supposed to win, though I've seen even that get broken by a team of underdog "heroes" who embarrassed the chosen team, and luckily senior management rewarded that.

It's similar to "red" vs "blue" pen-testing or wargaming, but you can have more than 2 teams and the goals can be aligned against the status quo (sometimes a tweaked current solution is the winner).

chuckadams2• 2 years ago

"Innovation Doctrine" sounds like something you put next to your Mission Statement. How about fostering an internal discussion board where employees can pitch ideas and get hooked up with others who know how to implement them? If you need guard rails around the anarchy, then you can tie action items to a ticket tracking system that's readable by the whole company. I can file a bug in JIRA against any product my company makes, why not the company itself?

MeetingsBrowser2• 2 years ago

Because in large companies, any helpful suggestions get lost in the noise. Everyone has opinions on how things could be done better, but very few of them are good.

chuckadams2• 2 years ago

The discussion forum still has to be managed (I said "anarchy" facetiously) and at least minimal standards of professionalism would still have to apply. And if the board still devolves into a swamp of griping and bickering, then well, they can nuke it and at least say they tried.

Most ideas are indeed crap, and the good ones have to be picked out. But reducing the total number of ideas doesn't raise the percentage of good ones.

justin_oaks3• 2 years ago

I remember a company I worked at that an internal website for suggesting ideas. The problem is that the management never looked at it. And because of that, employees stopped posting to it. And that was that.

praptak• 2 years ago

I could name some companies where you don't need to be an "innovation hero" to feel like the "innovation hero" from the anecdote and "Please ask next quarter" seems to be the motto :)

hayley-patton• 2 years ago

"Unhappy the land that has no heroes!" "No. Unhappy the land that needs heroes."

ghaff• 2 years ago

The basic idea here seems pretty sound. I remember years ago critiques about some Microsoft "Heroes" campaign along a similar line. While organizations often have superstars or whatever you want to call them, if you require them to at least minimally function you're probably doing something wrong.

That shouldn't mean that you don't celebrate those superstars. They're not a bad thing certainly--which is probably where I differ with the post a bit. But understand that we shouldn't be depending on them all the time.

Jun8• 2 years ago

This a million times! I’ve been in the position of the “innovation hero” but also was unfortunate enough to work on “innovation pipeline” implementations within a large company. These never work because:

1. (99% of) employees don’t care. They have their own job to do and working on other things is an eyebrow raise from their manager (see 3). And what’s the e benefit? Mostly it’s a pat on the back or some “points” in the company award system that you can use to buy shitty merch at the end of the year.

2. Executive leadership doesn’t care because for the most part they don’t trust their own tech team to innovate. Why take the risk when you can buy a startup which comes with a bona fide certificate of innovation.

3. But the real problem (as commonly identified) is middle mgmt, who not only don’t care but are generally hostile to nonstandard work. The reasons for this are complex, partly it’s the aging manager suffering from Peter Principle, partly it’s the fear of negative pushback from senior leadership.

PS: Steve is amazed, but 10 months for the sort of setup he describes which includes HW buy and setup, in a Big Corp is very fast.

cogman10• 2 years ago

This all, imo, is simply a trust problem primarily from leadership. Leadership does not trust the grunts to do productive work. So in order to make sure productive work is done, they build elaborate systems of cases, reviews, meetings, planning, scheduling, fighting, readjusting when the schedules are invariably missed, and finger pointing. All almost always completely devoid of input from the grunts.

The middle management problem is they are right in the worst place possible. They are removed from the actual work being done so they don't know what it actually takes to do anything and they are blamed for things not accomplished. Further, they are rewarded for every little stupid thing done. It hyperintensities them to do lots of small safe initiatives and vehemently oppose anything with any sort of risk. All while being almost completely disconnected from what actually needs to be done.

This all leads to a culture meant to squash innovation. Middle management isn't rewarded for implementing a grunt's idea, they are rewarded for delivering CEO initiatives. Anything that takes time away from that is seen as waste.

foobarian2• 2 years ago

Leadership does not trust the grunts to do productive work

Not only that, but unfortunately they are usually right. Without oversight the in-house team has high likelihood of building NIH spaghetti, which causes more problems down the line. To avoid the negative outcome leadership needs to be technically competent and resourced, and that's the other side of the coin - usually they don't have the expertise so in a way they also do not trust themselves to lead the project to a positive outcome.

danaris3• 2 years ago

This all, imo, is simply a trust problem primarily from leadership. Leadership does not trust the grunts to do productive work.

Which, IME, stems from a deep-seated classism that sees "grunts" today as being essentially no different than assembly-line workers in the Industrial Revolution: you're just a pair of hands who not only doesn't know enough to make changes in the process, you shouldn't even think of it, because it's not your place.

In this worldview, it's managers (and up) who have the education, intelligence, and breeding to know best, and lowly workers just need to shut up and do what they're told.

...This attitude is also responsible for a lot of other really destructive problems in the modern world of work.

jmull• 2 years ago

Mostly agree, but this

aging manager suffering from Peter Principle, partly it’s the fear of negative pushback from senior leadership

is not typically the reason, in my experience. I've seen a lack of full comprehension on the part of the team pushing the innovation as to the actual benefit and cost of the innovation to all the affected teams. And as a result, the lack of a plan to address those issues...

I've seen a lot of incomplete innovations, where the benefit is real and useful, but the plan leaves various concerns of different teams unaddressed -- no doubt due to the innovation team being unaware. Strangely, the innovation team often wants to push forward anyway, which is not good since the plan is basically unworkable with critical issues unresolved (I can tell they kind of think the issues aren't critical but only because they don't understand them -- leading with ignorance when the processes and products actually have to work at the end is always doomed to fail.)

The most common way plans are incomplete that I've seen is when they don't account for the schedule. The plan will take X time away from other work to implement, but the schedule for delivery of that other work isn't moved back X, nor are there other compensating measures. That's an unworkable plan, and any half-decent manager will push back on it.

(Schedule impact is usually a tough one... at least at larger places, in my experience, the high-level delivery schedule is negotiated at a high level, and it hard to change for political reasons. That means time for any innovation plans has to be included from the start. Yet slack in any schedule tends to get gobbled up at the team level or below, addressing their concerns -- who doesn't have tons of technical debt they are dying to resolve? There's a way to handle this, but it has to be planned for and done at a high level, and done correctly. The fruits of any innovation team that doesn't have this are gong to be minimal.)

specialist2• 2 years ago

Yes and:

As legendary methodologist Alistair Cockburn teaches us: People hate change. They really, really hate change. (paraphrasing)

not only don’t care but are generally hostile to nonstandard work

Most persons have an immunological-like response to any and all change. Our minds (personalities?) strive to maintain an equilibrium at all costs. Even when the status quo sucks. Even when we intellectually commit to changing. We keep getting pulled back into the rut, foiling our own best intentions.

This trait makes change very, very hard (for most people).

Now multiply that by 10, 100, or 1000 persons. Organizational (cultural) change is that much harder.

Noob me was belligerently intolerant of apathy and inaction. A total wrecking ball.

Elder me now appreciates notions like patience, deep listening, radical empathy. Like trying to empower people to improve their own situation, instead of always running them over.

tredigi2• 2 years ago

I agree to the overall tone, but there are also counter points.

One of them is the Google example. To get promoted beyond a certain level, you must have brought some new product over the finish line. Result? They have so many new things happening all the time, all of them suck, and then just move on to the next. Eg how many chat products do they need to invent before they settle on one and let it mature?

nutrie• 2 years ago

100 %. There's a sweet spot. I worked for both, start-ups and huge corporations. It's not one or the other, you want the right combination of maturity and fresh attitude. We often don't realize it can take years to steer back to find the balance again, but not oversteer, which is usually the default :) Just as with any other organism.

adverbly4• 2 years ago

To get promoted beyond a certain level, you must have brought some new product over the finish line

This always confused me. It looked from the outside like Google does so many things right on the innovation front, but after some early success they have had a rough streak.

I'd argue that one thing that Google is doing wrong is gatekeeping promotions based on (overly) well-defined criteria such as new products. Goodhart's Law applies to this situation: you're sure to see lots of new products if its highly rewarded - a lot more than you'd see naturally. As the author mentioned - this might still be desirable depending on the market conditions, but there is a lot more to this discussion, and it's not entirely clear that Google has it wrong.

I'd argue that they emphasize comparability of assessment results(e.g. being able to quantify someone's output like a percentage grade in a course) over the actual relevancy of the assessment criteria/work/kpi to the company's bottom line(e.g. does the course's test actually prepare students for the real world). This probably comes as a by-product of the organization's heavily academic-focused staff - so it might actually be the best culture choice for them given that context - but it might also lose to companies that can successfully put a bigger weighting on the right "intangibles".

immibis2• 2 years ago

You would think that. I would think that. But can you actually name more than a handful of things Google created that were good? Because I can't.

There's search, obviously. Gmail, if that wasn't an acquisition.... What else?

laidoffamazon2• 2 years ago

The promotion thing seems severely overstated. At the higher levels for IC and management this is basically how all tech companies that build products are run. But you don't see this said about Microsoft or Amazon, even though they also have hundreds of new features and discrete new products per year.

My theory for why Google is different remains unpopular, however.

p_j_w• 2 years ago

My theory for why Google is different remains unpopular, however.

May we hear it?

dekhn• 2 years ago

beyond just launching a product, the launch has to have some sort of "impact" (at least, this was true in the time period where I was trying to get promoted, roughly 2010-2013). Something that is not perceived as impactful by the promo committee is likely not going to count towards promotion. It's the job of the employee and manager to document the "impact".

If your manager is a director or higher, they can appeal the promo denial and an appeals committee can be manipulated into giving a promotion. That's what happened to me- promo saw no impact to my launch. Then my director went to appeals and basically said "promote him, he's doing good work".

Everything about Google messed up my expectations and planning around career. To work anywhere else (a startup, or a pharma) I had to unlearn all the bad habits of self-promotion and cookie-licking and impact-demonstration.

Of course many people joke the best way to get promoted at Google is to leave, get promoted elsewhere, and return to Google at a higher level (using all your newly learned negotiation skills).

loceng• 2 years ago

So that will result in an "innovation hero" improving on Google's model so the fundamentals don't create such waste, no?

bearjaws• 2 years ago

Why Innovation Heroes are a Sign of a Dysfunctional Organization

Because often you can solve 99% of companies problems with boring software.

I am reminded of this blog post from earlier this week:

Most organizations cannot ship the most basic applications imaginable with any consistency, and you're out here saying that the best way to remain competitive is to roll out experimental technology that is an order of magnitude more sophisticated than anything else your I.T department runs,

https://ludic.mataroa.blog/blog/i-will-fucking-piledrive-you...

graemep• 2 years ago

Updating a spreadsheet with data pulled from another system sounds more like "boring software" than "experimental technology".

verisimilidude• 2 years ago

"Innovation" in this context does not mean cutting-edge technology. It just means changing processes to deliver better results. The tech is often the easy part, and there's plenty of room for boring software.

The hard part is navigating the bureaucracy and building consensus toward a change. This management-craft is where the clever thinking and emergent solutions are found and deployed.

jjk166• 2 years ago

A backhoe is orders of magnitude more complex than a shovel. A team struggling to dig a pit with shovels would probably benefit immensely from a backhoe. The idea that workers shouldn't get better tools until they can succeed without them is insane.

shermantanktop2• 2 years ago

Reminds me of the Drucker quote about “culture eats strategy for breakfast.”

The problem is that large organizations naturally drift toward inertia and ossification. If a forward-looking leader wants the culture to change, they are faced with a conundrum:

- use a strategy like this (e.g. some top-down “innovation center” approved at the C level) which reinforces the rigidity and process-oriented thinking that needs to change, or

- create an insurgent skunkworks group that hopes to prove a different approach via undeniable results. This usually ends with back-alley knives getting unsheathed.

Pet_Ant• 2 years ago

Innovation means potential for failure and waste. The larger the company the more it's investors want it for steady predictable returns. Even shrinking returns as long as they are steady and foreseeable. A large company becomes about control. Look at summer blockbusters. They know the demographics that will see it and have a pretty good (not absolute) idea of how much they can make.

This is all by design.

shermantanktop2• 2 years ago

This can be by design, sure. But I think it is also an inexorable tendency for people in large groups. Once your company is 10x Dunbar’s number, it is very hard for “we” to mean the entire company.

Angostura• 2 years ago

I think there is possibly a third way.

I’m currently working with the NHS in a Quality Improvement role in a hospital. The team exists to give grass-roots folks the tools they need to make change happen when they spot something in the system which is less than ideal. We offer training, help with setting aims, running projects, finding stakeholders, analysing data etc. These aren’t huge system transformation projects, in fact some of them can be quite small - but as a way of working it feels quite effective.

withinboredom• 2 years ago

in fact some of them can be quite small

The biggest changes always start with someone taking one small step towards their goal.

Some of the biggest software (used by billions of people) I've ever worked on started as a simple script made by a teenager.

Mathnerd3143• 2 years ago

what we just witnessed was leadership rewarding and perpetuating a dysfunctional and broken system.

It's not clear that it is dysfunctional. If innovation is not a particularly high priority, but risk reduction is, then the system is working as designed - all of the checks are necessary. Compare to the "innovative" Boeing-type company which streamlines production by removing all safety checks.

adolph• 2 years ago

Lacking a method to implement continuous improvement is the dysfunction.

“When [W. Edwards Deming] came to spread the gospel of continuous improvement in 1950, he was preaching to the choir. Toyota already believed in it. [Deming] simply gave them a process to better understand how to progress from failure. The idea of ever-improving coupled with what they learned from Deming—especially the Theory of Knowledge and shorter feedback loops via the PDSA loop, as well as the Theory of Variation and the accompanying statistical process control—let them succeed in their failure.”

From “Deming’s Journey” by John Willis. Solid recommended new read.

https://www.amazon.com/Demings-Journey-Profound-Knowledge-In...

nlawalker• 2 years ago

This is exactly why the common advice is to automate your own tasks on your own time, with your own resources, don’t tell anyone, and enjoy working less.

debacle• 2 years ago

One addendum to this that I have observed: In many organizations, no one is "in charge." If change needs to happen, there are 100 checkboxes, 100 reasons not to change, 100 people who are concerned about their career, resume, budget, department, relevance, etc.

For every "innovation hero" that wins an award, there are 5 who are "managed" into leaving, marginalized, or disenfranchised. The high nail gets the hammer. I have been into and out of the startup space for the last 20 years, and most of the times that I was an "innovation hero" it was because there was one person in a corner office who was using me as a proxy for a change they wanted to see happen.

The American system of organizational management is...basically glue.

justin_oaks3• 2 years ago

100 reasons not to change

I'm reminded of the article "Layers of Management == Layers of Veto" [0]. In the article the author explains that each layer of management is likely to veto each idea coming from below. Each idea that didn't come from above is "insubordination" and likely to be vetoed.

[0] https://slott56.github.io/2010_02_12-layers_of_management_la...

lawlessone• 2 years ago

There was some anime/manga I recall from when I was younger that had the phrase "heroes require bad things and/villians to exist."

This reminds of that.

EncomLab• 2 years ago

Relate to the four stages of employment: 1) This is the new person "X" - they are amazing and are going to solve everything! 2) "X" is pretty good, but maybe not as good as we thought. 3) "X" turned out to just be another average performer. 4) "X" is obviously terrible or they would have left for someplace that would treat them better than we do.

quesera2• 2 years ago

I've never seen this written down, but I've definitely seen it in action.

Experienced leaders try to avoid deluding themselves at step 1, and work to prevent things transitioning from step 2 to step 3.

But sometimes it's impossible (or I'm not experienced enough!). Some employees really do start off strong and then fade out, even if the incentive structures are "good" / "above market" / etc.

Honestly, I've seen the roots of this in myself too, in classes, jobs, relationships: initial enthusiasm wanes and at some point it's time to make a different decision. Personally I prefer to move on instead of stagnate, but some people are perfectly happy to ride the suboptimal until a decision is made for them!

Viliam12342• 2 years ago

Sometimes the new employee's superpower is all the good ideas they have seen in their previous jobs that they are now bringing to your company. But the fact that they work for you now means that they no longer get new experience about how things work in other companies.

Sometimes it is the fact that as a new employee, they are expected to still be learning new things, so they are given less work and responsibilities, which leaves them some time to think. Later, they will probably be given as much work and responsibility as possible, which leaves no time and energy to think about things. Especially with daily agile meetings, which make sure that everyone is only thinking about the things the management wants them to think about (hint: fixing technical debt is not one of those things).

btbuildem• 2 years ago

Not sure that you can fix a calcified bureaucracy with "doctrine". I sort of get the angle, I think -- if the org operates within a rigid rule framework, you need to speak their language to get anywhere -- so, doctrine is best, because everyone is used to being told what to do? I feel like that approach is counter to the spirit of innovation, that's akin to being forced to have fun; nothing truly innovative will come from it.

I think it's more of a lost cause, really. If you want to work on cool new stuff, don't work at a large org. I play the "innovation hero" role often enough, and the diversity of pushback we encounter is impressive. It ranges from thinly veiled hostility, the sdev equivalent of NIMBY, lies through omission, through nonviolent noncompliance, all the way to blithely unaware absurdity.

One amazing moment stands out to me - we were jumping through the usual hurdles as described in the article, trying to get a prototype to prod, and in one meeting the head of IT indignantly exclaimed "I am not here to solve problems!". To paraphrase a scene from the Big Short: he wasn't confessing, he was bragging. The top brass exist to stifle any and all deviation from the norm.

Another commenter in this thread makes a very good point: most of these innovation initiatives die stillborn, because the existing power structures exist, their MO is to maintain, and the C-suites would rather buy a successful startup than take any political risks internally.

entropicdrifter• 2 years ago

"Innovation doctrine" is a contradictory phrase on its face, really. Going, "Oh we need a static, unchanging set of rules for innovation" doesn't exactly sound like a good way to attract and retain innovative thinkers lmao

adrianmonk• 2 years ago

so, doctrine is best, because everyone is used to being told what to do?

I think they mean doctrine as in military doctrine (https://en.wikipedia.org/wiki/Military_doctrine), which is a somewhat different idea than, say, religious doctrine. Religious doctrine can easily get rigid and legalistic, and compliance with it can become an end unto itself. Military doctrine is something a military would use in order to make the organization effective at a certain kind of task or endeavor.

didgetmaster• 2 years ago

The problem with this particular anecdote is that 'innovation' and 'government agency' are used in the same sentence.

Most people do not equate any government agency with an innovative environment. Bureaucracy is the watchword where the status quo is not only encouraged but fiercely protected.

Any new hire or politician who threatens to reform the process is treated as the enemy.

MichaelRo• 2 years ago

> Why is it that innovations require heroics to occur in our organization?

Why do we immediately assume that innovation = progress? Sure, the things that SURVIVE are useful, but that's just the tip of the iceberg. The vast majority of ideas are just like mutations in evolution more likely to be at best useless and probably damaging in various ways.

You see, social constructs are not as dumb as they appear to be to the armchair intellectual. "Why, we should embrace innovation and immediately adopt any idiocy that Mary from accounting is suggesting as our global company policy". I assure you that by natural law, if "random idea from random guy" were profitable on average, we'd have a system that would encourage such ideas. The sad fact is that they aren't and will never be.

Friction (named in the article as "Dysfunctional Organization") is an unfortunate but necessary process which ensures "survival of the fittest", even among "innovation". It's as simple as that.

ahstilde• 2 years ago

Innovation is required for progress. The alternative to innovation is stagnation. The end result of stagnation is death.

dmvdoug• 2 years ago

Pithy but, alas, meaningless.

First, define “progress.”

Second, innovation is a new idea, product, strategy, etc. Stagnation is lack of movement or lack of growth. There are alternatives to stagnation that are not innovation. Consider satisficing.

Third, the end result of stagnation may be other than death. It may be a steady state. Surviving, but not growing, is not death.

fuzzfactor2• 2 years ago

Don't sell short progress that has already been made.

I would estimate that for most people born in the 21st century, before they were born there were already more promising undeployed innovations that can't be beat, than new ones that will actually be deployed during their lifetime.

That's not all I do, but just about. Experimentation & discovery is like that. When I started out innovation was still as highly valued as it should be, up until the mid 1970's when people lost interest since nobody could afford to deploy anything any more anyway, unless they were among the most well-heeled who had weathered the economic devastation.

esafak• 2 years ago

Evolution requires mutation before selection can take place.

novagameco• 2 years ago

I worked for a large company and started seeing opportunities for automation immediately. I proposed some solutions to my boss, and he told me that he agreed that these tasks could be automated, but that we have 10,000 other tasks that could be automated, and each one takes a few months to get the resources provisioned and also set aside developer (me) time to get it done, which could be spent on other projects.

What was interesting to me was the self-fulfilling prophecy of dysfunction: because there was so much manual process and red tape, the cost of fixing a particular problem is larger than than the benefits (i.e: time spent on the task exceeds time saved by automating it).

But because the tasks do not get automated, the amount of time required to fix things increases bit by bit due to the processes in place. The cost of fixing a task increases marginally every day, and so the cost/benefit ratio increases every day, becoming further justification NOT to fix things. At a certain point you have to look at the bigger picture and recognize that there is a much larger problem in your company than a few excel spreadsheets that could be better automated.

JohnMakin2• 2 years ago

So much feel this and have seen it many times. A complete unwillingness to spend a few hours to save hundreds of hours later, because too much is urgent. It's a little bit like a thrashing OS, too many competing resources so the result is everything slows to a crawl or breaks.

I've clawed my way out of situations like this but it does require some heroics in the beginning and probably longer hours than your role requires. I am the kind of person that will ask to slow a project down or delay it if it means we get a chance to do things correctly and in a maintainable way, but certain types in management will not really understand enough to completely buy in.

theideaofcoffee2• 2 years ago

I know that feeling all too well. And it's such a hard truth that devoting just a little time, letting some projects slip just a bit, to fixing those systemic problems could make lots of others go away, it's just next to impossible to get people to want to change. I've found because people don't want to, they want to keep doing what they're doing and are scared of anything new because that may mean either they'd have to retrain, or more pathologically, that their position is threatened.

It all comes down to the people. The right people can make all the difference in something like that, the wrong people make it miserable for the rest.

treflop• 2 years ago

I honestly don't know where innovation exists that doesn't require "fighting the system" to some degree.

There are a lot of bad ideas out there and the "system" is there to weed out the bad ideas. However, you can have too much system where it will defeat even the good ideas too, so there's a sweet spot.

IMO it's comes down to how much leftover budget (time or money) the organization has. You don't need processes or procedures for innovation... you need people with enough leftover money and time who can "screw around" a little.

Also a place where everyone is not grumpy.

justin_oaks3• 2 years ago

There are a lot of bad ideas out there and the "system" is there to weed out the bad ideas. However, you can have too much system where it will defeat even the good ideas too, so there's a sweet spot.

Innovation means figuring out which are good and which are bad. It's hard to tell which is which from the beginning. Failure is inevitable. It's just whether there's a net benefit to the company instead of a net cost.

The problem is that the people and processes resist ANY failure, which necessarily throws out any innovation along with it.

you need people with enough leftover money and time who can "screw around" a little

And this is about right. There needs to be enough slack to not strangle ANY idea that can possibly fail. The author just went so far as to say that the "leftover" needed to be part of the budget.

Also a place where everyone is not grumpy.

Quite right. We need a place where some people are optimistic about new ideas instead of rejecting everything.

adverbly4• 2 years ago

To be fair, creating a process for innovation is hard and expensive - especially if you want it to apply across the board at a large organization.

I have been on multiple "incubator" development teams in the past and although we did move faster, my times at smaller startups still felt significantly more productive.

You want something that is the right balance of bespoke and standardized/transferable. Its not easy, but there is so much room for improvement at these bigger organizations that there is huge value to getting it right.

BenFranklin100• 2 years ago

Lack of innovation is of course a problem in industry, but it’s a particular problem in the public sector where many employees are attracted by the “can’t get fired” rather than the “let’s make things better” nature of the job.

dang• 2 years ago

Even this reply is too late but I want to make sure you see https://news.ycombinator.com/item?id=40783859. If you'd please read that moderation comment I'd appreciate it. You're welcome to participate on HN but we need you not to break the rules that badly.

solatic• 2 years ago

Author clearly has no understanding of why startups move quickly. Startups have the same Legal, Procurement, Security, etc. needs and responsibilities as any large company or government agency - its just that these responsibilities are looked after by generalist founder-executives, not whole departments filled with specialists. Any "innovation department", if it ever wants to actually ship anything, still needs to get BigBureaucracy on board, which is where the vast majority of time gets eaten up to ship anything. Startups don't need buy-in and alignment from whole departments full of specialists, just the executive-founders with unrelated titles and experience who are still nonetheless nominally responsible for those areas.

what we just witnessed was leadership rewarding and perpetuating a dysfunctional and broken system.

Leadership's first responsibility is to keep the system happy and running smoothly. The first responsibility is not to shareholders, not to attempting to seize potentially higher profits, but to the organization itself. The organization may be "dysfunctional" and "broken" but this is completely irrelevant - the Fortune 500 is still generating massive profits and the public institution is still nominally discharging its duties. This is why the vast majority of Fortune 500 CEOs are "caretaker" CEOs and why deep cultural change of public institutions is so difficult.

Culture is fundamentally a question of who you hire, who you promote, and who you fire. Those decisions do not happen overnight in healthy organizations. That's why it's slow.

smugglerFlynn• 2 years ago

Some orgs of Fortune 500 caliber have innovation units that help to process ideas and changes. To be efficient, these units must have buy in on senior leadership level and high level of exec sponsorship, ideally they would also have C-level representation in a form of dedicated “innovation officer” or similar role.

It is definitely not a new area, and author is on point with one key idea: organisations where heroes are praised as miracles generally don’t give a damn about continuous improvement, innovation units just never happen there. Praising individuals who swam against the flow for a year is pure virtue signalling in these orgs. In 2024 “running things smoothly” is almost a synonym for “continuous improvement”; building latter requires intent, not miracles.

pdonis• 2 years ago

Author clearly has no understanding of why startups move quickly.

He also appears to have no understanding of why government agencies and large corporations don't have to. Multiple times he says that such organizations need an innovation strategy or they will be out-competed. But government agencies and large corporations (which are mainly large because of the advantages of being large in an environment heavily regulated by government) don't have to worry about being out-competed, because they have insulated themselves from competition.

charles_f• 2 years ago

I fully agree with the sentiment, as I immediately get discouraged of doing anything as soon as it requires the approval from someone in another organization, which in large companies will often take weeks, if it happens at all.

There's an argument to be made for bureaucracy: once you reach a certain scale (of complexity and/or size), the law of large numbers makes that you're prone to more risks than when you're a startup. It's "fine" for a startup to not be aware of the latest development of privacy policies in Finland, and if you provision a couple resources in the cloud everything's fine because when looking at the bill, you can simply ask around "who owns that thing?". At enterprise size, you're much more prone to be audited by governments, to be attacked by hackers, to be the target of lawsuits, to have runaway costs that get hard to diagnose ; and the complexity of your systems (wet or hard) doesn't increase linearly. I don't think you can avoid having some processes to control that.

The problem is when the enterprise gets into a vicious cycle in which the only fix to any process issue is to slap on more process rather than rethink the existing ones. Such as illustrated in that post, where the solution to a lack of innovation is the creation of innovation heroes to incentivize the behaviour. I've been in three enterprise organizations, two of them have policies where you need to ask for permission, the other grants permission by default but reviews what's been done. That second policy is much better, as the level was quite high, people tend to respect the rule, and get caught if they don't. That allows for faster work.

But then there's also the matter of recruitment. As a startup you can handpick people, create a team that's functional and with an edge. In an enterprise, recruiting is done at an industrial scale, where your inbound fresh flesh exposes you to the simple truth that, by definition, half of humanity is below the mean.

JohnMakin2• 2 years ago

This is true, but I am unsure of the fix prescribed here.

For me, I had experience with a project like this - a large, extremely bureaucratic company tasked my team with a nearly impossible task, I think in an effort to lay off the team - which eventually happened. I fought with tooth and nails and was already doing fairly well in my role, so they gave me a shot and assigned the task, which I won't belabor the details of, but it was basically to implement a fundamental service that every application in the company would use to authenticate to the backend system (plus a lot of other reliability/availability guarantees).

The problem was, with their architecture being a central hub cluster of servers that provided core services to hundreds of edge servers around the world, it would have been fine to just throw this service into the central server and call it a day. However, the eggheads in executive leadership felt this was not acceptable, so the requirement was to make a new "central" cluster to connect to not only all the edge clusters in the company, but all the backend office/admin stuff as well.

Problem was - the people who wired up the networking and everything else with the central server had been gone for 10+ years. Barely anyone even knew how it worked, and unfortunately as I took this on, the network team got laid off, replaced by people who also had no clue how it worked. As we connected more and more services to this new hub, a bunch of skeletons emerged - the funniest being a server cluster in a region and account that no one had access to, or knew what it did, other than when it crashed other servers had issues. No one had accessed it for years. That was fun hacking into.

There were tons of hurdles like this at every step, basically being that this touched so many teams in so many different areas of the infrastructure, the organizational hurdles trying to even get the information you need to do the job required a ridiculous amount of heroics. I hated it. Basically, you need to create urgency any way you can - whether this is by breaking things, horse trading, political maneuvers, begging, intimidation - your livelihood is on the line. Other teams sensed what a difficult ask this was so early going was extremely difficult getting cooperation from the 30+ teams this touched.

Anyway at the end of the day I was able to do it by finally escalating urgency to the executive level and they made it a priority one quarter and it quickly got finished. If they hadn't, it would have been very difficult. They had the issue of having too many layers of management below them and had no idea, all they heard is "why is this project you said would take 2 months taking almost 2 years" and a bunch of manager speak trying to explain why.

I would never, ever want to work on any project like that again. 2 years to complete, should have taken 2 months if working as an IC, 2 weeks with a full competent team and good management. There were some good things that came out of it though - processes got improved, collaboration improved, and we were able to use it as a chance to refactor the IAC in a way you could deploy these hub servers again in a much easier way that didn't require the ridiculous amount of detective work to figure out the first time.

Oh yea, forgot to mention the actual application took about 2 days to configure. All the rest of it was what took 2 years.

Scubabear68• 2 years ago

I am currently working with a very large company that has this problem. They are highly risk adverse and are happy to pay $100 up front to avoid a $1 accidental loss.

Of course, all the processes don’t really protect them, they just get slowed down by a factor of 100 or more.

The successful people take on as many simultaneous projects as they can, because any one project will move at a snails pace.

I have told them tales of working on some internal document processing and classification systems in a fintech a few years ago where we would release to prod multiple times a day to test out different algorithms and approaches.

They flat out think I am lying.

groestl• 2 years ago

Large organizations don't improve by innovating. As the article mentions, they focus on sustaining the existing business model, anything that diverges from process potentially destroys the revenue source. Instead, they improve by acquisitions. Of innovative companies.

wonderwonder• 2 years ago

A long time ago I burned out and took a few years of leave from tech and instead took a document processing / generation job at a very large non tech travel / leisure focused company. Company used an archaic document generation system that had to have templates built by hand. then there was a significant amount of manual work moving data from microsoft excel to word. All Manual. My group consisted of myself, 4 other document processors, 2 leads and a director.

This team generated all of the sales documents for the company; thousands of templates.

Within weeks I wrote some VBA macros that automated 75% of the process and they promoted me to manager a couple months later.

What I found fascinating though was the group of existing document specialists were not suddenly capable of doing 2 - 3x the work. They just appeared to move slower. It was like pulling blood from a stone. They had no interest in learning new work which they were now free to do, they just wanted to clock in, turn their mind off for 8 hours, click the button and clock out.

40% of the staff at large companies are likely not needed and can be automated away.

Kind of made me feel sad.

drewcoo2• 2 years ago

What I found fascinating though was the group of existing document specialists were not suddenly capable of doing 2 - 3x the work. They just appeared to move slower. It was like pulling blood from a stone. They had no interest in learning new work which they were now free to do, they just wanted to clock in, turn their mind off for 8 hours, click the button and clock out.

The ability to barely function and slowly grind was what made them good at the job. Either they were hired for those "aptitudes" or they developed them on the job. what did you expect?

djaouen2• 2 years ago

I assure you, this slow grind of replying to non-relevant threads of yours will end after I post the following YouTube video: https://www.youtube.com/watch?v=MUfgAbFY4CA

agentultra• 2 years ago

“Engineering is doing well with $1 what another person could figure out with $2,” I heard from somewhere.

This often involves, in larger groups, communicating with folks and taking the time to understand the requirements.

Although I despise “5 whys,” almost as much as scrum; it does often boil down to dysfunctional organizations.

It saddens me when a highly productive development team gets pulled down by non-productive bureaucratic middle-management and politics. There often is a class of worker that doesn’t want to do meaningful work and sees the value generated by other people as an opportunity to take a ride to the top.

There should be a few check points in an organization where one needs to ask for permission but going too far down that road puts a ceiling on how productive any one group can be.

Good article; definitely need to remove barriers.

ecshafer2• 2 years ago

My goto example of a dysfunctional and bureaucratic organization is an example I saw in a finance company I worked for.

I was a on team that we were doing a major project. We basically ran Kanban but had to run "sprints" so we chose 4 week sprints to get it out of the way and we put everything on the board we had to get done that month to stay on track. Our pipeline was setup in such a way that you were required to have a jira ticket to push a commit. We were really crushing our timeline, doing 2x the work we were expected, a really great team honestly. But we opened up bugs and additional features in the middle of our sprints to track our work as we did it.

What this amounted to was maybe us saying we have to do 40 tickets this month, and wed be closing like 80. Everyone should have been thrilled by this development! Well we had a sprint review where some "Senior Project Manager" That wasn't really affiliated with our project but was some manager higher up was mad that us opening up new tickets mid sprint and closing them was ruining the org level burn down charts and expected delivery. They wouldn't give up on this, and said it was our fault for not estimating better (which sure, but we were beating expectations!). So we did the reasonable thing and improved our estimates! No of course not, we doubled our number of tickets and filled in "placeholder" on half of them, and used them as needed then closed everything out at the end of the sprint, where we were congratulated by everyone for our phenomonal estimation.

tokyolights2• 2 years ago

I have seen this multiple times before. Management chastising good work because some externality of that work shows up on a graph that they have to talk about at some weekly meeting. It doesn't even have to make the graph look bad, just stick out.

aiisjustanif2• 2 years ago

Could have spent that time more wisely by making a better graph that accounts for this…

FireBeyond• 2 years ago

I am dealing with this as a Senior Product Manager, explaining to execs, and defending my engineers when they started introducing metrics, and one of the keys was "Planned points delivered".

My team gets and triages all the escalated bugs.

It's not accurate, not fair, and demoralizing for that team to be "red" because the metrics look like: "Story Points Planned: 30. Planned points delivered: 5." when they delivered bug fixes on what amounted to 20-30 unplanned points.

"If they have to do that work, they need to get credit for it, planned or not. You can't have work that is required but counts for nothing in KPIs".

The issue of HOW MUCH unplanned work is being required is a separate and valid discussion, but not in the realm of "engineering cadence/velocity".

constantcrying4• 2 years ago

No of course not, we doubled our number of tickets and filled in "placeholder" on half of them, and used them as needed then closed everything out at the end of the sprint, where we were congratulated by everyone for our phenomonal estimation.

Are you sure this is a negative example? You even have a work around which makes everyone happy.

This isn't what a dysfunctional organization looks like.

loloquwowndueo2• 2 years ago

It’s called gaming the metrics and it’s totally a sign of dysfunction at the org level.

The team that decided to do this to keep the metrics happy rocks, though :)

danaris3• 2 years ago

It's still dysfunction even if the group in this case was able to finagle a happy ending to this specific story.

Charts and processes are there to serve people, not the other way around. If a particular group is clearly doing a great job, but the way your metrics are set up claims they're causing problems, then the problem is in your metrics.

The organization is dysfunctional because a higher-level manager is able to enforce their particular view of how these metrics are supposed to work at the expense of actually productive groups underneath them.

Note that if the GP had actually done what they were told, they would simply have not worked on all those bugs that came in mid-sprint until the next one.

apozem• 2 years ago

That is amazing. I worked with a (different?) finance company that did the same thing - every team was judged on their burndown charts.

From what I saw, it did no good. Teams simply did not use tickets to track work. If a story surfed sprints, its “ticket” was closed at the end of one sprint and a new one was opened for next. If a priority bug came in mid-sprint, we simply worked on it without a ticket.

cruffle_duffle2• 2 years ago

If a story surfed sprints, its “ticket” was closed at the end of one sprint and a new one was opened for next.

LOL, just joined a team that seems to be doing exactly what you describe. You start attaching performance incentives to how tickets are estimated and closed... people are gonna game the fuck out of them.

sailorganymede• 2 years ago

Strange… The finance company I work at also does this. And it’s so strange. I feel like they value Jira tickets more than actual work being done. It’s even funnier cause we get asked to close work when we want to go into prod and told to create new tickets to put our work in to prod. which kind of is annoying.

ryandrake2• 2 years ago

Careful, though, because you may encounter someone who can sniff out these tricks.

I worked at a place where, like many companies, tracked the bug burndown as one of the measures of project risk. So, the closer you got to the release, the fewer bugs were expected to be open. Having many bugs open early in the project was fine and encouraged. Having few bugs open towards the end was seen as On-track, and having many bugs open towards the end was a red flag, where the program might want to either step in and help organize things or at least raise the risk with higher management. This is a pretty standard scenario you get at a lot of software companies.

Well, one of the teams realized that if they just kept their bug count always at zero, they'd never be bothered by those nasty, annoying project managers. So that's what they did. They were constantly in crisis, fixing problems and committing code, but their bug counts always looked great. A trivial bug tracker query revealed what they were doing: They'd just keep a side bug tracker in a doc rather than filing them in the real tracker, and then seconds before committing the code, they'd quick create a bug and use that as the required bug in the commit message.

So, the following week, these "ninja bugs" were added to the automated reports, and that behavior ended quickly.

siva7• 2 years ago

Everyone should have been thrilled by this development! Well we had a sprint review where some "Senior Project Manager" That wasn't really affiliated with our project but was some manager higher up was mad that us opening up new tickets mid sprint and closing them was ruining the org level burn down charts and expected delivery.

Why should they be thrilled that you were running on Kanban but claiming to do otherwise and therefore ruining the burn down charts?

ecshafer2• 2 years ago

This was a team of 10 people, so costing the company $2M+ a year. How is halving the timeline on a very major, mission critical project not a good thing?

Some appendage management structure in the organization having charts and reports looking a certain way is secondary to how much value is being produced by a team of expensive engineers.

gopher_space• 2 years ago

Am I working for The Tacoma Chart Co. or are we developing actual products here?

adammarples• 2 years ago

Because the burn down charts don't matter and the real work being done twice as fast does matter.

riskable• 2 years ago

I also work at a large financial institution and have had many similar situations. Fortunately, I'm the one in charge (team lead of sorts) and I have a pretty standard response to such "high level" nonsense:

"Your inability to adequately track my team's weekly or monthly performance is not my problem."

Every "project" has plans, deliverables, and due dates and those are the ultimate arbiters of a team's performance. Not the weekly/monthly statistics! If we open 10 or 10,000 tickets it makes no difference. It's entirely arbitrary and only carries meaning for the team in question (not upper management).

Like some high-level manager/PM is going to be able to make any difference whatsoever on some software development task by watching weekly Jira ticket statistics. Sounds like they're giving themselves busywork to justify their role. Because having fancy charts and statistics at meetings of managers makes you look like you know what you're doing (LOL).

gspencley• 2 years ago

"Your inability to adequately track my team's weekly or monthly performance is not my problem."

I'm an engineer and I can certainly understand and empathize with where that sentiment comes from.

However, when people say things like this the first thought that goes through my head is "there is a culture problem."

That sentiment underscores an adversarial relationship between teams and leadership/management. That adversarial relationship should never exist (it does all too often, but it shouldn't).

Tracking timelines and deliverables is something that requires communication. That communication can be automated, but it's not up to leadership or management to implement that automation. They are not the engineers.

So if the process that is in place, which has worked for them despite inefficiencies (which they may not be aware of) is suddenly disrupted then no, it is not only "their problem." Some team went and did something differently than how things are usually done. The team [rightfully] recognizes that it is an improvement, but it was unsolicited and the communication / warning of the upcoming change was likely lacking.

Companies are called "companies" for a reason. They involve multiple people with varying skill-sets, responsibilities, understanding of how things do and should work, they have their own pressures and reporting structures (they need to hand things over to their management who expects a certain status quo as well) and most people have a default low tolerance for change.

This is no one person's fault. The company culture needs to facilitate iteration, improvement and innovation.

roenxi• 2 years ago

ecshafer's response seemed more productive. He was effectively providing an explicit maintenance budget in his estimates, which in practice seems to be exactly what Mr. Senior Project Manager wanted. No real cost to the team. Everyone a winner.

parkerhiggins• 2 years ago

outcomes > outputs

It’s proven time and time again by DX programs (S.P.A.C.E. etc) that focus time is really the only KPI that produces outcomes.

Aurornis• 2 years ago

we had a sprint review where some "Senior Project Manager" That wasn't really affiliated with our project but was some manager higher up was mad that us opening up new tickets mid sprint and closing them was ruining the org level burn down charts and expected delivery.

This is giving me flashbacks of a company I worked for (for a very short period of time).

For some reason, they decided that planning accuracy was the most important metric for the software organization. This was the headline metric they talked about in executive meetings and it was primarily how the program managers' bonus structure was determined.

So everything in the company revolved around planning accuracy. Opening new tickets within a sprint was strongly discouraged. Doing work from the next sprint was heavily discouraged. Trying to take on big projects within a sprint was heavily discouraged, because if you couldn't 100% guarantee that it would be done by the end of the sprint it posed an existential threat to the Program Managers' charts, and therefore their bonuses.

Program Managers wouldn't come out and say any of this, of course. They knew the situation was at odds with delivering software quickly. However, if you deviated from the plan they would pull you into meeting after meeting for hours and hours to try to keep you in line.

If you opened a new ticket mid-sprint, you'd get pulled into meetings with Program Managers to justify it. They'd argue and debate and cajole you into deleting the ticket and rolling it into something halfway related. They'd CC your manager on 5 different e-mails and check in multiple times a day to make sure you'd fallen in line. It was hell.

Weirdest part to me was how many people around me seemed to enjoy that structure. They recognized the game and gladly played along, delivering a couple hours of work each day and then sitting in meetings to talk about it for the rest of the time.

It all came crashing down about 18 months in, when management brought in someone who actually understood software development and started actually looking into what people were doing. I was gone by then, but they went slash and burn on the remaining department. They cut it down to 1/5th of the size and started delivering, as far as I can tell, the same amount of work.

nonameiguess• 2 years ago

I experienced something like this for many years across several of the big five defense contractors and smaller SBIR contractors working for the US DoD and IC. As someone who 20 years ago had a slightly better understanding of how computers and networks actually work than an average developer and was comfortable at the command line in an era when juniors increasingly couldn't leave their IDE without becoming hopelessly lost, as DevOps, SRE, and platform engineering started to become things, my career drifted in that direction.

The problem being on teams like that was always the same. You're responsible for developing software products of some sort, but they're software that runs, tests, or delivers other software or even orchestrates the operations of an entire environment shared by many different applications. This inevitably means a whole lot of your work is the classic incident response/post-mortem of operations, plus some level of customer support given to other development teams because internal platform teams never get a separate support organization. To this day, the government has no idea how to handle this in light of how DFARS (administrative law governing acquisition) works.

Every labor hour a contractor charges has to be tied to a specific line of accounting which is itself tied to some unit of planned work tracked in an issue tracker or project management system of some sort. Most of the time, this is an Epic in Jira. This is logical insofar as you consider the intention of acquisition as a category of appropriations bill. A budget proposal with line items tied to measurable product features is presented to and approved by Congress. You have to demonstrate you're spending the money they gave you on what they approved you to spend it on, because per the Constitution, that's how power of the purse works. The executive branch can't just do whatever it wants.

But it falls apart as contractor labor increasingly replaces federal civil servants. When your job is anywhere from half to all running and maintaining an operational system, that isn't really acquisitions any more. It's operations and maintenance, which is an entirely separate appropriation. Soldiers manning a defensive perimeter have no idea when they're going to be attacked or how much work they'll be putting in. Software operations is effectively just the less lethal civilian equivalent of that. The government arguably even recognizes this in many ways. Typically, running and maintaining the production system is a separate contract from the development contract and it uses separate work tracking systems, and if someone spends 8 hours watching a screen while producing no quantifiable work outputs, so be it. They charge 8 hours because that's what the contract says they're supposed to do.

But as we're increasingly expected to be modern software organizations with things like CI/CD, test and staging environments, and you inevitably need to run at least some of your own development infrastructure, well now what? You have a lot of people whose job is the same. Be on constant lookout and respond as needed, but now it's part of a development contract and they need to have quantifiable work outputs that be tied to a budgetary line item with an associated product feature.

So we convolute nonsense out of thin air like cloning the same monthly "Support" Epic in Jira, month after month. "As a developer, I want my tooling to work so I can do my job and deliver value to the government." Plug in some SWAG number vaguely guessed at based on how much of this kind of work you ended up doing last year. Every 90 days, play planning poker with it. LARP a product team even though that isn't what you are.

OldGuyInTheClub2• 2 years ago

Excellent and, to me, underappreciated points on HN. Now, expanding into the hardware world of A&D brings different levels of agony. If there's development risk, there's no good way to write a schedule that both accounts for it AND consistent with absolutely mandatory "Earned Value Management" where complex tasks have to be shredded into tiny 1-3 week chunks with metrics that the accounting team can track and report. If you need to fix a broken piece of equipment, if you need to buy a part, you're at the mercy of a procurement system that exists to survive audits and wants to know why the break or purchase wasn't in the plan. If the problem requires evaluating Plans A, B, and C, you have to pick one as the best, baseline, and execute it. Then if you need to consider the other two options that everyone knew up-front were possible, you get to explain why The One Chosen Plan didn't work. "You signed up for it."

The only way I've seen around it is a Risk/Opportunity Management system where risks/opportunities to whatever the baseline plan might be are enumerated, assessed, planned for, and maintained for when things go sideways or when there's a chance to save schedule and/or cost. But, that takes a VERY gifted _leader_ to run at the top level and those are rarer than hen's teeth. Ideally when a risk becomes a problem or an opportunity presents itself, the contingency plan is implemented. When it works, it is great and I've seen tough, ambiguous projects come in ahead of schedule and under budget. The common theme was one guy running all those projects and he's now retired. Usually the result is that the person reporting the problem gets stomped upon by a _manager_ who runs a cargo-cult process that s/he doesn't really understand.

Earned Value Management may be fine and dandy when development risk is done, the transition to manufacturing is complete, and the manufacturing process is relatively stable and the perturbations are small. Of course, that's when the contract SHOULD be Firm Fixed Price and not tracked by an army of accountants on both sides of the contract.

hinkley• 2 years ago

It's funny that my anecdote also comes from a kanban team, but amongst mostly waterfall people and one Scrum nutjob.

When you are showing people up they throw the book at you. Yes you're getting a lot done but you're cutting corners so that's cheating! In an org that is perpetually behind schedule they rely on one team being a scapegoat for why all the other teams are behind schedule, and when your team has never been more than 10 days behind it stings.

One of my proudest moments on that project was when I figured out a judo move to meet (and exceed) the documentation requirements for our part of the project with less than 3 man hours per milestone after I got it working (and half of that was typesetting fixes by one of our tech writers). I got the impression that the person who called it out was really hoping it would cost us 20-60 hours per milestone. I think I spent about 30 hours total, including the initial set up time for the tools and templates. So their gambit only worked for a single milestone and then it was back to the races.

specialist2• 2 years ago

Yes you're getting a lot done but you're cutting corners so that's cheating!

Yup.

Merits of Agile aside, I don't recall any of the project/product management books preparing noob me for the organizational (and personal) psychology stuff.

As you surely know, this is what the gray beards mean when they say most challenges (obstacles) to shipping software are not technical.

imtringued• 2 years ago

we doubled our number of tickets and filled in "placeholder" on half of them, and used them as needed then closed everything out at the end of the sprint

Seems like someone has become aware of the power of liquidity and delayed decision making giving one the ability to respond to future information.

araes• 2 years ago

I like how the response was "falsify the metric data" because an ape in a suit somewhere demanded a "correct" metric. And were rewarded for their falsification. There's an Aesop's Fables story in there somewhere. I wonder how many metrics people depend on, like jobs data, get the "change the data" to suit ape in suit's needs?

profsummergig• 2 years ago

Their organizations hemorrhage the very people they need to help them compete against aggressive adversaries or competitors who have them in their sights.

Govt is a monopoly. So it can't be bothered.

(The org. in charge of preventing monopolies is a monopoly.)

throwawayqqq11• 2 years ago

You are comparing companies and states. Sure, there might be similar machanics in both middle management at play but upper management is very different, at least in a democracy.

drewcoo2• 2 years ago

Heroics are responses to crises.

Why are there crises?

djaouen2• 2 years ago

I am only responding to this post because I cannot in my flagged thread: your bogus self-defense mechanism is not permissible. You will be loved, whether or not you intend for it! :D

lanstin• 2 years ago

The thing is there's a lot of dysfunctional organizations, and being an innovation hero is an easier and more fun career path than fixing dysfunctional organizations, which seems (as someone that only half-heartedly tries to keep insane decisions from the top from ruining things) to not necessarily be possible; there are often hidden agendas and people more interested in their cash out than the viability of the organization long term.

rednerrus• 2 years ago

If heroes are getting things done, it seems like it's possible. Wouldn't it make sense to study how the heroes are operating and try to make that part of the culture?

rqtwteye• 2 years ago

My company’s mantra is “change and innovation are great as long as nothing changes”.

anders30• 2 years ago

Last year, an Innovation Hero championed a completely new build system based on their homegrown version of asdf, essentially.

I was the one who had to stay late multiple Fridays making sure our Formal Builds would still work. I've been working to roll back the least well executed portions of this innovation for the past year because it doubled the amount of time required to release our software in the last stages of the pipeline (which was already five days, another rant, I work at VeryBigCo on a mixed HW/SW product).

I believe in this story the entrepreneur referenced worked to clear all similar hurdles, but it's hard to feel bad for folks who view themselves as Innovation Heroes when in many cases they're applying solutions in search of a problem. It's also very not fun to be "that person" aka "The Adult in the Room".

Yes, today the "Innovation" is part of our processes, but in stripped down form and it cost us two missed formal builds with the corresponding loss of credibility in our customer's eyes... in addition to my own sour grapes and missed dinners with my kid. It also cost me a personal friendship with the hero (my fault, but it's really hard for me to get past the fact this individual put their ego over my time with my family).

Agree it's a sign of a dysfunctional organization but it goes both ways. As an Innovation Hero, you should be aghast at how inefficient your org is, but as a part of that org, you should stop and ask, "How did we get this way and am I sure my solution truly solves all aspects of this situation?"

t0bia_s• 2 years ago

- All large organizations – both government and corporate—need an innovation doctrine or else risk being outpaced by competitors.

They not risk anything, because competition to government is forbidden by law. They have monopoly and will to use force if their system is threatened by alternative one. Innovation is not priority as it is in free market.

kepair• 2 years ago

In my company, awards and prizes always go to people that go "above and beyond" the processes the company sets in the first place.

theideaofcoffee2• 2 years ago

The idea of calcified culture and process isn't limited to large bureaucracies and government institutions. It's just as prevalent in orgs barely tipping over three figures' worth of people. In many ways, seeing it from both the large and the small, the smaller ones are often the most difficult to change because there are one or two people touching everything. Most often they have been there for a long time and such having the 'trust' of management so the bare minimum of 'new' ideas, and new being the state-of-the-art of the industry ten years ago, because they might have to adapt.

Playing the innovation hero in that kind of org is dangerous, and I have the battle scars, the burn-out and resentment to prove it. Others have pointed out that the point of the power structures is to self-perpetuate and any threat to that is swiftly dealt with.

It's just so sad to see, all because people fear change. I don't know how to change that other than replacing those lifers and swapping out management.

Now I know what to look for and am willing to bounce at the first sign of it.

obelos• 2 years ago

Counterpoint: most innovation sucks. Sure, innovation is necessary for improvement and adaptation to changing circumstances. Sure, sometimes innovative change is stifled to the point that organizations crumble under the stasis. But the larger and more stable the organization, the higher the bar should be between “I have a great idea that will make things better!” and its implementation. Most ideas of change do not take into account the complex, multiplicitous, overlapping, crossed-purpose systems that conspire to bring an organization to life. They account for only locally perceived changes, not weird, unpredictable knock-on effects that can arise from introducing an optimization. Adopting every ostensible innovation that comes along is inviting only chaos. Change should be hard. Not impossible, but hard.

contingencies• 2 years ago

I thought this was going to be about those people with colored glasses and ultra off-putting psychedelic sneakers (often matching) that turn up overcaffeinated with gelled hair as stage-dwelling experts, or multi-day workshopping nothing in particular with ramp-in/ramp-out platitudes and zero content. What do you call those? Innovation consultants?

web3-is-a-scam• 2 years ago

If you’re an innovation hero and you don’t own part of the company, you are a sucker

zmmmmm• 2 years ago

The saddest thing is that this "innovation" was mere automation of:

reentering data from one spreadsheet to another and annotating it with additional data from another system

Basically pretty much routine informatics - about the lowest bar you could possibly have.

Which is to say the whole article is really about simple baseline competence to implement process. If a place can't do that then they certainly won't be able to implement things that embody actual novel ideas. But framing it as "innovation" understates the severity of the problem.

satisfice• 2 years ago

Heroism is not a culture problem. Heroism is always a good thing. But process weenies like to knock it because it makes them seem grown up and sober.

The point the author wants to make can be made without denigrating the glorious matter of a person who is willing to overcome adversity to and ambiguity and chaos to do something wonderful.

When process mavens attack heroism, they are essentially saying that high performance and creative solutions should be an algorithm, a mechanism, something inhuman rather than fully human. They want the business to be an equation, and that should offend us, not comfort us.

By all means let’s do something to make innovation reasonably acceptable and not unreasonably expensive— but no organization should lightly enter into highly speculative work. Heroism is heroic partly because it’s also dangerous. There are good reasons to discourage it and good reasons to engage in it. Heroism is fraught— but it’s essential to any kind of technical work, since we are surrounded by obscure, “wicked” problems.

A good organization will reward heroism that doesn’t have a happy ending, as long as it was responsibly and competently pursued.

Log_out_• 2 years ago

The problem i see is that process designers do not have to justify the costs they inflict on economic +generating activity. Its purely designed by informal power struggles of "of course my department has to be looped in".. And thats the dilemma at the heart of all bureaucracy.. At the core its powerstruggles eating time and effectivity.

And to fight it, one would need sort of a controlling office that reports to and limits the damage done by powerstruggles. Never seen that in a company.

oweiler• 2 years ago

What's missing from the article: The innovation heroes will not only leave, they will most likely also burn out.

fuzzfactor2• 2 years ago

It could be worse.

It could be so dysfunctional, that the most visionary, capable, creative, and dedicated efforts need to be taken under the role of an anti-antihero ;)

jl2718• 2 years ago

“Why don’t big profitable companies (and government agencies) innovate?”

This was Clayton Christensen’s thesis, the most preeminent management academic of a generation, and still it’s been largely ignored or misinterpreted in favor of the cottage industry of fighting the obvious truths within it.

They do not exist to innovate; they exist to defend a business model from innovation. Innovation is termed “disruptive” in that it reduces margins, and expands access to markets that the organization has no advantage in. Innovation reduces profits and increases competition.

In the case of a government agency, their business model is monopolistic inefficiency: more budget to perform the same services. Internal departments in big companies operate similarly. The constant call for “innovation” is just another tactic to increase budget. Innovation in reality decreases budget, and the causality works best in the other direction, but curiously nobody is fighting for less. Why?

This is not just an internal phenomenon of departments versus budget planners or agencies versus congress. It’s also the relationship between the business or agency and the market it serves. The strategy of a monopolistic business model is to expand the captive market and extract higher margins for the same service. If they innovate, they do so only in defense, to prevent anybody else from establishing a profitable business from an innovative business model.

A good contemporary example of this is Google versus the LLMs. Google was founded to serve the market of people that love information. They found a business model in advertising, and used their profits in every imaginable way to expand the captive market of internet users, and the amount of browsing and searching they do. The problem with this is that their information-seeking users actually hate browsing, which is the activity that generates profits. Google also hired the majority of graduating AI researchers for two decades straight, who invented the solution to this problem, and published a version of it just strong enough so that nobody could create a profitable business from it. It should be obvious that still nobody else has even remotely the resources that they do to train an LLM. It’s perhaps likely that they do have a vastly superior LLM, it will be used only in service of their existing business model. The capital required for somebody else to train a model that could minimally compete with that business model was unprecedented by orders of magnitude. If Google were smart, they’d have calculated that amount versus the long-term profitability of a potential competitor, and release product updates and open source strategically to ensure that competing with them will never be profitable. And that’s probably exactly what they did. Yet somebody was willing to take that enormous loss, and now they’re at war. Now Google’s competitive LLM offerings are slightly inferior, and this appears as incompetence, but it’s actually excellent strategy to reduce competitor margins without advancing the state-of-the-art to affect the margins of their main business model. You should have no doubt that Google could easily produce a vastly superior LLM, and will continue to handicap themselves until such time as their advantage disappears. At that point, they will be forced to focus on higher margins at the top of the value chain of their business model, having lost the bottom, but also enjoy an expansion of that market from competition among low-margin or loss-leading innovators.

In Clayton’s thesis, it was steel mill technology. He explained why the big coal-fired mill businesses lost the market to electric mini-mills and eventually exited the steel mill business. He found that it was not due to incompetence, but profit-maximizing strategy. Every technology business model has a lifetime.

So don’t expect innovation from organizations that strategically demand the opposite. Tangentially, there is an interesting experiment going on at X where Elon has recreated half the conditions for innovation by cutting 90% of staff, but retained the user base and business model of the old business.

Innovation heroes are a sign of a dysfunctional organization

Related Stories