Why most product planning is bad and what to do about it

https://blog.railway.com/p/product-planning-improvement

by ndneighbor • 10 months ago

136 46 10 months ago

blog.railway.com

colonwqbang• 10 months ago

Meandering post which struggles to get to the point.

The font is extremely thin which makes it unnecessarily hard to read.

A lot of jargon and abbreviations also hinder understanding.

TuringTest• 10 months ago

You can skip to the "The Four-Day Process" at the end to get the gist.

But for people living in an organisation suffering from those problems, reading how other solutions were tried and how they failed is valuable, and it puts things in context of why the recommended approach may work best.

stackskipton• 10 months ago

As someone who has been at multiple organizations suffering from planning hell, most of the time the solution is pretty clear but outside factors prevent it.

Our Quarterly PI planning is nightmare but Sales wants to sell the backlog so fighting is different business units fighting over which items should be done and how fast they are done.

It's generally employees meta optimizing for themselves instead of larger business and business is too big for CEO to figure out. Railways career page says they are a team of 27 so CEO/Founder/Whoever has vision in their head and keep steering everyone towards that vision.

However, as Ops type, looks like could be interesting job. Digs further

apsurd4• 10 months ago

You got downvoted but then I read some and you're not wrong.

I think we should call out bad writing assuming English is their first language. Bad, lazy writing, doesn't respect the audience.

Instead of crowd sourcing the OKRs from the company and bubbling them up per function.

First sentence under the heading "Good Ole Projects". This is not a sentence.

edit: The charitable pov is that writing is very hard work and writing and publishing anything is a net good. I wish for more people to respect how hard writing is and also to take the time to write well! So that's why that sentence bothered me.

ndneighbor4• 10 months ago

Author here, not my intent! My deepest apologies. English is my first language but people do joke that they say I write English like I learned it as a second language.

I have fixed the sentence fragment and connected the two thoughts together. Thank you for keeping me honest.

datadrivenangel• 10 months ago

Weighted Shortest Job First with clear articulations of value is the way. This problem driven development approach seems like a decent take on that.

procaryote4• 10 months ago

Am I getting it wrong? It sounds like they're still doing quarterly planning just with a different ritual?

I had hoped they'd realise quarterly planning is a bad premise and asked themselves why they do it.

If you have a mature product where you add incremental features, you don't need that plan because it's just an arbitrary block of pretty fungible work.

If you're still looking for product market fit, that three month plan wont last a week before becoming obsolete.

If you need to build a bigger thing that is only valuable once it's all done, you A: need a project and B: probably don't because it will fail.

ndneighbor4• 10 months ago

Hello there! Author there, and surprised/delighted with the response. I don't think we had the issue with the cadence, the quarter is arbitrary, but we think it gives us the ability to just go heads down to focus.

With that said, one thing we did and I don't why we did it was that we would "re-justify" why we would want to work on something every three months which isn't great. There is a world where if we had more eng. resources we could have more people than problems and we could take stuff on board as it arrives, but for us deciding on what to work on is a hard decision.

I also agree that market fit is a key factor. I think Railway was lucky that we didn't have to pivot the product 3 to 5 times to get some latch.

What would be the post-quarterty planning process that you would like to see?

__alexs• 10 months ago

There is no world where you have more people than problems because the more people you hire the more problems they discover.

collingreen• 10 months ago

There is a world where if we had more eng. resources we could have more people than problems and we could take stuff on board as it arrives

I don't think this is a realistic world. The cavitation surface area that spawns new bubbles of unvetted fresh ideas grows as things are added. Adding eng resources to get more done makes it grow faster and I don't think there is ever such a thing as catching up.

tl;dr - deciding what to build (and what it looks like in detail) will always be a critical and fundamental function of product teams.

procaryote4• 10 months ago

Deciding what to work on is critical as you say, every startup I see have at least 10x the things they want to do compared to the number of people.

But you're working in a pretty well known area. You don't really need planning for all 1000ish people as you could take this well known area and divide it into pieces that are largely independent. These pieces are either just table stakes where you need to be good enough but not innovate, or they're part of your vision for how to differentiate from other PaaS companies. You can budget accordingly and let these pieces do their own planning with their importance for the overall vision as context. (If you don't have a vision, your company is probably a mistake. Why are you even there?)

If there's an area where you don't need to innovate much, but you need to catch up, they might actually do a 3 month plan. If there's an area vital for your vision, they are probably trying new things, learning and re-targeting. Three months is far too long to plan for them on a tactical level, and probably too short to actually deliver on the vision.

You can certainly have a quarterly checkup or something to see that all parts are working from an upper management level, but the useful planning horizon varies by problem area, and they don't really need to plan in the same way.

lemonlearnings2• 10 months ago

I think quarterly is used as a heartbeat across many teams. If you know your 15 adjacent teams are also quarter planning you can start asking for commitments. It makes things easier to figure out for higher ups too I imagine (never been a higher up though)

Why not 6 weeks? Not sure. Gut feeling I like 3 months. Maybe 6 week sprints are good.

QuercusMax• 10 months ago

3 months feels like a good cadence to get everybody in a room together, especially if you have geo-distributed teams or product / sales folks who are on the road a lot.

procaryote4• 10 months ago

If your 3 month plan relies on other teams delivering on commitments, you've coupled your teams success to that other team.

I usually tell people they should try to plan based on what exists. Occasionally it's not possible and two or more teams need to cooperate, but planning 15 teams with lots of interconnections about hypothetical things is too hard and brittle. If you need it, your teams are probably split in the wrong way.

goalieca• 10 months ago

I have a spicy take! OKRs every quarter don’t keep you focused on the prize, they keep you focused on your own performance for whatever the industry equivalent of publish or perish is called these days.

affidavid• 10 months ago

I believe that planning out 5 year vision, 2 year vision, and quarterly goals would be best, but it’s really difficult when the vision is just making more money and quarterly goals are either “I want you to develop a product it took another company 10 years to create but make it our idea and better” or it’s some boring feature or maintenance work.

I used to think we needed our company to explain their problems, so that we could fix them. I less and less believe this is true. You can’t define some things. Home Depot doesn’t sell pre-packaged solutions for every need. They just sell a bunch of stuff, and they have people you can talk with to order what you need and even drop it off or install it. That is what some people need- a selection of stuff and solution experts. And that’s why MCP sounds so attractive; it’s a bunch of tools, prompts, and resources- a type of Home Depot, and the LLM is the solutions expert.

nradov• 10 months ago

There is no one optimal planning horizon. If you have a small team and rapid requirements flux then you might not be able to plan more than a day ahead. But if you're trying to coordinate a huge program with a geographically distributed team, multiple suppliers, hardware components, regulatory compliance issues, etc. then you might have to plan even more than a quarter ahead to avoid complete paralysis.

theptip• 10 months ago

you don't need that plan because it's just an arbitrary block of pretty fungible work

This is an over-generalization. If you are a team of 5 selling a B2C product, sure.

But for B2B you tend to need to commit to a roadmap eventually, hence planning.

And for bigger orgs with multiple teams, if you want to commit to x-team initiatives then longer timeframes can help with coordination. (Ie- we want to deliver X in Q4, therefore A and B should be done in Oct, so that C can begin in Nov)

procaryote4• 10 months ago

Does the roadmap actually help in the B2B case though? In many companies it's just a way to disappoint the customer a bit now (because the thing they want isn't first on the roadmap) and a bit more later (because the roadmap failed).

X-team initiatives do need planning and coordination sometimes, but if you have a lot of x-team initiatives, your teams are probably set up wrong. If you fix that you can plan less.

zer00eyz• 10 months ago

Most planing is bad for two reasons that we never talk about.

1. The wrong metrics: Ultimately the only metric that matters is money. Is this saving us money, are we wasting money. Every feature has a cost, and we are decent about tracking its construction costs but not its operational costs. This matters when you're paying for every iota of infra on AWS.

2. No one ever gets promoted for ripping out a feature that costs too much to run or doesn't retain customers, or is under used. Heath of the product as a whole isnt always about growing the business, sometimes it's about running it.

apsurd4• 10 months ago

"Focus on problems, not solutions" is pretty standard practice. Seems like author cherry picks some abstract concept of "bigco" and their process-driven planning. I guess.

Came here to say, I don't deny those anecdotes, but just as many companies can and do intuit "what problem are we solving?" and then come up with some potential solutions in meetings across stakeholders. That's called planning. Time-box it and move on to some face-reality testing.

Planning anything, in any way, is imperfect. Partly fed by over-pontificating about the perfect way to plan!

p_v_doom• 10 months ago

"Focus on problems, not solutions" is pretty standard practice.

You'd be surprised. Especially as you go further from tech, or tech minded companies. Dont get me started on more regulated stuff in manufacturing. Ive met so many value stream managers, site managers and even directors who simply cannot comprehend how fancy system X wont solve everything

lovich• 10 months ago

I always found product planning went poorly because the bosses of the people making the plans would make heads roll if you couldn’t draw them 7 parallel red lines, 3 of them being perpendicular, and in green ink

They’d get a plan that made them happy in the moment, we’d inevitably go off the rails in a few weeks but in a politically satisfactory way. Mission accomplished would be declared and then we’d start all over again next quarter/year.

That is unless you are working under the SAFe framework in which case you cut out all that time wasted trying to execute on the impossible plan and just doing constant rolling, planning of planning

wrs2• 10 months ago

I must have missed something, because this seems to say you do capacity and headcount planning and publicly commit to the work before you know how to solve the problem? This seems like the “draw the rest of the owl” part of this process…

ndneighbor4• 10 months ago

I am more than happy to add color here, I am sorry, I try my best to write everything but my editor cuts as much as I add. We also tend to hire really autonomous engineers who tend to like just going off on their own to try to solve the issue.

There have been a few times where we would commit to the problem, assign a DRI, and then find out midway that... no we have to hire/consult our way out of the issue. I think that's okay, we then look back at the retro to see what we missed.

If interested, I think we can blog about what happens when a problem gets converted to an RFC and then we have more engineering discussions with the stakeholders but the piece was pushing a 10 min read time as it was...

dirtbag__dad• 10 months ago

This is interesting to me, too! My team is in the midst of opening new reqs. A leader wants to hire based on skillset. The team wants to hire based on upcoming work. We had a decent employee churn recently because the work wasn’t what they thought they were hired for.

I can see both sides. We don’t have work planned more than a quarter out (a good thing IMO). Generalist SWEs make a good fit. But we think we need someone specialized in AI/ML. Unclear to me if that’s the case… and how to plan for it if we don’t want to explore concrete features we _might_ build.

wrs2• 10 months ago

"Try to solve the issue" is the key phrase there. In my experience you're generally wrong about how long it will take to solve any nontrivial software problem, and that's exactly what's hard about results-based planning.

If this is a process to decide what to commit to start working on, not to finish, then that makes sense, and I congratulate you on having an environment that acknowledges and accepts the impossibility of planning results rather than planning activities.

0xbadcafebee2• 10 months ago

  > For most of my friends and colleagues at mature software companies,
    (randos at a company with a lot of money)
  
  > there are usually three ways for an item of work to get put on the board
    (we assume that everyone should always do work in the same basic ways and there's no reason to change the way you work other than personal preference)
  
  > Thats not to say that every company is a disorganized mess or a bureaucratic hell scape but
    (no, no, they all are)
  
  > and then at times get blindsided every now and then from a new business priority or an incident
    (your executive team is disorganized and your operations are unstable; again, normal)
  
  > we felt that reigniting the agile vs. waterfall armistice needed to be torn up

Wait... what? This article isn't trading on clickbait tropes about a black-and-white world for HN points? Ok, I'm listening.

  > We implemented them faithfully, we all read the John Doer book

So you read the 2018 book that was written after a Silicon Valley process used by mega-unicorns... but not the 1983 book that process was based on? Foreshadowing? I'm going to call it and say "they should've read Deming".

As a (lengthy) aside: planning is bad most places because most people don't have multiple perspectives. Anyone can have multiple perspectives, but it requires both an interest in other perspectives, and a means of communicating (and receiving) those perspectives. Those are rare commodities.

Sales and executives set deadlines for objectives without talking to the people who do the work. They don't have an accurate perspective of the engineering (or other) departments. If they knew there is no time, and they're crushing the morale of other teams (and why that's very important to avoid), they wouldn't commit to things when they don't know if it's feasible. Similarly, when engineering gets word they have to throw out their current work and rush to finish something else, and they had the executive/sales perspective, this demoralizing slog might not feel as bad.

Even within engineering, at every job I've had, splitting up teams creates dysfunction. Engineers stop understanding the entire system's architecture. They stop designing with the other pieces in mind. Their perspective becomes a laser beam shot at their own navels. This contributes to difficulty planning between teams, and invents new problems that have to be planned around. If they fully understood each other's perspectives, they wouldn't have those issues.

Today nearly all online products are developed with a 2010s-era "this is the only way you can make software" mentality. Your product is never finished, is constantly changing, re-architected, etc. It's cargo cult. You don't have to develop this way. You can make online products like physical products. But because people today are incapable of thinking outside this framework, planning is stuck in this bizarre world of only considering a few quarters at a time. This wastes time and money and creates more problems. Software architects, product owners, and executives, force themselves into working in crappy ways, and then struggle to find their footing.

This article is an example of people struggling to find footing. Rather than deal with the fact that the way they're working is causing them problems, they're instead focusing on how they can plan for the work that's causing them problems. It's like having a bum ankle while playing soccer, and rather than stop running on the ankle, you're trying to figure out how you can continue running on the ankle. Stop doing the thing that's causing you problems.

YZF• 10 months ago

This. I would also add that people are not motivated/incentivized to have those different perspectives.

All of this is very much related to: https://en.wikipedia.org/wiki/Conway%27s_law

Your software product is never finished is partly a reflection of how you structured your organization. That is often the root cause of your problem.

apsurd4• 10 months ago

Isn't software never being finished a unique trait of software and therefore intentionally used as a value?

I get that never-done software tends to justify shitty products.

But how would you suggest taking advantage of software's features vs do you really recommend building it like a bridge?

jason_oster• 10 months ago

You just cobble something together to sell. It need not be any good. As long as you can fool people into buying it, you can always try to make better versions later.
So then you get these version numbers, even with decimals: version 2.6 or 2.7. That nonsense. While version 1 should have been the finished product.

E.W. Dijkstra

Translated from the original Dutch [1] [2].

Dijkstra was convinced that programming was a formal application of mathematics. If the program has a bug, the math is wrong. If the program is missing a feature, the math is incomplete.

Personally, I feel that building software like a bridge is the better path. You don't want the bridge patched with new supports and "stability improvements" every time another fatal design flaw is discovered any more than you want to update your OS and system libraries every time a new CVE is announced. These scenarios are both disruptive and costly. But somehow, we have been collectively tricked into accepting it as an unchangeable fact of software.

The advantage of software is that I can "replace the whole bridge" with a completely different design if I wish. Not merely patching an existing bridge in place, with whatever poor aesthetics and integrity problems that leaves behind.

[1] https://www.cs.utexas.edu/~EWD/video-audio/NoorderlichtVideo...

[2] https://youtu.be/-Uae9_pgZzE?si=twwh7k7cPKRB2gvJ&t=50

0xbadcafebee2• 10 months ago

You could keep whittling on a chair leg forever, but I don't think that's a value.

I think software is a material, like steel. And like steel, there are properties of software you can take advantage of to build in different ways. For example, its ability to be turing-incomplete, or stateless/immutable, formally verified, interpreted vs compiled, composeable vs fully integrated, distributed vs isolated, etc. You can use the material many different ways, thus building a very different product.

It's also a material like a gas, in that it fills any container it's stored in. Or perhaps a flame, as it consumes all resources you give it? Or a clay, as it's easy to work, but needs to be specially treated for it to be stable long-term... Anyway, these properties of the material need to be better understood, so builders can understand the right ways to use it. If you are building a bridge, I sure as heck hope you're using the right methods.

shib71• 10 months ago

Real world example of:

    Patient: Doc, it hurts when I do this.
    Doctor: Then stop doing that.

florakel• 10 months ago

Sounds Like you rediscovered the “opportunity solution tree” (Teresa Torres) and were skipping a crucial step in product management / UX which is product discovery. I would suggest not to generalize your learnings by saying “why most product planning is bad…” and rather use a more humble title “why our product planning was bad and what we did about it”.

thenanyu• 10 months ago

This doesn’t sound very much like like OST beyond the general concept of caring about problems to me

sublinear• 10 months ago

The real answer is that the methodology really doesn't matter as long as you have clearly defined problems to solve, know that solving those problems would be lucrative, and the knowledge gaps across teams aren't too big.

Because we live in the real world full of people who are allergic to work that requires deeper and more tangible skills than "management", we must settle for some kind of methodology.

osigurdson• 10 months ago

> When Platform marks "Multi-mount volumes" as P1 and Product marks "HA DBs" as P1

Hilarious how closely aligned product and engineering are here. There must be essentially no delta in backgrounds at all. Might as well just merge the functions and have engineers talk to customers (who would be developers as well presumably) from time to time.

brightball2• 10 months ago

This sounds like PI Planning.

Fwiw, I have become a huge fan of the approach from ShapeUp with 6 week cycles followed by a 2 week cool down.

fredfli• 10 months ago

6 weeks is a long time. Is the benefit that you spend more time writing code rather than being in meetings?

brightball2• 10 months ago

It’s only long compared to sprints. With standard PI planning you’re looking at 8-12 weeks and with the authors post above they are going by quarters. 6 is short by comparison.

The idea is that there are numerous pieces of work that your team will do which take longer than 2 weeks and you’re better off planning on a longer window so that you can find the best way to execute with clearer scope. This is the approach of all of the longer timelines and it’s much more effective in my experience.

The 6 week approach advocated by ShapeUp tries to find a balance from years of trial and error that balances “enough time” with “short enough that there’s still a sense of urgency”.

The other hard rule for this system is one that I’m a huge believer in. Namely, “if you are doing something that’s too big for 2-3 people to ship something in 6 weeks it’s too big and you need to trim it.”

It doesn’t say “complete” it says “ship something”. Thats key. There will always be work that takes more than 6 weeks, but you should always be able to find a way to slice it into smaller pieces that can be shipped. There are subtle things happening under the hood of this simple dynamic that force a lot of good practices from everyone involved too.

adam_patarino• 10 months ago

What happens when the work isn’t finished at the start of the next planning?

I always found the hardest part of planning was “we need a few more days we are almost done” that stretches to infinity.

Why most product planning is bad and what to do about it

Related Stories