How to evaluate a product roadmap, for engineers

https://stephen.fm/how-to-evaluate-a-product-roadmap/

by BerislavLopac • 3 years ago

152 29 3 years ago

user_named2• 3 years ago

This is very idealistic and removed from reality. Most companies are so messed up that there is no coherent strategy, so you cannot require a single product manager to have an excellent roadmap, when the conditions for creating one is not there. You also may be working on a product or area that is far below the high level strategy, so it's goals would be different.

If an engineer really goes and thinks they should evaluate a roadmap like this, that's toxic.

xyzelement2• 3 years ago

I am a "Group Product Manager" and at least on skim, this article doesn't seem crazy. I would expect myself and folks on my team to articulate these things - or call out when we don't have something for a good reason (eg we're making a speculative bet.)

The thing that I don't expect is that the roadmap would stay static. Reality changes over time, you encounter new customers and opportunities, learn what's harder or easier than you thought, etc. So I would say "this is what we have today" and if that evolves somewhat even a few months down the line as we learn more, great. But at any given point, you should be able to articulate your best understanding in something that does hit against these checkpoints.

user_named2• 3 years ago

Right, so you're claiming that everything in your roadmap is scoped? That's what the article says to require.

mmmmpancakes• 3 years ago

It may be ideal to have all these items checked off. I think a productive way to look at this is "how many of these items does the roadmap check"?

If the answer is "very few" then that might be an early warning sign you're working for a product or org that is headed for serious problems. As a technical IC who can't hope to solve those large scale management problems, this can be useful a red flag. Don't wait for shit to hit the fan to leave.

I have seen a large scale failure of this kind and the items listed here line up very well with some of the root causes I observed.

- Is the roadmap flexible or iterative? The roadmap was hard, aggressive business targets.

- Are the roadmap initiatives scoped and prioritized based on evidence? The roadmap initiatives were derived by working backwards from business targets and then evidence was found after the fact.

- Does the roadmap identify major dependencies or risks? Many risks were identified much later because techincal teams were not part of input to initial planning.

- Does the roadmap feel aggressive but achievable? Aggressive but not physically possible.

- Does the roadmap take on appropriate risk? No, there was multiple possible independent points of failure.

Anyways, if you ever see this kind of product culture I suggest running for the hills unless you like having your time wasted. And if you are a technical manager I hope you push back like hell when presented with this situation.

spuiszis• 3 years ago

Author here, thanks for sharing your feedback. To your point, product is a high-stress, demanding role, and in reality, you might not hit every item from this list all the time. This ambitiously attempted to define greatness from when I've been fortunate to get there, see it, or be part of it. Getting to greatness can be done; it's just hard to achieve. I think it deserves that caveat up front, and I will update it.

I also agree that low-level teams might be very far removed from strategy, especially in large orgs. However, that team should have a clearly defined goal or mandate instead. So much of the product is context-driven, and the first few drafts were 3-4x longer to account for these differences between organizations, hierarchy, teams etc. I removed most of that to focus more on basic principles because this guide was starting to look like it was written by Charlie Kelly chasing Pepe Silvia accounting for all the edge cases :)

doctor_eval• 3 years ago

I did enjoy this article and agree with almost everything.

One piece of advice I’ve learned the hard way, however, is if the road map doesn’t make sense, it’s unlikely that making noise is going to make your life better.

I was reminded by this article of the time when my role as a PM was stymied by a lack of clear corporate strategy - I was told to make a road map without the support of any kind of goal or vision, told even that having a strategy would take “11 weeks” (yeah, an oddly specific number). Of course, I pushed back - but the outcome of doing so was very poor for me.

A road map doesn’t need to perfectly align with the very good points in your article, but there is a point at which it’s probably just better to cut and run.

johndhi• 3 years ago

Personally I've always found "risks" in product roadmap tend to not be great. There's often no risks for 80 percent of the project then at the end we'll discover blockers and risks from teams who weren't (or weren't thoroughly) consulted. It's really hard to know and communicate with all the stakeholders in these complex organizations.

pvdoom• 3 years ago

Most of the time risk analysis as it's done today is completely useless, as we tend to be pretty bad at risk assessment in general, and that is even worse in most organizations. Barry O'Reilly has a great talk on the topic somewhere on YouTube

vmfunction• 3 years ago

You are right, as a civilisation we have no idea how to do risk assessment, and often when sh*t hits the fan, it is something we can't do anything about. At the current rate it is better to NOT do any risk analysis, and just be prepared for the worse case scenario.

nitwit005• 3 years ago

A great roadmap is all signal and little noise, so much so that the team can easily understand it and get started today.

That's in the realm of dreams, unless you're starting totally fresh, and have no dependencies at all.

phillipcarter• 3 years ago

Moreover, most roadmaps I've been a part of have all involved items where the first step is discovery, both product and technical. There's usually strong signal that it's the right thing to do, but the first step isn't writing code, it's figuring the thing out from different angles.

kingkongjaffa• 3 years ago

Exactly this.

The roadmap is not for the engineers to know what they are going to code next.

That comes through discussion and sprint planning in a tool that has tickets/milestones/epics etc.

physicsguy• 3 years ago

Most roadmaps I've seen are usually a wishlist of features written with no consultation with the people who have to implement it, and shared with customers already.

j-a-a-p2• 3 years ago

(1) Does the roadmap clearly connect to the higher-level company or product mission, vision, and strategy?

Okay, engineers are now supposed to evaluate the roadmap to strategic goals. in principle all good that everybody is involved in lining up with strategy. But the caveat here is, the CPO and leadership team should do this. And if this does not resonate with the engineers, is it really wise to educate the product manager by sending him/her some literature on how to do strategy?

I would suggest to handle this if this occurs with more care. And also consider that many very successful companies did not have a written mission and strategy, it is usually introduced in the scale up phase when stakeholders demand more on the why and how.

proc02• 3 years ago

Unfortunately that is a common expectation for any engineer higher than junior level. It's a strange attempt to get engineers to own vertical slices of the company, as if they were managing their own business within the business.

The problem is that product owners and managers are not learning software engineering to better communicate, design etc., This is a one way street, engineers have more and more non-technical responsibilities until they are barely coding and then are forced to manage a team of junior engineers to do the code for them.

xctr942• 3 years ago

Cue ‘Developer Hegemony’, a great book on this topic. Yes, as an engineer, you should be interested in vision/strategy/product. Otherwise you’re just executing, and the opportunists will get promoted ahead of you. The book explains it much better than me.

aiunboxed• 3 years ago

Also engineers would need to manage their technical roadmaps - consisting of tech debt and other infra stuff. Every engineer should know the basics of product atleast

not_the_fda• 3 years ago

I've seen a lot of bad roadmaps. They usually have multiple projects, all set as the number one priority, all relying on the same fixed resources.

jschveibinz• 3 years ago

This is just systems engineering 101. There are volumes of material written on this topic including process, documentation, quality and organizational requirements. There is no need to reinvent the wheel. The product “roadmap” is the output of a good systems engineering approach applied to a timeline by a savvy, business- oriented project management team.

proc02• 3 years ago

I would think the roadmap is fully owned by managers just like the code is fully owned by engineers. Do we ask the product owners to be aware of the code the engineers are writing, just in case there are inconsistencies with their specification? Of course not.

eureka-belief• 3 years ago

But the caveat here is, the CPO and leadership team should do this.

In my experience, if you don’t have a smart lead engineer who is working directly with the product people and is involved with the higher level roadmap then life is more difficult for engineers. There needs to be someone advocating for engineering needs and setting the correct expectations about what reasonably can be built and at what cost. If engineering isn’t involved, then there ends up being this toxic one-way relationship where engineers are just receiving marching orders from someone else who might not know what is impossible or who might not have the technical vision to dream of what might be possible.

How to evaluate a product roadmap, for engineers

Related Stories