There may be no greater ongoing source of confusion, mis-set expectations, and outright deception than the ubiquitous software product road map. Are you tired of road map heartburn? Then take the Road Mapper’s Anonymous pledge and swear off bad habits forever!
The Term “Road Map” Means Different Things to Different People
How many times have you seen this scenario?
Act One: CEO or VP of Marketing says: “We’d like to create a product road map to outline our vision for the future of the product for the board.” (or for investors, or analysts, or a sales meeting) Product management dutifully prepares an illustration of what the product’s future might look like and says that all of this is subject to change. CEO agrees.
Act Two: People in various roles (CEO, sales, business development, marketing) obtain that road map presentation, show it to customers, prospects, and others, and describe it and its contents in various ways without product management present. They may describe some or all of the contents as firm commitments.
Act Three: Aggrieved account manager comes to product manager, upset to learn that a particular feature won’t be delivered on a particular date listed in some past road map. “That feature was committed. It was on the road map!” Product manager replies, “Oh that? That was just a road map.”
To some people, particularly CEOs and VPs of Marketing, a road map is a statement of vision at a point in time that is subject to change. They may ask in completely good faith, with good intentions, for a “road map” to be prepared on that basis. But the road map presentation deck usually does not define the meaning of the term “road map.” Others may interpret “road map” to mean a firm statement of binding commitments by the company. People inside and outside the company often interpret “road map” to mean what they want it to mean. In the short run, that makes everyone happy. In the long run, it guarantees unhappiness.
Road Maps Lack Appropriate Disclaimers and Are Incorrectly Positioned
Road maps usually lack any disclaimers describing the level of certainty about the road map as a whole or its individual items. So when the road map presentation that the CEO intended as a vision statement gets in to the hands of sales, marketing, or business development, it may be described as a hard-and-fast promise with 100% confidence attributed for any item or date in the document the customer asks about. This is particularly likely when an account manager is trying to interest a prospect or to use a feature on the road map to resolve an objection and close a deal.
Road Maps Go Too Far Into the Future
Software companies often prepare 18 month product road maps. If a person really had the ability to reliably predict the future 18 months in advance, they would not be working in the software industry. They would be raising money for a hedge fund and getting rich far faster! (Nassim Taleb makes the same point about financial analysts in The Black Swan.)
Clearly, many people who lack the ability to accurately predict the future 18 months in advance are nonetheless routinely creating product road maps that claim to do just that. Often, the people preparing the “road map” know they can’t predict the future. Why do they produce the document anyway? Sometimes, they’ve become cynical and accustomed to writing documents they don’t believe because they’ve been doing it for years and seeing others do the same. Other times, they’re just following orders.
Road Maps Are Too Detailed
Road maps also tend to provide too much detail at any point along the timeline. The further you go into the future, the less confidence you have in any particular prediction. A rationally-defined product road map would have more detail about items in the near future and less detail about items in the distant future. It would also document a decreasing confidence level the further it went into the future. Yet all too often you see a document that has the same level of detail, quarter by quarter, for the next 18 months and that also fails to indicate that there’s greater confidence about near-term items and delivery dates than longer-term ones.
Road Maps Are Overoptimistic
The situation would be bad enough if everyone were being rigorously hard-headed and completely honest with themselves and others when doing road map delivery date estimates. But how often does that happen?
There are too many incentives to be “optimistic” when doing a road map and estimate “best case or better” delivery dates. The company may be seeking new investment and trying to convince prospective investors that major new functionality is coming soon. It may be seeking to demonstrate it will have comparable functionality to competitors who are being overoptimistic in their own projections. The company may want to convince an existing or prospective customer that a specific feature is coming very soon. Product marketing between direct competitors often devolves into a competition to see who is willing to tell the biggest lie or give the most overoptimistic projection with a straight face.
Too many people are willing to give in to pressure and put dates in road maps in which they have low or no confidence. A product manager MUST be the voice of honesty, integrity, and reality in road map discussions because others usually won’t! Because the product manager understands both customer requirements from direct customer contact and a fair amount about the implementation challenges from working closely with engineering, they are uniquely positioned to push back against attempts by others to make overoptimistic assumptions.
Road Maps Usually Lack Adequate Schedule Buffer
You can’t predict which unexpected events will occur, but you can predict that some unexpected events will occur. Therefore, a rational person builds adequate buffer room into a schedule to accommodate unexpected events.
How much buffer is enough? At Kontiki, between April of 2004 and June of 2006, we assumed that engineers would spend 50% of each day on new feature development and the rest of their time on meetings, training, bug fixes, customer escalations, vacation, parental and sick leave, and other things. We estimated how long committed items would take on that basis. We then added on 30% buffer to the predicted release delivery date to accommodate changes in plan, development taking longer than expected, and so on. Using this approach, we shipped 13 product releases in a row on or ahead of schedule on a resource-constant basis. (Translation: there was one layoff during this period, after which we had to revise our delivery date for one release since the anticipated resources were no longer available. But we hit our revised delivery date after accounting for the new resource level.) Some people have criticized this approach as “sandbagging,” which is incorrect. Sandbagging is knowingly overestimating how long development will take, usually as a defensive move because engineering expects that bad managers will reduce estimates arbitrarily. Buffering is providing a reasonable amount of slip room to account for the fact that estimates may be wrong and unexpected things will occur or be added to the schedule. So far, none of the people who have criticized this approach have ever been able to say that they’ve shipped 13 releases in a row on or ahead of schedule.
Software Companies Are Generally Bad at Estimating Delivery Dates
Even when they are trying to be honest and accurate, software companies are notoriously bad at estimating how long it will take to deliver a particular piece of software. Microsoft made itself the poster child for this problem by shipping Windows Vista two years late without most of the originally-promised features. But to their credit, they took the negative feedback to heart and shipped Windows 7 on schedule as a very successful product.
Software Companies Rarely Stick to Their Plans
For a company to deliver on its road map, it would have to build in enough buffer that it could add other work after releasing the road map and still deliver the road map items by the dates they were scheduled, or it would have to be disciplined about sticking to its road map and refusing to make changes. Most software companies do neither. Companies need to be honest with themselves. Retaining flexibility is both good and intrinsic to Agile project management, but if companies want to be free to change their plans frequently, they shouldn’t issue lengthy committed road maps. There is no such thing as Agile project management using a 24 month firmly committed product road map!
Companies May Deliberately Use Misleading Road Maps
In the worst case, powerful companies may knowingly issue a misleading product announcement or road map to discourage competitors or startups from entering a space and venture capitalists and angels from funding competitors. During the bubble era, Microsoft was sometimes accused of this practice. Ironically, they might have trouble using this tactic today if they wanted to since many recent product releases have been late (Windows Vista), botched and dead on arrival (Kin), or irrelevant also-rans (Zune and whatever tablet computers they’re promoting these days).
Road Maps Have No Retrospective Validity
Want to eat some humble pie? Pull out your product road maps from 12, 18, and 24 months ago and compare them to what actually happened. If the predictions were inaccurate, don’t keep making unreliable predictions. Change what you do to improve the reliability of your predictions!
Customers Don’t Believe Road Maps Anyway
Customers aren’t stupid. They’ve been watching vendors trot out “road maps” for many years now, and they’ve noticed that the predictions are consistently overoptimistic and unreliable. Customers are tired of having vendors blow smoke in their face and are ready for honesty and change.
The Solution: From Road Maps to Reality
What’s the solution? Simple: a return to integrity. It’s time for product managers everywhere to insist on being honest at all times about what they do, don’t, and cannot know and to insist that everyone else in their company do the same.
What Product Managers Can Do To Stop Road Map Madness
- Call the road map something else like “a look ahead” to highlight to the customer that you’re taking a different approach in describing the future and to avoid some of the legacy confusion about the meaning of the term “road map.”
- Whatever you call it, clearly define what the road map is. Is it a vision subject to change or a binding commitment? Be explicit so people know how it is supposed to be interpreted.
- Make shorter road maps. If you don’t think you can predict what will happen between 15 and 18 months from now, be honest about that and don’t try. Refuse to do what you know you cannot do.
- Have less detail the further out in the future you’re making predictions.
- Provide greater ranges for dates the further into the future you’re making a prediction.
- Clearly distinguish between things that are and are not committed. I use two separate sections: what’s committed, and what’s not committed but is under consideration. I have seen enormous confusion around the meaning of the term “road map,” but I have yet to meet the customer who fails to understand the difference between “committed” and “not committed.”
- Permanently archive in your company wiki, intranet, or customer relationship management system a copy of every presentation you or anyone else gives to any customer. That way, if a customer says that they were promised something, you can go back and see what the document they were shown actually said.
- Periodically review your past road maps and see how accurate your predictions were.
What Customers Can Do To Stop Road Map Madness
- When a vendor starts showing you a “road map,” ask them to clearly state whether each item and date is or is not committed. This will put the vendor on notice that they need to be clear about what they are saying and will discourage the vendor from indulging in creative ambiguity and plausible deniability.
- Don’t ask a vendor to make predictions too far into the future. Reward vendor honesty and transparency by reasonableness in your expectations. Let vendors be honest with you, flexible in responding to the future, and free to use Agile project management and related methodologies that acknowledge, accept, and account for uncertainty about the future.
Road maps are a running joke and a chronic embarrassment in the software industry. It’s time to end the madness. Be honest with yourself, your company, and your customers, and demand that your vendors do the same!
Related posts:
voximate
RSS
[...] This post was mentioned on Twitter by Vin D'Amico and SolutionsIQ, Eric Krock. Eric Krock said: Most Software Product Road Maps Are Harmful Evil Lies! http://bit.ly/9dEJ0L #pm #pmot #prodmgmt #agile #prodmktg #project #marketing [...]
No one knows the future. There exist good road maps, that show what is maybe going to be shipped withing one or two years (comes close to 18 month).
Hi Michael – I agree with you. That’s why I titled the post “MOST Software Product Road Maps are Harmful Evil Lies” rather than “ALL Software Product Road Maps are Harmful Evil Lies.” I agree that if constructed, positioned, and qualified correctly, road maps (whatever you call them) can be a useful tool. Hopefully the guidelines here will help folks create, maintain, and defend reasonable, qualified, helpful road maps and prevent some of the heartburn we see all too often.
A roadmap can be a commitment OR a plan, but not both. While execs and product management see a roadmap as a rough plan, those in the field frequently present it as a guarantee. If it’s a plan, it can be changed; if it’s a commitment, it cannot be changed. And many want both–”I want the commitment and I want carte blanche to change it.”
One product manager found recently that her roadmap, delivered in PowerPoint format, was routinely being changed by sales people and the president, yet it still had her name on the roadmap. I suggested she use PDF format from now on.
Thanks for this excellent post.
Hi Steve – Many thanks for the link and for your excellent blog, by the way! Agree with your comments. I once summed up the “want it both” problem for the Silicon Valley Product Management Association as follows:
I also agree with you completely that it’s totally unacceptable for other employees to be changing the road map without the knowledge or consent of product management. That obviously makes it impossible for product management to ensure customer satisfaction when they don’t even know where the bar is being set. I’ve been fortunate enough not to have that problem with sales (at least to a material extent) in my own experience. There are probably a few reasons for that:
Account managers quickly realize that it’s easier, faster, and less painful to work WITH me and that blindsiding me has consequences that are excruciating for them, and then everything runs smoothly.
Your suggestion about doing road maps in PDF is a good practical one in a dysfunctional company environment to try to discourage unauthorized road map editing by making it functionally more difficult to accomplish. I would add this advice to the PM:
I agree wholeheartedly about the risks raised by answering every request for a “roadmap”. Labeling internal documents as such is key.
For outward-facing roadmap documents, to some degree you can mitigate these risks by putting yourself in the customer’s shoes, and also by considering where you are in the product lifecycle and what the competition is doing. In the case of my product, it’s a mature one by any standard (20+ years in the market) and yet continues with active and aggressive feature development. Much of our development is under-the-hood while some of it results in external features. When a customer asks me, “show me a roadmap”, what they really are asking for is some combination of the following:
“Is my investment backed up by R&D initiatives?”
“What disruptions should I expect in coming releases?”
“What new capabilities can I start planning for?”
It isn’t necessary to divulge sensitive (and preliminary) detail to answer the first two questions, which often are the most important. In the case of our competitive situation, we generally can address the third question by saying we don’t pre-announce, and use the question in person as an opportunity to gather market requirements. The first question is one we answer by pointing to releases of the recent past, and the second one we try to give an honest assessment if conversion issues loom (they generally don’t). Empathy for the customer is what it boils down to.
Great post, rings very true. I prefer describing the look ahead as “Product Themes”, tying them back to the nature of what customers are trying to do. Any one of those themes may come to fruition in the form of a release over the next 18 months.
Gives flexibility while giving context for what actually does make it in to future releases.
Thanks for the info
[...] a great article about roadmaps and how they generally, well, suck. My favorite quote: “You can’t predict which [...]
[...] [...]
[...] Most Software Product Road Maps Are Harmful Evil Lies! By Steve Johnson on October 18, 2010 There may be no greater ongoing source of confusion, mis-set expectations, and outright deception than the ubiquitous software product road map. Are you tired of road map heartburn? Then take the Road Mapper’s Anonymous pledge and swear off bad habits forever! via voximate.com [...]