In the classic film National Lampoon’s Vacation, Clark Griswold uses a simple computer program to map out the trip from Chicago to Los Angeles. He wants to get the family to WallyWorld and “maximize the fun time without missing out on any of the good stuff along the way.”

As Clark attempts to show what Day One will look like, the kids chase his car across the map with a pac-man-like character and a space ship, and Clark has to take evasive action to avoid being eaten or shot. 

 

What an apt metaphor for being a product manager. You get things all mapped out then reality comes along and things change fast. How do you stay on track in the midst of constant change? You focus on your customers.

You see, product development roadmaps are simply guides to how your product will evolve over time. It’s totally okay if they change when you learn new things about your customers’ needs, your business goals, and how they intersect. The roadmap just needs to be directionally accurate, not a blueprint for what must happen. For your roadmaps to remain directionally accurate you need to consistently check where you are in relation to where you want to go. And that’s best done by staying in tune with your customers’ needs — but not always in the ways you might assume.

 

Adding new features isn’t always the answer.

Most products have a backlog of new feature and update requests. Whether you’re building something new or breathing new life into an existing product, you have to be careful about what you choose to add. The truth is most features never get used (see the Pareto principle). Adding new things at the request of customers with hopes it will convince them to continue to pay for the product is a risky move.

usage-chart

The cost of adding new features is more than just the time and effort it takes to design, develop, and deploy it. You have to consider the effort it will take to train your team, update your documentation, create new sales or training materials, and the added effort of your support team answering questions about that feature for a long time. There’s also a chance that adding new features breaks something else in the product. Then there’s the opportunity cost of adding that feature instead of some other enhancement to the product.

Sometimes removing features is a good choice. As seen above, up to 24% of features are never used. That’s an opportunity to streamline and simplify the product so that your teams on focus on the things that are used, and that need either refinement or simply more promotion in the product.

 

Use customer insights to validate your roadmap.

Doing some basic research into how your product is used yields very helpful insights. A simple tool to uncover where you should be spending your efforts is the Feature Adoption Matrix. It splits features into one of four quadrants: Diamonds, Hidden Gems, Diamonds in the Rough, and Fool’s Gold.

feature-adoption-matrix

To use this matrix you need to know two things; Usage and Sentiment. First let’s talk about usage.

Gathering Usage Insights.

There are 4 steps to measure usage, and you want to understand the percentage of people who reach each step for each feature. There are lots of analytics tools available to track and report on feature usage. Find one that is a good fit for your technology and budget requirements and get it into your product as soon as possible. Understanding feature adoption without these analytics is like trying to drive cross country without a map. You might get there, but you’ll waste a lot of time and energy doing it.

The Feature Adoption Funnel

  1. Exposed - percentage of our users who were exposed to the feature (do they know it exists?)
  2. Activated* - percentage of users who actually activated the feature (did they turn it on?)
  3. Used - percentage of users who actually used the feature at least once
  4. Used again - percentage of users who used the feature more than once across separate sessions

*Note: Some features don’t require activation. If the feature you’re measuring doesn’t require the user to perform any action to "turn it on" or enable it, then you can skip this step entirely.

These steps only loosely align with the quadrants in the matrix. At first pass you might think features with a lot of exposure and low usage are Fool’s Gold, and features with a high reuse are Diamonds. You’re on the right track, but there’s more to it. You need User Sentiment for a more complete picture.

Collecting and Understanding User Sentiment

Sentiment can be difficult to understand. Where Adoption is a clear quantifiable number, Sentiment is a more qualitative measurement. To understand user sentiment for any given feature you have to look at several things:

  1. Support requests related to the feature
  2. Application analytics surrounding that feature, specifically:
    • Did users abandon using the product immediately after unsuccessfully using the feature
    • “RageClicks” - when a user furiously clicks on something trying to get it to perform the expected action
  3. Questions and comments about that feature in public forums and user groups
  4. Questions and comments about that feature on social media

As you can see, there can be a lot of information to gather, distill, and analyze. Fortunately, as with the analytics tools for measuring usage, there are a number of great tools for analyzing sentiment. Find one that suits your needs and budget and start using it as soon possible.

But what about new features?

Glad you asked. Mapping out new features is a slightly different task. You can’t really measure usage of something that doesn’t exist yet, but you can get an idea of what people need through doing research. We advise all our clients to form a Customer Advisory Council and use Empathy Mapping to understand what people need.

Once you have an idea for what people need you can build prototypes and test them with your Customer Advisory Council. From there you can get an idea for the value of the new feature and prioritize it against the rest of your roadmap.

Putting it all together.

Now that you have information about feature utilization and sentiment, you might find that a feature that isn’t used often is very important to a certain group of users. This is likely to be a Hidden Gem.

You might also find that a feature you thought was critical to the product isn’t used very often at all and users are indifferent about its use, which is likely Fool’s Gold.

Knowing this you can place these features on the Matrix and better prioritize where you'll spend your team’s efforts. This might seem like a lot of work, but what you’ve done is help defend yourself against the pac-man and space ship of uncertainty and change.

Takeaways:

  • Understand the requests you get from customers, and push back on them. Try to see what they REALLY need. Sometimes a feature exists that will solve their problem, they just didn’t know it.
  • Survey look-alike customers (people who have the same attributes as your target market) to measure how many people might use the feature.
  • Estimate the total effort and costs for the feature so you can prioritize it with others.
  • Build simple prototypes for a few ways to implement the feature/update and test them with the customers who requested the features.
  • Look at task completion % and overall satisfaction.
  • Retire features that aren’t used.
  • Promote the features that are underutilized.
  • Enhance the features that need it.

Note: Roadmaps and Release Plans are different things.

Sometimes the C-Suite just wants to know when something will be released. It’s a fair question to ask, especially since they are concerned with the overall strategy and have to coordinate efforts across multiple parts of the company. In these cases they are asking about a release plan, not the product roadmap.

Remember, a product roadmap outlines long term assumptions about what should be built. Your release plan is a short-term view of work that’s been agreed upon, defined, and planned for development and release to help the team coordinate delivery. The roadmap helps with strategy and discovery, checking assumptions long before you enter any sort of delivery mode. A release plan is all about delivery mode. New features or enhancements don’t go on the release plan until the team decides they will be built. 

You can avoid confusion and painful conversations by keeping these two documents separate. Then stakeholders won’t be tempted to ask when a new feature will be shipped before you’ve even validated that it should be developed in the first place.