How do you assess the impact of changes to your product?
Product Management
Perspectives from experts about the questions that matter in Product Management
This article was an early beta test. See all-new collaborative articles about Product Management to get expert insights and join the conversation.
When you make changes to your product, whether it's a new feature or a design tweak, you need to assess the impact of those changes, and determine whether they're moving your product in the right direction. This assessment is important because it helps you understand whether the changes you made were successful, and what you might want to consider for future iterations. Here are some ways product managers can measure the success of changes made to their product.
1. Track user engagement and satisfaction: Data around user engagement and satisfaction can be helpful in terms of understanding the success of your implemented changes. If you see a spike in user engagement or satisfaction rates after you release a new feature or make a design change, this can be a good indication that the changes you made resonated well with your target audience.
2. Conduct user research and surveys: User research, such as interviews or focus groups, can give you and your team valuable qualitative insights. It can also help you understand how people are using your product, what they like or dislike about it and what they think of your recent changes.
3. Consider the competitive landscape: Oftentimes, assessing the impact of changes isn't as straightforward as tracking metrics or conducting user research. Sometimes you need to consider other factors, such as how these changes may have affected your competitive landscape. Consider asking yourself, did you gain or lose market share after releasing a new feature? Or did your changes allow you to carve out a new niche in the market?
4. Go back to your goals: You should have specific goals in mind before you make any changes to your product. Having such objectives and returning to them as you introduce your changes can help you stay intentional about the decisions you make.
Explore more
- How to Measure the Impact of Product Launch or Key Product Change by Piotr Janecki for Startup House
- Measuring Impact — Picking the Right Metrics in Product by Curtis Stanier for Medium
- A Framework to Measure the Impact of Product Change by Hannah Feroz for HelloMeets
This article was edited by LinkedIn News Editor Felicia Hou and was curated leveraging the help of AI technology.
Product Manager | eCommerce, developer tools | T-Mobile, Expedia
2yTracking user engagement itself has a huge scope. You track funnel stages, drop-offs, user engagement heatmap, average time spent. Additionally, measure if the feature is contributing to the north-star/goal of the product. A/B testing leads to a hypothesized/expected value. Validating this value with the actual value gives a greater insight of the success of a new feature launch.
Customer segmentation to perform A/B testing, analyzing user behavior on various experience types, and allowing customers to switch back to the previous version(if possible) to understand user acceptance of new features and to help reduce the churn rate..
Putting LLMs to Work | x-Amazon, Adobe, SAP
2yWhat is the north star of your product? You have the measure the success of your product on that. Is your product aiming to get more user, make them post content, add impressions, catering relevant content, etc. Each can be measured using a different metric. The impact of incremental changes will also show up in your north star metric through A/B.
Strategic Visionary | Business Strategy & Roadmap Development | Catalyst for Industry-Leading Results
2yIt is really cumbersome to measure the impact of a product change especially in current fast-paced environment until and unless it is a big change impacting significant user experience. It is better to defined the entry criteria, toll gates and success criteria with clear metrics defined and how these metrics can be captured in apple to apple comparison. There are scenarios where external factors play a big role in measurement and will impact metrics measurement too.
Senior Staff Product Manager @ EA | API Platforms | Microservices | AI/ML | Data
2yWhile user engagement and market share are the two final leading KPI's , the below signals can provide incremental information around product success , in certain cases before the product is publicly launched. 1. What changes do we see in customer adoption rates during private preview ? Are pilot customers adopting the product earlier , faster and in higher numbers ? Also are previously laggard customers adopting the product earlier in the adoption cycle. 2. Whats the feedback from pilot customers ? 3. How is customer adoption of this feature impacting other common areas of interest we have with our customers ? 4. Has our internal engineering support burden on maintaining the product reduced or increased ? 5. Has the Cost of Good Sold(COGS) increased or decreased with the change ? How is gross margin impacted with the change ? 6. How is the product positioned now with respect competition? What's the cost to performance ratio ( if applicable ) and has it improved ? 7. How many customers on competitive offerings have migrated to your product after the changes were implemented ?