DevOpsDays SLC - Platform Engineers are Product Managers.pptx

Editor's Notes

• #4: We live in a world where software freedom principles have merged with cloud technologies and productivity theory, and given rise to these platforms. However, these platforms are increasingly complex, made of many different sources of automation and orchestration, some commercial, some open source, and all heterogeneous and, in the case of large enterprises, mostly sprawling. This is the familiar landscape diagram of the Cloud Native Computing Foundation, but lets be honest, its become an overwhelming picture. It’s this landscape that can be thought of as the problem statement for platform engineering, providing a layer of best practice and repeatable processes for engineers.
• #7: To do this well, Platform Engineers must adopt a composite mindset, taking knowledge from multiple areas of discipline and combining into the new practice of platform engineering. Platform engineers must take a blend of DevEx, DevOps, and, as we’ll explore in this presentation, Product Management Platform Engineering itself should not be thought of as mutually exclusive to DevOps, but rather the incorporation of DevOps principles, Developer Experience concerns, and product management discipline.
• #8: Despite knowing this, even the rise of DevOps didn’t fully include the experience of Developers. There’s a whole timeline, going back to the JIT Manufacturing in the 70s where we’ve tried to figure out ways to measure and optimize productivity, each phase adapting to new cultural norms, and iterating on and adapting what was learned in the past.
• #9: This makes the role much easier to understand, because we can apply decades of product management wisdom to the way we think about building and releasing our platform. When we think of the platform as a managed product, we can use the same principles for product packaging and release that we already use for our other products. Further, we can collect feedback from our users in much the same way that we would from our customers It’s key if you want to maximize the benefits of the platform to not treat the platform as a second-class product, but to give it the same priority that you would for products that you ship The business value cannot be overstated, by providing this level of focus you will build an excellent platform that will have major positive impacts to the overall throughput and productivity of your teams
• #11: Thinking about the key responsibilities of a product manager, when we treat our platform like a product, we can manage it in the same way that we’d manage any other product. The platform engineer should adopt the same responsibilities for the platform as they would for any other product. They should go well beyond the role of simply answering tickets, they should be evaluating feedback about the platform, solving pains and challenges proactively, defining releases and roadmap strategy, and reporting on platform performance and success metrics.
• #12: We should think about the development of our platform in the same way that we have learned to develop our other products This includes taking an Agile approach, releasing incrementally and often, and in a fully packaged and managed way Feedback should be gathered regularly, and used to inform new releases of the platform. Don’t build just for the sake of building, ensure that you understand the use case and value of each feature and iteration of the platform. Keep all platform configuration in SCM, and use automation tools to deploy the platform based on that configuration.
• #13: As a Product Manager of the platform, your role should expand beyond simply taking orders and delivering features to thinking forward and anticipating the needs of your users, just as you’d do for your business customers. Gather a range of feedback, both quantitative and qualitative about the use of the platform. UX is key, and because of that, every well designed platform needs an internal developer portal, and the product manager should be responsible for its upkeep. We would argue that a platform isn’t even really a platform without a portal, it might be a devops pipeline or a developer toolchain, but its not a platform without a UX layer.
• #14: The platform represents dozens of individual systems, each with their own metrics. No one set of metrics will paint a full story of your platform’s health, value, and performance. Combine multiple known metric frameworks, which becomes much easier to do when you have an Internal Developer Portal. DORA metrics and SPACE framework metrics are proven and potentially useful. Understand that onboarding metrics and platform adoption percentages are critical for understanding how well the platform is being received, and how much value its providing.
• #15: The platform represents dozens of individual systems, each with their own metrics. No one set of metrics will paint a full story of your platform’s health, value, and performance. Combine multiple known metric frameworks, which becomes much easier to do when you have an Internal Developer Portal. DORA metrics and SPACE framework metrics are proven and potentially useful. Understand that onboarding metrics and platform adoption percentages are critical for understanding how well the platform is being received, and how much value its providing.
• #16: DORA was really the first on the scene to try and measure DevOps outcomes. not only do the four key DORA metrics specifically apply to software engineering, but it gives a direction of what to do to get better. So this is where a lot of engineering leaders started, or do start today.
• #17: DORA is over a decade old, an in addition to these four key metrics there is a lot more to the organization. These metrics were popularized in Accelerate. This is a book about software delivery and devops, not about developer productivity. and since then, our understanding of it has changed a lot. What were orgs struggling with in 2015? We live in a different world now. Would you rely on a webapp book from 10 years ago?
• #22: And with this new focus on Developer Experience, we started to see the evolution of DORA. The industry got closer to a better definition of productivity in 2021 when the SPACE framework was released. SPACE was sort of a sleeper hit. Notice Nicole Forsgren who led the DORA initiative, and see that she is joined by a number of other researchers, Margaret-Anne (Peggy) Storey, and a number of researchers from Microsoft.
• #23: A pervasive challenge for platform engineers is properly and effectively measuring the impact of their platform on things like developer experience and productivity. We’ve learned that no single metric can tell us exactly how effective our platform is, so we know we need to measure against multiple metrics. But should we measure the quantitative qualities of our platform, or should we rely on qualitative metrics such as survey data?
• #24: SPACE expands upon DORA, learning that a single metric set will not be enough to gain a full understanding of developer experience and productivity. Rather, we must look at “a constellation of metrics in tension with one another”
• #27: Developer experience is all about going to the developers for the answers. So we can start an improvement cycle by asking devs about current conditions, setting targets, using other metrics to track progress, and then evaluate whether we’re reaching our goals.
• #31: Emphasize Cost of Qual metrics is high, time spent filling out surveys, etc, have to be repeated to get a trend
• #35: Now we are getting somewhere!! We have some great sets of metrics with precedent, validated by real engineering teams… But could we streamline this even further? Each of these metrics can influence other metrics, we learned that from SPACE, could we reduce these based on those correlations?
• #36: YES! Presenting: the most modern developer experience and productivity metric framework available, the DX Core 4. The most effective metrics have been pulled by these three major metric sets, and move into four new dimensions of Speed, Effectiveness, Quality, and Impact
• #38: Zooming in, we can see that our Key Metric for Effectiveness is something called the DXI
• #41: The DXI is a formative measure of how, and to what extent, the right conditions exist for developers to be able to deliver effectively. The DXI measures 14 dimensions (Figure 2) found to be actionable (and changeable) at both the manager and organizational level, such as deep work, local iteration speed, release process, and confidence in making changes. DX uses linear regression analysis to assess the strength of the relationship between the DXI and time waste, and for modeling the predictive relationship between them. The most recent analysis of DXI was conducted in early 2024, based on a sample of 50,000 responses from 200 organizations. The DXI is calculated by taking the mean 16 items corresponding to key dimensions of developer experience. Efficiency is calculated through a self-reported outcome measure of time loss due to inefficiencies in the work environment. Past research by DX identified a comprehensive set of factors underlying developer experience. In designing the 14 dimensions included in the DXI, our researchers took into account both the actionability of issues for management and predictive power against outcomes such as engineering velocity, quality, and efficiency.