Everyone wants to measure AI impact, but they need to start by measuring and driving adoption. Most companies I talk to don't have full adoption of these tools. Fewer feel confident that everyone's using them effectively. To track AI adoption in DX, we pull in adoption data directly from leading vendors like GitHub Copilot, Cursor, Amazon Q, and more. Once adoption data is in, you can view and analyze it in several ways: - Use the AI utilization report to understand overall and tool-specific AI adoption - Analyze AI usage on a team and persona level to spot gaps where adoption is lagging - Send event-based, in-the-moment surveys to see which AI use cases are working best - Identify power users who can promote successful approaches You can also create custom attributes for deeper comparisons. For example, you could tag developers in an enablement cohort and track their usage before and after the training. Once you understand how these tools are being used, how often, and who is using them, you can start measuring impact accurately.
Augment Code Pretty please give us an API so we can plug this into DX. :-)
This is great. Esp. last two points. I have used a ‘pioneer cohort’ approach to create ‘internal product champions’, and fine-tune my product/approach for new users based on their feedback.
FYI Oliver Latka and Sumeet Shetty
Well this is awesome. Much better than my current method of being able to see co-pilot use but otherwise have to ask every month or so what people are using and why 🤣
Pretty cool. I’m curious to understand how this would be tracked for data engineering and platform engineering. For example, I’m planning a large-scale Teradata migration spanning 8–12 months, with the primary focus on DevOps and data pipeline development.
PMO Consultant | Helping FAANG & FAANG-Aspiring PMs Land $200K+ Roles & Get Promoted | Reframe Your Narrative & Build Strategic Visibility, In the Room, Online & On Paper | Ex-Amazon
2wSuch an important distinction. Measuring AI impact without first ensuring adoption is like measuring ROI on a tool no one’s using. I’ve found the real breakthroughs come when you track how teams are using AI, then elevate the power users to model best practices.