![[-Peter]
Johannesburg
Basel
Vancouver](https://image.slidesharecdn.com/testinginproductionwithfeatureflags-180526052221/85/Testing-in-production-with-feature-flags-2-320.jpg)
Testing in production with feature flags
- 1. Visual Studio Team Services
- 4. testing
- 9. Init (user + hash) getHashKey (token, vstsaccount) hash checkToken getHashKey return all user flags
- 10. ISSUE VALUE test if ( flag ) else LaunchDarkly VSTS Service EndPoint VSTS Extension get flagquery flagON OFF SaaS Management
- 11. ISSUE VALUE if ( flag ) else LaunchDarkly VSTS Service EndPoint VSTS Extension get flagquery flagON OFF SaaS Management
- 12. testing ISSUE VALUE if ( flag ) else ON OFF SaaS Management LaunchDarkly VSTS Service EndPoint VSTS Extension get flag Azure Function set flag query flag REST API calls
- 13. testing production A|B testing production Symbiosis production FF FF
- 14. HYPOTHESIS - We believe that LaunchDarkly will … VALIDATED? Integrate seamlessly into our VSTS extensions Validated Enable us to decouple deployment and exposure of extension features Validated Enable us to provide control down to individual user for ext. features Validated Enable us to make changes without deploying our extensions Validated Support the OSS community Validated Allow us to quickly switch off / revert an extension feature Validated Enable us to support early testing, feedback, and experimentation of extension features Validated
Editor's Notes
- #4: So, why did we embrace feature flags?
- #5: They allow us to decouple deployment and exposure of FEATURES. They provide runtime control of FEATURES down to the individual user. They allow us to make FEATURE changes in production without reployment. They support early testing, feedback, and experimentation. And last but not least, they offer an emergency FEATURE power off switch.
- #6: https://blogs.msdn.microsoft.com/bharry/2013/11/25/a-rough-patch/ But before you move all your software behind feature flags, please read Brian Harry’s blog post, titled “a rough patch”. We flipped feature flags globally just before a keynote. It did NOT go well. You’ll notice the initial outage and ripples in availability as we tried to recover. It was a bad user experience.
- #7: For VSTS we’re using a custom feature flag solution. For VSTS extensions we looked at custom, open source, and service based feature flag solutions. We opted for LaunchDarkly as they offer a service, integrate with TFS and VSTS, and support the community space.
- #8: Here’s a few examples of VSTS extensions used on our dashboard. As the name implies, they extend VSTS.
- #10: ALL research and outcomes are shared on our team blog. We also shared JavaScript and Azure Function sample code on GitHub.
- #11: Let’s look at three scenarios we evaluated as part of our research. When we have a hidden feature, we want to avoid a risky re-deployment. “It’s working, don’t touch it”. Using feature flags we can flip a switch to expose and TEST the feature.
- #12: We enabled telemetry in all our VSTS extensions and flooded everyone’s mailbox and dashboards. To ensure we do not ignore telemetry as noise, we can use a feature flag to enable and disable telemetry as needed.
- #13: Lastly, we often need detailed logs when troubleshooting an issue. With feature flags we can avoid another re-deployment AND we can give the user the control to enable and disable verbose logging.
- #14: The result of our research demonstrated that feature flags enable: Granular feature releases and testing in production. Selected feature releases and A|B testing in production. Symbiosis with deployment rings in production. As shown, we’re deploying every release to Canary, Early Adopter, and end-users rings in production, using feature flags to fine tune features in each environment.
- #15: We had seven (7) hypothesis, all of which we were able to validate.
- #16: For more information, please refer to these links.
- #17: Questions?