Opticon 2017 How Developers Can Take Experimentation
Download as PPTX, PDF0 likes615 views
The document discusses how developers can take experimentation to new levels with full stack experimentation. It describes an architecture that allows experiments across platforms and flexibility to experiment deep within the technical stack. Examples are given of how full stack experimentation could be used for tasks like A/B tests, feature rollouts, and algorithm changes. The benefits of the SDK architecture are outlined as being flexible, stateless, and replaceable.
Opticon 2017 How Developers Can Take Experimentation
- 1. opticon2017 How Developers can take Experimentation to New Levels Michael Ng Software Eng, Optimizely Ali Rizvi Software Eng, Optimizely Josh Wang Software Eng, Optimizely
- 6. opticon2017 $100 million $50 million Mike Ng Business Impact
- 8. opticon2017 How Developers can take experimentation to new levels with Full Stack • Architecture that allows developer flexibility • Experiment across platforms • Experiment deep in the stack
- 10. opticon2017 How experimentation works with Full Stack Your servers SDKs Results Experiment management datafile events Flexible SDKs Optimizely Dashboard A B Node Java Ruby Python PHP C# Javascript iOS Android User devices
- 11. opticon2017 Search algorithms Redesigns Feature rollouts Sorting algorithms Price changes Backend changes Use cases for Full Stack experimentation
- 12. opticon2017
- 13. opticon2017 SDK architecture: Built for flexibility • Stateless • Consistent implementation across SDKs • Non-blocking API calls • Replaceable components
- 14. opticon2017 Creating an SMS experiment with Full Stack
- 15. opticon2017 SMS experiment “Are you interested in learning more about Optimizely Full Stack?” Variation B “Want to learn how to experiment easily with Optimizely Full Stack? Reply "YES" for a guide to best practices!” Variation A Hypothesis: Having a clear call to action will increase positive response rate.
- 16. opticon2017 Text “Hello” to (415)-980-8918 to participate in our live experiment
- 18. opticon2017
- 20. opticon2017
- 21. opticon2017 Problem • Email is being sent, but customers are not changing their password. • Potential risk because if email is compromised, then Optimizely account can be compromised. Potential solutions • Send a reminder email. • Have a call to action in the email. • Link to the right place to make it easier for the user to change the password. Hypothesis More people will change password.
- 22. opticon2017 Original email Variation email
- 23. opticon2017
- 27. opticon2017 Hypothesis We can use performance as a way to determine which tool to use. Problem Pick best tool to do cross-browser testing.
- 30. opticon2017 • Need to do code deploy to rollout feature. • Need to do code deploy to roll back feature. • Not easy to do feature randomization amongst your users. • No easy way to figure out feature access amongst users. • Generally, no access control. • No one source of truth. How we do feature management today
- 32. opticon2017 Takeaways • Full Stack enables experimentation across platforms • The Full Stack architecture allows for greater developer flexibility • Experimentation can happen in any part of the stack
- 33. opticon2017 Q& A
- 35. opticon2017