Atlassian's Mystique CLI, Minimizing the Experiment Development Cycle
0 likes410 views
The document outlines the challenges and solutions in using a command line interface (CLI) for managing experiments, emphasizing the issues with repetitive manual steps and the potential for defects. It suggests a customized CLI that streamlines the experiment development cycle, ensuring consistency and efficiency while integrating with systems like Optimizely. The document also includes code snippets and commands necessary for setting up projects, creating experiments, and handling audience targeting.
![{
"Project": {
"key_metrics": [ 12345, 45678, 98765 ],
"prefix": "www.website.com",
"project_id": 3232345,
"qa_audience": 2975832345,
},
"QA Project":
{
"key_metrics": [ 53682, 48154, 70962 ],
"prefix": "qa.stg.website.com",
"project_id": 592823671,
"qa_audience": 8275362345,
}…
}
JSON
everywhere
Project config
Experiment
Personalization
Events](https://image.slidesharecdn.com/opticon-mystiquecli-nathancaspar1-200916194542/85/Atlassian-s-Mystique-CLI-Minimizing-the-Experiment-Development-Cycle-32-320.jpg)
![{
"type": "ab_experiment",
"jira_ticket": "BXPSCI-123",
"optx_project": "QA Project",
"experiment_name":"Opticon",
"experiment_description": "Demo for Opticon”,
"variations": ["Opticon demo"],
"page_paths_ids": ["/", "/software"],
}
JSON
everywhere
Project config
Experiment
Personalizatio
n
Events](https://image.slidesharecdn.com/opticon-mystiquecli-nathancaspar1-200916194542/85/Atlassian-s-Mystique-CLI-Minimizing-the-Experiment-Development-Cycle-34-320.jpg)
![JSON
everywhere
Project config
Experiment
Personalizatio
n
Events
{
"type": "personalization_campaign",
"jira_ticket": "BXPSCI-123",
"optx_project": "QA Project",
"campaign_name": "Opticon campaign",
"page_paths_ids": ["/software","/try"],
"experiences": [{
"exp_name": "Opticon experience 1",
“variations": [ "first variation”,
"second variation”]
},{
"exp_name": "Opticon experience 2",
"variations": [ "first variation v2",
"second variation v2"]}]
}](https://image.slidesharecdn.com/opticon-mystiquecli-nathancaspar1-200916194542/85/Atlassian-s-Mystique-CLI-Minimizing-the-Experiment-Development-Cycle-35-320.jpg)
![JSON
everywhere
Project config
Experiment
Personalizatio
n
Events
{
"type": "custom_events",
"project": "QA Project",
"events": [
{"key":"api_key_1", "name":"event name 1"},
{"key":"api_key_2", "name":"event name 2"},
{"key":"api_key_3", "name":"event name 3"}
]
}](https://image.slidesharecdn.com/opticon-mystiquecli-nathancaspar1-200916194542/85/Atlassian-s-Mystique-CLI-Minimizing-the-Experiment-Development-Cycle-36-320.jpg)
Atlassian's Mystique CLI, Minimizing the Experiment Development Cycle
- 1. NATHAN CASPAR | SR. GROWTH ENGINEER | @NATHAN CASPAR Mystique CLI Minimizing the experiment development cycle
- 6. Why take the time and effort to build out a command line interface?
- 8. MANUALLY CREATE EXPERIMENT MANUALLY UPDATE EXPERIMENT COD MANUALLY DEPLOY EXPERIMENT MANUALLY TEST EXPERIMENT
- 10. MANUALLY CREATE EXPERIMENT MANUALLY UPDATE EXPERIMENT COD MANUALLY DEPLOY EXPERIMENT MANUALLY TEST EXPERIMENT • Create & add audience targeting • Create & add page targeting • Create & add events • Create variations • Set traffic allocations • Develop experiment code in local • For each variation • For each page • Copy local code into web editor • Save experiment • Publish experiment • For each targeted page • Test page conditions • Test audience conditions • Test experiment visuals • Test for bugs
- 11. MANUALLY CREATE EXPERIMENT MANUALLY UPDATE EXPERIMENT COD MANUALLY DEPLOY EXPERIMENT MANUALLY TEST EXPERIMENT • Create & add audience targeting • Create & add page targeting • Create & add events • Create variations • Set traffic allocations • Develop experiment code in local • For each variation • For each page • Copy local code into web editor • Save experiment • Publish experiment • For each targeted page • Test page conditions • Test audience conditions • Test experiment visuals • Test for bugs
- 12. ANY CHANGE MEANS A LOT OF SMALL MANUAL STEPS
- 13. WHICH ADD UP
- 14. AND UP…
- 15. AND UP!
- 16. MANUAL STEPS ALSO MEAN DEFECT OPPORTUNITIES
- 19. Why not use an existing CLI?
- 20. W Design Customization in how we design experiments Development Customization in how we build experiments Analytics Customization in how we perform post-analysis
- 22. Be the change you seek
- 23. The many faces of Optimizely
- 24. Operations Optimizely UI performance Optimizely snippet performance Optimizely maintenance Developer Simple development cycle Keeping repo and Optimizely in sync Quickly deploying and testing updates Understanding when/where experiments ran Accessing Optimizely results and raw data Communicating all things experimentation PM / Analyst
- 26. Developer Simple development cycle Keeping repo and Optimizely in sync Quickly deploying and testing updates PM / Analyst Understanding when/where experiments ran Accessing Optimizely results and raw data Communicating all things experimentation Operations Optimizely UI performance Optimizely snippet performance Optimizely maintenance
- 28. Developer Simple development cycle Keeping repo and Optimizely in sync Quickly deploying and testing updates
- 30. BASIC ACTIONS
- 31. SETUP Projects $ optx setup -pr TokensTokens GET /projects
- 32. { "Project": { "key_metrics": [ 12345, 45678, 98765 ], "prefix": "www.website.com", "project_id": 3232345, "qa_audience": 2975832345, }, "QA Project": { "key_metrics": [ 53682, 48154, 70962 ], "prefix": "qa.stg.website.com", "project_id": 592823671, "qa_audience": 8275362345, }… } JSON everywhere Project config Experiment Personalization Events
- 33. $ optx create -p=exp.json CREATE | UPDATE | PRUNE P13n Campaigns TokensA/B Experiments POST /experiments Events Pages Audiences
- 34. { "type": "ab_experiment", "jira_ticket": "BXPSCI-123", "optx_project": "QA Project", "experiment_name":"Opticon", "experiment_description": "Demo for Opticon”, "variations": ["Opticon demo"], "page_paths_ids": ["/", "/software"], } JSON everywhere Project config Experiment Personalizatio n Events
- 35. JSON everywhere Project config Experiment Personalizatio n Events { "type": "personalization_campaign", "jira_ticket": "BXPSCI-123", "optx_project": "QA Project", "campaign_name": "Opticon campaign", "page_paths_ids": ["/software","/try"], "experiences": [{ "exp_name": "Opticon experience 1", “variations": [ "first variation”, "second variation”] },{ "exp_name": "Opticon experience 2", "variations": [ "first variation v2", "second variation v2"]}] }
- 36. JSON everywhere Project config Experiment Personalizatio n Events { "type": "custom_events", "project": "QA Project", "events": [ {"key":"api_key_1", "name":"event name 1"}, {"key":"api_key_2", "name":"event name 2"}, {"key":"api_key_3", "name":"event name 3"} ] }
- 37. COMPLEX ACTIONS
- 39. EXP launch PROD DUPLICATION EXP launch PROD create We want confirmation duplicates Time to product ionize
- 40. QA
- 41. Future Vision Automating clean up Handling project JS changes Exposing experiment metadata Working to open source