ConversionSummit 2015 - Testing culture: making testing look easy!?
0 likes1,104 views
Martijn Scheybeler discusses how to establish and scale a testing culture within a marketing team, sharing insights from his experience at The Next Web. He details the team's structure, dynamics, and processes for creating and analyzing tests, while emphasizing the importance of collaboration and resource allocation. The presentation concludes with key takeaways for fostering a successful and innovative testing environment.
ConversionSummit 2015 - Testing culture: making testing look easy!?
- 1. Testing culture: making testing look easy!? Martijn Scheybeler / Lead Growth, SEO & Analytics / martijn@thenextweb.com / @MartijnSch
- 2. EASY, WHAT?
- 3. • Leads Marketing: Growth, SEO & Analytics • Marketing team The Next Web: 12 (60+) • Into testing for the last 3 years • Weekly running 10 new tests, we can do 15-20 :( • Scaling, coding and automating ConversionSUMMIT 2015 - @MartijnSch Who is this guy?
- 4. What is The Next Web? ConversionSUMMIT 2015 - @MartijnSch
- 5. 7 million unique users, 13 million pageviews
- 6. TNW Conference: Amsterdam & New York: 20.000 attendees
- 7. TNW Deals: The online marketplace for digital tastemakers
- 8. TNW Academy: Hand-picked online courses to improve your professional life
- 9. ConversionSUMMIT 2015 - @MartijnSch • From no testing to …? in 6 months • Setting up a testing culture from scratch • Scaled from: 1 test a month > 10 a week • 10 NEW tests as week, the ability to do: 15 What qualifies me to speak here?
- 10. ConversionSUMMIT 2015 - @MartijnSch • How to create a culture of testing? • How were / are we able to scale so fast? • How it influences our (company / team) culture? • Summary: How do YOU scale your testing program? What do I like to talk about…
- 11. ConversionSUMMIT 2015 - @MartijnSch • Team structure & dynamics • Creating experiment ideas + planning • Creating & coding experiments • Analysing experiments • What’s next? … and it goes something like this
- 12. ConversionSUMMIT 2015 - @MartijnSch • Design, Development, QA, Project Management, Legal > CEO approval: you’re doing it wrong! • Can you design, code & analyse? You don’t need a team. • Cross-functional or not? Team structure
- 13. ConversionSUMMIT 2015 - @MartijnSch TNW Style: Team structure
- 14. ConversionSUMMIT 2015 - @MartijnSch • 1 Web Analyst: Designing, Coding & Analysis • 2 Interns: Analysis + Coding • Me: Strategy, Coding, Analysis • Support: 1 designer + 1 front-end developer Not cross functional (at the moment) TNW Style: Team structure
- 15. ConversionSUMMIT 2015 - @MartijnSch • “Our developers won’t let us change the code” • “I need to run our tests against the legal and *BS* team” • “We’re still waiting for the design reviews” • “We don’t think A/B testing can improve our results” • “I asked my girlfriend to test the site, she didn’t have issues” Team dynamics
- 16. What does our culture looks like?
- 18. ConversionSUMMIT 2015 - @MartijnSch • “This is how you access our live servers, need GTM as well?” • “Legal, who?” • “We still have 5 design reviews laying around, DAMMIT!?” • “10 new tests this week! YOLO!” TNW Style: Team dynamics
- 19. ConversionSUMMIT 2015 - @MartijnSch • A lot of ideas, but no REAL ideas: button colours? • =AVG(TIME*0,272+EFFORT*0.123+HAIRCOLOUR*0,605) • A planning heavily discussed with the … Experiment ideas + planning
- 20. Make sure you don’t have a HIPPO in the room
- 21. Boris (CEO) “I had a very good idea this weekend! Can we make experiments more aggressive.”
- 22. ConversionSUMMIT 2015 - @MartijnSch • Ok, we do have a planning • “It doesn’t need to be rocket science” • We don’t have a huge backlog TNW Style: Planning, Planwhat?
- 23. ConversionSUMMIT 2015 - @MartijnSch • Design by committee, according to style guides • Won’t get you a lot further • Outsourced testing? In-house teams (with no time)? Creating & coding experiments
- 24. ConversionSUMMIT 2015 - @MartijnSch • Designer: “I created a new design, we should test it!” • Developer: “I looked at your A/B testing script: refactored it” • Analytics: “We have a backlog but we could use new ideas” • Me: “Can we do 20 tests a week?” TNW Style: Creating experiments
- 25. Deploying in 3..2..1! Flexibility v.s. Scale v.s. €0 Costs
- 26. ConversionSUMMIT 2015 - @MartijnSch • Designed by our CRO team • Working with our design team to test very big things • Responsible: no way of hiding behind bad code TNW Style: Creating experiments
- 27. ConversionSUMMIT 2015 - @MartijnSch • Google Tag Manager • A/B Testing & Multivariate Testing: http://tnw.to/testingJS • Custom JavaScript / jQuery for changes • Supported by the dataLayer for targeting / segmenting • You could do better: back-end testing! TNW Style: Coding experiments
- 28. ConversionSUMMIT 2015 - @MartijnSch • Google Analytics? Adobe Analytics? • You do make sure to check for power + significance, right? • You’re also tracking secondary KPIs? • You decide to ‘re-test’? Analysing experiments
- 29. Automated reporting via TNW Analytics: -20%
- 30. ConversionSUMMIT 2015 - @MartijnSch • Google Analytics API • Custom Dimensions / Event Tracking • Automated reporting • Ultimate pleasure and fun + no annoyed team members TNW Style: Analysing experiments
- 31. TNW Analytics: Adding experiments + ideasAutomated reporting via TNW Analytics: -20%
- 32. ConversionSUMMIT 2015 - @MartijnSch • Take it easy… do you know what to do? • First: make friends! • Second: make sure you know what you’re doing! • Third: map out your strategy, do you have the resources? • Worst case? Read back the slides! What’s next for you?
- 33. ConversionSUMMIT 2015 - @MartijnSch • Grow our team: 15 people? • Cross functional, a mistake? • Automate & Innovate even more • Drink more … TNW Style: What’s next?
- 34. ConversionSUMMIT 2015 - @MartijnSch • Create / Embrace a culture of testing: don’t create silos • Make the whole team + organisation come up with ideas • Design ideas yourself or ask your designer(s) • Learn to code so you can create experiments yourself • Want to scale? Make sure you have the resources to automate Take aways
- 35. ConversionSUMMIT 2015 - @MartijnSch Martijn Scheybeler Lead Growth, SEO & Analytics - TNW Email: martijn@thenextweb.com Twitter: @MartijnSch Have fun! Thank you! Slides: http://tnw.to/csummit Code: http://tnw.to/testingJS
- 36. ConversionSUMMIT 2015 - @MartijnSch