SlideShare a Scribd company logo
TITLE
Nice things
People (also $$$)Time (aka $$$)
☹
you
Nice things
People (also $$$)Time (aka $$$)
☺
X
yay
- GEO AT GOOGLE SCALE -
>1 billion active monthly users of Google Maps Services
>1 billion downloads of Google Maps on Android
>2 million active websites and apps use the Google Maps API every
week
10s of thousands of user edits to local data and map data per day
- QUALITY AT GOOGLE SCALE -
Google has the world mapped - 200 countries and territories
Street View imagery is available across 66 countries
The satellite imagery in Google Earth covers more than 45% of the
world's land surface and 80% of the world's population in sub-meter
high resolution.
A problem coming your way?
“We often find that when doing due diligence, that
the team got the thing up and running as fast as
possible and that's fine for getting to £1-5m revenue
but they’ve created scaling issues for themselves
after that.”
- Simon Menashy, MMC Ventures
- TYPICAL DECISION #1 -
Hiring a QA team to fix the quality
problem
Don’t hire the QA army
Quality issue? Quality assurance!
QA is accountable but has very little power to change anything
Manual testing is very slow
- TYPICAL DECISION #2 -
Hiring an automation team to fix the
testing problem
A bad sign
Subject: We’re hiring! TEST AUTOMATION ENGINEER
Join our fast-growing team of TEST AUTOMATION ENGINEERS!
• Low coding skill is okay
• Write all our tests for us
• We have a foosball table and a blimp!
- GOOGLE’S TURNING POINT -
7 years ago, Google teams released to production once or twice a
month, with the majority of the time taken by testing.
Today Google teams release to production daily or sub-daily.
There was no decrease in quality.
BoS2015 Trish Khoo – Engineering Manager, Google
The “Testing Phase”
• No idea when the product will actually launch
• Massive amount of human hours to manage effectively
• Huge waste in expensive engineering hours
BoS2015 Trish Khoo – Engineering Manager, Google
What testing really is
Testing provides expert driven feedback on the state of business
requirement gaps, user impact and overall project quality.
One of the ways we do this is by checking things. Everybody can
check things. It’s easy.
BoS2015 Trish Khoo – Engineering Manager, Google
Checking is easy. Testing is hard.
A testing expert in your development team will:
• Make sure people get information about your product at the optimal
time to make timely decisions
• Make your development process more efficient
• Design tools and infrastructure to make your developers more
productive
CHANGE YOUR CULTURE
- THE PUSHBACK -
We don’t have time to write tests!
We don’t know how!
That’s not the way we work at this company!
HIRE THE RIGHT SKILLS
ENGINEERING PRODUCTIVITY
Software Engineer, Tools & Infrastructure
Test Engineer
- EMPOWER YOUR TESTING EXPERT -
Hard to find, hard to retain!
• First class citizen within the team
• Similar job title and salary to developers
• Needs management support
LEARN, DON’T BLAME
- LEARNING, NOT BLAME -
• Cultivate a postmortem culture
• Trust your people
BoS2015 Trish Khoo – Engineering Manager, Google
>> FAST RELEASES
FAST FEEDBACK <<
EXPERIMENTS
&
DOGFOOD
What’s in it for me?
• Rock solid software
• Ability to quickly change product direction
• Freedom to innovate
THINGS TO DO
#1 Change your culture
#2 Hire the right skills
#3 Learn, don’t blame
#4 Launch and iterate

More Related Content

PPTX
BoS2015 Jeff Szczepanski – COO, Stack Exchange - Stack Overflow. Scaling a Te...
PDF
Agile: the Good, the Bad and the Ugly - Webinar by Clarke Ching Agile - Septe...
PPTX
Agile Israel 2017 bugs zero by Arlo Belshee
PPTX
Agile - A failure story
PPTX
Євген Лабунський: Agile in Enterprise. How do we do it
PPTX
Helping operations top-heavy teams the smart way
PPTX
How to do Estimates (well) in Agile?
PPTX
Lean conference 2014 Open Market - how we have benefited from the application...
BoS2015 Jeff Szczepanski – COO, Stack Exchange - Stack Overflow. Scaling a Te...
Agile: the Good, the Bad and the Ugly - Webinar by Clarke Ching Agile - Septe...
Agile Israel 2017 bugs zero by Arlo Belshee
Agile - A failure story
Євген Лабунський: Agile in Enterprise. How do we do it
Helping operations top-heavy teams the smart way
How to do Estimates (well) in Agile?
Lean conference 2014 Open Market - how we have benefited from the application...

What's hot (20)

PPTX
How pair programming can strengthen teams
PPTX
Anatomy of a Agile Product Lifecycle - Eilon Reshef - Agile Israel 2013
PPTX
Prioritise: Kanban Recipe for Success Step 5
PDF
Does this FizzGood? Improve velocity, predictability & agility by asking a si...
PDF
Іванна Заєць: Основи ПМа (PM’s Essentials)
PPTX
Code Yellow: Helping operations top-heavy teams the smart way
PDF
Spiking Your Way to Improved Agile Development - Anatoli Kazatchkov
PDF
Scrum and DevOps training
PDF
Growing games - hands on execution
PPTX
Agile Metrics Meetup: What to Measure and How?
PPTX
Agile at enterprice level
PDF
Indix Engineering Culture Code (2015)
PDF
Embracing the Consumerization of IT in Your Company
PDF
Does this Fizz Good?
PPTX
Paul Theyers (Assurity Consulting)
PDF
Being agile while standing in a waterfall
PPTX
The D Files: Debunking Myths About Distributed Teams
PPTX
Jira Portfolio by Elad Ben-Noam
PDF
GAC - Agile and Scrum Training
PPTX
Agile Israel 2017
How pair programming can strengthen teams
Anatomy of a Agile Product Lifecycle - Eilon Reshef - Agile Israel 2013
Prioritise: Kanban Recipe for Success Step 5
Does this FizzGood? Improve velocity, predictability & agility by asking a si...
Іванна Заєць: Основи ПМа (PM’s Essentials)
Code Yellow: Helping operations top-heavy teams the smart way
Spiking Your Way to Improved Agile Development - Anatoli Kazatchkov
Scrum and DevOps training
Growing games - hands on execution
Agile Metrics Meetup: What to Measure and How?
Agile at enterprice level
Indix Engineering Culture Code (2015)
Embracing the Consumerization of IT in Your Company
Does this Fizz Good?
Paul Theyers (Assurity Consulting)
Being agile while standing in a waterfall
The D Files: Debunking Myths About Distributed Teams
Jira Portfolio by Elad Ben-Noam
GAC - Agile and Scrum Training
Agile Israel 2017
Ad

Viewers also liked (15)

PPTX
BoS2015 Kristine Woolsey - Solve the Right Problem
PDF
BoS2015 Aaron Aycock - Making The Leap
PDF
BoS2015 David Heinemeier Hansson – Creator of Ruby on Rails, Founder of Basec...
PPT
Bos2015 - We Need to Talk About Unicorns. Values > Valuations.
PDF
BoS2015 - Sarah Allen – Co-Founder, Mightyverse, 18F
PPTX
BoS2015 Robert J Moore - Looking Good for Fun and Profit. When Does Your Comp...
PDF
BoS2015 Tania Katan - It Was Never a Dress
PDF
BoS2015 Claire Lew, Know Your Company. Don’t Be The Last to Know
PDF
BoS2015 James archer - software design mistakes that just wont die
PDF
BoS2015 - Steli Efti - How To Sell Software Using Sales
PPTX
BoS2015 Precious Lunga - how can we use existing tech to make a difference in...
PPTX
BoS2015 Art Papas - The Bullhorn Journey to Customer Focus
PPT
BoS2015 Paul Kenny - Difficult Conversations
PPTX
BoS2015 Matthew Bellows – CEO, Yesware. The Case for Mindfullness at Work
PPTX
BoS2015 Rich Mironov - The Four Laws of Software Economics
BoS2015 Kristine Woolsey - Solve the Right Problem
BoS2015 Aaron Aycock - Making The Leap
BoS2015 David Heinemeier Hansson – Creator of Ruby on Rails, Founder of Basec...
Bos2015 - We Need to Talk About Unicorns. Values > Valuations.
BoS2015 - Sarah Allen – Co-Founder, Mightyverse, 18F
BoS2015 Robert J Moore - Looking Good for Fun and Profit. When Does Your Comp...
BoS2015 Tania Katan - It Was Never a Dress
BoS2015 Claire Lew, Know Your Company. Don’t Be The Last to Know
BoS2015 James archer - software design mistakes that just wont die
BoS2015 - Steli Efti - How To Sell Software Using Sales
BoS2015 Precious Lunga - how can we use existing tech to make a difference in...
BoS2015 Art Papas - The Bullhorn Journey to Customer Focus
BoS2015 Paul Kenny - Difficult Conversations
BoS2015 Matthew Bellows – CEO, Yesware. The Case for Mindfullness at Work
BoS2015 Rich Mironov - The Four Laws of Software Economics
Ad

Similar to BoS2015 Trish Khoo – Engineering Manager, Google (20)

PDF
Nondeterministic Software for the Rest of Us
PPTX
Google Human Resource Management
PPTX
Google: Designed for Innovation
PPT
Agile for Me- CodeStock 2009
PPTX
2019 Top Lessons Learned Since the Phoenix Project Was Released
PDF
DevOps Patterns Distilled: Implementing The Needed Practices In Practical Steps
PDF
Tranformative Culture - The Shift From QA To Engineering Productivity - Selen...
PDF
iTexico Company Overview 2015
PDF
Google Product Dev Process
PPTX
The End of Projects & what to do about it
PDF
SFScon21 - Paolo d’Incau - Going to production in a few months – How we did it!
PDF
This stuff is cool, but HOW CAN I GET MY COMPANY TO DO IT? Businessing the S*...
PDF
Design Myths in Enterprise Software
PPTX
Customer Presentation: Digital Globe's road to Continuous Delivery
PPTX
Building an Excellent Web Startup
PPT
Webinar: 2 Billion Data Points Each Day
PDF
Google Apps for Work
PDF
Using data to guide product development
PPTX
Success story of google samarth
PPTX
No Projects / Beyond Projects (short version)
Nondeterministic Software for the Rest of Us
Google Human Resource Management
Google: Designed for Innovation
Agile for Me- CodeStock 2009
2019 Top Lessons Learned Since the Phoenix Project Was Released
DevOps Patterns Distilled: Implementing The Needed Practices In Practical Steps
Tranformative Culture - The Shift From QA To Engineering Productivity - Selen...
iTexico Company Overview 2015
Google Product Dev Process
The End of Projects & what to do about it
SFScon21 - Paolo d’Incau - Going to production in a few months – How we did it!
This stuff is cool, but HOW CAN I GET MY COMPANY TO DO IT? Businessing the S*...
Design Myths in Enterprise Software
Customer Presentation: Digital Globe's road to Continuous Delivery
Building an Excellent Web Startup
Webinar: 2 Billion Data Points Each Day
Google Apps for Work
Using data to guide product development
Success story of google samarth
No Projects / Beyond Projects (short version)

More from Business of Software Conference (20)

PPTX
BoSUSA24 | Austin Bouley | 3 Huge Levers That Got My SaaS To $500K ARR In 5 M...
PDF
BoSEU25 | Gareth Marlow | No One Knows What They’re Doing. Especially You. Le...
PDF
BoSEU25 | Mark Stephens | 25 Years of Heretical Thinking
PDF
BoSEU25 | Steve McLeod | How Bootstrapped Teams Really Choose What Feature to...
PPTX
BoSEU25 | Diego de Jódar | Why User Activation is the Key to Sustainable Growth
PDF
BoSEU25 | David Pereira | Product Management or BS Management?
PDF
BoSEU25 | Sara Gordon | The Seven Deadly Sins & Positioning
PPTX
BoSEU25 | Elizabeth Lawley | Lessons Learnt from my AI Girlfriend
PPTX
BoSEU25 | Robin Landy | Finding Features Customers Will Pay (More) For
PDF
BoSEU25 | Greg Baugues | AI You’ve Been Doing It All Wrong
PDF
BoSEU25 | Melissa Appel | The Founders’ Guide to What Product Teams Need
PPTX
BoSEU25 | Ryan Singer | Framing and Hard Conversations
PDF
BoSEU23 Ryan Singer Debugging the Product Development Process.pdf
PDF
BoSON23 | Jim Morris | Designing and Running Customer Interviews that Work
PPTX
BoSUSA24 | Tania Katan | Human Ingenuity Beats Artificial Intelligence as the...
PPTX
BoSUSA24 | Shawn Anderson | From Bootstrapped to a Billion – Five BoS Talks T...
PDF
BoSUSA24 | Georgiana Laudi | Data-Rich, Insight-Poor – The Real Reason Your G...
PPTX
BoSUSA24 | Joanna Wiebe | How Every Customer Interaction Can be Improved With...
PPTX
BoSUSA24 | Jim Benton | Building Iconic Teams – Achieving the Impossible Toge...
PPTX
BoSUSA24 | Stephen Steers | Four Questions to Ask Before You Tell a Story Tha...
BoSUSA24 | Austin Bouley | 3 Huge Levers That Got My SaaS To $500K ARR In 5 M...
BoSEU25 | Gareth Marlow | No One Knows What They’re Doing. Especially You. Le...
BoSEU25 | Mark Stephens | 25 Years of Heretical Thinking
BoSEU25 | Steve McLeod | How Bootstrapped Teams Really Choose What Feature to...
BoSEU25 | Diego de Jódar | Why User Activation is the Key to Sustainable Growth
BoSEU25 | David Pereira | Product Management or BS Management?
BoSEU25 | Sara Gordon | The Seven Deadly Sins & Positioning
BoSEU25 | Elizabeth Lawley | Lessons Learnt from my AI Girlfriend
BoSEU25 | Robin Landy | Finding Features Customers Will Pay (More) For
BoSEU25 | Greg Baugues | AI You’ve Been Doing It All Wrong
BoSEU25 | Melissa Appel | The Founders’ Guide to What Product Teams Need
BoSEU25 | Ryan Singer | Framing and Hard Conversations
BoSEU23 Ryan Singer Debugging the Product Development Process.pdf
BoSON23 | Jim Morris | Designing and Running Customer Interviews that Work
BoSUSA24 | Tania Katan | Human Ingenuity Beats Artificial Intelligence as the...
BoSUSA24 | Shawn Anderson | From Bootstrapped to a Billion – Five BoS Talks T...
BoSUSA24 | Georgiana Laudi | Data-Rich, Insight-Poor – The Real Reason Your G...
BoSUSA24 | Joanna Wiebe | How Every Customer Interaction Can be Improved With...
BoSUSA24 | Jim Benton | Building Iconic Teams – Achieving the Impossible Toge...
BoSUSA24 | Stephen Steers | Four Questions to Ask Before You Tell a Story Tha...

Recently uploaded (20)

PDF
Design an Analysis of Algorithms II-SECS-1021-03
PDF
medical staffing services at VALiNTRY
PPTX
Transform Your Business with a Software ERP System
PDF
Audit Checklist Design Aligning with ISO, IATF, and Industry Standards — Omne...
PPTX
Odoo POS Development Services by CandidRoot Solutions
PDF
Internet Downloader Manager (IDM) Crack 6.42 Build 42 Updates Latest 2025
PDF
top salesforce developer skills in 2025.pdf
PDF
Design an Analysis of Algorithms I-SECS-1021-03
PDF
Flood Susceptibility Mapping Using Image-Based 2D-CNN Deep Learnin. Overview ...
PPTX
VVF-Customer-Presentation2025-Ver1.9.pptx
PDF
Upgrade and Innovation Strategies for SAP ERP Customers
PPTX
Oracle E-Business Suite: A Comprehensive Guide for Modern Enterprises
PDF
PTS Company Brochure 2025 (1).pdf.......
PDF
Internet Downloader Manager (IDM) Crack 6.42 Build 41
PPT
Introduction Database Management System for Course Database
PDF
System and Network Administration Chapter 2
PPTX
ISO 45001 Occupational Health and Safety Management System
PPTX
Agentic AI Use Case- Contract Lifecycle Management (CLM).pptx
PPTX
CHAPTER 2 - PM Management and IT Context
PDF
SAP S4 Hana Brochure 3 (PTS SYSTEMS AND SOLUTIONS)
Design an Analysis of Algorithms II-SECS-1021-03
medical staffing services at VALiNTRY
Transform Your Business with a Software ERP System
Audit Checklist Design Aligning with ISO, IATF, and Industry Standards — Omne...
Odoo POS Development Services by CandidRoot Solutions
Internet Downloader Manager (IDM) Crack 6.42 Build 42 Updates Latest 2025
top salesforce developer skills in 2025.pdf
Design an Analysis of Algorithms I-SECS-1021-03
Flood Susceptibility Mapping Using Image-Based 2D-CNN Deep Learnin. Overview ...
VVF-Customer-Presentation2025-Ver1.9.pptx
Upgrade and Innovation Strategies for SAP ERP Customers
Oracle E-Business Suite: A Comprehensive Guide for Modern Enterprises
PTS Company Brochure 2025 (1).pdf.......
Internet Downloader Manager (IDM) Crack 6.42 Build 41
Introduction Database Management System for Course Database
System and Network Administration Chapter 2
ISO 45001 Occupational Health and Safety Management System
Agentic AI Use Case- Contract Lifecycle Management (CLM).pptx
CHAPTER 2 - PM Management and IT Context
SAP S4 Hana Brochure 3 (PTS SYSTEMS AND SOLUTIONS)

BoS2015 Trish Khoo – Engineering Manager, Google

  • 2. Nice things People (also $$$)Time (aka $$$) ☹ you
  • 3. Nice things People (also $$$)Time (aka $$$) ☺ X yay
  • 4. - GEO AT GOOGLE SCALE - >1 billion active monthly users of Google Maps Services >1 billion downloads of Google Maps on Android >2 million active websites and apps use the Google Maps API every week 10s of thousands of user edits to local data and map data per day
  • 5. - QUALITY AT GOOGLE SCALE - Google has the world mapped - 200 countries and territories Street View imagery is available across 66 countries The satellite imagery in Google Earth covers more than 45% of the world's land surface and 80% of the world's population in sub-meter high resolution.
  • 6. A problem coming your way? “We often find that when doing due diligence, that the team got the thing up and running as fast as possible and that's fine for getting to £1-5m revenue but they’ve created scaling issues for themselves after that.” - Simon Menashy, MMC Ventures
  • 7. - TYPICAL DECISION #1 - Hiring a QA team to fix the quality problem
  • 8. Don’t hire the QA army Quality issue? Quality assurance! QA is accountable but has very little power to change anything Manual testing is very slow
  • 9. - TYPICAL DECISION #2 - Hiring an automation team to fix the testing problem
  • 10. A bad sign Subject: We’re hiring! TEST AUTOMATION ENGINEER Join our fast-growing team of TEST AUTOMATION ENGINEERS! • Low coding skill is okay • Write all our tests for us • We have a foosball table and a blimp!
  • 11. - GOOGLE’S TURNING POINT - 7 years ago, Google teams released to production once or twice a month, with the majority of the time taken by testing. Today Google teams release to production daily or sub-daily. There was no decrease in quality.
  • 13. The “Testing Phase” • No idea when the product will actually launch • Massive amount of human hours to manage effectively • Huge waste in expensive engineering hours
  • 15. What testing really is Testing provides expert driven feedback on the state of business requirement gaps, user impact and overall project quality. One of the ways we do this is by checking things. Everybody can check things. It’s easy.
  • 17. Checking is easy. Testing is hard. A testing expert in your development team will: • Make sure people get information about your product at the optimal time to make timely decisions • Make your development process more efficient • Design tools and infrastructure to make your developers more productive
  • 19. - THE PUSHBACK - We don’t have time to write tests! We don’t know how! That’s not the way we work at this company!
  • 20. HIRE THE RIGHT SKILLS
  • 21. ENGINEERING PRODUCTIVITY Software Engineer, Tools & Infrastructure Test Engineer
  • 22. - EMPOWER YOUR TESTING EXPERT - Hard to find, hard to retain! • First class citizen within the team • Similar job title and salary to developers • Needs management support
  • 24. - LEARNING, NOT BLAME - • Cultivate a postmortem culture • Trust your people
  • 26. >> FAST RELEASES FAST FEEDBACK <<
  • 28. What’s in it for me? • Rock solid software • Ability to quickly change product direction • Freedom to innovate
  • 29. THINGS TO DO #1 Change your culture #2 Hire the right skills #3 Learn, don’t blame #4 Launch and iterate