SlideShare a Scribd company logo
!
!
K1#
Keynote!
6/24/2015! !
8:30!AM!
!
!
!
!
“How#We#NOW#Test#Software#at#Microsoft”##
Presented#by:#
Alan#Page#
Microsoft#
#
#
#
#
#
Brought#to#you#by:#
#
#
#
#
#
#
340!Corporate!Way,!Suite!300,!Orange!Park,!FL!32073!
888D268D8770!E!904D278D0524!E!sqeinfo@sqe.com!E!www.sqe.com!
!
!
!
!
!
!!!!! !
!!!!
Alan Page is a principal SDET—a fancy name for tester—on the Xbox console team at
Microsoft. Edging up on twenty years in software testing, Alan has previously worked on
a variety of Microsoft products including Windows, Windows CE, Internet Explorer, and
Office Lync. He spent some time as Microsoft’s director of test excellence where he
developed and ran technical training programs for testers throughout the company. Alan
was the lead author of How We Test Software at Microsoft and contributed chapters on
large-scale test automation to Beautiful Testing and Experiences of Test Automation:
Case Studies of Software Test Automation. You can follow Alan on his blog or on
Twitter @alanpage..
Alan Page
Microsoft
5/18/15%
1%
NOW%
Can$we$improve$the$way$we$make$
so2ware?$
5/18/15%
2%
Can$we$be$more$efficient?$
5/18/15%
3%
2006
5/18/15%
4%
5/18/15%
5%
HWTSAM$=$The$Beginning$
5/18/15%
6%
5/18/15%
7%
Nobody’s$Perfect$
1)  In the intro, I state that Ken approached me in the fall of 2007. On reflection, it was actually
late fall 2006. I began writing the first chapter in January 2007 (chapter 4 to be exact).
Minor issue, but an Author Bug nonetheless.
2)  The table on p. 19 is missing a line between USA (California) and Hyderabad (Production
Bug)
3)  On page 95, where the header says. “3BV”, it should say “3BC”
4)  There's a bit of weirdness with the code coverage data on page 119. Bj explains it in detail
on his blog here. (Author bug)
5)  On page 120, the explanation refers to variables names myString and myCharacter – but we
use different variable names in the code (s and c). (Author Bug).
6)  On page 123, the title of figure 5-6 is incorrect (the function is SimpleGetNT5ClientVersion).
7)  On page 125, the text refers to the Int.Parse() function, but it doesn’t exist in the code
fragment (oops - Author Bug)
8)  On page 202, the text reads “Total fixed found / total bugs fixed”, but it should read “Total
bugs fixed / total bugs found”
9)  On page 221, the “Positively False” sidebar is not a sidebar, and in fact the box around it
encompasses the additional points (Supported platforms, Complexity, and Other factors)
and section (We don’t have time to automate this) that aren’t part of that sidebar (Author
Bug)
10) On page 226, the sample using InvokePattern could use some more documentation (Author
Oversight)
11) In the code sample on the bottom of page 235, I have mismatched quotes in two of the
lines. (Author Bug).
5/18/15%
8%
5/18/15%
9%
People$and$Processes$
5/18/15%
10%
Senior%Vice%President%
GM%or%VP%of%Dev%
Dev%Managers% Architects%
GM%or%VP%of%Test%
Test%Managers% Test%Architects%
GM%or%VP%of%Program%
Mgmt%
PM%Managers% Etc.%
Vice%President%
Engineering%Manager% Engineering%Manager%
Engineer% Engineer% Engineer% Engineer% Engineer% Engineer% Engineer% Engineer% Engineer%
Engineering%Manager%
5/18/15%
11%
2008
Processes%iteraIve,%yet%slow%
Customer%behavior,%feedback%measured%postSrelease%
%
2015
Most%teams%use%agileSish%processes%
Customer%data%is%cri$cal'
2008
More%than%90,000%Employees%World%Wide%
More%than%38,000%SoTware%Engineers%%
Nearly%%10,000%Testers%
Test%to%Developer%RaIo%–%(almost)%1:1%
2015
120,000%Employees%World%Wide%
40,000%SoTware%Engineers%%
Testers?%
5/18/15%
12%
I$Know$What$You’re$Thinking…$
5/18/15%
13%
5/18/15%
14%
%The%implementaIon%described%
above%is%risky%and%invites%failure.%
5/18/15%
15%
5/18/15%
16%
5/18/15%
17%
FlightingRIPre CheckinDev SessionRealtime
5/18/15%
18%
5/18/15%
19%
5/18/15%
20%
Who$“owns”$Quality?$
What'Is'Quality?$
5/18/15%
21%
“Quality*is*value*to*some*person”%
S%Jerry%Weinberg%
Who%“owns”%creaIng%value%for%the%
customer?%
5/18/15%
22%
Dev%“stuff”%
5/18/15%
23%
Dev%“stuff”%
Project%
Management%
“stuff%
Dev%“stuff”%
Project%
Management%
“stuff%
The%
“Remainder”%
5/18/15%
24%
Data%Science%
Exploratory$TesPng$
“The%Remainder”%
5/18/15%
25%
Diversity$is$Important$CriPcal$
Specializing%Generalists%
Generalizing%Specialists%
Everyone%doesn’t%do%everything…%
…but%the%team%contains%“all%of%the%skills”%
5/18/15%
26%
By%staffing%a%team%with$people$who$have$an$
area$of$experPse,$but$can$do$anything,$you$
can$maximize$the$value$of$each$delivery$
cycle%
Nothing$is$easy…$
5/18/15%
27%
“That’s the way we’ve always done it.”
“That won’t work for us.”
“We tried that once, and it didn’t work.”
Things%I%Hear…%a%Lot%
“The old way won’t work here because….”
“This can work for us, because…”
“This will work for us, because…”
Things%I%Say…%a%Lot%
5/18/15%
28%
Before$I$Go…$
5/18/15%
29%
5/18/15%
30%
You’re$Not$As$Different$As$You$Think$
5/18/15%
31%
Thank$You$
$
hVp://angryweasel.com$
hVp://angryweasel.com/ABTesPng$
$

More Related Content

PPT
Software Engineering (Introduction to Software Engineering)
DOCX
Selenium Testing Project report
PPTX
Test cases for effective testing - part 1
DOC
PDF
Agile Methodology - Software Engineering
PPT
Software Process Models
PDF
RESUME_SUSHMA_BE_ECE_2015
PPT
Manual testing ppt
Software Engineering (Introduction to Software Engineering)
Selenium Testing Project report
Test cases for effective testing - part 1
Agile Methodology - Software Engineering
Software Process Models
RESUME_SUSHMA_BE_ECE_2015
Manual testing ppt

What's hot (20)

PPT
Software Testing
PPTX
Software testing ppt
PPT
Spice
PPTX
Software test automation
PPTX
1.fyp presentation
PPTX
Automation - web testing with selenium
PPTX
Software process
PPTX
Test automation framework
PPT
Final Year Project
PPTX
Object detection.pptx
PPT
Software metrics
PPT
Critical System Validation in Software Engineering SE21
PPTX
Software testing life cycle
PPT
Testing Metrics
PPTX
SDLC Model (Waterfall,Iterative Waterfall,Spiral)
DOC
Sandeep qa resume
PPT
Software engg. pressman_ch-9
PPT
Automatisation des tests
PDF
Summer Training Report SSPL , DRDO
PPTX
Test Automation and Selenium
Software Testing
Software testing ppt
Spice
Software test automation
1.fyp presentation
Automation - web testing with selenium
Software process
Test automation framework
Final Year Project
Object detection.pptx
Software metrics
Critical System Validation in Software Engineering SE21
Software testing life cycle
Testing Metrics
SDLC Model (Waterfall,Iterative Waterfall,Spiral)
Sandeep qa resume
Software engg. pressman_ch-9
Automatisation des tests
Summer Training Report SSPL , DRDO
Test Automation and Selenium
Ad

Viewers also liked (17)

PDF
Business Analysis: From Interviews through Implementation
PDF
Improve Your Test Process from the Bottom Up
PDF
Create Products That Customers Love: A Testing Perspective
PDF
Plotting a Better Beer with Geolocation: An Untappd Case Study
PDF
Continuous Testing through Service Virtualization
PDF
Security Testing: What Testers Can Do
PDF
Management Issues in Test Automation
PDF
The Art of Complex System Testing
PDF
Automate Legacy-System Testing: Easy, Reliable, and Extendible
PDF
Leadership for Test Managers and Testers
PDF
Testing the Internet of Everything
PDF
Automate Your Way to Sanity in the Chaos of Mobile App Development
PDF
Enough about Process, Let’s Use Patterns
PDF
What Do Defects Really Cost? Much More Than You Think
PDF
The Next Decade of Agile Software Development and Test
PDF
Lightning Strikes the Keynotes
PDF
Leadership for Test Managers and Testers
Business Analysis: From Interviews through Implementation
Improve Your Test Process from the Bottom Up
Create Products That Customers Love: A Testing Perspective
Plotting a Better Beer with Geolocation: An Untappd Case Study
Continuous Testing through Service Virtualization
Security Testing: What Testers Can Do
Management Issues in Test Automation
The Art of Complex System Testing
Automate Legacy-System Testing: Easy, Reliable, and Extendible
Leadership for Test Managers and Testers
Testing the Internet of Everything
Automate Your Way to Sanity in the Chaos of Mobile App Development
Enough about Process, Let’s Use Patterns
What Do Defects Really Cost? Much More Than You Think
The Next Decade of Agile Software Development and Test
Lightning Strikes the Keynotes
Leadership for Test Managers and Testers
Ad

Similar to How We NOW Test Software at Microsoft (20)

PDF
ITCamp 2013 - Florin Coros - Driving Your Team Towards Code Quality
PDF
Refactoring with Microsoft Visual Studio 2010 Peter Ritchie download pdf
PPTX
Tabara testare adrian matei
PDF
C Interview Guide Boost Your Confidence With Answers To Hundreds Of Secret In...
PPTX
SOFTWARE TESTING TRAFUNDAMENTALS OF SOFTWARE TESTING.pptx
PDF
Alan Page: On Testing
PPTX
Best pratice
PPT
Coding Standards
PPTX
Code quality
PPTX
Code Quality
PPTX
STARWEST 2011 - 7 Steps To Improving Software Quality using Microsoft Test Ma...
PPTX
A Lap Around Visual Studio 2010
PPT
Vsts 2
PPTX
Making the Unstable Stable - An Intro To Testing
PDF
Testingfor continuousdeliverywithvisualstudio2012
PPTX
STAREAST 2011 - 7 Steps To Improving Software Quality using Microsoft Test Ma...
PDF
CAJ-014 Rick Spiewak
PDF
Architecting a Large Software Project - Lessons Learned
PDF
5-Ways-to-Revolutionize-Your-Software-Testing
PDF
BlueHat v18 || Go build a tool - best practices for building a robust & e...
ITCamp 2013 - Florin Coros - Driving Your Team Towards Code Quality
Refactoring with Microsoft Visual Studio 2010 Peter Ritchie download pdf
Tabara testare adrian matei
C Interview Guide Boost Your Confidence With Answers To Hundreds Of Secret In...
SOFTWARE TESTING TRAFUNDAMENTALS OF SOFTWARE TESTING.pptx
Alan Page: On Testing
Best pratice
Coding Standards
Code quality
Code Quality
STARWEST 2011 - 7 Steps To Improving Software Quality using Microsoft Test Ma...
A Lap Around Visual Studio 2010
Vsts 2
Making the Unstable Stable - An Intro To Testing
Testingfor continuousdeliverywithvisualstudio2012
STAREAST 2011 - 7 Steps To Improving Software Quality using Microsoft Test Ma...
CAJ-014 Rick Spiewak
Architecting a Large Software Project - Lessons Learned
5-Ways-to-Revolutionize-Your-Software-Testing
BlueHat v18 || Go build a tool - best practices for building a robust & e...

More from TechWell (20)

PDF
Failing and Recovering
PDF
Instill a DevOps Testing Culture in Your Team and Organization
PDF
Test Design for Fully Automated Build Architecture
PDF
System-Level Test Automation: Ensuring a Good Start
PDF
Build Your Mobile App Quality and Test Strategy
PDF
Testing Transformation: The Art and Science for Success
PDF
Implement BDD with Cucumber and SpecFlow
PDF
Develop WebDriver Automated Tests—and Keep Your Sanity
PDF
Ma 15
PDF
Eliminate Cloud Waste with a Holistic DevOps Strategy
PDF
Transform Test Organizations for the New World of DevOps
PDF
The Fourth Constraint in Project Delivery—Leadership
PDF
Resolve the Contradiction of Specialists within Agile Teams
PDF
Pin the Tail on the Metric: A Field-Tested Agile Game
PDF
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
PDF
A Business-First Approach to DevOps Implementation
PDF
Databases in a Continuous Integration/Delivery Process
PDF
Mobile Testing: What—and What Not—to Automate
PDF
Cultural Intelligence: A Key Skill for Success
PDF
Turn the Lights On: A Power Utility Company's Agile Transformation
Failing and Recovering
Instill a DevOps Testing Culture in Your Team and Organization
Test Design for Fully Automated Build Architecture
System-Level Test Automation: Ensuring a Good Start
Build Your Mobile App Quality and Test Strategy
Testing Transformation: The Art and Science for Success
Implement BDD with Cucumber and SpecFlow
Develop WebDriver Automated Tests—and Keep Your Sanity
Ma 15
Eliminate Cloud Waste with a Holistic DevOps Strategy
Transform Test Organizations for the New World of DevOps
The Fourth Constraint in Project Delivery—Leadership
Resolve the Contradiction of Specialists within Agile Teams
Pin the Tail on the Metric: A Field-Tested Agile Game
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
A Business-First Approach to DevOps Implementation
Databases in a Continuous Integration/Delivery Process
Mobile Testing: What—and What Not—to Automate
Cultural Intelligence: A Key Skill for Success
Turn the Lights On: A Power Utility Company's Agile Transformation

Recently uploaded (20)

PDF
Navsoft: AI-Powered Business Solutions & Custom Software Development
PDF
Raksha Bandhan Grocery Pricing Trends in India 2025.pdf
PDF
PTS Company Brochure 2025 (1).pdf.......
PDF
medical staffing services at VALiNTRY
PPTX
Lecture 3: Operating Systems Introduction to Computer Hardware Systems
PDF
Digital Strategies for Manufacturing Companies
PDF
Upgrade and Innovation Strategies for SAP ERP Customers
PPTX
ISO 45001 Occupational Health and Safety Management System
PPTX
Transform Your Business with a Software ERP System
PDF
Which alternative to Crystal Reports is best for small or large businesses.pdf
PPTX
CHAPTER 12 - CYBER SECURITY AND FUTURE SKILLS (1) (1).pptx
PDF
System and Network Administration Chapter 2
PDF
Flood Susceptibility Mapping Using Image-Based 2D-CNN Deep Learnin. Overview ...
PPTX
CHAPTER 2 - PM Management and IT Context
PDF
AI in Product Development-omnex systems
PDF
How Creative Agencies Leverage Project Management Software.pdf
PPTX
Odoo POS Development Services by CandidRoot Solutions
PDF
Adobe Illustrator 28.6 Crack My Vision of Vector Design
PDF
How to Migrate SBCGlobal Email to Yahoo Easily
PDF
Wondershare Filmora 15 Crack With Activation Key [2025
Navsoft: AI-Powered Business Solutions & Custom Software Development
Raksha Bandhan Grocery Pricing Trends in India 2025.pdf
PTS Company Brochure 2025 (1).pdf.......
medical staffing services at VALiNTRY
Lecture 3: Operating Systems Introduction to Computer Hardware Systems
Digital Strategies for Manufacturing Companies
Upgrade and Innovation Strategies for SAP ERP Customers
ISO 45001 Occupational Health and Safety Management System
Transform Your Business with a Software ERP System
Which alternative to Crystal Reports is best for small or large businesses.pdf
CHAPTER 12 - CYBER SECURITY AND FUTURE SKILLS (1) (1).pptx
System and Network Administration Chapter 2
Flood Susceptibility Mapping Using Image-Based 2D-CNN Deep Learnin. Overview ...
CHAPTER 2 - PM Management and IT Context
AI in Product Development-omnex systems
How Creative Agencies Leverage Project Management Software.pdf
Odoo POS Development Services by CandidRoot Solutions
Adobe Illustrator 28.6 Crack My Vision of Vector Design
How to Migrate SBCGlobal Email to Yahoo Easily
Wondershare Filmora 15 Crack With Activation Key [2025

How We NOW Test Software at Microsoft