SlideShare a Scribd company logo
How we got to a CI/CD World:
A Vanguard Testing Journey
Craig Schwarzwald
About Craig Schwarzwald
• Worked at Vanguard for 12 years
– Many different roles:
• System Tester
• UI Developer
• Mobile Developer
• Automation Engineer
• Test Automation Coach
– Open Sourced Selenium Framework:
• https://github.com/vanguard/selenium-automation-framework.lib
• Details available: SauceCon2017:
https://www.youtube.com/watch?v=2K2M7s_Ups0
How Vanguard Got to a CD-CD World by Craig Schwarzwald
How Vanguard Got to a CD-CD World by Craig Schwarzwald
How Vanguard Got to a CD-CD World by Craig Schwarzwald
How Vanguard Got to a CD-CD World by Craig Schwarzwald
How did we get there?
2 of our biggest keys to success
❖An Automated Build Pipeline
❖A different mindset in testing
Microservice APIs.
Contract Testing with PACT
Investing in a CI Build Pipeline
•
•
•
•
•
What is the Microservice API Economy?
3rdPartyService
DownStreamJob
VendorService
???
3rdPartyService
DownStreamJob
VendorService
???
Traditional Testing - Microservices
•
•
•
•
Traditional Testing - Microservices
•
•
•
•
3rdPartyService
DownStreamJob
VendorService
???
Traditional Testing - Microservices
•
•
•
•
•
3rdPartyService
DownStreamJob
VendorService
???
Traditional Testing - Sliding Scale
Speed vs. Confidence
Changing Testing Mindset - Microservices
•
•
•
•
What is Contract Testing?
What if ALL our microservices used PACT?
•
•
Meteorology
Service
Vendor Service
What are the down sides with PACT Broker?
Your build is now dependent on multiple provider builds.
– No way to get End-to-End confidence without executing the providers.
• Because these are microservices, builds should be small, fast, and reliably passing (at
least on master).
Consumers have the ability to break Provider builds.
– Can be great to request new features of a provider
• But you need to set PACT up to run on tags/branches
– (you don’t want to break master builds).
– Good luck getting Vendors to implement PACT like this.
PACT on the sliding scale
Alternatives to PACT Broker
Swagger – Open API spec.
– Document how the API works
– Endpoints
– Response Objects
Can be utilized with PACT via:
Atlassian’s: swagger-request-validator-pact (JAVA)
swagger-mock-validator (JavaScript)
Meteorology
Service
Vendor Service
PACT Flows
Pact Broker
vs.
Swagger
•
•
Meteorology
Service
Vendor
Service
Swagger Validator
•
•
•
•
•
•
Summary
•
•
•
•

More Related Content

PDF
How to Measure Success in Continuous Testing by Fernando Vidal and Amir Rozen...
PDF
Overview the Challenges and Limitations of Android App Automation with Espres...
PDF
The Push From Within - A Journey of Transformation at Walmart Labs by Claude ...
PDF
Automated Testing at The Speed of Headless by Alissa Lydon and Samantha Coffman
PDF
Break Up the Monolith- Testing Microservices by Marcus Merrell
PPTX
ATAGTR2017 Wearable App Testing
PDF
Automating The New York Times Crossword by Phil Wells
PPTX
ATAGTR2017 Upgrading a mobile tester's weapons with advanced debugging
How to Measure Success in Continuous Testing by Fernando Vidal and Amir Rozen...
Overview the Challenges and Limitations of Android App Automation with Espres...
The Push From Within - A Journey of Transformation at Walmart Labs by Claude ...
Automated Testing at The Speed of Headless by Alissa Lydon and Samantha Coffman
Break Up the Monolith- Testing Microservices by Marcus Merrell
ATAGTR2017 Wearable App Testing
Automating The New York Times Crossword by Phil Wells
ATAGTR2017 Upgrading a mobile tester's weapons with advanced debugging

What's hot (20)

PDF
DevOps Continuous Integration & Delivery - A Whitepaper by RapidValue
PPTX
ATAGTR2017 Performance Automation in Dev-Ops
PDF
Testing AS A Container - Irfan Ahmad
PPTX
Continuous delivery applied
PPTX
Poster - DevOps Habits @ Microsoft
PPTX
CI/CD for mobile at HERE
PPTX
Continuous Delivery
PPTX
Poster - DevOps Planning @ Microsoft
PDF
"DevOps > CI+CD "
PPTX
Make the Shift from Manual to Automation with Open Source
PDF
Testing in a continuous delivery environment
PPTX
ATAGTR2017 Keeping pace with Product Evolution: UI Automation Framework Guide...
PDF
How to Introduce Continuous Delivery
PDF
Getting to Continuous Deployment (Webinar Slides)
PPTX
Appium vs Espresso and XCUI Test
PPTX
Introduction to Continuous Delivery (BBWorld/DevCon 2013)
PDF
Metrics-driven Continuous Delivery
PPTX
Testing in production with feature flags
PDF
Managers, Future Proof Your Automation
PDF
Continuous Delivery Agile Tour Beirut 2015
DevOps Continuous Integration & Delivery - A Whitepaper by RapidValue
ATAGTR2017 Performance Automation in Dev-Ops
Testing AS A Container - Irfan Ahmad
Continuous delivery applied
Poster - DevOps Habits @ Microsoft
CI/CD for mobile at HERE
Continuous Delivery
Poster - DevOps Planning @ Microsoft
"DevOps > CI+CD "
Make the Shift from Manual to Automation with Open Source
Testing in a continuous delivery environment
ATAGTR2017 Keeping pace with Product Evolution: UI Automation Framework Guide...
How to Introduce Continuous Delivery
Getting to Continuous Deployment (Webinar Slides)
Appium vs Espresso and XCUI Test
Introduction to Continuous Delivery (BBWorld/DevCon 2013)
Metrics-driven Continuous Delivery
Testing in production with feature flags
Managers, Future Proof Your Automation
Continuous Delivery Agile Tour Beirut 2015
Ad

Similar to How Vanguard Got to a CD-CD World by Craig Schwarzwald (20)

PPT
SOASTA Webinar: Process Compression For Mobile App Dev 120612
PDF
SanDiego_DevOps_Meetup_9212016
PPTX
SanDiego_DevOps_Meetup_9212016-v8
PDF
INTERFACE by apidays 2023 - Digital Platforms, Taking API to production, Micr...
PDF
Deep Dive on CI/CD NYC Meet Up Group
PPTX
Microservices
PDF
Enterprise QA and Application Testing Services
PPTX
Enterprise QA and Application Testing Services
PDF
Microservices the Good Bad and the Ugly
PDF
Dockercon State of the Art in Microservices
PDF
Testing in the 21st Century (ExpoQA)
PDF
Cloud Native Cost Optimization UCC
PDF
Automated Governance
PPTX
Webinar: “Continuous Delivery: il tuo primo passo verso il DevOps”
KEY
Introduction to Google App Engine
PDF
QAAgility Presentation - Cucumber with Appium
PDF
[WSO2Con EU 2017] Continuous Integration, Delivery and Deployment: Accelerate...
PPTX
CI/CD - A strategy for success (North Africa Dreamin' Prez)
PPT
SOASTA Webinar: Process Compression For Mobile App Dev 120612
PPTX
Micro Service Architecture
SOASTA Webinar: Process Compression For Mobile App Dev 120612
SanDiego_DevOps_Meetup_9212016
SanDiego_DevOps_Meetup_9212016-v8
INTERFACE by apidays 2023 - Digital Platforms, Taking API to production, Micr...
Deep Dive on CI/CD NYC Meet Up Group
Microservices
Enterprise QA and Application Testing Services
Enterprise QA and Application Testing Services
Microservices the Good Bad and the Ugly
Dockercon State of the Art in Microservices
Testing in the 21st Century (ExpoQA)
Cloud Native Cost Optimization UCC
Automated Governance
Webinar: “Continuous Delivery: il tuo primo passo verso il DevOps”
Introduction to Google App Engine
QAAgility Presentation - Cucumber with Appium
[WSO2Con EU 2017] Continuous Integration, Delivery and Deployment: Accelerate...
CI/CD - A strategy for success (North Africa Dreamin' Prez)
SOASTA Webinar: Process Compression For Mobile App Dev 120612
Micro Service Architecture
Ad

More from Sauce Labs (20)

PDF
Simplify Salesforce Testing with AI-Driven Codeless Tools
PDF
Testing on Mobile Devices with Location Services
PDF
Your Framework for Success: introduction to JavaScript Testing at Scale
PDF
Automating Hybrid Applications with Appium
PDF
Quality at Speed: More API Testing, Less UI Testing
PPTX
Creating Digital Confidence with Test Automation
PDF
Just Enough (Automated) Testing
PDF
Using Axe to Add Accessibility Checks to Your Existing Selenium Tests
PDF
How Open Source Helps to Bring Back Product Obsession
PDF
Webinar: A Sneak Peek at Selenium 4 with Simon Stewart
PDF
[Deu] Test Automatisierung Mit Web Driver.io
PDF
Accelerating Innovation: Leveraging Open Source to Optimize Your Shift-Left I...
PDF
Accelerating Your Digital Agenda with Continuous Testing ft. Forrester
PDF
How to Measure Success in Continuous Testing
PDF
From Zero to 2.7 Million - How Verizon Media Embraced Open Source to Accelera...
PDF
5 Steps to Jump Start Your Test Automation
PDF
Sauce Labs Webinar: Rising Importance of Software Testing
PDF
BDD With Selenide by Hima Bindu Peteti
PDF
Closer To the Metal - Why and How We Use XCTest and Espresso by Mario Negro P...
PDF
Continuous Delivery for "Mature" Codebases by Melisa Benua
Simplify Salesforce Testing with AI-Driven Codeless Tools
Testing on Mobile Devices with Location Services
Your Framework for Success: introduction to JavaScript Testing at Scale
Automating Hybrid Applications with Appium
Quality at Speed: More API Testing, Less UI Testing
Creating Digital Confidence with Test Automation
Just Enough (Automated) Testing
Using Axe to Add Accessibility Checks to Your Existing Selenium Tests
How Open Source Helps to Bring Back Product Obsession
Webinar: A Sneak Peek at Selenium 4 with Simon Stewart
[Deu] Test Automatisierung Mit Web Driver.io
Accelerating Innovation: Leveraging Open Source to Optimize Your Shift-Left I...
Accelerating Your Digital Agenda with Continuous Testing ft. Forrester
How to Measure Success in Continuous Testing
From Zero to 2.7 Million - How Verizon Media Embraced Open Source to Accelera...
5 Steps to Jump Start Your Test Automation
Sauce Labs Webinar: Rising Importance of Software Testing
BDD With Selenide by Hima Bindu Peteti
Closer To the Metal - Why and How We Use XCTest and Espresso by Mario Negro P...
Continuous Delivery for "Mature" Codebases by Melisa Benua

Recently uploaded (20)

PDF
How Creative Agencies Leverage Project Management Software.pdf
PPTX
CHAPTER 12 - CYBER SECURITY AND FUTURE SKILLS (1) (1).pptx
PDF
Audit Checklist Design Aligning with ISO, IATF, and Industry Standards — Omne...
PPTX
CHAPTER 2 - PM Management and IT Context
PDF
top salesforce developer skills in 2025.pdf
PDF
Internet Downloader Manager (IDM) Crack 6.42 Build 41
PDF
Digital Strategies for Manufacturing Companies
PDF
Claude Code: Everyone is a 10x Developer - A Comprehensive AI-Powered CLI Tool
PPTX
ManageIQ - Sprint 268 Review - Slide Deck
PPTX
Introduction to Artificial Intelligence
PDF
Design an Analysis of Algorithms I-SECS-1021-03
PDF
Nekopoi APK 2025 free lastest update
PPTX
ai tools demonstartion for schools and inter college
PPTX
Oracle E-Business Suite: A Comprehensive Guide for Modern Enterprises
PPTX
Odoo POS Development Services by CandidRoot Solutions
PDF
Design an Analysis of Algorithms II-SECS-1021-03
PDF
T3DD25 TYPO3 Content Blocks - Deep Dive by André Kraus
PDF
System and Network Administration Chapter 2
PDF
Addressing The Cult of Project Management Tools-Why Disconnected Work is Hold...
PPTX
VVF-Customer-Presentation2025-Ver1.9.pptx
How Creative Agencies Leverage Project Management Software.pdf
CHAPTER 12 - CYBER SECURITY AND FUTURE SKILLS (1) (1).pptx
Audit Checklist Design Aligning with ISO, IATF, and Industry Standards — Omne...
CHAPTER 2 - PM Management and IT Context
top salesforce developer skills in 2025.pdf
Internet Downloader Manager (IDM) Crack 6.42 Build 41
Digital Strategies for Manufacturing Companies
Claude Code: Everyone is a 10x Developer - A Comprehensive AI-Powered CLI Tool
ManageIQ - Sprint 268 Review - Slide Deck
Introduction to Artificial Intelligence
Design an Analysis of Algorithms I-SECS-1021-03
Nekopoi APK 2025 free lastest update
ai tools demonstartion for schools and inter college
Oracle E-Business Suite: A Comprehensive Guide for Modern Enterprises
Odoo POS Development Services by CandidRoot Solutions
Design an Analysis of Algorithms II-SECS-1021-03
T3DD25 TYPO3 Content Blocks - Deep Dive by André Kraus
System and Network Administration Chapter 2
Addressing The Cult of Project Management Tools-Why Disconnected Work is Hold...
VVF-Customer-Presentation2025-Ver1.9.pptx

How Vanguard Got to a CD-CD World by Craig Schwarzwald

  • 1. How we got to a CI/CD World: A Vanguard Testing Journey Craig Schwarzwald
  • 2. About Craig Schwarzwald • Worked at Vanguard for 12 years – Many different roles: • System Tester • UI Developer • Mobile Developer • Automation Engineer • Test Automation Coach – Open Sourced Selenium Framework: • https://github.com/vanguard/selenium-automation-framework.lib • Details available: SauceCon2017: https://www.youtube.com/watch?v=2K2M7s_Ups0
  • 7. How did we get there? 2 of our biggest keys to success ❖An Automated Build Pipeline ❖A different mindset in testing Microservice APIs. Contract Testing with PACT
  • 8. Investing in a CI Build Pipeline • • • • •
  • 9. What is the Microservice API Economy? 3rdPartyService DownStreamJob VendorService ???
  • 11. Traditional Testing - Microservices • • • • 3rdPartyService DownStreamJob VendorService ???
  • 12. Traditional Testing - Microservices • • • • • 3rdPartyService DownStreamJob VendorService ???
  • 13. Traditional Testing - Sliding Scale Speed vs. Confidence
  • 14. Changing Testing Mindset - Microservices • • • •
  • 15. What is Contract Testing?
  • 16. What if ALL our microservices used PACT? • • Meteorology Service Vendor Service
  • 17. What are the down sides with PACT Broker? Your build is now dependent on multiple provider builds. – No way to get End-to-End confidence without executing the providers. • Because these are microservices, builds should be small, fast, and reliably passing (at least on master). Consumers have the ability to break Provider builds. – Can be great to request new features of a provider • But you need to set PACT up to run on tags/branches – (you don’t want to break master builds). – Good luck getting Vendors to implement PACT like this.
  • 18. PACT on the sliding scale
  • 19. Alternatives to PACT Broker Swagger – Open API spec. – Document how the API works – Endpoints – Response Objects Can be utilized with PACT via: Atlassian’s: swagger-request-validator-pact (JAVA) swagger-mock-validator (JavaScript)
  • 20. Meteorology Service Vendor Service PACT Flows Pact Broker vs. Swagger • • Meteorology Service Vendor Service