SlideShare a Scribd company logo
How to Achieve and Maintain High
Quality SaaS Software in the Cloud


 Lessons learned and best practices on how to set up test
           automation and performance testing
Agenda

• Fire side chat

  – Introduction of the speakers
  – Conversation between QA experts
  – Q & A at the end
Howard Goldstein
VP of Marketing BlackLine Systems
           • Built BlackLine’s marketing program
             from the ground up.
           • Oversaw BlackLine’s expansion
             internationally into the EMEA and
             APAC markets.
           • Developed strategic partner
             relationships with leading global
             consulting firms.
Donna McCollum
QA Manager BlackLine Systems
          • Managed BlackLine’s Quality
            Assurance program for four years.
          • Served as the key liaison to XBOSoft
            overseeing functional and regression
            testing.
          • Previously was Manager of Financial
            Technology at industry leader
            SunGard Treasury Systems.
          • Developed institutional trading
            software.
Philip Lew
CEO and Founder XBOSoft
         • Leads XBOSoft’s direction and
           strategy as their CEO.
         • Ph.D. research in software quality
           and usability.
         • Published in several IEEE and
           ACM journal magazines.
         • Presenter at various academic
           and trade conferences.
         • Industry leader in software
           quality and measurement.
Housekeeping
• The audience will be on mute for a clear conversation
• Feel free to ask questions at anytime via the chat bar and we
  will fit them in where appropriate.
• Contact information will be given at the end and you will
  receive information on where to find the slide and the
  recording.


                   Let’s get Started!
BlackLine Company overview
LA-based financial software provider, operating
globally
First to market with automated account
reconciliation software
475+ clients, many of which are in the Fortune 500
More than 58,000 users worldwide
 ‘BlackLine Certified Implementation Professionals’ from all
the major advisory/consulting firms and all around the world
GLOBAL
    DEPLOYMENT




BlackLine OnDemand users
(89+ COUNTRIES)
BlackLine Offices
BLACKLINE CLIENTS (PARTIAL)
How to Achieve and Maintain High Quality SaaS Software in the Cloud
How to Achieve and Maintain High Quality SaaS Software in the Cloud
How to Achieve and Maintain High Quality SaaS Software in the Cloud
•SIMPLE, SECURE ONDEMAND/SAAS OFFERING
•REAL-TIME MONITORING AND REPORTING
•CONTROLLED PROCESS
•DATA INTEGRITY & INTEGRATION
BLACKLINE PROVIDES


EFFICIENCY, SPEED & VISIBILITY
REDUCED COSTS AND RISKS
CONTROLLED FINANCIAL CLOSE
ROI & CUTTING COSTS
• Auto-certification
                           MeadWestVaco Corporation

• Increased staff          ROI: 495% Payback: 2 Months


  productivity             Cox Communications

• Reduced paper,           ROI: 227% Payback: 6 Months


  printing, storage        Education Management Corporation

  overhead, auditor fees   ROI: 138% Payback: 10 Months



• Improved process
  management               Return on investment
                           is one of the most important
• Faster close             indicators of value
Regression Testing




         Image from: http://www.testfacebook.com
Regression Testing Key Points
• Impossible to test everything so implement
  regression strategy based on context
• Use Risk Assessment based on actual data
• Exploratory Testing
• Defect Analysis
Performance
Performance Key Points
• Understand your users so you can develop
  realistic performance scenario’s
• Benchmark and Improve
• Periodic Efforts
Test Automation




       Image from simply-the-test.blogspot.com
Key Points

• What to automate depends on maturity of the
  product and stability of modules and features
• Team is more important than the tool
• Set up a frame work
Best Practices & Lessons Learned
• Daily Build’s
• Goal 100% coverage, Risk Assessment when that’s not feasible.
• Fast and accurate: Accuracy is of primary importance, but
  performance is a very close second.
• Automation is used where the product is mature and stable.
• QTP is used, but the quality of your people and framework of
  the tests is more important.
• The remote XBOSoft QA team and the BlackLine development
  and support teams work very closely together.
• The follow-the-sun workflow works very well.
Q&A
Contact info

Information on where to find the slides and the
recording will be mailed.

Please follow us on twitter @xbosoft

Request for info: services@xbosoft.com
THANK YOU!


www.XBOSoft.com

www.blackline.com

More Related Content

PDF
Implementing Continuous Product Delivery
PDF
Building a guided analytics forecasting platform with Knime
PDF
AgilePM® V2 - Agile Project Management V2 - Foundation
PPTX
Agile methodology in cloud computing
PPTX
Easy to Know Software Dev. Models
PPTX
RPALM for SAP S/4 Journey
PPTX
PM, Scrum and TFS - Ivan Marković
PPTX
Dashlane Triple Track
Implementing Continuous Product Delivery
Building a guided analytics forecasting platform with Knime
AgilePM® V2 - Agile Project Management V2 - Foundation
Agile methodology in cloud computing
Easy to Know Software Dev. Models
RPALM for SAP S/4 Journey
PM, Scrum and TFS - Ivan Marković
Dashlane Triple Track

What's hot (20)

PPTX
Comparative study on agile software development
DOC
Resume2016
DOC
Resume2016
PDF
How to select the ALM solution that best suits your needs
PPTX
Top 5 mistakes in agile journey
PPTX
Contracting an outsourced agile test team
PPTX
Gaming Testing
PDF
Cprime Automated Jira and Confluence Health Checks: Boost your Operational Ef...
PPTX
A Top Down Approach to End-to-End Testing
PDF
CustomerCount Capabilities
DOC
Michael Osani_Resume
PPTX
Agile project management
PPTX
Application Modeller
PDF
Happy & Healthy - Automated Process and System Health Checks for your Atlassi...
PPTX
Quality strategies in Agile Transformation
PPTX
Daffodills - Corporate Profile
PPTX
TaaS Webinar
PPTX
Case metaconomy january2010
PPTX
Assure TotalView - Analytics for Application Delivery
PDF
Free Online Agile & SCRUM Study Training Material for PMI ACP Certification P...
Comparative study on agile software development
Resume2016
Resume2016
How to select the ALM solution that best suits your needs
Top 5 mistakes in agile journey
Contracting an outsourced agile test team
Gaming Testing
Cprime Automated Jira and Confluence Health Checks: Boost your Operational Ef...
A Top Down Approach to End-to-End Testing
CustomerCount Capabilities
Michael Osani_Resume
Agile project management
Application Modeller
Happy & Healthy - Automated Process and System Health Checks for your Atlassi...
Quality strategies in Agile Transformation
Daffodills - Corporate Profile
TaaS Webinar
Case metaconomy january2010
Assure TotalView - Analytics for Application Delivery
Free Online Agile & SCRUM Study Training Material for PMI ACP Certification P...
Ad

Similar to How to Achieve and Maintain High Quality SaaS Software in the Cloud (20)

PPTX
Enhancing Software Quality
PPTX
UNIt-4 UNIt-4 UNIt-4 UNIt-4 UNIt-4 UNIt-4 UNIt-4 UNIt-4 UNIt-4
PPTX
20151012_AQAA PPT
PDF
BlackLine-Executive-Summary-with-Product-Overviews
PDF
QA & Testing Capabilities - Sigma Infosolutions
PPTX
Software testing
DOCX
Some Commonly Asked Question For Software Testing
PPTX
SDLCTesting
PPTX
BenchmarkQA Software Quality Assurance Services Overview
PDF
Most Advanced Software Testing Solution Providers of 2022.pdf
PPT
Software Testing Presentation in Cegonsoft Pvt Ltd...
PPTX
Quality assurance by Sadquain
PPTX
Top 15 Interview Questions for Manual Testers
PDF
Corporate Profile(1)
PDF
Career Choice for Graduates
PPTX
Lecture 08 (SQE, Testing, PM, RM, ME).pptx
PPT
Savy training 01.11.2012
PDF
Ta3s - Testing Banking and Finance Applications
Enhancing Software Quality
UNIt-4 UNIt-4 UNIt-4 UNIt-4 UNIt-4 UNIt-4 UNIt-4 UNIt-4 UNIt-4
20151012_AQAA PPT
BlackLine-Executive-Summary-with-Product-Overviews
QA & Testing Capabilities - Sigma Infosolutions
Software testing
Some Commonly Asked Question For Software Testing
SDLCTesting
BenchmarkQA Software Quality Assurance Services Overview
Most Advanced Software Testing Solution Providers of 2022.pdf
Software Testing Presentation in Cegonsoft Pvt Ltd...
Quality assurance by Sadquain
Top 15 Interview Questions for Manual Testers
Corporate Profile(1)
Career Choice for Graduates
Lecture 08 (SQE, Testing, PM, RM, ME).pptx
Savy training 01.11.2012
Ta3s - Testing Banking and Finance Applications
Ad

More from XBOSoft (20)

PDF
Agile Metrics to Boost Software Quality improvement
PDF
Agile Test Management Using Jira and Zephyr
PDF
AI Based Test Automation Without AI
PDF
What Aircrews Can Teach Software Testing Teams - XBOSoft Webinar w/Peter Varhol
PDF
Agile User Acceptance Testing - Incorporating UAT into Agile
PDF
Challenges in Using Big Data for Software QA
PDF
Defect Patterns Analysis for Agile and Waterfall - XBOSoft Webinar with Micha...
PDF
Proactive SQA™ Shifting Left w/Proactive Software Quality Practices
PDF
Mobile Testing Challenges and Solutions XBOSoft Webinar
PDF
Heidi Araya - XBOSoft Webinar Guest Speaker - Working with Remote Agile Teams
PDF
XBOSoft webinar - How Did I Miss That Bug - Cognitive Biases in Software Testing
PDF
PSQT Keynote: Quality Challenges in the Internet of Things Era
PDF
7 Habits of Highly Effective Agile Testing - Test Istanbul
PDF
Managing Agile Software Projects With Risk and Uncertainty
PDF
Top IOT Testing Challenges Webinar with Jon Hagar
PDF
Testing in Agile with Coaching Agile Journeys and XBOSoft
PDF
Using JMeter and Google Analytics for Software Performance Testing
PDF
Storytelling: Discover the Big Picture for Agile Efforts Webinar - Tom Cagley...
PDF
ASTQB w/ XBOSoft CEO Phil Lew: Agile and Waterfall - What Do Testers Do Diffe...
PPT
When Agile is a Quality Game Changer Webinar - Michael Mah, Philip Lew
Agile Metrics to Boost Software Quality improvement
Agile Test Management Using Jira and Zephyr
AI Based Test Automation Without AI
What Aircrews Can Teach Software Testing Teams - XBOSoft Webinar w/Peter Varhol
Agile User Acceptance Testing - Incorporating UAT into Agile
Challenges in Using Big Data for Software QA
Defect Patterns Analysis for Agile and Waterfall - XBOSoft Webinar with Micha...
Proactive SQA™ Shifting Left w/Proactive Software Quality Practices
Mobile Testing Challenges and Solutions XBOSoft Webinar
Heidi Araya - XBOSoft Webinar Guest Speaker - Working with Remote Agile Teams
XBOSoft webinar - How Did I Miss That Bug - Cognitive Biases in Software Testing
PSQT Keynote: Quality Challenges in the Internet of Things Era
7 Habits of Highly Effective Agile Testing - Test Istanbul
Managing Agile Software Projects With Risk and Uncertainty
Top IOT Testing Challenges Webinar with Jon Hagar
Testing in Agile with Coaching Agile Journeys and XBOSoft
Using JMeter and Google Analytics for Software Performance Testing
Storytelling: Discover the Big Picture for Agile Efforts Webinar - Tom Cagley...
ASTQB w/ XBOSoft CEO Phil Lew: Agile and Waterfall - What Do Testers Do Diffe...
When Agile is a Quality Game Changer Webinar - Michael Mah, Philip Lew

Recently uploaded (20)

PDF
KodekX | Application Modernization Development
PDF
Review of recent advances in non-invasive hemoglobin estimation
PPTX
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
PDF
Empathic Computing: Creating Shared Understanding
PPTX
MYSQL Presentation for SQL database connectivity
PDF
Bridging biosciences and deep learning for revolutionary discoveries: a compr...
PDF
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
PDF
Modernizing your data center with Dell and AMD
PDF
Spectral efficient network and resource selection model in 5G networks
PDF
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
PPTX
20250228 LYD VKU AI Blended-Learning.pptx
PDF
Shreyas Phanse Resume: Experienced Backend Engineer | Java • Spring Boot • Ka...
PDF
Building Integrated photovoltaic BIPV_UPV.pdf
PDF
Agricultural_Statistics_at_a_Glance_2022_0.pdf
PDF
NewMind AI Weekly Chronicles - August'25 Week I
PDF
Encapsulation theory and applications.pdf
PDF
Diabetes mellitus diagnosis method based random forest with bat algorithm
PDF
NewMind AI Monthly Chronicles - July 2025
PPTX
Understanding_Digital_Forensics_Presentation.pptx
PDF
CIFDAQ's Market Insight: SEC Turns Pro Crypto
KodekX | Application Modernization Development
Review of recent advances in non-invasive hemoglobin estimation
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
Empathic Computing: Creating Shared Understanding
MYSQL Presentation for SQL database connectivity
Bridging biosciences and deep learning for revolutionary discoveries: a compr...
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
Modernizing your data center with Dell and AMD
Spectral efficient network and resource selection model in 5G networks
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
20250228 LYD VKU AI Blended-Learning.pptx
Shreyas Phanse Resume: Experienced Backend Engineer | Java • Spring Boot • Ka...
Building Integrated photovoltaic BIPV_UPV.pdf
Agricultural_Statistics_at_a_Glance_2022_0.pdf
NewMind AI Weekly Chronicles - August'25 Week I
Encapsulation theory and applications.pdf
Diabetes mellitus diagnosis method based random forest with bat algorithm
NewMind AI Monthly Chronicles - July 2025
Understanding_Digital_Forensics_Presentation.pptx
CIFDAQ's Market Insight: SEC Turns Pro Crypto

How to Achieve and Maintain High Quality SaaS Software in the Cloud

  • 1. How to Achieve and Maintain High Quality SaaS Software in the Cloud Lessons learned and best practices on how to set up test automation and performance testing
  • 2. Agenda • Fire side chat – Introduction of the speakers – Conversation between QA experts – Q & A at the end
  • 3. Howard Goldstein VP of Marketing BlackLine Systems • Built BlackLine’s marketing program from the ground up. • Oversaw BlackLine’s expansion internationally into the EMEA and APAC markets. • Developed strategic partner relationships with leading global consulting firms.
  • 4. Donna McCollum QA Manager BlackLine Systems • Managed BlackLine’s Quality Assurance program for four years. • Served as the key liaison to XBOSoft overseeing functional and regression testing. • Previously was Manager of Financial Technology at industry leader SunGard Treasury Systems. • Developed institutional trading software.
  • 5. Philip Lew CEO and Founder XBOSoft • Leads XBOSoft’s direction and strategy as their CEO. • Ph.D. research in software quality and usability. • Published in several IEEE and ACM journal magazines. • Presenter at various academic and trade conferences. • Industry leader in software quality and measurement.
  • 6. Housekeeping • The audience will be on mute for a clear conversation • Feel free to ask questions at anytime via the chat bar and we will fit them in where appropriate. • Contact information will be given at the end and you will receive information on where to find the slide and the recording. Let’s get Started!
  • 7. BlackLine Company overview LA-based financial software provider, operating globally First to market with automated account reconciliation software 475+ clients, many of which are in the Fortune 500 More than 58,000 users worldwide ‘BlackLine Certified Implementation Professionals’ from all the major advisory/consulting firms and all around the world
  • 8. GLOBAL DEPLOYMENT BlackLine OnDemand users (89+ COUNTRIES) BlackLine Offices
  • 13. •SIMPLE, SECURE ONDEMAND/SAAS OFFERING •REAL-TIME MONITORING AND REPORTING •CONTROLLED PROCESS •DATA INTEGRITY & INTEGRATION
  • 14. BLACKLINE PROVIDES EFFICIENCY, SPEED & VISIBILITY REDUCED COSTS AND RISKS CONTROLLED FINANCIAL CLOSE
  • 15. ROI & CUTTING COSTS • Auto-certification MeadWestVaco Corporation • Increased staff ROI: 495% Payback: 2 Months productivity Cox Communications • Reduced paper, ROI: 227% Payback: 6 Months printing, storage Education Management Corporation overhead, auditor fees ROI: 138% Payback: 10 Months • Improved process management Return on investment is one of the most important • Faster close indicators of value
  • 16. Regression Testing Image from: http://www.testfacebook.com
  • 17. Regression Testing Key Points • Impossible to test everything so implement regression strategy based on context • Use Risk Assessment based on actual data • Exploratory Testing • Defect Analysis
  • 19. Performance Key Points • Understand your users so you can develop realistic performance scenario’s • Benchmark and Improve • Periodic Efforts
  • 20. Test Automation Image from simply-the-test.blogspot.com
  • 21. Key Points • What to automate depends on maturity of the product and stability of modules and features • Team is more important than the tool • Set up a frame work
  • 22. Best Practices & Lessons Learned • Daily Build’s • Goal 100% coverage, Risk Assessment when that’s not feasible. • Fast and accurate: Accuracy is of primary importance, but performance is a very close second. • Automation is used where the product is mature and stable. • QTP is used, but the quality of your people and framework of the tests is more important. • The remote XBOSoft QA team and the BlackLine development and support teams work very closely together. • The follow-the-sun workflow works very well.
  • 23. Q&A
  • 24. Contact info Information on where to find the slides and the recording will be mailed. Please follow us on twitter @xbosoft Request for info: services@xbosoft.com

Editor's Notes

  • #2: This screen stays on while people wait for the webinar
  • #17: If you cannot test all test cases in a test cycle, how do you decide what test cases to run?What percentage of test cases do you run to get good regression coverage?How much time / resources do you dedicated to exploratory testing in your regression tests?How do you track defects from regression tests?Is performance testing part of regression testing?
  • #19: How important is performance for your product?What performance issues are typical for a SaaS product?How often do you run performance tests?
  • #21: What tools do you use to automate?What percentage of QA resources do you devote to test automation?What percentage of your regression tests do
  • #23: Testing on multiple devices, platforms and browsersDedicated QA, remote QA team, yet an integrated QA team