SlideShare a Scribd company logo
SaaS Testing Overview

                                              Ram Garg
                                                  CEO

                Software Performance Assurance (SPA)

Software Performance Assurance Confidential
Outline
           SaaS Introduction
                 What is it?
                 Why SaaS?

           SaaS Challenges
                 Key Challenges

           SaaS Testing
                 How to approach SaaS testing?
                 How SaaS Testing is different?
                 SaaS Test Tools




Software Performance Assurance Confidential
What is SaaS?

                 SaaS Introduction


Software Performance Assurance Confidential
SaaS: What is it?

Software-as-a-Service (SaaS) is the application
layer of the cloud computing model.

Application                 Software as a Service (SaaS)


Platform                    Platform as a Service (PaaS)


Infrastructure              Infrastructure as a Service (IaaS)

Software Performance Assurance Confidential
SaaS Definition


   “Software as a service (SaaS), sometimes
    referred to as "software on demand," is
    software that is 100% deployed over the
  internet … . With SaaS, a provider licenses
    an application to customers as a service
    on demand, through a subscription or a
            "pay-as-you-go" model.”
                                                  – Wikipedia definition




Software Performance Assurance Confidential
Architecture: Three Key Differentiators


                                      Multi-tenant
                                       Efficient


         Scalable                                      Configurable



                                           Key
                                      Architecture
                                     Differentiators


Software Performance Assurance Confidential
Why SaaS?
                 SaaS Introduction


Software Performance Assurance Confidential
The Software Business Model Changes …

                                      non -SaaS                SaaS

Software Ownership                        Customer             Provider


Infrastructure
                                          Customer             Provider
management Responsibility



Installation                           Client - Server         Central



Upgrades                           Larger, less frequent   Shorter, frequent



Platform                                Multi-version       Single-version



 Software Performance Assurance Confidential
Changing the Business Model cont …

                                      non -SaaS        SaaS

Configurability
                                               Low   High, Critical
Requirement



Multi-tenancy                                  No        Yes



Backward compatibility                         No        Yes



Maintenance                               Customer     Provider




 Software Performance Assurance Confidential
Changing the Business Model cont …

                                         non -SaaS                  SaaS



Cost, Allocation                        High, CapEx              Low, OpEx


                                      Close the deal,           Provide value,
Sales focus / Success
                                     License Revenue            Lack of Churn


Profits                         Initial sale + maintenance   Ongoing Subscription




 Software Performance Assurance Confidential
Ecosystem




SaaS (Software)        Starting point to every service within
                        cloud computing ecosystem
PaaS (Platform)        Highest on the maturity level and
                        growth curve amongst XaaS
 IaaS                  Highest revenue generating service
 (Infrastructure)
                        within cloud model for next several
                        years
SaaS Catalysts
Intrigue of access to unlimited Computing and storage
resources, made available and paid for only as needed (Pay as
you go)

Decreasing cost of bandwidth

Customer frustration with traditional software buying cycle
(license, maintenance contract, upgrades)

Perceived increase in control over the vendor relationship

Early SaaS success stories – Salesforce.com, WebEx, etc.
Demonstrate broader value proposition

You focus on your application and not on infrastructure


Software Performance Assurance Confidential
SaaS Early Adoption
                 SaaS Introduction


Software Performance Assurance Confidential
SaaS Adoption: Verticals
      AMI-Partners„ report -
      "Software-as-Service Hot                               Adoption rate of
      Spots” : Education,                                    SaaS in SMB is
      Healthcare, and Financial"                             higher than large
                                                                enterprises



                                              All research points out that
       In-Stat research on                    SaaS has been adopted
      vertical market adoption
      shows the healthcare                    heavily in :
      market is currently most                       HR applications (ERP,
      satisfied with hosted                           Salesforce automation,
      applications, with                              eRecruiting)
      education identified as an                     Online Collaboration (Web
      important potential target                      conferencing)
      market also.                                   CRM
                                                     Healthcare

Software Performance Assurance Confidential
SaaS Adoption Factors
                                                                   (source: Forrester)



            Availability & Reliability (71%)           Fast Deployment (45%)




              Reduced dependencies on
                                                     Flexible consumption (35%)
                  internal IT (24%)




                                  Simplified pricing model (32%)




Software Performance Assurance Confidential
The SaaS Market is and will
                  Continue to Grow…




Software Performance Assurance Confidential
Today and tomorrow …




Software Performance Assurance Confidential
Today and tomorrow …




Software Performance Assurance Confidential
Today and tomorrow …




Software Performance Assurance Confidential
Challenges
                 SaaS Challenges


Software Performance Assurance Confidential
Primary Concerns with the SaaS Model

Security concerns


Concerns over compliance with standards


Concerns over integration with legacy applications


Concerns over reliability and robustness of service

Not comfortable with putting mission-critical application on shared
infrastructure and have access over internet

SaaS can not be configured to meet my corporation‟s all specific
needs


Software Performance Assurance Confidential
Numerous Challenges



    Security              Performance           Availability          SLA


                                               Compliance
Data Integrity                 Privacy            with          Regulatory
                                               Standards


                                                           Lack of
               Integration             Multi-tenancy
                                                         confidence




Software Performance Assurance Confidential
SaaS Testing



Software Performance Assurance Confidential
Why SaaS Testing?
                 SaaS Testing


Software Performance Assurance Confidential
The average enterprise IT
                                                Why SaaS Testing?
   department devotes up to 50
   percent of its entire technology
   infrastructure to development                     A delay in website page
   and test, with up to 90 percent                   loading by even one second
   of the available test                             can impact conversions by
   infrastructure remaining idle.                    7% and customer satisfaction
                       IBM Research                  by 16%.
                                                                  Aberdeen Group


                                       Cloud
                                      Testing

                                                    Those using cloud facilities to
                                                    run applications, 49.8% said
                                                    they were doing so
  The average online                                experimentally or for
  business had 8 days of                            prototyping; 28.6% for non-
  downtime a year due to                            critical business applications
  performance issues                                and 21.6% for business
  leading to lost revenues                          critical applications
           Aberdeen Group                                       Evans Data Survey




Software Performance Assurance Confidential
Trend leads to trend …




Overall growth in Cloud Computing (investment,
 revenue, ROI, market cap, etc.) is leading to a
  marked increase in adoption of SaaS Testing
What is SaaS Testing?
                 SaaS Testing


Software Performance Assurance Confidential
Classifying SaaS Testing

                                                    “Testing The Cloud”
                                                                  SaaS”
                                              Testing applications which are in
                                                cloud or migrating to cloud.

      Testing                                 Test the performance, reliability &
     the SaaS                                       security of applications


                                  Cloud /
                                   SaaS          “Testing SaaS Using Cloud”
                                                    “Testing Using Cloud”
                                                Testing applications using the
                                  Testing     cloud-based hardware, software,
      Testing                                   infrastructure and computing
                                                          resources.
       SaaS
       Using                                  Tests performed like performance,
       Cloud                                     load & stress, security and
                                                         compatibility




Software Performance Assurance Confidential
How SaaS Testing is Different

            Easier                                Challenges
• No client-server                            • Frequent live upgrades
  installation
                                              • Too many pages
• No multiplatform support
                                              • Multiple privileges
• No multiple version
  upgrades                                    • Immature SaaS-specific
• No backward compatibility                     testing tools
                                              • Access partitions critical




Software Performance Assurance Confidential
Approach
                 SaaS Testing


Software Performance Assurance Confidential
SaaS Testing Preparation




                                               Prepare
    Assess           Understand                                         Prepare
                                              Test Plan    Prepare
    Testing            Usage                                              Test
                                              and Test    Test Data
    Needs             Patterns                                        Environment
                                                Suite




Software Performance Assurance Confidential
SaaS Testing Areas

                                      Infrastructure
                                          Testing

                Network                                Application
                testing                                  testing


                                         Secured
                                          SaaS
                                        Application



       Ensure SaaS testing does not damage
             the shared infrastructure
Software Performance Assurance Confidential
SaaS Functional Testing
  • Perform Manual and Exploratory test (similar to traditional
    applications)
  • Check Browser compatibility
  • Perform regression
    • SaaS regression must cover significant more functionality as single
      defect will impact all the customers

  • Functional and regression test automation.
    • This is important to comprehensive testing cost-effectively as above
      point indicates that, in case of SaaS, regression should cover more
      functionality.

  • Test SaaS on cloud-based staging environment
    • Testing on in-house environment and then directly deploying on cloud-
      based production environment may result in chaos

  • Test in a multi-user environment where different users are
    performing different activities.
Software Performance Assurance Confidential
SaaS Security Testing

   • Test application security
     • Cross Site scripting, SQL Injection, HTTP header injection

   • Test network security
     • Secured communication, Backup and storage policies, Controlled
       access to sites and servers

   • Test for possible attacks
   • Test roles and privileges in multi-tenant environment
     • Many roles and privilege levels

   • Test all the upgrades to detect broken security,
     Ensure security patches are not missed
   • Test data security / integrity among multiple tenants


Software Performance Assurance Confidential
SaaS Security Testing cont …

   • Identify scenarios where application might be
     vulnerable
   • Test PCI compliance
   • Test cookie setting, SSL configuration, information
     leak, etc.
   • Log errors, warnings and suspicious requests
   • Tests tampering with the licensing mechanism;
     gaining unauthorized access to application data,
     editing registry entities or configuration files



Software Performance Assurance Confidential
SaaS Performance Testing
 • Test performance extensively first time and for each upgrade
   • SaaS is revenue generating and business impacting applications.
     Single performance issue will impact all your customers
   • Test performance extensively as SaaS need to handle large number
     of users and we don‟t have luxury of re-booting or going down every
     once in a while

 • Test load characteristics, ramp up rate, protocols, load
   origination,
 • Establish ability to test performance of your SaaS isolated
   fashion so that network related issues can be identified and
   passed to network infrastructure and internet providers
 • Test key metrics and SLAs
 • Test performance for mobile users



Software Performance Assurance Confidential
In-Service Upgrade Testing
  • Ability to upgrade (full release, patches) system with zero-
    to-minimum down time is essential
  • Global users make this complicated as somewhere in the
    world someone might be accessing your application
  • Thoroughly test in staging environment before perform
    upgrade ass this is one of the most complex features


Remote Accessibility and Usage
  • Global access with acceptable response time
  • 24 hours availability


Software Performance Assurance Confidential
Disaster Recovery
  • Shutdown to bring up application quickly: server configuration;
    install and deploy software ; restore data

  • Roll back to last known stable data


Localization
  • Make sure it supports international characters as SaaS
    application can be accessed from anywhere in the world
  • Optimize by supporting target market language




Software Performance Assurance Confidential
Test Tools
                 SaaS Testing


Software Performance Assurance Confidential
Traditional and Cloud-based Test Tools

                                                     Cloud-based Test
                   Traditional Test Tools
                                                           Tools

                   QTP, WinRunner
                   Rational Robot
                   Rational Functional Tester
Functional
                   SilkTest
                                                       Gomez
                                                       PushToTest,
                   Selenium, STAFF, Fitness, Watir
                                                       SOASTA
                                                       CloudTesting
                   LoadRunner, Rational Robot,         uTest
                   Rational Performance Tester         Many others
                   Silk Performer
Performance
                   Jmeter, Grinder, WebLoad,
                   OpenSTA



Software Performance Assurance Confidential
Traditional Vs. Cloud-based Tools
           Testing SaaS Using                   Testing SaaS Using
            Traditional Tools                   Cloud-based Tools

         Many providers who provide            A few providers who provide
         SaaS testing using traditional        SaaS testing using their own
                 testing tools                    cloud-based test tools


          Many open source matured            Limited open source test tools
                   tools                                 options


         Many options for professional        Captive professional service by
           services by independent            test tools provider which is not
              service providers                even core for their business


         Professional service options          Captive profession service is
             allow cheaper cost                   sold at very high rate


Software Performance Assurance Confidential
Conclusion




Software Performance Assurance Confidential
Conclusion
Customers buy each month – keep them happy


SaaS is only a delivering model, customers will not compromise on
functionality, stability, security, and performance. They will not switch to
SaaS it if not better than traditional systems.


Save time on support, maintenance, and upgrade, etc.


You need to pay more attention to stuff like accessibility, security,
deployments. The way you earn reputation


Don‟t buy cloud-based test solution just because you are testing SaaS. In
most cases, traditional test solutions work just fine and save your 30% –
50% cost.


Software Performance Assurance Confidential

More Related Content

PPTX
Salesforce Deck Template
PDF
Low-Code App Development
PPTX
What Is Salesforce? | Salesforce Training - What Does Salesforce Do? | Salesf...
PDF
5G Service Assurance and Orchestration
PPTX
Architect your app modernization journey with containers on Microsoft Azure
PPTX
Salesforce Consulting Services
PPTX
Capgemini Cloud Assessment - A Pathway to Enterprise Cloud Migration
PDF
신규 미디어서비스 소개 : 손쉬운 라이브 소스 클라우드 입수부터 간편한 라이브 스트리밍 구축까지 – 임석영 AWS 솔루션즈 아키텍트:: A...
Salesforce Deck Template
Low-Code App Development
What Is Salesforce? | Salesforce Training - What Does Salesforce Do? | Salesf...
5G Service Assurance and Orchestration
Architect your app modernization journey with containers on Microsoft Azure
Salesforce Consulting Services
Capgemini Cloud Assessment - A Pathway to Enterprise Cloud Migration
신규 미디어서비스 소개 : 손쉬운 라이브 소스 클라우드 입수부터 간편한 라이브 스트리밍 구축까지 – 임석영 AWS 솔루션즈 아키텍트:: A...

What's hot (20)

PDF
금융 X 하이브리드 클라우드 플랫폼 - 한화생명 디지털 트랜스포메이션 전략 - 김나영 AWS 금융부문 사업개발 담당 / 박인규 AWS 금융...
PPTX
Salesforce Overview For Beginners/Students
PDF
Salesforce overview
PPTX
Introduction To Microservices
PDF
Enterprise Integration - Solution Patterns From the Field
PPTX
What Is Salesforce CRM? | Salesforce CRM Tutorial For Beginners | Salesforce ...
ODP
Bezpieczna postawa wobec psów
PDF
The Four Essential Pillars of Digital Transformation
PPTX
Azure Application Modernization
PDF
DevOps or DevSecOps
PDF
Planning Your Migration to the Lightning Experience
PDF
Microservice 4.0 Journey - From Spring NetFlix OSS to Istio Service Mesh and ...
PDF
Introduction to Salesforce | Salesforce Tutorial for Beginners | Salesforce T...
PPTX
Introduction to salesforce ppt
PPTX
Planning A Cloud Implementation
PDF
Fundamentals of Cloud Computing & AWS
PDF
Serverless Computing
PDF
APIConnect Security Best Practice
PPT
Presentation on software company
PDF
Android Mobile App Development Proposal PowerPoint Presentation Slides
금융 X 하이브리드 클라우드 플랫폼 - 한화생명 디지털 트랜스포메이션 전략 - 김나영 AWS 금융부문 사업개발 담당 / 박인규 AWS 금융...
Salesforce Overview For Beginners/Students
Salesforce overview
Introduction To Microservices
Enterprise Integration - Solution Patterns From the Field
What Is Salesforce CRM? | Salesforce CRM Tutorial For Beginners | Salesforce ...
Bezpieczna postawa wobec psów
The Four Essential Pillars of Digital Transformation
Azure Application Modernization
DevOps or DevSecOps
Planning Your Migration to the Lightning Experience
Microservice 4.0 Journey - From Spring NetFlix OSS to Istio Service Mesh and ...
Introduction to Salesforce | Salesforce Tutorial for Beginners | Salesforce T...
Introduction to salesforce ppt
Planning A Cloud Implementation
Fundamentals of Cloud Computing & AWS
Serverless Computing
APIConnect Security Best Practice
Presentation on software company
Android Mobile App Development Proposal PowerPoint Presentation Slides
Ad

Viewers also liked (20)

PDF
Testing a SaaS Platform
PDF
Testing SAAS, how to go about it?
PDF
Cloud testing: challenges and opportunities, TaaS, Integration Testing
PPT
Cloud Testing: The Future of software Testing
PDF
How to Develop Your SaaS Pricing Model
PPTX
The SaaS business model and metrics
PDF
Go to-market strategy for B2B SaaS companies
PPT
Cloud testing
PPTX
Impacted, oblique, spiral fractures
PDF
I Contratti di Sperimentazione Clinica - Studio Legale Pandolfini Assistenza ...
PDF
Franciscan Alliance Blazes New Trails in Healthcare Delivery
PPTX
Concept of market potential,crmmm market share & buying decision process
PDF
How To Build A Change Control System with SharePoint
PPTX
La valutazione ecocardiografica della funzione ventricolare destra
PPTX
Top 10 corporate communications manager interview questions and answers
PPT
Chapter 12 (motivation and work)
PDF
Avaya Network Management Overview
PDF
Top 5 Considerations for a Big Data Solution
PDF
introduction to commercial real estate development
PDF
Crm in retail
Testing a SaaS Platform
Testing SAAS, how to go about it?
Cloud testing: challenges and opportunities, TaaS, Integration Testing
Cloud Testing: The Future of software Testing
How to Develop Your SaaS Pricing Model
The SaaS business model and metrics
Go to-market strategy for B2B SaaS companies
Cloud testing
Impacted, oblique, spiral fractures
I Contratti di Sperimentazione Clinica - Studio Legale Pandolfini Assistenza ...
Franciscan Alliance Blazes New Trails in Healthcare Delivery
Concept of market potential,crmmm market share & buying decision process
How To Build A Change Control System with SharePoint
La valutazione ecocardiografica della funzione ventricolare destra
Top 10 corporate communications manager interview questions and answers
Chapter 12 (motivation and work)
Avaya Network Management Overview
Top 5 Considerations for a Big Data Solution
introduction to commercial real estate development
Crm in retail
Ad

Similar to SaaS Testing Overview - Foundation (20)

PDF
Estrategias para explotar las tendencias de SaaS y Cloud Computing
PPT
Overview of SaaS
PPTX
SaaS - Taking a Closer Look
PDF
Finance Technologies: Buy or Rent
PPT
Cloud Computing
PPTX
Introduction To Saa S Tech Ba Apr 2009
PPT
SaaS Presentation
PDF
CoreSite's 2019 SaaS Colocation Buyers Guide
PDF
eDocument Sciences SaaS 101
PDF
JD Edwards & Peoplesoft 3 _ Kristina Webb _ Seeing through the clouds - A gui...
PPSX
Saas & DBaas
PDF
SAS 70 in a Post-Sarbanes, SaaS World: Quest Session 52070
PDF
Running a World Class SaaS Organization
PPTX
Software as a service
PPSX
SaaS & DBaas
PPTX
Software as a Service
PDF
PDF
What Is SaaS: An Overview of Software as a Service
PDF
SaaS White Paper - An Overview
PDF
IRJET - An Overview of SaaS Model For Business Applications
Estrategias para explotar las tendencias de SaaS y Cloud Computing
Overview of SaaS
SaaS - Taking a Closer Look
Finance Technologies: Buy or Rent
Cloud Computing
Introduction To Saa S Tech Ba Apr 2009
SaaS Presentation
CoreSite's 2019 SaaS Colocation Buyers Guide
eDocument Sciences SaaS 101
JD Edwards & Peoplesoft 3 _ Kristina Webb _ Seeing through the clouds - A gui...
Saas & DBaas
SAS 70 in a Post-Sarbanes, SaaS World: Quest Session 52070
Running a World Class SaaS Organization
Software as a service
SaaS & DBaas
Software as a Service
What Is SaaS: An Overview of Software as a Service
SaaS White Paper - An Overview
IRJET - An Overview of SaaS Model For Business Applications

More from Ram Garg (10)

PDF
Si mobile app cover and cover story
PDF
Web Apps Development Services Brochure
PDF
Testing Services-Brochure
PDF
Mobile Apps Testing Services Brochure
PDF
Mobile App Development Brochure
PDF
Mobile App Developement and Testing Strategy Brochure
PPSX
SPAOnDemand podcast
PDF
Multisourcing the new global trend
PDF
Did you train your project management before you outsourced?
PDF
SpaOnDemand
Si mobile app cover and cover story
Web Apps Development Services Brochure
Testing Services-Brochure
Mobile Apps Testing Services Brochure
Mobile App Development Brochure
Mobile App Developement and Testing Strategy Brochure
SPAOnDemand podcast
Multisourcing the new global trend
Did you train your project management before you outsourced?
SpaOnDemand

Recently uploaded (20)

PDF
solutions_manual_-_materials___processing_in_manufacturing__demargo_.pdf
PPT
“AI and Expert System Decision Support & Business Intelligence Systems”
PDF
Unlocking AI with Model Context Protocol (MCP)
PDF
Per capita expenditure prediction using model stacking based on satellite ima...
PDF
NewMind AI Weekly Chronicles - August'25 Week I
PPTX
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
PDF
KodekX | Application Modernization Development
PDF
Advanced IT Governance
PDF
Review of recent advances in non-invasive hemoglobin estimation
PDF
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
PDF
Advanced Soft Computing BINUS July 2025.pdf
PDF
Reach Out and Touch Someone: Haptics and Empathic Computing
PPTX
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
PDF
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton
PPTX
Cloud computing and distributed systems.
PDF
Electronic commerce courselecture one. Pdf
PDF
Empathic Computing: Creating Shared Understanding
PDF
Network Security Unit 5.pdf for BCA BBA.
PDF
cuic standard and advanced reporting.pdf
solutions_manual_-_materials___processing_in_manufacturing__demargo_.pdf
“AI and Expert System Decision Support & Business Intelligence Systems”
Unlocking AI with Model Context Protocol (MCP)
Per capita expenditure prediction using model stacking based on satellite ima...
NewMind AI Weekly Chronicles - August'25 Week I
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
Dropbox Q2 2025 Financial Results & Investor Presentation
KodekX | Application Modernization Development
Advanced IT Governance
Review of recent advances in non-invasive hemoglobin estimation
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Advanced Soft Computing BINUS July 2025.pdf
Reach Out and Touch Someone: Haptics and Empathic Computing
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton
Cloud computing and distributed systems.
Electronic commerce courselecture one. Pdf
Empathic Computing: Creating Shared Understanding
Network Security Unit 5.pdf for BCA BBA.
cuic standard and advanced reporting.pdf

SaaS Testing Overview - Foundation

  • 1. SaaS Testing Overview Ram Garg CEO Software Performance Assurance (SPA) Software Performance Assurance Confidential
  • 2. Outline  SaaS Introduction  What is it?  Why SaaS?  SaaS Challenges  Key Challenges  SaaS Testing  How to approach SaaS testing?  How SaaS Testing is different?  SaaS Test Tools Software Performance Assurance Confidential
  • 3. What is SaaS? SaaS Introduction Software Performance Assurance Confidential
  • 4. SaaS: What is it? Software-as-a-Service (SaaS) is the application layer of the cloud computing model. Application Software as a Service (SaaS) Platform Platform as a Service (PaaS) Infrastructure Infrastructure as a Service (IaaS) Software Performance Assurance Confidential
  • 5. SaaS Definition “Software as a service (SaaS), sometimes referred to as "software on demand," is software that is 100% deployed over the internet … . With SaaS, a provider licenses an application to customers as a service on demand, through a subscription or a "pay-as-you-go" model.” – Wikipedia definition Software Performance Assurance Confidential
  • 6. Architecture: Three Key Differentiators Multi-tenant Efficient Scalable Configurable Key Architecture Differentiators Software Performance Assurance Confidential
  • 7. Why SaaS? SaaS Introduction Software Performance Assurance Confidential
  • 8. The Software Business Model Changes … non -SaaS SaaS Software Ownership Customer Provider Infrastructure Customer Provider management Responsibility Installation Client - Server Central Upgrades Larger, less frequent Shorter, frequent Platform Multi-version Single-version Software Performance Assurance Confidential
  • 9. Changing the Business Model cont … non -SaaS SaaS Configurability Low High, Critical Requirement Multi-tenancy No Yes Backward compatibility No Yes Maintenance Customer Provider Software Performance Assurance Confidential
  • 10. Changing the Business Model cont … non -SaaS SaaS Cost, Allocation High, CapEx Low, OpEx Close the deal, Provide value, Sales focus / Success License Revenue Lack of Churn Profits Initial sale + maintenance Ongoing Subscription Software Performance Assurance Confidential
  • 11. Ecosystem SaaS (Software)  Starting point to every service within cloud computing ecosystem PaaS (Platform)  Highest on the maturity level and growth curve amongst XaaS IaaS  Highest revenue generating service (Infrastructure) within cloud model for next several years
  • 12. SaaS Catalysts Intrigue of access to unlimited Computing and storage resources, made available and paid for only as needed (Pay as you go) Decreasing cost of bandwidth Customer frustration with traditional software buying cycle (license, maintenance contract, upgrades) Perceived increase in control over the vendor relationship Early SaaS success stories – Salesforce.com, WebEx, etc. Demonstrate broader value proposition You focus on your application and not on infrastructure Software Performance Assurance Confidential
  • 13. SaaS Early Adoption SaaS Introduction Software Performance Assurance Confidential
  • 14. SaaS Adoption: Verticals AMI-Partners„ report - "Software-as-Service Hot Adoption rate of Spots” : Education, SaaS in SMB is Healthcare, and Financial" higher than large enterprises All research points out that In-Stat research on SaaS has been adopted vertical market adoption shows the healthcare heavily in : market is currently most  HR applications (ERP, satisfied with hosted Salesforce automation, applications, with eRecruiting) education identified as an  Online Collaboration (Web important potential target conferencing) market also.  CRM  Healthcare Software Performance Assurance Confidential
  • 15. SaaS Adoption Factors (source: Forrester) Availability & Reliability (71%) Fast Deployment (45%) Reduced dependencies on Flexible consumption (35%) internal IT (24%) Simplified pricing model (32%) Software Performance Assurance Confidential
  • 16. The SaaS Market is and will Continue to Grow… Software Performance Assurance Confidential
  • 17. Today and tomorrow … Software Performance Assurance Confidential
  • 18. Today and tomorrow … Software Performance Assurance Confidential
  • 19. Today and tomorrow … Software Performance Assurance Confidential
  • 20. Challenges SaaS Challenges Software Performance Assurance Confidential
  • 21. Primary Concerns with the SaaS Model Security concerns Concerns over compliance with standards Concerns over integration with legacy applications Concerns over reliability and robustness of service Not comfortable with putting mission-critical application on shared infrastructure and have access over internet SaaS can not be configured to meet my corporation‟s all specific needs Software Performance Assurance Confidential
  • 22. Numerous Challenges Security Performance Availability SLA Compliance Data Integrity Privacy with Regulatory Standards Lack of Integration Multi-tenancy confidence Software Performance Assurance Confidential
  • 23. SaaS Testing Software Performance Assurance Confidential
  • 24. Why SaaS Testing? SaaS Testing Software Performance Assurance Confidential
  • 25. The average enterprise IT Why SaaS Testing? department devotes up to 50 percent of its entire technology infrastructure to development A delay in website page and test, with up to 90 percent loading by even one second of the available test can impact conversions by infrastructure remaining idle. 7% and customer satisfaction IBM Research by 16%. Aberdeen Group Cloud Testing Those using cloud facilities to run applications, 49.8% said they were doing so The average online experimentally or for business had 8 days of prototyping; 28.6% for non- downtime a year due to critical business applications performance issues and 21.6% for business leading to lost revenues critical applications Aberdeen Group Evans Data Survey Software Performance Assurance Confidential
  • 26. Trend leads to trend … Overall growth in Cloud Computing (investment, revenue, ROI, market cap, etc.) is leading to a marked increase in adoption of SaaS Testing
  • 27. What is SaaS Testing? SaaS Testing Software Performance Assurance Confidential
  • 28. Classifying SaaS Testing “Testing The Cloud” SaaS” Testing applications which are in cloud or migrating to cloud. Testing Test the performance, reliability & the SaaS security of applications Cloud / SaaS “Testing SaaS Using Cloud” “Testing Using Cloud” Testing applications using the Testing cloud-based hardware, software, Testing infrastructure and computing resources. SaaS Using Tests performed like performance, Cloud load & stress, security and compatibility Software Performance Assurance Confidential
  • 29. How SaaS Testing is Different Easier Challenges • No client-server • Frequent live upgrades installation • Too many pages • No multiplatform support • Multiple privileges • No multiple version upgrades • Immature SaaS-specific • No backward compatibility testing tools • Access partitions critical Software Performance Assurance Confidential
  • 30. Approach SaaS Testing Software Performance Assurance Confidential
  • 31. SaaS Testing Preparation Prepare Assess Understand Prepare Test Plan Prepare Testing Usage Test and Test Test Data Needs Patterns Environment Suite Software Performance Assurance Confidential
  • 32. SaaS Testing Areas Infrastructure Testing Network Application testing testing Secured SaaS Application Ensure SaaS testing does not damage the shared infrastructure Software Performance Assurance Confidential
  • 33. SaaS Functional Testing • Perform Manual and Exploratory test (similar to traditional applications) • Check Browser compatibility • Perform regression • SaaS regression must cover significant more functionality as single defect will impact all the customers • Functional and regression test automation. • This is important to comprehensive testing cost-effectively as above point indicates that, in case of SaaS, regression should cover more functionality. • Test SaaS on cloud-based staging environment • Testing on in-house environment and then directly deploying on cloud- based production environment may result in chaos • Test in a multi-user environment where different users are performing different activities. Software Performance Assurance Confidential
  • 34. SaaS Security Testing • Test application security • Cross Site scripting, SQL Injection, HTTP header injection • Test network security • Secured communication, Backup and storage policies, Controlled access to sites and servers • Test for possible attacks • Test roles and privileges in multi-tenant environment • Many roles and privilege levels • Test all the upgrades to detect broken security, Ensure security patches are not missed • Test data security / integrity among multiple tenants Software Performance Assurance Confidential
  • 35. SaaS Security Testing cont … • Identify scenarios where application might be vulnerable • Test PCI compliance • Test cookie setting, SSL configuration, information leak, etc. • Log errors, warnings and suspicious requests • Tests tampering with the licensing mechanism; gaining unauthorized access to application data, editing registry entities or configuration files Software Performance Assurance Confidential
  • 36. SaaS Performance Testing • Test performance extensively first time and for each upgrade • SaaS is revenue generating and business impacting applications. Single performance issue will impact all your customers • Test performance extensively as SaaS need to handle large number of users and we don‟t have luxury of re-booting or going down every once in a while • Test load characteristics, ramp up rate, protocols, load origination, • Establish ability to test performance of your SaaS isolated fashion so that network related issues can be identified and passed to network infrastructure and internet providers • Test key metrics and SLAs • Test performance for mobile users Software Performance Assurance Confidential
  • 37. In-Service Upgrade Testing • Ability to upgrade (full release, patches) system with zero- to-minimum down time is essential • Global users make this complicated as somewhere in the world someone might be accessing your application • Thoroughly test in staging environment before perform upgrade ass this is one of the most complex features Remote Accessibility and Usage • Global access with acceptable response time • 24 hours availability Software Performance Assurance Confidential
  • 38. Disaster Recovery • Shutdown to bring up application quickly: server configuration; install and deploy software ; restore data • Roll back to last known stable data Localization • Make sure it supports international characters as SaaS application can be accessed from anywhere in the world • Optimize by supporting target market language Software Performance Assurance Confidential
  • 39. Test Tools SaaS Testing Software Performance Assurance Confidential
  • 40. Traditional and Cloud-based Test Tools Cloud-based Test Traditional Test Tools Tools QTP, WinRunner Rational Robot Rational Functional Tester Functional SilkTest Gomez PushToTest, Selenium, STAFF, Fitness, Watir SOASTA CloudTesting LoadRunner, Rational Robot, uTest Rational Performance Tester Many others Silk Performer Performance Jmeter, Grinder, WebLoad, OpenSTA Software Performance Assurance Confidential
  • 41. Traditional Vs. Cloud-based Tools Testing SaaS Using Testing SaaS Using Traditional Tools Cloud-based Tools Many providers who provide A few providers who provide SaaS testing using traditional SaaS testing using their own testing tools cloud-based test tools Many open source matured Limited open source test tools tools options Many options for professional Captive professional service by services by independent test tools provider which is not service providers even core for their business Professional service options Captive profession service is allow cheaper cost sold at very high rate Software Performance Assurance Confidential
  • 43. Conclusion Customers buy each month – keep them happy SaaS is only a delivering model, customers will not compromise on functionality, stability, security, and performance. They will not switch to SaaS it if not better than traditional systems. Save time on support, maintenance, and upgrade, etc. You need to pay more attention to stuff like accessibility, security, deployments. The way you earn reputation Don‟t buy cloud-based test solution just because you are testing SaaS. In most cases, traditional test solutions work just fine and save your 30% – 50% cost. Software Performance Assurance Confidential