SlideShare a Scribd company logo
Mobile App Quality Roadmap for DevTest Teams
Dec 2015
© 2015, Perfecto Mobile Ltd. All Rights Reserved.
A few housekeeping items
1. Submit
questions using the
Q&A panel.
2. Slides and
recording sent
after the webinar.
3. Please complete
survey after the
presentation.
12/16/2015 3© 2015, Perfecto Mobile Ltd. All Rights Reserved.
The Common Agile Challenge
Maturity
Aspiration
Distress call!
• Release late by 2.5 months (from committed baseline)
• Low quality confidence
• Coverage gap concerns
• High defect rate
• “Stabilization” exposes more issues
• Low customer ratings after the delayed release
• Is this “Agile” or “Fragile”??? 
Reality … Testing Slows Down Agile 1.0 Teams ...
12/16/2015 5© 2015, Perfecto Mobile Ltd. All Rights Reserved.
“We are doing agile iterations for development
and testing and it is going quite well. One of our
greatest challenges is dealing with the stuff we
cannot finish within the iteration – currently its
one big pile at the end”
VP R&D for a 1000-
person group
“Let’s leave the serious testing for
the release testing phase”
Why?
A limited amount of feedback comes inside the development cycle (days)
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
done
done done
Late testing
 Slower velocity
 Higher cost
Test Execution
Agile – Take 2
7
In Real Agile – Specify Acceptance Tests FIRST
to DRIVE Design/Development
Elaborate
Requirements
Test Design
Technical
Design
Coding / Unit
Testing
Test
Execution +
Fixing
Done
Apply ATDD Thinking
Test scenarios guide code design
ATDD* = Build Quality Into Design to prevent defects rather than just finding them
http://www.agilesparks.com/test-first-reading-list
Test Automation
*ATDD: Acceptance Test Driven Design
Define acceptance tests before developers begin coding
Start earlier Done is Done
Testers create more detailed test
examples
Developers use them to guide technical
design/specification
Team pulls a few stories
Story is pulled into
development
Testers focus on detailed test plans,
automation preparation, data
preparation.
Developers code and do
developer/unit/small testing
Developer verifies the story passes the
team “ready for story testing” criteria:
• Code is “in the build”
• The acceptance tests for this story
pass in the dev env
• Good code coverage using
automated unit tests
• Sanity test passes
Result: Teams using ATDD
Thinking get much higher,
automated, test
coverage(some above 90%)
than the typical test-last
teams. Both due to discipline
as well as due to developers
awareness to the acceptance
tests
Developer marks story
as dev done, ready for
testing
Development done means the acceptance tests already
pass
Tester pulls story into
testing
• Tester finds significantly fewer defects
• Product Owner is happier with how the
story works & finds fewer surprises.
• Both as a result of deeper alignment
achieved in the specification workshops
with ATDD scenarios.
Developers meanwhile
pull more stories into
coding
UAT
Regression
Performance
Security
Functional
Progression
Exploratory
Achieve Agile Testing Flow at the Story AND
Feature/Epic/Iteration level
1. Add Feature/Epic
level testing
2. left-shift more and more testing through
automation, enabling teams using
environments/tools/knowhow and more.
The 3-level Test Strategy Pyramid -
Story+Feature/Epic/Iteration+Release
Across platforms and user conditions
Automation Ownership Poll
Who’s responsible for automation in your world?
1. Automation COE separate team/department
2. Automation COE engineers assigned to the Agile Teams
3. Test Engineers/Testers on Agile Teams with support by Automation COE experts
4. Developers/Testers on the Agile Team
5. Other
14
Continuous Integration/Delivery
Needs an Automation Engine
Complex Delivery Pipelines
Delivery of App and Config
Robust and Highly Available
Results Report
SOURCE
CODE
CONTROL
SYSTEM
Code
Commit
</>
Development Production
Continuous Integration/Delivery
Commit Build Test Stage Deploy



Development Production
Continuous Integration/Delivery
Commit Build Test Stage Deploy
Jenkins – #1 Automation
Platform for CI/CD
“Jenkins is for building, testing
and delivering software
projects continuously.”
10 Years of Open Source Innovation
http://www.jenkins-ci.org
• Pipeline-As-Code
• Version control, Code review, etc.
• Entire flow is one concise Groovy script using Workflow DSL
• For loops, try-finally, fork-join, etc
• Stages throttle concurrency of builds
• Allocate build nodes and workspaces
• As many as you want, when you want
• Human input/approval integrated into flow
Multi-Stage CI/CD: Enter Jenkins Workflow
Jenkins Workflow:
Powers Multi-Stage CI/CD Pipelines
Development Deploy/Publish
Commit Build Regression Release
? ?
 Branching
 Looping
 Restarts
 Checkpoints
 Manual Input
 Conditionals
Unit Test
Smoke Test
Jenkins Workflow
Continuous Feedback
CloudBees Jenkins
Workflow Stage View
Jenkins Workflow Stage View
SOURCE CODE
CONTROL
SYSTEM
Code
Commit
</>
Development Production
Jenkins Workflow
Mobile CI processes
Code
Test
Resource
IDE
Debug
Commit
Smoke
Regression
Edge
Perfecto CQ Lab
12/16/2015 21© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Easily Applied for Browsers …
Mobile Create Challenges …
UI
Integration
Unit Testing
Build Process
12/16/2015 23© 2015, Perfecto Mobile Ltd. All Rights Reserved.
5min 32s 37min 29s1 min 48S6 min 12s1 min 3S
•Connect WIFI
•Use iPhone
•Skip that OS update
•Connect to WIF
•Check Facebook
•Login your app
•Watch YouTube videos
•Answer the phone
•Clear emails
•Return to your app – check
status
•Accept LinkedIn invite
•Use navigation app
•…
UI
Integration
Unit Testing
Performance Security
Mobile Requires Shifting the Plan …
Target
Devices
Target
Env.OS
OS versions
Form factors
CPU & Memory
Manufacturers
Wearables
IOT
Networks
Orientation
Location
Notifications
Events
Device events
Edge tests
The Team and Project
12/16/2015 26© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Test Type Target Devices Time
Verification Tests
Verify the app can be installed on the devices and
basic flow works
All Few minutes
Smoke Tests
Test the “happy path” and validate the version
ready for regression (10%)
4
Up to 20
minutes
Regression Tests
Full functional tests
• 2 clean devices + 6 WT scenarios
• Happy path
8 Up to 5
hours
Edge tests
Full functional tests
• 4 different WT scenarios
• Extreme parameters
• Load with n Virtual users
16
Up to 12
hours
* Wind Tunnel devices
WT results
12/16/2015 27© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Android Galaxy 4.1Samsung Galaxy 6 5.1
Demo
Smoke
Regression
Edge
CloudBees Jenkins Platform
Editions
Enterprise Edition
Cloud/Container Support
Platinum Support Gold Support
Recap
• Minimize post-sprint testing
• ATDD is a Sprint 0 Automation enabler
• Manage the test workflow
• Continuously prioritize and refactor the test suite
• CD building blocks:
• ATDD to better achieve definition of done
• Automation that works
• Continuous optimization for fast (relevant) feedback
12/16/2015 30© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Thank You
© 2015,Perfecto Mobile Ltd. All Rights Reserved.

More Related Content

PPTX
Mobile and Web App Testing for Visual Studio Teams
PPTX
Responsive Web Design: Testing to Deliver a Consistent User Experience
PPTX
How to Test on the Right Mobile Platforms
PPTX
Testing Mobile Apps under Real User Conditions
PPTX
Testing Mobile Apps under Real User Conditions
PPT
Achieving a Winning Digital User Experience with Real World Testing
PPTX
Mobile test automation perfecto star east
PPTX
Overcoming Test Automation Obstacles
Mobile and Web App Testing for Visual Studio Teams
Responsive Web Design: Testing to Deliver a Consistent User Experience
How to Test on the Right Mobile Platforms
Testing Mobile Apps under Real User Conditions
Testing Mobile Apps under Real User Conditions
Achieving a Winning Digital User Experience with Real World Testing
Mobile test automation perfecto star east
Overcoming Test Automation Obstacles

What's hot (20)

PPTX
Virgin Media's Mobile Quality Transformation
PDF
Continuous Quality For a 5 Star Mobile Apps Delivery
PPTX
3 Free Tools That Will Help You Create the Right Mobile & Web Test Strategy
PPTX
How to Prevent App Failures with Real User Conditions
PPTX
Juc oct 2014 final
PDF
7 Keys for Unattended Test AUtomation webinar deck
POT
Mobile Test Coverage- Israel 4th meetup
PPTX
Why Apps Succeed: 4 Keys to Winning the Digital Quality Game
PDF
Cross Platform Mobile Test Automation using Selenium WebDriver by Perfecto Mo...
PPTX
How Digital Changed the Game... and how to cross platform test for it
PDF
The ultimate guide to mobile app testing with appium
PDF
Achieving 100% mobile test coverage perfecto mobile
PDF
iOS9 Launch - Mobile Dev & Test Implications
PDF
Mobile App Testing
PPTX
Testing Apps for Wearables
PDF
Webinar learn how to test any mobile app style from within eclipse using real...
PDF
Wearables meetup
PDF
Simon Bates, Manifesto Digital - Mobile Application Development: Past, Presen...
PDF
Meetup kickoff slides - Perfecto Mobile, September 2014
PDF
Hp perfecto webinar - UFT Mobile
Virgin Media's Mobile Quality Transformation
Continuous Quality For a 5 Star Mobile Apps Delivery
3 Free Tools That Will Help You Create the Right Mobile & Web Test Strategy
How to Prevent App Failures with Real User Conditions
Juc oct 2014 final
7 Keys for Unattended Test AUtomation webinar deck
Mobile Test Coverage- Israel 4th meetup
Why Apps Succeed: 4 Keys to Winning the Digital Quality Game
Cross Platform Mobile Test Automation using Selenium WebDriver by Perfecto Mo...
How Digital Changed the Game... and how to cross platform test for it
The ultimate guide to mobile app testing with appium
Achieving 100% mobile test coverage perfecto mobile
iOS9 Launch - Mobile Dev & Test Implications
Mobile App Testing
Testing Apps for Wearables
Webinar learn how to test any mobile app style from within eclipse using real...
Wearables meetup
Simon Bates, Manifesto Digital - Mobile Application Development: Past, Presen...
Meetup kickoff slides - Perfecto Mobile, September 2014
Hp perfecto webinar - UFT Mobile
Ad

Viewers also liked (17)

PPTX
Continuous Mobile - Testing Using Jenkins - A How To Guide
PPTX
Continuous delivery mobile application development
PPTX
Quality Dashboard Roadmap
PDF
Reliability roadmap using quality function deployment
PDF
Dev Ops and PaaS - Accelerate Application Delivery with OpenShift
PPTX
Managing Continuous Delivery of Mobile Apps - for the Enterprise
PPT
Qtp 80 Basics3561
PPTX
QTP Automation Testing Tutorial 2
PPTX
TechTalk: Report Bugs Like a Boss
PPTX
Selenium Testing
PPTX
Webinar: Appium & Perfecto: A Perfect Match
PDF
Ten reasons why now is the perfect time to get serious about the mobile web
PPT
Qtp 8.0 basic
PPTX
TechTalk: Taking the Mystery Out of Object ID Automation
PPTX
Qtp object repository
PDF
10 reasons why now is the perfect time to get started with the mobile web
PDF
Test Automation Tool comparison – HP UFT/QTP vs. Selenium
Continuous Mobile - Testing Using Jenkins - A How To Guide
Continuous delivery mobile application development
Quality Dashboard Roadmap
Reliability roadmap using quality function deployment
Dev Ops and PaaS - Accelerate Application Delivery with OpenShift
Managing Continuous Delivery of Mobile Apps - for the Enterprise
Qtp 80 Basics3561
QTP Automation Testing Tutorial 2
TechTalk: Report Bugs Like a Boss
Selenium Testing
Webinar: Appium & Perfecto: A Perfect Match
Ten reasons why now is the perfect time to get serious about the mobile web
Qtp 8.0 basic
TechTalk: Taking the Mystery Out of Object ID Automation
Qtp object repository
10 reasons why now is the perfect time to get started with the mobile web
Test Automation Tool comparison – HP UFT/QTP vs. Selenium
Ad

Similar to Mobile App Quality Roadmap for DevTest Teams (20)

PPTX
Test Automation in Agile
PPTX
The Journey to DevOps
PPTX
Real Testing Scenario Strategy - Bringing It All Together For Success
PDF
Building Quality In in SAFe – The Testing Organization’s Perspective
PPTX
Continuous Testing - The Final Frontier
PDF
Effective Test Automation in DevOps
PDF
Thomas Haver - Mobile Testing.pdf
PDF
From Zero to Hero in 205 Days!
PPT
How to develop a common sense of "DONE"?
PDF
High Performance Software Engineering Teams
PPTX
Creating testing tools to support development
PDF
Methodology: IT test
PPT
SOASTA Webinar: Process Compression For Mobile App Dev 120612
PPTX
Product quality in agile project
PDF
NYC MeetUp 10.9
PDF
Automate your way to agility
PPTX
MGM Agile Dec 28th 2022 (003).pptx
PDF
Testing in Agile Development
PDF
Enabling Continuous Quality in Mobile App Development
PPTX
Continuous test automation
Test Automation in Agile
The Journey to DevOps
Real Testing Scenario Strategy - Bringing It All Together For Success
Building Quality In in SAFe – The Testing Organization’s Perspective
Continuous Testing - The Final Frontier
Effective Test Automation in DevOps
Thomas Haver - Mobile Testing.pdf
From Zero to Hero in 205 Days!
How to develop a common sense of "DONE"?
High Performance Software Engineering Teams
Creating testing tools to support development
Methodology: IT test
SOASTA Webinar: Process Compression For Mobile App Dev 120612
Product quality in agile project
NYC MeetUp 10.9
Automate your way to agility
MGM Agile Dec 28th 2022 (003).pptx
Testing in Agile Development
Enabling Continuous Quality in Mobile App Development
Continuous test automation

More from Perfecto by Perforce (20)

PDF
Is BDD Worth It? Considerations for Advanced Test Automation
PDF
Yoda debunks the top 5 challenges of continuous testing in the cloud
PDF
5 Mobile App Trends & What They Mean for Dev & Testing
PDF
How to Scale Digital App Testing With Jenkins & Automation You Can Trust
PDF
Mastering Cross-Browser Test Automation With Cypress and Selenium
PDF
Cloud Testing Has Never Been Easier or More Accessible
PDF
Cognitive Engineering - Shifting Right with Gated.AI Testing - DevOps Next
PDF
The Rise and Benefits of Robotic Process Automation (RPA) - DevOps Next
PDF
The New Categories of Software Defects in the Era of AI and ML - DevOps Next
PDF
Moving to Modern DevOps with Fuzzing and ML - DevOps Next
PDF
Leveraging AI and ML in Test Management Systems - DevOps Next
PDF
How Does AIOps Benefit DevOps Pipeline and Software Quality? - DevOps Next
PDF
Classification of Advanced AI and ML Testing Tools - DevOps Next
PDF
Automated Code Reviews with AI and ML - DevOps Next
PDF
Advancing the State of The Art in AI and Testing - DevOps Next
PDF
How to Prepare Your Apps for iOS 14 - Test Strategy, Coverage, & Best Practices
PDF
How to Create a Risk Based Testing Strategy With Simulators, Emulators, and R...
PPTX
Fast Data, Fast Delivery: How Smart Analysis Accelerates App Testing
PDF
Best Practices for Shifting Left Performance and Accessibility Testing
PDF
Deliver Flawless Mobile Apps Faster with CI/CD & CT
Is BDD Worth It? Considerations for Advanced Test Automation
Yoda debunks the top 5 challenges of continuous testing in the cloud
5 Mobile App Trends & What They Mean for Dev & Testing
How to Scale Digital App Testing With Jenkins & Automation You Can Trust
Mastering Cross-Browser Test Automation With Cypress and Selenium
Cloud Testing Has Never Been Easier or More Accessible
Cognitive Engineering - Shifting Right with Gated.AI Testing - DevOps Next
The Rise and Benefits of Robotic Process Automation (RPA) - DevOps Next
The New Categories of Software Defects in the Era of AI and ML - DevOps Next
Moving to Modern DevOps with Fuzzing and ML - DevOps Next
Leveraging AI and ML in Test Management Systems - DevOps Next
How Does AIOps Benefit DevOps Pipeline and Software Quality? - DevOps Next
Classification of Advanced AI and ML Testing Tools - DevOps Next
Automated Code Reviews with AI and ML - DevOps Next
Advancing the State of The Art in AI and Testing - DevOps Next
How to Prepare Your Apps for iOS 14 - Test Strategy, Coverage, & Best Practices
How to Create a Risk Based Testing Strategy With Simulators, Emulators, and R...
Fast Data, Fast Delivery: How Smart Analysis Accelerates App Testing
Best Practices for Shifting Left Performance and Accessibility Testing
Deliver Flawless Mobile Apps Faster with CI/CD & CT

Recently uploaded (20)

PPTX
ISO 45001 Occupational Health and Safety Management System
PPTX
Introduction to Artificial Intelligence
PDF
AI in Product Development-omnex systems
PDF
medical staffing services at VALiNTRY
PDF
PTS Company Brochure 2025 (1).pdf.......
PDF
Navsoft: AI-Powered Business Solutions & Custom Software Development
PPTX
CHAPTER 2 - PM Management and IT Context
PPTX
VVF-Customer-Presentation2025-Ver1.9.pptx
PDF
Odoo Companies in India – Driving Business Transformation.pdf
PDF
Understanding Forklifts - TECH EHS Solution
PDF
Which alternative to Crystal Reports is best for small or large businesses.pdf
PDF
System and Network Administraation Chapter 3
PDF
Design an Analysis of Algorithms II-SECS-1021-03
PPTX
history of c programming in notes for students .pptx
PDF
Upgrade and Innovation Strategies for SAP ERP Customers
PDF
How to Choose the Right IT Partner for Your Business in Malaysia
PPTX
Lecture 3: Operating Systems Introduction to Computer Hardware Systems
PDF
Audit Checklist Design Aligning with ISO, IATF, and Industry Standards — Omne...
PDF
SAP S4 Hana Brochure 3 (PTS SYSTEMS AND SOLUTIONS)
PPTX
Oracle E-Business Suite: A Comprehensive Guide for Modern Enterprises
ISO 45001 Occupational Health and Safety Management System
Introduction to Artificial Intelligence
AI in Product Development-omnex systems
medical staffing services at VALiNTRY
PTS Company Brochure 2025 (1).pdf.......
Navsoft: AI-Powered Business Solutions & Custom Software Development
CHAPTER 2 - PM Management and IT Context
VVF-Customer-Presentation2025-Ver1.9.pptx
Odoo Companies in India – Driving Business Transformation.pdf
Understanding Forklifts - TECH EHS Solution
Which alternative to Crystal Reports is best for small or large businesses.pdf
System and Network Administraation Chapter 3
Design an Analysis of Algorithms II-SECS-1021-03
history of c programming in notes for students .pptx
Upgrade and Innovation Strategies for SAP ERP Customers
How to Choose the Right IT Partner for Your Business in Malaysia
Lecture 3: Operating Systems Introduction to Computer Hardware Systems
Audit Checklist Design Aligning with ISO, IATF, and Industry Standards — Omne...
SAP S4 Hana Brochure 3 (PTS SYSTEMS AND SOLUTIONS)
Oracle E-Business Suite: A Comprehensive Guide for Modern Enterprises

Mobile App Quality Roadmap for DevTest Teams

  • 1. Mobile App Quality Roadmap for DevTest Teams Dec 2015 © 2015, Perfecto Mobile Ltd. All Rights Reserved.
  • 2. A few housekeeping items 1. Submit questions using the Q&A panel. 2. Slides and recording sent after the webinar. 3. Please complete survey after the presentation.
  • 3. 12/16/2015 3© 2015, Perfecto Mobile Ltd. All Rights Reserved. The Common Agile Challenge Maturity Aspiration
  • 4. Distress call! • Release late by 2.5 months (from committed baseline) • Low quality confidence • Coverage gap concerns • High defect rate • “Stabilization” exposes more issues • Low customer ratings after the delayed release • Is this “Agile” or “Fragile”??? 
  • 5. Reality … Testing Slows Down Agile 1.0 Teams ... 12/16/2015 5© 2015, Perfecto Mobile Ltd. All Rights Reserved. “We are doing agile iterations for development and testing and it is going quite well. One of our greatest challenges is dealing with the stuff we cannot finish within the iteration – currently its one big pile at the end” VP R&D for a 1000- person group “Let’s leave the serious testing for the release testing phase”
  • 6. Why? A limited amount of feedback comes inside the development cycle (days) 0 20 40 60 80 100 120 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 done done done Late testing  Slower velocity  Higher cost Test Execution
  • 8. In Real Agile – Specify Acceptance Tests FIRST to DRIVE Design/Development Elaborate Requirements Test Design Technical Design Coding / Unit Testing Test Execution + Fixing Done Apply ATDD Thinking Test scenarios guide code design ATDD* = Build Quality Into Design to prevent defects rather than just finding them http://www.agilesparks.com/test-first-reading-list Test Automation *ATDD: Acceptance Test Driven Design Define acceptance tests before developers begin coding Start earlier Done is Done
  • 9. Testers create more detailed test examples Developers use them to guide technical design/specification Team pulls a few stories
  • 10. Story is pulled into development Testers focus on detailed test plans, automation preparation, data preparation. Developers code and do developer/unit/small testing
  • 11. Developer verifies the story passes the team “ready for story testing” criteria: • Code is “in the build” • The acceptance tests for this story pass in the dev env • Good code coverage using automated unit tests • Sanity test passes Result: Teams using ATDD Thinking get much higher, automated, test coverage(some above 90%) than the typical test-last teams. Both due to discipline as well as due to developers awareness to the acceptance tests Developer marks story as dev done, ready for testing Development done means the acceptance tests already pass
  • 12. Tester pulls story into testing • Tester finds significantly fewer defects • Product Owner is happier with how the story works & finds fewer surprises. • Both as a result of deeper alignment achieved in the specification workshops with ATDD scenarios. Developers meanwhile pull more stories into coding
  • 13. UAT Regression Performance Security Functional Progression Exploratory Achieve Agile Testing Flow at the Story AND Feature/Epic/Iteration level 1. Add Feature/Epic level testing 2. left-shift more and more testing through automation, enabling teams using environments/tools/knowhow and more. The 3-level Test Strategy Pyramid - Story+Feature/Epic/Iteration+Release Across platforms and user conditions
  • 14. Automation Ownership Poll Who’s responsible for automation in your world? 1. Automation COE separate team/department 2. Automation COE engineers assigned to the Agile Teams 3. Test Engineers/Testers on Agile Teams with support by Automation COE experts 4. Developers/Testers on the Agile Team 5. Other 14
  • 15. Continuous Integration/Delivery Needs an Automation Engine Complex Delivery Pipelines Delivery of App and Config Robust and Highly Available Results Report SOURCE CODE CONTROL SYSTEM Code Commit </> Development Production Continuous Integration/Delivery Commit Build Test Stage Deploy   
  • 16. Development Production Continuous Integration/Delivery Commit Build Test Stage Deploy Jenkins – #1 Automation Platform for CI/CD “Jenkins is for building, testing and delivering software projects continuously.” 10 Years of Open Source Innovation http://www.jenkins-ci.org
  • 17. • Pipeline-As-Code • Version control, Code review, etc. • Entire flow is one concise Groovy script using Workflow DSL • For loops, try-finally, fork-join, etc • Stages throttle concurrency of builds • Allocate build nodes and workspaces • As many as you want, when you want • Human input/approval integrated into flow Multi-Stage CI/CD: Enter Jenkins Workflow
  • 18. Jenkins Workflow: Powers Multi-Stage CI/CD Pipelines Development Deploy/Publish Commit Build Regression Release ? ?  Branching  Looping  Restarts  Checkpoints  Manual Input  Conditionals Unit Test Smoke Test Jenkins Workflow Continuous Feedback
  • 19. CloudBees Jenkins Workflow Stage View Jenkins Workflow Stage View SOURCE CODE CONTROL SYSTEM Code Commit </> Development Production Jenkins Workflow
  • 21. Perfecto CQ Lab 12/16/2015 21© 2015, Perfecto Mobile Ltd. All Rights Reserved.
  • 22. Easily Applied for Browsers … Mobile Create Challenges … UI Integration Unit Testing
  • 23. Build Process 12/16/2015 23© 2015, Perfecto Mobile Ltd. All Rights Reserved. 5min 32s 37min 29s1 min 48S6 min 12s1 min 3S
  • 24. •Connect WIFI •Use iPhone •Skip that OS update •Connect to WIF •Check Facebook •Login your app •Watch YouTube videos •Answer the phone •Clear emails •Return to your app – check status •Accept LinkedIn invite •Use navigation app •…
  • 25. UI Integration Unit Testing Performance Security Mobile Requires Shifting the Plan … Target Devices Target Env.OS OS versions Form factors CPU & Memory Manufacturers Wearables IOT Networks Orientation Location Notifications Events Device events Edge tests
  • 26. The Team and Project 12/16/2015 26© 2015, Perfecto Mobile Ltd. All Rights Reserved. Test Type Target Devices Time Verification Tests Verify the app can be installed on the devices and basic flow works All Few minutes Smoke Tests Test the “happy path” and validate the version ready for regression (10%) 4 Up to 20 minutes Regression Tests Full functional tests • 2 clean devices + 6 WT scenarios • Happy path 8 Up to 5 hours Edge tests Full functional tests • 4 different WT scenarios • Extreme parameters • Load with n Virtual users 16 Up to 12 hours * Wind Tunnel devices
  • 27. WT results 12/16/2015 27© 2015, Perfecto Mobile Ltd. All Rights Reserved. Android Galaxy 4.1Samsung Galaxy 6 5.1
  • 29. CloudBees Jenkins Platform Editions Enterprise Edition Cloud/Container Support Platinum Support Gold Support
  • 30. Recap • Minimize post-sprint testing • ATDD is a Sprint 0 Automation enabler • Manage the test workflow • Continuously prioritize and refactor the test suite • CD building blocks: • ATDD to better achieve definition of done • Automation that works • Continuous optimization for fast (relevant) feedback 12/16/2015 30© 2015, Perfecto Mobile Ltd. All Rights Reserved.
  • 31. Thank You © 2015,Perfecto Mobile Ltd. All Rights Reserved.

Editor's Notes

  • #3: Submit questions using the Q&A panel. We try to answer all questions during and after the presentation. Our experts do not have access to the chat interface Participate in the polls Your participation is critical to a relevant, meaningful discussion. Watch your email for a copy of the recording and slides Today’s presentation will be recorded and sent to you. Complete the survey following the presentation Your feedback helps us provide educational tools like this webinar.
  • #5: http://www.flickr.com/photos/andreasnilsson1976/1307180278/sizes/l/in/photostream/ slipping release due to long time finalizing quality. 
  • #16: Continuous Delivery needs to be managed by a robust automation engine. The Automation engine must orchestrate the full CD pipeline including execution of each stage of the pipeline and the best of breed DevOps tooling used within each stage. Developers can simply focus on creating code. When they check in code to the Source Code Control System, the CD automation engine will see that check-in, pull the code out, run the application compile and build process, execute and monitor unit tests and functional tests, possibly deploy to a staging environment and then run user acceptance tests and then, if desired, the CD automation engine can even automatically deploy to production. (Automatic deployment to production is optional as not every organization is ready for that. However, Continuous Delivery means that there will always be production-ready code available to push to production when appropriate.) A enterprise-ready CD automation engine will need to have three characteristics: 1) It needs to be able to create and manage complex, real-world CD Pipelines. These are pipelines that may not be just linear sequential steps. There may be parallel branches for parallel testing, there may be loops and restarts, and there may be a need for human-intervention within the pipelines. The automation engine needs to support all of these. 2) Continuous Delivery often is applied to not just the application code but the actual environment that the application runs in as well. This means defining the application environment configuration (OS parameters, etc) “as code” and managing that “configuration-as-code” through the CD Pipeline along with the Application code. This helps insure that the application will perform exactly in production as it did in Dev and Test. 3) The last characteristic of an enterprise CD automation engine is that it needs to have the capabilities expected of enterprise software including the ability to run in an HA, or highly available, configuration. Since the CD Pipelines are touching production systems and are automating the path to production, the CD Automation engine needs to be HA. So, how can we do CD?
  • #17: Jenkins is open source software that was originally developed over 10 years ago.
  • #18: Here are some really cool features in Jenkins Workflow. SAs talk here about coolness of Workflow.
  • #19: As we said, CD involves creating Pipelines for application code. This means that for every code “check-in” by every developer, the new code needs to go through the Pipeline that will take it from Dev to Prod. So, the CD Platform needs to manage the execution of many of these Pipelines. And the pipelines can be complex. They may have decision points or branches. They may need to be paused or restarted. Modeling the real-world process of application delivery requires that these pipelines be robust, flexible and powerful.
  • #20: The CloudBees Jenkins Platform also provides a dashboard that shows the current state of the pipeline activity, such as this example, which is critical to providing a full understanding of how things are performing.
  • #30: CloudBees Jenkins Platform Enterprise Edition: For Enterprise DevOps or Infrastructure teams that want to: Run Jenkins operations @scale to support a large number of developers and/or teams Optimize the performance and efficiency of the CD platform Monitor usage and share resources across their Jenkins implementation Have access to Jenkins support experts for quick problem resolution Ships with All Advanced Enterprise Features for both: CloudBees Jenkins Enterprise CloudBees Jenkins Operations Center CloudBees Jenkins Platform Team Edition: For small Dev teams or organizations with a limited number of developers that want to: Improve developer productivity by leveraging advanced developer features for Jenkins Easily get started with continuous integration (CI) and continuous delivery (CD) Have access to Jenkins Support experts for problem resolution Ships with a starter-pack of Enterprise Features focused on Developer Productivity: Developer Productivity Improved developer feedback loop and eliminate downtime with Validated Merge for Git and GitHub Pull Request Team Management Organize teams with folders Recoverability from failures using backup Security Roles-based access control Isolate team-sensitive slaves Analytics Build and performance dashboards Support Module Faster diagnosis for production issues