Why a Mobile Test Strategy is just Test Strategy
0 likes985 views
The document discusses how a mobile test strategy should be part of an organization's overall test strategy, not a separate strategy. It notes the challenges of testing across different devices, operating systems, and platforms. However, the key mission of testing is still to validate products and report risk. A tester's role is to provide input into an organization's mobile strategy and understand where new technologies might be accessed. A mobile test strategy should consider requirements, environments, test phases, and risk analysis for native apps, hybrid apps, responsive web designs, and other platforms like TVs and cars. The strategy must evolve to holistically address usability, performance, security and more across all potential user experiences.
Why a Mobile Test Strategy is just Test Strategy
- 1. Why a Mobile Test Strategy is just Test Strategy 27 | 06 | 2015 HCMC Software Testing Club 2015
- 2. 2
- 3. 3
- 4. 4
- 5. Are we scared of Mobile? Challenges: • Creating separate teams • Managing multiple devices • Webservers don’t run low on battery get taken home • Browser, OS, Device, Platform combinations are more than we have ever had to manage But it still boils down our mission – Validate products, report risk 5
- 6. Our role is greater than a mobile test strategy Building a mobile approach is a key point in your overall strategy…. …But we must do more! Our mission, to report risk, must include a focus on where risk is introduced We must have a seat at the table: - What is the mobile strategy of the organization? - What will the next technology be? - Where will our content and products be accessed? 6
- 7. What must we understand 7
- 8. What must we understand 8
- 9. What must we understand • For public and consumer applications, mobile does not just mean an “app” • For internal, private applications, our users or still customers and usability can’t be taken for granted 9
- 10. There are two primary drivers to Mobile Quality 10
- 11. There are two primary drivers to Mobile Quality 11
- 12. But our apps aren’t really meant for mobile 12
- 13. What are we really testing? • Native apps – Yes • Hybrid apps – Yes • Mobile web – Yes • Responsive design – Yes • What about: - TV’s? - Cars? - Refrigerators!!! 13
- 14. What are we really testing? 14
- 15. What are we really testing? 15
- 16. What are we really testing? 16
- 17. What are we really testing? 17
- 18. But, I just want to make sure it renders We have a lot more to test than functionality and usability 18
- 19. But, we don’t have the devices 19
- 20. What does this mean for the tester? • Defect escape risk is exponentially higher with changes in platform and technology • Our challenges are increasing and so are our options • Working with product teams on strategy can manage the risk • Knowing that you’re mobile even when you’re not • Communicate cost of defects and customer loss • Building for future needs will strengthen non-mobile aspects of your strategy 20
- 21. Mobile is not only about mobile • Building for future mobile needs will strengthen non-mobile aspects of your strategy: - Usability is not a uniquely mobile trait - Customer adoption and community acceptance isn’t either - Size, performance, battery consumption, and connectivity options all affect functionality - Biometrics - Augmented reality 21
- 22. How my strategy should change for mobile • Clearly, new techniques are needed • The functional tester is no longer only able to focus on functionality! • But, we can still break this down into similar buckets: 1. Estimation 2. Requirements 3. Environment 4. Test Phases 5. Risk Analysis 22
- 23. Strategy – Estimating for Mobile The complexity of combinations is a significant drain on test estimation We must focus on production statistics (if Available) or expected usage to drive initial, risk based estimations Permutations of device, OS, Browser will create distraction – These are important, but must be based on target usage AUT loading, charging, OS selection, and setup are also factors 23
- 24. Strategy – Managing Mobile Requirements This is a change in the role of the tester: • Functional requirements will not address the infrastructure and technical needs of a mobile application or web experience • We must create best practices for product owners and architects to follow that drive quality solutions • Testing must become the voice of the mobile strategy 24
- 25. Strategy – Managing Mobile Requirements Functional Requirements: • Usability, accessibility, notifications and interactions with OS Infrastructure Requirements: • Wi-Fi vs. Network, lost transmission, security, and on device storage Consumption requirements: • Battery, data, bandwidth Input requirements: • Camera, biometrics, keyboard types Error Handling: • Call interruption, multi-tasking, background usage 25
- 26. Strategy – Managing Mobile Environments Traditionally, the test environment is managed by the infrastructure or technical team But with mobile, who do we “request” an environment from? And what constitutes a mobile environment? It is now the tester’s job to load OS, AUT, prepare the environment, and ready for testing 26
- 27. Strategy – Managing Mobile Test Phases Traditional webdesktop testing has included a standard approach that usually consists of: Functional, System, Integration, NFR, Regression and UAT (along with many other phases) Often, with mobile, we are forced to only consider functional (does it render) when we must also include: • Usability • Performance • Security 27
- 28. Strategy – Managing Mobile QualityRisk Analysis Between nativehybrid apps, “m.”, responsive design, and mobile web, our risk reporting must be focused on financial impact, customer adoption, and usability Risk reporting now includes intended platformOSDevicebrowser combinations Our roles in this process will be continuing to change but we must define risk criteria as part of our strategy from the onset 28
- 29. Summary - Quick Hits • Mobile Strategy is as much a part of test strategy as regression, especially for customerpublic portals • Responsive design must be implemented in any external website and can be tested outside of the device • Test strategy must evolve to include influence to design and development based on intended use • Risk mitigation includes the identification of core aspects of product usage • Mobile is not just for phones and tablets! 29