Mobile App Testing Strategy by RapidValue Solutions
7 likes3,516 views
The document outlines a comprehensive mobile application testing strategy aimed at ensuring quality service and customer satisfaction amidst the growing demand for mobile applications. It details various testing types, including functionality, usability, and performance testing, along with challenges and methodologies such as cloud and emulator testing. The main objective is to verify that applications perform as expected across diverse devices and network conditions, ultimately fostering a positive user experience.
Mobile App Testing Strategy by RapidValue Solutions
- 1. © RapidValue Solutions Mobile Application Testing Strategy
- 2. © RapidValue Solutions Mobile App Testing Strategy – Adopted to Deliver Quality Service and Provide Utmost Consumer Satisfaction with the Perfect App ! There has been an increase in the adoption of smartphones, tablets and several mobile devices with the passage of time. And it has led to an enormous growth of mobile applications in recent years. Mobile device is considered to be the primary medium of interaction for the customers and also, businesses worldwide. And mobile applications are, actually, driving the communication. People, generally, do not give much importance to mobile application testing because of its expensive nature. But it is very much essential to ensure that the consumers have a great experience, every time they use the application. The aim of mobile application testing should be to acquire knowledge about the quality of the service that you are offering. Does it work properly or not? Will it provide services as per the expectation of the customer? These questions need to be answered to ensure that the customer comes back to you, for your service again. Mobile testing is becoming more and more complicated and complex with each passing day. Strategies are invented and used to simplify the mobile application testing. This presentation primarily addresses, the strategy to be adopted in mobile app testing, the types of mobile app testing and the stages to be followed, before the application is set live.
- 3. © RapidValue Solutions The main objective of mobile application testing is to help customers to gain assurance that the software applications are performing and functioning as expected with the devices or with the several other applications of the vital players in the mobility ecosystem. Objectives of Mobile Application Testing Parameters Details Functionality UI responsiveness, Peak load performance, Operational times Performance Network type (Wi-Fi, 2G, 3G, 4G), Impact of connectivity issues Network Security of data in motion and data at rest Security Mobile platform compatibility, Device model compatibility, Backward compatibility with previous app version Compatibility Marketplace guidelines’ compliance for Apple and Google stores Conformance Compare the developed product with user stories Usability User Experience Installation and Provisioning Installation process, Un-installation process, User provisioning and de- provisioning Service Testing Check that the services are up and running and also measure the performance Resource Testing Battery performance, Memory usage, Local database growth and Garbage in memory, come under this process Load Testing on Mobile With heavy loads of images, videos and also loading the app with more and heavy functions, use third party application like blaze meter to generate load Localization With this, the QA team checks the application’s multilingual capability
- 4. © RapidValue Solutions Emulators Cloud Testing Solution Physical Device oAll applications can be deployed and tested on emulator without investing in mobile handset for various OS oEmulators are mostly available free, and we can also perform UI, Stress and performance testing on that o30-40% test oThe Mobile devices can be accessed through web interface i.e. the browser oApplication can be deployed, tested and managed oAutomation module is available and the solution is secure, if private cloud is used with no maintenance oUp to 100% test coverage can be achieved oReal device testing will give most realistic view of test results oAll possible types of testing activities can be performed including those, that are dependent on hardware Mobile Testing Strategy is an essential element to ensure the smooth functioning of any application. Emulators are considered to be vital testing platforms while Cloud testing solution is very much secure. Real Device Testing is said to yield the most realistic and practical results. Mobile Testing Strategy
- 5. © RapidValue Solutions •Different Form Factors •Wide Variety of devices from different manufacturers, incase of Android •Different versions of Mobile OS •Different Screen Density •Infrastructure and Connectivity •Mobile testing tool availability – still on nascent stage •Cloud testing platform solves some of these issues, however: oIt is very expensive oLock – in with the platform oScripts cannot be used outside of the platform oOCR based platform. Because of automation, it is time consuming Challenges in Mobile Testing Mobile applications are considered to be “game-changing” technologies, which have further led to the development of various unique and beneficial testing challenges. The growth of the application market has increased the customer’s expectation of obtaining quality service. Companies are also coming up with new apps in the marketplace. Hence, challenges of mobile testing are also increasing each day. Whether it is an enterprise app or a consumer app, companies are finding out ample opportunities to boost their revenues and reaching out to a vast range of consumers. This ambition leads to a greater development of mobile testing strategies which face certain challenges:
- 6. © RapidValue Solutions •iOS 7 and above •iPhone 4, iPhone 4s •iPhone 5, iPhone 5s, iPhone 5c •iPhone 6 •iPad Air, iPad2 •iPod touch 4th generation •iPod 5th generation •Android 4.x and above •Samsung galaxy s4, s5 •Samsung Note 3 •Nexus 5 •HTC desire 816,one •Moto X •Sony Xperia Z Popular Mobile Devices for Testing Applications A great application certainly needs a greater customer service solution. And it is recommended for the application to support all the iOS and Android devices. The following are some of the devices and versions, on which its good to test your application.
- 7. © RapidValue Solutions The following statistical chart gives a clear picture of the screen sizes and densities for Android: Screen Sizes and Densities for Android
- 8. © RapidValue Solutions Simulator Testing Device Testing (On Wi-Fi Networks + Network Simulation Tools) Device Testing (In-Network) •Supports only device dependent use cases like camera, accelerometer, GPS etc. Device Testing (In –Network) •Ideal during the development phase •Best and economical option for functionality testing Simulator Testing Device Testing ( Wi-Fi ) •Use proxy tool Charles proxy to monitor the load and performance of the application There are certain distinctive features of Simulator and Device Testing. They are summarized below: Simulator vs. Device Testing
- 9. © RapidValue Solutions Advantages of Cloud Testing: •Variety of plans available, such as hourly, monthly or yearly subscriptions •Test incoming calls and sms without having any cellphone connection •Wide variety of devices •Automated test execution, can be played as videos •Test can run on several devices, in parallel •Build integration The performance testing is conducted on the applications and services provided via cloud computing. Cloud testing ensures optimal performance and scalability under a wide variety of conditions. Cloud testing monitors and reports on the real-world user traffic conditions. Cloud Testing User interacts with the device through studio or web front provided by the cloud provider Server receives the request and delegates that to the actual device Device sends back the result and that would be displayed on the studio or web front
- 10. © RapidValue Solutions The Application testing commences with Unit Testing and then proceeds with Functional Testing, Regression Testing and UI Testing. This is further followed by Automation Testing, Performance & Load Testing and the final stage is the Security Testing. Application Testing Flow Unit Testing Functional Testing Regression Testing UI Testing Automation Testing Performance & Load Testing Security Testing Performed by developers before releasing the build QA team perform the functional testing based on the test cases that are created using user stories Regression testing is a suite of testing, that include smoke test and execution of regression test cases UI testing is a usability testing, focusing on the UI design of the application, carried out by UI & QA teams Once the development is in shape, all the regression test cases should be automated QA team should check the UI response, peak load, transaction time as part of the performance testing. This should be conducted after every 3 sprints Security testing should be considered for data security, authentication and authorization
- 11. © RapidValue Solutions App Store Play Store Terms & Conditions Functionality/Reliability Metadata/Ratings & Location Intellectual Property Rights UI conformance & Contents Spam & Placement Ads & System Interference Contents Personnel & Confidential Information Content Ratings Conformance testing is considered to be critical. Conformance is essential and crucial to determine whether the product or the service meets the specified standards. Validation of the App Store and provision of the Google Store Guidelines are the integral part of the effective process, which plays a pivotal role. Conformance – Validate App Store & Google Play Store Guidelines
- 12. © RapidValue Solutions •The network testing should be carried out in physical device by the QA team to ensure the network performance under various technologies like GSM and CDMA •Cloud testing should also be leveraged to test it in the real environment •Separate test case suite should be designed to test the various signal strength under tele- signals using 4G,3G,2G etc. and the Wi-Fi signals •Charles proxy, a third-party tool can be used to control the network bandwidth by creating device and Charles proxy operate under one network •Applications should be tested under multiple signal strength and quality performance under live environment is ensured The most important thing is to ensure smooth and seamless network performance which can be evaluated with the help of efficient Network Testing. Performance is evaluated under technologies like CDMA and GSM. The applications are tried and tested under live environment to ensure utmost customer satisfaction. Network Testing
- 13. © RapidValue Solutions •Usability testing should be carried out by the following testing: oValidation of gray scale wire frames oTesting the compatibility with all devices and browsers oUI testing that focuses on user acceptance oUI comparison testing with benchmarked applications & standards oReview testing before every UAT by customer •Usability test should be conducted by UI/UX and QA team Another important testing procedure is the Usability Testing which should be ideally conducted by UI/UX team and the QA team. It is essential to test the compatibility factor with the various devices and browsers. UI Comparison Testing is also needed to know whether the benchmarked App is in conformity with the existing standards. Usability
- 14. © RapidValue Solutions •SOAP UI is a free testing tool, based on open-source standards as XML, XPATH and GROOVY (Java). It is stable and robust too. •Using SOAP UI, tester are able to perform oWeb service regression testing oWeb service test automation oWeb service load testing •SOAP, REST service will be covered as a part of service and all the responses will be evaluated using tools like GET, POST etc. Service testing will enable the providers to evaluate the performance of the application and work on it, according to the test results. It is essential to deliver a comprehensive testing as a service platform. There has been a rapid growth in the demand for effective application testing tools and services that will leverage mobility services. Service testing is performed to deliver effective and high-quality applications. Service Testing
- 15. © RapidValue Solutions •Resource utilization like memory, CPU, etc. is measured while doing Performance Testing. •Set criteria for Application/Device performance, Server Performance and Network performance oQuadrant and Linpack is used for measuring performance oLint (come with Android SDK), static code analyzer is used for performance and optimization oOC Lint (IOS), Static code analyzer is used for performance and optimization oiOS instrumentation tool is used to find the leaks and memory foot print in iOS application Testing is, indeed, an important aspect of development. It is crucial and critical for both, creating new applications and modifying the existing ones. Utilization of resources is measured effectively while going through the process of Performance Testing. Linpack and Quadrant prove to be effective in measuring performance of the application. Resource Testing
- 16. © RapidValue Solutions There are two methods to perform load testing - Manual & Automation Automation Testing: oLoad Testing is trying to simulate the work load of various user scenarios to the applications with third Jmeter oTesting the load within a specific pagination by monkey testing Manual Testing: oThe application under test are loaded with large images (high resolution) and videos to evaluate the peak point where the application can handle the maximum load oLoad testing are performed under various signal bandwidth to understand the minimum network signal strength to run the application with standard usage Load Testing
- 17. © RapidValue Solutions Automation testing is the key testing strategy to automate Regression testing which include all functionalities: oAppium is a good tool for Automation testing. And automation should start after the development is in the right stage oAppium is an open source, cross-platform test automation tool for native, hybrid and mobile web apps. It is tested on simulators (iOS, FirefoxOS), emulators (Android), and real devices (iOS, Android, FirefoxOS) oAutomation testing should be a part of every sprint to reduce the effort of Regression testing and Smoke testing oAutomation frame should be designed after couple of sprints One of the best characteristic of Automation Testing is that it does not require any manual intervention while executing the particular test. The test has certain benefits like greater consistency and the reduced nature of the repetitive work. Automation testing can be relied upon to provide the ease of accessibility to information about the tests. Automation Testing
- 18. © RapidValue Solutions •Automation of regression suite can be incremental starting from 1st sprint •Automated regression suite to check the impact of latest sprint deliverables on the previous sprint(s) deliverables, for continuous integration •Re-use the same (with additional manualautomated tests) for hardening sprint before application is launched Regression Testing is another essential procedure which deals in rerunning test cases just to ensure that the software changes do not have any side-effects. Regression Testing has become more effective and efficient with time. This testing is critical to check or detect whether new issues have cropped up after the change has been applied. Regression Testing is critical for large applications as it is a little difficult to figure out whether any part of the application has given rise to any issue or problem due to the change. Regression Testing
- 19. © RapidValue Solutions •Supporting types: Text, Images, Videos etc. •Size: Maximum size supported for each category •Size: Beyond maximum supported size for each category to see the stability / Large volume of data and measure performance •Mix of different data types with maximum size •Negative Data (Invalid data): The application is not intented to support •Data and network (2G,3G,Wifi) combination •Data and Device type/OS combinations The following gives a detailed description of the data types, data sizes, blend of different data types and sizes, supporting formats etc. It also provides information regarding the data and device/network combinations. One obtains knowledge about negative data also, which is a particular data type or data size that the application does not support. Test Data – Type and Size
- 20. © RapidValue Solutions •The app diagnostics and crashes on iOS and Android can be monitored using Crittercism •Google Analytics tool helps to monitor the exit points and identify the reason. It collects user interaction data, manages how the data is processed and provides a report. The post-production stage is equally important as the pre-production and the production stage. A report has to be provided after collecting user-interaction data and managing the data procedure. Crittercism and Google Analytics provide immense support for monitoring the application. Post Production
- 21. © RapidValue Solutions •Hardening sprint(s) - QA only sprint(s) for Go Live •On actual environment and configurations •Regression suite to focus on system level, Integration, Impacted (changes) areas, performance and security •Real-time data, text, videos and images with maximum supporting size •Go Live check lists based on the sprint level confidence oEntry Criteria oList of devices and configurations oActual environment & Infrastructure, Test Data oRegression Suite and Use cases oInstallation/ Uninstallation oExit criteria Testing stage holds utmost importance. It gives the clear picture of the task that has been carried out with such great effort. Testing is done in actual environment and configurations to know whether the application supports it or not. Go Live – Testing
- 22. © RapidValue Solutions The growth of the application market has increased the customer’s expectation of obtaining quality service. Companies are also coming up with new applications in the marketplace. Hence, challenges of mobile testing are also increasing day-by-day. Mobile application testing is, indeed, an integral and essential part of the production process. It is a procedure by which the application software, that is developed for the mobile devices, is tested for its supportive feature, its functionality, usability and for its consistency. The main objective of mobile application testing is to help and assist customers to gain assurance that the software applications are performing and functioning as expected. Mobile testing strategy is an essential element to ensure the smooth functioning of any application. •The most important thing is to ensure smooth and seamless network performance which can be evaluated with the help of efficient Network Testing. •Service testing enables the providers to evaluate the performance of the application and work on it, according to the test results. •The performance testing is conducted on the applications and services provided via cloud computing. •The Usability testing is equally essential to test the compatibility factor with the various devices and browsers. •Service testing, Load testing, Regression testing and Automation Testing - all the testing procedures form the significant part of the application testing strategy which is crucial and vital for the production of an application. Conclusion
- 23. © RapidValue Solutions Thank You
- 24. © RapidValue Solutions About RapidValue RapidValue is a leading provider of end-to-end mobility solutions to enterprises worldwide. Armed with a team of 250+ experts in mobility consulting and application development, along with experience delivering over 400 mobility projects, we offer a range of mobility services across industry verticals. RapidValue delivers its services to the world’s top brands and Fortune 1000 companies, and has offices in the United States and India. www.rapidvaluesolutions.com www.rapidvaluesolutions.com/blog +1 877.690.4844 contactus@rapidvaluesolutions.com This document contains information that is confidential and proprietary to RapidValue Solutions Inc. No part of it may be used, circulated, quoted, or reproduced for distribution outside RapidValue. If you are not the intended recipient of this report, you are hereby notified that the use, circulation, quoting, or reproducing of this report is strictly prohibited and may be unlawful. Disclaimer: Presentation by Rinish KN CTO, RapidValue Solutions