Native automation tooling for mobile application testing.pptx
Download as PPTX, PDF0 likes197 views
The document discusses native and non-native mobile automation tools. Native tools like Espresso and XCUITest work directly within a mobile app's codebase and development framework, allowing direct access to app objects. Non-native tools like Appium and Calabash interact with apps through a server, relying on instrumentation libraries. Native tools have benefits like better performance and object handling since they are part of the app, while non-native tools allow black-box testing but with slower server-based interactions. The document advocates using native tools and building cross-functional teams for more efficient test automation.
Native automation tooling for mobile application testing.pptx
- 1. Mesut Güneş www.testrisk.com May 17, 2022 Ankara Testing Days 22 #atd22
- 2. Mesut Güneş www.testrisk.com Motto: Be part of the development team by native way! NATIVE AUTOMATION TOOLING FOR MOBILE APPS
- 3. Mesut Güneş www.testrisk.com MESUT GÜNEŞ Staff SDET in Mobile in/gunesmes gunesmes story/gunesmes gunesmes www.testrisk.com gunesmes@gmail.com
- 4. Mesut Güneş www.testrisk.com ■ Test automation ■ Mobile testing ■ Mobile automation tools ■ Native mobile automation tools ■ Benefits of native tooling TALK INCLUDE THESE TOPICS
- 5. Mesut Güneş www.testrisk.com TABLE OF CONTENTS MOBILE AUTOMATION ECOSYSTEM What we have for automation NON-NATIVE SOLUTIONS How non-native tools work 01 03 02 04 05 06 NATIVE SOLUTIONS How native tools work BENEFITS: HANDLING OBJECTS What things are handled differently BENEFITS: PERFORMANCE How we can maximize the benefits BENEFITS: TEAM BUILDING What kind of benefits we can have
- 7. Mesut Güneş www.testrisk.com Mobile applications are packages/binaries that can be installed to the users mobile devices so that you can let them do what you want. Mobile applications testing is installing the packages to the target devices and performing test steps through the application in the devices. Automation means that all the thing required for manual testing are done by tools and scripts. So mobile applications automation requires the followings: ● Application packages (apk / app / ipa) ● Mobile devices/emulator/simulator (Android / iOS) ● Networks ● Permissions ● etc. WHAT IS MOBILE APPS AUTOMATION
- 8. Mesut Güneş www.testrisk.com What tests in what level we should test the apps. TEST TYPES AND LEVELS Test Level Test Types Functional Nonfunctional Unit unit tests load, stress, security, code coverage (metric) Integration unit integration tests, component integration tests, microservices testing, contract test load, stress, security System end-to-end test load, stress, security, reliability, maintainability, scalability Acceptance uat test, alpha - beta testing load, stress, security, usability, AB test
- 9. Mesut Güneş www.testrisk.com Identify the purpose of the automation to get a better result Automate everything that are possible and meaningful ● Classes/Methods ● Integrations ○ Unit ○ Component ○ System ● Requirements ○ Functional ○ Non-functional ● Processes ○ Development ○ Testing ○ Distributing WHAT TO AUTOMATE
- 10. Mesut Güneş www.testrisk.com Regarding the the end-to-end testing automation, it can be done by the tools that can interact to the element of the application inside the devices. Basically we can classify these tooling as follows ● Native toolings ○ Available inside the development framework ● Non-native tooling ○ External tools that make automation possible HOW TO AUTOMATE
- 12. Mesut Güneş www.testrisk.com Non-native mobile automation tools are external tools and interact with the mobile application object by a server connected to the mobile apps that are running inside a mobile device/simulator/emulator. Non-native tooling requires a server to translate automation commands to actions on apps for black-box testing ● Depends on server-client communication ● Depends on some native libraries that are always upgraded by manufacturer ● Totally black-box testing technique WHAT IS NON-NATIVE TOOLING
- 13. Mesut Güneş www.testrisk.com Non-native toolings work with http server to interact with mobile objects Interaction can be done by two ways ● Interact with instrumentation/UI library ○ Espresso / UIAutomator / UIAutomator2 ○ Robotium ○ XCUITests / UIAutomation ● Image recognition ○ Server take screenshot and find the relevant part of the screen HOW NON-NATIVE TOOLS WORK
- 14. Mesut Güneş www.testrisk.com APPIUM Appium is one of the most popular mobile automation solution. Its popularity comes from the following advantages: ● Familiarity with the Selenium Webdriver ● Supports most of the popular languages ● Supports multiple platforms such as Android, iOS, Windows ● Easily extent one automation code base for all projects
- 15. Mesut Güneş www.testrisk.com APPIUM ARCHITECTURE Appium is a non-native automation tool whose drivers use native automation libraries in the http server. Image: https://digital.ai/catalyst-blog/what-is-appium-server-a-complete-end-to-end-guide
- 16. Mesut Güneş www.testrisk.com APPIUM CALL STEPS - iOS A simple command reaches to the test devices by passing the following steps ● You can write test in your favourite language ● Then the command turns into different languages in different protocols ● This journey is two-way ● Each steps are not as mature as desired so that causes FLAKINESS ● This is a sample for over-engineered work ● Finally it uses the XCUITest ● Risk of non-working feature until new feature implemented by Appium community Reference: https://www.headspin.io/blog/understanding-appium-drivers-and-how-to-choose-which-one-to-use
- 17. Mesut Güneş www.testrisk.com APPIUM WORKS Appium is a non-native automation tool whose drivers use native automation libraries in the http server. Image: https://digital.ai/catalyst-blog/what-is-appium-server-a-complete-end-to-end-guide Reference: https://www.headspin.io/blog/understanding-appium-drivers-and-how-to-choose-which-one-to-use
- 18. Mesut Güneş www.testrisk.com CALABASH Calabash is another mobile automation tool for black-box testing. It was very popular until 2016 but now it almost become history. Its popularity comes from the following advantages: ● Behaviour Driven test framework for Mobile automation ● Open-Sourced project: Calabash-android and Calabash-ios ● Supports iOS and Android and Hybrid Apps (Webviews) - (Cross-platform) ● Supports simulators / emulators / real devices ● Supports CI / CD and Clouds - (CLI tools) ● Ruby Based ● Integrated Cucumber ● Calabash Console for real time automation development
- 19. Mesut Güneş www.testrisk.com CALABASH ARCHITECTURE Calabash is also a non-native automation tool. Android version uses Robotium which is an external automation library and iOS version uses XCUITest that is a native framework. Calabash-Android Calabash-iOS
- 21. Mesut Güneş www.testrisk.com Native mobile automation toolings are integrated libraries to the mobile application development framework so that developers can write UI tests for the products under development. Native toolings are part of the development frameworks so it is guarantee that the tools are always up to date with the development frameworks and upcoming new features. Since we are talking about two major mobile device operating system, we are just focus on the following platforms: ● Android ○ Espresso ● iOS ○ XCUITest WHAT IS NATIVE TOOLING
- 22. Mesut Güneş www.testrisk.com HOW NATIVE TOOLS WORK Native tools works natively, which means that they are providing direct methods via APIs to use inside the development project globally. Interactions are made by directly calling the UI objects inside project architecture by the APIs. So these are the main benefits: ● APIs know the state of the objects ● We are free to use tool as black-box as well as contributing the development code as white-box ● We are free to reach the UI objects ● If the app is the Matrix, native tools are the agents inside the matrix
- 23. Mesut Güneş www.testrisk.com ESPRESSO - WHAT Espresso is a test automation tool developed by Google for Android UI Tests. Espresso is open-source, very stable and there many frameworks have been developed around Espresso. ● It has full feature-set of Junit that works with Espresso ● Android studio provides everything you need ● It provides white-box testing
- 25. Mesut Güneş www.testrisk.com XCUITEST - WHAT XCUITest stands for Xcode User Interface Testing and it is UI automation framework iOS applications. XCUITest is developed on top of XCTest and Accessibility. It is launch by Apple in 2015. It supports all iOS functionality in terms of testing them by white-box testing techniques. ● XCUITest is native and inside the app source code with different target ● XCode provides everything you need ● It provides white-box testing ● XCTest provides everything you need for testing capabilities ● Accessibility provides UI interaction for all kind of XCode product
- 27. Mesut Güneş www.testrisk.com BENEFITS: HANDLING OBJECTS 04
- 28. Mesut Güneş www.testrisk.com Native toolings are inside the projects so no need to worry about the use external tool to get the related objects Espresso and XCUITest provides the following benefits ● APIs provide methods for all kind of object ● Methods are always up-to-date with latest improvements/updates ● Freely update the source code for the sake of ui-test NATIVELY AVAILABLE OBJECTS
- 29. Mesut Güneş www.testrisk.com IDEs ARE FRIENDLY
- 31. Mesut Güneş www.testrisk.com The performance is very important when you grow and have to handle hounderents tests. The main point for the performance is that how much time your tests require for running them locally and in a cloud services ● Simple UI frameworks interact with the objects so no need to translate with servers ● Native tooling know the state of the elements so no need to check them regularly ● Automation code is inside the test package so no need to handle it differently while testing on the cloud. This reduce the time drastically SERVERS --- NO
- 32. Mesut Güneş www.testrisk.com Simple test to compare performance of the tools shows that native toolings are always better than Appium. This was done by TestProject engineers. Reference: https://blog.testproject.io/2020/04/08/fastest-mobile-automation-testing-tool-appium-vs- xcuitest-vs-testproject-vs-uiautomator-vs-espresso/ COMPARE THE PERFORMANCE
- 33. Mesut Güneş www.testrisk.com On cloud; Appium-client is in the CI located in Houston but Appium-server is in the cloud service located in San Jose so this is around 1700 miles COMPARE THE PERFORMANCE Reference: https://www.youtube.com/watch?v=lPr5PGtWc6Y
- 34. Mesut Güneş www.testrisk.com BENEFITS: TEAM BUILDING 06
- 35. Mesut Güneş www.testrisk.com Better performance from an automation project can be only get when the developers and testers work cooperatively. Developers are are always result oriented people so they focus on the development of the features, but the QAs/SDETs must be keen on the quality. This quality not only the product quality but also quality of the tests, process, and the documentation. This can be achieved by cross-functional-team building. Cross-functional-team requires the followings: ● Team spirit ● Shared common knowledge, every team members should know basics about ○ What is being built ○ How the things are developed ○ How the things are being tested ● Absence of one person from a role can be filled by from another role ○ QA/SDET can be developer ○ Developer can be QA/SDET BUILD TEAM AROUND QUALITY
- 36. Mesut Güneş www.testrisk.com How I am lucky to get those feedbacks from experienced mobile developers :) ● DEV provides testable features ● QA/SDET writes test ● When QA/SDET get familiarity with whole project, s/he can update/add missing parts for deterministic test ● Dev are ready for making test framework better :) BUILD TEAM AROUND QUALITY
- 37. Mesut Güneş www.testrisk.com CREDITS: This presentation template was created by Slidesgo, including icons by Flaticon, and infographics & images by Freepik. Please keep this slide for attribution. Does anyone have any questions? THANKS References: ● https://developer.android.com/training/testing/espresso ● https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual /testing_with_xcode/chapters/09-ui_testing.html ● https://appium.io/docs/en/about-appium/intro/ ● https://www.browserstack.com/guide/getting-started-xcuitest-framework ● https://www.browserstack.com/guide/appium-vs-espresso ● https://blog.testproject.io/2020/04/08/fastest-mobile-automation-testing-tool-appium- vs-xcuitest-vs-testproject-vs-uiautomator-vs-espresso/ ● https://devexperts.com/blog/ios-ui-testing-frameworks-performance-comparison/
Editor's Notes
- #9: We mainly focus on doing the functional tests on system and acceptance level but we can also check that the performance of the apps during this test by keeping all dependent parts (such as network, backend services, devices, test data, test steps) stable so that we can catch the deviation in the performance of the app.
- #16: A simple automation code creates a request with a json-wire protocol to the Appium server then Appium server gets the capabilities and creates a driver accordingly then starts to connect to devices/emulators/simulators to interact with the Bootstrap framework inside the target devices and finally the Bootstrap tries to complete the requests. Once the requests completed or failed for any reasons it sends back the log to the client so there is mutual communication during execution of the simple automation command. Appium created based on the Selenium Webdriver that was created for web automation so making this technology for mobile solution requires a lot of engineering. Appium includes lot of development to cover many edge cases and it will be also updated/refactored by upcoming updates in native automation frameworks, Espresso and XCUITest, that are updated every year by Google and Apple. As a result, Appium is an over-engineered tool.
- #18: Reference: https://www.headspin.io/blog/understanding-appium-drivers-and-how-to-choose-which-one-to-use
- #20: Same as in the Appium, a simple automation command that is initiated by step definition send requests to the server inside the devices so it can interact with the app UI objects. This interaction is provided by Robotium for android and by XCUITest for iOS. There is also a detail that the iOS server should be inside the build app which means that you need to rebuild iOS app for the Calabash target. The main differences between Appium and Calabash is that the server is inside the devices for the Calabash and you are sending query for the UI objects.
- #25: Talk about followings: What native means Where app code, UI code, unit test code How to apply white-box testing How Espresso is initiated How Espresso can be architectured POM
- #27: Talk about followings: What native means Where app code, UI code, unit test code How to apply white-box testing How Espresso is initiated How Espresso can be architectured POM