Parallel Test Runs with Appium on Real Mobile Devices – Hands-on Webinar
Download as PPTX, PDF13 likes15,123 views
The document provides an overview of a webinar on parallel test runs with Appium on real mobile devices, covering updates on Testdroid, mobile app testing statistics for Q1 2015, and the significance of real device testing versus emulation. It highlights the various deployment options and pricing plans for Testdroid, details on Appium client-server architecture, and discusses the importance of test automation for agile methodologies. Additionally, the document outlines the functionality, frameworks, and a demonstration of Appium capabilities with real devices.
![Multiple Devices – Client Side
python testscript.py –device <devicename>1
python testscript.py –device <devicename>2
python testscript.py –device <devicename>n
Test Script
Test Cases
Instigator Script
deviceArray=
[
“iPad 4 6.0.1 A1458”,
“iPad mini 7.0.4 A1432”,
.
.
.
“iPhone 4S 6.1.3 A1387”,
]](https://image.slidesharecdn.com/paralleltestrunswithappiumonrealmobiledeviceshands-onwebinar-150513165345-lva1-app6891/85/Parallel-Test-Runs-with-Appium-on-Real-Mobile-Devices-Hands-on-Webinar-37-320.jpg)
Parallel Test Runs with Appium on Real Mobile Devices – Hands-on Webinar
- 1. 12 May 2015 Ville-Veikko Helppi Technical Product Manager ville-veikko.helppi@bitbar.com Parallel Test Runs with Appium on Real Mobile Devices W E B I N A R
- 2. Agenda • Testdroid Update • Statistics from Q1'2015 • Testing and Mobile Test Automation • Appium – Client vs. Server Side Execution • Demonstration • Q&A
- 4. Public Device Cloud on-demand devices (multitenant) Mobile app testing on thousands of real Android and iOS devices hosted by Bitbar Private Device Cloud Reserved Devices Hosted by Bitbar in the US and/or Europe Devices chosen by and reserved only for the Customer On-Premise Device Cloud Automated mobile app testing devices hosted by the customer, usually 30-500 devices 1 Product – 3 Deployment Options Testdroid Cloud Testdroid Enterprise Testdroid PrivateCloud
- 5. New Pricing Plans for Cloud *) = Prices for annual commitment
- 6. Appium • Client-Side Execution • Server-Side Execution
- 8. Q1'2015 – Test Run Stats
- 10. iOS 8.0 iOS 8.0.2 iOS 8.1 iOS 8.1.1 iOS 8.2 iOS 8 ALL KitKat 4.4 KitKat 4.4.2 KitKat 4.4.3 KitKat 4.4.4 KitKat ALL Lollipop 5.0 Lollipop 5.0.1 Lollipop 5.0.2 Lollipop ALL 0 5 10 15 20 25 30 35 Failed test runs. Percentage (%).
- 14. Android Fragmentation 2014 by Open Signal report July 2014
- 15. How Many Devices is Enough? ~90% market coverage can be achieved with 128 devices ~20% market coverage can be achieved with 12 devices US Market 25 Android devices = ~2/3 market Global Market 60 Android devices = ~1/2 market
- 16. TESTING AND MOBILE TEST AUTOMATION
- 17. OS versions Chipsets CPU + GPU Tens of OEMs Memory Displays (resolutions, physical hw) OEM mods Other hardware (connectivity calibration) Relation to other software Where Test Automation Can Help
- 18. Correct behaviour across platforms and browsers Integration with web back-ends Typically need to fully utilize HW (CPU+GPU) Resource (e.g. battery) consumption OpenGL ES 2/3 Functionality and usability Screen orientations, connectivity, user profiles Robustness Robustness and security! Brand Compliances, verification with back- ends and data Different Mobile 'App Verticals'
- 19. Manual Testing is not Agile
- 20. Test Automation is Agile
- 21. Why Real Devices Are Must-to-Have • Emulators cannot help you to test... • User Experience • Usability • Hardware • Software • Infrastructure 0 % = the percentage of your app users that use emulator to run your app!
- 22. Manual vs. Automation Manual Automation
- 23. Manual vs. Automation Smaller coverage, More money burnt & time wasted, Error-prone Manual Automation Large coverage, quickly completed, Less money & time wasted, Exact results.
- 24. APPIUM – CLIENT VS. SERVER SIDE EXECUTION
- 25. Family Tree of Android Test Automation Frameworks JUnit Android Instrumentation Framework Robotium Espresso and Espresso v2 uiautomator Appium ExtSolo Calabash
- 26. What Framework Works You The Best? Robotium uiautomator Espresso Appium Calabash Android Yes Yes Yes Yes Yes iOS No No No Yes Yes Mobile web Yes (Android) Limited to x.y clicks No Yes (Android & iOS) Yes (Android) Scripting Language Java Java Java Almost any Ruby Test creation tools Testdroid Recorder UI Automator viewer Hierarchy Viewer Appium.app CLI Supported API levels All 16 => 8, 10, 15- All All Community Contributors Google Google Active Pretty quiet
- 28. Client Side Appium at Testdroid CloudTest Script Test Case Desired Capabilities { “device”: “Android”, “app”: “/Users/user/ApiDemos.apk” “app-package”: “com.example.android.apis” “app-activity”: “.ApiDemos” }
- 29. Test Script Test Case Desired Capabilities WebDriver http://localhost:4723/wd/hub Appium Server 4723 Device Localhost (DesiredCaps)
- 30. http://localhost:4723/wd/hub *Testdroid Caps http://appium.testdroid.com/wd/hub (DesiredCaps) Test Script Test Case Desired Capabilities WebDriver From Localhost to Testdroid
- 31. Client Side Execution Go to cloud.testdroid.com
- 32. Client Side Execution Get a Device Name Go to cloud.testdroid.com
- 33. Client Side Execution Add Testdroid Desired Caps to test script { “testdroid_username”: “user@domain.com”, “testdroid_password”: “p4s$w0rd”, “testdroid_project”: “My First Project”, “testdroid_testrun”: “Test 1”, “testdroid_device”: “iPad Mini 7.0.4 A1432”, “testdroid_app”: “http://domain.com/app_v1.ipa” . . “app”: “com.bitbar.testdroid.BitbarIOSSample” } Get a Device Name Go to cloud.testdroid.com
- 34. Client Side Execution driver = webdriver.Remote("http://appium.testdroid.com/wd/hub", desired_caps); Point the Webdriver to http://appium.testdroid.com/wd/hub Add Testdroid Desired Caps to test script Get a Device Name Go to cloud.testdroid.com
- 35. Client Side Execution Run the Test ScriptGet Results from Testdroid Cloud Point the Webdriver to http://appium.testdroid.com/wd/hub Add Testdroid Desired Caps to test script Get a Device Name Go to cloud.testdroid.com
- 36. Client Side Execution Pull the Results from the Result URL driver = webdriver.Remote("http://appium.testdroid.com/wd/hub", desired_caps); Run the Test ScriptGet Results from Testdroid Cloud Point the Webdriver to http://appium.testdroid.com/wd/hub Add Testdroid Desired Caps to test script Get a Device Name Go to cloud.testdroid.com
- 37. Multiple Devices – Client Side python testscript.py –device <devicename>1 python testscript.py –device <devicename>2 python testscript.py –device <devicename>n Test Script Test Cases Instigator Script deviceArray= [ “iPad 4 6.0.1 A1458”, “iPad mini 7.0.4 A1432”, . . . “iPhone 4S 6.1.3 A1387”, ]
- 38. 21 3 4 5 6 7 WebDriver Session Request @http://appium.testdroid.com/wd/hub WebDriver Session Response Testrun Configure Project Appium Ready Wait for Device to Become Available Device 1 Device 2 Device 3 Session Map Proxy Device Cluster Start Appium Desired Capabilities, .apk / .ipa SessionID SessionID Sessionid Test Script Appium Broker Client Side – Behind the Scenes
- 39. Introducing "Server Side Appium"
- 41. Demonstration Appium with Real Devices using Image Recognition for Hill Climb Racing
- 42. Setup • Using real Android devices at Testdroid Cloud • Parallel test runs without a need to configure desired capabilities • Device groups (= set of devices used for runs) can be manually created and configured
- 43. File Structure • pom.xml (maven) • testdroid.properties (overwritten after submitted to Cloud) • run-test.sh (shell script for execution) • image files
- 48. 15 seconds
- 50. Image Recognition • Resolution agnostic implementation • Identifies stretched and rotated images as well
- 51. Let's See How it Works!
- 53. THANK YOU! More information about frameworks, devices and testing at www.testdroid.com