Mobility testing day_2_3_ppt
- 1. Mobile App Testing Day 2 & 3
- 2. Agenda • How to write optimal test cases for Mobility testing? • Test Plan – Mobile App Testing • Mobility Test Methodology o App Install – Uninstall Testing o Build Acceptance/Sanity Testing o Crash Discovery Testing o Device Interruption Testing o Device Orientation Testing o Device Resolution Testing o Network Connectivity Testing o UI-UX Testing o Splash Screen Testing o Device Input Testing • Mobility Test Scenarios/Test Cases - FAQ’s o App Install – Uninstall Testing o Splash/Login/Registration Screen Testing o Build Acceptance/Sanity Testing o Crash Discovery Testing o Device Interruption Testing o Device Orientation Testing o Device Resolution Testing o Network Connectivity Testing o UI-UX Testing • Workshop – Reusable Test Cases in the Mobile Space • Introduction to App Store, Google Play and submission guidelines • List of Automation Tools • Mobile App Testing - Practicals
- 3. Optimized Mobile App Test Cases
- 4. Mobile App Test Techniques Mobility Testing UX/UI (Human Factors) Functional Crash Discovery Resolution Based Compliance OS Platform Coverage (Compatibility ) Device Orientation Response Time Installation/U ninstallation Network Connectivity Interruption
- 5. Test Plan - Mobile App Testing
- 6. App Install – Uninstall Testing – Test Methodology iOS - .ipa Installation ways • App Store • iTunes • Xcode • OTA/URL •Test Flight Andriod - .apk Installation ways •Play Store • Data cable • OTA/URL • apkinstall.com BB - .cod Installation ways • App World • Data cable • OTA Windows Phone - .xap Installation ways •Market Place •Data cable • OTA Symbian - .sis, .sisx Installation ways • Ovi Store • Data cable • OTA
- 7. App Install – Uninstall Testing – Test Methodology Considerations during Installation/Un- Installation testing Successful installation (of New, Complete, Custom, Upgrade installation) without errors & with version check Memory space checking Installation/Un- Installation under low- battery/chargin g mode/ fluctuating network Installation/Un- Installation under different Interruption ex: incoming call, receiving call Battery consumption Timeline taken for the process of Installation/Un- Installation Effect on other application Clean un- installation without any files left behind Reboot of the device should be done manually after Installation/Un- Installation Installation over older versions Un-usual application performance
- 8. Splash Screen Testing – Test Methodology SPLASH SCREEN TESTING Time Period of Display Orientation Functionality Branding/Logo Network Factors UI/UX App Minimize/Closed/Resto red Interruption
- 9. Build Acceptance/Smoke Testing – Test Methodology • Unit Testing Report • Unit Testing Status • Unit Testing Approval • Unit Testing Test Data • Unit Testing Validation UNIT TESTING • New Functionality/CR's • Bug Fixes/Status • Known Issues • BRM tool usage • Additional Information INTERNAL RELEASE NOTES • Verify Build versioning • App Install/Uninstall • Successful login to App • Build Acceptance Test Cases Execution n Status BUILD ACCEPTANCE TEST CASES
- 10. App Crash Discovery Testing – Test Methodology Crash Discovery Testing Network Connectivity Factors • WiFi Secured/Unsecured • GPRS . EDGE • 2G 3G 4G • Hot Spot Tethering • VPN External Interruptions • Sleep Mode • Voice/Data/Texting • GPS ON/OFF • Camera ON/OFF • Audio/Video ON/OFF • Low Signal /Battery • Reminders/Alerts Multiple Platforms • iOS • Android • Blackberry • Windows Phone • Symbian • BREW Numerous OEM's • Samsung . HTC • Google Nexus . Sony • Motorola . LG • Apple iPhone/iPod/iPad • Others, On Demand Screen Resolutions • 480*800 . 540*960 • 480*854 . 240*320 • 320*480 . 320*240 • 480*360 . 360*400 • Others, On Demand Orientation Factors • Landscape • Portrait • Device Orientation Lock internally or externally Compatibility Factors • Software • Hardware • Input Method (Keypad, Touch, Voice) • Notifications ON/OFF • Airplane mode Stress Testing • Multi Tasking • Heavy Input • Upload/Download huge data • Memory at low Screen Tap Factors • Single Tap • Multiple/Double Taps • Multi Touch Taps • Multi Touch Double Taps • Pinch In/Out • Swipe Up/Down/Left/Right Telecom Carriers • Airtel • Vodafone • Verizon • AT & T • Others, On Demand
- 11. Device Interruption Testing – Test Methodology Device Interruption Testing External Interruptions • Device Power Cycle • Sleep Mode/Idle State • Device Lock/Unlock • Voice/Data/SMS/Push Message • GPS ON/OFF • Camera ON/OFF • Audio/Video ON/OFF • Low Signal /Battery/Memory • Reminders/Alerts • Batter Charger ON/OFF/Removal • Notifications • USB Cable ON/OFF Network Interruptions • WiFi Interruption • Cellular Data Interruption • VPN Interruption Orientation Interruption • Landscape to Portrait • Portrait to Landscape • Device Orientation Lock internally or externally
- 12. Device Orientation Testing – Test Methodology Device Orientation •Portrait to Landscape •Landscape to Portrait •Alpha, Beta, Gamma view •0, 90, -90, 180 Degree View Landscape Orientation •Device Lock/Unlock •App Lock/Unlock • Keypad ON/OFF •Split Keypad •Sensor Motion • Device Interruption • Single Pane view • Split Pane view Portrait Orientation •Device Lock/Unlock •App Lock/Unlock •Keypad ON/OFF •Split Keypad •Sensor Motion •Device Interruption • Single Pane view •Split Pane View
- 13. Device Resolution Testing – Test Methodology DEVICE RESOLUTION TESTING Screen Size(small, normal, large, Xlarge) Device Density (ldpi, mdpi, hdpi, xhdpi) UI/UX Image Quality Animation Text Alignment Font Consistenc y Auto resize behavior Size Density Low density (120), ldpi Medium density (160), mdpi High density (240), hdpi Extra high density (320), xhdpi Small screen Normal screen Large scren Extra Large screen
- 14. Network Connectivity Testing – Test Methodology Network Connectivity Testing WiFi •Secured •Unsecured •Weak Network Strength Cellular Data Internet •2G/GPRS •2.75G/EDGE •3G •4G Telecom Carriers •Airtel •Vodafone •Verizon •AT & T •Others, on demand Others •VPN •Hotspot •Tethering •Weak Network Strength Interruptions •WiFi enabled/disabled •GPRS enabled/disabled •EDGE enabled/disabled •3G enabled/disabled •GPRS to WiFi to 3G
- 15. Network Connectivity Testing – Test Methodology Network & Connectivity Wi-Fi •Wi-Fi enabled •Wi-Fi disabled •Weak network(Wi-Fi) strength RFID •Short Range •Long Range GPRS •GPRS enabled •GPRS disabled •Weak network( GPRS) strength Bluetooth •Bluetooth enabled •Bluetooth disabled NFC •NFC enabled •NFC disabled Switching •Wi-Fi/GPRS enabled •Wi-Fi & GPRS enabled •Wi-Fi & GPRS disabled
- 16. UI/UX Testing – Test Methodology UI/UX Testing Smooth Animations Wow Simplicity Correct Consistency (Retina/Non Retina Display) Familiarity/Accessi bility Obviousness Encouragement Satisfaction Availability Portrait/Landscape Versatility Personalization
- 17. Notifications Testing – Test Methodology NOTIFICATIONS TESTING ON @ APP LEVEL OFF @ APP LEVEL ON @ DEVICE LEVEL OFF @ DEVICE LEVEL LOGOUT APP BACKGROUNDED APP CLOSED INTERRUPTION + NOTIFICATION ANOTHER APP FOREGROUNDED NO NETWORK RESTART MOBILE LOGGED OUT USER NOTIFICATION BELL & COUNT NOTIFICATION COUNT ON THE APP ICON NAVIGATION TO THE NOTIFICATION SCREEN
- 18. Device Input Testing – Test Methodology DEVICE INPUT TESTING TOUCH SCREEN - KEYPAD SPLIT KEYPAD VOICE COPY+PASTE EXTERNAL DEVICE INPUT (BLUE-TOOTH, SCANNER, STYLUS) NON TOUCH SCREEN – PHYSICAL KEYPAD TRACK BALLS/QWERTY KEYPAD HEAVY INPUT UNACCEPTABLE FORMAT ACCEPTABLE FORMAT
- 19. App Install - Uninstall – Test Scenarios/Cases
- 20. Splash/Login/Registration Screen – Test Scenarios/Cases
- 21. Build Acceptance/Smoke – Test Scenarios/Cases
- 22. App Crash Discovery – Test Scenarios/Cases
- 23. App Crash Discovery – Insights
- 24. Device Interruption – Test Scenarios/Cases
- 25. Device Orientation – Test Scenarios/Cases
- 26. Device Resolution – Test Scenarios/Cases
- 27. Network Connectivity – Test Scenarios/Cases
- 28. UI/UX Testing – Test Scenarios/Cases
- 29. UI/UX Testing – Test Scenarios/Cases To keep controls as unobtrusive as possible for instance by fading them out if they are not used for a while. Make it possible for users to go back to a previous screen for instance by adding a back or cancel button The main function of the app should be apparent immediately. It should speak for itself. Use at most one action on the screen that is highlighted as the most likely for the user. (Example: in iOS a blue button represents the default or most likely action).
- 30. UI/UX Testing – Test Scenarios/Cases Minimize user actions by using a picker or a table view where users can select a certain choice over a data entry field where users have to type a choice In an app, the user should not be able to store files locally, outside the app sandbox. In an app, the user should not be exposed to the permissions of a specific file If there is a long list of data to scroll trough, provide a search option above the list.
- 31. UI/UX Testing – Test Scenarios/Cases If performance is slow, indicate a progress status icon (“Loading…”), preferably with specific message. In case of ‘live’ filtering of data while the user enters his search query, verify the performance. The appearance of buttons that perform standard actions are not altered in the app (for instance: refresh, organize, trash, Reply, back, etc.) Do not use standard buttons for other functions then that they are normally used for
- 32. UI/UX Testing – Test Scenarios/Cases The app should respond to all changes in device orientation, as per the design Tappable elements should be about 7x7 mm in size, using the pixel density of the target device you can calculate the amount of pixels (chapter documentation contains a link to different devices compared). Do not redefine gestures in your app that have a standard meaning (example: swiping from top to bottom enables the notification center) Requirement to login is delayed in the app as long as possible
- 33. UI/UX Testing – Test Scenarios/Cases If the app is stopped at an unexpected time, user data should be saved locally and available at start-up. Users should be warned of the consequences of deleting a document Keyboard adjusts to expected input (for instance numbers/letters when expected). Are inactive buttons clearly distinguished from active buttons?
- 34. Notifications Testing – Test Scenarios/Cases
- 35. Mobile App Test Plan
- 36. MOBILE TESTING – OVERALL METHODOLOGY PHASES OF MOBILE QUALITY ATTRIBUTES IN CONSIDERATION
- 37. iOS App Store – Submission Guidelines
- 38. iOS App Store – Rejection Criteria
- 39. Android Google Play Store – Submission Guidelines Before you publish your apps on Google Play and distribute them to users, you need to get the apps ready, test them, and prepare your promotional materials. Here is Google's official checklist to get your app ready for successful launch on Google Play: • Understand the Publishing Process • Understand Google Play Policies and Agreements • Test for Quality • Determine your App’s Content Rating • Determine Country Distribution • Confirm the App's Overall Size • Confirm the App's Platform and Screen Compatibility Ranges • Decide Whether your App will be Free or Priced • Consider using In-app Billing • Set Prices for your Products • Start Localization • Prepare Promotional Graphics, Screenshots, and Videos • Build and Upload the Release-ready APK • Plan a Beta Release • Complete the Apps’ Store Listing • Use Google Play Badges and Links in your Promotional Campaigns • Final Checks and Publishing • Support Users after Launch
- 40. Android Google Play Store –Rejection Criteria
- 41. Submission Guidelines Apple App Store Google Play Store App Name 255 character limit No limit Description 4,000 characters limit 4,000 character limit Keywords Required, 100 character limit Category Primary required, secondary optional Required Application Type “Application” or “Game” Organization Name Required Required URL to Help Content Required Support Information Email required Email, Phone, or URL (you can provide more than one) Privacy Policy URL Optional Optional Marketing URL Optional Promo Text Optional. Accompanies promo graphic in featured spots
- 42. iOS Submission Guidelines – Mind map
- 43. App Submission Results Application Status At the top of your application’s profile page, you will find the status of your application submission. The following labels will explain the status of your application. Once your application status changes, an email will be sent informing you of this change and any issues or reasons for the change if applicable.
- 44. App Submission Results To Be Approved: Your application has not been reviewed or is currently being reviewed. Note that if you do not set your application to be “Published” (on the Edit node), it will not be reviewed. It may take up to 4 business days for your application status to change.
- 45. App Submission Results Approved: Your application has met our guidelines and is now live in our catalogue if you have set the application to be published. Once an application has been approved, it can take a variable amount of time before it appears on the SlideME website and SAM catalogs. Please wait at least 24 hours before submitting a support ticket regarding approved applications that are not appearing in the catalog.
- 46. App Submission Results To Be Reviewed: Your application is currently queued for an additional review to ensure the changes or updates you have made meet our guidelines or the editor’s requests. Changing the application title or changing/adding permissions will place your application back into the review queue. Note that you cannot make any changes to your application once it has been set "To Be Reviewed". If you wish to make any additional corrections while it is in the review queue, you can request your application be placed in Denied status by opening a support ticket and including the URL of your application. It may take up to 2 business days for your application status to change.
- 47. App Submission Results Denied: Your application review was completed, but your submission did not meet our guidelines. You can re-submit the app with the required changes.
- 48. App Submission Results Forbidden: Your application was rejected and is not permitted on app store. The reason for rejection is sent via email and may be due to our content guidelines or developer behavior. Forbidden applications are not open for appeal and should not be re- submitted again (even with a different packageName or under a different account) or your account may be flagged for spamming and/or result in loss of developer privileges. Once an application is Forbidden, we will no longer accept it no matter how many times it is re-submitted.
- 49. App Rejection Examples S.No Project Reason for App Store Rejections 1 SoCoSports Playbook External browser is launched while making payments by a playbook user Reason from Apple: Apps that link to external mechanisms for purchases or subscriptions to be used in the app, such as a “buy" button that goes to a web site to purchase a digital book, will be rejected 2 CA Clarity Private API’s are used which are outside Apple SDK 3 Digidata Leap drive The amount of data to be downloaded is more & the user has chosen to do so through GPRS/3G. As per the HIG, the app has to prompt the user with a message to switch to WiFi or any other faster mechanism 4 Expense Manager The app used standard logos from outside such as Google logo, Microsoft logo 5 Red Eye Keypad is not handled properly (appropriate keypad is not shown to the user i.e., when the user is expected to enter a mobile #, the keypad allows the user to enter alpha-special characters & vice versa behavior) 6 Seven Wonders Network connectivity problem i.e., the app doesn’t show appropriate message in the absence of network connectivity & it continues to search for the network and hangs the app after a while leading to a crash 7 eMDs The user is asked to BUY/PURCHASE (the app/a feature) from inside the app which is against the HIG rules 9 Furrer Jacot FJ app has very limited functionality & contains nothing other than jewellery images (this app has never made to app store) Reason from Apple: We found that your app provides a limited set of features and functionality, and focuses primarily on marketing or promoting your business, which is not in compliance with the App Store Review Guidelines. 9 Red Eye V1 The app we submitted to Apple doesn’t say that Apple is not a sponsor for the sweepstakes & contests. Hence, it is rejected by App store. Reason from Apple: The app was rejected as per the Official rules for sweepstakes and contests, must be presented in the app and make it clear that Apple is not a sponsor or involved in the activity in any manner.
- 50. List of Automation Tools
- 52. Thank You