How to Reliably Measure and Optimize Graphics Performance of Your Android Games
Download as PPTX, PDF3 likes2,406 views
The document outlines a webinar focused on measuring and optimizing the graphics performance of Android games. It includes statistics on mobile game testing, the importance of real devices for accurate user experience, and various types of game testing methodologies. Additionally, it highlights the challenges of testing mobile games compared to regular applications, emphasizing the critical nature of performance in user experience.
How to Reliably Measure and Optimize Graphics Performance of Your Android Games
- 1. 11 August 2015 Ville-Veikko Helppi Technical Product Manager ville-veikko.helppi@bitbar.com How to Reliably Measure and Optimize Graphics Performance of Your Android Games W E B I N A R Sharif Sakr Director of Business Dev. sharif@gamebench.net
- 2. Agenda • Testdroid Statistics from Q2'2015 • Testing Mobile Games – Why and How? • Optimize Graphics Performance with Help of Gamebench - Introduction • Demonstration • Q&A
- 4. Q2'2015 – Test Run Stats
- 5. Why Mobile Games Fail?
- 6. iOS 8.0 iOS 8.0.2 iOS 8.1 iOS 8.1.1 iOS 8.2 iOS 8 ALL KitKat 4.4 KitKat 4.4.2 KitKat 4.4.3 KitKat 4.4.4 KitKat ALL Lollipop 5.0 Lollipop 5.0.1 Lollipop 5.0.2 Lollipop ALL Android 'M' 0 5 10 15 20 25 30 35 Failed test runs. Percentage (%).
- 10. Android Fragmentation 2015 by Open Signal report August 2015 Testdroid Cloud >95%
- 11. Android Fragmentation 2015 by Open Signal report August 2015 Testdroid Cloud + Gamebench >85%
- 12. How Many Devices is Enough? ~90% market coverage can be achieved with 128 devices ~20% market coverage can be achieved with 14 devices US Market 30 Android devices = ~2/3 market Global Market 80 Android devices = ~1/2 market
- 14. Correct behaviour across platforms and browsers Integration with web back- ends Typically need to fully utilize HW (CPU+GPU) Resource (e.g. battery) consumption OpenGL ES 2/3 Performance! Functionality and usability Screen orientations, connectivity, user profiles Robustness Robustness and security! Brand Compliances, verification with back-ends and data Different 'Verticals'
- 15. Manual Testing is not Agile
- 16. Test Automation is Agile
- 17. OS versions Chipsets CPU + GPU Tens of OEMs Memory Displays (resolutions, physical hw) OEM mods Other hardware (connectivity calibration) Relation to other software Where Test Automation Can Help
- 18. Why Real Devices Are Must-to-Have • Emulators cannot help you to test... • User Experience • Usability • Hardware • Software • Infrastructure 0 % = the percentage of your app users that use emulator to run your app!
- 19. Why Testing Games Matter? • Graphics performance capability is the most important thing for mobile games when forming UX • Too many games fail to work properly with low-end/mid- range devices due hardware performance • The real understanding of game's performance capability can be only figured out by testing on real devices Understanding How quickly gamer assimilates what the game is about, how to fully play with it, what are the functionalities. Usability The main User Interface, menus, design and how usable those are Playing Experience Intuition to follow the game and go forward with its instructions and game-play Monetization Are gamers willing to buy – invest – in game
- 20. Types of Game Testing • Functionality of the game-play, featuresFunctional Testing • Done always when new features /regressions are includedRegression Testing • How game runs on different configurationsCompatibility Testing • Different languages, geo-focused materialsLocalization Testing • Endurance test to determine if system can handle the loadSoak Testing • Measures the capacity of the systemStress Testing • Simplest form of performance testing, measures how system handles certain loadsLoad Testing • Isolation of the environment (e.g. from network) to see how game worksHermetic Testing Feature -based testing Performance testing End-user testing
- 21. Gameflow - Performance • Compared to regular app testing, mobile game testing may seem to be more challenging • e.g. OpenGL ES content not action-identifiable • Graphics and UI – Identical look&feel across all devices and variants • Usability controls in the context of game • Navigation/game-play, Progress, Delays between "game-stages" • Animations, Fonts, Other graphics assets • Screen Resolution and Orientation
- 22. • Studies: Majority of users leave game due sluggish, general slowness or other bad performance behavior • It's very important to understand how game behaves on different devices, under different circumstances Performance Makes +/- UX
- 23. Performance Testing • Performance, Load, Stress, Spike, Soak • Measure the real user experience • Execute tests with all possible devices (SW&HW) • Discover the impact of server loads, app performance (CPU/GPU utilization)
- 24. Components in Game Testing App Middleware Platform Connectivity/Network Back-end GAME Game Engine Android iOS ... API & services Server Database
- 25. Starts at Design Level App Middleware Platform Connectivity/Network Back-end GAME Game Engine Android iOS ... API & services Server Database Game Design, GFX Assets, Files, Optimization
- 26. Performance Testing w/HW App Middleware Platform Connectivity/Network Back-end GAME Game Engine Android iOS ... API & services Server Database Real Devices and Platform
- 27. Compatibility Testing App Middleware Platform Connectivity/Network Back-end GAME Game Engine Android iOS ... API & services Server Database Variations
- 28. Reference Testing App Middleware Platform Connectivity/Network Back-end GAME Game Engine Android iOS ... API & services Server Database
- 29. Back-End Performance App Middleware Platform Connectivity/Network Back-end GAME Game Engine Android iOS ... API & services Server Database Game with Back-End Integration / Google Play Services
- 31. Trends in Mobile Game Testing • Image recognition 15 seconds
- 32. GAMEBENCH
- 33. A traffic light system for mobile apps and devices
- 34. 50%
- 40. OEM and Chip makers Developers – Games, Apps, VR Journalists Testing / QA 40k+ downloads in less than 1 yr since launch, >250 users any day Market traction
- 41. App and Game Developers Chip makers OEM Consumers Journalists and Enthusiasts Used to certify performance before release Used by retailers to match consumers’ usage profiles to mobile devices Helps with device and app performance reviews
- 43. Who are the people behind GameBench •Sri Kannan Iyer: CEO & Founder (London) •Over 15 years experience in the mobile •ARM, MediaTek, Analog Devices and Intel •MSEE (UT Austin), MBA (Oxford) •4 patents, 1 book chapter, speaker at several conferences •Karthik Hariharakrishnan: CTO & Founder (Bristol) •Over 10 years in mobile engineering •ARM, Motorola, National Instruments •MSEE (University of Illinois), 3 patents Six employees based in Bristol & London Get in touch!
- 45. Demonstration Gamebench with Real Devices from Testdroid Cloud
- 47. THANK YOU! More information about frameworks, devices and testing at www.testdroid.com