Game On: Automating Sports Video Game Testing
- 1. T24 Special Topics 5/8/2014 3:00:00 PM Game On: Automating Sports Video Game Testing Presented by: Fazeel Gareeboo EA Sports Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
- 2. Fazeel Gareeboo EA Sports A software development director at EA Sports Tiburon, Fazeel Gareeboo manages a team that provides automated testing for the studio. Before moving into management, he developed computer aided design (CAD) software and wrote device drivers for Windows and MicroStation, including the first 2048x2048 Windows device driver-for Bill Gates’ house. Fazeel has a special interest in automating any repetitive work and in creating great teams for the kinds of work that computers cannot do yet. He grew up on the island of Mauritius and worked in Europe before settling in the United States. Reach Fazeel at fgareeboo@ea.com.
- 3. 4/26/2014 1 Fazeel Gareeboo, EA Sports Sports Video games Game development Game stability during development Automated smoke testing Technology Q&A
- 4. 4/26/2014 2 Key differences from other software: ◦ Requirements Usually not ‘given’ Internally generated by game designers ◦ ‘Metric’ for success – Sales – hard to predict ◦ Platforms Consoles are closed Need licensing Fixed release date – tied to sports’ season Game subject to licensing authorities’ approval
- 5. 4/26/2014 3 Fixed release date Requirements TBD Multiple external approvals (consoles, licensors) Pick a set of features that would make a great game Develop and polish the features in time for the release date
- 6. 4/26/2014 4 Alpha • Feature Complete Beta • 0 bugs Final • Ship it ! ‘unknown’ number of defects + Fixed release date Hard to add more resources to project “Crunching”
- 7. 4/26/2014 5 Make sure the game is stable all the way during development “Stop the line” – Toyota way manufacturing ◦ Freeze the depot Automated testingAutomated testingAutomated testingAutomated testing is key because manual testing cannot keep up with the pace of changes
- 8. 4/26/2014 6 Given the existing challenges of developing the game, investing in automated testing does not rank high on the list Case for a stable game throughout development is received more favorably by those underwriting the game – company executives/publishers, … ◦ CostCostCostCost Invest a known amount in setting up Automated testing ◦ Benefits Lower development costs: Team not blocked during development by game instability No need to crunch during Alpha Lower risk to ship date: Game stable throughout development
- 9. 4/26/2014 7 Central team for Automated Smoke Testing Central technology created to support automated testing on consoles Game teams supported by central team, and we run set of smoke tests on every build of the game and provide results to the game teams New Game Built Copy to game console Run test scripts on game Report results to Dashboard
- 10. 4/26/2014 8 No, not really Having test results is not sufficient. Need effective policing of breaks… ◦ Individuals You checked something in that broke the game Did you not test this ? From now on, we need you to run all the tests before you check in (we have a dashboard to verify) ◦ Team Create and publish a stability policy “If we don’t have a full set of passes in 2 days, we freeze the depot”
- 11. 4/26/2014 9 Major pieces: ◦ Automation infrastructure Runtime component that is integrated into the game Server component ◦ Test scripts Easily edited Stored in game depot ◦ UI automation layer ‘Blind navigation’ leads to brittle scripts E.g. move to x,y then press ‘A’ Need to provide a way for the test script to talk to the UI layer What UI components are on screen ? Select component Button | ‘Ok’ Is UI ready to accept input ? ‘industry standards’ / ‘open source’ – not readily available, because consoles are closed systems Applies to : ◦ Automated testing technology ◦ UI automation layer
- 12. 4/26/2014 10 Automated testing of automated testing tech highly recommended Making the case for Automated smoke testing may be easier higher up the food-chain Having the test results is not enough. You have to drive through the culture of stability at the individual and team levels
- 13. 4/26/2014 11