Battlestorm Game Evaluation Summary
1 like1,072 views
The evaluation found that the Battlestorm social impact game in Biloxi, MS had mixed results in increasing hurricane preparedness. [1] The game engaged youth and sparked conversations about preparedness between players and their families, but had limited success in directly improving players' knowledge. [2] Parents of players did become more aware and take some preparedness actions after learning from their children. [3] The evaluation provided recommendations to strengthen community partnerships, game design, and implementation for replicating similar preparedness games in other locations.
Battlestorm Game Evaluation Summary
- 1. Summary of Evaluation Findings Evaluation conducted by: Cause Communications, Network Impact
- 3. Games as a Tool for Engagement Playing games can bring together fundamental aspects of psychology, sociology and technology to engage people for social change. 72% of American households play computer and video games. Have seen the rise of games in popular culture with video games and reality television competitions Games engage the public’s imagination
- 4. Why fund social impact games? Knight Foundation aims to increase the ability of individuals to engage in change Seeking innovative ways to make that happen Research on digital games shows they can improve learning and influence behavior Knight wanted to see if the same was true for games that took place in the real-world, with real- people, in real-time
- 5. Why fund social impact games? Funded two pilot projects: – Battlestorm in Biloxi, MS, – Macon Money in Macon, GA Macon Money – Knight partnered with local organizations and residents on a game that fosters new connections between people and with local businesses by creating a local currency to spend in participating stores and restaurants.
- 6. Macon Money Connecting residents and spurring economic growth
- 7. Goal: Engaging youth and their families in hurricane preparedness with activities focused on youth as leaders.
- 8. A new way to engage kids around hurricane preparedness and get them interested – Give them a new language to use to talk about hurricanes Influence the behavior of parents and community Empower youth to act safely and take responsibility during dangerous storms and their aftermath
- 9. Why the Mississippi Gulf Coast? In the Gulf Coast Region – levels of preparedness had not increased in past 5 years Could tap into Boys and Girls Clubs’ afterschool programming Committed to recovery and resiliency through previous grantmaking efforts – Gulf Coast Community Foundation, Habitat for Humanity, Knight Nonprofit Center
- 10. Partners and Cities IP Center Forest Heights East Biloxi Qatar Center Hancock County
- 11. What can be learned about potential uses of real world social games? How did the game operate as a tool for community engagement? How can this game be used as a template for other communities or issues?
- 12. Advisory Board: Consulted on Game from Soup to Nuts James Paul Gee, Arizona State University, Digital Literacies, Situational Learning Beth Kolko, Univ of Washington , Berkman Center, Digital Games Research Group Tracy Fullerton, USC, Electronic Arts Games Innovation Lab, Ben Stokes, USC, Co-founder, Games for Change
- 13. New game: combination of dodgeball and freeze tag 493 players total in 5 communities on Gulf Coast 10 - 14 year-old boys and girls, after school at Boys and Girls Clubs March – May, 2011
- 14. Iterative Approach to Game Design
- 15. Iterative Approach to Game Design Tap into community character Intensive development process for designing the game that started by understanding the community Fun is paramount! Kids have to want to play the game Tap into existing local networks for players, team support and big event Make hurricane prep a public, social effort.
- 16. How to Play Youth from different neighborhoods play on teams representing the “Town” against Navy See Bees representing the “Hurricane” Town teams can earn tokens as people in their community upload photos of hurricane prep kits to the game website Tokens can be cashed in to buy extra powers like “Shelter” (which goves teams a safe zone on the court) or “Flashlight” (which removes a Hurricane player) Months of practice culminate in a tournament
- 18. Community Involvement Big Event tournament -- youth played against Team Hurricane of Navy Seabees Photos of family’s and friend’s prep kits uploaded to game website give “power boost” to Attendees took home a prep kit from watching the tournament and the winning team took an additional 500 kits home to their neighborhood to hand out
- 19. How it worked: 4 Potential Pathways for Change and Impact
- 20. Pathways for Change Players would learned about hurricane prep from game which would also prepare them for future learning Game would leverage youth as “superconductors” for information for their families and communities Parents of players would become more knowledgeable and prepared Game would trigger conversation, catharsis, healing
- 21. Pathway 1: Knowledge Gain and Prep for Future Learning Least effective results (although important to note that game was not designed around learning or knowledge gain) Surveys confirmed that players learned little about hurricane characteristics or behavior Game did not prime kids to learn about hurricane prep more easily in the future Knowledge gains limited
- 22. Knowledge Gain and Prep for Future Learning Three exceptions… 1. Linking game strategy to specific concepts was connected to learning retention 2. Small but significant increase in knowledge of hurricane prep kits items 3. Higher quality of questions asked about hurricanes and hurricane prep post-play
- 23. Pathway 2: Youth as “Superconductors” of Information
- 24. Youth as “Superconductors” of Information Graphic Slide here
- 25. Youth as “Superconductors” of Information Research has shown that conversations can lead directly to action, which is why this potential pathway is so important. Game was a trigger for conversation and preparation (disaster prep needs a trigger)
- 26. Youth as “Superconductors” – Talking with Parents 68% of Battlestorm players started/continued talking with parents about hurricanes vs. only 38% in control group One third of parents reported learning something new about hurricane prep from their child who played the game
- 27. Youth as “Superconductors” – Talking with Parents
- 28. Youth as “Superconductors” – Talking with Parents Example: After a conversation with his daughter, one parent put their family’s important paperwork in plastic bags Example: One player interviewed his grandparents to make sure they had all of the materials they needed to make a prep kit Example: a girl from East Biloxi told her father that she was playing a game about hurricanes, which reminded him to update the family’s flood insurance
- 29. Youth as “Superconductors” – Sharing with Friends
- 30. Youth as “Superconductors” – Sharing with Friends After the game… – 44% of players had spoken with a friend about how to be prepared for a hurricane vs. only 23% at baseline – 40% of players had spoken with a friend about what goes into a hurricane kit vs. only 10% at baseline
- 31. Pathway 3: Parents More Knowledgeable and Prepared
- 32. Parents More Knowledgeable and Prepared 25% of parents learned something new from watching the game Families of players had more hurricane kit items after the game After the game… – 65% of parents had spoken with their kids about how to be prepared, what goes into a prep kit, and about having a family communication plan – 59% of parents had spoken with kids about a family evacuation plan
- 33. Pathway 4: Conversation as Catharsis
- 34. Conversation as Catharsis Talking about games allowed families to process experiences and talk about hurricanes in an empowered way – More than half of Battlstorm parents surveyed confirmed that their family was directly effected by Katrina – Of those, nearly 1 in 5 said their children continue to experience symptoms caused by the storm – 75% of players’ parents believe that games like Battlestorm can help children withemotional or psychological problems related to hurricanes
- 37. Challenges: Submit your Kit 33% of families received kits at Big Event rather than through self-preparation Kit assembly component not promoted adequately with instructors and families Digital divide did play a small role Additional training on how to take pictures of kits and how to upload photos to the website could have helped this component
- 38. Challenges: Community Partner Engagement
- 39. Challenges: Community Partner Engagement Partner contributions could have been more strategically integrated or coordinated Participation in the Big Event could have been leveraged more effectively in community/with partners Needed more explicit explanation of game’s goals and purpose with audiences such as instructors, parents, and partners
- 40. Challenges: Community Partner Engagement Positives: – Partners did see opportunity to promote preparedness AND physical activity at the same time – Partners saw value of engaging young people as levers for community change
- 41. Best Practices: Set Up Games require significant local staff who have both time and talent! Multi-stakeholder approach is key in game development and implementation (e.g., Game designers, foundation, community leaders, community partners) Don’t leave partnerships to chance - Intra- organizational partnerships need orchestration and coordination
- 42. Best Practices: Game Design Games can be a catalyst for awareness, attitudinal, and/or behavior change Balance fun and Education - Fun is paramount, but tie context and concepts explicitly not implicitly Link game strategies to learning outcomes Be thoughtful about conveying the social purpose of game to players and partners
- 44. Best Practices: Implementation Evaluators can play an important role in mid-course corrections; monitoring implementation is critical Strategic communication can be key to engaging players, partners and community in game and Big Event
- 45. Replication Recommendations Customize for other types of disaster preparedness; not “generic hazard” Create a discussion guide for instructors to bridge in-game and out-of-game experience Consider integrating into schools vs. after- school programs Improve “Submit Your Kit” mechanism with early outreach and greater visual clarity; game mechanic may still be valid
- 46. Replication Recommendations Improve Big Event coordination, logistics, and promotion; focus as much on spectator experience as on player experience to engage community Think about follow-up communications that can leverage increased learning preparedness from game – Could include a scavenger hunt element for kit creation that is done as part of team
- 47. Battlestorm In Your Community Documentation designed to help anyone interested in creating Battlestorm in their community can be found at this link http://battlestormgame.org/battlestorm-in-your- community/
- 48. More on Knight and Social Impact Games Please visit: Knightfoundation.org/games/
- 49. Evaluation and Assessment Methodology Pre- and post-game survey to players with a control group Post-game focus group with players Pre- and post-game survey to players’ parents/adults care- givers Participation observation of game practices and Big Event Exit survey to Big Event audience Baseline and follow-up interviews with community partners Interviews with game advisory board and Area / Code Surveys as part of a process to test the game’s effect on “preparation for learning”
- 50. Questions?