Is it working

Editor's Notes

• #2: Thank you for joining us this morning. This session poses the question we all want to answer about our mobile apps. Is it working? But in order to answer this question we must first answer a much more complex one:
• #3: How do we define success? Before going into that, I’d like to introduce our session panelists.
• #4: I am Matthew Fisher, president of Night Kitchen Interactive. Our panelists are LoicTallon, of Pocket Proof, Nancy Proctor, Head of Mobile Strategy & Initiatives at the Smithsonian, and Matthew Petrie, of Fusion Research and Analytics. Each of the three of our panelists brings a unique and compelling perspective to the questions of how and perhaps more importantly what do we measure in determining the success of museum mobile apps? Before we hear from our panelists, allow me to briefly provide my perspective on this question and why it led me to not only organize this session,
• #5: but to co-author a paper for the museums and the web conference last month entitled Rousing the Mobile Herd: Apps that Encourage Real Space Engagement.
• #6: I became interested in this topic last summer when, together with my team at Night Kitchen, we downloaded over 75 of the top museum apps in the app store in an effort to familiarize ourselves with the current state of the industry. My primary interest was in understanding how these apps fulfill what I see as a key strength that museums hold in the continuum of informal learning environments.
• #7: Museums and their peers are unique in that continuum, in that they inhabit the rare intersection of real space and social space. They provide direct access to material culture and natural phenomena on the one hand AND they support a wide range of social behaviors on the other. In short, they are what I call “real social”.
• #8: To tease out this concept of “real social”, we mapped social behaviors into four quadrants with anti-social on the left, social on the right, the virtual below and the real above. Those activities in the lower left quadrant, that are both anti-social and virtual, take the least advantage of the museum space. Imagine, for example, a tween sitting on a bench playing angry birds. Those activities in the upper right quadrant, that are both social and real, take the greatest advantage of the museum space. Imagine a family on a walk through the nature center, sharing thoughts and reflections as all five senses come alive with the experience.
• #9: But don’t take my word for it. As you probably know, there is a wellspring of research identifying social engagement as a key ingredient in effective museum experiences, as our paper explores in greater depth. I’ll simply reference two key points, one being that: “conversation is a primary mechanism of knowledge construction and distributed meaning-making”, and the other, thatgroups observe each other to learn, to understand exhibit interactions and model behavior.
• #10: To promote“real social” engagement, we considered the types of behaviors to be encouraged in the museum space, such as talking about the exhibit, playing a game in the space, engaging in facilitated discussions, participating in multi-person interactives, and photographing or observing others. Yet when we analyzed the apps that were available last summer, we found that the vast majority of them provided features that did not promote real social engagement. Instead, by and large the apps provided features that were anti-social and virtual, such as audio and media tours, single-person games and research, and features that fell into the social and virtual quadrant, such as social sharing and commenting.
• #11: In our paper we were not questioning the benefits that many of these apps provide. We would say that the majority of them effectively meet the goals of engaging visitors in the space, providing visitors with context and insights, and allowing visitors to share and engage with others online. But on a cautionary note, we would say that the more that museums define success in terms of app usage and social media metrics, the more we will design apps that encourage virtual, and often anti-social, behaviors, rather than “real social” behaviors.
• #12: In our paper we simply suggest that IF you agree that “real social” engagement is a key strength for museums, and if this sort of visitor engagement is central to your museum’s mission, then its important to design mobile apps with real social goals in mind, and measure the effectiveness of your apps in meeting those goals. There are several examples in our paper and available on slideshare that I will not go into here.
• #13: So, back to the question at hand: How do we define success?
• #14: There are many ways to do this, so many, in fact, that it is difficult to know where to begin. Our panel gathered a range of metrics and I have attempted to place them into categories, including usage analytics, visitor participation, critical assessment, museum goals, and the bottom line.
• #15: There are usage analytics, such as downloads, visitor tracking, and pre, during and post visit usage comparisons and comparisons of online and onsite visitors.
• #16: You can look at visitor participation, including in-app commenting and social sharing, contributed content when solicited, or even across the entire engagement pyramid.
• #17: There are also critical assessments, including press and social media reviews, the quality and quantity of reviews and ratings in the app stores, visitor surveys, and tracking against other apps.
• #18: There are also many museum goals to track against, including the desire to innovate and provide unique functionality, integrate with museum offerings and systems, reach new audiences and improve audience diversity, and meeting both onsite and beyond the walls requirements.
• #19: And finally there are some bottom line considerations, such as what is the cost per user, is it sustainable, and does mobile usage translate to ticket sales and contributions?