Masteral summative report
1 like144 views
This 3-sentence summary provides an overview of the key points from the document: The document presents a summative evaluation of the Technoquest (now called Innovation Station) exhibit at the Oregon Museum of Science and Industry (OMSI) that analyzed how visitors experienced the exhibit. The evaluation used exit interviews and targeted interviews to assess whether the exhibit achieved its goals of being engaging for all ages and genders, making technology personally meaningful, and providing educational opportunities. The findings showed that the exhibit was widely appealing and successful in connecting to visitors' everyday experiences of technology while also conveying educational content.
Masteral summative report
- 1. Technoquest (Innovation Station) Exhibit A Summative Evaluation Research People, Places & Design Research Jeff Hayward & Jolene Hart Northampton, Massachusetts August 2005
- 2. Technoquest and the evaluation process that led to this report were funded with the support of the National Science Foundation’s Informal Science Education program.
- 3. • This summative evaluation of the Technoquest project (the new exhibition name is Innovation Station) provides a thorough analysis of how visitor audiences experience this exhibition in relationship to the project’s objectives. Ideally, the conclusions will facilitate the planning team’s opportunity • to reflect on the project’s strengths and weaknesses, • to consider some directions for their ongoing commitment • to update and revise the exhibits, to apply any lessons here to future projects, and • to contribute to general knowledge about the effectiveness of exhibitions that promote informal science education.
- 4. six goals that guided this evaluation • The exhibits should make technology personally meaningful by connecting to visitors’ everyday experiences outside the museum. Visitors to Innovation Station will find topics to explore that they find interesting or familiar to them
- 5. • The exhibits should be engaging for people of all ages and with a variety of interests and abilities. There should be activities for everyone in the family to enjoy. • The exhibits should be equally engaging for male and female visitors. Both boys and girls should find activities that they enjoy and should feel comfortable in the exhibit space.
- 6. • The exhibits should provide opportunities for visitors to consider the trade-offs associated with technology. Visitors should understand that technology gives them more choices but these choices may have good and bad consequences. The exhibit space should feel inviting and fun to visitors.
- 7. OMSI’s administrative representative asked reasonable questions about whether the big move to transform Turbine Hall with the new technology exhibits was worth it. Did visitors perceive a difference and, if so, was that for the better or not? These practical questions also informed the selection of methods and questions for this study.
- 8. Research Methods This summative evaluation used a multi-method approach to investigate the experiences of children and adults, including these parts of the research strategy:
- 9. Method 1 • Exit interviews were conducted with 358 visitor groups leaving Turbine Hall. Some questions were answered by an adult in the group while other questions were directed at both adults and children in the target age range of 8-14. The purpose of this strategy was to explore visitors’ overall experience of the exhibition in terms of enjoyment, awareness of the change in Turbine Hall exhibits, feelings about the ambiance, and perceptions of interpretive messages.
- 10. Method 2 Photoboard interviews were conducted with 150 children (aged 8-14) as they were leaving Turbine Hall. The purpose of this strategy was to obtain more in-depth information about children’s use and perceptions of selected exhibits and to explore potential gender differences.
- 11. Method 3 Exhibit-focused interviews were conducted with 259 adults who stopped at one of 6 exhibits selected for in-depth study. The purpose of this strategy was to get systematic data about some exhibits that the planning team felt could be problematic or ineffective in some way (not well-used, hard to use, hard to understand, fun but not educational).
- 12. Summary of Findings • Many of this project’s goals and questions raised by the exhibit team were addressed by the summative evaluation. In the following summaries, the goal or challenge is stated first and then the findings are presented about visitors’ experiences.
- 13. Ambiance of the Space: • Although there are no systematic data from visitors about their perceptions of the old Tech Hall, it was most frequently described by OMSI staff (n=12, using a staff questionnaire) as “dark,” “electronic,” and “outdated.” One goal for the new Technoquest exhibits in Turbine Hall was to create a lighter, more colorful appearance and thereby signify a more whimsical and user-friendly image of technology.
- 14. • The results indicate that a large majority of repeat visitors noticed a change in Turbine Hall and said it was “for the better.” Visitors most often chose the words ‘fun’ and ‘exciting’ to describe the atmosphere of this hall (top two descriptors from a list of nine phrases). About half of the adults and children selected ‘friendly’ or ‘inviting’ to describe the feeling here (even though this isn’t a huge proportion, it seems positive for exhibits about technology and engineering). About half of the adults (but not many children) felt that the space was ‘loud’ (possibly reflecting the crowds and high level of activity in this hall). Overall, then, visitors’ perceptions of the ambiance of technology exhibits was clearly positive.
- 15. Broad Appeal • The exhibit team wanted to communicate with a broad audience about technology but realized that there may be gender issues involved–that technology may be less interesting to girls and women. The formative studies documented some differences between men and women in their comfort with technology (such results were correlated with occupation, with fewer women in technology-related jobs). So, the exhibit development and design process continuously focused on how the exhibits • would be perceived by women and girls.
- 16. • The findings very clearly show that this exhibition hall is equally appealing to women and men and to girls and boys. Both genders gave similar ratings of the overall experience, chose mostly the same words to describe the feeling of this hall, had similar patterns of use (of the sample of exhibits and areas we asked about), gave similar ratings to selected exhibits, and had similar opinions about the complexity of some exhibits. Meanwhile, the perception of gender differences continues to exist: girls as well as boys thought that girls in their class would be less interested in these exhibits than boys in their class.
- 17. “Doing” Technology – Engaging in the Process of Designing and Inventing • A goal of this exhibition was to broaden people’s impressions and understanding of technology, with an emphasis on the process of invention and design rather than on “high-tech” products. One of the formative studies for this project suggested that people are fairly informed about the process of inventing/designing, but they don’t necessarily connect these ideas with technology. Interviews with visitors in the frontend research showed that their definitions of technology tend to focus on electronic products such as computers, TVs, and cell phones.
- 18. • The results show that most visitors do not come away from this exhibition with a conscious perception of technology as a main theme. However, many people do recognize one or more of the process-words inherent in technology, selecting the words designing, inventing, or engineering from a list of possible themes. Further supporting this perception, a majority of children felt that they had designed or invented something in this Hall. The examples they gave included both Technoquest (Innovation Station) and Engineer It! exhibits, suggesting that it was appropriate to put these exhibits together in one space as they reinforce each other. Repeat visits appear to enhance the sense of inventing and designing things, perhaps as people discover some of the less-used exhibits that round out the experience or perhaps as they move beyond basic patterns of use (figuring out how to use an exhibit) to explore how to use an exhibit differently (becoming more conscious of their options or role in creating something).
- 19. Relevance to Everyday Life • The exhibit team was initially concerned that people think technology is high-tech stuff that’s mechanical, complicated, and outside of the range of everyday life. Therefore, another goal of this exhibition was to make technology personally meaningful by connecting to visitors’ everyday experiences. In their exposure to the variety of exhibits here (many of which are “low-tech”), it’s clear that visitors associate technology with a wide range of everyday things. Twothirds of the children felt that they had seen familiar things that relate to everyday life, giving examples such as computers, paper cups, airplanes, bottle rockets, toilets, bridges, balls, and light bulbs. Adults perceived some exhibits as being relevant to everyday life (Tech Choices, Oregon Inventors), while some other exhibits were not (Ball Room, Float Table, Program a Robot).
- 20. Educational Value of the Exhibits • The project team had concerns about the effectiveness of some of the new exhibits. Mini-studies of six exhibits (method 3) provide some systematic information about the educational value and visitors’ understanding of the interpretive content for the following exhibits:
- 21. Ball Room • Is anyone learning anything here or are people just throwing balls at each other? The findings suggest that it’s some of each. About half of adults, and somewhat fewer children, were able to reasonably explain the main point of this exhibit in terms of content or principles (learning about air pressure, vacuum, designing tubes for a purpose). This activity especially conveyed the ideas that “you can play around without caring which way works better” and “there’s more than one way to do things.”
- 22. Float Table • Do people understand this activity, or are they just having fun • floating things in the air with no particular purpose? Although only about half of reasonable explanation of the main point of the activity. This exhibit was • perceived as “encouraging creativity,” “showing there’s more than one way to do • things,” and “playing around without caring which way works better.” Nearly all • the adults who stopped here did participate in the activity (with their children).
- 23. Program a Robot: • Are people using this activity? do they like it and understand it? The results indicate that this is not a highly used exhibit: only 25% of the children recalled stopping here. The activity had moderate appeal among adults and children. However, of those who did use it, most adults (79%) had a reasonable understanding of the main point. Some people enjoyed the challenge while others got frustrated; some adults complained that the light level made it hard to see, while some children had difficulty telling left and right.
- 24. Technology Choices • Are people engaged by a serious quiz-type game on a computer monitor? Do they understand that there are a lot of ways that we make choices about technology in our lives? The findings indicate that this exhibit serves a smaller proportion of the visitors (only 20% of kids interviewed said they stopped here), but among the people who did use it, the results suggest that children liked this computer “game” more than adults. The main ideas are not entirely clear–about 50% of adults got a reasonable message.
- 25. Oregon Inventors Panels • Is it worthwhile to have a non-interactive exhibit that may not attract much use? Designers say that there are always requests for some type of text and graphics panels that are not interactive, and they wonder if these are worth doing. The findings do support the notion of lower use, at least among children: only 18% stopped there. However, among adults who looked at these panels, three-quarters indicated a reasonable understanding of the main point (the process of invention), and a similar proportion felt that it related to everyday life. Perhaps this is a successful supplemental exhibit in that it provides context and relevance.
- 26. Build an Aqueduct: • Do visitors understand this activity? Adults and children understood the point of this activity, although a substantial portion of adults suggested that it could be easier to operate or have better instructions (about 60% said they read the labels).
- 27. The Unique Challenge of the Space of Turbine Hall • The decision to blend new technology exhibits with the previous Engineer It! Exhibits in a larger space made sense. However, the planning team worried about the huge turbine in the middle of the space – that this would be an obstacle to people’s use of the entire space, seeing all the exhibits. The openness and size of the architectural space may have been a useful cue for people that there was more to see beyond the turbine, perhaps aided by the strategy of marking a pathway along the floor past the turbine. The findings indicate that most visitors did go beyond the turbine to the far end of the hall.