Midlands hub tech in evaluation workshop 26 feb2010
- 1. Dr Emma Gillaspy Vitae NW Hub Manager www.vitae.ac.uk/nwhub http://twitter.com/vitaenwhub http://vitaenwhub.posterous.com/
- 2. Objectives To introduce key applications of web 2.0 and other technologies in evaluating training programmes To explore how web 2.0 can be of benefit in your own environment To ease the administrative burden of evaluation To add value to your current evidence of impact To help influence key stakeholders
- 3. Awareness How aware are you about the range of technologies that could be incorporated into your training? Absolutely unaware OK but am I missing anything? Totally on top of it
- 4. Practice How comfortable do you feel incorporating technology in your training? Fairly comfortable I’d rather swim with hungry sharks They call me Dr Gadget
- 5. Tools Aside from PowerPoint have you used or considered using other technologies in your training? How about when evaluating your training?
- 6. What is web 2.0? The second generation of the World Wide Web, especially the movement away from static webpages to dynamic and shareable content and social networking. Web 2.0 websites allow users to do more than just retrieve information. Web 2.0 harnesses the collective intelligence
- 7. Digital Age Social bookmarking Wiki’s RSS Podcasting (sharing audio) Blogging Chat Slide sharing Searching
- 8. Using Google Docs Prequel questionnaires to gauge audience expectations and/or create a baseline Online ‘happy sheets’ Review changes in behaviour over time Review whole training programmes http://vitaenwhub.posterous.com/evaluation-questionnaire-for-creating-highly http://docs.google.com Can also use Survey Monkey (plus others)
- 9. Slideshare
- 10. The power of blogs “ The experience was quite enriching, I must say… I was happily surprised with the creativity of the team of facilitators, as well as the strategies and ideas they developed to constantly engage their difficult audience” “ Some ‘less serious’ activities could have easily come across as a ‘waste of time’. But I am happy to say that neither me (maybe especially me) nor my fellow students seemed to think it was not worth to spend those two days away from their desks doing something totally different” “ If you have the opportunity to attend GRADschool then I strongly recommend you do. Aside from improving your communication skills, meeting new and likeminded people, boosting your confidence and learning about yourself, the scheme was actually lots of fun and I guarantee you will walk away with a buzz. I am still unsure about what I would like to do post my PhD, but I now have a much greater self awareness and an understanding of what is important to me”
- 11. … and Twitter Advertising Using the #tag Increasing importance? Metrics http://mashable.com/guidebook/twitter/
- 13. Evaluation activities What methods do you currently use to evaluate your training? What evaluation would you like to do in an ideal world?
- 14. A matter of perspective…
- 15. Integrating technology Can technology help you collect evidence of the impact of your training? What are the pros and cons of using technology for evaluation?
- 16. Now what?
- 17. Want to explore more? Integrating technology into researcher training: A Vitae NW Hub good practice workshop 27 May 2010, Manchester, 10:30-15:30 Case studies A practical session Group discussions www.vitae.ac.uk/nwhub
- 18. Don’t use technology for technologies sake, use it only when it adds value!
Editor's Notes
- #2: I have a particular interest in the application of technology in researcher training.
- #3: The objectives of today’s session are… Are there any other burning issues you would like to cover in the next hour?
- #4: Just to gauge how to pitch this session I wanted to start with a couple of questions. Where do you think you fit on this scale?
- #5: And how comfortable are you with using tech in your training at the moment?
- #6: Have a quick chat in your groups about what technologies you have used in your training, in particular if you have used any in evaluating your training. Video in training e.g. presentation skills Online activities before or after a session Electronic rather than paper handouts Voting handsets
- #7: So for a bit of background, Web 2.0 is a phrase that covers all the internet technologies that incorporate more than just static text. This includes things like being able to add comments or content to a site and networking methods. Web 2.0 works on the principle that collective intelligence is better than individual intelligence. For example the online encyclopaedia britannica is web 1.0 but wikipedia is web 2.0.
- #8: Web 2.0 includes all of these technologies and more We can take advantage of these technologies in our training and evaluation of training
- #9: One example is using Google Docs to do some online evaluation. This could be to… An example of what an evaluation questionnaire can look like is… Would anyone like me to go through the basics of creating a questionnaire? There are plenty of other ways of evaluating online using Survey Monkey, poll daddy and others, just find which one works best for you.
- #10: Does anyone use Slideshare? It is excellent for sharing/distributing content and receive feedback from participants, peers and anyone else. It’s really easy to use and upload and you can post any uploads straight to your blog or Twitter here. One of my colleagues ran this workshop for researchers and posted the slides on Slideshare and linked it to their blog and twitter accounts. There were 35 participants at the session but the slides have now had over 1000 views.
- #11: Blogs can be used for advertising your training programmes but can also document some powerful comments by participants.
- #12: Twitter is a fantastic advertising tool for your training but it can be used to gather evidence as well. Encouraging participants to use a #tag at a large event allows them to post about it. You can also use the #tag for a resource for example #TED Twitter is becoming an increasingly important way of distributing and receiving information and making contacts. Recently someone was offered an internship when they Tweeted they were looking for one. You can also gather metrics on Twitter. As well as knowing who follows you, you can also tell things like how many people have clicked on a link in your tweets. There’s a great guide to using Twitter here.
- #13: Videos can provide valuable evidence of personal experience on courses. This can be done easily using mobile phones or you could enlist help from others in your department. This is a video made about last year’s Manchester GRADschool which was opened up to researchers around the region. I wont play the whole video but will skip to a couple of comments by participants at the end of the course. (9min 43)
- #14: Right so now its over to you. First I want you to identify all the methods you currently use to evaluate your training. Now think about what evaluation you would like to do if there were no time or resource restrictions Depending on time, get them to categorise activities onto RTIF
- #15: Now I want to your to think about things from a slightly different perspective. Who are your stakeholders and who do you need to influence? Now imagine you are that stakeholder What do you want from researcher development training? What would influence you?
- #16: Now think about whether technology can help you collect evidence of training that will influence the stakeholders we have identified. What are the pros and cons of using technology to collect the evidence? Depending on time, get them to categorise activities onto RTIF Examples: Online happy sheet: Pros - less admin time analysing and more considered answers Cons – might not get everyone completing it
- #17: So what are you going to take away from this session? Have you got any easy wins that you can go back to the office and implement straight away? Are there actions you might want to do longer term to help evidence your programmes?
- #18: A bit of shameless marketing but if you want to explore more…
- #19: My parting message is don’t……