Right Things Right: ZEF Evaluation
Download as PPT, PDF0 likes335 views
The document discusses Pekka Silvén's background and work in evaluation, quality and feedback processes. It notes some common problems with traditional evaluation methods and introduces the "Zef-method" as an alternative that provides clearer results and facilitates improvements. The Zef-method is used in various feedback questionnaires and evaluation engines to analyze responses and compare views of different stakeholders.
![Pekka Silvén Director Professional Development Services Oulu University of Applied Sciences [email_address] +358 10 27 23195](https://image.slidesharecdn.com/saskatoon2010-finalb-100318162434-phpapp01/85/Right-Things-Right-ZEF-Evaluation-39-320.jpg)
Right Things Right: ZEF Evaluation
- 1. Pekka Silvén Director Professional Development Services Evaluation, Quality and Feedback
- 2. Pekka’s ”cv” Evaluation, quality and feedback Use of Technology in Healthcare Services in Technology Health Centers International training programs and projects EU-projects Developmental neuropsychology Teacher Freelance radio journalist
- 3. We all agree on… that feedback (queries) has an essential role in improving the teaching, learning, arrangements, management, well being and all the other important processes in any ( educational) organisation But….. … do we really succeed well with this task in real life? Feedback / Evaluation processes
- 4. 1/2001 The common problems of evaluation / feedback processes expensive… lots of work and paper wasted… the whole process takes too much time… results are unclear and difficult to make conclusions… lack of improvements based on the feedback… goal and purpose of the process / study remains unclear.. difficulties to see the effectiveness…
- 5. “ Right Things Right” “ the Zef-method”
- 10. Zef-method vs. Likert Scale ZEF Solutions Oy Course feedback - Company visits Study material - On-line support Interactivity with the other students 1 2 3 4 5 X = 1,6
- 11. Averages and Gaussian curve OR Priotising with Z-score
- 12. Excel/SPSS? Index? Ranking list? Average or the direction / priorities Reference material(X) Reference material(Y) Home Assignments (X) Home Assignments (Y) Interactivity with the teacher(X) Interactivity with the teacher (Y) Equipments (X) Equipments (Y) 1,86 3,62 1,97 4,12 3,83 3,83 4,15 3,45 4,83 4,8 4,71 4,88 4,71 4,85 4,46 4,75 4,45 4,51 4,23 4,77 4,66 4,82 3,14 4,37 4,74 4,8 3,75 3,78 4,29 4,42 4,57 3,72 4,68 4,66 4,66 4,51 4,46 4,65 4,11 4,65 3,85 4,23 4,12 3,78 3,45 3,38 4,46 4,46 4,46 4,37 4,17 4,11 4,38 4,12 4,68 4,8 2,15 4,72 2,92 3,85 1,62 4,66 2,55 4,83 4,94 4,91 4,92 4,92 4,92 4,92 4,92 4,92 4,92 4,65 4,92 4,35 4,71 4,46 4,74 4,08 4,71 4,72 4,71 4,72 4,65 4,74 4,72 4,68 2,09 4,15 2,31 3,92 3,89 4,23 2,28 4,58 4,42 4,37 3,78 4,11 4,15 4,52 4,23 4,69 5 5 5 5 5 5 5 5 3,94 3,85 3,95 4,03 4,85 4,92 4,83 4,92 3,29 4,75 3,43 4,66 3,91 4,68 3,48 4,57 4,74 4,86 4,68 4,8 4,69 4,83 3,69 4,92 4,55 3,43 2,38 3,29 4,28 4,51 3,97 4,49 3,94 4,02 4,95 4,95 4,31 4,48 4,14 4,97 4,89 4,85 4,94 4,94 4,98 4,97 3,89 4,97 3,55 4,8 3,68 4,94 3,52 4,89 4,55 4,92 4,71 4,72 4,25 4,43 4,69 4,85 4,06 4,83 4,75 4,85 4,25 4,25 4,8 4,86 4,82 4,86 1,92 3,58 2,17 3,74 2,35 3,85 2,08 4,31
- 13. Euros Quality Key figures Present state Target Feedback Action plan attractiveness of the course ( the number of students applying for the school / courses) 2,5 3,8 Report –case1 action plan how many students will complete the studies in scheduled time 67 % 85 % Report –case2 action plan How many students will interrupt the studies 25 % 5 % Report –case3 action plan Employment rate 74 % 92 % Report –case4 action plan
- 14. Benchmarking
- 15. Customer satisfaction Response rate 83% How have we succeeded in serving you? Project Management Project Planning Systematic Execution Communication Keeping our promises Experience and know-how Cooperation Dealing with errors and problems Flexibility Satisfactory results
- 16. PwC: Recession survey Blue: SMEs Red: Large companies 1. International networks 2. Focusing on real customer needs 3. Marketing of innovative products and services 4. Role of suppliers in the value chain of your company 5. Public and university research in your innovation activity
- 17. Employee well-being 1. I am valued and trusted 2. Fairness in the workplace 3. Giving and receiving feedback 4. Communication in the workplace 5. Compliance with rules and regulations 6. Development opportunities 7. Communication on difficult topics 8. I support my manager in management 9. My line manager values feedback 10. Use of satisfaction surveys 11.My work produces satisfaction
- 18. Hazard and risk assessments Response rate: 73% Statutory hazard and risk assessments To what extent do you see the following items as hazards/risk factors: 1. Effect of the work environment on job satisfaction 2. Threat of violence and substance abuse 3. Cooperation between managers and employees 4. Communication in the workplace 5. Uncertainty regarding job security 6. Ambiguity in work delegation and responsibilities 7. Deficiencies in job orientation 8. Lack of support from manager 9. Lack of valuation of your work 10. Inappropriate treatment in the workplace 11. Substance abuse
- 19. Innovation evaluations Which innovations are put into practice? Confidential ideas.
- 20. Organisation's self-evaluation Section 4: Information and knowledge management : 1. We monitor the fulfilment and development of our business objectives with appropriate indicators and set target values, in relation to the whole business and to individual processes. 2. We monitor the impact and efficiency of our business processes. 3. We can easily acquire analytical, up-to-date information on our company's performance and its development in relation to our strategic objectives and other points of comparison. 4. Our real-time internal accounting is predictive. 5. Our employees measure, analyse and evaluate the quality of their own work in order to make decisions and improvement measures related to their work in line with the whole company's development measures. 6. Our employees and the company's most important stakeholders receive accurate information as to how the company operates as a whole and in relation to themselves, and how its objectives are fulfilled. 7. We comprehensively manage the measurement data on our company and its functions, and the know-how of our people. 8. In knowledge management we utilise effective, creativity-oriented employer-employee negotiations.
- 21. Strategy analysis Response rate 88% Case: TEKES, $200M FinnWell program Components of the Vision Increase customer orientation in products and services Innovate new diagnostic methods and products Innovate new patient care methods and products Innovate new biodiagnostics methods and products Innovate new biotech patient care products Increase user friendliness of the tech and products Innovate new preventive care technologies Digitalize patient transactions Digitalize professional transactions Create new services utilizing new technologies Increase the use of new technologies in healthcare processes Innovate intelligent decision support systems Increase pilot cooperation between companies and hospitals Facilitate the integration of health care systems Facilitate more flexible hospital environments Create new logistics systems and solutions
- 22. Values of the Centre Party 15 largest towns and cities vs. other towns 1. Patriotism 2. Family and friends 3. Equality 4. Nature and environment
- 25. All Answers Explanations Blogs Wikis Podcasting Forums Instant messaging Audio / video conferences Emails RSS feeds Social bookmarking Shared web based calendar services Sharing of learning materials produced by teachers and students.
- 26. Greatest Potential All Answers Explanations Blogs Wikis Podcasting Forums Instant messaging Audio / video conferences Emails RSS feeds Social bookmarking Shared web based calendar services Sharing of learning materials produced by teachers and students . Start with These Services
- 27. Revised Competence might help to recognize the Potential All Answers Explanations Blogs Wikis Podcasting Forums Instant messaging Audio / video conferences Emails RSS feeds Social bookmarking Shared web based calendar services Sharing of learning materials produced by teachers and students. Revise your Competence
- 28. Revise your Competence All Answers Explanations Blogs Wikis Podcasting Forums Instant messaging Audio / video conferences Emails RSS feeds Social bookmarking Shared web based calendar services Sharing of learning materials produced by teachers and students. Re-evaluate the Potential for Learning
- 29. IT Specialists Teachers with eLearning Experience Mobile Technology Questionnaire: Clear Need For Communication
- 30. Evaluation Engine query excample: http://www.zef.fi/service/user/?q=425 Shall we try “Zefing” EduZef example: http://www.zef.fi/arviointikone/player/?q=837-af98fk7e How to use Zef-tool
- 31. Services / Solutions: ZEF® Evaluation Engine ZEF® Comparison Engine Other solutions Consultant services 1995 - Idea 1997 - First evaluation solution 2001 - Founded 2005 - Innofinland award 2007 - Biggest survey ever 2008 - Office founded in silicon valley 18 employees 5 offices in Finland, 1 in California ZEF Solutions Ltd.
- 36. MTV3's European Election Engine (EU-vaalikone) helps you find the best MEP candidate to vote for. By answering the questions in the service, you can compare your opinions to those of candidates . MTV3: European Election Engine
- 37. MTV3: Presidential Election Engine More than half a million users. Should fertility treatments be allowed for female couples? Shown are the responses from the presidential candidates. Also shown are the averages for all users, and all male and female users
- 38. Talouselämä newspaper: Economic Election Engine The Economic Election Engine (Talousvaalikone) focuses on the hard core of parliamentary power: economic administration. In deciding on the State budget, members of parliament also decide on the taxation and benefits of private individuals and organisations. Question shown: Should corporate taxation be reduced? Shown are the responses of the parties and the voters.
- 39. Pekka Silvén Director Professional Development Services Oulu University of Applied Sciences [email_address] +358 10 27 23195
Editor's Notes
- #4: But before that…let´s go back to basics… It all matters a good evaluation and feedback method… please don’t run away! I know that you might be thinking that this is waste of time and boring! That is the way I also used to think before. Today I am more than excited about this!
- #5: I know all about this too well and it makes me mad every time when I see people struggling with the feedback forms and senseless queries.
- #9: This is a simple example of the course feedback! Course feedback is typically made at the end of the course. I would like to ask why. Also the students might like to ask what is it good for them? Why not to ask already at the beginning of the course or after the few lessons. That is the only way to be able to affect also the on-going course members to benefit of the feedback. The results should be public (at least for the group).
- #14: In Finland each student will bring about 7000 dollars each year max 4 years The maximum number of students for each year is defined by the goverment We can have more, but we won’t get the money of extra persons. So, you must understand that these are some of the key figures in our University. Without a good success in figures, we would be in trouble. What are the reasons for the students to apply for our school? Where and how did they received the information and made them to choose our University? What are they worried about at the starting phase of the studies reason for the delays of the studies reason for quitting the studies