Bridging the Gap: Examining a Model for Evaluator Training

Editor's Notes

• #4: Educational opportunities and training is great for developing skills but traditional classroom environments and constrains in many of the programs do not allow for the students to have an opportunity to apply the foundational knowledge and skills they are taught; in order for people to be marketable & employable we need to build a bridge between training and programs available to emerging evaluators and formal opportunities in which they can apply their skills and demonstrate their competenciesCouse work allows one to build a solid understanding of theory, an evaluator’s roles and methods specific to evaluation; course can develop competencies (e.g., design a project; learning how to conduct specific types of analysis, etc) but classroom constraints limit the types of courses that can be offered and the projects that can be assigned within those courses (i.e., timeframes, ethical considerations, feasibility, etc); Cheryl’s current challenge of her courses needing to appeal to a broader audience due to the lack of a critical mass of “evaluation students” within her department. Additional constraints facing Cheryl include missing or a lack of resources (sorry didn’t catch all these Cheryl/ Andrew)Link to personal experience
• #5: How can the applied work experience develop competencies? The employer provides the practice based environment to apply the theories and concepts that one acquires through formal training. It is not just mentoring and training, but learning on the job, when an individual is working on complex, real-world problems or evaluation questions where one has the opportunity to apply the knowledge and skills they acquired through school – adult learners can build on their competencies and theory suggests that adults learn best through “doing” - adults learn by doing.An internship or similar work experience setting that provides real world problems to solve (would like to give some examples of the work our interns have done, and perhaps speak to the importance of the problems being real – or maybe Andrew should speak to this], and a safe and supportive environment to practice the learning, engage in discussion and dialogue along with the guidance of a supportive and accessible mentor may offer the ideal arrangement for new evaluators because it provides and opportunity for them to move from acquiring knowledge and skills from their formal training programs, to building upon and enhancing those qualities to eventually become a professionally competent and confident evaluator.Benefits to the employer: In a knowledge generation economy, at a time when interest in “highly skilled or qualified people” is at a peak, securing professional talent can be a challenge. One of the solutions we wanted to explore as a way to address this challenge was to introduce this internship program (based on our experience with past summer student contracts); after establishing a relationship with the Cheryl at the University of Alberta (identifying a target candidate pool), the internship has enabled our organization to recruit and in some instances retain HQP for our organization. In our experience, this type of recruitment has been far less expensive (in terms of recruitment costs and time) then traditional methods, with a better guarantee or outcome of a high quality placement, then the traditional process. An additional benefit of the internship is, for our organization who has been in a hiring freeze for 2 years means that because we have adopted a ‘contract’ based placement, the hiring freeze has not precluded use for getting these talented people from the universities. Finally, one of the most obvious benefits to our organization has been the influx of these HQP and the interest, passion, new ideas, insights and perspectives they bring to the table - How we did it: What made this possible was a supportive work environment coupled with some additional environment drivers – including the introduction of the CES credentialing certification and lack of experienced recruits, need to highly skilled people and resource/ hiring constraints. More about how we make this happen so not taxing on the employer (i.e., low stakes) but there is still a learning componentWhat’s missing from the work experience- we provide targeted PD based on our business need and the personal development needs of the intern, but the foundational knowledge is needed at entryLink to personal experience
• #6: Framework verification projects involved tests of a model before broad adoption; In addition to the project work, at AIHS we also try to include the interns in the planning meetings and brainstorming sessions: this is where we can really tap into each others ideas, and we can learn and gain new insights from their fresh perspectives, and the interns benefit as well because they are exposed and learn about the “additional contextual” factors that are present in ‘real world’ scenarios and
• #7: Talk specifically to experiences in both internshipsDriven my a practical need = needed a job yet, most positions require previous work experience in addition to formal training/ educationEvaluation, by nature, is a very practical field; while classroom education is important, it doesn’t fulfill all of the core competencies (i.e, only really targets technical practice)The AIHS Internship offered a protective & supportive environment to apply skills and a work place mentor to provide guidance and oversight to the new evaluator How did this differ (if at all) from your previous internshipOffered me the chance to put my theory into practiceCompare & contrast internship experiencesHaving been in an internship without any eval training as well as being in an internship after eval training, it’s very easy to see how the two compliment each other
• #9: TAKE HOME MESSAGE
• #10: Upcoming plans: A follow-up interview re: our PM internship; details to TBD with Cheryl/ Heidi
• #11: Other thoughts:What should people/ organizations expect bring to the table or otherwise contribute to the successful development of professional and competent evaluators?Who’s role is it to develop competent evaluators? What are the boundary roles? Where does the responsibility start?