Outputs, Outcomes, and Logic Models
Download as PPTX, PDF3 likes15,089 views
This document discusses using outcomes and logic models to evaluate programs. It defines key terms like outputs, outcomes, indicators, and impacts. It emphasizes measuring outcomes over just outputs. A logic model is presented as a visual way to connect a program's resources and activities to its intended outputs, outcomes, and impacts. The document provides examples and has interactive activities for participants to develop the components of a logic model for their own program.
Outputs, Outcomes, and Logic Models
- 1. Outputs, Outcomes, and Logic Models Community Training Institute 2013 Ashley Brundage, Senior Director of Community Impact Grants United Way of Metropolitan Dallas
- 2. Topics and Objectives Why Use Outcomes? • Learn value of outcome measurement Logic Models • Clearly explain how to construct the logic model Outputs • Clearly explain outputs Outcomes • Provide an in-depth discussion on developing outcomes 2
- 3. Why Use Outcomes? 3
- 4. The importance of Evaluation and Outcome based programing The Accountability measurement of Results Action & Analysis & Improvement Learning 4
- 5. Benefits of Outcome Measurement • Increased accountability to donors and the community • Enhance marketing and fundraising messages • An improved image and greater visibility in the community • Greater success in retaining, maintaining and/or increasing dollars “The American public has extended you the right to avoid taxation in exchange for your commitment to create real social value. You owe it to the public to make a real effort to track your performance and report to them on how you are doing in your work.” from Enterprising Nonprofits 5
- 6. Questions that Outcomes Answer • Did the program achieve is intended goals/impact/etc.? • How does the program contribute to the agency‟s strategic goals? • Is this program worth the cost? • Did the program reach the target population? • How can implementation be improved? • Is this program still relevant or viable? • Is what was promised being achieved? • Is the program working? • What is the program accomplishing? • How does this program compare to others? • Did we reasonably accomplish what we committed to do? • How have participants changed as a result of our program? 6
- 7. Logic Models
- 8. Logic Models: Purpose What is a Logic Model? • If Then • Visual representation of the program‟s intent Why are Logic Models necessary? • Help to assess the program‟s effectiveness. • Illustrate the client‟s progress through the program • Developing a realistic picture of what the program can expect to accomplish • Helps tell your story and the difference you‟re making 8
- 9. Logic Models: Resources & Activities Resources Activities • Identify all inputs that • What will participants be make the activities doing? possible. • Prioritize, what activities • List all the “what” and will help to achieve the “who” that is necessary to outcomes? delivering the activities. • People, materials, meeting space, etc. 9
- 10. Resources & Activities Example Resources Activities • Staff • Child abuse awareness • Curriculum presentations • Funding • Distribution of • Awareness handouts awareness handouts • Conference room • Projector 10
- 11. ACTIVITY BREAK 11
- 12. Activity Break: Resources and Activities Think about a program at your agency to evaluate. • Needs to have a clear target population. • Needs to have a clear method of providing services. • Focus on one for the purposes of this training today. Fill in the Resources and Activities of your program. Work in pairs to help each other. What did you find difficult about this part? Easy? 12
- 13. Outputs 13
- 14. Logic Model Component: Outputs Primary What do Activities clients do? What is Outputs produced? 14
- 15. Outputs Example Outputs: • Educators will teach 200 people about the signs of child abuse. • Educators will give 10 hour long presentations. • Outreach staff will distribute 1,000 child abuse awareness handouts. 15
- 16. Activity Break: Develop Outputs With your partner, Discuss your program‟s activities Develop your outputs Fill in the output column of your logic model Share with the group one output from each of your programs What did you find difficult about this part? Easy? 16
- 17. Outcomes
- 18. What is outcomes based planning and evaluation? “Not how many worms the bird feeds its young, but how well the fledgling flies” United Way of America, 2002 18
- 19. Logic Model Component: Outcomes • What is an Outcome? • Outcomes are the benefits/client change for participants during or after their involvement with a program. • Outcomes may relate to a change in knowledge, skills, attitudes, values, behavior, condition, or status. • Satisfaction is not an outcome. Example: Educators increase the audience‟s awareness about signs of child abuse. 19
- 20. Indicators • What is an Indicator? • Indicators are specific observable, measurable characteristics or changes that represent achievement of the overall impact. • Specific statistics calculated to summarize the program‟s level of achievement. Example: Percent of target audience that remembers key points of lecture. 20
- 21. Building an Outcome How you What your want your Indicator of Outcome clients do clients to Change Statement change Outputs Outcome Indicator 21
- 22. Building an Outcome Audience will have increased awareness of child Increased abuse as Attend Increased Pre/Post measured by an presen- Awarenes Test increased score tations s scores from their pre to post test taken before and after Outputs Outcome Indicator the presentation. 22
- 23. Activity Break: Outcome Statements • Look at your activities and outputs you have for your program so far. • What outcomes do you want to see? • What indicators will determine those outcomes? • Develop at least one Outcome Statement for your program and share with the group. 23
- 24. Outcomes: SMART S • Specific M • Measurable A • Action-Oriented R • Realistic T • Timed 24
- 25. Outcomes: The SMART Format X% (number) of clients will improve by X% or achieve X in a specific time frame. 25
- 26. Outcomes: Example 80% (160) of 200 clients will increase their post test score by 20% above their pre test score after attending one child abuse awareness presentation. 26
- 27. Activity Break: Outputs vs Outcomes Check under seat Read out loud Is yours an Output or an Outcome? 27
- 28. Activity Break: SMART Outcomes • Using the Outcomes statement developed earlier • Add SMART Criteria • Fill in the Outcomes column on Logic Model • Report to Group Examples 28
- 29. Logic Model Components: Impact Impacts are organizational, community, and/or system level changes expected to result from program activities, which might include improved conditions, increased capacity, and/or changes in the policy arena. Ex: Improved health and well being in North Texas. 29
- 30. Activity Break: Add your impact statement to your logic model. Share examples with the group. 30
- 31. Logic Models: Tips A common problem is that activities and strategies often do not lead to the desired outcomes. Check your „if-then‟ statements and ensure that they make sense and lead to the outcomes you want to achieve. A logic model makes the connections EXPLICIT. “I think you should be more explicit here in Step Two.” 31
- 32. Activity Break: Logic Model Exercise Using the logic model example provided • Individually circle what you think needs adjustment • Share with the group what you identified as problem areas and why 32
Editor's Notes
- #2: 6 years at UWMD2 years at UWTCSeveral grant review committees: CDBG, Homeless Review comm for Continuum of Care in Dallas and Tarrant, Dallas Women’s Foundation
- #3: Format of training today – will leave session with a logic model developed of your program, lots of sharing of examples and working together.
- #5: OutcomeMEASUREMENT quantifies the extent to which program clients achieve intended benefitsIt helps agencies better understand both clients and programs and discover which practices are more effective than others with different client groupsThe LEARNING that comes from thoughtful ANALYSIS of program outcome data helps programs IMPROVE their services to achieve even better results for clientsWhen agencies feature programs’ results in their marketing and campaign messages, they exhibit ACCOUNTABILITY to investors and the community and assert leadership in results-oriented programming.
- #9: It is an “If – Then” sequence of changes that the program intends to set in motion through inputs, activities, and outputs.A graphic representation of your program showing the intended relationships between investments and results.Logic Models help to identify the key program components that must be tracked to assess the program’s effectiveness.Logic Models illustrate the client’s progress through the program, and developing a realistic picture of what the program can expect to accomplish.Logic Models provide the program description that guides the evaluation processHelps the audience match evaluation to the programHelps organizations to know what and when to measure
- #10: Gather Resources:Identify all inputs that make the activities possible.List all the “what” and “who” that is necessary to delivering the activities.Primary activities – the main activities provided by your program, the activities that will make the desired change in the client
- #13: 10 mins3 min on own3 min with partner4 minutes report out
- #15: Outputs are the tangible, direct results of program activities – things you can count easilyWhat is an output?Direct products of program activities and usually are measured in terms of volume of work accomplishedThink in units (i.e. time, people, things)Be specific, list the actual numbersExample:Educators taught 200 people about the signs of child abuse.
- #17: 12 mins:5 min withparnter7 min report out
- #19: Want to know the impact of feeding the bird the worms
- #20: Satisfaction: Satisfaction is not an outcome because being “satisfied” with something doesn’t mean that someone has changed or improved. Participants may be satisfied with the program, or like you as a person, or want to come again, but such reactions do not indicate that they have changed or benefited in any way.
- #21: How can I measure that they are remembering key points of the lecture?
- #22: Outputs - What your clients do - what are the outputs of your program activities, what can be counted/measured?Outcome - How you want your clients to change -– How do you want to change your clients’ lives? What do you want to target?Indicator – what can you measure to show the achievement of the outcome?Outcome Statement - Statement connecting the three–– What can you measure that will represent the change in your client? Includes the indicator and the output.
- #23: What your clients do - Program Activities- what are the outputs of your program activities, what can be counted/measured?How you want your clients to change - Outcome– How do you want to change your clients’ lives? What do you want to target?Statement connecting the two – Outcome Statement– What can you measure that will represent the change in your client? Includes the indicator and the output.
- #24: 10 mins5 on own5 share w group
- #28: 7 mins
- #29: 7 minsOutcome developed earlier, SMART criteria, check for adherence and make changes as needed.
- #31: 7 mins
- #32: Start at the end, identify the outcomes and goals.Avoid Logic Models that are too long or too vague.Group Process includes: Staff and volunteers, fund providers, administrators, board members, clients, or a content expert.Examples of What Not to do:Safe sex 1 time presentation => students will be prepared to graduate1 time financial education seminar => clients will improve their credit score2 week summer camp => students will be prepared to graduateWeek long Nutrition course => Reduced BMIYou need to have a long term engagement or monitoring plan to prove these types of results Also need lots of good strong research to back up that this type of intervention will result in those impacts.
- #33: 5 mins