Communities for Children evidence-based program requirement – where to from here?
- 1. Communities for Children evidence- based program requirement – where to from here? Elly Robinson FRSA Conference, 10-12 November 2015 Brisbane, QLD
- 2. Communities for Children Facilitating Partners - objectives To improve the health and wellbeing of families and the development of young children (0-12 years), paying special attention to: Healthy young families Supporting families and parents Early learning School transition and engagement
- 3. The evidence-based program requirement New grant agreements From 1 July 2015 at least 30% of funding used for direct service delivery should be used to purchase high quality, evidence-based programs as identified by the Expert Panel. Increases to 50% from 1 July 2017
- 4. Dual approach The “guidebook” Evidence-based programs that meet a rigorous set of criteria “Other” programs Recognise that good programs are already being implemented A slightly less rigorous set of criteria
- 5. The “Guidebook” Criteria for inclusion Objectives of program are in line with CfC FP model. Targets children aged 0-12 years and their families. Documented information on program is available: Aims, objectives and theoretical basis; Program logic or similar; Target group for program; and Activities of the program and why they are important.
- 6. The “Guidebook” Criteria for inclusion of programs Training manual or documentation that allows for replication in Australia is available. A good quality evaluation with at least 20 participants. Evaluation shows positive outcomes (and no negative effects reported). Program has been replicated (or has potential for replication). 30 profiles now available
- 7. The “Guidebook” – recent additions Healthy & Safe A home‐based education resource designed to equip parents with learning difficulties with the knowledge and skills necessary for managing home dangers, accidents and childhood illness. Parent-Child Mother Goose A group program for parents/carers and their babies/young children. Supports the development of secure parent-child attachment, promotes children’s speech development and enhances families’ community inclusion. https://www.sd79.bc.ca/schools- programs/programs/early-learning/parent-child- mother-goose/
- 8. The “Guidebook” – recent additions Smalltalk A service enhancement that can be embedded within community-based supported playgroups. Designed for participating parents experiencing vulnerable circumstances with children aged 6-months to 4 years. Through the Looking Glass A therapeutic parenting program based on attachment theory principles which aims to strengthen attachment relationships between children aged 0- 5 years and their parents/carers in cases where attachment has been compromised.
- 9. Assessment of “other” programs Criteria for inclusion - must be documented A theoretical and/or research background to the program. A program logic (or theory of change, or logic model). Activities in the program which generally match good practice in meeting the needs of the target group. An evaluation (with at least 20 participants) has established that the program has positive benefits for the target group. Staff members are qualified and/or trained to run the program. Needs to tell the “story” of your program http://bit.ly/1MLW411
- 10. 3. Provisional assessment In recognition of when more time is needed to meet criteria for 30% requirement. Must be able to: Articulate research and/or theoretical background of the programme; show that some form of evaluation has been conducted in the past (may not meet the rigorous evaluation criteria yet); and outline your plan for meeting the criteria in the online guide by 30 June 2016.
- 11. Assessment of programs - summary Wide variation of quality and documented effectiveness Total programs so far: 13 programs fully approved 42 provisionally approved 7 not approved https://aifs.gov.au/cfca/expert-panel-project/list- programmes-submitted-approval-cfca
- 12. Can we deliver programs on approved list? Fully approved programs meet a minimum standard for the purposes of the 30/50% requirement. Not evidence-based as per the “guidebook” programs – would still prefer these were used. If FPs wish to explore further, we can provide contact details for the program developers. Provisionally assessed programs – suggest waiting.
- 13. Same program – different locations In some cases, the same program is being run in multiple locations Program fidelity = staying true to the original program design Evidence/evaluation relates to particular program To what extent is the alternative version different?
- 14. Program adaptation Acceptable Changing language, translations Replacing cultural references Adding relevant, evidence based materials to make the program more appealing to potential participants Risky/unacceptable Reducing number/length of sessions (“dosage”) Removing topics Changing the theoretical approach Lowering the level of participant engagement
- 15. Building the evidence base Intention is to continue to build our understanding of “what works” - with your help “Nudge” programs along the evidence-base continuum Avoid recreating the wheel where possible. Three things over next two years: Move provisionally approved programs to fully approved. More fully approved programs into guidebook. Focus on 50% requirement, including increasing the pool of available programs.
- 16. Theory/research basis for programs - feedback Importance Does something already exist that is proven to work, or can reliably inform our work? What is already known about addressing the needs of our target group? Many variations on how addressed. Best examples: Used target group needs as basis Explored what was known already about addressing these needs Used this to frame a response Clear pathway to activities and outcomes
- 17. Theory/research basis for programs - feedback Examples Well-known programs that have no or limited theoretical/research background, “good practice” activities, evidence of effectiveness (often commercial products) Longstanding programs with limited/dated research basis Programs developed using target group input & practitioner knowledge Drawing from different evidence “pots”– the playgroup example
- 18. Other criteria Program logic Varied quality and generally indicative of quality of program Program pathway needs to make sense – “if-then”. The “roadmap” for the program Activities Need to match “good practice” - e.g. group vs individual service, who delivers? Often a lack of detail provided.
- 19. Evaluation Good examples Reasonable sample size to draw conclusions Some level of rigour and appropriate methodology Use of validated tools Not just “feedback sheets” – demonstrated, measured change from start to end of program Different approaches – degree of discretion for the purposes of the requirement
- 20. How can we help? CFCA inquiry helpdesk Provides assistance with enquiries related to the latest stats and facts, evaluation advice, research findings, practice/policy pointers or many other topics. To access the research and information helpdesk call 03 9214 7888 or use our contact form at https://aifs.gov.au/cfca/contact
- 21. Industry List Helps to connect FaC service providers with research, practice and evaluation experts. “Screened” members of Industry List: Quality services, fixed price, help to draft proposal. Guide to use: https://aifs.gov.au/cfca/expert-panel- project/information-service-providers/guide-using- industry-list Or contact us to talk!
- 22. Overall reflections Terrific opportunity to work with sector – thank you! Main feedback Need a coherent pathway through program (what is the program’s “story”?) Most difficult seemed to be longstanding programs Some commercial products have a weak evidence base – be careful Use the Industry List – they have a track record
- 23. Important points for provisionally approved programs Purpose of project: Build capacity of organisations to deliver quality, evidence-based programs that work. Other programs, innovations – outside of the 30-50% All documentation must be submitted by 30 June 2016. Make sure plans reflect this. Due to volume, please submit complete documentation to CFCA.
- 24. Contact us Expert Panel project https://aifs.gov.au/cfca/expert-panel-project Industry list and CfC queries fac-expert-panel@aifs.gov.au Phone us 03 9214 7888 Elly, Sharnee or Kat
Editor's Notes
- #2: Please add the title of the paper and author names in the File > Properties dialogue box.
- #6: Use this slide to create normal text slides. This is the default. If you are adding a table or graphics, please add a description of it in the slide notes (here) and also add alternative text to images and tables (right-click on the image, select Format Picture and type the title of the image into the Title box).
- #7: Use this slide to create normal text slides. This is the default. If you are adding a table or graphics, please add a description of it in the slide notes (here) and also add alternative text to images and tables (right-click on the image, select Format Picture and type the title of the image into the Title box).
- #8: Use this slide to create normal text slides. This is the default. If you are adding a table or graphics, please add a description of it in the slide notes (here) and also add alternative text to images and tables (right-click on the image, select Format Picture and type the title of the image into the Title box).
- #9: Use this slide to create normal text slides. This is the default. If you are adding a table or graphics, please add a description of it in the slide notes (here) and also add alternative text to images and tables (right-click on the image, select Format Picture and type the title of the image into the Title box).
- #10: Use this slide to create normal text slides. This is the default. If you are adding a table or graphics, please add a description of it in the slide notes (here) and also add alternative text to images and tables (right-click on the image, select Format Picture and type the title of the image into the Title box).