Planning Slides.pdf (1).pdf

The Context of Health Program
Development and Evaluation
Chapter 1
‫وتقييمها‬ ‫الصحية‬ ‫البرامج‬ ‫تطوير‬ ‫سياق‬
Translated by
Khaled
Good Luck

Learning Objectives
With this chapter, readers will be able to:
1. Articulate a definition of evaluation.
2. Understand the relevance of the public
health pyramid to health program planning
and evaluation.
3. List circumstances under which an
evaluation is not recommended.
• ‫من‬ ‫القراء‬ ‫سيتمكن‬ ، ‫الفصل‬ ‫هذا‬ ‫مع‬
:
• ‫التقييم‬ ‫تعريف‬ ‫توضيح‬
.
• ‫وتقييمها‬ ‫الصحية‬ ‫البرامج‬ ‫تخطيط‬ ‫في‬ ‫العامة‬ ‫الصحة‬ ‫هرم‬ ‫أهمية‬ ‫فهم‬
.
• ‫بالتقييم‬ ‫فيها‬ ‫يوصى‬ ‫ال‬ ‫التي‬ ‫الظروف‬ ‫اذكر‬
.

Presentation Outline
• Key terminology
• Concept of health
• Generational
milestones of
evaluation
• Evaluation standards
• The planning and
evaluation cycle
• The program cycle
• Evaluation types
• When not to evaluate
• The public health
pyramid
• Healthy People 2010
objectives
• Internet resources
•
‫الرئيسية‬ ‫المصطلحات‬
•
‫الصحة‬ ‫مفهوم‬
•
‫التقييم‬ ‫من‬ ‫األجيال‬ ‫معالم‬
•
‫التقييم‬ ‫معايير‬
•
‫والتقييم‬ ‫التخطيط‬ ‫دورة‬
•
‫البرنامج‬ ‫دورة‬
•
‫التقييم‬ ‫أنواع‬
•
‫لتقييم‬ ‫ال‬ ‫عندما‬
•
‫العامة‬ ‫الصحة‬ ‫هرم‬
•
‫األصحاء‬ ‫األشخاص‬ ‫أهداف‬
2010
•
‫االنترنت‬ ‫موارد‬

Key Terminology
• Aggregate
• Community needs assessment
• Comprehensive evaluation
• Cost evaluation
• Effect evaluation
• Evaluation
• Formative evaluation
• Health
• Health program
• Health services
• Impact evaluation
• Institutionalized program
• Meta-evaluation
• Model program
• Outcome evaluation
• Pilot program
• Planning
• Process evaluation
• Process theory
• Program
• Project
• Prototype program
• Summative evaluation
• Service
•
‫مجموع‬
•
‫المجتمع‬ ‫احتياجات‬ ‫تقييم‬
•
‫شامل‬ ‫تقييم‬
•
‫التكلفة‬ ‫تقييم‬
•
‫التأثير‬ ‫تقييم‬
•
‫تقييم‬
•
‫التكويني‬ ‫التقييم‬
•
‫الصحة‬
•
‫الصحة‬ ‫برنامج‬
•
‫صحية‬ ‫خدمات‬
•
‫األثر‬ ‫تقييم‬
•
‫مؤسسي‬ ‫برنامج‬
•
‫التقييم‬ ‫ميتا‬
•
‫نموذجي‬ ‫برنامج‬
•
‫النتائج‬ ‫تقييم‬
•
‫تجريبي‬ ‫برنامج‬
•
‫تخطيط‬
•
‫العملية‬ ‫تقييم‬
•
‫العملية‬ ‫نظرية‬
•
‫برنامج‬
•
‫مشروع‬
•
‫األولي‬ ‫النموذج‬ ‫برنامج‬
•
‫تلخيصي‬ ‫تقييم‬
•
‫الخدمات‬

Health as a Concept
• WHO definition (1947)
– Absence of illness and presence of well-being
• Health viewed as holistic
– Physical, mental, developmental social, and financial
aspects
– “Health problems” may affect one domain while the
others remain healthy
• Health viewed as political
– Health policy around access to care
– Implications for program planning
•
‫العالمية‬ ‫الصحة‬ ‫منظمة‬ ‫تعريف‬
(
1947
)
•
‫الكلية‬ ‫والمالية‬ ‫واالجتماعية‬ ‫والتنموية‬ ‫والعقلية‬ ‫المادية‬ ‫الجوانب‬ ‫أنها‬ ‫على‬ ‫إليها‬ ‫ينظر‬ ‫الصحة‬ ‫ووجود‬ ‫المرض‬ ‫غياب‬
•
‫تؤثر‬ ‫قد‬
"
‫الصحية‬ ‫المشكالت‬
"
‫سليمة‬ ‫األخرى‬ ‫المجاالت‬ ‫تظل‬ ‫بينما‬ ‫واحد‬ ‫مجال‬ ‫على‬
•
‫سياسية‬ ‫تعتبر‬ ‫الصحة‬
•
‫الرعاية‬ ‫على‬ ‫الحصول‬ ‫حول‬ ‫الصحية‬ ‫السياسة‬
•
‫البرنامج‬ ‫تخطيط‬ ‫على‬ ‫المترتبة‬ ‫اآلثار‬

Generational Milestones of Evaluation
• Technical – growth of scientific management, statistics,
and research methodologies
• Descriptive – use of goals and objectives as the basis of
evaluation to establish program merit (outcome-focused)
• Responsiveness – concern for the usefulness of the
evaluation to stakeholders; participation of stakeholders
in the evaluation process (utilization-focused)
• Meta-evaluation – evaluation of evaluations across
similar programs
•
‫التقنية‬
-
‫البحث‬ ‫ومنهجيات‬ ‫واإلحصاء‬ ‫العلمية‬ ‫اإلدارة‬ ‫نمو‬
•
‫وصفي‬
-
‫الجدارة‬ ‫لتحديد‬ ‫للتقييم‬ ‫كأساس‬ ‫والغايات‬ ‫األهداف‬ ‫استخدام‬
‫البرنامجية‬
(
‫النتائج‬ ‫على‬ ‫تركز‬ ‫التي‬
)
•
‫االستجابة‬
-
‫التقييم‬ ‫عملية‬ ‫في‬ ‫المصلحة‬ ‫أصحاب‬ ‫مشاركة‬ ‫؛‬ ‫المصلحة‬ ‫ألصحاب‬ ‫التقييم‬ ‫بفائدة‬ ‫االهتمام‬
(
‫االستخدام‬ ‫على‬ ‫تركز‬ ‫التي‬
)
•
‫التلوي‬ ‫التقييم‬
-
‫مماثلة‬ ‫برامج‬ ‫عبر‬ ‫التقييمات‬ ‫تقييم‬
‫التقييم‬ ‫في‬ ‫األجيال‬ ‫معالم‬

Outcome- vs. Utilization-Focused
Evaluations
Focus Purpose Audience Method
Outcome Show program effect Funders, researchers,
other external
audience
Research methods;
usually external
evaluators
Utilization Get stakeholders to use
evaluation findings for
decisions regarding program
improvements and future
program development
Program people
(internal audience),
funders
Research methods,
participatory
‫االستخدام‬ ‫على‬ ‫تركز‬ ‫التي‬ ‫التقييمات‬ ‫مقابل‬ ‫النتائج‬
‫البرنامج‬ ‫تأثير‬ ‫إظهار‬
‫والجمهو‬ ‫والباحثين‬ ‫الممولين‬
‫ر‬
‫الخارجي‬
‫عادة‬ ‫البحث؛‬ ‫طرق‬
‫خارجيين‬ ‫مقيمين‬
‫التق‬ ‫نتائج‬ ‫يستخدمون‬ ‫المصلحة‬ ‫أصحاب‬ ‫اجعل‬
‫ييم‬
‫البرنا‬ ‫بتحسينات‬ ‫المتعلقة‬ ‫القرارات‬ ‫التخاذ‬
‫مج‬
‫المستقبل‬ ‫في‬ ‫وتطويره‬
‫الناس‬ ‫برنامج‬
(
‫الجمهور‬
‫الداخلي‬
)
‫الممولين‬ ،
‫التشاركي‬ ‫البحث‬ ‫طرق‬

Standards Established by the Joint Commission
on Standards for Educational Evaluation
Adapted from the American Evaluation Association (2002)
Standard Description
Utility To ensure that the evaluation will meet the content needs of
those involved
Feasibility To ensure that the evaluation will be realistic, prudent,
diplomatic, and frugal
Propriety To ensure that the evaluation will be conducted in a legal and
unbiased way
Accuracy To ensure that the evaluation will communicate appropriate
and accurate information
‫للمشاركين‬ ‫المحتوى‬ ‫احتياجات‬ ‫التقييم‬ ‫يلبي‬ ‫أن‬ ‫لضمان‬
‫ًا‬‫د‬‫ومقتص‬ ‫ا‬ً‫ي‬‫ودبلوماس‬ ‫ا‬ً‫م‬‫وحكي‬ ‫ا‬ً‫ي‬‫واقع‬ ‫التقييم‬ ‫يكون‬ ‫أن‬ ‫لضمان‬
‫متحيزة‬ ‫وغير‬ ‫قانونية‬ ‫بطريقة‬ ‫التقييم‬ ‫إجراء‬ ‫لضمان‬
‫والدقيقة‬ ‫المناسبة‬ ‫المعلومات‬ ‫ينقل‬ ‫سوف‬ ‫التقييم‬ ‫أن‬ ‫لضمان‬
‫التربوي‬ ‫التقييم‬ ‫لمعايير‬ ‫المشتركة‬ ‫اللجنة‬ ‫وضعتها‬ ‫التي‬ ‫المعايير‬

The Planning and Evaluation Cycle

The Program Life Cycle:
Assessment, Implementation, and Evaluation
Program
Stage
Community Assessment Program Implementation Program Evaluation
Pilot Generic information about
the health problem & target
audience
‫الصحية‬ ‫المشكلة‬ ‫عن‬ ‫عامة‬ ‫معلومات‬
‫المستهدف‬ ‫والجمهور‬
Few participants; strict
guidelines & protocols
‫إرشادا‬ ‫المشاركين‬ ‫من‬ ‫قليل‬ ‫عدد‬
‫ت‬
‫صارمة‬ ‫وبروتوكوالت‬
Rigorous impact
evaluation & process
monitoring
‫العمل‬ ‫ومراقبة‬ ‫الدقيق‬ ‫األثر‬ ‫تقييم‬
‫ية‬
Model Greater information about
the target audience
‫الجمهور‬ ‫حول‬ ‫المعلومات‬ ‫من‬ ‫مزيد‬
‫المستهدف‬
Realistic number of
participants; use
previously set procedures
‫استخدام‬ ‫المشاركين‬ ‫من‬ ‫واقعي‬ ‫عدد‬
‫مسبقا‬ ‫المحددة‬ ‫اإلجراءات‬
Outcome & impact
assessment; rigorous
process monitoring
‫عم‬ ‫مراقبة‬ ‫؛‬ ‫والنتائج‬ ‫النتائج‬ ‫تقييم‬
‫لية‬
‫صارمة‬
Prototype Very specific information
about the target audience &
local variations in the health
problem
‫الجمهور‬ ‫حول‬ ‫للغاية‬ ‫محددة‬ ‫معلومات‬
‫المشك‬ ‫في‬ ‫المحلية‬ ‫والتغيرات‬ ‫المستهدف‬
‫لة‬
‫الصحية‬
Some flexibility and
adaptation to local needs;
realistic enrollment
‫االحتياجا‬ ‫مع‬ ‫والتكيف‬ ‫المرونة‬ ‫بعض‬
‫ت‬
‫واقعي‬ ‫التحاق‬ ‫؛‬ ‫المحلية‬
Outcome & impact
assessment; rigorous
process monitoring
‫البرنامج‬ ‫حياة‬ ‫دورة‬
:
‫والتقييم‬ ‫والتنفيذ‬ ‫التقييم‬

The Program Life Cycle, Continued
Program Stage Community
Assessment
Program
Implementation
Program Evaluation
Organizationally
Institutionalized
More attention to
assessment of
organizational
resources for
program
sustainability
‫الموار‬ ‫لتقييم‬ ‫االهتمام‬ ‫من‬ ‫مزيد‬
‫د‬
‫البرنامج‬ ‫الستدامة‬ ‫التنظيمية‬
Use standard
operating procedures;
organization specific
‫التشغيل‬ ‫إجراءات‬ ‫استخدام‬
‫محددة‬ ‫منظمة‬ ‫؛‬ ‫القياسية‬
Outcome & impact
assessment based on
objectives; routine
process monitoring
‫ا‬ ‫على‬ ً‫ء‬‫بنا‬ ‫واألثر‬ ‫النتائج‬ ‫تقييم‬
‫؛‬ ‫ألهداف‬
‫روتينية‬ ‫عملية‬ ‫رصد‬
Professionally
institutionalized
Rarely any detail,
more assessment of
competitors and
professional norms
‫والمزيد‬ ، ‫تفاصيل‬ ‫أي‬ ‫ما‬ ‫نادرا‬
‫والقواع‬ ‫المنافسين‬ ‫تقييم‬ ‫من‬
‫د‬
‫المهنية‬
Standards for
professional practice;
may involve
certification
‫قد‬ ‫؛‬ ‫المهنية‬ ‫الممارسة‬ ‫معايير‬
‫شهادة‬ ‫على‬ ‫تنطوي‬
Use professionally set
standards or benchmarks
of outcome & impact
assessment
‫المحددة‬ ‫المعايير‬ ‫أو‬ ‫المعايير‬ ‫استخدم‬
‫واألثر‬ ‫النتائج‬ ‫لتقييم‬ ‫ا‬ً‫ي‬‫احتراف‬

Evaluation Types
– Outcome evaluation
– Impact evaluation
• Cost evaluation
• Comprehensive evaluation
• Meta-evaluation
•
•
‫العملية‬ ‫تقييم‬
•


‫األثر‬ ‫تقييم‬
•
‫التكلفة‬ ‫تقييم‬
•
‫شامل‬ ‫تقييم‬
•
‫ميتا‬ ‫تقييم‬

When Not to Evaluate
• There are no questions about the program
• The program has no clear direction
• Stakeholders cannot agree on the program
objectives
• There is not enough money to conduct a
sound evaluation
•
‫البرنامج‬ ‫حول‬ ‫أسئلة‬ ‫توجد‬ ‫ال‬
•
‫واضح‬ ‫اتجاه‬ ‫لديه‬ ‫يوجد‬ ‫ال‬ ‫البرنامج‬
•
‫البرنامج‬ ‫أهداف‬ ‫على‬ ‫االتفاق‬ ‫المصلحة‬ ‫ألصحاب‬ ‫يمكن‬ ‫ال‬
•
‫سليم‬ ‫تقييم‬ ‫إلجراء‬ ‫المال‬ ‫من‬ ‫يكفي‬ ‫ما‬ ‫يوجد‬ ‫ال‬
‫لتقييم‬ ‫ال‬ ‫عندما‬

The Pyramid as an Ecological Model

Healthy People 2010 Objectives in 28 Focus Areas
1. Access to quality health services
2. Arthritis, osteoporosis, and chronic
back conditions
3. Cancer
4. Chronic kidney disease
5. Diabetes
6. Disability and secondary conditions
7. Educational and community-based
programs
8. Environmental health
9. Family planning
10. Food safety
11. Health communication
12. Heart disease and stroke
13. HIV
1. Immunization and infectious disease
2. Injury and violence prevention
3. Maternal, infant, and child health
4. Medical product safety
5. Mental health and mental disorders
6. Nutrition and overweight
7. Occupational safety and health
8. Oral health
9. Physical activity and fitness
10. Public health infrastructure
11. Respiratory diseases
12. Sexually transmitted diseases
13. Substance abuse
14. Tobacco use
15. Vision and hearing
•
‫خدمات‬ ‫على‬ ‫الحصول‬
‫الجودة‬ ‫عالية‬ ‫صحية‬
•
‫المفاصل‬ ‫التهاب‬
، ‫العظام‬ ‫وهشاشة‬
‫الظهر‬ ‫وظروف‬
‫المزمنة‬
•
‫سرطان‬
•
‫مزمن‬ ‫كلوي‬ ‫فشل‬
•
‫السكري‬ ‫داء‬
•
‫والحاالت‬ ‫اإلعاقة‬
‫الثانوية‬
•
‫التعليمية‬ ‫البرامج‬
‫والمجتمعية‬
•
‫البيئية‬ ‫الصحة‬
•
‫العائلة‬ ‫خطة‬
•
‫الغذاء‬ ‫سالمة‬
•
‫الصحي‬ ‫التواصل‬
•
‫والسكتة‬ ‫القلب‬ ‫أمراض‬
‫الدماغية‬
•
‫المناعة‬ ‫نقص‬ ‫فيروس‬
‫البشرية‬
•
‫المعدية‬ ‫واألمراض‬ ‫التحصين‬
•
‫العنف‬ ‫ومنع‬ ‫اإلصابات‬
•
‫واألطفال‬ ‫والرضع‬ ‫األم‬ ‫صحة‬
•
‫الطبية‬ ‫المنتجات‬ ‫سالمة‬
•
‫النفس‬ ‫واالضطرابات‬ ‫العقلية‬ ‫الصحة‬
‫ية‬
•
‫الوزن‬ ‫وزيادة‬ ‫التغذية‬
•
‫المهنية‬ ‫والصحة‬ ‫السالمة‬
•
‫الفم‬ ‫صحة‬
•
‫البدنية‬ ‫واللياقة‬ ‫البدني‬ ‫النشاط‬
•
‫العامة‬ ‫للصحة‬ ‫التحتية‬ ‫البنية‬
•
‫التنفسي‬ ‫الجهاز‬ ‫أمراض‬
•
‫جنسيا‬ ‫المنقولة‬ ‫األمراض‬
•
‫المخدرات‬ ‫تعاطي‬
•
‫التبغ‬ ‫تعاطي‬
•
‫والسمع‬ ‫الرؤية‬

Internet Resources
• American Evaluation Association resources:
http://www.eval.org/resources.asp
• Centers for Disease Control and Prevention online
evaluation textbook:
http://www.cdc.gov/mmwr/preview/mmwrhtml/
rr4811a1.htm
• Western Michigan University Evaluation Center
resources: http://ec.wmich.edu/resources/
• The Evaluation Exchange periodical:
http://www.gse.harvard.edu/hfrp/eval.html

Relevance of Diversity and
Disparity to Health Programs
Chapter 2
‫الصحية‬ ‫البرامج‬ ‫في‬ ‫والتفاوت‬ ‫التنوع‬ ‫أهمية‬
Translated by
Khaled
Good Luck

Learning Objectives
1. Describe how diversity may be defined and measured.
2. Appreciate the complexity inherent in having diversity
in program targets, program staff, and advisory bodies.
3. Assess the level of cultural competency of staff and
self.
4. Develop a plan of action to address diversity as it
applies to program planning and evaluation.
•
‫من‬ ‫القراء‬ ‫سيتمكن‬ ، ‫الفصل‬ ‫هذا‬ ‫مع‬
:
•
‫وقياسه‬ ‫التنوع‬ ‫تعريف‬ ‫يمكن‬ ‫كيف‬ ‫صف‬
.
•
‫االستشارية‬ ‫والهيئات‬ ‫البرنامج‬ ‫وموظفي‬ ‫البرنامج‬ ‫أهداف‬ ‫في‬ ‫تنوع‬ ‫وجود‬ ‫في‬ ‫الكامن‬ ‫التعقيد‬ ‫تقدير‬
.
•
‫والنفس‬ ‫للموظفين‬ ‫الثقافية‬ ‫الكفاءة‬ ‫مستوى‬ ‫تقييم‬
.
•
‫وتقييمه‬ ‫البرنامج‬ ‫تخطيط‬ ‫على‬ ‫ينطبق‬ ‫كما‬ ‫التنوع‬ ‫لمعالجة‬ ‫عمل‬ ‫خطة‬ ‫وضع‬
.

• Key terminology
• Effects of diversity in the
planning and evaluation
cycle
• Tailoring throughout the
cycle
• Diversity among the health
professions
• The cultural competency
continuum
• Domains of cultural
competence for healthcare
organizations
• Characteristics of effective
coalitions
• Across the pyramid
•
•
‫والتقييم‬ ‫التخطيط‬ ‫دورة‬ ‫في‬ ‫التنوع‬ ‫آثار‬
•
‫الدورة‬ ‫طوال‬ ‫الخياطة‬
•
‫الصحية‬ ‫المهن‬ ‫بين‬ ‫التنوع‬
•
‫الثقافية‬ ‫الكفاءة‬ ‫استمرارية‬
•
‫الصحية‬ ‫الرعاية‬ ‫لمنظمات‬ ‫الثقافية‬ ‫الكفاءة‬ ‫مجاالت‬
•
‫الفعالة‬ ‫التحالفات‬ ‫خصائص‬
•
‫الهرم‬ ‫عبر‬
•

Key Terminology
• Acculturation
• Cultural competency
• Cultural leverage
• Culture
• Culture bound syndromes
• Disparities
• Diversity
• Ethnicity
• Health disparities
• Interventions
• Multicultural
• Nationality
•
‫التثاقف‬
•
‫الثقافية‬ ‫الكفاءة‬
•
‫الثقافي‬ ‫النفوذ‬
•
‫حضاره‬
•
‫متالزمات‬
‫الثقافة‬
•
‫الفوارق‬
•
‫تنوع‬
•
‫العرقي‬ ‫األصل‬
•
‫الصحية‬ ‫الفوارق‬
•
‫التدخالت‬
•
‫الثقافات‬ ‫متعدد‬
•
‫جنسية‬

Effects of Diversity in the Planning and
Evaluation Cycle

Examples of Tailoring Throughout the
Cycle
Stage in the Planning
and Evaluation Cycle
Examples of Tailoring for Cultural and Ethnic
Diversity
Community needs
assessment
Definitions of health and illness; willingness to reveal
needs or wants; self-definition in terms of culture,
race, or ethnicity; health disparities; experience of
disparities in access to or quality of health care
‫الت‬ ‫؛‬ ‫الرغبات‬ ‫أو‬ ‫االحتياجات‬ ‫عن‬ ‫للكشف‬ ‫االستعداد‬ ‫؛‬ ‫والمرض‬ ‫الصحة‬ ‫تعاريف‬
‫عريف‬
‫الفوارق‬ ‫تجربة‬ ‫الصحية؛‬ ‫الفوارق‬ ‫؛‬ ‫العرق‬ ‫أو‬ ‫العرق‬ ‫أو‬ ‫الثقافة‬ ‫حيث‬ ‫من‬ ‫الذاتي‬
‫في‬
‫جودتها‬ ‫أو‬ ‫الصحية‬ ‫الرعاية‬ ‫إلى‬ ‫الوصول‬
Program theory and
development
‫وتطويره‬ ‫البرنامج‬ ‫نظرية‬
Identification of contributing and determinant factors
of health disparities; role of discrimination and
culturally bound health behaviors in the disparities;
culturally acceptable and appropriate interventions
‫والسلوك‬ ‫التمييز‬ ‫دور‬ ‫؛‬ ‫الصحية‬ ‫للتفاوتات‬ ‫والحاسمة‬ ‫المساهمة‬ ‫العوامل‬ ‫تحديد‬
‫يات‬
‫وا‬ ‫ثقافيا‬ ‫المقبولة‬ ‫التدخالت‬ ‫؛‬ ‫التباينات‬ ‫في‬ ً‫ا‬‫ثقافي‬ ‫المرتبطة‬ ‫الصحية‬
‫لمالئمة‬

Examples of Tailoring, Continued
Stage in the Cycle Examples of Tailoring
Process or program
implementation
‫البرنامج‬ ‫تنفيذ‬ ‫أو‬ ‫عملية‬
Culturally and ethnically adjusted program targets;
cultural, racial, and ethnic representations and
appropriateness of materials; modality of
distribution; enticement used
‫والعرقي‬ ‫الثقافي‬ ‫التمثيل‬ ‫؛‬ ً‫ا‬‫وعرقي‬ ً‫ا‬‫ثقافي‬ ‫المعدلة‬ ‫البرنامج‬ ‫أهداف‬
‫واإلث‬
‫ني‬
‫مالءمة‬ ‫ومدى‬
‫المستخدمة‬ ‫إغراء‬ ‫؛‬ ‫التوزيع‬ ‫طريقة‬ ‫؛‬ ‫المواد‬
Program intervention
delivery ‫التدخل‬ ‫برنامج‬ ‫تسليم‬
Intervention type; number and length of sessions
‫الجلسات‬ ‫وطول‬ ‫عدد‬ ‫التدخل‬ ‫نوع‬
Program effect
evaluation
‫البرنامج‬ ‫تأثير‬ ‫تقييم‬
Language(s) of survey questionnaires; culturally
appropriate enticements to participate; access to
culturally and ethnically equivalent “control” groups
‫لغة‬
(
‫لغات‬
)
‫؛‬ ‫للمشاركة‬ ً‫ا‬‫ثقافي‬ ‫المناسبة‬ ‫اإلغراءات‬ ‫؛‬ ‫االستبيان‬ ‫استبيانات‬
‫إلى‬ ‫الوصول‬
‫مجموعات‬
"
‫السيطرة‬
"
ً‫ا‬‫وعرقي‬ ً‫ا‬‫ثقافي‬ ‫المكافئة‬

Diversity among the Health Professions
Discipline Education Primary Focus Licensure /
Certificate
Program Contribution Est. # in
the US
Dentistry Dental doctorate Tooth & gum diagnosis
& treatment
Licensure Oral health knowledge 196,000
Community health
worker
HS or baccalaureate Education, advocacy,
community member
assistance
Certification
varies by state
Shares ethnic, linguistic, SES,
life experiences with
community members
86,000
Dietitian,
nutritionist
Baccalaureate Dietary elements of
health
Licensure &
certificate
Nutrition knowledge,
influence of nutrition on
health
96,000
Health
administration
Master’s Health care org
leadership &
management
Certificate Management &
administration
N/A
Health education Baccalaureate Development of
materials designed to
impart health
knowledge & change
behavior
Certificate Social and behavioral
knowledge
N/A

Diversity among the Health
Professions, Continued
Discipline Education Primary Focus Licensure /
Certificate
Program Contribution Est. # in
the US
Medicine Medical doctorate Differential diagnosis &
treatment of illness
Licensure &
certificate
Medical pathology,
treatment knowledge
863,000
Nursing Baccalaureate
(min. associate’s
degree)
Promotion of health &
well-being
Licensure &
certificate
Integration of behavioral &
medical knowledge
2,529,000
Physical therapy Baccalaureate Body strength &
flexibility restoration &
maintenance
Licensure Focus on enhancing
capability within limitations
198,000
Social work Master’s Licensure Focus on family &
psychological factors
698,000

Features of the Cultural Competency Continuum
Cultural
Destructiveness
Cultural
Incapacity
Cultural
Blindness
Cultural Openness Cultural
Competence
Cultural
Proficiency
Attitude toward
other cultures
Hostility Dislike, separate
but equal
Ambivalence,
treat all alike
Curious, cultural
awareness
Respect and
tolerance,
cultural
sensitivity
Fully
comfortable,
cultural
attunement
Knowledge of
other cultures
Active avoidance
of knowledge
None Little or none Some Fair amount Extensive
Degree of
integration across
cultures
None None None Consideration of
potential benefits
of integration
Some
integration
Fully
multicultural,
fusion of
cultures
Implications for
health programs
of participants at
each stage
Programs address
consequences of
cultural
destructiveness
Need to have
programs
provided to
separate groups
If have
multicultural
elements, may
need to justify
and explain
Can provide
program to
participants from
multiple cultures
but need to
provide
competence info &
role modeling
Can provide
program to
participants
from multiple
cultures with
minimal
adjustments
Can provide
multilingual,
multicultural
interventions
in one
program

Domains of Cultural Competence for
Healthcare Organizations
• Acceptance and respect of the role that values and beliefs play in health and
illness
• Sensitivity toward cultural nonverbal language
• Consumer participation in decision making
• Cultural appropriateness of the environment, materials, and resources
• Staff hiring that reflects the community’s linguistic and cultural diversity
• Health professionals’ self-awareness of their own beliefs, values, and
knowledge about diversity
• Staff cultural competence training and development
•
‫والمرض‬ ‫الصحة‬ ‫في‬ ‫والمعتقدات‬ ‫القيم‬ ‫تلعبه‬ ‫الذي‬ ‫الدور‬ ‫واحترام‬ ‫القبول‬
•
‫اللفظية‬ ‫غير‬ ‫الثقافية‬ ‫اللغة‬ ‫تجاه‬ ‫الحساسية‬
•
‫القرار‬ ‫صنع‬ ‫في‬ ‫المستهلك‬ ‫مشاركة‬
•
‫والموارد‬ ‫والمواد‬ ‫للبيئة‬ ‫الثقافية‬ ‫المالءمة‬
•
‫للمجتمع‬ ‫والثقافي‬ ‫اللغوي‬ ‫التنوع‬ ‫يعكس‬ ‫الموظفين‬ ‫توظيف‬
•
‫بالتنوع‬ ‫ومعرفتهم‬ ‫وقيمهم‬ ‫بمعتقداتهم‬ ‫الصحيين‬ ‫للمهنيين‬ ‫الذاتي‬ ‫الوعي‬
•
‫للموظفين‬ ‫الثقافية‬ ‫الكفاءات‬ ‫وتطوير‬ ‫تدريب‬
‫الثقافية‬ ‫الكفاءة‬ ‫مجاالت‬
‫الصحية‬ ‫الرعاية‬ ‫لمنظمات‬

Characteristics of Effective Coalitions
• Formal rules and procedures
• Inclusive leadership style
• Member participation
• Diverse membership
• Collaboration with agencies
• Group cohesion
‫الفعالة‬ ‫التحالفات‬ ‫خصائص‬
•
‫الرسمية‬ ‫واإلجراءات‬ ‫القواعد‬
•
‫الشامل‬ ‫القيادة‬ ‫أسلوب‬
•
‫األعضاء‬ ‫مشاركة‬
•
‫متنوعة‬ ‫عضوية‬
•
‫الوكاالت‬ ‫مع‬ ‫التعاون‬
•
‫المجموعة‬ ‫تماسك‬

Diversity and Disparities Across the Pyramid
• Direct services level
– Disparities affect individuals’ health status
– Staff cultural competence implementation
– Culturally tailored and targeted programs
• Enabling services level
– Disparities affect families and aggregates
– Interventions tailored to the characteristics and preferences of
the target aggregate
‫الهرم‬ ‫عبر‬ ‫والتفاوتات‬ ‫التنوع‬
•
‫المباشرة‬ ‫الخدمات‬ ‫مستوى‬
•
‫لألفراد‬ ‫الصحية‬ ‫الحالة‬ ‫على‬ ‫التباينات‬ ‫تؤثر‬
•
‫للموظفين‬ ‫الثقافية‬ ‫الكفاءة‬ ‫تنفيذ‬
•
‫وموجهة‬ ‫ثقافيا‬ ‫مصممة‬ ‫برامج‬
•
‫الخدمات‬ ‫مستوى‬ ‫تمكين‬
•
‫والركام‬ ‫العائالت‬ ‫على‬ ‫التباينات‬ ‫تؤثر‬
•
‫المستهدف‬ ‫التجميع‬ ‫وتفضيالت‬ ‫لخصائص‬ ‫المخصصة‬ ‫التدخالت‬

Diversity and Disparities Across the Pyramid, Continued
• Population-based services level
– Disparities reflected in vital statistics or care utilization
– Interventions may be multiple, targeted messages for different subgroups,
or a single, generic message
• Infrastructure level
– Disparities and diversity reflected in health policy and priorities, and
organizational processes and culture
– Interventions address workforce or organizational diversity and cultural
competence
– Legal considerations include accessibility and antidiscrimination laws
•
‫السكان‬ ‫على‬ ‫القائمة‬ ‫الخدمات‬ ‫مستوى‬
•
‫الرعاية‬ ‫استخدام‬ ‫أو‬ ‫الحيوية‬ ‫اإلحصاءات‬ ‫في‬ ‫تنعكس‬ ‫التباينات‬
•
‫واحدة‬ ‫عامة‬ ‫رسالة‬ ‫أو‬ ، ‫مختلفة‬ ‫فرعية‬ ‫لمجموعات‬ ‫مستهدفة‬ ‫ورسائل‬ ، ‫متعددة‬ ‫التدخالت‬ ‫تكون‬ ‫قد‬
•
‫التحتية‬ ‫البنية‬ ‫مستوى‬
•
‫والثقافة‬ ‫التنظيمية‬ ‫والعمليات‬ ، ‫الصحية‬ ‫واألولويات‬ ‫السياسة‬ ‫في‬ ‫ينعكس‬ ‫والتنوع‬ ‫التباين‬
•
‫الثقافية‬ ‫والكفاءة‬ ‫التنظيمي‬ ‫التنوع‬ ‫أو‬ ‫العاملة‬ ‫القوى‬ ‫التدخالت‬ ‫تتناول‬
•
‫التمييز‬ ‫ومكافحة‬ ‫الوصول‬ ‫إمكانية‬ ‫قوانين‬ ‫القانونية‬ ‫االعتبارات‬ ‫تشمل‬

Planning for Health Programs
and Services
Chapter 3
Translated by
Khaled
Good Luck
‫للبرامج‬ ‫التخطيط‬
‫والخدمات‬ ‫الصحية‬

Learning Objectives
1. Describe commonly used program planning models.
2. Articulate the differences among six ethical
frameworks.
3. Understand the value of drawing upon a variety of
approaches to the health planning process.
•
:
•
‫االستخدام‬ ‫الشائعة‬ ‫البرامج‬ ‫تخطيط‬ ‫نماذج‬ ‫وصف‬
.
•
‫أخالقية‬ ‫أطر‬ ‫ستة‬ ‫بين‬ ‫االختالفات‬ ‫توضيح‬
.
•
‫الصحي‬ ‫التخطيط‬ ‫لعملية‬ ‫النهج‬ ‫من‬ ‫متنوعة‬ ‫مجموعة‬ ‫على‬ ‫االعتماد‬ ‫قيمة‬ ‫فهم‬
.

• Key terminology
• Definitions of planning
• Public health planning
models
• Triggering the planning
and evaluation cycle
• Health planning
paradoxes
• Health planning
assumptions
• Challenges to a rational
planning process
• Ethical frameworks
• Approaches to planning
• Planning steps and
stages
•
•
‫التخطيط‬ ‫تعاريف‬
•
‫العامة‬ ‫الصحة‬ ‫تخطيط‬ ‫نماذج‬
•
‫والتقييم‬ ‫التخطيط‬ ‫دورة‬ ‫بدء‬
•
‫الصحي‬ ‫التخطيط‬ ‫مفارقات‬
•
‫الصحي‬ ‫التخطيط‬ ‫افتراضات‬
•
‫الرشيد‬ ‫التخطيط‬ ‫عملية‬ ‫تواجه‬ ‫التي‬ ‫التحديات‬
•
‫األخالقية‬ ‫األطر‬
•
‫للتخطيط‬ ‫نهج‬
•
‫والمراحل‬ ‫الخطوات‬ ‫تخطيط‬
•
•

Key Terminology
• Active protection
• Ambiguity
• Autonomy
• Bioethics
• Consortium
• Control
• Criticality
• Egalitarian
• Interventions
• Needs based
• Pandemic
• Passive protection
• Planning
• Resource sensitive
• Risk
• Uncertainty
• Utilitarian
•
‫النشطة‬ ‫الحماية‬
•
‫التباس‬
•
‫الذاتي‬ ‫الحكم‬
•
‫األحياء‬ ‫علم‬ ‫أخالقيات‬
•
‫التحالف‬
•
‫مراقبة‬
•
‫الحرجية‬
•
‫متساو‬
•
•
‫القائمة‬ ‫االحتياجات‬
•
‫جائحة‬
•
‫السلبية‬ ‫الحماية‬
•
‫تخطيط‬
•
‫للموارد‬ ‫حساسة‬
•
‫خطر‬
•
‫ريبة‬
•
‫المنفعي‬

Definitions of Planning
• Blum (1974): “the deliberate introduction of desired social change
in orderly and acceptable ways”
• Marris (1982): the effort to control uncertainty by taking action
now to secure the future
• Nutt (1984): visible as forecasting, problem solving, programming,
design, policy analysis, and as a response to a problem; creating
change
• Hoch (1994): the popular adoption of democratic reforms in
providing public goods
•
Blum
(
1974
" :)
‫ومقبولة‬ ‫منظمة‬ ‫بطرق‬ ‫المنشود‬ ‫االجتماعي‬ ‫للتغيير‬ ‫المتعمد‬ ‫اإلدخال‬
"
•
Marris
(
1982
:)
‫المستقبل‬ ‫لتأمين‬ ‫اآلن‬ ‫إجراءات‬ ‫اتخاذ‬ ‫خالل‬ ‫من‬ ‫اليقين‬ ‫عدم‬ ‫على‬ ‫للسيطرة‬ ‫جهد‬
•
Nutt
(
1984
:)
‫التغيير‬ ‫خلق‬ ‫؛‬ ‫ما‬ ‫لمشكلة‬ ‫وكاستجابة‬ ‫السياسات‬ ‫وتحليل‬ ‫والتصميم‬ ‫والبرمجة‬ ‫المشكالت‬ ‫وحل‬ ‫التنبؤ‬ ‫مثل‬ ‫مرئي‬
•
Hoch
(
1994
:)
‫العامة‬ ‫المنافع‬ ‫توفير‬ ‫في‬ ‫الديمقراطية‬ ‫لإلصالحات‬ ‫الشعبي‬ ‫التبني‬

Public Health Planning Models
PATCH APEX-PH MAPP CHIP PACE-EH
Current practice No Yes Yes, very widely
used
Yes, but not
widely used
Yes
Developers CDC CDC & NACCHO NACCHO CDC & NACCHO CDC & NACCHO
Appropriate for
which unit/level
City and county Local health
dept.,
community
assessments
Community
health at city,
county, and
state
Local health
dept. and
county
City, county,
and state
Distinguishing
emphasis
Health
promotion
programs &
community
capacity
building
Core PH
functions
(assessment,
assurance,
policy
development)
Strategic
planning,
community
involvement in
the process
Local health
dept.
performance,
data used for
planning
Environmental
health, legal
advocacy

Triggering the Planning and Evaluation Cycle

Common Triggers
• Funding announcement or renewal
• Staff interest in a particular issue
• Strategic planning process
• News coverage
• Local activism
• Information generated from an evaluation
•
‫التجديد‬ ‫أو‬ ‫التمويل‬ ‫إعالن‬
•
‫معينة‬ ‫قضية‬ ‫في‬ ‫الموظفين‬ ‫مصلحة‬
•
‫االستراتيجي‬ ‫التخطيط‬ ‫عملية‬
•
‫األخبار‬ ‫تغطية‬
•
‫المحلي‬ ‫النشاط‬
•
‫التقييم‬ ‫من‬ ‫الناتجة‬ ‫المعلومات‬

Health Planning Paradoxes
• Forces shaping health problems also shape the planning
process
• Prosperity may lead to poor health outcomes
• Easier solutions may be less acceptable
• Health professionals, as planners, may not have the best
ideas for addressing the problem
• Biases toward success blind planners to unanticipated
consequences
•
‫القوى‬
‫تشكل‬
‫أيضا‬ ‫الصحية‬ ‫المشاكل‬
‫تشكل‬
‫التخطيط‬ ‫عملية‬
•
‫الرفاهيه‬
‫سيئة‬ ‫صحية‬ ‫نتائج‬ ‫إلى‬ ‫يؤدي‬ ‫قد‬
•
‫قبوال‬ ‫أقل‬ ‫تكون‬ ‫قد‬ ‫أسهل‬ ‫حلول‬
•
‫المشكلة‬ ‫لمعالجة‬ ‫األفكار‬ ‫أفضل‬ ، ‫كمخططين‬ ، ‫الصحيين‬ ‫المهنيين‬ ‫لدى‬ ‫يكون‬ ‫ال‬ ‫قد‬
•
‫متوقعة‬ ‫غير‬ ‫عواقب‬ ‫إلى‬ ‫األعمى‬ ‫المخططين‬ ‫نجاح‬ ‫نحو‬ ‫التحيز‬
‫الصحي‬ ‫التخطيط‬ ‫مفاهيم‬

Health Planning Assumptions
• A solution can be identified and provided
• Available resources are sufficient to implement the
solution
• Planning leads to the necessary resource allocation
• Planning is a rational process
•
‫وتوفيره‬ ‫الحل‬ ‫تحديد‬ ‫يمكن‬
•
‫الحل‬ ‫لتنفيذ‬ ‫كافية‬ ‫المتاحة‬ ‫الموارد‬
•
‫الالزمة‬ ‫الموارد‬ ‫تخصيص‬ ‫إلى‬ ‫يؤدي‬ ‫التخطيط‬
•
‫عقالنية‬ ‫عملية‬ ‫التخطيط‬

Challenges to a Rational Planning
Process
• Ambiguity and uncertainty
• Conflict
• Risk
• Control
‫التخطيط‬ ‫عملية‬ ‫تواجه‬ ‫التي‬ ‫التحديات‬
‫العقالني‬
•
‫اليقين‬ ‫وعدم‬ ‫الغموض‬
•
‫نزاع‬
•
‫خطر‬
•
‫مراقبة‬

Ethical Frameworks for Planning
Health Programs
Approach Principle Health Application Health Examples
Autonomy Personal right to self-
determination and choice
Individual choice takes
priority, avoidance of coercion
Pro-choice, living will
Criticality (contractarian) The worst off benefit the
most
Greatest problem, severest
health risk
WIC, Medicaid, SCHIP
Egalitarian Everyone valued equally,
minimize disparities
Hardest to reach, most
marginalized, most in need
Healthy People 2010 goals
Needs based Equal opportunity to meet
own needs
Future oriented, strategies to
promote health and prevent
illness
Early childhood intervention
programs
Resource sensitive Resources are scarce Cost-effectiveness Oregon Health Plan
Utilitarian Greatest good for the
greatest number; ends
justify means
Collective benefits outweigh
individual choices
Required immunization

Six Approaches to Planning
Approach Assumptions Consequences of Use Public Health Example
Incremental
‫تدريجي‬
Not feasible to do more than
small portions at a time; the
parts are greater than the whole
‫صغير‬ ‫أجزاء‬ ‫من‬ ‫بأكثر‬ ‫القيام‬ ‫الممكن‬ ‫من‬ ‫ليس‬
‫ة‬
‫الكل‬ ‫من‬ ‫أكبر‬ ‫األجزاء‬ ‫؛‬ ‫واحد‬ ‫وقت‬ ‫في‬
Can be done quickly; results in
plans that may be redundant or
leave gaps; no guarantee that
parts will build on one another
‫بسرعة‬ ‫به‬ ‫القيام‬ ‫يمكن‬
.
‫تك‬ ‫قد‬ ‫خطط‬ ‫عنه‬ ‫ينتج‬
‫ون‬
‫يوجد‬ ‫ال‬ ‫؛‬ ‫فجوات‬ ‫تترك‬ ‫أو‬ ‫الحاجة‬ ‫عن‬ ‫زائدة‬
‫البعض‬ ‫بعضها‬ ‫على‬ ‫ستبني‬ ‫األجزاء‬ ‫بأن‬ ‫ضمان‬
Specific programs
implemented that reflect
discrete, categorical funding,
despite potential overlap with
existing similar programs
‫ا‬
‫تمويال‬ ‫تعكس‬ ‫تنفيذها‬ ‫تم‬ ‫محددة‬ ‫برامج‬
‫احتمال‬ ‫من‬ ‫الرغم‬ ‫على‬ ، ‫ا‬‫ا‬‫ع‬‫وقاط‬ ‫ا‬‫ال‬‫منفص‬
‫الموجودة‬ ‫المماثلة‬ ‫البرامج‬ ‫مع‬ ‫تداخلها‬
Apolitical
‫سياسية‬ ‫غير‬
Options are known;
technicalizes the problem; the
means to the ends are known;
can anticipate all caveats
‫الغا‬ ‫وسائل‬ ‫المشكلة‬ ‫تقني‬ ‫معروفة‬ ‫الخيارات‬
‫يات‬
‫معروفة‬
.
‫المحاذير‬ ‫جميع‬ ‫توقع‬ ‫يمكن‬
Plans may fail because of
unforeseen or unaccounted-for
factors
‫مت‬ ‫غير‬ ‫عوامل‬ ‫بسبب‬ ‫الخطط‬ ‫تفشل‬ ‫قد‬
‫وقعة‬
‫معروفة‬ ‫غير‬ ‫أو‬
Evidence-based practice
‫األدلة‬ ‫على‬ ‫القائمة‬ ‫الممارسة‬
Advocacy An external expert can
accurately speak for those with
less power
‫بدق‬ ‫يتحدث‬ ‫أن‬ ‫الخارجي‬ ‫للخبير‬ ‫يمكن‬
‫عن‬ ‫ة‬
‫أقل‬ ‫طاقة‬ ‫لديهم‬ ‫من‬
Experts may not accurately speak
for others; media attention is
likely to focus on the
spokesperson rather than the
issue
‫من‬ ‫؛‬ ‫اآلخرين‬ ‫عن‬ ‫بدقة‬ ‫التحدث‬ ‫للخبراء‬ ‫يجوز‬ ‫ال‬
‫على‬ ‫اإلعالم‬ ‫وسائل‬ ‫اهتمام‬ ‫يركز‬ ‫أن‬ ‫المرجح‬
‫القضية‬ ‫من‬ ‫ا‬‫ال‬‫بد‬ ‫الرسمي‬ ‫المتحدث‬
Environmental activists
‫البيئة‬ ‫نشطاء‬

Six Approaches to Planning, Continued
Approach Assumptions Consequences of Use Public Health Example
Communication
action
‫االتصاالت‬ ‫عمل‬
Language is powerful; those
with the problem have the
capability to enact the
solution
‫مش‬ ‫لديهم‬ ‫الذين‬ ‫أولئك‬ ‫قوية‬ ‫اللغة‬
‫كلة‬
‫الحل‬ ‫سن‬ ‫على‬ ‫القدرة‬ ‫لديهم‬
An increased sense of confidence
and an increased ability to solve
one’s own problems; less
potential for conflict
‫على‬ ‫القدرة‬ ‫وزيادة‬ ‫بالثقة‬ ‫الشعور‬ ‫زيادة‬
‫حل‬
‫للصراع‬ ‫احتمال‬ ‫أقل‬ ‫؛‬ ‫الفرد‬ ‫مشاكل‬
Community coalitions that
take on a program or become
not-for-profit organizations
‫أو‬ ‫ا‬ً‫ج‬‫برنام‬ ‫تأخذ‬ ‫مجتمعية‬ ‫تحالفات‬
‫ربحية‬ ‫غير‬ ‫منظمات‬ ‫تصبح‬
Comprehensive
rational
‫عقالني‬ ‫شامل‬
System feedback loops are
contextual and can be
known; rational choices are
preferred
‫سياق‬ ‫هي‬ ‫النظام‬ ‫مالحظات‬ ‫حلقات‬
‫ويفض‬ ‫؛‬ ‫معروفة‬ ‫تكون‬ ‫أن‬ ‫ويمكن‬
‫ل‬
‫العقالنية‬ ‫الخيارات‬
Takes considerable time and
effort to implement; likely to
have more dissent to overcome;
results in an encompassing,
intertwined set of actions
‫؛‬ ‫للتنفيذ‬ ‫والجهد‬ ‫الوقت‬ ‫من‬ ‫الكثير‬ ‫يستغرق‬
‫من‬
‫المعارضة‬ ‫من‬ ‫المزيد‬ ‫هناك‬ ‫يكون‬ ‫أن‬ ‫المرجح‬
‫ومتشابك‬ ‫شاملة‬ ‫مجموعة‬ ‫ينتج‬ ‫؛‬ ‫عليها‬ ‫للتغلب‬
‫ة‬
‫اإلجراءات‬ ‫من‬
Community-focused initiatives
‫المجتمع‬ ‫على‬ ‫تركز‬ ‫التي‬ ‫المبادرات‬
Strategic planning Can anticipate and predict
the future; stability is more
pervasive than change
‫المستقبل‬ ‫وتوقع‬ ‫توقع‬ ‫يمكن‬
.
‫التغ‬ ‫من‬ ‫انتشارا‬ ‫أكثر‬ ‫االستقرار‬
‫يير‬
Lacks flexibility to respond to
emerging issues; a costly process
to arrive at a plan
‫للق‬ ‫االستجابة‬ ‫في‬ ‫المرونة‬ ‫إلى‬ ‫يفتقر‬
‫ضايا‬
‫خط‬ ‫إلى‬ ‫للوصول‬ ‫مكلفة‬ ‫عملية‬ ‫؛‬ ‫الناشئة‬
‫ة‬
Healthy People series; state
two- to five-year plans; Title V
two-year plans
‫وضع‬ ‫؛‬ ‫األصحاء‬ ‫األشخاص‬ ‫سلسلة‬
‫؛‬ ‫سنوات‬ ‫خمس‬ ‫إلى‬ ‫سنتان‬ ‫مدتها‬ ‫خطط‬
‫عامين‬ ‫لمدة‬ ‫خطط‬ ‫الخامس‬ ‫الباب‬

Planning Steps and Stages
• Team formation and development
• Vision creation
• Investigation
• Prioritization
• Decision
• Implementation and continuation
•
‫وتطوير‬ ‫فريق‬ ‫تشكيل‬
•
‫الرؤية‬ ‫خلق‬
•
‫تحقيق‬
•
‫األولويات‬ ‫ترتيب‬
•
‫قرار‬
•
‫واالستمرار‬ ‫التنفيذ‬

Planning Across the Pyramid
– May be clinical or affect small aggregates
– Prioritizes among available services for
aggregates
– Creates new services for high-priority problems
– Involves a wider body of constituents
•
•
‫الصغيرة‬ ‫المجاميع‬ ‫على‬ ‫تؤثر‬ ‫أو‬ ‫سريرية‬ ‫تكون‬ ‫قد‬
•
•
‫للمجاميع‬ ‫المتاحة‬ ‫الخدمات‬ ‫بين‬ ‫أولويات‬
•
‫العالية‬ ‫األولوية‬ ‫ذات‬ ‫للمشاكل‬ ‫جديدة‬ ‫خدمات‬ ‫يخلق‬
•
‫المكونات‬ ‫من‬ ‫أوسع‬ ‫مجموعة‬ ‫يتضمن‬

Planning Across the Pyramid, Continued
– Most evident in state health plans
– Requires a wider array of stakeholders
– Issues of resource allocation, planning for the
planning, and data collection for the planning
– May focus on health personnel and resource planning
•
•
‫الصحية‬ ‫الدولة‬ ‫خطط‬ ‫في‬ ‫وضوحا‬ ‫األكثر‬
•
‫المصلحة‬ ‫أصحاب‬ ‫من‬ ‫واسعة‬ ‫مجموعة‬ ‫يتطلب‬
•
•
‫للتخطيط‬ ‫البيانات‬ ‫وجمع‬ ‫للتخطيط‬ ‫والتخطيط‬ ‫الموارد‬ ‫تخصيص‬ ‫قضايا‬
•
‫الموارد‬ ‫وتخطيط‬ ‫الصحة‬ ‫مجال‬ ‫في‬ ‫العاملين‬ ‫على‬ ‫تركز‬ ‫قد‬

Community Health Assessment for
Program Planning
Chapter 4
‫البرامج‬ ‫لتخطيط‬ ‫المجتمع‬ ‫صحة‬ ‫تقييم‬
Translated by
Khaled
Good Luck

Learning Objectives
1. Articulate the benefits of incorporating components of various
approaches to conducting a needs assessment.
2. Critique a plan for conducting a community health assessment.
3. Appreciate the interaction among different types of needs.
•
:
•
‫االحتياجات‬ ‫تقييم‬ ‫إلجراء‬ ‫المختلفة‬ ‫النهج‬ ‫مكونات‬ ‫دمج‬ ‫فوائد‬ ‫توضيح‬
.
•
‫المجتمع‬ ‫صحة‬ ‫تقييم‬ ‫إلجراء‬ ‫خطة‬ ‫نقد‬
.
•
‫االحتياجات‬ ‫من‬ ‫مختلفة‬ ‫أنواع‬ ‫بين‬ ‫التفاعل‬ ‫نقدر‬
.

• Key terminology
• Elements of community
• Community as context and
recipient
• Connections among program,
agency, and community
• Community based, focused, and
driven
• Types of needs
models
• Epidemiology rates
• Assessment types
• Assessment steps
• Principles of data collection
•
•
‫المجتمع‬ ‫عناصر‬
•
‫ومتلقي‬ ‫كسياق‬ ‫المجتمع‬
•
‫والمجتمع‬ ‫والوكالة‬ ‫البرنامج‬ ‫بين‬ ‫االتصاالت‬
•
‫ويقودها‬ ، ‫تركز‬ ، ‫المجتمع‬ ‫أساس‬ ‫على‬
•
‫االحتياجات‬ ‫أنواع‬
•
‫المجتمع‬ ‫احتياجات‬ ‫تقييم‬ ‫نماذج‬
•
‫الوبائيات‬ ‫معدالت‬
•
‫التقييم‬ ‫أنواع‬
•
‫التقييم‬ ‫خطوات‬
•
‫البيانات‬ ‫جمع‬ ‫مبادئ‬
•
•

Key Terminology
• Aggregate
• Community
• Community based
• Community competence
• Community driven
• Community focused
• Emic
• Etic
• Expressed need
• High risk
• Needs assessment
• Normative need
• Perceived need
• Population at risk
• Recipient audience
• Relative need
• Reliability
• Target audience
• Target population
• Validity
•
‫مجموع‬
•
‫اجتماعي‬ ‫تواصل‬
•
‫القائم‬ ‫المجتمع‬
•
‫المجتمع‬ ‫كفاءة‬
•
‫مدفوعة‬ ‫المجتمع‬
•
‫المجتمع‬ ‫ركز‬
•
EMIC
•
ETIC
•
‫معبرة‬ ‫حاجة‬
•
‫عالية‬ ‫مخاطرة‬
•
‫االحتياجات‬ ‫تقييم‬
•
‫معيارية‬ ‫حاجة‬
•
‫المدركة‬ ‫الحاجة‬
•
‫خطر‬ ‫في‬ ‫السكان‬
•
‫مستلم‬ ‫جمهور‬
•
‫النسبية‬ ‫الحاجة‬
•
‫الموثوقية‬
•
‫المستهدف‬ ‫الجمهور‬
•
‫المستهدفين‬ ‫السكان‬
•
‫صالحية‬

Three Elements of Community
Element Examples
People Values, beliefs, behaviors, size, membership, demographic
characteristics, social and economic status, sense of power or
influence, sense of belonging
Place Geography, boundaries, housing, industry, air, water, land, virtual
presence
Interaction Communication, familial, education, religious based, political,
recreational, virtual

Community as the Context and
Recipient of Health Programs
• The community must be defined to determine the health
program recipients
• The community is also the context
– Sociopolitical and economic factors influence program planning and
implementation
– Community members participating in the planning process are the
immediate context of the intervention
‫الصحية‬ ‫للبرامج‬ ‫والمستلم‬ ‫السياق‬ ‫باعتباره‬ ‫المجتمع‬
•
‫الصحي‬ ‫البرنامج‬ ‫مستلمي‬ ‫لتحديد‬ ‫المجتمع‬ ‫تحديد‬ ‫يجب‬
•
‫السياق‬ ‫أيضا‬ ‫هو‬ ‫المجتمع‬
•
‫وتنفيذها‬ ‫البرامج‬ ‫تخطيط‬ ‫على‬ ‫واالقتصادية‬ ‫والسياسية‬ ‫االجتماعية‬ ‫العوامل‬ ‫تؤثر‬
•
‫للتدخل‬ ‫المباشر‬ ‫السياق‬ ‫هم‬ ‫التخطيط‬ ‫عملية‬ ‫في‬ ‫المشاركين‬ ‫المجتمع‬ ‫أعضاء‬

Connections Among Program, Agency, and
Community

Community Based, Focused, and
Driven
• Not mutually exclusive
• Degree of the three factors varies
• The designations can also apply to families, populations, and
other aggregates
•
‫بعضا‬ ‫بعضها‬ ‫يستبعد‬ ‫ال‬
•
‫تختلف‬ ‫الثالثة‬ ‫العوامل‬ ‫درجة‬
•
‫األخرى‬ ‫والمجاميع‬ ‫والسكان‬ ‫العائالت‬ ‫على‬ ‫ًا‬‫ض‬‫أي‬ ‫التعيينات‬ ‫تنطبق‬ ‫أن‬ ‫يمكن‬

Venn Diagram of Community Based,
Focused, and Driven

Types of Needs
• Expressed
• Normative
• Perceived
• Relative
•
‫دقيق‬ ‫أو‬ ‫تعبيري‬
•
‫معياري‬
•
‫محسوس‬
-
‫ملموس‬
•
‫نسبيا‬

Community Needs Assessment Models
Epidemio-
logical
Public Health Social Asset Rapid
Population
assessed
Populations State and
communities
Populations,
selected
aggregates
Community,
neighborhoods
Community,
neighborhoods
Data sources Registries,
national
probability sample
surveys, existing
national
databases
State and local
agencies, vital
records
Individual or
national surveys
Agency rosters,
focus groups,
maps
Windshield
surveys, existing
data, interviews
Examples NHIS, HCUP APEX-PH, PATCH,
MAPP
U.S. Census Assets-Based
Community
Development
Institute
RAR, RARE

Community Needs Assessment
Models, Continued
Epidemio-
logical
Public Health Social Asset Rapid
Types of needs
assessed
Normative, expressed,
and relative can be
estimated
Normative and
relative can be
estimated
Relative can be
estimated; perceived
are directly
determined
Perceived needs and
strengths
Normative and
perceived
Advantages Statistically sound and
generalizable
Administratively
sound; includes focus
on constituent
concerns
Statistically sound;
provides information
on factors
contributing to
health problem
Existing resources
are identified
Quickly completed
and provides basic
information
Disadvan-tages No info on perceived
needs; local variations
not captured
Relies on other data
sources; perceived
needs not directly
captured
Doesn’t directly
measure extent of
the health problem
Doesn’t measure
extent of the health
problem
Doesn’t measure
extent of the health
problem; may miss
problems or causes

Numerators and Denominators for
Selected Epidemiology Rates
Rate Numerator Denominator Per
Crude death rate
‫اإلجمالي‬ ‫الوفيات‬ ‫معدل‬
Total # of deaths in a given period Total population 1000
Cause-specific death rate # of deaths due to a given cause in a
given period
Total population 100, 000
Birth rate # of live births in a given period Total population 1000
Fetal death rate # of fetal deaths of > 28 weeks
gestation in a given period
# of fetal deaths of > 28 weeks gestation
+ # of live births in a given period
1000
Neonatal death rate # of infant deaths of < 28 days old in a
given period
# of live births in a given period 1000
Infant mortality rate # of infant deaths of < 1 year old in 1
year
# of live births in 1 year 1000

Types of Assessments
• Organizational
– What is the capability and willingness of the organization to provide
the health program?
• Marketing
– What will draw the target audience into the program?
• Needs
– What health problems exist, and to what extent?
•
‫التنظيمية‬
•
‫الصحي؟‬ ‫البرنامج‬ ‫لتقديم‬ ‫المنظمة‬ ‫واستعداد‬ ‫قدرة‬ ‫هي‬ ‫ما‬
•
‫تسويق‬
•
‫البرنامج؟‬ ‫إلى‬ ‫المستهدف‬ ‫الجمهور‬ ‫سيجذب‬ ‫الذي‬ ‫ما‬
•
‫االحتياجات‬
•
‫مدى؟‬ ‫أي‬ ‫وإلى‬ ، ‫الموجودة‬ ‫الصحية‬ ‫المشاكل‬ ‫هي‬ ‫ما‬

Types of Assessments, Continued
• Community health
– What are the health problems, and which resources are available to address those
health problems?
• Rapid
– What are the most immediate and pressing needs that can be addressed with readily
available resources?
• Workforce
– Which human resources exist at which level of expertise to address the health needs?
•
‫المجتمع‬ ‫صحة‬
•
‫الصحية؟‬ ‫المشاكل‬ ‫تلك‬ ‫لمعالجة‬ ‫المتاحة‬ ‫الموارد‬ ‫هي‬ ‫وما‬ ، ‫الصحية‬ ‫المشاكل‬ ‫هي‬ ‫ما‬
•
‫سريعون‬
•
‫بسهولة؟‬ ‫المتاحة‬ ‫الموارد‬ ‫خالل‬ ‫من‬ ‫تلبيتها‬ ‫يمكن‬ ‫والتي‬ ‫ا‬ً‫ح‬‫إلحا‬ ‫االحتياجات‬ ‫أكثر‬ ‫هي‬ ‫ما‬
•
‫العاملة‬ ‫القوى‬
•
‫الصحية؟‬ ‫االحتياجات‬ ‫لتلبية‬ ‫الخبرة‬ ‫من‬ ‫مستوى‬ ‫أي‬ ‫في‬ ‫الموجودة‬ ‫البشرية‬ ‫الموارد‬ ‫هي‬ ‫ما‬

Steps in Conducting the Assessment
• Involve community members
• Define the population
• Define the problem to be assessed
• Anticipate data-related and methodological issues
•
‫المجتمع‬ ‫أعضاء‬ ‫إشراك‬
•
‫السكان‬ ‫تحديد‬
•
‫تقييمها‬ ‫المراد‬ ‫المشكلة‬ ‫تحديد‬
•
‫والمنهجية‬ ‫بالبيانات‬ ‫المتعلقة‬ ‫القضايا‬ ‫توقع‬
‫التقييم‬ ‫إجراء‬ ‫خطوات‬

Principles of Data Collection for a Community Health Assessment
• Collect data from more than one source
• Involve community members in all aspects
• Give full disclosure and get informed consent from those providing data
• Collect data from unlikely sources
• Be as scientifically rigorous as possible
‫المجتمع‬ ‫صحة‬ ‫لتقييم‬ ‫البيانات‬ ‫جمع‬ ‫مبادئ‬
•
‫مصدر‬ ‫من‬ ‫أكثر‬ ‫من‬ ‫البيانات‬ ‫جمع‬
•
‫الجوانب‬ ‫جميع‬ ‫في‬ ‫المجتمع‬ ‫أعضاء‬ ‫إشراك‬
•
‫البيانات‬ ‫يقدمون‬ ‫الذين‬ ‫أولئك‬ ‫من‬ ‫مستنيرة‬ ‫موافقة‬ ‫على‬ ‫والحصول‬ ‫الكامل‬ ‫الكشف‬ ‫إعطاء‬
•
‫مرجحة‬ ‫غير‬ ‫مصادر‬ ‫من‬ ‫البيانات‬ ‫جمع‬
•
‫ممكن‬ ‫علميا‬ ‫صارمة‬ ‫تكون‬ ‫أن‬

Needs Assessment Across the Pyramid
– Focus on a specific medical problem
– Community providers’ diagnostic/treatment capabilities
– Social context for those with the health problem
– Infrastructure capabilities
•
•
‫محددة‬ ‫طبية‬ ‫مشكلة‬ ‫على‬ ‫التركيز‬
•
‫التشخيص‬ ‫قدرات‬
/
‫المجتمعية‬ ‫الخدمات‬ ‫لمقدمي‬ ‫العالج‬
•
•
‫صحية‬ ‫مشكلة‬ ‫من‬ ‫يعانون‬ ‫الذين‬ ‫ألولئك‬ ‫االجتماعي‬ ‫السياق‬
•
‫التحتية‬ ‫البنية‬ ‫قدرات‬

Needs Assessment Across the Pyramid,
Continued
– Epidemiological; magnitude of various problems as well as antecedent and
contributing factors
– Community health assessment to understand problems, social context, and assets
– Organizational assessment to identify resources, capabilities, and mission
•
•
‫وبائية‬
.
‫واإلسهامات‬ ‫السابقة‬ ‫العوامل‬ ‫وكذلك‬ ‫المختلفة‬ ‫المشاكل‬ ‫حجم‬
•
•
‫واألصول‬ ‫االجتماعي‬ ‫والسياق‬ ‫المشكالت‬ ‫لفهم‬ ‫المجتمع‬ ‫صحة‬ ‫تقييم‬
•
‫والمهمة‬ ‫والقدرات‬ ‫الموارد‬ ‫لتحديد‬ ‫التنظيمي‬ ‫التقييم‬

Characterizing and Defining the
Health Problem
Chapter 5
‫وتعريفها‬ ‫الصحية‬ ‫المشكلة‬ ‫وصف‬
Translated by
Khaled
Good Luck

Learning Objectives
1. Establish program priorities based on an existing methodology.
2. Critique techniques used in public health for establishing program priorities.
3. Develop a causal statement of health problems chosen for attention.
4. Apply basic epidemiologic concepts to gain a quantification of health
problems.
5. Articulate the benefits and limitations of the main approaches to prioritizing
health problems.
•
:
•
‫الحالية‬ ‫المنهجية‬ ‫أساس‬ ‫على‬ ‫البرنامج‬ ‫أولويات‬ ‫تحديد‬
.
•
‫البرنامج‬ ‫أولويات‬ ‫لتحديد‬ ‫العامة‬ ‫الصحة‬ ‫في‬ ‫المستخدمة‬ ‫النقد‬ ‫تقنيات‬
.
•
‫لالنتباه‬ ‫المختارة‬ ‫الصحية‬ ‫للمشاكل‬ ‫سببي‬ ‫بيان‬ ‫تطوير‬
.
•
‫الصحية‬ ‫للمشاكل‬ ‫تقدير‬ ‫على‬ ‫للحصول‬ ‫األساسية‬ ‫الوبائية‬ ‫المفاهيم‬ ‫تطبيق‬
.
•
‫الصحية‬ ‫المشاكل‬ ‫أولويات‬ ‫لتحديد‬ ‫الرئيسية‬ ‫النهج‬ ‫وقيود‬ ‫فوائد‬ ‫توضيح‬
.

• Key terminology
• Data sources for
community health
assessment
• Information categories
for health planning
• Haddon’s typology
• Quality of life
• Descriptive statistics
• Small numbers and
small areas
• Elements of a causal
theory
• Relationship of problem
definition to program
design and evaluation
• Prioritizing health
problems
• BPRS criteria

Key Terminology
• Acceptability
• Causal factors
• Confidence interval
• Denominator
• Economic
• Incidence
• Legality
• Mediating factors
• Moderating factors
• Numerator
• Odds ratio
• Prevalence
• Propriety
• Relative risk
• Required antecedent
factors
• Resources
• Sensitivity
• Specificity
• Synthetic estimation
• Years of life lost
• Years of potential life lost
•
‫المقبولية‬
•
‫المسببة‬ ‫العوامل‬
•
‫الثقة‬ ‫فاصل‬
•
‫المقام‬
-
‫مشتركة‬ ‫صفة‬
-
‫حالة‬
•
‫الوصفي‬ ‫اإلحصاء‬
•
‫اقتصادي‬
•
‫سقوط‬
•
‫شرعية‬
•
‫الوساطة‬ ‫عوامل‬
•
‫المعتدلة‬ ‫العوامل‬
•
‫عداد‬
•
‫االحتماالت‬ ‫نسبة‬
•
‫انتشار‬
•
‫استقامة‬
•
‫نسبي‬ ‫خطر‬
•
‫المطلوبة‬ ‫السابقة‬ ‫العوامل‬
•
‫مصادر‬
•
‫حساسية‬
•
‫النوعية‬
•
‫االصطناعية‬ ‫تقدير‬
•
‫المفقودة‬ ‫الحياة‬ ‫سنوات‬
•
‫المفقو‬ ‫المحتملة‬ ‫الحياة‬ ‫من‬ ‫سنوات‬
‫دة‬

Data Sources for Community Health Assessment
• Archival data
• Public data
• Proprietary data
• Primary data
• Observational data
• Published literature
• Other data sources (i.e., “beyond the street lamp”)
‫المجتمع‬ ‫صحة‬ ‫لتقييم‬ ‫البيانات‬ ‫مصادر‬
•
‫األرشفة‬ ‫بيانات‬
•
‫العامة‬ ‫البيانات‬
•
‫الملكية‬ ‫بيانات‬
•
‫األولية‬ ‫البيانات‬
•
‫الرصد‬ ‫بيانات‬
•
‫المنشور‬ ‫األدب‬
•
‫األخرى‬ ‫البيانات‬ ‫مصادر‬
(
‫مثل‬
"
‫الشارع‬ ‫مصباح‬ ‫وراء‬ ‫ما‬
)"

Information Categories for Health
Planning
• Magnitude of the problem
– Incidence and prevalence
• Dynamics leading to the problem
– Agent, host, problem
• Population characteristics
• Attitudes and behaviors
‫الصحي‬ ‫للتخطيط‬ ‫المعلومات‬ ‫فئات‬
•
‫المشكلة‬ ‫حجم‬
•
‫واالنتشار‬ ‫اإلصابة‬
•
‫ديناميات‬
‫المشكلة‬ ‫إلى‬ ‫تؤدي‬
•
‫المشكلة‬ ، ‫المضيف‬ ، ‫الوكيل‬
•
‫السكانية‬ ‫الخصائص‬
•
‫والسلوكيات‬ ‫المواقف‬

Haddon’s Typology for Analyzing an Event, Using Highway Safety as an Example
Agent factors Human factors Physical
environment
Sociocultural
environment
Health system
environment
Pre-event Latency Genetic
makeup,
motivation,
knowledge
Proximity,
transportation,
availability of
agent (i.e., alcohol
or drugs)
Norms, policy and
laws, cultural
beliefs about
causes, family
dynamics
Accessibility,
availability,
acceptability
Event
(behavior)
Virulence,
addictiveness,
difficulty of
behavior
Susceptibility,
vulnerability,
hardiness,
reaction
Force Peer pressure Iatrogenic
factors,
treatments
Post-event Resistance to
treatment
Motivation,
resilience, time
for recovery
Proximity,
availability of
agent
Meaning of event,
attribution of
causality, sick role
Resources and
services,
treatment
options,
emergency
response
Source: Adapted from Haddon, W., Jr. (1972). A logical framework for categorizing highway
safety phenomena and activity. Journal of Trauma, 12, 193–207. Cited in D. C. Grossman
(2000), The history of injury control andthe epidemiology of child and adolescent injuries.
Future of Children, 10(1), 23–52.
‫كمثال‬ ‫السريعة‬ ‫الطرق‬ ‫على‬ ‫السالمة‬ ‫واستخدام‬ ، ‫ما‬ ‫حدث‬ ‫لتحليل‬ ‫هادون‬ ‫نموذج‬

Quality-of-Life Acronyms and Definitions
Acronym Spelled-out form Definition
QALYs Quality-adjusted life years # years of life expected at a given level of health
and well-being
DALYs Disability-adjusted life years # years of life lost from living with a given level of
morbidity or disability
YLL Years of life lost # years a person is estimated to have remained
alive if the disease hadn’t occurred
YPLL Years of potential life lost # years of life lost before a specific age (often 65
or 75); assigns additional value to deaths at
earlier ages
HYE Healthy years equivalent # years in perfect health considered equivalent to
a particular health state
YHL Years of healthy life # healthy years of life lived, adjusted for health
status
‫الحياة‬ ‫نوعية‬
‫االختصارات‬
‫والتعاريف‬

Descriptive Statistics
• Frequency
• Mean
• Standard deviation, variance
• Odds ratio
• Relative risk
• Confidence interval
• Synthetic estimate
• Mapping
•
‫تكرر‬
•
‫تعني‬
•
‫التباين‬ ، ‫المعياري‬ ‫االنحراف‬
•
‫االحتماالت‬ ‫نسبة‬
•
‫نسبي‬ ‫خطر‬
•
‫الثقة‬ ‫فاصل‬
•
‫االصطناعية‬ ‫تقدير‬
•
‫الخرائط‬ ‫رسم‬

Small Numbers and Small Areas
• Small denominators mean that small changes
in the numerator lead to large changes in the
rate or proportion
• One strategy is to pool multiple years of data
•
‫ال‬ ‫في‬ ‫كبيرة‬ ‫تغييرات‬ ‫إلى‬ ‫تؤدي‬ ‫البسط‬ ‫في‬ ‫الصغيرة‬ ‫التغييرات‬ ‫أن‬ ‫تعني‬ ‫الصغيرة‬ ‫القواسم‬
‫النسبة‬ ‫أو‬ ‫معدل‬
•
‫البيانات‬ ‫من‬ ‫سنوات‬ ‫عدة‬ ‫تجميع‬ ‫في‬ ‫االستراتيجيات‬ ‫إحدى‬ ‫تتمثل‬

Elements of a Causal Theory
• Existing factors
• Causes
• Mediating factors
• Moderating factors
‫عناصر‬
‫السببية‬ ‫النظرية‬
•
‫الحالية‬ ‫العوامل‬
•
‫األسباب‬
•
‫الوساطة‬ ‫عوامل‬
•
‫المعتدلة‬ ‫العوامل‬

Generic Model of a Causal Theory

Example Elements for Two Health
Problems
Required
antecedent
factors
Moderating
factors
Key causal
factors
Mediating
factors
Health problem Health impact
Age, existing
health
conditions,
pathogens in
environment
Adult
immunization
knowledge,
media attention,
medical care
quality
Motivation to be
vaccinated, fear
of the
communicable
disease,
perceived
susceptibility
Vaccine supply
and distribution,
cost
Vaccination Preventable
hospitalizations
Developmental
stage, local
history of
violence, local
lack of jobs, state
gun laws
Parental
supervision,
school
antiviolence
program,
community
action
Lack of conflict
resolution skills,
school dropout
rate, local gang
activity, gun
availability
Individual
resilience,
inadequate
policing, quality
of emergency
care
Admissions for
gunshot wounds
Adolescent death
rate due to
gunshot wound

Causal Theory Diagram for Adult
Immunization

Causal Theory Diagram for Adolescent
Deaths from Gunshot Wounds

Causal Theory Statement Templates
• Risk of health problem among population /
community is indicated by health outcome indicators
and results from causative factors.
• Health problem among population / community,
indicated in health outcome indicators, is caused by
causative factors, but is mediated by mediating
factors given that moderating factors moderate the
causes and that required antecedent factors exist
prior to the causes.
•
‫السكان‬ ‫بين‬ ‫صحية‬ ‫مشكلة‬ ‫حدوث‬ ‫خطر‬ ‫إلى‬ ‫يشار‬
/
‫المسببة‬ ‫العوامل‬ ‫ونتائج‬ ‫الصحية‬ ‫النتائج‬ ‫مؤشرات‬ ‫خالل‬ ‫من‬ ‫المجتمع‬
.
•
‫السكان‬ ‫بين‬ ‫الصحية‬ ‫المشكلة‬ ‫إن‬
/
‫يت‬ ‫ولكن‬ ، ‫مسببة‬ ‫عوامل‬ ‫عن‬ ‫ناتجة‬ ، ‫الصحية‬ ‫النتائج‬ ‫مؤشرات‬ ‫في‬ ‫إليها‬ ‫المشار‬ ، ‫المجتمع‬
‫فيها‬ ‫التوسط‬ ‫م‬
‫األسباب‬ ‫قبل‬ ‫مطلوبة‬ ‫سابقة‬ ‫عوامل‬ ‫وتوجد‬ ‫األسباب‬ ‫بتخفيف‬ ‫تقوم‬ ‫المعتدلة‬ ‫العوامل‬ ‫أن‬ ‫إلى‬ ‫بالنظر‬ ‫الوساطة‬ ‫عوامل‬ ‫خالل‬ ‫من‬
.

Relationship of Problem Definition to
Program Design and Evaluation
Diagnosis Problem  Program  Evaluation
Risk of: Health problem or
condition
Program goal Outcome variables
Among: At-risk population or
group, target
audience
Recipients Intervention group
As demonstrated in: Health indicators Program objectives Outcome and impact
variables
Resulting from causal
factors:
Specific processes,
conditions, and
factors
Interventions or
treatments for the
target population
Outcome evaluation

Relationship of Problem Definition to
Program Design and Evaluation, Continued
Diagnosis Problem  Program  Evaluation
But is mediated by: Factors that must be
present for the health
problem to occur
Possible intervention Possible control
variables
Given moderation of
the causes by:
Factors that increase
or decrease the
potency of the
causative factors
Possible intervention Possible control
variables
And required existing
factors of:
Sociodemographic
characteristics and
social ecological
factors
Program eligibility
criteria
Control variables or
comparison groups

Prioritizing Health Problems
• Nominal group technique
• Basic priority rating system (BPRS)
– Basic priority rating = (A + 2B) * C
– A = magnitude, B = seriousness, C = intervention
effectiveness
• Propriety, economic, acceptability, resources,
and legality (PEARL)
• Importance and changeability
‫الصحية‬ ‫المشاكل‬ ‫أولويات‬ ‫تحديد‬
•
‫االسمية‬ ‫مجموعة‬ ‫تقنية‬
•
‫األساسي‬ ‫األولويات‬ ‫تصنيف‬ ‫نظام‬
(
BPRS
)
•
‫األساسية‬ ‫األولويات‬ ‫تصنيف‬
=
+ A(
2
B( * C
•
A
=
، ‫الحجم‬
B
=
، ‫الجدية‬
C
=
‫التدخل‬ ‫فعالية‬
•
‫والشرعية‬ ، ‫الموارد‬ ، ‫القبول‬ ، ‫االقتصادية‬ ، ‫المالءمة‬
(
PEARL
)
•
‫التغيير‬ ‫قابلية‬ ‫و‬ ‫أهمية‬

BPRS Criteria
A B C
BPRS
factor
Size Urgency Severity Economic
consequences
Willingness or
involvement
of others
Intervention
effectiveness
Rating
scale
1 (small) – 10
(endemic)
1 (not at all) –
10 (extremely
urgent)
1 (low) – 10
(high)
1 (low) – 10
(high)
1 (low) – 5
(high)
1 (low) – 5
(high)
Factors to
consider
Stability of
incidence or
prevalence
over time
Extent to
which QALYs
and DALYs are
affected;
virulence of
the problem
Healthcare
costs; extent to
which YLL and
YPLL are
affected
Political
support for
addressing
the problem;
popular
awareness
Resistance to
change;
entrenchment
of contributing
factors

Causal Theory Model with BPRS Score
Elements

Prioritization Based on Importance
and Changeability
Highly important health
problem
Less important health
problem
Highly changeable health
problem, more effective
intervention
High priority for
developing a program
Low priority, unless
resources are available
for developing a program
Less changeable health
problem, less effective
intervention
High priority, if an
innovative program can
be developed
No program
development is
warranted

Health Problem Data and Prioritization Across the Pyramid
– Data from individuals or clinic archives
– Problem statement is comprehensive medical diagnosis
including sociocultural factors
– Data from local news media, observation, list of local
services
– Characteristics sorted into different aspects of the problem
statement •
•
‫العيادة‬ ‫أرشيف‬ ‫أو‬ ‫األفراد‬ ‫من‬ ‫بيانات‬
•
‫والثقافية‬ ‫االجتماعية‬ ‫العوامل‬ ‫ذلك‬ ‫في‬ ‫بما‬ ‫الشامل‬ ‫الطبي‬ ‫التشخيص‬ ‫هو‬ ‫المشكلة‬ ‫بيان‬
•
•
‫المحلية‬ ‫الخدمات‬ ‫وقائمة‬ ، ‫والمراقبة‬ ، ‫المحلية‬ ‫اإلعالم‬ ‫وسائل‬ ‫من‬ ‫البيانات‬
•
‫المشكلة‬ ‫بيان‬ ‫من‬ ‫مختلفة‬ ‫جوانب‬ ‫في‬ ‫الخصائص‬ ‫فرز‬ ‫يتم‬

Health Problem Data and Prioritization
Across the Pyramid, Continued
– Data from epidemiological sources, trends
– Problem statement includes physical or social factors
– Data from organization, legal, and workforce records
– Characteristics sorted into different aspects of the
problem statement •
•
‫واالتجاهات‬ ، ‫الوبائية‬ ‫المصادر‬ ‫من‬ ‫البيانات‬
•
‫االجتماعية‬ ‫أو‬ ‫المادية‬ ‫العوامل‬ ‫المشكلة‬ ‫بيان‬ ‫يتضمن‬
•
•
‫العاملة‬ ‫والقوى‬ ‫والقانونية‬ ‫التنظيمية‬ ‫السجالت‬ ‫من‬ ‫بيانات‬
•
‫المشكلة‬ ‫بيان‬ ‫من‬ ‫مختلفة‬ ‫جوانب‬ ‫في‬ ‫الخصائص‬ ‫فرز‬ ‫يتم‬

ProgramTheory and
Interventions Revealed
Chapter 6
Translated by
Khaled
Good Luck
‫البرنامج‬ ‫نظرية‬
‫وكشفت‬

Learning Objectives
1. Understand the implications of program theory, espoused
theories, and theories-in-use for planning and evaluating a
health program.
2. Develop a program theory and portray that theory in a variety of
formats.
3. Value the development of program theory as a communication
and consensus building activity.
•
:
•
‫الصح‬ ‫البرنامج‬ ‫وتقييم‬ ‫للتخطيط‬ ‫االستخدام‬ ‫قيد‬ ‫والنظريات‬ ، ‫المتبنى‬ ‫النظريات‬ ، ‫البرنامج‬ ‫نظرية‬ ‫على‬ ‫المترتبة‬ ‫اآلثار‬ ‫فهم‬
‫ي‬
.
•
‫األشكال‬ ‫من‬ ‫متنوعة‬ ‫مجموعة‬ ‫في‬ ‫النظرية‬ ‫هذه‬ ‫وتصوير‬ ‫البرنامج‬ ‫نظرية‬ ‫تطوير‬
.
•
‫والتوافق‬ ‫التواصل‬ ‫بناء‬ ‫كنشاط‬ ‫البرنامج‬ ‫نظرية‬ ‫تطوير‬ ‫قيمة‬
.

Learning Objectives, Continued
4. Develop program theory that encompasses
both process and effect theory.
5. Distinguish between process and effect
components of a health program.
6. Articulate how program interventions vary
across the levels of the public health pyramid.
•
‫والتأثير‬ ‫العملية‬ ‫من‬ ‫كل‬ ‫نظرية‬ ‫تشمل‬ ‫التي‬ ‫البرنامج‬ ‫نظرية‬ ‫تطوير‬
.
•
‫الصحي‬ ‫البرنامج‬ ‫في‬ ‫والتأثير‬ ‫العملية‬ ‫مكونات‬ ‫بين‬ ‫التمييز‬
.
•
‫العامة‬ ‫الصحة‬ ‫هرم‬ ‫مستويات‬ ‫عبر‬ ‫البرنامج‬ ‫تدخالت‬ ‫تختلف‬ ‫كيف‬ ‫توضيح‬

• Key terminology
• Model of a program theory
• Process theory components
• Model of an effect theory
• Effect theory components
• Intervention typologies
• Intervention types across
the pyramid
• Elements of dosage
• Criteria for good
interventions
• Steps to outcomes and
impacts
• Effect theory example
• Effect theory, espoused
theory, and theory-in-use
• Types of theories relevant
to developing causative
theories
• Types of theories relevant
to developing process
theories
• Validity assumptions
• Program theory functions
• Two roots of program
failure

Key Terminology
• Active protection
• Administration credibility
• Causal theory
• Dosage
• Duration
• Effect
• Effect theory
• Espoused theory
• Frequency
• Impact
• Impact theory
• Intervention theory
• Interventions
• Manipulability
• Organizational plan
• Outcome
• Passive protection
• Primary prevention
• Process theory
• Program theory
• Route of administration
• Secondary prevention
• Service utilization plan
• Strength
• Tertiary prevention
• Theory
• Theory-in-use
•
‫النشطة‬ ‫الحماية‬
•
‫اإلدارة‬ ‫مصداقية‬
•
•
‫جرعة‬
•
‫الزمنية‬ ‫المدة‬
•
‫تأثير‬
•
‫التأثير‬ ‫نظرية‬
•
‫نظرية‬
‫ية‬ّ‫ن‬‫متب‬
•
‫تكرر‬
•
‫تأثير‬
•
•
‫التدخل‬ ‫نظرية‬
•
•
Manipulability
•
‫التنظيمية‬ ‫الخطة‬
•
‫نتيجة‬
•
‫السلبية‬ ‫الحماية‬
•
‫األولية‬ ‫الوقاية‬
•
‫العملية‬ ‫نظرية‬
•
‫البرنامج‬ ‫نظرية‬
•
‫اإلدارة‬ ‫مسار‬
•
‫الثانوية‬ ‫الوقاية‬
•
‫الخدمة‬ ‫استخدام‬ ‫خطة‬
•
‫قوة‬
•
‫الثالثية‬ ‫الوقاية‬
•
‫نظرية‬
•
‫االستخدام‬ ‫في‬ ‫نظرية‬

Process Theory Components
– Personnel, organization resources, capacity
– How to reach the target audience and deliver the
intervention
• Specifications of their outputs
‫مكونات‬
‫النظرية‬
‫العملية‬
•
‫التنظيمية‬ ‫الخطة‬
•
‫القدرات‬ ، ‫المنظمة‬ ‫موارد‬ ، ‫الموظفين‬
•
‫الخدمة‬ ‫استخدام‬ ‫خطة‬
•
‫التدخل‬ ‫وتقديم‬ ‫المستهدف‬ ‫الجمهور‬ ‫إلى‬ ‫الوصول‬ ‫كيفية‬
•
‫مخرجاتها‬ ‫مواصفات‬

Effect Theory Components
• Causal theory
• Intervention theory
• Impact theory
•
•
•

Intervention Typologies
• Prevention levels
– Primary, secondary, tertiary
• Current Procedure Terminology codes
• Nursing Intervention Classification
• Grobe and Hughes (1993) and Issel (1997)
‫التدخل‬ ‫أنواع‬
•
‫الوقاية‬ ‫مستويات‬
•
‫والجامعي‬ ‫والثانوي‬ ‫االبتدائي‬
•
‫الحالية‬ ‫اإلجراءات‬ ‫مصطلحات‬ ‫رموز‬
•
‫التمريض‬ ‫تدخل‬ ‫تصنيف‬
•
‫جروب‬
‫وهيوز‬
(
1993
)
‫وإيسيل‬
(
1997
)

Eight Intervention Types across the
Public Health Pyramid
Intervention
type
Direct services level Enabling services level Population level
Treating Medical or dental procedures,
medications, physical
manipulations, tertiary
prevention, aromatherapy
Respite care, exercise classes Water treatment and
fluoridation, mass
immunizations
Assessing Determining needs and
preferences by asking
individuals, secondary
prevention
Determining needs and
preferences by needs
assessment
Using epidemiological data
identify trends and rates of
illnesses and conditions
Coordinating Care coordination, client
advocacy, referral, linking to
services
Case coordination, local
provider networks and
collaborations
Systems integration, records
and data sharing, state child
health insurance programs
Monitoring Reassessment, follow-up Local trends and news reports Trends analysis

Eight Intervention Types across the
Public Health Pyramid, Continued
Intervention
type
Direct services level Enabling services level Population level
Educating Skills building, information
giving
GED programs, job training
programs
Media campaigns
Counseling Psychotherapy, emotional
support, marital counseling,
cognitive behavioral therapy
Group counseling, family
counseling, grief counseling
for groups
News alerts and advice
Coaching Role modeling, motivational
interviewing, empowerment,
encouragement, stress
management
Community development Policy formation
Giving tangibles Giving vouchers for food or
clothing
Medial supplies loan programs Income supplements,
insurance supplements

Elements of Dosage
• Frequency
• Duration
• Strength
• Route of administration
• Administration credibility
•
‫تكرر‬
•
‫الزمنية‬ ‫المدة‬
•
‫قوة‬
•
‫اإلدارة‬ ‫مسار‬
•
‫اإلدارة‬ ‫مصداقية‬

Criteria for Good Interventions
• Evidence based
• Tailored to the target population
• Conducive to health gains
• Manipulable
• Technologically and logistically feasible
• Reasonable cost
• Politically feasible
• Addresses societal priorities
•
‫األدلة‬ ‫على‬ ‫القائم‬
•
‫المستهدفين‬ ‫للسكان‬ ‫خصيصا‬ ‫مصممة‬
•
‫صحية‬ ‫مكاسب‬ ‫إلى‬ ‫يفضي‬
•
Manipulable
•
‫واللوجستية‬ ‫التكنولوجية‬ ‫الناحية‬ ‫من‬
•
‫معقولة‬ ‫تكلفة‬
•
‫سياسيا‬ ‫مجدية‬
•
‫المجتمعية‬ ‫األولويات‬ ‫يعالج‬
‫الجيدة‬ ‫التدخالت‬ ‫معايير‬

Steps to Outcomes and Impacts
• Generate the effect theory
– Causal, intervention, and impact theories
• Involve key stakeholders
• Draw upon the scientific literature
• Diagram the causal chain of events
• Check against assumptions
‫واآلثار‬ ‫النتائج‬ ‫نحو‬ ‫خطوات‬
•
‫التأثير‬ ‫نظرية‬ ‫توليد‬
•
‫والتأثير‬ ‫والتدخل‬ ‫السببية‬ ‫نظريات‬
•
‫الرئيسيين‬ ‫المصلحة‬ ‫أصحاب‬ ‫إشراك‬
•
‫العلمي‬ ‫األدب‬ ‫على‬ ‫االعتماد‬
•
‫السببية‬ ‫األحداث‬ ‫من‬ ‫سلسلة‬ ‫رسم‬
•
‫االفتراضات‬ ‫ضد‬ ‫تحقق‬

Effect Theory Example for Congenital
Anomalies

Effect Theory, Espoused Theory, and
Theory-in-Use
Effect theory Espoused theory Theory-in-use
What it is Explanation of how
program interventions
affect participants
What staff say about how
the program affects
participants
What staff do to affect
participants
Where it resides Manuals and procedures;
program descriptions
Minds of program staff;
program manuals and
descriptions
Actions of program staff;
on-the-job training
How it is identified Review of scientific
literature, program
materials
Listen to staff describe the
program, read program
materials
Watch what staff do in
providing the program
Importance Guides program and
evaluation; basis for
claiming outcomes
Becomes what staff, clients,
and stakeholders believe
and expect of the program
Is the actual cause of
program outcomes
‫االستخدام‬ ‫ونظرية‬ ، ‫التبني‬ ‫نظرية‬ ، ‫التأثير‬ ‫نظرية‬

Types of Theories Relevant to
Developing Causative Theories
Physical health Psychosocial health Knowledge and
abilities
Self-care and
lifestyle behaviors
•Pathophysiology
•Immunology
•Endocrinology
•Pharmacology
•Wound healing
•Biochemistry
•Metabolism
•Psychopathology
•Social cognition
•Stress and coping
•Family functioning
•Addiction
•Violence
•Resilience
•Learning
•Communication
•Cognition
•Attention
•Memory
•Diffusion of
innovation
•Acculturation
•Peer pressure
•Decision making
•Self-efficacy
•Self-worth
•Risk taking
•Social stratification
•Motivational
‫المسببة‬ ‫النظريات‬ ‫بتطوير‬ ‫المتعلقة‬ ‫النظريات‬ ‫أنواع‬

Types of Theories Relevant to
Developing Process Theories
Organizational plan Service utilization plan
•Social network
•Communication
•Leadership
•Accounting
•Quality improvement
•Social marketing
•Marketing
•Queuing

Validity Assumptions
• The theory is really about the phenomenon of
interest
• Parsimony
‫الصالحية‬ ‫افتراضات‬
•
‫االهتمام‬ ‫ظاهرة‬ ‫حول‬ ‫تدور‬ ‫النظرية‬
•
‫تقتير‬
(
‫الشح‬
)

Program Theory Functions
• Provide guidance
• Enable explanations
• Distinguish between process or effect theory
failure
• Form a basis for communication
• Make a scientific contribution
•
‫التوجيه‬ ‫تقديم‬
•
‫التفسيرات‬ ‫تمكين‬
•
‫التأثير‬ ‫نظرية‬ ‫أو‬ ‫عملية‬ ‫فشل‬ ‫بين‬ ‫التمييز‬
•
‫للتواصل‬ ‫أساسا‬ ‫تشكل‬
•
‫علمية‬ ‫مساهمة‬ ‫تقديم‬

Theories Across the Pyramid
– Individual behavior and intra-individual responses
to treatment or pathology
– Interactions of individuals with family or community
characteristics
•
•
‫األمراض‬ ‫أو‬ ‫للعالج‬ ‫الفردية‬ ‫واالستجابات‬ ‫الفردي‬ ‫السلوك‬
•
•
‫المجتمع‬ ‫أو‬ ‫األسرة‬ ‫خصائص‬ ‫مع‬ ‫األفراد‬ ‫تفاعالت‬

Theories Across the Pyramid, Continued
– Group responses that lead to the health problem
– Cultural theories that explain behaviors or beliefs that lead
to the health problem
– Social theories about interactions among groups
– Organizational behavior, management and leadership
style, personnel motivation, political action,
communication •
•
‫صحية‬ ‫مشكلة‬ ‫إلى‬ ‫تؤدي‬ ‫التي‬ ‫المجموعة‬ ‫ردود‬
•
‫الصحية‬ ‫المشكلة‬ ‫إلى‬ ‫تؤدي‬ ‫التي‬ ‫المعتقدات‬ ‫أو‬ ‫السلوكيات‬ ‫تشرح‬ ‫التي‬ ‫الثقافية‬ ‫النظريات‬
•
‫المجموعات‬ ‫بين‬ ‫التفاعالت‬ ‫حول‬ ‫اجتماعية‬ ‫نظريات‬
•
•
‫التواصل‬ ، ‫السياسي‬ ‫العمل‬ ، ‫الموظفين‬ ‫تحفيز‬ ، ‫والقيادة‬ ‫اإلدارة‬ ‫أسلوب‬ ، ‫التنظيمي‬ ‫السلوك‬

Program Objectives and
SettingTargets
Chapter 7
Translated by
Khaled
Good Luck
‫البرنامج‬ ‫أهداف‬
‫وتحديد‬
‫األهداف‬

Learning Objectives
1. Write clear, measurable process and effect
objectives.
2. Distinguish between process and effect objectives.
3. Develop realistic and achievable target values for
objectives.
•
:
•
‫للقياس‬ ‫وقابلة‬ ‫واضحة‬ ‫وأثر‬ ‫عملية‬ ‫أهداف‬ ‫اكتب‬
.
•
‫وأثرها‬ ‫العملية‬ ‫أهداف‬ ‫بين‬ ‫التمييز‬
.
•
‫لألهداف‬ ‫للتحقيق‬ ‫وقابلة‬ ‫واقعية‬ ‫الهدف‬ ‫القيم‬ ‫تطوير‬
.

• Key terminology
• Goals vs. objectives
• Program theory
elements
• Effect theory elements
• Health domains and
indicators
• Criteria for selecting
indicators
• Effect objective
examples
• SMART objectives
• Decision options for
target setting
• Data source availability
and consistency for
target setting
• Options for calculating
target values
• Caveats to setting goals
and objectives

Key Terminology
• Goal
• Indicator
• Objective

Goals vs. Objectives
Goals
• Broad, encompassing statements
• Generally one per program
• Longer time horizon
• Not stated in quantifiable terms
Objectives
• Specific statements
• Shorter time horizon
• Measurable
• Process and effect
• “By when, who will achieve what, by how much”
•
‫البيانات‬ ‫وتشمل‬ ، ‫واسعة‬
•
‫برنامج‬ ‫لكل‬ ‫واحد‬ ‫عموما‬
•
‫أطول‬ ‫وقت‬ ‫أفق‬
•
‫الكمي‬ ‫للقياس‬ ‫قابلة‬ ‫شروط‬ ‫في‬ ‫يرد‬ ‫لم‬
•
‫محددة‬ ‫بيانات‬
•
‫أقصر‬ ‫وقت‬ ‫أفق‬
•
‫للقياس‬ ‫قابل‬
•
‫وتأثير‬ ‫عملية‬
•
"
‫وكم‬ ، ‫ما‬ ‫سيحقق‬ ‫من‬ ، ‫متى‬
"

Program Theory Elements as the Basis
for Writing Objectives

Program Theory Elements and Process
and Outcome Objectives
Organizational plan Service utilization
plan
Theory outputs
Format By when, who will
obtain or organize
how much of which
types of resources in
what ways
By when, who will
have how many
interactions with
participants of what
type
By when, how
many of what types
of outputs or
products will be
created or finalized
by whom
Examples By [date], the program
manager will secure
funding for 3 new
computers with
electronic clinical
record software
By [date], the health
educator will
identify 3 evidence-
based interventions
for improving
diabetes self-
management
By [date], staff will
distribute #
brochures to
women receiving
genetic counseling
at the clinic

Relationship Between Effect Theory
Elements and Objectives

Health Domains and Indicators
Impact domain
Indicators to measure objectives
General measures Standardized measures
Physical health Weight, dental or medical diagnoses Normal range on laboratory tests, ICD-10
Mental health Motivation, attitudes, medical diagnosis of
mental illness
CES-D, Daily Hassles Scale
Cognitive processes Decision making, judgments, cognitive
development or impairment
Bayley Scales of Infant Development, IQ tests
Behavior Smoking, exercise, seat belt wearing
Knowledge Skill, ability, performance, recall or synthesis
of facts
Social health Marital status, social network, recreation
activities, volunteerism
Norbeck Social Support Questionnaire
Resources Income, insurance coverage, housing,
employment status
Hollingshead Index, Dissimilarity Index

Criteria for Selecting Indicators
• Funding agency requirements
• Data collection is feasible
• Scientifically defendable
• Easy to analyze
‫المؤشرات‬ ‫اختيار‬ ‫معايير‬
•
‫التمويل‬ ‫وكالة‬ ‫متطلبات‬
•
‫ممكن‬ ‫هو‬ ‫البيانات‬ ‫جمع‬
•
‫علميا‬ ‫دافع‬
•
‫التحليل‬ ‫سهل‬

Bowe County Health Problems with
Indicators, Outcomes, and Impacts
Problem Indicator Outcome Goal or impact
Vaccination Rates of underimmunized,
by age group
Vaccine-preventable illness Decrease vaccine
preventable hospitalizations
Presence of neural
tube defect
Rates of neural tube defects
and congenital anomalies
Maintain current rate of
congenital anomalies
Diagnosis of
pregnancy
Pregnancy rate, by age
group
Child abuse rate Reduced child abuse related
to unwanted pregnancy
Hospital admissions
for gunshot wounds
Rate of admissions for
gunshot injuries at local
hospitals, number of police
reports
Adolescent death rate due to
gunshot wounds
Reduce adolescent death
rate due to gunshot wounds
Diagnosis of type 2
diabetes
Incidence and prevalence
rates of diabetes
Morbidity due to chronic
illness
Reduce rates of amputation
and vision loss due to
diabetes
‫لمقاطعة‬ ‫الصحية‬ ‫المشاكل‬
Bowe
‫واآلثار‬ ‫والنتائج‬ ‫المؤشرات‬ ‫مع‬

Effect Objective Examples for
Congenital Anomalies
Intervention objective from
intervention theory
Outcome objective from
causal theory
Impact objective from
impact theory
Format By when, what proportion of
recipients will have how
much effect from program
interventions on which causal
factors that lead to the health
problem
By when, what proportion
of recipients will have how
much effect from program
interventions on the
immediate health problem
By when, long-term or
global health change or
status among the target
population
Example By [date], [target #] women in
the program will have a
decrease by [target %] in
exposure to teratogenic
environmental hazards
By [date], [target #] women
in the program will have
normal newborns (no
neural tube defects)
By [date], [target rate] of
congenital anomalies
among residents of Bowe
County

SMART Objectives
• Specific
• Measurable
• Achievable
• Realistic
• Time
•
‫محدد‬
•
‫للقياس‬ ‫قابل‬
•
‫للتحقيق‬ ‫قابل‬
•
‫واقعي‬
•
‫زمن‬

Decision Options for Target Setting
Population indicator
trend
Current value of the indicator in the target audience
Better than long-term
objective
Meets long-term
objective
Worse than long-term
objective
Improving Set target to maintain
current level
Set target to surpass
objective; continue the
trend
Set target to a better
level; accelerate the
trend
No change Set target to a slightly
better level
Set target to surpass
objective
Set target to a slightly
better level
Deteriorating Set target to maintain
current level; stop the
trend
Set target to maintain
current level
Set target to maintain
current level or a
slightly lower one; stop
or decelerate the trend

Data Source Availability and
Consistency for Target Setting
One or only a few sources Many sources
Consistent information
across sources
Consider the data quality and
relevance to program and
objective
Can use any of the data
sources
Inconsistent information
across sources
If the source is markedly
different from the literature,
need to either change the
objective or verify the data
Need to decide which source
to use, given their strengths
and weaknesses; consider
which is most relevant to the
program and objective

Options for Calculating Target Values
Option Description Ideal program type Pros Cons
1 Default, no change Mature, stable Doesn’t require
historical data
Doesn’t require
improvement
2 Change based on results of
statistical test
Population-based with
many recipients
Supports argument that
improvement was more
than by chance
Sensitive to sample
size; may result in
unreasonable target
3 % change based on current
trend, literature, or guess
Stable program and target
population
Easy to understand; can
account for trend data
Requires some
statistical knowledge
4 Use existing benchmark to
project for several years
Program must show
improvement
Comparable programs
can be compared
Requires existing long-
term objective and
long-term program
5 Mean rate across geographic
areas
Population based Easily understood Requires data from
each area

Options for Calculating Target Values,
Continued
Option Description Ideal program type Pros Cons
6 Median rate across
geographic areas
Population based Easily understood Requires data from each
area
7 Overall rate from best 50%
across geographic areas
Population based or
multisite
Considers all values in
population and moves entire
population to an achievable
value
Requires data from each
area; harder to
understand
8 Overall rate from best 75%
across geographic areas
Population based or
multisite
value
area; harder to
understand
9 Rate for best
sociodemographic stratum
Population based or diverse
target audience with
disparities
value
group; harder to
understand
10 Overall rate based on
different targets for strata
Population based or diverse
target audience with
disparities
Program must show
improvement; more intense
intervention for worst-off group
group; harder to
understand

Caveats to Setting Goals and Objectives
• Tenuous effectiveness of using objectives to guide work
– Need staff buy-in, using resources to support achieving
objectives, and a reward system
• Need for spontaneity
– Complexity theory highlights the importance of flexibility
• Messy interface between objectives and performance
measures
– More on this in Chapter 10
‫والغايات‬ ‫األهداف‬ ‫لتحديد‬ ‫تحذيرات‬
•
‫العمل‬ ‫لتوجيه‬ ‫األهداف‬ ‫الستخدام‬ ‫ضعيفة‬ ‫فعالية‬
•
‫المكافآت‬ ‫ونظام‬ ، ‫األهداف‬ ‫تحقيق‬ ‫لدعم‬ ‫الموارد‬ ‫واستخدام‬ ، ‫الموظفين‬ ‫موافقة‬ ‫إلى‬ ‫تحتاج‬
•
‫العفوية‬ ‫إلى‬ ‫الحاجة‬
•
‫المرونة‬ ‫أهمية‬ ‫تبرز‬ ‫التعقيد‬ ‫نظرية‬
•
‫األداء‬ ‫ومقاييس‬ ‫األهداف‬ ‫بين‬ ‫فوضوي‬ ‫واجهة‬
•
‫الفصل‬ ‫في‬ ‫هذا‬ ‫عن‬ ‫المزيد‬
10

Objectives Across the Pyramid
– Process objectives on interactions between providers and
participants
– Effect objectives on client behavior or health status change
– Targets may adapt national objectives for the local setting
•
•
‫والمشاركين‬ ‫الخدمات‬ ‫مقدمي‬ ‫بين‬ ‫التفاعالت‬ ‫على‬ ‫العملية‬ ‫أهداف‬
•
‫الصحية‬ ‫الحالة‬ ‫تغيير‬ ‫أو‬ ‫العميل‬ ‫سلوك‬ ‫على‬ ‫التأثير‬ ‫أهداف‬
•
‫المحلي‬ ‫لإلعداد‬ ‫الوطنية‬ ‫األهداف‬ ‫تكييف‬ ‫لألهداف‬ ‫يمكن‬

– Process objectives on involving community resources and
encouraging interagency collaboration
– Effect objectives on behavior or health status of families or
other aggregates
– Targets based on past experience, similar programs, and
community assessment data
Objectives Across the Pyramid,
Continued
•
•
‫الوكاالت‬ ‫بين‬ ‫التعاون‬ ‫وتشجيع‬ ‫المجتمع‬ ‫موارد‬ ‫بإشراك‬ ‫المتعلقة‬ ‫العملية‬ ‫أهداف‬
•
‫األخرى‬ ‫المجاميع‬ ‫أو‬ ‫لألسر‬ ‫الصحية‬ ‫الحالة‬ ‫أو‬ ‫السلوك‬ ‫على‬ ‫األهداف‬ ‫تأثير‬
•
‫المجتمع‬ ‫تقييم‬ ‫وبيانات‬ ، ‫مماثلة‬ ‫وبرامج‬ ، ‫السابقة‬ ‫الخبرة‬ ‫إلى‬ ‫تستند‬ ‫أهداف‬

Objectives Across the Pyramid, Continued
– Process objectives on coordinating implementation efforts and
getting adequate resources
– Effect objectives with an outcome or impact focus
– Targets will draw heavily from national data
– Process objectives dominate
– Effect objectives on service effectiveness and efficiency or the
outcomes on the infrastructure itself
•
•
‫الكافية‬ ‫الموارد‬ ‫على‬ ‫والحصول‬ ‫التنفيذ‬ ‫جهود‬ ‫تنسيق‬ ‫على‬ ‫العملية‬ ‫أهداف‬
•
‫التأثير‬ ‫أو‬ ‫النتيجة‬ ‫على‬ ‫التركيز‬ ‫مع‬ ‫التأثير‬ ‫أهداف‬
•
‫الوطنية‬ ‫البيانات‬ ‫من‬ ‫بشدة‬ ‫األهداف‬ ‫ستستمد‬
•
•
‫تهيمن‬ ‫العملية‬ ‫أهداف‬
•
‫نفسها‬ ‫التحتية‬ ‫البنية‬ ‫على‬ ‫النتائج‬ ‫أو‬ ‫وكفاءتها‬ ‫الخدمة‬ ‫فعالية‬ ‫على‬ ‫التأثير‬ ‫أهداف‬

Program Implementation
Chapter 8
Translated by
Khaled
Good Luck
‫تنفيذ‬
‫البرنامج‬

Learning Objectives
1. Articulate the inputs and outputs of a service utilization plan
of a program.
2. Justify the inputs and outputs of the organizational plan of a
program.
•
:
•
‫لبرنامج‬ ‫الخدمة‬ ‫استخدام‬ ‫خطة‬ ‫من‬ ‫والمخرجات‬ ‫المدخالت‬ ‫توضيح‬
.
•
‫للبرنامج‬ ‫التنظيمية‬ ‫الخطة‬ ‫ومخرجات‬ ‫مدخالت‬ ‫تبرير‬
.

• Key terminology
• Effort across the life of a
health program
• Organizational and
services utilization plan
elements
• Organizational plan inputs
• Human resources
considerations
• Physical resources
• Informational resources
• Managerial resources
outputs
• Service utilization plan inputs
• Social marketing
• Overinclusion and
underinclusion
• Scope of coverage
• Intervention delivery
outputs
• Process and effect theory
examples
• Business plans vs. process
theories and logic models

Key Terminology
• Business plan
• Full-coverage programs
• Indirect costs
• Logic model
• Overinclusion
• Partial-coverage
programs
• Place
• Price
• Product
• Promotion
• Recipient
• Sensitivity
• Social marketing
• Specificity
• Target audience
• Target population
• Timeline
• Underinclusion
• Units of service
•
‫عمل‬ ‫خطة‬
•
‫الكاملة‬ ‫التغطية‬ ‫برامج‬
•
‫المباشرة‬ ‫غير‬ ‫التكاليف‬
•
‫المنطق‬ ‫نموذج‬
•
Overinclusion
•
‫الجزئية‬ ‫التغطية‬ ‫برامج‬
•
‫مكان‬
•
‫السعر‬
•
‫المنتج‬
•
‫وظيفية‬ ‫ترقية‬
•
‫مستلم‬
•
‫حساسية‬
•
‫التسويق‬
‫اإلجتماعي‬
•
‫النوعية‬
•
•
‫المستهدفين‬ ‫السكان‬
•
‫الزمني‬ ‫الجدول‬
•
Underinclusion
•
‫الخدمة‬ ‫وحدات‬

Effort across the Life of a Health
Program

Organizational and Services Utilization Plan
Elements of the Process Theory

Organizational Plan Inputs
1. Human resources
2. Physical resources
3. Transportation
4. Informational resources
5. Time
6. Managerial resources
7. Monetary resources
‫التنظيمية‬ ‫الخطة‬ ‫مدخالت‬
•
‫البشرية‬ ‫الموارد‬
•
‫مادية‬ ‫موارد‬
•
‫النقل‬ ‫وسائل‬
•
‫المعلوماتية‬ ‫الموارد‬
•
‫زمن‬
•
‫االدارية‬ ‫الموارد‬
•
‫النقدية‬ ‫الموارد‬
‫مهم‬

Human Resources Considerations
• Human resources encompass the quantity and quality of personnel
needed to carry out the program, in terms of their expertise,
experience and capabilities.
• Licensure and/or certification
– Program needs and regulatory requirements
• Training
– Aligns theory-in-use and espoused theory with effect theory
• Cultural diversity and sensitivity
• Volunteers
•
‫م‬ ، ‫البرنامج‬ ‫لتنفيذ‬ ‫الالزمين‬ ‫الموظفين‬ ‫ونوعية‬ ‫كمية‬ ‫البشرية‬ ‫الموارد‬ ‫تشمل‬
‫حيث‬ ‫ن‬
‫وقدراتهم‬ ‫وخبراتهم‬ ‫خبرتهم‬
.
•
‫الترخيص‬
‫و‬
/
‫الشهادة‬ ‫أو‬
•
‫التنظيمية‬ ‫والمتطلبات‬ ‫البرنامج‬ ‫احتياجات‬
•
‫تدريب‬
•
‫التأثير‬ ‫نظرية‬ ‫مع‬ ‫التبني‬ ‫ونظرية‬ ‫االستخدام‬ ‫قيد‬ ‫النظرية‬ ‫محاذاة‬
•
‫والحساسية‬ ‫الثقافي‬ ‫التنوع‬
•
‫المتطوعين‬
‫مهم‬

Considerations for Health Disciplines
Discipline State Licensure Required? State Regulation of Scope of
Practice
Professional Certification Exists
Dentistry Yes Yes Yes
Community health worker No No Certificate programs
Dietitian Yes, as RD Yes Yes
Health administration No (except for long-term care) No Yes
Health education No No Yes
Industrial hygiene Yes Yes Yes
Medicine Yes Yes Yes
Nursing Yes, as RN Yes Yes
Physical therapy Yes Yes Yes
Social work Optional Yes, for those with licensure Yes

Physical Resources
1. Material resources are those tangible items needed to provide
intervention and program support.
2. Facilities is another material resources needed for intervention
and program support.
3. Supplies
4. Equipment
•
‫الب‬ ‫ودعم‬ ‫التدخل‬ ‫لتوفير‬ ‫الالزمة‬ ‫الملموسة‬ ‫العناصر‬ ‫تلك‬ ‫هي‬ ‫المادية‬ ‫الموارد‬
‫رنامج‬
.
•
‫البرنامج‬ ‫ودعم‬ ‫للتدخل‬ ‫الالزمة‬ ‫األخرى‬ ‫المادية‬ ‫الموارد‬ ‫هي‬ ‫المرافق‬
.
•
‫المستلزمات‬
•
‫المعدات‬
‫مهم‬

Informational Resources
1. Professional networks
2. Street smarts
3. Professional knowledge and experience
4. Important to maximize staff retention
•
‫المهنية‬ ‫الشبكات‬
•
‫الشارع‬ ‫ذكاء‬
•
‫المهنية‬ ‫والخبرة‬ ‫المعرفة‬
•
‫الموظفين‬ ‫استبقاء‬ ‫تعظيم‬ ‫المهم‬ ‫من‬

Managerial Resources
1. Organizational abilities
2. Communication skills, including
negotiation
3. Team-building skills
4. Leadership qualities
5. Coping skills for complex tasks
6. Technical skills
•
‫التنظيمية‬ ‫القدرات‬
•
‫التفاوض‬ ‫ذلك‬ ‫في‬ ‫بما‬ ، ‫االتصال‬ ‫مهارات‬
•
‫الفريق‬ ‫بناء‬ ‫مهارات‬
•
‫القيادية‬ ‫الصفات‬
•
‫المعقدة‬ ‫المهام‬ ‫مع‬ ‫التعامل‬ ‫مهارات‬
•
‫تقنية‬ ‫مهارات‬

Organizational Plan Outputs
1. Timeline
2. Operations manual (contains the policies, procedures, guideline
and protocols related to the health program)
3. Organizational chart: graphic representation describe
relationships among work units, departments and individuals.
4. Information system: has outputs that are part of the
organization plan.
5. Budget
‫التنظيمية‬ ‫الخطة‬ ‫مخرجات‬
‫مهم‬
•
‫الزمني‬ ‫الجدول‬
•
‫العمليات‬ ‫دليل‬
(
‫الصحي‬ ‫بالبرنامج‬ ‫المتعلقة‬ ‫والبروتوكوالت‬ ‫التوجيهية‬ ‫والمبادئ‬ ‫واإلجراءات‬ ‫السياسات‬ ‫على‬ ‫يحتوي‬
)
•
‫التنظيمي‬ ‫الهيكل‬
:
‫واألفراد‬ ‫واإلدارات‬ ‫العمل‬ ‫وحدات‬ ‫بين‬ ‫العالقات‬ ‫يصف‬ ‫البياني‬ ‫التمثيل‬
.
•
‫المعلومات‬ ‫نظام‬
:
‫التنظيمية‬ ‫الخطة‬ ‫من‬ ‫ا‬ً‫ء‬‫جز‬ ‫تشكل‬ ‫مخرجات‬ ‫على‬ ‫يحتوي‬
.
•
‫ميزانية‬

Budgeting Terminology
• Fixed costs: Don’t vary with the number of clients served. include rent, salaries of administrative personnel, and insurance
costs.
• Variable costs: vary with number of clients served. include copying program handouts, program advertising, and
refreshments for participants.
• Direct costs: reflect those resources used directly in the delivery of the program.eg. materials or supplies used with clients.
• Indirect costs: are those costs not associated with the delivery of program, but more generally with supporting the
program. Example telephone charges, and staff travel expenses to present the program at scientific conferences. Indirect
costs, as a percentage of direct costs, can vary from 8% as limited by funding agencies up to 51%.
Indirect costs associated with overhead expenses (e.g., rent, utility, facilities management, shared clerical support staff, office equipment)
• Charges: include the cost plus a profit margin and administrative costs. example is printing costs given as an in kind
donation;
• volunteer time is another in-kind donation for staff time
• Opportunity costs: refer to purchases that cannot be made because having spent the money on something else.
• The majority of costs is related to the program implementation and includes the resources utilized by the program staff
and participants.
‫مهم‬
•
‫الثابتة‬ ‫التكاليف‬
:
‫تقديمهم‬ ‫يتم‬ ‫الذين‬ ‫العمالء‬ ‫عدد‬ ‫مع‬ ‫تختلف‬ ‫ال‬
.
‫التأمين‬ ‫وتكاليف‬ ، ‫اإلداريين‬ ‫الموظفين‬ ‫رواتب‬ ، ‫اإليجار‬ ‫تشمل‬
.
•
‫المتغيرة‬ ‫التكاليف‬
:
‫خدم‬ ‫العمالء‬ ‫من‬ ‫عدد‬ ‫مع‬ ‫تختلف‬
.
‫نسخ‬ ‫تشمل‬
‫نسخ‬
‫للمشاركين‬ ‫والمرطبات‬ ، ‫البرنامج‬ ‫عن‬ ‫اإلعالن‬ ، ‫البرنامج‬
.
•
‫المباشرة‬ ‫التكاليف‬
:
‫تقديم‬ ‫في‬ ‫مباشرة‬ ‫المستخدمة‬ ‫الموارد‬ ‫تلك‬ ‫تعكس‬
program.eg
.
‫العمالء‬ ‫مع‬ ‫المستخدمة‬ ‫اللوازم‬ ‫أو‬ ‫المواد‬
.
•
‫المباشرة‬ ‫غير‬ ‫التكاليف‬
:
‫البرنامج‬ ‫دعم‬ ‫مع‬ ‫أعم‬ ‫بشكل‬ ‫ولكن‬ ، ‫البرنامج‬ ‫بتسليم‬ ‫المرتبطة‬ ‫غير‬ ‫التكاليف‬ ‫تلك‬ ‫هي‬
.
‫البرنام‬ ‫لتقديم‬ ‫الموظفين‬ ‫سفر‬ ‫ونفقات‬ ، ‫المثال‬ ‫سبيل‬ ‫الهاتف‬ ‫رسوم‬
‫العلمية‬ ‫المؤتمرات‬ ‫في‬ ‫ج‬
.
‫التكاليف‬
‫من‬ ‫تختلف‬ ‫أن‬ ‫يمكن‬ ، ‫المباشرة‬ ‫التكاليف‬ ‫من‬ ‫مئوية‬ ‫كنسبة‬ ، ‫المباشرة‬ ‫غير‬
8
‫التمويل‬ ‫وكاالت‬ ‫تقتصر‬ ‫كما‬ ٪
‫حتى‬
51
.%
•
‫العامة‬ ‫بالنفقات‬ ‫المرتبطة‬ ‫المباشرة‬ ‫غير‬ ‫التكاليف‬
(
‫المكتبية‬ ‫والمعدات‬ ‫المشترك‬ ‫الكتابي‬ ‫الدعم‬ ‫وموظفي‬ ‫المرافق‬ ‫وإدارة‬ ‫والمرافق‬ ‫اإليجار‬ ‫مثل‬
)
•
‫الرسوم‬
:
‫اإلدارية‬ ‫والتكاليف‬ ‫الربح‬ ‫هامش‬ ‫إلى‬ ‫باإلضافة‬ ‫التكلفة‬ ‫تشمل‬
.
‫؛‬ ‫عيني‬ ‫كتبرع‬ ‫المقدمة‬ ‫الطباعة‬ ‫تكاليف‬ ، ‫المثال‬ ‫سبيل‬ ‫على‬
•
‫الموظفين‬ ‫لوقت‬ ‫آخر‬ ‫عيني‬ ‫تبرع‬ ‫هو‬ ‫التطوعي‬ ‫الوقت‬
•
‫البديلة‬ ‫الفرصة‬ ‫تكاليف‬
:
‫آخر‬ ‫شيء‬ ‫على‬ ‫المال‬ ‫أنفقت‬ ‫ألنك‬ ‫إجراؤها‬ ‫يمكن‬ ‫ال‬ ‫التي‬ ‫المشتريات‬ ‫إلى‬ ‫تشير‬
.
•
‫والمشاركون‬ ‫البرنامج‬ ‫موظفو‬ ‫يستخدمها‬ ‫التي‬ ‫الموارد‬ ‫وتشمل‬ ‫البرنامج‬ ‫بتنفيذ‬ ‫التكاليف‬ ‫غالبية‬ ‫ترتبط‬
.

Break-Even Analysis
• After the program budget is complete and nearly final, it is possible to do a
breakeven analysis . A break-even analysis is the mathematical
determination of the point at which the expenses related to providing the
program are equal to or less than the revenues generated for or from the
program .
• This type of analysis uses the price of the service (the charge), the variable
costs of program, and the fixed costs of the program.
• Point at which expenses ≤ revenues
• Qty of services = fixed cost / (price per client – variable cost per client)
•
‫التعادل‬ ‫تحليل‬ ‫إجراء‬ ‫يمكن‬ ، ‫ا‬ً‫ب‬‫تقري‬ ‫والنهائية‬ ‫البرنامج‬ ‫ميزانية‬ ‫اكتمال‬ ‫بعد‬
.
‫الب‬ ‫بتوفير‬ ‫المتعلقة‬ ‫النفقات‬ ‫فيها‬ ‫تكون‬ ‫التي‬ ‫للنقطة‬ ‫الرياضي‬ ‫التحديد‬ ‫هو‬ ‫التعادل‬ ‫تحليل‬
‫اإليرادات‬ ‫من‬ ‫أقل‬ ‫أو‬ ‫مساوية‬ ‫رنامج‬
‫منه‬ ‫أو‬ ‫البرنامج‬ ‫عن‬ ‫الناتجة‬
.
•
‫الخدمة‬ ‫سعر‬ ‫التحليل‬ ‫من‬ ‫النوع‬ ‫هذا‬ ‫يستخدم‬
(
‫الرسوم‬
)
‫للبرنامج‬ ‫الثابتة‬ ‫والتكاليف‬ ، ‫للبرنامج‬ ‫المتغيرة‬ ‫والتكاليف‬ ،
.
•
‫فيها‬ ‫تكون‬ ‫التي‬ ‫النقطة‬
‫النفقات‬
≤
‫اإليرادات‬
•
‫الخدمات‬ ‫كمية‬
=
‫الثابتة‬ ‫التكلفة‬
( /
‫عميل‬ ‫لكل‬ ‫السعر‬
-
‫عميل‬ ‫لكل‬ ‫المتغيرة‬ ‫التكلفة‬
)
‫مهم‬

More on Budgets
• Must include evaluation expenses: at a minimum, a meaningful evaluation
cannot be done for less than 10% of the direct program costs
• Must include evaluation expenses
– Generally ≤ 10% of direct program costs
• Budget justifications are required for funding proposals
• Most grant proposal budgets focus on the major categories of direct costs.
•
‫التقييم‬ ‫نفقات‬ ‫تشمل‬ ‫أن‬ ‫يجب‬
:
‫من‬ ‫بأقل‬ ‫حقيقي‬ ‫تقييم‬ ‫إجراء‬ ‫يمكن‬ ‫ال‬ ، ‫أدنى‬ ‫كحد‬
10
‫المباشرة‬ ‫البرنامج‬ ‫تكاليف‬ ‫من‬ ٪
•
‫يجب‬
‫التقييم‬ ‫نفقات‬ ‫تشمل‬ ‫أن‬
•
‫عموما‬
≥
10
‫المباشرة‬ ‫البرنامج‬ ‫تكاليف‬ ‫من‬ ٪
•
‫لمقترحات‬ ‫مطلوبة‬ ‫الميزانية‬ ‫مبررات‬
‫التمويل‬
•
‫المباشرة‬ ‫للتكاليف‬ ‫الرئيسية‬ ‫الفئات‬ ‫على‬ ‫المنح‬ ‫مقترحات‬ ‫ميزانيات‬ ‫معظم‬ ‫تركز‬
.
‫مهم‬

Service Utilization Plan Inputs
1. Social marketing
2. Eligibility screening
3. Queuing
4. Intervention delivery
•
•
‫األهلية‬ ‫فحص‬
•
‫الطابور‬
•
‫التدخل‬ ‫تسليم‬
‫الخدمة‬ ‫استخدام‬ ‫خطة‬ ‫مدخالت‬
‫مهم‬

1- Social marketing:
defined social marketing as the design, implementation, and control of the
program calculated to Influence classical marketing: product, price,place,
and promotionthe acceptability of social ideas .
A.Product: refers to the service, tangibles
B. price: focus on the secondary costs, such as transportation or loss of peer
group status .
C. Place : refers to where the product is available
•
:
•
‫الكالسيكي‬ ‫التسويق‬ ‫على‬ ‫التأثير‬ ‫على‬ ‫المحسوب‬ ‫البرنامج‬ ‫ومراقبة‬ ‫وتنفيذ‬ ‫تصميم‬ ‫بأنه‬ ‫االجتماعي‬ ‫التسويق‬ ‫ف‬ّ‫ُعر‬‫ي‬
:
‫والترو‬ ‫والمكان‬ ‫والسعر‬ ‫المنتج‬
‫االجتماعية‬ ‫األفكار‬ ‫لمقبولية‬ ‫يج‬
.
•
‫المنتج‬
:
‫الملموسة‬ ، ‫الخدمة‬ ‫إلى‬ ‫يشير‬
•
‫السعر‬
:
‫النظراء‬ ‫مجموعة‬ ‫حالة‬ ‫فقدان‬ ‫أو‬ ‫النقل‬ ‫مثل‬ ، ‫الثانوية‬ ‫التكاليف‬ ‫على‬ ‫التركيز‬
.
•
‫المكان‬
:
‫المنتج‬ ‫توفر‬ ‫مكان‬ ‫إلى‬ ‫يشير‬
‫مهم‬

Social Marketing
• Also called health marketing
• Four P’s of classic marketing
– Product, price, place, promotion
• Additional P’s
– Partnership, policy
•
‫الصحي‬ ‫التسويق‬ ‫أيضا‬ ‫وتسمى‬
•
‫الكالسيكي‬ ‫التسويق‬ ‫من‬ ‫ف‬ ‫أربعة‬
•
‫والترويج‬ ‫والمكان‬ ‫والسعر‬ ‫المنتج‬
•
‫إضافية‬ ‫ف‬
•
‫سياسة‬ ، ‫شراكة‬
‫مهم‬

2- Eligibility screening :
One of the first decisions facing program planners is to define for whom the program is designed .
A. Target population: is the entire population in need of the program .
B. Target audience: is the segment of the population for whom the program is specifically intended.
C. Recipient: refers to those individuals who actually receive or participate in the program .
3- Queuing:
Waiting to be seen for services, being on hold, and having to wait until services become available are all aspects
of being put in a queue.
4- Intervention delivery:
Delivery of the intervention ought to follow the protocols and procedures developed specifically for the health
program ‫األهلية‬ ‫فحص‬
:
‫البرنامج‬ ‫صمم‬ ‫من‬ ‫تحديد‬ ‫هو‬ ‫البرامج‬ ‫مخططي‬ ‫تواجه‬ ‫التي‬ ‫األولى‬ ‫القرارات‬ ‫أحد‬
.
‫المستهدفون‬ ‫السكان‬
:
‫البرنامج‬ ‫إلى‬ ‫يحتاجون‬ ‫الذين‬ ‫السكان‬ ‫مجموع‬ ‫هو‬
.
:
‫لهم‬ ‫ا‬ً‫ص‬‫خصي‬ ‫البرنامج‬ ‫تصميم‬ ‫تم‬ ‫الذين‬ ‫السكان‬ ‫شريحة‬ ‫هو‬
.
‫المستلم‬
:
‫البرنامج‬ ‫في‬ ‫يشاركون‬ ‫أو‬ ‫بالفعل‬ ‫يتلقون‬ ‫الذين‬ ‫األفراد‬ ‫إلى‬ ‫يشير‬
.
3
-
‫االنتظار‬
:
‫انتظار‬ ‫قائمة‬ ‫في‬ ‫وضعها‬ ‫يتم‬ ‫جوانب‬ ‫كلها‬ ، ‫متوفرة‬ ‫الخدمات‬ ‫تصبح‬ ‫حتى‬ ‫االنتظار‬ ‫إلى‬ ‫والحاجة‬ ، ‫والتعليق‬ ، ‫الخدمات‬ ‫ظهور‬ ‫انتظار‬ ‫في‬
.
4
-
‫التدخل‬ ‫تسليم‬
:
‫الصحي‬ ‫للبرنامج‬ ‫ا‬ً‫ص‬‫خصي‬ ‫الموضوعة‬ ‫واإلجراءات‬ ‫البروتوكوالت‬ ‫التدخل‬ ‫تسليم‬ ‫يتبع‬ ‫أن‬ ‫يجب‬
‫مهم‬

Overinclusion vs. Underinclusion
• Overinclusion
– May dilute the estimated program effects on the target audience’s outcomes
– May falsely increase the estimated size of the target audience
• Underinclusion
– Too few participants may make it difficult to detect program effects
– More intensive services to a small group may inflate the true program effect
– Increases cost per participant
•
‫المستهدف‬ ‫الجمهور‬ ‫نتائج‬ ‫على‬ ‫المقدرة‬ ‫البرنامج‬ ‫آثار‬ ‫من‬ ‫يخفف‬ ‫قد‬
•
‫المستهدف‬ ‫للجمهور‬ ‫المقدر‬ ‫الحجم‬ ‫كذبا‬ ‫يزيد‬ ‫قد‬
•
‫البرنامج‬ ‫آثار‬ ‫اكتشاف‬ ‫الصعب‬ ‫من‬ ‫تجعل‬ ‫قد‬ ‫المشاركين‬ ‫من‬ ‫قلة‬
•
‫ا‬ ‫البرنامج‬ ‫تأثير‬ ‫تضخيم‬ ‫إلى‬ ‫صغيرة‬ ‫لمجموعة‬ ‫كثافة‬ ‫األكثر‬ ‫الخدمات‬ ‫تؤدي‬ ‫قد‬
‫لحقيقي‬
•
‫مشارك‬ ‫لكل‬ ‫التكلفة‬ ‫يزيد‬

Minimizing Over- and Underinclusion
• Specifying how recipients get into the program
– Sensitive and specific screening tools
• Sensitivity (identifying true cases) reduces under inclusion
• Specificity (identifying true non-cases) reduces over inclusion
– Screening enhances program efficiency and effectiveness
• Good marketing plan
•
‫البرنامج‬ ‫إلى‬ ‫المستلمين‬ ‫وصول‬ ‫كيفية‬ ‫تحديد‬
•
‫والمحددة‬ ‫الحساسة‬ ‫الفحص‬ ‫أدوات‬
•
‫الحساسية‬
(
‫الحقيقية‬ ‫الحاالت‬ ‫تحديد‬
)
‫اإلدراك‬ ‫من‬ ‫تقلل‬
•
‫خصوصية‬
(
‫الحقيقية‬ ‫غير‬ ‫الحاالت‬ ‫تحديد‬
)
‫من‬ ‫يقلل‬
overinclusion
•
‫وفعاليته‬ ‫البرنامج‬ ‫كفاءة‬ ‫يعزز‬ ‫الفحص‬
•
‫جيدة‬ ‫تسويقية‬ ‫خطة‬
‫مهم‬

Relationship of Sensitivity and
Specificity to Inclusion
Sensitivity Specificity
High Low
High Ideal inclusion and
coverage
Overinclusion
Low Underinclusion Over- and
underinclusion

Scope of Coverage
• Partial-coverage programs: designed to serve some portion of the target
population.
– Often at the direct care or enabling services levels of the pyramid.
– Example include early childhood intervention programs for children at developmental
risk
• Full-coverage programs: designed to serve the entire of the target
population.
– Population services level of the pyramid.
– Example include seat belt laws and water fluoridation
‫مهم‬
•
‫الجزئية‬ ‫التغطية‬ ‫برامج‬
:
‫المستهدفين‬ ‫السكان‬ ‫من‬ ‫جزء‬ ‫لخدمة‬ ‫مصممة‬
.
•
‫الهرم‬ ‫الخدمات‬ ‫مستويات‬ ‫تمكين‬ ‫أو‬ ‫المباشرة‬ ‫الرعاية‬ ‫في‬ ‫األحيان‬ ‫من‬ ‫كثير‬ ‫في‬
.
•
‫النمو‬ ‫لخطر‬ ‫المعرضين‬ ‫لألطفال‬ ‫المبكرة‬ ‫الطفولة‬ ‫مرحلة‬ ‫في‬ ‫التدخل‬ ‫برامج‬ ‫تشمل‬ ، ‫المثال‬ ‫سبيل‬ ‫على‬
•
‫الكاملة‬ ‫التغطية‬ ‫برامج‬
:
‫المستهدفين‬ ‫السكان‬ ‫جميع‬ ‫لخدمة‬ ‫مصممة‬
.
•
‫للهرم‬ ‫السكانية‬ ‫الخدمات‬ ‫مستوى‬
.
•
‫الماء‬ ‫وفلورة‬ ‫األمان‬ ‫حزام‬ ‫قوانين‬ ‫ذلك‬ ‫على‬ ‫مثال‬

Examples of Partial and Full Coverage
Programs, by Pyramid Level
Pyramid level Partial coverage Full coverage
Individual – direct services •Early intervention for children at
developmental risk
•Hospice care
•Ambulance and emergency medical
care
•Immunization clinics available to all
Aggregate – enabling
services
•Needle exchange for some substance
abusers
•Head Start for low income children
•Medicaid coverage for dialysis of
those with kidney failure
Population-based •WIC
•SCHIP
•Seat belt laws
•Medicare for all age 65+
•Water fluoridation
Infrastructure Laptop computers for nurses making
home visits
•Licensure for physicians, nurses, etc.
•National cancer registry

Intervention Delivery
• Requires the most effort
• Made easier via:
– Thorough planning
– Well-conceived process theory
• Must follow the developed protocols
• Pilot testing prior to implementation
– Intervention itself
– Evaluation instruments
•
‫جهد‬ ‫أقصى‬ ‫يتطلب‬
•
‫عبر‬ ‫أسهل‬ ‫أصبح‬
:
•
‫شامل‬ ‫تخطيط‬
•
‫التصميم‬ ‫جيدة‬ ‫عملية‬ ‫نظرية‬
•
‫المتقدمة‬ ‫البروتوكوالت‬ ‫اتباع‬ ‫يجب‬
•
‫التنفيذ‬ ‫قبل‬ ‫تجريبي‬ ‫اختبار‬
•
‫نفسه‬ ‫التدخل‬
•
‫التقييم‬ ‫أدوات‬
‫مهم‬

Service Utilization Plan Outputs
1. Units of service provided:(what was provided, such as hours
per client, number of inpatient visits,)
2. Quantity of service completions.
3. Materials developed to provide the health program: Program
educational videos, annual reports.
4. Work flow: the extent to which program staff have work over
a given time period or that work is done in a coordinated
manner.
‫الخدمة‬ ‫استخدام‬ ‫خطة‬ ‫مخرجات‬
‫مهم‬
•
‫المقدمة‬ ‫الخدمة‬ ‫وحدات‬
( :
، ‫الداخليين‬ ‫المرضى‬ ‫زيارات‬ ‫وعدد‬ ، ‫عميل‬ ‫لكل‬ ‫ساعات‬ ‫مثل‬ ، ‫توفيره‬ ‫تم‬ ‫ما‬
)
•
‫الخدمة‬ ‫استكمال‬ ‫كمية‬
.
•
‫الصحي‬ ‫البرنامج‬ ‫لتوفير‬ ‫المطورة‬ ‫المواد‬
:
‫سنوية‬ ‫تقارير‬ ، ‫تعليمي‬ ‫فيديو‬ ‫برنامج‬
.
•
‫العمل‬ ‫تدفق‬
:
‫منسقة‬ ‫بطريقة‬ ‫العمل‬ ‫بهذا‬ ‫القيام‬ ‫أو‬ ‫معينة‬ ‫زمنية‬ ‫فترة‬ ‫خالل‬ ‫البرنامج‬ ‫موظفي‬ ‫عمل‬ ‫مدى‬
.

Examples of Tracking Service
Utilization Plan Outputs
Intervention activity Target audience: Persons
with health problem
Target audience: Clinics Target audience: City
Individual education 100 persons at risk 76 providers visited
Individual screening 600 persons 30 providers 1000 persons screened at
health fairs
Group education 15 groups at hospital 8 groups at 3 clinics 15 groups at library; 6
groups at school
Population education 2000 flyers; 600 stickers 50 posters 25 public service
announcements; 10
health fairs
Individual support for
behavior
125 persons at risk in 22
groups
32 providers in 8 groups

Process Theory Example for NTD
Prevention Program

Effect and Process Theory Example for NTD Prevention Program

Hypothetical Program Logic Model for Reducing
Congenital Anomalies
Assumptions Health problem is important to community
of providers & residents
Residents have access to health care and
food sources
Inputs Program personnel: RNs, primary care MDs,
geneticists
Funding for 2 years, information system for
tracking participants
Activities Create a timeline and tracking system Conduct staff training; create program
protocol and manual; create PSAs and
recruitment materials
Outputs # of education materials; program manual; #
of staff trained
# women screened for type of employment
and serum folic acid; % women who start
prenatal folic acid supplementation
Immediate
outcome
Improved folic acid knowledge % births with NTD or cleft lip/palate
Long-term
outcome
Rate of congenital anomalies

Business Plans vs. Process Theories
and Logic Models
Business plan format Purpose Process theory element Logic model element
Title/cover page First impressions
Executive summary First impressions
Business concept Describes program design
with goals and objectives
Entire program theory Entire logic model
Market analysis Analyzes demand, need,
competition, & effect on
existing services & health
status
Community needs
assessment
Financial analysis Projects revenues &
expenses; states fiscal
assumptions used in
analyses
Organizational plan inputs:
monetary resources,
budget

Business Plans vs. Process Theories
and Logic Models, Continued
Business plan format Purpose Process theory element Logic model
element
Risk and competitive
analysis
Discloses sources and types of possible
failures, with alternatives to avoid those
failures; balances failure risks with
program merits
Operational plan Shows how personnel, management,
space, and equipment come together;
delineates resource requirements
Organizational plan inputs: human,
informational, physical, managerial,
time resources, transportation
Inputs, activities
Marketing plan Describes strategy to reach target
audience, branding, distribution, price,
promotion
Service utilization plan: social
marketing, participants
Activities
Milestones Time frame to accomplish key tasks and
outcomes
Process and outcome objectives Immediate and
long-term
outcomes

Program Implementation across the Pyramid
– Tailor human resources to the intervention
– Social marketing plan will target individuals and individual behavior
– Providers likely to need a social services background
– More likely to use volunteers
– Social marketing tailored to the targeted aggregate •
•
‫للتدخل‬ ‫البشرية‬ ‫الموارد‬ ‫تخصيص‬
•
‫الفردي‬ ‫والسلوك‬ ‫األفراد‬ ‫االجتماعي‬ ‫التسويق‬ ‫خطة‬ ‫تستهدف‬ ‫سوف‬
•
•
‫االجتماعية‬ ‫الخدمات‬ ‫خلفية‬ ‫إلى‬ ‫يحتاجوا‬ ‫أن‬ ‫المحتمل‬ ‫من‬ ‫الخدمات‬ ‫مقدمي‬
•
‫المتطوعين‬ ‫الستخدام‬ ‫عرضة‬ ‫أكثر‬
•
‫المستهدف‬ ‫للمجموع‬ ‫ا‬ً‫ق‬‫وف‬ ‫المصمم‬ ‫االجتماعي‬ ‫التسويق‬

Program Implementation across the
Pyramid, Continued
– Also requires a skills match between providers and the intervention
– Social marketing likely to use mass media
– Strong organizational and services utilization plans are indicators of a solid
infrastructure
– Adequate infrastructure is essential to program success
•
•
‫والتدخل‬ ‫الخدمات‬ ‫مقدمي‬ ‫بين‬ ‫المهارات‬ ‫تطابق‬ ‫ًا‬‫ض‬‫أي‬ ‫يتطلب‬
•
‫اإلعالم‬ ‫وسائل‬ ‫تستخدم‬ ‫أن‬ ‫المرجح‬ ‫من‬ ‫االجتماعي‬ ‫التسويق‬
•
•
‫قوية‬ ‫تحتية‬ ‫بنية‬ ‫وجود‬ ‫على‬ ‫مؤشرات‬ ‫بمثابة‬ ‫القوية‬ ‫والخدمات‬ ‫التنظيمي‬ ‫االستخدام‬ ‫خطط‬ ‫تعد‬
•
‫البرنامج‬ ‫لنجاح‬ ‫ضرورية‬ ‫المناسبة‬ ‫التحتية‬ ‫البنية‬

Monitoring Implementation
Through Budgets & Information
Systems
Chapter 9
‫المعلومات‬ ‫وأنظمة‬ ‫الميزانيات‬ ‫خالل‬ ‫من‬ ‫التنفيذ‬ ‫مراقبة‬
Translated by
Khaled
Good Luck

Learning Objectives
• With this chapter, readers will be able to:
• Evaluate the extent to which the service utilization plan outputs were achieved.
• Select key methods for collecting process evaluation data.
• Explain the importance of intervention fidelity.
• Quantify coverage of the program.
• Critique a program cost evaluation with regard to its rigor.
• Distinguish between cost description, cost-effectiveness, cost-benefit, and cost-utility
analyses.
• Relate the process theory to the program costs.
•
:
•
‫الخدمة‬ ‫استخدام‬ ‫خطة‬ ‫مخرجات‬ ‫تحقيق‬ ‫مدى‬ ‫تقييم‬
.
•
‫العملية‬ ‫تقييم‬ ‫بيانات‬ ‫لجمع‬ ‫الرئيسية‬ ‫الطرق‬ ‫حدد‬
.
•
‫اإلخالص‬ ‫تدخل‬ ‫أهمية‬ ‫اشرح‬
.
•
‫البرنامج‬ ‫تغطية‬ ‫تحديد‬
.
•
‫بصرامته‬ ‫يتعلق‬ ‫فيما‬ ‫البرنامج‬ ‫تكلفة‬ ‫تقييم‬ ‫نقد‬
.
•
‫التكلفة‬ ‫وفائدة‬ ‫التكلفة‬ ‫وتحليل‬ ‫التكلفة‬ ‫وفعالية‬ ‫التكلفة‬ ‫تحليل‬ ‫بين‬ ‫التمييز‬
.
•
‫البرنامج‬ ‫بتكاليف‬ ‫العملية‬ ‫نظرية‬ ‫ربط‬
.

Key Terminology
• Break-Even Analysis
• Budget variance
• Charges
• Cost
• Cost analysis
• Cost-benefit analysis
• Cost comparison
• Cost description
• Cost-effectiveness analysis
• Cost minimization
• Cost-utility analysis
• Data Dictionary
• Direct cost
• Discounting
• Electronic Medical Record
(EMR)
• Electronic Health Record (EHR)
• Externalities
• Fixed costs
• Geographic Information Systems (GIS)
• Health Information Technology (HIT)
• Indirect cost
• In-kind donation
• Information systems
• Information technology
• Management information systems (MIS)
• Opportunity costs
• Variable costs

Two key elements exist that contribute to the success of a program and to the
ability to evaluate a program:
1- Budget.
2- The information systems.
‫تقييم‬ ‫على‬ ‫القدرة‬ ‫وفي‬ ‫البرنامج‬ ‫نجاح‬ ‫في‬ ‫يساهمان‬ ‫أساسيان‬ ‫عنصران‬ ‫يوجد‬
‫البرنامج‬
:
1
-
‫الميزانية‬
.
2
-
‫المعلومات‬ ‫نظم‬
.
‫مهم‬

BUDGETING AS PART OF PLANNING
Monetize and Compute Program Costs:
The majority of costs is related to the program implementation and includes the resources utilized by the program staff and
participants. Most program participants will want to know what it will cost them to participate. The charge for the program or
service is the most obvious cost. Other costs for program participants may exist, such as new or special equipment (e.g.,
handrails or cooking utensils), educational materials (e.g., books or magazine subscriptions), transportation, or child care while
attending the program.
Budget for Start-Up and Evaluation Costs:
For large-scale health programs that are being implemented for the first time, some costs will be specifically related to the
start-up phase and scaling-up. These cost include the one-time purchase of large or small equipment, the extra time needed to
find locations or negotiate contracts, or other one-time purchases needed to launch a program. (9-1).
Break-Even Analysis: After the program budget is complete and nearly final, it is possible to do a breakeven analysis.
Budget Justification:is a requirement for virtually all grant proposals, although the degree of detail expected varies by funding
agency. A safe rule of thumb is to provide a very detailed budget justification; more detailed budget justifications demonstrate
a more thorough program implementation and evaluation plan.
•
‫البرنامج‬ ‫تكاليف‬ ‫وحساب‬ ‫نقد‬
:
•
‫والمشاركون‬ ‫البرنامج‬ ‫موظفو‬ ‫يستخدمها‬ ‫التي‬ ‫الموارد‬ ‫وتشمل‬ ‫البرنامج‬ ‫بتنفيذ‬ ‫التكاليف‬ ‫غالبية‬ ‫ترتبط‬
.
‫المشاركة‬ ‫تكلفة‬ ‫معرفة‬ ‫في‬ ‫البرنامج‬ ‫في‬ ‫المشاركين‬ ‫معظم‬ ‫سيرغب‬
.
‫ال‬ ‫هي‬ ‫الخدمة‬ ‫أو‬ ‫البرنامج‬ ‫تكلفة‬
‫ا‬ً‫ح‬‫وضو‬ ‫األكثر‬ ‫تكلفة‬
.
‫قد‬
‫الخاصة‬ ‫أو‬ ‫الجديدة‬ ‫المعدات‬ ‫مثل‬ ، ‫البرنامج‬ ‫في‬ ‫للمشاركين‬ ‫أخرى‬ ‫تكاليف‬ ‫توجد‬
(
‫الطهي‬ ‫أواني‬ ‫أو‬ ‫الدرابزين‬ ‫مثل‬
)
‫التعليمية‬ ‫المواد‬ ‫أو‬
(
‫المجالت‬ ‫أو‬ ‫الكتب‬ ‫اشتراكات‬ ‫مثل‬
)
‫البر‬ ‫حضور‬ ‫أثناء‬ ‫األطفال‬ ‫رعاية‬ ‫أو‬ ‫النقل‬ ‫أو‬
‫نامج‬
.
•
‫والتقييم‬ ‫التشغيل‬ ‫بدء‬ ‫تكاليف‬ ‫ميزانية‬
:
•
‫والتوسعة‬ ‫البدء‬ ‫بمرحلة‬ ‫خاص‬ ‫بشكل‬ ‫مرتبطة‬ ‫التكاليف‬ ‫بعض‬ ‫ستكون‬ ، ‫مرة‬ ‫ألول‬ ‫تنفيذها‬ ‫يتم‬ ‫التي‬ ‫النطاق‬ ‫واسعة‬ ‫الصحية‬ ‫للبرامج‬ ‫بالنسبة‬
.
‫الص‬ ‫أو‬ ‫الكبيرة‬ ‫المعدات‬ ‫من‬ ‫واحدة‬ ‫لمرة‬ ‫شراء‬ ‫التكلفة‬ ‫هذه‬ ‫وتشمل‬
‫والوقت‬ ، ‫غيرة‬
‫ل‬ ‫الالزم‬ ‫اإلضافي‬
•
‫البرنامج‬ ‫إلطالق‬ ‫الالزمة‬ ‫واحدة‬ ‫لمرة‬ ‫الشراء‬ ‫عمليات‬ ‫من‬ ‫غيرها‬ ‫أو‬ ، ‫العقود‬ ‫على‬ ‫التفاوض‬ ‫أو‬ ‫مواقع‬ ‫على‬ ‫العثور‬
( .
9
-
1
.)
•
‫التعادل‬ ‫تحليل‬
:
‫الميزانية‬ ‫تكون‬ ‫أن‬ ‫بعد‬
‫التعادل‬ ‫تحليل‬ ‫إجراء‬ ‫يمكن‬ ، ‫نهائية‬ ‫وشبه‬ ‫كاملة‬
.
•
‫الموازنة‬ ‫تبرير‬
:
‫التمويل‬ ‫وكالة‬ ‫حسب‬ ‫تختلف‬ ‫المتوقعة‬ ‫التفاصيل‬ ‫درجة‬ ‫أن‬ ‫من‬ ‫الرغم‬ ‫على‬ ، ‫ا‬ً‫ب‬‫تقري‬ ‫المنح‬ ‫مقترحات‬ ‫لكل‬ ‫شرط‬ ‫هو‬
.
‫م‬ ‫توضح‬ ‫؛‬ ‫للميزانية‬ ‫للغاية‬ ‫مفصل‬ ‫تبرير‬ ‫تقديم‬ ‫في‬ ‫اآلمنة‬ ‫القاعدة‬ ‫تتمثل‬
‫األكثر‬ ‫الموازنة‬ ‫بررات‬
ً‫ل‬‫شمو‬ ‫أكثر‬ ‫وتقييم‬ ‫تنفيذ‬ ‫خطة‬ ً‫ال‬‫تفصي‬
.
‫مهم‬

Relevance of Process Theory to
Economic Evaluations
‫مهم‬
‫أهمية‬
‫النظرية‬
‫االقتصادية‬ ‫التقييمات‬ ‫في‬ ‫العملية‬

Using Causal Theory to Identify
Potential Costs and Benefits

Criteria for Choosing Economic Evaluations
• Comparing 2 programs for the same health problem
• Whether only costs are being considered
• Whether costs and outcomes of different health programs are being
compared
•
‫المشكلة‬ ‫لنفس‬ ‫برنامجين‬ ‫مقارنة‬
‫الصحية‬
•
‫ما‬
‫التكاليف‬ ‫في‬ ‫النظر‬ ‫يتم‬ ‫كان‬ ‫إذا‬
‫فقط‬
•
‫ما‬
‫المختلفة‬ ‫الصحية‬ ‫البرامج‬ ‫ونتائج‬ ‫تكاليف‬ ‫مقارنة‬ ‫يتم‬ ‫كان‬ ‫إذا‬

Steps in an Economic Analysis
1. Define the problem
2. Stipulate comparison parameters
3. Develop decision rules
4. Choose an accounting perspective
5. Monetize and compute program costs
6. Adjust for time
7. Identify and measure program effects
8. Monetize the effects
9. Conduct an analysis
10. Conduct a sensitivity analysis
11. Disseminate the findings
‫مهم‬
.
‫المشكلة‬ ‫عرف‬ 1
.
‫ستيب‬ 2
‫المقارنة‬ ‫معلمات‬
.
‫القرار‬ ‫قواعد‬ ‫تطوير‬ 3
.
‫المحاسبة‬ ‫منظور‬ ‫اختيار‬ 4
.
‫البرنامج‬ ‫تكاليف‬ ‫وحساب‬ ‫نقد‬ 5
.
‫الوقت‬ ‫ضبط‬ 6
.
‫البرنامج‬ ‫آثار‬ ‫وقياس‬ ‫تحديد‬ 7
.
‫اآلثار‬ ‫نقد‬ 8
.
‫تحليل‬ ‫إجراء‬ 9
.
‫الحساسية‬ ‫تحليل‬ ‫إجراء‬ 01
.
‫النتائج‬ ‫نشر‬ 11
‫خطوات‬
‫االقتصادي‬ ‫التحليل‬

Stipulate Comparison Parameters
• N/A for cost descriptions or simple cost analyses
• If comparing 2 programs, justify their selection and the
selected variables
• State programs’ goals and objectives
•
N / A
‫البسيط‬ ‫التكلفة‬ ‫تحليل‬ ‫أو‬ ‫التكلفة‬ ‫لوصف‬
•
‫المحددة‬ ‫والمتغيرات‬ ‫اختيارهما‬ ‫بتبرير‬ ‫قم‬ ، ‫برنامجين‬ ‫لمقارنة‬
•
‫وأهدافها‬ ‫الدولة‬ ‫برامج‬

Develop Decision Rules
• Definitions of key parameters
• Time frame for measuring costs and outcomes
• Breadth of program costs
• Participant characteristics included in analyses
• Interpretation of the results
– Ceiling on cost per unit of benefit
– Minimum acceptable amount of effect
– Program budget limit
•
‫الرئيسية‬ ‫المعلمات‬ ‫تعاريف‬
•
‫والنتائج‬ ‫التكاليف‬ ‫لقياس‬ ‫الزمني‬ ‫اإلطار‬
•
‫البرنامج‬ ‫تكاليف‬ ‫اتساع‬
•
‫التحليالت‬ ‫في‬ ‫المدرجة‬ ‫المشاركين‬ ‫خصائص‬
•
‫النتائج‬ ‫تفسير‬
•
‫فائدة‬ ‫وحدة‬ ‫لكل‬ ‫التكلفة‬ ‫على‬ ‫السقف‬
•
‫تأثير‬ ‫من‬ ‫مقبول‬ ‫األدنى‬ ‫الحد‬
•
‫البرنامج‬ ‫ميزانية‬ ‫حد‬

Relationship Between Cost and Effectiveness

Accounting Perspectives
Accounting
perspective
Costs Benefits and outcomes
Participants Medical costs, treatment & medication costs,
participation costs
Program-specific impacts, secondary effects on
family, quality of life
Program Program costs, opportunity costs Visibility and goodwill (marketing value), increased
program or agency funding
Payers Program costs, loss of revenues, increases in
expenditures
Increased revenues or decreased expenditures
Society Lost taxes, disability support, criminal justice Gain in taxes, reduced public service costs (e.g.,
police, fire), reductions in costs to family members
related to health problem, reductions in costs to
support family due to health problem

Monetize Program Costs
• Attach dollar amounts to:
– Antecedents
– Causal factors
– Moderators
– Mediators
– Outcomes
– Impacts-,Costs/expenses of participants
•
‫إلى‬ ‫بالدوالر‬ ‫المبالغ‬ ‫إرفاق‬
:
•
‫أسالف‬
•
‫المسببة‬ ‫العوامل‬
•
‫المشرفون‬
•
‫وسطاء‬
•
‫النتائج‬
•
‫التكاليف‬ ، ‫اآلثار‬
/
‫نفقات‬
‫المشاركين‬
‫مهم‬

Using Causal Theory to Identify Potential Costs and Benefits

Adjust for Time
• Discounting
- is the process of converting future dollars, and future health benefits, into the
present dollar value.
- It involves decreasing the current value by a rate, usually between 1% and 3%,
on an annual basis. In a sense, discounting is the reverse of interest.
– Usually 1 – 3% annually
• Inflation
– Applied to program expenses
• Depreciation(decreasing the value of assets over time .
– Decreasing value of capital (e.g., computers).
•
‫خصم‬
•
-
‫الحالية‬ ‫الدولر‬ ‫قيمة‬ ‫إلى‬ ، ‫المستقبلية‬ ‫الصحية‬ ‫والفوائد‬ ، ‫المستقبلية‬ ‫الدولرات‬ ‫تحويل‬ ‫عملية‬ ‫هي‬
.
•
-
‫بين‬ ‫عادة‬ ‫يتراوح‬ ‫بمعدل‬ ‫الحالية‬ ‫القيمة‬ ‫خفض‬ ‫على‬ ‫ذلك‬ ‫ينطوي‬
1
‫و‬ ٪
3
‫سنوي‬ ‫أساس‬ ‫على‬ ٪
.
‫الهتمام‬ ‫عكس‬ ‫هو‬ ‫الخصم‬ ، ‫ما‬ ‫بمعنى‬
.
•
‫عادة‬
1
-
3
‫سنويا‬ ٪
•
‫التضخم‬
•
‫البرنامج‬ ‫مصاريف‬ ‫على‬ ‫تطبق‬
•
‫الستهالك‬
(
‫الوقت‬ ‫مرور‬ ‫مع‬ ‫األصول‬ ‫قيمة‬ ‫تقليل‬
.
•
‫المال‬ ‫رأس‬ ‫قيمة‬ ‫انخفاض‬
(
‫الكمبيوتر‬ ‫أجهزة‬ ، ‫المثال‬ ‫سبيل‬ ‫على‬
.)
‫مهم‬

Identify and Measure Program Effects
It is include two things:
1. Use data from outcome evaluation
2. Consider externalities(secondary effect) Identifying externalities is
important because they then become effects (if beneficial) or costs (if
harmful or in some way costly).
• Secondary effects, whether anticipated or not, may occur, and the effects can
spill over to individuals not participating in the program
• Externalities may be indirect or even unanticipated consequences of the
program, whether beneficial or harmful.
•
‫شيئين‬ ‫تشمل‬ ‫وهي‬
:
•
‫النتائج‬ ‫تقييم‬ ‫من‬ ‫البيانات‬ ‫استخدام‬
•
‫الخارجية‬ ‫العوامل‬ ‫في‬ ‫النظر‬
(
‫الثانوي‬ ‫التأثير‬
)
‫ا‬ً‫آثار‬ ‫تصبح‬ ‫ألنها‬ ‫مهم‬ ‫أمر‬ ‫الخارجية‬ ‫العوامل‬ ‫تحديد‬
(
‫مفيدة‬ ‫كانت‬ ‫إذا‬
)
‫تكاليف‬ ‫أو‬
(
‫ما‬ ‫حد‬ ‫إلى‬ ‫مكلفة‬ ‫أو‬ ‫ضارة‬ ‫كانت‬ ‫إذا‬
.)
•
‫البرنامج‬ ‫في‬ ‫المشاركين‬ ‫غير‬ ‫األفراد‬ ‫إلى‬ ‫التأثيرات‬ ‫تمتد‬ ‫أن‬ ‫ويمكن‬ ، ‫ل‬ ‫أم‬ ‫متوقعة‬ ‫كانت‬ ‫سواء‬ ، ‫ثانوية‬ ‫تأثيرات‬ ‫تحدث‬ ‫قد‬
•
‫ضارة‬ ‫أو‬ ‫مفيدة‬ ‫كانت‬ ‫سواء‬ ، ‫للبرنامج‬ ‫متوقعة‬ ‫غير‬ ‫أو‬ ‫مباشرة‬ ‫غير‬ ‫عواقب‬ ‫الخارجية‬ ‫العوامل‬ ‫تكون‬ ‫قد‬
.
‫مهم‬

Monetizing Outcomes and Impacts
‫النقدية‬ ‫واآلثار‬ ‫النتائج‬ ‫تحقيق‬
Approach Method
Market value (of health state or ongoing medical
treatment for illnesses)
Obtain price for treatments, wages
Client willingness to pay Observe choices that are made by program
participants or members of the target audience
Policy maker’s view of the value Ask policy maker to estimate the dollar value of the
impact
Practitioner’s view of the value Ask health professionals to estimate the dollar
value of the impact

Conduct the Analysis and Sensitivity Analysis
• Simple analyses can be done in Excel; more complex ones may
need special software
• Purpose of sensitivity analysis
1. Identify the most influential parameter
2. Establish the range in which program or outcome costs have minimal
effect on conclusions.
‫مهم‬
•
‫في‬ ‫بسيطة‬ ‫تحليالت‬ ‫إجراء‬ ‫يمكن‬
Excel
‫خاصة‬ ‫برامج‬ ‫تحتاج‬ ‫قد‬ ‫تعقيدا‬ ‫أكثر‬ ‫؛‬
•
‫الحساسية‬ ‫تحليل‬ ‫من‬ ‫الغرض‬
•
‫نفوذا‬ ‫األكثر‬ ‫المعلمة‬ ‫تحديد‬
•
‫االستنتاجات‬ ‫على‬ ‫ضئيل‬ ‫تأثير‬ ‫فيه‬ ‫النتائج‬ ‫أو‬ ‫البرنامج‬ ‫لتكاليف‬ ‫يكون‬ ‫الذي‬ ‫النطاق‬ ‫حدد‬
.

BUDGET AS A MONITORING TOOL
• A budget developed during the planning stage becomes a tool to monitor implementation
after the program has started. In addition, the careful attention to tracking expenses and
income can be helpful in doing different types of cost analyses as part of a monitoring and
subsequent planning cycle.
• Budget Variance:On an ongoing and regular basis (usually monthly), the
program manager ought to determine the extent to which current
expenditures exceed (or not) the projected program expenditures.
• The difference between the budgeted and actual expenditures or income is
called the budget variance.
•
‫البرنامج‬ ‫بدء‬ ‫بعد‬ ‫التنفيذ‬ ‫لمراقبة‬ ‫أداة‬ ‫التخطيط‬ ‫مرحلة‬ ‫خالل‬ ‫تطويرها‬ ‫تم‬ ‫التي‬ ‫الميزانية‬ ‫تصبح‬
.
‫م‬ ‫واإليرادات‬ ‫النفقات‬ ‫بتتبع‬ ‫الدقيق‬ ‫االهتمام‬ ‫يكون‬ ‫قد‬ ، ‫ذلك‬ ‫إلى‬ ‫باإلضافة‬
‫من‬ ‫مختلفة‬ ‫أنواع‬ ‫إجراء‬ ‫في‬ ‫ًا‬‫د‬‫في‬
‫الالحق‬ ‫والتخطيط‬ ‫الرصد‬ ‫دورة‬ ‫من‬ ‫كجزء‬ ‫التكلفة‬ ‫تحليالت‬
.
•
‫الميزانية‬ ‫تباين‬
:
‫ومنتظم‬ ‫مستمر‬ ‫أساس‬ ‫على‬
(
‫ًا‬‫ي‬‫شهر‬ ‫يكون‬ ‫ما‬ ً‫ة‬‫عاد‬
)
‫الحالية‬ ‫النفقات‬ ‫تجاوز‬ ‫مدى‬ ‫تحديد‬ ‫البرنامج‬ ‫مدير‬ ‫على‬ ‫يجب‬ ،
(
‫عدم‬ ‫أو‬
)
‫المتوقعة‬ ‫البرنامج‬ ‫نفقات‬
.
•
‫الميزانية‬ ‫في‬ ‫الفرق‬ ‫الميزانية‬ ‫في‬ ‫المدرجة‬ ‫الفعلية‬ ‫اإليرادات‬ ‫أو‬ ‫المصروفات‬ ‫بين‬ ‫الفرق‬ ‫على‬ ‫ُطلق‬‫ي‬
.
‫مهم‬

Types of Cost Analyses
One program Two or more programs
Costs only Cost description
Cost analysis
Cost minimization
Cost comparison
Costs and effects Cost analysis Cost-effectiveness (CEA)
Cost-benefit (CBA)
Cost-utility (CUA)
‫مهم‬

CEA, CBA, and CUA
Type of analysis Basic formula Outcomes, impacts,
benefits
Costs, expenses
Cost-effectiveness analysis
(CEA)
CEA = (total cost $) / (health
effect unit)
Program direct health
impacts
Program direct + indirect
costs
Cost-benefit analysis (CBA) Net benefit = total benefit $ -
total cost $
Dollar values of program
direct impacts + indirect
program outcomes + long-
term social consequences,
willingness to pay, life
expectancy
Program direct + indirect
costs + medical costs + non-
medical costs + opportunity
costs + other societal costs
Cost-utility analysis (CUA) CUA = (total cost $) / (utility
costs)
Preferences for health state Program costs + medical
costs + non-medical costs
‫مهم‬

Cost Description, Analysis, Minimization, and Comparison
• Cost description
– Direct program costs, no effects
– Process monitoring
• Cost analysis
– Costs by element of organizational or services utilization plan
– Process monitoring
• Cost minimization
– Program delivery at the lowest possible cost
– Part of budgeting process
• Above can be used during program planning
• Cost comparison
– Costs per participant without examining outcomes or impacts
– May be used to compare varied programs within an agency
‫مهم‬
•
‫التكلفة‬ ‫وصف‬
•
‫آثار‬ ‫أي‬ ، ‫المباشرة‬ ‫البرنامج‬ ‫تكاليف‬
•
‫الرصد‬ ‫عملية‬
•
‫التكلفة‬ ‫تحليل‬
•
‫الخدمات‬ ‫استخدام‬ ‫أو‬ ‫التنظيمية‬ ‫الخطة‬ ‫عنصر‬ ‫حسب‬ ‫التكاليف‬
•
•
‫التكلفة‬ ‫تقليل‬
•
‫ممكنة‬ ‫تكلفة‬ ‫بأقل‬ ‫البرنامج‬ ‫تسليم‬
•
‫الميزانية‬ ‫وضع‬ ‫عملية‬ ‫من‬ ‫جزء‬
•
‫البرنامج‬ ‫تخطيط‬ ‫أثناء‬ ‫استخدامها‬ ‫يمكن‬ ‫أعاله‬
•
‫التكاليف‬ ‫مقارنة‬
•
‫اآلثار‬ ‫أو‬ ‫النتائج‬ ‫فحص‬ ‫دون‬ ‫مشارك‬ ‫لكل‬ ‫التكاليف‬
•
‫الوكالة‬ ‫داخل‬ ‫المختلفة‬ ‫البرامج‬ ‫لمقارنة‬ ‫استخدامها‬ ‫يمكن‬

Cost-Effectiveness Analysis
• 2 programs compared by cost per unit of the same outcome:
(total costs of program A) – (total costs of program B)
(effects of program A) – (effects of program B)
• Alternative is incremental improvement in CE:
program cost per participant
amount of change in an indicator
• Applications:
– Planning phase for choosing an intervention
– Post-intervention to compare program vs. no program
•
‫النتيجة‬ ‫نفس‬ ‫من‬ ‫وحدة‬ ‫لكل‬ ‫بالكلفة‬ ‫مقارنة‬ ‫برنامجان‬
:
•
(
‫أ‬ ‫البرنامج‬ ‫تكاليف‬ ‫إجمالي‬
)
-
(
‫ب‬ ‫البرنامج‬ ‫تكاليف‬ ‫إجمالي‬
)
•
(
‫البرنامج‬ ‫تأثيرات‬
A
)
-
(
‫البرنامج‬ ‫تأثيرات‬
B
)
•
‫في‬ ‫تدريجي‬ ‫تحسين‬ ‫هو‬ ‫البديل‬
CE
:
•
‫مشارك‬ ‫لكل‬ ‫البرنامج‬ ‫تكلفة‬
•
‫المؤشر‬ ‫في‬ ‫التغيير‬ ‫مقدار‬
•
‫التطبيقات‬
:
•
‫التدخل‬ ‫لختيار‬ ‫التخطيط‬ ‫مرحلة‬
•
‫برنامج‬ ‫وجود‬ ‫عدم‬ ‫مقابل‬ ‫البرنامج‬ ‫لمقارنة‬ ‫التدخل‬ ‫بعد‬
‫مهم‬

Example of Incremental CEA Program A (usual care) Program B (new
program)
Program C (modified
new)
Amt of improvement in impact objective 1 (↑
perceived health status score)
2 units 5 units 10 units
Cost / unit of higher perceived health status $2 $2 $0.70
Amt of improvement in impact objective 2 (addt’l mins
of exercise)
3 min 20 min 15 min
Cost / addt’l min of exercise $1.33 $0.50 $0.47
Amt of improvement in impact objective 3 (↓ diastolic
BP)
2 mm Hg 4 mm Hg 4 mm Hg
Cost / ↓ mm Hg $2.00 $2.50 $1.75
Example of Simple CEA Program A (usual care) Program B (new program) Program C (modified new)
Total program costs $400 $1000 $700
# participants / program 100 100 100
Total program cost / participant $4 $10 $7
Total amt of overall improvement 5% 25% 20%
Overall CE $800 $400 $350
Overall CE / participant = (cost /
participant) ± (amt of
improvement)
$80 $40 $35

Decision Matrix for Choosing Between
Programs
Costs: Program A costs…
Effect: Program A does…
More than B
(A > B)
Same as B
(A = B)
Less than B
(A < B)
More than B (A > B) Undecided Choose B Choose B
Same as B (A = B) Choose A Either Choose B
Less than B (A < B) Choose A Choose A Undecided

Cost-Effectiveness Analysis
• 2 programs compared on cost/dollar value of benefits:
($ value of tangible benefits) + ($ value of intangible benefits)
total program costs
• May also use net benefit:
($ value of tangible benefits + $ value of intangible benefits) – (direct costs + indirect costs)
• Applications:
– Prioritization phase
– Compare programs for different health problems or across pyramid levels
‫مهم‬
•
‫بالتكلفة‬ ً‫ة‬‫مقارن‬ ‫برنامجان‬
/
‫للفوائد‬ ‫بالدوالر‬ ‫القيمة‬
:
•
(
‫الملموسة‬ ‫للفوائد‬ ‫دوالر‬ ‫قيمة‬
( + )
‫الملموسة‬ ‫غير‬ ‫للفوائد‬ ‫دوالر‬ ‫قيمة‬
)
•
‫البرنامج‬ ‫تكاليف‬ ‫مجموع‬
•
‫الصافية‬ ‫الفائدة‬ ‫ا‬ً‫ض‬‫أي‬ ‫تستخدم‬ ‫قد‬
:
•
(
‫الملموسة‬ ‫للفوائد‬ ‫دوالر‬ ‫قيمة‬
+
‫الملموسة‬ ‫غير‬ ‫للفوائد‬ ‫دوالر‬ ‫قيمة‬
)
-
(
‫المباشرة‬ ‫التكاليف‬
+
‫المباش‬ ‫غير‬ ‫التكاليف‬
‫رة‬
)
•
:
•
‫األولويات‬ ‫تحديد‬ ‫مرحلة‬
•
‫الهرم‬ ‫مستويات‬ ‫عبر‬ ‫أو‬ ‫المختلفة‬ ‫الصحية‬ ‫للمشاكل‬ ‫البرامج‬ ‫بين‬ ‫قارن‬

Cost-Utility Analysis
• Cost per unit of participant preference for a given health state (utility):
total costs associated with the program
utility costs
• Utilities obtained through judgment, literature, or from participants
– Measured via scales, standard gamble, time trade-off, or paired comparisons
• Applications:
– QOL is an important programmatic outcome
– Difficult to use for planning or policy purposes
‫مهم‬
•
‫معينة‬ ‫صحية‬ ‫لحالة‬ ‫المشارك‬ ‫تفضيل‬ ‫من‬ ‫وحدة‬ ‫لكل‬ ‫التكلفة‬
(
‫فائدة‬
:)
•
‫بالبرنامج‬ ‫المرتبطة‬ ‫التكاليف‬ ‫إجمالي‬
•
‫المرافق‬ ‫تكاليف‬
•
‫المشاركين‬ ‫من‬ ‫أو‬ ، ‫واألدب‬ ، ‫الحكم‬ ‫خالل‬ ‫من‬ ‫عليها‬ ‫الحصول‬ ‫تم‬ ‫التي‬ ‫المرافق‬
•
‫المزدوجة‬ ‫المقارنات‬ ‫أو‬ ‫الزمنية‬ ‫المقايضة‬ ‫أو‬ ‫القياسية‬ ‫المقامرة‬ ‫أو‬ ‫المقاييس‬ ‫عبر‬ ‫تقاس‬
•
:
•
QOL
‫نتيجة‬ ‫هو‬
‫برنامجية‬
‫مهمة‬
•
‫السياسة‬ ‫أو‬ ‫التخطيط‬ ‫ألغراض‬ ‫االستخدام‬ ‫صعب‬

Criteria for Assessing Economic Evaluations
• Well-defined economic Q
• Descriptions of comparison programs
• Costs and effects identified
• Credible costs assigned
• Time adjustments
• Software used
• Limitations and implications acknowledged and explained
•
‫س‬ ‫المعالم‬ ‫واضحة‬ ‫االقتصادية‬
•
‫المقارنة‬ ‫برامج‬ ‫وصف‬
•
‫المحددة‬ ‫واآلثار‬ ‫التكاليف‬
•
‫المعينة‬ ‫موثوقة‬ ‫تكاليف‬
•
‫الوقت‬ ‫تعديالت‬
•
‫المستخدمة‬ ‫البرمجيات‬
•
‫والموضحة‬ ‫بها‬ ‫المعترف‬ ‫واآلثار‬ ‫القيود‬

The strategic goals of these pieces of legislation, essentially, are to have:
(1) a nationally adopted standard terminology for health data elements;
(2) secure mechanisms for electronically sharing and transmitting health
information.
(3) a data repository that can be used to monitor in real-time health events
and trends.
• The following introduction to health informatics and public health
informatics forms a base for understanding the issues and processes
relevant to health program managers and developers.
INFORMATION SYSTEMS
•
‫هي‬ ، ‫األساس‬ ‫في‬ ، ‫التشريعات‬ ‫لهذه‬ ‫االستراتيجية‬ ‫األهداف‬
:
•
‫؛‬ ‫الصحية‬ ‫البيانات‬ ‫لعناصر‬ ‫الوطني‬ ‫المستوى‬ ‫على‬ ‫المعتمدة‬ ‫القياسية‬ ‫المصطلحات‬
•
ً‫ا‬‫إلكتروني‬ ‫ونقلها‬ ‫الصحية‬ ‫المعلومات‬ ‫لتبادل‬ ‫آليات‬ ‫تأمين‬
.
•
‫الفعلي‬ ‫الوقت‬ ‫في‬ ‫الصحية‬ ‫واالتجاهات‬ ‫األحداث‬ ‫لرصد‬ ‫استخدامه‬ ‫يمكن‬ ‫بيانات‬ ‫مستودع‬
.
•
‫البرامج‬ ‫ومطوري‬ ‫بمديري‬ ‫الصلة‬ ‫ذات‬ ‫والعمليات‬ ‫القضايا‬ ‫لفهم‬ ‫ا‬ً‫س‬‫أسا‬ ‫العامة‬ ‫الصحة‬ ‫ومعلوماتية‬ ‫الصحية‬ ‫للمعلوماتية‬ ‫التالية‬ ‫المقدمة‬ ‫تشكل‬
‫ال‬
‫صحية‬
.

Health Informatics Terminology
Knowing how to use a computer and the Internet is not the same as understanding health
informatics. In the broadest sense, an information system encompasses both the hardware and
the software used to collect, store, and exchange digital information for a work unit, whether a
healthcare organization or a utilities company. Information technology refers the products used
within the information system, such as Microsoft
Office and Ethernet Internet connections. Most organizations have a management information
system (MIS), which a subset of the information system designed to address managerial, fiscal,
and human resources processes and data. Appointment scheduling software might be a
component of the MIS for a healthcare organization, as would be any accounting software.
Increasingly, geographic information systems (GIS) are used to collect, analyze, and display
data using maps. John Snow, who mapped new cases of cholera in London in 1854, was essentially
the first to use GIS in public health.
•
‫الصحية‬ ‫المعلوماتية‬ ‫فهم‬ ‫عن‬ ‫يختلف‬ ‫ال‬ ‫واإلنترنت‬ ‫الكمبيوتر‬ ‫استخدام‬ ‫كيفية‬ ‫معرفة‬
.
‫ال‬ ‫وتبادل‬ ‫وتخزين‬ ‫لجمع‬ ‫المستخدمة‬ ‫والبرامج‬ ‫األجهزة‬ ‫من‬ ‫كل‬ ‫المعلومات‬ ‫نظام‬ ‫يشمل‬ ، ‫أوسع‬ ‫بمعنى‬
‫لوحدة‬ ‫الرقمية‬ ‫معلومات‬
‫مساعدة‬ ‫أدوات‬ ‫شركة‬ ‫أو‬ ‫صحية‬ ‫رعاية‬ ‫مؤسسة‬ ‫أكانت‬ ‫سواء‬ ، ‫العمل‬
.
‫مثل‬ ، ‫المعلومات‬ ‫نظام‬ ‫في‬ ‫المستخدمة‬ ‫المنتجات‬ ‫إلى‬ ‫المعلومات‬ ‫تكنولوجيا‬ ‫تشير‬
Microsoft
•
‫اإلنترنت‬ ‫وإيثرنت‬ ‫مكتب‬ ‫اتصاالت‬
.
‫إدارة‬ ‫معلومات‬ ‫نظام‬ ‫المؤسسات‬ ‫معظم‬ ‫لدى‬
(
MIS
)
‫والبشرية‬ ‫والمالية‬ ‫اإلدارية‬ ‫الموارد‬ ‫وبيانات‬ ‫عمليات‬ ‫لمعالجة‬ ‫مصممة‬ ‫المعلومات‬ ‫نظام‬ ‫من‬ ‫فرعية‬ ‫مجموعة‬ ‫وهو‬ ،
.
‫قد‬
‫محاسبة‬ ‫برنامج‬ ‫أي‬ ‫مثل‬ ، ‫الصحية‬ ‫الرعاية‬ ‫لمؤسسة‬ ‫اإلدارية‬ ‫المعلومات‬ ‫نظام‬ ‫مكونات‬ ‫أحد‬ ‫المواعيد‬ ‫جدولة‬ ‫برنامج‬ ‫يكون‬
.
‫الجغرافية‬ ‫المعلومات‬ ‫أنظمة‬ ‫استخدام‬ ‫يتم‬
(
GIS
)
‫وتحليلها‬ ‫البيانات‬ ‫لجمع‬ ‫متزايد‬ ‫بشكل‬
‫الخرائط‬ ‫باستخدام‬ ‫وعرضها‬
.
‫عام‬ ‫لندن‬ ‫في‬ ‫للكوليرا‬ ‫الجديدة‬ ‫الحاالت‬ ‫رسم‬ ‫الذي‬ ، ‫سنو‬ ‫جون‬ ‫كان‬
1854
‫العامة‬ ‫الصحة‬ ‫في‬ ‫الجغرافية‬ ‫المعلومات‬ ‫نظام‬ ‫استخدم‬ ‫من‬ ‫أول‬ ،
.

health information technology (HIT) broadly encompasses all of the technology in use within
healthcare organizations, from computerized imaging to tweets of health messages. Until the passage of
the American Recovery and Reconstruction Act (ARRA) in 2009, focus centered on medical technology
linked to imaging or devices or on the documents related to the medical record. The ARRA provided
financial incentives for physicians and healthcare organizations to speed up the adoption and use of HIT.
Thus, emphasis has shifted from the electronic medical record (EMR), the analog to the more
traditional paper medical record maintained by a physician, to the electronic health record (EHR),
which has been defined as the longitudinal electronic record of a patient’s health information generated
by one or more encounters in any care delivery setting .
•
‫الصحية‬ ‫المعلومات‬ ‫تكنولوجيا‬ ‫تشمل‬
(
HIT
)
‫إلى‬ ‫المحوسب‬ ‫التصوير‬ ‫من‬ ، ‫الصحية‬ ‫الرعاية‬ ‫مؤسسات‬ ‫داخل‬ ‫المستخدمة‬ ‫التكنولوجيا‬ ‫كل‬ ‫واسع‬ ‫نطاق‬ ‫على‬
‫التغريدات‬
‫الصحية‬ ‫بالرسائل‬ ‫الخاصة‬
.
‫ح‬
‫إصدار‬ ‫تى‬
‫األمريكي‬ ‫اإلعمار‬ ‫وإعادة‬ ‫التعافي‬ ‫قانون‬
(
ARRA
)
‫عام‬ ‫في‬
2009
‫الطبي‬ ‫بالسجل‬ ‫المتعلقة‬ ‫الوثائق‬ ‫على‬ ‫أو‬ ‫األجهزة‬ ‫أو‬ ‫بالتصوير‬ ‫المرتبطة‬ ‫الطبية‬ ‫التكنولوجيا‬ ‫على‬ ‫التركيز‬ ‫تركز‬ ،
.
‫قدمت‬
ARRA
‫الحوافز‬
‫واستخدام‬ ‫اعتماد‬ ‫لتسريع‬ ‫الصحية‬ ‫الرعاية‬ ‫ومؤسسات‬ ‫لألطباء‬ ‫المالية‬
HIT
.
‫اإللكتروني‬ ‫الطبي‬ ‫السجل‬ ‫من‬ ‫التركيز‬ ‫تحول‬ ، ‫وهكذا‬
(
EMR
)
‫به‬ ‫يحتفظ‬ ‫الذي‬ ‫تقليدية‬ ‫األكثر‬ ‫الورقي‬ ‫الطبي‬ ‫السجل‬ ‫إلى‬ ‫النظير‬ ،
‫اإللكتروني‬ ‫الصحي‬ ‫السجل‬ ‫إلى‬ ، ‫الطبيب‬
(
EHR
)
‫ال‬ ‫لتقديم‬ ‫إعداد‬ ‫أي‬ ‫في‬ ‫أكثر‬ ‫أو‬ ‫واحدة‬ ‫مواجهة‬ ‫عن‬ ‫الناتجة‬ ‫المعلومات‬ ‫المريض‬ ‫لصحة‬ ‫الطولي‬ ‫اإللكتروني‬ ‫السجل‬ ‫أنه‬ ‫على‬ ‫تعريفه‬ ‫تم‬ ‫والذي‬ ،
‫رعا‬
‫ي‬
‫ة‬
.

Throughout the program planning and evaluation cycle, the information system
plays an important role as the means to retrieve existing or collect new data,
store data and other program data elements, and then utilized the data
elements in the form of reports. Developing and using a data dictionary, a list of
the names of data elements to be collected with definitions of those date
elements, helps assure a standardized definition and consistency in use and
(AHIMA e-HIM Work Group on EHR Data Content, 2006).
Information Systems Considerations
•
‫البي‬ ‫وتخزين‬ ، ‫الجديدة‬ ‫أو‬ ‫الموجودة‬ ‫البيانات‬ ‫السترداد‬ ‫كوسيلة‬ ‫ا‬ً‫م‬‫مه‬ ‫ا‬ً‫دور‬ ‫المعلومات‬ ‫نظام‬ ‫يلعب‬ ، ‫وتقييمها‬ ‫البرامج‬ ‫تخطيط‬ ‫دورة‬ ‫طوال‬
‫انا‬
‫وعناص‬ ‫ت‬
‫البرنامج‬ ‫بيانات‬ ‫ر‬
‫تقارير‬ ‫شكل‬ ‫في‬ ‫البيانات‬ ‫عناصر‬ ‫استخدام‬ ‫ثم‬ ، ‫األخرى‬
.
‫جم‬ ‫سيتم‬ ‫التي‬ ‫البيانات‬ ‫عناصر‬ ‫بأسماء‬ ‫قائمة‬ ، ‫بيانات‬ ‫قاموس‬ ‫واستخدام‬ ‫تطوير‬ ‫يساعد‬
‫لعناصر‬ ‫تعريفات‬ ‫مع‬ ‫عها‬
‫و‬ ‫االستخدام‬ ‫في‬ ‫وتناسق‬ ‫موحد‬ ‫تعريف‬ ‫ضمان‬ ‫على‬ ، ‫هذه‬ ‫التاريخ‬
(
‫عمل‬ ‫مجموعة‬
AHIMA e-HIM
‫بيانات‬ ‫محتوى‬ ‫على‬
EHR
،
2006
.)

Economic Analyses Across the Pyramid
– CEA more likely
– Medical economic evaluations usually at this level
– CBA or CUA more likely
– Challenges at this level:
• Benefits are intangible
• Large indirect costs
• Considerable/varied family costs and benefits
• Often hidden within other programs
•
•
CEA
‫األرجح‬ ‫على‬
•
‫المستوى‬ ‫هذا‬ ‫على‬ ‫عادة‬ ‫الطبية‬ ‫االقتصادية‬ ‫التقييمات‬
•
•
CBA
‫أو‬
CUA
‫األرجح‬ ‫على‬
•
‫المستوى‬ ‫هذا‬ ‫على‬ ‫التحديات‬
:
•
‫ملموسة‬ ‫غير‬ ‫الفوائد‬
•
‫كبيرة‬ ‫مباشرة‬ ‫غير‬ ‫تكاليف‬
•
‫كبيرة‬ ‫عائلية‬ ‫وفوائد‬ ‫تكاليف‬
/
‫متنوعة‬
•
‫أخرى‬ ‫برامج‬ ‫داخل‬ ‫األحيان‬ ‫من‬ ‫كثير‬ ‫في‬ ‫مخبأة‬

Economic Analyses Across the Pyramid, Continued
– CEA used to compare screening programs
– CBA used to compare dissimilar full coverage programs
– CUA used when overall health of a community is a concern
– All analyses are appropriate, with infrastructure as focus
•
•
‫تستخدم‬
CEA
‫الفحص‬ ‫برامج‬ ‫لمقارنة‬
•
‫تستخدم‬
CBA
‫المتباينة‬ ‫الكاملة‬ ‫التغطية‬ ‫برامج‬ ‫لمقارنة‬
•
‫تستخدم‬
CUA
‫قلق‬ ‫مصدر‬ ‫للمجتمع‬ ‫العامة‬ ‫الصحة‬ ‫تكون‬ ‫عندما‬
•
•
‫التحتية‬ ‫البنية‬ ‫على‬ ‫التركيز‬ ‫مع‬ ، ‫مناسبة‬ ‫التحليالت‬ ‫جميع‬

Implementation Evaluation:
Measuring Inputs and Outputs
Chapter 10
‫التنفيذ‬ ‫تقييم‬
:
‫والمخرجات‬ ‫المدخالت‬ ‫قياس‬
Translated by
Khaled
Good Luck

Learning Objectives
1. Evaluate the extent to which the service utilization plan
outputs were achieved.
•
:
•
‫الخدمة‬ ‫استخدام‬ ‫خطة‬ ‫مخرجات‬ ‫تحقيق‬ ‫مدى‬ ‫تقييم‬
.

• Key terminology
• Process theory elements in a
process evaluation
• Assessing implementation
• Purposes of process evaluation
• Roots of program failure
• Efficacy, effectiveness, and
efficiency
• Data collection methods
• Measuring organizational plan
inputs and outputs
• Measuring service utilization
plan inputs and outputs
• Intervention fidelity
• Under- and over-coverage
• Units of service
• Dosage monitoring
• Satisfaction
• Challenges in measuring
satisfaction
• Program logistics

Key Terminology
• Ceiling effect
• Coverage
• Coverage efficiency
• Decision drift
• Effectiveness
• Efficacy
• Efficiency
• Implementation assessment
• Implementation
documentation
• Implementation evaluation
• Intervention fidelity
• Over-coverage
• Process monitoring
• Program coverage
efficiency index
• Reach
• Satisfaction
• Service completion
• Under-coverage
• Unit of service

Process Theory Elements Included in a Process
Evaluation

Assessing Implementation
• Implementation documentation
– Tallying the activities and processes of the program implementation
• Implementation assessment
– Timely data collection to make corrections to the process theory during
implementation
– is called Program or process monitoring
• Implementation evaluation
– Retrospective determination of whether the program was implemented as designed
– Process evaluation entails systematic research to assess the extent to which the
program was delivered as intended.
•
‫التنفيذ‬ ‫وثائق‬
•
‫البرنامج‬ ‫تنفيذ‬ ‫وعمليات‬ ‫أنشطة‬ ‫حصر‬
•
•
‫أثناء‬ ‫العملية‬ ‫لنظرية‬ ‫تصحيحات‬ ‫إلجراء‬ ‫المناسب‬ ‫الوقت‬ ‫في‬ ‫البيانات‬ ‫جمع‬
‫التنفيذ‬
•
‫الرصد‬ ‫عملية‬ ‫أو‬ ‫برنامج‬
•
•
‫تصميمه‬ ‫تم‬ ‫كما‬ ‫تنفيذه‬ ‫تم‬ ‫قد‬ ‫البرنامج‬ ‫كان‬ ‫إذا‬ ‫ما‬ ‫رجعي‬ ‫بأثر‬ ‫تحديد‬
•
‫المنشود‬ ‫النحو‬ ‫على‬ ‫البرنامج‬ ‫تسليم‬ ‫مدى‬ ‫لتقييم‬ ‫منهجية‬ ‫أبحاث‬ ‫إجراء‬ ‫يستلزم‬ ‫العملية‬ ‫تقييم‬
‫مهم‬

Purposes of Process Evaluation
1. Gather Data about program delivery provides context for the
effect evaluation
– May help to identify whether program failure may be due to
implementation failure
• Operational information for program replication
(dissemination or replication of the health program)
‫مهم‬
•
‫يوفرجمع‬
‫البيانات‬
‫التأثير‬ ‫لتقييم‬ ‫ا‬ً‫ق‬‫سيا‬ ‫البرنامج‬ ‫تنفيذ‬ ‫حول‬
•
‫التنفيذ‬ ‫فشل‬ ‫بسبب‬ ‫يكون‬ ‫قد‬ ‫البرنامج‬ ‫فشل‬ ‫كان‬ ‫إذا‬ ‫ما‬ ‫تحديد‬ ‫في‬ ‫تساعد‬ ‫قد‬
•
‫البرنامج‬ ‫لتكرار‬ ‫التشغيل‬ ‫معلومات‬

Efficacy, Effectiveness, and Efficiency
• Efficacy
– Maximum potential effect under ideal conditions
– Because ideal conditions are difficult to create, the efficacy of an intervention is determined
through rigorous studies—usually clinical trials, especially randomized clinical trials.
• Effectiveness
– Realistic potential under real life conditions
– The degree of effectiveness of an intervention reflects what can be expected given the normally
messy situations that occur when health programs are delivered to real-world clients.
• Efficiency
– Amount of output achieved from the inputs
– Cost per unit of output
– higher outputs that are achieved with fewer inputs being deemed more efficient
‫مهم‬
•
‫فعالية‬
•
‫مثالية‬ ‫ظروف‬ ‫ظل‬ ‫في‬ ‫محتمل‬ ‫تأثير‬ ‫أقصى‬
•
‫دقيقة‬ ‫دراسات‬ ‫خالل‬ ‫من‬ ‫التدخل‬ ‫فعالية‬ ‫تحديد‬ ‫يتم‬ ، ‫المثالية‬ ‫الظروف‬ ‫تهيئة‬ ‫لصعوبة‬ ‫ا‬ً‫نظر‬
-
‫العشوائية‬ ‫السريرية‬ ‫التجارب‬ ‫وخاصة‬ ، ‫السريرية‬ ‫التجارب‬ ً‫ة‬‫عاد‬
.
•
‫فعالية‬
•
‫الحقيقية‬ ‫الحياة‬ ‫ظروف‬ ‫ظل‬ ‫في‬ ‫واقعية‬ ‫إمكانات‬
•
‫ا‬ ‫العالم‬ ‫في‬ ‫العمالء‬ ‫إلى‬ ‫الصحية‬ ‫البرامج‬ ‫تسليم‬ ‫عند‬ ً‫ة‬‫عاد‬ ‫تحدث‬ ‫التي‬ ‫الفوضوية‬ ‫للحاالت‬ ‫ا‬ً‫نظر‬ ‫توقعه‬ ‫يمكن‬ ‫ما‬ ‫التدخل‬ ‫فعالية‬ ‫درجة‬ ‫تعكس‬
‫لحق‬
‫يقي‬
.
•
‫كفاءة‬
•
‫المدخالت‬ ‫من‬ ‫المحققة‬ ‫المخرجات‬ ‫مقدار‬
•
‫االنتاج‬ ‫من‬ ‫وحدة‬ ‫لكل‬ ‫التكلفة‬
•
‫كفاءة‬ ‫أكثر‬ ‫تعتبر‬ ‫التي‬ ‫المدخالت‬ ‫من‬ ‫أقل‬ ‫عدد‬ ‫مع‬ ‫تحققت‬ ‫التي‬ ‫المخرجات‬ ‫ارتفاع‬

Data Collection Methods
seven categories of methods of data collection are appropriate
for process evaluations
Method When to use Examples of measures Pros Cons
Activity log List of discrete actions;
quantitative data
# sessions or
participants, inquiry call
log, press release data
Can be tailored to
program activities; easy
to use and analyze;
applicable across
pyramid levels
May become too long;
may not be completed
regularly
Checklist List of observable
actions or behaviors
Set up room for session,
gave supportive
comments, distributed
program materials
Simple; can include a
timeframe; easy data
entry; applicable across
pyramid levels
Difficult to narrow list
items; challenging to
write items for
consistent
interpretation; need to
establish reliability
Organizational
records
Have existing records
capturing the needed
data that can be legally
accessed; quantitative
data
Length of time on
waiting list, # purchased
computers, # hours
worked
Information accessibility;
applicable across
pyramid levels
Need a data
abstraction form; may
require complex data
linking and analysis
‫مهم‬

Data Collection Methods, Continued
Method When to use Examples of measures Pros Cons
Client records Have existing records
capturing the needed
data that can be legally
accessed; quantitative
data
Program attendance,
client compliance with
program elements
Accessibility to the
information
Need a data
abstraction form; may
require complex data
linking and analysis
Observation Data on interpersonal
interactions or event
sequences
Use of materials
produced, # of
advertising events
Data may reveal
unexpected results;
naturalistic; quantifiable
Time intensive; need
observation checklist;
complex data analysis
Questionnaire Need to quickly collect
data from reliable
respondents; have
reliable and valid Qx;
quantitative data
Degree of program
satisfaction, degree of
compliance with
interventions
Can collect pencil-and-
paper versions for many
programs; applicable
across pyramid levels
Respondents must
have good reading
skills; data are useless
if Qx is poorly written;
expensive for
population-level
programs
‫مهم‬

Data Collection Methods, Continued
Method When to use Examples of
measures
Pros Cons
Interview Have time and need
qualitative data;
respondents for
whom Qx is not
appropriate
Staff commitment to
program and
intervention
Able to get detailed
descriptions during
one-on-one interview;
possibly new insights
Time intensive; need
private place for
interviewing; more
complex data
analysis
Case study Need to understand
full set of
interactions around
program and its
context
Degree to which
managerial personnel
make changes to
assure intervention
fidelity
Gives thorough picture
of program; provides
new insights
Extremely complex
because it uses
multiple methods
over time; time
intensive; very
complex data
analysis
‫مهم‬

Organizational Plan Inputs and Outputs that Can Be
Measured

Example Measures of Organizational Plan Inputs and
Outputs
Inputs Input measures Output measures
Human resources # FTEs, new hires, or volunteers; %
personnel with licenses or certification;
staff education level; hours of training
and orientation
# hours worked; staff-to-recipient ratio;
hours per client contact per staff; staff
job satisfaction or commitment
Informational resources # computers bought; # program
recruitment efforts; communication
hardware and software availability; ease
of process and impact data entry and
retrieval
Degree to which computer and
telecommunication systems facilitate
intervention delivery; personnel
accessibility and availability; budget,
operating, meeting, and other reports
Monetary resources Grant and donation amounts; amount of
indirect costs deducted from program; #
proposals submitted for funding
Dollars or % variance from budget per
line item

Example Measures of Organizational Plan Inputs and
Outputs, Continued
Inputs Input measures Output measures
Physical resources # and type of capital, office, or clinical
equipment; office space square footage
Extent to which changes are made to
physical resources needed for
intervention delivery; replacement of
aged equipment
Transportation Parking fees; mileage per month; # bus
passes used; program vehicle expenses
Mileage per staff; # clients receiving
transportation; transportation cost per
staff or program recipient
Managerial resources Place in organizational chart; years of
experience; education level;
communication skills
Extent to which staff view managers as
controlling or delegating; degree to
which managers adjust to ensure
program fidelity
Time resources Timeline developed; presence of
deadline dates
# days delayed; % deadlines met; #
repeated requests

Service Utilization Plan Inputs and
Outputs that Can Be Measured

Example Measures of Service
Utilization Plan Inputs and Outputs
Element Input measures Output measures
Program reach # program requests % under- and over-coverage
Participants # recipients/participants, persons denied, or
not qualified for program
Efficiency index; degree of satisfaction with
program
Queuing and
program logistics
# on waiting list; presence of system to move
those on wait list into program or program
alternatives
Length of time on wait list; evenness of work
flow; # and types of materials produced
Social marketing Type and quality of social marketing; extent of
social marketing analysis
# advertising events and program requests
based on social marketing efforts
Intervention # meetings to standardize program; extent of
revisions based on previous intervention
delivery cycle or new research evidence
Fidelity to intervention plan; # sessions,
hours, and persons completing intervention,
requests for additional program delivery; use
of materials produced

Intervention Fidelity(validity)
• Intervention fidelity indicates whether the intervention was provided as designed
and planned.
• Degree to which program was offered as designed.
• 3 threats to fidelity:
1. No program provided: an intervention can fail because of the lack of a program (non
program, no treatment): meaning that the program was not provided.
2. Different program provided (activities and/or dosage): an intervention other than the
one designed and planned is provided.
3. Inconsistent program provided: intervention can fail is if the intervention is provided in
an inconsistent manner, resulting in a non standardized treatment over time or among
recipients
‫مهم‬
‫ال‬ ‫أم‬ ‫له‬ ‫مخطط‬ ‫هو‬ ‫كما‬ ‫تصميمه‬ ‫تم‬ ‫قد‬ ‫التدخل‬ ‫كان‬ ‫إذا‬ ‫ما‬ ‫إلى‬ ‫اإلخالص‬ ‫تدخل‬ ‫يشير‬
‫التصميم‬ ‫حسب‬ ‫البرنامج‬ ‫تقديم‬ ‫تم‬ ‫التي‬ ‫الدرجة‬
3
‫لإلخالص‬ ‫تهديدات‬
‫برنامج‬ ‫تقديم‬ ‫يتم‬ ‫لم‬
:
‫برنامج‬ ‫وجود‬ ‫عدم‬ ‫بسبب‬ ‫التدخل‬ ‫يفشل‬ ‫أن‬ ‫يمكن‬
(
‫عالج‬ ‫يوجد‬ ‫ال‬ ، ‫برنامج‬ ‫غير‬
:)
‫البرنامج‬ ‫توفير‬ ‫يتم‬ ‫لم‬ ‫أنه‬ ‫يعني‬ ‫مما‬
.
‫مقدم‬ ‫مختلف‬ ‫برنامج‬
(
‫و‬ ‫األنشطة‬
/
‫الجرعة‬ ‫أو‬
:)
‫وتخطيطه‬ ‫تصميمه‬ ‫تم‬ ‫الذي‬ ‫غير‬ ‫تدخل‬ ‫تقديم‬ ‫يتم‬
.
‫المقدم‬ ‫المتسق‬ ‫غير‬ ‫البرنامج‬
:
‫المتلقين‬ ‫بين‬ ‫أو‬ ‫الوقت‬ ‫مرور‬ ‫مع‬ ‫قياسي‬ ‫غير‬ ‫عالج‬ ‫إلى‬ ‫أدى‬ ‫مما‬ ، ‫متسقة‬ ‫غير‬ ‫بطريقة‬ ‫التدخل‬ ‫تقديم‬ ‫تم‬ ‫إذا‬ ‫التدخل‬ ‫يفشل‬ ‫أن‬ ‫يمكن‬

Matrix of Under- and Over-Coverage
Persons not served
by program
Persons served by
program
Persons not in need
of program
Ideal coverage Over-coverage
Persons in need of
program
Under-coverage Ideal coverage

Formulas for Measures of Coverage
• % over-coverage =
# not in need but served
# served
• % under-coverage =
# in need but not served
# in need
• Coverage efficiency =
(# served - # over-covered)
# in need
• Efficiency index =
(# in need & served / # in need) x 100
Standard stated in objectives
• Program efficiency index =
Sum of efficiency indices
# components in the program

Units of Service
• More commonly used in programs at the direct or enabling services
level
• Predetermined unit
• Stated in process objective
• Examples: # contact hours, # clients seen, # education sessions,
caseload size
•
‫أو‬ ‫المباشرة‬ ‫الخدمات‬ ‫مستوى‬ ‫على‬ ‫البرامج‬ ‫في‬ ‫ا‬ً‫م‬‫استخدا‬ ‫أكثر‬
‫التمكينية‬
•
‫سلفا‬ ‫محددة‬ ‫وحدة‬
•
‫العملية‬ ‫الهدف‬ ‫في‬ ‫ذكرت‬
•
‫أمثلة‬
:
#
‫القضايا‬ ‫حجم‬ ، ‫التعليم‬ ‫جلسات‬ ‫عدد‬ ، ‫رؤيتهم‬ ‫تم‬ ‫الذين‬ ‫العمالء‬ ‫عدد‬ ، ‫االتصال‬ ‫ساعات‬

Dosage Monitoring
• 2 of 5 dosage elements most relevant to monitoring
– Frequency, duration, strength, route of administration, administration credibility.
– Frequency and Duration have the greatest relevance for ongoing monitoring. Frequency of the
intervention—whether hourly, daily, weekly, or monthly—and duration of the intervention—whether
one session, eight weeks of classes, or 6 months of exposure—are the elements of dosage that may
vary from the objectives laid out in the plan because of either program or participant factors.
• Service completion: which program participants completedthe health program.
• Level of participation: determining if participants received the appropriate “dose” of the
intervention
‫مهم‬
•
2
‫من‬
5
‫للرصد‬ ‫أهمية‬ ‫األكثر‬ ‫الجرعة‬ ‫عناصر‬
•
‫اإلدارة‬ ‫مصداقية‬ ، ‫اإلدارة‬ ‫مسار‬ ، ‫القوة‬ ، ‫المدة‬ ، ‫التكرار‬
.
•
‫المستمرة‬ ‫للمراقبة‬ ‫أهمية‬ ‫أكبر‬ ‫لها‬ ‫والمدة‬ ‫التردد‬
.
‫التدخل‬ ‫تواتر‬
-
‫ًا‬‫ي‬‫شهر‬ ‫أو‬ ‫ًا‬‫ي‬‫أسبوع‬ ‫أو‬ ‫ًا‬‫ي‬‫يوم‬ ‫أو‬ ‫الساعة‬ ‫في‬ ‫سواء‬
-
‫التدخل‬ ‫ومدة‬
-
‫م‬ ‫أسابيع‬ ‫ثمانية‬ ‫أو‬ ‫واحدة‬ ‫جلسة‬ ‫كانت‬ ‫سواء‬
‫أو‬ ‫الدراسية‬ ‫الفصول‬ ‫ن‬
6
‫التعرض‬ ‫من‬ ‫أشهر‬
-
‫المشارك‬ ‫أو‬ ‫البرنامج‬ ‫عوامل‬ ‫بسبب‬ ‫الخطة‬ ‫في‬ ‫المحددة‬ ‫األهداف‬ ‫عن‬ ‫تختلف‬ ‫قد‬ ‫التي‬ ‫الجرعة‬ ‫عناصر‬ ‫هي‬
.
•
‫الخدمة‬ ‫إتمام‬
:
‫الصحي‬ ‫البرنامج‬ ‫أكملوا‬ ‫البرنامج‬ ‫في‬ ‫المشاركين‬ ‫أي‬
.
•
‫المشاركة‬ ‫مستوى‬
:
‫تلقوا‬ ‫قد‬ ‫المشاركون‬ ‫كان‬ ‫إذا‬ ‫ما‬ ‫تحديد‬
"
‫جرعة‬
"
‫المناسبة‬ ‫التدخل‬

Satisfaction
• Part of process evaluation, not an outcome
• satisfaction as a process output stems from the definition of satisfaction:
the degree to which participants receive what they expect to receive and
the extent to which their expectations are met with regard to how they
are treated
• Related to elements of the service utilization plan
– What is received
– What is expected
– How one is treated
•
‫وليس‬ ، ‫التقييم‬ ‫عملية‬ ‫من‬ ‫جزء‬
‫نتيجة‬
•
‫الرضا‬ ‫تعريف‬ ‫من‬ ‫عملية‬ ‫كناتج‬ ‫الرضا‬ ‫ينبع‬
:
‫الح‬ ‫يتوقعون‬ ‫ما‬ ‫بها‬ ‫المشاركون‬ ‫يتلقى‬ ‫التي‬ ‫الدرجة‬
‫عليه‬ ‫صول‬
‫معالجتهم‬ ‫بكيفية‬ ‫يتعلق‬ ‫فيما‬ ‫توقعاتهم‬ ‫تلبية‬ ‫ومدى‬
•
‫الخدمة‬ ‫استخدام‬ ‫خطة‬ ‫بعناصر‬ ‫المتعلقة‬
•
‫تلقى‬ ‫ما‬
•
‫متوقع‬ ‫هو‬ ‫ما‬
•
‫واحد‬ ‫عالج‬ ‫يتم‬ ‫كيف‬

Challenges in Measuring Satisfaction
• Ceiling effect occurs when the measurement tool is constructed so that
respondents do not have an opportunity to distinguish among levels at the high
end of the scale.
– High mean value and small standard deviation
• Measure must be culturally sensitive
• Discrepancies between paper-and-pencil questionnaire responses and interview
responses
• Measure must capture participants’ expectations and experiences
• Scope can be difficult to define
– Relate items to the organizational and service utilization plans
•
‫تأثير‬
‫األعلى‬ ‫الحد‬
(
‫السقف‬
)
‫فر‬ ‫للمستجيبين‬ ‫تتاح‬ ‫ال‬ ‫بحيث‬ ‫القياس‬ ‫أداة‬ ‫إنشاء‬ ‫يتم‬ ‫عندما‬ ‫يحدث‬
‫بين‬ ‫للتمييز‬ ‫صة‬
‫المقياس‬ ‫نهاية‬ ‫في‬ ‫المستويات‬
.
•
‫الصغير‬ ‫المعياري‬ ‫واالنحراف‬ ‫القيمة‬ ‫متوسط‬ ‫ارتفاع‬
•
‫ًا‬‫ي‬‫ثقاف‬ ‫ا‬ً‫س‬‫حسا‬ ‫اإلجراء‬ ‫يكون‬ ‫أن‬ ‫يجب‬
•
‫على‬ ‫والردود‬ ‫والقلم‬ ‫الورق‬ ‫استبيان‬ ‫ردود‬ ‫بين‬ ‫التناقضات‬
‫المقابلة‬
•
‫المشاركين‬ ‫وتجارب‬ ‫توقعات‬ ‫التدبير‬ ‫يستوعب‬ ‫أن‬ ‫يجب‬
•
‫النطاق‬ ‫تحديد‬ ‫الصعب‬ ‫من‬ ‫يكون‬ ‫أن‬ ‫يمكن‬
•
‫والخدمي‬ ‫التنظيمي‬ ‫االستخدام‬ ‫بخطط‬ ‫العناصر‬ ‫ربط‬

Program Logistics
• Work flow: Measures of work flow are one indicator of the amount of work done by the program staff and
the queuing of participants.
• Queuing: the volume and queuingwill greatly affect.
• the work flow of program staff
– Amount of work done by staff
– Involve staff in developing these measures
• Materials produced
– Both the quantity and the quality of the materials produced for the health program need to be
considered in the process evaluation.
– Data on materials can be challenging to generate
– Program-specific
• Data about production of materials can be difficult to obtain and will be very program specific.
‫مهم‬
•
‫العمل‬ ‫تدفق‬
:
‫المشاركين‬ ‫انتظار‬ ‫وقائمة‬ ‫البرنامج‬ ‫موظفو‬ ‫به‬ ‫قام‬ ‫الذي‬ ‫العمل‬ ‫مقدار‬ ‫مؤشرات‬ ‫أحد‬ ‫العمل‬ ‫تدفق‬ ‫مقاييس‬ ‫تعد‬
.
•
‫الطوابير‬
:
‫كبير‬ ‫بشكل‬ ‫تؤثر‬ ‫سوف‬ ‫طابور‬ ‫و‬ ‫حجم‬
.
•
‫البرنامج‬ ‫موظفي‬ ‫عمل‬ ‫تدفق‬
•
‫الموظفون‬ ‫به‬ ‫قام‬ ‫الذي‬ ‫العمل‬ ‫مقدار‬
•
‫التدابير‬ ‫هذه‬ ‫تطوير‬ ‫في‬ ‫الموظفين‬ ‫إشراك‬
•
‫المنتجة‬ ‫المواد‬
•
‫العملية‬ ‫تقييم‬ ‫في‬ ‫الصحي‬ ‫للبرنامج‬ ‫المنتجة‬ ‫المواد‬ ‫ونوعية‬ ‫كمية‬ ‫مراعاة‬ ‫يجب‬
.
•
‫التوليد‬ ‫صعبة‬ ‫بالمواد‬ ‫المتعلقة‬ ‫البيانات‬ ‫تكون‬ ‫أن‬ ‫يمكن‬
•
‫خاص‬ ‫برنامج‬
•
‫للبرنامج‬ ‫للغاية‬ ‫محددة‬ ‫وستكون‬ ‫المواد‬ ‫إنتاج‬ ‫حول‬ ‫بيانات‬ ‫على‬ ‫الحصول‬ ‫يصعب‬ ‫قد‬
.

Process Monitoring and Evaluation Across the Pyramid
– Units of service
– Similar to direct services level, but modified to be program-specific and using different
data sources
– Units of service at community level (e.g., # agencies involved or households reached)
– Units of service may be # employees involved, # procedures updated, etc.
•
•
‫الخدمة‬ ‫وحدات‬
•
•
‫مختلفة‬ ‫بيانات‬ ‫مصادر‬ ‫ويستخدم‬ ‫للبرنامج‬ ‫ا‬ً‫ص‬‫مخص‬ ‫ليكون‬ ‫تعديله‬ ‫يتم‬ ‫لكن‬ ، ‫المباشرة‬ ‫الخدمات‬ ‫مستوى‬ ‫يشبه‬
•
•
‫المجتمع‬ ‫مستوى‬ ‫على‬ ‫الخدمة‬ ‫وحدات‬
(
‫إليها‬ ‫الوصول‬ ‫تم‬ ‫التي‬ ‫األسر‬ ‫أو‬ ‫المشاركة‬ ‫الوكاالت‬ ‫عدد‬ ، ‫المثال‬ ‫سبيل‬ ‫على‬
)
•
•
‫الخدمة‬ ‫وحدات‬ ‫تكون‬ ‫قد‬
#
‫تحديث‬ ‫تم‬ ، ‫مشاركين‬ ‫موظفين‬
#
‫إلخ‬ ، ‫إجراءات‬
.

Example Process Evaluation Measures Across the
Pyramid
Direct services Enabling services Population services Infrastructure
Organizational plan
input
Provider credentials,
location
physical resources
(e.g., cars)
managerial
resources
Personnel
qualifications,
managerial
resources, fiscal
resources
Organizational plan
output
Protocols and
procedures for
service delivery, data
about individual
participants
Protocols and
procedures for
service delivery, data
about participants
Protocols and
procedures for
service delivery
Budget variance,
fiscal accountability,
data and
management
information systems
Service utilization
plan input
Wait times,
participant
characteristics
Wait times,
participant
characteristics
Population
characteristics
Workforce
characteristics
Service utilization
plan output
Measures of
coverage
Measures of
coverage
Measures of
coverage
Materials produced,
number of
participants

Program Quality and Fidelity:
Managerial and Contextual
Considerations
Chapter 11
‫واإلخالص‬ ‫البرنامج‬ ‫جودة‬
:
‫والسياقية‬ ‫اإلدارية‬ ‫االعتبارات‬
Translated by
Khaled
Good Luck

Learning Objectives
1. Explain the different types of accountability to which
program managers are held
2. Select and apply the appropriate quality improvement visual
display of processes
3. Take managerial actions to assure implementation fidelity.
•
:
•
‫البرامج‬ ‫مديرو‬ ‫يتحملها‬ ‫التي‬ ‫للمساءلة‬ ‫المختلفة‬ ‫األنواع‬ ‫اشرح‬
•
‫للعمليات‬ ‫الجودة‬ ‫لتحسين‬ ‫المناسب‬ ‫المرئي‬ ‫العرض‬ ‫وتطبيق‬ ‫تحديد‬
•
‫التنفيذ‬ ‫لضمان‬ ‫اإلدارية‬ ‫اإلجراءات‬ ‫اتخاذ‬
‫التنفيذ‬
.

• Key terminology
• Types of program
accountability
• Approaches to quality and
performance
• QI tools
• Effects of QI on evaluation
• Improvement
methodologies vs.
process evaluation
• Performance
measurement
• Informatics and
information technology
• Interpreting
implementation data
• Using implementation
data for program quality
and fidelity
• Managing group
processes for program
quality and fidelity
• When not to change
• Formative evaluations

Key Terminology
• Accountability
• Control chart
• Coverage accountability
• Efficiency accountability
• Fiscal accountability
• Fishbone diagram
• Flowchart
• Formative evaluation
• Histogram
• Impact accountability
• Legal accountability
• Pareto chart
• Performance measures
• PERT chart
• Professional
accountability
• Quality assurance
• Responsibility
• Scatter diagram
• Service delivery
accountability

Accountability
• Accountability and Responsibility are Cornerstones for Program
Implementation.
• Accountability means being held answerable for actions taken and the
subsequent success or failure of the program
• Responsibility means having the charge to ensure that things are done,
and done within the specified parameters.
• Program managers are generally both accountable for the program and
responsible for seeing that the program is carried out.
•
‫البرنامج‬ ‫تنفيذ‬ ‫في‬ ‫الزاوية‬ ‫حجر‬ ‫هما‬ ‫والمسؤولية‬ ‫المساءلة‬
.
•
‫للبرنامج‬ ‫الالحق‬ ‫الفشل‬ ‫أو‬ ‫والنجاح‬ ‫المتخذة‬ ‫اإلجراءات‬ ‫عن‬ ً‫ال‬‫مسؤو‬ ‫تكون‬ ‫أن‬ ‫تعني‬ ‫المساءلة‬
•
‫المحددة‬ ‫المعايير‬ ‫ضمن‬ ‫وتنفيذها‬ ، ‫األشياء‬ ‫إنجاز‬ ‫من‬ ‫للتأكد‬ ‫الرسوم‬ ‫تحمل‬ ‫تعني‬ ‫المسؤولية‬
.
•
‫البرنامج‬ ‫تنفيذ‬ ‫رؤية‬ ‫عن‬ ‫ومسؤولون‬ ‫البرنامج‬ ‫عن‬ ‫عام‬ ‫بشكل‬ ‫مسؤولين‬ ‫البرامج‬ ‫مديرو‬ ‫يكون‬
.
‫مهم‬

1. Fiscal accountability refers to the need for sound accounting, careful documentation of
expenses, and tracking of revenues.
2. Legal accountability encompasses staff acting in accordance with local, state, and
federal laws and within their professional licensure limits.
3. Efficiency accountability means that the program is delivered with efficient use of the
resources.
4. Coverage accountability relates to the program reaching the intended recipients.
5. Service delivery accountability comprises the extent to which the intervention is
provided as planned.
6. Impact accountability is concerned with the program having an outcome and impact on
the target audience and recipients. The indicators for impact accountability are highly
tailored to reflect the effect theory of the program.
Types of program accountability
.
‫اإليرادات‬ ‫وتتبع‬ ‫للمصروفات‬ ‫دقيق‬ ‫وتوثيق‬ ‫سليمة‬ ‫محاسبة‬ ‫إلى‬ ‫الحاجة‬ ‫إلى‬ ‫المالية‬ ‫المساءلة‬ ‫تشير‬ 1
.
.
‫بهم‬ ‫الخاصة‬ ‫المهني‬ ‫الترخيص‬ ‫حدود‬ ‫وضمن‬ ‫واالتحادية‬ ‫والوالئية‬ ‫المحلية‬ ‫للقوانين‬ ‫ا‬ً‫ق‬‫وف‬ ‫يعملون‬ ‫الذين‬ ‫الموظفين‬ ‫القانونية‬ ‫المساءلة‬ ‫تشمل‬ 2
.
.
‫للموارد‬ ‫الفعال‬ ‫االستخدام‬ ‫مع‬ ‫تقديمه‬ ‫يتم‬ ‫البرنامج‬ ‫أن‬ ‫الكفاءة‬ ‫مساءلة‬ ‫تعني‬ 3
.
.
‫المقصودين‬ ‫المستلمين‬ ‫إلى‬ ‫يصل‬ ‫الذي‬ ‫بالبرنامج‬ ‫التغطية‬ ‫تغطية‬ ‫تتعلق‬ 4
.
.
‫له‬ ‫مخطط‬ ‫هو‬ ‫كما‬ ‫التدخل‬ ‫تقديم‬ ‫فيه‬ ‫يتم‬ ‫الذي‬ ‫المدى‬ ‫الخدمات‬ ‫تقديم‬ ‫عن‬ ‫المساءلة‬ ‫تشمل‬ 5
.
.
‫المستهدفين‬ ‫والمستفيدين‬ ‫الجمهور‬ ‫على‬ ‫وتأثير‬ ‫نتائج‬ ‫له‬ ‫الذي‬ ‫بالبرنامج‬ ‫المساءلة‬ ‫تأثير‬ ‫يتعلق‬ 6
.
‫نظر‬ ‫لتعكس‬ ‫كبير‬ ‫بشكل‬ ‫التأثير‬ ‫على‬ ‫المساءلة‬ ‫مؤشرات‬ ‫تصميم‬ ‫تم‬
‫البرنامج‬ ‫تأثير‬ ‫ية‬
‫مهم‬

Types of Program Accountability
Accountability type Definition: The extent to which… Examples of indicators
Organizational plan related
Efficiency Resources are utilized without waste
or redundancy
Dollars spent on the program, cost per client
served, cost per unit of outcome
Fiscal Resources are managed according
to the budget
Existence of receipts and bills paid, # errors
found during annual audit, % budget variance
Legal Legal, regulatory, and ethical
standards are met
# malpractice suits, investigations, or
personnel with licensure
‫مهم‬

Types of Program Accountability, Continued
Accountability type Definition: The extent to
which…
Examples of indicators
Service utilization plan related
Coverage Target population is reached Coverage efficiency, efficiency index, %
coverage efficiency target achieved
Service delivery Intervention is provided as
planned
# units of service provided, breaches of
protocol, or intervention modifications
Effect theory related
Impact Participants change or are
changed because of the
intervention
Very program-specific health and
behavior indicators
‫مهم‬

Professional accountability
• Professional Accountability : refers to an individual from a health
profession being bound or binding by the corresponding
professional norms and codes(laws), including the moral and
ethical codes related to serving the public interest
• Personal Professional Accountability: encompasses situationsin
which an individual is not performing according to professional
standards and norms
‫مهم‬
•
‫المهنية‬ ‫المساءلة‬
:
‫المقابلة‬ ‫المهنية‬ ‫والقواعد‬ ‫بالقواعد‬ ‫ملزم‬ ‫أو‬ ‫ملتزم‬ ‫الصحة‬ ‫مهنة‬ ‫من‬ ‫الفرد‬ ‫أن‬ ‫إلى‬ ‫تشير‬
)
‫القوانين‬
(
‫المصل‬ ‫بخدمة‬ ‫المتعلقة‬ ‫واألخالقية‬ ‫األخالقية‬ ‫القواعد‬ ‫ذلك‬ ‫في‬ ‫بما‬ ،
‫العامة‬ ‫حة‬
•
‫الشخصية‬ ‫المهنية‬ ‫المساءلة‬
:
‫المهنية‬ ‫والقواعد‬ ‫للمعايير‬ ‫ا‬ً‫ق‬‫وف‬ ‫الفرد‬ ‫يؤديها‬ ‫ال‬ ‫التي‬ ‫المواقف‬ ‫تشمل‬

Approaches to Quality and Performance
• Quality Assurance: Entails using the minimum acceptable requirements for processes and
standards for outpMuts as the criteria for taking corrective action.
• Continuous Quality Improvement (CQI) and Total Quality Management (TQM): were adopted by
healthcare organizations as tools to reduce costs while improving the quality of services by:
- Address problems by focusing on whole system
– Employees are best source of possible solutions
– Data-driven
– Examples: Six Sigma, Balanced Score Card
• Six sigma: is a process to reduce variation in clinical and business processes.
• Balanced Score Card:BSC was developed by Kaplan and Norton (1992, 1996) as a tool for
businesses that integrates financial performance measures with measures of customer
satisfaction, internal processes, and organizational learning.
•
‫الجودة‬ ‫ضمان‬
:
‫التصحيحي‬ ‫اإلجراءات‬ ‫التخاذ‬ ‫كمعايير‬ ‫الخارجية‬ ‫باآلثار‬ ‫الخاصة‬ ‫والمعايير‬ ‫للعمليات‬ ‫المقبولة‬ ‫المتطلبات‬ ‫من‬ ‫األدنى‬ ‫الحد‬ ‫استخدام‬ ‫يستلزم‬
‫ة‬
.
•
‫للجودة‬ ‫المستمر‬ ‫التحسين‬
(
CQI
)
‫الشاملة‬ ‫الجودة‬ ‫وإدارة‬
(
TQM
:)
‫طريق‬ ‫عن‬ ‫الخدمات‬ ‫جودة‬ ‫تحسين‬ ‫مع‬ ‫التكاليف‬ ‫لخفض‬ ‫كأدوات‬ ‫الصحية‬ ‫الرعاية‬ ‫مؤسسات‬ ‫قبل‬ ‫من‬ ‫اعتمادهما‬ ‫تم‬
:
•
-
‫بأكمله‬ ‫النظام‬ ‫على‬ ‫التركيز‬ ‫خالل‬ ‫من‬ ‫المشاكل‬ ‫معالجة‬
•
‫الممكنة‬ ‫للحلول‬ ‫مصدر‬ ‫أفضل‬ ‫هم‬ ‫الموظفون‬
•
‫البيانات‬ ‫على‬ ‫تعتمد‬
•
‫أمثلة‬
:
‫ستة‬
‫سيغما‬
‫نقاط‬ ‫بطاقة‬ ،
‫متوازنة‬
•
‫ستة‬
‫سيغما‬
:
‫التجارية‬ ‫والعمليات‬ ‫السريرية‬ ‫العمليات‬ ‫في‬ ‫االختالف‬ ‫من‬ ‫للحد‬ ‫عملية‬ ‫هي‬
.
•
‫المتوازنة‬ ‫النتيجة‬ ‫بطاقة‬
:
‫تطوير‬ ‫تم‬
BSC
‫بواسطة‬
Kaplan and Norton (1992
،
1996)
‫التنظيمي‬ ‫والتعلم‬ ‫الداخلية‬ ‫والعمليات‬ ‫العمالء‬ ‫رضا‬ ‫مقاييس‬ ‫مع‬ ‫المالي‬ ‫األداء‬ ‫مقاييس‬ ‫تدمج‬ ‫التي‬ ‫للشركات‬ ‫كأداة‬
.
‫مهم‬

Quality Improvement Tools
•The tools used by improvement committees rely on statistical analyses
and graphic displays of the statistical information .
Seven basic tools are used to statistically control the processes) Figure
11
-
1
.( These tools are Easy to use and Require Minimal Statistical
Knowledge, which probably accounts for their wide application.
‫مهم‬

Program Evaluation Review Technique (PERT) charts diagram the sequence of events against a specific
timeline, thereby showing when tasks need to be accomplished.
1. Fishbone diagrams, or cause diagrams, are representations of sequential events and major factors
at play at each stage.
2. Check sheet.
3. Control charts show an average, with upper and lower confidence limits and standard deviations.
They indicate whether a variable is within the acceptable parameters, and they result in a heavy
focus on setting and staying within control limits and parameters for a select set of outcome
indicators.
4. Histograms are simple bar graphs showing the frequency of a value for one variable.
5. Pareto charts are a bit more complicated—they use a bar graph to identify the major source of a
problem.
6. Scatter diagrams show the relationship between two variables by using the data from each
individual. They make it easy to see the direction of the relationship.
7. Flowcharts diagram the sequence of activities from start to outcome.
Seven basic tools of the statistical information
‫مهم‬
•
‫البرامج‬ ‫تقييم‬ ‫مراجعة‬ ‫مخططات‬ ‫ُظهر‬‫ت‬
(
PERT
)
‫المهام‬ ‫إنجاز‬ ‫يجب‬ ‫متى‬ ‫يوضح‬ ‫مما‬ ، ‫محدد‬ ‫زمني‬ ‫لجدول‬ ‫ا‬ً‫ق‬‫وف‬ ‫األحداث‬ ‫تسلسل‬
.
•
‫مخططات‬ ‫تمثل‬
Fishbone
‫مرحلة‬ ‫كل‬ ‫في‬ ‫تلعبها‬ ‫التي‬ ‫الرئيسية‬ ‫والعوامل‬ ‫المتتابعة‬ ‫لألحداث‬ ‫تمثيالت‬ ، ‫المخططات‬ ‫تسبب‬ ‫أو‬ ،
.
•
‫ورقة‬ ‫من‬ ‫تحقق‬
.
•
‫قياسية‬ ‫وانحرافات‬ ‫وأسفل‬ ‫أعلى‬ ‫ثقة‬ ‫حدود‬ ‫مع‬ ، ‫ا‬ً‫ط‬‫متوس‬ ‫التحكم‬ ‫مخططات‬ ‫ُظهر‬‫ت‬
.
‫لمجم‬ ‫التحكم‬ ‫ومعلمات‬ ‫حدود‬ ‫ضمن‬ ‫والبقاء‬ ‫اإلعداد‬ ‫على‬ ‫شديد‬ ‫تركيز‬ ‫عنه‬ ‫وينتج‬ ، ‫المقبولة‬ ‫المعلمات‬ ‫ضمن‬ ‫المتغير‬ ‫كان‬ ‫إذا‬ ‫ما‬ ‫إلى‬ ‫تشير‬ ‫فهي‬
‫وعة‬
‫مؤشرا‬ ‫من‬ ‫مختارة‬
‫النتائج‬ ‫ت‬
.
•
‫واحد‬ ‫لمتغير‬ ‫قيمة‬ ‫وتيرة‬ ‫تظهر‬ ‫التي‬ ‫البسيطة‬ ‫الشريطية‬ ‫البيانية‬ ‫الرسوم‬ ‫هي‬ ‫البيانية‬ ‫الرسوم‬
.
•
‫مخططات‬ ‫تعد‬
Pareto
‫ًا‬‫د‬‫تعقي‬ ‫أكثر‬
-
‫للمشكلة‬ ‫الرئيسي‬ ‫المصدر‬ ‫لتحديد‬ ‫شريطي‬ ‫بياني‬ ‫رسم‬ ‫تستخدم‬ ‫فهي‬
.
•
‫فرد‬ ‫كل‬ ‫من‬ ‫البيانات‬ ‫باستخدام‬ ‫متغيرين‬ ‫بين‬ ‫العالقة‬ ‫المبعثرة‬ ‫المخططات‬ ‫ُظهر‬‫ت‬
.
‫العالقة‬ ‫اتجاه‬ ‫رؤية‬ ‫السهل‬ ‫من‬ ‫تجعل‬ ‫أنها‬
.
•
‫النتيجة‬ ‫إلى‬ ‫البداية‬ ‫من‬ ‫األنشطة‬ ‫تسلسل‬ ‫تخطيطي‬ ‫بياني‬ ‫رسم‬
.

Effects of Continuous quality improvement (CQI) and total
quality management ( TQM|) on Evaluations
• Relevance to Health Programs: Evaluations can be affected by the presence of CQI/TQM in several
ways.
• Easier to involve employees in the evaluation
– Used to using data for improvement
– Better sense of developing program theory
• Methods can help with designing process evaluations
• Facilitates identifying problems with service utilization plan implementation or organizational plan
deficiencies.
• Program Evaluation Review Technique (PERT) charts:
• Fishbone diagrams, and control charts. These skills help personnel articulate and then construct
diagrams of underlying processes, especially when program planners are developing the process
theory. They also make the CQI/TQM way of thinking and methods useful in designing and
conducting process monitoring evaluations.
‫مهم‬
•
‫الصحية‬ ‫بالبرامج‬ ‫الصلة‬
:
‫بوجود‬ ‫التقييمات‬ ‫تتأثر‬ ‫أن‬ ‫يمكن‬
CQI / TQM
‫طرق‬ ‫بعدة‬
.
•
‫التقييم‬ ‫في‬ ‫الموظفين‬ ‫إلشراك‬ ‫أسهل‬
•
‫للتحسين‬ ‫البيانات‬ ‫الستخدام‬ ‫تستخدم‬
•
‫البرنامج‬ ‫نظرية‬ ‫لتطوير‬ ‫أفضل‬ ‫شعور‬
•
‫العمليات‬ ‫تقييمات‬ ‫تصميم‬ ‫في‬ ‫األساليب‬ ‫تساعد‬ ‫أن‬ ‫يمكن‬
•
‫التنظيمية‬ ‫الخطة‬ ‫عيوب‬ ‫أو‬ ‫الخدمة‬ ‫استخدام‬ ‫خطة‬ ‫تنفيذ‬ ‫مع‬ ‫المشاكل‬ ‫تحديد‬ ‫يسهل‬
.
•
‫البرامج‬ ‫تقييم‬ ‫مراجعة‬ ‫مخططات‬
(
PERT
:)
•
‫السيطرة‬ ‫البيانية‬ ‫والرسوم‬ ، ‫السمكة‬ ‫هيكل‬ ‫المخططات‬
.
‫العملية‬ ‫نظرية‬ ‫بتطوير‬ ‫البرامج‬ ‫مخططو‬ ‫يقوم‬ ‫عندما‬ ً‫خاصة‬ ، ‫األساسية‬ ‫للعمليات‬ ‫مخططات‬ ‫إنشاء‬ ‫ثم‬ ‫توضيح‬ ‫على‬ ‫الموظفين‬ ‫المهارات‬ ‫هذه‬ ‫تساعد‬
.
‫كم‬
‫تجعل‬ ‫أنها‬ ‫ا‬
‫التفكير‬ ‫طريقة‬
CQI / TQM
‫العمليات‬ ‫مراقبة‬ ‫تقييمات‬ ‫وإجراء‬ ‫تصميم‬ ‫في‬ ‫مفيدة‬ ‫وأساليب‬ ‫مفيدة‬
.

Improvement Methodologies vs. Program Process
Evaluation
Improvement methodologies Program process evaluation
Philosophy Organizations can be more effective if use staff
expertise to improve services and products
Programs need to be justified in terms of their
effect on participants
Purpose Systems analysis and improvement focus on
indentified problem areas from the point of
view of customer needs
Evaluators determine whether a program was
provided as planned and if it made a difference
to the participants (customers)
Approach Team-based approach to identifying and
analyzing the problem
Evaluator-driven approach to data collection
and analysis
Who does it Employees from any or all depts., mid-level
managers, top-level executives
Evaluators and program managers, with or
without employee or stakeholder participation
Methods Engineering approaches to systems analysis Scientific research methods

Performance measures
Performance measures are indicators of process, output, or outcomes that have been developed to
be used as standardized indicators by health programs, initiatives, practitioners, or organizations.
• performance measures vary across accrediting bodies and the various performance
measurement systems
Criteria for Good Performance Measures
1. Evidence-based
2. Interpretable by practitioners
3. Actionable by improvement committees
4. Rigorous (must have validity and reliability)
5. Specify who is included in numerators and denominators
6. Measuring performance must be feasible
‫مهم‬
•
‫كمؤشرات‬ ‫الستخدامها‬ ‫تطويرها‬ ‫تم‬ ‫التي‬ ‫النتائج‬ ‫أو‬ ، ‫المخرجات‬ ‫أو‬ ، ‫للعملية‬ ‫مؤشرات‬ ‫هي‬ ‫األداء‬ ‫مقاييس‬
‫المنظمات‬ ‫أو‬ ‫الممارسين‬ ‫أو‬ ‫المبادرات‬ ‫أو‬ ‫الصحية‬ ‫البرامج‬ ‫قبل‬ ‫من‬ ‫موحدة‬
.
•
‫المختلفة‬ ‫األداء‬ ‫قياس‬ ‫وأنظمة‬ ‫االعتماد‬ ‫هيئات‬ ‫بين‬ ‫األداء‬ ‫مقاييس‬ ‫تختلف‬
•
‫الجيد‬ ‫األداء‬ ‫مقاييس‬ ‫معايير‬
•
‫على‬ ‫القائمة‬ ‫األدلة‬
•
‫الممارسين‬ ‫قبل‬ ‫من‬ ‫للتفسير‬ ‫قابلة‬
•
‫التحسين‬ ‫لجان‬ ‫قبل‬ ‫من‬ ‫للتنفيذ‬ ‫قابلة‬
•
‫صارمة‬
(
‫وموثوقية‬ ‫صالحية‬ ‫لها‬ ‫يكون‬ ‫أن‬ ‫يجب‬
)
•
‫والمقام‬ ‫البسط‬ ‫في‬ ‫تضمينه‬ ‫يتم‬ ‫من‬ ‫حدد‬
•
‫ا‬ً‫ن‬‫ممك‬ ‫األداء‬ ‫قياس‬ ‫يكون‬ ‫أن‬ ‫يجب‬

Terms Used in Performance Measurement
Term Definition
Measure type Broad health status that the performance measure is intended to describe
Measure Statement is measurable in terms of the desired health status or behavior, as it relates to the measurement
type
Numerator Definition used to assign individuals into the numerator to quantify the measure
Denominator Definition used to assign individuals into the denominator to quantify the measure
Rationale for
measure
Brief explanation of relationship of the measure to the measurement type; includes reference to the
appropriate Healthy People 2010 objective
Limitations of
measure
Statement of which key factors might contribute to the potential failure of program, which would be captured
in the measure
Use of measure How the measure might be used in program development or policy making
Data resources List of relevant sources of data for estimating the measure
Limitations of data Brief statement of those factors that may contribute to inaccurate, not valid, or not reliable data from the
data resource listed
‫مهم‬

Selected Performance Measurement Systems Used by Healthcare
Organizations
Name Sponsoring organization
Capacity Assessment for Title V (CAST-5) Association of Maternal and Child Health
Programs and Johns Hopkins University
Consumer Assessment of Health Plans (CAHPS®) Agency for Healthcare Research and Quality
Healthcare Effectiveness Data and Information Set
(HEDIS)
National Commission on Quality Assurance
Joint Commission Joint Commission on the Accreditation of Health
Care Organizations
National Public Health Performance Standards Collaborative of CDC and a variety of public health
organizations

Informatics and Information Technology
• Types
– Electronic medical records
– Data warehousing
– Web-based patient education
– Wireless applications
– Telemedicine
– Biometrics
– Handheld technology
• Advantages
– Makes current data available
– Data needed for monitoring or effect
evaluation may be collected already
to meet Healthcare Effectiveness
Data and Information Set (HEDIS) or
Joint Commission on Accreditation
of Healthcare Organizations (JCAHO)
requirements
•
‫األنواع‬
•
‫اإللكترونية‬ ‫الطبية‬ ‫السجالت‬
•
‫البيانات‬ ‫تخزين‬
•
‫المريض‬ ‫اإلنترنت‬ ‫شبكة‬ ‫على‬ ‫التعليم‬
•
‫الالسلكية‬ ‫التطبيقات‬
•
‫بعد‬ ‫عن‬ ‫التطبيب‬
•
‫الحيوية‬ ‫القياسات‬
•
‫المحمولة‬ ‫التكنولوجيا‬
•
‫المزايا‬
•
‫المتاحة‬ ‫الحالية‬ ‫البيانات‬ ‫يجعل‬
•
‫و‬ ‫بيانات‬ ‫مجموعة‬ ‫لتلبية‬ ‫األثر‬ ‫تقييم‬ ‫أو‬ ‫للمراقبة‬ ‫الالزمة‬ ‫البيانات‬ ‫جمع‬ ‫بالفعل‬ ‫يتم‬ ‫قد‬
‫فعالية‬ ‫معلومات‬
‫الصحية‬ ‫الرعاية‬
HEDIS
‫الصحية‬ ‫الرعاية‬ ‫منظمات‬ ‫العتماد‬ ‫المشتركة‬ ‫اللجنة‬ ‫أو‬
JCAHO
‫مهم‬

CREATING CHANGE FOR QUALITY AND FIDELITY
The purpose of collecting, analyzing, and reviewing process data is to create a feedback
loop within the program that both leads to improvements that strengthen the program
implementation and maintains the optimally functioning elements of the program. The
feedback loop consists of first interpreting the process data, then for mulating and
implementing corrective managerial or programmatic actions. In other words, changes are
made to ensure program quality and fidelity.
‫و‬ ‫البرنامج‬ ‫تنفيذ‬ ‫تقوي‬ ‫تحسينات‬ ‫إلى‬ ‫تؤدي‬ ‫البرنامج‬ ‫داخل‬ ‫مرتدة‬ ‫تغذية‬ ‫حلقة‬ ‫إنشاء‬ ‫هو‬ ‫ومراجعتها‬ ‫وتحليلها‬ ‫العملية‬ ‫بيانات‬ ‫جمع‬ ‫من‬ ‫الغرض‬
‫تحا‬
‫عل‬ ‫فظ‬
‫المثلى‬ ‫التشغيل‬ ‫عناصر‬ ‫ى‬
‫للبرنامج‬
.
‫أو‬ ‫اإلدارية‬ ‫اإلجراءات‬ ‫وتنفيذ‬ ‫محاكاة‬ ‫أجل‬ ‫من‬ ‫ثم‬ ، ً‫ال‬‫أو‬ ‫العملية‬ ‫بيانات‬ ‫تفسير‬ ‫من‬ ‫المالحظات‬ ‫حلقة‬ ‫تتكون‬
‫التصحيحية‬
.
‫آ‬ ‫بمعنى‬
‫تغييرات‬ ‫إجراء‬ ‫يتم‬ ، ‫خر‬
‫وإخالصه‬ ‫البرنامج‬ ‫جودة‬ ‫لضمان‬
.
‫مهم‬

Interpreting Implementation Data
• Compare process monitoring data with targets stated in the process
objectives
• Attention to coverage is critical
• Get staff insight into congruence between objectives and
achievements
• Consider extent to which program components are provided
•
‫العملية‬ ‫أهداف‬ ‫في‬ ‫المحددة‬ ‫باألهداف‬ ‫العملية‬ ‫مراقبة‬ ‫بيانات‬ ‫مقارنة‬
•
‫األهمية‬ ‫بالغ‬ ‫أمر‬ ‫التغطية‬ ‫إلى‬ ‫االنتباه‬
•
‫واإلنجازات‬ ‫األهداف‬ ‫بين‬ ‫التطابق‬ ‫في‬ ‫الموظفين‬ ‫ثاقبة‬ ‫نظرة‬ ‫على‬ ‫الحصول‬
•
‫البرنامج‬ ‫مكونات‬ ‫توفير‬ ‫مدى‬ ‫في‬ ‫النظر‬

Using Implementation Data for Program Quality and Fidelity
• Implementation-focused actions
– Seek more funds to address under-coverage
– Update needs assessment for ongoing programs
• Process theory-focused actions
– Modify aspects of the process theory other than the interventions themselves
• e.g., change immunization clinic hours to better reach the target population
•
‫التنفيذ‬ ‫على‬ ‫تركز‬ ‫التي‬ ‫اإلجراءات‬
•
‫التغطية‬ ‫نقص‬ ‫لمعالجة‬ ‫األموال‬ ‫من‬ ‫المزيد‬ ‫عن‬ ‫ابحث‬
•
‫الجارية‬ ‫للبرامج‬ ‫االحتياجات‬ ‫تقييم‬ ‫تحديث‬
•
‫العملية‬ ‫نظرية‬ ‫على‬ ‫تركز‬ ‫التي‬ ‫اإلجراءات‬
•
‫نفسها‬ ‫التدخالت‬ ‫بخالف‬ ‫العملية‬ ‫نظرية‬ ‫جوانب‬ ‫تعديل‬
•
‫أفضل‬ ‫بشكل‬ ‫المستهدفين‬ ‫السكان‬ ‫إلى‬ ‫للوصول‬ ‫التحصين‬ ‫عيادة‬ ‫ساعات‬ ‫تغيير‬ ، ‫المثال‬ ‫سبيل‬ ‫على‬

Managing Group Processes for Quality and Fidelity
• Address staff defensiveness with group process skills
– Win over a central staff member
– Use theory of group formation
– Allow a professional, motivated staff to self-organize
•
‫المجموعة‬ ‫عملية‬ ‫مهارات‬ ‫طريق‬ ‫عن‬ ‫الموظفين‬ ‫دفاع‬ ‫معالجة‬
•
‫المركزي‬ ‫الموظف‬ ‫على‬ ‫الفوز‬
•
‫المجموعة‬ ‫تشكيل‬ ‫نظرية‬ ‫استخدام‬
•
‫الذاتي‬ ‫بالتنظيم‬ ‫والمتحمسين‬ ‫المحترفين‬ ‫للموظفين‬ ‫اسمح‬

When Not to Change
• Activities and objectives are on track, on target, on time
• Stakeholders are resistant
• Funds are insufficient
• Process data are ambiguous
• Program replicates an existing, successful program and outcome
data are not yet available
•
‫المحدد‬ ‫الوقت‬ ‫في‬ ، ‫الهدف‬ ‫على‬ ، ‫الصحيح‬ ‫المسار‬ ‫على‬ ‫واألهداف‬ ‫األنشطة‬
•
‫مقاومون‬ ‫المصلحة‬ ‫أصحاب‬
•
‫كافية‬ ‫غير‬ ‫األموال‬
•
‫غامضة‬ ‫العملية‬ ‫بيانات‬
•
‫بعد‬ ‫المتوفرة‬ ‫غير‬ ‫الحالية‬ ‫الناجحة‬ ‫والنتائج‬ ‫البرنامج‬ ‫بيانات‬ ‫البرنامج‬ ‫يكرر‬

Formative Evaluations
• A formative evaluation is an assessment during the initial stages of the implementation
process and preliminary outcomes.
• Assessment of processes and early outcomes It is performed if a program:
– New or experimental programs
– Programs under political scrutiny
• Diagnostic of early problems with the program process theory.
• Often use multiple methods (both quantitative and qualitative)
• Other requirement for a useful formative evaluation is that it is done in a timely manner,
and with strong feedback and reporting to the program manager.
•
‫األولية‬ ‫والنتائج‬ ‫التنفيذ‬ ‫لعملية‬ ‫األولية‬ ‫المراحل‬ ‫خالل‬ ‫تقييم‬ ‫هو‬ ‫التكويني‬ ‫التقييم‬
.
•
‫البرنامج‬ ‫كان‬ ‫إذا‬ ‫تنفيذها‬ ‫يتم‬ ‫المبكرة‬ ‫والنتائج‬ ‫العمليات‬ ‫تقييم‬
:
•
‫تجريبية‬ ‫أو‬ ‫جديدة‬ ‫برامج‬
•
‫السياسي‬ ‫التدقيق‬ ‫تحت‬ ‫برامج‬
•
‫البرنامج‬ ‫عملية‬ ‫نظرية‬ ‫في‬ ‫المبكرة‬ ‫المشكالت‬ ‫تشخيص‬
).
•
‫متعددة‬ ‫أساليب‬ ‫تستخدم‬ ‫ما‬ ‫ا‬ً‫ب‬‫غال‬
(
‫ونوعية‬ ‫كمية‬
)
•
‫البرنامج‬ ‫مدير‬ ‫إلى‬ ‫التقارير‬ ‫وتقديم‬ ‫قوية‬ ‫فعل‬ ‫ردود‬ ‫مع‬ ، ‫المناسب‬ ‫الوقت‬ ‫في‬ ‫يتم‬ ‫أنه‬ ‫هو‬ ‫مفيد‬ ‫تكويني‬ ‫تقييم‬ ‫إلجراء‬ ‫اآلخر‬ ‫الشرط‬
.
‫مهم‬

Quality and Fidelity Across the Pyramid
– Seen in array of accrediting bodies and professional certification
– Quality assurance and fidelity related to individual providers or overall program staff
– Accrediting bodies for organizations
– Similar challenges to the direct services level, though corrective actions may be
more difficult
•
•
‫المهنية‬ ‫والشهادات‬ ‫االعتماد‬ ‫هيئات‬ ‫من‬ ‫مجموعة‬ ‫في‬ ‫ينظر‬
•
‫عام‬ ‫بشكل‬ ‫البرنامج‬ ‫موظفي‬ ‫أو‬ ‫الفردية‬ ‫بالجهات‬ ‫يتعلق‬ ‫فيما‬ ‫واإلخالص‬ ‫الجودة‬ ‫ضمان‬
•
•
‫للمنظمات‬ ‫االعتماد‬ ‫هيئات‬
•
‫صعوبة‬ ‫أكثر‬ ‫تكون‬ ‫قد‬ ‫التصحيحية‬ ‫اإلجراءات‬ ‫أن‬ ‫من‬ ‫الرغم‬ ‫على‬ ، ‫المباشرة‬ ‫الخدمات‬ ‫لمستوى‬ ‫مماثلة‬ ‫تحديات‬

Quality and Fidelity Across the Pyramid, Continued
– Timely implementation monitoring and corrective action can be difficult
– Another challenge is program decentralization
– Management is an infrastructure element
– Most QI approaches are directed toward the workforce
– Balanced Score Card may be particularly relevant
•
•
‫صعبا‬ ‫يكون‬ ‫أن‬ ‫يمكن‬ ‫التصحيحية‬ ‫واإلجراءات‬ ‫التنفيذ‬ ‫رصد‬ ‫المناسب‬ ‫الوقت‬ ‫في‬
•
‫البرنامج‬ ‫في‬ ‫الالمركزية‬ ‫تطبيق‬ ‫هو‬ ‫اآلخر‬ ‫التحدي‬
•
•
‫التحتية‬ ‫البنية‬ ‫في‬ ‫أساسي‬ ‫عنصر‬ ‫اإلدارة‬
•
‫العاملة‬ ‫القوى‬ ‫نحو‬ ‫الجودة‬ ‫تحسين‬ ‫أساليب‬ ‫معظم‬ ‫توجيه‬ ‫يتم‬
•
‫خاص‬ ‫بشكل‬ ‫صلة‬ ‫ذات‬ ‫المتوازنة‬ ‫النقاط‬ ‫بطاقة‬ ‫تكون‬ ‫قد‬

Planning the Intervention
Effects Evaluation
Chapter 12
Translated by
Khaled
Good Luck
‫للتدخل‬ ‫التخطيط‬
‫اآلثار‬ ‫تقييم‬

Key Terminology
• Evaluation
• Evaluation research
• Inter-rater agreement
• Interval variable
• Measure sensitivity
• Mediating variable
• Moderating variable
• Nominal variable
• Ordinal variable
• Reliability
• Research
• Validity
• Learning Objectives
• With this chapter, readers will be
able to: 1. Distinguish among
outcome documentation,
assessment, and evaluation. 2.
Articulate the advantages and
disadvantages of using different
levels of measurement. 3. Describe
important factors to consider when
designing the effects evaluation.

Learning Objectives
1. Distinguish among outcome documentation, assessment, and
evaluation.
2. Articulate the advantages and disadvantages of using different
levels of measurement.
3. Describe important factors to consider when designing the effects
evaluation.
•
:
•
‫والتقييم‬ ‫الختامية‬ ‫الوثائق‬ ‫بين‬ ‫التمييز‬
‫والتقييم‬
.
•
‫القياس‬ ‫من‬ ‫مختلفة‬ ‫مستويات‬ ‫استخدام‬ ‫وعيوب‬ ‫مزايا‬ ‫توضيح‬
.
•
‫التأثيرات‬ ‫تقييم‬ ‫تصميم‬ ‫عند‬ ‫مراعاتها‬ ‫يجب‬ ‫التي‬ ‫الهامة‬ ‫العوامل‬ ‫صف‬
.

• The intervention effect evaluation deserves the same degree of attention during
program planning as does development of the program interventions; ideally, it
should be designed concurrently with the program.
• All too often, it is only after the goals and objectives are finalized and the program
is up and running that attention is focused on developing the evaluation.
• Well-articulated program outcome goals and outcome objectives facilitate
development of the evaluation.
• but insights about the program process can be gained from developing an evaluation
plan. As highlighted in the planning and evaluation cycle (Figure 12-1), the planning
and decisions about the effect evaluation ought to occur as the program is being
developed.
•
‫يجب‬ ، ‫المثالية‬ ‫الناحية‬ ‫من‬ ‫؛‬ ‫البرنامج‬ ‫تدخالت‬ ‫تطوير‬ ‫يستحق‬ ‫مثلما‬ ‫البرنامج‬ ‫تخطيط‬ ‫أثناء‬ ‫االهتمام‬ ‫درجة‬ ‫نفس‬ ‫التدخل‬ ‫تأثير‬ ‫تقييم‬ ‫يستحق‬
‫أن‬
‫تصميمه‬ ‫يتم‬
‫البرنامج‬ ‫مع‬ ‫متزامن‬ ‫بشكل‬
.
•
‫والغايات‬ ‫األهداف‬ ‫من‬ ‫االنتهاء‬ ‫بعد‬ ‫إال‬ ‫التقييم‬ ‫تطوير‬ ‫على‬ ‫التركيز‬ ‫يتم‬ ‫ال‬ ، ‫األحيان‬ ‫من‬ ‫كثير‬ ‫في‬
.
•
‫التقييم‬ ‫تطوير‬ ‫تسهل‬ ‫النتائج‬ ‫وأهداف‬ ‫المعالم‬ ‫واضحة‬ ‫البرنامج‬ ‫نتائج‬ ‫أهداف‬
.
•
‫التقييم‬ ‫خطة‬ ‫وضع‬ ‫من‬ ‫البرنامج‬ ‫عملية‬ ‫حول‬ ‫رؤى‬ ‫اكتساب‬ ‫يمكن‬ ‫ولكن‬
.
‫والتقييم‬ ‫التخطيط‬ ‫دورة‬ ‫في‬ ‫موضح‬ ‫هو‬ ‫كما‬
(
‫الشكل‬
12
-
1
)
‫بتقييم‬ ‫المتعلقة‬ ‫والقرارات‬ ‫التخطيط‬ ‫يحدث‬ ‫أن‬ ‫يجب‬ ،
‫البرنامج‬ ‫تطوير‬ ‫أثناء‬ ‫التأثير‬
.
‫مهم‬

Effect Evaluation in the Planning and Evaluation Cycle

Planning the evaluation begins with selecting the evaluation questions and
then proceeds to developing the details of the evaluation implementation
plan, similar to the details of the program organization plan. Aspects of the
evaluation plan related to data collection are discussed next—namely,
levels of measurement and levels of analysis, as well as techniques to
collect data.
•
‫خ‬ ‫تفاصيل‬ ‫غرار‬ ‫على‬ ، ‫التقييم‬ ‫تنفيذ‬ ‫خطة‬ ‫تفاصيل‬ ‫تطوير‬ ‫في‬ ‫يبدأ‬ ‫ثم‬ ، ‫التقييم‬ ‫أسئلة‬ ‫باختيار‬ ‫التقييم‬ ‫تخطيط‬ ‫يبدأ‬
‫البرنامج‬ ‫تنظيم‬ ‫طة‬
.
‫مناقشة‬ ‫تتم‬
‫ذلك‬ ‫بعد‬ ‫البيانات‬ ‫بجمع‬ ‫المتعلقة‬ ‫التقييم‬ ‫خطة‬ ‫جوانب‬
-
‫ج‬ ‫تقنيات‬ ‫إلى‬ ‫باإلضافة‬ ، ‫التحليل‬ ‫ومستويات‬ ‫القياس‬ ‫مستويات‬ ‫وهي‬
‫البيانات‬ ‫مع‬
.
‫مهم‬

Basis for Decisions about Evaluation Focus and Purpose
• The first step in planning the evaluation is deciding which questions the effect evaluation must be
able to answer.
• The first place to start in developing the evaluation questions is:
• Effect theory
• Logic model
• Outcome objectives (TREW (Timeframe, what portion of Recipients experience what Extent of Which type
of change) objectives— they ought to be the basis for decisions about the focus and purpose of the
intervention evaluation).
• Who the evaluation is for
– e.g., funders, stakeholders, research
•
‫اإلجابة‬ ‫على‬ ً‫ا‬‫قادر‬ ‫التأثير‬ ‫تقييم‬ ‫يكون‬ ‫أن‬ ‫يجب‬ ‫التي‬ ‫األسئلة‬ ‫تحديد‬ ‫هي‬ ‫للتقييم‬ ‫التخطيط‬ ‫في‬ ‫األولى‬ ‫الخطوة‬
‫عليها‬
.
•
‫هو‬ ‫التقييم‬ ‫أسئلة‬ ‫تطوير‬ ‫في‬ ‫للبدء‬ ‫مكان‬ ‫أول‬
:
•
•
‫المنطق‬ ‫نموذج‬
•
‫النتائج‬ ‫أهداف‬
(
TREW
(
‫مدى‬ ‫يختبرون‬ ‫المتلقين‬ ‫من‬ ‫جزء‬ ‫أي‬ ، ‫الزمني‬ ‫اإلطار‬
‫مدى‬
‫التغيير‬ ‫من‬ ‫نوع‬ ‫أي‬
)
‫األهداف‬
-
‫أن‬ ‫يجب‬
‫التدخل‬ ‫تقييم‬ ‫من‬ ‫والغرض‬ ‫بالتركيز‬ ‫المتعلقة‬ ‫للقرارات‬ ‫ا‬ً‫س‬‫أسا‬ ‫تكون‬
.)
•
‫ل‬ ‫التقييم‬ ‫الذي‬
•
‫البحث‬ ، ‫المصلحة‬ ‫أصحاب‬ ، ‫الممولين‬ ، ‫المثال‬ ‫سبيل‬ ‫على‬
‫مهم‬

Characteristics of the Right Question
• Relevant data can be collected
• More than 1 answer is possible
• Produces info that decision makers want and feel they need
•
‫جمعها‬ ‫يمكن‬ ‫الصلة‬ ‫ذات‬ ‫البيانات‬
•
‫ممكنة‬ ‫واحدة‬ ‫إجابة‬ ‫من‬ ‫أكثر‬
•
‫إليها‬ ‫بحاجة‬ ‫أنهم‬ ‫ويشعرون‬ ‫القرار‬ ‫صانعو‬ ‫يريدها‬ ‫معلومات‬ ‫تنتج‬

Outcome Documentation, Assessment, and Evaluation
• Documentation
– To what extent were the outcome objectives met?
– To answer this question, an outcome documentation evaluation will use data collection methods that are
very closely related to the objectives. In this way, the TREW objectives that flowed from the effect theory
become the cornerstone of an outcome documentation evaluation.
• Assessment
– To what extent is any noticeable change or difference in participants related to having received the
program interventions?
• Evaluation
– Were the changes or differences due to participants having received the program and nothing else?
•
‫توثيق‬
•
‫النتائج؟‬ ‫أهداف‬ ‫تحقيق‬ ‫تم‬ ‫مدى‬ ‫أي‬ ‫إلى‬
•
‫باألهداف‬ ‫ا‬ً‫ق‬‫وثي‬ ‫ا‬ً‫ط‬‫ارتبا‬ ‫المرتبطة‬ ‫البيانات‬ ‫جمع‬ ‫أساليب‬ ‫النتائج‬ ‫وثائق‬ ‫تقييم‬ ‫سيستخدم‬ ، ‫السؤال‬ ‫هذا‬ ‫على‬ ‫لإلجابة‬
.
‫أهداف‬ ‫تصبح‬ ، ‫الطريقة‬ ‫وبهذه‬
TREW
‫وثائق‬ ‫تقييم‬ ‫في‬ ‫الزاوية‬ ‫حجر‬ ‫التأثير‬ ‫نظرية‬ ‫من‬ ‫انبثقت‬ ‫التي‬
‫النتائج‬
.
•
‫تقدير‬
•
‫البرنامج؟‬ ‫تدخالت‬ ‫على‬ ‫بالحصول‬ ‫المشاركين‬ ‫في‬ ‫ملحوظ‬ ‫اختالف‬ ‫أو‬ ‫تغيير‬ ‫أي‬ ‫يرتبط‬ ‫مدى‬ ‫أي‬ ‫إلى‬
•
‫تقييم‬
•
‫ذلك؟‬ ‫غير‬ ‫شيء‬ ‫وال‬ ‫للبرنامج‬ ‫المشاركين‬ ‫تلقي‬ ‫عن‬ ‫ناتجة‬ ‫االختالفات‬ ‫أو‬ ‫التغييرات‬ ‫كانت‬ ‫هل‬

Three Levels of Intervention Effects Evaluations
Outcome
documentation
Outcome assessment Outcome evaluation
Purpose Show that outcome and
impact objectives were
met
Determine whether
participants in the
program experienced
any change/benefit
Determine whether the
program caused a
change or benefit for the
recipients
Relationship to program
effect theory
Confirms reaching
targets set in the
objectives that were
based on the theory
Supports the theory Verifies the theory
Level of rigor Minimal Moderate Maximum
Data collection Data type and collection
timing based on
objectives being
measured
Data type based on
effect theory; timing
based on feasibility
Data type based on
effect theory; baseline
and post-intervention
data are required

Evaluation vs. Research
Characteristic Research Evaluation
Goal or purpose Generating new knowledge for prediction Social accounting and program or policy
decision making
Questions addressed Scientist's own questions Derived from program goals and impact
objectives
Problem addressed Areas where knowledge is lacking Program impacts and outcomes
Guiding theory Theory used as basis for hypothesis testing Theory underlying the program interventions,
theory of evaluation
Appropriate techniques Sampling, statistics, hypothesis testing, etc. Whichever research techniques fit with the
problem
Setting Anywhere that is appropriate to the research
question
Anywhere evaluators can access the program
recipients and controls
Dissemination Scientific journals Internal and externally viewed reports, scientific
journals
Allegiance Scientific community Funding source, policy preference, scientific
community

The differences between research and evaluation are important to appreciate
for two reasons:
1. communicating the differences to stakeholders and program staff helps
establish realistic expectations about implementing the evaluation and
about the findings of the evaluation.
2. easier to gain their cooperation of generating knowledge about the
effectiveness of a program and, as such, represents the blending of research
and evaluation.
•
‫لسببين‬ ‫والتقييم‬ ‫البحث‬ ‫بين‬ ‫االختالفات‬ ‫تقدير‬ ‫المهم‬ ‫من‬
:
•
‫و‬ ‫التقييم‬ ‫تنفيذ‬ ‫حول‬ ‫واقعية‬ ‫توقعات‬ ‫تحديد‬ ‫على‬ ‫يساعد‬ ‫البرنامج‬ ‫وموظفي‬ ‫المصلحة‬ ‫أصحاب‬ ‫إلى‬ ‫االختالفات‬ ‫نقل‬ ‫إن‬
‫التقييم‬ ‫نتائج‬
.
•
‫والتقييم‬ ‫البحث‬ ‫بين‬ ‫يمزج‬ ، ‫النحو‬ ‫هذا‬ ‫وعلى‬ ، ‫البرنامج‬ ‫فعالية‬ ‫حول‬ ‫المعرفة‬ ‫توليد‬ ‫في‬ ‫تعاونهم‬ ‫اكتساب‬ ‫في‬ ‫أسهل‬
.

Rigor and Identifying a Program’s Net Effects
• The purpose of the effect evaluation is to identify the net effects, so rigor is used to minimize
the inclusion of nonintervention effects and design effects.
•
‫وتأ‬ ‫التدخل‬ ‫عدم‬ ‫تأثيرات‬ ‫إدراج‬ ‫لتقليل‬ ‫الدقة‬ ‫استخدام‬ ‫يتم‬ ‫لذلك‬ ، ‫الصافية‬ ‫التأثيرات‬ ‫تحديد‬ ‫هو‬ ‫التأثير‬ ‫تقييم‬ ‫من‬ ‫الغرض‬
‫التصميم‬ ‫ثيرات‬
.
‫مهم‬

• The net effects are those that are attributable only to the program,
• Total change includes effects from the intervention as well as effects
that are artifacts of the evaluation design.
Rigor in Evaluation
•
، ‫البرنامج‬ ‫إلى‬ ‫فقط‬ ‫تعزى‬ ‫التي‬ ‫تلك‬ ‫هي‬ ‫الصافية‬ ‫اآلثار‬
•
‫التقييم‬ ‫تصميم‬ ‫آثار‬ ‫من‬ ‫هي‬ ‫التي‬ ‫التأثيرات‬ ‫إلى‬ ‫باإلضافة‬ ‫التدخل‬ ‫من‬ ‫تأثيرات‬ ‫الكلي‬ ‫التغيير‬ ‫يشمل‬
.
‫مهم‬

Three Theories Comprising the Program Effect Theory
1. Causal theory
– Existing and causal factors, moderators and mediators, and health outcome
2. Intervention theory
– How the interventions affect the causal, moderating, and mediating factors
3. Impact theory
– How immediate outcomes become long-term impact
• At minimum, evaluation should measure causal factors and outcomes
‫مهم‬
•
•
‫الصحية‬ ‫والنتائج‬ ، ‫والوسطاء‬ ‫المشرفين‬ ، ‫والسببية‬ ‫الحالية‬ ‫العوامل‬
•
•
‫والوساطة‬ ‫والمعتدلة‬ ‫السببية‬ ‫العوامل‬ ‫على‬ ‫التدخالت‬ ‫تؤثر‬ ‫كيف‬
•
•
‫األجل‬ ‫طويل‬ ‫تأثير‬ ‫الفورية‬ ‫النتائج‬ ‫تصبح‬ ‫كيف‬
•
‫السببية‬ ‫والنتائج‬ ‫العوامل‬ ‫التقييم‬ ‫يقيس‬ ‫أن‬ ‫يجب‬ ، ‫األقل‬ ‫على‬

Nomenclature for Effect Evaluation Variables
• The outcome and impact variables are designated as y ،
• the dependent variable, and the variables that precede the impact are designated as x, independent variables .
• Strictly speaking, any antecedent existing, causal, moderating, or mediating factor is an independent variable, as is the
intervention.
•
‫أنها‬ ‫على‬ ‫واألثر‬ ‫النتائج‬ ‫متغيرات‬ ‫تعيين‬ ‫تم‬
y
،
•
‫مستقلة‬ ‫كمتغيرات‬ ‫تحديدها‬ ‫يتم‬ ‫التأثير‬ ‫تسبق‬ ‫التي‬ ‫والمتغيرات‬ ‫التابع‬ ‫المتغير‬
x
.
•
‫التدخل‬ ‫مثل‬ ‫مثله‬ ، ‫مستقل‬ ‫متغير‬ ‫هو‬ ‫ا‬ً‫ط‬‫وسي‬ ‫أو‬ ً
‫معتدال‬ ‫أو‬ ‫ًا‬‫ي‬‫سبب‬ ‫أو‬ ‫ا‬ً‫ق‬‫ساب‬ ‫موجود‬ ‫عامل‬ ‫أي‬ ، ‫للكلمة‬ ‫الدقيق‬ ‫بالمعنى‬
.

Dependent (y) Variables
• Need to choose most important outcome objectives, not a
“fishing expedition”
• Typically from the 6 health and well-being domains:
– Knowledge, lifestyle behaviors, cognitive processes, mental health,
social health, resources
‫مهم‬
•
‫وليس‬ ، ‫النتائج‬ ‫أهداف‬ ‫أهم‬ ‫اختيار‬ ‫إلى‬ ‫بحاجة‬
"
‫صيد‬ ‫رحلة‬
"
•
‫الستة‬ ‫والرفاهية‬ ‫الصحة‬ ‫مجاالت‬ ‫من‬ ً‫ة‬‫عاد‬
:
•
‫الموارد‬ ، ‫االجتماعية‬ ‫الصحة‬ ، ‫العقلية‬ ‫الصحة‬ ، ‫المعرفية‬ ‫العمليات‬ ، ‫الحياة‬ ‫نمط‬ ‫سلوكيات‬ ، ‫المعرفة‬

Independent (x) Variables
• Called “independent” because they are not influenced by the
outcome
• Start by measuring causal factors
• May be measured before and/or after a program, in
participants and/or controls
‫مهم‬
•
‫عليها‬ ‫يطلق‬
"
‫مستقلة‬
"
‫بالنتيجة‬ ‫تتأثر‬ ‫ال‬ ‫ألنها‬
•
‫السببية‬ ‫العوامل‬ ‫بقياس‬ ‫ابدأ‬
•
‫و‬ ‫قبل‬ ‫قياسه‬ ‫يمكن‬
/
‫و‬ ‫المشاركين‬ ‫في‬ ، ‫البرنامج‬ ‫بعد‬ ‫أو‬
/
‫التحكم‬ ‫عناصر‬ ‫أو‬

Moderating and Mediating Variables
• Mediating – intervene between x and y
• Moderating – change strength or direction of relationship between x
and y
• Including them in the evaluation helps in understanding what influences
intervention effectiveness
•
‫التوسط‬
-
‫بين‬ ‫التدخل‬
x
‫و‬
y
•
‫اإلشراف‬
-
‫بين‬ ‫العالقة‬ ‫اتجاه‬ ‫أو‬ ‫قوة‬ ‫تغيير‬
x
‫و‬
y
•
‫التدخل‬ ‫فعالية‬ ‫على‬ ‫يؤثر‬ ‫ما‬ ‫فهم‬ ‫في‬ ‫يساعد‬ ‫التقييم‬ ‫في‬ ‫إدراجهم‬

Congenital Anomalies Effect Theory
Example with Variables

Measurement Considerations
• Unit of observation must match level of program
– e.g., individuals, schools, communities
• Levels of measurement for variables
- Nominal: are the simplest, in that the information indicates only yes/no, absent/present, or a name.
- ordinal: provide slightly more information by indicating an order, a sequence, or a rank. The most common ordinal
variables are generated from a Likert-type scale, such as good, fair, and poor.
- interval: in which the intervals between the values are equal on an absolute scale.E.G Temperature ,IQ distance.
• Measurement timing
• Sensitivity of measures
•
‫البرنامج‬ ‫مستوى‬ ‫مع‬ ‫المراقبة‬ ‫وحدة‬ ‫تتطابق‬ ‫أن‬ ‫يجب‬
•
‫والمجتمعات‬ ‫والمدارس‬ ‫األفراد‬ ، ‫المثال‬ ‫سبيل‬ ‫على‬
•
‫للمتغيرات‬ ‫القياس‬ ‫مستويات‬
•
-
‫االسمية‬
:
‫نعم‬ ‫فقط‬ ‫المعلومات‬ ‫تشير‬ ‫حيث‬ ، ‫أبسط‬ ‫هي‬
/
‫غائب‬ ، ‫ال‬
/
‫اسم‬ ‫أو‬ ، ‫حاضر‬
.
•
-
‫ترتيبي‬
:
‫ترتيب‬ ‫أو‬ ‫تسلسل‬ ‫أو‬ ‫ترتيب‬ ‫إلى‬ ‫اإلشارة‬ ‫طريق‬ ‫عن‬ ً‫ال‬‫قلي‬ ‫أكثر‬ ‫معلومات‬ ‫بتوفير‬ ‫قم‬
.
‫نوع‬ ‫مقياس‬ ‫من‬ ‫ًا‬‫ع‬‫شيو‬ ‫األكثر‬ ‫الترتيبية‬ ‫المتغيرات‬ ‫إنشاء‬ ‫يتم‬
‫ليكرت‬
‫الجيد‬ ‫مثل‬ ،
‫والفقير‬ ‫والعادل‬
.
•
-
‫الزمني‬ ‫الفاصل‬
:
‫مطلق‬ ‫مقياس‬ ‫على‬ ‫متساوية‬ ‫القيم‬ ‫بين‬ ‫الزمنية‬ ‫الفواصل‬ ‫تكون‬ ‫حيث‬
.
‫مثال‬
:
‫الذكاء‬ ‫مسافة‬ ، ‫الحرارة‬ ‫درجة‬
.
•
‫القياس‬ ‫توقيت‬
•
‫التدابير‬ ‫حساسية‬

Pros and Cons of Levels of Measurement
Type Examples Advantage Disadvantage
Nominal,
categorical
ZIP code, race, yes/no Easy to understand Limited information from
the data
Ordinal, rank Social class, Likert scale,
“top 10” list (worst to best)
Considerable information,
can collapse into nominal
categories
Sometimes statistically
treated as a nominal
variable, ranking can be a
difficult task for
respondents
Interval,
continuous
Temperature, IQ,
distances, dollars, inches,
age
Most information, can
collapse into nominal or
ordinal categories
Can be difficult to construct
valid and reliable interval
variables

Examples of Nominal, Ordinal, and Interval Variables
Outcome variable Nominal Ordinal Interval
Childhood
immunization
Yes/no up-to-date None required, 1
immunization required,
>1 required
Rubella titer
Breastfeeding Yes/no breastfed Category for how long
breastfed: <2 weeks, 2-6
weeks, >6 weeks
# of days breastfed
Housing situation Homeless or not Housing autonomy (own,
rent monthly, rent
weekly, homeless)
# of days living at current
residence

Example Timeline of Intervention and Evaluation Activities
Month Intervention activity Evaluation activity
1 Pilot intervention with small group Conduct focus group to refine intervention acceptability and elements of services
utilization plan
2 Recruit into program, screen for eligibility Randomly assign to program or wait list, collect data for baseline and comparison
•Participants n=150
•Wait listed controls n=150
3 Provide intervention to 1st group of
participants
Analyze baseline, pre-intervention data
4 Recruit into program, screen for eligibility Collect post-intervention data
•Participants (time 1) who completed program n=125
•New nonparticipant controls from wait list n=130
5 Repeat intervention Analyze data
6 Collect post-intervention data
•Previous program participants (time 1) n=95
•Current program participants (time 2) n=120
•Current nonparticipant controls n=110
Analyze data

Threats to Data Quality
1. Missing data: A very common problem is missing data. Data can be
missing on single items in a survey or variables in existing records.
2. Reliability: refers to the extent to which the data are free of errors.
Several sources of errors can diminish the reliability of the data. –
Instrument issues, individual variability day-to-day, interrater
agreement, data entry
3. Validity: The validity of a measure is the degree to which the tool
captures what it purports to measure—in other words, the extent to
which the tool measures what it is intended to measure. A measure is
valid if it truly measures the concept.
•
‫المفقودة‬ ‫البيانات‬
:
‫البيانات‬ ‫فقدان‬ ‫في‬ ‫تتمثل‬ ‫ًا‬‫د‬‫ج‬ ‫شائعة‬ ‫مشكلة‬ ‫هناك‬
.
‫الموج‬ ‫السجالت‬ ‫في‬ ‫متغيرات‬ ‫أو‬ ‫استطالع‬ ‫في‬ ‫مفردة‬ ‫عناصر‬ ‫في‬ ‫مفقودة‬ ‫البيانات‬ ‫تكون‬ ‫أن‬ ‫يمكن‬
‫ودة‬
.
•
:
‫األخطاء‬ ‫من‬ ‫البيانات‬ ‫خلو‬ ‫مدى‬ ‫إلى‬ ‫تشير‬
.
‫البيانات‬ ‫موثوقية‬ ‫من‬ ‫لألخطاء‬ ‫مصادر‬ ‫عدة‬ ‫تقلل‬ ‫أن‬ ‫يمكن‬
.
-
‫اتفاقية‬ ، ‫ا‬ً‫ي‬‫يوم‬ ‫الفردي‬ ‫التباين‬ ، ‫الصك‬ ‫مشاكل‬
interrater
‫إدخال‬ ،
•
‫الصالحية‬
:
‫قياسه‬ ‫إلى‬ ‫تهدف‬ ‫ما‬ ‫األداة‬ ‫بها‬ ‫تجسد‬ ‫التي‬ ‫الدرجة‬ ‫هي‬ ‫المقياس‬ ‫صالحية‬
-
‫قياسه‬ ‫المقصود‬ ‫ما‬ ‫األداة‬ ‫يقيس‬ ‫مدى‬ ‫أي‬ ‫إلى‬ ، ‫آخر‬ ‫بمعنى‬
.
‫يقيس‬ ‫كان‬ ‫إذا‬ ‫صحيح‬ ‫التدبير‬
‫ا‬ً‫ق‬‫ح‬ ‫المفهوم‬
.
‫مهم‬

Contextual Considerations in Evaluation Planning
1. Evaluation budget
– Roughly 10 – 20% of implementation budget
2. Evaluation standards: Criteria for a good evaluation were established by the American Evaluation
Association. Patton (1997) discussed these criteria in terms of four issues. The first criterion is the
process by which decision making occurs regarding the evaluation. That is, a good evaluation is
generated from a decision process that is inclusive and thoughtful. The second criterion is that
stakeholders need to be able to believe the evaluation results and the evaluator. The third
criterion is that stakeholders need to be able to trust the evaluation as being scientifically and
ethically conducted and trust the evaluator as a person to do what is ethical and scientifically
sound. The fourth criterion is that the most feasible and reasonable design is used,
3. Evaluation ethics
4. Stakeholders’ interests
‫مهم‬
•
‫التقييم‬ ‫ميزانية‬
•
‫من‬ ‫يقرب‬ ‫ما‬
10
-
20
‫التنفيذ‬ ‫ميزانية‬ ‫من‬ ٪
•
‫التقييم‬ ‫معايير‬
:
‫األمريكية‬ ‫التقييم‬ ‫جمعية‬ ‫قبل‬ ‫من‬ ‫الجيد‬ ‫التقييم‬ ‫معايير‬ ‫وضع‬ ‫تم‬
.
‫باتون‬ ‫ناقش‬
(
1997
)
‫قضايا‬ ‫أربع‬ ‫حيث‬ ‫من‬ ‫المعايير‬ ‫هذه‬
.
‫بالتقييم‬ ‫يتعلق‬ ‫فيما‬ ‫القرارات‬ ‫اتخاذ‬ ‫بها‬ ‫يتم‬ ‫التي‬ ‫العملية‬ ‫هو‬ ‫األول‬ ‫المعيار‬
.
‫ه‬ ‫وهذا‬
، ‫و‬
‫ومدروس‬ ‫شاملة‬ ‫القرارات‬ ‫اتخاذ‬ ‫عملية‬ ‫من‬ ‫جيد‬ ‫تقييم‬ ‫إنشاء‬ ‫يتم‬
.
‫م‬ِّ‫والمقي‬ ‫التقييم‬ ‫نتائج‬ ‫تصديق‬ ‫على‬ ‫قادرين‬ ‫يكونوا‬ ‫أن‬ ‫إلى‬ ‫بحاجة‬ ‫المصلحة‬ ‫أصحاب‬ ‫أن‬ ‫هو‬ ‫الثاني‬ ‫المعيار‬
.
‫أصح‬ ‫أن‬ ‫هو‬ ‫الثالث‬ ‫المعيار‬
‫أن‬ ‫يجب‬ ‫المصلحة‬ ‫اب‬
‫علم‬ ‫وسليم‬ ‫أخالقي‬ ‫هو‬ ‫بما‬ ‫للقيام‬ ‫كشخص‬ ‫التقييم‬ ‫في‬ ‫يثقوا‬ ‫وأن‬ ً‫ا‬‫وأخالقي‬ ً‫ا‬‫علمي‬ ً‫ء‬‫إجرا‬ ‫باعتباره‬ ‫التقييم‬ ‫في‬ ‫الوثوق‬ ‫على‬ ‫قادرين‬ ‫يكونوا‬
ً‫ا‬‫ي‬
.
‫األكث‬ ‫التصميم‬ ‫استخدام‬ ‫يتم‬ ‫أنه‬ ‫هو‬ ‫الرابع‬ ‫المعيار‬
، ‫ومعقولة‬ ‫جدوى‬ ‫ر‬
•
‫التقييم‬ ‫أخالقيات‬
•
‫المصلحة‬ ‫أصحاب‬ ‫مصالح‬

• For each element of an evaluation, both scientific and programmatic
considerations apply (Table 12-6). These considerations have the
potential to influence the ultimate design and implementation of the
effect evaluations. Reviewing these differences and considerations with
stakeholders can help establish realistic expectations and identify points
on which consensus is needed.
•
‫العلمية‬ ‫االعتبارات‬ ‫تنطبق‬ ، ‫التقييم‬ ‫عناصر‬ ‫من‬ ‫عنصر‬ ‫لكل‬
‫والبرنامجية‬
(
‫الجدول‬
12
-
6
.)
‫التأثير‬ ‫على‬ ‫القدرة‬ ‫لها‬ ‫االعتبارات‬ ‫هذه‬
‫التأثيرات‬ ‫وتقييمات‬ ‫النهائي‬ ‫التصميم‬ ‫في‬
.
‫المص‬ ‫أصحاب‬ ‫مع‬ ‫واالعتبارات‬ ‫االختالفات‬ ‫هذه‬ ‫مراجعة‬ ‫تساعد‬ ‫أن‬ ‫يمكن‬
‫تحديد‬ ‫في‬ ‫لحة‬
‫عليها‬ ‫اإلجماع‬ ‫يتطلب‬ ‫التي‬ ‫النقاط‬ ‫وتحديد‬ ‫واقعية‬ ‫توقعات‬
.
Summary of Evaluation Elements

Summary of Evaluation Elements
Elements of effect
evaluation
Science considerations Program considerations
What to evaluate Impact & outcome variables most likely to
demonstrate the strength of the evidence
for the effect theory
Highest-priority impact and outcome
objectives, variables that meet funding
agency requirements
Who to evaluate Sample representativeness & comparability
to non-participants, ethics of assignment to
program or not
Accessibility of program participants,
availability of easily accessed target
audience members
When to evaluate Effect onset and duration Convenience and accessibility of program
participants
Why evaluate Scientific contributions and knowledge
generation
Program promotion, program refinement,
funding agency requirements
How to evaluate Maximize rigor through choice of
measures, design, and analysis
Minimize intrusion of evaluation into
program through seamlessness of
evaluation with program implementation

Effect Evaluation Across the Pyramid
– Evaluation of individuals may be most straightforward
– Questionnaire construction and secondary data analysis are main considerations
– Similar to direct services level
– How to identify participants and choosing the right unit of observation are main
issues
•
•
‫ا‬ً‫ح‬‫وضو‬ ‫أكثر‬ ‫األفراد‬ ‫تقييم‬ ‫يكون‬ ‫قد‬
•
‫الرئيسية‬ ‫االعتبارات‬ ‫هي‬ ‫الثانوية‬ ‫البيانات‬ ‫وتحليل‬ ‫االستبيانات‬ ‫بناء‬
•
•
‫المباشرة‬ ‫الخدمات‬ ‫لمستوى‬ ‫مماثلة‬
•
‫الرئيسية‬ ‫القضايا‬ ‫هي‬ ‫المراقبة‬ ‫من‬ ‫المناسبة‬ ‫الوحدة‬ ‫واختيار‬ ‫المشاركين‬ ‫تحديد‬ ‫كيفية‬

Effect Evaluation Across the Pyramid, Continued
– Major issues are aggregation of data and selecting the unit of observation
– Evaluation itself is an infrastructure process
– If the program affects infrastructure, then may need to collect individual-level data
– May need to develop infrastructure measures
•
•
‫المراقبة‬ ‫وحدة‬ ‫واختيار‬ ‫البيانات‬ ‫تجميع‬ ‫هي‬ ‫الرئيسية‬ ‫القضايا‬
•
•
‫أساسية‬ ‫بنية‬ ‫عملية‬ ‫ذاته‬ ‫حد‬ ‫في‬ ‫التقييم‬
•
‫الفردي‬ ‫المستوى‬ ‫على‬ ‫البيانات‬ ‫جمع‬ ‫إلى‬ ‫يحتاج‬ ‫فقد‬ ، ‫التحتية‬ ‫البنية‬ ‫على‬ ‫يؤثر‬ ‫البرنامج‬ ‫كان‬ ‫إذا‬
•
‫التحتية‬ ‫البنية‬ ‫تدابير‬ ‫تطوير‬ ‫إلى‬ ‫تحتاج‬ ‫قد‬

Choosing Designs for
Effect Evaluation
Chapter 13
‫التأثير‬ ‫لتقييم‬ ‫التصاميم‬ ‫اختيار‬
Translated by
Khaled
Good Luck

1. Select an appropriate design for an effect evaluation given
constraints on data collection.
2. Identify strategies to avoid possible threats to internal and
external validity.
3. Describe the pros and cons of different sampling strategies.
4. Explain the importance of sample size in assessing the effect of a
program.
•
:
•
‫البيانات‬ ‫جمع‬ ‫على‬ ‫ًا‬‫د‬‫قيو‬ ‫المعطاة‬ ‫التأثيرات‬ ‫لتقييم‬ ‫ا‬ً‫م‬‫مالئ‬ ‫ا‬ً‫م‬‫تصمي‬ ‫حدد‬
.
•
‫والخارجية‬ ‫الداخلية‬ ‫الصالحية‬ ‫على‬ ‫المحتملة‬ ‫التهديدات‬ ‫لتجنب‬ ‫االستراتيجيات‬ ‫تحديد‬
.
•
‫المختلفة‬ ‫العينات‬ ‫أخذ‬ ‫استراتيجيات‬ ‫وسلبيات‬ ‫إيجابيات‬ ‫وصف‬
.
•
‫البرنامج‬ ‫تأثير‬ ‫تقييم‬ ‫في‬ ‫العينة‬ ‫حجم‬ ‫أهمية‬ ‫اشرح‬
.
Learning Objectives

Key Terminology
• Attrition
• Baseline
• Bias
• Bounded outcome
• Cluster trial
• Cohort
• Community trial
• Control, comparison, or
unexposed group
• Design
• Ecological fallacy
• Experimental designs
• Experimental,
intervention, or exposed
group
• External validity
• History threats
• Instrumentation
• Internal validity

• Intervention designs
• Longitudinal designs
• Manipulation
• Methods
• Nonequivalent
• Observational designs
• Post-test
• Pre-test
• Prospective designs
Key Terminology, Continued
• Quasi-experimental
• Random assignment
• Random selection
• Regression to the mean
• Repeated measures
design
• Retrospective designs
• Selection bias
• Testing effect
• Unbounded outcome

• Effect evaluations can be conducted from the perspective of a number
of different disciplines, each of which has its own terminology for
describing designs (Table 13-1).
• The social science of psychology typically focuses on individuals and
uses experimental and quasi-experimental as the terminology to
describe the major classification of designs.
• Health education mostly uses social sciences terminology for designs.
•
‫التصاميم‬ ‫لوصف‬ ‫به‬ ‫خاصة‬ ‫مصطلحات‬ ‫منها‬ ‫لكل‬ ، ‫المختلفة‬ ‫التخصصات‬ ‫من‬ ‫عدد‬ ‫منظور‬ ‫من‬ ‫التأثيرات‬ ‫تقييمات‬ ‫إجراء‬ ‫يمكن‬
(
‫الجدول‬
13
-
1
.)
•
‫يركز‬
‫للت‬ ‫الرئيسي‬ ‫التصنيف‬ ‫لوصف‬ ‫كمصطلحات‬ ‫التجريبية‬ ‫وشبه‬ ‫التجريبية‬ ‫ويستخدم‬ ‫األفراد‬ ‫على‬ ً‫ة‬‫عاد‬ ‫االجتماعي‬ ‫النفس‬ ‫علم‬
‫صميمات‬
.
•
‫للتصاميم‬ ‫االجتماعية‬ ‫العلوم‬ ‫مصطلحات‬ ‫الغالب‬ ‫في‬ ‫الصحي‬ ‫التثقيف‬ ‫يستخدم‬
.
Introduction
‫مهم‬

Contribution of Disciplines to Health Program Evaluation
‫الصحية‬ ‫البرامج‬ ‫تقييم‬ ‫في‬ ‫التخصصات‬ ‫مساهمة‬
Discipline Typical impact or outcome question Typical design terminology
Psychology Are recipients’ outcomes different from non-recipients’
outcomes?
Experimental, quasi-
experimental
Sociology Are there changes over time that might be related to the
program implementation?
Epidemiology Are cases (individuals with the outcome characteristic)
less likely to have had exposure to the program than
controls (individuals without the outcome characteristic)?
Observational
Health services
research
Does differential utilization of services by enrollees
(target audience) and non-enrollees (non-target
audience) lead to differential outcomes?
Experimental, quasi-
experimental, clinical trial

Characteristics of an Ideal Design
The aim in choosing a design is to come as close as possible to a design that
can demonstrate an effect actually caused by the intervention, program or
policy.
An ideal design has three salient characteristics:
1. Comparison/control group similar to the experimental/exposed group
2. Measurement of unbounded outcomes before and after the intervention
3. Minimal threats to internal and external validity
‫مهم‬ ‫المثالي‬ ‫التصميم‬ ‫خصائص‬
•
‫ا‬ ‫أو‬ ‫البرنامج‬ ‫أو‬ ‫التدخل‬ ‫عن‬ ‫ا‬ً‫ي‬‫فعل‬ ‫الناجم‬ ‫التأثير‬ ‫يثبت‬ ‫أن‬ ‫يمكن‬ ‫الذي‬ ‫التصميم‬ ‫من‬ ‫اإلمكان‬ ‫قدر‬ ‫االقتراب‬ ‫هو‬ ‫التصميم‬ ‫اختيار‬ ‫من‬ ‫الهدف‬
‫لسي‬
‫اسة‬
.
•
‫بارزة‬ ‫خصائص‬ ‫ثالث‬ ‫له‬ ‫المثالي‬ ‫التصميم‬
:
•
‫مقارنة‬ ‫مجموعة‬
/
‫التجريبية‬ ‫للمجموعة‬ ‫مماثلة‬ ‫تحكم‬
/
‫المكشوفة‬
•
‫التدخل‬ ‫وبعد‬ ‫قبل‬ ‫المحدودة‬ ‫غير‬ ‫النتائج‬ ‫قياس‬
•
‫والخارجية‬ ‫الداخلية‬ ‫للصالحية‬ ‫التهديدات‬ ‫من‬ ‫األدنى‬ ‫الحد‬

Design Considerations
factors in the choice of a design:
1. Causality
2. Bias
3. Retrospective vs. prospective orientation
4. Time span
5. Groups
6. Bounded vs. unbounded outcome
7. Individual-vs. population-level intervention
‫مهم‬
‫التصميم‬ ‫اختيار‬ ‫في‬ ‫العوامل‬
:
1
.
‫السببية‬
2
.
‫التحيز‬
3
.
‫المحتملين‬ ‫التوجه‬ ‫مقابل‬ ‫رجعي‬ ‫بأثر‬
4
.
‫زمنية‬ ‫فترة‬
5
.
‫المجموعات‬
6
.
‫محدودة‬ ‫غير‬ ‫نتائج‬ ‫مقابل‬ ‫من‬ ‫يحد‬
7
.
‫مقابل‬ ‫الفردية‬
.
‫السكان‬ ‫مستوى‬ ‫على‬ ‫تدخل‬

• Causality: The first decision in choosing a design is to decide whether it is important to
determine if a cause-and-effect relationship exists between receiving the health
program interventions and the health outcomes.
• Bias: The choice of an evaluation design is also influenced by the need to have a design
that is as free of bias as possible given the realities of the evaluation. Bias in design
refers to the extent to which the design is flawed and, therefore, more likely to lead to
an inaccurate conclusion about the effectiveness of the health program.
• External Validity : The flaws are categorized based on whether they affect the ability to
generalize the findings to other populations.
• Internal Validity: whether they affect the ability to say that the intervention made a
difference.
•
‫السببية‬
:
‫ا‬ ‫تدخالت‬ ‫تلقي‬ ‫بين‬ ‫والنتيجة‬ ‫السبب‬ ‫بين‬ ‫عالقة‬ ‫هناك‬ ‫كانت‬ ‫إذا‬ ‫ما‬ ‫تحديد‬ ‫المهم‬ ‫من‬ ‫كان‬ ‫إذا‬ ‫ما‬ ‫تحديد‬ ‫هو‬ ‫التصميم‬ ‫اختيار‬ ‫في‬ ‫األول‬ ‫القرار‬
‫لبر‬
‫والنتائج‬ ‫الصحي‬ ‫نامج‬
‫الصحية‬
.
•
‫التحيز‬
:
‫التقييم‬ ‫لواقع‬ ‫ا‬ً‫نظر‬ ‫اإلمكان‬ ‫قدر‬ ‫التحيز‬ ‫من‬ ٍ‫ل‬‫خا‬ ‫تصميم‬ ‫إلى‬ ‫بالحاجة‬ ‫ا‬ً‫ض‬‫أي‬ ‫التقييم‬ ‫تصميم‬ ‫اختيار‬ ‫يتأثر‬
.
‫الت‬ ‫عيوب‬ ‫مدى‬ ‫إلى‬ ‫التصميم‬ ‫في‬ ‫التحيز‬ ‫يشير‬
‫يؤدي‬ ‫أن‬ ‫المرجح‬ ‫من‬ ، ‫وبالتالي‬ ، ‫صميم‬
‫الصحي‬ ‫البرنامج‬ ‫فعالية‬ ‫حول‬ ‫دقيق‬ ‫غير‬ ‫استنتاج‬ ‫إلى‬
.
•
‫الخارجية‬ ‫الصالحية‬
:
‫األخرى‬ ‫المجموعات‬ ‫على‬ ‫النتائج‬ ‫تعميم‬ ‫على‬ ‫القدرة‬ ‫على‬ ‫تؤثر‬ ‫كانت‬ ‫إذا‬ ‫ما‬ ‫على‬ ً‫ء‬‫بنا‬ ‫العيوب‬ ‫تصنيف‬ ‫يتم‬
.
•
‫الداخلية‬ ‫الصالحية‬
:
‫ا‬ً‫ق‬‫فر‬ ‫أحدث‬ ‫التدخل‬ ‫إن‬ ‫القول‬ ‫على‬ ‫القدرة‬ ‫على‬ ‫تؤثر‬ ‫كانت‬ ‫إذا‬ ‫ما‬
.
‫مهم‬

• Retrospective Designs : Entail gathering data from the point of intervention
backward in time.
• Prospective designs : Entail gathering data forward in time, beginning from a point
prior to the initiation of the intervention
• Time Span: The length of time during which the evaluation is conducted—whether
looking back or looking forward—needs to be considered A design can be described
as longitudinal such as weeks, months, years, or decades, over which the pretest and
the posttest data are collected. although in reality data might be collected multiple
times during one day, a few days, or a week and The two factors that determine
whether a longitudinal design is appropriate are the intervention theory and the
evaluation budget.
‫مهم‬
•
‫رجعي‬ ‫بأثر‬ ‫التصميمات‬
:
‫المناسب‬ ‫الوقت‬ ‫في‬ ‫الوراء‬ ‫إلى‬ ‫التدخل‬ ‫نقطة‬ ‫من‬ ‫البيانات‬ ‫جمع‬ ‫استلزم‬
.
•
‫المستقبلية‬ ‫التصميمات‬
:
‫التدخل‬ ‫بدء‬ ‫قبل‬ ‫نقطة‬ ‫من‬ ‫ا‬ً‫ء‬‫بد‬ ، ‫المناسب‬ ‫الوقت‬ ‫في‬ ‫لألمام‬ ‫البيانات‬ ‫جمع‬ ‫استنبط‬
•
Time Span
:
‫التقييم‬ ‫إجراء‬ ‫خاللها‬ ‫يتم‬ ‫التي‬ ‫الزمنية‬ ‫المدة‬
-
‫المستقبل‬ ‫إلى‬ ‫نتطلع‬ ‫أو‬ ‫الوراء‬ ‫إلى‬ ‫نظرنا‬ ‫سواء‬
-
‫ا‬ ‫مثل‬ ‫طوالني‬ ‫بأنه‬ ‫وصفه‬ ‫يمكن‬ ‫التصميم‬ ‫اعتبار‬ ‫يجب‬
‫أو‬ ‫األشهر‬ ‫أو‬ ‫ألسابيع‬
‫البيانات‬ ‫جمع‬ ‫يتم‬ ‫خاللها‬ ‫واختبارها‬ ‫القبلي‬ ‫االختبار‬ ‫اختبار‬ ‫تم‬ ‫والتي‬ ، ‫العقود‬ ‫أو‬ ‫السنوات‬
.
‫واح‬ ‫يوم‬ ‫خالل‬ ‫مرات‬ ‫عدة‬ ‫البيانات‬ ‫جمع‬ ‫يتم‬ ‫قد‬ ‫الواقع‬ ‫في‬ ‫أنه‬ ‫من‬ ‫الرغم‬ ‫على‬
‫أو‬ ‫أيام‬ ‫بضعة‬ ‫أو‬ ‫د‬
‫التقييم‬ ‫وميزانية‬ ‫التدخل‬ ‫نظرية‬ ‫هما‬ ‫ًا‬‫ب‬‫مناس‬ ‫الطولي‬ ‫التصميم‬ ‫كان‬ ‫إذا‬ ‫ما‬ ‫يحددان‬ ‫اللذان‬ ‫والعامالن‬ ‫أسبوع‬
.

• Groups: Yet another consideration in choosing an evaluation design is whether it is
possible to identify and distinguish between participants and nonparticipants
• Bounded and Unbounded Outcomes: Another consideration in choosing an
evaluation design is the type of outcome being evaluated.
 Examples of unbounded outcomes include knowledge, attitudes, behavioral
intentions, behaviors, and health or physical conditions that can exist before (and
after) the program, such as blood pressure, immune status, weight, or depression.
Birth and death are the prime.
 Examples of bounded health outcomes, as are low birth weight, adolescent
pregnancy, or amputation of toes for diabetics.
 In epidemiology, these results are called discrete health outcomes.
‫مهم‬
•
‫المجموعات‬
:
‫المشاركين‬ ‫وغير‬ ‫المشاركين‬ ‫بين‬ ‫وتمييز‬ ‫تحديد‬ ‫الممكن‬ ‫من‬ ‫كان‬ ‫إذا‬ ‫ما‬ ‫وهو‬ ‫التقييم‬ ‫تصميم‬ ‫اختيار‬ ‫في‬ ‫آخر‬ ‫اعتبار‬ ‫هناك‬
•
‫المحددة‬ ‫وغير‬ ‫المحدودة‬ ‫غير‬ ‫النتائج‬
:
‫تقييمها‬ ‫يتم‬ ‫التي‬ ‫النتيجة‬ ‫نوع‬ ‫هو‬ ‫التقييم‬ ‫تصميم‬ ‫اختيار‬ ‫في‬ ‫آخر‬ ‫اعتبار‬ ‫هناك‬
.
•
‫يم‬ ‫التي‬ ‫البدنية‬ ‫أو‬ ‫الصحية‬ ‫والظروف‬ ، ‫والسلوكيات‬ ، ‫السلوكية‬ ‫والنوايا‬ ، ‫والمواقف‬ ، ‫المعرفة‬ ‫المحدودة‬ ‫غير‬ ‫النتائج‬ ‫على‬ ‫األمثلة‬ ‫تشمل‬
‫كن‬
‫البرنامج‬ ‫قبل‬ ‫توجد‬ ‫أن‬
(
‫وبعده‬
)
‫أو‬ ‫الدم‬ ‫ضغط‬ ‫مثل‬ ،
‫االكتئاب‬ ‫أو‬ ‫الوزن‬ ‫أو‬ ‫المناعية‬ ‫الحالة‬
.
‫الوزراء‬ ‫رئيس‬ ‫هما‬ ‫والموت‬ ‫الوالدة‬
.
•
‫السكر‬ ‫لمرضى‬ ‫القدمين‬ ‫أصابع‬ ‫بتر‬ ‫أو‬ ، ‫المراهقات‬ ‫وحمل‬ ، ‫الوالدة‬ ‫عند‬ ‫الوزن‬ ‫انخفاض‬ ‫مثل‬ ، ‫المربوطة‬ ‫الصحية‬ ‫النتائج‬ ‫على‬ ‫أمثلة‬
.
•
‫المنفصلة‬ ‫الصحية‬ ‫بالنتائج‬ ‫النتائج‬ ‫هذه‬ ‫تسمى‬ ، ‫األوبئة‬ ‫علم‬ ‫في‬
.

Relationship between Causality and Design Costs and Complexity

When Designs are Used
• Experimental or quasi-experimental: (Designs that use random assignment of potential participants to
either receive or not receive the program are called experimental, while designs that do not use random
assignment but are more robust than nonexperimental designs are called quasi-experimental designs).
 in epidemiology and clinical medicine, intervention designs are called clinical trials;
 in the social sciences, they are called experimental designs or quasi-experimental designs).
 In program evaluation, experimental and quasi-experimental designs at the individual level are typically used
when four conditions exist.
1. Information on the outcome exists before the program is delivered
2. Outcome information is available for members of > 2 groups
3. The intervention is received by members of one of the groups
4. After the intervention, data are collected from members of the same groups
• Observational: used to study what constitutes environmental or lifestyle risks. use of observational designs
to examine the relationship between receiving health program interventions and health outcomes has
grown. * Bounded health outcome (e.g., birth or death)
•
‫تجريبية‬ ‫شبه‬ ‫أو‬ ‫تجريبية‬
( :
‫التصميمات‬ ‫تسمى‬ ‫حين‬ ‫في‬ ، ‫تجريبية‬ ، ‫البرنامج‬ ‫تلقي‬ ‫عدم‬ ‫أو‬ ‫لتلقي‬ ‫إما‬ ‫المحتملين‬ ‫للمشاركين‬ ‫عشوائية‬ ‫مهمة‬ ‫تستخدم‬ ‫التي‬ ‫التصميمات‬ ‫تسمى‬
‫الت‬
‫التجري‬ ‫غير‬ ‫التصميمات‬ ‫من‬ ‫أقوى‬ ‫ولكنها‬ ‫العشوائي‬ ‫التعيين‬ ‫تستخدم‬ ‫ال‬ ‫ي‬
‫التجريبية‬ ‫شبه‬ ‫التصميمات‬ ‫بية‬
.)
•
‫؛‬ ‫السريرية‬ ‫التجارب‬ ‫التدخل‬ ‫تصاميم‬ ‫تسمى‬ ، ‫السريري‬ ‫والطب‬ ‫األوبئة‬ ‫علم‬ ‫في‬
•
‫التجريبية‬ ‫شبه‬ ‫التصميمات‬ ‫أو‬ ‫التجريبية‬ ‫التصميمات‬ ‫عليهم‬ ‫يطلق‬ ، ‫االجتماعية‬ ‫العلوم‬ ‫في‬
.)
•
‫شروط‬ ‫أربعة‬ ‫وجود‬ ‫عند‬ ً‫ة‬‫عاد‬ ‫الفردي‬ ‫المستوى‬ ‫على‬ ‫التجريبية‬ ‫وشبه‬ ‫التجريبية‬ ‫التصميمات‬ ‫ُستخدم‬‫ت‬ ، ‫البرنامج‬ ‫تقييم‬ ‫في‬
.
•
‫البرنامج‬ ‫تسليم‬ ‫قبل‬ ‫النتيجة‬ ‫حول‬ ‫معلومات‬ ‫توجد‬
•
‫ألعضاء‬ ‫النتائج‬ ‫معلومات‬ ‫تتوفر‬
>
‫مجموعتين‬
•
‫المجموعات‬ ‫إحدى‬ ‫أعضاء‬ ‫قبل‬ ‫من‬ ‫التدخل‬ ‫تلقي‬ ‫يتم‬
•
‫المجموعات‬ ‫نفس‬ ‫من‬ ‫أعضاء‬ ‫من‬ ‫البيانات‬ ‫جمع‬ ‫يتم‬ ، ‫التدخل‬ ‫بعد‬
•
‫المالحظة‬
:
‫الحياة‬ ‫نمط‬ ‫أو‬ ‫بيئية‬ ‫مخاطر‬ ‫يشكل‬ ‫ما‬ ‫لدراسة‬ ‫تستخدم‬
.
‫الصحية‬ ‫والنتائج‬ ‫الصحي‬ ‫البرنامج‬ ‫تدخالت‬ ‫تلقي‬ ‫بين‬ ‫العالقة‬ ‫لفحص‬ ‫الرصدية‬ ‫التصميمات‬ ‫استخدام‬
.
•
‫المربوطة‬ ‫الصحية‬ ‫النتائج‬
(
‫الوفاة‬ ‫أو‬ ‫الوالدة‬ ‫مثل‬
)
‫مهم‬

When Designs are Used
CHOOSING THE EVALUATION DESIGN
designs can be more easily understood if we think of them as having
three levels of ability to attribute effects to the program, yielding three
groups of designs:
1. outcome documentation,
2. outcome assessment,
3. outcome evaluation.
•
‫التأ‬ ‫إسناد‬ ‫على‬ ‫القدرة‬ ‫من‬ ‫مستويات‬ ‫ثالثة‬ ‫لديهم‬ ‫أن‬ ‫اعتقدنا‬ ‫إذا‬ ‫أكبر‬ ‫بسهولة‬ ‫التصميمات‬ ‫فهم‬ ‫يمكن‬
‫إلى‬ ‫ثيرات‬
‫التصميمات‬ ‫من‬ ‫مجموعات‬ ‫ثالث‬ ‫على‬ ‫الحصول‬ ‫إلى‬ ‫يؤدي‬ ‫مما‬ ، ‫البرنامج‬
:
•
‫النتائج‬ ‫توثيق‬
،
•
،‫النتائج‬ ‫تقييم‬
•
.

• several of the designs can be used at either the individual or population level (Table 13-2). Using the design at the
individual or population level does not change the assumptions that must be met, the timing of data collection, or
the comparability of the groups.
•
‫السكاني‬ ‫أو‬ ‫الفردي‬ ‫المستوى‬ ‫على‬ ‫التصميمات‬ ‫من‬ ‫العديد‬ ‫استخدام‬ ‫يمكن‬
(
‫الجدول‬
13
-
2
.)
‫استخدام‬ ‫يؤدي‬ ‫ال‬
‫توق‬ ‫أو‬ ‫بها‬ ‫الوفاء‬ ‫يجب‬ ‫التي‬ ‫االفتراضات‬ ‫تغيير‬ ‫إلى‬ ‫السكاني‬ ‫أو‬ ‫الفردي‬ ‫المستوى‬ ‫على‬ ‫التصميم‬
‫البيانات‬ ‫جمع‬ ‫يت‬
‫المجموعات‬ ‫بين‬ ‫المقارنة‬ ‫قابلية‬ ‫أو‬

Individual or Aggregate/Population Designs for Effect
Evaluations
Design
Level at which design can be used
Individual Aggregate, population
One group, pre-test/post-test X X
Comparison groups, post-test only X X
Ecological study X
One group, repeated measures or time series X X
Multiple group, repeated measures or time series X X

Individual or Aggregate/Population Designs for
Effect Evaluations, Continued
Design
Level at which design can be used
Individual Aggregate, population
Two group, retrospective; case-control X
Two group, prospective; cohort X
Patched-up cycle X
Two group, pre-test/post-test X X
Two group, pre-test/post-test, with random assignment;
randomized trial
X X (Cluster trial if the groups
are separate populations)

Outcome Documentation Designs
• One group, post-test only
– Major threats: history, maturation
• history(are specific events that happen to participants between the beginning and the end of the program)
• Maturation(arise when participants mature physically or emotionally between the beginning and the end of the
program, independent of the program)
• One group, pre-test/post-test
– Major threats: history, maturation, testing, instrumentation
• Testing: (occurs when the process of being involved in providing the pretest data in some way affects the posttest
data)
• Instrumentation: (in which the concern focuses on possible changes or alterations to how or which data are collected
for the posttest as compared to the pretest.)
‫مهم‬
•
‫فقط‬ ‫االختبار‬ ‫بعد‬ ، ‫واحدة‬ ‫مجموعة‬
•
‫الرئيسية‬ ‫التهديدات‬
:
‫النضج‬ ، ‫التاريخ‬
•
‫التاريخ‬
(
‫البرنامج‬ ‫بداية‬ ‫بين‬ ‫للمشاركين‬ ‫تحدث‬ ‫محددة‬ ‫أحداث‬
‫ونهايةه‬
)
•
‫النضج‬
(
‫البرنامج‬ ‫بداية‬ ‫بين‬ ‫ًا‬‫ي‬‫عاطف‬ ‫أو‬ ‫ًا‬‫ي‬‫جسد‬ ‫المشاركون‬ ‫ينضج‬ ‫عندما‬ ‫ينشأ‬
‫ونهايةه‬
‫البرنامج‬ ‫عن‬ ‫النظر‬ ‫بغض‬ ،
)
•
‫االختبار‬ ‫قبل‬ ، ‫واحدة‬ ‫مجموعة‬
/
‫االختبار‬ ‫بعد‬
•
:
‫األجهزة‬ ، ‫االختبار‬ ، ‫النضج‬ ، ‫التاريخ‬
•
‫االختبار‬
( :
‫البعدي‬ ‫االختبار‬ ‫بيانات‬ ‫على‬ ‫ما‬ ‫بطريقة‬ ‫التمهيدي‬ ‫االختبار‬ ‫بيانات‬ ‫توفير‬ ‫في‬ ‫المشاركة‬ ‫عملية‬ ‫تؤثر‬ ‫عندما‬ ‫يحدث‬
)
•
‫األجهزة‬
( :
‫االختبار‬ ‫مع‬ ‫بالمقارنة‬ ‫البعدي‬ ‫لالختبار‬ ‫جمعها‬ ‫يتم‬ ‫التي‬ ‫البيانات‬ ‫أو‬ ‫لكيفية‬ ‫المحتملة‬ ‫التعديالت‬ ‫أو‬ ‫التغييرات‬ ‫على‬ ‫االهتمام‬ ‫يركز‬ ‫حيث‬
‫ال‬
‫قبلي‬
).

• Comparison group, post-test only
Major threats: attrition, selection bias
Attrition:(is the loss of participants over time due to their dropping out of the
program and/or evaluation, moving away, being lost to follow-up, or death)
selection bias:(refers to the fact that program participants may differ
significantly from those not in the program:
• Ecological design
Major threat: ecological fallacy
ecological fallacy(that is, the evaluators may wrongly assume that group characteristics
apply to all individuals in the group)
Outcome Documentation Designs
‫مهم‬
•
‫فقط‬ ‫االختبار‬ ‫بعد‬ ، ‫المقارنة‬ ‫مجموعة‬
•
:
‫االختيار‬ ‫في‬ ‫والتحيز‬ ‫االستنزاف‬
•
‫االستنزاف‬
( :
‫و‬ ‫البرنامج‬ ‫من‬ ‫تسربهم‬ ‫بسبب‬ ‫الوقت‬ ‫بمرور‬ ‫المشاركين‬ ‫فقدان‬ ‫هو‬
/
‫الوفاة‬ ‫أو‬ ، ‫للمتابعة‬ ‫الضياع‬ ‫أو‬ ، ‫االبتعاد‬ ‫أو‬ ، ‫التقييم‬ ‫أو‬
)
•
‫االختيار‬ ‫تحيز‬
( :
‫البرنامج‬ ‫في‬ ‫الموجودين‬ ‫غير‬ ‫المشاركين‬ ‫عن‬ ‫ا‬ً‫كثير‬ ‫يختلفون‬ ‫قد‬ ‫البرنامج‬ ‫في‬ ‫المشاركين‬ ‫أن‬ ‫حقيقة‬ ‫إلى‬ ‫يشير‬
:
•
‫البيئي‬ ‫التصميم‬
•
‫الرئيسي‬ ‫التهديد‬
:
‫البيئية‬ ‫المغالطة‬
•
‫بيئية‬ ‫مغالطة‬
(
‫المجموعة‬ ‫في‬ ‫األفراد‬ ‫جميع‬ ‫على‬ ‫تنطبق‬ ‫المجموعة‬ ‫خصائص‬ ‫أن‬ ً‫خطأ‬ ‫يفترضون‬ ‫قد‬ ‫ِّمين‬‫المقي‬ ‫أن‬ ‫أي‬
)

Outcome Assessment Designs
• One group, time series
– Major threats: history, instrumentation, maturation, selection, regression to
the mean.
• Multiple group, time series
– Challenges: need same measures on both populations, complex analysis
• Case-control (two group, retrospective)
– Challenges: need good data on who received how much of the program,
may have selection bias
•
‫زمنية‬ ‫سلسلة‬ ، ‫واحدة‬ ‫مجموعة‬
•
‫التهديدات‬ ‫أهم‬
:
‫الوسط‬ ‫إلى‬ ‫االنحدار‬ ، ‫االختيار‬ ، ‫النضج‬ ، ‫األجهزة‬ ، ‫التاريخ‬
.
•
‫زمنية‬ ‫سلسلة‬ ، ‫متعددة‬ ‫مجموعة‬
•
‫التحديات‬
:
‫معقد‬ ‫وتحليل‬ ، ‫السكان‬ ‫كال‬ ‫على‬ ‫التدابير‬ ‫نفس‬ ‫إلى‬ ‫تحتاج‬
•
‫الحالة‬ ‫مراقبة‬
(
‫رجعي‬ ‫بأثر‬ ، ‫مجموعتان‬
)
•
:
‫االختيار‬ ‫في‬ ‫تحيز‬ ‫له‬ ‫يكون‬ ‫قد‬ ، ‫البرنامج‬ ‫مقدار‬ ‫تلقى‬ ‫الذي‬ ‫من‬ ‫حول‬ ‫جيدة‬ ‫بيانات‬ ‫إلى‬ ‫تحتاج‬

• Cohort (two group, prospective)
- Challenges: need to track people and collect data long-term, may have
attrition bias
• Patched-up cycle
- Major threats: maturation, history, regression to the mean
• Two group, pre-test/post-test
- Major threats: nonequivalence, selection, regression to the mean
Outcome Assessment Designs,
Continued
•
‫الفوج‬
(
‫مستقبلية‬ ، ‫مجموعتان‬
)
•
-
:
‫االستنزاف‬ ‫انحياز‬ ‫لها‬ ‫يكون‬ ‫قد‬ ، ‫الطويل‬ ‫المدى‬ ‫على‬ ‫البيانات‬ ‫وجمع‬ ‫الناس‬ ‫تتبع‬ ‫إلى‬ ‫الحاجة‬
•
‫مصححة‬ ‫دورة‬
•
-
:
‫الوسط‬ ‫إلى‬ ‫االنحدار‬ ، ‫التاريخ‬ ، ‫النضج‬
•
‫االختبار‬ ‫قبل‬ ، ‫المجموعة‬ ‫من‬ ‫اثنين‬
/
‫االختبار‬ ‫بعد‬
•
-
:
‫الوسط‬ ‫إلى‬ ‫واالنحدار‬ ‫واالختيار‬ ‫التكافؤ‬ ‫عدم‬

Outcome Evaluation Research Designs
• Random assignment practical issues:
– Ethical concerns (depends on the control used)
– Need more resources
– Target population must be large
– Intervention must be robust to conduct the evaluation properly
• Two group, pre-test/post-test with RA
– Major threat: differential attrition
– Random assignment is the process of determining on a random basis who does and
does not receive the health program/intervention.
– Random selection, which refers to the random identification from the target
population of those who will be in the program and/or evaluation.
•
‫العشوائية‬ ‫االحالة‬ ‫عملية‬ ‫قضايا‬
:
•
‫األخالقية‬ ‫المخاوف‬
(
‫المستخدم‬ ‫التحكم‬ ‫على‬ ‫يعتمد‬
)
•
‫الموارد‬ ‫من‬ ‫مزيد‬ ‫إلى‬ ‫بحاجة‬
.
•
‫ا‬ً‫كبير‬ ‫المستهدف‬ ‫السكان‬ ‫عدد‬ ‫يكون‬ ‫أن‬ ‫يجب‬
•
‫التدخل‬ ‫يكون‬ ‫أن‬ ‫يجب‬
‫إلجراء‬ ‫قويا‬
‫صحيح‬ ‫بشكل‬ ‫التقييم‬
•
‫االختبار‬ ‫قبل‬ ، ‫مجموعتان‬
/
‫مع‬ ‫االختبار‬ ‫بعد‬
RA
•
‫الرئيسي‬ ‫التهديد‬
:
‫التفاضلي‬ ‫االستنزاف‬
•
‫الصحي‬ ‫البرنامج‬ ‫يتلقى‬ ‫من‬ ‫تحديد‬ ‫عملية‬ ‫هو‬ ‫العشوائي‬ ‫التعيين‬
/
‫عليه‬ ‫يحصل‬ ‫وال‬ ‫التدخل‬
.
•
‫و‬ ‫البرنامج‬ ‫في‬ ‫سيكونون‬ ‫الذين‬ ‫ألولئك‬ ‫المستهدفين‬ ‫السكان‬ ‫من‬ ‫العشوائي‬ ‫التحديد‬ ‫إلى‬ ‫يشير‬ ، ‫عشوائي‬ ‫اختيار‬
/
‫التقييم‬ ‫أو‬
.
‫مهم‬

Three Sources of Program Failure

Minimizing Program Failure
Process theory failure Effect theory failure Evaluation failure
Definition Interventions were not
sufficiently implemented to
affect the health problem
Interventions did not or
could not affect the
health problem
Evaluation methods,
design, or sample was
inappropriate so that true
effects were not detected
Design and
methods
considerations
Ability to link process data
with effect data at individual
level
Ideally use random
assignment; consider
timing with regard to
finding maximum
program effect
Tailor instruments to
participants and expected
impact; maximize internal
and external validity
Sample
considerations
Ideally include all program
participants
Ideally use random
assignment and random
selection
Select equivalent
intervention and control
groups; have adequate
sample size

Evaluation Designs Across the Pyramid
– All individual-level designs can be used; may range from inexpensive to costly
– May be most challenging level to evaluate outcomes
– May not be suitable for experimental designs
•
•
‫مكلفة‬ ‫إلى‬ ‫مكلفة‬ ‫غير‬ ‫من‬ ‫تتراوح‬ ‫قد‬ ‫؛‬ ‫استخدامها‬ ‫يمكن‬ ‫الفردي‬ ‫المستوى‬ ‫على‬ ‫التصاميم‬ ‫جميع‬
•
•
‫النتائج‬ ‫لتقييم‬ ‫تحديا‬ ‫األكثر‬ ‫المستوى‬ ‫يكون‬ ‫قد‬
•
‫التجريبية‬ ‫للتصاميم‬ ‫مناسبة‬ ‫تكون‬ ‫ال‬ ‫قد‬

Evaluation Designs Across the Pyramid,
Continued
– Time series designs may be best
– Evaluation outcome question determines whether the focus is on
infrastructure or health problem change  determines design
•
•
‫األفضل‬ ‫تكون‬ ‫قد‬ ‫الزمنية‬ ‫السالسل‬ ‫تصاميم‬
•
•
‫صحية‬ ‫مشكلة‬ ‫أو‬ ‫التحتية‬ ‫البنية‬ ‫تغيير‬ ‫على‬ ‫التركيز‬ ‫كان‬ ‫إذا‬ ‫ما‬ ‫التقييم‬ ‫نتيجة‬ ‫سؤال‬ ‫يحدد‬
-
‫التصميم‬ ‫يحدد‬

Sampling Designs and Data
Sources for Effect Evaluation
Chapter 14
‫التأثير‬ ‫لتقييم‬ ‫البيانات‬ ‫ومصادر‬ ‫العينات‬ ‫تصاميم‬
Translated by
Khaled
Good Luck

1. Select an optimal sampling strategy for the effect evaluation
2. Explain the importance of sample size
3. Assess secondary data sources for validity and reliability
4. Develop a plan for addressing missing data
Learning Objectives
•
:
•
‫التأثير‬ ‫لتقييم‬ ‫األمثل‬ ‫العينات‬ ‫أخذ‬ ‫استراتيجية‬ ‫حدد‬
•
‫العينة‬ ‫حجم‬ ‫أهمية‬ ‫اشرح‬
•
‫وموثوقيتها‬ ‫صحتها‬ ‫من‬ ‫للتأكد‬ ‫الثانوية‬ ‫البيانات‬ ‫مصادر‬ ‫تقييم‬
•
‫البيانات‬ ‫لمعالجة‬ ‫خطة‬ ‫وضع‬
‫المفقودةت‬

Key Terminology
• Attrition
• Convenience sample
• Designs
• Effect size
• Hard-to-reach
populations
• Methods
• Nonprobability or
nonrandom sample
• Nonresponse bias
• Power
• Power analysis
• Probability or random sample
• Purposive sample
• Questionnaire
• Quota sample
• Response bias
• Response rate
• Sample
• Secondary data
• Snowball sample
• Social desirability
• Survey

Sampling Steps and Realities
• Steps:
– Identify program participants and target population
– Develop a plan to select an unbiased sample from each group
• Limitations:
– # people who can/did participate in the program
– Can’t always distinguish participants from nonparticipants, eligibles from ineligibles
•
‫خطوات‬
:
•
‫المستهدفين‬ ‫والسكان‬ ‫البرنامج‬ ‫في‬ ‫المشاركين‬ ‫تحديد‬
•
‫مجموعة‬ ‫كل‬ ‫من‬ ‫متحيزة‬ ‫غير‬ ‫عينة‬ ‫لتحديد‬ ‫خطة‬ ‫وضع‬
•
‫العيوب‬
:
•
‫يمكنهم‬ ‫الذين‬ ‫األشخاص‬ ‫عدد‬
/
‫البرنامج‬ ‫في‬ ‫شاركوا‬
•
‫المؤهلين‬ ‫غير‬ ‫عن‬ ‫واألهلية‬ ‫المشاركين‬ ‫غير‬ ‫عن‬ ‫المشاركين‬ ‫تمييز‬ ‫ا‬ً‫م‬‫دائ‬ ‫يمكن‬ ‫ال‬

Probability and Nonprobability Samples
Sample type Key characteristics Situations when preferred
Probability (Random) Each population element has
a chance of being selected;
known probabilities of
selection
‫لالخت‬ ‫فرصة‬ ‫لديه‬ ‫السكان‬ ‫عناصر‬ ‫من‬ ‫عنصر‬ ‫كل‬
‫يار‬
‫لالختيار‬ ‫المعروفة‬ ‫االحتماالت‬ ‫؛‬
Evaluation must demonstrate
causation; effect evaluation of
a novel program
‫تأث‬ ‫تقييم‬ ‫؛‬ ‫السببية‬ ‫التقييم‬ ‫يثبت‬ ‫أن‬ ‫يجب‬
‫ير‬
‫جديد‬ ‫برنامج‬
Nonprobability (Nonrandom) Unknown probabilities of
selection
‫لالختيار‬ ‫معروفة‬ ‫غير‬ ‫احتماالت‬
Outcome documentation and
assessment designs; no
sampling frame; small or
hard-to-reach population
‫أخذ‬ ‫إطار‬ ‫يوجد‬ ‫ال‬ ‫؛‬ ‫التقييم‬ ‫وتصاميم‬ ‫نتائج‬ ‫توثيق‬
‫إليه‬ ‫الوصول‬ ‫يصعب‬ ‫أو‬ ‫صغير‬ ‫سكان‬ ‫عدد‬ ‫؛‬ ‫العينات‬
‫مهم‬

Sampling Strategies for Hard-to-Reach Populations
• Random-digit dialing:
involves generating phone numbers based on randomly generating the last four digits of the
telephone number.
• Capture-recapture:
involves using two or more lists or observational periods to identify unique individuals who might be
eligible for participation
• Multiplicity, referral, or snowball sampling:
a key informant provides a referral to other potential participants, each of whom then provides
additional referrals, and so on. Over time, the list of potential participants snowballs, growing
through a multiplicity of referrals.
• Venue-based sampling:
This strategy entails going to specific types of locations to find potential participants. Similar to time-
space sampling.
•
‫العشوائي‬ ‫االتصال‬
:
•
‫ا‬ً‫ي‬‫عشوائ‬ ‫الهاتف‬ ‫رقم‬ ‫من‬ ‫األخيرة‬ ‫األربعة‬ ‫األرقام‬ ‫توليد‬ ‫على‬ ً‫ء‬‫بنا‬ ‫هواتف‬ ‫أرقام‬ ‫إنشاء‬ ‫يتضمن‬
.
•
‫العينات‬ ‫فحص‬ ‫إعادة‬ ‫على‬ ‫القبض‬
:
•
‫للمشاركة‬ ‫مؤهلين‬ ‫يكونون‬ ‫قد‬ ‫الذين‬ ‫الفريدين‬ ‫األفراد‬ ‫لتحديد‬ ‫رصد‬ ‫فترات‬ ‫أو‬ ‫أكثر‬ ‫أو‬ ‫قائمتين‬ ‫استخدام‬ ‫يتضمن‬
•
‫الثلج‬ ‫كرة‬ ‫عينات‬ ‫أخذ‬ ‫أو‬ ‫اإلحالة‬ ‫أو‬ ‫التعددية‬
:
•
‫جرا‬ ‫وهلم‬ ، ‫إضافية‬ ‫إحاالت‬ ‫ذلك‬ ‫بعد‬ ‫يقدم‬ ‫منهم‬ ‫كل‬ ، ‫اآلخرين‬ ‫المحتملين‬ ‫للمشاركين‬ ‫إحالة‬ ‫الرئيسي‬ ‫المخبر‬ ‫يقدم‬
.
‫ا‬ ‫من‬ ‫كبير‬ ‫عدد‬ ‫خالل‬ ‫من‬ ‫ينمو‬ ، ‫الثلج‬ ‫كرات‬ ‫المحتملين‬ ‫المشاركين‬ ‫قائمة‬ ، ‫الوقت‬ ‫مرور‬ ‫مع‬
‫إلحاالت‬
.
•
‫المكان‬ ‫أساس‬ ‫على‬ ‫العينات‬ ‫أخذ‬
:
•
‫المحتملين‬ ‫المشاركين‬ ‫على‬ ‫للعثور‬ ‫المواقع‬ ‫من‬ ‫محددة‬ ‫أنواع‬ ‫إلى‬ ‫االنتقال‬ ‫االستراتيجية‬ ‫هذه‬ ‫تستلزم‬
.
‫الزمكان‬ ‫العينات‬ ‫أخذ‬ ‫غرار‬ ‫على‬
.

Factors Affecting Response Rates
1. Whether those who were invited are actually eligible to participate
2. Whether eligible individuals who were invited actually participate
3. Whether participants complete all components of the evaluation
4. Thus, need to track invitations, eligibility, refusals, and completions
Response rate:
is the percentage of individuals who were invited to participate in the evaluation and who
actually participated in the evaluation
•
‫للمشاركة‬ ‫بالفعل‬ ‫مؤهلون‬ ‫دعوتهم‬ ‫تمت‬ ‫الذين‬ ‫أولئك‬ ‫كان‬ ‫إذا‬ ‫ما‬
•
‫المشاركة‬ ‫فعال‬ ‫دعوتهم‬ ‫تمت‬ ‫الذين‬ ‫المؤهلين‬ ‫األفراد‬ ‫سواء‬
•
‫التقييم‬ ‫مكونات‬ ‫جميع‬ ‫إكمال‬ ‫المشاركون‬ ‫كان‬ ‫إذا‬ ‫ما‬
•
‫واالنتهاء‬ ، ‫والرفض‬ ، ‫واألهلية‬ ، ‫الدعوات‬ ‫تتبع‬ ‫إلى‬ ‫تحتاج‬ ، ‫وبالتالي‬
•
‫االستجابة‬ ‫معدل‬
:
•
‫التقييم‬ ‫في‬ ‫بالفعل‬ ‫شاركوا‬ ‫والذين‬ ‫التقييم‬ ‫في‬ ‫للمشاركة‬ ‫دعوتهم‬ ‫تمت‬ ‫الذين‬ ‫لألفراد‬ ‫المئوية‬ ‫النسبة‬ ‫هي‬
‫مهم‬

Nonresponse Concerns
Nonresponse:
may be due to attrition, meaning that program participants or control
subjects are no longer part of the evaluation or the program.
• Increased cost through needing a larger starting sample size
• Nonresponse bias
– Attrition: death, becoming ineligible, refusing to continue, lost to follow-up
•
‫اإلجابة‬ ‫عدم‬
:
•
‫البرنامج‬ ‫أو‬ ‫التقييم‬ ‫من‬ ‫ا‬ً‫ء‬‫جز‬ ‫يعدوا‬ ‫لم‬ ‫للمراقبة‬ ‫الخاضعين‬ ‫األشخاص‬ ‫أو‬ ‫البرنامج‬ ‫في‬ ‫المشاركين‬ ‫أن‬ ‫يعني‬ ‫مما‬ ، ‫االستنزاف‬ ‫بسبب‬ ‫يكون‬ ‫قد‬
.
•
‫البداية‬ ‫عينة‬ ‫أكبر‬ ‫حجم‬ ‫إلى‬ ‫الحاجة‬ ‫خالل‬ ‫من‬ ‫التكلفة‬ ‫زيادة‬
•
‫التحيز‬ ‫االستجابة‬ ‫عدم‬
•
‫االستنزاف‬
:
‫للمتابعة‬ ‫وخسر‬ ، ‫االستمرار‬ ‫ورفض‬ ، ‫مؤهل‬ ‫غير‬ ‫يصبح‬ ، ‫الموت‬
‫مهم‬

Incentives
• May increase response rates
• Consider both the total amount and the payment schedule
• May be monetary or non-monetary
•
‫االستجابة‬ ‫معدالت‬ ‫من‬ ‫تزيد‬ ‫قد‬
•
‫للدفع‬ ‫الزمني‬ ‫والجدول‬ ‫اإلجمالي‬ ‫المبلغ‬ ‫من‬ ‫كل‬ ‫في‬ ‫النظر‬
•
‫نقدية‬ ‫غير‬ ‫أو‬ ‫نقدية‬ ‫تكون‬ ‫قد‬
‫الحوافز‬

Nonprobability Sample Types
Sample type Implementation ease Representativeness Sampling frame
Convenience Easiest None assured, but may occur by
chance
Willingness to
participate in the
evaluation
Purposive Easy None Specific characteristics
of interest
Quota Moderately easy None assured, but by chance
may be representative of those
with chosen characteristics
Specific characteristics
of interest
Snowball Somewhat difficult None; likely to be biased Network of initial
participants
‫مهم‬

Probability Sample Types
Sample type Implementation ease Representativeness Sampling frame
Simple random Easy-to-use random number table High Entire population
Stratified random Moderate because must first
choose stratification variable &
categories
High Entire population, but must
have info to assign
individuals to strata
Systematic Easy to select each nth from a list Moderately high; lower if listing
sequence is not random
List of possible evaluation
participants
Random route Difficult because must define area,
construct route, and choose nth
houses
Moderate to poor depending on
diversity and availability of area
residents
Geographically accessible
area
Cluster or nested Moderate once the cluster has
been identified
Moderately high if clusters &
individuals within clusters are
selected randomly
Population with naturally
occurring clusters
‫مهم‬

Sampling for Outcome Assessment and Evaluation
• Outcome assessment relies on nonprobability samples
– Strategies are generally easy to implement and explain to stakeholders
• Outcome evaluation relies on probability samples
– Increases external validity
– Costs generally increase with complexity
•
‫محتملة‬ ‫غير‬ ‫عينات‬ ‫على‬ ‫النتائج‬ ‫تقييم‬ ‫يعتمد‬
•
‫المصلحة‬ ‫ألصحاب‬ ‫والتفسير‬ ‫التنفيذ‬ ‫سهلة‬ ‫االستراتيجيات‬
•
‫االحتماالت‬ ‫عينات‬ ‫على‬ ‫النتائج‬ ‫تقييم‬ ‫يعتمد‬
•
‫الخارجية‬ ‫الصالحية‬ ‫من‬ ‫يزيد‬
•
‫تعقيد‬ ‫مع‬ ‫عام‬ ‫بشكل‬ ‫التكاليف‬ ‫زيادة‬
‫مهم‬

Data Collection Methods
1. Primary
– Generating new data.
– The most common form of primary data collected is through the use
of surveys and questionnaires.
2. Secondary
– Using existing data
•
‫اولي‬
•
‫جديدة‬ ‫بيانات‬ ‫توليد‬
.
•
‫استخدام‬ ‫هو‬ ‫جمعها‬ ‫يتم‬ ‫التي‬ ‫األولية‬ ‫للبيانات‬ ‫ا‬ً‫ع‬‫شيو‬ ‫األكثر‬ ‫الشكل‬
‫المسوحات‬
‫واالستبيانات‬
.
•
‫ثانوي‬
•
‫الموجودة‬ ‫البيانات‬ ‫باستخدام‬
‫مهم‬

Data Sources for Health Domains
Health domain Example data sources
Physical health Survey data: Self-report
Secondary data: Medical records for diagnoses
Physical data: Scale for weight; lab tests
Observation: Response to physical activity
Knowledge Survey data: Self-report, standardized tests
Secondary data: School records
Physical data: N/A
Observation: Performance of task
Lifestyle behavior Survey data: Self-report
Secondary data: Police records
Physical data: Lab tests related to behaviors (e.g., nicotine, cocaine)
Observation: Behaviors in natural settings
Cognitive processes Survey data: Self-report, standardized tests of cognitive development or problem solving
Secondary data: School records
Physical data: Brain activity imaging
Observation: Problem-solving tasks, narrative

Data Sources for Health Domains, Continued
Health domain Example data sources
Mental health Survey data: Self-reported motivation, values, or attitudes
Secondary data: Medical records diagnostic categories
Physical data: Self-inflicted wounds, lab values
Observation: Emotional bonding
Social health Survey data: Self-report, social network questionnaires, reports from others
Secondary data: Recreational activities attendance records
Physical data: N/A
Observation: Interpersonal interactions
Resources Survey data: Self-report
Secondary data: Employer records, marriage records, school records
Physical data: Address
Observation: Possessions

Questionnaire Construction Considerations
• Use existing, validated items and scales if possible
• Cultural sensitivity
– Translation and/or cultural adaptation
• Avoid using # client goals achieved as an outcome measure
• Pilot test the questionnaire before the evaluation starts
•
‫أمكن‬ ‫إن‬ ‫صحتها‬ ‫من‬ ‫التحقق‬ ‫تم‬ ‫التي‬ ‫الحالية‬ ‫والمقاييس‬ ‫العناصر‬ ‫استخدم‬
•
‫الثقافية‬ ‫الحساسية‬
•
‫و‬ ‫الترجمة‬
/
‫الثقافي‬ ‫التكيف‬ ‫أو‬
•
‫استخدام‬ ‫تجنب‬
#
‫للنتائج‬ ‫كمقياس‬ ‫المحققة‬ ‫العميل‬ ‫أهداف‬
•
‫التقييم‬ ‫بدء‬ ‫قبل‬ ‫لالستبيان‬ ‫تجريبي‬ ‫اختبار‬

• A threat to the quality of questionnaire data, and especially to
self-report data from individuals, comes from the various types
of response bias, the intentional or unconscious systematic way
in which individuals select responses.
• One of the most common types of response bias, known as
social desirability, is answering questions in a manner intended
to make a favorable impression
Social Desirability
•
‫المن‬ ‫والطريقة‬ ، ‫االستجابة‬ ‫تحيز‬ ‫من‬ ‫مختلفة‬ ‫أنواع‬ ‫من‬ ، ‫األفراد‬ ‫من‬ ‫الذاتي‬ ‫اإلبالغ‬ ‫بيانات‬ ‫وخاصة‬ ، ‫االستبيان‬ ‫بيانات‬ ‫جودة‬ ‫تهديد‬ ‫يأتي‬
‫هجي‬
‫أو‬ ‫المقصودة‬ ‫ة‬
‫الالواعية‬
‫يختار‬ ‫التي‬
‫الردود‬ ‫األفراد‬ ‫بها‬
.
•
‫إ‬ ‫انطباع‬ ‫خلق‬ ‫إلى‬ ‫تهدف‬ ‫بطريقة‬ ‫األسئلة‬ ‫على‬ ‫اإلجابة‬ ‫هو‬ ، ‫االجتماعية‬ ‫الرغبة‬ ‫باسم‬ ‫والمعروفة‬ ، ‫ا‬ً‫ع‬‫شيو‬ ‫االستجابة‬ ‫تحيز‬ ‫أنواع‬ ‫أكثر‬ ‫أحد‬
‫يجا‬
‫بي‬
‫مهم‬

Response Bias and Variable Error
Bias
Variable error
Low High
Low Ideal: high range of honest
responses on good measure
Questionable but acceptable
data from high range of
honest responses on poor
measure
High Questionable but acceptable
data from skewed responses
on good measure
Unusable data due to skewed
responses on poor measure

Secondary Data Sources
• Vital records
– Birth certificates, death certificates, disease registries
• Medical records, case files, or insurance claims
• National surveys
– NHANES, NFPS, etc.
•
‫الحيوية‬ ‫السجالت‬
•
‫المرض‬ ‫سجالت‬ ، ‫الوفاة‬ ‫شهادات‬ ، ‫الميالد‬ ‫شهادات‬
•
‫التأمين‬ ‫مطالبات‬ ‫أو‬ ، ‫القضية‬ ‫ملفات‬ ، ‫الطبية‬ ‫السجالت‬
•
‫المسوحات‬
‫الوطنية‬

Physical Data Sources
• Biological samples
– Blood, urine, hair
• Anthropometric measures
– Height, weight, BMI
• Environmental samples
– Ozone levels, bacteria counts
•
‫البيولوجية‬ ‫العينات‬
•
‫والشعر‬ ‫والبول‬ ‫الدم‬
•
‫البشرية‬ ‫القياسات‬ ‫تدابير‬
•
‫الجسم‬ ‫كتلة‬ ‫ومؤشر‬ ‫والوزن‬ ‫الطول‬
•
‫البيئية‬ ‫العينات‬
•
‫والبكتيريا‬ ‫األوزون‬ ‫مستويات‬

Sampling and Data Collection Across
the Pyramid
– More likely to have sampling frame, so probability sample of participants is possible;
may be harder for nonparticipants
– Frame less likely to be knowable or accessible
– Response biases a concern
•
•
‫ا‬ ‫لغير‬ ‫صعوبة‬ ‫أكثر‬ ‫يكون‬ ‫قد‬ ‫؛‬ ‫ممكنة‬ ‫المشاركين‬ ‫احتمال‬ ‫عينة‬ ‫فإن‬ ‫لذلك‬ ، ‫العينات‬ ‫أخذ‬ ‫إطار‬ ‫لها‬ ‫يكون‬ ‫أن‬ ‫المرجح‬ ‫من‬
‫لمشاركين‬
•
•
‫إليها‬ ‫الوصول‬ ‫يمكن‬ ‫أو‬ ‫معروفة‬ ‫لتكون‬ ‫عرضة‬ ‫أقل‬ ‫اإلطار‬
•
‫قلق‬ ‫مصدر‬ ‫التحيز‬ ‫استجابة‬

Sampling and Data Collection Across
the Pyramid, Continued
– Sampling and data collection limited to what can be implemented population-wide
– Whether the evaluation is of the workforce or health outcomes determines
sampling and data collection approaches
•
•
‫السكان‬ ‫مستوى‬ ‫على‬ ‫تنفيذه‬ ‫يمكن‬ ‫ما‬ ‫على‬ ‫البيانات‬ ‫وجمع‬ ‫العينات‬ ‫أخذ‬ ‫تقتصر‬
•
•
‫البيانات‬ ‫وجمع‬ ‫العينات‬ ‫أخذ‬ ‫نهج‬ ‫يحدد‬ ‫الصحية‬ ‫النتائج‬ ‫أو‬ ‫العاملة‬ ‫للقوى‬ ‫التقييم‬ ‫كان‬ ‫إذا‬ ‫ما‬

Quantitative Data Analysis
and Interpretation
Chapter 15
Translated by
Khaled
Good Luck
‫الكمية‬ ‫البيانات‬ ‫تحليل‬
‫وتفسيرها‬

1. Select an appropriate indicator of the amount of change from a program.
2. Identify the elements of a persuasive statistical argument.
3. Choose a statistical procedure given the level of measurement.
Learning Objectives
•
:
•
‫البرنامج‬ ‫من‬ ‫التغيير‬ ‫لمقدار‬ ‫ًا‬‫ب‬‫مناس‬ ‫ا‬ً‫مؤشر‬ ‫حدد‬
.
•
‫المقنعة‬ ‫اإلحصائية‬ ‫الحجة‬ ‫عناصر‬ ‫حدد‬
.
•
‫القياس‬ ‫مستوى‬ ‫إلى‬ ‫بالنظر‬ ‫إحصائي‬ ‫إجراء‬ ‫اختيار‬

Key Terminology
• Aggregation
• Backlash
• Backsliding
• Change
• Clinical significance
• Correlation
• Data cleaning
• Data dredging
• Diffusion
• Ecological correlation
• Ecological fallacy
• Effectiveness

Key Terminology, Continued
• Efficiency
• Historical effect
• Linked data
• Nonparametric tests
• Outliers
• Parametric tests
• Score
• Skip patterns
• Sleeper effect
• Spurious finding
• Statistical significance
• Surprise finding
• Trigger effect

Data Entry and Management
• Choose software based on users’ skills (The software to be used for data analysis must be
chosen before beginning data entry.)
• Need systems to:
– Track data entry
– Manage and store paper documents
– Create backup files
• (1) Data cleaning: is a critical step that can take a noticeable amount of time and effort.
This process involves checking the data for obvious data entry errors
• (2) skip patterns—that is, systematic non-responses to items on a questionnaire.
•
‫المستخدمين‬ ‫مهارات‬ ‫على‬ ‫يعتمد‬ ‫ا‬ً‫ج‬‫برنام‬ ‫اختر‬
(
‫يجب‬
‫إدخال‬ ‫بدء‬ ‫قبل‬ ‫البيانات‬ ‫لتحليل‬ ‫استخدامه‬ ‫سيتم‬ ‫الذي‬ ‫البرنامج‬ ‫اختيار‬
.)
•
‫إلى‬ ‫األنظمة‬ ‫تحتاج‬
:
•
‫البيانات‬ ‫إدخال‬ ‫تتبع‬
•
‫الورقية‬ ‫المستندات‬ ‫وتخزين‬ ‫إدارة‬
•
‫االحتياطي‬ ‫النسخ‬ ‫ملفات‬ ‫إنشاء‬
•
(
1
)
‫البيانات‬ ‫تنظيف‬
:
‫والجهد‬ ‫الوقت‬ ‫من‬ ً‫ا‬‫ملحوظ‬ ً‫ا‬‫قدر‬ ‫تستغرق‬ ‫أن‬ ‫يمكن‬ ‫حاسمة‬ ‫خطوة‬ ‫هي‬
.
‫البيانات‬ ‫إدخال‬ ‫في‬ ‫واضحة‬ ‫أخطاء‬ ‫عن‬ ‫ا‬ً‫ث‬‫بح‬ ‫البيانات‬ ‫فحص‬ ‫العملية‬ ‫هذه‬ ‫تتضمن‬
•
(
2
)
‫األنماط‬ ‫تخطي‬
-
‫االستبيان‬ ‫في‬ ‫للعناصر‬ ‫المنهجية‬ ‫االستجابة‬ ‫عدم‬ ‫أي‬
.

Outliers
• (3) Outliers:
are those variables with reasonable, plausible, yet extraordinary values; they lie outside
the normal or at the extreme ends on the tails of a distribution curve on other words
Plausible values at the extreme ends of the distribution
• Plausible values at the extreme ends of the distribution
• Sources:
1. Instrument or measurement errors
2. Data entry errors
3. Unusual but accurate values
• Must decide whether to include outliers on a case-by-case basis
•
(
3
)
‫المتطرفة‬ ‫القيم‬
:
•
‫التوزيع‬ ‫منحنى‬ ‫ذيول‬ ‫على‬ ‫القصوى‬ ‫األطراف‬ ‫في‬ ‫أو‬ ‫الطبيعية‬ ‫الحدود‬ ‫خارج‬ ‫تقع‬ ‫؛‬ ‫عادية‬ ‫غير‬ ‫ولكنها‬ ‫ومعقولة‬ ‫معقولة‬ ‫بقيم‬ ‫المتغيرات‬ ‫تلك‬ ‫هي‬
‫بمع‬
‫للتوزيع‬ ‫القصوى‬ ‫األطراف‬ ‫في‬ ‫المعقولة‬ ‫القيم‬ ، ‫آخر‬ ‫نى‬
•
‫للتوزيع‬ ‫القصوى‬ ‫األطراف‬ ‫في‬ ‫المعقولة‬ ‫القيم‬
•
‫مصادر‬
:
•
‫القياس‬ ‫أو‬ ‫الصك‬ ‫أخطاء‬
•
‫البيانات‬ ‫إدخال‬ ‫أخطاء‬
•
‫دقيقة‬ ‫ولكنها‬ ‫عادية‬ ‫غير‬ ‫قيم‬
•
‫حدة‬ ‫على‬ ‫حالة‬ ‫كل‬ ‫أساس‬ ‫على‬ ‫المتطرفة‬ ‫القيم‬ ‫تضمين‬ ‫تريد‬ ‫كنت‬ ‫إذا‬ ‫ما‬ ‫تقرر‬ ‫أن‬ ‫يجب‬

Linked Data
• (4) Linked data refers to a data set that results from merging data from more than one
source
• Example uses:
– Connect program participation data with outcome data
– Link program outcome data with process data
– Validate self-reported responses
• Caveats:
– Confidentiality (need unique identifiers to match records)
– Accuracy (duplicate names, dates of birth, etc.) •
(
4
)
‫مصدر‬ ‫من‬ ‫أكثر‬ ‫من‬ ‫البيانات‬ ‫دمج‬ ‫عن‬ ‫تنتج‬ ‫بيانات‬ ‫مجموعة‬ ‫إلى‬ ‫المرتبطة‬ ‫البيانات‬ ‫تشير‬
•
‫المثال‬ ‫يستخدم‬
:
•
‫النتائج‬ ‫ببيانات‬ ‫البرنامج‬ ‫في‬ ‫المشاركة‬ ‫بيانات‬ ‫ربط‬
•
‫العملية‬ ‫ببيانات‬ ‫البرنامج‬ ‫نتائج‬ ‫بيانات‬ ‫ربط‬
•
‫ذاتيا‬ ‫عنها‬ ‫المبلغ‬ ‫الردود‬ ‫صحة‬ ‫من‬ ‫التحقق‬
•
‫تحفظات‬
:
•
‫السرية‬
(
‫السجالت‬ ‫لمطابقة‬ ‫فريدة‬ ‫معرفات‬ ‫إلى‬ ‫تحتاج‬
)
•
‫الدقة‬
(
‫إلخ‬ ، ‫الميالد‬ ‫تواريخ‬ ، ‫المكررة‬ ‫األسماء‬
)

Measuring Change
• One group, pre-test/post-test design:
Amt. of change =
sum (each post-test – each pre-test)
# paired scores
• Nonequivalent group, post-test only design:
Amt. of change =
mean participants’ post-test – mean nonparticipants’ post-test
• Two group, pre-test/post-test designs:
Amt. of change =
(mean participants’ post-test – mean participants’ pre-test) –
(mean nonparticipants’ post-test – mean nonparticipants’ pre-test)

Issues in Quantifying Change
• Direction of the outcome
The direction of the change becomes important for selecting appropriate calculation
techniques, interpreting the statistical results, and finally presenting the findings
• High level of the desired outcome at pre-test:
This process is referred to as diffusion of the innovation through the population.
• Relationship to outcome objective target value
– Effectiveness ratio
– Target adequacy index
• Relationship to intervention effort
– Intervention efficiency index
•
‫النتيجة‬ ‫اتجاه‬
•
، ‫اإلحصائية‬ ‫النتائج‬ ‫وتفسير‬ ، ‫المناسبة‬ ‫الحساب‬ ‫أساليب‬ ‫الختيار‬ ‫ا‬ً‫م‬‫مه‬ ‫التغيير‬ ‫اتجاه‬ ‫يصبح‬
‫النتائج‬ ‫تقديم‬ ً‫ا‬‫وأخير‬
•
‫المسبق‬ ‫االختبار‬ ‫في‬ ‫المرجوة‬ ‫النتيجة‬ ‫من‬ ٍ‫ل‬‫عا‬ ‫مستوى‬
:
•
‫السكان‬ ‫خالل‬ ‫من‬ ‫االبتكار‬ ‫نشر‬ ‫باسم‬ ‫العملية‬ ‫هذه‬ ‫إلى‬ ‫يشار‬
.
•
‫الهدف‬ ‫القيمة‬ ‫الهدف‬ ‫بالنتائج‬ ‫العالقة‬
•
‫الفعالية‬ ‫نسبة‬
•
‫الهدف‬ ‫كفاية‬ ‫مؤشر‬
•
‫التدخل‬ ‫بجهود‬ ‫العالقة‬
•
‫التدخل‬ ‫كفاءة‬ ‫مؤشر‬

Calculating Effectiveness and Adequacy Indices
Clinic A Clinic B Interpretation
Outcome objective target value 98% 98% Value established for the
outcome objective
% taking prenatal vitamins at baseline 60% 85% Baseline or control values
% taking prenatal vitamins post-
program
70% 95% Outcome values
Pre/post-program change 70% - 60% = 10% 95% - 85% = 10% Simple amount of change in
outcome
Effectiveness ratio (70% - 60%)
(98% - 60%)
= .26
(95% - 85%)
(98% - 85% )
= .77
Ratio of actual to planned effect
Target adequacy index 1 – (98% - 70%)
(98% - 60%)
= .27
1 – (98% - 95%)
(98% - 85%)
= .77
Relative closeness to reaching
target value
Intervention efficiency (70% - 60%)
(10 min – 5 min)
= 2% / minute
(95% - 85%)
(15 min – 5 min)
= 1% / minute
Improvement per unit of
intervention effort

Intervention Efficacy
Causal size
Effect size
Small effect Large effect
Small cause Neutral intervention efficacy High intervention efficacy
Large cause Low intervention efficacy Neutral intervention efficacy

Factors Contributing to Total Amount
of Change

Significance
• Statistical significance
– refer to or define as Likelihood of getting the evaluation results by pure
chance
• Clinical significance
– Likelihood of the intervention being beneficial to participants
•
‫إحصائية‬ ‫داللة‬
•
‫البحتة‬ ‫الصدفة‬ ‫طريق‬ ‫عن‬ ‫التقييم‬ ‫نتائج‬ ‫على‬ ‫الحصول‬ ‫إمكانية‬ ‫تحديد‬ ‫أو‬ ‫إلى‬ ‫الرجوع‬ ‫يمكنك‬
•
‫سريرية‬ ‫أهمية‬
•
‫للمشاركين‬ ‫ًا‬‫د‬‫مفي‬ ‫التدخل‬ ‫يكون‬ ‫أن‬ ‫احتمال‬

Aggregating Data
• Aggregation means or refers to summarizing data from across
participants within one nested unit so as to create a variable at the
unit level of analysis
• Data from individuals are combined to create a new measure at the
aggregate level
– Aggregate when those within units are similar to each other
– Intraclass correlation (ICC)
•
‫التحليل‬ ‫وحدة‬ ‫مستوى‬ ‫على‬ ‫متغير‬ ‫إلنشاء‬ ‫وذلك‬ ‫واحدة‬ ‫متداخلة‬ ‫وحدة‬ ‫داخل‬ ‫المشاركين‬ ‫جميع‬ ‫من‬ ‫البيانات‬ ‫تلخيص‬ ‫إلى‬ ‫يشير‬ ‫أو‬ ‫يعني‬ ‫التجميع‬
•
‫الكلي‬ ‫المستوى‬ ‫على‬ ‫جديد‬ ‫مقياس‬ ‫إلنشاء‬ ‫األفراد‬ ‫من‬ ‫البيانات‬ ‫دمج‬ ‫يتم‬
•
‫البعض‬ ‫بعضها‬ ‫مع‬ ‫متشابهة‬ ‫الوحدات‬ ‫داخل‬ ‫الوحدات‬ ‫تكون‬ ‫عندما‬ ‫التجميع‬
•
‫الطبقة‬ ‫داخل‬ ‫العالقة‬

Analysis Procedures
Level of analysis
Level of program intervention
Individual Aggregate Population Infrastructure
Individual Comparison,
association, and
prediction tests
If individuals can be
identified:
comparison and
association tests
identified:
comparison and
association tests
identified:
comparison and
association tests
Aggregate N/A Comparison,
association, and
prediction tests
If subgroups can be
identified:
comparison and
association tests
If subgroups can be
identified:
comparison and
association tests
Population N/A N/A Comparison,
association, and
prediction tests
Comparison,
association, and
prediction tests
Infrastructure N/A N/A Comparison,
association, and
prediction tests
Comparison,
association, and
prediction tests

Analytic purpose - Descriptive Statistics
• Measures of frequency, mean, dispersion, etc.
• Questions answered:
– What were the characteristics of program participants and
nonparticipants?
– How were pre- and post-test scores distributed?
– What percentage of participants reached the objective target values?
– Are the data normally distributed?
‫التحليلي‬ ‫الغرض‬
-
‫الوصفي‬ ‫اإلحصاء‬
•
‫إلخ‬ ، ‫التشتت‬ ، ‫المتوسط‬ ، ‫التردد‬ ‫قياسات‬
.
•
‫األسئلة‬ ‫على‬ ‫اإلجابة‬
:
•
‫المشاركين؟‬ ‫وغير‬ ‫البرنامج‬ ‫في‬ ‫المشاركين‬ ‫خصائص‬ ‫هي‬ ‫ما‬
•
‫االختبار؟‬ ‫وبعد‬ ‫قبل‬ ‫النتائج‬ ‫توزيع‬ ‫تم‬ ‫كيف‬
•
‫الموضوعية؟‬ ‫المستهدفة‬ ‫القيم‬ ‫إلى‬ ‫وصلوا‬ ‫الذين‬ ‫للمشاركين‬ ‫المئوية‬ ‫النسبة‬ ‫هي‬ ‫ما‬
•
‫طبيعي؟‬ ‫بشكل‬ ‫موزعة‬ ‫البيانات‬ ‫هل‬

Common Parametric and Nonparametric Statistical Tests
Type of data
Complexity of question about effect
Comparison Association Prediction
Parametric •Difference scores
•t-tests of difference of
means
•Analyses of variance
(ANOVA, ANCOVA)
•Correlation
•Hierarchical analyses
•Time series
•Regression analyses
•Logistic regression
analyses
Nonparametric Chi-square tests based
on contingency tables
•Chi-square tests based
on contingency tables
•Odds ratio
•Relative risk
•Other (sign test,
Wilcoxon, Kruskal-
Wallis)
•Log-linear regression
analysis
•Probit regression
analysis

Analytic purpose - Comparison
– Are post-test scores different from pre-tests?
– Do outcomes differ for participants vs. nonparticipants?
– Is one group more likely to have a characteristic than another?
Net program effect refers to the degree of intervention effect on participants compared
to any effect on the comparison group, given the amount of error due to the design and
measures used
•
:
•
‫السابقة؟‬ ‫االختبارات‬ ‫عن‬ ‫االختبار‬ ‫بعد‬ ‫ما‬ ‫نتائج‬ ‫تختلف‬ ‫هل‬
•
‫المشاركين؟‬ ‫غير‬ ‫مقابل‬ ‫للمشاركين‬ ‫بالنسبة‬ ‫النتائج‬ ‫تختلف‬ ‫هل‬
•
‫أخرى؟‬ ‫مجموعة‬ ‫من‬ ‫أكثر‬ ‫بميزة‬ ‫ما‬ ‫مجموعة‬ ‫تتمتع‬ ‫أن‬ ‫المحتمل‬ ‫من‬ ‫هل‬
•
‫ال‬ ‫الخطأ‬ ‫مقدار‬ ‫إلى‬ ‫بالنظر‬ ، ‫المقارنة‬ ‫مجموعة‬ ‫على‬ ‫تأثير‬ ‫بأي‬ ‫مقارنة‬ ‫المشاركين‬ ‫على‬ ‫التدخل‬ ‫تأثير‬ ‫درجة‬ ‫إلى‬ ‫البرنامج‬ ‫تأثير‬ ‫صافي‬ ‫يشير‬
‫نات‬
‫التص‬ ‫عن‬ ‫ج‬
‫المستخدمة‬ ‫والتدابير‬ ‫ميم‬

Comparison Analyses
Level of
measurement
Comparison-focused analyses
Analysis procedures Measures of magnitude
Tests of
significance
Nominal by
nominal data
•Difference scores
•Chi-square tests
•Percent or mean difference
•Phi coefficient
•Cramer’s V
•Lambda
p value
Ordinal by ordinal
data
•Median test
•Mann-Whitney U test
•Kruskal-Wallis test
•Sign test
•Wilcoxon matched-pairs, sign-
rank test
•Friedman two-way analysis
•Lambda
•Uncertainty coefficient
•Goodman and Kruskal’s gamma
•Sommer’s d
•Eta coefficient
p value
Interval by interval
data
•t-test of independent samples
•Paired t-test
Difference between means •p value
•Confidence
interval

Analytic purpose - Association
– Was getting more interventions related to a greater amount of change?
– Is the amount of change associated with certain participant characteristics?
• When running many correlation analyses, consider adjusting the p-value to avoid
“false positives.”
• Include moderating and mediating variables if they are in the causal theory.
•
:
•
‫التغيير؟‬ ‫من‬ ‫أكبر‬ ‫بقدر‬ ‫الصلة‬ ‫ذات‬ ‫التدخالت‬ ‫من‬ ‫مزيد‬ ‫على‬ ‫الحصول‬ ‫كان‬
•
‫المشاركين؟‬ ‫خصائص‬ ‫ببعض‬ ‫التغيير‬ ‫مقدار‬ ‫يرتبط‬ ‫هل‬
•
‫القيمة‬ ‫ضبط‬ ‫في‬ ‫فكر‬ ، ‫االرتباط‬ ‫تحليالت‬ ‫من‬ ‫العديد‬ ‫إجراء‬ ‫عند‬
p
‫لتجنب‬
"
‫الخاطئة‬ ‫اإليجابيات‬
."
•
‫السببية‬ ‫النظرية‬ ‫في‬ ‫كانت‬ ‫إذا‬ ‫الوسيطة‬ ‫المتغيرات‬ ‫بتضمين‬ ‫قم‬
.

Association Analyses
Level of
measurement
Tests of
significance
Nominal by
nominal data
•Fisher’s exact for 2x2 table
•Chi-square or independent
samples
•McNemar or Cochran Q for
related samples
•Relative risk
•Coefficient of contingency
•Phi coefficient
•Cramer’s V
•Lambda
•p value
•Confidence
interval
Ordinal by ordinal
data
•Chi-square
•Spearman rank order
•Kendall’s coefficient of
concordance
•Kendall’s tau a, b, or c
•Somer’s d
p value
Interval by interval
data
Multiple regression analyses Eta coefficient p value
Mixed ANOVA for nominal by interval Eta coefficient p value for F
statistic

Example Association Analyses, Using
Worksite Safety
Independent predictor
variables
Dependent outcome variables
Nominal (e.g., ICD-10 for
injury)
Ordinal (e.g., rank among
schools for # of injuries)
Interval (e.g., # students
with emergency
admissions)
Nominal (e.g., race of
student)
•Cramer’s C (> 2 groups)
•Phi (2 groups)
•Chi-square
•Chi-square
•Mann-Whitney
•t-test
•ANOVA
•Kruskal-Wallis
Ordinal (e.g., rank among
schools for test scores)
Probit regression •Spearman’s rho
•Kendall’s tau
•Eta coefficient
•Spearman’s rank
•Linear regression
Interval (e.g., # hours
anger management
education)
Logistic regression Pearson’s r •Pearson’s r
•Multiple correlation
coefficient
•Linear regression

Analytic purpose - Prediction
– How much of an effect did the intervention have?
– How much more improvement might occur with more intervention?
– Prediction is a type of evaluation or Analytic purpose-that difficult to answer the question during
evaluation.
• Answerable only with a robust evaluation design
– Quasi-experimental
– Experimental
ANOVA : is statistical test, If the level of measurement is at the interval level, then ANOVA tests are
appropriate
Prediction is Questions about how much of an effect a programmatic intervention might have on
individuals, aggregates, or populations are basically questions of causation.
‫األساس‬ ‫في‬ ‫وهي‬ ، ‫السكان‬ ‫أو‬ ‫المجاميع‬ ‫أو‬ ‫األفراد‬ ‫على‬ ‫برنامجي‬ ‫تدخل‬ ‫أي‬ ‫يحدثه‬ ‫قد‬ ‫الذي‬ ‫التأثير‬ ‫مقدار‬ ‫حول‬ ‫أسئلة‬ ‫هو‬ ‫التنبؤ‬
‫بالسببية‬ ‫تتعلق‬ ‫أسئلة‬
.
•
:
•
‫للتدخل؟‬ ‫كان‬ ‫التأثير‬ ‫من‬ ‫كم‬
•
‫التدخل؟‬ ‫من‬ ‫المزيد‬ ‫مع‬ ‫التحسن‬ ‫من‬ ‫المزيد‬ ‫يحدث‬ ‫قد‬ ‫كم‬
•
‫التحليلي‬ ‫الغرض‬ ‫أو‬ ‫التقييم‬ ‫من‬ ‫نوع‬ ‫هو‬ ‫التنبؤ‬
-
‫التقييم‬ ‫أثناء‬ ‫السؤال‬ ‫على‬ ‫اإلجابة‬ ‫الصعب‬ ‫من‬
.
•
‫قوي‬ ‫تقييم‬ ‫تصميم‬ ‫مع‬ ‫فقط‬ ‫مسؤولة‬
•
‫التجريبية‬ ‫شبه‬
•
‫تجريبي‬
•
ANOVA
:
‫اختبارات‬ ‫فإن‬ ، ‫الفاصل‬ ‫مستوى‬ ‫عند‬ ‫القياس‬ ‫مستوى‬ ‫كان‬ ‫إذا‬ ، ‫إحصائي‬ ‫اختبار‬ ‫هو‬
ANOVA
‫مناسبة‬

Prediction Analyses
Level of
measurement
Tests of
significance
Nominal by
nominal data
Probit regression analyses Correlation coefficient F statistic
Ordinal by ordinal
data
Trend analyses Correlation coefficient F statistic
Interval by
interval data
•Time series
•Regression discontinuity
Beta coefficient •F statistic
•Confidence
interval
Nominal by
interval data
Logistic regression Entropy concentration F statistic

Interpretation
• Watch out for spurious and surprise findings
• Fallacies of interpretation
– Equating effectiveness with efficiency
– Assuming a constant rate of progress
– Assuming that ongoing improvement can be achieved
– Underestimating the complexity of the change process
– Ecological fallacy •
‫والمفاجئة‬ ‫الزائفة‬ ‫النتائج‬ ‫من‬ ‫احترس‬
•
‫التفسير‬ ‫مغالطات‬
•
‫بالكفاءة‬ ‫الفعالية‬ ‫مساواة‬
•
‫التقدم‬ ‫من‬ ‫ثابت‬ ‫معدل‬ ‫افتراض‬ ‫على‬
•
‫مستمر‬ ‫تحسن‬ ‫تحقيق‬ ‫يمكن‬ ‫أنه‬ ‫افتراض‬ ‫على‬
•
‫التغيير‬ ‫عملية‬ ‫تعقيد‬ ‫من‬ ‫التقليل‬
•
‫البيئية‬ ‫مغالطة‬

Four Fallacies of Interpretation
 One fallacy is equating effectiveness with efficiency.
1. Effectiveness is the extent to which results are achieved.
2. Efficiency is a ratio of amount of effort or resources to the amount of
effect achieved
3. Another fallacy is assuming that a constant rate of progress or health
improvement will or has occurred.
4. The fourth fallacy is underestimating the complexity of the change
process. Behavioral change within individuals involves multiple stages
•
‫بالكفاءة‬ ‫الفعالية‬ ‫مساواة‬ ‫هي‬ ‫واحدة‬ ‫مغالطة‬
.
•
‫النتائج‬ ‫تحقيق‬ ‫مدى‬ ‫هي‬ ‫الفعالية‬
.
•
‫تحقيقه‬ ‫تم‬ ‫الذي‬ ‫التأثير‬ ‫مقدار‬ ‫إلى‬ ‫الموارد‬ ‫أو‬ ‫الجهد‬ ‫مقدار‬ ‫نسبة‬ ‫هي‬ ‫الكفاءة‬
•
‫حدث‬ ‫قد‬ ‫أو‬ ‫يحدث‬ ‫سوف‬ ‫الصحة‬ ‫تحسن‬ ‫أو‬ ‫التقدم‬ ‫من‬ ‫ثابت‬ ‫معدل‬ ‫أن‬ ‫افتراض‬ ‫هي‬ ‫أخرى‬ ‫مغالطة‬
.
•
‫التغيير‬ ‫عملية‬ ‫تعقيد‬ ‫من‬ ‫تقلل‬ ‫الرابعة‬ ‫المغالطة‬
.
‫متعددة‬ ‫مراحل‬ ‫على‬ ‫ينطوي‬ ‫األفراد‬ ‫داخل‬ ‫السلوكي‬ ‫التغيير‬

Ecological Fallacy
• The ecological fallacy is refers to the assumption that a
group characteristic applies to all individuals within that
group
•
‫المجموع‬ ‫تلك‬ ‫داخل‬ ‫األفراد‬ ‫جميع‬ ‫على‬ ‫تنطبق‬ ‫المجموعة‬ ‫خاصية‬ ‫أن‬ ‫افتراض‬ ‫إلى‬ ‫اإليكولوجية‬ ‫المغالطة‬ ‫يشير‬
‫ة‬

Analysis and Interpretation Across the
Pyramid
– Individual as the unit of analysis
– Programs may be small, prompting concerns about sample size for statistical
analysis
– Level of analysis and possible aggregation are big considerations
•
•
‫التحليل‬ ‫كوحدة‬ ‫فردي‬
•
‫اإلحصائي‬ ‫للتحليل‬ ‫العينة‬ ‫حجم‬ ‫بشأن‬ ‫المخاوف‬ ‫يثير‬ ‫مما‬ ، ‫صغيرة‬ ‫البرامج‬ ‫تكون‬ ‫قد‬
•
•
‫كبيرة‬ ‫اعتبارات‬ ‫المحتمل‬ ‫والتجميع‬ ‫التحليل‬ ‫مستوى‬

Analysis and Interpretation Across the Pyramid, Continued
– Less likely to have control groups, so trend analyses are more common
– Focus may be changes within individuals who make up the infrastructure, or changes in
the population after altering the infrastructure
– Analysis may be at individual or higher levels of aggregation
•
•
‫ا‬ً‫ع‬‫شيو‬ ‫أكثر‬ ‫االتجاهات‬ ‫تحليالت‬ ‫فإن‬ ‫لذلك‬ ، ‫تحكم‬ ‫مجموعات‬ ‫لديك‬ ‫يكون‬ ‫أن‬ ‫المحتمل‬ ‫غير‬ ‫من‬
•
•
‫التحتية‬ ‫البنية‬ ‫تغيير‬ ‫بعد‬ ‫السكان‬ ‫في‬ ‫تغييرات‬ ‫أو‬ ، ‫التحتية‬ ‫البنية‬ ‫يشكلون‬ ‫الذين‬ ‫األفراد‬ ‫داخل‬ ‫تغييرات‬ ‫التركيز‬ ‫يكون‬ ‫قد‬
•
‫التجميع‬ ‫من‬ ‫أعلى‬ ‫أو‬ ‫فردية‬ ‫مستويات‬ ‫على‬ ‫التحليل‬ ‫يكون‬ ‫قد‬

Qualitative Methods for Planning
and Evaluation
Chapter 16
‫والتقييم‬ ‫للتخطيط‬ ‫النوعية‬ ‫الطرق‬
Translated by
Khaled
Good Luck

Learning Objectives
1. Select an appropriate qualitative design for process and effect
evaluations.
2. Describe the basic procedures for analyzing qualitative data.
3. Develop qualitative methods to address evaluation questions.
•
:
•
‫والتأثير‬ ‫العمليات‬ ‫لتقييم‬ ‫المناسب‬ ‫النوعي‬ ‫التصميم‬ ‫حدد‬
.
•
‫النوعية‬ ‫البيانات‬ ‫لتحليل‬ ‫األساسية‬ ‫اإلجراءات‬ ‫صف‬
.
•
‫التقييم‬ ‫أسئلة‬ ‫لمعالجة‬ ‫نوعية‬ ‫أساليب‬ ‫تطوير‬
.

Key Terminology
• Case study
• Category saturation
• Confirmability
• Credibility
• Dependability
• Exhaustive
• Focus group
• Implied meanings
• Key informant
• Manifest meanings
• Mutually exclusive
• Photovoice
• Theoretical sample
• Transferability
• Triangulation

Qualitative Methods in the Planning and Evaluation Cycle
• Assessment
– Insight into needs, barriers, etc. of target population
• Program and effect theory development
– Incorporate target audience’s explanatory models
– Staff’s theory-in-use during program revision
• Process monitoring
– Adds different types of information
– Enhance explanatory power of evaluation design and data
– Explain why/how certain outcomes occurred
•
‫تقدير‬
•
‫المستهدفين‬ ‫السكان‬ ‫من‬ ‫وغيرها‬ ‫والحواجز‬ ‫االحتياجات‬ ‫ثاقبة‬ ‫نظرة‬
•
‫البرنامج‬ ‫وتأثير‬ ‫نظرية‬ ‫تطوير‬
•
‫المستهدف‬ ‫للجمهور‬ ‫التوضيحية‬ ‫النماذج‬ ‫دمج‬
•
‫البرنامج‬ ‫مراجعة‬ ‫أثناء‬ ‫الموظفين‬ ‫استخدام‬ ‫نظرية‬
•
•
‫المعلومات‬ ‫من‬ ‫مختلفة‬ ‫أنواع‬ ‫يضيف‬
•
•
‫والبيانات‬ ‫التقييم‬ ‫لتصميم‬ ‫التوضيحية‬ ‫القوة‬ ‫تعزيز‬
•
‫لماذا‬ ‫اشرح‬
/
‫النتائج‬ ‫بعض‬ ‫حدثت‬ ‫كيف‬

Comparing Qualitative Perspectives
Perspective Basic question addressed Planning and evaluation relevance
Content analysis Which themes are in the text? Thoughts & perspectives revealed in text and
dialogues
Critical analysis How has power shaped it? Participants’ view of their ability to be in control
of the health problem & solution; staff’s view of
their autonomy in improving the program
Ethnography What are the norms and values
(culture)?
Participants’ cultural forces that contribute to
the problem and acceptance of the program
Grounded theory What are the relationships (theory)? Explanations that participants and staff have for
the health problem and possible solutions
Phenomenology What does it mean to the person? Participants’ meaning of content and the
problem being addressed
• The first challenge faced by evaluators and program planners is to select the best
qualitative method to answer the questions at hand.
•
‫المطروحة‬ ‫األسئلة‬ ‫على‬ ‫لإلجابة‬ ‫نوعية‬ ‫طريقة‬ ‫أفضل‬ ‫اختيار‬ ‫هو‬ ‫البرامج‬ ‫ومخططو‬ ‫المقيمون‬ ‫يواجه‬ ‫الذي‬ ‫األول‬ ‫التحدي‬

Qualitative Perspectives and Methods
Perspective Typical methods
Content analysis Focus groups, surveys with open-ended questions,
narrative designs
Critical analysis Individual in-depth interviews
Ethnography Case study, participant observation, observations
Grounded theory Individual in-depth interviews
Phenomenology Individual in-depth interviews

Comparing Qualitative Methods
Method Key benefits Key challenges
Case study Understanding context as an
influence on the program or
participant
Complex, overwhelming
amount of data; definition of
case
Observations Identify sequence of causes
and effects; may identify new
behaviors or events
Difficult to obtain reliable
data without recording
devices; sampling frame
difficult to establish
Individual in-depth interviews Rich insights into personal
thoughts, values, meanings,
and attributions
Identifying individuals who
are willing to be open

Comparing Qualitative Methods,
Continued
Method Key benefits Key challenges
Focus groups Inexpensive given amount & type of
data collected; get collective views
rather than individual views
Need training in managing group
process; need good data recording
method
Survey with open-
ended questions
Very inexpensive Poor handwriting & unclear
statements can make data useless
Narrative designs Very inexpensive; insights into social
& cultural influences on thoughts &
actions
Need training in data analysis; may
not have credibility with
stakeholders; difficult to select text
most relevant to problem or
program

Individual In-Depth Interviews
• Interviews: The most widely known and used of the qualitative methods.
• Can be used throughout planning and evaluation cycle
– Key informants especially useful in assessment and planning
• Focus on meanings and attributions, or insights into poorly understood
phenomenon
• Interviewer skills are critical
•
‫المقابالت‬
:
‫النوعية‬ ‫األساليب‬ ‫واستخدام‬ ‫شهرة‬ ‫األكثر‬
.
•
‫والتقييم‬ ‫التخطيط‬ ‫دورة‬ ‫طوال‬ ‫استخدامها‬ ‫يمكن‬
•
‫والتخطيط‬ ‫التقييم‬ ‫في‬ ‫خاص‬ ‫بشكل‬ ‫مفيدة‬ ‫الرئيسيين‬ ‫المخبرين‬
•
‫الفهم‬ ‫سيئة‬ ‫ظاهرة‬ ‫ثاقبة‬ ‫نظرة‬ ‫أو‬ ، ‫والصفات‬ ‫المعاني‬ ‫على‬ ‫التركيز‬
•
‫ضرورية‬ ‫المقابلة‬ ‫مهارات‬

Open-Ended Survey Questions
• Can be used during assessment, process monitoring, or impact
questionnaires
• Examples:
– Anything else?
– What was the best/worst…?
– What suggestions do you have?
• Challenges include illegible handwriting and overly brief responses
•
‫التأثير‬ ‫استبيانات‬ ‫أو‬ ‫العملية‬ ‫مراقبة‬ ‫أو‬ ‫التقييم‬ ‫أثناء‬ ‫استخدامها‬ ‫يمكن‬
•
‫أمثلة‬
:
•
‫آخر؟‬ ‫شيء‬ ‫من‬ ‫هل‬
•
‫األفضل‬ ‫هو‬ ‫ما‬
/
‫األسوأ‬
...
‫؟‬
•
‫لديك؟‬ ‫التي‬ ‫االقتراحات‬ ‫هي‬ ‫ما‬
•
‫مفرط‬ ‫بشكل‬ ‫مختصرة‬ ‫وردود‬ ‫مقروء‬ ‫غير‬ ‫اليد‬ ‫خط‬ ‫التحديات‬ ‫وتشمل‬

Focus Groups
• The focus group method of collecting qualitative data involves conducting an interview
with a group of individuals.
• Can be used during assessment and program planning; possibly process monitoring
– Not appropriate for outcomes assessment
• Group dynamics during the interview may produce new insights
• Same data analysis steps as for individual in-depth interviews
• The focus group method of collecting qualitative data involves conducting an interview
with a group of individuals, thereby taking advantage of the group dynamic, which itself can
lead to discussions and revelations of new information
•
‫األفراد‬ ‫من‬ ‫مجموعة‬ ‫مع‬ ‫مقابلة‬ ‫إجراء‬ ‫النوعية‬ ‫البيانات‬ ‫لجمع‬ ‫التركيز‬ ‫مجموعة‬ ‫طريقة‬ ‫تتضمن‬
.
•
‫الرصد‬ ‫عملية‬ ‫ربما‬ ‫؛‬ ‫البرنامج‬ ‫وتخطيط‬ ‫التقييم‬ ‫أثناء‬ ‫استخدامها‬ ‫يمكن‬
•
‫النتائج‬ ‫لتقييم‬ ‫مناسب‬ ‫غير‬
•
‫جديدة‬ ‫رؤى‬ ‫تنتج‬ ‫قد‬ ‫المقابلة‬ ‫خالل‬ ‫المجموعة‬
•
‫المتعمقة‬ ‫الفردية‬ ‫المقابالت‬ ‫في‬ ‫كما‬ ‫نفسها‬ ‫البيانات‬ ‫تحليل‬ ‫خطوات‬
•
‫المجموعة‬ ‫ديناميكية‬ ‫من‬ ‫االستفادة‬ ‫وبالتالي‬ ، ‫األفراد‬ ‫من‬ ‫مجموعة‬ ‫مع‬ ‫مقابلة‬ ‫إجراء‬ ‫النوعية‬ ‫البيانات‬ ‫لجمع‬ ‫التركيز‬ ‫مجموعة‬ ‫طريقة‬ ‫تتضمن‬
،
‫مناقشات‬ ‫إلى‬ ‫ذاتها‬ ‫حد‬ ‫في‬ ‫تؤدي‬ ‫أن‬ ‫يمكن‬ ‫والتي‬

Focus Group Data Collection
• Ideally 8 – 12 participants
– Balance between homogeneity and heterogeneity
– Scheduling can be challenging
• Interviewer must elicit responses from quiet participants
• Recording must be unobtrusive
– Use > 1 method (e.g., audio recording plus handwritten notes)
•
‫مثاليا‬
8
-
12
‫مشاركا‬
•
‫التجانس‬ ‫وعدم‬ ‫التجانس‬ ‫بين‬ ‫التوازن‬
•
‫تحديا‬ ‫يكون‬ ‫أن‬ ‫يمكن‬ ‫جدولة‬
•
‫الهادئين‬ ‫المشاركين‬ ‫من‬ ‫استجابات‬ ‫إجراء‬ ‫الباحث‬ ‫على‬ ‫يجب‬
•
‫مزعج‬ ‫غير‬ ‫التسجيل‬ ‫يكون‬ ‫أن‬ ‫يجب‬
•
‫استخدم‬
>
‫واحدة‬ ‫طريقة‬
(
‫ال‬ ‫بخط‬ ‫المكتوبة‬ ‫المالحظات‬ ‫إلى‬ ‫باإلضافة‬ ‫الصوتي‬ ‫التسجيل‬ ، ‫المثال‬ ‫سبيل‬ ‫على‬
‫يد‬
)

Observation
• Observation using one’s own eyes to collect data is another qualitative method
• May be non-participatory or participatory, in a variety of settings
• Data collection
– May use audio-visual recording
– Coding events as they occur with standardized tool
– Keeping detailed notes after observation
•
‫أخرى‬ ‫نوعية‬ ‫طريقة‬ ‫هي‬ ‫البيانات‬ ‫لجمع‬ ‫المرء‬ ‫عيون‬ ‫باستخدام‬ ‫المالحظة‬
•
‫اإلعدادات‬ ‫من‬ ‫متنوعة‬ ‫مجموعة‬ ‫في‬ ، ‫تشاركية‬ ‫أو‬ ‫المشاركة‬ ‫على‬ ‫قائمة‬ ‫غير‬ ‫تكون‬ ‫قد‬
•
‫البيانات‬ ‫جمع‬
•
‫والمرئي‬ ‫الصوتي‬ ‫التسجيل‬ ‫تستخدم‬ ‫قد‬
•
‫قياسية‬ ‫أداة‬ ‫باستخدام‬ ‫حدوثها‬ ‫عند‬ ‫الترميز‬ ‫أحداث‬
•
‫المالحظة‬ ‫بعد‬ ‫التفصيلية‬ ‫المالحظات‬ ‫حفظ‬

Case Study
• A case study is an empirical inquiry into existing phenomena in their real-life
contexts when the boundary between what is being studied and its context
is not clearly evident.
• Address Qs of how/why something occurred
• If used in effect evaluation, program implementation or program itself is the
case
– Program participants may also be cases in effect evaluations
•
‫واضحة‬ ‫وسياقه‬ ‫دراسته‬ ‫يجري‬ ‫ما‬ ‫بين‬ ‫الحدود‬ ‫تكون‬ ‫ال‬ ‫عندما‬ ‫الواقعية‬ ‫سياقاتها‬ ‫في‬ ‫الموجودة‬ ‫الظواهر‬ ‫في‬ ‫تجريبي‬ ‫بحث‬ ‫هي‬ ‫الحالة‬ ‫دراسة‬
.
•
‫عنوان‬
Qs
‫كيف‬
/
‫ما‬ ‫شيء‬ ‫حدث‬ ‫لماذا‬
•
‫الحال‬ ‫هو‬ ‫نفسه‬ ‫البرنامج‬ ‫أو‬ ‫البرنامج‬ ‫تنفيذ‬ ‫يكون‬ ، ‫التأثير‬ ‫تقييم‬ ‫في‬ ‫استخدامه‬ ‫تم‬ ‫إذا‬
•
‫التنفيذ‬ ‫قيد‬ ‫حاالت‬ ‫ا‬ً‫ض‬‫أي‬ ‫البرنامج‬ ‫في‬ ‫المشاركون‬ ‫يكون‬ ‫قد‬

Case Study Data Collection
• Defining the case is critical
– Consider what makes the case very typical or very unique
• Multiple data sources
– Primary: interviews, observations, surveys
– Secondary: agency documents, program promotional materials, policies
• Overwhelming amounts of data  “analysis paralysis”
•
‫األهمية‬ ‫بالغ‬ ‫أمر‬ ‫القضية‬ ‫تحديد‬
•
‫للغاية‬ ‫نوعها‬ ‫من‬ ‫فريدة‬ ‫أو‬ ‫للغاية‬ ‫نموذجية‬ ‫القضية‬ ‫يجعل‬ ‫ما‬ ‫في‬ ‫النظر‬
•
‫متعددة‬ ‫بيانات‬ ‫مصادر‬
•
‫األولية‬
:
‫والمالحظات‬ ‫المقابالت‬
‫والمسوحات‬
•
‫الثانوية‬
:
‫والسياسات‬ ، ‫للبرنامج‬ ‫الترويجية‬ ‫والمواد‬ ، ‫الوكالة‬ ‫وثائق‬
•
‫البيانات‬ ‫من‬ ‫هائلة‬ ‫كميات‬
-
"
‫التحليل‬ ‫شلل‬
"

Photovoice
• Photovoice is a relatively new approach to data collection that involves the
use of photography by the participants and analysis of their photographs
to understand the phenomena and lives of the participants who took the
photographs.
• Participants’ photos are analyzed
• Interpretation through dialogue and discussion
• Useful when there are language difficulties
•
Photovoice
‫ا‬ ‫الذين‬ ‫المشاركين‬ ‫وحياة‬ ‫ظواهر‬ ‫لفهم‬ ‫صورهم‬ ‫وتحليل‬ ‫الفوتوغرافي‬ ‫للتصوير‬ ‫المشاركين‬ ‫استخدام‬ ‫يتضمن‬ ‫البيانات‬ ‫لجمع‬ ‫ًا‬‫ي‬‫نسب‬ ‫جديد‬ ‫نهج‬ ‫هو‬
‫لتق‬
‫الصور‬ ‫طوا‬
.
•
‫المشاركين‬ ‫صور‬ ‫تحليل‬ ‫يتم‬
•
‫والمناقشة‬ ‫الحوار‬ ‫خالل‬ ‫من‬ ‫التفسير‬
•
‫لغوية‬ ‫صعوبات‬ ‫هناك‬ ‫تكون‬ ‫عندما‬ ‫مفيد‬

Narrative Designs
• Can be used in assessment, process monitoring, and identifying
outcomes of policy changes
• Used text as data
– Personal diaries, agency/program records, media reports
• Often follow content analysis approach
•
‫السياسة‬ ‫تغييرات‬ ‫نتائج‬ ‫وتحديد‬ ‫العملية‬ ‫ومراقبة‬ ‫التقييم‬ ‫في‬ ‫استخدامها‬ ‫يمكن‬
•
‫النص‬ ‫استخدم‬
‫كبيانات‬
•
‫الوكالة‬ ‫وسجالت‬ ، ‫الشخصية‬ ‫اليوميات‬
/
‫اإلعالم‬ ‫وسائل‬ ‫تقارير‬ ، ‫البرنامج‬
•
‫المحتوى‬ ‫تحليل‬ ‫نهج‬ ‫اتبع‬ ‫األحيان‬ ‫من‬ ‫كثير‬ ‫في‬

Multiple Methods and Triangulation
• Triangulation generically refers to the use of more than one method for the purpose of
confirming, disconfirming, or modifying information gained through one of the Methods.
• Using > 1 method to compensate for limitations of each approach
– Can use qualitative and/or quantitative methods together
– May reveal entirely different results
• Triangulation
– Confirming, disconfirming, or modifying info gained from one method
• Challenges in how to analyze and synthesize data
•
‫إحدى‬ ‫خالل‬ ‫من‬ ‫المكتسبة‬ ‫المعلومات‬ ‫تعديل‬ ‫أو‬ ‫تأكيد‬ ‫إلغاء‬ ‫أو‬ ‫تأكيد‬ ‫لغرض‬ ‫طريقة‬ ‫من‬ ‫أكثر‬ ‫استخدام‬ ‫إلى‬ ‫عام‬ ‫بشكل‬ ‫التثليث‬ ‫يشير‬
‫الطرق‬
.
•
‫باستخدام‬
>
1
‫نهج‬ ‫كل‬ ‫على‬ ‫المفروضة‬ ‫القيود‬ ‫عن‬ ‫للتعويض‬ ‫طريقة‬
•
‫و‬ ‫النوعية‬ ‫الطرق‬ ‫استخدام‬ ‫يمكن‬
/
‫ا‬ً‫ع‬‫م‬ ‫الكمية‬ ‫أو‬
•
‫تماما‬ ‫مختلفة‬ ‫نتائج‬ ‫عن‬ ‫تكشف‬ ‫قد‬
•
‫التثليث‬
•
‫واحدة‬ ‫طريقة‬ ‫من‬ ‫المكتسبة‬ ‫المعلومات‬ ‫تعديل‬ ‫أو‬ ‫تأكيد‬ ‫إلغاء‬ ‫أو‬ ‫تأكيد‬
•
‫البيانات‬ ‫وتوليف‬ ‫تحليل‬ ‫كيفية‬ ‫في‬ ‫التحديات‬

scientific Rigor in Qualitative Methods
• The following four elements of scientific rigor in qualitative methods :
1. Credibility
– Triangulation
– Use outsiders to aid in interpretation
– Refine hypotheses with negative cases
2. Transferability (also known as applicability)
– Applicability to other contexts or respondents
3. Dependability
- Document analysis steps
- Use reliability statistics
4. Confirm-ability
- Document analysis steps
- Document researcher’s perceptions
- the findings are truly from the respondents,
rather than reflecting the researcher’s
perceptions or biases.
•
‫مصداقية‬
•
‫التثليث‬
•
‫التفسير‬ ‫في‬ ‫للمساعدة‬ ‫الغرباء‬ ‫استخدم‬
•
‫السلبية‬ ‫الحاالت‬ ‫مع‬ ‫الفرضيات‬ ‫صقل‬
•
‫إنتقال‬
‫إلى‬ ‫من‬ ‫الملكية‬
•
‫اآل‬ ‫المجيبين‬ ‫أو‬ ‫السياقات‬ ‫على‬ ‫التطبيق‬ ‫قابلية‬
‫خرين‬
‫النوعية‬ ‫األساليب‬ ‫في‬ ‫العلمية‬ ‫للصرامة‬ ‫التالية‬ ‫األربعة‬ ‫العناصر‬
:
•
‫الجدارة‬
•
-
‫الوثيقة‬ ‫تحليل‬ ‫خطوات‬
•
‫إحصائيات‬ ‫استخدام‬
•
4
.
‫القدرة‬ ‫تأكيد‬
•
-
‫الوثيقة‬ ‫تحليل‬ ‫خطوات‬
•
-
‫تصورات‬ ‫وثيقة‬
‫الباحث‬
•
‫تحيز‬ ‫أو‬ ‫الباحث‬ ‫تصورات‬ ‫تعكس‬ ‫أن‬ ‫من‬ ‫بدال‬ ، ‫المستطلعين‬ ‫من‬ ‫حقا‬ ‫هي‬ ‫النتائج‬
‫اته‬
.

Sampling for Qualitative Methods
• Two key considerations
1. Design
2. Sampling strategy to be used
• Purposive samples
– Random selection and power analysis not relevant
•
‫الرئيسية‬ ‫االعتبارات‬ ‫من‬ ‫اثنين‬
•
‫التصميم‬
•
‫الستخدامها‬ ‫العينات‬ ‫أخذ‬ ‫استراتيجية‬
•
‫هادفة‬ ‫عينات‬
•
‫صلة‬ ‫ذي‬ ‫غير‬ ‫الطاقة‬ ‫وتحليل‬ ‫العشوائي‬ ‫االختيار‬

Sampling for Qualitative Methods
Method Sampling considerations
Case study Choice of case based on being either “usual” or “unusual”; maximum # of cases feasible
to conduct
Observations Ability to sample behaviors or events without altering their quality; need to obtain
category saturation
Individual in-depth
interviews
Need to obtain category saturation; choice of individuals based on theoretical sampling
Focus groups Participant representativeness within & across groups; max size of each focus group; min
# focus groups to capture diversity of views
Survey with open-
ended questions
Linked to sampling strategy for the survey; likelihood of write-in responses
Narrative designs Quantity and quality of existing documents available for review; access to existing
documents

Qualitative Sampling Strategies
Sampling
strategy
Types of cases used Use
Convenience Those that are accessible and wiling Saves time and recruitment money
Critical cases Exemplar cases; those that are
unique in an important way
Permits generalization to similar
cases
Deviant cases Highly unusual cases Reveals factors associated with
unique or extreme conditions and
may lead to new theory
Maximum
variation
Cases with differing experiences Fosters category saturation with
most possible categories

Qualitative Sampling Strategies,
Continued
Sampling
strategy
Types of cases used Use
Random
purposeful
Cases randomly selected from large
sampling pool
Adds credibility to the sample and
thus some indication of
generalizability
Typical cases Usual or normal cases Broadly applicable theory or
categories, but doesn’t address full
breadth of program effects
Theory based Cases with theoretical construct Elaborates or refines the theory

Overview of the Analytic Process
• Determine codable units and identify them in the data
– e.g., paragraphs, facial expressions
• Understand manifest and implied meanings
• Category formation and naming
• Define mutually exclusive and exhaustive categories
• Present findings to participants to confirm or revise interpretation
• Generate working theories based on results
•
‫البيانات‬ ‫في‬ ‫وتحديدها‬ ‫المشفرة‬ ‫الوحدات‬ ‫تحديد‬
•
‫الوجه‬ ‫تعبيرات‬ ، ‫الفقرات‬ ، ‫المثال‬ ‫سبيل‬ ‫على‬
•
‫والضمنية‬ ‫الواضحة‬ ‫المعاني‬ ‫فهم‬
•
‫وتسمية‬ ‫الفئة‬ ‫تشكيل‬
•
‫وشاملة‬ ‫متبادلة‬ ‫حصرية‬ ‫فئات‬ ‫تحديد‬
•
‫التفسير‬ ‫مراجعة‬ ‫أو‬ ‫لتأكيد‬ ‫للمشاركين‬ ‫النتائج‬ ‫قدم‬
•
‫النتائج‬ ‫أساس‬ ‫على‬ ‫العمل‬ ‫نظريات‬ ‫توليد‬

Data Analysis Software
• Features:
– Diagram relationships among categories, count units of analysis, coding
text, theory building
• Software facilitates analysis work, but does not do the work
• Examples: Atlas.ti, NUD*IST, Ethnograph, AnSWR
•
‫المميزات‬
:
•
‫النظرية‬ ‫بناء‬ ، ‫الترميز‬ ‫نص‬ ، ‫التحليل‬ ‫وحدات‬ ، ‫الفئات‬ ‫بين‬ ‫العالقات‬ ‫تخطيط‬
•
‫بالعمل‬ ‫يقوم‬ ‫ال‬ ‫لكنه‬ ، ‫التحليل‬ ‫عمل‬ ‫البرنامج‬ ‫يسهل‬

Analysis Issues to Consider
• Tempting to count # of occurrences
– What is the numerator? Denominator?
• Iterative and sometimes messy data analysis
• Cost
– Travel costs to study sites
– Transcription costs (1 hour of interview = 3 hours transcription)
• Staff training for data collection and analysis •
‫الحوادث‬ ‫من‬ ‫عدد‬ ‫لحساب‬ ‫المغري‬
•
‫المقام‬ ‫البسط؟‬ ‫هو‬ ‫ما‬
-
‫مشتركة‬ ‫صفة‬
-
‫حالة؟‬
•
‫البيانات‬ ‫تحليل‬ ‫فوضوي‬ ‫وأحيانا‬ ‫تكراري‬
•
‫كلفة‬
•
‫المواقع‬ ‫لدراسة‬ ‫السفر‬ ‫تكاليف‬
•
‫النسخ‬ ‫تكاليف‬
(
‫المقابلة‬ ‫من‬ ‫واحدة‬ ‫ساعة‬
=
3
‫النسخ‬ ‫من‬ ‫ساعات‬
)
•
‫وتحليله‬ ‫البيانات‬ ‫جمع‬ ‫على‬ ‫الموظفين‬ ‫تدريب‬

Presenting Findings
• Transferability: describe data collection contexts
• Dependability and confirmation: document category development
• Confirmation: include participants’ words to humanize findings
• Diagram relationships among categories
• Relate findings to program theory or logic model, if possible
•
‫النقل‬ ‫قابلية‬
:
‫البيانات‬ ‫جمع‬ ‫سياقات‬ ‫وصف‬
•
‫والتأكيد‬ ‫االعتمادية‬
:
‫المستند‬ ‫فئة‬ ‫تطوير‬
•
‫التأكيد‬
:
‫النتائج‬ ‫على‬ ‫اإلنساني‬ ‫الطابع‬ ‫إلضفاء‬ ‫المشاركين‬ ‫كلمات‬ ‫تضمين‬
•
‫الفئات‬ ‫بين‬ ‫العالقات‬ ‫تخطيط‬
•
‫أمكن‬ ‫إن‬ ، ‫المنطق‬ ‫نموذج‬ ‫أو‬ ‫البرنامج‬ ‫بنظرية‬ ‫النتائج‬ ‫اربط‬

Qualitative Methods Across the
Pyramid
Services level Assessment Planning Monitoring Outcome
evaluation
Direct Interview,
photovoice
Interview Observation Interview,
observation
Enabling Focus group,
observation
Focus group Observation Interview
Population Focus group Focus group Focus group Focus group
Infrastructure Case study,
narrative
Case study Case study Case study,
interview, focus
groups

Program Evaluators’
Responsibilities
Chapter 17
Translated by
Khaled
Good Luck
‫البرنامج‬ ‫مقيم‬ ‫مسؤوليات‬

1. Develop an informed consent document that includes the eight required elements.
2. Describe the evaluator’s ethical and reporting responsibilities.
3. Develop a final evaluation report with an executive summary and
recommendations
Learning Objectives
•
:
•
‫المطلوبة‬ ‫الثمانية‬ ‫العناصر‬ ‫تتضمن‬ ‫مستنيرة‬ ‫موافقة‬ ‫وثيقة‬ ‫تطوير‬
.
•
‫األخالقية‬ ‫المسؤوليات‬ ‫وصف‬
‫واإلبالغية‬
‫للمقيم‬
.
•
‫وتوصيات‬ ‫تنفيذي‬ ‫ملخص‬ ‫مع‬ ‫النهائي‬ ‫التقييم‬ ‫تقرير‬ ‫تطوير‬

Key Terminology
• Articulate
• Credibility
• Ethics
• Executive summary
• Generalizability
• Informed consent
• Interesting
• Magnitude
• Meta-analysis
• Sense making

• Ethics remains a foremost concern in health care.
• Ethics is the discipline or study of rights, morals, and principles that guide human
behavior.
• Issues become ethical when:
 basic human rights are involved
 dilemmas arise as to what might be the moral and principled course of action.
ETHICAL RESPONSIBILITIES
•
‫الصحية‬ ‫الرعاية‬ ‫في‬ ‫رئيسي‬ ‫قلق‬ ‫مصدر‬ ‫األخالق‬ ‫تزال‬ ‫ال‬
.
•
‫اإلنسان‬ ‫سلوك‬ ‫توجه‬ ‫التي‬ ‫والمبادئ‬ ‫واألخالق‬ ‫الحقوق‬ ‫دراسة‬ ‫أو‬ ‫االنضباط‬ ‫هي‬ ‫األخالق‬
.
•
‫عندما‬ ‫أخالقية‬ ‫القضايا‬ ‫تصبح‬
:
•
‫متورطة‬ ‫األساسية‬ ‫اإلنسان‬ ‫حقوق‬
•
‫والمبدئي‬ ‫األخالقي‬ ‫العمل‬ ‫مسار‬ ‫يكون‬ ‫قد‬ ‫لما‬ ‫معضالت‬ ‫تنشأ‬ ‫عندما‬
.

•
‫الصحية‬ ‫الرعاية‬ ‫في‬ ‫رئيسي‬ ‫قلق‬ ‫مصدر‬ ‫األخالق‬ ‫تظل‬
.
‫اإلنسان‬ ‫سلوك‬ ‫توجه‬ ‫التي‬ ‫والمبادئ‬ ‫واألخالق‬ ‫الحقوق‬ ‫دراسة‬ ‫أو‬ ‫االنضباط‬ ‫هي‬ ‫األخالق‬
.
‫أخال‬ ‫القضايا‬ ‫تصبح‬
‫يتعلق‬ ‫عندما‬ ‫قية‬
‫والمبدئي‬ ‫األخالقي‬ ‫العمل‬ ‫مسار‬ ‫يكون‬ ‫قد‬ ‫ما‬ ‫حول‬ ‫معضالت‬ ‫تنشأ‬ ‫عندما‬ ‫أو‬ ‫األساسية‬ ‫اإلنسان‬ ‫بحقوق‬ ‫األمر‬
.

Levels of IRB Review
Type of review Definition-criteria Process
Full (review by all
IRB members)
Involves > minimal risk, knowing the
identity of participants, and whether
data are sensitive or may put the
participant at risk
Requires completing a full IRB
application; researcher must provide
copies of all materials (e.g.,
questionnaires, consents, recruitment
flyers, etc.) that will be used in the
research
Expedited
(reviewed by 2 IRB
members)
Involves < minimal risk and may
involve knowing the identity of
participants
Requires completing a full IRB
application; researcher must provide
copies of all materials that will be used
in the research
Exempt (from IRB
review)
Involves < minimal risk, the identity of
participants is not known or knowable,
routine educational or food tasting
research
A brief form that describes the research
is completed and submitted; usually
reviewed by < 1 member of the IRB

Eight Elements of Informed Consent
1. States that it is research and participation is voluntary
2. Explain Research purpose
3. Description of procedures (task length, type of data collected)
4. Possible risks or discomforts
•
‫طوعية‬ ‫والمشاركة‬ ‫البحث‬ ‫هي‬ ‫التي‬ ‫الدول‬
•
‫البحث‬ ‫الغرض‬ ‫شرح‬
•
‫اإلجراءات‬ ‫وصف‬
(
‫جمعها‬ ‫تم‬ ‫التي‬ ‫البيانات‬ ‫نوع‬ ، ‫المهمة‬ ‫طول‬
)
•
‫المضايقات‬ ‫أو‬ ‫المحتملة‬ ‫المخاطر‬

Eight Elements of Informed Consent, Continued
5. Explanation of direct benefits
6. Statement of confidentiality
7. Description of compensation for participation
8. Contact information about the study, research rights, and
assistance if there is a problem resulting from participating
•
‫المباشرة‬ ‫الفوائد‬ ‫شرح‬
•
‫السرية‬ ‫بيان‬
•
‫المشاركة‬ ‫عن‬ ‫التعويض‬ ‫وصف‬
•
‫المشاركة‬ ‫عن‬ ‫ناتجة‬ ‫مشكلة‬ ‫هناك‬ ‫كانت‬ ‫إذا‬ ‫والمساعدة‬ ‫البحث‬ ‫وحقوق‬ ‫الدراسة‬ ‫حول‬ ‫االتصال‬ ‫معلومات‬

Vulnerable Populations
• Children
– Usually requires parental consent and child assent
– Participation rates may be lower
• Other groups where ethics are a concern
– e.g., domestic violence prevention programs, program staff as study
participants.
Prisoners vulnerable” for the purposes of protecting human subjects.
•
‫األطفال‬
•
‫الطفل‬ ‫وموافقة‬ ‫الوالدين‬ ‫موافقة‬ ‫يتطلب‬ ‫ما‬ ‫عادة‬
•
‫أقل‬ ‫المشاركة‬ ‫معدالت‬ ‫تكون‬ ‫قد‬
•
‫قلق‬ ‫مصدر‬ ‫هي‬ ‫األخالق‬ ‫حيث‬ ‫أخرى‬ ‫مجموعات‬
•
‫الدراسة‬ ‫في‬ ‫كمشاركين‬ ‫البرنامج‬ ‫وموظفو‬ ، ‫المنزلي‬ ‫العنف‬ ‫منع‬ ‫برامج‬ ، ‫المثال‬ ‫سبيل‬ ‫على‬
.
•
‫عرضة‬ ‫السجناء‬
"
‫البشر‬ ‫حماية‬ ‫ألغراض‬
.

HIPAA and Evaluations
• The purpose of HIPAA is to protect personal information
• Related to having received health care. Personal information includes the individual’s birth
date, any element of an address, dates on which services were received, and diagnoses.
• Health Insurance Portability and Accountability Act
– Provisions to protect personal health information
• Implications may differ by whether:
– Organization falls under HIPAA requirements
– Evaluator is within or outside the organization
– The evaluation is part of routine care
– Results will be made public
•
‫من‬ ‫الغرض‬
HIPAA
‫الشخصية‬ ‫المعلومات‬ ‫حماية‬ ‫هو‬
•
‫الصحية‬ ‫الرعاية‬ ‫على‬ ‫بالحصول‬ ‫تتعلق‬
.
‫الشخص‬ ‫المعلومات‬ ‫تتضمن‬
‫تاريخ‬ ‫ية‬
‫والتشخ‬ ‫الخدمات‬ ‫تلقي‬ ‫وتواريخ‬ ‫العنوان‬ ‫في‬ ‫عنصر‬ ‫وأي‬ ‫الفرد‬ ‫ميالد‬
‫يصات‬
.
•
‫والمساءلة‬ ‫الصحي‬ ‫التأمين‬ ‫قابلية‬ ‫قانون‬
•
‫الشخصية‬ ‫الصحية‬ ‫المعلومات‬ ‫لحماية‬ ‫أحكام‬
•
‫كان‬ ‫إذا‬ ‫عما‬ ‫تختلف‬ ‫قد‬ ‫اآلثار‬
:
•
‫متطلبات‬ ‫تحت‬ ‫المؤسسة‬ ‫تقع‬
HIPAA
•
‫المنظمة‬ ‫خارج‬ ‫أو‬ ‫داخل‬ ‫مقيم‬
•
‫الروتينية‬ ‫الرعاية‬ ‫من‬ ‫جزء‬ ‫التقييم‬
•
‫النتائج‬ ‫عن‬ ‫اإلعالن‬ ‫سيتم‬

Dissemination Considerations
• Making a persuasive argument based on the data
• Report format
• Possible misuse of the evaluation findings
•
‫البيانات‬ ‫على‬ ً‫ء‬‫بنا‬ ‫مقنعة‬ ‫حجة‬ ‫عمل‬
•
‫التقرير‬ ‫شكل‬
•
‫التقييم‬ ‫لنتائج‬ ‫المحتمل‬ ‫االستخدام‬ ‫سوء‬

Properties of Persuasiveness
• Magnitude
• Articulateness
• Generalizability
• Interest level
• Credibility
•
‫الحجم‬
•
‫التحديد‬
•
‫التعميم‬
•
‫االهتمام‬ ‫مستوى‬
•
‫مصداقية‬

Effects of Rigor and Importance of Claims on Decision Making
Strength of Claim
Quality of Claim
Major Importance, High
Interesting
Minor Importance, Low
Interesting
High Rigor and Effect Ideal for making decisions Becomes “factoid”
Low Rigor and Effect Cause for concern, need to study
further, tentativeness to decisions
Ignored, forgotten, unspoken
Source: Adapted from Patton, M. Q. (1997). Utilization focused evaluation (3rd ed.). Thousand Oaks, CA: Sage
Publications.

Factors Affecting Sense Making
• Heuristics and hindsight bias
• Cultural values, experiences, and ways of thinking
•
‫األوان‬ ‫فوات‬ ‫بعد‬ ‫والتحيز‬ ‫االستدالل‬
•
‫التفكير‬ ‫وطرق‬ ‫والخبرات‬ ‫الثقافية‬ ‫القيم‬

Report Contents
1. Executive summary
2. Program and evaluation background
3. Program and evaluation description
4. Summary findings
– Match to process and outcome objectives
– Use graphs to convey key points
5. Recommendations
•
‫تنفيذي‬ ‫ملخص‬
•
‫والتقييم‬ ‫البرنامج‬ ‫خلفية‬
•
‫والتقييم‬ ‫البرنامج‬ ‫وصف‬
•
‫النتائج‬ ‫ملخص‬
•
‫والنتائج‬ ‫العملية‬ ‫أهداف‬ ‫تطابق‬
•
‫الرئيسية‬ ‫النقاط‬ ‫لنقل‬ ‫البيانية‬ ‫الرسوم‬ ‫استخدام‬
•
‫توصيات‬

Making Graphs Interpretable
• Limit graphs to key points and variables
• Only use as many dimensions in the graph as there are in the data
• Don’t let labels interfere with seeing the data
• Choose ranges on axes that are reasonable
• Use category names for nominal or ordinal data, not numeric values
• Use line graphs only for trend data
• Titles must be sufficiently explanatory
• Choosing the right variables and scale is more important than the color scheme
•
‫والمتغيرات‬ ‫الرئيسية‬ ‫النقاط‬ ‫على‬ ‫البيانية‬ ‫الرسوم‬ ‫قصر‬
•
‫البيانات‬ ‫في‬ ‫الحال‬ ‫هو‬ ‫كما‬ ‫البياني‬ ‫الرسم‬ ‫في‬ ‫األبعاد‬ ‫من‬ ‫ممكن‬ ‫عدد‬ ‫أكبر‬ ‫فقط‬ ‫استخدم‬
•
‫البيانات‬ ‫رؤية‬ ‫مع‬ ‫تتداخل‬ ‫العالمات‬ ‫تدع‬ ‫ال‬
•
‫معقولة‬ ‫محاور‬ ‫على‬ ‫نطاقات‬ ‫اختر‬
•
‫الرقمية‬ ‫القيم‬ ‫وليس‬ ، ‫الترتيبية‬ ‫أو‬ ‫االسمية‬ ‫للبيانات‬ ‫الفئات‬ ‫أسماء‬ ‫استخدم‬
•
‫االتجاه‬ ‫لبيانات‬ ‫فقط‬ ‫الخطية‬ ‫البيانية‬ ‫الرسوم‬ ‫استخدم‬
•
‫الكفاية‬ ‫فيه‬ ‫بما‬ ‫توضيحية‬ ‫العناوين‬ ‫تكون‬ ‫أن‬ ‫يجب‬
•
‫األلوان‬ ‫نظام‬ ‫من‬ ‫أهمية‬ ‫أكثر‬ ‫الصحيح‬ ‫والقياس‬ ‫المتغيرات‬ ‫اختيار‬ ‫يعد‬

Making Recommendations
• Base in data
• Link to organizational and services utilization plans
• Link to program theory
• Collect ideas throughout planning, implementation, and evaluation
• Draw on outside sources as justification
• Link to larger social, political, or organizational contexts
• Make them easy to understand
•
‫البيانات‬ ‫في‬ ‫قاعدة‬
•
‫التنظيمية‬ ‫الخدمات‬ ‫استخدام‬ ‫لخطط‬ ‫رابط‬
•
‫البرنامج‬ ‫لنظرية‬ ‫رابط‬
•
‫والتقييم‬ ‫والتنفيذ‬ ‫التخطيط‬ ‫خالل‬ ‫من‬ ‫األفكار‬ ‫جمع‬
•
‫كمبرر‬ ‫الخارجية‬ ‫بالمصادر‬ ‫استعن‬
•
‫أكبر‬ ‫تنظيمية‬ ‫أو‬ ‫سياسية‬ ‫أو‬ ‫اجتماعية‬ ‫سياقات‬ ‫إلى‬ ‫رابط‬
•
‫الفهم‬ ‫سهلة‬ ‫اجعلها‬

Misuse of Evaluations
• Manipulation to distort findings or compromise integrity
• May occur at any point in the evaluation process
• Biases may be driven by resource constraints, self-interest, political pressure,
etc.
• May be intentional or subconscious
• Minimize misuse by educating all stakeholders about the evaluation and
appropriate use
•
‫بالنزاهة‬ ‫المساس‬ ‫أو‬ ‫النتائج‬ ‫لتشويه‬ ‫التالعب‬
•
‫التقييم‬ ‫عملية‬ ‫في‬ ‫وقت‬ ‫أي‬ ‫في‬ ‫يحدث‬ ‫قد‬
•
‫إلخ‬ ، ‫السياسي‬ ‫والضغط‬ ، ‫الشخصية‬ ‫والمصالح‬ ، ‫الموارد‬ ‫قيود‬ ‫التحيزات‬ ‫وراء‬ ‫الدافع‬ ‫يكون‬ ‫قد‬
.
•
‫الالوعي‬ ‫أو‬ ‫متعمدا‬ ‫يكون‬ ‫قد‬
•
‫المناسب‬ ‫واالستخدام‬ ‫التقييم‬ ‫حول‬ ‫المصلحة‬ ‫أصحاب‬ ‫جميع‬ ‫تثقيف‬ ‫خالل‬ ‫من‬ ‫االستخدام‬ ‫سوء‬ ‫من‬ ‫قلل‬

Broader Dissemination
• Report made available beyond stakeholders
• Evaluator website
• Academic journals
• Potential conflict of interest if published evaluation was conducted by
program developer
• May need to negotiate with stakeholders up front
– Authorship, publication outlet, manuscript content •
‫المصلحة‬ ‫أصحاب‬ ‫بعد‬ ‫متاح‬ ‫التقرير‬
•
‫الموقع‬ ‫مقيم‬
•
‫األكاديمية‬ ‫المجالت‬
•
‫البرنامج‬ ‫مطور‬ ‫بواسطة‬ ‫المنشور‬ ‫التقييم‬ ‫إجراء‬ ‫تم‬ ‫إذا‬ ‫المصالح‬ ‫في‬ ‫محتمل‬ ‫تضارب‬
•
‫مقدما‬ ‫المصلحة‬ ‫أصحاب‬ ‫مع‬ ‫التفاوض‬ ‫إلى‬ ‫تحتاج‬ ‫قد‬
•
‫المخطوطة‬ ‫محتوى‬ ، ‫النشر‬ ‫منفذ‬ ، ‫التأليف‬

Organization-Evaluator Relationship
• Potential third-party dynamics among organization, funder, and evaluator
• Different prioritization of the program
– Differing timelines, expectations, levels of commitment
• Different professional and organizational cultures; different vocabulary
•
‫ِّم‬‫والمقي‬ ‫والممول‬ ‫التنظيم‬ ‫بين‬ ‫المحتملة‬ ‫الثالث‬ ‫الطرف‬
•
‫للبرنامج‬ ‫المختلفة‬ ‫األولويات‬ ‫تحديد‬
•
‫االلتزام‬ ‫ومستويات‬ ‫والتوقعات‬ ‫الزمنية‬ ‫الجداول‬ ‫اختالف‬
•
‫مختلفة‬ ‫مفردات‬ ‫؛‬ ‫المختلفة‬ ‫والتنظيمية‬ ‫المهنية‬ ‫الثقافات‬

Organization-Evaluator Relationship,
Continued
• Broader purpose of the evaluation
– Evaluation for management vs. leadership
• Blurriness between roles as evaluator vs. organizational development
consultant
• Negotiate early on who owns the evaluation data and how it will be
disseminated
•
‫للتقييم‬ ‫األوسع‬ ‫الغرض‬
•
‫القيادة‬ ‫مقابل‬ ‫اإلدارة‬ ‫تقييم‬
•
‫التنظيمي‬ ‫التطوير‬ ‫مستشار‬ ‫مقابل‬ ‫كمقيم‬ ‫األدوار‬ ‫بين‬ ‫التشويش‬
•
‫نشرها‬ ‫سيتم‬ ‫وكيف‬ ‫التقييم‬ ‫بيانات‬ ‫يملك‬ ‫من‬ ‫حول‬ ‫مبكر‬ ‫وقت‬ ‫في‬ ‫التفاوض‬

Evaluation and Health Policy
• When stakeholders want to use the findings to change policy
– Strategies:
• Communication networks involving legislators
• Communicate findings that are understandable and actionable
• When evaluation is contracted for a government agency of a government-funded program
– Political pressure may be intense
•
‫السياسة‬ ‫لتغيير‬ ‫النتائج‬ ‫استخدام‬ ‫المصلحة‬ ‫أصحاب‬ ‫يريد‬ ‫عندما‬
•
‫االستراتيجيات‬
:
•
‫المشرعين‬ ‫تشمل‬ ‫التي‬ ‫االتصال‬ ‫شبكات‬
•
‫للتنفيذ‬ ‫وقابلة‬ ‫مفهومة‬ ‫هي‬ ‫التي‬ ‫النتائج‬ ‫التواصل‬
•
‫الحكومة‬ ‫من‬ ‫ممول‬ ‫لبرنامج‬ ‫حكومية‬ ‫وكالة‬ ‫مع‬ ‫التعاقد‬ ‫يتم‬ ‫عندما‬
•
‫ًا‬‫د‬‫شدي‬ ‫السياسي‬ ‫الضغط‬ ‫يكون‬ ‫قد‬

Assessing Evaluation Quality
• AEA(American Evaluation Association) Program Evaluation Standards:
– Utility, feasibility, accuracy, propriety
• Confer with users of the evaluation to determine its usefulness to them
• Scientific rigor
• Evaluator characteristics
– Maintain independence, disclose conflicts of interest, maintain information
confidentiality, keep records of the evaluation process
•
‫برنامج‬ ‫تقييم‬ ‫معايير‬
AEA
(
‫األمريكية‬ ‫التقييم‬ ‫جمعية‬
:)
•
‫المالءمة‬ ، ‫الدقة‬ ، ‫الجدوى‬ ، ‫فائدة‬
•
‫لهم‬ ‫فائدتهم‬ ‫مدى‬ ‫لتحديد‬ ‫التقييم‬ ‫مستخدمي‬ ‫مع‬ ‫التشاور‬
•
‫العلمية‬ ‫الصرامة‬
•
‫المقيم‬ ‫خصائص‬
•
‫التقييم‬ ‫لعملية‬ ‫بسجالت‬ ‫واالحتفاظ‬ ، ‫المعلومات‬ ‫سرية‬ ‫على‬ ‫والحفاظ‬ ، ‫المصالح‬ ‫تضارب‬ ‫عن‬ ‫والكشف‬ ، ‫االستقالل‬ ‫على‬ ‫الحفاظ‬

Keep Up with Current Practice
• RE-AIM and other evaluation models
– Organize flowchart of activities, compare 2 programs on summary indices
– Appropriate for individual behavior change programs
• Meta-analysis
– Synthesis of findings across evaluations to generate summative statistics
– Include a quality rating
•
RE-AIM
‫األخرى‬ ‫التقييم‬ ‫ونماذج‬
•
‫الملخص‬ ‫مؤشرات‬ ‫على‬ ‫برنامجين‬ ‫ومقارنة‬ ، ‫لألنشطة‬ ‫انسيابي‬ ‫مخطط‬ ‫تنظيم‬
•
‫الفردي‬ ‫السلوك‬ ‫تغيير‬ ‫لبرامج‬ ‫مناسبة‬
•
‫البعدي‬ ‫التحليل‬
•
‫موجزة‬ ‫إحصائيات‬ ‫لتوليد‬ ‫التقييمات‬ ‫عبر‬ ‫للنتائج‬ ‫توليف‬
•
‫الجودة‬ ‫تصنيف‬ ‫تشمل‬

Evaluator Responsibilities Across the
Pyramid
– Informed consent of individuals is easiest
– Meta-analysis and evidence-based reviews of these programs because of their
popularity
– Informed consent is usually feasible
– Group sense making may differ from individual sense making
•
•
‫أسهل‬ ‫هو‬ ‫األفراد‬ ‫من‬ ‫مستنيرة‬ ‫موافقة‬
•
‫شعبيتها‬ ‫بسبب‬ ‫البرامج‬ ‫لهذه‬ ‫األدلة‬ ‫على‬ ‫القائمة‬ ‫والمراجعات‬ ‫التلوي‬ ‫التحليل‬
•
•
‫ممكنا‬ ‫يكون‬ ‫ما‬ ‫عادة‬ ‫مستنيرة‬ ‫موافقة‬
•
‫الفردي‬ ‫اإلحساس‬ ‫صنع‬ ‫عن‬ ‫يختلف‬ ‫قد‬ ‫الجماعي‬ ‫اإلحساس‬ ‫صنع‬

Evaluator Responsibilities Across the Pyramid, Continued
– Informed consent becomes blurred with the implementation of health policy, where
consent is implied
– Meta-evaluations of health policy studies
– Collective processes by which organizations and policy makers achieve a shared
understanding of evaluation findings
– May lead to program implementation or policy adoption
– Procedures for ethical issues including IRB and HIPAA come from this level
•
•
‫ضمنية‬ ‫الموافقة‬ ‫تكون‬ ‫حيث‬ ، ‫الصحية‬ ‫السياسة‬ ‫تنفيذ‬ ‫عند‬ ‫واضحة‬ ‫غير‬ ‫المستنيرة‬ ‫الموافقة‬ ‫تصبح‬
•
‫الصحية‬ ‫السياسة‬ ‫لدراسات‬ ‫التلوية‬ ‫التقييمات‬
•
•
‫التقييم‬ ‫لنتائج‬ ‫ا‬ً‫ك‬‫مشتر‬ ‫ا‬ً‫م‬‫فه‬ ‫السياسات‬ ‫وصانعي‬ ‫المنظمات‬ ‫تحقق‬ ‫خاللها‬ ‫من‬ ‫التي‬ ‫الجماعية‬ ‫العمليات‬
•
‫السياسة‬ ‫اعتماد‬ ‫أو‬ ‫البرنامج‬ ‫تنفيذ‬ ‫إلى‬ ‫يؤدي‬ ‫قد‬
•
‫ذلك‬ ‫في‬ ‫بما‬ ‫األخالقية‬ ‫بالقضايا‬ ‫الخاصة‬ ‫اإلجراءات‬ ‫تأتي‬
IRB
‫و‬
HIPAA
‫المستوى‬ ‫هذا‬ ‫من‬

Planning Slides.pdf (1).pdf

More Related Content

Similar to Planning Slides.pdf (1).pdf (20)

Recently uploaded (20)

Planning Slides.pdf (1).pdf