APCICT M&E Framework Initiative (Part 1)

APCICT M&E Framework Initiative
Incheon 3rd
November 2010
Presented by Prof John Ure
Director, Telecommunications Research Project,
University of Hong Kong
Director, TRPC Pte Ltd, Singapore
http//www.trpc.com.hk
Title slide

Introduction to Part 1
Section Divider

Why M&E?
 Short term: test for …
 course improvements (delivery, organization, etc)
 course relevance of content to stakeholders
(sponsors, clients, participants, others?)
 Long term: sustainability
 Results/course outcomes (practical measures)
 Returns on investment or expectations (value for
money for the sponsor or client)
3

APCICT Business Strategy and Plan
4
 APCICT’s medium-long term Strategy and Plan
 Expand and deepen Academy programs at national and
sub-regional levels
 Reach out towards FHE institutions to promote ICT
learning for development
 A harmonized or common standard M&E Framework would
help achieve
 Quality of training programs
 Relevance of training programs to the needs of
stakeholders
 A bridge to the development community

M&E in Recent Times
 Traditionally FHE evaluated the students not the courses
 Peer review of courses is now commonplace
 Development agencies spend trillions of dollars a year
globally
 They want accountability
 Evidence of value for money
 Evidence of results
 Many different methodologies have sprung up to cater
for different project and course types
 Some highly quantitative, others totally qualitative
5

As M&E strives to achieve more, it costs more
6

Stakeholders
7
Different stakeholders have different interests. It is important to (a)
identify the principal stakeholders and gear the M&E process towards
them, and (b) to engage with them as closely as possible to so they
share the “ownership” and therefore commitment to the course or
program

What is M&E?
 Many definitions – this one is straightforward
“The monitoring process looks at what is being done
and how it is being done… Evaluation looks at
performance against goals”
(Daniel Wagner, InfoDev 2005)
 My definition (quite similar)
 Monitoring is how well the learning process is going
(gives rise to “happiness”)
 Evaluation is how relevant that learning process is to
the stakeholders (gives rise to sustainability)
8

Who is Using M&E and How?
Section Divider

APCICT Country Profiles
10
 APCICT’s has programs in 17countries – of the
7 country profiles currently available
 2 countries offer modules 1-3
 5 offered all modules
 All used the standard end-of-course evaluation form
of APCICT… but questions worth asking:
• In principle allows for cross country comparisons,
but in practice how meaningful are the
comparisons? Quantitatively? Qualitatively?
• Is it always suitable for local situations?

Local lessons from country reports from Mongolia,
Myanmar, Pakistan, Philippines and Uzbekistan
14

15

16

17

Frameworks, Methodologies
and Assessment Tools
Section Divider

M&E Frameworks, Methodologies
and Assessment Tools
19
An M&E
Framework can
operate with
different
methodologies
and different
assessment
tools

The (Revised) Kirkpatrick Model
21

Methods: Examples in Practice
22

Tools: Examples in Practice
23

Conclusion
 Current APCICT M&E practice is restricted to end-of-
course M&E questionnaire
 Is this sufficient?
 Does it reach out towards the development community and its
use of more thorough M&E frameworks and methodologies?
 M&E for APCICT’s Virtual Academy (AVA) is online and
voluntary, but responses are few (4-5 replies)
 There is a need for a practical, flexible-in-use M&E
framework for APCICT and its partner academies
 Needs to be doable in terms of resources
 Needs to be meaningful in terms of comparative data/lessons
 Needs to be sustainable, to attract sponsor/client buy-in (PME)
25

Building the Leaders of
Tomorrow!
Section Divider

APCICT M&E Framework Initiative (Part 1)

More Related Content

What's hot (20)

Similar to APCICT M&E Framework Initiative (Part 1) (20)

More from TRPC Pte Ltd (18)

Recently uploaded (20)

APCICT M&E Framework Initiative (Part 1)