SlideShare a Scribd company logo
Working Paper # 172
Measuring Institutional
Performance in Pakistan: Can
governance indices help?
By:
Shehryar Khan Toru, Syed Mohsin Ali, Vaqar Ahmed
All rights reserved.
No part of this paper may be reproduced or transmitted in any form or by any means, electronic or
mechanical, including photocopying, recording or information storage and retrieval system, without
prior written permission of the publisher.
A publication of the Sustainable Development Policy Institute (SDPI).
The opinions expressed in the papers are solely those of the authors, and publishing them does not in
any way constitute an endorsement of the opinion by the SDPI.
Sustainable Development Policy Institute is an independent, non-profit research institute on
sustainable development.
First edition: July 2019
© 2019 by the Sustainable Development Policy Institute
Mailing Address:
PO Box 2342, Islamabad, Pakistan
Telephone: 0092-51-2278134, 2278136, 2277146, 2270674-76
Fax: 0092-51-2278135,
URL: www.sdpi.org
Acknowledgement
We acknowledge the data related support provided by Ahmad Durrani, Maryam Waqar, and Syeda
Rubab. We are grateful for support and cooperation by United Nations Development Program (UNDP),
Pakistan Office, NADRA, CDA and NHMP. Ashfaq H. Khan, Nohman Ishtiaq and Shahid Sattar
provided experts’ input on the methodology. The team is grateful for their support and services. The
usual disclaimer applies.
Abstract
In Pakistan, several efforts have been made to measure the government performance at national and
sub-national levels. Our efforts build on an existing framework for measuring service delivery. Though
the previous tool, which has been applied to study the select federal government ministries includes
supply side governance and accountability indicators, there is, however, a need to also incorporate
demand side indicators to capture community satisfaction from publicly provided services. In this study,
an effort has been made to formulate an augmented index and by using survey data, application of this
index on three public sector entities has been analyzed. Results show that this approach has the benefit
of validating supply and demand side information and perceptions.
Keywords: Governance, BGI, Public sector institutions,
Table of Contents
1. Introduction.....................................................................................................................................1
2. Literature Review............................................................................................................................2
3. Methodology...................................................................................................................................6
4. Construction of Performance Index ................................................................................................8
4.1. Performance Indicators for Demand Side...............................................................................8
4.2. Performance Indicators for Supply Side.................................................................................8
5. Findings-I: Citizen Feedback survey ............................................................................................10
6. Findings-II: Results from augmented performance index ............................................................12
7. Conclusion ....................................................................................................................................14
References.............................................................................................................................................15
Annexure-I: (Detailed computation of index).......................................................................................16
a. Construction of demand side governance index............................................................................16
b. Construction of supply side index.................................................................................................22
1
1. Introduction
The study aims to measure the performance of a select number of public sector institutions using both
supply and demand side indicators. Our starting point is Better Governance Index (BGI) framework
developed by Moriani and Rehman1
. We supplement the effort from 2015 using a demand side client
satisfaction survey to be explained later. The study, on the one hand, provides a conceptual tool to
merge internal and external outcome measures, while on the other it goes on to apply this tool on a
select number of select public sector entities, including National Database & Registration Authority
(NADRA), National Highways and Motorway Police (NHMP), and Capital Development Authority
(CDA). These institutions have been selected on the basis of their public interface as well as the high
frequency with which their services are demanded by the society.
At the outset, the performance measurement framework adopted in BGI2
sets the basis of engaging
different frameworks for measuring institutional performance in public sector institutions. The
importance of inputs, outputs and strategic goals are mentioned to illustrate -institutional efficiency in
more general terms. The terms and process indicators of BGI are explained to gain familiarity with
types of performance frameworks. The rationale behind gauging performance is to benchmark
performance indicators on a set of pre-selected indicators, namely quality of governance, accountability,
inclusion and openness, effectiveness and responsiveness.
Measurements of institutional efficiency generally focus on the relationship between institutional
resources and activities on the one hand, and institutional outcomes, on the other (Cristescu et al. 2010;
Castro 2011; Gokey et al. 2013; Khattri and Roberts, 2012. Systems of measuring performance
efficiency have indicators functioning both at the individual level such as the qualitative service delivery
survey (Bold et al. 2011; Dehn et al. 2003), and the institutional level such as the results framework
(Trivedi 2014).
Measurements of institutional effectiveness focus on the relationship between the intended and actual
outcomes and impacts, or the ratio between outputs and outcomes (Cristescu et al. 2010). However,
while such calculations are often straightforward in the private sector, they are harder to make in the
public sector due to the difficulty of conclusively establishing causal link between the level of activities
and outcomes (Davies 1999).
Owing to the considerable difficulties in measuring institutional efficiency and effectiveness within the
public sector, evaluation experts have developed another set of indicators that measure the nature of
policy implementation such as whether it supports accountability and citizens participation. These are
known as process indicators (Landman 2003). The Weberian Comparative State Data project is a
comparative assessment framework that includes a number of process indicators with its focus on
measuring various dimensions of bureaucratic structure and institutional meritocracy.
With its twofold objective, the study would first augment BGI framework for Pakistan with inclusion
of citizen satisfaction indicators, and then make an effort to apply the augmented index to gauge the
performance of select public sector entities. to the study also suggests as to how such performance index
can be institutionalized by the government institutions concerned.
1
Farrukh Moriani and Hira Hafeez ur Rehman (2015) produced a report in 2015 on “Better Governance Index: a conceptual
framework for measuring institutional performance”. This report uses a conceptual framework having process indicators
(………) for measuring institutional performance of select institutions.
2 Adopted from Better Governance Index report authored by (Moriani & Rehman, 2015).
2
2. Literature Review
This section provides a brief literature review with case studies from the select countries. We also
provide current gaps in literature and how some of these are addressed. This is followed by findings
and conclusion section.
How countries measure institutional performance?
A review of literature was carried out to understand governance indices used by other studies, and
countries in South Asia and elsewhere. Existing literature on institutional performance and their utility
in different country contexts are cited as useful performance frameworks. The intention behind this
effort is to strengthen the BGI framework. Efforts have been made to see in-depth at least three case
studies with respect to objectives, methodology, rollout and sustainability of performance measurement
frameworks (see Table 1).
The criteria used for selecting country examples consists of i) governance context and relevance (to
Pakistan) which captures accountability and other attributes (similar to local milieu); ii) indicators
description, i.e. process and micro level indicators (similar to processes seen in Pakistan’s public
sector); and iii) sustainability and institutionalization. A number of developed and developing countries
have designed and implemented performance measurement systems within their domains and varying
contexts. This section includes a brief review of such systems, which are: i) applicable to agency-level
public sector performance; ii) broad enough to facilitate inter-agency comparison of performance; based
on both subjective and objective means of data collection.3
Table 1: Select Examples of Performance Measurement Frameworks
Framework
Details
Results Framework
Document [RFD]
(Khattri and
Roberts, 2007;
Trivedi, 2014)
Rwanda Joint Governance
Assessment [JGA]
(Kwakwa and Musoni,
2008; Rwanda Governance
Advisory Council, 2011)
Open
Government
Directive [OGD]
(Bertot et al.
2012)
Governance
Report 2013/2014
(Lodge and
Wegrich, 2013)
Description Used in India, this
tool compares the
performance of
individual
departments against
standardized
indicators,
objective targets
and measurement
criteria (Khattri and
Roberts, 2012)
A country-specific tool
developed with the aim of
creating a common
understanding of
governance issues and
measurement frameworks
in Rwanda, and identifying
future areas for
performance improvement
and priority action.
Measurement areas
include: Ruling Justly,
Investment Climate, and
Government Effectiveness.
Majority of indicators
geared towards country-
level governance
performance.
Rolled-out in
USA to measure
the success of
transparency and
open government
initiatives within
different public-
sector agencies.
Assessments
were undertaken
in three key
areas:
Transparency,
Participation, and
Collaboration.
A highly
adaptable
assessment
framework that
functions at the
national, sub-
national and
agency levels of
governance.
Measures
government’s
administrative
capacity
(resilience and
preparedness) to
address
contemporary
3
There may also be other measurement systems, e.g. see Hertie School of Governance Indicators:
http://www.governancereport.org/home/governance-indicators/2016-governance-indicators/
3
challenges,
including the
identification of
problems, the use
of tools to address
problems, and
types of responses
generated towards
problems.
Assessments are
carried out on the
basis of aggregate
subjective
indicators
measuring
capacity for
Delivery,
Regulation,
Coordination and
Analytical
capacities.
Objectives a. Planning
strategies and
selecting
appropriate
interventions to
address
challenges in
question
b. Building
consensus and
ownership
between
different
stakeholders of
a policy
intervention
c. Providing
objective
performance
data to inform
programme
decisions
d. Improved intra
and inter-
agency
communication
and reporting
e. Performance
evaluation
Harmonization and
rationalization of
procedures
a. Consolidating peace
and security in the
country
b. Promoting inclusive
governance
c. Strengthening rule of
law and accountability
d. Linking government
operations and
activities to broader
development goals
Assessing the
impact of
transparency and
open government
initiatives within
government
agencies.
Developing an
understanding of
how
administrative
capacity of
governments can
be improved in
terms of
understanding and
responding to
emerging
challenges in
better and
innovative ways
Methodology
& roll-out
i. Consultation
between
Ministers and
line
i. Consultations with
diverse base of
government and non-
government
i. Experts from
government
and non-
government
i. Governance
and public
administration
and
4
departments on
departmental
objectives,
proposed
activities to
achieve
objectives, and
corresponding
success
indicators
(micro, macro
and process)
and time-bound
targets
ii. Devising
weighted
scores for each
success
indicator and
target
iii. Defining level
of coordination
required with
outside
departments for
achievement of
objectives
iv. Critique and
approval by
Cabinet
Secretariat
v. Review by
centralized
Performance
Management
Division and
non-
government
sector experts
vi. Review by
Cabinet
Committee on
Government
Performance
Final RFD placed
on departmental
website
stakeholders to design
draft indicators
framework
ii. Management structure
established, including
Steering Committee to
oversee roll-out and
indicators
development,
Technical Committee
to advise on
methodological and
analytical issues, and
Consultant team
reporting to Steering
Committee on
implementation
iii. Regular meetings
between different tiers
in management
structure to discuss
progress, earlier drafts
and way forward
iv. Quality Control by
consultant team
during roll-out
v. Production of
evaluation reports by
Consultant Team
Training of Governance
Advisory Council (GAC)
staff for implementation.
GAC to function as
centralized secretariat for
JGA implementation after
pilot.
sectors, and
employees
from
agencies
being
assessed
brought
together in
Open
Government
Working
Group to
devise
assessment
criteria and
data
collection
methods and
tools
ii. Pre-testing
of
assessment
methods and
tools, and
incorporation
of feedback
iii. 3-point
ordinal scale
developed to
eliminate
problems
resulting
from poor
goal
description
or inter-rater
variations in
scoring
iv. Expert
evaluators
employed to
assess
agency
compliance
with OGD
requirements
v. Agency-level
self-
assessments
also
conducted
Performance data
displayed
publicly
management
experts design
indicators
framework
and selected
data sources
ii. Aggregation
of subjective
and objective
data to
generate
composite
scores in each
assessment
area
iii. Borda
scoring:
scores
assigned to
individual
institutions
and then
aggregated to
provide
country-level
scores
Scores
represented at
three different
levels: agency,
national and
transnational
Consultative
process
Consultation and
review at different
levels, with
multiple
Consultation and review at
different levels, with
multiple stakeholders, both
Consultation and
review at
different levels,
with multiple
Consultations
among experts of
Hertie School
5
stakeholders, both
government and
non-government
government and non-
government
stakeholders,
both government
and non-
government
Sustainability Consultations at
multiple levels of
government to
ensure stakeholder
buy-in and
ownership
Creation of
electronic Results-
Framework
Management
System, that aids in
RFD preparation
and monitoring on
monthly and annual
basis
Consultations at multiple
levels of government to
ensure stakeholder buy-in
and ownership
Centralized Secretariat for
monitoring of JGA
implementation
Consultations at
multiple levels of
government to
ensure
stakeholder buy-
in and
ownership.
Sustainability
hampered due to
lack of
centralized
control, and
resistance at
agency-level
against
assessments by
non-government
sector
Lack of
evaluation
criteria,
especially at the
level of impacts,
prevents
comparison of
effectiveness of
OGD
implementation
between different
agencies
No information
provided
Evidence of
impact
Covers 80
departments of
Government of
India and being
implemented in 18
Union territories
Improved
Grievance Redress
after mandatory
inclusion of
corresponding
indicators within
departmental RFDs
Improved social
indicators
Transparency in public
procurement
Increase in competitive
bidding for award of
government contracts
Decrease in corruption
Determination of new
salary schemes
Increase in number of
public sector reform
initiatives
Open Data 500
US studies the
variety of
different
companies and
innovators using
the data made
accessible by
different
government
agencies.
Positive
correlation among
different
capacities
identified
Relevance to
BGI
Allows
comparisons of
inter-agency
governance
performance
Aside from indicators
measuring Government
Effectiveness, JGA is
unsuited to measuring
agency-level governance
Allows for
comparison of
inter-agency
compliance with
Allows
comparisons of
inter-agency
governance
6
Objective means of
data collection
Includes process
indicators such as
effectiveness of
public grievance
redress mechanisms
Objective and subjective
means of data collection
Includes process indicators
OGD
requirements
Lack of
evaluation
criteria at level of
impacts prevents
comparisons of
agency
performance in
terms of
producing social
benefits
Includes process
indicators
Objective means
of data collection
Flexible enough
to allow same
capacity to be
measured through
both objective and
subjective means.
In the case of Pakistan and to our best knowledge, the BGI framework encompasses supply side
concepts that include quality of governance, internal accountability, inclusion and openness,
effectiveness (e.g. in the context of usage of financial resources), and responsiveness (e.g. in the context
of ability to respond to the expectations of citizens), however to make such an index truly
comprehensive indicators which reflect citizen satisfaction need to be incorporated. We make this effort
in the sections ahead.
3. Methodology
Methodology encompasses mixed methods’ approach where quantitative and qualitative research tools
are employed. After undertaking a literature review to learn about the utilization, roll-out and
institutionalization of performance measurement tools, we embarked upon the construction of index.
Our point of departure was to revise BGI through revisiting the criteria of governance index, sources of
data available on indicators, and suggest possibilities of future research on BGI framework, with a
particular focus on possible inclusion of qualitative indicators.
Data collected through different sources (such as official documents, websites, and secondary research)
was triangulated. An added feature of validating data pertains to the feedback of local experts on
governance and institutional performance. This particular activity entails the constitution of a feedback
mechanism, thus allowing critical reflection of public sector institutions on governance indicators.
While supply side indicators remain similar to BGI (and frameworks from other countries mentioned
in section 2), a survey was designed to gauge the perception of citizens on the functions of select
institutions. The survey included specific questions regarding experiences of citizens, process of
service, outcome, and grievance redressal processes. To validate findings, key informant interviews
were held with experts who have in the past either developed or used performance measurement
frameworks.
There are certain challenges in the way of implementation of this methodology. This also gives rise to
limitations of this study. First, since governance index involves performance assessment of public sector
entities, the project team faced informational and public disclosure challenges. The public officials were
sometimes reluctant to provide information on matters which fall in official domain. For example,
seeking information on employees might raise eyebrows. The institution may not like to share such
information. Similarly, some specific budget-related information at a disaggregated level is difficult to
obtain.
7
The public officials asked SDPI for completing unanticipated written approvals. This resulted in
unnecessary delay which may have changed the political economy context of our analysis. This
however should not affect the conceptual value of the tool developed in this study.
The following public-sector institutions are selected for applying the augmented performance index.
Highway and Motorway Police are mandated to ensure safe and citizen centric safety environment for
drivers. Their core functions are issuance and renewal of driving licenses, assurance of road safety,
enforcement of traffic rules, and assistance of driver in accidents and emergences.
National Database & Registration Authority (NADRA) is a federal institution having branches
across the country. It provides services related to: issuance of Computerized National Identity Cards;
Juvenile card4
; National Identity Card for Overseas Pakistanis; Child registration certificate;
Cancellation certificate5
; Family registration certificate6
; and Pakistan Origin Card7
.
Capital Development Authority (CDA) provides municipal services in Islamabad which include (but
are not limited to): residential and commercial property registration and transfer; refuse (waste)
collection; approval of construction of houses; collection of property tax; ensuring health and safety of
residents; approval for the construction of commercial buildings; imposition and collection of water
charge; development of new residential sectors; water supply connection; land accusation for new
schemes; and streetlighting.
The aforesaid institutions are selected on the basis of following criteria:
Access: One of the most influential criteria of acquiring information from government institutions is
access. The criterion of access is selected on the basis of good professional relations of SDPI with
selected institutions and on the basis of receptivity and ownership for the pilot study amongst the
institutions.
Public Interface and service provision: The selected institutions are public sector entities mandated
to provide public and collective services to clients. As a result of state officials’ interaction with citizens
on the diverse services, these institutions have a public interface considered essential to understand
governance perceptions and benchmarking institutional performance.
Regarding the citizen feedback survey, a quantitative research methodology was adopted to acquire
answers to multiple options of inquiry. In some instances, descriptive qualitative inquiry is also
included. A purposive sampling technique is adopted to ensure the selection of stakeholders from pre-
determined pool. This pool includes beneficiaries of the select public sector entities. One of the
advantages of such sampling approach is that it allows targeted questions specific only to the users of
public services. The overall sample size for the survey is 300, selected from the stratum of service users.
The team ensured at least 100 responses per institution. This minimum response rate can increase if a
single respondent also wishes to answer questions related to multiple public sector entities and services,
accessed during the past. All respondents are above 18 years of age and 50 per cent respondents for
each institution are women. The final gender mix of our respondents varies depending upon the refusal
rate.
4 “Juvenile card is an identity card issued to children under the age of 18 years”.
5
Cancellation certificate is a registration document issued to register the termination of ID card in case of death of any
citizen.
6
“Family Registration Certificate (FRC) is a mean of being identified with your NADRA’s record”.
7 “Pakistan Origin Card is issued to facilitate the eligible foreigners. This card is for those foreigners who have some roots in
Pakistan i.e., they are natives of Pakistan.
8
Prior to the pre-testing and implementation, enumerators were trained on how to use the survey
instrument. To ensure accuracy of data and its analysis, enumerators ensured the elimination of
inconsistencies and incorrect responses during the phase of data collection.
4. Construction of Performance Index
4.1. Performance Indicators for Demand Side
The criteria used for each measurement area are presented in Table 2. A total of 19 indicators are
proposed for eight criteria under five measurement areas. Since in the pilot phase of index is to be
developed for the selected three institutions, (e.g. NADRA, CDA, and NHMP), the indicators aim at
measuring performance at that level.
Table 2: Demand Side Indicators
Measurement
Area
Criteria (C) Indicators (I)
Quality of
Governance
No Corruption
No informal payments or gifts in lieu of accessing services
Transparency in service delivery
Accountability
Customer
Feedback
Availability of criteria for Customer Service Feedback
Inclusion and
Openness
Inclusion Not internally-deprived during accessing service
Openness
Complaint procedure is easy and accessible
No social discrimination
Effectiveness
Service
Delivery
Availability of services in the vicinity
Quality of service
Institution and its staff are trustworthy
Payment system is very convenient and transparent
Cost of services are affordable
Accuracy in the cost of service
Responsiveness
Mechanisms
for facilitating
access to
information
Designated official nominated under Right to Information Act 2010
Accessibility of source of information
Standards for access to information services
Mechanisms
for public
grievance
redress
Complaint mechanism is available for desired service
Acknowledgement of complaints by relevant department
Standards for grievance redress services
Use of ICT
tools
Availability of online services for complaints, information, registration,
social media
4.2. Performance Indicators for Supply Side
List of performance indicators given in table 3, for measuring supply side index are proposed on the
basis of:
i) classification of each indicator;
ii) ii) an evaluation of each indicator with respect to its relevance, coverage and cost-
effectiveness.
9
Table 3: Supply Side Indicators
Measurement
Area
Criteria Indicators
Indicator
Classification
Evaluation
(relevance,
coverage,
cost)
Quality of
Governance
Clarity of
institutional vision
and mission
Statements are clear, brief and
reflect the values, philosophy and
culture of the institution
Process
indicator
Relevant,
global, and
cost effective
Vision statement expresses in
sensory terms, the future position
or image of the institution
Process
indicator
Relevant,
global, and
cost-effective
Mission statement focuses on one
or a few strategic goals
Process
indicator
Relevant,
global, and
cost-effective
Budget Accuracy
Variance between revised budget
and budget estimates
Micro-level
Relevant,
global, and
cost-effective
Gender Equity in
HR
Proportion of female employees in
total working staff
Process
indicator
Relevant,
global, and
cost-effective
Investment in
Capacity Building
Training budget as a proportion of
total Personnel budget
Micro-level
Relevant,
global and
cost-effective
Political
Accountability
Regular submission of briefings to
the Standing Committees of
Parliament
Process
indicator
Relevant,
global and
cost-effective
Accountability
Accountability to
citizens
Publication of Annual Budget on
institution’s website
Process
indicator
Relevant,
global and
cost-effective
Publication of Yearbook
Process
indicator
Relevant,
global, and
cost-effective
Majority of tenders follows PPRA
rules
Process
indicator
Relevant,
global and
cost-effective
Use of social accountability tools
Process
indicator
Relevant
Citizens’ Charter issued
Process
indicator
Relevant and
cost-effective
Inclusion and
Openness
Participatory and
inclusive decision
making and
oversight
No. of forums on which citizens,
private sector, academia or civil
society are represented
Process
indicator
Relevant,
global, and
cost-effective
Proactive public
disclosure
Publication of updated policies,
strategies on website
Process
indicator
Relevant,
global, and
cost-effective
Availability of expenditure data
Process
indicator
Relevant,
global, and
cost-effective
Effectiveness Budget execution
Proportion of released annual
budget actually utilized
Micro-level
Relevant,
global, and
cost-effective
10
Implementation of
priority programs
Completion level of priority
programs
Micro-level
Relevant and
global
Responsiveness
Linkage with
National/Provincial
priorities and
strategies
Explicit linkage of priority
programs with the Vision 2025
Process
indicator
Relevant and
global
Explicit linkage of priority
programs with relevant sector
policy
Process
indicator
Relevant and
global
Mechanisms for
facilitating access
to information
Designated Official nominated
under Right to Information Act
Micro-level
Relevant,
global, and
cost-effective
Standards for access to
information services clearly
mentioned on website
Process
indicator
Relevant,
global, and
cost-effective
Mechanisms for
public grievance
redress
Designated Official for public
grievance redress and addressing
references from Ombudsmen
Micro-level
Relevant,
global, and
cost-effective
Standards for grievance redress
services clearly mentioned
Process
indicator
Relevant,
global, and
cost-effective
Use of ICT tools
Availability of online services for
complaints, information,
registration, social media
Micro-level
Relevant,
global, and
cost-effective
5. Findings-I: Citizen Feedback survey
The highest percentage of respondents had availed services related to NADRA. For example, 72% male
and 58% female respondents availed the services of acquiring Computerized National Identity Card
(CNIC). Furthermore, 10% males and 19% females acquired Child Registration Certificate. Only 3%
males and 2% females availed the service of cancellation certificate; whereas only 2% males and 3%
females availed family registration certificate. Getting Juvenile card was the least availed service as
only one female respondent opted for it. The reason for the higher percentage of service acquisition in
the case of CNIC is its requirement on daily basis; the Juvenile card is not in common usage as most of
the people are not aware of its utility. The quality of services provided by NADRA was also termed
very satisfactory.
In the case of CDA, the highest percentage of respondents benefitted from ‘Residential or Commercial
Property Registration and Transfer’, which accounts for 59% males and 48% females. ‘Approval of
construction of houses’ is the second most commonly availed service by 25% males and 19% females.
None of the male respondents availed facilities of ‘collection of property tax’, ‘Ensuring health and
safety of residents’ and ‘Approval of construction of commercial building’, only 4% (constitutes one
female) availed the aforementioned services. Similarly, none of the female respondents benefitted from
services of ‘imposition and collection of water charges’ and ‘refuse (waste) collection,’ and only 03%
(constitutes only 1 man) availed the aforementioned services. The service of ‘water supply
connection/reconnection’ was availed by 6% males and 7% females. Similarly, 03% males and 15%
females availed the service of land accusation for new schemes.
11
Upon inquiring about the quality of services provided by CDA, more than half of the respondents
indicated low priority for services (including Residential or commercial property registration and
transfer, Collection of property tax, Water supply connection/reconnection, Ensuring health and safety
of residents, Land accusation for new schemes, Approval of construction of houses, and Approval of
construction of commercial buildings) due to high rate of corruption and informal payments. Around
25% respondents ranked residential and commercial property registration and transfer, and Approval of
construction of houses somewhat lowest due to unspecified duration of time span. Few respondents
held a neutral stance, asserting keeping quite as their specific request for a service may be unnecessarily
delayed by CDA.
From the services being offered by National Highways and Motorway Police (NHMP), issuance of
driving license was availed by 63% males and 64% females. The issuance of international standard
driving licenses had been the least common among the respondents as they accounted for only 4%
female. On the other hand, 17% males and 32% females availed the service of renewal of driving license
and 20% respondents availed the service “acquittal from challan”.
Considering the quality of services, 16% respondents were of the view that the quality was somewhat
satisfactory. In the case of 6% respondents, the services of ‘Issuance of driving license’ and ‘Acquittal
from challan’ were least satisfactory. Whereas 44% beneficiaries indicated entire services of ‘highest’
rank. In the case of 14% respondents, issuances of driving license, renewal of driving license and
acquittal from challan were ranked less satisfactory.
In terms of lodging of a complaint, majority complaints were lodged against NHMP as they
accounted amongst 20% male and 48% females. The least number of complaints was lodged against
NADRA, i.e. 4% amongst female and 28% amongst male respondents. In the case of CDA, 18%
respondents lodged complaints. The reason for lodging least complaints against NADRA is the
effective, timely and transparent provision of services.
Around 88% NHMP service users were satisfied with their complaints as they were addressed within a
reasonable time. Similar response concerning timely redressal of complaints was observed in the case
of NADRA. A glaring observation relates to CDA’s non-redressal of complaints in more than 70% male
and 60% female respondents.
When probed for not lodging a compliant, 19% consumers in the case of CDA were not aware of the
complaint procedure; whereas 5% consumers made reference to a complicated procedure for lodging
complaint. Similarly, only 3% respondents found complaint procedure of NHMP complex. In the case
of CDA, 36% consumers found the complaint procedure ineffective, which constitutes the highest
proportion of complaints when compared with 19% NHMP and 2% NADRA. In the case of CDA, 19%
service users lodged complaints because of the fear of harassment.
Concerning the source of information, 74% respondents learned about NHMP through the word of
mouth, followed by 62% in the case of NADRA and CDA. Upon inquiring about accessibility of
information, 98% respondents in the case of NADRA were able to access information. On the other
hand, 30% respondents in the case of CDA opted for 'somewhat accessible' option. The following
reasons were cited for not having comprehensive access to information
 Physical access to information faced by 94% respondents in the case of CDA.
 Non-cooperation of NHMP’s staff faced by 26% respondents.
12
 88% respondents do not have access to the offices of NADRA in their vicinity, so they often have
to travel a long distance to access the nearest office. Similar response was recorded in the case of
CDA. In the case of NHMP, 46% users opined that they have no concerning offices in their vicinity.
About half of the respondents travel between 30-60 minutes to reach NADRA, 30% users travel
between 1-2 hours and 10% users travel more than two hours to reach CDA and NHMP respectively.
Concerning distance travelled, 68% users travel more than 10 kilometers to reach CDA which is the
highest percentage when compared with 44% users in the case of NADRA and 38% users in the case
of NHMP. Almost 66% customers of NADRA stated that their required service is being delivered within
stipulated time. Within the same category, the percentage was lower in the case of NHMP users.
However, 50% service users in the case of CDA narrated procrastination or unwarranted delay. 42%
CDA customers were of the view that the delay in the provision of services is due to corruption. In the
case of NHMP, 48% service users found unpredictable service delivery time. More than half (56%) of
the CDA service users felt intentionally deprived because of inordinate delay in pushing files through
the counter to higher authorities for approval. To push files, they have to resort to informal payments
or use the right kind of reference. 67% NADRA users found high work load to be the main cause of
intentional deprivation; whereas, 46% respondents reported rent seeking to be main factor of
deprivation. On the other hand, 40% NHMP customers found “low economic status” a stagnating factor
in acquiring the desired level of service, 40% users believed not to have the right kind of political
affiliation to be the main factor of intentional deprivation. 75% NADRA service users never reported
an incident of deprivation. According to them, reporting an incident would fall on deaf ears. Similar
reasons were found in the case of 89% CDA and 67% NHMP service users.
Majority users of NADRA and NHMP services were of the view that incidents of withholding services
rarely take place. Whereas, more than half (58%) CDA customers experience such incidents more often.
According to 84% users, CDA has no dedicated feedback channels to deal with customer complaints.
More than 45% users of NHMP and NADRA did observe feedback channels concerning customers’
complaints and feedback.
According to 88% users of NADRA, customer payment system was found to be very convenient,
followed by 60% users of NHMP. With regard to CDA, 60% service users found payment system to be
somewhat convenient. Majority users of all three institutions found the payment system in accordance
with expected cost of service. However, in the case of CDA, more than 60% female showed
apprehension concerning a mismatch between the cost of service and the available information. In the
case of NADRA, 67% users found cost of services as per their expectations followed by NHMP. In the
case of CDA, the response was divided between those who found the service in accordance with their
expectations and those who are unsatisfied with the services. More than 56% respondents in the case of
CDA cited frequent informal payments from service users. Similar experience is found in 40% service
users of NHMP. Majority respondents cited lack of internal and external accountability which directly
contributes to less than optimal performance. Some respondents cited political pressure which
undermines performance. Very few attributed bad performances to understaffing.
6. Findings-II: Results from augmented performance index
On the basis of the measurement criteria and weights provided in Annexure-I, ranking of institutional
performance is given below8
:
8 See also technical note on scoring method for demand and supply side computation in Annex-II.
13
Table 4: Ranking of Demand Side Indicators
Name of Institution(s) Average Score Ranking
National Highways & Motorway Police 2.86 2
National Database & Registration Authority (NADRA) 3.249 1
Capital Development Authority (CDA) 1.053 3
The above table elucidates the total scores and ranking of each institution on the basis of the selected
measurement areas of demand side variables. In this regard, NADRA scored the highest, i.e. 3.249, and
ranked 1st, for having good quality of governance, being least corrupted and no deprivation and
discrimination of its service beneficiaries. NADRA is followed by NHMP scoring 2.860. However,
CDA accounted for least average score, i.e. 1.053, because its service users felt internally deprived of
getting their required services, corruption rate is high, and there is a lack of availability of criteria for
customer service feedback.
Table 5: Ranking of Supply Side Indicators
Institution Average Score Ranking
National Highways & Motorway Police 0.845 2
National Database & Registration Authority (NADRA) 0.358 3
Capital Development Authority (CDA) 0.88 1
The aforementioned table of supply side score demonstrates the recorded scores and the ranking of
three institutions, based on the selected measurement areas, according to which CDA leads by scoring
0.880 for having full budgets available for all the three years (Fiscal Year 2011-14) in English only, for
updated policies, strategies and plans, and for budget execution, if not up-to the mark but at least, 65-
80% and execution of priority programs less than 50%. This is followed by NHMP scoring 0.845 and
NADRA is at third position with 0.358 average score. Moreover, it is interesting to note that all the
three institutions performed very poorly on accountability and responsiveness, which confirms the
intuitive analysis that Pakistan’s public-sector institutions are unaccountable to the public and
stakeholders and unresponsive to their needs.
Table 6: Final Rank
Institution Average Score Ranking
National Highways & Motorway Police 0.04576 2
National Database & Registration Authority (NADRA) 0.04619 1
Capital Development Authority (CDA) 0.02232 3
Combining all the scores of demand side and supply side together, it is evident from the above table
that NADRA is top-ranked, having average score of 0.04619, followed by NHMP which ranked second
the score of 0.04576. However, CDA accounted for least average score which caused its third position
in ranking table.
14
7. Conclusion
The paper makes an effort to combine supply and demand side accountability measures and apply an
augmented performance index framework on select public institutions in Pakistan. This pilot phase (i.e.
application of three government entities) helped us in analyzing performance indicators on the one hand,
and citizens’ feedback on the other. The supply side indicators are useful as they provide information
on how institutions perform core functions. The demand side indicators are useful in knowing how
citizens evaluate performance through access and informational indicators. Based on the strengths of
this index and learning from the weaknesses such as the actual performance may be political in nature,
the scope of the index would be enhanced, by including in it other public-sector institutions as well as
the provincial governments. It will be made a comprehensive index, which would work as a mirror for
the institutions. It is hoped that some of the learning from our findings could motivate the federal and
provincial governments towards more objective measurement of service delivery.
15
References
1. Bertot, JC, McDermott, P and Smith, T 2012, Measurement of Open Government: Metrics and
Processes, 45th Hawai International Conference on System Sciences, Hawai
2. Bold, T, Svensson, T, Gauthier, B, Mæstad, Q and Wane, W 2011, Service Delivery Indicators:
Pilot in Education and Health Care in Africa. Chr. Michelsen Institute, Norway.
3. Brown, K and Repucci, S 2009, A User’s Guide to Measuring Public Administration Performance.
Oslo Governance Centre, United Nations Development Programme, Oslo.
4. Burall, S, Gavelin, K and Wilson, R 2009, Open Government: beyond static measures. Organization
of Economic Cooperation and Development,
5. Castro, MF 2011, Defining and Using Performance Indicators and Targets in Government M&E
Systems. Special Series on The Nuts & Bolts of M&E Systems (no. 12). The World Bank,
Washington, D.C.
6. Cristescu, M, Mihaiu, D and Opreana, A 2010, Efficiency, Effectiveness and Performance of the
Public Sector, Romanian Journal of Economic Forecasting, vol. 4.
7. Davies, I 1999, Evaluation and Performance Management in Government. Evaluation, SAGE
Publications, London.
8. Dehn, J, Reinikka, R and Svensson, J 2003, Survey Tools for Assessing Performance in Service
Delivery, Toolkit for Evaluating the Poverty and Distributional Impact of Economic Policies.
World Bank, Washington, D.C.
9. Gokey, C, Parsons, J and Thornton, M 2013, Indicators of Inputs, Activities, Outputs, Outcomes
and Impacts in Security and Justice Programming, Department for International Development,
London.
10. Khattri, N and Roberts, D 2012, Designing a Results Framework for Achieving Results: A How-
To Guide. Independent Evaluation Group, Washington, D.C.
11. Kwakwa, V and Musoni, P 2008, Rwanda: Joint Governance Assessment report. Rwanda Joint
Governance Assessment Steering Committee, Kigali.
12. Landman, T 2003, Map-making and Analysis of the Main International Initiatives on Developing
Indicators on Democracy and Good Governance. University of Essex – Human Rights Centre,
Essex.
13. Lodge, M and Wegrich, K 2013, Setting the Scene: Challenges to the State, Government Readiness
and Administrative Capacities, Hertie School of Governance, Berlin.
14. Trivedi, P 2014, India: Performance Monitoring and Evaluation System, Proceedings of Global
Roundtable on Government Performance Management, Cabinet Secretariat, Government of India,
New Delhi.
16
Annexure-I: (Detailed computation of index)
a. Construction of demand side governance index
Table 20: Results for NADRA
BETA TEST RESULTS FOR NADRA
Measurement
Area
Criteria (C) Criteri
a
Weigh
t (CW)
Weighte
d Score
of
Criteria
(WSC)=
Sum of
WSI ×
CW
Indicators (I) Raw
Scor
e on
a
Scal
e of
1-10
(RS)
Indicato
r
Weight
(IW)
Weighte
d Score
of
Indicator
(WSI)=
RS × IW
Sum
of
WSI
Quality of
Governance
No
Corruption
1.00 9.5 No informal
payments or gifts
in lieu of
accessing
services
10.0 0.50 5 9.5
Transparency in
service delivery
9.0 0.50 4.5
Accountabilit
y
Customer
Feedback
1.00 2.3 Availability of
criteria for
Customer
Service Feedback
4.6 0.50 2.3 2.3
Inclusion and
Openness Inclusion 0.50 4.7
Not internally
deprived during
accessing service
9.4 1.00 9.4
9.4
Openness 0.50 0
Complaint
Procedure is easy
and accessible
0.0 0.50 0
0
No social
discrimination
0.0 0.50 0
Effectiveness
Service
Delivery
1.00 6.969
Availability of
services in the
vicinity
1.2 0.17 0.204
6.96
9Quality of service 6.6 0.17 1.122
17
Institution and its
staff is
trustworthy
9.4 0.15 1.41
Payment system
is very
convenient and
transparent
8.8 0.17 1.496
Cost of service is
affordable
6.7 0.17 1.139
Accuracy in cost
of service
9.4 0.17 1.598
Responsivene
ss
Mechanism
s for
facilitating
access to
information
0.34 2.45956
Designated
official
nominated under
Right to
Information Act
2010
4.0 0.34 1.36
7.23
4
Accessibility of
source of
information
9.8 0.33 3.234
Standards for
access to
information
services
8.0 0.33 2.64
Mechanism
s for public
grievance
redress
0.33 0
Complaint
mechanism is
available for
desired service
0.0 0.34 0
0
Acknowledgeme
nt of complaints
by relevant
department
0.0 0.33 0
Standards for
grievance
redress services
0.0 0.33 0
Use of ICT
tools
0.33 0.066
Availability of
online services
for complaints,
information,
0.2 1.00 0.2
0.2
18
Total Score
(Simple
Average of
WSC)
registration,
social media
3.249
Table 21: Results for CDA
BETA TEST RESULTS FOR CDA
Measurement
Area
Criteria ( C
)
Criteri
a
Weigh
t (CW)
Weighte
d Score
of
Criteria
(WSC)=
Sum of
WSI ×
CW
Indicators ( I )
Raw
Scor
e on
a
Scal
e of
1-10
(RS)
Indicato
r
Weight
(IW)
Weighte
d Score
of
Indicator
(WSI)=
RS × IW
Sum
of
WSI
Quality of
Governance
No
Corruption
1.00 1.7
No informal
payments or gifts
in lieu of
accessing
services
3.0 0.50 1.5
1.7
Transparency in
service delivery
0.4 0.50 0.2
Accountabilit
y
Customer
Feedback
1.00 0.3
Availability of
criteria for
Customer
Service
Feedback
0.6 0.50 0.3
0.3
Inclusion and
Openness
Inclusion 0.50 2.2
Not internally
deprived during
accessing service
4.4 1.00 4.4
4.4
Openness 0.50 0
Complaint
Procedure is easy
and accessible
0.0 0.50 0
0
No social
discrimination
0.0 0.50 0
Effectiveness
Service
Delivery
1.00 2.625
Availability of
services in the
vicinity
1.8 0.17 0.306
2.62
5
Quality of
service
0.0 0.17 0
19
Institution and its
staff is
trustworthy
2.2 0.15 0.33
Payment system
is very
convenient and
transparent
1.4 0.17 0.238
Cost of service is
affordable
5.0 0.17 0.85
Accuracy in cost
of service
5.3 0.17 0.901
Responsivene
ss
Mechanism
s for
facilitating
access to
informatio
n
0.34 1.3
Designated
official
nominated under
Right to
Information Act
2010
2.4 0.34 0.816
3.91
8
Accessibility of
source of
information
6.4 0.33 2.112
Standards for
access to
information
services
3.0 0.33 0.99
Mechanism
s for public
grievance
redress
0.33 0
Complaint
mechanism is
available for
desired service
0.0 0.34 0
0
Acknowledgeme
nt of complaints
by relevant
department
0.0 0.33 0
Standards for
grievance redress
services
0.0 0.33 0
Use of ICT
tools
0.33 0.264
Availability of
online services
for complaints,
information,
registration,
social media
0.8 1.00 0.8
0.8
Total Score (Simple Average of WSC) 1.053
20
Table 22: Results for Motorway Police
BETA TEST RESULTS FOR Motorway Police
Measurement
Area
Criteria
(C)
Criteria
Weight
(CW)
Weighted
Score of
Criteria
(WSC)=
Sum of
WSI ×
CW
Indicators ( I )
Raw
Score
on a
Scale
of 1-
10
(RS)
Indicator
Weight
(IW)
Weighted
Score of
Indicator
(WSI)=
RS × IW
Sum
of
WSI
Quality of
Governance
No
Corruption
1.00 7.4
No informal
payments or
provides gifts in
lieu of accessing
services
9.0 0.50 4.5
7.4
Transparency in
service delivery
5.8 0.50 2.9
Accountability
Customer
Feedback
1.00 2.4
Availability of
criteria for
Customer Service
Feedback
4.8 0.50 2.4
2.4
Inclusion and
Openness
Inclusion 0.50 4.5
Not internally
deprived during
accessing service
9.0 1.00 9
9
Openness 0.50 0
Complaint
Procedure is easy
and accessible
0.0 0.50 0
0
No social
discrimination
0.0 0.50 0
Effectiveness
Service
Delivery
1.00 6.366
Availability of
services in the
vicinity
5.2 0.17 0.884
6.366
Quality of service 4.2 0.17 0.714
Institution and its
staff is trustworthy
6.4 0.15 0.96
Payment system is
very convenient
and transparent
6.0 0.17 1.02
21
Cost of service is
affordable
8.2 0.17 1.394
Accuracy in cost
of service
8.2 0.17 1.394
Responsiveness
Mechanisms
for
facilitating
access to
information
0.34 1.5
Designated
official nominated
under Right to
Information Act
2010
1.2 0.34 0.408
4.368
Accessibility of
source of
information
7.0 0.33 2.31
Standards for
access to
information
services
5.0 0.33 1.65
Mechanisms
for public
grievance
redress
0.33 0
Complaint
mechanism is
available for
desired service
0.0 0.34 0
0
Acknowledgement
of complaints by
relevant
department
0.0 0.33 0
Standards for
grievance redress
services
0.0 0.33 0
Use of ICT
tools
0.33 0.726
Availability of
online services for
complaints,
information,
registration, social
media
2.2 1.00 2.2
2.2
Total Score (Simple Average of WSC) 2.860
Based on the above-mentioned results of all the three selected institutions, main results of the Demand
Side Better Governance Index (BGI) is as follow:
22
Table 23: Scoring of Demand Side
Institution Average Score
National Highways & Motorway Police 2.86
National Database & Registration Authority (NADRA) 3.249
Capital Development Authority (CDA) 1.053
Scoring methods can be found in annexure I (Table 1)
b. Construction of supply side index
Table 24: Results for NADRA
BETA TEST RESULTS FOR NADRA
Measurement
Area
Criteria
(C)
Criteri
a
Weigh
t (CW)
Weighte
d Score
of
Criteria
(WSC)=
Sum of
WSI ×
CW
Indicators
(I)
Raw
Scor
e on
a
Scal
e of
1-10
(RS)
Indicato
r
Weight
(IW)
Weighte
d Score
of
Indicato
r
(WSI)=
RS × IW
Su
m
of
WS
I
Quality of
Governance
Clarity of
institutional
vision and
mission
0.35 0
Statements
are clear,
brief and
reflect the
values,
philosophy
and culture
of the
institution
0 0.4 0
0
Vision
statement
expresses, in
sensory
terms, the
future
position or
image of the
institution
0 0.3 0
Mission
statement
focuses on
one or a few
strategic
goals
0 0.3 0
Budget Accuracy 0.40 0
Variance
between
revised
budget and
budget
estimates
0 1.0 0
0
23
Gender Equity in
HR
0.25
Proportion
of female
employees
in total
working
staff
0 1.0 0
0
Investment in
Capacity
Building
1.00
Training
budget as a
proportion
of total
Personnel
budget
0 1.0 0
0
Accountabilit
y
Political
Accountability
0.40 0
Timely and
complete
responses to
PAC review
queries
0 0.5 0
0
Regular
submission
of briefings
to the
Standing
Committees
of
Parliament
0 0.5 0
Accountability to
citizen
0.60 0
Publication
of Annual
Budget on
institution’s
website
0 0.2 0
0
Publication
of Yearbook
0 0.2 0
Majority of
tenders
follow
PPRA rules
0 0.2 0
Use of
social
accountabilit
y tools
0 0.2 0
Citizens’
Charter
issued
0 0.2 0
Inclusion and
Openness
Participatory and
inclusive decision
making and
oversight
0.50 0
Forums on
which
citizens,
private
sector,
academia or
civil society
are
represented
0 1.0 0
0
24
Proactive public
disclosure
0.50 0.75
Publication
of updated
policies,
strategies on
website
3 0.5 1.5
1.5
Availability
of
expenditure
data
0 0.5 0
Effectiveness
Budget execution 0.40 0
Proportion
of released
annual
budget
actually
utilized
0 1.0 0
0
Implementation
of priority
programs
0.40 0.4
Completion
level of
priority
programs
1 1.0 1
1
Strengthening
institutional
capacities for
implementation
0.20 0
Training
expenditure
as a
proportion
of total
Personnel
expenditure
0 1.0 0
0
Responsivene
ss
Linkage with
National/Provinci
al priorities and
strategies
0.25 0
Explicit
linkage of
priority
programs
with the
Vision 2025
0 0.5 0
0
Explicit
linkage of
priority
programs
with
relevant
sector policy
0 0.5 0
Mechanisms for
facilitating access
to information
0.25 0.5
Designated
official
nominated
under Right
to
Information
Act 2010
0 0.5 0
2
Standards
for access to
information
services
clearly
mentioned
4 0.5 2
Mechanisms for
public grievance
redress
0.25 0.5
Designated
official for
public
0 0.5 0
2
25
grievance
redress and
addressing
references
from
Ombudsmen
Standards
for
grievance
redress
services
clearly
mentioned
4 0.5 2
Use of ICT tools 0.25 2.5
Availability
of online
services for
complaints,
information,
registration,
social media
10 1.0 10
10
Total Score (Simple Average of WSC) 0.358
Table 25: Results for CDA
BETA TEST RESULTS FOR CDA
Measurement
Area
Criteria (C)
Criteria
Weight
(CW)
Weighted
Score of
Criteria
(WSC)=
Sum of
WSI ×
CW
Indicators ( I )
Raw
Score
on a
Scale
of 1-
10
(RS)
Indicator
Weight
(IW)
Weighted
Score of
Indicator
(WSI)=
RS × IW
Sum
of
WSI
Quality of
Governance
Clarity of
institutional
vision and
mission
0.35 0
Statements are
clear, brief and
reflect the
values,
philosophy
and culture of
the institution
0 0.4 0
0
Vision
statement
expresses, in
sensory terms,
the future
position or
image of the
institution
0 0.3 0
Mission
statement
focuses on one
0 0.3 0
26
or a few
strategic goals
Budget
Accuracy
0.40
2
Variance
between
revised budget
and budget
estimates
5 1.0 5
5
Gender
Equity in
HR
0.25
Proportion of
female
employees in
total working
staff
0 1.0 0
0
Investment
in Capacity
Building
1.00
Training
budget as a
proportion of
total Personnel
budget
0 1.0 0
0
Measureme
nt Area
Criteria ( C )
Criteri
a
Weigh
t (CW)
Weighte
d Score
of
Criteria
(WSC)=
Sum of
WSI ×
CW
Indicators ( I
)
Raw
Scor
e on
a
Scal
e of
1-10
(RS)
Indicato
r
Weight
(IW)
Weighte
d Score
of
Indicato
r (WSI)=
RS × IW
Sum of
WSI
Accountabili
ty
Political
Accountabili
ty
0.40 0
Timely and
complete
responses to
PAC review
queries
0 0.5 0
0
Regular
submission
of briefings
to the
Standing
Committees
of
Parliament
0 0.5 0
Accountabili
ty to citizens
0.60 0.24
Publication
of Annual
Budget on
institution’s
website
2 0.2 0.4
0.4
27
Publication
of Yearbook
0 0.2 0
Majority of
tenders
follow PPRA
rules
0 0.2 0
Use of social
accountabili
ty tools
0 0.2 0
Citizens’
Charter
issued
0 0.2 0
Inclusion
and
Openness
Participatory
and inclusive
decision
making and
oversight
0.50 0
Forums on
which
citizens,
private
sector,
academia or
civil society
are
represented
0 1.0 0
0
Proactive
public
disclosure
0.50 2.25
Publication
of updated
policies,
strategies
on website
5 0.5 2.5
4.5
Availability
of
expenditure
data
4 0.5 2
Effectivenes
s
Budget
execution
0.40 0.8
Proportion
of released
annual
budget
actually
utilized
2 1.0 2
2
Measuremen
t Area
Criteria ( C )
Criteri
a
Weig
ht
(CW)
Weight
ed
Score of
Criteria
(WSC)=
Indicators (
I )
Raw
Scor
e on
a
Scal
Indicat
or
Weight
(IW)
Weight
ed
Score of
Indicato
Sum of
WSI
28
Sum of
WSI ×
CW
e of
1-10
(RS)
r (WSI)=
RS × IW
Implementation
of priority
programs
0.40 0.4
Completio
n level of
priority
programs
1 1.0 1
1
Strengthening
institutional
capacities for
implementation
0.20 0
Training
expenditur
e as a
proportion
of total
Personnel
expenditur
e
0 1.0 0
0
Responsiven
ess
Linkage with
National/Provin
cial priorities
and strategies
0.25 0.75
Explicit
linkage of
priority
programs
with the
Vision
2025
0 0.5 0
3
Explicit
linkage of
priority
programs
with
relevant
sector
policy
0 0.5 0
Mechanisms for
facilitating
access to
information
0.25 1.25
Designate
d official
nominated
under
Right to
Informatio
n Act 2010
6 0.5 3
5
Standards
for access
to
informatio
n services
clearly
mentioned
4 0.5 2
29
Mechanisms for
public grievance
redress
0.25 1.25
Designate
d official
for public
grievance
redress
and
addressing
references
from
Ombudsm
en
6 0.5 3
5
Standards
for
grievance
redress
services
clearly
mentioned
4 0.5 2
Use of ICT tools 0.25 2.5
Availability
of online
services
for
complaints
,
informatio
n,
registratio
n, social
media
10 1.0 10
10
Total Score (Simple Average of WSC) 0.880
Table 26: Results for Motorway Police
BETA TEST RESULTS FOR Police
Measure
ment
Area
Criteria
( C )
Crit
eria
Wei
ght
(C)
Weight
ed
Score
of
Criteria
(WSC)
= Sum
of WSI
× CW
Indicators ( I )
Raw
Scor
e on
a
Scale
of 1-
10
(RS)
Indicato
r
Weight
(IW)
Weighte
d Score
of
Indicator
(WSI)=
RS × IW
Su
m
of
WS
I
Quality
of
Clarity
of
0.35 0.7175 Statements are clear, brief
and reflect the values,
4.0 0.4 1.6
30
Governa
nce
institutio
nal
vision
and
mission
philosophy and culture of
the institution
2.0
5
Vision statement expresses
in sensory terms, the future
position or image of the
institution
0.0 0.3 0
Mission statement focuses
on one or a few strategic
goals
1.5 0.3 0.45
Budget
Accurac
y
0.40
2
Variance between revised
budget and budget estimates
5.0 1.0 5 5
Gender
Equity
in HR
0.25
Proportion of female
employees in total working
staff
3.0 1.0 3 3
Investm
ent in
Capacity
Building
1.00
Training budget as a
proportion of total Personnel
budget
0.0 1.0 0 0
Accounta
bility
Political
Account
ability
0.40 0
Timely and complete
responses to PAC review
queries
0.0 0.5 0
0
Regular submission of
briefings to the Standing
Committees of Parliament
0.0 0.5 0
Account
ability
to
citizens
0.60 0.12
Publication of Annual
Budget on institution’s
website
0.0 0.2 0
0.2
Publication of Yearbook 1.0 0.2 0.2
Majority of tenders follow
PPRA rules
0.0 0.2 0
Use of social accountability
tools
0.0 0.2 0
Citizens’ Charter issued 0.0 0.2 0
Inclusion
and
Participa
tory and
inclusiv
e
0.50 0
Forums on which citizens,
private sector, academia or
civil society are represented
0.0 1.0 0 0
31
Opennes
s
decision
making
and
oversigh
t
Proactiv
e public
disclosu
re
0.50 2
Publication of updated
policies, strategies on
website
5.0 0.5 2.5
4
Availability of expenditure
data
3.0 0.5 1.5
Effective
ness
Budget
executio
n
0.40 0.4
Proportion of released
annual budget actually
utilized
1.0 1.0 1 1
Impleme
ntation
of
priority
program
s
0.40 0
Completion level of priority
programs
0.0 1.0 0 0
Strength
ening
institutio
nal
capacitie
s for
impleme
ntation
0.20 0
Training expenditure as a
proportion of total Personnel
expenditure
0.0 1.0 0 0
Responsi
veness
Linkage
with
National
/Provinc
ial
prioritie
s and
strategie
s
0.25 0.75
Explicit linkage of priority
programs with the Vision
2025
0.0 0.5 0
3
Explicit linkage of priority
programs with relevant
sector policy
0.0 0.5 0
Mechani
sms for
facilitati
ng
access
to
0.25 1.25
Designated Official
nominated under Right to
Information Act 2010
6.0 0.5 3
5
Standards for access to
information services clearly
mentioned
4.0 0.5 2
32
informat
ion
Mechani
sms for
public
grievanc
e redress
0.25 1.25
Designated Official for
public grievance redress and
addressing references from
Ombudsmen
6.0 0.5 3
5
Standards for grievance
redress services clearly
mentioned
4.0 0.5 2
Use of
ICT
tools
0.25 2.5
Availability of online
services for complaints,
information, registration,
social media
10.0 1.0 10 10
Total Score (Simple
Average of WSC) 0.845

More Related Content

PDF
Minning of Bitcoin Technology
PDF
Organization citizenship behaviour as a determining Factor in Business outcome
DOCX
Research proposal of sohail tariq (ms management science) uol
PDF
Improvement of IT Governance Case Study Government Institution Region X
PDF
Analysis of Performance Appraisal Systems on Employee Job Productivity in Pub...
PDF
The Effect of Participation in Budgeting on the Performance of Regional Gover...
PDF
Analyzing E-Government Development in Kudus Local Government Using SWOT Analysis
PPTX
PhD Presentation on 23rd April 2020 Before Viva Voce Bord
Minning of Bitcoin Technology
Organization citizenship behaviour as a determining Factor in Business outcome
Research proposal of sohail tariq (ms management science) uol
Improvement of IT Governance Case Study Government Institution Region X
Analysis of Performance Appraisal Systems on Employee Job Productivity in Pub...
The Effect of Participation in Budgeting on the Performance of Regional Gover...
Analyzing E-Government Development in Kudus Local Government Using SWOT Analysis
PhD Presentation on 23rd April 2020 Before Viva Voce Bord

What's hot (20)

PDF
Exploring the Impact of the use of Business Information systems BIS on the or...
PDF
IRJET- Campus E-Voting in a Developing Nation: An Application of the Unified ...
PDF
INVESTIGATING TANZANIA GOVERNMENT EMPLOYEES’ ACCEPTANCE AND USE OF SOCIAL MED...
PDF
FACTORS AFFECTING ACCEPTANCE OF WEB-BASED TRAINING SYSTEM: USING EXTENDED UTA...
PDF
The Effect Of Planning And Control To Bureaucracy Behavior In The Improving S...
PDF
Evaluation of Factors Affecting the Adoption of Smart Buildings Using the Tec...
PDF
Klibel5 acc 53_
PDF
The Effect of Information Technology, User Technical Skills, Education and Tr...
PDF
WHY KNOWLEDGE MANAGEMENT SYSTEM FAILED by DANIEL DONI SUNDJOJO
PDF
Record Management and Goals Achievement in the Public Sector: A Study of Sele...
PDF
Klibel5 bus 37
PDF
Factors affecting the introduction of ic ts for ‘healthcare
PDF
Klibel5 bus 23
PDF
Klibel5 acc 36_
PDF
Klibel5 acc 38_
PDF
Social media for collaborative learning
PDF
The Influence of Personal Information Capability and Management Commitment on...
PDF
Klibel5 acc 37_
PDF
Technology Acceptance Model for Mobile Health Systems
PDF
Knowledge management and organizational innovation sjbss
Exploring the Impact of the use of Business Information systems BIS on the or...
IRJET- Campus E-Voting in a Developing Nation: An Application of the Unified ...
INVESTIGATING TANZANIA GOVERNMENT EMPLOYEES’ ACCEPTANCE AND USE OF SOCIAL MED...
FACTORS AFFECTING ACCEPTANCE OF WEB-BASED TRAINING SYSTEM: USING EXTENDED UTA...
The Effect Of Planning And Control To Bureaucracy Behavior In The Improving S...
Evaluation of Factors Affecting the Adoption of Smart Buildings Using the Tec...
Klibel5 acc 53_
The Effect of Information Technology, User Technical Skills, Education and Tr...
WHY KNOWLEDGE MANAGEMENT SYSTEM FAILED by DANIEL DONI SUNDJOJO
Record Management and Goals Achievement in the Public Sector: A Study of Sele...
Klibel5 bus 37
Factors affecting the introduction of ic ts for ‘healthcare
Klibel5 bus 23
Klibel5 acc 36_
Klibel5 acc 38_
Social media for collaborative learning
The Influence of Personal Information Capability and Management Commitment on...
Klibel5 acc 37_
Technology Acceptance Model for Mobile Health Systems
Knowledge management and organizational innovation sjbss
Ad

Similar to Measuring Institutional Performance: Can governance indices help? (20)

PPTX
Presentation. Workshop to discuss the joint EC/SIGMA comments to the draft Bi...
PDF
Presentation, Keit Kasemets, SIGMA, Paris, 4 Dec 2015_English
PDF
Assessment of Local Governance and Development Performance in Indonesia
PDF
The Ibrahim Index of African Governance (IIAG) Methodology
PDF
61506_Capstone_Report_DFID_FINAL_Quantifying_Governance__Indicators-4
PDF
F265475
PPTX
Gender Concerns in New Paradigms of Governance.pptx
PDF
A Critical Review Of Literature On Influence Of Good Governance On Service Qu...
PDF
Monitoring Anti-Corruption Reforms in Bulgaria
PDF
Citizen satisfaction and transperancy
PDF
Optimising performance through C3I (coordination, collaboration, communicatio...
PDF
Handbook On Measuring Governance Elgar Handbooks In Public Administration And...
PPTX
BRAINPOoL Workshop 24 March 2014: E-frame presentation
PDF
What is governance
PDF
Dimensions features and indicators of Governance.pdf
PDF
Dimensions features and indicators of Governance.pdf
PPT
Indicators: Levels, Types, Existing and New
PDF
The Role of Transparency in Mediating the Determinants of Government Accounta...
PDF
Performance accountability of local government
PDF
Transparency_international_Nigeria
Presentation. Workshop to discuss the joint EC/SIGMA comments to the draft Bi...
Presentation, Keit Kasemets, SIGMA, Paris, 4 Dec 2015_English
Assessment of Local Governance and Development Performance in Indonesia
The Ibrahim Index of African Governance (IIAG) Methodology
61506_Capstone_Report_DFID_FINAL_Quantifying_Governance__Indicators-4
F265475
Gender Concerns in New Paradigms of Governance.pptx
A Critical Review Of Literature On Influence Of Good Governance On Service Qu...
Monitoring Anti-Corruption Reforms in Bulgaria
Citizen satisfaction and transperancy
Optimising performance through C3I (coordination, collaboration, communicatio...
Handbook On Measuring Governance Elgar Handbooks In Public Administration And...
BRAINPOoL Workshop 24 March 2014: E-frame presentation
What is governance
Dimensions features and indicators of Governance.pdf
Dimensions features and indicators of Governance.pdf
Indicators: Levels, Types, Existing and New
The Role of Transparency in Mediating the Determinants of Government Accounta...
Performance accountability of local government
Transparency_international_Nigeria
Ad

More from Vaqar Ahmed (20)

DOCX
2025 UN High-level Committee on South-South Cooperation Side-event: Empowerin...
PPTX
How Can Regional Cooperation Under CAREC Enhance Cross-Border Freelancing Opp...
PPTX
Population Growth and Economic Progress.
PPTX
Personnel Exchanges as a Cooperation Model for South-South and Triangular Coo...
PDF
Global South Researchers and Inequality in Development Consulting
DOCX
Swabi Master Plan & City Strategic Management Plan
PDF
History of the Economy of Pakistan (1947-2024)
PDF
Economic Revitalization for Pakistan: An Overview
PDF
Fiscal Sustainability and Pension Liabilities
PPTX
Sustaining Public Finance and Social Protection Reforms in Pakistan
PDF
Chapter 1 Delivering Economic Reforms Amid Political Uncertainty
PPTX
Solid Waste Management – Effective Waste Strategies for a Greener Tomorrow
PPTX
State of Pakistan's Economy: Identifying Opportunities
PDF
Promoting Sustainable Urban Development: Lessons Learned from Khyber Pakhtunk...
DOCX
Equity in research practices: a southern-led action agenda
PDF
Personnel Exchanges as a Cooperation Model for South-South and Triangular Coo...
PPTX
Institutional Developments & Reforms in Pakistan
PDF
Financial Appraisal With Risk Assessment Lens
DOCX
Mobilizing Local Government Revenues: The Case of Khyber Pakhtunkhwa
DOCX
Local Government Reforms: Case of Own Source Revenue Mobilization
2025 UN High-level Committee on South-South Cooperation Side-event: Empowerin...
How Can Regional Cooperation Under CAREC Enhance Cross-Border Freelancing Opp...
Population Growth and Economic Progress.
Personnel Exchanges as a Cooperation Model for South-South and Triangular Coo...
Global South Researchers and Inequality in Development Consulting
Swabi Master Plan & City Strategic Management Plan
History of the Economy of Pakistan (1947-2024)
Economic Revitalization for Pakistan: An Overview
Fiscal Sustainability and Pension Liabilities
Sustaining Public Finance and Social Protection Reforms in Pakistan
Chapter 1 Delivering Economic Reforms Amid Political Uncertainty
Solid Waste Management – Effective Waste Strategies for a Greener Tomorrow
State of Pakistan's Economy: Identifying Opportunities
Promoting Sustainable Urban Development: Lessons Learned from Khyber Pakhtunk...
Equity in research practices: a southern-led action agenda
Personnel Exchanges as a Cooperation Model for South-South and Triangular Coo...
Institutional Developments & Reforms in Pakistan
Financial Appraisal With Risk Assessment Lens
Mobilizing Local Government Revenues: The Case of Khyber Pakhtunkhwa
Local Government Reforms: Case of Own Source Revenue Mobilization

Recently uploaded (20)

PDF
eVerify Overview and Detailed Instructions to Set up an account
PPTX
The DFARS - Part 251 - Use of Government Sources By Contractors
PDF
PPT Item # 4 - 328 Albany St compt. review
PDF
CXPA Finland Webinar: Rated 5 Stars - Delivering Service That Customers Truly...
PPTX
Developing_An_Advocacy_Agenda_by_Kevin_Karuga.pptx
PPTX
Part II LGU Accreditation of CSOs and Selection of Reps to LSBs ver2.pptx
PPTX
Portland FPDR Oregon Legislature 2025.pptx
PDF
Item # 3 - 934 Patterson Final Review.pdf
PDF
ESG Alignment in Action - The Abhay Bhutada Foundation
PDF
About Karen Miner-Romanoff - Academic & nonprofit consultant
PPTX
dawasoncitcommunityroolingadsAug 11_25.pptx
PDF
Abhay Bhutada Foundation’s ESG Compliant Initiatives
DOCX
EAPP.docxdffgythjyuikuuiluikluikiukuuuuuu
PPTX
Weekly Report 17-10-2024_cybersecutity.pptx
PPTX
SOMANJAN PRAMANIK_3500032 2042.pptx
PPTX
Parliamentary procedure in meeting that can be use
PDF
PPT - Primary Rules of Interpretation (1).pdf
PDF
Population Estimates 2025 Regional Snapshot 08.11.25
PPTX
Neurons.pptx and the family in London are you chatgpt
PDF
Building Bridges (of Hope) over Our Troubled Waters_PART 1
eVerify Overview and Detailed Instructions to Set up an account
The DFARS - Part 251 - Use of Government Sources By Contractors
PPT Item # 4 - 328 Albany St compt. review
CXPA Finland Webinar: Rated 5 Stars - Delivering Service That Customers Truly...
Developing_An_Advocacy_Agenda_by_Kevin_Karuga.pptx
Part II LGU Accreditation of CSOs and Selection of Reps to LSBs ver2.pptx
Portland FPDR Oregon Legislature 2025.pptx
Item # 3 - 934 Patterson Final Review.pdf
ESG Alignment in Action - The Abhay Bhutada Foundation
About Karen Miner-Romanoff - Academic & nonprofit consultant
dawasoncitcommunityroolingadsAug 11_25.pptx
Abhay Bhutada Foundation’s ESG Compliant Initiatives
EAPP.docxdffgythjyuikuuiluikluikiukuuuuuu
Weekly Report 17-10-2024_cybersecutity.pptx
SOMANJAN PRAMANIK_3500032 2042.pptx
Parliamentary procedure in meeting that can be use
PPT - Primary Rules of Interpretation (1).pdf
Population Estimates 2025 Regional Snapshot 08.11.25
Neurons.pptx and the family in London are you chatgpt
Building Bridges (of Hope) over Our Troubled Waters_PART 1

Measuring Institutional Performance: Can governance indices help?

  • 1. Working Paper # 172 Measuring Institutional Performance in Pakistan: Can governance indices help? By: Shehryar Khan Toru, Syed Mohsin Ali, Vaqar Ahmed
  • 2. All rights reserved. No part of this paper may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording or information storage and retrieval system, without prior written permission of the publisher. A publication of the Sustainable Development Policy Institute (SDPI). The opinions expressed in the papers are solely those of the authors, and publishing them does not in any way constitute an endorsement of the opinion by the SDPI. Sustainable Development Policy Institute is an independent, non-profit research institute on sustainable development. First edition: July 2019 © 2019 by the Sustainable Development Policy Institute Mailing Address: PO Box 2342, Islamabad, Pakistan Telephone: 0092-51-2278134, 2278136, 2277146, 2270674-76 Fax: 0092-51-2278135, URL: www.sdpi.org
  • 3. Acknowledgement We acknowledge the data related support provided by Ahmad Durrani, Maryam Waqar, and Syeda Rubab. We are grateful for support and cooperation by United Nations Development Program (UNDP), Pakistan Office, NADRA, CDA and NHMP. Ashfaq H. Khan, Nohman Ishtiaq and Shahid Sattar provided experts’ input on the methodology. The team is grateful for their support and services. The usual disclaimer applies.
  • 4. Abstract In Pakistan, several efforts have been made to measure the government performance at national and sub-national levels. Our efforts build on an existing framework for measuring service delivery. Though the previous tool, which has been applied to study the select federal government ministries includes supply side governance and accountability indicators, there is, however, a need to also incorporate demand side indicators to capture community satisfaction from publicly provided services. In this study, an effort has been made to formulate an augmented index and by using survey data, application of this index on three public sector entities has been analyzed. Results show that this approach has the benefit of validating supply and demand side information and perceptions. Keywords: Governance, BGI, Public sector institutions,
  • 5. Table of Contents 1. Introduction.....................................................................................................................................1 2. Literature Review............................................................................................................................2 3. Methodology...................................................................................................................................6 4. Construction of Performance Index ................................................................................................8 4.1. Performance Indicators for Demand Side...............................................................................8 4.2. Performance Indicators for Supply Side.................................................................................8 5. Findings-I: Citizen Feedback survey ............................................................................................10 6. Findings-II: Results from augmented performance index ............................................................12 7. Conclusion ....................................................................................................................................14 References.............................................................................................................................................15 Annexure-I: (Detailed computation of index).......................................................................................16 a. Construction of demand side governance index............................................................................16 b. Construction of supply side index.................................................................................................22
  • 6. 1 1. Introduction The study aims to measure the performance of a select number of public sector institutions using both supply and demand side indicators. Our starting point is Better Governance Index (BGI) framework developed by Moriani and Rehman1 . We supplement the effort from 2015 using a demand side client satisfaction survey to be explained later. The study, on the one hand, provides a conceptual tool to merge internal and external outcome measures, while on the other it goes on to apply this tool on a select number of select public sector entities, including National Database & Registration Authority (NADRA), National Highways and Motorway Police (NHMP), and Capital Development Authority (CDA). These institutions have been selected on the basis of their public interface as well as the high frequency with which their services are demanded by the society. At the outset, the performance measurement framework adopted in BGI2 sets the basis of engaging different frameworks for measuring institutional performance in public sector institutions. The importance of inputs, outputs and strategic goals are mentioned to illustrate -institutional efficiency in more general terms. The terms and process indicators of BGI are explained to gain familiarity with types of performance frameworks. The rationale behind gauging performance is to benchmark performance indicators on a set of pre-selected indicators, namely quality of governance, accountability, inclusion and openness, effectiveness and responsiveness. Measurements of institutional efficiency generally focus on the relationship between institutional resources and activities on the one hand, and institutional outcomes, on the other (Cristescu et al. 2010; Castro 2011; Gokey et al. 2013; Khattri and Roberts, 2012. Systems of measuring performance efficiency have indicators functioning both at the individual level such as the qualitative service delivery survey (Bold et al. 2011; Dehn et al. 2003), and the institutional level such as the results framework (Trivedi 2014). Measurements of institutional effectiveness focus on the relationship between the intended and actual outcomes and impacts, or the ratio between outputs and outcomes (Cristescu et al. 2010). However, while such calculations are often straightforward in the private sector, they are harder to make in the public sector due to the difficulty of conclusively establishing causal link between the level of activities and outcomes (Davies 1999). Owing to the considerable difficulties in measuring institutional efficiency and effectiveness within the public sector, evaluation experts have developed another set of indicators that measure the nature of policy implementation such as whether it supports accountability and citizens participation. These are known as process indicators (Landman 2003). The Weberian Comparative State Data project is a comparative assessment framework that includes a number of process indicators with its focus on measuring various dimensions of bureaucratic structure and institutional meritocracy. With its twofold objective, the study would first augment BGI framework for Pakistan with inclusion of citizen satisfaction indicators, and then make an effort to apply the augmented index to gauge the performance of select public sector entities. to the study also suggests as to how such performance index can be institutionalized by the government institutions concerned. 1 Farrukh Moriani and Hira Hafeez ur Rehman (2015) produced a report in 2015 on “Better Governance Index: a conceptual framework for measuring institutional performance”. This report uses a conceptual framework having process indicators (………) for measuring institutional performance of select institutions. 2 Adopted from Better Governance Index report authored by (Moriani & Rehman, 2015).
  • 7. 2 2. Literature Review This section provides a brief literature review with case studies from the select countries. We also provide current gaps in literature and how some of these are addressed. This is followed by findings and conclusion section. How countries measure institutional performance? A review of literature was carried out to understand governance indices used by other studies, and countries in South Asia and elsewhere. Existing literature on institutional performance and their utility in different country contexts are cited as useful performance frameworks. The intention behind this effort is to strengthen the BGI framework. Efforts have been made to see in-depth at least three case studies with respect to objectives, methodology, rollout and sustainability of performance measurement frameworks (see Table 1). The criteria used for selecting country examples consists of i) governance context and relevance (to Pakistan) which captures accountability and other attributes (similar to local milieu); ii) indicators description, i.e. process and micro level indicators (similar to processes seen in Pakistan’s public sector); and iii) sustainability and institutionalization. A number of developed and developing countries have designed and implemented performance measurement systems within their domains and varying contexts. This section includes a brief review of such systems, which are: i) applicable to agency-level public sector performance; ii) broad enough to facilitate inter-agency comparison of performance; based on both subjective and objective means of data collection.3 Table 1: Select Examples of Performance Measurement Frameworks Framework Details Results Framework Document [RFD] (Khattri and Roberts, 2007; Trivedi, 2014) Rwanda Joint Governance Assessment [JGA] (Kwakwa and Musoni, 2008; Rwanda Governance Advisory Council, 2011) Open Government Directive [OGD] (Bertot et al. 2012) Governance Report 2013/2014 (Lodge and Wegrich, 2013) Description Used in India, this tool compares the performance of individual departments against standardized indicators, objective targets and measurement criteria (Khattri and Roberts, 2012) A country-specific tool developed with the aim of creating a common understanding of governance issues and measurement frameworks in Rwanda, and identifying future areas for performance improvement and priority action. Measurement areas include: Ruling Justly, Investment Climate, and Government Effectiveness. Majority of indicators geared towards country- level governance performance. Rolled-out in USA to measure the success of transparency and open government initiatives within different public- sector agencies. Assessments were undertaken in three key areas: Transparency, Participation, and Collaboration. A highly adaptable assessment framework that functions at the national, sub- national and agency levels of governance. Measures government’s administrative capacity (resilience and preparedness) to address contemporary 3 There may also be other measurement systems, e.g. see Hertie School of Governance Indicators: http://www.governancereport.org/home/governance-indicators/2016-governance-indicators/
  • 8. 3 challenges, including the identification of problems, the use of tools to address problems, and types of responses generated towards problems. Assessments are carried out on the basis of aggregate subjective indicators measuring capacity for Delivery, Regulation, Coordination and Analytical capacities. Objectives a. Planning strategies and selecting appropriate interventions to address challenges in question b. Building consensus and ownership between different stakeholders of a policy intervention c. Providing objective performance data to inform programme decisions d. Improved intra and inter- agency communication and reporting e. Performance evaluation Harmonization and rationalization of procedures a. Consolidating peace and security in the country b. Promoting inclusive governance c. Strengthening rule of law and accountability d. Linking government operations and activities to broader development goals Assessing the impact of transparency and open government initiatives within government agencies. Developing an understanding of how administrative capacity of governments can be improved in terms of understanding and responding to emerging challenges in better and innovative ways Methodology & roll-out i. Consultation between Ministers and line i. Consultations with diverse base of government and non- government i. Experts from government and non- government i. Governance and public administration and
  • 9. 4 departments on departmental objectives, proposed activities to achieve objectives, and corresponding success indicators (micro, macro and process) and time-bound targets ii. Devising weighted scores for each success indicator and target iii. Defining level of coordination required with outside departments for achievement of objectives iv. Critique and approval by Cabinet Secretariat v. Review by centralized Performance Management Division and non- government sector experts vi. Review by Cabinet Committee on Government Performance Final RFD placed on departmental website stakeholders to design draft indicators framework ii. Management structure established, including Steering Committee to oversee roll-out and indicators development, Technical Committee to advise on methodological and analytical issues, and Consultant team reporting to Steering Committee on implementation iii. Regular meetings between different tiers in management structure to discuss progress, earlier drafts and way forward iv. Quality Control by consultant team during roll-out v. Production of evaluation reports by Consultant Team Training of Governance Advisory Council (GAC) staff for implementation. GAC to function as centralized secretariat for JGA implementation after pilot. sectors, and employees from agencies being assessed brought together in Open Government Working Group to devise assessment criteria and data collection methods and tools ii. Pre-testing of assessment methods and tools, and incorporation of feedback iii. 3-point ordinal scale developed to eliminate problems resulting from poor goal description or inter-rater variations in scoring iv. Expert evaluators employed to assess agency compliance with OGD requirements v. Agency-level self- assessments also conducted Performance data displayed publicly management experts design indicators framework and selected data sources ii. Aggregation of subjective and objective data to generate composite scores in each assessment area iii. Borda scoring: scores assigned to individual institutions and then aggregated to provide country-level scores Scores represented at three different levels: agency, national and transnational Consultative process Consultation and review at different levels, with multiple Consultation and review at different levels, with multiple stakeholders, both Consultation and review at different levels, with multiple Consultations among experts of Hertie School
  • 10. 5 stakeholders, both government and non-government government and non- government stakeholders, both government and non- government Sustainability Consultations at multiple levels of government to ensure stakeholder buy-in and ownership Creation of electronic Results- Framework Management System, that aids in RFD preparation and monitoring on monthly and annual basis Consultations at multiple levels of government to ensure stakeholder buy-in and ownership Centralized Secretariat for monitoring of JGA implementation Consultations at multiple levels of government to ensure stakeholder buy- in and ownership. Sustainability hampered due to lack of centralized control, and resistance at agency-level against assessments by non-government sector Lack of evaluation criteria, especially at the level of impacts, prevents comparison of effectiveness of OGD implementation between different agencies No information provided Evidence of impact Covers 80 departments of Government of India and being implemented in 18 Union territories Improved Grievance Redress after mandatory inclusion of corresponding indicators within departmental RFDs Improved social indicators Transparency in public procurement Increase in competitive bidding for award of government contracts Decrease in corruption Determination of new salary schemes Increase in number of public sector reform initiatives Open Data 500 US studies the variety of different companies and innovators using the data made accessible by different government agencies. Positive correlation among different capacities identified Relevance to BGI Allows comparisons of inter-agency governance performance Aside from indicators measuring Government Effectiveness, JGA is unsuited to measuring agency-level governance Allows for comparison of inter-agency compliance with Allows comparisons of inter-agency governance
  • 11. 6 Objective means of data collection Includes process indicators such as effectiveness of public grievance redress mechanisms Objective and subjective means of data collection Includes process indicators OGD requirements Lack of evaluation criteria at level of impacts prevents comparisons of agency performance in terms of producing social benefits Includes process indicators Objective means of data collection Flexible enough to allow same capacity to be measured through both objective and subjective means. In the case of Pakistan and to our best knowledge, the BGI framework encompasses supply side concepts that include quality of governance, internal accountability, inclusion and openness, effectiveness (e.g. in the context of usage of financial resources), and responsiveness (e.g. in the context of ability to respond to the expectations of citizens), however to make such an index truly comprehensive indicators which reflect citizen satisfaction need to be incorporated. We make this effort in the sections ahead. 3. Methodology Methodology encompasses mixed methods’ approach where quantitative and qualitative research tools are employed. After undertaking a literature review to learn about the utilization, roll-out and institutionalization of performance measurement tools, we embarked upon the construction of index. Our point of departure was to revise BGI through revisiting the criteria of governance index, sources of data available on indicators, and suggest possibilities of future research on BGI framework, with a particular focus on possible inclusion of qualitative indicators. Data collected through different sources (such as official documents, websites, and secondary research) was triangulated. An added feature of validating data pertains to the feedback of local experts on governance and institutional performance. This particular activity entails the constitution of a feedback mechanism, thus allowing critical reflection of public sector institutions on governance indicators. While supply side indicators remain similar to BGI (and frameworks from other countries mentioned in section 2), a survey was designed to gauge the perception of citizens on the functions of select institutions. The survey included specific questions regarding experiences of citizens, process of service, outcome, and grievance redressal processes. To validate findings, key informant interviews were held with experts who have in the past either developed or used performance measurement frameworks. There are certain challenges in the way of implementation of this methodology. This also gives rise to limitations of this study. First, since governance index involves performance assessment of public sector entities, the project team faced informational and public disclosure challenges. The public officials were sometimes reluctant to provide information on matters which fall in official domain. For example, seeking information on employees might raise eyebrows. The institution may not like to share such information. Similarly, some specific budget-related information at a disaggregated level is difficult to obtain.
  • 12. 7 The public officials asked SDPI for completing unanticipated written approvals. This resulted in unnecessary delay which may have changed the political economy context of our analysis. This however should not affect the conceptual value of the tool developed in this study. The following public-sector institutions are selected for applying the augmented performance index. Highway and Motorway Police are mandated to ensure safe and citizen centric safety environment for drivers. Their core functions are issuance and renewal of driving licenses, assurance of road safety, enforcement of traffic rules, and assistance of driver in accidents and emergences. National Database & Registration Authority (NADRA) is a federal institution having branches across the country. It provides services related to: issuance of Computerized National Identity Cards; Juvenile card4 ; National Identity Card for Overseas Pakistanis; Child registration certificate; Cancellation certificate5 ; Family registration certificate6 ; and Pakistan Origin Card7 . Capital Development Authority (CDA) provides municipal services in Islamabad which include (but are not limited to): residential and commercial property registration and transfer; refuse (waste) collection; approval of construction of houses; collection of property tax; ensuring health and safety of residents; approval for the construction of commercial buildings; imposition and collection of water charge; development of new residential sectors; water supply connection; land accusation for new schemes; and streetlighting. The aforesaid institutions are selected on the basis of following criteria: Access: One of the most influential criteria of acquiring information from government institutions is access. The criterion of access is selected on the basis of good professional relations of SDPI with selected institutions and on the basis of receptivity and ownership for the pilot study amongst the institutions. Public Interface and service provision: The selected institutions are public sector entities mandated to provide public and collective services to clients. As a result of state officials’ interaction with citizens on the diverse services, these institutions have a public interface considered essential to understand governance perceptions and benchmarking institutional performance. Regarding the citizen feedback survey, a quantitative research methodology was adopted to acquire answers to multiple options of inquiry. In some instances, descriptive qualitative inquiry is also included. A purposive sampling technique is adopted to ensure the selection of stakeholders from pre- determined pool. This pool includes beneficiaries of the select public sector entities. One of the advantages of such sampling approach is that it allows targeted questions specific only to the users of public services. The overall sample size for the survey is 300, selected from the stratum of service users. The team ensured at least 100 responses per institution. This minimum response rate can increase if a single respondent also wishes to answer questions related to multiple public sector entities and services, accessed during the past. All respondents are above 18 years of age and 50 per cent respondents for each institution are women. The final gender mix of our respondents varies depending upon the refusal rate. 4 “Juvenile card is an identity card issued to children under the age of 18 years”. 5 Cancellation certificate is a registration document issued to register the termination of ID card in case of death of any citizen. 6 “Family Registration Certificate (FRC) is a mean of being identified with your NADRA’s record”. 7 “Pakistan Origin Card is issued to facilitate the eligible foreigners. This card is for those foreigners who have some roots in Pakistan i.e., they are natives of Pakistan.
  • 13. 8 Prior to the pre-testing and implementation, enumerators were trained on how to use the survey instrument. To ensure accuracy of data and its analysis, enumerators ensured the elimination of inconsistencies and incorrect responses during the phase of data collection. 4. Construction of Performance Index 4.1. Performance Indicators for Demand Side The criteria used for each measurement area are presented in Table 2. A total of 19 indicators are proposed for eight criteria under five measurement areas. Since in the pilot phase of index is to be developed for the selected three institutions, (e.g. NADRA, CDA, and NHMP), the indicators aim at measuring performance at that level. Table 2: Demand Side Indicators Measurement Area Criteria (C) Indicators (I) Quality of Governance No Corruption No informal payments or gifts in lieu of accessing services Transparency in service delivery Accountability Customer Feedback Availability of criteria for Customer Service Feedback Inclusion and Openness Inclusion Not internally-deprived during accessing service Openness Complaint procedure is easy and accessible No social discrimination Effectiveness Service Delivery Availability of services in the vicinity Quality of service Institution and its staff are trustworthy Payment system is very convenient and transparent Cost of services are affordable Accuracy in the cost of service Responsiveness Mechanisms for facilitating access to information Designated official nominated under Right to Information Act 2010 Accessibility of source of information Standards for access to information services Mechanisms for public grievance redress Complaint mechanism is available for desired service Acknowledgement of complaints by relevant department Standards for grievance redress services Use of ICT tools Availability of online services for complaints, information, registration, social media 4.2. Performance Indicators for Supply Side List of performance indicators given in table 3, for measuring supply side index are proposed on the basis of: i) classification of each indicator; ii) ii) an evaluation of each indicator with respect to its relevance, coverage and cost- effectiveness.
  • 14. 9 Table 3: Supply Side Indicators Measurement Area Criteria Indicators Indicator Classification Evaluation (relevance, coverage, cost) Quality of Governance Clarity of institutional vision and mission Statements are clear, brief and reflect the values, philosophy and culture of the institution Process indicator Relevant, global, and cost effective Vision statement expresses in sensory terms, the future position or image of the institution Process indicator Relevant, global, and cost-effective Mission statement focuses on one or a few strategic goals Process indicator Relevant, global, and cost-effective Budget Accuracy Variance between revised budget and budget estimates Micro-level Relevant, global, and cost-effective Gender Equity in HR Proportion of female employees in total working staff Process indicator Relevant, global, and cost-effective Investment in Capacity Building Training budget as a proportion of total Personnel budget Micro-level Relevant, global and cost-effective Political Accountability Regular submission of briefings to the Standing Committees of Parliament Process indicator Relevant, global and cost-effective Accountability Accountability to citizens Publication of Annual Budget on institution’s website Process indicator Relevant, global and cost-effective Publication of Yearbook Process indicator Relevant, global, and cost-effective Majority of tenders follows PPRA rules Process indicator Relevant, global and cost-effective Use of social accountability tools Process indicator Relevant Citizens’ Charter issued Process indicator Relevant and cost-effective Inclusion and Openness Participatory and inclusive decision making and oversight No. of forums on which citizens, private sector, academia or civil society are represented Process indicator Relevant, global, and cost-effective Proactive public disclosure Publication of updated policies, strategies on website Process indicator Relevant, global, and cost-effective Availability of expenditure data Process indicator Relevant, global, and cost-effective Effectiveness Budget execution Proportion of released annual budget actually utilized Micro-level Relevant, global, and cost-effective
  • 15. 10 Implementation of priority programs Completion level of priority programs Micro-level Relevant and global Responsiveness Linkage with National/Provincial priorities and strategies Explicit linkage of priority programs with the Vision 2025 Process indicator Relevant and global Explicit linkage of priority programs with relevant sector policy Process indicator Relevant and global Mechanisms for facilitating access to information Designated Official nominated under Right to Information Act Micro-level Relevant, global, and cost-effective Standards for access to information services clearly mentioned on website Process indicator Relevant, global, and cost-effective Mechanisms for public grievance redress Designated Official for public grievance redress and addressing references from Ombudsmen Micro-level Relevant, global, and cost-effective Standards for grievance redress services clearly mentioned Process indicator Relevant, global, and cost-effective Use of ICT tools Availability of online services for complaints, information, registration, social media Micro-level Relevant, global, and cost-effective 5. Findings-I: Citizen Feedback survey The highest percentage of respondents had availed services related to NADRA. For example, 72% male and 58% female respondents availed the services of acquiring Computerized National Identity Card (CNIC). Furthermore, 10% males and 19% females acquired Child Registration Certificate. Only 3% males and 2% females availed the service of cancellation certificate; whereas only 2% males and 3% females availed family registration certificate. Getting Juvenile card was the least availed service as only one female respondent opted for it. The reason for the higher percentage of service acquisition in the case of CNIC is its requirement on daily basis; the Juvenile card is not in common usage as most of the people are not aware of its utility. The quality of services provided by NADRA was also termed very satisfactory. In the case of CDA, the highest percentage of respondents benefitted from ‘Residential or Commercial Property Registration and Transfer’, which accounts for 59% males and 48% females. ‘Approval of construction of houses’ is the second most commonly availed service by 25% males and 19% females. None of the male respondents availed facilities of ‘collection of property tax’, ‘Ensuring health and safety of residents’ and ‘Approval of construction of commercial building’, only 4% (constitutes one female) availed the aforementioned services. Similarly, none of the female respondents benefitted from services of ‘imposition and collection of water charges’ and ‘refuse (waste) collection,’ and only 03% (constitutes only 1 man) availed the aforementioned services. The service of ‘water supply connection/reconnection’ was availed by 6% males and 7% females. Similarly, 03% males and 15% females availed the service of land accusation for new schemes.
  • 16. 11 Upon inquiring about the quality of services provided by CDA, more than half of the respondents indicated low priority for services (including Residential or commercial property registration and transfer, Collection of property tax, Water supply connection/reconnection, Ensuring health and safety of residents, Land accusation for new schemes, Approval of construction of houses, and Approval of construction of commercial buildings) due to high rate of corruption and informal payments. Around 25% respondents ranked residential and commercial property registration and transfer, and Approval of construction of houses somewhat lowest due to unspecified duration of time span. Few respondents held a neutral stance, asserting keeping quite as their specific request for a service may be unnecessarily delayed by CDA. From the services being offered by National Highways and Motorway Police (NHMP), issuance of driving license was availed by 63% males and 64% females. The issuance of international standard driving licenses had been the least common among the respondents as they accounted for only 4% female. On the other hand, 17% males and 32% females availed the service of renewal of driving license and 20% respondents availed the service “acquittal from challan”. Considering the quality of services, 16% respondents were of the view that the quality was somewhat satisfactory. In the case of 6% respondents, the services of ‘Issuance of driving license’ and ‘Acquittal from challan’ were least satisfactory. Whereas 44% beneficiaries indicated entire services of ‘highest’ rank. In the case of 14% respondents, issuances of driving license, renewal of driving license and acquittal from challan were ranked less satisfactory. In terms of lodging of a complaint, majority complaints were lodged against NHMP as they accounted amongst 20% male and 48% females. The least number of complaints was lodged against NADRA, i.e. 4% amongst female and 28% amongst male respondents. In the case of CDA, 18% respondents lodged complaints. The reason for lodging least complaints against NADRA is the effective, timely and transparent provision of services. Around 88% NHMP service users were satisfied with their complaints as they were addressed within a reasonable time. Similar response concerning timely redressal of complaints was observed in the case of NADRA. A glaring observation relates to CDA’s non-redressal of complaints in more than 70% male and 60% female respondents. When probed for not lodging a compliant, 19% consumers in the case of CDA were not aware of the complaint procedure; whereas 5% consumers made reference to a complicated procedure for lodging complaint. Similarly, only 3% respondents found complaint procedure of NHMP complex. In the case of CDA, 36% consumers found the complaint procedure ineffective, which constitutes the highest proportion of complaints when compared with 19% NHMP and 2% NADRA. In the case of CDA, 19% service users lodged complaints because of the fear of harassment. Concerning the source of information, 74% respondents learned about NHMP through the word of mouth, followed by 62% in the case of NADRA and CDA. Upon inquiring about accessibility of information, 98% respondents in the case of NADRA were able to access information. On the other hand, 30% respondents in the case of CDA opted for 'somewhat accessible' option. The following reasons were cited for not having comprehensive access to information  Physical access to information faced by 94% respondents in the case of CDA.  Non-cooperation of NHMP’s staff faced by 26% respondents.
  • 17. 12  88% respondents do not have access to the offices of NADRA in their vicinity, so they often have to travel a long distance to access the nearest office. Similar response was recorded in the case of CDA. In the case of NHMP, 46% users opined that they have no concerning offices in their vicinity. About half of the respondents travel between 30-60 minutes to reach NADRA, 30% users travel between 1-2 hours and 10% users travel more than two hours to reach CDA and NHMP respectively. Concerning distance travelled, 68% users travel more than 10 kilometers to reach CDA which is the highest percentage when compared with 44% users in the case of NADRA and 38% users in the case of NHMP. Almost 66% customers of NADRA stated that their required service is being delivered within stipulated time. Within the same category, the percentage was lower in the case of NHMP users. However, 50% service users in the case of CDA narrated procrastination or unwarranted delay. 42% CDA customers were of the view that the delay in the provision of services is due to corruption. In the case of NHMP, 48% service users found unpredictable service delivery time. More than half (56%) of the CDA service users felt intentionally deprived because of inordinate delay in pushing files through the counter to higher authorities for approval. To push files, they have to resort to informal payments or use the right kind of reference. 67% NADRA users found high work load to be the main cause of intentional deprivation; whereas, 46% respondents reported rent seeking to be main factor of deprivation. On the other hand, 40% NHMP customers found “low economic status” a stagnating factor in acquiring the desired level of service, 40% users believed not to have the right kind of political affiliation to be the main factor of intentional deprivation. 75% NADRA service users never reported an incident of deprivation. According to them, reporting an incident would fall on deaf ears. Similar reasons were found in the case of 89% CDA and 67% NHMP service users. Majority users of NADRA and NHMP services were of the view that incidents of withholding services rarely take place. Whereas, more than half (58%) CDA customers experience such incidents more often. According to 84% users, CDA has no dedicated feedback channels to deal with customer complaints. More than 45% users of NHMP and NADRA did observe feedback channels concerning customers’ complaints and feedback. According to 88% users of NADRA, customer payment system was found to be very convenient, followed by 60% users of NHMP. With regard to CDA, 60% service users found payment system to be somewhat convenient. Majority users of all three institutions found the payment system in accordance with expected cost of service. However, in the case of CDA, more than 60% female showed apprehension concerning a mismatch between the cost of service and the available information. In the case of NADRA, 67% users found cost of services as per their expectations followed by NHMP. In the case of CDA, the response was divided between those who found the service in accordance with their expectations and those who are unsatisfied with the services. More than 56% respondents in the case of CDA cited frequent informal payments from service users. Similar experience is found in 40% service users of NHMP. Majority respondents cited lack of internal and external accountability which directly contributes to less than optimal performance. Some respondents cited political pressure which undermines performance. Very few attributed bad performances to understaffing. 6. Findings-II: Results from augmented performance index On the basis of the measurement criteria and weights provided in Annexure-I, ranking of institutional performance is given below8 : 8 See also technical note on scoring method for demand and supply side computation in Annex-II.
  • 18. 13 Table 4: Ranking of Demand Side Indicators Name of Institution(s) Average Score Ranking National Highways & Motorway Police 2.86 2 National Database & Registration Authority (NADRA) 3.249 1 Capital Development Authority (CDA) 1.053 3 The above table elucidates the total scores and ranking of each institution on the basis of the selected measurement areas of demand side variables. In this regard, NADRA scored the highest, i.e. 3.249, and ranked 1st, for having good quality of governance, being least corrupted and no deprivation and discrimination of its service beneficiaries. NADRA is followed by NHMP scoring 2.860. However, CDA accounted for least average score, i.e. 1.053, because its service users felt internally deprived of getting their required services, corruption rate is high, and there is a lack of availability of criteria for customer service feedback. Table 5: Ranking of Supply Side Indicators Institution Average Score Ranking National Highways & Motorway Police 0.845 2 National Database & Registration Authority (NADRA) 0.358 3 Capital Development Authority (CDA) 0.88 1 The aforementioned table of supply side score demonstrates the recorded scores and the ranking of three institutions, based on the selected measurement areas, according to which CDA leads by scoring 0.880 for having full budgets available for all the three years (Fiscal Year 2011-14) in English only, for updated policies, strategies and plans, and for budget execution, if not up-to the mark but at least, 65- 80% and execution of priority programs less than 50%. This is followed by NHMP scoring 0.845 and NADRA is at third position with 0.358 average score. Moreover, it is interesting to note that all the three institutions performed very poorly on accountability and responsiveness, which confirms the intuitive analysis that Pakistan’s public-sector institutions are unaccountable to the public and stakeholders and unresponsive to their needs. Table 6: Final Rank Institution Average Score Ranking National Highways & Motorway Police 0.04576 2 National Database & Registration Authority (NADRA) 0.04619 1 Capital Development Authority (CDA) 0.02232 3 Combining all the scores of demand side and supply side together, it is evident from the above table that NADRA is top-ranked, having average score of 0.04619, followed by NHMP which ranked second the score of 0.04576. However, CDA accounted for least average score which caused its third position in ranking table.
  • 19. 14 7. Conclusion The paper makes an effort to combine supply and demand side accountability measures and apply an augmented performance index framework on select public institutions in Pakistan. This pilot phase (i.e. application of three government entities) helped us in analyzing performance indicators on the one hand, and citizens’ feedback on the other. The supply side indicators are useful as they provide information on how institutions perform core functions. The demand side indicators are useful in knowing how citizens evaluate performance through access and informational indicators. Based on the strengths of this index and learning from the weaknesses such as the actual performance may be political in nature, the scope of the index would be enhanced, by including in it other public-sector institutions as well as the provincial governments. It will be made a comprehensive index, which would work as a mirror for the institutions. It is hoped that some of the learning from our findings could motivate the federal and provincial governments towards more objective measurement of service delivery.
  • 20. 15 References 1. Bertot, JC, McDermott, P and Smith, T 2012, Measurement of Open Government: Metrics and Processes, 45th Hawai International Conference on System Sciences, Hawai 2. Bold, T, Svensson, T, Gauthier, B, Mæstad, Q and Wane, W 2011, Service Delivery Indicators: Pilot in Education and Health Care in Africa. Chr. Michelsen Institute, Norway. 3. Brown, K and Repucci, S 2009, A User’s Guide to Measuring Public Administration Performance. Oslo Governance Centre, United Nations Development Programme, Oslo. 4. Burall, S, Gavelin, K and Wilson, R 2009, Open Government: beyond static measures. Organization of Economic Cooperation and Development, 5. Castro, MF 2011, Defining and Using Performance Indicators and Targets in Government M&E Systems. Special Series on The Nuts & Bolts of M&E Systems (no. 12). The World Bank, Washington, D.C. 6. Cristescu, M, Mihaiu, D and Opreana, A 2010, Efficiency, Effectiveness and Performance of the Public Sector, Romanian Journal of Economic Forecasting, vol. 4. 7. Davies, I 1999, Evaluation and Performance Management in Government. Evaluation, SAGE Publications, London. 8. Dehn, J, Reinikka, R and Svensson, J 2003, Survey Tools for Assessing Performance in Service Delivery, Toolkit for Evaluating the Poverty and Distributional Impact of Economic Policies. World Bank, Washington, D.C. 9. Gokey, C, Parsons, J and Thornton, M 2013, Indicators of Inputs, Activities, Outputs, Outcomes and Impacts in Security and Justice Programming, Department for International Development, London. 10. Khattri, N and Roberts, D 2012, Designing a Results Framework for Achieving Results: A How- To Guide. Independent Evaluation Group, Washington, D.C. 11. Kwakwa, V and Musoni, P 2008, Rwanda: Joint Governance Assessment report. Rwanda Joint Governance Assessment Steering Committee, Kigali. 12. Landman, T 2003, Map-making and Analysis of the Main International Initiatives on Developing Indicators on Democracy and Good Governance. University of Essex – Human Rights Centre, Essex. 13. Lodge, M and Wegrich, K 2013, Setting the Scene: Challenges to the State, Government Readiness and Administrative Capacities, Hertie School of Governance, Berlin. 14. Trivedi, P 2014, India: Performance Monitoring and Evaluation System, Proceedings of Global Roundtable on Government Performance Management, Cabinet Secretariat, Government of India, New Delhi.
  • 21. 16 Annexure-I: (Detailed computation of index) a. Construction of demand side governance index Table 20: Results for NADRA BETA TEST RESULTS FOR NADRA Measurement Area Criteria (C) Criteri a Weigh t (CW) Weighte d Score of Criteria (WSC)= Sum of WSI × CW Indicators (I) Raw Scor e on a Scal e of 1-10 (RS) Indicato r Weight (IW) Weighte d Score of Indicator (WSI)= RS × IW Sum of WSI Quality of Governance No Corruption 1.00 9.5 No informal payments or gifts in lieu of accessing services 10.0 0.50 5 9.5 Transparency in service delivery 9.0 0.50 4.5 Accountabilit y Customer Feedback 1.00 2.3 Availability of criteria for Customer Service Feedback 4.6 0.50 2.3 2.3 Inclusion and Openness Inclusion 0.50 4.7 Not internally deprived during accessing service 9.4 1.00 9.4 9.4 Openness 0.50 0 Complaint Procedure is easy and accessible 0.0 0.50 0 0 No social discrimination 0.0 0.50 0 Effectiveness Service Delivery 1.00 6.969 Availability of services in the vicinity 1.2 0.17 0.204 6.96 9Quality of service 6.6 0.17 1.122
  • 22. 17 Institution and its staff is trustworthy 9.4 0.15 1.41 Payment system is very convenient and transparent 8.8 0.17 1.496 Cost of service is affordable 6.7 0.17 1.139 Accuracy in cost of service 9.4 0.17 1.598 Responsivene ss Mechanism s for facilitating access to information 0.34 2.45956 Designated official nominated under Right to Information Act 2010 4.0 0.34 1.36 7.23 4 Accessibility of source of information 9.8 0.33 3.234 Standards for access to information services 8.0 0.33 2.64 Mechanism s for public grievance redress 0.33 0 Complaint mechanism is available for desired service 0.0 0.34 0 0 Acknowledgeme nt of complaints by relevant department 0.0 0.33 0 Standards for grievance redress services 0.0 0.33 0 Use of ICT tools 0.33 0.066 Availability of online services for complaints, information, 0.2 1.00 0.2 0.2
  • 23. 18 Total Score (Simple Average of WSC) registration, social media 3.249 Table 21: Results for CDA BETA TEST RESULTS FOR CDA Measurement Area Criteria ( C ) Criteri a Weigh t (CW) Weighte d Score of Criteria (WSC)= Sum of WSI × CW Indicators ( I ) Raw Scor e on a Scal e of 1-10 (RS) Indicato r Weight (IW) Weighte d Score of Indicator (WSI)= RS × IW Sum of WSI Quality of Governance No Corruption 1.00 1.7 No informal payments or gifts in lieu of accessing services 3.0 0.50 1.5 1.7 Transparency in service delivery 0.4 0.50 0.2 Accountabilit y Customer Feedback 1.00 0.3 Availability of criteria for Customer Service Feedback 0.6 0.50 0.3 0.3 Inclusion and Openness Inclusion 0.50 2.2 Not internally deprived during accessing service 4.4 1.00 4.4 4.4 Openness 0.50 0 Complaint Procedure is easy and accessible 0.0 0.50 0 0 No social discrimination 0.0 0.50 0 Effectiveness Service Delivery 1.00 2.625 Availability of services in the vicinity 1.8 0.17 0.306 2.62 5 Quality of service 0.0 0.17 0
  • 24. 19 Institution and its staff is trustworthy 2.2 0.15 0.33 Payment system is very convenient and transparent 1.4 0.17 0.238 Cost of service is affordable 5.0 0.17 0.85 Accuracy in cost of service 5.3 0.17 0.901 Responsivene ss Mechanism s for facilitating access to informatio n 0.34 1.3 Designated official nominated under Right to Information Act 2010 2.4 0.34 0.816 3.91 8 Accessibility of source of information 6.4 0.33 2.112 Standards for access to information services 3.0 0.33 0.99 Mechanism s for public grievance redress 0.33 0 Complaint mechanism is available for desired service 0.0 0.34 0 0 Acknowledgeme nt of complaints by relevant department 0.0 0.33 0 Standards for grievance redress services 0.0 0.33 0 Use of ICT tools 0.33 0.264 Availability of online services for complaints, information, registration, social media 0.8 1.00 0.8 0.8 Total Score (Simple Average of WSC) 1.053
  • 25. 20 Table 22: Results for Motorway Police BETA TEST RESULTS FOR Motorway Police Measurement Area Criteria (C) Criteria Weight (CW) Weighted Score of Criteria (WSC)= Sum of WSI × CW Indicators ( I ) Raw Score on a Scale of 1- 10 (RS) Indicator Weight (IW) Weighted Score of Indicator (WSI)= RS × IW Sum of WSI Quality of Governance No Corruption 1.00 7.4 No informal payments or provides gifts in lieu of accessing services 9.0 0.50 4.5 7.4 Transparency in service delivery 5.8 0.50 2.9 Accountability Customer Feedback 1.00 2.4 Availability of criteria for Customer Service Feedback 4.8 0.50 2.4 2.4 Inclusion and Openness Inclusion 0.50 4.5 Not internally deprived during accessing service 9.0 1.00 9 9 Openness 0.50 0 Complaint Procedure is easy and accessible 0.0 0.50 0 0 No social discrimination 0.0 0.50 0 Effectiveness Service Delivery 1.00 6.366 Availability of services in the vicinity 5.2 0.17 0.884 6.366 Quality of service 4.2 0.17 0.714 Institution and its staff is trustworthy 6.4 0.15 0.96 Payment system is very convenient and transparent 6.0 0.17 1.02
  • 26. 21 Cost of service is affordable 8.2 0.17 1.394 Accuracy in cost of service 8.2 0.17 1.394 Responsiveness Mechanisms for facilitating access to information 0.34 1.5 Designated official nominated under Right to Information Act 2010 1.2 0.34 0.408 4.368 Accessibility of source of information 7.0 0.33 2.31 Standards for access to information services 5.0 0.33 1.65 Mechanisms for public grievance redress 0.33 0 Complaint mechanism is available for desired service 0.0 0.34 0 0 Acknowledgement of complaints by relevant department 0.0 0.33 0 Standards for grievance redress services 0.0 0.33 0 Use of ICT tools 0.33 0.726 Availability of online services for complaints, information, registration, social media 2.2 1.00 2.2 2.2 Total Score (Simple Average of WSC) 2.860 Based on the above-mentioned results of all the three selected institutions, main results of the Demand Side Better Governance Index (BGI) is as follow:
  • 27. 22 Table 23: Scoring of Demand Side Institution Average Score National Highways & Motorway Police 2.86 National Database & Registration Authority (NADRA) 3.249 Capital Development Authority (CDA) 1.053 Scoring methods can be found in annexure I (Table 1) b. Construction of supply side index Table 24: Results for NADRA BETA TEST RESULTS FOR NADRA Measurement Area Criteria (C) Criteri a Weigh t (CW) Weighte d Score of Criteria (WSC)= Sum of WSI × CW Indicators (I) Raw Scor e on a Scal e of 1-10 (RS) Indicato r Weight (IW) Weighte d Score of Indicato r (WSI)= RS × IW Su m of WS I Quality of Governance Clarity of institutional vision and mission 0.35 0 Statements are clear, brief and reflect the values, philosophy and culture of the institution 0 0.4 0 0 Vision statement expresses, in sensory terms, the future position or image of the institution 0 0.3 0 Mission statement focuses on one or a few strategic goals 0 0.3 0 Budget Accuracy 0.40 0 Variance between revised budget and budget estimates 0 1.0 0 0
  • 28. 23 Gender Equity in HR 0.25 Proportion of female employees in total working staff 0 1.0 0 0 Investment in Capacity Building 1.00 Training budget as a proportion of total Personnel budget 0 1.0 0 0 Accountabilit y Political Accountability 0.40 0 Timely and complete responses to PAC review queries 0 0.5 0 0 Regular submission of briefings to the Standing Committees of Parliament 0 0.5 0 Accountability to citizen 0.60 0 Publication of Annual Budget on institution’s website 0 0.2 0 0 Publication of Yearbook 0 0.2 0 Majority of tenders follow PPRA rules 0 0.2 0 Use of social accountabilit y tools 0 0.2 0 Citizens’ Charter issued 0 0.2 0 Inclusion and Openness Participatory and inclusive decision making and oversight 0.50 0 Forums on which citizens, private sector, academia or civil society are represented 0 1.0 0 0
  • 29. 24 Proactive public disclosure 0.50 0.75 Publication of updated policies, strategies on website 3 0.5 1.5 1.5 Availability of expenditure data 0 0.5 0 Effectiveness Budget execution 0.40 0 Proportion of released annual budget actually utilized 0 1.0 0 0 Implementation of priority programs 0.40 0.4 Completion level of priority programs 1 1.0 1 1 Strengthening institutional capacities for implementation 0.20 0 Training expenditure as a proportion of total Personnel expenditure 0 1.0 0 0 Responsivene ss Linkage with National/Provinci al priorities and strategies 0.25 0 Explicit linkage of priority programs with the Vision 2025 0 0.5 0 0 Explicit linkage of priority programs with relevant sector policy 0 0.5 0 Mechanisms for facilitating access to information 0.25 0.5 Designated official nominated under Right to Information Act 2010 0 0.5 0 2 Standards for access to information services clearly mentioned 4 0.5 2 Mechanisms for public grievance redress 0.25 0.5 Designated official for public 0 0.5 0 2
  • 30. 25 grievance redress and addressing references from Ombudsmen Standards for grievance redress services clearly mentioned 4 0.5 2 Use of ICT tools 0.25 2.5 Availability of online services for complaints, information, registration, social media 10 1.0 10 10 Total Score (Simple Average of WSC) 0.358 Table 25: Results for CDA BETA TEST RESULTS FOR CDA Measurement Area Criteria (C) Criteria Weight (CW) Weighted Score of Criteria (WSC)= Sum of WSI × CW Indicators ( I ) Raw Score on a Scale of 1- 10 (RS) Indicator Weight (IW) Weighted Score of Indicator (WSI)= RS × IW Sum of WSI Quality of Governance Clarity of institutional vision and mission 0.35 0 Statements are clear, brief and reflect the values, philosophy and culture of the institution 0 0.4 0 0 Vision statement expresses, in sensory terms, the future position or image of the institution 0 0.3 0 Mission statement focuses on one 0 0.3 0
  • 31. 26 or a few strategic goals Budget Accuracy 0.40 2 Variance between revised budget and budget estimates 5 1.0 5 5 Gender Equity in HR 0.25 Proportion of female employees in total working staff 0 1.0 0 0 Investment in Capacity Building 1.00 Training budget as a proportion of total Personnel budget 0 1.0 0 0 Measureme nt Area Criteria ( C ) Criteri a Weigh t (CW) Weighte d Score of Criteria (WSC)= Sum of WSI × CW Indicators ( I ) Raw Scor e on a Scal e of 1-10 (RS) Indicato r Weight (IW) Weighte d Score of Indicato r (WSI)= RS × IW Sum of WSI Accountabili ty Political Accountabili ty 0.40 0 Timely and complete responses to PAC review queries 0 0.5 0 0 Regular submission of briefings to the Standing Committees of Parliament 0 0.5 0 Accountabili ty to citizens 0.60 0.24 Publication of Annual Budget on institution’s website 2 0.2 0.4 0.4
  • 32. 27 Publication of Yearbook 0 0.2 0 Majority of tenders follow PPRA rules 0 0.2 0 Use of social accountabili ty tools 0 0.2 0 Citizens’ Charter issued 0 0.2 0 Inclusion and Openness Participatory and inclusive decision making and oversight 0.50 0 Forums on which citizens, private sector, academia or civil society are represented 0 1.0 0 0 Proactive public disclosure 0.50 2.25 Publication of updated policies, strategies on website 5 0.5 2.5 4.5 Availability of expenditure data 4 0.5 2 Effectivenes s Budget execution 0.40 0.8 Proportion of released annual budget actually utilized 2 1.0 2 2 Measuremen t Area Criteria ( C ) Criteri a Weig ht (CW) Weight ed Score of Criteria (WSC)= Indicators ( I ) Raw Scor e on a Scal Indicat or Weight (IW) Weight ed Score of Indicato Sum of WSI
  • 33. 28 Sum of WSI × CW e of 1-10 (RS) r (WSI)= RS × IW Implementation of priority programs 0.40 0.4 Completio n level of priority programs 1 1.0 1 1 Strengthening institutional capacities for implementation 0.20 0 Training expenditur e as a proportion of total Personnel expenditur e 0 1.0 0 0 Responsiven ess Linkage with National/Provin cial priorities and strategies 0.25 0.75 Explicit linkage of priority programs with the Vision 2025 0 0.5 0 3 Explicit linkage of priority programs with relevant sector policy 0 0.5 0 Mechanisms for facilitating access to information 0.25 1.25 Designate d official nominated under Right to Informatio n Act 2010 6 0.5 3 5 Standards for access to informatio n services clearly mentioned 4 0.5 2
  • 34. 29 Mechanisms for public grievance redress 0.25 1.25 Designate d official for public grievance redress and addressing references from Ombudsm en 6 0.5 3 5 Standards for grievance redress services clearly mentioned 4 0.5 2 Use of ICT tools 0.25 2.5 Availability of online services for complaints , informatio n, registratio n, social media 10 1.0 10 10 Total Score (Simple Average of WSC) 0.880 Table 26: Results for Motorway Police BETA TEST RESULTS FOR Police Measure ment Area Criteria ( C ) Crit eria Wei ght (C) Weight ed Score of Criteria (WSC) = Sum of WSI × CW Indicators ( I ) Raw Scor e on a Scale of 1- 10 (RS) Indicato r Weight (IW) Weighte d Score of Indicator (WSI)= RS × IW Su m of WS I Quality of Clarity of 0.35 0.7175 Statements are clear, brief and reflect the values, 4.0 0.4 1.6
  • 35. 30 Governa nce institutio nal vision and mission philosophy and culture of the institution 2.0 5 Vision statement expresses in sensory terms, the future position or image of the institution 0.0 0.3 0 Mission statement focuses on one or a few strategic goals 1.5 0.3 0.45 Budget Accurac y 0.40 2 Variance between revised budget and budget estimates 5.0 1.0 5 5 Gender Equity in HR 0.25 Proportion of female employees in total working staff 3.0 1.0 3 3 Investm ent in Capacity Building 1.00 Training budget as a proportion of total Personnel budget 0.0 1.0 0 0 Accounta bility Political Account ability 0.40 0 Timely and complete responses to PAC review queries 0.0 0.5 0 0 Regular submission of briefings to the Standing Committees of Parliament 0.0 0.5 0 Account ability to citizens 0.60 0.12 Publication of Annual Budget on institution’s website 0.0 0.2 0 0.2 Publication of Yearbook 1.0 0.2 0.2 Majority of tenders follow PPRA rules 0.0 0.2 0 Use of social accountability tools 0.0 0.2 0 Citizens’ Charter issued 0.0 0.2 0 Inclusion and Participa tory and inclusiv e 0.50 0 Forums on which citizens, private sector, academia or civil society are represented 0.0 1.0 0 0
  • 36. 31 Opennes s decision making and oversigh t Proactiv e public disclosu re 0.50 2 Publication of updated policies, strategies on website 5.0 0.5 2.5 4 Availability of expenditure data 3.0 0.5 1.5 Effective ness Budget executio n 0.40 0.4 Proportion of released annual budget actually utilized 1.0 1.0 1 1 Impleme ntation of priority program s 0.40 0 Completion level of priority programs 0.0 1.0 0 0 Strength ening institutio nal capacitie s for impleme ntation 0.20 0 Training expenditure as a proportion of total Personnel expenditure 0.0 1.0 0 0 Responsi veness Linkage with National /Provinc ial prioritie s and strategie s 0.25 0.75 Explicit linkage of priority programs with the Vision 2025 0.0 0.5 0 3 Explicit linkage of priority programs with relevant sector policy 0.0 0.5 0 Mechani sms for facilitati ng access to 0.25 1.25 Designated Official nominated under Right to Information Act 2010 6.0 0.5 3 5 Standards for access to information services clearly mentioned 4.0 0.5 2
  • 37. 32 informat ion Mechani sms for public grievanc e redress 0.25 1.25 Designated Official for public grievance redress and addressing references from Ombudsmen 6.0 0.5 3 5 Standards for grievance redress services clearly mentioned 4.0 0.5 2 Use of ICT tools 0.25 2.5 Availability of online services for complaints, information, registration, social media 10.0 1.0 10 10 Total Score (Simple Average of WSC) 0.845