Construction of composite index: process & methods

CONSTRUCTION OF COMPOSITE
INDEX
III Seminar
Gopichand, B
III Ph. D.
PALB - 8025
Dept of Agril. Extension

Acknowledgement
Joint Research Centre-European Commission, 2008, Handbook on
constructing composite indicators: methodology and user guide.
OECD publishing.
BANDURA, R., 2008, A survey of composite indices measuring
country performance: 2008 update. New York: United Nations
Development Programme, Office of Development Studies (UNDP/ODS
Working Paper).
MAZZIOTTA, M., & PARETO, A., 2013, Methods for constructing
composite indices: One for all or all for one. Rivista Italiana di
Economia Demografia e Statistica, 67(2), 67-80.

INTRODUCTION
Social science thrives to measure the appropriate human
relations, behavior, growth, impact, comparison…
Different methodologies used
Scale and index are most valid Agricultural Extension
methodologies

Scale v/s Index
Dr. Sivakumar PS, Scientist ICAR

OBJECTIVES
To know the concept of Composite
Index
To understand different normalization,
weighted and aggregation methods in
index construction
To review the related research studies

Concepts
•Objective could be maximization of economic growth
Economic
Social
Environment
R&D Performance
No. of patents
per million of
inhabitants

Dimension
A composite indicator or index is an aggregate of
all dimensions, objectives, individual indicators and
variables used. This implies that what formally defines a
composite indicator is the set of properties underlying
its aggregation convention.
Objective
Indicator

INDEX- Meaning
• A composite indicator is formed when individual
indicators are compiled into a single index on the
basis of an underlying model.
• The composite indicator should ideally measure
multidimensional concepts which cannot be captured
by a single indicator.
• Construction owes more craftsmanship of modeler
than to universally accepted scientific rules for
encoding
- OECD

Importance
Simplify
interpretation
Statistical aggregate
to measure change
Rank Overall
GII
Institutio
ns
Human
capital &
research
Market
sophistic
aton
Business
sophistio
cation
Knowled
ge &
Technolo
gy inputs
Creative
outputs
48 61 60 75 31 55 27 64
General
Public
Benchmarking
country
performance
Big picture for
policy makers
Means for initiating
discussion
Multi -
dimensionality

Importance
• Common public to interpret common trends
• Quantitative or qualitative measure from observed facts that reveal
relative position
• 7,92,000 results of Google scholar till 1.12.2020 from 2016 on
Composite index
• NITI Ayog develops SDG India, Health Index, Indian Innovation
Index, School Education Quality Index…

Characteristics of an Index
❖ Robust – Not thrown off by random or partial
variations
❖ Discriminating – Able to distinguishes between
different case
❖ Efficient – Reasonably easy to build and measure
❖ Effective : Captures what we want to measure

Guidelines for selection of indicators
1. Higher level of measurement - interval or ratio
2. Ordinal level of measurement - try to maintain at least five
continuum
3. If it is not possible to have five continuum, then, use
polychoric correlation coefficients as the input data for PCA
instead of using raw/normalized data
4. Avoid the indicators with nominal level of measurement

Choices made in one step can have important
implications for others: therefore, the developer of
composite indicator has not only to make the most
appropriate methodological choices in each step, but also to
identify whether they fit together well.

Steps in Construction of
Composite Index

Theoretical Frame work
Data Selection
Missing data
Normalization
Weightage & Aggregation
Uncertainty & Sensitivity
Back to real data
Links to other variables
Presentation & Visualization

Data Selection
Missing data
Normalization
Back to real data
Step 1

Step 1. Developing a theoretical
framework
Provides the basis for the selection and combination
of variables into a meaningful composite indicator under a
fitness-for-purpose principle (involvement of experts and
stakeholders is envisaged at this step)
Define phenomenon and its sub components: Badly defined
badly measure
Dimensions& indicators: Importance or desirable than
availability

At the end of Step 1
•A clear understanding and definition of the multi-
dimensional phenomenon to be measured.
•A nested structure of the various sub-groups of the
phenomenon if needed.
•A list of selection criteria for the underlying variables, e.g.
input, output, process.
•Clear documentation of the above.

Data Selection
Missing data
Normalization
Back to real data
Step 2

Step 2 : Data Selection/ Selecting
Variables
Quality of underlying variables: relevance, analytical
soundness, timeliness, accessibility…..
•Lack of relevant data limit developer
•Absence of quantitative data indicators often include qualitative
data from surveys or policy reviews
•Proxy measures: eg: No of people who use computer measured
by proxy access to computer. This should be checked through
correlation and sensitivity analysis
•Type of variable match definition of composite indicator

Data Selection
Missing data
Normalization
Back to real data
Checked the quality of
the available indicators.
• Discussed the strengths
and weaknesses of
indicator.
• Summary table on data
characteristics, e.g.
availability, source, type
(hard, soft or input,
output, process).

Data Selection
Missing data
Normalization
Back to real data
Step 3

Step 3: Imputation of Missing
Data
Missing data often hinder the development of robust
composite indicators.
Reasons for missing data;
Data entry, data extraction and data collection

Step 3: Imputation of Missing
Data
Missing Patterns Could be;
1. Missing completely at random (MCAR). Missing values do not
depend on the variable of interest or on any other observed
variable in the data set.
2. Missing at random (MAR). Missing values do not depend on the
variable of interest, but are conditional on other variables in the data
set.
3. Not missing at random (NMAR). Missing values depend on the
values themselves. For example, high income households are less
likely to report their income.

Methods to deal with Missing
Data
Three general methods for dealing with missing
data;
(i) Case deletion,
(ii) Single imputation or
(iii) Multiple imputation

Data Selection
Missing data
Normalization
Back to real data
• A complete data
set without missing
values.
• Reliability of
imputed method
• Discussed the
presence of outliers
in the dataset
• Documented
selected imputation
procedures and the
results

Step 3(a) Multivariate Analysis
•To identify groups of indicators or groups of
countries that are statistically “similar” and
provide an interpretation of the results.
•To compare the statistically determined
structure of the data set to the theoretical
framework and discuss possible differences.

Method Description Pros Cons
PCA Change in relation &
associated.
Correlated variable to
new set of un correlated
variables
Summarize individual
indicators while
preserving variation
in dataset
Sensitive to modifications in
data
Sensitive to outliers
Small sample problem
Minimize individual indicators
that don’t move with other
Cronbach's
alpha
Most common estimate
of internal consistency of
items in a model or
survey.
Measure internal
consistency
Correlation between
indicators not mean the
contribution of indicator.
Meaningful only when
composite indicator
computed as scale
Cluster
analysis
To group information on
countries based on their
similarity on different
individual indicators.
Different ways to
group countries
Diagnostic
Without loosing
indicators
Purely descriptive, not
transparent if methodology
not clearly explained

Data Selection
Missing data
Normalization
Back to real data
Step 4

Step 4:Normalization
Transforming indicators in order to bring common
scaling.
A number of normalisation methods exist but selection of
technique depends on purpose, data properties, importance
of extreme values, variance in data etc..

1. Ranking
Ranking is the simplest normalization technique.
Useful when no precise data & ordinal data.
Example: The Information and Communications
Technology Index (Fagerberg, 2001)

2. Standardization

3. Min Max Technique
Normalizes indicators to have an identical range [0, 1]
by subtracting the minimum value and dividing by the
range of the indicator values.
will be used for the indicators which have
positive implication with the construct
will be used for the indicators which have
negative implication with the construct

4. Distance to a reference
Measures the relative position of a given indicator
with a reference point.
This could be a target to be reached in a given time frame.
1. External benchmark
2. Average of a group and assigned 1 hence above avg.
performance above 1
3. Reference might be group leader and receives one,
reference country might be unreliable outlier
4. Instead of being centered on 1, it is centered on 0. In the
same way, the reference country can be the average country,
the group leader, or an external benchmark.

5. Categorical scale
Rewarding the best performing countries and penalizing the
worst.
Change in definition of indicator not affects transformed
variable
difficult to follow increases over time.
Exclude large amounts of information about the variance of
the transformed indicators.

Data Selection
Missing data
Normalization
Back to real data
•Appropriate normalization
procedure(s) with reference
to the theoretical framework
and to the properties of the
data.
• Made scale adjustments, if
necessary.

Data Selection
Missing data
Normalization
Back to real data
Step 5

Step 5 (a): Weightage
The normalised variables are weighted using
various weighting techniques. Most important techniques
are as follows;
1. Iyengar-Sudarshan Method
2. Unobserved Component Analysis
3. Principal Component Analysis and

Iyengar and Sudarshan (1982) linked the weight to
variance across the indicators. More precisely, they
postulated that

The choice of the weights in this manner would
ensure that large variation in any one of the indicators
would not unduly dominate the contribution of the rest of
the indicators and distort the overall ranking of the
countries (Iyengar and Sudarshan, 1982).

2. Principal Component Analysis
The steps include;
• checking the correlation structure of the data.
• identification of certain no of latent factors.
• rotation of factors.
• construction of weights from the matrix of factor
loadings after rotation
The weights following this method are computed
using SPSS

3. Unobserved Component Model
This method is applied if a set of sub-indicators are
out to measure an unknown component. The weight
obtained will be set to minimize the error in the
composite index
This method resembles with regression analysis but
the main difference lies in the dependent variable,
which is unknown and hence named as the
Unobserved Component Model

3. Unobserved Component Model
Using the method, the weight Wj corresponding to
the j thindicator is given by,
Where, j is the standard deviation of the values of
the j th indicator. Thus, the indicator with less precision
lower will be the weight assigned to that indicator.

4. Analytic hierarchy process (AHP)
• Considers experts opinion
• Complex problem into hierarchy or simple groups
• Experts evaluate the elements two at a time
comparison: various combinations

Step 5 (b): Aggregation
The Weighted indicators were aggregated to form a
composite index, The different methods of aggregation
are ;
1. Linear aggregation
2. Geometric aggregation and
3. Weighted Displaced Ideal Method

1. Linear aggregation
It is the summation of weighted and normalized sub-
indicators

2. Geometric aggregation
In case of geometric aggregation of sub-indicators we
use the original data set instead of the normalized values
and for main indicators we use normalized values.
weight
normalised value of the indicator

2. The Weighted Displaced Ideal Method
The weighted displaced ideal (WDI) method is based
on the concept that the best system should have the least
distance from the ideal system. The WDI method can
then be formulated as ;

Composite Indices Formed by Linear Aggregation Method Using Different
Weighting Techniques

Composite Indices Formed by Geometric Aggregation Method Using Different
Weighting Techniques

Data Selection
Missing data
Normalization
Back to real data
Step 6

Step 6 Uncertainty & Sensitivity
These analysis can help gauge the robustness of
the composite indicator and improve transparency.
Uncertainty analysis focuses on how uncertainty in
the input factors propagates through the structure of the
composite indicator and affects the composite indicator
values.
Sensitivity analysis assesses the contribution of the
individual source of uncertainty to the output variance.

Step 7 Back to Data
•To check for correlation and causality (if possible).
• to identify if the composite indicator results are overly
dominated by few indicators and to explain the relative
importance of the sub-components of the composite
indicator.
• The tools such as Path Analysis and Bayesian Networks
(the probabilistic version of path analysis) could be of
some help in studying the many possible causal
structures and removing those which are strongly
incompatible with the observed correlations.

Quality Dimensions Of Composite
Index

Limitations of Composite Index
• Non-availability of data on individual indicator
• Send misleading policy messages if poorly constructed or
misinterpreted.
• Invite simplistic policy conclusions.
• May be misused, e.g. to support a desired policy, if the
construction process is not transparent and/or lacks
sound statistical or conceptual principles.
• The selection of indicators and weights could be the
subject of political dispute.
• May lead to inappropriate policies if dimensions of
performance that are difficult to measure are ignored.

Study 1: Work styles, Best practices and Productivity
of Agricultural Scientists
Sudipta Paul
(2012)
Research Design: Ex-post facto
Study Area: IARI and
CSAUA&T
Sample size: 220

Research Productivity Index
Steps Followed are;
1. Operationalization of variables
2. Measurement and scoring of indicators
3. Assigning weightage to independent variables: Mean
weightage
4. Derivation of formula
5. Assessing validity and reliability of the measurement
instrument : test-retest

Research Productivity Index
Indicator M.W. Sub.I M.W.
Scientific publication
Product development
Research activities undertaken
Teaching activities undertaken
Extension activities undertaken
Awards received
Recognition achieved

Study 2: Development Programmes and their
Impact on Farmers’ Welfare in Kerala
State
Darsana
(2018)
Research Design: simulated
research design with control-
randomisation
Study Area: Palakkad district,
Kerala
Sample size: 240

Study 3: A Critical Analysis of Vulnerability and
Adaptation Strategies for Climate Change Among
the Farmers of Hyderabad Karnataka Region
Shanabhoga M B
( 2019)
Research Design: Descriptive
and Diagnostic research design
Study Area: Hyderabad
Karnataka Region
Sample size: 180

Vulnerability to Climate Change
Index
Steps Followed are;
1. Identification of dimensions
2. Collection of indicators
3. Relevancy weightage
4. Computation of scale values: Centile
5. Measurement procedures of indicators
6. Reliability & validity
7. Calculation
VI= (Exposure Index + Sensitivity Index)- Adaptive capacity

Vulnerability to Climate Change
Index
Normalization method used : Min – Max Method
Weighted method : Linear Aggregation
Quartile analysis was carried out to classify districts in
four groups indicating ‘Very high', ‘High', ‘Medium' and
‘Low' degree of exposure, sensitivity, adaptive capacity and
vulnerability.

Conclusion
Composite index serves as an handy tool in assessing
the overall performance of any selected dependent variable
on which the particular index was constructed. The
irreplaceable contribution of composite index in policy
decisions and simplicity in representation makes it front-
runner in measurement of socio-economic development.
However, the universal validity of index depends on
objective of the measurement, dimensions selected and
transparency of process

Construction of composite index: process & methods

More Related Content

What's hot (20)

Similar to Construction of composite index: process & methods (20)

Recently uploaded (20)

Construction of composite index: process & methods