Performance Assessment of Agricultural Research Organisation Priority Setting by using Balanced Score Card (BSC) & Analytic Hierarchy Process (AHP) technique

IOSR Journal of Business and Management (IOSR-JBM)
e-ISSN: 2278-487X, p-ISSN: 2319-7668. Volume 17, Issue 12 .Ver. II (Dec. 2015), PP 62-74
www.iosrjournals.org
DOI: 10.9790/487X-171226274 www.iosrjournals.org 62 | Page
Performance Assessment of Agricultural Research Organisation
Priority Setting by using Balanced Score Card (BSC) & Analytic
Hierarchy Process (AHP) technique
Morla. Raja Krishna Murthy
Outcome & Impact Assessment Specialist,
Programme Performance and Outcome Monitoring Unit (PPOMU),
Finance Department, Government of Odisha (GoO), Bhubaneswar, Odisha
Abstract: Assessment of research priority setting in agriculture is important in planning for organizational
development. The potential of agricultural research, education, and training has attracted considerable
attention in many developing countries. However, without clear focus and coherent research policies the public
sector is unlikely to reap the full benefits of the new technology. This paper reports on a priority-setting exercise
for agricultural research in National Academy of Agricultural Research Management (NAARM) - ICAR, based
on the Analytic Hierarchy Process (AHP). Three main functions performance evaluation of R&D, Training and
Education have been prioritized using Balanced Score Card (BSC) model that consists of three hierarchies to
evaluate their potential contribution to organizational development objectives, chances of research success, and
expected adoption rates, respectively. The final scoring shows a clear grouping of the research alternatives. The
conceptual framework is to develop decision criteria. All proved to be useful tools to improve the priority setting
process.
Keywords: analytic hierarchy process, agricultural research, education, training, priority setting, public
sector, research management.
I. Introduction:
Priority setting is as critical step for an efficient research planning. This could help the decision making
process of what to invest, where to invest, when to invest, and how much to invest; in other words, towards an
optimal agricultural science resources allocation (Braunschweig, 2000). A formal approach for priority setting
could furthermore improve efficiency and credibility of decision makers. However, several priority settings in
the public sector have not undergone systematically (Setboonsarng et al., 1991).
Over the years, several frameworks have been developed to address the management of organizational
assets, both tangible and intangible. For tangible Balanced Scorecard (BSC) and the Intangible Assets Monitor
(IAM) and the Skandia Navigator can mainly used to framework. Kaplan and Norton (1996) defined Balanced
Scorecard (BSC) as a multidimensional framework for describing, implementing and managing strategy at all
levels of an enterprise by linking through a logical structure, objectives, initiatives and measures to an
organization‟s strategy. The resulting scorecard provides the details of an enterprise view of an organization‟s
overall performance. It complements the financial measures with other key performance indicators around
customer perspectives and internal business process, and around organizational growth, learning and innovation.
It may be noted that the BSC is not a static list of measures, but rather a logical framework for implementing
and aligning complex programs of change, indeed for managing strategy focused organizations. In summary, a
scorecard is to be used to facilitate the translation of strategy into action.
The four basic perspectives of the organizational BSC are: 1. Financial; 2. Customer; 3. Internal Business
Process; 4. Learning & Growth.
1. Financial: This perspective typically relates to profitability - It is measured for instance by plan allocation
of total budget, non-plan allocation of total budget, plan expenditure of total budget, non-plan expenditure
of total budget, revenue generation on total budget. Finance is essential in summarizing the economic
consequences of strategy implementation.
2. Customer: This perspective includes several core or genetic measures of successful outcomes from the
organizational strategy, like for instance, international consultancy projects, national consultancy projects,
on-going research projects, R&D projects time over run, R&D projects cost overrun. It identifies the
customer and research segments in which the business desires to compete.
3. Internal processes: This perspective focuses on the internal process that will have the greatest impact on
research projects on achieving an organizations financial objective. It is measured for instance staff
strength, frequency of review meetings, books/bulletin/Manual/Policy papers, collaborations with national

Performance Assessment of Agricultural Research Organisation Priority Setting by using …
institutes, collaborations with international institutes. It identifies the critical internal processes for which
the organization must excel in implementing its strategy.
4. Learning and Growth: This perspective identifies that the organization has to build and manage to create
long-term growth and improvement through people, systems and organizational procedures that the
organization must build to create long-term growth and improvement. It is a measure for instance scientists
participation in seminars/ workshops/symposia/conference/meeting/foreign visits, papers presented per
scientist, programs on new skills, awards and recognitions, scientists published articles in journals.
The execution of this strategy is then monitored through an internal performance measurement
framework with a set of goals; drivers and indicators (both lag and lead types) grouped into each of the four
perspectives. The other key objective of a BSC is to tell the story of the organizations strategy.
Lead vs. Lag indicators:
For developing a Balanced Scorecard (or any other performance management system), it is
recommended to use a combination of Leading and Lagging Indicators. Kaplan and Norton call these
“Performance Drivers” and “Outcome Measures”.
The idea is that Lagging Indicators without Leading Indicators tell you nothing about how the
outcomes will be achieved, nor can you have any early warnings about being on track to achieve your strategic
goals.
Similarly, Leading Indicators without Lagging Indicators may enable you to focus on short-term
performance, but you will not be able to confirm that broader organizational outcomes have been achieved.
Leading Indicators should enable you to take pre-emptive actions to improve your chances of achieving strategic
goals.
Leading Indicators are often captured at the level of individual processes, whereas Lagging Indicators
may be the result of changes in a number of Leading Indicators. So, a process cycle-time or error rate might be
Leading Indicators, measured at the process level and Customer Satisfaction would be a Lagging Indicator,
measured at the organization level.
If you are measuring “activity” (i.e. at a process level), it is more likely that you are using Leading
Indicators. The closer you move to process inputs and activities, the closer you get to Leading Indicators of
downstream, (Lagging) performance. If you are measuring aggregated effects, or outcomes, at an organizational
level, you are more likely to be using Lagging Indicators.
Remember, the overall purpose of selecting metrics is to enable you to track performance towards your
goals. So, you should aim to identify and then control those metrics that drive you towards your ultimate goals.
Lead and Lag relations among indicators of performance:
Source: Author
The above figure represents the measures of final outcome of management plans and their execution.
Research Priority Settings:
NAARM has made a strong commitment to set clear and rational agricultural research priorities and to
translate those priorities into resource allocation decisions. Fulfilling this commitment is crucial to the institute‟s
future success. As the resources available for agricultural development become increasingly scarce, it is
essential that the institute can clearly and quantifiably demonstrate the value of agricultural research in
promoting national agricultural development objectives, particularly in comparison to competing claims for
public and contributor resources. Further, the clear and rational presentation of priorities will allow NAARM to
take a more pro-active approach in soliciting contributor support for areas identified as vital to national
agricultural development efforts. Such a presentation will, in turn, increase the institute‟s control over its
research agenda and further increase the flow of innovations with a demonstrable impact on agricultural
development. The plan of action is formulated with recommendations on how NAARM should processed to
develop institutional structures, criteria, methods and information based within an integrated priority setting
framework.
Institutional structure:
The academy is changed with identifying the long range research needs in various areas, including
agriculture, training, education, and assessing technology transfer linkages between the private sector and public

universities. The competitive edge in R&D though closer interactions between academic institutions and the
private sector in the setting of research priorities and most likely in making available adequate support of funds.
We recognized that our competitive strength in national and international level depends on the quality and
effectiveness of a carefully planned research and development, communication system. The academy is
confident that the efficiency of our resource utilization and effectiveness of training programs will benefit as a
result. It is important to review existing programs and make sure that there is a process for applying knowledge
and technology already available as it is to set priorities for future research.
Over time period the demands on agricultural research and education system have been fast changing.
Priority setting in any system requires professional expertise. The agricultural system is now faced with the task
of not only increasing the productivity of selected crops but also to address issues such as increasing disparity in
agricultural growth, maintaining and enhancing the quality of natural resource base and reducing environmental
degradation, poverty alleviation, emerging trade regimes, etc. These developments are useful for rationalization
of allocation of current and future resources for enhanced research efficiency. Institutionalizing a systematic
analysis of agricultural research priorities and integrating it with an effective monitoring and evaluation (M&E)
system, therefore, holds a key to make the system efficient and effective.
Strategic research prioritization techniques:
Agricultural research prioritization is a process of ranking a set of potential research activities so that
the scarce research resources may be allocated to these activities as per the ranking. This process is undertaken
to align the resource allocation with the national objectives and to improve the effectiveness and efficiency of
the scarce research resources. It also aims to improve the transparency and accountability in the research system
with respect to resource allocation, monitoring and evaluation.
Foundation of Research prioritization:
1. The primary objective of prioritizing options is to allocate scarce research resources efficiently, i.e., to
maximize the benefit for the society expected from public expenditures for research.
2. A good priority setting approach will allow information on national policy objectives to flow downward.
3. Information on researchable problems to flow from the bottom up to influence higher levels of priority
setting.
4. Institutionalization of priority assessment mechanism at different levels would enhance transparency,
ownership and accountability, thereby contributing to the effectiveness and efficiency of the research
system.
5. Systematic and objective approaches lead to a more efficient resource allocation than the informal
mechanisms such as collective judgment or beliefs of individual scientists.
6. There‟s always a higher demand for research resources like scientific and technical manpower, capital
assets and contingencies then their availability.
Research prioritization being a relatively interesting concept, there are apprehensions among some scientists and
researchers about it. The main arguments against research prioritization are summarized.
1. Research is an uncertain economic activity with unpredictable outcomes. Any pre-judgment of research
activity is not fair. If scientists have to convince their peers or researchers about the desirability of the
research activities they wish to take up, they may tend to shift from basic and upstream research to applied
and downstream research.
2. The informal mechanisms used for research resource allocation in the past have been quite effective in
meeting the national priorities.
3. The best research prioritization approaches cannot guarantee a close correspondence between the ex-ante
assessment and ex-post impact of any research activity.
4. The methodologies suggested by social scientists and economists are only a plan to take away the decision
making powers and prerogatives of the researchers.
Despite such apprehensions and objectives voiced by some scientists and researchers, the priority
assessment procedures are finding increasing application and acceptance in the western countries. The National
Agricultural Research system (NARS) has decided to make use of the funding under National Agricultural
Technology Project (NATP) for Organizational and Management (O&M) reforms to improve the transparency,
accountability and efficiency of the system. The process of priority assessment, monitoring and evaluation
(PME) are sought to be strengthened through sensitization, capacity building and institutionalization of the
necessary procedures. Priority assessment at different levels of research planning is the first step in ushering the
O&M reforms in the NARS.

Research Resources allocation and Research Prioritization:
Research Resources allocation and Research Prioritization processes are interlinked. In every research
resource allocation, there may be an implicit prioritization, whether it is made explicit or not. In general, a
method of percentage increase is followed in allocating resources in the government departments. If the research
organization gets a ten per cent higher allocation in the current year over that of the previous year, the tendency
is to enhance the allocations to different institutes or research programs by the same ten percent. However, some
marginal variations can be made here and there as exceptions. But this method of allocation places immense
faith on the past resource allocation decisions. Another reason for this method of allocation is that the normal
increases in the committed expenditures like salaries, labour charges etc. are to be taken care of.
Analytical Framework for Assessment of Agricultural Research Priority Setting:
The Balanced Score Card (BSC) framework has been used effectively in the past in research
development and academic development studies as it provides a means to quantify characterize structure and
analyze strategic measurement systems. For this reason, the framework is also adopted in this study to measure
the performance of a research organization system through appropriately designed indicators and indices.
Balanced Score Card framework:
The Balanced Score Card (BSC) framework is scalable, and enables better judgment, insight, and
accountability in planning for organizational development. BSC translates the vision and strategy of a business
unit into objectives, measures in the financial, customer, internal business process, learning and growth
perspectives. Balance between four capital assets, and the various components of each type of asset, is also
important, because minimum levels of one asset may be necessary to effectively make use of another.
Fig 1. Balanced Scorecard strategic framework for R&D in NAARM
Based on this research, it has been found that a Balanced Scorecard framework using about 20-25
measures is the usual recommended best practice. We selected 20 measures for Research and Development in
NAARM, Hyderabad. The following table presents the selected measures under four different perspectives.
Table 1. Research Performance measure through BSC technique
Perspective No. of measures Weight
Financial 5 33.83 %
Customer 5 17.99 %
Internal Business Process 5 26.42 %
Learning and Growth 5 21.75 %
Total 20 100 %

Fig 2. Web-based Research Performance on 4 key perspectives
The above figure represents the need for balancing the number of measures, because the process
perspective is the primary domain through which organizational strategy is implemented. According to goal
orientation the measurable indicators considered in BSC, the financial aspects covering (33.83%), internal
business process (26.42%), learning and growth (21.75%) and customer perspective (17.99%) performance
weighted. There is a big gap observed in between customer and organizational learning & growth is about to 30-
40 per cent, which gives an opportunity to expand the research development in these two areas in NAARM.
Future focus should not be limited to the finance and internal business process and it could be concentrated on
other two aspects also.
Strategy Mapping:
Fig. 3. The strategy map represents how an organization creates value
The strategy map brings together the strategic objectives of organization to illustrate causal linkages
within their relationships. Within the business application, strategy maps and the Balanced Scorecard
approaches have been proven to be effective tools of business communication and strategic management in
aligning and integrating the strategic goals of various levels within an organization (Chan, 2009: 349-363;
Paladino, 2007b:38; Wu & Liu, 2010: 27-47).
The strategy map allows senior scientist and managers to see how attaining strategic objectives at the
departmental or employee level assists an organization to achieve better internal business processes, customer
satisfaction and ultimately its financial objectives. A strategy map identifies the specific capabilities in the
organizations intangible assets such as human capital, information capital and organization capital that are
required to deliver exceptional performance in the critical internal process. The given analysis indicates that the

strategy map is a critical component in the design of performance measurement systems based on the Balanced
Scorecard model. The strategy map has turned out to be as important innovation as the original Balanced
Scorecard model itself. The visual representation of an organizations strategy is both natural and powerful, as
one sees the progression of the interrelationships of perspectives towards meeting organizational objectives.
The strategy map acts as an instrument in understanding the relationship of perspectives better, as one
can easily study the map critically. A strategy map gives a clear direction and shows the linkages between all the
critical elements in each corporate perspective of the Balanced Scorecard model.
Conceptual Framework of NAARM Performance:
The measurement of NAARM performance requires measurement along many dimensions. In fact,
there is no consensus about what are the key indicators for catch-up in the multidimensional ICAR institutes
structure. We need an indicator system that responds clearly to certain general principles such as relevance and
feasibility in relation to the available time and resources (Oakes, 1989). We need an indicator system that is
reliable and provides accurate information and allows comparison across time periods and among research
organizations. Recognition is the need to measure multiple dimensions of NAARM has finally led to the attempt
to get some agreement on an indicator system to enlighten national agricultural research policies. The present
paper contributes by establishing classification criteria for some of the existing indicators on which there is no
widespread agreement. The focus in the analyses is on achieving a degree of consensus among a group of
experts about which are the best/most appropriate indicators to analyze NAARM performance. To achieve
agreement about what indicators are the key ones for evaluating performance, the designed set of quantitative
and qualitative indicators used to evaluate. We compiled a comprehensive list of indicators to evaluate NAARM
activities, with special attention to Indian Agricultural system. The list of indicators was constructed on the basis
of whether they were informative about:
 Research, training and educational knowledge transfer (the three main missions of NAARM);
 Resources, processes or results (type of indicator);
 Level of importance in an evaluation process.
II. Methodology:
Identify which from a group of 20 indicators used for the evaluation of NAARM are most appropriate:
 To analyze NAARM missions (research, training and education knowledge transfer)
 To indicate the typology of each indicator (input, process or output indicator);
 To score their relevance for the evaluation process.
In this study, data was analyzed using Analytical hierarchy process (AHP) which continues to be the most
highly regarded and widely used decision making process.
NAARM Vision and Mission:
 Vision: NAARM will be India‟s premier institute in agricultural research management that enables NARS
adapt to change through continuous innovation.
 Mission: To enhance leadership, governance and innovation capacities of NARS through capacity
strengthening, education, research, consultancy and policy support.
Balanced Score Card (BSC):
The Balanced Scorecard (BSC) is a method that could easily connect directly to various performance
measures (criteria) of a BSC with the stated goals and objectives of NAARM. Specifically, it explains in great
detail how the multi criteria method of Analytical Hierarchy Process (AHP) could practically facilitate this
connection. It analyses how an organization could arrange the various performance criteria in such a way that
could be capable of controlling its stated goals and objectives through the implementation of its strategy.
The first step is to connect two methods of BSC and AHP and the second step is the formation of the
proposed framework, which actually facilitates the formal and quantitative links between the organization‟s
stated performance criteria and its overall strategic planning process: its mission, goals, objectives, and the
specific strategy it follows for the attainment of these goals and objectives.
Balanced Scorecard should have an approximate mix of outcomes (covering indicators) and
performance drivers (leading indicators) of the organizational development strategy. BSC translates the
organizational development strategy into a linked set of measures that define the long term strategic objectives,
as well as the mechanisms for achieving those objectives.
Analytical Hierarchy Process (AHP) allows decision makers to model a complex problem in
hierarchical structure showing the relationships of the goal, objectives (criteria), sub objectives, and alternatives.
Thus, a typical hierarchy consists of at least three levels, the goals, the objectives, and the alternatives.

The proposed BSC – AHP framework:
Table 2 below describes the steps required to link the performance measures of the BSC to (1) the
overall mission (or overall Goal) and the objectives (or strategic challenges) of the organization, and (2) a
specific strategy designed to help the organization achieve its objectives and mission. This process begins with
the construction of an AHP model that directly links, as we shall analyze shortly, the organization‟s
performance measures to its objectives and mission (step 1). After creating this linked hierarchy, we then use the
AHP to determine the relative weight (or importance) of each individual performance measure to the
organization‟s ability to succeed in its objectives and mission (step 2). The relative weights of the performance
measures determined in step 2 then allow us to develop an index, which collectively uses the key performance
measures to track the organization‟s performance (step 3).
Table 2. Steps required linking the performance measures of the BSC
Steps Task
Step 1 Construct AHP model linking organization mission and objectives to BSC
Step 2 Use AHP to determine the „weights‟ or relative importance of individual key performance measures.
Step 3 Compute the index of the performance measures to monitor overall organization‟s performance.
Prioritize operations relative to strategic plan:
Table 3. Balanced Scorecard indicators for R&D in NAARM
S.No. Research Indicator Calculated priority
weight
Measures/
weightage
Financial (F)
1 Plan allocation % of total budget 100% 5
2 Non plan allocation % of total budget 100% 5
3 Plan Expenditure % of total budget 99.99% 5
4 Non plan Expenditure % of total budget 99.95% 5
5 Revenue generated % of total budget 21.30% 1
Customer (C)
1 % of International consultancy projects 7.69% 1
2 % of national consultancy projects 68% 4
3 No. of ongoing research projects (Externally-aided projects) 41.00% 3
4 % of R&D projects time over run 7.14% 1
5 % of R&D projects cost over (Nil) 100% 5
Internal Business Process (IBP)
1 % of staff filled 74.24% 4

2 Frequency of review meetings 50% 3
3 Books/bulletin/Manual/ Policy paper 80% 5
4 Collaborations with national institutes 50% 3
5 Collaborations with international institutes 75% 4
Learning & Growth (L&G)
1 % of scientists participation in seminars/workshops/
symposia/conference/meeting/Foreign visits
68.97% 4
2 % of papers presented per scientist 30% 3
3 % of programmes in new skills 67.00% 4
4 Awards and recognitions 60.00% 4
5 % of scientists published articles in journals 44.83% 3
The performance of research indicators and priority weight was done through a 5-point rating scale in which 1
stands for Poor performance (Upto 20%), 2 for Fair (21% - 40%), 3 for Good (41% - 60%), 4 for Very Good
(61% - 80% ), and 5 for Excellent (81% - 100%).
AHP Pair wise comparison matrix:
Goal: Prioritize valuation approach of NAARM performance assessment
Step I
A three –level hierarchy to link the performance measures of the BSC to the organization‟s mission and
objectives.
1. The organization‟s mission or overall goal
2. The organization‟s objectives or strategic challenges (Hamel and Prahalad, 1996: 149)
3. The key performance measures of the BSC
Priority Weights of Performance Criteria:
j
i
F C IBP L&G GM W
F 1 8 9 5 4.3559 0.6637
C 1/8 1 6 4 1.3161 0.2005
IBP 1/9 1/6 1 7 0.6000 0.0914
L&G 1/5 1/4 1/7 1 0.2907 0.0443
TOTAL 6.5627 1.0000
1. We prioritize five research and control gaps
2. Set up a table - each pair wise comparison table – each element is paired with every other element
3. We need only fill in the yellow shaded area
n(n-1)/2 = 5(4)/2 = 10
Number of gaps 1 2 3 4 5
number of comparisons 0 1 3 6 10
Why Analytical Hierarchy Process (AHP):
The AHP, developed by the mathematician, Thomas L. Saaty (1980), is a practical and effective
method for solving multi-criteria decision problems. It is a flexible and powerful tool for handling both
qualitative and quantitative multi-criteria problems. Its decision model is based on structuring the elements of
the observed problem in terms of how the alternative solutions influence decision criteria, satisfaction of which
will show how much particular solution contributes to the accomplishment of the main objective of a decision
problem. The AHP uses a multi-level hierarchical structure of objectives, criteria, sub-criteria and alternatives.
The typical problem examined by the AHP consists of a set of decision criteria. The pertinent data are derived
by using a set of pair wise comparisons. These comparisons are used to obtain the weights of importance of the
decision criteria and the relative performance measurements of the alternatives in terms of each individual
decision criterion. The hierarchical organization structure of the AHP arranges a decision problem in clusters at
different levels; higher level elements transmit influence to lower ones, or lower level elements contribute to the
functioning of the higher level ones (Guo and He, 1999).
AHP as a “structured technique for organizing and analyzing complex decisions”. It further mentions
that AHP “has particular application in group decision making, and is used around the world in a wide variety of
decision making situations, in fields such as government, business, industry, healthcare and education”. The
AHP is a multi objective, multi criteria decision making approach which employs a pair wise comparison
procedure to arrive at a scale of preferences among a set of alternatives, it is necessary to break down a complex
unstructured problem into the component parts into a hierarchic order. It is a process involving discussions,

learning and checking and also a powerful tool to support good decision making. Unique feature is possibility
to calculate a measure of inconsistency for each set of judgments. The property enables to identify errors and to
revise adjustment.
Principles of AHP:
 decomposition of a complex unstructured problem
 comparative judgments about its components
 synthesis of priorities derived from the judgments
The goal, criteria, sub criteria and options (projects) identified, judgments are made to compare the all
criteria in pairs with respect to goal and options (projects) with respect to each of the criteria at the next higher
level. Relative comparisons are based on hard data as well as on the intuition, experience and expertise of
decision-makers. AHP explicitly allows for subjective judgments, recognizing their legitimate role in ex ante
analysis. Once the verbal judgments are made they are translated into numbers by means of a fundamental scale
presented in Table 4.
For qualitative data such as preference, ranking and subjective opinions, it is suggested to use scale 1 to 9.
Table 4. Fundamental scale for AHP
Intensity of Value Interpretation
1 Requirement i and j are of equal value
3 Requirement i has a slightly higher value than j
5 Requirement i has a strongly higher value than j
7 Requirement i has a very strongly higher value than j
9 Requirement i has an absolutely higher value than j
2, 4, 6, 8 These are intermediate scales between two adjacent judgments
Reciprocal scale i.e. 1/3, 1/5,
1/8 etc If requirement i has a lower value than j
 In the yellow highlighter cells, type in the intensity value number
 If "i" is higher than "j", type the integer
 If "i" is lower than "j", type an equal sign follows reciprocal (=1/5)
Table 5. Performance made on 1- 9 scale
AHP scale of importance for comparison
pair (aij)
Explanation Numeric
Rating
Reciprocal
(decimal)
Extreme Importance The evidence favoring one activity over another is of the
highest possible order of affirmation
9 1/9 (0.111)
Very strong to extremely 8 1/8 (0.125)
Very strong Importance An activity is favored very strongly over another; its
dominance demonstrated in practice
7 1/7 (0.143)
Strongly to very strong 6 1/6 (0.167)
Strong Importance Experience and judgment strongly favor one activity
over another
5 1/5 (0.200)
Moderately to strong 4 1/4 (0.250)
Moderate Importance Experience and judgment slightly favor one activity over
another
3 1/3 (0.333)
Equally to moderately Two activities contribute equally to the objective 2 1/2 (0.500)
Equal Importance 1 1 (1.000)
Reciprocals of above (If activity i has one of
the above non-zero numbers assigned to it
when compared with activity j, then j has the
reciprocal value when compared with i)
A reasonable assumption
If the activities are very close May be difficult to assign the best value but when
compared with other contrasting activities the size of the
small numbers would not be too noticeable, yet they can
still indicate the relative importance of the activities.
1.1 – 1.9
The paired comparison scale between the comparison pair (aij) of two items (items i and item j) is as follows:
(item i) 9-8-7-6-5-4-3-2-1-2-3-4-5-6-7-8-9 (item j)
The preference scale for pair-wise comparisons of two items ranges from the maximum value 9 to 1/9 (0.111 in
decimal from). Let aij represent the comparison between item-i (left) and item-j (right). If item-i is 5 times
(strong importance) more important than item-j for a given criteria or product, then the comparison aji = 1/aij =
1/5 (0.200) or the reciprocal value for the paired comparison between both items.
To find out corresponding sets of weights, the pair-wise comparison is done where element i with element j is
placed in the position of aij in the reciprocal matrix A as shown below:

A= a11 a12.. a1n
a21 a22.. a2n
.
.
.
.
an1 an2.. ann
The consistency ratio is determined and sensitivity analysis can be done by AHP computer programme
known as “Expert Choice” (Forman and Satty, 1991).
In developmental context to prioritize options, first we identify problems and opportunities that deal
with the causes and consequences. The next step is screening of options that examines the proposed solutions
against the relative impact of each of them in terms of income generation and improved resource development,
chances of research success, and expected adoption rates, respectively. It is most effectively organized in a
stepwise manner by following 3 steps one make sure to arrive at the best decision in the best manner.
Step 1 : Defining objectives and options
A sector or sub sector analysis followed by defining research objectives e.g. at national level;
objectives have often been grouped in 4 categories i.e. income generation and improved resource development,
chances of research success, and expected adoption rates, respectively. At research level, efficiency of research
technology management, address emerging concerns in agriculture, facilitate organizational effectiveness,
enhance administrative and financial management in the system, forge and strengthen partnerships, linkages and
networking at regional, national and global levels may be the research objectives. The research alternatives must
be identified and defined by analyzing constraints and opportunities. To assess the research alternatives the
evaluation criteria need to be formulated. These criteria normally correspond with measurable indicators for
research objectives. When there are several criteria, criteria weighing may be required.
Step 2 : Choosing right method
With limited resources, research should clearly be taken only on those alternatives that contribute most
to the research objectives. To choose critically, one should estimate how the research alternatives perform on
the measurable criteria, often the initial rank order is submitted to sensitivity analysis.
Step 3 : Preparing for implementation
Priorities are set in order to be implemented. The priorities thus need to be transformed into a resource
allocation plan. Finally the process followed and results obtained into here should be submitted to the wider
validation.
Procedure of prioritizing options at organization level:
Following steps to be followed to prioritize options and allocate resources at organization level
1. Preferential ranking of problems of the research, use analytical tools.
2. Selected problem examined for Problem-Causal analysis, identify intervention points.
3. Formulate research questions based on research hypotheses.
4. Identify research alternatives, resource allocation plans.
5. Generate criteria to evaluate options, check relevancy and applicability of these criteria, find out indicators
for these criteria which can be measured either using data, experiences of other countries, expertise and
intuitions of team members.
6. Evaluate options using analytic methods, which can be understood by following hypothetical example of
Analytic Hierarchy Process (AHP).

Step I: Decomposition of unstructured problem into hierarchy level
Step II: Judgment matrix for pair wise comparison of criteria (C) with respect to goal
Finance - C1, Customer - C2, Internal Business Process - C3, Learning & Growth - C4
From the Introduction that the organization has four factors in mind: finance, customer, internal
business process, learning and growth; F, C, IBP, L&G respectively. The factors chosen should be independent,
as required by Saaty‟s mathematics.
First we provide an initial matrix for the organization‟s pair wise comparisons in which the principal
diagonal contains entries of 1, as each factor is as important as itself.
Performance measures (criteria) with respect to Goal
C1 C2 C3 C4 GM W
C1 (A11) 1 (A12) (A13) (A14) a X1 =a/T
C2 (A21) (A22) 1 (A23) (A24) b X2 =b/T
C3 (A31) (A32) (A33) 1 (A34) c X3 =c/T
C4 (A41) (A42) (A43) (A44) 1 d X4 =d/T
TOTAL T 1.00
Step III: Option with respect to criteria-1 .. To.. 4
Judgment matrix for pair wise comparison of options with respect to criteria
Options of valuation approaches to be prioritized (P1, P2, P3, P4, P5….)
P1 - Enhancing institute revenue generation
P2 - Professional methodology of valuation
P3 -Transparency in valuation process
P4 – Efficiency of resources
P5 – Improved collaboration in research
1. Options with respect to first criteria - C1. ------------------
P1 P2 P3 P4 P5 GM
National
priority
P1 1 5 1/8 6 1/7 0.8826 0.159
P2 1/5 1 1/7 1/9 6 0.4529 0.081
P3 8 7 1 5 1/9 1.9888 0.357
P4 1/6 9 1/5 1 7 1.1600 0.208
P5 7 1/6 9 1/7 1 1.0845 0.195
TOTAL 5.5687 1.000
2. Options with respect to second criteria - C2-------------------
P1 P2 P3 P4 P5 GM
National
priority
P1 1 3 1/7 1/5 1/9 0.3942 0.0628
P2 1/3 1 1/5 1/4 7 0.6507 0.1037
P3 7 5 1 8 1/6 2.1568 0.3437
P4 5 4 1/8 1 1/8 0.7924 0.1263
P5 9 1/7 6 8 1 2.2807 0.3635
TOTAL 6.2749 1.0000

3. Options with respect to third criteria - C3---------------------
P1 P2 P3 P4 P5 GM
National
priority
P1 1 1/6 1/3 9 1/5 0.6310 0.0840
P2 6 1 4 8 5 3.9487 0.5254
P3 3 1/4 1 1/7 1/9 0.4122 0.0549
P4 1/9 1/8 7 1 1/4 0.4755 0.0633
P5 5 1/5 9 4 1 2.0477 0.2725
TOTAL 7.5151 1.0000
4. Options with respect to forth criteria - C4----------------------
P1 P2 P3 P4 P5 GM
National
priority
P1 1 4 1/8 6 1/9 0.8027 0.1407
P2 1/4 1 1/5 9 7 1.2579 0.2204
P3 8 5 1 1/4 1/7 1.0739 0.1882
P4 1/6 1/9 4 1 1/5 0.4307 0.0755
P5 9 1/7 7 5 1 2.1411 0.3752
TOTAL 5.7064 1.0000
Step IV: Synthesize to obtain Priority
G1 = X1C1.X1 + X1C2.X2 + X1C3.X3 + X1C4.X4
G2 = X2C1.X1 + X2C2.X2 + X2C3.X3 + X2C4.X4
G3 = X3C1.X1 + X3C2.X2 + X3C3.X3 + X3C4.X4
G4 = X4C1.X1 + X4C2.X2 + X4C3.X3 + X4C4.X4
G5 = X5C1.X1 + X5C2.X2 + X5C3.X3 + X5C4.X4
Step IV: Synthesize to obtain Priority
Matrix of Local priorities, weights and final ranking of the options
Thus P3 and P5 is most feasible followed by P1, P2, P4.

Research priority of direct score can clearly compare values on an absolute scale and in case of cumulative score
2 of 5 gaps meet almost 75% of our total value of global priority.
III. Conclusion:
In this paper, indicators of research practices are classified into four categories: financial, customer,
internal business process, learning and growth. The criteria are both qualitative and quantitative for our case;
AHP is used to evaluate decision process. According to AHP‟s results, the criterion research quality is
determined as the most important criterion and the criterion professional methodology of valuation is
determined as the least important sub-criterion. The ranking of criteria is determined research quality -
transparency in valuation process, improved collaboration in research, efficiency of resources from the most
important to the least important.
In this paper, AHP methodology is used to determine the best practice for organizational development.
According to obtained results, the transparency in valuation process is determined as the best alternative, while
improved collaboration in research is determined as second best alternative and professional methodology of
valuation is the worst alternative.
Acknowledgements:
The Author is grateful to the NAARM staff members in cooperating during performance appraisal, Scientists
and Director of NAARM for support.
References:
[1]. Braunschweig, T. (2000) Priority Setting in Agricultural Biotechnology Research: Supporting Public Decisions in Developing
Countries with the Analytic Hierarchy Process. ISNAR Research Report No. 16. The Hague: International Service for National
Agricultural Research.
[2]. Chan YL (2009), How strategy map works for Ontario‟s health system. Int. J. Pulic. Sect. Manage., 22(4): 349-363.
[3]. Guo, L.S., He, Y.S. (1999): Integrated Multi-criterial Decision Model: a Case Study for the allocation of facilities in Chinese
Agriculture. Journal of agricultural Engineering, 73:87-94.
[4]. Oakes, Jeannie 1989. What educational indicators? The case for assessing the school context. Educational, Evaluation and Policy
Analysis, 11 (2), 181-199.
[5]. Robert S. Kaplan and David P. Norton, "Using the Balanced Scorecard: Translating Strategy into Action". Harvard Business
School Press, 1996.
[6]. Saaty, T.L. (1980): The Analytic Hierarchy Process. New York, McGraw – Hill.
[7]. Setboonsarng, S., Pochanukul, P., Areepakorn, K. Kaoparisuthi, C., Namahong, C. (1991) Priority Setting for Crop Research in
Thailand, Thailand Development and Research Institute, Bangkok: Thailand Development and Research Institute Foundation. (in
Thai).
[8]. Wu S, Liu S (2010). The performance measurement perspective and causal relationship for ISO certified companies. Int. J. Qual.
Reliab. Manage., 27(1): 27-47.

Performance Assessment of Agricultural Research Organisation Priority Setting by using Balanced Score Card (BSC) & Analytic Hierarchy Process (AHP) technique

More Related Content

What's hot (17)

Similar to Performance Assessment of Agricultural Research Organisation Priority Setting by using Balanced Score Card (BSC) & Analytic Hierarchy Process (AHP) technique (20)

More from iosrjce (20)

Recently uploaded (20)

Performance Assessment of Agricultural Research Organisation Priority Setting by using Balanced Score Card (BSC) & Analytic Hierarchy Process (AHP) technique