Analysing Policy Arguments

1
ANALYSING POLICY ARGUMENTS
Des Gasperi
December 1995.
Chapter for Arguing Development Policy - Frames And Discourses, eds. R. Apthorpe and D.
Gasper, London: Frank Cass, 1996, and a special issue of The European Journal Of
Development Research, 1996 (June), pp. 36-62.
Abstract The complexity and distinctiveness of policy discourse bring a need for methods
and advice in both specifying and assessing policy arguments. The paper reviews, links and
systematizes work in three areas: (1) general advice from `informal logic' on the exploration
and analysis of sets of propositions that make up broad arguments; (2) commentaries on
important elements and tactics in policy argumentation in particular, with special attention
to aspects of `framing'; and (3) proposed methods to specify and appraise whole positions in
policy argument, including the `logical framework approach' and Fischer's `Logic of Policy
Questions'.
1. Argumentation and policy analysis
While training in systematic practical argumentation is valuable for almost any subject, it may
be especially important for policy-oriented work. Public policy discourse is notably complex,
and further has important distinctive features, including the need to incorporate value inputs,
considerations of legitimacy, and assessments of the constraints on public action. I propose
that we should not only stress any distinctive components and structures, but emphasize some
themes that reflect the field's complexity. These themes include the underdefinition of
problems and criteria, and the resulting importance of activities of problem formulation and
closure.
There is a temptation in discourse analysis to plunge deep into philosophy and
linguistic and literary theory. Yet the test does not lie in the elegance of an abstracted case, but
in our ability to provide new insights in particular instances, and to develop and convey to
others some means by which they can generate insights themselves. Much current discourse
theory seems of limited use to students of public policy. Its distinctions do not reflect the
specifics of our field, require specialist background and become inaccessible to nearly all
policy analysis readers and practitioners. Further, while the complexity of policy
argumentation calls for systematic methods of examination, it limits the value of any generic
format for analysing all arguments or even all policy arguments. We have partly to develop
our own variants of argument analysis.
The paper is a brief overview of some tools for analysing and assessing policy
arguments. I will suggest that the following - overlapping, complementary, and non-
exhaustive - contributions are useful in analysing and teaching policy argumentation:- (i)
some general tools of so-called `informal logic' that have been systematized in the past

2
generation; (ii) work that identifies and discusses important elements of specifically policy
argumentation, and associated tactics, dangers, and stylistic devices; and (iii) methods to try to
firstly, specify, and secondly appraise, overall systems or structures of policy argument.
We need to increase sensitivity, to the definition and handling of concepts, and to
patterns and types of inference, but also in less familiar areas: firstly, `formulation', the overall
shaping of extended arguments in complex fields, which involves issues such as the
specification of boundaries and alternatives; and secondly, `colligation' and synthesis, the
pulling together of available elements towards a conclusion.
2. Elements of argument analysis
Toulmin (1958) and Scriven (1976) suggested that when we try to upgrade practical
argumentation we must largely look outside formal logic, for that subject has been concerned
with laws, not maxims or tentative assessments. It rarely analyses multi-part real arguments,
but instead generally restricts itself to assessing given, isolated parts or artificial arguments; in
other words, to quite limited or already clearly ordered sets of propositions, a far cry from real
wide-ranging policy arguments. In contrast, Scriven noted that practical argument analysis
(also known as `informal logic') has two phases: firstly, specifying the content and structure of
a real, untidy position, and only secondly assessing them. The first phase is neither trivial nor
already given to us. In analysing policy arguments we normally have to spend considerable
time identifying their components and linkages, before deciding which parts should be
appraised in detail.
Scriven's own format for analysis reflects direct experience in the tangled argumentative
world of programme evaluation, and deserves attention. He indicated the following steps
(which can be iterative) in the process of specification of arguments and positions. A careful,
even plodding, approach proves justified here. The steps help us to look for components and
structures, but do not pre-empt our findings.
1.Clarification of meanings (not total definitions but distinctions sufficient to the use-in-
context).
2.Identification of conclusions (including unstated implications).
3.Portrayal of structure (the precise connection of premises, inferences and conclusions).
4.Identification of unstated assumptions (the most defensible premises needed to make the
argument complete and consistent).
Typically it will be worth first enumerating significant propositions and terms (e.g.
concepts like `efficient' or `fair' may need examination). One can then construct a diagram or
set of diagrams to show interconnections in the overall argument (see e.g. Dubnick, 1978).
Argument assessment involves criticism of the premises and inferences that have been
identified. Criteria here include: (a) (sufficient) clarity; (b) consistency - both logically and
with accepted facts; (c) scope - i.e. the range of the consistency; (d) simplicity - including
absence of special pleading; (e) applicability and refutability; (f) comparison with other
relevant arguments. This final criterion might appear to unduly extend the scope of
assessment; in fact it keeps matters more manageable and relevant, by setting practical
parameters, and making assessment relative not absolute. It is easier to defend or attack

3
things if they are considered in isolation; but to reject one argument may mean having none or
implicitly accepting another, and should be assessed in the light of that.
Figure 1 - Scriven's schema for argument analysis
1. Clarify meaning of components
2. Identify conclusions
- stated
- unstated
3. Clarify meaning of overall argument
4. Portray structure
5. Extract unstated assumptions
6. Evaluate premises and inferences
7. Consider other relevant arguments
8. Overall evaluation of the argument
In Figure 1 I summarize Scriven's procedure for specifying arguments, as a flow-diagram.
We should distinguish his procedure from the use of flow-diagrams for summarizing already
specified arguments, as Dubnick suggested and as is increasingly practised (e.g. Cole, 1995;
Cole, Cameron and Edwards, 1983; Edwards, 1985). That sort of summary helps in clear
presentation of a case, but may not force one to study the logic of the connections or whether
the summary is a good one. Scriven's procedure emphasises painstaking specification, with
stages of feedback and iteration, and formalized assessment as well as description.
So to assess arguments we must first work out what they say and how. Even prior to
that we need to find and identify them. Stakeholder analysis helps in spotting different
parties and viewpoints involved in an issue. Similarly, unless one identifies the positions
which are being responded to, one may miss the meaning of an argument. One's initial listing
of viewpoints may of course be refined during later analysis.
3. Specification of policy arguments
Many authors list types of element or component in policy arguments. Pen (1985) noted these:
(1) Proposed observations - though these may be defective, and are bound to be selective; (2)
Logical statements; (3) Empirical statements, on relations between observables; (4)
Methodological statements; (5) Images and metaphors - used for the integration or `colligation'
of the above elements into `stories'; (6) Value judgements - which could be expressions of taste,
or truly judgements; (7) Policy recommendations - which are produced by a further stage that
Pen calls super-colligation, pulling together values and facts towards normative conclusions.
What we need besides such lists is a way to look at how elements function and connect
within an argument as a whole. Toulmin's `The Uses of Argument' (1958) gave something

4
more helpful here than the classical syllogism, and has been applied in later policy analysis
theory and cases (e.g. Mason & Mitroff, 1981). Classical logic's focus on determinate inference
mean that it is less relevant for us than jurisprudence, which has non-definitive conclusions,
exceptions to rules, and so on. Toulmin thus spoke of `warrants', which support a belief but
do not deductively oblige it. He also added to the standard syllogism the categories of
`qualifier' and `rebuttal', and distinguished between `grounds' and `warrants' as types of
premise. Grounds are (purportedly) factual statements, whereas warrants are the
considerations used to move from these empirical particulars towards a conclusion. Warrants
in turn require backing of some sort. Figure 2 shows a modified variant of the format adapted
by Dunn (1981, 1993a) for policy analysis.
Figure 2 - Toulmin-Dunn schema for specifying policy arguments
Given
Data/GROUNDS CONCLUSION
(policy-relevant (policy proposal
information) Therefore QUALIFIER / CLAIM)
(inference) (modulates strength of
the conclusion)
And Since
WARRANT Unless
(justification for the inference) -__________________ REBUTTAL to
warrant / its
backing / claim
Because Because
BACKING for warrant BACKING for rebuttal
The format illustrates that arguments have structures, and gives one generalized picture
of structure. It is a tool for looking at any case presently at issue, but not a universally
sufficient pattern.
Toulmin et al (1979) use it to show how the character of `grounds', `warrants' and so on
vary between fields.ii Even within one field there are various types of warrant. For policy
analysis, Dunn (1981) illustrates the use of each of: i) authority; ii) insight, judgement and
intuition; iii) established analytical methods; iv) standard general propositions; v) pragmatic
comparisons with other cases; vi) ethics. An exercise for students is to look for the types of
warrant, as well as of backings, grounds etc., in a selected study or report.
Goldstein (1984) confirms how eclectic are the warrants and backings used in arguments
in planning. This reflects its inter-disciplinary history and tasks, and the need to combine
empirical and normative considerations. In drawing inferences from grounds (data about
current or projected situations, such as about needs and preferences), the warrants used can
include positive theories, normative theories about appropriate allocations or procedures, and
existing laws. Their proposed backing may include views on method, broad positive and
normative conceptions of man and society, the legal constitution, established professional
approaches, and so on.iii

5
Toulmin and Dunn's format remains too simple to be a complete layout for arguments.
`Grounds' may themselves need backing; there can be several types and layers of warrants
and backing; and discourses contain whole sets of propositions. Nor is the format a full
procedure for analysing arguments to identify their detailed structures. There is a danger that
after students learn it they try to force everything into a single straitjacket, and become
diverted from analysing the particular features of an argument. One must stress that it is just a
starting point.iv
Toulmin-Dunn provide a way of looking at a particular proposition or well-knit set of
propositions, that is more relevant than the standard syllogisms in logic texts, though those
still offer us something important. One cannot however analyse complex policy positions or
debates in the way one can dissect a single page or poem; one must be selective. From Scriven
and others we have some advice for analyzing the more extended sets of propositions that
make up broad arguments. What we require are approaches that combine the insights of
Toulmin-Dunn and Scriven, and then more thoroughly connect them to key particulars of the
policy analysis field.
One helpful proposal is Hambrick's `Guide for the Analysis of Policy Arguments' (1974),
which probes further the nature of warrants, backings, rebuttals and qualifiers in policy
arguments. It tries to identify the types of premise involved in any claim that a policy
measure will lead to desired impacts and be worthy of support. Such a proposition involves a
combination, explicit or implicit, of cause-effect claims and normative claims. This is obvious
(though not highlighted in Dunn's format, which simply re-labelled Toulmin's format devised
for any type of argument). Hambrick adds other categories: (a) `grounding propositions', i.e.
intellectual background (not the same as Toulmin-Dunn's `grounds'); (b) `time-place
propositions', i.e. the context of application for the proposed claims; (c) `constraints
propositions', i.e. claims about feasibility; and (d) comparison with alternatives. We will look
further at alternatives, constraints and `grounding' in a subsequent section.
In Figure 3 I take Hambrick's guide beyond a list of elements, by putting it in a
diagrammatic sequence. In Stage 1 an initial if-then proposition draws on positive data and
warrants, and on grounding warrants and backings from theory and methodology (as do
other propositions). In Stage 2 normative warrants and backing turn the if-then proposition
into a means-ends proposition, which in Stage 3 is tested by a number of possible rebuttals
(e.g. whether there are better alternatives available), to see how far it is sustainable as an action
proposal.

6
Figure 3 - Hambrick's categories for the structure of policy arguments, in a flow-diagram
Stage 1 - Cause-Effect Propositions
Causal proposition
(about impact on a Grounding propositions
target variable)
(also to other propositions)
Instrumental
propositions
(extending cause-effect
chains back to a policy
instrument)
If-then
(action-impact) proposition
Stage 2 - Means-Ends
Normative
propositions
Means-ends
(action - valued impacts) proposition
Stage 3 - Tests
Time-place propositions
Constraints (concerning contextual
propositions relevance)
Comparative External impact
propositions propositions (about other
impacts, esp. undesired)
POLICY ACTION PROPOSAL
Hambrick's main distinctive stress is thus on what I have called Stage 3, the testing of a means-
ends proposition in various ways. Two other increasingly well-known approaches focus
somewhat differently: the `logical framework' approach elaborates the cause-effect chain in
Stage 1; and Taylor's model of normative argument, as applied by Fischer (1980) to policy
arguments, goes deeper into the normative backing in Stage 2, and can be used similarly to
examine grounding propositions. Both models however represent to some extent similar
elements as in Hambrick's guide. We can consider them in turn.
The `logical framework approach' (LFA or `logframe') in project analysis was developed
around 1970 for USAID, and has since been adopted by most of the major official international
aid agencies. It attempts: (i) to distinguish various levels of project objectives, starting with

7
inputs, through to those about eventual or broader development impacts; (ii) to specify and
check the causal linkages between the different levels; (iii) to identify the assumptions about
the other factors that are needed for the connections between the different levels to be valid;
and (iv) to specify means of measuring the degrees of fulfilment of the various levels of
objectives. Elements (i), (ii) and (iii) concern us here. They try to indicate the sequence and
structure of argument implied in a project or project proposal.
The original USAID version has four levels of objectives (Inputs-Outputs-Purpose-Goal).
The project's argument has to be put in the following form (see e.g. Coleman, 1987):
1. If the specified Inputs are provided and the specified Assumptions (notably about
important conditions external to the project which could interfere with achievement of
the Outputs) are fulfilled, then the specified Outputs will be achieved;
2. if the specified Outputs are present and a second set of specified Assumptions are fulfilled,
then the specified Purpose will be achieved; and
3. if the specified Purpose is achieved and a third set of specified Assumptions are fulfilled,
then the specified Goal will be achieved.
Assumptions analysis in LFA clearly corresponds to the `Unless' clauses in Toulmin-
Dunn, and to stage 3 in Hambrick's format, the testing of a means-ends proposition; but the
logframe format does not include a built-in set of questions to support this analysis. Many
authors observe how, for this and other reasons, the assumptions column is typically
completed perfunctorily and unhelpfully.v Similarly the objectives column is frequently
confused and arbitrary. Many completed logframes thus have an appearance of logic but lack
the substance. The increasingly popular GTZ (German) version, which rejoices in the acronym
ZOPP, is more useful. It distinguishes five levels, while reducing the degree of stress on
means of measurement; but the relevant difference here is that it specifies the project's chain of
means-ends links by first undertaking a problem analysis to establish a chain (or branching
`tree'-diagram) of hypothesized cause-effect links. The problem analysis gives a reasoned,
non-arbitrary basis for the logframe's means-ends hierarchy.
Compared to Hambrick's guide, the LFA gives a more concrete means-ends format, but
often less advice on how then to proceed with the analysis. Hambrick offers more in the way
of procedure, questions and categories. The two approaches are thus fairly complementary.
Fischer's model involves 12 groups of questions implied in policy arguments (Box 1). The
twelve are themselves grouped into four levels derived from Taylor (1961), moving up from
(1) technical analysis in terms of given objectives, to (2) assessing the objectives in terms of
available policy goals, to (3) assessing the goals in terms of accepted social ideals, and finally
to (4) assessing the ideals. Some details of the model and the ways it is used are open to
criticism (see e.g. Gasper, 1989); but those require a separate discussion. For our present
purposes the model valuably suggests the scope for ordering and systematizing normative
policy arguments, and is becoming quite widely used.

8
BOX 1 - Fischer's model of the levels of policy evaluation
I have paraphrased Fischer (1980:206-12). The comments in brackets are
Fischer's on how to tackle the questions.
Level 1 (Technical verification of program objectives):-
1)Program Objectives - Is the program objective logically derived from the
relevant policy goals? (Refer to logical rules.)
2)Empirical Consequences - Does the program empirically fulfil its stated
objectives? (Refer to empirical knowledge of consequences.)
3)Unanticipated Effects - Does the empirical analysis uncover secondary effects
that offset the program objectives? (Refer to knowledge of
consequences.)
4)Alternative Means - Does the program fulfill the objectives more efficiently
than alternative means available? (Refer to knowledge of alternative
means.)
Level 2 (Situational validation of policy goals):-
5)Relevance - Are the policy goals relevant? Can they be justified by appeal to
higher principles or established causal knowledge? (Refer to knowledge
of established norms and to causal conditions and laws.)
6)Situational Context - Are there any circumstances in the situation which
require that an exception be made to the policy goal or criterion? (Refer
to particular facts of the situation.)
7)Multiple goals - Are two or more goals equally relevant to the situation? (Refer
to normative logic.)
8)Precedence - Does the decision-maker's value system place higher precedence
on one of the conflicting criteria? Or does it lead to some contradictory
prescriptions? (Refer to normative logic.)
Level 3 (Vindication of political choice):-
9)System Consequences - comparison of goal-system's consequences with
accepted social ideals in the situation. (Refer to causal conditions and
laws.)
10)Equity. (Refer to normative logic and accepted social ideals.)
11)Ideological conflict. (Ditto.)
For questions 10 & 11, there should be discussion of how far the policy's goals
and the supporting social ideals are compatible with equitable resolution
of conflicts.
Level 4 (Choice of Social Order):-
12)Alternative Social Orders - comparison with alternative social orders, if 10 &
11 so imply. (Refer to knowledge of fundamental needs and to
normative logic.)

9
Fischer's model was devised more for assessment than for description, and so we look at
it further later. However, checklists of elements or issues that policy arguments should cover
can help us also in description, for they give questions with which to probe what is covered in
an argument and what not. (Hoppe has used Fischer's model in this way.) They similarly
help us in generating propositions. The Fischer-Taylor approach of distinguishing broad levels
helps in describing the levels of normative backing to an argument, systems of grounding
propositions, and the worldviews and `paradigms' used in different policy stances.
The Fischer model provides more questions than does the LFA. It bears a superficial
similarity to LFA, for the prevalent USAID version of LFA also contains four levels. However
the top two USAID levels seem to roughly correspond to the bottom two Fischer levels:-
FISCHER USAID
Higher principles
Predominant social ideals
Policy goals Goal
Programme objective Purpose
Outputs
Inputs
So Fischer's model is stronger for probing the Goal level and beyond, and LFA for levels of
Purpose and below. This is consistent with the acknowledged weakness around Goal level in
the use of LFA (Cracknell & Rednall, 1986). The use of four levels in both cases probably
reflects the attraction of four as a reasonable balance between on one hand (i) complexity and
highlighting of layers and interconnections, and on the other hand (ii) simplicity and
manageability, especially for large-scale use. Dividing a programme into four (or five) levels
is usually a simplification; correspondingly in practice the dividing lines between levels are
often left unclear.vi
How can one counter the real danger that, once people have learnt an approach, they
will insist on `finding' examples of its categories in every argument, and only those categories?
And for example, in use of the LFA, insist on identifying a project's means-ends chain(s) as
three (or four if they have been taught the German version), regardless of the specifics of the
case; or, because only one slot is provided in a diagram, identify only one Purpose or Output
when there are several.
I suggest that a picture of imputable sequences of argument in actual project and policy
analyses makes a useful complement. The idea is to minimize leading people to preset
answers, and to emphasize the creative and variable stages of formulation and final synthesis,
which can produce a range of structures and tactics. Hopefully this picture balances
specificity (and hence practical usefulness) with generality (and so a wide range of
application).
Here is a set of activities into which one can analyze the contents, explicit and implicit,
of appraisal and evaluation arguments:

10
1. Formulation and framing
1a - definition of the exact type of assessment question asked
1b - interpretation of key terms and categories
1c - defining the scope of materials to which the question is
put (this involves further questions about alternatives,
constraints, boundaries)
2. Objectives and criteria
2a - specification of objectives and criteria
2b - justification of them (but the presence and form of this
stage depend on 1a above)
3. Data collection: on outcomes, etc.
4. Data analysis and interpretation.
5. Synthesis
5a - partial synthesis; as in making predictions, or interpret-
ing what are costs and benefits; etc.
5b - full synthesis: evaluation, prescription, conclusion.
The stress on formulation and synthesis is seen by the first and last stages, which we will
examine in Sections 4 and 5 below. Much under Hambrick's propositions on grounding,
constraints and alternatives corresponds to `formulation'. There is little in his schema to
match `synthesis'. Yet policy conclusions generally do not follow simply or unequivocally
from the materials mustered in their support; further stages of interpretation and judgement
are required.
The set of stages above can be useful in both analysing and improving arguments. It is
not a necessary sequence for actually constructing, specifying or assessing positions; those
sequences could well be more complicated.vii It is more a format for grouping the multiple
strands and clusters that argument analysis will reveal, or that brainstorming produces. (See
Apthorpe & Gasper, 1982, with reference to evaluations of service cooperatives in developing
countries). The stages cited are interactive; for instance, definition of scope and specification
of objectives affect each other. Which stages are worth distinguishing depends on the case; for
example, interpretation of what are costs and benefits can be sufficiently creative to deserve
separate attention. (See Gasper, 1987, on project analyses.)
One should look broadly at a position in and across its stages, not only at isolated
statements. Specifications of positions can reflect an approach to their assessment, e.g. a
predisposition to defend or condemn (see my paper on essentialism, later in this collection).
Scriven argues that one should lean toward charitable interpretations and hence more
defensible specifications. This gives priority to developing knowledge rather than to
criticising as an end in itself. Attacks on weaker specifications of arguments are also more
easily dismissed.
In this section I have argued that Toulmin's popular format is not enough to show the
mechanics of policy argumentation. We need too the more open-ended exploratory methods
of Scriven and others, to look at individual cases and patterns; and more reference to the
specifics of policy, where Pen and others offer lists of typical components. I have extended

11
Hambrick's work into a sketch of how components fit into an overall policy argument; and
compared this with the `logical framework' approach and Fischer's model. The three have
different emphases and can be seen as complementary.
In all of these cases there remains the danger that approaches devised to help us think
more carefully will lull us into not thinking. I have therefore added a complementary sketch
of an imputable set of activities in evaluation arguments, with emphasis on how positions are
framed and synthesized from available materials. It directs attention to key issues rather than
relying on a fixed format that may be thoughtlessly imitated. The next two sections look
further at this framing and synthesis.
4. Formulation and framing
Policy argument includes as intersecting activities the framing of techniques and the
application of social philosophies (Braybrooke & Lindblom, 1963; MacIntyre, 1977). We see
that clearly in Fischer's schema. Techniques can only operate within a frame of assumptions;
and on the other hand, general principles only become operational through many technical
steps. This intersection needs to be handled in an orderly consistent way. Policy argument
analysis tries to unravel the linkages involved. Important issues can otherwise fall neglected
between the disciplinary stools of the social scientist and the social philosopher, and
sophisticated lower-level materials and higher-level philosophies can be sloppily combined.
Rein's influential discussions of `framing' in policy analysis take `frame' as a
comprehensive category, like `paradigm' in its `disciplinary-matrix' sense, i.e. the core set of
shared ideas on which the work of a group is based.viii While examining such paradigms has
become common in universities, `the university environment and culture... has the tendency
of driving [such] inquiry into a metalevel where the philosophical assumptions ... [are]
examined... and runs the risk of deepening the separation between thought and action' (Rein,
1983:106).
I will therefore highlight aspects of argument framing which have an active and
practical nature: matters of inclusion, exclusion and attention, including how the burden of
proof is distributed, and the perception of alternatives and constraints. There is a case for
limiting the word `framing' to these areas, to help differentiate terms. We earlier used
`formulation' as label for a broader concept. The more limited meaning seems better fitted too
to the root metaphor for framing that Rein employs -- that of visual perspective (1983:96-7) --
than is a sense that covers every aspect of one's approach.
Since choice of broad theoretical/policy paradigm does not settle all other choices, one
can highlight these overlapping component areas in formulation: (a) the choice of purpose and
standpoint of an argument; (b) the above sense of `framing'; (c) choice of terms and categories,
and (d) the key images and metaphors used. The first concerns for example at what level of a
`logical framework' one is presently arguing, and why; who is the client/audience, and what
is the context. It is now relatively familiar, as are issues of paradigm choice. Choice of key
images (such as `growth pole') has also been well treated (e.g. Rein & Schon, 1977; McCloskey,
1985). Here I will look instead at framing and the boundaries adopted in analysis.
Policy analysts do well to consider the following seven connected aspects of framing when
they examine a policy position, more than how to fit it into the categories of say speech-act

12
theory or classical or modern rhetoric. The seven concern the sort of often unstated
assumptions that were referred to in Scriven's checklist. Limits of space preclude extended
examples.
(1) In tracing and weighing of effects, issues of time horizons and discount rates have
become prominent, thanks to environmentalists' attack on the valuations in mainstream
economics (see e.g. Goodin, 1982). The issues are vital because, to exaggerate only slightly,
àny argument can be turned to any effect by juggling with the time scale' (Elster, 1984:34). All
institutions and movements argue for example that bad short-term effects will be outweighed
by longterm improvements. In the extreme, a supposedly ìnfinitely' better future justifies
anything that furthers its eventual attainment. At the other extreme, use of a high discount
rate may in effect render the longterm future of no importance.
(2) The overall issue of the proper scope of analyses has been considered in planning
theory's debates about comprehensive versus bounded rationality, the range of factors to
include in project analyses, and so on. Where in open systems should one draw the
boundaries of analysis? Matters of scope in identifying causes link closely to choices of level of
analysis and of broad policy paradigm. Some types of possible effect are often ignored if it is
believed that all options can equally plausibly claim them, or that action on such matters is
better assigned to other policies. On the other hand, Hirschman (1967) noted that while
analysis cannot go on forever, it must consider those `side effects' of projects, such as impacts
on self reliance, management capacity, commitment and adaptability, which are in fact vital in
their future operation.
Also enlightening here is work by historians conscious of the necessity of selection in
their accounts and analyses, and of corresponding criteria. Some connections are traced, not
others; some contributory factors are highlighted as causes, not others. The issue is not that
this is done - for neither causes nor effects can be traced in totality or without end - but how
consistently and defensibly (Stretton, 1969). The convenience of an author, rather than his or
her special insight, may underlie claims of necessary bounds to the argument, as signalled by
ìn the final analysis' and similar phrases. These require probing to see if they have more
content than just `where I want to stop (or happen to have stopped) the analysis'.
Critics of foreign aid from both Right and Left have insisted that aid be seen as
contributing to a wider system, which they respectively identify as public sector socialism or
dependent capitalism; and they then simply assess aid by assessing that system. Whereas
liberals who have sought to identify precise project impacts are faced with problems such as
that aid flows can release domestic resources for other uses which are very hard to identify.
They sometimes concluded (e.g. Wood & Morton, 1977) that evaluation is an unending task,
with rapidly diminishing returns. The greater the scope in any policy analysis, the greater
often is the indeterminateness. The need then is for defensible principles of delimitation.
These can be legal, political, ethical, administrative, economic, and procedural-
methodological. And this is the heart of policy analysis (Self, 1975).
(3) Focussing includes distribution of the burden of proof. Has a policy or a danger to be
sufficiently proved or sufficiently disproved? What is the balance adopted between the risks
of Type I and Type II errors? (i.e. the risks of rejecting a true claim / good proposal, and of
accepting a false claim / bad proposal; Gasper, 1987). Similarly, why are certain things taken
as not needing explanation, as being `normal' or `natural' or so on?

13
(4) These designations of a `base case' are of special interest in valuative argument: what
is the baseline against which something is judged positive or negative, an improvement or a
retrogression? It is a commonplace that revolutions are more likely after an earlier period of
improvement, just as loneliness can become intolerable after a taste of contact. Conflicts often
involve different views over how things would be in the absence of the feature that is in
dispute, and these typically depend much on conceptions of how things could and should be.ix
In investment analysis for example, the specification of the `without-project' case is generally
more important for the results than are the refinements of cost-benefit analysis that have
received so much attention from theorists (Stewart, 1978).
(5) Fifthly, we need to ask more generally what comparisons are made. The conceived
alternatives often determine the evaluation of a situation. Thus Bienefeld (1982), when arguing
how relatively casual some academic assessments of Tanzanian performance have been, drew
out a variety of tacit or underargued optimistic assumptions used by critics from Left and
Right on the feasible alternatives that supposedly were available.
Struggles to legitimate one base case and set of categories rather than another sometimes
involve controlled comparisons of the performance of different systems, but with the control
used to ensure certain conclusions. The alternative to the inherently efficient and enlightened
system that the advocate favours is often depicted as a a collage of its bad features, so that it
can be ruled out from further consideration. The actual performance of one's preferred system
can then be more easily defended. Rulers have a special fondness for TINA, the claim that
`There Is No Alternative' to their current policy. TINA is also an assertion about constraints,
and leads to the Panglossian claim that one is in the best of all possible worlds.
(6) A linked aspect of alternatives concerns the ideas used about the range of means that
are available and legitimate, and how they work. In practice `policies in taxation, welfare,
subsidization and so forth... [operate in ways that are] complex, unintended, hard to alter'
(Schaffer & Lamb, 1981:75-6). Past disputes in cost-benefit theory thus often centred --
implicitly-- on what are the constraints in using the other policy tools: which variables are
manipulable and which must be accepted as given (Fitzgerald, 1978).
(7) Layard (1972) has suggested more generally that underlying most normative
disputes is the question of which factors are to be taken as given, as constraints. Definition of
constraints is thus the other crucial face of framing. The answers necessarily reflect both
theories and values, for to call something a constraint means claiming that there is an
unsatisfactory rate-of-return from trying to change it; and so they are often controversial.
Compared to the project level, some argue that at broad policy levels the criteria of (economic)
efficiency and optimization become more obscure and potentially misleading, and constraints
more numerous, so that analysis can usefully center on them (Majone, 1975; Seers, 1983). Neo-
liberals have instead believed they can define, and enforce, efficiency economy-wide, nation-
wide, and world-wide (see Gore in this collection).x
One needs to be suspicious of alleged constraints; but assertions, or presumptions, of
absence of constraint must be treated sceptically too (Goodin, 1977; Gasper, 1987). The
allegations are likely to be emphatic, in an attempt to avoid debate. `The government of India,
for example, simply cannot command the resources that would guarantee each one of [the]
inhabitants of India a standard of living adequate for the health and well-being of himself and
his family [sic]' (Cranston, 1973:67; my emphasis). Yet the governments of Sri Lanka and

14
China can substantially do it. As Bentham noted, `the plea of impossibility offers itself at
every step, in justification of injustice'.
Having noted in sections 3 and 4 some key structural elements, we can turn more to
how they link up in arguments. Just as the scope of policy argumentation means we must
study its framing, so when we look at extended discourses we must consider overall texturing
and pattern, not just the strength of individual links.xi
5. Style, story-telling, and colligation
One can investigate `style' in various ways. Sillince (1986) tries to identify language
forms corresponding to different policy contexts. He discusses four language styles he sees in
urban and regional planning: hortatory, administrative, legal, and bargaining. It is useful too
to simply list various stylistic devices, for sensitization rather than memorization. McCloskey
(1985) and others give illustrations, and government plans are a rich source. (See Box 2.)
George (1990) shows some distorting devices used to present `Government as Intrusive Alien'
and `Taxes as Burdensome Impositions' in many of the most used introductory American
economics textbooks; the cases hinge on the `details' of choice of terms and choice of
comparisons.
Of special interest are the devices by which authors rule out certain things as needing to
be discussed or taken further. Exclusion can be tacit, or by invoking precedents, authority,
`likeliness', òbviousness', or so on. Exclusion is part of framing, but is especially marked in
the concluding stages of extended arguments, when authors may be in a hurry to arrive at
BOX 2: PURPOSEFUL TALK - THE BOTSWANA TRIBAL GRAZING LAND POLICY
Èfforts will be made to develop means for the progressive restriction of
TGLP herds to their ranches' - i.e. for enforcement of a longstanding legal
condition in the ranch-leases - says the sixth National Plan for Botswana (1985:
para.4.65). We can break this sentence into three parts:-
1. Efforts will be made
2. to develop means
3. for the progressive restriction of...
Each part contains words of action, and the overall effect is thus one of
purposefulness. (Mention of the past non-implementation of legal requirements
is left until later.)
But suppose we remove the first part of the sentence, to leave something
like `Means will be developed for the progressive restriction of TGLP herds...'
Interestingly, the result now seems more purposeful, not less. Alternatively we
could remove the second part, which gives us Èfforts will be made to
progressively restrict..' Finally, suppose we remove both the first and second
parts, and just say `TGLP herds will be progressively restricted to their ranches
[as required by law]...' There are now fewer purposeful words and yet a more
powerful meaning.

15
claims.
The project of McCloskey and others to identify the forms of `the rhetoric of the human
sciences' (Nelson et al, eds. 1987) has encountered some resistance, not least the belief that it is
`merely looking at style'. But the way the components of an argument are selected and
handled is both style and substance of the argument. If one does distinguish, then McCloskey
suggests that we see style as `the details of substance' rather than as surface ornament
(1988:286). Until recently their research project has not given much attention to the
organization of positions in policy-oriented argumentation (though see Throgmorton, 1991).xii
So in the previous sections, especially that on formulation, I have highlighted some major
`details' of the substance of policy argumentation.
Two further important lines are the work on `colligation' and `storytelling'. Practical
argument often involves synthesis of judgements from a variety of experiences and of types of
argument. The latter type of synthesis is especially common in policy, since one must relate
disparate sets of ideas (economic, sociological, political, administrative). Colligation refers to
the construction of overall arguments, where one has `to connect or link together, tie, join, to
relate (isolated facts, observations etc.) by a general hypothesis' (Collins Dictionary). It `is
ubiquitous...because only [so] can one tie... research to the uses to which [social science] is put'
(Ward, 1972:181). In policy argument it involves further layers, with the use of ideas about
purposes and values, as implied in Pen's ugly label, `normative super-colligation'.
The term `storytelling' likewise conveys the need for selection and construction from
what materials are actually available (rather than just hoped for). Unfortunately its
connotations of narrative sequence, and of casualness or even deception, make it controversial
as a general label. We refer here though to the sort of disciplined interpretive description
practiced by the good historian. (In contrast, mathematical economists' `stories' are often
fables.) Explaining actions requires relating them to contexts which can never be fully or non-
interpretively described. These inevitably partial explanations are colligated into an overall
account of behaviour. Story-telling is then not just one more phase or technique, but the
purposeful knitting together and application of all the relevant phases and techniques,
including resolving disputes between them. Besides being used to understand past events,
and in scenarios of possible future implementation problems, narratives can help in coming to
agreed policy proposals (Kaplan, 1986).xiii
Analysis of the available `stories' in a particular case involves analysing arguments by
means such as we mentioned earlier, and something more. Criticism and improvement of
`story-telling' require awareness that selections, and hence criteria of selection, are inevitable
and may need to be clarified; and likewise for the stitching together of incomplete materials
into conclusions.
`Storytelling' is empirical, for without information there is little to select or stitch.
Unfortunately the term has obscured this to some. Several commentators have been enraged
into inattention, as are others by McCloskey's use of the word `rhetoric'.xiv Many authors have
presented ways of countering the dangers and for assessing `stories' (e.g. Ward, 1972; Rein &
Schon, 1977; Wilber, 1978; Kaplan, 1986). The criteria in their lists are largely familiar - for
example Kaplan speaks of truth, richness, consistency, congruency and unityxv; though not all
the criteria (e.g. fertility, elegance, consistency with felt convictions and expert belief) are part
of older philosophy of science, and they are often overlooked in the enraged attacks that allege

16
rejection of all criteria. Kaplan and others instead propose to make our actual practice of
argumentation more explicit and systematic.
6. Argument assessment
Aspects of assessment have emerged in earlier sections. Much follows directly from careful
argument specification: e.g. seeing how far terms are used consistently, or what alternatives
have been considered. Often one may take it as a compliment to be told one has `only'
identified `simple' errors; for in reality, inconsistencies and failings can involve elements
which are spread wide and deep within positions, amongst thousands of words, not already
collated and displayed. All errors appear simple when broken into steps, but the
identification may be far from simple; having some methods is important. Flawed policy
arguments are not in general knowingly constructed and then defended; instead, appealing or
convenient arguments are defended, and then need to be deconstructed.
Corresponding to universal criteria of assessment are standard lists of fallacies. Toulmin
et al (1979:158-185), following Aristotle, distinguish `fallacies of ambiguity', which trade on
obscurity in key terms; and `fallacies of unwarranted assumptions' or unwarranted inferences,
which involve inconsistency with rules of logic or accepted facts or other premises. Thouless
(1974) and others provide similar compilations. It is not sufficient, nor indispensable, to study
such lists; much of the important pitfall orientation is subject-specific. But they help, as much
to increase alertness as to provide specific tools. Toulmin for example lists 18 fallacies, with
some sub-variants, and Thouless gives 38: these are practical lists, neither uselessly vague nor
immensely academic. They should be combined with study of subject-specific pitfalls, as in
Majone's proposal for `craft training' in policy analysis (1989).xvi
We noted earlier that the content and character of `grounds' and `warrants' vary
between fields. Toulmin concluded that: `all the canons for the criticism and assessment of
arguments are in practice field-dependent' (1958:38), and illustrated this at length in a later
textbook. It can be misplaced, for example, to criticize ethical or policy arguments for not
being mathematical or natural science arguments. `Context determines criteria' (Toulmin et al,
1979:120).
Some authors argue that certain criteria are implicit in the (or a) particular context of
policy argument. `Political evaluation, as a [specific] subuniverse of evaluative discourse,
specifies a "point of view"' (Fischer, 1980:113). There are various interpretations of what this
entails: for example, reference to `the public interest'; or, according to Anderson (1979), to
efficiency, justice, and appropriate authority (though the meaning of each is of course open to
debate).xvii
MacRae (1993) offers five worthy though pedestrian criteria for assessing policy
proposals relative to given goals and values, in other words within a group of like-minded
people. Proposals should: (1) not omit reference to relevant valuative criteria; (2) not elevate
means into ends; (3) compare alternative policies, including the status quo and doing nothing;
(4) not omit relevant information; and (5) consider relative quantities. A multi-criteria matrix
can ensure that the pros and cons of each relevant alternative are considered for each relevant
criterion. He also offers a few points, equivalent to pieces of a professional ethics, concerning
what tactics are unacceptable in debate between groups with different goals, and how to

17
constrain them.
The Fischer model (see Box 1 above) shows this layering of types of policy discourse in a
deeper and clearer way, and much more. His `A Logic of Policy Questions' (1980:206-212)
provides a general heuristic for considering how particular families of questions correspond to
certain levels of the evaluation of policy arguments. (See Box 1 above.) We noted that some of
its questions match Hambrick's, but with more probing of normative and grounding
propositions, and more on warrant-backing hierarchies than in the Toulmin-Dunn model. It
gives one serviceable frame for looking at whole positions, not just the mechanics of single
propositions or proposals. The `Logic of Policy Questions' is now quite extensively used in
teaching and research in the Netherlands (see e.g. de Graaf & Hoppe, 1989; Hoppe et al, 1990;
Hoppe, 1993.)
Its emphasis on desirable elements of a policy argument still needs to be complemented
by methods to identify actual tactics, links and structures, like the formats we saw earlier. In
addition, full operational classifications of questions have to be specific to particular policy
contexts. Finally, Fischer's framework reflects a certain philosophy of valuation, derived from
Taylor (1961) and others. Thus he speaks of `A' rather than `The' logic of policy questions. By
going in depth into one widely used philosophy he is able to give more precise, even if not
universally binding, illustrations of how policy argumentation synthesizes positive and
normative aspects.xviii
After noting criteria of assessment, we must consider practical matters of strategy.
Outside the closed worlds of mathematics and formal logic, and given the presence of
multiple criteria which can pull in different directions, assessment is a matter of reasoned
judgement and not always of decisive demonstration; and what is reasonable generally
depends on the context.
Firstly, since it usually does not suffice to point out an individual fallacy here or there in
a component argument of a position, Scriven's canons of assessment in section 2 above went
beyond clarity and consistency, to matters like scope and comparison with other relevant
arguments. The criteria for assessing wider story-telling come in here too. Some
quantification and weighting of criteria may sometimes be useful as a check, since our
unaided powers of aggregation tend to be unreliable. Assessment is not absolute, but can
clarify differences and their sources, shift balances, and help sift out arguments.
Secondly, for real-world debates, where the strands of argument may spread virtually
without limit, Scriven advised focussing one's energies. Concentrate on the parts of an
argument which support the main conclusions, not necessarily on the weakest parts; and
concentrate on strong variants of positions. Since positions are frequently underspecified,
priority should usually go to encouraging their improvement rather than trying to totally tie
them down for absolute judgement at one moment. In practice most positions will implicitly
accept some criticism and an assessment of `could do better', as is seen by their evolving after
the criticism.
Next, immanent criticism is often helpful, using the position's own terms, as opposed to
transcendent criticism, which draws on clearly external criteria and intellectual resources.
Immanent criticism works within a position, but does not imply acceptance of the whole
position. Positions are layered; parts of them, like methodological criteria, can have priority
and be used to rule against some other parts. Also the immanent/ transcendent distinction is

18
not absolute, since positions are usually not sharply defined and bounded. The layered
systems have fuzzy bounds. Positions exist as both (A), the stated representations, and (B), the
representations' sources, which are somewhat vague, and neither totally stated nor perfectly
represented by (A). Appeal to the quasi-internal resources in (B) is an important aspect of
criticism which aims to persuade (Nussbaum & Sen, 1987; Walzer, 1987). Whether it is then
called immanent or transcendent could vary with context. Scriven's advice was thus to give
less priority to judging the current representations than to improving their portrayal of their
sources. Rigorous criticism is still vital in this, but some of the emotions it arouses might
thereby be reduced.
Fourthly, the debating aspect to assessments is fundamental, as it is for policy
argumentation itself. Indeed we saw that debate, in the form of possible rebuttals and
comparison with alternative views, is integral to all practical argumentation as elucidated by
Toulmin and Scriven. Arguments attempt to persuade an audience; they respond, even if
implicitly, to previously stated views; and their meanings always go beyond the conscious
control of the author (Throgmorton, 1993). And assessments of the quality of arguments
involve interchanges between parties, where `face' can be as important as `faiths'.
The selective interchange in debates is potentially a way of concentrating on matters that
are more important in the given context; but it can also facilitate conflict. People usually
cannot resist concentrating on others' peripheral apparent inconsistencies, to imply their own
superiority, since that is often easier than demonstrating it.
Also common are crossed transactions, where responses are directed at implications it is
feared others might draw from a statement, rather than directed (as in complementary
transactions) to its actual content.
Examples of complementary transactions:
Stimulus - co-operatives aren't suitable in cases A to H.
Responses - (i) I agree; (ii) I don't agree; (iii) I agree for cases A to D, but not for cases E to H; (iv)
for reasons x,y, and z, I think you should review your arguments and criteria.
Examples of crossed transactions:
Stimulus - co-operatives aren't suitable in cases A to H.
Responses - (i) co-operatives aren't to blame for lack of suitability; (ii) I deny that co-operatives are
a flawed and inferior form of organization; (iii) so you reject co-operatives. (See Apthorpe
& Gasper, 1982.)
Many a time, transactions reinforce differences rather than reduce them; so some policy
analysis techniques keep parties out of direct contact, to reduce the peripheral conflicts. The
above set of crossed responses is prone to be followed by `you renegade', and more. Provoked
by the tactics and tone of debates around evaluations of formal rural service co-operatives in
India, Baviskar & George (1988) provided a thoughtful set of suggestions on controversy
management. Debates around other policies and organizations are equally heated and in need
of help.
Finally, assessors must not be surprised by the apparent persistence of identified
fallacies, nor by the denial of change even if and when fallacies are abandoned. On one
occasion one thing is said, on another something evidently inconsistent; but the two may have

19
a common inspiration: a belief in the essential desirability of the policy being discussed, and a
common effect - namely, its vindication. Even a feeling of consistency may then be retained.
In the short run, the `open texture' of conceptsxix may help cover inconsistencies and defence
of idealized essences; but in the longer run it can provide a yet more valuable service. For
when adaptation is felt advisable, open texture eases an evolution under cover of claiming -
indeed perhaps still comfortingly feeling - that one's essential position has not changed, even
though by operational tests one's effective position clearly has. `But of course we have always
believed in [our new position]'.
7. Conclusion
I have presented argumentation in policy analysis and planning as selective and
creative. So I have tried to outline an active flexible approach in argument analysis: not a
grand theory but some usable themes and methods that should help to draw forth skills, not
depress them under a supposedly total map of policy argumentation. The approach aims to
be flexible where there can be differences in structure and elements between cases; but to
stress issues in `opening' and `closing' which arise in every case. I have tried too to present
argument specification and assessment in a broader context, to help in identification,
generation and improvement of arguments, not just the examination of given and fixed
positions.
In a complex field, none of the methods mentioned, nor others, nor all put together, will
guarantee correct or insightful analyses; and they may be misused. But in general they should
help. Many can be viewed as like the aids one uses when learning a language or some other
skill. Once one has mastered such aids, one can go on to more novel tasks and refer back only
occasionally.
My concern has been to introduce modest but helpful tools. There is little point in
asking most policy analysts to master the latest models from speech-act theory or wherever,
elaborated without reference to the specifics of their field. We need instead to mobilize and
complement their `trade'-skills. With students too, we should foster skills that will help them
to grapple with cases by themselves, during their courses and later. While the literature on
policy argumentation has more to offer (see e.g. Fischer & Forester, eds. 1993), I have
considered that one priority is to present a practical approach to analysing arguments. The
second priority has been to highlight a few important issue clusters that can be vividly
conveyed in teaching. A kit of `watch-out-fors' and `think-carefully-abouts' should
supplement the standard tools. Argument analysis must include sensitivity both to use of
language and to wider structures. The issue cluster of framing was accordingly emphasized.
There and elsewhere, policy argument analysis can and should give attention both to broad
conceptions and precise details; and, vitally, show how those conceptions are realized through
the mere `details' of argument.
*****
i Institute of Social Studies, PO Box 29776, 2502 LT The Hague, The Netherlands; e-mail: gasper@iss.nl.

20
ii. Toulmin has said he would, if now rewriting `The Uses of Argument', stress further how the very
substance of the things at stake varies between fields, and hence how each field is distinctive (Toulmin,
1990).
iii. Goldstein suggests that in utilitarian arguments the grounds include individual preferences, market
prices, etc.; the warrants are from orthodox welfare economics; and the backing is utilitarian social
philosophy (with more or less liberal amendment). In systems arguments, grounds include estimated
and/or projected system conditions; the warrants are the models used of the system, and the posited
goals, usually for system maintenance; backed by the associated methodology. In procedural arguments,
the grounds are facts about the procedure by which a conclusion was reached (e.g. majority voting held
under certain conditions); the warrant is a purported principle of good procedure; and the backing is a
supportive social philosophy. He noted too the prominence of qualifiers and rebuttals in policy and
planning arguments, given our limited theories, orientation to an uncertain future, and typical need to
act before research can fully mature.
iv. Figure 2 already refines the schema by noting that there can be a rebuttal not only to a warrant but
also to a backing.
v. See e.g. Cracknell & Rednall, 1986; Wiggins & Shields, 1995. Location of the assumptions column at
the far right of the logical framework diagram contributes to its neglect, at least in the USAID and related
versions which place two other columns between the means-ends hierarchy and the associated
assumptions. People obliged to fill in a logframe are in addition often exhausted by the time they reach
this fourth column. A more sensible location would be directly adjacent to the means-ends hierarchy, on
its left.
vi. The orthodox case for standardization, and associated simplification, holds that unless principles of
analysis are incorporated into compulsory administrative routines, they are unlikely to have widespread
and sustained impact (e.g. Cracknell & Rednall, 1986). Bureaucracy favours standardization, for large-
scale use by ordinary staff, and for ease of comparison and central supervision; one's judgement on
standardization may depend on how much faith one has in people's independent thinking. The LFA
also seems more used in aid programmes than in domestically funded programmes in either rich or poor
countries; the resolution (on paper) of objectives is easier to enforce when the players have such unequal
power.
vii. Thomas (1972) presents a similar picture, and notes that it is a `reconstructed logic'. He then
contrasts it with a pragmatic `logic-in-use' for actually making and testing value judgements.
viii. In fact Kuhn's other, neglected, sense of `paradigm' is equally important here: `exemplar' paradigms
are concrete instances of exemplary practice, which embody ideas that can be recognized but are not
easily or satisfactorily formalizable, including ways of `seeing' situations or of deciding which
formalizations should be applied and how. The meaning of exemplars is notably open-textured and
open to evolution (Kuhn, 1977). Exemplar is not only the older meaning, but not prone to the potential
danger with the other sense (i.e. a way-of-thinking or disciplinary matrix) of inducing lack of care when
we examine others' ideas.
ix. The choice of base case can be vital in bids to define the term `justice'. In the classic liberal view of
Locke, a man has exclusive right to the products of his own labour, including land that he opens up,
provided that there be `enough and as good left in common for others'. Many enclosure movements in
fact reduced the standard of life of those who had benefited from communal rights of access. Hence
property holders and their advocates often seek a modified reading of Locke's proviso. The relevant
baseline for those who are excluded by the privatization of common property is taken as their well-being
in an original `state-of-nature', in which resources were not scarce but where living standards were
conveniently low. It is easier to defend privatization when `the baseline for comparison is so low'

21
(Nozick, 1974:181). Yet if one argues - as do many defenders of privatization - that the long-run position
of the excluded be considered, and that short-term losses may be outweighed by longer-term gains, then
the baseline for the comparison should not be static. It should be what others' standard of life would be
over time without the privatization, e.g. with socialization when that has been an historically relevant
alternative. The baseline then could well rise over time and not flatter private holding in the way that
Nozick's argument does (Paul ed., 1982.)
x. While economists are trained to think of constraints, they can forget even economic constraints, e.g. the
need for states to raise revenue on a scale beyond the scope of allocatively `neutral' methods. There are
also numerous constraints of law and administrative procedure; and political, behavioural and
administrative capacity limits. Some development economists find absurd the idea that a government
could wish to pursue an objective (e.g. redistribution) but be effectively barred from using an available
means (say taxation; Stewart, 1975); they read this as just a disguised statement of lack of concern, for
they presume that government is like an individual. But existing tools may be so burdened with other
tasks, or so inert and hard to change, that introduction of a new tool (like social cost-benefit analysis in
investment choice) might enable those using it to advance the objective to some degree.
xi. I have not discussed the social framing of argumentation, including selection and grouping of
participants, a key aspect of argument generation. Mason & Mitroff (1981) and Fischer & Forester (1993)
put argument analysis into this perspective.
xii. For an example of `the new rhetoric' - i.e. an integration of classical rhetoric's study of style with
analysis of other aspects of argumentation - applied to programme evaluations, see Appendix A of
House (1980). Much of the early work on `rhetoric of economics' was on the Keynesian versus
monetarist versus new classical controversies in macroeconomics (e.g. Klamer, 1984), but emphasised
rather universally posed policy issues of intervention versus non-intervention and discretion versus
fixed rules. Attention to more intricate and substantive policy argumentation becomes unavoidable in
work on sectors such as health or roads; cf. Colvin (1985).
xiii. Although often consensus on objectives and criteria is unattainable, agreement on actions is
sometimes still attainable through composing and scrutinizing a narrative that reviews past experience,
explains current dilemmas, and presents corresponding proposals. Value criteria can be left partly
implicit, just suggested by the account; for the proposals may be compatible with a range of values and
objectives and hence acceptable to a range of people.
xiv. See e.g. reviews of Pen (1985) by Cohen (1986); and by Blaug (1989) of Klamer et al (eds., 1988).
McCloskey (1994) gives a magisterial rejoinder to his critics.
xv. Incongruency is when adjacent elements seem to conflict, without adequate explanation being given.
Unity means that the sections of a story clearly show their relationship to each other.
xvi. Sillince (1986) gives examples from planning, for 18 types of fallacy, mostly based on Thouless. In
some instances one might query whether the fallacy is well presented, distinct from others, and well
exemplified, or even whether what is described is always a fallacy. This is partly inevitable when one
comes to complex real cases; and it offers an opportunity to engage students, not spoonfeed them. They
can also consider what type of fallacy each is: 1. conceptual, 2. inferential; or 3. failure to look at relevant
factors, 4. introduction of irrelevant factors, or 5. a `Type III error' of addressing the wrong question.
Dunn (1993b:261) contrasts `first-order errors.. the choice of the less valid of two or more causal
inferences [e.g. Type I and Type II errors], and `second-order errors.. the selection of the less appropriate
of two or more world-views, frames of reference, or problem definitions'.
xvii. Dunn (1981:232-9) has a similar list of criteria, but is non-committal on whether they are obligatory.
He also presents extensive sets of more general criteria relevant to knowledge claims, covering (a) types

22
of possible warrant or backing, and (b) types of possible rebuttal (Dunn, 1993b:270-82). Similarly, Dunn
(1990) collates a large number of possible weaknesses: the classical fallacies in deduction, plus the
fallacies possible in asserting causality (standard in discussions of experimental design), plus the sorts of
`practical' criteria (clarity, scope, etc.) we saw earlier from Scriven. He does not advance as clearly as
Fischer does to substantive considerations of what a policy argument should have covered but did not.
xviii. Taylor's proposal was that testing a normative judgement requires four stages: `verification',
`validation', `vindication', `rational choice'. Fischer valuably suggests which types of social science and
philosophy correspond to the four levels. It is debatable though to take Taylor's model as a universal
prescription (let alone description). It reflects the work in moral philosophy of Kurt Baier, Herbert Feigl,
and R.M. Hare; and is consistent with a type of rule-consequentialism. This is why it gives four rather
than more or less levels. It fits some socio-cultural-political set-ups and issue areas better than others;
and `it is possible to have as many forms of policy analysis as there are systems of political thought'
(Anderson, 1987:26). Even given its philosophy, the four levels in fact overlap and flow into each other,
rather than being sharply distinct. See the appendix to Gasper (1989). Some applications mainly use just
two aggregated levels (Hoppe et al, 1990): policy discourse that employs given normative beliefs
concerning a policy, and discourse concerning those beliefs.
xix. This means our inability to specify a full and exact set of necessary conditions which together are the
sufficient condition for use of a term; instead meanings develop in the process of use. See Gasper on
essentialism, in this collection.

23
REFERENCES
Anderson C.W., 1979. The Place of Principles in Policy Analysis, American Political Science
Review, 73, 711-23.
----, 1987. Political Philosophy, Practical Reason and Policy Analysis. Pp.22-44. in F. Fischer &
J. Forester (eds), 1987, Confronting Values in Policy Analysis, Sage, Beverly Hills.
Apthorpe, R.J., & Gasper, D.R., 1982. Policy Evaluation and Meta-Evaluation: the Case of Rural
Cooperatives, World Development, 10, 8, 651-69.
Baviskar, B., & George, S., 1987. Development and Controversy. Economic & Political Weekly,
XXIII (13), A.35-43.
Bienefeld, M., 1982. Tanzania: Model or Anti-Model?; in M. Bienefeld & M. Godfrey (eds.),
The Struggle for Development, Wiley, Chichester.
Blaug, M., 1989. Book review. Economic Journal, 99 (Sept.), 856-7.
Braybrooke, D. and Lindblom, C., 1963. A Strategy of Decision, Free Press of Glencoe, New
York.
Cohen, A.J., 1986. Book review, J. of Economic Literature, XXIV, Sept., 1218-9.
Cole, K., 1995. Understanding Economics. London: Pluto Press.
----, Cameron, J., & Edwards, C., 1983. Why Economists Disagree. Harlow: Longman.
Coleman, G., 1987. Logical framework approach. Project Appraisal, 2, 4, 251-9.
Colvin, P., 1985. The Economic Ideal in British Government. Cambridge: Cambridge Univ.
Press.
Cracknell, B. & Rednall, J., 1986. Defining objectives and measuring objectives in aid projects
and programmes. EV 384, Evaluation Department, U.K. Overseas Development
Administration.
Cranston, M., 1973. What are Human Rights?. Bodley Head, London.
Dubnick, M., 1978. Comparing Policy Alternatives, Policy Studies Journal, 6, 3, 368-375.
Dunn, W.N., 1981 (& 1993a, 2nd edition). Public Policy Analysis: An Introduction, Prentice
Hall, Englewood Cliffs, N.J.
----, 1990. Justifying Policy Arguments. Evaluation and Program Planning, vol.13, 321-9.
----, 1993b. Policy Reforms as Arguments. Pp.254-90 in Fischer & Forester (eds.), 1993.
Edwards, C., 1985. The Fragmented World. London: Methuen.
Elster, J., 1984. Ulysses and the Sirens, Cambridge U.P., Cambridge.
Fischer, F., 1980. Politics, Values and Public Policy, Westview Press,
Boulder, Colorado.
Fischer, F., & J. Forester (eds), 1993. The Argumentative Turn in Policy Analysis and Planning.
Durham, N.C.: Duke U.P.
Fitzgerald, E.V.K., 1978. Public Sector Investment Planning for Developing Countries.
Macmillan, London.
Gasper, D.R., 1987. Motivations and Manipulations: Practices of Project Appraisal and
Evaluation, Manchester Papers on Development, 3,1, 24-70.
----, 1989. Policy Argument. Working Paper 47, Institute of Social Studies, The Hague.
George, D., 1990. The Rhetoric of Economics Texts. J. of Economic Issues, XXIV, 3, 861-878.
Goldstein, H.A., 1984. Planning as argumentation. Environment and Planning B: Planning
and Design, 11, 297-312.

24
Goodin, R., 1977. Ethical Perspectives on Political Excuses, Policy and
Politics, 5, 71-78.
----, 1982. Discounting discounting. J. of Public Policy, 2, 53-72.
de Graaf, H., & Hoppe, R., 1989. Beleid en Politiek, Coutinho, Muiderberg.
Hambrick, R.S., 1974. A Guide for the Analysis of Policy Arguments, Policy Sciences, vol.5,
no.4, 469-78.
Hirschman, A., 1967. Development Projects Observed. Brookings, Washington.
Hoppe, R., 1993. Political Judgement and the Policy Cycle: the Case of Ethnicity Policy
Arguments in the Netherlands. Pp.77-100 in Fischer & Forester (eds.), 1993.
Hoppe, R., et al, 1990. Belief Systems and Risky Technologies. Industrial Crisis Quarterly, 4,
121-140.
House, E., 1980. Evaluating with Validity, Sage, Beverly Hills.
Kaplan, T.J., 1986. The Narrative Structure of Policy Analysis. J. of Policy Analysis and
Management, 5, 4, 761-778.
Klamer, A., 1984. Levels of discourse in new classical economics. History of Political
Economy, 16,2, 263-90.
---- et al. (eds.), 1988. The Consequences of Economic Rhetoric, Cambridge Univ. Press,
Cambridge.
Kuhn, T.S., 1977. The Essential Tension. Univ. of Chicago Press, Chicago.
Layard, R., 1972. Introdn. to Layard (ed.), Cost-Benefit Analysis, Harmondsworth: Penguin.
MacIntyre, A., 1977. Utilitarianism and Cost Benefit Analysis, pp.217-37 in K. Sayre (ed.),
Values in the Electric Power Industry, Notre Dame Univ. Press, Notre Dame, Indiana.
MacRae, D., 1993. Guidelines for Policy Discourse. Pp.291-318 in Fischer & Forester (eds.),
1993.
Majone, G., 1975. The Feasibility of Social Policies, Policy Sciences, 6, 1, 49-70.
----, 1989. Evidence, Argument and Persuasion in the Policy Process. Yale Univ. Press, New
Haven.
Mason, R.O., and I.I. Mitroff, 1981. Challenging Strategic Planning Assumptions, Wiley, New
York.
McCloskey, D., 1985. The Rhetoric of Economics. Madison: Univ. of Wisconsin Press.
----, 1988. The consequences of rhetoric. Pp.280-293 in Klamer et al (eds.).
----, 1994. Knowledge and Persuasion in Economics. Cambridge: Cambridge Univ. Press.
Nelson, J.S., et al, (eds.), 1987. The Rhetoric of the Human Sciences. Univ. of Wisconsin Press,
Madison.
Nozick, R., 1974. Anarchy, State and Utopia. Basic Books, New York.
Nussbaum, M. & Sen, A., 1987. Internal Criticism and Indian Rationalist Traditions. Working
Paper 30, World Inst. for Development Economics Research, Helsinki.
Paul, J. (ed.), 1982. Reading Nozick, Blackwell, Oxford.
Pen, J. 1985. Among Economists, North Holland, Amsterdam.
Rein, M., 1983. Value-Critical Policy Analysis. Pp.83-112 in D. Callahan & B. Jennings (eds.,
1983), Ethics, the Social Sciences & Policy Analysis, Plenum, New York.
----, & Schon, D., 1977. Problem Setting in Policy Research. Pp.235-251 in C. Weiss (ed., 1977),
Using Social Research in Public Policy Making, Heath, Lexington.
Schaffer, B., & Lamb, G., 1981. Can Equity Be Organized? Gower, U.K.

25
Scriven, M., 1976. Reasoning, McGraw-Hill, New York.
Seers, D. 1983. The Political Economy of Nationalism, Oxford U.P.
Self, P., 1975. Econocrats and the Policy Process, Macmillan, London.
Sillince, J., 1986. A Theory of Planning, Gower, U.K.
Stewart, F., 1975. A Note on Social Cost-Benefit Analysis and Class Conflict in LDCs. World
Development, 3(1), 31-9.
----, 1978. Social Cost-Benefit Analysis in Practice, World Development, 6, 153-165.
Stretton, H., 1969. The Political Sciences, Routledge, London.
Taylor, P. 1961. Normative Discourse, Prentice Hall, Englewood Cliffs.
Thomas, L.G., 1972. A Model for Making and Testing Value Judgements. In L.G. Thomas
(ed.), 1972, Philosophical Redirection of Educational Research, Univ.of Chicago Press,
Chicago.
Thouless, R., 1974. Straight and Crooked Thinking, Pan, London.
Throgmorton, J., 1991. The rhetorics of policy analysis. Policy Sciences, 24, 153-179.
----, 1993. Survey Research as Rhetorical Trope: Electric Power Planning Arguments in
Chicago. Pp.117-44 in Fischer & Forester (eds.), 1993.
Toulmin, S., 1958. The Uses of Argument, Cambridge: Cambridge Univ. Press.
----, 1990. Logic, Rhetoric and Reason. Keynote address, 2nd ISSA International Conference
on Argumentation, Univ. of Amsterdam, June 19-22.
----, et al, 1979. An Introduction to Reasoning, Macmillan, New York.
Walzer, M., 1987. Interpretation and Social Criticism. Cambridge, Mass.: Harvard U.P.
Ward, B., 1972. What Is Wrong With Economics, Macmillan, London.
Wiggins, S., & Shields, D., 1995. Clarifying the Logical Framework. Project Appraisal, 10(1), 2-
12.
Wilber, C.K., 1978. The Methodological Basis of Institutional Economics: Pattern Model,
Storytelling, Holism, J. of Econ. Issues, 12,1, 61-89.
Wood, R., & Morton, K., 1977. Has British Aid Helped Poor Countries? ODI Review, 1977,
no.1, 40-67.

Analysing Policy Arguments

More Related Content

Similar to Analysing Policy Arguments (20)

More from Courtney Esco (20)

Recently uploaded (20)

Analysing Policy Arguments