Producing Data: Analyzing Material, Culture & Documents

Producing Data
Analyzing Material,
Culture & Documents
Aitza M. Haddad, J.D., LL.M.
MCMS 702
March 7, 2013

Producing Data: Analyzing Material,
Culture & Documents
Chapter 7 of Taylor, B. C., & Lindlof, T. R. (2010). Qualitative communication research methods. Sage
Publications, Incorporated.
• Material culture
• As well as documents are “mute evidence”
• They are unable to respond to the researcher’s questioning.
• They have a lot to say when they are read alongside the
living voices of informants and other social actors.
• Researcher must be ready to pay close attention to all the
ways in which people interact with things and texts in order
to gather evidence about how the material world evokes
meaning.
• Collecting, reading and interpreting
• Any humanly produced artifact; the corporeal, tangible
objects constructed by humans; and the material
manifestations of the social realities understood to be
relevant and powerful.
• Things owe their existence to a human source
• Homo faber – human beings are always about the business of
making things, maintaining a degree of dominion over the material
world.

 Objects also posses a degree of agency
 They make their influence felt as soon as they appear.
 They do not evoke meaning in isolation but only in its
relation to the meanings of other objects.
 They have the power to reconnect people with the traditions
and myths of their distant past or to serve as sense-making
devices for future generations.
 While material culture may be expressive, it cannot formulate a
complex idea in the way as spoken language.
 A material object is looked not as the entity that communicates but as an
element in the process of communication.
 The study of material culture has a long history
 Is not the synonymous of folklore studies.
 Folklorists study, and sometimes act as collectors of, the indigenous
arts, crafts, tools and built environment of subcultures.
 Archeology, social anthropology, sociology and consumer
behavior have used it and contributed to it.
 Communication is a latecomer to the field.
 Tendency of the communication researchers to regard objects and
the built environment as a mere backdrop or staging area for speech
acts.
 Perceived nature of the objects.

MAKING MATERIAL CULTURE VISIBLE
 Reasons rooted in human history and the ongoing human
relationship with the material world.
1. Antimateriality – the theme of the body and other material objects as
having nonessential, impure, vulgar, or even false status.
 Define both humanity and its higher pursuits as in direct opposition to the
vulgarity of the material because they do not depend on the provisions of
material forms.
 People are generally less comfortable with the idea that their interests are defined by what
they consume or that objects help create the need for security.
2. Social actors overlook the material culture
 Inconspicuous quality – hails our attention without resorting to language and
rarely calls attention to itself, which makes it easier to take their physical
character for granted.
 Overfamiliarity with concrete elements of a scene tends to breed undersensitivity to their
presence and rhetorical effects.
 Great propagandistic value in the creation of a world of meaning – operates on us by subterfuge,
evoking our memory of the discourses and cultural associations that have built up around it over
time.
3. People often engage with an object in terms of its fitness for a particular
project.
 Theory of affordances – an object has built into its design a range of “action
and possibilities,” which affords certain kinds of interactions with capable
human actors.
 Extends the concept of affordance from perceptual psychology into the realm of social

 Material items do become explicit and visible in their own
right when they malfunction, go missing, are singled out in
conversation, or become problematic in social relationships.
 Interviewing tactics can be used to aid people in “waking up” to
what they know and feel about their surroundings.
 Steps for making material culture visible for analysis:
1. Identifying one or more objects for study.
 One common strategy is to choose an object with strong symbolic
value in a particular context.
2. Evaluate the similarities and differences of the object of study in
relation to objects in the same context.
 They will arise from comparing the discursive references to these
objects in conversations, speeches, textual materials, ads, etc.
 Important for three reasons:
1. It sensitizes you to those features of an object that elicit the most powerful or
prominent meanings.
2. You will be on firmer ground arguing for the boundaries of a particular context.
3. You start to develop interpretations grounded in multiple levels of meaning.
3. Expand the interpretations made within larger frameworks of
history, ideology, biography, and/or theory.
 The goal is to develop a deeper, multidimensional understanding
 The specific choice of frameworks to apply depends (a) on the nature
of the place or objects being studied, (b) on the interests brought to the
study by the researcher, or (3) as the result of ongoing encounters.

DOCUMENTS
 The study of documents can be very useful and a vital
source of field data.
 You should always be on the look for them in the predictable
places of a cultural scene.
 Familiarization will give you a “leg up” in framing sensible
questions.
 Four uses in qualitative communication research:
1. Often useful to study the “career” of a document
 Can lead to insights about the ways in which documents help
coordinate interpretations and behaviors.
2. Official documents are a site of claims of power, legitimacy, and
reality.
 It is important to study these texts in relation to the institutional settings
in which they are constructed, interpreted, and used.
 Emphasis in the spatial, temporal, and practical contingencies associated
with the texts.
3. Communication events are encoded and preserved as documents.
4. Many organizations and groups create documents of some sort for
public consumption.

TYPES OF DOCUMENTS
 Primary sources – comprises the testimony of eyewitnesses
of the event described.
 Usually preferred when the facts of an event need to be establish.
 Secondary sources – based on indirect (hearsay) evidence
 Media stories about an event
 Expressed opinion of social actors affected by but not directly
involved in an event.
 Records – any written or recorded statement prepared by or
for an individual organization for the purpose of attesting to
an event or providing an account.
 Personal document – any written or recorded material other
than a record that was not prepared specifically in response
to a request from an inquirer.
 Less constrained by organizational demands and requirements
that can be read as genres of self-expession.
 A text considered primary in one context could be a
secondary source in another context.

ADVANTAGES AND DISADVANTAGES
 Informational richness – contextually relevant and
grounded in the contexts they represent.
 Richness of documents derive not only from the amount of
information but also the quality.
 Availability – documents are almost always available,
on a low-cost or free basis.
 Transition from stand-alone, analogical media to networked,
digital media has had a major impact.
 Physical and online archives are both potential sources of bias and
are both at the same risk of being incomplete, difficult to search, or
purged.
 The entity that controls access may impose conditions on
the ability to obtain, photocopy, or quote them.
 Ethical and legal questions about how it can or should be used for
research purposes.
 Researcher should try to get the originals, but photocopies will suffice
for most purposes.
 Describe the document’s origins and history, who issued it, when and

 Nonreactivity of the data – Many documents came into being
at the end of a lively social process or as a result of a deliberate
and/or creative thought process by an individual.
 May have gone through many iterations during it span of life
and maybe even morphed into something different.
 In most cases by the time the research arrives at a scene the
document is a relatively inert, stable object.
 Major advantage – endures unchanged across time and space.
 A document’s meaning may vary across times, places, and readers
 Truth value – To the extent that the information contained in
documents is vetted for accuracy, is used as a reliable basis for
organizational decisions and actions, and/or is validated by
internal or external authorities, it may be regarded as a
trustworthy source.
 Unless a problem with a document’s accuracy is noted,
challenged, and repaired, or unless the users are indifferent
about its truth value, it may still be regarded as a trustworthy
source.
 The evaluation of a document’s truthfulness depends on your

The Method of History
Smith, M. Y. (1989). The method of history. Research methods in mass communication. Englewood
Cliffs, NJ: Prentice-Hall, Inc.
• The issue of quantification in history raises the
false dichotomy of whether the historian may
properly use quantitative methods.
• Whether historians will indulge in impressionistic research
and the subsequent necessary empty rhetoric.
• Impressionistic history – random note-taking without
applying system and rigor to the data gathering.
• Whether communication historians will discipline
themselves to the rigor of empirical study.
• Empirical history – the application of system and rigor to the
study of the past.
• The empirical historian fully recognizes that a systematic account
of relationships among events and/or persons is only a history of
communication and not the history of communication.
• Seeks a verisimilitude rather than objective truth.
• David Fischer – An empirical historian is one who asks an open-ended
question about past events and answers it with selected facts which are
arranged in the form of an explanatory paradigm.

FORMULATING THE RESEARCH QUESTION
 Thorough knowledge of general history and
communication history.
 Practical approach: Saturation in historical readings and
immersion in communication history data.
 Begins when individual interest leads a researcher to see
material about a general communication subject and then,
when fortified by a general understanding of events, immerse
itself in a general topic.
 Who, What, When, Where, Why & How
 Pursuit of Evidence
 Historians cannot properly begin formulating a
research hypothesis before the immersion process
narrows into “guided entry”
 The researcher focuses in a more specific part of the
data delimiting the data to be studied.

GUIDED ENTRY
 The process of general question forming begins.
 A question must be worth asking
 A historical question ought to be neutral
 Receptive to both yes and no answers.
 All terms in the historical research question must be
capable of being operationally defined
 The operational definition establishes categories or
classifications.
 The language of the question ought to be explicit and
precise
 Questions should not be based on unproved assumptions.
 The research question ought to be flexible enough that
it can be refined as the study progresses.
 Questions should be conceived with imagination and
honesty to define clearly the relationship being
observed within certain parameters of time and place in

AUTHENTICATING THE DATA
 An authentic document is the product of the eyewitness, not a
forgery.
 Evidence must be affirmative.
 All evidence must be clearly established by the testimony of independent
witnesses.
 The historian works by accretion of evidence from primary sources.
 Primary sources are eyewitness testimony with expertise in the area – no
need to be original as long as is firsthand testimony – person or thing
present at the events and a narrator of those events who lacks bias and
motives of self-interest.
 Records – intentional eyewitness testimony; may function to obscure the
interpretation of a historical event. The motivation for the recordkeeping must be
examined, as well as the content.
 Relics – although also eyewitness testimony, were recorded for purposes other
than to form a database for future historians.
 The historian should first try to establish the date of the document and
determine whether the materials (ink, paper, type of film) were available at
that time.
 The source’s style of writing should be analyzed.
 The historian should ask whether the views expressed in the document are in

 Anachronistic references to events in the document
should also be searched.
 The document should have come from a line of well-
authenticated, reputable owners, or it should have been
found where it logically ought to nave been.
 Once data have been authenticated, the historian must
ask how much of the authentic evidence is credible and
to what degree.
 “Credible testimony is not that it is actually what happened,
but that it is as close to what actually happened as we can
learn from a critical examination of the best available
sources.” Louis Gottschalk
 History of American Revolution – George Washington’s papers
 History of Watergate – memoirs of Richard M. Nixon
 The researcher should consider the tone of the news
medium and how that affects reporting.
 The news medium’s general adherence to reasonably
acceptable standards of ethical and professional journalism,
and the degree to which the publisher controls the
newsroom.

 A particular caution in weighing credibility of evidence is the
effect of language, which as any cultural phenomenon,
changes over time.
 Primary documents must be studied in context.
 Texts tell us what an actor did, not necessarily what it thought.
Therefore, subjective documents may intentionally obscure what
the actor did (Behavioral evidence v. Ideational evidence)
 Circumstantial evidence
 Should be used with care – Historians need to consider carefully
the deceptive possibilities of circumstantial evidence
 Should be considered inferior to direct evidence.
 Secondary sources – can be useful in establishing the
historical context for the research problem.
 Research question and hypothesis that might be tested.
 The techniques for recording evidence should be mentioned
briefly.
 A full bibliographic reference must accompany every shred of
evidence.
 The important aspects of any system of data recording are
accuracy, completeness, and permanency.

SELECTING ANALYTICAL STRATEGIES
 Decisions on data analysis are made in response to the
research question.
 The decision should have been made whether the historian uses the
more traditional documentary analysis or quantitative analysis or both.
 Ideally, multiple measures are used to reduce uncertainty in interpreting results
when the historian reaches the same general conclusions through several
measures.
 Historians can use social science methods as a tool only if they have mastered
historiography first.
 In considering data the historian must consider the nature of
historical reasoning, the documentary approach to data
analysis and its advantages with incomplete data, the
applicability of quantitative methods, forms of data presentation,
and literary style.
 Historical reasoning process is a process of adductive
reasoning in the simple sense that a satisfactory explanatory fit
is obtained
 Adduction permits the historian to respond to research questions with

 To arrive at a verisimilitude of the past, the historian
applies traditional documentary analysis:
 To evidence too skimpy to permit quantification.
 To parts of quantified evidence to enhance and illuminate the
historical record.
 In the course of extracting the credible material from the
authentic data, the historian:
 Notes the primary source’s interpretation of the situation and
the source’s behavior in the situation.
 The historian’s analysis includes reports on the feedback to the
primary source
 It should be noted whether the feedback was the intended result of the
source’s activity or whether it disagreed with the source’s aims.
 Defines other observer’s views of the situation and traces
the anticipated and the unanticipated consequences of the
source’s activity
 Noting the relationships among those views and the primary sources
ideational and behavioral aspects, as well as the feedback.
 The information collected is organized in meaningful
fashion, depending on the structure of the research
question.

 Two cautions should be exercise by the historian:
 Evidence should be interpreted in terms of the postulated
past time
 Not to fall into the excesses of psychohistorians
 Quantification complements the traditional documentary
method of analysis injecting rigor and system into it to
make more reliable generalizations.
 Content analysis permits systematic and objective
classification of communication content into categories
according to specified characteristics so that inferences can
be made from the data.
 Every historian makes some leap of faith between
evidence and generalization. However, the historian
cannot predict on the basis of generalizations because
its legitimate inferences are incapable of being tested by
means of the full hypothesis.
 Exception – Case study, where little generalization, if any, is
possible. Has only limited value as a pretest for further

 In making that leap of faith the historian goes show
high probability
 The balance of chances that, given such and such
evidence, the event it records happened in a certain
way, or, in other cases, that a supposed event did not in
fact take place.
 The historian is the judge of what constitutes high
probability
 In a quantitative approach, this is indicated in advance
and tested statistically
 In either case, the reader should be able to accept or
reject the judgment made by the historian from the
evidence presented.
 Difficulties in dealing with causal antecedents of
historical events can be controlled in the question-
forming process when operational definitions are
explicated.

REPORT AND DATA PRESENTATION
 Should reflect the rigor and precision applied to the data,
without, however, losing the literary quality of the report.
 Case studies – limited range of data
 Article, series, monographs or books – broad range of data
 Should begin with a conceptualization of the problem
and a clear statement of the research question, followed
by a literary survey appropriately integrated into the
narrative and an explanation of the research method, full
and precise presentation of the evidence responsive to
the research question, and finally, the generalizations in
response to the research question.
 “You tell’em what you are gonna tell’em; you tell’em; and
then you tell’em what you told’em
 Rigorous, systematic history does not have to be dull
history
 Communication history should be written in the best literary

Evaluating Information: Historical/Rhetorical-
Critical Methodology
Chapter 5 of Hocking, J.E., et al., (2003) Communication Research. Allyn and Bacon. Pp. 104-121
• Three traditional classifications of research methods:
• Historical/Rhetorical critical – general way of studying communications.
• Studies a concept, event or person in great detail.
• It is used by all researchers at some stage.
• Reflects the careful and systematic assembling of all the authentic facts that may have a
bearing on the target of study.
• Descriptive
• Experimental
• Your ability to generalize and to establish or test scientific theory is expanded.
• These methods can be divided into two groups:
• Humanistic – Qualitative
• Criticism = creative activity that relies on the interpretative and observational powers of the
critic.
• Scientific – Quantitative
• These methods function by examining communication in a way so as to point
out aspects of the communication that the reader should be aware of.
• In choosing a type of research and method the researcher indicates a philosophical
stance toward the phenomenon studied.
• The historian’s methodology is tied to how the information is gathered.

AN OVERVIEW OF HISTORICAL RESEARCH
 Requires following systematic rules and
procedures rigorously in order to order
knowledge in such a way as to pass several
tests of critical analysis.
 Correct
 Appropriate
 Builds upon known facts and evidence in such
a way as to understand better the quality of
what happened and how certain events may
have contributed to that event.

UNDERSTANDING THE MEANING OF HISTORY
 The means by which the researcher deals with
the latent meaning of history.
 Must get as close to the original source of
information about the original event as possible.
 Chronology of events or reflection of an event
occurring in time and space
 Traces the events as they occurred in some time pattern
 Road map that provides insights into cause-effect relationships but
not to understand what happened.
 Spatial dimension of history – the events had to occur
somewhere
 Helps to better key in on the unique contributions of the person,
event, or institutions and make analysis.

AREAS OF POTENTIAL HISTORICAL RESEARCH
 Historical study falls into seven established areas and two suggested
potential areas of study:
1. Biographical or biographical/critical studies – concentrates on
the life of a particular person exploring not only what this person has
said or done but also its biographers.
2. Movement or idea studies – center on the development of social
movements with a more sociological focus.
3. Regional studies – examine the impact of geographical location.
4. Institutional studies – center on the history of particular
institutions.
5. Case study – center on specific events, institutions, or persons at a
specific point in time.
6. Selective studies – examine one particular aspect of a complex
process.
7. Editorial studies – focuses on translation of texts or discovery of
new texts.
8. *Bibliographic study – focuses on documentary research and the
creation of an information base.
9. *Study of sources – study of those with whom the person studied

ADVANTAGES AND DISADVANTAGES
 Advantages
 Ability to know how the event turned out.
 Can ascertain whether or not the source agrees or disagrees
with history as it happened.
 Certain patterns that may help to explain why others saw or
reported the event or person differently may become evident.
 Disadvantages
 The problem of determining accuracy
 Records may be inaccurate, incomplete or lost
 The result of new sources of data acquisition and
retrieval available.
 Rather than reading all the documents identified, we may rely
on the services of others.
 We may wait forever to conclude

HISTORICAL METHODOLOGY
 Historical Facts
 The researcher is seeking the facts as they might have
been known at that time, which may be different from
those we believe or know today.
 Facts are created through human perception, perceptions that
may change as times or locations change. Are derived directly
or indirectly from historical documents and verified by others.
 A historical fact is something the researcher sees and uses to make
a particular event clear that is dependent upon verification.
 Historical Documents
 Any evidence used to establish or help explain the
target of study.
 Something the researcher seeks that may add to its knowledge
of the target of study.

 Historical Research
 Blends the data obtained from various sources into a lucid and
flowing narrative in a manner that illuminates the communication
and its effects. Its major functions are:
1. Set out to find all available information
2. Scrutinize that information in terms of accuracy
3. Get the story out
 Sources, Information, and the Test of Evidence
 Primary evidence – evidence from the source itself.
 Testimony, documents, and evidence that comes from those actually engaged
in the activities of the period.
 Secondary evidence – evidence from those of the time period
who may have observed but did not actually engage in the
activities in question.
 Sources whose credibility or testimony cannot be verified through primary
sources.
 Researcher must establish the context of the source and should carefully
cross-check it for accuracy.
 Tertiary evidence – evidence based on the accounts of the
primary sources.
 May yield important clues to securing secondary or primary evidence.
However, is accepted less frequently than secondary sources. Therefore, it
must be approached with greater caution and must be viewed skeptically it.

 External evidence
 Finding two or more forms of the same material is not un
common.
 Authenticity could be established through scientific tests on
the documents found.
 Examination of handwriting, paper and ink used, and whether the
language reflects the period under study.
 The source’s training as an observer, and establish the source’s
relationship to the event, person, or institution.
 Ascertain whether you are working with the source’s own words.
 Speeches – be sure that the text you are working with is the one actually presented.
 Speeches before the Congress – be sure that the indicated speaker actually delivered
the speech.
 Internal evidence – evaluates the statements made within the
material following external testing.
 Source’s literal meaning – credibility of the document.
 The researcher must delve into the symbols and slang of the period
to gain an understanding of what the source was trying to say.
 Must take into account ambiguity, the use of figurative language, statements taken out
of context, whether the statement reflects the target of the study, and whether or not is
believable.
 Believability is based upon confirmation

 Skepticism
 The acceptance or rejection of evidence must be
approached with skepticism.
 Conflicting testimony and information, repression of the truth
by the ones in power.
 Analytical Strategies
 Deductive approach – reasoning from an idea or thesis,
looking for evidence to support a thesis that would lead
to new ideas or thesis statements.
 Inductive approach – when a particular piece of
evidence derives thesis or ideas concerning
relationships or events.
 Adductive approach (Lateral thinking) – the process of
adducing answers to specific questions so that a
satisfactory explanatory fit is obtained.
 Holistic approach that examines the interactive impact of all
possible causes rather than singling out for specific attention.
 When the RQ suggests it is appropriate to do so, the
historian can turn from a humanistic, qualitative approach
to a quantitative, social scientific, behavioral approach.

RHETORICAL/CRITICAL METHODOLOGY
 Critical method is part of the humanistic approach to
historical research.
 Is also considered an extension of the tests of internal evidence or
criticism the researcher uses when examining the question of how
well the act was accomplished.
 In conducting a critical study the researcher should
examine at least four aspects of the communication:
1. the results
 What was the purpose of the communication? What was the true
purpose of the communication? What connection can be found
between the communication and the results?
2. the artistic standards
 Refer to the use of appropriate rhetorical methods and principles
to examine how effectively or gracefully the intended affect of the
communication was accomplished.
3. the basis of the idea
4. the motives and ethics behind the communication.

RHETORICAL CRITICISM/CRITICAL DISCOURSE
 Farrell’s three methods of performance of discourse, each
slightly different in its approach to the communication
event or person:
1. Symptom criticism – approaches the communication
event or person as a symptom of some larger social
concern.
 Tends to be more sociological examining the social features or
themes underlying the culture of the communication.
2. Didactic criticism – a more traditional model that follows
the notion that the study of the event or person is the
model.
 Interested in the particular communicator and how this uses
communication as a tool.
3. Thematic criticism – is interested in how the
communication is constructed.

CONDUCTING CRITICAL EXAMINATION OF
COMMUNICATION
 The researcher begins by describing what has happened.
It focuses on two sets of factors tied to the rigor of the
method:
1. Extrinsic factors – the researcher seeks to:
 Describe and analyze the period in which the communication
occurred.
 Describe the audience both in terms of the intended and the
real audience.
 Describe and analyze the occasion for the speech
 The researcher describe and analyze the speaker giving
attention to its background and training.
 The communication itself is analyzed.
2. Intrinsic factors – analysis of the communication itself
and the evidence with which it was constructed.
 How the logic fits with the period which the communication
was presented.
 Intrinsic features stress the individual’s skills at creating and
presenting the communication.

LIMITATIONS OF HISTORICAL RESEARCH
 The researcher must be prepared for
fragmentary and incomplete records.
 Facts change as language and meaning
change.
 The researcher’s personal bias, selective
perception of what is right and what is not, and
faulty interpretation.

Describing Information:
Content Analysis
Chapter 8 of Hocking, J.E., et al., (2003) Communication Research. Allyn and Bacon. Pp. 170-192
• Content analysis is the systematic study and
quantification of the content or meaning of
communication messages that involves the
creation of categories that are designed to allow
a particular research question to be answered by
counting the instances of content within a
message that fall within a certain predetermined
category.
• There must be a good reason to use it – it must help
us answer an important research question.
• Can be used to add accuracy and precision to
observations.
• Provides a method for quantifying the messages that
were produced during the event.
• Can be used as a starting point for establishing the
effects of particular messages.

STEPS IN CONTENT ANALYSIS
 The methods of follow directly from and are largely guided by
the research question. The steps are:
1. A literature review, which would help to focus and clarify the
research questions or hypothesis that guide the study.
 The research question delimits the content to be analyzed in such a
way as to make the research doable and valuable.
2. Define exactly what messages are to be studied.
 Here your population or universe of messages is defined.
3. The selection of a sample.
 Messages that are representative of the entire body of messages
available for analysis.
 Must be large enough to be representative yet small enough ti be studied
feasibly with available resources.
4. Define the units of analysis
 The thing that is actually counted and assigned to categories.
5. Create categories into which each unit of analysis would be
assigned.
6. The coding of the messages
 Carefully, systematically, and objectively

DEFINING THE METHOD
 Two directions that the researcher can take:
1. Manifest aspect of the communication – the content
exactly as it appears.
2. Latent aspect of the communication – the deeper
meanings that are intended or perceived.
 At the coding level the researcher must be limited to manifest
content, the latent question comes in the question goes
beyond the pragmatic aspect of the messages.
 In preparing to begin a research study you must:
1. Decide what you will be studying. What would be your
unit of analysis?
2. How to sort the data. What would be your category
system?
3. Establish a sampling procedure.
4. Acquire the data.

 Delimiting the population
 Establish certain boundary conditions that will limit
your search.
 Which one is appropriate?
 This will limit the generalizability of the findings.
 Units of analysis
 What you actually count and assigned categories.
 Their definition depends on how the variable(s) under
study are operationally defined.
 Barelson’s five major units (Table 8.1 p.178):
Unit Description Example
Word Manifest content Nouns, Proper names, Utterances (for example, “uh-huh”)
Theme Latent content An idea, concept or thesis (Self-esteem, Democracy)
(Subthemes)
Character An individual Male/Female, Occupation, Ethnicity (Communicative
behaviors)
Time/Space Physical or temporal
measure
Column inches, Type size, Air time provided a program

 How to separate the units of analysis into meaningful
categories.
 Five general guidelines – underlie the systematic and
objective nature of content analysis.
 Content analysis categories should:
1. Reflect the purpose of the research
2. Be exhaustive
3. Be mutually exclusive
 All units fit into only one of the categories
4. Allow for independence
 The assignment of a unit to one category should not influence the assignment
of other units.
5. All categories should reflect one single classification principle
 Counting – how many
 The most obvious and simple quantification is to count the units
being analyzed in each category.
 Availability of data – decisions should be considered at
the research question/hypothesis stage.
CREATING CATEGORIES FOR ANALYSIS

SAMPLING
 Census – conducted when the population to be
studied is either not large or is composed by a
limited number of messages.
 Random and Systematic Sampling
 Random sampling – research question would
concentrate on understanding the messages of this
particular person.
 Limited application.
 Systematic sampling – involves compiling a lost of
messages and the selection of a random starting
point and a constant skip interval.
 Not concerned with change in messages over time.

QUANTIFICATION
 Decisions on how to quantify or “code” the
categories should be approached early in the
planning process.
 One of the major goals is that all data should be
potentially quantifiable.
 Nominal quantification - The researcher is
merely naming (nominal) the characteristics of the
variables.
 Ethnicity, gender, job, category, region of the country…
 Ordinal quantification – requires some decision
regarding ranking the data.
 Requires the content to be placed in categories that are
mutually independent and ordered in some way.

CODING RELIABILITY
 Coders should always use a standardized coding
form to ensure reliability and accuracy from coder
to coder.
 Researcher must be prepared to establish that the
coders are consistently placing data into the correct
categories.
 Intercoder reliability – can be enhanced by making certain that
all coders understand the definition of the categories and how
and why the units of analysis were chosen.
 Coders should not be able to know what the researcher is seeking -
they could introduce their bias (even unintentionally)
 Coding reliability – is enhanced when the units of analysis are
clearly defined and they are assigned to mutually exclusive
and exhaustive categories.
 Pilot-test phase of research
 Practice sessions are important – is helpful to have the categories and
the operational definitions tested as well as the coders trained and

RELIABILITY COEFFICIENTS
 Once the data have been gathered some estimate of
coding reliability must be determined.
 Holsti’s reliability formula – an example of a handwritten
reliability coefficients.
 Most researchers want to achieve a coefficient if about 90 –
This assumes that independent coders have been employed
 In some cases you may use only one coder.
 A drawback of this method is coding the message which can
be quite complex and requires both extensive sessions and
checking of the coding by the researcher.
Reliability = __2M__
Ni +Nj
M = Total items agreed upon
Ni = Total items i selected
Nj = Total items j selected

CODING VALIDITY
 Whether a coding system measured what you
want it to measure. Three possible sources of
invalidity:
 (1) Definition; (2) Category; and (3) Sample
1. Definitional Sources of Invalidity
 When units of analysis are defined, a set of boundary
conditions are created which serve to limit what you will accept
in the analysis.
 Your theoretical perspective will serve as the guiding force in
definition – You may or may not reflect the same definitions others
would have chosen.
2. Category Sources of Invalidity
 Definition is closely tied to category selection
 Category systems should be exhaustive and mutually exclusive and
must reflect the purpose of the research.
3. Sampling Sources of Invalidity
 If your sample represents the universe of the population from
which it was drawn and the analysis of the sample is valid,

 Use of Computers in Content Analysis
 Can provide speed and precision not possible by
human beings.
 It does not replace anything that the researcher could do,
but it does make it easier to accomplish the chores
associated with coding the data – can’t establish
meaning.
 Advantages and Limitations of Content
Analysis
 Time consuming and complex.
 Does not, and cannot, asses the effects of
communications messages.
 Provides the manifest content, which may lead to
future research on the latent meaning of that
content.

Fallacies of Factual Significance
Chapter 3 of Fischer, D. H. (1970). Historian's Fallacies. Pp.64-100. Harper Perennial.
• To write and read history is to engage in an endless process of
selection
• The process of selection and the criteria of factual significance
can and must be clarified
• Sampling – selection of representative facts of a certain
predetermined kind, within a closed universe of investigation
• Selection of the kinds of facts to be sampled – determination of
fundamental criteria of factual significance
• The purpose of historical inquiry is not to vindicate a method but
to discover what actually happened.
• Every efficient means to this end is legitimate, but none alone can be
erected into a standard of legitimacy.
• Few criteria of factual significance are inconsistent with empiricism in
history
• The fallacy consists not in the criteria per se but rather in an attempt to
combine them with methods and objects of empirical inquiry.

CRITERIA OF FACTUAL SIGNIFICANCE
 A true standard of factual significance is one which is
generated by a sound model of historical explanation.
 A historical explanation is an attempt to relate some
historical phenomenon in a functional way to other historical
phenomena.
 Nothing is self-explanatory – a properly executed explanation relates
the unknown to the known in a series of orderly inferences.
 A fact becomes significant in proportion to its relevance to an
explanation model.
 A significant fact is one which helps historians to make a case for their
explanation and to communicate its nature to the reader.
 Criteria of factual significance should not be
methodological, but substantive in nature
 Grounded on the nature of the problem itself and not in the
tools of problem solving
 It should be empirical, capable of fulfillment, must be
made explicitly, and must not violate what philosophers
call the principle of nonvacuous contrast.

THE HOLIST FALLACY
 Is the mistaken idea that a historian should
select significant details from a sense of the
whole thing.
 It would prevent a historian from knowing anything
until it knows everything.
 Its evidence is always incomplete
 Its perspective is always limited
 The thing itself is a vast expanding universe
 A historian who swears to tell nothing but the whole truth, would
thereby take a vow of eternal silence
 The researcher who promises to find the whole secret for himslef
condemns himself to perpetual failure.
 The whole truth cannot ever be attained –
Historians are bound to tell the best and biggest

THE FALLACY OF ESSENCES
 Begins with the old idea that everything has
something deep inside, a profound inner core
reality called essence.
 Facts are significant in the degree to which they display
the essence of the entity in question.
 Supplies a sense of completeness and it encourages a
sense of certainty.
 The existence of essences cannot be disproved by any rational
method.
 What it is possible to demonstrate is that a belief in essences
involves an empiricist in certain difficulties
 Essentialism is very common in historical writings
– closely related to holism.
 Knowledge of the essence of a thing implies knowledge
of the whole thing – Impossible

THE FURTIVE FALLACY
 Erroneous idea that facts of special significance
are dark and dirty things and that history itself is
a story of causes mostly insidious and results
mostly invidious.
 Causal reduction where reality is a sordid secret
thing and history happens on the back – Reality is
always the underlying fact; always something more
(or sometimes less) than meets the eye
 Combines a naïve epistemological assumption that things
are never what they seem to be, with a firm attachment to
the doctrine of the original sin
 Mental illness commonly called paranoia
 Self-fulfilling quality – Men who believe it begin to act furtively
 Errors and distortions in interpretation and assumptions

THE MORALISTIC FALLACY
 Selects edifying facts – It would make history a
handmaid of moral philosophy.
 This fallacy is exceedingly difficult to define precisely;
 All historians must and would make value judgments in their
work
 Some historians moralize upon past events in ways which
are inconsistent with empiricism
 Important to distinguish between acceptable moral judgment (a priori)
and unacceptable moralizing.
 The historian can adjust its project to its values in such a
fashion as to neutralize or to control its moral
preferences:
 Make its values as fully explicit to himself and others as
possible
 Design a research problem in which its values allow an open

 Selects immediately and directly useful facts in
the service of a social cause. How utility can
function in history?
 An attempt to combine scholarly monographs and
social manifestoes in a single operation.
 Doubled trouble : distorted monographs and dull
manifestoes
 Scholars who take a pragmatic view of their
task and collect facts that are weapons for a
cause are faced with the problem that some
facts exist which are useful to their enemies.
THE PRAGMATIC FALLACY

 Selects beautiful facts, or facts that can be built
into a beautiful story, rather than facts that are
functional to the empirical problem at hand.
 An attempt to organize an empirical inquiry upon
aesthetic criteria of significance, or conversely in an
attempt to create an object of art by an empirical
method.
 Art ≠ history – for art external reality is irrelevant
while history is an empirical search for external
truths.
 Any attempt to conduct research according to aesthetic
standards of significance is either to abandon
empiricism or to contradict it.
 A historian who omits all the ugly “buts,” “excepts,” “perhaps,”
THE AESTHETIC FALLACY

 Idea that the facts which count best count most.
Assumes that facts are important in proportion
to their susceptibility to quantification.
 Not quantification per se – to quantify is merely to
count.
 Counting has always been useful and it will ever be.
 Historians should quantify everything they can, by the
best statistical method.
 Conclusions tend to be inaccurate and a little
superficial.
THE QUALITATIVE FALLACY

 Erroneous idea that facts which count best, count
least. Begins with the assumption that regularities
do not exist in history, or do not exist significantly
 Hostility to quantification – counting does not coexist
with ideological preconceptions
 Every historical event is unique
 Commits to holism and essentialism
 Most apparent among conservative scholars where
attendant metaphysical bias is most clearly visible
 No historian really treats all facts as unique; it treats
them as particular
 Every fact and event require reference to commonalities before
they acquire meaning
THE ANTINOMIAN FALLACY

 Committed by any scholar who abdicates its
arduous responsibility of rational selection and
allows the task to be performed for it by time and
accident.
 This would reduce scholarship to mere sciolism
 A smattering of superficial nuggets of knowledge without point
or plan or purpose
 Careless, casual impressionism
 Exceedingly ineffectual. The reader is diverted by
it, but so is the historian, from its difficult obligation
to select factual statements according to explicit
criteria of significance and to tell truths which are
as clear and comprehensive as mortal intelligence
THE FORTUITOUS FALLACY

Producing Data: Analyzing Material, Culture & Documents

More Related Content

What's hot (20)

Similar to Producing Data: Analyzing Material, Culture & Documents (20)

More from Dr. Aitza Haddad Nuñez (20)

Recently uploaded (20)

Producing Data: Analyzing Material, Culture & Documents