Ch04 General Issues in Research Design
- 1. 1 General Issues in Research Design
- 2. OUTLINE Introduction Causation in the Social Science Validity and Causal Inference Units of Analysis The Time Dimension How to Design a Research Project The Research Proposal
- 3. 3 •Causation, units, and time are key elements in planning a research study •As social scientists, we seek to explain the causes of some phenomenon (e.g., crime) •Often, criminal justice researchers want to find out something that involves questions of cause and effect
- 4. 4 •Causation is the focus of explanatory research •Cause in social science is inherently probabilistic •Certain factors make crime/delinquency more or less likely within groups of people •Two models of explanation •Ideographic – Lists the many, perhaps unique considerations behind an action •Nomothetic – Lists the most important (and fewest) considerations/variables that best explain general patterns of cause and effect
- 5. 5 •Assessing an idiographic explanation – posited by Maxwell (2005) •How credible and believable it is •Whether alternative explanations (“rival hypotheses) were seriously considered and found wanting •Assessing a nomothetic explanation – posited by Shadish, Cook, & Campbell (2002) •Empirical relationship between variables •Temporal order (cause precedes effect) •No alternative explanations – no spurious other variable(s) affecting the initial relationship •Any relationship that satisfies all these criteria is causal
- 6. 6 •Within the probabilistic model, two types: •Necessary cause – Represents a condition that must be present for the effect to occur (being charged before being convicted) •Sufficient cause – Represents a condition that, if it is present, will pretty much guarantee that the effect will occur (pleading guilty before being convicted)
- 7. 7 •Scientists assess the truth of statements about cause by considering threats to validity. •When we make a cause-and-effect statement, we are concerned with its validity – whether it is true and valid •Certain threats to the validity of our inference exist •These are reasons why we might be incorrect in stating that some cause produces some effect
- 8. 8 •Refers to our ability to determine whether a change in the suspected cause is statistically associated with a change in the suspected effect •Are two variables related to each other? •Researchers cannot have much confidence in statements about cause if their findings are based on a small number of cases
- 9. 9 •An observed association between two variables has internal validity if the relationship is, in fact, causal and not due to the effects of one or more other variables •Generally due to non-random or systemic error •The threat to IV results when the relationship between two variables arises from the effect of some third variable •Example: drug users sentenced to probation over prison recidivate less
- 10. 10 •Concerned with whether research findings in one study can be replicated in another study, often under different conditions •Do the findings apply equally in different settings (locales, cities, populations)? •Kansas City evaluation found sharp reductions in gun-related crimes in hot spots that had been targeted for focused police patrols •Indianapolis and Pittsburgh launched similar projects
- 11. 11 •Concerned with how well an observed relationship between variables represents the causal process •Refers to generalizing from what we observe and measure to the real-world things in which we are interested •e.g., close supervision of officers more tickets? •e.g., Kansas City Preventive Patrol Experiment, “police visibility”
- 12. 12 •The four types of validity threats can be grouped into these two categories •Bias – Internal Validity and Statistical Conclusion Validity threats are related to systematic and nonsystematic bias •Generalizability – Construct Validity and External Validity are concerned with generalization to real-world behaviors and conditions
- 13. 13 •Temporal order: which comes first? •A statistical relationship exists, but underlying causes affect both drug use and crime (Internal Validity threat) •What constitutes drug use? Crime? (Construct Validity threat) •How will policy affect drug use and crime? •A crackdown on all drugs among all populations will do little to reduce serious crime
- 14. •Bridges idiographic and nomothetic approaches to explanation by seeking to understand how causal mechanisms operate in specific contexts •Studies how such influences are involved in cause-and-effect relationships •Exhibits both ideographic & nomothetic approaches to explanation •"Can the design of streets and intersections be modified to make it more difficult for street drug markets to operate?"
- 15. 15 •What or who is studied •Individuals - Police, victims, defendants, inmates, gang members, burglars, etc. •Groups - Multiple persons with same characteristics (gangs, cities, counties, etc.) •Organizations - Formal groups with established leaders and rules (prisons, police departments, courtrooms, drug treatment facilities, etc.) •Social artifacts - Products of social beings and their behavior (stories in newspapers, posts on the Internet, photographs of crime scenes, incident reports, police/citizen interactions)
- 16. 16 •Ecological fallacy - Danger of making assertions about individuals based on the examination of groups or aggregations •Poor areas = more crime, therefore poor people commit more crime •Individual fallacy – Using anecdotal evidence to make an argument •O.J. Simpson court resources •Reductionism - Failing to see the myriad of possible factors causing the situation being studied
- 17. 17 •Time sequence is critical in determining causation •Time is also involved in the generalizability of research findings •Observations can either be made more or less at one point, or stretched over a longer period •Observations made at more than one time point can look forward or backward
- 18. 18 •Observing a single point in time (cross-section) •Simple and least costly way to conduct research •Typically descriptive or exploratory in nature •A single wave of the National Crime Victimization Survey (NCVS) is a descriptive cross-sectional study that estimates how many people have been victims of crime in a given time
- 19. 19 •Permit observations over time •Trend – Those that study changes within some general population over time (UCR) •Cohort – Examine more specific populations as they change over time (Wolfgang study) •Panel – Similar to trend or cohort, but the same set of people is interviewed on two or more occasions (NCVS) (panel attrition)
- 20. 20 •Gun ownership and violence study by Swiss researcher Martin Killias (1993) •Compared rates of gun ownership as reported in an international crime survey to rates of homicide and suicide committed with guns •May be possible to draw approximate conclusions about processes that take place over time, even when only CS data is available •When time order of variables is clear, logical inferences can be made about processes taking place over time
- 21. 21 •Asks people to recall their past for the purpose of approximating observations over time •People have faulty memories; people lie •Analysis of past records also suffer from problems – records may be unavailable, incomplete, or inaccurate •Prospective research – longitudinal study that follows subjects forward in time (Widom – child abuse/drug use)
- 22. 22 •Cross-sectional study = snapshot – an image at one point in time •Trend study = slide show – a series of snapshots in sequence over time, allows us to tell how some indicator varies over time •Panel study = motion picture – gives information about individual observations over time
- 23. 23 •Designing research requires planning several stages, but the stages do not always occur in the same sequence •Beginning points for a line of research •e.g., interests, ideas, theories, new programs •Why does something occur? •Why is this how it is? •What about this possible program? •These questions may lead to others you might like to explore
- 24. 24 •Find out what research has been done •Read newspaper stories, journal articles, check out the Internet, talk to relevant people •Figure out your objective & intended audience •Generally, your purpose for undertaking research can be expressed in a report •You should be clear about the kinds of statements you will want to make when the research is complete
- 25. 25 •Specify what you mean by the term in your research - this doesn't necessarily mean you have to settle for a single definition •What do you mean by the concept being studied? •If you are going to study fear of violent crime: •What is considered “violent crime”? •What is considered “fear”? •You need to specify ALL the concepts you wish to study
- 26. 26 •A variety are available, each has strengths and weaknesses, choose one after considering the specific concept you want to study •Interviews, surveys, field research, content analysis, official records •The best studies utilize more than one research method
- 27. 27 •Create concrete ways of actually measuring your concept •Fear and violent crime: •Questionnaire item: how safe do you feel in your house, downtown at night, etc. •Official records of violent crime incidents in that neighborhood, downtown, etc.
- 28. 28 •Exactly who or what will you study? •Population – Group (e.g., of people) about whom we want to be able to draw conclusions •Since it is generally unfeasible to study ALL members of that population, how will you take a representative sample? •Fear and violent crime: Will you include elderly? Teenagers and kids? A particular race or gender?
- 29. 29 •Collect the empirical data •Make the observations that you will process, analyze, interpret, apply, and review
- 30. 30 •You have amassed a volume of observations in a form that usually is not easily interpretable •How will you make sense of the observations? •Code the data somehow
- 31. 31 •Manipulate the data •Goal is to draw conclusions that reflect on the interests, ideas, and theories which initiated our research project •Calculate percentages of those who have been victimized by violent crime, those who fear violent crime, differences among population subsets, etc.
- 32. 32 •Utilize the research you’ve conducted and the conclusions you’ve reached •Make your findings known to others •Develop policy to address your findings •Determine what mistakes were made that could be corrected in the future •Determine how your research might feed into future research
- 33. 33 •Begin by assessing: •Your interests •Your abilities •The resources available to you •What are you interested in studying? •What information is needed & how to obtain it •Review prior research in journals, books and governmental reports •“Triangulation”
- 34. 34 •Problem/Objective •Data Collection Methods •Literature Review •Analysis •Research Questions •References •Subjects for Study •Schedule •Measurement •Budget