Materials and methods
- 1. MATERIALS AND METHODS Chapter III Adopted by: CHARLANE M. DIASNES, LPT
- 2. MATERIALS AND METHODS • Research Design • Sampling Techniques • Research Instruments • Collecting the Data • Statistical Tools • Descriptive • Inferential
- 3. Limitations • Science is imperfect, especially when it is applied to human behavior and performance
- 4. RESEARCH DESIGN • Refers to the overall plan and scheme for conducting the study • Descriptive Comparative Correlation • Experimental
- 5. Types of Research • Descriptive Comparative Correlation • Case study • Cross-sectional study • Qualitative study • Experimental • True experimental designs • Quasi-experimental designs
- 6. Descriptive Research • Descriptive research usually makes some type of comparison contrasts and correlation and sometimes, in carefully planned and orchestrated descriptive researchers, cause-effect relationship may be established to some extent.
- 7. Descriptive Research • The purpose of this design is to describe the status of events, people or subjects as they exist • Present oriented • Not able to explain the cause-effect relationship but is able to provide clues to such relationships • Describes and interprets what is currently prevailing
- 8. Descriptive Research • Descriptive: investigator attempts to describe a group of individuals on a set of variables or characteristics. • Enables classification and understanding • Methods: survey research, case study, qualitative, developmental (natural history of something, patterns of growth and change), normative, evaluation
- 9. • Aims to establish significant differences between two or more groups of subjects on the basis of a criterion measure (ex. Compare the managerial effectiveness of three groups of managers A,B,C) • Limitations • The lack of control variables making them less reliable in terms of actual hypothesis testing • Unless the normative survey where the entire population is considered , conclusions drawn from descriptive designs are at best tentative Descriptive Research
- 10. Descriptive Research • Case study • Cross-sectional study • Qualitative study
- 11. Case Study Design • Often a description of a individual case’s condition or response to an intervention • can focus on a group, institution, school, community, family, etc. • data may be qualitative, quantitative, or both • Case series: observations of several similar cases are reported
- 12. Case Study Example • In 1848, young railroad worker, Phineas Gage, was forcing gun powder into a rock with a long iron rod when the gun powder exploded. The iron rod shot through his cheek and out the top of his head, resulting in substantial damage to the frontal lobe of his brain. Incredibly, he did not appear to be seriously injured. His memory and mental abilities were intact, and he could speak and work. However, his personality was markedly changed. Before the accident, he had been a kind and friendly person, but afterward he became ill-tempered and dishonest. • Phineas Gage’s injury served as a case study for the effects of frontal lobe damage. He did not lose a specific mental ability, such as the ability to speak or follow directions. However, his personality and moral sense were altered. It is now known that parts of the cortex (called the association areas) are involved in general mental processes, and damage to those areas can greatly change a person’s personality.
- 13. Case Study Design • Strengths • Enables understanding of the totality of an individual’s (or organization, community) experience • The in-depth examination of a situation or ‘case’ can lead to discovery of relationships that were not obvious before • Useful for generating new hypotheses or for describing new phenomena • Weaknesses • No control group • Prone to selection bias and confounding • The interaction of environmental and personal characteristics make it weak in internal validity • Limited generalizability
- 14. Cross-sectional Study • Researcher studies a stratified group of subjects at one point in time • Draws conclusions by comparing the characteristics of the stratified groups • Well-suited to describing variables and their distribution patterns • Can be used for examining associations; determination of which variables are predictors and which are outcomes depends on the hypothesis • eg. Does lead paint ingestion cause hyperactivity or does hyperactivity lead to lead paint ingestion?
- 15. Cross-sectional Study • Example: What is the prevalence of chlamydia in women age 18-35 in Cleveland, and is it associated with the use of oral contraceptives? • Select a sample of 100 women attending an STD clinic in the city of Cleveland • Measure the predictor and outcome variables by taking a history of oral contraceptive use and sending a cervical swab to the lab for chlamydia culture • A questionnaire may be used to gather information abut oral contraceptive history
- 16. Cross-sectional Study • Strengths • Fast and inexpensive • No loss to follow-up (no follow-up) • Ideal for studying prevalence • Convenient for examining potential networks of causal links • e.g., in analysis, examine age as a predictor of oral contraceptive use, and then examine oral contraceptive use as a predictor for chlamydia infection • Weaknesses: • Difficult to establish a causal relationship from data collected in a cross-sectional time-frame (Lack of a temporal relationship between predictor variables and outcome variables - Does not establish sequence of events) • Not practical for studying rare phenomena
- 17. Qualitative Study • Seeks to describe how individuals perceive their own experiences within a social context • Emphasizes in-depth, nuanced understanding of human experience and interactions • Methods include in-depth interviews, direct observations, examining documents, focus groups • Data are often participants’ own words and narrative summaries of observed behavior
- 18. Qualitative Study Example A researcher wants to understand how provision of healthcare to undocumented persons affects the people and institutions involved • In 3 communities, information is gathered from undocumented patients, FQHC primary care clinicians, specialists, and hospital administrators • Methods: in-depth interviews, key informant interviews, participant observations, case studies, focus groups
- 19. Qualitative Study Strengths • Data based on the participants’ own categories of meaning • Useful for studying a limited number of cases in depth or describing complex phenomena • Provides understanding and description of people’s personal experiences of phenomena • Can describe in rich detail phenomena as they are embedded in local contexts • The researcher can study dynamic processes (i.e., document sequential patterns/change) Weaknesses • Knowledge produced might not generalize to other people or other settings • It is difficult to make quantitative predictions • It might have lower credibility with some administrators and commissioners of programs • Takes more time to collect and analyze the data when compared to quantitative research • The results are more easily influenced by the researcher’s personal biases and idiosyncrasies
- 20. Quantitative/Qualitative • Quantitative research involves measurement of outcomes using numerical data under standardized conditions • May be used along the continuum of research • Qualitative research is concerned with narrative information under less structured conditions that often takes the research context into account • Descriptive and exploratory research • Purposes: describing conditions, exploring associations, formulating theory, generating hypotheses
- 21. Exploratory Research • Cohort study • Case control study
- 22. Cohort Study • A group of individuals who do not yet have the outcome of interest are followed together over time to see who develops the condition • Participants are interviewed or observed to determine the presence or absence of certain exposures, risks, or characteristics • May be simply descriptive • May identify risk by comparing the incidence of specific outcomes in exposed and not exposed participants
- 23. Cohort Study • Example To determine whether exercise protects against coronary heart disease (CHD). 1. Assemble the cohort: 16,936 Harvard alumni were enrolled 2. Measure predictor variables: Administer a questionnaire about activity and other potential risk factors , collected data from college records 3. 10 years later, sent a follow-up questionnaire about CHD and collected data about CHD from death certificates
- 24. Cohort Study • Strengths • Powerful strategy for defining incidence and investigating potential causes of an outcome before it occurs • Time sequence strengthens inference that the factor may cause the outcome • Weaknesses • Expensive – many subjects must be studied to observe outcome of interest • Potential confounders: eg, cigarette smoking might confound the association between exercise and CHD
- 25. Case-Control Study • Generally retrospective • Identify groups with or without the condition • Look backward in time to find differences in predictor variables that may explain why the cases got the condition and the controls did not • Assumption is that differences in exposure histories should explain why the cases have the condition • Data collection via direct interview, mailed questionnaire, chart review
- 26. Case-Control Study • Strengths • Useful for studying rare conditions • Short duration & relatively inexpensive • High yield of information from relatively few participants • Useful for generating hypotheses • Weaknesses • Increased susceptibility to bias: • Separate sampling of cases and controls • Retrospective measurement of predictor variables • No way to estimate the excess risk of exposure • Only one outcome can be studied
- 27. Case-Control Study • Example Purpose: To determine whether there is an association between the use of aspirin and the development of Reye’s syndrome in children. 1. Draw the sample of cases – 30 patients who have had Reye’s syndrome 2. Draw the sample of controls – 60 patients from the much larger population who have had minor viral illnesses without Reye’s syndrome 3. Measure the predictor variable: ask patients in both groups about their use of aspirin
- 29. Experimental Research • Provides a basis for comparing 2 or more conditions • Controls or accounts for the effects of extraneous factors, providing the highest degree of confidence in the validity of outcomes • Enables the researcher to draw meaningful conclusions about observed differences • Randomized controlled trials, single subject designs, sequential clinical trials, evaluation research, quasi- experimental research, meta-analysis
- 30. Experimental Research • Often regarded as the most rigid and scientific of all research methods • When used properly it can provide conclusions that are beyond questions • Future oriented • Characterized by its strict adherence to the scientific process
- 31. Features of Experimental Research 1. Presence of dependent and independent variable 2. The presence of control and 3. The measurement of effect of an independent variable, on the dependent variable neglecting the effects of all other variables
- 32. • ID and D • Control • The Measurement
- 33. Experimental Research • True experimental designs • Quasi-experimental designs
- 34. Efficacy vs. Effectiveness • Efficacy: the benefit of an intervention compared to a control or standard program under controlled, randomized conditions • Randomized controlled trial (RCT) design often used • Effectiveness: the benefit of an intervention under less controlled ‘real world’ conditions • Quasi-experimental design often used
- 35. Experimental Design • True experimental design: Subjects are randomly assigned to at least 2 comparison groups • Purpose is to compare 2 or more groups that are formed by random assignment • The groups differ solely on the basis of what occurs between measurements (ie, intervention) • Changes from pretest to posttest can be reasonably attributed to the intervention • Most basic is the pretest-posttest control group design (randomized controlled trial, RCT)
- 36. Experimental Design Example: • Researchers conducted an RCT to study the effect of progressive resistance exercises in depressed elders. They studied 35 volunteers who had depression. • Participants were randomly assigned to an exercise group, which met three times per week for 10 weeks, or a control group which met 2 times per week for an interactive health education program. • The outcome variables were: level of depression, functional status, and quality of life, using standardized instruments. • Pretest and posttest measures were taken for both groups and differences were compared.
- 37. Experimental Design Strengths • Controls the influence of confounding variables, providing more conclusive answers • Randomization eliminates bias due to pre-randomization confounding variables • Blinding the interventions eliminates bias due to unintended interventions Weaknesses • Costly in time and money • Many research questions are not suitable for experimental designs • Usually reserved for more mature research questions that have already been examined by descriptive studies • Experiments tend to restrict the scope and narrow the study question
- 38. Quasi-Experimental Design • Quasi-Experimental designs do not use randomized assignments for comparisons
- 39. Quasi-Experimental Design • Example: • A study was designed to examine the effect of electrical stimulation on passive range of motion of wrist extension in 16 patients who suffered a stroke. • Outcomes: effects of treatment on sensation, range of motion, & hand strength. • Patients were given pretest and posttest measurements before and after a 4-week intervention program. • Note: No randomization, and no comparison group
- 40. Quasi-Experimental Design Strengths • Q-E designs are a reasonable alternative to RCT • Useful where pre-selection and randomization of groups is difficult • Saves time and resources vs. experimental designs Weaknesses • Nonequivalent groups may differ in many ways -- in addition to the differences between treatment conditions, introducing bias • Non-blinding allows the possibility of unintended interventions; blinding can be used in some Q-E studies • Must document participant characteristics extensively • Potential biases of the sample must be acknowledged when reporting findings • Causal inferences are weakened by the potential for biases vs. experimental designs
- 41. EXPERIMENTAL DESIGN 1. Single Treatment Design 1. One Group Pretest-Posttest Design 2. Two Group Pretest-Posttest Design 3. Solomon Four Group Design 4. Posttest Only Control Group Design
- 42. 1. One group pretest-post-test design T1 P T2 T1 = Pretest (treatment group) T2 = Posttest (treatment group) P = Program or intervention 42 Types of designs
- 43. One group pretest-post-test design • In experimental conditions where a limited number of subjects are available • The group is first given a pretest followed by the usual treatment and then a posttest is administered • This design is very delicate because the researcher must see to it that the situations are equivalent before and during the experimental factor is introduced • More open to threats to internal validity such as the Hawthorne Effects (Test wiseness), maturation and attrition
- 44. 2. 2 Group Pretest-Posttest Design T1 P T2 C1 C2 • T1 = Pretest (treatment group) • T2 = Posttest (treatment group) • P = Program or intervention • C1 = Pretest (comparison group) • C2 = Posttest (comparison group) 44 Types of designs
- 45. Pre-Test/Post-Test Control Group Design • After the experimental period, both groups are again given the same posttest • The researcher may now conduct a comparison of the posttest results or the gain in scores (posttest- pretest) between the experimental and control group • This design is threatened by certain factors, maturation, test wiseness and natural attribution (natural death or drop outs)
- 46. 3. Solomon Four Group Design - Employ four equivalent groups - The first two groups obey the pretest-posttest control group design, the third group is given no pretest with treatment and a posttest - The last group is given no pretest, no treatment but with posttest - The design can eliminates the Hawthorne effect, effects of maturation and attrition, but has the main disadvantages of requiring a large number of respondents
- 47. • To analyze the data in Solomon four-group design, one uses a two factor analysis of variance • Factor A is the effect of treatment while the factor B is the effect of the pretest • The interaction affect AB would indicate if the treatment works well with pretesting or without pretesting
- 48. 4. Posttest Control Group Only Design P T2 P = Program or intervention T2 = Posttest 48 Types of designs
- 49. How much of the effect is due to the program? 49 Time (X) Desired Outcome (Y) T C Gross Effect Pre Post Net Effect
- 50. EXPERIMENTAL DESIGN 2. Doubt or Multiple Treatment Design 1. RCBD – Randomized Complete Block Design ex. Block treatment design RCBD treatment – 2 block design 2. CRD - Complete Randomized Design
- 51. RCBD • The RCBD assumes that a population of experimental units can be divided into a number of relatively homogenous subpopulations or blocks. • The treatments are then randomly assigned to experimental units such that each treatment occurs equally (usually once) in each block (each block contains all treatments) • Block usually represent levels of naturally-occurring differences or sources of variation that are not related to the treatments, and the characterization of these differences is not of interest of to the researcher
- 52. RCBD 1 B A C 2 A B C 3 A C B 4 A C B BLOCK NORTH END OF THE FIELD High N SOUTH END OF THE FIELD Low N
- 53. CRD • Is the basic single factor design. In this design the treatments are assigned completely at random so that each experiment unit has the same chance of receiving any one treatment. • But CRD is appropriate to only when the experimental material is homogeneous. As there is generally large variation among experimental plots due to many factors CRD is not preferred in field experiments.
- 54. example TREATMENT 1 2 1 3 2 1 2 3 3 3 1 2 REPLICATION
- 55. SAMPLING • Refers to the design for getting the respondents of the study with minimum cost and such that the resulting observation will be representative of the entire population
- 56. SAMPLING PLANS AND SAMPLING DESIGNS • Sample – is the small that being observed • Population – is the larger group about which your generalization is made • Sampling –is the process of obtaining information from a proper subset of a population • The values calculated from this sample will not be too far from the actual values of the population
- 57. Example: Population MALE FEMALE TOTAL High School 200 350 550 College 150 300 450 Total 350 650 1000 Sampling Plan MALE FEMALE TOTAL High School 57 100 157 College 143 86 129 Total 100 186 286
- 58. If we know that the underlying population is normally distributed, then if we have some estimate of the variability of the population such as variance (s2) then the formula for the sample size is: N=4s2/e2 where e=error tolerance (0.05 or 0.01) for a confidence coefficient of a=.05 For a=.01, then the formula becomes N=9s2/e2
- 59. How to calculate value of the sample size? Slovin’s Formula can be used if the population is more than 500. n= N/(1+Ne2) Where N= population size e=error of tolerance or desired margin of error n=a sample size