1.3 Experimental Design and Observational Studies
- 1. Section 1.3: Experimental Design & Observational Studies • Experiment = A controlled study conducted to determine the effect of varying one or more factors (explanatory variables) has on a response variable. – Intended to establish cause-and-effect. – Treatment = Any combination of the values of the factors. – A Control group serves as a baseline treatment that can be used to compare to other treatments. • Observational Study – when the investigator collects data merely by watching or asking questions or uses existing data, such as medical or sports records. – The investigator doesn’t influence the data. – The investigator doesn’t change anything. – An observational study generally can’t establish cause-and-effect . 1
- 2. Guidelines for Planning an Experiment 1. Specify the variable (or variables). – The variable must be something you cam measure. 2. Specify the population. 3. Identify the individuals or items of interest. – These are sometimes called the “experimental units” 4. Specify the method for measuring or observing. 5. Determine the sampling method. 6. Specify the hypothesis that you want to test and the significance level you will use. (See chapter 7) 7. Collect the data. 8. Use inference (a statistical hypothesis test) and make a conclusion. – E.g. Did the treatment make a significant difference. 9. Note any concerns you might have about your data collection methods and list any recommendations for future. 2
- 3. Experimental Options Randomized 2-Treatment Design The experimental group gets the treatment of interest. The control group gets either no treatment (or a “placebo” ) or is given the standard treatment (the usual teaching method, medicine, or fertilizer). Randomization reduces differences between the 2 groups. Randomized Block Design Example: Block 1 = full-time students. Block 2 = part-time students. Randomly choose some from each block . Then randomly assign ½ of those chosen in each block to get the treatment ½ to be in the control group. Matched Pairs Design The two groups that are matched with each other in some way. Examples: Twins Same subject before and after a treatment is applied (weight loss) Left & right halves of a field for an agricultural experiment Rigorously Controlled Design Group similar subjects by characteristics that are important to the experiment (e.g. age, health, gender). Assign equal numbers of each group to each treatment group, so that the different treatments are the only significant difference. 3
- 4. Other aspects of experimentation • Replication – Nobody is going to make major changes based on a single study. That what graduate students are for. • “Blind” Experiments – Single-Blind: subjects does not know if treatment or placebo – Double-Blinded: an experiment in which both the subject and the provider do not know which treatment is being applied. • This is the Gold Standard for medical studies 4 Placebo: A pill or treatment that has no effect, but looks, tastes, and smells like the experimental medication.
- 5. Experiment Example: Lipitor: a Cholesterol-Lowering Drug Study of 2838 people, ages 40 to 75, with Type 2 Diabetes Double blind experiment: people randomly assigned to 2 groups & tracked for 4 years. 1428 received Lipitor: 83 cardio events. 61 deaths. 1410 received a placebo: 127 cardio events. 82 deaths. Cardio event = heart attack, stroke, etc.
- 6. Observational Study Types • Cross-sectional study – data observed, measured, or collected at one point in time. • Retrospective (or case-control) study – data collected from the past using records, interviews, and other similar artifacts. • Prospective (or longitudinal or cohort) study – data collected in the future from groups sharing common factors. 6 An Observational Study does not try to change or influence the individuals or items. It tries to survey or measure things they way they are. It may research existing data.
- 7. Types of Survey Questions Open questions allow free responses • Allows the respondent to state exactly how they feel. • Harder to process the data, since there may be many possible answers • Can be useful to do a small open-question survey, in order to get possible answers for a closed-question survey Example: What do you consider to be the most important issue in the next election? Closed Questions have only a pre-determined list of possible answers: • Simpler to analyze, but may not have the best selection of answers. • In a closed-question survey, use multiple versions of poll with the order of the possible answers randomized. Example: Which of the following do you consider to be the most important issue in the next election? 1. Safe train crossings 2. Crime 3. Housing growth 4. New water sources
- 8. Review Questions 1. In a study of upper respiratory tract infections, 337 Children were given Echinacea, & 370 were given placebos. Is this an observational study or an experiment? 2. A student collected measurements of arm lengths from her family members. Is this random, systematic, convenience, stratified, or cluster sampling? 3. ABC conducted exit polls. Specific polling stations were randomly selected & all voters were surveyed. Is this random, systematic, convenience, stratified, or cluster sampling? 4. At a sobriety check-point, every 5th driver was stopped. Is this random, systematic, convenience, stratified, or cluster sampling? Experiment Convenience Cluster Systematic End of Section