Methodology and IRB/URR
Download as PPTX, PDF3 likes1,329 views
Dr. James Lani discusses research methodology, emphasizing the importance of a structured design that includes research methods, operational constructs, and sampling strategies. He outlines quantitative and qualitative approaches to data collection, the need for precise definitions of constructs and variables, and the significance of reliability and validity in research. Additionally, he highlights ethical considerations and provides practical advice for conducting and analyzing studies.
![Brief Review: Units of Analyses
Quantitative (e.g., Age)
Nominal-level (Latin for name). Gender (M/F), Grouped (Old =
65+, middle age = 36-64, young = 35 or younger). Assign any
number of groups (old = 1, middle = 2, young = 3).
Ordinal-level is ranked (Latin for showing order). GPA (A-F), or
age (group 1 = age 1–15, group 2 = age 16-25, group 3 = age 25-54,
group 4 = age 65+)
Interval/Ratio-level (also named scale or continuous; Latin for
[equal] space [between numbers]). What is your age today in
years? ____ (a number from 1-105)](https://image.slidesharecdn.com/methodologyandirbpp-141120120633-conversion-gate02/85/Methodology-and-IRB-URR-17-320.jpg)
Methodology and IRB/URR
- 1. Methodology and IRB By Dr. James Lani
- 2. Methodology and IRB/URR Dr. James Lani Take Away Message Research design is a blueprint with several components: • Research method: Quant/Qual/Mixed • Operational constructs: How constructs are measured • Sampling strategy and procedure What is Research Research follows the scientific method. What’s the scientific method? • Formulation of testable questions or hypotheses • It’s organized knowledge: logical (theory) and evidence based (observable) • Precise constructs • Can be disproven (falsifiable) • Parsimonious (simplest explanation)
- 3. Methodology: The Cookbook Metaphor Cooking Researching Making a stew… Examining research questions… Ingredients: 2 lbs beef, I clove garlic… Preparation: Cube beef, mince garlic… Cooking Instructions: Bake at 350 for one hour Celebrate: You’ve made a replicable) stew! Ingredients: 20 participants, 15 item questionnaire, semi-structured interview. Preparation: Administer questionnaire before and after lecture; semi-structure interview with participants for 20 minutes using a tape recorder. Data analysis plan: Conduct dependent sample t-test; transcribe interviews then thematize participants responses. Celebrate: You’ve conducted a (replicable) research study! Where the recipe can be replicated
- 4. Methodology Essential Ingredients Restate research question and hypotheses Overview Research Design Participants Materials/ Instruments Data Collection Procedures Sample Size Data Limitations Analysis Plan
- 5. Methods Quantitative Qualitative Mixed You can count it Not quantitative Both Goal Tend to answer “What questions” (What is relationship or differences) or “When questions” (when is theory supported…after intervention Tend to answer “Why questions” (understand why people feel that way) or “How questions” (explore how they see things) Both Research methods/ Strategies of data collection Experimental (random assignment) and Non-experimental methods (no control group), Observations (time participant)… Semi-structured interviews, Archive data, Observations (write down positive and negative feeling words)… Both Operationalize variables Define constructs/ instruments Define constructs/ instruments Both Sampling strategy Discuss sampling process Discuss sampling process Both
- 6. Theoretical (or Conceptual) Framework Theory is a systematic explanation of behavior of phenomena. • Theory guides analyses • List existing theories and how your research questions relate to those theories • E.g., Theory of mind: the ability to understand that others have their own beliefs, desires, intentions. Empathy. Tested by Faux Pas Task (ability to recognize a faux pas). Research question: Does alcohol abuse impact empathy?
- 7. Population The population is the group you want to generalize to. • Describe characteristics of population • Why is population relevant to problem (look at other peer reviewed study’s justification) • Distinguish the population from the sample
- 8. Sampling Frame and Sample Universe: theoretical population to generalize to Population: largest target population from universe you have access to or “Sampling Frame” Original Sample Final Sample Attrition
- 9. Survey Item Old (one’s own SE) Change in item New (perceptions of others’ SE) 1 I feel that I have a number of good qualities. “I have” to “she has” I feel that she has a number of good qualities. 2 I feel I do not have much to be proud of. “I do not” to “she does not” I feel she does not have much to be proud of. 3 On the whole, I am satisfied with myself. “I am” to “she is” and “myself” to “herself” On the whole, she is satisfied with herself. … If you need to amend instrument, use a change matrix; do not create your own instrument!
- 10. Materials: Informed Consent • State purpose of project • State procedure and how long it will take • State voluntary nature of participation • State risks (if any) • Have them sign or state that by filing our survey they are agreeing to participate.
- 11. Constructs vs. Variables Empirical Theoretical Construct A: Social Environment Construct B: Personality Variable A: Birth-Order Variable B: Introversion/ extroversion
- 12. Constructs vs. Variables Constructs are the invisible abstract things we’re measuring (e.g., personality), while variables are the way we’re assessing (measure/operationalize) those invisible things. Constructs and variables need to be precise (is personality measured by introversion scale or by conscientiousness scale?) e.g., Intelligence is a construct, while the number of words remembered is a way of assessing intelligence. e.g., Personality is a construct, while the scores on an introversion/extroversion test is a way to assess an aspect of personality.
- 13. Variables Operationalize Constructs, Make Distinction between IV’s/DV’s, and Describe Level of Measurement Example: Does Empathy differ by group (alcohol abuse vs. no alcohol abuse)? Empathy is my dependent variable and Group is my independent variable. Empathy (my construct) is measured by scores on the Faux Pas task. Alcohol abuse (my construct) is measured by 5 or more drinks in one day. Empathy is a ratio-level variable measured with scores ranging from 0- 25, while my Group variable is a nominal-level (categorical-level) variable because participants re in one of two groups (alcohol abuse group or not in alcohol abuse group).
- 14. Reliability and Validity Valid but not Reliable Reliable but not Valid Reliable and Valid
- 15. Validity • Internal Validity: IV causes a change in the DV (not time or other covariates, etc. • External validity: can be generalized to the population • Construct validity: does the scale measure the theoretical construct • Translational validity: • Face validity (items are reasonable) • Content validity (items match the domains of interest) • Criterion related validity: measures behave as theory predicts • Convergent validity: how close the variable aligns with the construct (use EFA) • Concurrent validity: construct relates to established instruments • Predictive validity: measure can predict an outcome (GPA→Income)
- 16. Reliability Internal consistency: Cronbach alpha. Average inter-item correlation Inter-rater: if interval, correlate; if dichotomous, kappa Test-retest: administer same test at two times Split half: divide instrument into 2 parts and calculated totals, then correlate totals.
- 17. Brief Review: Units of Analyses Quantitative (e.g., Age) Nominal-level (Latin for name). Gender (M/F), Grouped (Old = 65+, middle age = 36-64, young = 35 or younger). Assign any number of groups (old = 1, middle = 2, young = 3). Ordinal-level is ranked (Latin for showing order). GPA (A-F), or age (group 1 = age 1–15, group 2 = age 16-25, group 3 = age 25-54, group 4 = age 65+) Interval/Ratio-level (also named scale or continuous; Latin for [equal] space [between numbers]). What is your age today in years? ____ (a number from 1-105)
- 18. Types of Methodology Models Theories explain phenomena, Models represent Phenomena. • SEM and Path models • Regression models (linear, logistic, ordinal) • ANOVA models (repeated-measures) • Time-series • Heirarchical Linear Models (HLM) • Correlational Models
- 19. Relationship Among Variables Employee Tenure Leadership Style Employee Satisfaction Customer Satisfaction Mediator Moderator
- 20. Data Collection Method -- Overview • Describe the procedures used to administer the materials to the participants • Remember to be as detailed as necessary so someone can literally replicate your study
- 21. Data Collection Qualitative • Procedure for accessing participants • Selection of data collected • Number and duration of interviews • How and when data is collected • How data is recorded (e.g., hand notes, audiotaped) • Role of researcher-relationship to researcher Quantitative • Procedure to administer measures
- 22. Pilot Test (only if you develop instrument) • Detect potential issues in the instrument • Allows you to get feedback and to finalize your survey/interview items • Makes sure participants understand survey items • Assess typical responses to survey items or interview questions (were participants comfortable, long-winded, defensive, etc. • Can you access data?
- 23. Data Analysis Plan • Quantitative: Describe the analysis plan used to test each hypothesis, the assumptions of the statistical analyses, and a justification of the appropriateness of the analysis for each research question. • Qualitative: Describe how the data will be analyzed (or thematized) • Phenomenological • Case Study • Grounded theory
- 24. Sample Size Quantitative Varies by type of statistical analysis 1. Research questions in statistical language 2. Level of measurement of variables 3. Determine statistical analysis 4. Conduct power analysis Qualitative Varies by which qualitative approach is taken • Phenomenological • Case Study • Grounded theory
- 25. Assumptions, Limitation, Delimitations Assumptions: Discuss things out of your control about the population and design, then justify assumptions (e.g., participants will answer honestly.) Limitations: Are out of your control and describe weaknesses in design, threats to validity (e.g., generalizability). Delimitations: Are in your control and relate to choices you will make to narrow the scope of the study (e.g., variables, research questions).
- 26. Ethical Considerations • Describe informed consent procedures • State whether your study will be anonymous or confidential with respect to the participants • Describe considerations for children or vulnerable participants
- 27. See yourself graduate in 2014! 1-1 Personalized Dissertation Consulting 877-437-8600 Info@StatisticsSolutions.com
- 28. Thank you for your participation and attention! Join us for our next webinar on Wednesday, December 17th at 8:30pm ET. Results and Discussion Chapters, Editing, and Defense
- 29. Join us for our 1-day Dissertation Workshop Dallas/Fortworth, Texas December 6th 9am-5pm $199/student $89 lodging/night