validity_reliability_trustworthiness.pdf
- 1. Validity-Reliabil ity and Trustworthiness PhD. Olga Camila Hernández M
- 2. Validity □ Validity Validity refers to the degree to which evidence and theory support the interpretations of test scores entailed by proposed uses of tests. *Internal validity is the extent to which research findings are free from bias and effects. *External validity is the extent to which the findings can be generalized.
- 3. Types of Validity 3. Content Validity: Content-related evidence of validity focuses on the content and format of an instrument. Is it appropriate? Comprehensive? Is it logical? How do the items or questions represent the content? Is the format appropriate? Consult other experts who rate the items. Rate items, eliminating or changing those that do not meet the specified content. Repeat until all raters agree on the questions and answers.
- 4. Types of Validity 4. Face Validity: Face validity occurs where something appears to be valid. This depends very much on the judgment of the observer. 5. Test Validity: Criterion: This refers to the relationship between the scores obtained using the instrument and the scores obtained using one or more other instruments or measures. For example, are students’ scores on teacher made tests consistent with their scores on standardized tests in the same subject areas? 6. Construct Validity: Construct validity is defined as “establishing correct operational measures for the concepts being studied” (Yin, 1984). For example, if one is looking at problem solving in leaders, how well does a particular instrument explain the relationship between being able to problem solve and effectiveness as a leader.
- 5. Reliability Reliability *The consistency of scores or answers from one administration of an instrument to another, or from one set of items to another. *A reliable instrument yields similar results if given to a similar population at different times. *Reliability coefficient - a number that tells us how likely one instrument is to be consistent over repeated administrations.
- 6. Types of reliability Reliability Stability Reliability Equivalence Reliability Test- retest Inter term Parallel forms Inter observer Agreement
- 7. Types of Reliability 1. Stability Reliability: Test-retest: ➢ ➢ ➢ ➢ Test-retest reliability is the degree to which scores are consistent over time. It indicates score variation that occurs from testing session to testing session as a result of errors of measurement. Same test- different Times Only works if phenomenon is unchanging Example: Administering the same questionnaire at 2 different times.
- 8. Types of reliability 2. Equivalence Reliability: a) Inter-item reliability: (internal consistency) ➢ ➢ The association of answers to set of questions designed to measure the same concept. Cronbach’s alpha is a statistic commonly used to measure inter-item reliability which is based on the average of all the possible correlations of all the split 1/2 s of set of questions on a questionnaire.
- 9. Types of Reliability b) Parallel form of Reliability Split-Half Reliability: □ □ Especially appropriate when the test is very long. The most commonly used method to split the test into two is using the odd-even strategy. Since longer tests tend to be more reliable, and since split-half reliability represents the reliability of a test only half as long as the actual test. C) Inter observer Reliability: Correspondence between measures made by different observers.
- 10. Quantitative vs. Qualitative Traditional Criteria for Judging Quantitative Research Alternative Criteria for Judging Qualitative Research Internal validity External validity Reliability Objectivity Credibility Transferability Dependability Confirmability