Lesson 11 Understanding Data and Ways to Systematically Collect Data.ppt
- 1. MEASUREMENT AND SCALING OF MEASUREMENT Practical Research 2 CAF Reballos
- 2. Measurement in Research • Quantification of a characteristic or attribute of a person, object, or event • Provides for a consistent and meaningful interpretation of the nature of an attribute when the same measurement process or instrument is used • Systematic process that uses rules to assign numbers to persons, objects, or events which represent the amount or kind of a specified attribute
- 3. Sources of Error of Measurements • Respondents • Situations • Measurer • Instrument
- 4. Brief Overview • Reliability and Validity are important concepts in research as they are used for enhancing the accuracy of the assessment and evaluation of a research work (Tavakol and Dennick, 2011) • They have different meanings under the different types of research, i.e. quantitative and qualitative research (Creswell, 2014) • It is possible for measurement to be reliable but invalid and vice-versa
- 5. a. Validity • Ability of an instrument to measure what it is designed to measure • Refers to the accuracy of responses on self-report, norm-referenced measures of attitudes and behavior • It is the extent to which any measuring instrument measures what it is intended to measure (Thatcher, 2010)
- 6. a. Validity: Types 1. Face Validity • Refers to whether the instrument looks as though it is measuring the appropriate construct • It is helpful for a measure to have face validity if other types of validity have also been demonstrated
- 7. a. Validity: Types 2. Content Validity • Concerns the degree to which an instrument has an appropriate sample of items for the construct being measured • Relevant in development of both affective measures and cognitive measures
- 8. a. Validity: Types 3. Criterion-Related Validity • Involves determining the relationship between an instrument and an external criterion • Relates to our ability to predict some outcome or estimate the existence of some current condition • The instrument is said to be valid if its scores correlates highly with scores on the criterion
- 9. a. Validity: Types 3.1 Predictive Validity • Refers to the adequacy of an instrument in differentiating between people’s performance on some future criterion 3.2. Concurrent Validity • Refers to an instrument’s ability to distinguish individuals who differ on a present criterion
- 10. a. Validity: Types 4. Construct Validity • Degree to which scores on a test can be accounted for by the explanatory constructs of a sound theory • For determining construct validity, we associate a set of other propositions with the results received from using our measurement instrument
- 11. a. Validity: Types 5.1. Convergent Validity • Evidence that different methods of measuring a construct yield similar results • Correlations between two different methods measuring the same trait 5.2. Divergent Validity • Ability to differentiate the construct form other similar construct
- 12. b. Reliability • Refers to the consistency of responses on self-report, norm-referenced measures of attitudes and behavior • Refers to the consistency, stability and repeatability of results i.e. the result of a researcher is considered reliable if consistent results have been obtained in identifical situations but different circumstances (Twycross and Shields, 2004)
- 13. b. Reliability: Factors • Factors affecting the reliability of a research instrument – Wording of questions – Physical setting – Respondent’s mood – Nature of interaction – Regression effect of an instrument
- 14. b. Reliability: Types 1. Stability – this is when a researcher obtains the same results in repeated administrations or when the same test tools are used on the same sample size more than once, and when there is a reliability coefficient that provides an indication of how reliable the tool is
- 15. b. Reliability: Types Test of Stability • Test-retest – Researchers administer the same measure to a sample on two occasions and then compare the scores – The comparison is performed objectively by computing a reliability coefficient (index of magnitude of the test’s reliability) – Reliability coefficient ranges from -1.00 through 0.00 to +1.00
- 16. b. Reliability: Types 2. Homogeneity – This is a measure of the internal consistency of the scales. Cronbach’s alpha is used to measure the reliability of a tool
- 17. b. Reliability: Types Test of Homogeneity • Internal Consistency – The most widely used reliability approach among the researchers – If is economical and is the best means of assessing an especially important source of measurement error in instruments – Types: 1. Split-half technique 2. Cronbach’s alpha
- 18. b. Reliability: Types Test of Homogeneity • Split Half Technique (Internal Consistency) – Oldest methods for assesing internal consistency – Items on a scale are split into two groups and scored independently. Scores on the two hald tests then are used to compute a correlation coefficient
- 19. b. Reliability: Types Test of Homogeneity • Cronbach’s alpha (Internal Consistency) – Most widely used method – Preferable because it gives an estimate of the split-half correlation for all possible ways of dividing the measure into two halves – Normal range values is between 0.00 and +1.00
- 20. b. Reliability: Types 3. Equivalence – this is level of agreement among researchers using the same data collection tool. The ratings of two or more researchers are compared by calculating a correlation coefficient
- 21. b. Reliability: Types Test of Equivalence • Interrater (or interobserver) Reliability – Estimated by having two or more trained observers watching an event simultaneously, and independently recording data according to the instrument’s instructions. – The data can be used to compute an index of equivalence or agreement between observers
- 22. b. Reliability: Types • Techniques such as Cohen’s kapa, analysis of variance, intraclass correlations and rank-orer correlations assess this reliability
- 23. b. Reliability: Types Test of Equivalence • Alternate Form – Consisting of two sets of similar questons designed to measure the same trait – The two tests are based on the same content, but the individual items are different.
- 24. c. Other Criteria for Assesing Quantitative Measures • Sensitivity and Specificity – Sensitivity = the ability of an instrument to identify a “case correctly”, that is, to screen in or diagnosis is its rate of yielding “true positives” – Specificity = the intrument’s ability to identify non-cases correctly, that is, to screen out those without condition correctly, yielding “true negatives”
- 25. c. Other Criteria for Assesing Quantitative Measures • Receiver Operating Characteristic (ROC) curve – The best cutoff point (score value used to distinguish cases and noncases) used – A tradeoff between sensitivity and specificity of an instrument – Sensitivity is plotted against the false-positive rate (rate of incorrectly diagnosing someone as a case, inverse of specificity)
- 26. c. Other Criteria for Assesing Quantitative Measures