Validity and Reliability.pdf
- 1. Validity and Reliability in Research Zubaedah Wiji Lestari Universitas Pasundan Bandung 2022
- 2. DEFINITION • Validity refers to the appropriateness, meaningful- ness, correctness, and usefulness of the inferences a researcher makes. • Reliability refers to the consistency of scores or answers from one administration of an instrument to another, and from one set of items to another
- 3. Validity • Validity is the most important idea to consider when preparing or selecting an instrument for use. • More than anything else, researchers want the information they obtain through the use of an instrument to serve their purposes. • For example: to find out what teachers in a particular school district think about a recent policy passed by the school board, researchers need both an instrument to record the data and some sort of assurance that the information obtained will enable them to draw correct conclusions about teacher opinions. The drawing of correct conclusions based on the data obtained from an assessment is what validity is all about.
- 4. Three main types of validity • Content-related evidence of validity refers to the content and format of the instrument. How appropriate is the content? How comprehensive? Does it logically get at the intended variable? How adequately does the sample of items or questions represent the content to be assessed? Is the format appropriate? The content and format must be con- sistent with the definition of the variable and the sample of subjects to be measured. • Criterion-related evidence of validity refers to the relationship between scores obtained using the instrument and scores obtained using one or more other instruments or measures (often called a criterion). How strong is this relationship? How well do such scores estimate, present, or predict future performance of a certain type? • Construct-related evidence of validity refers to the nature of the psychological construct or characteristic being measured by the instrument. How well does a measure of the construct explain differences in the behavior of individuals or their performance on certain tasks?
- 5. Content – Related Evidence • Content related validity is a matter of determining if the content of the instrument contains an adequate sample of the domain of content it is supposed to represent. The other aspect of content validity has to do with the format of the instrument such as the clarity of printing, size of type, adequacy of work space (if needed), appropriateness of language, clarity of directions, and so on. • Regardless of the adequacy of the questions in an instrument, if they are presented in an inappropriate format (such as giving a test written in English to children whose English is minimal), valid results can- not be obtained. For this reason, it is important that the characteristics of the intended sample be kept in mind. How does one obtain content-related evidence of validity? A common way to do this is to have someone look at the content and format of the instrument and judge whether or not it is appropriate. • The “someone,” of course, should not be just anyone, but rather an individual who can be expected to render an intelligent judgment about the adequacy of the instrument—in other words, someone who knows enough about what is to be measured to be a competent judge
- 6. Criterion – Related Evidence • To obtain criterion-related evidence of validity, researchers usually compare performance on one instrument (the one being validated) with performance on some other, known as independent criterion. A criterion is a second test or other assessment procedure presumed to measure the same variable. • There are two forms of criterion-related validity: predictive and concurrent. To obtain evidence of predictive validity, researchers allow a time interval to elapse between administration of the instrument and obtaining the criterion scores. On the other hand, when instrument data and criterion data are gathered at nearly the same time, and the results are compared, this is an attempt by researchers to obtain evidence of concurrent validity.
- 7. Construct-related evidence validity • Construct-related evidence of validity is the broadest of the three categories of evidence for validity. • There is no single piece of evidence that satisfies construct-related validity. Rather, researchers attempt to collect a variety of different types of evidence (the more and the more varied the better) that will allow them to make warranted inferences—to assert. • Usually, there are three steps involved in obtaining construct-related evidence of validity: (1) the variable being measured is clearly defined, (2) hypotheses, based on a theory underlying the variable, are formed about how people who possess a lot versus a little of the variable will behave in a particular situation, (3) the hypotheses are tested both logically and empirically.
- 8. RELIABILITY • Reliability refers to the consistency of the scores obtained—how consistent they are for each individual from one administration of an instrument to another and from one set of items to another • Reliability and validity always depend on the context in which an instrument is used.
- 9. Distinction between reliability and validity The bull’s-eye in each target represents the information that is desired. Each dot represents a separate score obtained with the instrument. A dot in the bull’s-eye indicates that the information obtained (the score) is the information the researcher desires.
- 10. Three best-known ways to obtain a reliability coefficient • THE TEST-RETEST METHOD involves administering the same test twice to the same group after a certain time interval has elapsed. A reliability coefficient is then calculated to indicate the relationship between the two sets of scores obtained. • EQUIVALENT-FORMS METHOD. When the equivalent-forms method is used, two different but equivalent (also called alternate or parallel ) forms of an instrument are administered to the same group of individuals during the same time period. • INTERNAL-CONSISTENCY METHODS, The methods mentioned so far all require two administration or testing sessions. There are several internal- consistency methods of estimating reliability, however, that require only a single administration of an instrument. There are two procedures in conducting internal consistency methods: - --- - Split-half Procedure. The split-half procedure involves scoring two halves (usually odd items versus even items) of a test separately for each person and then calculating a correlation coefficient for the two sets of scores. - Kuder-Richardson Approaches. Perhaps the most frequently employed method for determining internal consistency is the Kuder-Richardson approach , particularly formulas KR20 and KR21. The latter formula requires only three pieces of information—the number of items on the test, the mean, and the standard deviation.
- 11. VALIDITY AND RELIABILITY IN QUALITATIVE RESEARCH • While many qualitative researchers use many of the procedures we have described, some take the position that validity and reliability, as we have discussed them, are either irrelevant or not suited to their research efforts because they are attempting to describe a specific situation or event as viewed by a particular individual. • They emphasize instead the honesty, believability, expertise, and integrity of the researcher. • We maintain that all researchers should ensure that any inferences they draw that are based on data obtained through the use of an instrument are appropriate, credible, and backed up by evidence of the sort we have described in this chapter.
- 12. Source: Creswell. J.W. (2012). Educational research: planning, conducting and evaluating quantitative and qualitative research. London: Pearson. Fraenkel J. R., Wallen & Hyun. (2012). How to design and evaluate research in education. New York: McGraw-Hill Companies.
- 13. ☺