Topic 7 measurement in research
Download as PPT, PDF18 likes17,265 views
This document discusses measurement in research and provides examples and guidelines. It covers topics such as selecting observable events, assigning numbers or symbols to represent aspects of events, applying mapping rules, and different levels of measurement including nominal, ordinal, interval and ratio scales. Reliability and validity are important criteria for good measurement. The document also discusses sampling methods like probability and non-probability designs as well as factors to consider for determining sample size.
Topic 7 measurement in research
- 1. 03/17/1503/17/15 RESEARCH INRESEARCH IN INFORMATION SYSTEMSINFORMATION SYSTEMS MANAGEMENTMANAGEMENT (IMS 603)(IMS 603) Topic 7:Topic 7: Measurement in ResearchMeasurement in Research
- 2. IntroductionIntroduction Measurement in research consists of assigning numbers to empirical events in compliance with a set of rules. 1)Selecting observable empirical events 2)Using numbers or symbols to represent aspects of the events 3)Applying a mapping rule to connect the observation to the symbol 03/17/1503/17/15
- 3. Introduction (cont.)Introduction (cont.) Example 1: To study people whom attend a computer exhibition at PWTC where all of the computer’s new models are on display. You are interested in learning the male-to-female ratio among visitors of the exhibition. You observe those enter the exhibition area. • Record male as ‘m’ and female as ‘f’ or • Record male as ‘1’ and female as ‘2’. 03/17/1503/17/15
- 4. Introduction (cont.)Introduction (cont.) Example 2: To measure the opinion of people on several new computer models. This can be achieved by interviewing a sample of visitors and assign their opinions to scales ranging from Strongly Agree (1) … Neutral (3) … to Strongly Disagree (5). 03/17/1503/17/15
- 5. What is measured?What is measured? Concepts used in research may be classified as: Objects •Include the things of ordinary experience such as people, automobiles, food etc. Phenomena •Things that are not concrete such as attitudes, perception, opinion, satisfaction etc. Properties •Characteristics of the objects
- 6. What is measured? (cont.)What is measured? (cont.) • A person’s physical properties may be stated in terms of weight, height, posture. • Psychological properties include attitudes and intelligence. • Social properties include leadership, ability, class affiliation or status.
- 7. Rules of MeasurementRules of Measurement A rule is a guide that instructs us on what to do. An example of a rule of measurement might be: •Assign the numerals 1 through 7 to individuals according to how productive they are. If the individual is an unproductive worker with little output, assign the numeral 1. •If a study on office computer systems is not concerned with a person’s depth of experience but defines people as users or nonusers, a ‘1’ for experience with the system and a ‘0’ for non experience with the system can be used.
- 8. Levels of MeasurementLevels of Measurement Variables can be further differentiated in terms of the ‘level’ or nature of measurement that are ‘continuous’ or ‘discrete’ in their form. Continuous variables •Have an infinite number of values that flow along a continuum. •On a continuum, values can be divided and sub- divided indefinitely in mathematical theory. •Even a five-point scale could be divided into a larger number of smaller units by sub-dividing between each pair of points on the scale.
- 9. Levels of MeasurementLevels of Measurement Discrete variables • Have relatively fixed set of separate values or variable attributes. • Instead of a smooth continuum of values, discrete variables contain distinct categories (eg. Gender: Male and Female)
- 10. Measurement LevelsMeasurement Levels Continuous and discrete variables yield four levels of measurement (degree of precision of measurement). The four levels of measurement are: 1.Nominal 2.Ordinal 3.Interval 4.Ratio
- 11. Measurement LevelsMeasurement Levels Discrete / Categorical (Frequency) Continuum / Continuous/ Scale (Score) Nominal Ordinal Interval Ratio Categories with no order. Categories with some order. Arranges objects according to their magnitudes in units of equal interval. Arranges objects according to their magnitudes in units of equal interval & has a true zero point.
- 12. Nominal ScaleNominal Scale • The simplest type of scale. • A scale in which the numbers of letters assigned to objects serve as labels for identification or classification. GENDER Males : 1 Females : 2 RACE Malays : 1 Chinese : 2 Indian : 3
- 13. Ordinal ScaleOrdinal Scale • A scale that arranges objects or alternatives according to their magnitudes. • A typical ordinal scale, example to rate services, brands, and so on as ‘excellent’, ‘good’, ‘fair’, or ‘poor’. • We know ‘excellent’ is higher than ‘good’ but we do not know by how much nor would we know whether the gaps between ranks are the same or different.
- 14. Interval ScaleInterval Scale • A scale that not only arranges objects according to their magnitudes, but also distinguished this ordered in units of equal interval. • Example 1: Ratings of radio programs would involve program evaluations using a five- or seven-point scale. • Hence, it would be possible not only to determine which program was best liked, second best liked, third best liked, etc. but also the amount by which one program was more liked than another.
- 15. Interval Scale (cont.)Interval Scale (cont.) • Example 2: If a temperature is 90 degree Celsius, it cannot be said that it is twice as hot as 45 degree Celsius. • The reason for this is that 0 degree Celsius does not represent the lack of temperature but a relative point on the Celsius scale. • Due to the lack of an absolute zero point, the interval scale does not allow the conclusion that 90 is twice as great as the number 45, only that the interval distance is two times greater.
- 16. Ratio ScaleRatio Scale • At the ratio level, it is possible to measure the extent to which one variable exceeds another on a particular dimension, and in addition, the scale of measurement has a true zero point. • Example: when measuring distance in meters, zero means no distance at all. It is an absolute and non-arbitrary zero point. • When measuring money in currency values, again zero means no money at all. The absolute zero point is an important factor because such scales also have exactly equal intervals between the separate points on the scale.
- 17. Criteria for good measurementCriteria for good measurement 1. Reliability The degree to which measures are free from error and therefore yield consistent results. The reliability of a measure indicates the stability and consistency with which the instrument measures the concept. Example: imperfections in the measuring process that affect the assignment of scores or numbers in different ways each time a measure is taken, such as a respondent who misunderstands a question are the cause of low reliability.
- 18. Criteria for good measurementCriteria for good measurement 2. Validity Is a test of how well an instrument that is concerned with whether we measure the right concept. There are two type of validity: Internal and external validity. Internal validity: concerned about issue of the authenticity of the cause-and-effect relationships External validity: concerned about issue of the generalizability to the external environment.
- 19. 03/17/1503/17/15 Goodness of MeasuresGoodness of Measures 1. Item Analysis Test whether items in the instruments should belong there. Steps: 1. Calculate Total Score 2. Divide respondents into high and low score 3. Compute t-test for each item 4. Use only items that are significant 2. Reliability Analysis Is the measure without bias (error free) and therefore consistent across time and across items in the instrument? i.e. is it stable and consistent? 3. Validity Analysis Is the instrument measuring the concept it sets out to measure and not something else?
- 20. 03/17/1503/17/15 Goodness of MeasuresGoodness of Measures GOODNESS OF DATA Reliability (Accuracy) Validity (Actuality) Stability Consistency Test-retest Parallel form Interitem consistency Split-half Logical (content) Criterion related Congruent (construct) Face Predictive Concurrent Convergent Discriminant
- 21. 03/17/1503/17/15 Reliability and ValidityReliability and Validity Valid but Unreliable Valid & Reliable Reliable but NOT Valid
- 22. 03/17/1503/17/15 ReliabilityReliability Observed scores may reflect true scores,Observed scores may reflect true scores, but it may reflect other factors as well:but it may reflect other factors as well: stable characteristics: two people having thestable characteristics: two people having the same opinion may circle different responsessame opinion may circle different responses transients personal factors such as moodtransients personal factors such as mood situational factors, time pressure, timesituational factors, time pressure, time variations in administration and mechanicalvariations in administration and mechanical factorsfactors Reliability: Stability and consistencyReliability: Stability and consistency StabilityStability – over time, conditions, state of– over time, conditions, state of respondentsrespondents ConsistencyConsistency – Homogeneity of times; items can– Homogeneity of times; items can measure the construct independentlymeasure the construct independently
- 23. 03/17/1503/17/15 Reliability of MeasuresReliability of Measures RELIABILITY Stability Consistency Test-retest Parallel form Repeated measures on the same respondent; high correlation – high reliability Two comparable sets of measures for the same construct; same items, same response format but different wording; Analysis - correlation Interitem Split-half Consistency of respondents’ answer to all the items; high correlation among responses to the items – Cronbach α Correlation between two- halves of a measure; correlation between the two halves
- 24. 03/17/1503/17/15 ValidityValidity Multiple indicators: - often used to capture aMultiple indicators: - often used to capture a given construct e.g. attitude; togiven construct e.g. attitude; to cover the domain of the constructcover the domain of the construct robust - reduce random errorrobust - reduce random error Cronbach alpha - measures intercorrelationCronbach alpha - measures intercorrelation between indicators - they should be positivelybetween indicators - they should be positively correlated but not perfectly correlatedcorrelated but not perfectly correlated Construct ValidityConstruct Validity Face validityFace validity Convergent validity (Correlation to assess it)Convergent validity (Correlation to assess it) Divergent validityDivergent validity
- 25. 03/17/1503/17/15 ValidityValidity VALIDITY Logical (content) Criterion related Congruent (construct) Face Ensures adequate and representative set of items that tap the concept Panel of judges – face validity Predictive Concurrent Does measure differentiate to predict a future criterion variable Analysis – Correlation Does measure differentiate to predict a criterion variable currently Analysis – Correlation Convergent Discriminant Do the two instruments measuring the concept correlate highly? Does the measure have low correlation with an unrelated variable?
- 26. 03/17/1503/17/15 Data Source: SamplingData Source: Sampling Two Central QuestionsTwo Central Questions Do weDo we samplesample oror censuscensus?? If sample:If sample: How to identifyHow to identify Who/whatWho/what to include into include in the sample? - sampling designthe sample? - sampling design HowHow manymany to include in the sample? -to include in the sample? - sample sizesample size
- 27. 03/17/1503/17/15 What is a Good Sample?What is a Good Sample? RepresentativeRepresentative of the Populationof the Population Estimates from sample areEstimates from sample are accurateaccurate Estimates from sample areEstimates from sample are preciseprecise
- 28. 03/17/1503/17/15 Steps in Sampling DesignSteps in Sampling Design What is the relevantWhat is the relevant populationpopulation?? What are theWhat are the parametersparameters of interest?of interest? What is theWhat is the sampling framesampling frame?? WhatWhat sizesize sample is needed?sample is needed? What is theWhat is the typetype of sample?of sample? How much will itHow much will it costcost??
- 29. 03/17/1503/17/15 Types of SamplingTypes of Sampling DesignDesign Non- probability Design Probability Design Convenience Judgement Quota Snowball Simple Random Systematic Stratified Cluster Simple Random Stratified Combination Sampling Design One-stage design Multistage design
- 30. 03/17/1503/17/15 Choosing a SamplingChoosing a Sampling DesignDesign Is REPRESENTATIVENESS critical? Area samples Only experts have information Info from special interest groups QuotaJudgement Quick, unreliable information Relevant information about certain groups Convenience Simple random Systematic Cluster if not enough RM Double samples Equal sized subgroups? Proportionate stratified samples Disproportionate stratified samples YES NO Choose PROBABILITY design Choose NON-PROBABILITY design NOYES Generaliza bility Subgroup Differences Collect localized information Information about subsets of sample
- 31. 03/17/1503/17/15 Sample Size: FactorsSample Size: Factors HomogeneityHomogeneity of sampling unitsof sampling units ConfidenceConfidence levellevel PrecisionPrecision Analytical ProcedureAnalytical Procedure Cost, Time and PersonnelCost, Time and Personnel
- 32. 03/17/1503/17/15 Roscoe’s Rule of ThumbRoscoe’s Rule of Thumb Larger than 30 and less than 500Larger than 30 and less than 500 appropriate for most researchappropriate for most research A minimum of 30 for each sub samplesA minimum of 30 for each sub samples Multivariate research: At least 10 timesMultivariate research: At least 10 times the number of variablesthe number of variables Simple Experiments with tight controlsSimple Experiments with tight controls - samples as small as 10 to 20- samples as small as 10 to 20