Introduction to measures of relationship: covariance, and Pearson r
15 likes1,796 views
The document provides an introduction to measuring relationships between variables through covariance and Pearson's r. It defines correlation as a statistical method to describe and measure the relationship between two variables. Covariance is introduced as a way to measure how two variables vary together or oppose each other by taking the average of their cross-product deviations from the mean. However, covariance depends on the scale of measurement. Pearson's r standardizes the covariance by converting the variables to z-scores based on their standard deviations, allowing comparison of relationships between variables measured on different scales. It represents the covariability of the two variables divided by their separate variability.
Introduction to measures of relationship: covariance, and Pearson r
- 1. Introduction to measures of relationship: covariance, and Pearson r Ivan Jacob Agaloos Pesigan
- 2. Outline • • • • • • Defining correlation Describing relationships: Scatter plots Measuring relationships: Covariance Measuring relationships: Pearson r Measuring relationships: Issues Uses of Correlation 2
- 3. Defining correlation Introduction to measures of relationship: covariance, and Pearson r 3
- 4. Defining correlation • A correlation is a statistical method used to describe and measure the relationship between two variables. • A relationship exists when changes in one variable tend to be accompanied by consistent and predictable changes in the other variable. 4
- 5. Defining correlation Although you should not make too much of the distinction between relationships and differences (if treatments have different means, then means are related to treatments), the distinction is useful in terms of the interests of the experimenter and the structure of the experiment. 5
- 6. Describing Relationships: Scatter Plots Introduction to measures of relationship: covariance, and Pearson r 6
- 8. 16 14 12 Digit Symbol 18 20 A strong, positive linear relationship 5 6 7 8 9 10 Vocabulary 8
- 10. 40 35 30 25 20 Time to complete 45 50 55 A negative linear relationship 5.0 5.5 6.0 6.5 7.0 7.5 Age 10
- 11. Describing relationships: Scatter plots • • • • • Linear relationship Positive linear relationship Negative linear relationship Curvilinear relationship No linear relationship 11
- 12. Linear relationship A negative linear relationship 40 30 35 Time to complete 16 25 14 20 12 Digit Symbol 45 18 50 55 20 A strong, positive linear relationship 5 6 7 8 Vocabulary 9 10 5.0 5.5 6.0 6.5 7.0 7.5 Age 12
- 13. Curvilinear Relationship A curvilinear relationship 14 8 10 12 Number of Errors 15 10 5 Time to Complete the Test 16 18 20 A negative, curved line relationship 1 2 3 Motivation 4 5 4.0 4.5 5.0 5.5 6.0 6.5 Age 13
- 14. No linear relationship 70 60 50 Depression Score 80 No linear relationship 60 80 100 IQ 120 140 14
- 15. Measuring relationships: Covariance Introduction to measures of relationship: covariance, and Pearson r 15
- 16. Measuring relationships: Covariance 85 80 75 70 Exam 90 95 100 IQ and final examination grade 100 105 110 115 IQ 120 125 16
- 17. Measuring relationships: Covariance • The simplest way to look at whether two variables are associated is to look at whether they covary. • To understand what covariance is, we first need to think back to the concept of variance. 17
- 18. Measuring relationships: Covariance • Remember that the variance of a single variable represents the average amount that the data vary from the mean. variance = s 2 X 2 X ∑( X − M ) = variance = s = i 2 X nX −1 ∑( X − X ) 2 i nX −1 18
- 19. Measuring relationships: Covariance • If we are interested in whether two variables are related, then we are interested in whether changes in one variable are met with similar changes in the other variable. • Therefore, when one variable deviates from its mean we would expect the other variable to deviate from its mean in a similar way or the directly opposite way. 19
- 20. Measuring relationships: Covariance Final Exam Scores per Student 80 85 Final Exam Scores 115 110 75 105 70 100 IQ Scores 90 120 95 125 100 IQ Scores per Student 2 4 6 Student 8 10 2 4 6 8 10 Student 20
- 21. Measuring relationships: Covariance 15 10 5 0 Exam 20 25 30 IQ and Errors in the Final Examination 100 105 110 115 IQ 120 125 21
- 22. Measuring relationships: Covariance Final Exam Errors per Student 10 15 Final Exam Errors 115 110 5 105 0 100 IQ Scores 20 120 25 125 30 IQ Scores per Student 2 4 6 Student 8 10 2 4 6 8 10 Student 22
- 25. Student IQ X - Mx 1 2 3 4 5 6 7 8 9 10 SUM MEAN 97 98 108 114 115 119 120 121 122 127 1141 114.1 -17.1 -16.1 -6.1 -0.1 0.9 4.9 5.9 6.9 7.9 12.9 Final Exam Score 71 71 74 84 86 86 87 90 96 99 844 84.4 Y - MY -13.4 -13.4 -10.4 -0.4 1.6 1.6 2.6 5.6 11.6 14.6 25
- 26. Measuring relationships: Covariance Final Exam Scores per Student 100 IQ Scores per Student 14.6 120 125 12.9 5.9 6.9 95 11.6 1.6 1.6 2.6 -0.4 75 105 80 -6.1 5.6 85 Final Exam Scores 115 0.9 110 -0.1 100 -17.1 -16.1 70 IQ Scores 90 4.9 7.9 2 4 6 Student 8 10 -10.4 -13.4 -13.4 2 4 6 8 10 Student 26
- 27. Measuring relationships: Covariance • When we multiply the deviations of one variable by the corresponding deviations of a second variable, we get what is known as the cross-product deviations. • As with the variance, if we want an average value of the combined deviations for the two variables, we must divide by the number of observations (n− 1) 27
- 28. Student IQ 1 97 2 98 3 108 4 114 5 115 6 119 7 120 8 121 9 122 10 127 SUM 1141 MEAN 114.1 X - Mx -17.1 -16.1 -6.1 -0.1 0.9 4.9 5.9 6.9 7.9 12.9 Final Exam Score 71 71 74 84 86 86 87 90 96 99 844 84.4 Y - MY -13.4 -13.4 -10.4 -0.4 1.6 1.6 2.6 5.6 11.6 14.6 (X - Mx) (Y – MY) 229.14 215.74 63.44 0.04 1.44 7.84 15.34 38.64 91.64 188.34 851.6 cov = 94.62 28
- 29. Measuring relationships: Covariance • This averaged sum of combined deviations is known as the covariance. covariance = cov X ,Y ∑ ( X − M ) (Y − M ) = covariance = cov X ,Y = i X i Y n −1 ∑ ( X − X ) (Y − Y ) i i n −1 29
- 30. Measuring relationships: Covariance covariance = cov X ,Y cov X ,Y ∑ ( X - M ) (Y - M ) = i X i Y n -1 851.6 851.6 = = ≈ 94.622 10 -1 9 30
- 31. Measuring relationships: Covariance • Calculating the covariance is a good way to assess whether two variables are related to each other. • A positive covariance indicates that as one variable deviates from the mean, the other variable deviates in the same direction. • On the other hand, a negative covariance indicates that as one variable deviates from the mean (e.g. increases), the other deviates from the mean in the opposite direction (e.g. decreases). 31
- 32. Measuring relationships: Covariance • There is, however, one problem with covariance as a measure of the relationship between variables and that is that it depends upon the scales of measurement used. • So, covariance is not a standardized measure. 32
- 33. Measuring relationships: Covariance • For example, if we use the data above and assume that they represented two variables measured in miles then the covariance is 94.62 (as calculated above). If we then convert these data into kilometers (by multiplying all values by 1.609) and calculate the covariance again then we should find that it increases to 244.97. 33
- 34. Measuring relationships: Covariance • This dependence on the scale of measurement is a problem because it means that we cannot compare covariances in an objective way – so, we cannot say whether a covariance is particularly large or small relative to another data set unless both data sets were measured in the same units. 34
- 35. Measuring Relationships: Pearson r Introduction to measures of relationship: covariance, and Pearson r 35
- 36. Measuring relationships: Pearson r • To overcome the problem of dependence on the measurement scale, we need to convert the covariance into a standard set of units. • This process is known as standardization. 36
- 37. Measuring relationships: Pearson r • A very basic form of standardization would be to insist that all experiments use the same units of measurement, say meters – that way, all results could be easily compared. • However, what happens if you want to measure attitudes – you’d be hard pushed to measure them in meters! 37
- 38. Measuring relationships: Pearson r • Therefore, we need a unit of measurement into which any scale of measurement can be converted. The unit of measurement we use is the standard deviation. • If we divide any distance from the mean by the standard deviation, it gives us that distance in standard deviation units. 38
- 39. Measuring relationships: Pearson r X i − MX X i − X zX = = sX sX Yi − MY Yi − Y zY = = sY sY 39
- 40. Measuring relationships: Pearson r • Since the properties of z scores form the foundation necessary for understanding the Pearson product moment correlation coefficient (r) they will be briefly reviewed: 1. The sum of a set of z score (Σ z) and therefore also the mean equal 0. 2. The variance (s2) of the set of z scores equals 1, as does the standard deviation (s). 3. Neither the shape of the distribution of X, nor of its relationship to any other variable, is affected by transforming it to zX 40
- 41. Measuring relationships: Pearson r rX ,Y ∑z = where z X Y n −1 X i − MX X i − X zX = = sX sX cov X ,Y = Yi − MY Yi − Y zY = = sY sY rX ,Y = ∑ ( X − M ) (Y − M ) = X i ( )( ) sX sY n −1 ∑ ( X − X ) (Y − Y ) i i (s s ) (n −1) i X i Y (n −1) ∑ ( X − X ) (Y − Y ) i i (n −1) then then i ∑ ( X − M ) (Y − M ) = rX ,Y = Y cov X ,Y sX sY covariability of X and Y variability of X and Y separately degree to which X and Y vary together = degree to which X and Y vary separately rX ,Y = rX ,Y X Y 41
- 42. Student IQ zx = (X – Mx)/sx 1 97 -1.69 2 98 -1.59 3 108 -0.60 4 114 -0.01 5 115 0.09 6 119 0.48 7 120 0.58 8 121 0.68 9 122 0.78 10 127 1.27 SUM 1141 MEAN 114.1 s = 10.14 Final Exam Score 71 71 74 84 86 86 87 90 96 99 844 84.4 zy = (Y – MY)/sy -1.37 -1.37 -1.06 -0.04 0.16 0.16 0.27 0.57 1.19 1.49 s = 9.77 zxzy 2.31 2.18 0.64 0.00 0.01 0.08 0.15 0.39 0.93 1.90 8.60 r = 0.96 42
- 43. Student IQ X - Mx 1 97 -17.1 2 98 -16.1 3 108 -6.1 4 114 -0.1 5 115 0.9 6 119 4.9 7 120 5.9 8 121 6.9 9 122 7.9 10 127 12.9 SUM 1141 MEAN 114.1 s = 10.14 Final Exam Score 71 71 74 84 86 86 87 90 96 99 844 84.4 Y - MY -13.4 -13.4 -10.4 -0.4 1.6 1.6 2.6 5.6 11.6 14.6 (X - Mx) (Y – MY) 229.14 215.74 63.44 0.04 1.44 7.84 15.34 38.64 91.64 188.34 851.6 s = 9.77 cov = 94.62 43
- 44. Measuring relationships: Pearson r rX ,Y = rX ,Y cov X ,Y sX sY 94.622 = 10.14 9.77 ( )( ) 94.622 = = 0.96 99.00331457 44
- 45. Measuring relationships: Pearson r radj ( =1− 1− r 2 (n −1) ) n−2 45
- 46. Measuring relationships: Pearson r There are three general strategies for determining the size of the population effect which a research is trying to detect: 1. To the extent that studies which have been carried out by the current investigator or others are closely related to the present investigation, the ESs found in these studies reflect the magnitude which can be expected. 2. In some research areas an investigator may posit some minimum population effect that would have either practical of theoretical significance. 3. A third strategy in deciding what ES values to use in determining the power of a study is to use certain suggested conventional definitions of “small”, “medium”, and “large” effects. 46
- 47. Measuring relationships: Pearson r Size of Effect / Magnitude r r2 (% of variance) small 0.1 0.01 (1%) medium 0.3 0.09 (9%) large 0.5 0.25 (25%) 47
- 48. Measuring relationships: Pearson r r = +1.00 r = -1.00 Perfect negative linear relationship 0 Y -3 -3 -2 -2 -1 -1 0 Y 1 1 2 2 3 Perfect positive linear relationship -3 -2 -1 0 1 X 2 3 -2 -1 0 1 2 3 X 48
- 49. Measuring relationships: Pearson r r = .80 r = .30 Moderate positive linear relationship Y -3 -2 -2 -1 -1 0 Y 0 1 1 2 2 3 Strong positive linear relationship -3 -2 -1 0 X 1 2 -2 -1 0 1 2 3 X 49
- 50. Measuring relationships: Pearson r r = .10 r = 0 No linear relationship -3 -3 -2 -2 -1 -1 Y Y 0 0 1 1 2 2 Weak positive linear relationship -3 -2 -1 0 X 1 2 -2 -1 0 1 2 3 X 50
- 51. Measuring relationships: Pearson r • Pearson Product Moment Correlation Coefficient: – It is a pure number and independent of the units of measurement. – Its absolute value varies between zero, when the variables have no linear relationship, and 1, when each variable is perfectly predicted by the other. The absolute value thus gives the degree of relationship. – Its sign indicates the direction of the relationship. A positive sign indicates a tendency for high values of one variable to occur with high values of the other, and low values to occur with low. A negative sign indicates a tendency for high values of one variable to be associated with low values of the other. Reversing the direction of measurement of one of the variables will produce a coefficient of the same value bur of opposite sign. Coefficients with equal value but opposite sign (for example, +.50 and -.50) thus indicate equally strong linear relationship, but in opposite directions. 51
- 52. Measuring Relationships: Issues Introduction to measures of relationship: covariance, and Pearson r 52
- 53. Restriction of Range -3 -2 -1 0 Y 1 2 3 Restriction of range -2 -1 0 1 X 2 53
- 54. Heterogeneous Subsamples male female 160 140 120 100 Weight 180 200 Relationship of Height and Weight Blue Line = Males; Red Line = Females; Black Line = Overall 62 64 66 68 70 72 74 Height 54
- 55. Outliers 55
- 57. Uses of Correlation Introduction to measures of relationship: covariance, and Pearson r 57
- 58. Uses of Correlation • Prediction. If two variables are known to be related in a systematic way, then it is possible to use one of the variables to make accurate predictions about the other. 58
- 59. Uses of Correlation • Validity. Suppose that a psychologist develops a new test for measuring intelligence. How could you show that this test truly measures what it claims; that is, how could you demonstrate the validity of the test? One common technique for demonstrating validity is to use a correlation. 59
- 60. Uses of Correlation • Reliability. In addition to evaluating the validity of a measurement procedure, correlations are used to determine reliability. A measurement procedure is considered reliable to the extent that it produces stable, consistent measurements. 60
- 61. Uses of Correlation • Theory Verification. Many psychological theories make specific predictions about the relationship between two variables. 61
- 62. People behind the concepts Scatter plot Bivariate plot Pearson Product Moment Correlation Coefficient 62
- 63. Slides Prepared by: Ivan Jacob Agaloos Pesigan Lecturer, Psychology Department Ateneo de Manila University Assistant Professor Lecturer, Psychology Department De La Salle University Lecturer, College of Education, Psychology Department, Mathematics Department Miriam College 63