Matthew_Sullivan_Honors_Thesis
- 1. The Economic Determinants of SAT Scores Matthew Sullivan April 10, 2015
- 2. Page | 2 I Introduction The United States has been the world’s largest economic power for over half a century. The United States is also home to some of the most powerful companies and innovations that have changed the world. A large part of The United States’ economic success is due to American students’ performance in school relative to the rest of the world. Many of the world’s best universities are in the United States, and students from all over the world come to the US to study. However, students in the American public school system have been struggling in international competitiveness relative to other developed countries. In PISA1 standardized testing, American students rank 25th in math scores, 17th in science, and 14th in reading scores among 34 OECD nations2 . There are many varying opinions as to why American students are not performing as well as their foreign counterparts, making educational reform one of the hottest issues today. Inadequate spending, bad teachers, misallocation of resources, and poor district-level control are some of the most popular theories, each having a different solution to the problem. There have been many proposed types of reform, but the most common are private school vouchers, merit- based pay, and charter schools. Private school vouchers and charter schools are proposed programs that seek to decrease the amount of students in overpopulated and underperforming school districts. Vouchers are scholarships that are paid to students so that they are able to afford and attend certain private schools.3 Charter schools are public schools where families choose to enroll their kids and are granted more flexibility on operations in exchange for greater accountability for student 1 Program for International Student Assessment (PISA) is a worldwide evaluation of 15-year-old students 2 Highlights from PISA 2009: Performance of U.S. 15-Year-Old Students in Reading, Mathematics, and Science Literacy in an International Context 3 National Council of State Legislatures, School Voucher Laws: State-By-State Comparison
- 3. Page | 3 performance4 . Merit-based pay attempts to promote better student performance by structuring salaries based on how well a teacher’s students perform on certain criteria. The purpose of this paper is to look at some of the factors in the current education system that some of the proposed reforms aim to change. To get the best measure of the effect on the quality of education, I look at how these variables impact a state’s average score on the SAT. The independent variables used in this paper are the relative strength of teachers unions, average class size, enrollment per district, the cost of living adjusted average starting teacher salary, and the percent of people with a bachelor’s degree, all measured at the state-level. II Sample The sample I will use in this study is public school data from 20 states in the US5 . The reasoning behind the selection of these states will be explained in the following section. III Dependent Variable The dependent variable in this study is the average SAT scores by state for the year 2013. The SAT is an aptitude test that is designed to asses a student’s readiness for college. I will use this measure as a proxy for student achievement because standardized scores are perceived as better measures than other measures whose criteria vary by district or state (such as grade point average or graduation rates). The SAT is also one of the most recognized and widely used tests for college entrance and has been used since 19266 . The test is divided into three sections, and scores for each section are out of 800 possible points. The total score is calculated as the sum of the three scores. However, some universities take 4 Uncommon Schools, Frequently Asked Questions About Public, Charter Schools 5 Public school data was chosen because private schools have more freedom to exercise their own standards for the variables used in this study. 6 The College Board: About the SAT
- 4. Page | 4 the highest score for each section that the student has taken in their lifetime. This means that a student’s score of one complete test may not be the score colleges use for acceptance. To control for this, I have elected to run each score in its own regression. I also chose to look at state-level scores because the average scores vary widely across the country and many educational decisions are made by state and local governments rather than by the federal government. The data on SAT scores was collected from the Commonwealth Foundation of Pennsylvania7 . I chose this source because the data was collected at the state-level, and the participation rate by state is included. The participation rate is of particular importance because even though the SAT is one of the most commonly used standardized tests, is not the only test that universities use as a basis for acceptance. This means that there are many college-bound students who do not take the SAT, and the state-by-state participation rate varies significantly. A large difference in the SAT participation rate across the states could create a potential problem with the results of this study. When a higher amount of students take the exam, there are more lower-ability students taking the exam, which drives the average score for the state down. Therefore, states with lower participation rates will have higher average SAT scores. To get the best distribution of states with students taking the exam, I chose the 20 states with participation rates from 25% to 75%. IV Independent Variables Union Membership (Union) Though private sector union membership is falling, teachers unions are some of the most powerful organizations in all of American politics. Unions are formed to promote teacher benefits, 7 The Commonwealth Foundation: SAT Scores by State 2013
- 5. Page | 5 but the political reach of these organizations extends far beyond educational lobbying by supporting many special interest groups that are not education-oriented. The two strongest unions are the National Education Association and the American Federation of Teachers, which account for over $600 million combined receipts, $70 million in political activities and lobbying, over 4 million members, and more than $92 million in contributions, gifts, and grants8 . Many supporters of education reform believe teachers unions specifically lobby against reform legislation to protect bad teachers and keep union membership high. My hypothesis on this variable is that a higher union strength will negatively affect average SAT scores due to inefficiencies and the protection of underperforming teachers. The variable for Union Strength is measured as the relative strength of union power in the 50 states and Washington DC. This data is gathered from a three year study conducted by the Thomas B. Fordham Institute9 . In the study, they look at the involvement of unions in over three dozen union characteristics, including membership, involvement in politics, scope of policies, state politics, and perceived influence to determine how powerful unions are in each state. In the Fordham report, the strongest state union is given a value of 51, the second strongest a value of 50, and so on to a value of 1. Although only 20 states were used in this paper, I kept the original ranking from the study rather than reassigning the values from 1-20 for two reasons. The first was that Union strength was relatively evenly distributed in the sample used for my regressions10 . The second reason was that reassigning the values on a 1-20 scale would not accurately reflect the disparity of power the strongest unions have compared to the weakest unions. 8 Teachers Union Exposed: Union Profiles 9 Thomas B. Fordham Institute: How Strong Are U.S. Teacher Unions? A State-By-State Comparison 10 The mean value (located on Figure 1) is 27.4, with the 5 strongest unions and 4 of the top 5 weakest unions represented in the sample.
- 6. Page | 6 Average Class Size (ACS) Average class size was chosen because it has been a popular topic for union lobbying and impacts both students and teachers. This data is obtained from the American Legislative Exchange Council’s (ALEC) Report Card on American Education, with data reported for 201011 . Though the SAT data is for 2013, I chose the ALEC report card because the report itself is one of the most comprehensive reviews of American education and has data for all 50 states. Additionally, the Fordham Institute study of 2012 covers 3 years of prior data to determine their union strength rating scale, and 2010 falls within their study timeframe. The logic of supporting a smaller average class size is that a smaller student to teacher ratio will increase teacher interaction and therefore student performance. Smaller classes are also easier environments to teach and are part of increasing teacher benefits. Because of the inherent positive impacts associated with smaller class sizes for the teacher and the student, this variable will be measured using a one-tailed test. I therefore hypothesize that states with smaller average class sizes have higher average SAT scores. Enrollment per District (EpD) Enrollment per district seeks to capture the efficiency of each school district. Districts with higher enrollments have more students and fewer resources. With larger districts, school faculty and parents have less of a voice on policy issues and the best interests of certain groups may be left out. However, it may be possible that a larger district creates an economy of scale by cutting down on some of the administrative costs, leaving more money to go to teachers and students. Taking both of these factors into account, I believe that the diluted voice a teacher or parent will outweigh the 11 NEA: Rankings and Estimates Report: Rankings of the States 2012 and Estimates of School Statistics 2013
- 7. Page | 7 effects of any efficiency gains created with larger districts. Thus, I predict that a higher enrollment per district will adversely affect a state’s average SAT score. This data is defined by taking the total enrollment for each state in 2011 and dividing it by the total number of school districts for each state in 2011. Enrollment and district data were collected from the National Educational Association’s Rankings and Estimates Report12 . I chose this report because it fell in the time frame of the study and is only a year later than the previous variable that was concerned with enrollment figures. Though the variables are intended to describe two different phenomena, I ran a correlation on enrollment per district and average class size to combat any concerns for multicolinearity. The correlation returned a value of 0.005388, so both of the variables’ presence should not interfere with the results of the regression. Cost of Living Adjusted Average Starting Teacher Salary (Tpay) Teacher salary level is one of the biggest points of contention among educational lobbyists and proponents of reform. Public school teacher salaries are relatively low in the United States when compared to other careers that require similar educational attainment, driving many people away from becoming a teacher. Intuitively, higher salaries should make teachers generally happier and perform their jobs better. In the United States however, most public school teacher salaries are based on years of experience and the highest degree the teacher attained, not on the performance of their students. Because salaries are not based on student achievement, average salary level for all teachers would not be an ideal measure to relate to SAT scores since there is no economic incentive for better student performance. However, I have elected to use starting salary because theoretically teachers will want to maximize their starting salaries since they will get paid more as their career 12 National Education Association: 2011-2012 Average Starting Teacher Salaries by State
- 8. Page | 8 progresses. This means districts with higher starting salaries can attract more teachers, and should be able to hire the best applicants. Therefore, I predict that as teacher salary increases, SAT scores increase as well. Because the cost of living varies from state to state, I divided the teacher salary by the state’s cost of living index. Data on the average teacher salary starting pay for teachers across the country is collected from the National Education Association13 for 2013 and the cost of living is gathered from Missouri Economic Research and Information Center14 for Q2 of 2014. Percent of People with a Bachelor’s Degree (PBD) The percent of people with a bachelor’s degree was chosen as a proxy for the role parents have on student success. Though education reform doesn’t have any control over the amount of people that currently have a bachelor’s degree or higher, this variable was included because parent involvement is a key indicator of student success. Parents with advanced degrees tend to have higher paying jobs, giving their children access to more resources and mobility to purchase houses in more desirable school districts. These parents could also provide students with an additional resource for understanding schoolwork and going through school with the expectation of going to college. This variable will be measured using a one-tail test as well because of the inherent positive benefits of highly educated parents. The data was collected from the US Census’s 2012 Statistical Abstract.15 I predict that as the percentage of people with a bachelor’s degree increases, the average SAT score will also increase. 13 National Education Association: 2011-2012 Average Starting Teacher Salaries by State 14 Missouri Economic Research and Information Center: Cost of Living Data Series Second Quarter 2014 15 United States Census Bureau: The 2012 Statistical Abstract: Educational Attainment: Table 233
- 9. Page | 9 V Results Union Strength (Union) This variable was insignificant in all three regressions, with p-values of 0.6733, 0.9996, and 0.6806 for Math, Reading, and Writing, respectively. Therefore, the strength of a teachers union in a state does not affect the average SAT score. Additionally, due to the variable’s insignificance, the coefficient and impact are also inconsequential. Average Class Size (ACS) This variable was also insignificant in all three regressions, with one-tail p-values of 0.2855, 0.3206, and 0.4884. This result shows that the average class size in a state does not affect the average SAT score, and that its impact and sign on the coefficient can be ignored. Enrollment per District (EpD) As predicted, the coefficient of enrollment per district has a negative sign with marginally significant p-values for Math (at the 10% level) and highly significant p-values (at the 1% level) for Reading and Writing. Referring to Figure 2 for the amount of the variable’s impact, a one standard deviation increase in enrollment per district causes SAT scores to decrease by 6.01 points on Math scores, 10.02 points on Reading scores, and 9.62 points on Writing scores. This confirms the original hypothesis that a lower enrollment per district leads to a higher state-level average SAT score. Cost of Living Adjusted Average Starting Teacher Salary (Tpay) The cost of living adjusted salary proved significant (at the 5% level) for Math and Reading and highly significant for Writing. Though the variable was significant, it did not have the coefficient
- 10. Page | 10 I had anticipated. A one standard deviation increase salary leads to a decrease of 7.47 points on Math scores, 9.39 points on Reading scores, and 8.22 points on Writing scores. Percent of People with a Bachelor’s Degree (PBD) The percent of people with a bachelor’s degree also proved marginally significant for Math and Reading and highly significant for Writing. This variable also had the predicted positive coefficient, with a one standard deviation increase leading to an increase of 4.46 points for Math scores, 5.46 points on Reading scores, and 8.04 points on Writing scores. VI Conclusion The research presented in this paper addresses economic factors that affect student performance using state-level average SAT scores on Math, Reading, and Writing as a proxy for student achievement. Each section of the SAT was run in its own regression to control for any effects of aggregate scoring. Of the five variables tested in this paper, enrollment per district, cost of living adjusted average starting teacher salary, and percent of people with a bachelor’s degree proved to be significant with similar impacts on SAT scores. Enrollment per district had the predicted negative coefficient, confirming the hypothesis that smaller relative representation in school districts overpowers any positive gains from economies of scale in larger districts. Though charter schools and private school vouchers seek to lower the enrollment in school districts, increasing the number of districts is not a widespread campaign for reform. I believe that this is because of the large range and standard deviation among states, and more districts may only be beneficial in certain states. Based on the results, the two states in this sample that would benefit most from more school district would be Florida and Hawaii. Florida has
- 11. Page | 11 over 2.6 million students enrolled in public schools and only 67 school districts16 . Hawaii has over 177,000 students and only 1 school district that encompasses all eight major islands, making representation in district meetings much more difficult and costly than other states. While I have noted some problems with the coefficient on this variable, the results for the cost of living adjusted average teacher starting salary could be caused by more than purely economic phenomena. It is possible that the best teachers are simply not motivated by money, and chose their career without giving much consideration to differences in income across districts and states. This would also account for the possibility of some teachers going to poorer performing districts because they are purely incentivized by the opportunity to help underachieving students succeed over a higher salary. These results are unclear to whether the sign on this coefficient would be positive if there were a different method for teacher compensation, or if the best teachers are indifferent to their salaries regardless of how they are determined. The significance of the percent of people with a bachelor’s degree demonstrates that higher educated states have higher performing students. However, since the variable measures the entire population of the state and not only the parents, there may be some room for interpretation on the true impact on SAT scores. Unfortunately, with the lack of a nation-wide uniform standardized achievement test, there is no proxy available that perfectly assesses student performance. However, the results of this study are able to support the claims that more school districts may be beneficial for certain states, teachers respond differently to their current compensation than expected, and that more educated states have higher performing students. 16 A comparable state in the size of enrollment would be New York, which has 695 school districts.
- 12. Page | 12 VII Output Figure 1: Descriptive Statistics Mean507.5Mean502.15Mean486.65Mean27.4 StandardError3.379894112StandardError3.707939561StandardError3.644227591StandardError4.20551017 Median504.5Median496Median484Median28 Mode528Mode492Mode468Mode#N/A StandardDeviation15.11534598StandardDeviation16.58240983StandardDeviation16.29748124StandardDeviation18.80761324 SampleVariance228.4736842SampleVariance274.9763158SampleVariance265.6078947SampleVariance353.7263158 Kurtosis-0.554703498Kurtosis-0.497327975Kurtosis-0.90607505Kurtosis-1.688192732 Skewness0.459747574Skewness0.538550368Skewness0.253558061Skewness-0.144408871 Range53Range62Range55Range50 Minimum487Minimum477Minimum461Minimum1 Maximum540Maximum539Maximum516Maximum51 Sum10150Sum10043Sum9733Sum548 Count20Count20Count20Count20 Mean15.8185Mean17555.8844Mean33659.03106Mean0.2822 StandardError0.644827302StandardError8816.050534StandardError979.0994131StandardError0.008401942 Median14.975Median3801.433079Median33583.02388Median0.2745 Mode#N/AMode#N/AMode#N/AMode#N/A StandardDeviation2.883755364StandardDeviation39426.57658StandardDeviation4378.665689StandardDeviation0.037574627 SampleVariance8.316045SampleVariance1554454940SampleVariance19172713.22SampleVariance0.001411853 Kurtosis-0.782581317Kurtosis16.17123147Kurtosis-0.55735822Kurtosis-0.522395302 Skewness0.290151733Skewness3.897068554Skewness0.013503665Skewness0.255953431 Range10.28Range177440.2657Range15360.0762Range0.139 Minimum10.47Minimum293.7342657Minimum25819.38326Minimum0.218 Maximum20.75Maximum177734Maximum41179.45946Maximum0.357 Sum316.37Sum351117.6881Sum673180.6211Sum5.644 Count20Count20Count20Count20 MathScore ACS ReadingScoreWritingScoreUnion EpDTpayPBD
- 13. Page | 13 Figure 2: Variable Impact A variable’s impact is calculated by multiplying its Standard Deviation by its Regression Coefficient. A variable’s impact is defined as the change on the average SAT score by a 1 standard deviation increase in a significant independent variable. Math Impact Reading Impact Writing Impact Union 1.455827 Union -0.00148 Union 1.1017477 ACS 3.355743 ACS 2.915263 ACS 1.6668106 EpD -6.0146 EpD -10.0229 EpD -9.615793 Tpay -7.47117 Tpay -9.38995 Tpay -8.223388 PBD 4.461549 PBD 5.455162 PBD 8.0407849