A Framework for Examining Tailored Longitudinal Data to Advance Institutional Effectiveness and Student Completion
- 1. A Framework for Examining Tailored Longitudinal Data to Advance Institutional Effectiveness and Student Completion Center for Research on Educational Access and Leadership 2015-2016 Colloquium Series October 7, 2015 Dr. David El Fattal Vice President of Business Services Cerritos Community College District
- 2. 1 Topic Background Public scrutiny of colleges and universities has escalated to a high level as taxpayers, students, and policymakers alike want to know they are receiving quality, value, and institutional effectiveness for the price they pay (Alfred, 2011; Bailey, Jenkins, & Leinbach, 2005; Boggs, 2011; Ewell, 2008a, 2011; Head, 2011; Shulock, Offenstein, & Esch, 2011). Increasing core student completion outcomes such as certificate, transfer, and graduation rates is a critical public policy goal (Assembly Bill 1417, 2004; Douglass, 2010; Ewell, 2011; Gansemer-Topf & Schuh, 2006; Little Hoover Commission, 2012; Obama, 2009; Senate Bill 1456, 2012; Shulock & Jenkins, 2011; Titus, 2006; Tschechtelin, 2011).
- 3. 2 Topic Background A limited number of higher education studies have examined the relationships between institutional expenditures and student achievement outcomes (Hamrick, Schuh, & Shelley, 2004; Pike, Smart, Kuh & Hayek, 2006; Pike, Kuh, McCormick, Ethington, & Smart, 2011; Ryan, 2005; Scott, Bailey, & Kienzl, 2006; Walpole, 2003; Webber & Ehrenberg, 2010). To What Degree Does Money Matter for Student Success? Dissertation research was first study of California Community Colleges examining the relationships between institutional expenditures and student attainment milestones outcomes (El Fattal, 2014).
- 4. 3 Purpose of the Research The purpose of the research was to identify whether, or to what degree, the allocation of institutional expenditures for instructional, student support, administrative, and total educational and general activities influenced the student success outcomes and indicators of SPAR, thirty-unit completion, and persistence at the 72 districts and 112 institutions that comprise California Community Colleges (California Community Colleges Chancellor’s Office, 2010).
- 5. 4 Research Methodology and Design Conducted a quantitative, non-experimental, correlational study with an explanatory research design and post-positivist theoretical foundation. Highly skewed data resulted in a series of bivariate analyses instead of the initial plan for hierarchical linear regression. Underlying research focus was to better understand the relationships between institutional expenditures and student attainment outcomes. Sought to identify the nature, direction, and strength of the associations between 46 pairs of variables. Utilized Astin’s (1993) Input-Environment-Output (I-E-O) model as conceptual framework.
- 6. 5 Input Variables Mediating Environment Variables Output Variables 1 Socioeconomic status Instructional expenditures per FTES SPAR—student progress and achievement rate 2 District Type – Single-college district – Multi-college district Student support expenditures per FTES Thirty-unit Completion—percentage of students who earned at least 30 units 3 Administrative expenditures per FTES Persistence—persistence rate 4 Total educational and general expenditures per FTES 5 Ratio of instructional cost to total cost 6 Ratio of student support cost to total cost 7 Ratio of administrative cost to total cost 8 Ratio of instructional cost to student support cost Research Methodology and Design
- 7. 6 Research Methods Data collection and management All source data were publicly available from the U.S. Census Bureau (2013), or the California Community Colleges Chancellor’s Office (2010, 2011a, 2011b). All data were accurately combined into one dataset of records for individual colleges and college districts. Multi-college district SES and outcomes variables were computationally weighted from the values of individual colleges. Highly skewed data resulted in a series of bivariate analyses instead of the initial plan for hierarchical linear regression. All data (except district type) were transformed into ordinal variables of low, middle, and high status.
- 9. 8 Key Findings and Interpretations (a) SES was the key influencing factor for the attainment of each outcome: SPAR, thirty-unit completion, and persistence. Low SES resulted in low outcomes. High SES produced high outcomes. (b) Resource allocations for administrative, student support, and instructional expenditure categories were significantly different for low SES and high SES institutions. (c) Low SES districts spent comparatively more than high SES districts on administrative functions and student support activities, and comparatively less on direct classroom instruction. Meanwhile, high SES districts did the inverse and spent comparatively less than low SES districts on administrative functions and student support activities, and comparatively more on direct classroom instruction.
- 10. 9 Key Findings and Interpretations (d) Higher proportional spending on instructional cost was related to high outcomes. (e) Higher levels of spending on student support activities or administrative activities in relation to total costs were related to lower levels of SPAR and thirty-unit completion. (f) The SES of the student population drove the allocation of resources among expenditure categories. (g) Low SES institutions typically serve large populations of underprepared students and have reason to allocate resources differently than do higher level SES institutions.
- 11. 10 Abridged Implications Substantially changing student demographics will continue to bring an influx of underprepared students to community colleges (Crisp & Nora, 2010; Prescott, 2013). Annual budgets of California Community Colleges will continue to be intermittently impacted by potentially volatile revenue swings, providing crucial opportunities to reprioritize spending in alignment with legislative, system, and institutional imperatives. State and federal governments, and other stakeholders, will continue to pressure institutions to increase student completion outcomes, especially degrees, through the efficient and effective use of finite taxpayer dollars. SES may account for 60%–70% of student achievement (Coleman et al., 1966; Marion & Flanigan, 2001), leaving little room for other variables to have much impact on the attainment of outcomes.
- 12. 11 Abridged Implications SES is widely accepted as a leading influencing factor for student achievement. In studies that have examined SES as a variable of interest, the impact of SES on outcomes has been considerable (Astin, 1993; Coleman et al., 1966; Marion & Flanigan, 2001; Pacheco, 2012; Titus, 2006). Future research should purposefully and consistently integrate SES into quantitative and qualitative student attainment research. Astin’s (1993) I-E-O model has been a durable, prominent, and influential research framework utilized for decades to study the impact of college on the outcomes of how students grow or change, cognitively and non-cognitively. Traditionally, this framework has not been used to examine the allocation of expenditures or completion outcomes. Future research should consider doing so.
- 13. 12 Central Recommendations The modest zone outside the impact of SES is where researchers, policymakers, and practitioners should focus improvement efforts to increase student attainment. Each college, on its own or through a consortium, should become a self-directed I-E-O (Astin, 1993) research institute to gain a tailored, longitudinal, empirical basis for improving institutional effectiveness, cost effectiveness, student development, student completion, and other self-identified outcomes that are important for fulfilling the mission of the organization. The California Community Colleges Chancellor’s Office should provide start-up funding through a grant process to establish up to 10 internally focused I-E-O research institutes for interested and capable districts.
- 15. 14 Input Variables Mediating Environment Variables Output Variables 1 2 3 4 5 6 7 8 9 10 Longitudinal Data Discussion Guide
Editor's Notes
- #6: P. 76 In his nationwide longitudinal study that is presented in What Matters in College? Four Critical Years Revisited, Astin (1993) empirically analyzed complex relationships among variables by conducting 82 stepwise regressions on 146 input variables, 192 environmental variables, and 82 outcome variables on a sample of 24,847 students who in 1985 were first-time, full-time, traditional-aged freshman undergraduate students at bachelor’s degree-granting institutions. Community colleges and community college students were excluded from the study, in part, to ameliorate any potential confounding effects. Astin (1993) utilized the large number of variables in this study as a strategy to control for as much bias and error as possible in order that reasonable conclusions could be made about the causal relationships that emerged from correlational data. The findings of this substantial study are too numerous to delineate in my dissertation.
- #11: Elaborate on F
- #16: Examples SES: Student / Peer Group Disaggregated Expenditure Data Number of Degrees Earned Student Equity Metrics Scorecard Metrics IEP Standards