Using Data to Improve Minority-Serving Institution Success
1 like431 views
This document discusses how Minority-Serving Institutions (MSIs) can better use data to improve student and institutional success and meet national college completion goals. It outlines that MSIs have traditionally educated underserved student populations but some are not fully utilizing data to assess areas for improvement. The document recommends that MSIs adopt interim measures to track student progress and success at multiple points along their educational pathway, such as placement upon entry, persistence in continuous enrollment, progression toward earning a credential, and ultimate completion. These interim measures can provide a more comprehensive view of student outcomes than traditional metrics like graduation rates alone.
Using Data to Improve Minority-Serving Institution Success
- 1. To meet our nation’s college completion goals by 2025, postsecondary institutions must graduate a total of 23 million more students over the next 13 years.1 As the higher education sector continues to consider strategies to meet this ambitious goal, it is crucial that higher education institutions use data effectively to analyze where they are, where they need to be, and what steps will get them there. Many institutions that serve large numbers of 21st century students who are crucial to meeting the goal and have been traditionally underserved in the past—such as students of color, low- income students, and first-generation college students—have extensive knowledge of how to best support students and reduce barriers from enrollment to graduation. Minority-Serving Institutions (MSIs) in particular have historically educated and graduated a large proportion of underserved students. Therefore, MSIs have great potential for graduating an even larger number of college graduates over the next decade. INTRODUCTION This brief highlights how MSIs can better identify, collect, and use data2 for internal decision making and provide external audiences with a deeper understanding of how MSIs contribute to the higher education landscape. Specifically, this brief highlights how MSIs from the Lumina MSI-Models of Success project (see BOX 1) have used data to implement policy and programmatic changes on their campuses in support of student and institutional success. The goal of this brief is to continue a conversation about ways MSIs can build upon their data work to improve future reporting, analyses, and decision making. In addition, the lessons shared in the brief have broad application to other institutions, especially those that serve students from disadvantaged backgrounds. This is the second in a series of briefs by the Institute for Higher Education Policy (IHEP) to feature emerging themes from the Lumina MSI-Models of Success program. MSIs are well known for their academic and social supports, strong cultural understanding of the populations they serve, and targeted missions to educate and graduate students of color.3 Some, however, are not fully using data to their advantage. Finding better ways to measure success, progress, and the exceptional benefits of MSIs USING DATA TO IMPROVE MINORITY-SERVING INSTITUTION SUCCESS By Bi Vuong and Christen Cullum Hairston October 2012 1 Lumina Foundation Goal 2025. Retrieved from http://www.luminafoundation.org/goal_2025/goal3.html. 2 Unless otherwise indicated, the term “data” in this brief refers to student-level longitudinal data (i.e., data collected on individual students that can be linked over time to help assess student progress. 3 N. Harmon. 2012. The Role of Minority-Serving Institutions in National College Completion Goals. Washington, DC: Institute for Higher Education Policy. 1
- 2. 2 BOX 1: LUMINA MSI-MODELS OF SUCCESS MSIs include Historically Black Colleges and Universities (HBCUs), Tribal Colleges and Universities (TCUs), Hispanic-Serving Institutions (HSIs), and as of 2008, Asian American and Native American Pacific Islander -Serving Institutions (AANAPISIs). The Lumina MSI-Models of Success project began in fall 2009 and will continue until fall 2012. Seeking to dramatically increase college completion— especially among first-generation students, low -income students, and students of color—the Lumina MSI-Models of Success project is partnering with more than 25 MSIs and other organizations to improve and document increased postsecondary attainment. Participating institutions and organizations embrace a collective MSI success agenda. The Lumina MSI-Models of Success project has five objectives: 1. Improve the capacity of MSIs to collect, analyze, and use data to inform decisions that will promote student success. 2. Create a collective voice for policy advocacy on behalf of MSIs. 3. Strengthen policy and practice to improve developmental education. 4. Increase MSIs’ commitment to transparency and effectiveness in improving student learning outcomes. 5. Increase the postsecondary completion of traditionally underserved students, especially men of color. As the key intermediary for the initiative, IHEP provides technical assistance and support for eight grantees. IHEP also assists with the documentation and dissemination of project findings to inform the higher education success and policy agenda at the federal, state, and institutional levels. beyond the typical data points is vital to illustrating their value to higher education and the college completion agenda. Traditional data points such as graduation rates do not always show the complete picture of MSIs’ successes, as they do not always adequately measure MSIs’ unique educational experiences and targeted missions. Some solutions not only improve measures at a student’s entrance and exit from the university but also include interim measures—a concept that this brief will describe in detail. Data collection and analyses at all points of a student’s educational pathway are at the core of providing quantitative, tangible benchmarks and points of improvement. In line with this goal, MSIs are paying closer attention to how they can use data to identify opportunities and develop strategies for improving the delivery and support of institutional goals. Historically, most colleges (not just MSIs) have generated data primarily to meet federal or state reporting requirements.4 However, changes in the national policy landscape, projected trends in the labor market, and the growing focus of many states on postsecondary completion have compelled MSIs and other institutions to consider how they can develop and sustain an internal culture of data-informed decision making on campus—all in support of college student success. Some MSIs are using data in new and notable ways that can inform other efforts. The change from collecting data for compliance purposes to using data to inform decision making requires a paradigm shift. To facilitate this shift, institutional leaders at MSIs must demonstrate that research and evaluation for institutional improvement is a priority. They can do so by dedicating institutional resources to data collection and analyses, and by changing institutional policies and programs based on the findings. Institutions will also have to dedicate time and resources to collecting the right data, implementing data systems and processes that ensure data quality and developing internal data analysis capacity (see Box 2). Absent these conditions, institutions may have only anecdotal evidence, rather than data, to draw on in their efforts to assess policy and program effectiveness, and they may struggle to assess areas of weakness and strategies for improvement. “Finding better ways to measure success, progress, and the unique benefits that MSIs provide beyond the typical data points is vital to illustrating the value of MSIs to higher education and the college completion agenda.” 4 L. Allen and R. Kazis. 2007. Building a Culture of Evidence in Community Colleges: Lessons from Exemplary Institutions. Boston, MA: Jobs for the Future. Retrieved from http://www.jff.org/publications/education/building-culture-evidence-community-coll/208. 2
- 3. USING WHAT YOU HAVE Federal data, such as those collected in Integrated Postsecondary Education Data Survey (IPEDS), is often used for compliance purposes. For example, the standard definition of graduation rates, as mandated in federal data collections, is calculated using first-time, full-time students who graduate from the same institution.5 Although this measurement is not unique to MSIs, it is problematic because MSIs educate and graduate many part-time, nontraditional, and transfer students who may not be counted in the standard graduation rate calculation. Consequently, MSIs may appear to have low graduation rates, which can affect their funding and peer competitiveness. But several MSIs have used these data in novel ways to illustrate students’ progress from enrollment to degree completion. For example, the University of Texas-El Paso (UTEP), an HSI and participant in the Lumina MSI-Models of Success project, not only measures the traditional four- and six-year graduation rates but also has created a “degree production ratio.” This ratio looks at the relationship between baccalaureate graduates (the grand total of baccalaureate degrees awarded in the first major) and the total number of full-time equivalent (FTE) undergraduates enrolled four years earlier (12-month FTE undergraduate enrollment). This type of measurement can provide a different perspective to BOX 2: THE DIMENSIONS OF DATA-INFORMED DECISION MAKING Capacity for Data Use and Analysis Collecting the right data and ensuring their accuracy are basic to data-informed policymaking. Equally critical is the capacity to analyze and use these data effectively. To build this capacity, institutions must hire staff with the right set of data skills and the policy and program knowledge to conduct and interpret analysis, and enable policymakers and institutional leaders to integrate findings into their decision-making processes. In sum, using data effectively for policymaking requires investing the time and effort to look at the analysis and assess the implications, and being willing to create or adjust policies based on the findings. SOURCE: B. Vuong. 2011. Leveraging Data for College Completion. Pathways to College Network. Washington, DC: Institute for Higher Education Policy. 5 IPEDS defines graduation rates as follows: “Data are collected on the number of students entering the institution as full-time, first-time, degree/certificate -seeking undergraduate students in a particular year (cohort), by race/ethnicity and gender; the number completing their program within 150 percent of normal time to completion; the number that transfer to other institutions if transfer is part of the institution's mission.” Retrieved from http://nces.ed.gov/ipeds/glossaryindex.asp?id=812. 3 THE DIMENSIONS OF DATA-INFORMED DECISION MAKING Data can be used in multiple ways to inform decision making and subsequently drive policy and programmatic change. Data-informed decision making has three dimensions: (1) Data collection, (2) data management, and (3) capacity for data use and analysis. Data Collection The purpose of data collection is straightforward: To obtain information. Yet since education data can be collected for various reasons and in various ways, it is important that policymakers, institutional leaders, and practitioners identify the policy, programmatic, and operational issues they would like to address before using the data. A significant challenge to data collection is gathering the right data on the front end. Decision makers play a central role in providing the context within which staff members collect data points; encouraging their input and commitment at an early stage will make data collection and subsequent analysis most effective for a variety of purposes. Data Management Proper data collection is essential for proper analysis. Data management, in contrast, can ensure the accuracy of the data collected, as well as proper data analysis. Among the components that contribute to sound data management are having robust data governance structures in place and addressing issues of data access, privacy, and security. By supporting solid data management practices, decision makers will help ensure that their institutions collect accurate data. Thus, any decisions informed by data analysis will be based on sound information.
- 4. 4 UTEP’s data by adding to the cohort graduation rates and showing all graduates as a ratio of those who entered college four years before. Morgan State University President David Wilson6 recently discussed how he used standard compliance data to illustrate the success of students at his institution. To an audience of MSI leaders, Wilson shared how he used disaggregated student data to make comparisons among Morgan State University students and comparable students from institutions across Maryland. By disaggregating these data, he was able to show that students from low-income backgrounds were succeeding at Morgan State University at comparable (and sometimes higher) rates compared with their peers across the state. Disaggregating these data allowed Wilson to make two strong statements: First, Morgan State University is making good progress in educating its students, and second, state policymakers and institutional leaders should have a more comprehensive depiction of Morgan State University’s outcomes beyond the limitations of traditional graduation rates. MSIs and other institutions that seek to better illustrate their success should consider the same strategy. These examples, amongst others, demonstrate the importance and power of using existing data. In March 2012, the U.S. Department of Education highlighted this point in its Action Plan for Improving Measures of Postsecondary Student Success. This plan outlines the ways in which the federal government wishes to “help institutions, systems, and states to increase their capacity for collecting and disseminating data on student success” and to “improve the quality and availability of student success data at the federal level for consumers, institutions, policymakers, and researchers.”7 Among other things, the plan calls for graduation data to be expanded to account for the diversity of students enrolled (i.e., part-time, adult, certificate) at two-year institutions, and suggests improvements to data elements and reporting on student success and completion in both the IPEDS and the National Student Loan Data System. Although the efforts outlined in the Action Plan are designed specifically for two-year colleges, they are also applicable to four-year institutions. The plan is suitable for all types of institutions of higher education considering the need for improved data systems, collection, and reporting to meet national college completion goals. It also reinforces the need for state and federal systems and institutions to work together to better reflect student success at colleges and universities nationwide. EDUCATION PATHWAY: MEASURES OF SUCCESS ALONG THE WAY Many MSI leaders recognize that students’ educational paths can be varied and not easily captured through federal data. They have begun to use their own institutional data to better assess student success and track progress toward degree completion. The indicators used are sometimes referred to as “interim measures.” Recent research8 recommends using interim measures to mark intermittent steps or show students’ momentum. This study focuses on interim measures in the context of community colleges; however, this framework is easily applied to four-year institutions as well. Both two- and four-year MSIs should consider creating their own interim measures to better measure and communicate students’ progress. This type of information could be beneficial for both external and internal decision makers. The overarching goal is not only to collect front-end (e.g., admission test scores) and back-end (e.g., graduation rates) data, but also to have a more consistent and systematic data collection process. This approach will allow the institution a comprehensive picture of students as they move along their education pathway. Four interim measures that all types of institutions can identify and measure along a student’s education pathway are: (1) Placement, (2) persistence, (3) progression toward a credential, and (4) completion. These measures are defined as follows: Placement measures are used to assess students upon entry to the institution, a major field of study, or a specific course. Placement is key because it evaluates where students “begin,” and helps them get placed in the appropriate courses and academic programs. Placement tests, pre-course assessments, and prior learning credentials are examples of indicators that could serve as baseline measures to ensure that students are appropriately placed. Correctly assessing students at the point of entry heightens the likelihood of success and provides the foundation upon which to examine future progress. Persistence measures indicate continuous enrollment. Although persistence is commonly measured by semester-to- semester enrollment, it can be measured in various ways, including course-to-course (especially for gateway, and 6 David Wilson was a keynote speaker at the 2012 Lumina MSI-Models of Success Summer Grantee Meeting. In attendance were representatives from all of the Lumina MSI-Models of Success grantee institutions and partner institutions, who met to discuss, collaborate, and celebrate the successes of their projects. Other keynote speakers included Karen Gross, senior policy advisor from the U.S. Department of Education; and Clay Pell, White House Fellow, National Se- curity staff, and grandson of the late Senator Claiborne Pell. 7 U.S. Department of Education. 2012. Action Plan for Improving Measures of Postsecondary Student Success. Retrieved from http://www.accjc.org/wp-content/uploads/2010/09/Action-Plan-for-Improving-Measures-of-Postsecondary-Student-Success-FINAL2.pdf. 8 J. Offenstein and N. Shulock. 2010. Taking the Next Step: The Promise of Intermediate Measures for Meeting Postsecondary Completion Goals. Boston, MA: Jobs for the Future. Retrieved from http://www.jff.org/sites/default/files/ATD_TakingtheNextStep_092810.pdf. 4
- 5. 9 M. Gasman. 2012. “The Government’s New Way of Measuring Student Success: Implications for Black Colleges,” Huffington Post. Retrieved from http://www.huffingtonpost.com/marybeth-gasman/the-governments-new-way-o_b_1420456.html. 10 In September 2010, the Bill & Melinda Gates Foundation Released the concept paper “Completion by Design” and the accompanying framework “Support Student Success: Preventing Loss, Creating Momentum” to support community colleges. Although closely related, the four points identi- fied in the framework have been modified for this brief. The original framework can be accessed at: http://completionbydesign.org/sites/default/files/cdat_loss&momentum_r101_0.pdf. 5 major course sequences), credit-hour accumulation, year-to-year, academic standing (i.e., moving from junior to senior year), or transfer. Institutions use various indicators to assess students’ progress on these pathways. Some of the more common markers include early warning or interim progress reports, enrollment and course-taking patterns, and grade point average (GPA). Institutional leaders often analyze the risk factors associated with negative student performance and use those factors to predict problems and develop strategies to ensure continuous enrollment. Progression toward a credential includes measures that determine whether students are making satisfactory progress toward graduation. Academic audits highlight this type of progress, but they are often done at the end of a student’s academic experience. More periodic audits should occur to ensure that students are progressing toward their degree by obtaining the appropriate credits and grades. Progression indicators can also be used to support and reinforce students’ transfer readiness and development of non-cognitive skills that support academic success. Completion refers to attaining a postsecondary credential. Although it has been largely defined by graduation rates, new measures of completion, such as those discussed in the U.S. Department of Education’s Action Plan9 are being encouraged. Collecting data incrementally can help faculty and staff better identify and support students, particularly those in need of additional or specialized academic and social supports, early and at various intervals. Some institutions may find that it is difficult to use data in this manner at the outset, as it shifts when data is collected and how it is used to inform decision making. The following section highlights examples from the Lumina MSI-Models of Success project that show how data are used to measure momentum at the four aforementioned intervals.10 Each example reflects a particular institution’s needs, but MSIs and other institutions can all replicate these strategies. Although each institutional experience is different, by assessing the factors that affect student progress in these four areas, institutional leaders can address some of the key challenges to degree completion.
- 6. 6 Based on their work in the Lumina MSI-Models of Success project, UTEP, El Paso Community College, Texas A&M University, and Prairie View A&M University collectively established an iterative and dynamic process of data collection and analysis to address issues related to persistence of first-time students. The UTEP team was able to refine and expand its research questions, supplement its data collection, and develop a Risk Stratification Model to identify factors that negatively affect student success, such as academic background, enrollment patterns, financial aid, and time to degree. UTEP supplemented its quantitative analyses with qualitative data gained from student focus groups designed to provide a deeper examination of the factors contributing to persistence challenges. UTEP now regularly generates system-wide reports that identify first-time students who may require additional support, and offers these students follow-up services, including workshops on developing academic strategies to help them persist beyond the first year and succeed in college. As part of the Lumina MSI-Model of Success Project, Salish Kootenai College (SKC) and Fort Peck Community College undertook an action research project focused on American Indian students in need of developmental course work in mathematics, English, and reading. Currently, 70 percent of all SKC students start in developmental education. The research was intended to identify the factors that contribute to retention and success. An initial review of course completion and retention data showed high dropout rates, low persistence rates in gateway courses, and low long-term persistence rates for students enrolled in developmental education. An analysis of student placement exams revealed that a significant number of students had not been placed in appropriate developmental courses. Based on these results, SKC modified its placement process to ensure that students were enrolled in the correct academic and support courses. To implement this new process, the Department of Academic Success (DAS) and Developmental Task Force trained DAS staff regarding placement and worked with faculty to help them understand placement results. Initial results have been promising, as the percentage of students incorrectly placed has declined from 30 percent to 8 percent. In addition, the change for developmental students from fall 2007 to fall 2011 coincided with an increase in passing remedial courses of 22 percentage points over four years. Overall, the proper identification of students in need of developmental education and the subsequent placement and support for these students have resulted in an increase in student success at SKC and Fort Peck Community College over the past four years. 6 PLACEMENT PERSISTENCE
- 7. To assess the efficacy of the Dual Degree program, in which students transfer from Miami Dade College (MDC) to Florida International University (FIU), the institutions established a process to collect and analyze relevant data associated with students who initially applied to FIU but were not ready to enroll, and students who completed an associate’s degree in three years at FIU. Several analyses have been conducted, resulting in changes to institutional practices. For example, the FIU team determined that students who declined offers to participate in the program were less informed or more confused about the program than students who accepted offers. Consequently, the admissions office has changed its invitation letter, is updating its website to include additional program information, and is making follow- up calls to students who do not respond to the initial offer. A review of data on students who transferred to FIU revealed that some were taking fewer courses at FIU than at MDC, thus slowing their credit accumulation and degree completion rates. To understand this trend, FIU interviewed 25 of these students, many of whom communicated a lack of clarity about FIU’s degree expectations prior to transfer. As a result, students often faced problems getting placed in their major of choice, stemming from such issues as a lack of prerequisite courses, GPA requirement barriers, and challenges related to course availability. An FIU advisor is now available on MDC’s campus several days a week to assist students. The program data available from MDC (fall 2006 to spring 2010) and FIU (fall 2006 to spring 2011), show that 18,286 students were invited to the program, 4,470 accepted, 4,122 enrolled, and 3,222 (78.2 percent of enrolled students) took remedial courses at MDC. Of this last group, 1,389 transitioned to FIU. These data show that by identifying the additional 18,286 potential students for the program, even in its early years the program added 1,224 associate’s degree graduates and 140 bachelor’s degree graduates, with many more anticipated in the future. The data points highlighted above exemplify how MSIs can use data both to improve student outcomes and to report out their successes. Across the policymaking spectrum, the push to use data more systematically for decision making continues to gain momentum. The Obama administration, state governments, and many major foundations (notably Lumina Foundation) have all emphasized the need for evidence-based reform in higher education, and specifically as a tool for meeting the national completion agenda, as mentioned previously within the U.S. Department of Education’s Action Plan. The support for data-based inquiry presents an extraordinary opportunity for institutions and policymakers to take advantage of sound data in key decision making around student success in college. SKC, UTEP, and FIU are strong models of how data can be used to analyze barriers to student success and to inform decision making. Although campuses participating in the Lumina MSI-Models of Success Project have collected data linked to various interim measures, in most cases completion data are not yet available. Nevertheless, since participating in this project, all institutions have begun using data to better evaluate and support students as well as rethink prior assumptions and practices related to data and decision making. PROGRESS COMPLETION 7
- 8. 8 BOX 3: LUMINA MSI-MODELS OF SUCCESS GRANTEES American Indian Higher Education Consortium, Alexandria, Va. California State University-Monterey Bay, Monterey, Calif. Hartnell College, Salinas, Calif. Cabrillo College, Aptos, Calif. Florida International University, Miami, Fla. Miami Dade College, Miami, Fla. Jackson State University, Jackson, Miss. Alcorn State University, Alcorn, Miss. Dillard University, New Orleans, La. Hinds Community College, Utica, Miss. Miles College, Fairfield, Ala. Tougaloo College, Tougaloo, Miss. Salish Kootenai College, Pueblo, Mont. Fort Peck Community College, Poplar, Mont. Southern Education Foundation, Atlanta, Ga. University of North Carolina System, Chapel Hill, N.C. Elizabeth City State University, Elizabeth City, N.C. Fayetteville State University, Fayetteville, N.C. North Carolina A&T State University, Greensboro, N.C. North Carolina Central University, Durham, N.C. UNC-Pembroke, Pembroke, N.C. Winston-Salem State University, Winston-Salem, N.C. University of Texas-El Paso, El Paso, Texas El Paso Community College, El Paso, Texas Prairie View A&M University, Prairie View, Texas Texas A&M International University, Laredo, Texas *Those in bold indicate the lead institution or organization. The Institute for Higher Education Policy (IHEP) is a nonpartisan, nonprofit organization committed to promoting access to and success in higher education for all students. Based in Washington, D.C., IHEP develops innovative policy- and practice-oriented research to guide policymakers and education leaders, who develop high-impact policies that will address our nation’s most pressing education challenges. 1825 K Street, NW, Suite 720 Washington, DC 20006 202 861 8223 TELEPHONE 202 861 9307 FACSIMILE www.ihep.org WEB CONCLUSION Using data well at MSIs requires investing the time and effort to collect the data, look at the analysis, and assess the implications, as well as to create or adjust policies based on the findings. With the right type of high-quality data, the ability to analyze these data, and the capacity to translate analytical findings into policy solutions, MSIs can embark on a continuous and sustainable process of improvement. The postsecondary sector has a long history of collecting longitudinal data. Contextual analyses can help institutions identify areas that need improvement, while deeper analyses can reveal institutional strengths and suggest strategies for removing barriers to student success. This is only the first step. Institutions should continue to engage their leadership and strengthen their data systems to enable continuous improvement and sustainability of effective policies and practices. In this time of tight budgetary constraints and intense college completion initiatives, it is imperative for MSIs to highlight the uniqueness of their experiences and successes and to use their data not only for internal decision making and improvement but also to educate external stakeholders. Now and into the future, MSIs should collaborate on and coordinate their data collection efforts to create a unified MSI message. It is important to consider developing new measures of success that reflect the effective work of MSIs. As institutions continue to improve their data systems, researchers, practitioners, and policymakers can better understand the MSI experience. In addition, improved data collection can provide stronger structures and efforts toward our nation’s goals of increased college completion and student success.