The Education Department’s “framework” for its college-ratings plan is surprisingly tentative, filled with verbs like “exploring” and “considering.” It can be seen as a smart move: Kick the ratings can down the road, telegraph what might be coming, get more stakeholder involvement, and so on. But it can also be seen as an OMG moment: After so much effort, so many meetings, and so much chatter, there remain far too many questions unanswered and far too many ratings criteria ill-defined.
“As a nation, we have to make college more accessible and affordable and ensure that all students graduate with an education of real value. Our students deserve to know, before they enroll, that the schools they’ve chosen will deliver this value. With the guidance of thousands of wise voices, we can develop a useful ratings system that will help more Americans realize the dream of a degree that unleashes their potential and opens doors to a better life.”
In today’s world, college is not a luxury that only some Americans can afford to enjoy; it is an economic, civic and personal necessity for all Americans. Expanding opportunity for more students to enroll and succeed in college, especially low-income and underrepresented students, is vital to building a strong economy with a thriving middle class and critical to ensuring a strong democracy. That is why President Obama has set the course for the United States to once again world in college attainment, as we did a generation ago.
Since the President took office, the Administration has worked to increase Pell Grants by more than $1,000 a year, created the new American Opportunity Tax Credit worth up to $10,000 over four years of college, capped student loan payments to 10 percent of monthly income, and laid out an ambitious agenda to keep college affordable. We have focused on improving college performance, promoting innovation and competition that can lead to breakthroughs on cost and quality, and helping students and families manage their student loan debt after college.
As a key part of this plan, the President directed the U.S. Department of Education to develop and publish a new college ratings system by the 2015-16 school year that would expand college opportunity by recognizing institutions that: excel at enrolling students from all backgrounds, focus on maintaining affordability, and succeed at helping all students graduate.
In response, the Department set out to design a ratings system that is clear, fair, and focused on a few key critical measures of institutional performance, while accounting for the diversity and complexity of the nation’s rich system of higher education. That said, many of the factors that contribute to a high quality postsecondary education are intangible, not amenable to simple and readily comparable quantitative measures, and not the subject of existing data sources that could be used across all institutions. Foremost among these are learning outcomes, which are central to understanding the value of an education but vary widely across programs and institutions and are communicated in many different ways. For many reasons – including the desire for simplicity of the ratings system, institutional autonomy and differences, and lack of shared approaches and data — it seems preferable at this time to concentrate on the core data elements addressed above and to allow institutions to provide additional information to prospective students as discussed below.
Based on extensive consultation with stakeholders and experts, the Department has now released a draft framework for the ratings system for further public comment. We will continue to refine the ratings system over time based on user and institutional experience, input from the field, and the availability of additional data.
The purposes of the ratings system are:
To help colleges and universities measure, benchmark, and improve across shared principles of access, affordability, and outcomes.
To provide better information about college value to students and families to support them as they search for select a college,
To generate reliable, useful data that policymakers and the public can use to hold America’s colleges and universities accountable for key performance measures. In the future this can be used to help align incentives for colleges to serve students from all backgrounds well by focusing on the shared principles of access, affordability, and outcomes; ensuring wise and effective use of $150 billion in financial aid.
In additional to federal efforts, and those of individual institutions, we believe the ratings system can help inform policy, accreditation and funding decisions by states education authorities, policies and practices of accreditors and others.
What Does the Draft Framework Cover?
In addition to this summary, the Department has released a detailed explanation of the ratings framework.
Institutions that will be included in the ratings system: The Department has received thoughtful feedback on the implications of determining which institutions are included in the ratings system, and which students within those institutions should contribute to an institution’s “rating.” In the first version of the ratings, the Department is planning to include:
Predominantly four-year institutions – colleges that primarily award baccalaureate degrees and above
Predominantly two-year institutions – degree-granting colleges that primarily award associate degrees and/or certificates
Graduate-degree-only and non-degree-granting institutions will not be included in the initial version of the ratings.
Rating Categories: This will be a ratings system, using broad categories to highlight significant success and weakness, not a numerical ranking of institutions. The Department has received many comments that the ratings should be focused, clear, simple and avoid “rankings” and false precision. The Department is considering three rating levels: high-performing, low-performing, and those in the middle. This will allow the ratings system to identify institutions that are performing very well, and those that are weak or struggling, without suggesting more nuanced distinctions than the data can meaningfully support. For consumers, policymakers, and the public, the Department feels that identifying extremely high- and low-performing institutions using a simple set of key measures is the most valuable action the Department can take.
Institutional Groupings: The Department has also received thoughtful feedback about differences in institutional characteristics and mission, and how those differences ought to be taken into account when rating institutions. Consistent with this feedback and the President’s goal to compare schools with similar missions, the Department will, at a minimum, have separate groupings for two-year and four-year institutions. The Department is also exploring several other strategies for grouping institutions together for comparison purposes. In addition to presenting four-year institutions separately from two-year institutions, the Department is considering accounting for differences in institutional characteristics such as degree and program mix and selectivity and are currently conducting data analyses to determine what, if any, more granular groupings could be used.
Recognizing Improvement: Recognizing performance improvement over time would encourage and support progress toward the key measures. The Department is interested in exploring how the system should give credit to colleges that are demonstrating such improvement.
What Data Will The Department Use?
This first version of college ratings will rely primarily on data from federal administrative data systems and data collections, including the Integrated Postsecondary Education Data System (IPEDS), the National Student Loan Data System (NSLDS), and earnings information. All data used for the ratings will be used and maintained according to the highest federal privacy standards. No student-level data will ever be released to the public; all student data will be presented only in aggregate.
“Designing a new college ratings system is an important step in improving transparency, accountability, and equity in higher education. The public should know how students fare at institutions receiving federal student aid, and this performance should be considered when we assess our investments and set priorities. We also need to create incentives for schools to accelerate progress toward the most important goals, like graduating low-income students and holding down costs – and shine a spotlight on schools that are effective in improving student success. We’re working with everyone – students, parents, states, the higher education community, researchers and experts – to make sure we do this well.”
What Metrics Are Being Considered?
The Department is exploring and seeking comment to weigh the pros and cons of possible metrics that could be used to narrow down the measures that will actually be used in the ratings system including:
Percentage of students receiving Pell. The percentage of a college’s enrolled students who receive federal Pell grants would offer a simple, clear measure of access for low-income students.
Expected family contribution (EFC) Gap. The Department is considering this metric as another way to gauge the degree to which institutions enroll students from lower-income family backgrounds. The EFC of students is generally considered to be a strong measure of whether a student comes from a low socio-economic status or high socio-economic status background.
Family Income Quintiles. This is another possible measure of the institution’s enrollment of low- to moderate-income students. Like percentage of students receiving Pell, this would be simple and easy for consumers to understand.
First-Generation College Status. The percentage of enrolled students who did not have a parent who attended college would reflect the extent to which institutions serve underrepresented populations in ways not measured by metrics based on EFC or family income.
Average Net Price. This measure – the cost of attendance after accounting for all federal, state, and institutional grant aid – is one we’re considering using to determine affordability. It comes closer to approximating the amount that students actually pay to attend an institution than published tuition and fees.
Net Price by Quintile. This measure offers a more accurate measure of the actual price paid by students of various family income backgrounds than average net price but is only available for those students who are federal student aid recipients.
Completion Rates. College completion rates including the IPEDS Graduation Rate measure – which includes only first-time full-time degree- or certificate-seeking undergraduates – aims to capture of how well institutions support students in persisting and achieving their educational goals. Students who complete college also have higher earnings and are less likely to default on student loan debt than those who don’t. The Department is scheduled to publicly release IPEDS graduation rates for a broader cohort of students (including part-time and transfer students) beginning in 2017.
The Department is also exploring the feasibility and reliability of an alternative to the IPEDS–based completion rate using data from NLSDS. An NSLDS constructed completion rate would allow us to incorporate completion outcomes for all federally aided students at an institution, including part-time students and transfer-in students. The data also allow us to construct completion measures for student sub-groups, such as Pell students, based on data from the FAFSA.
Transfer Rates. Since, for many students, a two-year college is a step toward completion of a bachelor’s degree, ED is exploring the viability of measuring transfer rates as a positive outcome for students.
Labor Market Success, such as Short-term “Substantial Employment” Rates and Long-term Median Earnings. Another important higher education outcome is ensuring students are prepared to students to successfully enter the workforce. Schools with comparable missions, student bodies and costs may have very different student employment and earnings rates, in both the short and longer term. The Department is exploring a variety of employment outcome measures that can provide critical employment information that appreciate those differences in ways that are sensitive to educational, career, work force, and other variables. The Department is considering looking short-term at measures that would determine if former students are making above an established threshold, such as 200 percent of the federal poverty line, while also stressing the importance of an individual’s potential lifetime earnings. In either case, the Department is committed to establishing measures that recognize the important contributions of relatively low-paying fields, such as public service, make to society.
Graduate School Attendance. In addition to completion and earnings, the Department is considering including a measure of graduate school attendance rates of former students within a period, like ten years, from entry into a school’s undergraduate program.
Loan Performance Outcomes. Relatively simple metrics like the percentage of students repaying their loans on time might be important as consumers weigh whether or not they will be able to handle their financial obligations after attending a specific school.
How it Fits Together
The Department would like feedback on the best way to present the metrics. Rating an institution on each metric separately could make it easier to identify where an institution may excel and where it is relatively weak and may need improvement. As a result keeping the metrics separate may allow for a more nuanced picture of varied institutions. On the other hand, without aggregating the metrics in some way, information about an institution’s performance across multiple measures relative to the principles identified for ratings that comprise “value” may be difficult to assess for consumers and difficult to map in order to align incentives for an accountability purpose.
Next Steps, College Ratings Website and Transparency Tools
Overall, the Department is considering how best to present the information about access, affordability, and outcomes in a way that increases transparency and meaningfully informs student choice. We welcome input from students, institutions and the public about the types of tools and formats that will meet this objective, as part of the system that the Department plans to release by the 2015-2016 school year. We will also conduct consumer testing on these tools, to ensure that the ratings can be easily understood by prospective students and other users.
The Department invites comment from the public on the draft framework by Tuesday, Feb. 17, 2015. Submissions can be submitted through this online form or by email to collegefeedback@ed.gov.
We will also arrange and participate in further meetings about the framework in the months ahead.
Submit comments on the framework. All comments, including those submitted by email, will be made public. All opinions, ideas, suggestions and comments are considered informal input. As such, the Department will not provide formal responses to ideas submitted and submissions may or may not be reflected in the final decisions. Information should be accessible to all individuals, including individuals with disabilities, and should not include links to advertisements or endorsements. Any advertisements and/or endorsements will be deleted before submissions are posted.
In addition to this summary, the Department has released a 
Overall, the Department is considering how best to present the information about access, affordability, and outcomes in a way that increases transparency and meaningfully informs student choice. We welcome input from students, institutions and the public about the types of tools and formats that will meet this objective, as part of the system that the Department plans to release by the 2015-2016 school year. We will also conduct consumer testing on these tools, to ensure that the ratings can be easily understood by prospective students and other users.
Teaching Blog of Ioannis Souldatos 