College is now the second-largest financial expenditure for many families, exceeded only by buying a home. So it isn’t surprising that parents and students are taking a harder look at the costs and payoffs of any college they consider.
To help, MONEY has drawn on the research and advice of dozens of the nation’s top experts on education quality, financing, and value, to develop a new, uniquely practical analysis of more than 700 of the nation’s best-performing colleges.
Here is the methodology behind our new 2016 rankings, first in a short version, then in a more comprehensive one.
MONEY’s Methodology in Brief
To make our initial cut, a college had to
- Have at least 500 students.
- Have a graduation rate that was at or above the median for its institutional category (public or private).
- Have sufficient data to be analyzed.
- Not be identified as in financial difficulty by the U.S. Department of Education or bond ratings agencies.
However, if a school’s graduation rate was below the median but was in the top 25% of our “comparative value” analysis (graduation rate after accounting for the average test scores and percentage of low income students among its enrollees), we made an exception and included it. We screened out military academies that require a commitment of service in exchange for free tuition.
That left 705 schools, which we ranked on 24 factors in three categories:
Quality of education (1/3 weighting), which was calculated using:
- Six-year graduation rate (35%), widely recognized as one of the most important indicators of a college’s quality.
- Peer quality (15%), measured by the standardized test scores of entering freshman (10%) and the percentage of accepted students who enroll in that college, known as the “yield” rate (5%).
- Instructor quality (15%), measured by the student-to-faculty ratio (.
- Value-added graduation rate (35%), or the difference between a school’s actual graduation rate and its expected rate, based on the economic and academic profile of the student body (measured by the percentage of attendees receiving Pell grants given to low-income students and the average standardized test scores of incoming freshmen).
Affordability (1/3 weighting), which was calculated using:
- Net price of a degree (35%), or the estimated amount a typical freshman starting in 2016 will pay to earn a degree, taking into account the college’s sticker price; how much the school awards in grants and scholarships; and the average time it takes students to graduate from the school, all as reported to the federal government. (The data on merit and need-based grants you’ll see reported on our website are numbers reported by the colleges to Peterson’s, and are for your information but are not used in the rankings.)
- Debt (20%), taking into account both student debt upon graduation (15%) and parent federal PLUS loans (5%).
- Student loan repayment and default risk (15%),the percent of students who have paid down at least $1 in principal in five years of payments, as reported on the federal College Scorecard, and the Student Loan Default Risk Index, which is a calculation that compares the number of of borrowers who default on their federal student loans as a percentage of the school’s enrollment.
- Value-added student loan repayment measures (15%), the school’s performance on the student loan repayment and default measures after adjusting for the economic and academic profile of the student body.
- Affordability for low- income students (15%), based on federally collected data on the net price that students from families earning $0 to $30,000 pay.
Outcomes (1/3 weighting), which was calculated using:
- Graduates’ earnings (15%), as reported by alumni to PayScale.com; early career earnings within five years of graduation (10%) and mid-career earnings (5%).
- Earnings adjusted by majors (15%). To see whether students at a particular school earn more or less than would be expected given the subjects students choose to study, we adjusted PayScale.com’s data for the mix of majors at each school; for early career earnings (10%) and mid-career earnings (5%).
- College Scorecard 10-year earnings (10%). The earnings of federal financial aid recipients at each college as reported to the IRS 10 years after the student started at the college.
- Brookings Institution analysis of the market value of alumni skills (10%), the market value of the 25 skills that alumni of each school most commonly list on their LinkedIn profile as valued by Burningglass.com.
- Comparative value earnings (15%). To see if a school is helping launch students to better-paying jobs than competitors that take in students with similar academic and economic backgrounds, we adjusted PayScale.com’s earnings data for the student body’s average test scores and the percentage of low-income students at each school; for early career earnings (10%) and mid-career earnings (5%).
- Career services (15%), measured by staffing per 1,000 students (7.5%) and the presence of a program that connects job-seeking students with alumni (7.5%).
- Job meaning (20%). We used the average score of each school’s alumni on PayScale.com’s survey question of “Does your work make the world a better place?”
Finally, we used statistical techniques to turn all the data points into a single score and ranked the schools based on those scores.
For a more detailed description of the methodology, see the next section.
NEWSLETTER: COLLEGE_PLANNERSign up for COLLEGE_PLANNER and more View Sample
MONEY’s Methodology in Detail
MONEY’s Best Colleges for Your Money rankings are the first to combine the most accurate pricing estimates available with students’ likely earnings after graduation and a unique analysis of how much “value” a college adds when compared to other schools that take in similar students.
We estimate a college’s comparative value by calculating its performance on important measures such as graduation rates, student loan repayment and default rates, and post-graduation earnings, after adjusting for the types of students it admits. We believe this analysis gives students and parents a much better indication of which colleges will provide real value for their tuition dollars.
We developed our ratings in partnership with one of the nation’s leading experts on higher education data and accountability metrics, Mark Schneider. The former commissioner of the National Center of Educational Statistics, he is currently a vice president at the American Institutes for Research (AIR) and president of College Measures, which collects and publishes public data comparing a student’s educational record and later earnings.
The final methodology decisions were made by the MONEY editorial team, in consultation with Schneider and College Measures.
In building our rankings, MONEY focused on the three basic factors that surveys show are the most important to parents and students:
- Quality of education
Because these three factors are so interrelated and crucial to families, we gave them equal weights in our ranking.
In each of these three major categories, we consulted with our advisers to identify the most reliable and useful data to assess a school’s performance. We also balanced the basic data in each category with at least one “value-added” measure.
To gauge how well a college is performing relative to its peers, we gathered federal and college data on the admissions selectivity of each school such as average test scores and grade point averages of students at each college, as well as the percentage of each graduating class receiving degrees in each major, and the percentage of students with incomes low enough to qualify for Pell Grants (about 90% of which go to families with incomes below $50,000). We then used the statistical technique of regression analysis to determine the impact of a student’s test scores, economic background, and college major on key factors, such as graduation rates and future earnings. That enables us to see how much better or worse a particular college performed than would is typical for schools with similar students.
We used this estimate of comparative performance by the college as an important part of the ranking, as you’ll see below.
MONEY assigned each indicator a weight based on our analysis of the importance of the factor to families, the reliability of the data, and our view of the value of the information the data provided.
To avoid overloading readers with too many choices or too much data, and to ensure that we fairly compared apples with apples, we decided to analyze only those colleges that, in our view, passed these minimal quality and data tests. We decided that to be included in our rankings, a college must meet these five criteria:
- Be a public or not-for-profit four-year college or university.
- Have enough cost, quality, and outcomes data available to provide at least moderate confidence in our assessment.
- Have at least 500 full-time undergraduates.
- Not be financial trouble, as indicated by a below-investment grade rating for its bonds by bond rating agencies, or by inclusion on the U.S. Department of Education’s list of schools under the strictest level of “Heightened Cash Monitoring” because of indications of low “financial responsibility.”
- Have a graduation rate at or above the median for its type of school (public or private), or if the rate is below the median, have a comparative value graduation rate in the top 25% of all schools.
This eliminated some colleges that may be good values, but might be facing temporary financial difficulties or be too small for us to evaluate. . But it left us with a robust universe of more than 700 colleges. In our view, even the lowest-ranked of the schools on our list demonstrate that they provide at least some value for your tuition dollars.
We then used the following data and methodologies in our three basic categories to create our rankings:
QUALITY OF EDUCATION: 33.3% weighting
For this factor, we used five indicators that provide meaningful information about the quality of a school’s instruction, weighted as shown:
(1) Graduation rates: 35%. Education experts and college officials generally agree that one of the most important reflections of a college’s quality is its graduation rate. (The U.S. Department of Education calls them “helpful indicators of institutional quality”.) Many rankings use this commonly cited federal statistic on the percentage of freshmen that graduate within six years. Because of its importance and wide acceptance, we assigned this measure a comparatively heavy weight.
(2) Value-added graduation rate: 35%. Many education experts and college officials point out that the basic graduation rate number, while useful, is an insufficient indicator of a college’s value because research shows that wealthier students and students who got good grades in high school are more likely to finish whatever college they attend. So elite, expensive schools such as, say, Harvard, would be expected to have high graduation rates. For that reason, we also calculated each school’s relative performance after accounting for the economic background and academic preparation of its students. The higher a school’s graduation rate was above the rate that would predicted for a school with that particular mix of students, the more value that particular college is assumed to have added. This “value-added” graduation rate analysis is widely accepted. (A 2013 OECD paper found that such “value- added measurement provides a ‘fairer’ estimate of the contribution educational institutions.”) Because of its reliability and acceptance, we weighed this factor heavily.
(3 & 4) Peer quality: 15%. Decades of research have shown that undergraduates have a major impact on their peers. Students who room with better students get better grades, for example. By contrast, students surrounded by less conscientious peers—for example, heavy drinkers, video game players, etc.—study less and get worse grades. And students who room or socialize with more successful students tend to get better jobs upon graduation.
Our peer quality measure consists of these two indicators:
- Academic preparation of students (10%). We gathered data on the admissions selectivity of each school, as well as the school’s reported average grade point average and test scores of incoming freshmen to estimate the academic qualifications of the student body. While there is controversy over the usefulness and validity of standardized tests, the SAT and ACT tests currently provide the only nationally comparable data on student abilities. In addition, many studies have found a high correlation between test scores and academic success.
- Yield (5%). The federally reported “yield” is the percentage of accepted students who enroll in a given college. The higher the yield, the more likely it is that the school was the student’s first choice, or best option, and that applicants have perceive the college’s quality as high.
(5 ) Faculty: 15%. Research shows that students who get more individual attention from faculty tend to achieve more, both in college and after graduation. (See, for example, the recent Gallup-Purdue study.) So we include each school’s student-to-faculty ratio.
Change from our 2015 rankings: For 2016, we removed the 5% of quality weight we had put on each school’s average grade on Ratemyprofessors.com. Research shows that the ratings of professors on the website track well with the internal ratings of professors given by students, but because we added several other powerful new measures (as you’ll see below) we wanted to trim some of the smaller measures.
AFFORDABILITY: 33.3% weighting
For this factor we used eight indicators, weighted as shown:
(1) Net Price of a degree: 35%. MONEY has developed a unique, and many experts tell us, more accurate estimate, of college prices. We started with the “sticker” price provided by the college to the federal government. The full cost of attendance includes tuition, fees, room, board, books, travel, and miscellaneous costs. For public colleges, we used the in-state tuition and fees.
We then subtracted the average amount of institutional aid provided per student by the college. That gave us an estimate of the net price the college charged an average student for the most recent academic year these data were available. (The data on merit and need-based grants you’ll see reported on our website are numbers reported by the colleges to Peterson’s, and are for your information but are not used in the rankings.)
Next, we used federal data on the percentage of students who graduate from that college in four, five, and six years to calculate an average time to degree, which for the vast majority of schools is now more than 4 years.
Judith Scott-Clayton of Columbia University, for example, has found that the average college graduate pays for 4.5 years of college, not just 4. However, because MONEY’s 2016 ranking includes only those schools with the best graduation rates, the average time to degree for all of the schools on our list is just 4.3 years.
We generated an estimated net price for a series of six years, inflating each slightly since tuition prices typically rise each year and then applying a small inflation adjustment. We then created a weighted average total net price, based on those prices and the proportion of students who complete their degrees within four years, in their fifth year, and in their sixth year.
College counselors and financial aid experts have told us that the MONEY calculation is more realistic and helpful to parents and students than other popular price estimates, most of which use the cost of a single year, not the full degree. Sandy Baum, a senior fellow at the Urban Institute and one of the nation’s leading researchers on college costs and aid, says our estimates of the net prices of individual institutions “provide good benchmarks.” She added an important caveat, however: “Students at each institution face a wide range of net prices, so no individual student should assume that the schools on this list with highest net prices would end up being most expensive for them.”
The federal net price estimate for a year’s costs is typically lower than MONEY’s estimate because the Department of Education subtracts almost all grants—federal, state and institutional—while we only subtract aid provided by the school, for reasons described below.
MONEY gives the net price sub-factor a very heavy weighting because surveys show that the cost of college is now one of families’ biggest worries. A 2014 Harvard Institute of Politics survey reported, for example, that 70% of young adults say finances were a major factor in their college decision.
(2 & 3) Educational debt: 20%. Surveys show debt to be another of most families’ biggest worries.
Our educational debt assessment is based on these two indicators:
- Student borrowing (15%). As reported by the federal government.
- Parent borrowing (5%). The federal government reports the total amount of parent PLUS loans awarded to parents at each college each year. We divided this number by the school’s enrollment to calculate an average parent PLUS debt per student. While other organizations generally don’t include parental debt in their rankings, MONEY believes parent educational borrowing is a financial burden, and should be an important consideration.
(4 – 7) Ability to repay: 30%. The ability to repay loans taken out to finance a college education is another indication of a school’s affordability for its students.
We evaluated ability to repay using these four indicators:
- Student loan default risk index (5%). Each year, the federal government publishes the number of former students who left college three years ago and have since defaulted on their federal student loans. Using a methodology proposed by The Institute for College Access and Success (TICAS), MONEY adjusts these numbers for the share of students at the college who take out federal student loans. TICAS says this is a fairer and more accurate indicator of the odds that any particular student at the college will end up defaulting on a student loan.
- Student loan default risk index comparative value (5%). We calculated the relationship between schools’ SLDRI and average student test scores and the percentage of the student body from low-income households. Schools that had a lower default rate than is typical given their student population were ranked more highly.
- Federal student loan repayment (10%). We used the new data released on the federal College Scorecard on the percentage of student borrowers who pay down at least $1 on their principal in their first five years of payments. Jordan Matsudaira, the Cornell economist who oversaw the development of the College Scorecard when he served as chairman of the White House Council of Economic Advisers, says repayment rates are better indicators of who can afford to repay their student loans, since some schools have started to game default rates by helping students to take advantage of hardship programs that delay defaults until after the three-year measuring window is closed.
- Student loan repayment comparative value (10%). We analyzed how much above or below the school’s repayment rate was compared to schools with similar student bodies.
(8) Affordability for low-income families (15%). The federal government reports the average price paid at each school by federal aid recipients who are in the lowest income group. MONEY uses this as an indicator of how well the college targets need-based financial aid. At some schools, those from the lowest-income group are expected to come up with more than $20,000 per year—which may be more than the entire annual income of the family.
Changes from our 2015 rankings: To make room for our new student loan repayment indicators, we reduced the weighting of our default indicator. We also shifted a total of 10% of the weight of the affordability factor from the borrowing indicators to the two net price indicators because the debt numbers are an incomplete indicator of how a family is paying for college; families who take out, say, a home equity line, or borrow against a retirement account are not counted as borrowers in the federal debt data.
OUTCOMES: 33.3% weighting
For our third category, we used a total of 11 indicators, weighted as shown below.
One data point we could not use was the percentage of students who find jobs within a year of graduation. Although many colleges claim that a high percentage of their new grads are employed, the data are generally not considered reliable enough to compare one college against another. That’s because each college has a different method of surveying graduates, and unemployed grads may be less likely to answer such surveys, which would tend to make a college’s employment rate look better than it really is.
(1 & 2) College career services: 15%. We used one quantitative and one qualitative indicator to assess the value of a college’s career services:
- Caseload of career services staffers (7.5%). Surveys show that the one of the most important reasons students now give for attending college is to increase their odds of landing a good job and launching a career. Unfortunately, most colleges don’t provide much career coaching or job assistance. At the typical four-year college, there is only one professional career services staffer for every 1,000 undergraduates, which equates to an average availability of just one hour of personal attention per undergraduate per year. While we couldn’t get reliable data on the quality of the career services that schools provide, we could at least tell parents how overworked the staff is. So MONEY included in its ranking the number of full-time equivalent staffers in each college’s career services office as reported to Peterson’s. (We called several hundred schools that hadn’t reported the data to Peterson’s to fill in the missing numbers.) We then calculated a caseload per staffer, based on each school’s enrollment.) The lower the caseload, the higher the college is ranked.
- Formal programs linking alumni with job-seeking students (7.5%). Personal referrals provide a huge advantage in the job market. (A 2010 Federal Reserve study found, for example, that job applicants who received a personal referral from an employee were more likely to get interviews, get hired, and be offered higher starting salaries. Colleges that help students make connections with their working alumni provide a valuable service.
(3, 4, 5, & 6) Post-graduation earning power: 35%. We used four indicators to assess this factor:
- 2011-12 earnings of freshmen federal aid recipients who started at the school in 2001, as reported to the IRS and on the federal College Scorecard. The federal government last year released data on the median earnings of Americans who started college in 2001-2. This is the most comprehensive national data on how much students are earning.
- Payscale.com earnings for students with less than five years’ work experience (10%). PayScale provided MONEY with aggregated data on more than 1.4 million people who in the last three years have filled out an online form that asks what their job is, what their salary is, where they went to college, what they majored in, and when they graduated. This adds additional important information to the new federal data because it accounts for only students who earned a degree from the school, not the dropouts, and screens out those who have earned graduate degrees, and thus may be earning more because of their graduate, rather than undergraduate, education. While the two data sets cover different populations, more than 80 % of the schools that have high earnings on one of the measures also has high earnings on the other.
- Payscale.com earnings for mid-career workers (5%). We used the average earnings reported by those who do not have graduate degrees and have at least 10 years’ work experience. The typical worker in this group has 15 years of work experience. We weighted early earnings more heavily than mid-career earnings because Trey Miller, a RAND Corporation economist who has studied the relationship between college choice and earnings, noted that a college choice has a much stronger impact on the type and pay of a first job after graduation than it does on job type and pay for a person who has been in the workforce for, say, 10 years. By then, experience and skills acquired since graduation will also play an important role.
- Estimated market value of alumni skills (10%). In April 2015, the Brookings Institution published an analysis of new data shedding light on the value added by each college. One of Brookings’ indicators was an estimate of the market value of the 25 most commonly cited skills listed by alumni of each college in their LinkedIn profiles. Jonathan Rothwell, the author of the Brookings study, said this new measure is based on millions of LinkedIn profiles and is a new and potentially better way to discover earning potential. In addition, the data is for all graduates, including those who have earned additional degrees, so it captures the earnings of some schools’ higher earners.
(7) Job meaning: 20%. Our 2016 MONEY/Barnes & Noble College survey found that parents and students alike put a higher emphasis on finding a “fulfilling” career, than on finding a high-paying job. So we added to this year’s rankings, alumni’s response to a PayScale.com survey question about whether their job “makes the world a better place.”
( 8, 9, 10, & 11) Earnings comparative value: 30%. We used four numbers to calculate how much more or less the graduates of each school are earning, compared with graduates of similar colleges.
- Early-career PayScale earnings after adjusting for majors (10%). In other analyses of which colleges seem to produce the highest earners, engineering and tech-oriented colleges dominate because computer scientists and engineers tend to earn very high salaries. But what if your child isn’t fated to be an engineer? Which school produces the highest earners for other majors? MONEY used a regression analysis to estimate the average impact on reported earnings of each of three “buckets” of majors: Business, STEM (science, technology, engineering, and math) and “everything else,” which is mostly made up of humanities, education, visual and performing arts, and behavioral and social sciences. We calculated the average earnings for each group of majors. Then, using federal data on the number of graduates in each major at a college, we calculated what the expected earnings would be for that school if each student earned an average salary for his or her field of study. Colleges where graduates earn more than the predicted salary are ranked more highly.
- Mid-career PayScale earnings after adjusting for majors (5%). We weighed mid-career earnings less heavily because, as previously noted, where a person went to school generally has a greater impact on salary earlier in a career.
- Comparative value early-career PayScale earnings (10%). We also conducted the same kind of ”comparative value“ analysis we did in the educational quality and affordability categories. To find colleges that accomplish what education is supposed to do—help hardworking students from any background get ahead—we calculated the impact of test scores and of coming from a low-income family on graduates’ earnings.
- Comparative value mid-career PayScale earnings (5%).
Changes from our 2015 rankings: To make room for the new indicators, we reduced the weighting we had given to various PayScale measures. We also this year evenly divided the career services weight between our two measures, rather than overweighting staffing, to reflect the equal importance we give each factor.
How we calculated these rankings: For each of our data points, for each college, we calculated a “Z-score”—a statistical technique that turns lots of seemingly different kinds of data into a standard set of numbers that are easily comparable. Each Z-score tells you how far any particular number—such as, say, a graduation rate of 75% —is from the average for the entire group under consideration. To reward colleges that demonstrate generalized excellence (and dampen the effect of a school’s outperformance on any one particular measure) we limited the range of the Z-scores to three standard deviations. We then rescaled each of the three main factors so that they had the same ranges. Finally, we ranked the schools according to their total score on our three main factors.
While we believe our rankings are better than any others out there, we know they are not perfect. Here are some caveats that we are aware of and hope to address in future rankings.
Student learning. What students actually learn is a big unanswered question in higher education. Very few colleges try to measure learning, and very few of the ones that do release the results publicly. In addition, we were not able to find good data on basic indicators of academic rigor, such as the number of pages of writing or reading required per assignment. We will continue to explore the data in hopes of finding useful indicators of student learning.
Geographical cost of living adjustments. Some colleges in low-wage, low-living cost areas, such as parts of the South and Midwest, may get lower rankings because we have not adjusted the earnings data cost of living. But several factors added to the MONEY rankings in 2015 and 2016 dramatically lessen the impact of any geographic wage differential. The Brookings Institution skills value measure, for example, is based on national averages, so treats all schools equally. The new student loan repayment data shows which alumni have sufficient earnings to repay their loans, in theory raising the disposable income of students in low cost areas. In fact, our data show no variation in repayment rates by urban density or regional location. Finally, we shifted weight from the PayScale earnings numbers to the PayScale “job meaning” numbers. And those show no regional variation.
Alumni satisfaction. The information that’s currently available on alumni satisfaction—based on surveys and donation rates—is incomplete for many of the colleges on our list, so we were unable to include it as a measure. We are looking for ways to improve the alumni data and make it part of future rankings.
Out-of-state-public college tuition. Many students are interested in attending public colleges out of state. But public colleges charge higher tuition to out-of-state students. We will consider developing a cost and value comparison for out-of-state students.
Net prices. MONEY’s estimated net price is likely to be higher than the average price actually paid by most families. It is crucial to understand that while the MONEY net price estimate is based on the average price charged by the college, you and your family will pay less than that if your student receives any federal, state, or private scholarships. As an analogy, if you’re buying a can of soup, you have to pay what the grocery store charges, unless you have a coupon. Just as coupons can be used at competing supermarkets, most federal, state, and private scholarships can be used at many competing colleges. So we help you identify which college has the lowest net price at which you can apply any additional scholarships. In addition, our net price is based on the average student’s time-to-degree. Your student may finish in four years. And while many students take more than four years to finish a degree, they aren’t necessarily paying full tuition for the five or six years before they graduate, since they may, for example, take a year off to work. MONEY attempted to account for this by adjusting the estimated time to degree for all schools with large and established co-op work programs, such as Northeastern University. In addition, MONEY is not adding to the cost of a degree any amount for “opportunity cost,” which is the amount in earnings a student loses by not finishing a degree on time and delaying entry into the higher-paying job market of college graduates. So, while we may, in some cases, be overestimating the price of a degree, we are also underestimating the total economic expense to a student of schools that don’t speed them to graduation.
ACKNOWLEDGEMENTS: Matthew Soldner, chief researcher, and Cameron Smither, researcher, at the American Institutes for Research (AIR), did the bulk of our data collection and analysis.
For more of MONEY’s Best Colleges 2016-2017 coverage, check out: