College Rankings: Looking Beyond the Numbers
In part three of our continuing series, we’ll further explore the different evaluation systems of Colleges of Distinction and U.S. News & World Report. If you missed part one or two, check them out to learn about the difference between qualitative and quantitative evaluation systems for choosing a college.
This entry is a critical look at quantitative college rankings, in which we examine known issues with individual statistics and number manipulation. We’ll also explain how the overall scoring system is negatively affecting colleges.
“There are three kinds of lies: lies, damned lies, and statistics” – Mark Twain (quoting Benjamin Disraeli)
U.S. News & World Report puts out a quantitative school ranking each year. And each year, those ranking are treated as the only way to know what college is the best college. When the statistics, math, and logic used in the U.S. News ranking is assessed, many holes and problems appear.
More Than Just Numbers
One main issue with the U.S. News ranking system is that numbers aren’t everything. Numbers and statistics can be used to quantify class size and acceptance rate, but not student experience, internships, networking opportunities, or friendships.
The limitations of quantifiable data distort which schools truly are “the best” schools. Another major issue with quantitative ranking is purposeful or accidental number tampering. Either way, ranking may not reflect reality. As a result, some types of school are underrepresented on the U.S. News rankings.
With all this in mind, we invite prospective students, parents, and colleges to join us as we question the assumptions that lead many to hold the U.S. News & World Report ranking system in high regard.
Individual Statistics Manipulation
Peer Assessment Survey (22.5% of total score, 15% for national schools)
For U.S. News’ rankings, school reputation is measured based on the average score administrators of other schools award to them. But if the goal is to always move up in the ranking, what’s the incentive to honestly or highly rank your peers?
As there is minimal incentive, schools likely rank their peers with a low score. As well, it’s unlikely that school administrators can intimately know the quality or lack of quality at other schools—let alone be objective about it, due to their employment. So not only is there the chance for intentional manipulation, but the measurement system has unintentional biases.
Student Selectivity (12.5% of total score)
Another part of a school’s prestige is their level of selectivity. For U.S. News, this is a measurement of student ambitions and ability based on admissions test scores, percentage of freshmen graduating at the top of their class, and the ratio of admitted students to applicants.
It appears the intent of the measurement is to illustrate the character of the student body. Indeed, many schools like to flaunt such numbers to attract out-of-state students. But once again, the type of statistics measured make it easy for schools to inflate the numbers in their favor, distorting readers’ perceptions of them.
For example, Clemson University did it by:
- Accepting fewer students from their applicant pool.
- Advertising to increase the size of their applicant pool.
Since U.S. News is only measuring ratios and percentages, tactics like these can be easily employed to inflate the appearance of selectivity. In this way, the supposed quality of student community is easily undermined. In the end, this superficial understanding of the measurement and its underlying assumptions does more to deceive than inform.
Class Size (8% of total score)
In a piece for The New Yorker criticizing the U.S. News ranking system, acclaimed author Malcolm Gladwell writes:
“According to educational researchers, arguably the most important variable in a successful college education is a vague but crucial concept called student “engagement”—that is, the extent to which students immerse themselves in the intellectual and social life of their college—and a major component of engagement is the quality of a student’s contacts with faculty.”
He goes on to explain how some colleges will shrink their class sizes during the fall semester, when the numbers are recorded, to manipulate the rankings. Then, in the spring, the class sizes jump back up.
Schools do this by shifting spots to classes that are already large. For example, a class that may have had 24 people may be changed to 19, and the class that had 100 spots would shift to 105. Class size is an important indicator for student engagement and success. When that is manipulated simply for a better rank on the U.S. News site, the point is missed. Small classes should be used to better teach and engage with students, not raise prestige.
Percentage of Faculty with Terminal Degree (3% of total score)
Here is a case where the framing of a statistic appears to be useful, but is actually deceptive upon further investigation. U.S. News attributes 15% of their faculty resources score to the school’s percentage of faculty with a terminal degree (i.e. the highest degree attainable in one’s field of study). On the surface, this seems like an important indicator for the quality of education one might receive at a given school. After all, you’d probably prefer a physics professor with a doctorate to a professor without.
But consider this: not every terminal degree offers inherent incentives for earning it. Therefore, terminal degrees are not always the best measure of expertise and quality teaching.
Let’s take the example of journalism. It’s easy to imagine a journalist earning their master’s degree with a focus on journalism to rise in the ranks in their career. Now imagine how many career paths there would be for someone who completed their doctoral degree in journalism. For career purposes, a master’s degree would be sufficient—and a doctorate might be overkill.
With that in mind, who is really going to be the better teacher? The teacher with no working experience and a doctoral degree, or the former career journalist with a master’s degree and years of experience on the job?
When all’s said and done, we at Colleges of Distinction are more concerned with the following question: What benefits do students gain by learning from a teacher with a terminal degree versus a teacher with experience, but no terminal degree? In recognizing and respecting the context of this issue, we ensure our readers that we are focusing on what matters: great teaching and engaged students, not just fancy titles.
Cheating/Gaming the System
As we’ve alluded to, the U.S. News college ranking system is significantly vulnerable to manipulation. In fact, the sections we just covered amount to 46% (38.5% for national schools) of the total score! Furthermore, there are schools that have admitted to outright cheating and number inflation to raise their rank on the U.S. News site. For more on that, you can look up the scandals at Claremont-Mckenna College, Emory University, George Washington University, and many more.
Without a doubt, your college education is one of the largest investments of time and money you’ll make in your lifetime. Common sense demands that such an important decision be based on accurate information, not inflated statistics.
The Important Statistics That Aren’t Represented
Possibly more troubling than anything else we’ve mentioned, is that key elements of the higher education experience aren’t represented in U.S. News’ statistics and rankings. Some of these elements would be difficult to rank with numbers as they’re more qualitative, but that’s exactly why we at Colleges of Distinction find them so valuable.
Outcomes
U.S. News’ methods focus solely on what happens before college (prospective student test scores), student performance in the first year (retention rate), and how many students graduate (graduation rate). Unfortunately, this completely overlooks what happens to students after graduation. Are they able to find jobs? Are they participating and contributing members of the community?
At Colleges of Distinction, we care about what happens to students after graduation. We want to make sure that graduates end up in rewarding jobs and as invested members of their communities. That’s why we focus on the internship programs and alumni networks our schools implement to ensure successful outcomes.
These points aren’t addressed in the U.S. News ranking, leaving prospective students to wonder what happens to graduates from these schools.
Student Experience
Another aspect of student life that isn’t represented on the U.S News ranking system is that of the student experience. There’s acceptance and graduation, but nothing in between except class size and student selectivity.
But there’s so much more to the student experience than just the size of the classes! What should be represented are extracurricular activities, student groups, study groups, the number of students living on campus, campus events, and study abroad opportunities.
Colleges of Distinction’s unique evaluation system highlights these High-Impact Practices in higher education. As a result, our readers get a far more detailed picture of day-to-day campus life than they would elsewhere.
Failures of The Overall Ranking System
Ranking System Statistics
U.S. News changes the weight of statistics used in their ranking system every year or every other year. Supposedly, these changes are meant to make the ranking system more accurate, but the schools ranked at the top rarely change. When the schools at the top do change, it’s only by a rank or two.
Additionally, the regular changes present another systemic problem: They leave us with no way to compare schools and their ranking from year to year, making the meaning of any changes in rank vague at best and dubious at worst.
With subtle changes from year to year, U.S. News can easily keep prospective students and parents buying their guide every year. This marketing ploy guarantees that their guide is always fresh. It creates it’s own demand.
Pushed to Participate
There have also been rumors that if schools choose not to participate, their scores are reduced or manipulated to look worse. This, for many schools, is motivation enough to continue participating. Some, though, have been brave enough to opt out.
Problems with Statistics
In his piece for The New Yorker, Malcolm Gladwell beautifully illustrates this problem with an example from Car and Driver. If you use the same metrics to rank a minivan and a sports car, is it truly a fair ranking? The cars are so widely different in purpose and design, it’s difficult to fairly compare them.
This holds true when comparing colleges and universities. Is it even fair to compare an engineering- and technology-focused school with a small liberal arts college? UC Berkeley is widely regarded as a great example of this at play. Just like standardized tests don’t fully represent each student’s ability and knowledge, the U.S. News ranking can’t fully represent the best schools.
A Faulty System
We are not the first to write about the problems facing the U.S. News’ ranking system, and we will not be the last. With such diversity, from public to private or from liberal arts to engineering and technology, you can’t capture greatness at every level with one ranking system. In recent years, there have been numerous articles written to explore the faults with the U.S. News system.
That being said, there are faults in any system. We believe that our qualitative evaluation system really takes into account the individuality of each college as well as the uniqueness of the student seeking education. The right fit is out there for each person. The right fit for that student is where they will learn to the best of their ability, and graduate with new skills to thrive in work and life, and become a contributing member of society.
Quantitative ranking systems can give schools a false sense of prestige and competitiveness. This leads to number tampering, whether it be accidental or on purpose. Why then do schools continue to seek the prestige of being highly ranked by the U.S. News? There are many outside pressures at play, which is why we are firm in our belief that rankings are not everything. There are so many great unranked schools where we truly believe that students will receive an education of equal, if not better, quality than some of the highest ranked schools. We hope you’ll join us in our mission to shift the focus from rankings to the outcome and student experience.
If you’re interested in learning more about Colleges of Distinction’s qualitative system for evaluating schools, check out which schools make our list. Next week, we’ll help you figure out which school might be right for you. If you have any questions about our qualitative evaluation of schools or our evaluation of U.S. News ranking system, leave them in the comments.
