The Nitty Gritty: Quantitative vs. Qualitative College Evaluation Methods

This blog series compares and contrasts our qualitative college evaluation method used at Colleges of Distinction with U.S. News & World Report’s quantitative ranking method. In our first article, we identified the qualifications that schools must meet to become a College of Distinction, and compared them to the statistics that are necessary to be highly ranked on U.S. News & World Report’s list of great colleges.

Here, in part two, we’ll begin to explore the nitty gritty of these evaluation systems: their strengths and their weaknesses.

Colleges of Distinction evaluates colleges based on four categories:

• great teaching
• successful outcomes
• engaged students
• vibrant communities

U.S. News has seven main ranking categories:

• graduation and retention rates
• undergraduate academic reputation
• student selectivity for the previous year
• faculty resources
• financial resources
• alumni giving
• graduation rate performance

Both Colleges of Distinction and U.S. News have subsections of their main categories.

Great Teaching vs. Faculty Resources

Let’s start with the differences in evaluating teaching quality.

For a school to become a College of Distinction, it must have great teaching. This means classes are small and personal, teachers are innovative, and students are encouraged to think and learn critically and independently.

In comparison, U.S. News judges the quality of teaching on faculty resources. They weigh it quite highly, too. Faculty resources account for 20% of a school’s total score. Comprising this score are five subsections: faculty compensation, percent of faculty who have their terminal degree, percent of faculty that is full time, student to faculty ratio, and class size.

There are a lot of similarities between U.S. News and Colleges of Distinction. Both value small classes, low student-to-faculty ratios, and teachers who have plenty of time to focus on students (full time teachers). For both, the desired teaching environment focuses on learning, not a professor’s research.

Successful Outcomes vs. Graduation and Retention Rates

Colleges of Distinction defines Successful Outcomes as what happens in a student’s life after graduation. This can be much more than finding a job out of college. It encompasses the idea that a college education extends beyond their major. Life skills, workplace readiness, professional adaptability, and good citizenship all factor into our definition of success.

There are three sections from U.S. News that can compare with College of Distinction’s Successful Outcomes category: graduation rates, retention rates, and graduation rate performance. Together, those sections account for 30% of the total.

Retention rate is defined as the percentage of students who return to the same college or university the following fall after their first year. Graduation rate is the percentage of first-year students who graduate in six-years or less. Finally, graduation rate performance compares the actual graduation rate with the predicted graduation rate to make sure the school isn’t underperforming.

Graduating from college is only the first step in success as an adult. Just because a high percentage of students graduate from a college, doesn’t necessarily mean those same students will find and excel at meaningful jobs. It also doesn’t mean that a college’s graduates are contributing community members with the ability to adapt to a changing world.

That’s why we find it more useful to look at things like internship programs, capstone projects, and collaborative learning opportunities. These are the sorts of school features that ensure real life success—not just the attainment of a degree.

Engaged Students vs. Student Selectivity

For Colleges of Distinction, an engaged student body is full of active learners gaining new knowledge and discussing it, instead of memorizers who simply regurgitate facts. Colleges with such students encourage learning through things like personalized course plans, undergraduate research, capstones, and study abroad programs. This gives students a wide range of opportunities to learn a broad range of skills.

U.S. News evaluates student populations in a fairly different way. Making up 12.5% of the total evaluation score, their measurements focus on what they call: “student selectivity.” This consists of three different subsections: acceptance rate, high school class standing, and scores from the  critical reading and math portions of standardized college entry exams (SAT and ACT). This type of ranking can give a basic view of students’ ability to score well on tests and perform well academically in high school.

More importantly colleges’ student selectivity statistic fails to effectively describe the student experience on campus. Colleges of Distinction looks at more varied qualities to assess student engagement, such as how students interact on campus, the opportunities afforded to them, and the encouragement they receive from faculty members. Engaged Students stand out by representing skills such as group collaboration, critical thought, adaptability in diverse populations, and excellent written and verbal communication.

Vibrant Communities: Exclusive to Colleges of Distinction

Among Colleges of Distinction’s evaluation categories, Vibrant Communities most thoroughly explores the student experience on campus. As Colleges of Distinction defines it, a campus with a vibrant community doesn’t shut down after classes. It offers students opportunities other than classes, such as active student life activities, guest lectures, leadership organizations, networking events, sports events, and internships.

Within the U.S. News ranking methodologies, there is nothing that can compare to Colleges of Distinction’s Vibrant Communities section. U.S. News focuses only on what happened before and right after college. They look at what students are accepted and how many graduate, but nothing during the four years that students are on campus nor what happens to the students after graduation.

Overall, the biggest, most concerning difference between the two evaluation systems we’ve explored are that U.S. News looks only at the numbers that are supposed to represent the experience, but not the actual experience itself. When applying to college, it’s easy to forget that getting into college isn’t the end-all be-all. Applying to college is just the beginning of your college journey.

College is a stepping stone for learning skills that will help you to live a successful life, have a rewarding job, and make an impact in the community. When choosing a college, it’s important to look at the entire picture—to look at what the four (or more) years that are spent on campus living and learning will be like. Many ranking systems, including the one created by U.S. News & World Report, focus only on the before and after and nothing in between.

A student’s education is more than just going to class, it’s the people they meet, the experiences they have, the networking the do, how much of the world they get to see, what they accomplish in extracurricular activities, and what their experience of the working world is during internships.

The next article in this series will investigate some of the problems that arise when evaluating a school solely by the numbers. Stay tuned! What do you think about these different methods for evaluating colleges? If you ever have any questions, we’re here to help.