Recently Inside Higher Ed had an article about a study (PDF here) coming out of the University of California on the predictive power of the SAT with respect to grades in college courses. The study, by Saul Geiser and Maria Veronica Santelices at the UC-Berkeley Center for Studies in Higher Education, followed the performance (which is to say, grades) of students at all UC campuses for four years and found that “high school grades are consistently the strongest predictor of any factor of success through four years in college”. Indeed, the study found high school grades a stronger predictor of grades past the first year.
The SAT turned out to be not such a great predictor of college grades, but a dandy predictor of socioeconomic status of the kids taking the SAT.
What interests me about this story is not the perennial debate about whether the SAT tells admissions officers anything useful about applicants, but the larger question of what admissions officers — and folks in society at large — imagine should be the guiding principle in deciding who ought to win the competition for scarce spaces in a college class.
From the Inside Higher Ed article:
Geiser, one of the two authors of the study, noted in an interview that defenders of standardized testing always like to say that it is needed to compensate for the fact that high schools have widely varying quality. But what the researchers found is that there isn’t such a problem — even in a state as large and diverse as California. “How you perform in college prep biology is a justifiable and appropriate way” to decide whom to admit, Geiser said.
While Geiser said that the results clearly point to the need to “emphasize” grades and to “de-emphasize” the SAT (a direction in which the University of California has moved), he stopped short of saying that the findings suggest that universities should abandon the SAT. He said he did not want to be drawn into that debate.
Rather, he said he hoped people would consider the meaning of the finding that only 30 percent of the grade variance in college could be explained by the factors admissions officers examine. If so much of the grade variance can’t be explained, Geiser said, that raises a tough question: “Why are we emphasizing prediction [of college success] as the central value in admissions if we do it so poorly?”
(Bold emphasis added.)
Geiser seems to take for granted that the goal of the admissions officer is to accept the applicants most likely to score the highest grades in college courses. Thinking back to when I was in high school, I suppose we (and our parents and teachers) we operating under an assumption that was at least in the same ballpark — namely, that we had to find ways to make our grades, test scores, letters of recommendation, and such speak to how smart (and hardworking) we were.
Of course, for the high school student, this is tied with seeing college acceptances (or rejections) as part of some big meritocracy that was set up to reward us for being awesome and having worked our butts off. It was really hard for the typical 17-year-old to wrap her head around the idea that admissions officers not only couldn’t arrive at some perfectly accurate objective measure of our awesomeness on the basis of grades, SAT, rec letters, essays, etc., but also that even if such an objective ranking of all the applicants was possible they had an interest in building a class, not just rewarding us for a well-played high school career.
Anyway, it seems sensible that colleges probably don’t want to admit classes full of people who there is good reason to believe will not be able to handle college work. But I wonder whether there’s an argument for admitting the students who are most likely to benefit from a college education — even if these are not precisely the students most likely to earn the highest grades in college.*
Frankly, there are a lot of kids who haven’t really worked out the right balance of effort, motivation, and study skills during the stretch of high school where they’re earning the grades admissions officers care about. (I’m told, here in California, that sophomore and junior year grades are the ones that count for college admissions within the UC and CSU systems — so if you have a rough sophomore year, you can end up with no reasonable chance of admission to any of those schools.) Some of the kids who are getting it together by senior year could, arguably, benefit hugely from a college education. It seems wasteful to write them off because of bad timing.
And the challenge, really, is trying to make good predictions about future performance on the basis of past performance. This isn’t surprising given how many factors can play a role in how a person does in a class: What’s her prior preparation like? How well does the presentation of the course interact with her learning style? Is she interested and motivated? How’s her time management? What’s going on with her family and personal life? Is she getting enough sleep? Healthy? Happy?
By the way, do we have data on whether college grades are an especially good predictor of anything else (like getting a good job, being a useful member of your community, or being happy and fulfilled)? Should that question have any impact on how we think about what considerations should drive college admissions decisions?
I don’t really have a settled view on how admissions decisions ought to be made, so if you do, I’d be thrilled if you shared it with us.
______
*Also, I have this worry about the logic of the claim that good high school grades are predictive of good college grades in the case where a college decided to make admissions decisions solely on the basis of the high school grades. In the (wildly oversimplified) case where a high school A-average predicts a college A-average, if a class was admitted that consisted just of A-students in high school, are they all going to get As in college? Will we get sucked into another vortex of soul-searching about grade inflation? Or, will this create a boundary condition where the high school grades fail to have predictive power?
A primary goal of state run university admissions should be to admit those students that would most benefit society by getting that education. (so, for example, I don’t think the students ultimate happiness and fulfilment should be, separately, a legitimate factor)
A criticism of a similar study of Harvard students many years ago was that GPA alone tells you nothing about how students with different SAT scores faired head to head in the same courses. And this study has the same weakness, as it uses only cumulative GPA as it measure of performance.
A further criticism of the Harvard study was that they were (duh!) comparing students that had SAT scores that got them into Harvard, so the range of scores was very narrow. This study is a bit better since it evaluated students at all the UC campuses, which overall means a wider spread of scores, but on the other hand these still are the top CA schools, and grades were earned on the campus they were admitted to.
Not surprisingly, the results of their analysis are the same (HS GPA is a better predictor of college GPA than SAT scores), and they seem to come to the same conclusion, that the SAT is, at best, only marginally useful as an admission criteria.
The alternative hypothesis floated in response to the Harvard study was that after lessons learned the first year, students might then have chosen classes that were more suited to their abilities for the remainder of their college career. This was coupled with the further criticism that the apparent utility of the SAT was artificially deflated by chosing a population of students that had a limited range (they all had high scores!) and were selected in part *because* of those high scores. Neither study was comparing the pool of *all* US HS grads in a given year (e.g., the 4.0 student with 500’s on the SAT from the rural HS in Mississippi against the 4.0 700+ student from the New England prep school).
I know that a high HS GPA is a sign of positive personal attributes and intangibles that are generally applicable (and valuable) outside the classroom. And I have no doubt that, all other things being equal, those with high HS GPAs will get higher college GPAs. But of course, all other things are never the same. The question is, how much of a lower GPA will you accept and still select that 700+ SAT math student over the 4.0 GPA 500 SAT math student for a slot in your math program? (Assuming, of course, that your math department would even consider someone who only got a 500 SAT math score).
At this point, I think it’s fairly clear that as a society, we have absolutely no clue what we need or what would help us, so trying to make a judgment about “who would best benefit society” is not only prohibitively difficult but pointless.
Standards that can’t be described succinctly or operationalized in a clear and objective way tend to encourage bias on both an individual and an institutional level. “Who would best benefit society” is sounds good, but only because it’s composed of high-level concepts that don’t actually specify anything about the standards that would be involved.
I think a clarification of the purpose of institutions of higher learning would be helpful in determining what admissions standards should be.
For a couple of years, we were able to select 20% of our first-year students (computing program) on the basis of interviews. Of course, since this was so successful, it has been banned on account of not being “objective” and we are back to grades-or-waiting-time, as is usual in German.
We selected quite a variety of people: ones who failed math, ones who traveled around the world, ones who learned a trade, ones who spoke French as a mother tongue. I actually came up with a formula for the most successful class (40 +/4 people) – and we pretty well hit it before interviewing was banned. It was a fantastic group a people!
* Half men, half women
* One third should have learned a trade and worked between high school and college.
* The trades should be varied, i.e. not all computer related.
* At least one artist in the crowd.
* At least one business-oriented person.
* A party organizer
* At least a quarter should be in their late 20s, but there should be at least one “Mom” or “Dad”, i.e. old enough to be parents to the youngest ones.
* A few fresh-out-of-school-never-worked-a-job-in-my-life-before (but not too many, they tend to focus on partying and not on studying).
This mix of people works real well – the older people, back to school for a degree, will balance the young ones. The people who have worked have a good idea about what Real Life ™ looks like. The men will learn that there are competent women, and the women will learn that not all men in computing are sloppy nerds.
But no one made me dictator at the school, so we continue on grades-only. …..