April 6, 2001Recently, you received yet another letter from Professor Emeritus Lee Hansen, dated March 26, 2001. In this letter, and in all his previous letters, op-ed pieces, letters to the editor, and public utterances on the subject of UW-Madison Freshman admissions, Professor Hansen relies on at least two misapplications (errors) of statistical reasoning and one major erroneous assumption. They are as follows:
- Professor Hansen analyzes and compares various averages and percentages for majority and minority subsets of our applicants, admitted students, and enrolled students. For his comparisons to be valid, these subsets would have to be “the same in all other relevant respects” (other than minority status). In statistical terms, they would have to be random samples of their respective populations. They are not, so his comparisons have no face validity.
- Professor Hansen assumes that the well-known statistical correlation between (any of, or any combination of) those numerical indicators and first-year grades in college can be applied to individual applicants. That is another bit of fallacious reasoning. What is statistically and accurately true for a large group is not necessarily true for individual members of that group. In fact, counterexamples are easy to find. Some of us (you and I) may well be counterexamples — people who did much better or much worse in college than our entering numerical indicators would have predicted. We do not admit groups. We admit individuals.
- Professor Hansen assumes implicitly (sometimes asserts explicitly) that “admissibility” or “qualification for admission” can and should be judged primarily, if not solely, on some combination of high school grades, high school class rank, and standardized test scores. This is either an incorrect assumption or a mistaken belief.
I will amplify (briefly) on each of these in turn, but I don’t propose to consume any more of your or my time on this subject than absolutely necessary. In 1996, I wrote a 20-page paper for the Regents in connection with a briefing on our admissions policies. I have located the original text on an archive disk, and could bring it up to date for you if you were interested in “more than you ever wanted to know about admissions.” Otherwise, I will probably just leave it with this brief rebuttal.
Point #1 is the most glaring fallacy in Professor Hansen’s analysis. Many, if not most, of the minority applicants whose high school ranks are below (say) the 80th percentile have been encouraged to apply based on personal interviews, guidance counselor advice, or other information that makes us believe they can do well here despite their class rank. Thus, they are not a random cross-section of the “low percentile” minority population; they are a prescreened subset of that population. Because we are actively recruiting to improve our diversity, low-scoring minority applicants are more likely to have been pre-screened than are majority applicants in those same percentile bins. It is neither surprising nor is it evidence of “unfair discrimination” that (in this particular semester — the numbers are small, in any case, and fluctuate greatly) we admitted 37% of the minority applicants who are below the 50th percentile, but only 6% of the majority applicants from those ranges. All of the “bottom half” admissions are in the Regent-defined “exception” categories, and the number of majority exceptions usually greatly exceeds the number of minority exceptions.
To points #2 and 3: Regent policy states that we are to admit preferentially (but not exclusively) those applicants deemed “most likely to succeed.” Common sense and Regent policy also dictate that we should pay attention to past academic performance as a predictor of future academic success. So we do. But if we relied solely on numerical indicators, we would make lots more admissions mistakes than we do now. These indicators are simply not as powerful as many people (including Professor Hansen) seem to believe. At best, they enable us to “explain” (predict accurately) only about 25% of the observed variance in the first-year grades of admitted students, and less than 10% of the variances in graduation. Since Professor Hansen has focused on class rank bins, I’ll point out that the “average” entering student who was in the 20th percentile range is predicted to earn a first-year GPA of about 2.5 and the “average” entering student who was in the 80th percentile range should earn about 3.1. Overall, each one percentile increase in class rank “predicts” an increase of about 0.01GPA points in first-year grades. That’s very accurately true for a group of 1000 students in any of these bins. But the standard deviation in those predictions is about 0.5GPA points, and the GPAs of individuals in each rank bin (even the top 10% bin!) will range from 0.0 to 4.0. If all I know about an individual is that her class rank is at the 20th percentile, I can say that the most probable first-year GPA for someone we admit at this level is about 2.5, but that her actual GPA will be somewhere between 0.0 and 4.0. That’s why we look at lots of other things in addition to class rank — things that we hope and believe are relevant to the other 75% of the observed variance in grades.
I also find it interesting that Professor Hansen does not point out that his own table shows we admitted 98% of the majority applicants who were in the top ten percent, but only 93% of the minority applicants in that bin. I wonder how he explains that? It seems to me that if we were practicing (in his usual terminology — a term I find both offensive and intentionally misleading) “race preference,” and if we were admitting based solely on numerical indicators, we would be admitting 100% of the high-scoring minority applicants. I haven’t checked his numbers with original data, but we do turn down both majority and minority applicants who have high grades, ranks, or test scores when our overall review of the application leads us to conclude they would not be likely to succeed here.
Finally, I can tell you that more than two thirds of the students who fail to graduate are in good academic standing when they drop out. That means they have dropped out for nonacademic reasons (financial reasons, health, discouragement over the campus climate, etc.), and minority students experience more of these challenges than do majority students. All the students we admit are, to the very best of our ability to judge, capable of succeeding academically, and most of them do. We have the highest graduation rate in the System, and are far above the national average for both majority and minority students.
— John Wiley
Letter to the Board of Regents from Chancellor Wiley Rebutting Hansen Letter
April 6, 2001Recently, you received yet another letter from Professor Emeritus Lee Hansen, dated March 26, 2001. In this letter, and in all his previous letters, op-ed pieces, letters to the editor, and public utterances on the subject of UW-Madison Freshman admissions, Professor Hansen relies on at least two misapplications (errors) of statistical reasoning and one major erroneous assumption. They are as follows:
I will amplify (briefly) on each of these in turn, but I don’t propose to consume any more of your or my time on this subject than absolutely necessary. In 1996, I wrote a 20-page paper for the Regents in connection with a briefing on our admissions policies. I have located the original text on an archive disk, and could bring it up to date for you if you were interested in “more than you ever wanted to know about admissions.” Otherwise, I will probably just leave it with this brief rebuttal.
Point #1 is the most glaring fallacy in Professor Hansen’s analysis. Many, if not most, of the minority applicants whose high school ranks are below (say) the 80th percentile have been encouraged to apply based on personal interviews, guidance counselor advice, or other information that makes us believe they can do well here despite their class rank. Thus, they are not a random cross-section of the “low percentile” minority population; they are a prescreened subset of that population. Because we are actively recruiting to improve our diversity, low-scoring minority applicants are more likely to have been pre-screened than are majority applicants in those same percentile bins. It is neither surprising nor is it evidence of “unfair discrimination” that (in this particular semester — the numbers are small, in any case, and fluctuate greatly) we admitted 37% of the minority applicants who are below the 50th percentile, but only 6% of the majority applicants from those ranges. All of the “bottom half” admissions are in the Regent-defined “exception” categories, and the number of majority exceptions usually greatly exceeds the number of minority exceptions.
To points #2 and 3: Regent policy states that we are to admit preferentially (but not exclusively) those applicants deemed “most likely to succeed.” Common sense and Regent policy also dictate that we should pay attention to past academic performance as a predictor of future academic success. So we do. But if we relied solely on numerical indicators, we would make lots more admissions mistakes than we do now. These indicators are simply not as powerful as many people (including Professor Hansen) seem to believe. At best, they enable us to “explain” (predict accurately) only about 25% of the observed variance in the first-year grades of admitted students, and less than 10% of the variances in graduation. Since Professor Hansen has focused on class rank bins, I’ll point out that the “average” entering student who was in the 20th percentile range is predicted to earn a first-year GPA of about 2.5 and the “average” entering student who was in the 80th percentile range should earn about 3.1. Overall, each one percentile increase in class rank “predicts” an increase of about 0.01GPA points in first-year grades. That’s very accurately true for a group of 1000 students in any of these bins. But the standard deviation in those predictions is about 0.5GPA points, and the GPAs of individuals in each rank bin (even the top 10% bin!) will range from 0.0 to 4.0. If all I know about an individual is that her class rank is at the 20th percentile, I can say that the most probable first-year GPA for someone we admit at this level is about 2.5, but that her actual GPA will be somewhere between 0.0 and 4.0. That’s why we look at lots of other things in addition to class rank — things that we hope and believe are relevant to the other 75% of the observed variance in grades.
I also find it interesting that Professor Hansen does not point out that his own table shows we admitted 98% of the majority applicants who were in the top ten percent, but only 93% of the minority applicants in that bin. I wonder how he explains that? It seems to me that if we were practicing (in his usual terminology — a term I find both offensive and intentionally misleading) “race preference,” and if we were admitting based solely on numerical indicators, we would be admitting 100% of the high-scoring minority applicants. I haven’t checked his numbers with original data, but we do turn down both majority and minority applicants who have high grades, ranks, or test scores when our overall review of the application leads us to conclude they would not be likely to succeed here.
Finally, I can tell you that more than two thirds of the students who fail to graduate are in good academic standing when they drop out. That means they have dropped out for nonacademic reasons (financial reasons, health, discouragement over the campus climate, etc.), and minority students experience more of these challenges than do majority students. All the students we admit are, to the very best of our ability to judge, capable of succeeding academically, and most of them do. We have the highest graduation rate in the System, and are far above the national average for both majority and minority students.
— John Wiley