An important part of the admission process to a competitive college is the admissions interview. I'm against it. And that isn't just because interviews were originally instituted to keep Jews out of Harvard. It's because they are poor predictors of future performance and, even worse, they are poor predictors that people weight very heavily.
I was first clued into this by none other than Google. Google recently revamped the way it chooses new hires, and an important part of the overhaul was minimizing the importance of the interview. As Laszlo Bock, Google's vice president for people operations said, "Interviews are a terrible predictor of performance."
This stands to reason. We all know people who make great first impressions but then turn out to be lousy employees/students/friends/etc. Similarly, we know people who originally struck us as dull but turned out to be our best employee/student/friend/etc. However, it would be nice to have something quantitative to back up this observation, and so I've been on the lookout ever sense.
It is in this context that I read this following quote from a classic Science paper by Tversky and Kahneman:
Tversky and Kahneman have an explanation for why people care so much about interviews. Across the board, people believe that small samples are much more reliable than they are. I recommend the original paper if you want the full argument, but they bring up many examples. For instance, participants believe a random sample of 10 men is just as likely to have an average height of 6 feet as a random sample of 1000. This is not mathematically possible, but even experts in statistics can, under the right circumstances, fall for this.
This is why I think the admissions interview, as well as the job interview, should be scrapped. It takes place over a short period of time, which means it is an inherently unreliable predictor of future performance. It's unreliable, but, even knowing that, the information gleaned from it irresistible.
Tversky, A., Kahneman, D. (1974). Judgment under uncertainty: heuristics and biases. Science, 185(4157), 1124-1131.
I was first clued into this by none other than Google. Google recently revamped the way it chooses new hires, and an important part of the overhaul was minimizing the importance of the interview. As Laszlo Bock, Google's vice president for people operations said, "Interviews are a terrible predictor of performance."
This stands to reason. We all know people who make great first impressions but then turn out to be lousy employees/students/friends/etc. Similarly, we know people who originally struck us as dull but turned out to be our best employee/student/friend/etc. However, it would be nice to have something quantitative to back up this observation, and so I've been on the lookout ever sense.
It is in this context that I read this following quote from a classic Science paper by Tversky and Kahneman:
It is a common observation that psychologists who conduct selection interviews often experience considerable confidence in their predictions, even when they know of the vast literature that shows selection interviews to be highly fallible. The continued reliance on the clinical interview for selection, despite repeated demonstrations of its inadequacy, amply attests to the strength of this effect.Tversky and Kahneman probably did not think this was a problem with the clinical interview per se. They give several other examples, including a study in which participants read a short description of a particular lesson a student teacher gave. Some participants were asked to evaluate the quality of the lesson, giving it a percentile score. Others were asked to guess the percentile score of that student teacher's overall abilities 5 years in the future. The judgments in both conditions were identical. That is, the participants believed that the quality of a single lesson fully predicted how good a future teacher would be. They don't take into consideration that the student teacher might be having a bad or good day.
Tversky and Kahneman have an explanation for why people care so much about interviews. Across the board, people believe that small samples are much more reliable than they are. I recommend the original paper if you want the full argument, but they bring up many examples. For instance, participants believe a random sample of 10 men is just as likely to have an average height of 6 feet as a random sample of 1000. This is not mathematically possible, but even experts in statistics can, under the right circumstances, fall for this.
This is why I think the admissions interview, as well as the job interview, should be scrapped. It takes place over a short period of time, which means it is an inherently unreliable predictor of future performance. It's unreliable, but, even knowing that, the information gleaned from it irresistible.
Tversky, A., Kahneman, D. (1974). Judgment under uncertainty: heuristics and biases. Science, 185(4157), 1124-1131.
4 comments:
So what do you suggest instead? Put people's CVs on the wall and throw darts?
Interviews are as much about determining who will fit in better on a social level. Someone might look good on paper, but if they are a jerk, then everyone's productivity is affected. This certainly applies in the workplace, perhaps not so much in admissions.
I think the interview process is better than letters of recommendation. I/O Psych literature suggests that people are not going to ask for a recommendation from someone who will say something bad about them. You may argue people may still write a non-glowing recommendation - but I/O Psych shows this is rarely the case and the recommendation letter is one of the poorest predictors of performance.
It is true in admissions processes that typically they ask for sealed letters, however, I know people who asked for several (more than they needed - for more schools than needed). So they picked a school they weren't going to apply for and then was free to take home the sealed envelopes read the recommendations and figure out who was giving the best recommendations and then use those folks' recommendations for the schools they actually applied for.
At some point, all pieces of a selection process can be criticized and I think the interview may have problems but does add something (Can the person prepare for an interview? Do they care enough to prepare for an interview?) At least, it may be better than a recommendation letter which someone else prepares.
I am not an expert, but when I was running a volunteer organization, I found that interviews could help us shrink the number of candidates that would eventually "bail out" and abandon their volunteer position in a very short period of time.
The interview might not say much about actual abilities, but it tends to say a lot about your commitment. References (especially when contacted directly or given online, and not coming from a letter) can be a good way to attest to the supposed abilities of the candidate.
It's also important to remember that interviews are helpful to vet out the information on the CV - if a candidate is fluffing their background/experience, that comes out in about 5 minutes.
Post a Comment