Skip to Content

 

Contact Us

P.O. Box 400308
Booker House
Charlottesville, VA 22904-4308

Phone:
434-924-3728
Fax:
434-982-2920
Email:
provost@virginia.edu

Provost's Search Committee Tutorial: A Primer

Please Note: This is not the tutorial but a narrative version of tutorial materials, a primer if you will. Reading through this material will NOT give the necessary certification for serving on a faculty search committee. It is designed instead to offer all who are interested a chance to become familiar with the material covered in the tutorial prior to taking the tutorial and/or a chance to review material and references for individual topics after you have taken the tutorial.

If you would like to actually take the tutorial please click here.

 

8. Cognitive Errors


JoAnn Moody has spent the last couple of decades as a consultant to colleges and universities across the country, offering expertise on recruitment, mentoring and retention of faculty. In her monograph, Rising Above Cognitive Errors: Guidelines for Search, Tenure Review, and Other Evaluation Committees, she writes, “cognitive scientists, are proving definitively that many of the selection and evaluation tasks we undertake on a daily basis are alarmingly ‘contaminated.’ The contaminants—what can be generically termed cognitive shortcuts and errors—are present in academia as we gather and sort through information, interpret it, and then come to decisions about, for instance, job candidates, tenure and promotion cases, grant and fellowship applicants” (p.1).  

Throughout the evaluation process, search committee members and chairs can avoid or minimize the severity of these errors—if they learn to recognize and steer clear of them and agree on the ground rules for candidate discussion. The dialogue below is based on Moody's work and illustrates several cognitive errors.

We drop in on the middle of a search committee meeting at Eminent University…

Which cognitive errors do you detect in this conversation?

Luanne [search committee chair]: John, we’re discussing Dr. Houston now, not Dr. Smith.
John: But Smith is the obvious choice—I mean, he went to Harvard undergrad, then did his Ph.D. at Johns Hopkins. What else do we need to know?
Ed: [rushing in and sitting down] Sorry I’m late—one of my students stopped me in the hall. [waves some papers around] I happened to pull up Lucia Arroyo’s file. I was looking at her references. You know, I have to say they seem almost too good, and they're from her advisors and co-researchers. I have to wonder...Maybe we should get some more references for her, maybe from people who are more, I don’t know, at an arm's length or something.
Luanne: [looking puzzled] Her references are too good? I don’t think I’ve ever heard anyone say that before. And who else besides advisors and co-workers should we be getting references from?
John: Hmmm, interesting point, Ed. Besides, would she feel comfortable here? I mean, it would probably be too humid for her; given that she’s from New Mexico.
Luanne: [to herself] Am I really hearing this? [to the others] Come on, guys. We’ve gotten off track; we were talking about candidates strengths. Before you came in, Ed, I was saying that Houston had done some really interesting research on—.
Sally: [interrupting] It seems like a waste of time to spend the first meeting looking at the strengths of all the candidates. I know the vice-provost suggested it, but really, we’re all very busy people. A lot of these candidates just aren’t that good. And we all know which one is best. Smith first, then Mullins, and ah, then, ah [she looks up at Ed] maybe Arroyo.
John: Absolutely. Smith is the one, and if we can’t get him, I’d agree that Mullins is the next best, though not nearly as good as Smith. Forget the rest.
Ed: I agree about Smith, and Mullins.
Luanne: Wait. Wait. This is absurd. We need to look at all of them, strengths first , and not think about ranking them until after we’ve given all of them an honest shake, and carefully, and thoughtfully, looked at their strengths, and then their limitations, against the criteria that we developed before we posted the position.
John: Well, my mind’s made up.
Ed: [nodding in agreement]
Sally: [looking a little sheepish] Yeah, I agree Smith is really good.

Cognitive Errors in this Scenario
 
a) elitism - assuming that the best candidates always come from schools/social classes/regions that have traditionally been considered "the best," without careful attention to CVs, recommendations, needs of the department, etc.

b) raising & lowering the bar - setting higher/lower standards for some candidates based on negative/positive stereotypes

c) seizing a pretext - giving excessive weight to a relatively minor point, in order to justify disqualifying a candidate

d) premature ranking
– a rush to rank candidates; a focus on filtering out rather than filtering in

e) momentum of the groupif most group members have rallied together for their favorite candidate, it may be difficult to encourage people to step back and look more objectively at other qualified candidates

 



 

Moody also recommends other techniques for preventing or dealing with cognitive errors, including:

  1. keeping reminders of common cognitive errors on index cards visible during search committee meetings
  2. establishing ground rules for search committee processes prior to the first meeting
  3. set criteria for evaluation prior to receiving candidate applications
  4. utilize a matrix for keeping track of how well candidates meet those criteria


The scenario below is a duplicate of the one given above; in this version, however, the cognitive errors are in red and are identified by name in the [blue brackets] which follow.

Luanne: [search committee chair]: John, we’re discussing Dr. Houston now, not Dr. Smith.
John: But Smith is the obvious choice—I mean, he went to Harvard undergrad, then did his Ph.D. at Johns Hopkins. What else do we need to know? [elitism]
Ed: [rushing in and sitting down] Sorry I’m late—one of my students stopped me in the hall. [he waves some papers around] I happened to pull up Lucia Arroyo’s file. I was looking at her references. You know, I have to say they seem almost too good, and they're from her advisors and co-researchers. I have to wonder...Maybe we should get some more references for her, maybe from people who are more, I don’t know, at an arm's length or something. [raising the bar]
Luanne: [looking puzzled] Her references are too good? I don’t think I’ve ever heard anyone say that before. And who else besides advisors and co-workers should we be getting references from?
John: Hmmm, interesting point, Ed. Besides, would she feel comfortable here? I mean, it would probably be too humid for her; given that she’s from New Mexico. [seizing a pretext]
Luanne: [to herself] Am I really hearing this? [to the others]. Come on, guys. We’ve gotten off track; we were talking about candidates strengths. Before you came in, Ed, I was saying that Houston had done some really interesting research on—.
Sally: [interrupting] It seems like a waste of time to spend the first meeting looking at the strengths of all the candidates. I know the vice-provost suggested it, but really, we’re all very busy people. A lot of these candidates just aren’t that good. And we all know which one is best. Smith first, then Mullins, and ah, then, ah [she looks up at Ed] maybe Arroyo. [premature ranking]
John: Absolutely. Smith is the one, and if we can’t get him, I’d agree that Mullins is the next best, though not nearly as good as Smith. Forget the rest. [premature ranking, momentum of the group]
Ed: I agree about Smith, and Mullins. [premature ranking, momentum of the group]
Luanne: Wait. Wait. This is absurd. We need to look at all of them, strengths first , and not think about ranking them until after we’ve given all of them an honest shake, and carefully, and thoughtfully, looked at their strengths, and then their limitations, against the criteria that we developed before we posted the position.
John: Well, my mind’s made up. [momentum of the group]
Ed: [nodding in agreement] [momentum of the group]
Sally: [looking a little sheepish] Yeah, I agree Smith is really good. [momentum of the group]

Cognitive Errors in this Scenario
 
a) elitism - assuming that the best candidates always come from schools/social classes/regions that have traditionally been considered "the best," without careful attention to CVs, recommendations, needs of the department, etc.

b) raising & lowering the bar - setting higher/lower standards for some candidates based on negative/positive stereotypes

c) seizing a pretext - giving excessive weight to a relatively minor point, in order to justify disqualifying a candidate

d) premature ranking
– a rush to rank candidates; a focus on filtering out rather than filtering in

e) momentum of the groupif most group members have rallied together for their favorite candidate, it may be difficult to encourage people to step back and look more objectively at other qualified candidates



 

JoAnn Moody lists fifteen common cognitive errors that can occur during searches, promotion and tenure, and other evaluative situations. For a complete description of all fifteen cognitive errors, and her suggestions for avoiding or mitigating these ways of thinking or behaving, see Rising Above Cognitive Errors: Guidelines for Search, Tenure Review, and Other Evaluation Committees which can be ordered from Dr. Moody’s Web site, http://www.diversityoncampus.com/id13.html

 

References

Moody, J. 2004. Faculty diversity: problems and solutions. New York: Routledge. UVa call number LB2332.6.M66 2004.

______. 2007. Rising above cognitive errors: Guidelines for search, tenure review, and other evaluation committees. JoAnn Moody. To order this monograph go to JoAnn Moody's Web site, http://www.diversityoncampus.com/id13.html  

 

NEXT


Table of Contents