During their career people go through a professional socialization process in which they learn the theories of their field and adopt its metaphysical assumptions. This process defines for the members of the profession the phenomena that are worthy of observation, and the type of information that is relevant to their professional judgments. While this process is an integral part of the development of professional skills, it may also lead professionals to base their judgments on perceptions derived from the dominant theories in their field, and ignore, to some extent, important relationships among variables in the environment. This phenomenon has been demonstrated in domains such as managerial decision making and scientific research.

For many years, an important characteristic of the professional socialization process of clinical psychologists has been an emphasis on the psycho-pathological aspects of the mind. The training of clinical psychologists concentrated on issues such as the origins of psycho-pathology, its development, and its remedy; it focused on the diagnosis of the pathological rather than on the identification of the benign; and it centered on the study of the deviant rather than the understanding of the normal. Furthermore, to a certain extent, even normal behavior were often understood by clinical psychologists to be the result of unconscious pathological aspects of the mind, such as murder impulses, incestuous fantasies, and death wishes.

Did a socialization process that emphasize pathology influence the professional judgment of clinicians? A number of studies which have dealt with this question concluded that the answer is positive. Rosenham’s (1972) paper entitled ‘On being sane in insane places’ is the most well-known. In this paper, Rosenham reports that behaviors, which would otherwise appear normal, were judged as pathological by the staff of psychiatric hospitals. However, Rosenham’s results could be attributed to initial false information which was supplied to the judges in his study, or to the high base-rate probability of pathology for inmates of psychiatric hospitals.

The current paper examines the hypothesis that in making clinical judgments, psychologists assigne excessively heavy weight to information regarding the presence or absence of severe pathology (which will be labelled pathological information), in comparison to information regarding the presence or absence of mild pathology (which, within the context of the current study, could be labelled non-pathological information). The paper presents evidence suggesting that such biased weighing influence clinical judgment, and explores the outcomes of this bias. Finally, the cognitive processes underlying this bias are discussed in terms of a confirmation bias in hypothesis testing – the tendency to overemphasize information confirming, rather than disconfirming, expectations; and it is suggested that clinicians’ confirmatory hypothesis are associated with the existence of severe, rather than mild, pathology.

The data used in the paper were collected in the mid-1950s by Meehl. The analysis of these data played a major role in the study of the validity of clinical judgments. In these studies, researchers were primarily interested in the actuarial validity of the judgment, and in particular, in whether clinical judgments have a higher correlation with the criterion than the predictions of a linear model. The approach of the current paper to the study of the validity of clinical judgment is different. The paper focuses on analyzing and explaining biases, or systematic deviations, from optimal actuarial validity. As a result, the method used in the paper is also different from the method used in the previous studies. While in previous studies the validity of clinical judgment was studied by correlating judgment with the criterion, in the current study it is studied by comparing models of the judgment to models of the criterion. In particular, the study compares the weight of information associated with severe pathology to the weight of information associated with mild pathology in these models.

Meehl’s data include 861 MMPI profiles of psychiatric patients – the patients’ scores on the 11 most commonly used scales of the MMPI. They also include the criterion – the diagnosis given to the patient in the clinic in which he/she received treatment. 47% of the patients were diagnosed as psychotics and 53% diagnosed as neurotics. These diagnoses were based primarily on information about the patient’s past and present behavior, and, to a certain extent, on the results of various psychological tests. For some of the patients, the information on which the diagnosis was based did not include the MMPI profiles, but for some, the MMPI profiles were available when the clinic’s diagnosis was made.

The data also include evaluations of the 861 profiles that were made by 29 clinicians, whose schooling represented a wide variety of approaches to Clinical Psychology at the time Meehl’s experiment took place (Meehl, personal communication). Each clinician judged the 861 MMPI profiles on an 11-step forced normal distribution scale from least psychotic (1) to most psychotic (11). The clinicians were instructed that the patients could be either psychotics or neurotics.

One aspect of the data which is particularly important to the current study is that the MMPI scales of the 861 profiles have a clear dimensional organization, for the results of a factor analysis of the scales.) One dimension is associated with the neurotic scales of the MMPI, another with the psychotic scales, and a third with scales that identify defensiveness in test taking. These dimensions, and in particular the neurotic and the psychotic dimensions, were likely to have played an important role in the process by which the clinicians used the MMPI profiles in their diagnostic judgments in Meehl’s experiment.

The criterion was modeled using a logistic regression, where Y is the log of the odds of having been diagnosed as psychotic by the clinic. The judgment was modeled in three ways. First, by modeling the mean judgment, which was created by averaging the judgments of all 29 clinicians to each profile. Second, by modeling the judgments of each of the 29 judges. And third, by modeling a binary variable, called the MMPI diagnosis, created by rank ordering the profiles according to their mean judgment, and labeling the top 47% as having diagnosis of psychosis and the bottom 53% as having a diagnosis of neurosis. The MMPI diagnosis represents the diagnosis that would have been assigned to the patients based on a consensus judgment of the MMPI profiles, and it could be compared to the criterion, or the clinic’s diagnosis, which was made primarily on the basis of actual behavior. Although some loss of information is involved in using the MMPI diagnosis rather than the untransformed judgment, I discuss the results primarily in terms of this binary variable, because it makes the judgment directly comparable to the criterion. However, the results of the models of the untransformed judgments are reported as well.