Does the DSM-IV Clinical Significance Criterion for Major Depression Reduce False Positives? Evidence From the National Comorbidity Survey Replication

The National Comorbidity Survey Replication (NCS–R) (19) is a community-based epidemiological survey of a nationally representative U.S. sample. The survey was administered between February 2001 and December 2002 to 9,282 persons aged 18 years or older. In the present study, we restricted our analyses to the sample respondents ages 18 to 54 years (N=6,707). Demographic and other survey data are available elsewhere (19).

DSM-IV assesses clinical significance in those individuals whose conditions satisfy the symptom-duration criterion A. The NCS–R followed this approach for questions regarding impairment, but questions pertaining to distress were asked earlier in the interview of any respondent who reported adequately persistent sadness or the equivalent. This complicated our analysis because, given different subsamples resulting from skipped interview questions, the distress and impairment components had to be analyzed separately. However, examining distress in the broader persistent-sadness group was advantageous because it posed a more difficult challenge to the redundancy prediction. This group would be expected to include many individuals reporting milder, less symptomatic experiences that might not be distressful.

The NCS–R major depression diagnosis required 2 weeks of five or more DSM-IV-based symptoms, which had to include symptoms of sadness/emptiness/depression or of being discouraged or uninterested (not as the result of organic causes), and no lifetime mania or hypomania. In multiple-episode cases, symptoms were assessed for a "target" episode identified as the most severe. Mixed-episode and psychotic disorder exclusions were not operationalized. The survey's clinical significance requirement for major depression used multiple measures for distress and impairment, reflecting the broad DSM-IV criterion.

The NCS–R did not operationalize DSM-IV's bereavement exclusion. Thus, it allowed false positive diagnoses over and above the usual ones. These diagnoses were excluded in the NCS (20) at known rates, offering a comparison point for clinical significance in the replicated version of the survey.

The major depression portion of the interview was administered only if the respondent answered "yes" to one or more screener questions regarding persistent sadness (e.g., "Have you ever in your life had a period lasting several days or longer when most of the day you felt sad, empty or depressed"?), with additional screeners confirming that the symptom occurred at least 1 hour nearly every day for at least 2 weeks or multiple 3-day periods over 1 year. The distress questions followed this screening for persistent sadness. Those who reported distress were further evaluated on whether at least five relevant symptoms occurred during the same 2-week period (satisfying DSM-IV criterion A). Those reporting no distress were eliminated from the interview. Questions regarding impairment and various exclusions followed questions pertaining to symptoms.

A positive response to any one of 10 distress and impairment questions satisfied the survey's major depression clinical significance criterion.

Statistical analyses were performed using the survey estimation procedures in Stata10 (22), which calculate weighted coefficients and use Taylor series linearization to determine standard errors to correct for the sampling design.

Despite lack of a clinical significance criterion in the initial, DSM-III-R-based NCS and the addition of one in the DSM-IV-based NCS–R, the prevalence rates of adult depression in the 18- to 54-year-old age range rose substantially from 15.2% to 18.3%, respectively (Table 1).

The redundancy hypothesis for distress was strongly confirmed. Of the 2,071 respondents who experienced at least one mood symptom for 2 weeks (or multiple 3-day periods), 2,016 (97.2%) satisfied the distress clinical significance criterion. Only 55 individuals (2.8%) reported a mood symptom that did not satisfy the distress criterion.

Among the 2,071 case subjects evaluated for distress, 1,254 (60.5%) went on to qualify for a diagnosis of major depression, whereas 817 (39.5%) did not qualify for various reasons, mostly as a result of being subthreshold. Of those who did not qualify, 765 (93.5%) satisfied NCS–R distress criteria. Thus, distress was an exceptionally poor indicator of diagnostic status.

Minor variations in distress criteria did not make a substantial difference to prevalence. For example, the threshold for the response of "rarely" for the third distress question seemed low. However, raising the threshold to "sometimes" had almost no effect (10 subjects with depression were eliminated).

NCS–R impairment questions were asked of respondents satisfying symptom-duration criterion A requirements for depression. The redundancy prediction was strongly confirmed for impairment. Of the 1,542 individuals who satisfied criterion A requirements, 1,487 (96.2%) also satisfied clinical significance requirements for impairment (Table 1). The four Sheehan Disability Scale questions regarding impairment had little effect, adding only 19 impairment case subjects to those who already qualified on the basis of the interference with life and activities questions.

Despite false negative concerns, we recalculated the prevalence rates after raising the response thresholds for the distress and impairment questions to "severe," "often," and "a lot." Raising the threshold for distress to "severe" eliminated some subjects but left many subthreshold sadness cases satisfying the criterion. Of 2,071 respondents reporting sadness episodes, 1,346 (64.9%) reported severe distress. Notably, of 817 respondents who reported non-major depression sadness, 461 (56.5%) reported severe distress. Using severe impairment alone for clinical significance, our results are consistent with those of previous studies. Of 1,542 subjects satisfying criterion A, 1,277 (82.7%) reported severe impairment. The overall effect on major depression was minimal. Of 1,254 subjects qualifying for the diagnosis of major depression, 1,168 (92.9%) satisfied "severe" clinical significance.

The goal of the depression clinical significance criterion is to distinguish depressive disorder from symptomatically similar normal reactions. We found that significant distress accompanied almost all NCS–R cases of persistent sadness, and significant impairment accompanied almost all symptomatic cases satisfying criterion A. A disjunction of the two would cover even more cases.

The regular association of distress with sadness renders the distress criterion all but useless in distinguishing normal sadness from major depression. The clinical significance criterion's disjunctive character means that the distress component's weakness undermines the criterion's overall validity, perhaps explaining why many researchers forgo the distress component (9, 15, 18). We found that only 4% of subjects satisfying the requirements for NCS–R criterion A were not impaired, which is less than the percentage of case subjects with uncomplicated bereavement who were excluded from NCS but would have likely represented false positive diagnoses satisfying NCS–R criterion A requirements in the present study.

Thus, the belief that clinical significance effectively addresses false positive diagnoses for major depression and reduces rates of community prevalence appears to be unsubstantiated. Strengthening the criterion is not necessarily a solution. Our analyses indicated that modestly raising low thresholds has little effect. Substantially raising severity thresholds reduces prevalence (4, 9, 18), but, as we demonstrated, does not eliminate subthreshold false positives and only minimally reduces the prevalence of major depression, whereas the number of false negatives may substantially increase (4).

Our findings raise questions about the common use of clinical significance to justify diagnosis of subthreshold depression under the category of "mood disorder not otherwise specified." Such diagnoses, while allowing individuals who do not satisfy major depression criteria to get needed help, may be made without attention to whether an individual's symptoms are caused by a dysfunction, as required by DSM-IV's definition of disorder. The fact that virtually all individuals reporting extended sadness also reported significant distress suggests that the use of clinical significance to justify subthreshold diagnosis increases the risk of massive false positives.

Our finding is contrary to claims made by some NCS–R researchers. Kessler et al. (19) noted that "previous methodological research…found that the NCS CIDI [Composite International Diagnostic Interview] overdiagnosed MDD [major depressive disorder] because of false positive assessments of dysphoria and anhedonia" (19, p. 3097) and that the NCS–R's clinical significance requirement for depression was an attempt to correct these problems. However, our results indicate that the NCS–R clinical significance criterion eliminates almost no case subjects reporting persistent dysphoria and anhedonia because distress is almost always present. Thus, any progress with false positives must be a result of other NCS–R features.

Was there such progress? Kessler et al. (19) reported, as evidence for false positive reduction, a lower NCS–R major depression prevalence rate relative to that of NCS: "The NCS–R MDD [major depressive disorder] prevalence estimates are intermediate between the ECA [Epidemiologic Catchment Area] and NCS estimates….The lower CIDI prevalence estimates than those in the NCS are consistent with the fact that these modifications operated largely by reducing false positive assessments" (19, p. 3103). This might apply to 1-year prevalence rates. However, our results indicate that the NCS–R adult rate of prevalence for lifetime major depression (18.3%) is higher than the NCS prevalence rate (15.2%). Published rates for lifetime NCS and NCS–R major depression are 14.9% (23) and 16.2% (19), respectively. This increase occurred despite changes that made the replicated survey symptom criteria more demanding than that of the initial survey (e.g., requiring observed psychomotor retardation and feelings of complete worthlessness). Thus, the claim of improved lifetime depression false positive reduction remains unsupported.

How should DSM-V's re-evaluation of the clinical significance criterion proceed? Should we accept the ICD's generic rationale that diagnostic symptom/dysfunction assessment should be separated from considerations of role impairment?

To the contrary, the need for disorder-by-disorder evaluation is suggested by the ICD's own continued use of social role functioning in some criteria sets. For example, conduct disorder requires "major violations of age-appropriate social expectations," and reading disorder must "significantly interfere with academic achievement or activities of daily living that require reading skills." These exceptions suggest that clinical significance is diagnostically useful in some categories. For some disorders, the only harm caused by the symptoms might lie in social role impairment, as in conduct disorder. Moreover, some role capacities are biologically shaped, and thus role impairment implies biological dysfunction. For example, if social phobia when interacting with family members impairs basic role functions such as parenting and sexual interaction, this is also biological dysfunction. Finally, sometimes role failure, even if it is not itself a biological dysfunction, is the only way to infer an underlying dysfunction. For example, failure to learn to read, an invented social function, despite opportunity may imply an underlying dysfunction, and because the inferred dysfunction has no other known harmful effects the social impairment is important to diagnosis.

Even concerning redundancy, there are cross-category nuances. One might expect that a specific phobia entails distress in the form of object-triggered or anticipatory anxiety (in successful avoidance). The specific phobia criterion avoids redundancy by requiring role impairment or marked distress about having the condition. However, this solution creates a new problem. Distress about having a condition is of questionable validity in discriminating disorder from nondisorder (as in "ego-dystonic homosexuality"). Regarding role impairment, it might be argued that many specific phobias are instances of biologically prepared, normally distributed fears and thus are pathological only when they subvert other basic, biologically shaped functions, manifested in role impairment. Thus, within this category, impairment, if properly qualified, is diagnostically relevant.

These examples suggest that rather than wholesale removal that might yield unintended consequences, a disorder-by-disorder re-evaluation of the clinical significance criterion is preferable.