Results of DSM Field Trials Available on AJP in Advance

Three papers discussing the results of the DSM-5 field trials were posted October 30 by AJP in Advance. These papers describe the methods and results of the 23 diagnoses that were assessed.

In these first analyses of data from the field trials, 14 of the 23 adult or child psychiatric diagnoses had “very good” or “good” reliability. Among these were autism spectrum disorder and ADHD in children and posttraumatic stress disorder and binge-eating disorder in adults.

Another paper describes the study design, sampling strategy, and analytic approach used in the field trials. The third paper reports results of reliability tests for “cross-cutting symptom measures”—a set of symptom measures that may be included in supplemental material to DSM-5 to assist clinicians in forming diagnoses for patients with comorbid conditions. These measures were also generally found to have good to excellent reliability.

The proposed DSM-5 criteria are still under review and will not be final until they are approved by the APA Board of Trustees.

The criteria were tested in field trials October 2010 through February 2012 by 279 clinicians at 11 academic centers in the United States and Canada. (A second set of data from small-group practices and private practices will be reported early next year.) Each center was assigned several target diagnoses frequently treated in that setting.

Consecutive patients visiting a site during the study were screened and stratified on the basis of DSM-IV diagnoses or symptomatic presentations. Patients were randomly assigned to two clinicians for a diagnostic interview, and clinicians were blind to any previous diagnosis. All data were entered directly via an Internet-based software system to a secure central server.

There were 15 adult and eight child/adolescent diagnoses for which adequate sample sizes were obtained to report adequately precise estimates of kappa—the statistical term for reliability. Overall, five diagnoses were in the “very good” range with kappa scores between 0.60 and 0.79, and nine were in the “good” range with scores between 0.40 and 0.59.

Diagnoses for which the scores were “very good” included autism spectrum disorder and ADHD in children, and PTSD, complex somatic symptom disorder (revised), and major neurocognitive disorder in adults. The reliability scores for autism spectrum disorder in children were quite high at both centers where the criteria were tested, and at one institution the score was 0.72; similarly high scores were found for ADHD.

Diagnoses for which the scores were in the “good” range included avoidant-restrictive food intake disorder and oppositional defiant disorder in children, and schizophrenia, schizoaffective disorder, bipolar disorder, binge-eating disorder, alcohol use disorder, mild neurocognitive disorder, and borderline personality disorder in adults.

Six of the diagnoses had relatively low reliability scores, which the report characterized as “questionable” but acceptable; among these diagnoses were major depressive disorder and generalized anxiety disorder, two of the most common psychiatric diagnoses.

DSM-5 Task Force Chair David Kupfer, M.D., said, “We believe at least some of the reason why the scores were not at the level of previous field trials is the attention we devoted to new diagnoses and criteria sets that are often comorbid with depression and anxiety. Moreover, depression and anxiety represent symptom clusters that can fluctuate over the course of several weeks, and the rigorous trial design of the field trials—employing a test-retest design requiring two separate clinicians to evaluate the same patient on different days—may have contributed to a lower reliability score than was found in previous field trials.”

Just three diagnoses had “unacceptable” rates of reliability, and these have been substantially revised or are no longer proposed for inclusion in the manual.

Kupfer said that the field trials provide critical new data for the ongoing review of proposed diagnostic criteria for DSM-5 (see What We Learned From DSM-5 Field Trials).

“We accomplished what we set out to do—which was to look at a limited group of diagnoses in which either a new disorder was being proposed or there were significant changes to existing criteria and to test both reliability and clinical acceptance,” he said. “The good to very good reliability of most diagnoses underscores that clinicians are comfortable using these new criteria.”

#x201C; DSM-5 Field Trials in the United States and Canada, Part I: Study Design, Sampling Strategy, Implementation” is posted at http://ajp.psychiatryonline.org/Article.aspx?ArticleID=1387935. “DSM-5 Field Trials in the United States and Canada, Part II: Test-Retest Reliability of Selected Categorical Diagnoses and Analytic Approaches” is posted at http://ajp.psychiatryonline.org/article.aspx?articleID=1387906. “DSM-5 Field Trials in the United States and Canada, Part III: Development and Reliability Testing of a Cross-Cutting Symptom Assessment for DSM-5 ” is posted at http://ajp.psychiatry online.org/Article.aspx?ArticleID=1387907 .