Skip to main content
To the Editor: The DSM-5 criteria incorporate many changes to improve psychiatric diagnosis compared with DSM-IV. While test-retest reliability was not a major DSM-5 focus, valid measurement requires adequate reliability (1). For alcohol use disorders, a 2006 review (2) indicated good to excellent reliability of the DSM-IV criteria for alcohol dependence (kappa values, 0.66–0.82). Reliability of DSM-IV alcohol abuse was lower, but the criteria were more reliable when analyzed independently of dependence (2). In the DSM-5 field trial, alcohol use disorder reliability was much lower (kappa=0.40) (1). Why the precipitous drop in reliability?
We considered three explanations. First, compared with DSM-IV, DSM-5 field trial cases could have been more complex because of comorbidity that reduced reliability. This appears unlikely, however, because the DSM-IV studies did not exclude comorbid cases, and they recruited participants from settings in which comorbidity was common. Second, DSM-IV kappas could have been inflated because multisite studies did not adjust for data pooled across sites. The reanalysis of data from one of the studies (3) after making this adjustment (4) suggests that this is also unlikely. After pooling data from sites with more than 50 patients, the unadjusted and adjusted kappas did not differ dramatically (Table 1).
TABLE 1. Test-Retest Reliability of Pooled Data From DSM-IV Trial Sites With >50 Patients (N=257)a
DiagnosisKappa Adjusted for Site
 NoYes
Alcohol dependence  
 Current0.820.82
 Lifetime0.760.62
Cocaine dependence  
 Current0.900.87
 Lifetime0.870.87
Cannabis dependence  
 Current0.730.64
 Lifetime0.630.60
Heroin dependence  
 Current0.930.93
 Lifetime0.960.92
Major depressive disorder, primary  
 Current0.770.84
 Lifetime0.720.80
Major depressive disorder, substance induced  
 Current0.680.71
 Lifetime0.740.74
Antisocial personality disorder0.730.74
Borderline personality disorder0.660.66
a
Data from Hasin et al. (3)
Third, error variance could have differed between the DSM-IV studies and the DSM-5 field trial. Two main types of error variance reduce reliability (57): criterion variance and information variance. Criterion variance is minimized by having clear guidelines to evaluate symptoms. Information variance is minimized by standardizing 1) content, through fully or semistructured interviews, and 2) procedures, such as investigator training in the use of the interviews. While information variance can never be eliminated completely, studies using standard methodology (structured interviews and training) (7) to control information variance provide more information on criterion variance than studies that do not.
In terms of criterion variance, did the DSM-5 criteria for alcohol use disorder become less clear than those in DSM-IV? This appears unlikely, since the biggest change in alcohol use disorder from DSM-IV to DSM-5 was the combination of dependence and abuse criteria into a single disorder. Except for the addition of craving (new to DSM-5) and the deletion of legal problems, the alcohol use disorder criteria are identical in DSM-IV and DSM-5.
Did information variance increase in the DSM-5 field trial compared with studies in the 2006 review (2)? Here, the answer appears to be yes. All DSM-IV reliability studies reviewed in 2006 used standard methodology to minimize information variance. The DSM-5 field trials were very different (1, 4). After a brief introduction to DSM-5 criteria, clinicians conducted unstructured diagnostic interviews, rating criteria checklists for any disorder they deemed appropriate. Because clinician training was minimal and diagnostic assessment and checklist selection were unstructured, information variance in the DSM-5 alcohol use disorder field trial was not controlled.
Given the design differences in the DSM-IV studies and the DSM-5 field trial, the reliability of DSM-5 alcohol use disorder cannot be directly compared with DSM-IV; studies of DSM-5 alcohol use disorder that minimize information variance are needed. We think such a study should be conducted to accurately make comparisons between the DSM-IV and DSM-5 criteria for alcohol use disorder.

References

1.
Regier DA, Narrow WE, Clarke DE, Kraemer HC, Kuramoto SJ, Kuhl EA, Kupfer DJ: DSM-5 field trials in the United States and Canada, part II: test-retest reliability of selected categorical diagnoses. Am J Psychiatry 2013; 170:59–70
2.
Hasin D, Hatzenbuehler ML, Keyes K, Ogburn E: Substance use disorders: Diagnostic and Statistical Manual of Mental Disorders, fourth edition (DSM-IV) and International Classification of Diseases, tenth edition (ICD-10). Addiction 2006; 101(suppl 1):59–75
3.
Hasin D, Samet S, Nunes E, Meydan J, Matseoane K, Waxman R: Diagnosis of comorbid psychiatric disorders in substance users assessed with the Psychiatric Research Interview for Substance and Mental Disorders for DSM-IV. Am J Psychiatry 2006; 163:689–696
4.
Clarke DE, Narrow WE, Regier DA, Kuramoto SJ, Kupfer DJ, Kuhl EA, Greiner L, Kraemer HC: DSM-5 field trials in the United States and Canada, part I: study design, sampling strategy, implementation, and analytic approaches. Am J Psychiatry 2013; 170:43–58
5.
Spitzer RL, Endicott J, Robins E: Clinical criteria for psychiatric diagnosis and DSM-III. Am J Psychiatry 1975; 132:1187–1192
6.
Spitzer RL, Fleiss JL: A re-analysis of the reliability of psychiatric diagnosis. Br J Psychiatry 1974; 125:341–347
7.
Endicott J, Spitzer RL: A diagnostic interview: the schedule for affective disorders and schizophrenia. Arch Gen Psychiatry 1978; 35:837–844

Information & Authors

Information

Published In

Go to American Journal of Psychiatry
Go to American Journal of Psychiatry
American Journal of Psychiatry
Pages: 442 - 443
PubMed: 23545795

History

Accepted: February 2013
Published online: 1 April 2013
Published in print: April 2013

Authors

Details

Deborah S. Hasin, Ph.D.
From the Department of Psychiatry, College of Physicians and Surgeons, Columbia University, New York State Psychiatric Institute/Columbia University, New York; the Department of Addiction Psychiatry, Bordeaux Segalen University, Bordeaux, France; National Institute of Psychiatry, Mexico City; Washington University, St. Louis; the Center for Addiction Research, Department of Psychiatry, Dartmouth University, University of Colorado School of Medicine, Aurora; National Institute on Alcohol Abuse and Alcoholism, Bethesda, Md.; the Department of Psychiatry, University of Pennsylvania School of Medicine, Philadelphia; the Department of Psychiatry, University of Connecticut Health Center, Farmington; the Department of Psychiatry, San Diego VA Medical Center; and the Department of Psychiatry, College of Physicians and Surgeons, Columbia University.
Marc Auriacombe, M.D.
From the Department of Psychiatry, College of Physicians and Surgeons, Columbia University, New York State Psychiatric Institute/Columbia University, New York; the Department of Addiction Psychiatry, Bordeaux Segalen University, Bordeaux, France; National Institute of Psychiatry, Mexico City; Washington University, St. Louis; the Center for Addiction Research, Department of Psychiatry, Dartmouth University, University of Colorado School of Medicine, Aurora; National Institute on Alcohol Abuse and Alcoholism, Bethesda, Md.; the Department of Psychiatry, University of Pennsylvania School of Medicine, Philadelphia; the Department of Psychiatry, University of Connecticut Health Center, Farmington; the Department of Psychiatry, San Diego VA Medical Center; and the Department of Psychiatry, College of Physicians and Surgeons, Columbia University.
Guilherme Borges, Ph.D.
From the Department of Psychiatry, College of Physicians and Surgeons, Columbia University, New York State Psychiatric Institute/Columbia University, New York; the Department of Addiction Psychiatry, Bordeaux Segalen University, Bordeaux, France; National Institute of Psychiatry, Mexico City; Washington University, St. Louis; the Center for Addiction Research, Department of Psychiatry, Dartmouth University, University of Colorado School of Medicine, Aurora; National Institute on Alcohol Abuse and Alcoholism, Bethesda, Md.; the Department of Psychiatry, University of Pennsylvania School of Medicine, Philadelphia; the Department of Psychiatry, University of Connecticut Health Center, Farmington; the Department of Psychiatry, San Diego VA Medical Center; and the Department of Psychiatry, College of Physicians and Surgeons, Columbia University.
Kathleen Bucholz, Ph.D.
From the Department of Psychiatry, College of Physicians and Surgeons, Columbia University, New York State Psychiatric Institute/Columbia University, New York; the Department of Addiction Psychiatry, Bordeaux Segalen University, Bordeaux, France; National Institute of Psychiatry, Mexico City; Washington University, St. Louis; the Center for Addiction Research, Department of Psychiatry, Dartmouth University, University of Colorado School of Medicine, Aurora; National Institute on Alcohol Abuse and Alcoholism, Bethesda, Md.; the Department of Psychiatry, University of Pennsylvania School of Medicine, Philadelphia; the Department of Psychiatry, University of Connecticut Health Center, Farmington; the Department of Psychiatry, San Diego VA Medical Center; and the Department of Psychiatry, College of Physicians and Surgeons, Columbia University.
Alan Budney, Ph.D.
From the Department of Psychiatry, College of Physicians and Surgeons, Columbia University, New York State Psychiatric Institute/Columbia University, New York; the Department of Addiction Psychiatry, Bordeaux Segalen University, Bordeaux, France; National Institute of Psychiatry, Mexico City; Washington University, St. Louis; the Center for Addiction Research, Department of Psychiatry, Dartmouth University, University of Colorado School of Medicine, Aurora; National Institute on Alcohol Abuse and Alcoholism, Bethesda, Md.; the Department of Psychiatry, University of Pennsylvania School of Medicine, Philadelphia; the Department of Psychiatry, University of Connecticut Health Center, Farmington; the Department of Psychiatry, San Diego VA Medical Center; and the Department of Psychiatry, College of Physicians and Surgeons, Columbia University.
Thomas Crowley, M.D.
From the Department of Psychiatry, College of Physicians and Surgeons, Columbia University, New York State Psychiatric Institute/Columbia University, New York; the Department of Addiction Psychiatry, Bordeaux Segalen University, Bordeaux, France; National Institute of Psychiatry, Mexico City; Washington University, St. Louis; the Center for Addiction Research, Department of Psychiatry, Dartmouth University, University of Colorado School of Medicine, Aurora; National Institute on Alcohol Abuse and Alcoholism, Bethesda, Md.; the Department of Psychiatry, University of Pennsylvania School of Medicine, Philadelphia; the Department of Psychiatry, University of Connecticut Health Center, Farmington; the Department of Psychiatry, San Diego VA Medical Center; and the Department of Psychiatry, College of Physicians and Surgeons, Columbia University.
Bridget F. Grant, Ph.D., Ph.D.
From the Department of Psychiatry, College of Physicians and Surgeons, Columbia University, New York State Psychiatric Institute/Columbia University, New York; the Department of Addiction Psychiatry, Bordeaux Segalen University, Bordeaux, France; National Institute of Psychiatry, Mexico City; Washington University, St. Louis; the Center for Addiction Research, Department of Psychiatry, Dartmouth University, University of Colorado School of Medicine, Aurora; National Institute on Alcohol Abuse and Alcoholism, Bethesda, Md.; the Department of Psychiatry, University of Pennsylvania School of Medicine, Philadelphia; the Department of Psychiatry, University of Connecticut Health Center, Farmington; the Department of Psychiatry, San Diego VA Medical Center; and the Department of Psychiatry, College of Physicians and Surgeons, Columbia University.
Charles O’Brien, M.D., Ph.D.
From the Department of Psychiatry, College of Physicians and Surgeons, Columbia University, New York State Psychiatric Institute/Columbia University, New York; the Department of Addiction Psychiatry, Bordeaux Segalen University, Bordeaux, France; National Institute of Psychiatry, Mexico City; Washington University, St. Louis; the Center for Addiction Research, Department of Psychiatry, Dartmouth University, University of Colorado School of Medicine, Aurora; National Institute on Alcohol Abuse and Alcoholism, Bethesda, Md.; the Department of Psychiatry, University of Pennsylvania School of Medicine, Philadelphia; the Department of Psychiatry, University of Connecticut Health Center, Farmington; the Department of Psychiatry, San Diego VA Medical Center; and the Department of Psychiatry, College of Physicians and Surgeons, Columbia University.
Nancy M. Petry, Ph.D.
From the Department of Psychiatry, College of Physicians and Surgeons, Columbia University, New York State Psychiatric Institute/Columbia University, New York; the Department of Addiction Psychiatry, Bordeaux Segalen University, Bordeaux, France; National Institute of Psychiatry, Mexico City; Washington University, St. Louis; the Center for Addiction Research, Department of Psychiatry, Dartmouth University, University of Colorado School of Medicine, Aurora; National Institute on Alcohol Abuse and Alcoholism, Bethesda, Md.; the Department of Psychiatry, University of Pennsylvania School of Medicine, Philadelphia; the Department of Psychiatry, University of Connecticut Health Center, Farmington; the Department of Psychiatry, San Diego VA Medical Center; and the Department of Psychiatry, College of Physicians and Surgeons, Columbia University.
Marc Schuckit, M.D.
From the Department of Psychiatry, College of Physicians and Surgeons, Columbia University, New York State Psychiatric Institute/Columbia University, New York; the Department of Addiction Psychiatry, Bordeaux Segalen University, Bordeaux, France; National Institute of Psychiatry, Mexico City; Washington University, St. Louis; the Center for Addiction Research, Department of Psychiatry, Dartmouth University, University of Colorado School of Medicine, Aurora; National Institute on Alcohol Abuse and Alcoholism, Bethesda, Md.; the Department of Psychiatry, University of Pennsylvania School of Medicine, Philadelphia; the Department of Psychiatry, University of Connecticut Health Center, Farmington; the Department of Psychiatry, San Diego VA Medical Center; and the Department of Psychiatry, College of Physicians and Surgeons, Columbia University.
Melanie M. Wall, Ph.D.
From the Department of Psychiatry, College of Physicians and Surgeons, Columbia University, New York State Psychiatric Institute/Columbia University, New York; the Department of Addiction Psychiatry, Bordeaux Segalen University, Bordeaux, France; National Institute of Psychiatry, Mexico City; Washington University, St. Louis; the Center for Addiction Research, Department of Psychiatry, Dartmouth University, University of Colorado School of Medicine, Aurora; National Institute on Alcohol Abuse and Alcoholism, Bethesda, Md.; the Department of Psychiatry, University of Pennsylvania School of Medicine, Philadelphia; the Department of Psychiatry, University of Connecticut Health Center, Farmington; the Department of Psychiatry, San Diego VA Medical Center; and the Department of Psychiatry, College of Physicians and Surgeons, Columbia University.

Competing Interests

Dr. Auriacombe has received research and educational grants from Reckit Benkiser and advisory board fees from D&A Pharma and Mundipharma. Dr. Budney has received consulting fees from G.W. Pharmaceuticals. Dr. O'Brien has been an occasional consultant to companies that are conducting research to develop new medications for the treatment of addictive disorders. All authors have received travel expenses from APA for DSM work group meetings.

Metrics & Citations

Metrics

Citations

Export Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

For more information or tips please see 'Downloading to a citation manager' in the Help menu.

Format
Citation style
Style
Copy to clipboard

View Options

View options

PDF/EPUB

View PDF/EPUB

Login options

Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.

Personal login Institutional Login Open Athens login
Purchase Options

Purchase this article to access the full text.

PPV Articles - American Journal of Psychiatry

PPV Articles - American Journal of Psychiatry

Not a subscriber?

Subscribe Now / Learn More

PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.

Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).

Media

Figures

Other

Tables

Share

Share

Share article link

Share