Application of DSM-5 Criteria for Autism Spectrum Disorder to Three Samples of Children With DSM-IV Diagnoses of Pervasive Developmental Disorders
Publication: American Journal of Psychiatry
Abstract
Objective
Substantial revisions to the DSM-IV criteria for autism spectrum disorders (ASDs) have been proposed in efforts to increase diagnostic sensitivity and specificity. This study evaluated the proposed DSM-5 criteria for the single diagnostic category of autism spectrum disorder in children with DSM-IV diagnoses of pervasive developmental disorders (PDDs) and non-PDD diagnoses.
Method
Three data sets included 4,453 children with DSM-IV clinical PDD diagnoses and 690 with non-PDD diagnoses (e.g., language disorder). Items from a parent report measure of ASD symptoms (Autism Diagnostic Interview–Revised) and clinical observation instrument (Autism Diagnostic Observation Schedule) were matched to DSM-5 criteria and used to evaluate the sensitivity and specificity of the proposed DSM-5 criteria and current DSM-IV criteria when compared with clinical diagnoses.
Results
Based on just parent data, the proposed DSM-5 criteria identified 91% of children with clinical DSM-IV PDD diagnoses. Sensitivity remained high in specific subgroups, including girls and children under 4. The specificity of DSM-5 ASD was 0.53 overall, while the specificity of DSM-IV ranged from 0.24, for clinically diagnosed PDD not otherwise specified (PDD-NOS), to 0.53, for autistic disorder. When data were required from both parent and clinical observation, the specificity of the DSM-5 criteria increased to 0.63.
Conclusions
These results suggest that most children with DSM-IV PDD diagnoses would remain eligible for an ASD diagnosis under the proposed DSM-5 criteria. Compared with the DSM-IV criteria for Asperger’s disorder and PDD-NOS, the DSM-5 ASD criteria have greater specificity, particularly when abnormalities are evident from both parents and clinical observation.
The proposed changes to the DSM-IV diagnostic criteria for pervasive developmental disorders (PDDs) include shifting from a multicategorical model to a single diagnostic category of autism spectrum disorder (ASD), replacing the three-domain model with a two-domain model, relaxing the criteria for age at onset, and adding symptoms not previously included in DSM-IV, such as sensory interests and aversions. Although these changes are based on empirical data (e.g., 1, 2), little is known about the sensitivity and specificity of the new criteria. In particular, it is unclear whether the revised criteria will inadvertently narrow the definition of PDDs. This is of major significance to families concerned that their affected children might not meet the proposed criteria for ASD and might therefore lose necessary services.
To date, various empirical studies have found support for a two-domain ASD symptom model (3–5). One difference from the original model is that communication deficits are subsumed under social impairments. Mandy and colleagues (6) tested this model, including sensory behaviors as part of the restricted and repetitive behavior criterion, and found that this model has an excellent fit. In contrast, the original DSM-IV model did not meet statistical criteria for an acceptable fit. Although this work confirms the conceptual validity of the proposed changes to DSM-IV, it tells us little about the sensitivity of the new criteria.
Because of the newness of the proposed criteria, only a handful of studies have examined the DSM-5 criteria, and all have examined slightly different versions of the criteria under consideration. McPartland and colleagues assessed the sensitivity and specificity of the proposed DSM-5 criteria by using the DSM-IV field trial checklist items and found DSM-5 to perform quite poorly (7). Using existing data from parent questionnaires, the Autism Diagnostic Interview–Revised (8), and the Autism Diagnostic Observation Schedule (9), Mattila et al. (10) examined an early draft of the criteria (2010) and found that only 46% of children with PDD diagnoses were identified as meeting the ASD criteria. Notably, when the authors used criteria more similar to the current DSM-5 criteria, approximately 96% of the children with PDD diagnoses were classified correctly.
The poor sensitivity of the early draft criteria and the remarkable increase in sensitivity with the new draft are likely explained by Mattila and colleagues’ stringent interpretation of the 2010 criteria. For example, sensitivity was improved when they required “routines AND/OR rituals” instead of “routines AND rituals.” Furthermore, unlike the early draft, the improved model included “unusual sensory behaviors” and the removal of an onset criterion of 36 months. This revision, which has been implemented in the latest DSM-5 draft, increased sensitivity, particularly in the “high-functioning” subgroup (i.e., children with full scale IQs of 70 or higher).
In another study, Frazier et al. (11) mapped items from the Social Communication Questionnaire (12) and the Social Responsiveness Scale (13) to DSM-5 criteria and found that 19%–22% of children with DSM-IV PDD diagnoses did not meet the proposed criteria. Notably, these analyses were based on criteria from DSM field trial phase 1, which required a greater number of symptoms than the currently proposed criteria. When the authors required fewer symptoms within each criterion (as in the current DSM-5 proposal), the sensitivity was comparable to that of DSM-IV and there was a slight improvement in specificity. This pattern of results was similar across many of the subgroups, such as girls, verbal youth, and multiplex families. Nevertheless, while the study group of Frazier et al. was large (N=14,744), the methods of the study limit the interpretability of their findings. For example, analyses included items based on past behavior (“When she/he was 4 to 5, did she/he smile back if someone smiled at her/him?”), whereas the proposed DSM-5 social communication criteria relate to current functioning and behavior.
Though specificity is an important focus of the proposed revisions, it is not yet clear that it will improve with the DSM-5 criteria. Frazier and colleagues’ recent analyses of the proposed criteria suggest improved specificity for the DSM-5 criteria over DSM-IV (11), particularly with a relaxed version of the DSM-5 criteria, using one less symptom per domain. However, these results were obtained from siblings of affected children, of which only about 30% had a caregiver-reported non-PDD diagnosis. Additional evidence from children with non-PDD diagnoses is necessary to make claims about DSM-5’s specificity.
The proposed change to a single ASD category, as well as the requirement that there must be a history of restricted and repetitive behaviors, has led some to believe that DSM-5 will make it more difficult for some individuals with PDDs to qualify for a diagnosis. The comprehensive review by Wing et al. of the proposed criteria articulates some of these concerns, explaining that DSM-5 could inadvertently exclude subgroups of affected people, including very young children, girls, and those with diagnoses of Asperger’s disorder (14). The introduction of “social communication disorder” in DSM-5 raises additional concerns that children currently diagnosed with PDDs will be misclassified with this disorder if they do not meet the DSM-5 requirement for restricted and repetitive behavior.
In sum, from the existing empirical work, the sensitivity of the proposed DSM-5 criteria remains unclear. In addition, relatively little attention has been paid to questions about specificity. Thus, before the proposed diagnostic changes go forward, it is critical to use the recent availability of large and well-characterized groups of children with PDDs and non-PDD diagnoses to attempt to shed light on these issues.
In the current study we sought to provide additional insights into DSM-5’s sensitivity and specificity by assigning individual items from well-established autism diagnostic measures to the proposed criteria and then using symptom counts to estimate how many children with previous DSM-IV diagnoses of PDDs or non-PDD disorders (e.g., language disorder, attention deficit hyperactivity disorder) would meet the DSM-5 ASD criteria. We applied these same methods to the DSM-IV criteria and to slight variations of the DSM-5 criteria, including those of Frazier et al. (11). We also completed domain-specific analyses to examine whether any children with clinical diagnoses of PDDs might meet the criteria for DSM-5 social communication disorder.
Method
This study was not a field trial for DSM-5. It used previously collected data to evaluate DSM-5 criteria in groups of children with DSM-IV clinical diagnoses.
Study Group
Participant data were obtained from three sources: 1) proband data from the Simons Simplex Collection, a genetic consortium study focusing on families having just one child with an ASD (“simplex” families) (15), 2) the Collaborative Programs of Excellence in Autism (subsequently referred to as “Collaborative Programs”), a multicenter study of ASD (4), and 3) the University of Michigan Autism and Communication Disorders Center data bank (subsequently referred to as “University of Michigan”), which consists of research participants and patients clinically referred for assessment of ASD (16). All groups have been previously described in detail (4, 15, 16). Institutional review board approval was obtained at each site, and written informed consent was obtained from the participants’ legal guardians.
DSM-IV Diagnostic Confirmation
All study participants had previously undergone diagnostic testing that included the Autism Diagnostic Interview–Revised, the Autism Diagnostic Observation Schedule, and cognitive or developmental testing. Clinical best-estimate diagnoses were determined by experienced clinicians (e.g., psychologists, psychiatrists) on the basis of all available information from the parent interview and child assessment.
Operationalizing of DSM Criteria
For the study analyses, we relied primarily on the Autism Diagnostic Interview–Revised, a 96-item parent report measure. It includes items assessing current and past behaviors and covers a wide range of ASD-related impairments (e.g., use of idiosyncratic language).We also used the Autism Diagnostic Observation Schedule, a clinician-based measure of ASD impairments. These two instruments were particularly well suited for the current study because they include items based on current behavior and they take into account developmental level in their design. This is consistent with the DSM-5 criteria, which operationalize symptoms differently for individuals of different ages in order to account for the effect of development on ASD symptoms (17–20).
As a first step in our analyses, items from the Autism Diagnostic Interview and the Autism Diagnostic Observation Schedule were mapped onto the DSM-5 criteria. Before the items were assigned to each criterion, the subjects were divided into age by language groups. Groups for children under the age of 4 and children over the age of 10 were created to be consistent with the age-based routing rules of the Autism Diagnostic Interview. Children were assigned to language groups depending on which module of the Autism Diagnostic Observation Schedule was administered. After consensus was reached among all study authors about item assignments for the DSM-5 criteria, this process was repeated for DSM-IV criteria (item assignments are available in supplemental Tables S1 and S2 in the data supplement accompanying the online version of this article).
For each item included in the DSM-IV and DSM-5 item maps, a score of 1, 2, or 3 indicated the presence of a symptom, whereas a score of 0 indicated the absence of a symptom. DSM-IV and DSM-5 guidelines were then followed to determine whether each participant met or did not meet the DSM-5 criteria for ASD and the DSM-IV criteria for autistic disorder, Asperger’s disorder, and/or pervasive developmental disorder not otherwise specified (PDD-NOS). Initially we established classifications, such as meeting versus not meeting the DSM-5 ASD criteria, by extracting symptom information from only the Autism Diagnostic Interview. We then established classifications by using information from both the Autism Diagnostic Interview and the Autism Diagnostic Observation Schedule, i.e., allowing evidence of symptoms to come from parent report, direct observation, or both. Unfortunately, it was not practical to attempt to establish classifications by using only information from the Autism Diagnostic Observation Schedule, because there are no relevant items for certain subdomains (see online supplemental Tables S1 and S2). However, because there are adequate numbers of items on both instruments that assess DSM-5 criteria A1 and A2, we were able to examine sensitivity and specificity when symptoms in these domains were required from both measures.
To ensure that the creation of both the DSM-5 and DSM-IV item assignments agreed with other clinicians’ interpretations of the criteria, these were reviewed by two psychologists and one psychiatrist who were not otherwise involved in the design or execution of the current study. All have extensive experience with ASD diagnosis and the study instruments. As a result of their feedback, two items were reassigned and one item was removed from the DSM-5 criteria (for details, see online supplemental Tables S1 and S2). The majority of the study authors and the independent experts noted some overlap between DSM-5 criterion A1 and criterion A3. For example, whereas a poor-quality social overture or initiation could be considered evidence of “abnormal social approach” (A1), it could also reflect “difficulties adjusting behavior to suit different social contexts” (A3). In general, however, the group agreed that the items were easier to map onto the DSM-5 criteria than DSM-IV criteria.
Statistical Analysis
Analyses were restricted to participants ages 2 to 17 for whom Autism Diagnostic Interview and Autism Diagnostic Observation Schedule data and DSM-IV clinical diagnoses were available. All statistical analyses were run by using SPSS 17.0 (SPSS, Chicago).
Analyses examined the sensitivity and specificity of the proposed DSM-5 ASD criteria and the DSM-IV PDD criteria in the three study groups, individually and combined, and in specific subgroups of children (i.e., children with DSM-IV Asperger’s disorder or PDD-NOS diagnoses, girls, young children). For each clinical diagnosis of a PDD, McNemar’s test was used to compare the proportions of non-PDD children who, according to the clinical best-estimate diagnosis, were correctly classified by DSM-5 and by DSM-IV. Domain-specific analyses were also conducted to explore whether children who did not meet the DSM-5 criteria for ASD might meet the proposed criteria for social communication disorder.
Results
Demographic data and mean IQs for the study groups are displayed in Table 1. The participants ranged in age from 2 to 17 years, 11 months. They represented a wide range of nonverbal and verbal ability; approximately 30% across all three groups had nonverbal IQs under 70. The nonverbal IQ of the PDD group from the University of Michigan was significantly lower than the nonverbal IQs of both other groups. The majority of the participants were Caucasian and male, but the three groups had significantly different male-to-female ratios.
| Collaborative Programs of Excellence in Autism | University of Michigan Autism and Communication Disorders Center | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Characteristic | All PDDs (N=858) | Non-PDD Disorders (N=163) | All PDDs (N=1,465) | Non-PDD Disorders (N=527) | Simons Simplex Collection: All PDDs (N=2,130) | |||||
| Male-to-female ratio | 5.3:1b | 2.1:1c | 3.8:1d | 2.0:1c | 6.7:1b,d | |||||
| Mean | SD | Mean | SD | Mean | SD | Mean | SD | Mean | SD | |
| Age (years) | 6.4b,e | 3.6 | 5.8 | 3.4 | 6.6d,e | 3.5 | 7.3 | 3.7 | 9.4b,d | 3.5 |
| Verbal IQ | 68.4b | 29.0 | 74.4c | 26.2 | 66.2d | 33.9 | 84.2c | 26.7 | 77.7b,d | 31.4 |
| Nonverbal IQ | 83.3c | 26.6 | 80.3f | 24.1 | 77.0c,d | 28.8 | 86.0f | 26.2 | 84.8d | 26.1 |
| N | % | N | % | N | % | N | % | N | % | |
| Gender | ||||||||||
| Male | 639 | 84.1 | 110 | 67.5 | 1,162 | 79.3 | 353 | 67.1 | 1,854 | 87.0 |
| Female | 121 | 15.9 | 53 | 32.5 | 303 | 20.7 | 173 | 32.9 | 276 | 12.9 |
| Race | ||||||||||
| Caucasian | 721 | 86.6 | 144 | 88.3 | 621 | 80.4 | 316 | 73.8 | 1,687 | 79.2 |
| African American | 23 | 2.8 | 4 | 2.5 | 59 | 7.6 | 57 | 13.3 | 76 | 3.6 |
| Asian | 15 | 1.8 | 1 | 0.6 | 20 | 2.6 | 7 | 1.6 | 79 | 3.7 |
| Biracial or multiracial | 63 | 7.6 | 12 | 7.4 | 47 | 6.1 | 29 | 6.8 | 169 | 7.9 |
| Other | 11 | 1.3 | 2 | 1.2 | 20 | 2.6 | 15 | 3.5 | 103 | 4.6 |
| Clinical diagnosis | ||||||||||
| Autistic disorder | 780 | 90.9 | 0 | 0.0 | 975 | 66.6 | 0 | 0.0 | 1,466 | 68.8 |
| Pervasive developmental disorder not otherwise specified | 78 | 9.1 | 0 | 0.0 | 465 | 31.7 | 0 | 0.0 | 428 | 20.1 |
| Asperger’s disorder | 0 | 0.0 | 0 | 0.0 | 25 | 1.7 | 0 | 0.0 | 236 | 11.1 |
a
Full data not available for all subjects. Analyses were conducted to compare the three ASD groups with each other and to compare the two non-ASD groups (Collaborative Programs and Michigan) with each other. Comparisons were not completed for race and clinical diagnosis.
b
Simons and Collaborative Programs groups are significantly different from one another (p<0.05).
c
Michigan and Collaborative Programs groups are significantly different from one another (p<0.001).
d
Michigan and Simons groups are significantly different from one another (p<0.001).
e
Michigan and Collaborative Programs groups are significantly different from one another (p<0.01).
f
Michigan and Collaborative Programs groups are significantly different from one another (p<0.05).
DSM-IV PDD and DSM-5 ASD Classifications Compared With Best-Estimate Diagnoses
As outlined in Table 2, on the basis of parent-reported symptoms only, in children with a clinical best-estimate diagnosis of any PDD, the sensitivity of the proposed DSM-5 criteria ranged from 0.89 to 0.93 (Table 2, “ASD, One Symptom” column). On the basis of either parent report or clinical observation, DSM-5 sensitivity ranged from 0.97 to 0.99 for any PDD. In every sample, sensitivity was highest for children with a DSM-IV clinical diagnosis of autism. Not surprisingly, given that the subjects in the Simons Simplex Collection were the only group in which participants’ initial eligibility partially depended on scores from the Autism Diagnostic Observation Schedule and Autism Diagnostic Interview, sensitivity was highest in that group. Overall, the sensitivity values of the DSM-5 and DSM-IV criteria were similar (see Table 2).
| DSM-5 Criteria | DSM-IV Criteria | ||||||
|---|---|---|---|---|---|---|---|
| Data Set, Clinical Diagnosis, and Symptom Measurea | ASD, One Symptomb | ASDc | ASD, Two Symptomsd | ASD, Two Symptoms, One Less Subdomaine | Autistic Disorderf | Asperger’s Disorderg | PDD NOSh |
| Combined groups | |||||||
| Sensitivity: all PDDs combined (N=4,435–4,453) | |||||||
| Parent report | 0.91 | 0.91 | 0.97 | 0.98 | |||
| Either parent or clinical measure | 0.99 | 0.88 | 0.99 | 0.97 | 0.99 | 0.99 | |
| Both measures | 0.90 | 0.92 | 0.96 | 0.96 | |||
| Specificity: non-PDD diagnoses (N=683–690) | |||||||
| Parent report | 0.53 | 0.53 | 0.34 | 0.24 | |||
| Either parent or clinical measure | 0.33 | 0.66 | 0.42 | 0.33 | 0.19 | 0.10 | |
| Both measures | 0.63 | 0.61 | 0.50 | 0.47 | |||
| Collaborative Programs of Excellence in Autism | |||||||
| Sensitivity | |||||||
| Autism (N=741–780) | |||||||
| Parent report | 0.94 | 0.89 | 0.97 | 0.99 | |||
| Either parent or clinical measure | 0.99 | 0.85 | 0.99 | 0.98 | 0.99 | 1.00 | |
| Both measures | 0.93 | 0.93 | 0.97 | 0.97 | |||
| PDD or Asperger’s disorder (N=77–78) | |||||||
| Parent report | 0.76 | 0.69 | 0.86 | 0.92 | |||
| Either parent or clinical measure | 0.94 | 0.51 | 0.92 | 0.84 | 0.94 | 0.99 | |
| Both measures | 0.69 | 0.62 | 0.77 | 0.82 | |||
| All PDDs combined (N=818–858) | |||||||
| Parent report | 0.93 | 0.87 | 0.96 | 0.98 | |||
| Either parent or clinical measure | 0.99 | 0.82 | 0.98 | 0.96 | 0.99 | 0.99 | |
| Both measures | 0.91 | 0.90 | 0.95 | 0.96 | |||
| Specificity: non-PDD diagnoses (N=158–163) | |||||||
| Parent report | 0.63 | 0.72 | 0.51 | 0.36 | |||
| Either parent or clinical measure | 0.50 | 0.78 | 0.58 | 0.46 | 0.34 | 0.18 | |
| Both measures | 0.67 | 0.72 | 0.60 | 0.54 | |||
| University of Michigan Autism and Communication Disorders Center | |||||||
| Sensitivity | |||||||
| Autism (N=973–975) | |||||||
| Parent report | 0.95 | 0.95 | 0.99 | 0.99 | |||
| Either parent or clinical measure | 0.99 | 0.86 | 0.99 | 0.99 | 0.99 | 1.00 | |
| Both measures | 0.96 | 0.96 | 0.99 | 0.99 | |||
| PDD or Asperger’s disorder (N=488–490) | |||||||
| Parent report | 0.78 | 0.78 | 0.90 | 0.93 | |||
| Either parent or clinical measure | 0.93 | 0.70 | 0.94 | 0.90 | 0.98 | 0.99 | |
| Both measures | 0.72 | 0.76 | 0.85 | 0.86 | |||
| All PDDs combined (N=1,461–1,465) | |||||||
| Parent report | 0.89 | 0.89 | 0.96 | 0.97 | |||
| Either parent or clinical measure | 0.97 | 0.80 | 0.97 | 0.96 | 0.99 | 0.99 | |
| Both measures | 0.88 | 0.89 | 0.94 | 0.95 | |||
| Specificity: non-PDD diagnoses (N=525–527) | |||||||
| Parent report | 0.49 | 0.48 | 0.30 | 0.20 | |||
| Either parent or clinical measure | 0.28 | 0.62 | 0.37 | 0.29 | 0.15 | 0.07 | |
| Both measures | 0.62 | 0.57 | 0.47 | 0.44 | |||
| Simons Simplex Collection: sensitivity | |||||||
| Autism (N=1,465–1,466) | |||||||
| Parent report | 0.93 | 0.95 | 0.99 | 0.99 | |||
| Either parent or clinical measure | 0.99 | 0.97 | 0.99 | 0.99 | 0.99 | 1.00 | |
| Both measures | 0.93 | 0.96 | 0.99 | 0.99 | |||
| PDD (N=428) | |||||||
| Parent report | 0.83 | 0.88 | 0.92 | 0.96 | |||
| Either parent or clinical measure | 0.99 | 0.87 | 0.99 | 0.98 | 0.99 | 1.00 | |
| Both measures | 0.80 | 0.89 | 0.94 | 0.95 | |||
| Asperger’s disorder (N=233–236) | |||||||
| Parent report | 0.94 | 0.93 | 0.99 | 0.98 | |||
| Either parent or clinical measure | 1.00 | 0.95 | 0.99 | 0.97 | 1.00 | 1.00 | |
| Both measures | 0.92 | 0.92 | 0.97 | 0.97 | |||
| All PDDs combined (N=2,126–2,130) | |||||||
| Parent report | 0.91 | 0.93 | 0.97 | 0.98 | |||
| Either parent or clinical measure | 0.99 | 0.95 | 0.99 | 0.99 | 0.99 | 1.00 | |
| Both measures | 0.90 | 0.94 | 0.98 | 0.98 | |||
a
Parent report, Autism Diagnostic Interview–Revised. Clinician-based measure, Autism Diagnostic Observation Schedule.
b
At least one symptom from each A subdomain and at least one symptom from two or more B subdomains.
c
At least one symptom on both measures from A1 and A2, at least one symptom on either measure from A3, and at least one symptom on either measure from two or more B subdomains.
d
At least two symptoms from each A subdomain and at least two symptoms from two or more B subdomains.
e
At least two symptoms from each A subdomain and at least two symptoms from one or more B subdomains; or at least two symptoms from two or more A subdomains and at least two symptoms from two or more B subdomains (see reference 11).
f
Collaborative Programs and University of Michigan: at least one symptom from two or more A subdomains, at least one symptom from one or more B subdomains, at least one symptom from one or more C subdomains, and at least one parent-reported symptom in six or more subdomains across A, B, and C. Simons: at least one symptom on both measures from two or more A subdomains, at least one symptom on both measures from one or more B subdomains, at least one symptom on either measure from one or more C subdomains, and at least one symptom on either measure in six or more subdomains across A, B, and C.
g
NOS, not otherwise specified. Collaborative Programs and University of Michigan: at least one symptom from two or more A subdomains and at least one symptom from one or more C subdomains. Simons: at least one symptom on both measures from two or more A subdomains and at least one symptom on either measure from one or more C subdomains.
h
Collaborative Programs and University of Michigan: at least one symptom from two or more A subdomains and at least one symptom from one or more B or C subdomains. Simons: At least one symptom on both measures from two or more A subdomains and at least one symptom on both measures from one or more B subdomains; or at least one symptom on both measures from two or more A subdomains and at least one symptom on either measure from one or more C subdomains.
Sensitivity was also examined within the separate PDD diagnostic groups. DSM-5 sensitivity based on parent report only in the subjects with a clinical diagnosis of Asperger’s disorder or PDD-NOS ranged from 0.76 to 0.94 (see Table 2), while DSM-5 sensitivity in those with autism ranged from 0.93 to 0.95. The sensitivity of the DSM-5 criteria was also examined within PDD phenotypic subgroups based on sex, IQ, and age. As shown in Table 3, the sensitivity for girls ranged from 0.88 to 0.93. For those in the “high-functioning” range of cognitive ability (nonverbal IQ>70), DSM-5 sensitivity was between 0.86 and 0.91, while among those with a nonverbal IQ of ≤70, sensitivity ranged between 0.93 and 0.97. In children under 4, sensitivity ranged between 0.90 and 0.98.
| Collaborative Programs of Excellence in Autism | University of Michigan Autism and Communication Disorders Center | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| PDDs | Non-PDD Disorders | PDDs | Non-PDD Disorders | Simons Simplex Collection: All PDDs | ||||||
| Phenotype | N | Sensitivity | N | Specificity | N | Sensitivity | N | Specificity | N | Sensitivity |
| Male | 633 | 0.92 | 110 | 0.56 | 1,158 | 0.89 | 350 | 0.48 | 1,850 | 0.91 |
| Female | 121 | 0.91 | 53 | 0.76 | 303 | 0.88 | 173 | 0.51 | 276 | 0.93 |
| Over age 10 with fluent language | 147 | 0.90 | 26 | 0.77 | 194 | 0.81 | 99 | 0.57 | 622 | 0.89 |
| Under age 4 | 306 | 0.98 | 74 | 0.53 | 396 | 0.90 | 117 | 0.40 | 0 | |
| Nonverbal IQ >70 | 571 | 0.91 | 103 | 0.68 | 843 | 0.86 | 374 | 0.54 | 1,584 | 0.90 |
| Nonverbal IQ ≤70 | 280 | 0.97 | 60 | 0.53 | 574 | 0.93 | 140 | 0.37 | 542 | 0.95 |
a
At least one parent-reported symptom (on the Autism Diagnostic Index–Revised) from each A subdomain and at least one parent-reported symptom from two or more B subdomains.
Table 2 includes specificity values for the Collaborative Programs and University of Michigan study groups (the Simons Simplex Collection was restricted to children with PDDs). In the Collaborative Programs group, on the basis of parent-reported items DSM-IV specificity was as high as 0.72 for the autistic disorder criteria and as low as 0.36 for the PDD-NOS criteria. In the University of Michigan group, DSM-IV specificity was 0.20 for PDD-NOS, 0.30 for Asperger’s disorder, and 0.48 for autistic disorder. In contrast, when the DSM-5 ASD criteria were applied, specificity was 0.49 in the University of Michigan group and 0.63 in the Collaborative Programs group.
When evidence of impairments in social reciprocity and nonverbal behavior was required from both parent report and clinical observation, the specificity of the DSM-5 criteria improved (Table 2). This improvement was most clinically meaningful in the University of Michigan subjects, of whom approximately 26% had non-PDD diagnoses. In this group, DSM-5 specificity increased to 0.62. Specificity in the Collaborative Programs sample increased to 0.69 with the requirement that symptoms be evident on both instruments. On the other hand, this requirement led to a decrease in sensitivity across all groups, most strikingly for children with clinical diagnoses of PDD-NOS or Asperger’s disorder (see Table 2). As in the study by Frazier et al. (11), requiring one less subdomain from either domain, with either the parent or clinical report, provided the best balance of sensitivity and specificity, although specificity remained low.
McNemar’s chi-square tests were used to investigate whether DSM-5’s proportion of correct classification of the non-PDD cases was significantly different than that of DSM-IV. When parent-reported items were used, the proportion of individuals with non-PDD diagnoses who were correctly classified by DSM-5 but misclassified by DSM-IV as having PDD-NOS was significantly higher (χ2=117.55, df=1, p<0.001) than the proportion who were misclassified by DSM-5 and accurately classified by DSM-IV (34.9% versus 5.9%). Misclassification by DSM-IV of a non-PDD condition as Asperger’s disorder was also significantly higher (χ2=161.54, df=1, p<0.001) than misclassification by DSM-5 (29% versus 11%).
DSM-5 Domains
The DSM-5 domains were examined individually to assess how many children might meet the criteria for social communication disorder and to better understand why some were misclassified in relation to their clinical diagnoses (see Figure 1). In the Simons Simplex Collection (N=2,130), eight subjects who had clinical diagnoses of PDDs failed to meet the DSM-5 criteria for ASD because they did not exhibit enough symptoms in the restricted and repetitive behavior domain, and 178 did not meet the criteria because they did not exhibit enough symptoms in the social communication domain. Similarly, in the Collaborative Programs group (N=1,021); 14 subjects did not meet the criteria in the DSM-5 restricted and repetitive behavior domain, while 45 did not meet the DSM-5 criteria in the social communication domain. In the University of Michigan group (N=1,992), 53 children did not meet the DSM-5 criteria for restricted and repetitive behavior, while 97 did not meet the social communication criteria. In total, 75 of 5,143 subjects met the criteria in the social communication domain only.
FIGURE 1. Percentage of Children in Three Data Sets With DSM-IV Clinical Diagnoses of Pervasive Developmental Disorders (PDDs) or Other Disorders Who Met Proposed DSM-5 and Domain Criteria for Autism Spectrum Disordera
a Percents based on available data.
Discussion
In this study we explored the proposed DSM-5 criteria for ASD in three groups of children with clinical DSM-IV PDD or non-PDD diagnoses. The majority of children with clinical diagnoses of PDDs met the DSM-5 ASD criteria according to item scores on the Autism Diagnostic Interview–Revised and the Autism Diagnostic Observation Schedule. Notably, application of the DSM-5 criteria demonstrated adequate sensitivity across all groups, as well as in phenotypic subgroups, including young children, girls, and children denoted as cognitively “higher functioning.” These results, together with those of Frazier et al. (11), provide evidence that the proposed criteria would likely be able to correctly classify a phenotypically wide range of children with ASD. What is more, the results of the current study indicate that the specificity of the DSM-5 criteria is improved when compared with the DSM-IV criteria for Asperger’s disorder and PDD-NOS. Overall, the accuracy of nonspectrum classification made by DSM-5 was better than that of DSM-IV. Thus, although there is much room for improvement with respect to specificity, the proposed criteria appear to meet the stated goal of the DSM-5 committee to create criteria that better distinguish ASD from nonspectrum disorders such as language disorders, intellectual disability, attention deficit hyperactivity disorder, and anxiety disorders. Our results further indicate that requiring evidence of clinician-observed social communication deficits, in addition to parent-reported deficits, can increase the specificity of the proposed criteria. However, the inevitable tradeoff between specificity and sensitivity occurred when evidence was required from both parent report and direct observation.
Given concerns that the restricted and repetitive behavior requirement might lead to reduced identification of children previously diagnosed with ASDs under DSM-IV and possible misclassification as social communication disorder, we examined why some children with PDDs did not meet the DSM-5 criteria. In all three groups, most children who did not meet the criteria did so because they did not demonstrate the required impairments in social and communication functioning, and not because they did not meet the restricted and repetitive behavior criteria. In fact, few children did not meet the restricted and repetitive behavior requirement in DSM-5. These results suggest that few children with ASDs are likely to be misclassified as having social communication disorder and lend further support to the addition of the restricted and repetitive behavior criterion.
Finally, the process of matching individual symptom items to the diagnostic criteria revealed potential challenges in the interpretation of DSM-5 criterion A3. In addition to the reduced number of items (especially on the Autism Diagnostic Observation Schedule) that could be applied to A3, it was also sometimes difficult to determine whether an item should be placed in A3 (“difficulties adjusting behavior to suit different social contexts”) or A1 (“abnormal social approach”). Although the difficulty in assigning specific items may have partially resulted from the fact that the study measures were based on the DSM-IV criteria and therefore not designed to map directly onto the DSM-5 criteria, it will be critical to ensure that the final wording of the DSM-5 criteria lends itself to being clearly and reliably interpreted by ASD diagnosticians.
Limitations
Replication of our findings in other study groups (including adults), by means of both retrospective data analysis and prospective field trial methods, is needed. Two of our study groups may not be fully representative of children typically referred for assessment of ASDs. Our groups may represent extremes in terms of ASD phenotypes: on the one hand, clinical cases at the University of Michigan with complex presentations, and on the other, clearer cases of ASDs in the Simons Simplex Collection.
The results obtained here may not reflect the proposed criteria’s true sensitivity and specificity. Using archival data and symptom counts is not comparable to clinical diagnosis. As the study instruments were largely based on the DSM-IV criteria, it is likely that behaviors that might fit into the DSM-5 criteria are not currently captured by these methods. In spite of the breadth of the Autism Diagnostic Interview, analyses of existing data cannot begin to approximate a field trial. Conducting evaluations in real time and making determinations about whether a child meets the DSM-5 criteria on the basis of all information gathered during that evaluation is the only way to assess the true sensitivity and specificity of the DSM-5 criteria. Nevertheless, though in practice it would be inappropriate to make diagnoses solely on the basis of symptom counts, our use of these methods allows comparisons with other researchers’ analyses of the DSM-5 criteria.
Conclusions
To our knowledge, this study is the most comprehensive assessment to date of the newly proposed DSM-5 ASD criteria. Based on symptom extraction from previously collected data, our findings indicate that the majority of children with DSM-IV PDD diagnoses would continue to be eligible for an ASD diagnosis under DSM-5. Additionally, these results further suggest that the revisions to the criteria, when applied to records of children with non-PDD diagnoses, yield fewer misclassifications. Our findings also contribute to literature that supports the use of both parent report and clinical observation for optimal classification accuracy.
Acknowledgments
The authors thank Mandy Steiman, Ph.D., Stephen Kanne, Ph.D., and Edwin Cook, M.D.; Nicole Saghy and Carrie Thomas, who helped prepare the manuscript; the families who participated in the three studies; and the SFARI Simplex Collection sites and investigators.
The principal investigators of the Simons Simplex Collection were A. Beaudet, R. Bernier, J. Constantino, E. Cook, E. Fombonne, D. Geschwind, D. Grice, A. Klin, D. Ledbetter, C. Lord, C. Martin, D. Martin, R. Maxim, J. Miles, O. Ousley, B. Peterson, J. Piggot, C. Saulnier, M. State, W. Stone, J. Sutcliffe, C. Walsh, and E. Wijsman. Approved researchers can obtain a subset of the data described in this study at the Simons Foundation Autism Research Initiative (SFARI) web site (https://base.sfari.org).
Supplementary Material
Supplementary Material (1056_ds001.pdf)
- View/Download
- 105.24 KB
References
1.
Lord C, Petkova E, Hus V, Gan W, Lu F, Martin DM, Ousley O, Guy L, Bernier R, Gerdts J, Algermissen M, Whitaker A, Sutcliffe JS, Warren Z, Klin A, Saulnier C, Hanson E, Hundley R, Piggot J, Fombonne E, Steiman M, Miles J, Kanne SM, Goin-Kochel RP, Peters SU, Cook EH, Guter S, Tjernagel J, Green-Snyder LA, Bishop S, Esler A, Gotham K, Luyster R, Miller F, Olson J, Richler J, Risi S: A multisite study of the clinical diagnosis of different autism spectrum disorders. Arch Gen Psychiatry 2012; 69:306–313
2.
Billstedt E, Gillberg IC, Gillberg C: Autism in adults: symptom patterns and early childhood predictors: use of the DISCO in a community sample followed from childhood. J Child Psychol Psychiatry 2007; 48:1102–1110
3.
Snow AV, Lecavalier L: Sensitivity and specificity of the Modified Checklist for Autism in Toddlers and the Social Communication Questionnaire in preschoolers suspected of having pervasive developmental disorders. Autism 2008; 12:627–644
4.
Gotham K, Risi S, Dawson G, Tager-Flusberg H, Joseph R, Carter A, Hepburn S, McMahon W, Rodier P, Hyman SL, Sigman M, Rogers S, Landa R, Spence MA, Osann K, Flodman P, Volkmar F, Hollander E, Buxbaum J, Pickles A, Lord C: A replication of the Autism Diagnostic Observation Schedule (ADOS) revised algorithms. J Am Acad Child Adolesc Psychiatry 2008; 47:642–651
5.
Frazier TW, Youngstrom EA, Kubu CS, Sinclair L, Rezai A: Exploratory and confirmatory factor analysis of the Autism Diagnostic Interview-Revised. J Autism Dev Disord 2008; 38:474–480
6.
Mandy WP, Charman T, Skuse DH: Testing the construct validity of proposed criteria for DSM-5 autism spectrum disorder. J Am Acad Child Adolesc Psychiatry 2012; 51:41–50
7.
McPartland JC, Reichow B, Volkmar FR: Sensitivity and specificity of proposed DSM-5 diagnostic criteria for autism spectrum disorder. J Am Acad Child Adolesc Psychiatry 2012; 51:368–383
8.
Rutter M, Le Couteur A, Lord C: Autism Diagnostic Interview–Revised (ADI-R). Los Angeles, Western Psychological Services, 2003
9.
Lord C, Rutter M, DiLavore PC, Risi S: Autism Diagnostic Observation Schedule (ADOS). Los Angeles, Western Psychological Services, 1999
10.
Mattila ML, Kielinen M, Linna SL, Jussila K, Ebeling H, Bloigu R, Joseph RM, Moilanen I: Autism spectrum disorders according to DSM-IV-TR and comparison with DSM-5 draft criteria: an epidemiological study. J Am Acad Child Adolesc Psychiatry 2011; 50:583–592
11.
Frazier TW, Youngstrom EA, Speer L, Embacher R, Law P, Constantino J, Findling RL, Hardan AY, Eng C: Validation of proposed DSM-5 criteria for autism spectrum disorder. J Am Acad Child Adolesc Psychiatry 2012; 51:28–40
12.
Rutter M, Bailey A, Lord C: Social Communication Questionnaire (SCQ). Los Angeles, Western Psychological Services, 2003
13.
Constantino JN, Gruber CP: Social Responsiveness Scale (SRS). Los Angeles, Western Psychological Services, 2005
14.
Wing L, Gould J, Gillberg C: Autism spectrum disorders in the DSM-V: better or worse than the DSM-IV? Res Dev Disabil 2011; 32:768–773
15.
Fischbach GD, Lord C: The Simons Simplex Collection: a resource for identification of autism genetic risk factors. Neuron 2010; 68:192–195
16.
Gotham K, Risi S, Pickles A, Lord C: The Autism Diagnostic Observation Schedule: revised algorithms for improved diagnostic validity. J Autism Dev Disord 2007; 37:613–627
17.
Gotham K, Pickles A, Lord C: Standardizing ADOS scores for a measure of severity in autism spectrum disorders. J Autism Dev Disord 2009; 39:693–705
18.
Hus V, Pickles A, Cook EH, Risi S, Lord C: Using the Autism Diagnostic Interview—Revised to increase phenotypic homogeneity in genetic studies of autism. Biol Psychiatry 2007; 61:438–448
19.
Bishop DVM, Maybery M, Wong D, Maley A, Hallmayer J: Characteristics of the broader phenotype in autism: a study of siblings using the Children’s Communication Checklist-2. Am J Med Genet B Neuropsychiatr Genet 2006; 141B:117–122
20.
Joseph RM, Tager-Flusberg H, Lord C: Cognitive profiles and social-communicative functioning in children with autism spectrum disorder. J Child Psychol Psychiatry 2002; 43:807–821
Information & Authors
Information
Published In
History
Received: 27 February 2012
Revision received: 6 May 2012
Accepted: 24 May 2012
Published online: 1 October 2012
Published in print: October 2012
Authors
Funding Information
Dr. Lord reports receiving royalties for the Autism Diagnostic Interview–Revised and the Autism Diagnostic Observation Schedule and donating all royalties to nonprofit autism charities; Dr. Lord is a member of the DSM-5 Neurodevelopmental Disorders Work Group. The other authors report no financial relationships with commercial interests.
Supplementary Material
Supported by NIMH grants R01 MH-081873 and RC1 MH-089721 to Dr. Lord, grant R01 HD-065277 from the National Institute of Child Health and Human Development to Dr. Bishop, and graduate fellowships from the Simons Foundation and Autism Speaks and a Dennis Weatherstone Predoctoral Fellowship to Ms. Hus.Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBGet Access
Login options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).