The Highs and Lows of Counting Autism

The article by Kim et al. (1) in this issue is certain to attract widespread interest, comment, and debate among the clinical and academic psychiatry communities but also across the media and public more generally. The reason for this high level of interest will be the same in both communities—the remarkable new high for a prevalence figure for autism spectrum disorders (ASDs) of 2.64% in mid-childhood (and as high as 3.74% in boys). However, I am confident that the content of the debate within the psychiatric community will differ somewhat from that within the media. Kim and colleagues' article is the latest of several prevalence studies published over the past 5 years to show that the prevalence of ASDs broadly defined is greater than 1% (2) and may even approach 2% (3); so in this sense it is just another turn of the ratchet.

Much of the public debate, particularly in the United States, has been discussion of what has caused the “epidemic of autism.” Various environmental causes have been suggested, often without an accompanying scientific evidence base—most notably thimerosal-containing vaccines (4) and, in the United Kingdom, the measles, mumps, and rubella (MMR) vaccination (5). There may have been a real rise in autism prevalence, and (unknown) environmental factors may have played a role in this rise, but, put simply, we do not know that for sure. In the scientific literature, the possible contributory factors that might explain higher measured prevalence in more recent epidemiological studies have been well rehearsed, and some of them are summarized in the article. In brief, over time the diagnostic criteria have broadened, the diagnosis of ASDs has been applied to children previously excluded (those with severe intellectual disability, above-average intelligence, genetic syndromes, or sensory impairments), and prevalence study designs have improved and therefore capture more cases. One reason why the question “Has there been a real increase in autism prevalence?” remains unanswered is that we cannot quantify whether these factors account for all of the apparent increase in measured prevalence over time. However, do these methodological factors help us understand the high prevalence figure found by Kim et al., and do they help us answer the question of why it's so hard to count autism?

The other remarkable feature of the article is that this is a report of a prevalence study from South Korea. There are no known genetic, biological, or environmental factors that would lead us to expect a higher prevalence of ASDs in Southeast Asia than in North America and Europe, where the majority of studies have been conducted to date. However, with respect to the methodological factors that we know can systematically affect prevalence estimates, are there reasons to doubt the study's finding? In other words, is 2.64% an overestimate?

The authors detail in an online data supplement the care they took in the translation and back-translation of the screening and diagnostic instruments, the cross-checking of cases between the Korean team and North American experts on a random sample of cases, the training and experience of the research team, and the procedures put in place for quality control and reliability. As with many psychiatric disorders, diagnosis of ASDs is an uncertain art, and in the end one has to make a clinical judgment of whether the number, quality, severity, and accompanying impairment of symptoms are sufficient to fulfill DSM criteria. With respect to the diagnostic component of the study, which is critical to counting autism, the authors have paid close attention to avoid bias.

It is instructive in an autism prevalence study to look at the ratios of the prevalence of the core disorder (autistic disorder in DSM-IV; childhood autism in ICD-10) and of other ASDs (all the other pervasive developmental disorder subcategories). In the Kim et al. study, the figures are 0.94% and 1.70% (a ratio of 1:1.8) for autistic disorder and other ASDs, respectively. In our own study (2), we found prevalence estimates of 0.39% and 0.77% (a ratio of 1:2) for childhood autism and other ASDs, respectively. However, we also derived a “narrow autism” category whereby children had to meet ICD criteria for childhood autism and the established thresholds for autism on the Autism Diagnostic Interview and the Autism Diagnostic Observation Schedule, which were also used by Kim et al. Using these criteria, prevalence estimates were 0.25% and 0.77% (1:3.1) for the two categories, respectively. For those of us interested in autism prevalence, such figures are important because they remind us of how estimates vary within the same study depending on the threshold set for “caseness”—when other design issues such as sampling, instruments, and participation are invariant. In our study, the prevalence estimate varied by more than a factor of 4 from 0.25% for narrow autism to 1.16% for all ASDs. The narrow autism group had a lower mean IQ as well as more severe symptoms compared to the other ASDs group. However, while the notion that there might be true phenotypic subgroups on the autism spectrum remains attractive, none has been securely identified to date (6).

There is also recent evidence that different expert groups apply the DSM subclassifications in the diagnostic systems very differently (7). One reason why counting autism is so difficult is that when you set a high threshold for who has autism, the count is lower, and when you set a low threshold, the count is higher. There is also some evidence that culture drift in our understanding of what constitutes autism has changed over the decades in ways that are hard to quantify and study (8).

The proposed revision to the diagnostic classification in DSM-5 (www.dsm5.org/ProposedRevisions/Pages/Default.aspx) removes the subclassifications that existed in DSM-IV in favor of an all-encompassing classification of autism spectrum disorder. While the notion of a spectrum or dimensional approach to autism has been generally welcomed and reflects the biological and phenotypic heterogeneity of what have been termed “the autisms” (9), there is an urgent need to establish the reliability of clinical judgment of where the lower threshold of the spectrum is (10).

Another critical aspect of design that can lead to an overestimate of the prevalence rate is that of sample participation. In the Kim et al. study, opt-in at the early stages of recruitment in terms of schools, parents completing the screen, and agreement to assessment were relatively low. The authors describe how they examined whether measured child and school factors, such as grade, sex, and school size, affected participation at each stage, and broadly they did not. However, there are a host of possible unmeasured factors that might have influenced participation in such a way as to introduce bias—in the direction of overestimating prevalence. These include school and parent knowledge and interest in participating in an autism study and parental concerns about their child's development and behavior. Such factors are very difficult to measure in a screening-sampling-diagnostic epidemiological design, but they matter greatly. For example, in the Kim et al. study, the estimated ASD prevalence of 2.64% is based on approximately 1,460 children with ASDs from the sample of 55,266, but this is estimated from direct clinical assessment of only 201 children. The measured participation variables are broadly captured in the confidence intervals around the prevalence estimates, but the unmeasured variables are not.

Finally, conducting such a large and ambitious prevalence study in a country like South Korea will likely have cultural ramifications into the future. Kim et al. discuss the stigma in some aspects of South Korean culture associated with genetic conditions such as autism. The authors say that most ASD cases that met diagnostic criteria in their research study were “undiagnosed and unrecognized.” It seems very likely that this is as true for many other neurodevelopmental and neuropsychiatric conditions in this community. By ages 7–12 years, only 294 children were in the disability register from the total sample of 55,266—about one-half of one percent. This is an identification of disability very much lower than in many Western societies. A final intriguing and perhaps more hopeful point is made in Kim and colleagues' discussion of how it is that so many pupils with an (undiagnosed) ASD can apparently manage to be educated and integrated in mainstream classrooms. This is a useful reminder that developmental conditions such as autism are affected by the environment in which a child develops and that accommodations to such environments can sometimes have a positive impact on children with a “disorder.”

Within any community, it is important to recognize children with developmental problems in order to provide the best treatment, support to families, and advice to educators. Perhaps the most remarkable figure in the Kim et al. article, one not directly highlighted by the authors, is the disparity between the half percent of children recognized by midchildhood by the community health and education services as having any disability and the two-and-a-half percent identified as having an ASD in the research study. From an external perspective, I suspect that many readers will think that the former figure is an underestimate of need. The outstanding question, particularly because counting autism is very difficult to do and to validate, is whether the latter figure is an accurate estimate of need in South Korea or indeed in other communities.