Effect of Treatment Modality on Long-Term Outcomes in Attention-Deficit/Hyperactivity Disorder: A Systematic Review

An extensive and systematic search was conducted of 12 literature databases to identify peer-reviewed, primary studies that reported long-term outcomes of individuals with ADHD. The databases were Academic Search Premier, CINAHL, Cochrane CRCT (including EMBASE), Criminal Justice Abstracts, ERIC, MEDLINE, Military & Government collection, NHS Economic Evaluation database, PsycARTICLES, PsycINFO, SocINDEX, and Teacher Reference Center. All 12 databases were searched in two search waves conducted in May 2010 and March 2011. A third search of 9 databases was conducted in March 2012. Three databases were not utilized in the third search (Academic Search Premier, NHS Economic Evaluation database, and PsycARTICLES) because no unique useful citations had been derived from those databases in the first searches. All other methods were held constant between the 3 search waves (see S1 Appendix for search string and limits). Search limits included English-language and publication date from January 1980 through December 2011 inclusive. Duplicates were eliminated electronically and manually. Based primarily on title and abstract, these studies were reviewed manually and inclusion was agreed on by two researchers. All disagreements between researchers on study inclusion were resolved by examining the full text of the study. Inclusion required that studies be published as peer-reviewed, primary research articles in the English language with full text available. Inclusion also required that the study had a comparator group (e.g., individuals with untreated ADHD) or a comparison measure (e.g., pre-treatment baseline), and that ADHD was a primary disorder under study. Treatments included pharmacological, non-pharmacological, or combination treatments/interventions intended for treatment of ADHD. Only studies reporting long-term outcomes of 2 years or more (follow-up time, not necessarily treatment duration) were included. The 2-year long-term outcome criterion could be met by longitudinal studies with prospective follow-up measurements ≥2 years, retrospective studies with a time period ≥2 years, cross-sectional studies comparing two ages differing by 2 years or more, or cross-sectional studies of individuals age 10 years or older. Age 10 was chosen as the minimum age for singleage, cross-sectional studies, based on the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision (DSM-IV-TR) diagnosis criteria that symptoms be present before age 7 years, thus conservatively allowing at least 2 years to pass before outcomes were measured. Meta-analyses, case studies, and literature reviews were excluded. A checklist of Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) is provided in S2 Appendix and a PRISMA flow diagram of the study selection process [22] is provided in Fig. 1.

Reported data from each study was manually extracted from the full text of the study to a database, including: (1) study location, (2) study sample size, (3) study length (e.g., years to follow-up), (4) participant sex and age range, (5) diagnostic criteria or symptom assessment, (6) study type (longitudinal, cross-sectional, prospective, retrospective, randomized), (7) outcome results, (8) comparator type, (9) treatment type, (10) treatment initiation age, and (11) treatment duration.

All reported long-term outcomes were organized into 9 main categories/domains based on common characteristics: 1) academic (e.g., achievement test scores, grades, length of education, repeated grades, education level), 2) antisocial behavior (e.g., school expulsion, delinquency, police contacts, arrests, convictions, incarceration, self-reported crimes, types or severity of offenses, age at first incident, repeat convictions), 3) driving (e.g., traffic violations, automobile accidents, license status, driving simulation rating), 4) non-medicinal drug use/addictive behavior (e.g., substance use, abuse, and/or dependence—from caffeine to illicit drugs; age at initiation; quitting substance use; amount of substance used; non-substance addictions such as gambling), 5) obesity (body mass index, weight), 6) occupation (e.g., employment, military service, income/debt, job performance, job loss/changes, occupation level, socioeconomic status), 7) services use (e.g., school services, health services, emergency room visits, work-related services, financial assistance, justice system), 8) self-esteem (self-esteem questionnaires, suicide ideation, suicide attempts, suicide rate), and 9) social function (e.g., peer, family, and romantic relationships; peer nomination scores; marital status; divorce rate, social skills, living arrangements, activities/hobbies). Note that ADHD symptoms were not considered to be a type of long-term outcome.

Except in the analysis of individual effect sizes, the individual measures reported in each study were summarized as a single “improved” or “no-benefit” outcome for each outcome category. Clinical significance was rarely reported, so an outcome was considered “improved” with treatment when a measure was reported to have statistically significant improvement associated with treatment compared with individuals with untreated ADHD or pre-treatment baseline. An outcome was considered to have “no benefit” with treatment when measures were either poorer or not statistically significantly different compared with pre-treatment baseline or individuals with untreated ADHD. Proportions of improved and no-benefit outcomes reported were then compared non-statistically. A risk of Type 2 error must be acknowledged for some studies (e.g., some differences may not have reached statistical significance due to small sample size).

The effect sizes of improved outcomes associated with pharmacological, non-pharmacological, or combination treatment were compared through analysis of measures that were reported to be statistically significantly improved with treatment, and either the effect size was reported or sufficient information was reported so that effect size could be calculated. Cohen’s d was derived from continuous measures for which results were reported as means (± standard deviation), such as scores on a math achievement test. Cohen’s w was derived from discrete measures of incidence, such as number of people who smoked cigarettes last month, often derived from the related reported χ² (chi squared) statistic. Cohen’s f was derived from results of analysis of variance (ANOVA) or regression analyses reported as the η² or η (eta squared or eta) statistic, such as the likelihood of cocaine use related to stimulant medication history controlling for the presence of conduct disorder. Effect size derivation was according to methods in Cohen, 1988 [23]: The difference in means divided by the pooled standard deviation of the means for Cohen’s d, square root of reported χ² divided by the study sample or from a 2×2 contingency table for Cohen’s w, and Cohen’s f was converted from reported η² (or η) using conversion tables in Cohen, 1988 [23]. Effect sizes were evaluated according to Cohen’s conventional criteria for small, medium, and large effect sizes (Cohen’s d: 0.2, 0.5, 0.8; Cohen’s f: 0.1, 0.25, 0.4; Cohen’s w: 0.1, 0.3, 0.5, respectively).

For each of the nine outcome categories, the proportion of studies of each treatment modality (pharmacological, non-pharmacological, and combination treatment) that reported significant improvement was analyzed. The effect of treatment duration was examined by dividing studies into those with shorter treatment duration (mid-range treatment of individuals ≤2 years) or longer treatment duration (mid-range >2 years) and determining the proportion of studies demonstrating improvement or no benefit on outcomes with treatments from either group. Treatment duration of 2 years was chosen for this analysis to correspond with several previous studies of treatment duration of 2 years or less [24–31]. Effect of participant age at initiation of treatment was analyzed by dividing studies into three groups according to mid-range age at initiation of treatment (<8 years old, 8–12 years old, and ≥13 years old) and determining the proportion of studies demonstrating benefit or no benefit for each initiation age category. These age category divisions were chosen to correspond with the minimum age of adolescence used in this analysis for other age categorizations (13 years) and to provide a sample of the youngest age-of-initiation studies available in this dataset (<8 years old). The influence of several aspects of study design and participant characteristics was also examined. Outcomes reported in studies of specific study designs (e.g., randomized treatment studies) were examined. The effect of total study length (i.e., time to follow-up measurement <3 years versus ≥3 years) was examined. The effect of participant sex and age at follow-up were examined. Age at follow-up was also analyzed for each outcome domain by categorizing study groups by mid-range age at the longest follow-up time as children, (6–12 years old), adolescents (13–17 years old), or adults (18–64 years old), regardless of age at treatment initiation or treatment duration.

There were 51 studies (111 outcomes total; see S3 Appendix for complete list of citations) that reported outcomes associated with ADHD treatment in comparison with no treatment for ADHD (either pre-treatment baseline or an untreated group with ADHD). There were more outcomes than studies because many studies reported more than one outcome. The effect of different treatment modalities (pharmacological, non-pharmacological, and combination treatment) was examined (see S4 Appendix for a complete list of specific treatments included in the studies). Table 1 shows the proportion of outcomes reported to improve significantly or not with the three different ADHD treatment modalities. The majority of the reported outcomes (56%, 62/111) were from studies of the effects of pharmacological treatment, and 50% (31/62) of these outcomes were reported to exhibit significant improvement. For non-pharmacological treatment, 65% (17/26) of the outcomes were reported to exhibit significant improvement. The highest proportion of outcomes exhibiting significant improvement was reported for combination treatment, 83% (19/23).

The degree of improvement with treatment for those outcomes that were statistically significantly improved was expressed as effect sizes that were then grouped by comparator type, treatment modality, and effect size index (d, w, or f) for comparison (Fig. 2). All effect sizes that were reported or could be derived for statistically significant improvements are included (numerical listing in S5 Appendix).

Effect sizes ranged from below the small threshold to above the large threshold (Fig. 2A,B). All but one of the effect sizes falling above the large threshold were for outcomes within the academic, self-esteem, or social function domains, suggesting that these outcomes exhibited the greatest improvements associated with treatment. Note that these effect sizes are only for outcomes reported to be statistically significantly improved.

Separating the results by type of outcome allowed examination of which treatment types were most commonly associated with improvement for each outcome (Fig. 3A). The proportion of outcomes that improved varied among treatment modalities, as did the number of available supporting studies and outcomes. Of the outcomes most amenable to treatment, all treatment modalities were reported to be associated with improvement. For example, pharmacological, non-pharmacological, and combination treatments all contributed to the 100% of studies reporting improved outcomes for driving. Similarly, a high percentage of beneficial results for social function outcomes were reported with all three treatment modalities: pharmacological (67%, 6/9), non-pharmacological (83%, 5/6), and combination treatment (86%, 6/7). All treatment modalities also contributed to improved outcomes in the self-esteem, academic, and antisocial behavior domains.

The number of studies meeting inclusion criteria was limited for some outcome domains and treatment modalities. For example, in the domain of drug use/addictive behavior outcomes, effects of pharmacological treatment represented most of the studies (81%, 17/21) with 47% (8/17) of outcomes exhibiting improvement. Non-pharmacological treatment was assessed in only 3 drug-use/addictive behavior studies and was associated with improved outcomes in only 33% (1/3) of outcomes reported. Combination treatment for drug use/addictive behavior outcomes was evaluated in one study and was not significantly associated with improvement. For self-esteem, social function, academic, and antisocial behavior outcomes, a greater proportion of outcomes following combination treatment showed improvement compared with outcomes following pharmacological or non-pharmacological treatment. In the cases of obesity, services use, and occupation outcomes, only studies with pharmacological treatment were identified, so treatment modalities could not be compared.

Considering that obesity, services use, and occupation outcomes were not reported for all three treatment modalities, we re-analyzed the overall improved outcomes of the three treatment modalities without these outcomes included. This sensitivity analysis examined outcomes measured for all three treatment modalities (102 total outcomes), to reduce possible bias of association between treatment modality and outcome type. The results were similar to those presented in Table 1 with all outcomes included, with percentages changing by 1–3 percentage points. The proportion of significantly improved pharmacological treatment outcomes increased from 50% to 51%.

The proportion of outcomes that improved with each treatment type examined in children and adolescents at follow-up were similar to results in all age groups (Fig. 3B), and these results did not support any strong age-group by treatment-modality interaction. Adults were not presented separately in this analysis because only pharmacological treatment was assessed in studies of adults. Approximately half (47–53%) of the outcomes were reported to improve with pharmacological treatment for each age group, including adults. A high percentage of outcomes were improved with combination treatment, measured only in children or adolescents (78–86%).

The age at which therapy is initiated may be an important factor influencing long-term outcomes. Of the total 51 studies reporting treatment effects compared with untreated ADHD, 40 studies reported the age of initiation of treatment, often as a range. These studies were divided into three groups according to mid-range age at initiation of treatment (<8 years old, 8 to 12 years old, and ≥13 years old).

A high proportion of outcomes (60–75%) were reported to improve with treatment regardless of age of treatment initiation (Fig. 4A). Most of the studies (80%, 32/40) described findings with age of initiation of 8 through 12 years old. Findings with this age-of-initiation group included all treatment modalities and all types of outcomes. Although there was a tendency for the proportion of beneficial outcomes to increase with age of treatment initiation, a majority of outcomes benefitted with treatment (all modalities represented) even with the younger ages of treatment initiation and encompassed outcome domains of self-esteem, social function, and academics (4 studies). Treatment benefits with the older ages of treatment initiation included obesity and drug use outcomes (4 studies). Of note, studies with younger ages of treatment initiation also had younger ages of follow up, which necessarily limited the type of outcomes measured (e.g., neither driving nor occupation) [3].

Treatment duration was reported in 40 of 51 studies comparing treated ADHD to untreated ADHD. Depending upon the study design, a range of treatment durations may be reported in one study. Of note, presence or absence of treatment and type of treatment utilized during the intervening time to follow-up were often not monitored in the studies making up this analysis; only treatment durations reported in a study were used in the present analysis. To examine the effect of treatment duration, studies were divided into two groups according to the mid-range duration of treatment (≤2 years or >2 years). Studies of both treatment-duration groups reported a high proportion of outcomes with beneficial effects: 72% (21/29) for ≤2 years treatment duration and 62% (21/34) for treatment duration >2 years (Fig. 4B). A majority (53–75%) of each of the three age-of-treatment-initiation categories were studies of shorter treatment duration (≤2 years).

The follow-up time considered here is the time from study start until when the last follow-up measurement was taken, regardless of treatment duration or time from end of treatment (lag time). The follow-up times of all 51 studies reporting treatment outcomes ranged from 2 to 13 years. To examine the influence of time-to-follow-up measurement [16,27,32–34], we categorized the studies by follow-up time (<3 years versus ≥3 years). Fig. 4C shows the proportion of outcomes demonstrating improvement or no benefit associated with treatment for each of these follow-up time categories. Both follow-up time categories were associated with a high proportion of outcome improvements, however, the percentage of outcomes reported to improve with treatment was greater with the shorter follow-up time (93%) than with the longer follow-up time (57%) as shown in 13 and 30 studies, respectively.

Considering age-of-treatment-initiation (Fig. 4A), treatment duration (Fig. 4B), and time to follow-up (Fig. 4C), the greatest difference in proportion of outcomes reported to have treatment benefit was observed with shorter versus longer time to follow-up. In a post-hoc analysis of the difference observed with shorter follow-up times, two relationships were noted. Studies with shorter treatment duration (≤2 years) also had a shorter mean time to follow-up measurement (mean 3.2 years compared with 7.1 years for studies with longer treatment duration). We also examined the possibility that the lag time (time between end of treatment and time of follow-up measurement) may explain the greater proportion of treatment benefit with shorter follow-up times. Lag time was estimated for 37 studies by subtracting the duration of treatment from the follow-up time. This estimate was made with the assumption that treatment started at the start of the study. For studies in which treatment started at some point before the study start, lag time was underestimated. The estimated lag time (mean = 0.4 years) was markedly shorter for studies with short <3 year follow-up times compared with studies with long ≥3 year follow-up times (mean lag time = 3.2 years). This indicates that studies with shorter treatment duration (and coincidentally shorter follow-up time) had follow-up assessment closer to the cessation of treatment.

A limitation of this systematic review is the inclusion of studies of widely varied characteristics, for example, different study designs, study population types and numbers, types of informant or rater, follow-up intervals, diagnostic criteria, and treatment types. In recognition of this limitation, conclusions were maintained at a general level. The purpose of this review was to include as many studies as possible that met simple, basic criteria and to provide a comprehensive summary of the included results. Nonetheless, for at least minimal criteria of quality/credibility, all studies included in the present analysis passed peer review, included a control or comparison condition, and only statistically significant improvements were counted in the present analysis as indicating benefit associated with treatment. In our assessment of the influence of study design, we found that prospective studies and RCTs showed a greater proportion of improved outcomes with treatment, compared with retrospective and cross-sectional studies. The limitations inherent to retrospective and cross-sectional studies may influence the reported outcomes. For example, self-reported historical data may be less accurate than data regarding recent or present status. Furthermore, historical data from medical and criminal records, although accurate, may not be complete (e.g., only crimes that resulted in police encounter would be in official records).

Another important limitation is that “treatment benefit” was defined as any significant improvement compared with a group of untreated individuals with ADHD or from pre-treatment baseline to follow-up, even without separate within-study untreated comparator groups. Many things aside from treatment, including maturation, history, placebo response, rater bias, practice effect on assessment instruments, and regression to the mean, may account for measurement of significant improvement from baseline to follow-up. Four follow-up studies of untreated individuals with ADHD compared with study-start baseline, however, reported a deterioration from baseline. Also, studies with a within-study untreated comparison group showed a similar rate of significant treatment-related improvement overall (60%). Similarly, for the few studies with a randomized untreated comparison group, the percentage of outcomes reported to improve significantly with treatment was 57% (4/7 outcomes). Finally, those who apparently got more treatment (combination of both pharmacological and nonpharmacological treatment) improved more than those who received apparently less treatment (only one type of treatment), suggesting that the amount of treatment makes a difference, and thus that treatment itself makes a difference.

There are many comorbidities associated with ADHD, including oppositional defiant disorder, conduct disorder, anxiety disorders (i.e., generalized anxiety disorder, obsessive-compulsive disorder, and posttraumatic stress disorder), and depression. The presence of comorbidities may influence the effect of treatment. Many of the studies in this dataset, however, either excluded individuals with particular comorbidities (24 of 51 studies) or controlled for comorbidities through methods such as regression analysis or comparison of outcomes of individuals with ADHD with/without the comorbidity (17 of 51 studies). Outcomes adjusted for comorbidities by some method were used in the present analysis whenever possible. The influence of comorbidites on the long-term outcomes associated with ADHD is a complex issue, especially in the consideration of the effects of treatment on these long-term outcomes. For example, this analysis showed that there were proportionally fewer studies of antisocial outcomes reporting benefit with treatment. There is evidence that, while poor antisocial outcomes appear to be associated with the presence of ADHD specifically [39,40], conduct disorder (a common comorbidity) also has a strong role in this association [19]. Thus, it may be expected that, for those individuals with comorbid ADHD and conduct disorder, treatment for ADHD specifically may not result in significant improvement in antisocial outcomes. Proper treatment and rehabilitation of such individuals is challenging [41]. Recent studies of cognitive-behavioral group therapy (the Reasoning and Rehabilitation ADHD program) either alone or in combination with pharmacological treatment, however, have reported promising results for improvement in ADHD symptoms and comorbidities [42,43]. A large Swedish study of individuals with ADHD found a significant reduction in criminality (32% for men, 41% for women) during periods on pharmacological treatment compared to periods without medication [44]. The population included patients with and without diagnosed comorbidities (conduct, oppositional-defiant, antisocial, personality, or substance-use disorder).

One bias that may occur in studies that report more than one outcome each, is that all the outcomes reported within the study may tend to be of the same type (poorer or improved; referred to as the “halo” effect), essentially giving greater weight to those studies reporting many outcomes. Analyses after collapsing all the outcomes into a single result for each study showed results similar to that presented here for the analyses of the untreated and treated outcomes, indicating that the “halo” effect was negligible for these analyses.

Another bias that may occur with the collapsing of several measures reported in a study into a single outcome group result for that study (improved or no benefit), is that for studies in which many measures were taken within a single outcome group, there is a higher likelihood of an improvement being observed for a single measure that then was summarized for the outcome group as a reported improvement (similar to the “multiple comparisons” issue in statistical comparisons). Analysis of the treated versus untreated outcomes utilizing only those outcomes stemming from consistently improved or consistently no significant improvement measures yeilded results (55% improvement associated with treatment) similar to those presented here utilizing all the outcomes reported, indicating that this effect was negligible for these analyses.

Publication bias is a potential limitation of all literature reviews, including this one. It can promote a more encouraging impression from published literature than may be gleaned from inclusion of all studies performed. The outcomes analyzed in the present analysis included every long-term outcome presented in each study, including outcomes that were reported but not necessarily the main focus of the study, thus minimizing publication bias.