Genetic, Clinical, and Sociodemographic Factors Associated With Stimulant Treatment Outcomes in ADHD
Publication: American Journal of Psychiatry
Abstract
Objective:
Stimulant medications are effective for treating attention deficit hyperactivity disorder (ADHD), yet discontinuation and switch to nonstimulant ADHD medications are common. This study aimed to identify genetic, clinical, and sociodemographic factors influencing stimulant treatment initiation, discontinuation, and switch to nonstimulants in individuals with ADHD.
Methods:
The authors obtained genetic and national register data for 9,133 individuals with ADHD from the Danish iPSYCH2012 sample and defined stimulant treatment initiation, discontinuation, and switch from prescriptions. For each stimulant treatment outcome, they examined associations with polygenic risk scores (PRSs) for psychiatric disorders and clinical and sociodemographic factors using survival analyses, and conducted genome-wide association studies (GWASs) and estimated single-nucleotide polymorphism heritability (h2SNP).
Results:
Eighty-one percent of the sample initiated stimulant treatment. Within 2 years, 45% discontinued stimulants and 15% switched to nonstimulants. Bipolar disorder PRS (hazard ratio=1.05, 95% CI=1.02, 1.09) and schizophrenia PRS (hazard ratio=1.07, 95% CI=1.03, 1.11) were associated with discontinuation. Depression, bipolar disorder, and schizophrenia PRSs were marginally but not significantly associated with switch (hazard ratio range, 1.05–1.07). No associations were observed for ADHD and autism PRSs. Individuals diagnosed with ADHD at age 13 or older had higher rates of stimulant initiation, discontinuation, and switch (hazard ratio range, 1.27–2.01). Psychiatric comorbidities generally reduced rates of initiation (hazard ratio range, 0.84–0.88) and increased rates of discontinuation (hazard ratio range, 1.19–1.45) and switch (hazard ratio range, 1.40–2.08). h2SNP estimates were not significantly different from zero. No GWAS hits were identified for stimulant initiation or discontinuation. A locus on chromosome 16q23.3 reached genome-wide significance for switch.
Conclusions:
The study findings suggest that individuals with ADHD with higher polygenic liability for mood and/or psychotic disorders, delayed ADHD diagnosis, and psychiatric comorbidities have a higher risk for stimulant treatment discontinuation and switch to nonstimulants. Despite the study’s limited sample size, one putative GWAS hit for switch was identified, illustrating the potential of utilizing genomics linked to prescription databases to advance ADHD pharmacogenomics.
Attention deficit hyperactivity disorder (ADHD) is a neurodevelopment disorder affecting 5%–10% of children and 2.5%–5% of adults (1). Stimulant medications, particularly methylphenidate, are the first-line recommended treatment for ADHD and have proven to be effective in reducing ADHD core symptoms in clinical trials and meta-analyses. Given lower effect sizes, nonstimulant medications (e.g., atomoxetine) are second-line treatment and are primarily prescribed to individuals with poor stimulant treatment response or tolerance (2–4). Despite the high efficacy of stimulants, many patients discontinue treatment or switch to nonstimulant ADHD medications, with the most common reasons reported being poor treatment response and adverse effects (5–7). Because stimulant treatment has been associated with positive effects on important functional outcomes (8), it is of clinical importance to identify genetic, clinical, and sociodemographic factors that influence stimulant treatment initiation, discontinuation, and switch to nonstimulant medications in ADHD.
Genetics likely contribute to stimulant treatment response and risk of adverse treatment effects, yet few genetic variants have been robustly linked to stimulant treatment outcomes in ADHD. A meta-analysis of candidate gene studies reported replicated associations of methylphenidate efficacy with variants in the ADRA2A, COMT, SLC6A2, SLC6A3, and DRD4 genes (9). However, candidate gene studies are known to be problematic, with many identified variants failing to replicate in genome-wide association studies (GWASs) (10, 11). Two small GWASs (N<200) have been conducted on methylphenidate response (12, 13), without any genome-wide significant hits identified, likely because of the limited sample sizes. Utilizing genetic data linked to individual-level electronic health records (EHRs), including prescriptions, is a promising avenue to obtain larger, representative samples for pharmacogenomic research. However, treatment outcomes are rarely reported in EHRs and must be approximated from prescriptions (14, 15). Numerous pharmacoepidemiological studies have used discontinuation and switch to nonstimulants, defined from prescriptions, as proxies of suboptimal treatment outcomes in ADHD (16–23). Nevertheless, we are not aware of any studies investigating the genetic contributions to these stimulant treatment outcomes, through GWAS, single-nucleotide polymorphism (SNP) heritability (h2SNP), or polygenic risk score (PRS) analyses.
PRSs, which capture the weighted sum of an individual’s phenotype-associated risk alleles, may be a useful component in predicting treatment outcomes (14, 15). A recent study of 214 ADHD patients found that higher ADHD PRS was associated with symptom improvement following stimulant treatment (24). Moreover, ADHD is genetically correlated with other psychiatric disorders, including autism spectrum disorder (ASD), depression, bipolar disorder, and schizophrenia (11, 25). Thus, it can be hypothesized that higher PRSs for these disorders in individuals with ADHD may also influence stimulant treatment outcomes, for example, through increased risk of adverse treatment effects. Nevertheless, given the low predictive ability of current psychiatric PRSs, identification of patients at high risk of suboptimal stimulant treatment outcomes will likely depend on combining genetic, clinical, and sociodemographic information (14). Psychiatric comorbidities can affect stimulant treatment outcomes in ADHD (2, 3); for example, comorbid ASD has been linked to more adverse treatment effects (26), and substance misuse to lower stimulant efficacy (27). There are also concerns that stimulants may induce or exacerbate tics, compulsive behaviors, anxiety, depression, or psychotic symptoms (3, 28, 29). Hence, individuals with ADHD and these comorbidities may be less likely to receive stimulants and more likely to discontinue or switch treatment. Age, sex, parental psychiatric illness, and socioeconomic status have also been linked to ADHD treatment initiation, discontinuation, and switch, albeit with mixed findings (5, 7, 23, 30–32).
In this study, we aimed first to identify PRSs for psychiatric disorders and clinical and sociodemographic factors associated with stimulant treatment initiation, discontinuation, and switch to nonstimulant medications, defined from prescriptions. Second, we explored the feasibility of conducting GWASs and estimating h2SNP for these stimulant treatment outcomes. To do so, we utilized genetic and Danish national register data in a large sample of individuals with ADHD from the iPSYCH2012 case cohort (33).
Methods
Study Population
The Lundbeck Foundation Initiative for Integrative Psychiatric Research (iPSYCH2012) case cohort was identified from all singletons born in Denmark between May 1, 1981, and December 31, 2005 (33). iPSYCH2012 consists of a random population sample of 30,000 control subjects and all individuals with a major psychiatric disorder diagnosed by December 31, 2012. This included 18,726 individuals with ADHD identified by a discharge diagnosis with ICD-10 code F90.0 (34) in the Danish Psychiatric Central Research Register, which contains inpatient care since 1969 and outpatient since 1995 (35). iPSYCH2012 is linked to Danish national registers, with data available through December 31, 2016. We restricted our analysis to individuals with ADHD diagnosed between January 1, 2005, and December 31, 2012, as ADHD prevalence and treatment have increased markedly since the early 2000s in Denmark (36), and because atomoxetine, the main nonstimulant ADHD medication, was first approved in Denmark in 2006. Genotypes in iPSYCH2012 were obtained for 78,050 samples, and 554,360 SNPs were genotyped in 23 waves on the Illumina PsychChip v1.0 array. The procedure has been described elsewhere (33). For imputation, principal component analyses, and quality control, see Schork et al. (37). Briefly, imputed best-guess genotypes were filtered on INFO score >80%, minor allele frequency >1%, Hardy-Weinberg equilibrium (p>1 × 10−6), association with genotyping wave (p>5 × 10−8), and imputation batch (p>5 ×10−8), leaving 6,361,597 autosomal SNPs. Analyses were restricted to unrelated individuals of European ancestry identified using SmartPCA (38). After phenotype and genotype exclusions, 9,133 individuals with ADHD were retained for analyses. The study population selection is described in Figure S1 in the online supplement .
Stimulant Treatment Outcomes
ADHD medication prescriptions were identified from the Danish National Prescription Registry, which includes all prescriptions redeemed at pharmacies since January 1, 1995, classified according to Anatomical Therapeutic Chemical (ATC) codes (39). We included all stimulant (methylphenidate [N06BA04], dexamfetamine [N06BA02], lisdexamfetamine [N06BA12]), and nonstimulant (atomoxetine [N06BA09], guanfacine [C02AC02]) ADHD medications approved in Denmark. Prescriptions for modafinil (N06BA07) and bupropion (N06AX12) following treatment with a licensed ADHD medication were also included, as these are sometimes used as off-label ADHD treatment (40). ADHD medication treatment before age 3 is not recommended, so prescriptions before this age were excluded (2, 3).
ADHD stimulant treatment outcomes were stimulant initiation, discontinuation, and switch to nonstimulant treatment within a 2-year observation window (Figure 1).
FIGURE 1. Flowchart depicting the 2-year observation period for medication initiation in 9,133 individuals with ADHD and the 2-year observation period for discontinuation and switch in the 7,247 individuals with ADHD who initiated stimulant treatmenta
a Stimulant treatment initiation was evaluated within 2 years of first ADHD diagnosis. Censoring due to death or emigration is reported separately for the analyses of discontinuation and switch, as these were modeled as separate, non–mutually exclusive outcomes. Individuals could switch to nonstimulant medications prior to discontinuation, to then go on to either comedicate (augmentation) or fully discontinue stimulants. Conversely, individuals could discontinue stimulants and then switch to nonstimulant ADHD medication. In total, 934 individuals had both a discontinuation (i.e., a gap >180 days between stimulant prescriptions) and a switch (i.e., a prescription of a nonstimulant ADHD medication) during follow-up. Among individuals who switched to nonstimulants, 203 (18%) augmented their stimulant treatment (i.e., switch without discontinuation during follow-up). ADHD=attention deficit hyperactivity disorder.
Initiation of stimulants was defined as the date of the first prescription for any stimulant ADHD medication. Individuals were followed from the date of their first ADHD diagnosis until the date of initiation, censoring (due to death or emigration), or end of follow-up (i.e., 730 days after first ADHD diagnosis), whichever came first. To allow for a delay between first treatment contact and diagnosis, individuals with a prescription within 6 months prior to first registered ADHD diagnosis were considered to have initiated at start of follow-up. Individuals who initiated with a nonstimulant were excluded from the main analyses (N=568, 5.6% of the eligible sample) but were contrasted to stimulant initiators in supplementary analyses (see Statistical Analysis below).
Individuals with ADHD who initiated stimulant treatment were followed from the date of their first stimulant prescription until discontinuation or switch, censoring, or end of follow-up (i.e., 730 days after first dispensed stimulant). Discontinuation of stimulants was defined as a gap between stimulant prescriptions of ≥180 days, in line with previous research (19–21). Date of discontinuation was set to 30 days after the final dispensed prescription (i.e., the median length of time between stimulant prescriptions [interquartile range, 17–51 days]), to account for consumption of the final prescription. Switch to a nonstimulant was defined as the date of the first prescription of a nonstimulant ADHD medication. Discontinuation and switch were treated as non–mutually exclusive outcomes because of the challenge of determining the exact duration of each prescription, and because individuals who discontinue may later switch to nonstimulants. For the same reasons, we did not differentiate between switch and augmentation (i.e., coprescribing of stimulant and nonstimulant ADHD medications). See Figure 1 for details.
Polygenic Risk Scores
We included PRSs for ADHD, as well as ASD, depression, bipolar disorder, and schizophrenia, based on their reported genetic correlations with ADHD (11, 41) and availability of sufficiently powered GWASs. We derived externally trained PRSs using LDPred (42), with SNP weights obtained from external GWAS summary statistics excluding iPSYCH2012 (see Table S1 in the online supplement ). We also leveraged having individual-level SNP data on a large number of individuals with ADHD, ASD, and depression in iPSYCH2012 by deriving another set of internally trained PRSs using SNP weights obtained from a best linear unbiased prediction of SNPs in the iPSYCH2012 sample (43). The final ADHD, ASD, and depression PRSs were constructed as a linear combination of the internally and externally trained PRSs. All PRSs were standardized to the mean and standard deviation for the iPSYCH2012 control subjects. For details on the derivation of PRSs, see Albiñana et al. (43) and the online supplement .
Clinical and Sociodemographic Factors
Clinical and sociodemographic factors were obtained from Danish national registers and were chosen on the basis of treatment guidelines and previous studies (3, 7). We included sex, age at first register-based ICD diagnosis for any ADHD subtype (ICD-10 codes F90.x, F98.8, to capture earliest observable diagnosis), maternal education and paternal income in birth year of the index child, parental history of any psychiatric disorder, and comorbid register-based diagnosis of ASD, intellectual disability, oppositional defiant or conduct disorder, tic disorder, obsessive-compulsive disorder (OCD), anxiety, depression, bipolar disorder, and substance use disorder. Comorbid schizophrenia was not included as there were too few cases. Data sources and definitions are outlined in the online supplement .
Statistical Analysis
Associations with PRSs and clinical and sociodemographic factors.
We used Cox proportional hazards models to estimate associations of PRSs and clinical and sociodemographic factors with each stimulant treatment outcome. Associations are expressed as hazard ratios with 95% confidence intervals. All models were adjusted for sex, age at first ADHD diagnosis (divided into five categories: 1–6, 7–9, 10–14, 15–19, and 20–32 years), and birth year (divided into five categories: 1981–1985, 1986–1990, 1991–1995, 1996–2000, and 2001–2005). PRS associations were further adjusted for genotyping wave and the first four principal components. Sex, age at first ADHD diagnosis, birth year, and parental education, income, and psychiatric history were modeled as time-fixed covariates. Psychiatric comorbidities were modeled as time-varying covariates to capture psychiatric problems emerging after start of follow-up. PRSs were modeled both as continuous covariates, with hazard ratios estimated by one-standard-deviation increase, and as quintiles, with hazard ratios estimated in each PRS quintile compared with the lowest.
We performed three supplementary analyses. For discontinuation and switch, we first ran multivariate Cox models, including each PRS separately in a model with all clinical and sociodemographic factors, to evaluate the impact of covariate adjustment on PRS associations. Second, we ran analyses stratified by age at first ADHD diagnosis to evaluate whether associations differed between children (under age 13) and adolescents and adults (age 13 and older). Finally, we ran logistic regression to evaluate differences in PRSs and clinical and sociodemographic factors between individuals with ADHD who initiated treatment with a nonstimulant (i.e., those excluded from the main analyses) and those who initiated with stimulants.
Analyses were conducted in R, version 3.6.0.
GWASs and h2SNP.
We conducted within-case GWAS and estimated h2SNP for stimulant treatment initiation, discontinuation, and switch, defined by treatment status at the end of follow-up. Stimulant initiators (N=7,427) were compared with noninitiators (N=1,706). Individuals with ADHD who discontinued (N=3,370) or switched (N=1,137) were compared with individuals who remained in stimulant treatment (i.e., no discontinuation or switch, N=3,854). GWASs were performed using BOLT-LMM (44), and h2SNP estimated using BOLT-REML (45). Analyses were adjusted for sex, birth year, age at first ADHD diagnosis, genotyping wave, and the first 10 principal components. We used FUMA for functional mapping and annotation of GWAS results (46). For details on GWAS analyses and h2SNP derivation, see the online supplement .
This study was approved by the Danish Scientific Ethics Committee, the Danish Health Data Authority, the Danish Data Protection Agency, and the Danish Newborn Screening Biobank Steering Committee. The Danish Scientific Ethics Committee, in accordance with Danish legislation, waived the need for informed consent in biomedical research for this study based on existing biobanks (33).
Results
Among 9,133 individuals with ADHD, 29% were female, and the median age at first ADHD diagnosis was 12 years (interquartile range, 8–17). Baseline descriptive statistics are presented in Table S2 in the online supplement . Within 2 years of ADHD diagnosis, 7,427 individuals (81%) had initiated stimulant treatment. Among stimulant initiators, 3,370 (45%) had discontinued stimulants and 1,137 (15%) switched to nonstimulants within 2 years of stimulant initiation.
Associations With PRSs and Clinical and Sociodemographic Factors
PRS associations expressed by standard deviations (Table 1) showed that ADHD and ASD PRSs were not associated with any stimulant treatment outcome. PRSs for bipolar disorder (hazard ratio=1.05, 95% CI=1.02, 1.09) and schizophrenia (hazard ratio=1.07, 95% CI=1.03, 1.11) were associated with discontinuation. PRSs for depression (hazard ratio=1.06, 95% CI=0.99, 1.13), bipolar disorder (hazard ratio=1.05, 95% CI=0.99, 1.12), and schizophrenia (hazard ratio=1.07, 95% CI=1.00, 1.13) were marginally but not significantly associated with switch to nonstimulants. PRS associations in the fully adjusted multivariate Cox models (see Table S3 in the online supplement ) were nearly identical to the main results.
| Initiation | Discontinuation | Switch | ||||
|---|---|---|---|---|---|---|
| Measure | Hazard Ratio | 95% CI | Hazard Ratio | 95% CI | Hazard Ratio | 95% CI |
| Polygenic risk scores | ||||||
| ADHD (per 1 SD) | 1.02 | 0.99, 1.04 | 0.99 | 0.96, 1.03 | 1.01 | 0.95, 1.07 |
| ASD (per 1 SD) | 0.99 | 0.97, 1.02 | 1.02 | 0.99, 1.05 | 1.00 | 0.94, 1.06 |
| Depression (per 1 SD) | 0.99 | 0.97, 1.02 | 1.00 | 0.96, 1.04 | 1.06 | 0.99, 1.13 |
| Bipolar disorder (per 1 SD) | 0.99 | 0.97, 1.01 | 1.05 | 1.02, 1.09 | 1.05 | 0.99, 1.12 |
| Schizophrenia (per 1 SD) | 0.99 | 0.96, 1.01 | 1.07 | 1.03, 1.11 | 1.07 | 1.00, 1.13 |
| Clinical and sociodemographic factors | ||||||
| Female | 1.02 | 0.97, 1.07 | 0.99 | 0.92, 1.06 | 1.17 | 1.03, 1.33 |
| ADHD diagnosis at age ≥13 years | 1.27 | 1.17, 1.38 | 2.01 | 1.77, 2.27 | 1.91 | 1.53, 2.39 |
| Parental psychiatric history | 0.96 | 0.89, 1.04 | 1.12 | 1.00, 1.26 | 1.14 | 0.94, 1.38 |
| Mother, low education | 0.94 | 0.90, 0.99 | 1.05 | 0.98, 1.13 | 0.92 | 0.82, 1.04 |
| Father, low income | 0.97 | 0.92, 1.02 | 1.11 | 1.03, 1.20 | 0.97 | 0.85, 1.12 |
| ASD | 0.95 | 0.89, 1.03 | 1.19 | 1.07, 1.33 | 1.07 | 0.89, 1.28 |
| Intellectual disability | 1.07 | 0.91, 1.26 | 1.06 | 0.89, 1.26 | 1.03 | 0.78, 1.36 |
| Oppositional defiant or conduct disorder | 1.17 | 0.97, 1.39 | 1.13 | 0.95, 1.35 | 1.44 | 1.12, 1.86 |
| Tic disorder | 1.04 | 0.93, 1.17 | 1.15 | 0.97, 1.38 | 2.08 | 1.65, 2.63 |
| OCD | 0.84 | 0.72, 0.97 | 1.22 | 1.00, 1.48 | 1.27 | 0.92, 1.76 |
| Anxiety disorder | 0.83 | 0.75, 0.92 | 1.18 | 1.04, 1.35 | 1.44 | 1.17, 1.77 |
| Depressive disorder | 0.90 | 0.82, 1.00 | 1.00 | 0.88, 1.13 | 1.20 | 0.98, 1.48 |
| Bipolar disorder | 0.59 | 0.43, 0.80 | 1.45 | 1.04, 2.02 | 1.08 | 0.60, 1.97 |
| Substance use disorder | 0.88 | 0.79, 0.97 | 1.24 | 1.10, 1.41 | 1.40 | 1.13, 1.73 |
a
Significant associations are in boldface. Sex, parental characteristics, and age at first ADHD diagnosis were modeled as time-fixed exposures, and psychiatric comorbidities were treated as time-varying exposures. All models were adjusted for sex, age at first ADHD diagnosis split into five categories (1–6, 7–9, 10–14, 15–19, and 20–32 years; except when age at ADHD diagnosis was the exposure), and birth year in five categories (1981–1985, 1986–1990, 1991–1995, 1996–2000, and 2001–2005). Polygenic risk score models are further adjusted for genotyping wave and the first four principal components. ADHD=attention deficit hyperactivity disorder; ASD=autism spectrum disorder; OCD=obsessive-compulsive disorder.
PRS associations across quintiles (Figure 2; see also Table S4 in the online supplement ) showed that individuals with ADHD in the highest PRS quintile for bipolar disorder (hazard ratio=1.21, 95% CI=1.09, 1.35) and schizophrenia (hazard ratio=1.24, 95% CI=1.11, 1.39) had higher rates of discontinuation compared with those in the lowest quintile. Further, those in the highest PRS quintile for depression (hazard ratio=1.26, 95% CI=1.05, 1.52) and bipolar disorder (hazard ratio=1.25, 95% CI=1.04, 1.51) had higher rates of switch. For schizophrenia PRS, rates were elevated across the second to the fifth quintiles but only significantly so at the fourth quintile (hazard ratio=1.33, 95% CI=1.10, 1.61). Confidence intervals for the remaining estimates were near to or included 1, and overlapped across quintiles.
FIGURE 2. Hazard ratios of stimulant treatment initiation, discontinuation, and switch to nonstimulants across polygenic risk scores quintiles in individuals with ADHDa
a The first quintile was set to reference. All models were adjusted for sex, age at first ADHD diagnosis split into five age categories (1–6, 7–9, 10–14, 15–19, and 20–32 years), birth year in five categories (1981–1985, 1986–1990, 1991–1995, 1996–2000, and 2001–2005), genotyping wave, and the first four principal components. Error bars indicate 95% confidence interval. ADHD=attention deficit hyperactivity disorder; ASD=autism spectrum disorder; PRS=polygenic risk score.
Associations with clinical and sociodemographic factors are presented in Table 1 and Figure 3. Sex was not associated with initiation or discontinuation; however, females had higher rates of switch (hazard ratio=1.17, 95% CI=1.03, 1.33). Being diagnosed with ADHD at age 13 or older was associated with higher rates of initiation (hazard ratio=1.27, 95% CI=1.17, 1.38), discontinuation (hazard ratio=2.01, 95% CI 1.77–2.27), and switch (hazard ratio=1.91, 95% CI=1.53, 2.39) compared with diagnosis before age 13. Low maternal education was marginally associated with lower rates of initiation (hazard ratio=0.94, 95% CI=0.90, 0.99) and low paternal income with higher rates of discontinuation (hazard ratio=1.11, 95% CI=1.03, 1.20).
FIGURE 3. Hazard ratios expressing the association of clinical and sociodemographic factors with stimulant treatment outcomes in ADHDa
a Sex, age at first ADHD diagnosis, parental education, income, and psychiatric history were treated as time-fixed exposures. Psychiatric comorbidities were treated as time-varying exposures. All models were adjusted for sex, age at first ADHD diagnosis split in five age categories (1–6, 7–9, 10–14, 15–19, and 20–32 years; except when age at ADHD diagnosis was the exposure), and birth year in five categories (1981–1985, 1986–1990, 1991–1995, 1996–2000, and 2001–2005). Error bars indicate 95% confidence interval. ADHD=attention deficit hyperactivity disorder; ASD=autism spectrum disorder; OCD=obsessive-compulsive disorder.
Comorbid OCD, anxiety, bipolar disorder, and substance use disorder were associated with lower rates of initiation (hazard ratio range, 0.59–0.88). Further, ASD, anxiety, bipolar disorder, and substance use disorder were associated with higher rates of discontinuation (hazard ratio range, 1.18–1.46), and oppositional defiant/conduct disorder, tics, anxiety, and substance use disorder with higher rates of switch (hazard ratio range, 1.41–2.08), compared with individuals with ADHD without these comorbidities. Estimates stratified by age at first ADHD diagnosis (see Figure S2 and Table S5 in the online supplement ) showed that comorbid ASD was associated with discontinuation only in children (hazard ratio=1.41, 95% CI=1.24, 1.60). Similarly, comorbid depression was associated with discontinuation (hazard ratio=3.05, 95% CI=1.76, 5.28) and switch (hazard ratio=3.43, 95% CI=1.53, 7.71) in children but not in adolescents and adults. Associations of comorbid bipolar disorder and substance use disorder with stimulant treatment outcomes in the main analysis were driven by individuals diagnosed with ADHD during adolescence or adulthood, as there were too few comorbid cases (N<10) to estimate hazard ratios in children. Confidence intervals were overlapping for the remaining age-stratified estimates.
Supplementary analyses (see Table S6 in the online supplement ) showed that individuals with ADHD who initiated treatment with a nonstimulant medication (N=568) were more likely to be female (odds ratio=0.77, 95% CI=0.64, 0.93 [male as reference]), to be diagnosed with ADHD at age 13 or older (odds ratio=6.12, 95% CI=4.20, 9.00), and to have comorbid ASD, tics, anxiety, or substance use disorder (odds ratio range, 1.47–3.56) compared with stimulant initiators.
GWAS and h2SNP
h2SNP on the observed scale was estimated to be 0.08 (SE=0.06) for initiation, 0.13 (SE=0.08) for discontinuation, and 0.09 (SE=0.11) for switch. N and liability-scale converted estimates are provided in Table S7 in the online supplement . No genome-wide significant hits were detected for initiation and discontinuation (p>5 × 10−8) (see Figures S3 and S4 in the online supplement ). For switch, one locus on chromosome 16q23.3 (p<4.7 × 10−8, lead SNP rs58543609, CHR16:82376003 [GRCh37]) reached genome-wide significance (Figure 4). Using FUMA (see supplementary note 3 in the online supplement ), AC024590.1 and RN7SKP190 were identified as the most proximal genes of the rs58543609 locus. Expression quantitative trait loci annotation identified one putatively associated gene, MPHOSPH6. A lookup in the GWAS Catalog (47) identified no prior reports of the lead SNP, although a locus scan (CHR16:82315555–82397332) revealed suggestive associations in prior GWASs of cognitive performance, seasonal depression, face morphology, and bone mineral density. PheWAS implemented in the GWAS ATLAS (48) revealed two putative associations with a measure of weekly alcohol intake and cerebellar volume. Associations for the 18 independent genomic loci reaching suggestive genome-wide significance (p<10−5) and FUMA results are presented in Tables S8 and S9 in the online supplement .
FIGURE 4. Manhattan plot and quantile-quantile (QQ) plot of the association p values, and regional plot of top locus on chromosome 16q23.3, from GWAS of switch to nonstimulants in individuals with ADHDa
a Individuals with ADHD who switched to nonstimulants (N=1,137) were compared with those who remained on stimulants (N=3,854) (i.e., no discontinuation and no switch) in the 2 years following stimulant treatment initiation. In the Manhattan plot (panel A), the −log10(p) for each SNP is plotted on the y-axis against the genomic position on the x-axis. The horizontal red line represents the threshold for genome-wide significance and the gray dashed line the suggestive genome-wide significance (p<10-5). In the QQ-plot (panel B), the black dots represent observed p values and the red line represents expected p values under the null hypothesis; the shaded area indicates the 95% confidence band. The regional association plot (panel C) around the lead SNP rs58543609 shows the chromosomal base-pair positions on the x-axis, the significance of the association expressed as −log10(p) on the left y-axis, and the recombination rate indicated by the blue curve (right y-axis). The horizontal gray line represents the threshold for genome-wide significance. Linkage disequilibrium estimates of surrounding SNPs with the index SNP are indicated by color (the color bar in the upper left corner indicates r2 values). The nearest genes are shown below the plot. ADHD=attention deficit hyperactivity disorder; GWAS=genome-wide association study.
Discussion
This is, to our knowledge, to first study thus far utilizing genetic and prescription data to investigate stimulant treatment initiation, discontinuation, and switch to nonstimulant medication in ADHD. We present novel findings suggesting that individuals with ADHD and higher polygenic liability for bipolar disorder, schizophrenia, and possibly depression may be at increased risk of stimulant treatment discontinuation and switch. We also identified several clinical factors contributing to stimulant treatment outcomes, including delayed ADHD diagnosis and certain psychiatric comorbidities. We also present the first GWASs and h2SNP estimates of stimulant treatment outcomes in ADHD defined from prescription data, identifying one putative locus associated with switch to nonstimulants.
We found that a majority (81%) of individuals with ADHD initiated stimulant treatment, yet within 2 years, nearly half (45%) had discontinued treatment and 15% had switched to nonstimulants. Similar rates of discontinuation have previously been reported in Denmark, Sweden, the United States, South Korea, and Taiwan (19–23), highlighting that prescription databases provide relatively consistent estimates of ADHD treatment patterns and that discontinuation of medication is an important issue in the management of ADHD globally.
Individuals with ADHD in the higher PRS quintiles for bipolar disorder and schizophrenia had 17%–25% higher rates of discontinuation and switch compared with those in the lowest PRS quintiles. We also found that higher polygenic liability for depression may increase the rate of switch. These novel findings require replication, as some confidence intervals included 1, and the pattern of associations was not entirely consistent with a dose-response relationship across all PRS quintiles. Nevertheless, it can be hypothesized that elevated genetic liability for mood and psychotic disorders in individuals with ADHD may increase the risk of adverse effects of stimulant treatment (e.g., mood destabilization or psychotic symptoms) (3, 29). In support of this, we also found that comorbid depression in children with ADHD, and comorbid bipolar disorder in adolescents and adults, was associated with higher rates of discontinuation. Another possibility is that prodromal mood or psychotic symptoms may, in some cases, be misdiagnosed as ADHD, potentially rendering stimulant treatment inappropriate (49). Nevertheless, given the high validity of ADHD diagnoses in the Danish registers (50) and evidence of familial and genetic overlap between ADHD and mood and psychotic disorders (11, 51), this is unlikely to fully explain our results. Regardless, these findings underscore the importance of screening for family history and symptoms of psychosis, mania, and depression prior to initiating stimulants and suggest that PRSs may complement such screening in the future, assuming that their predictive validity can be substantially improved (14). This is especially notable as we found that PRS associations, although modest in effect size compared with psychiatric comorbidities, were largely unchanged after adjusting for comorbidities and sociodemographic factors. This suggests that higher PRSs for mood and psychotic disorders in ADHD may act as an independent risk factor for stimulant treatment discontinuation and switch, which is measurable prior to the (potential) onset of such disorders and in patients with symptoms below the diagnostic threshold.
We found limited evidence for the contribution of ADHD and ASD PRSs to stimulant treatment outcomes, possibly because of the lower power in the discovery GWASs of these disorders or the challenge of using PRS for ADHD to predict secondary associations in a sample selected for ADHD (i.e., “index event bias”) (52). It is also possible that risk variants for ADHD are not necessarily the same variants influencing treatment outcomes in ADHD. Interestingly, a recent study (53) found no correlation between 23 genes identified as targets of ADHD medications and ADHD GWAS summary statistics. Instead, associations were observed for GWAS of alcohol consumption, schizophrenia, and depression with the DRD2, CYP2D6, and CHRM2 genes. This, together with our findings, suggests that the effectiveness and/or tolerance of ADHD medications may be mediated by pathways other than those underlying ADHD, and that genetic liability for psychiatric disorders emerging in adolescence and adulthood may be of importance for stimulant treatment outcomes (53).
Beyond bipolar disorder and depression (discussed above), several psychiatric comorbidities stood out as risk factors for discontinuation and switch. Comorbid OCD and anxiety increased the risk of discontinuation and switch, and ASD increased the risk of discontinuation, although the latter was only significant in children. Although evidence is low, these comorbidities have historically been suggested to increase the risk of adverse effects of stimulant treatment (2, 3). Our data may either support the comorbidities’ association with adverse effects or merely confirm that this clinical perception is common. Among individuals with ADHD diagnosed in adolescence or adulthood, substance use disorder increased the risk of both discontinuation and switch, while comorbid oppositional defiant or conduct disorder was associated only with switch. The former may relate to evidence of lower stimulant efficacy in ADHD with comorbid substance misuse and clinical concerns of misuse or diversion (27). Finally, comorbid tic disorder increased the risk of switch to nonstimulants across ages, and the risk of discontinuation in children, which could reflect the fact that stimulant treatment can, although rarely, exacerbate or induce tics (2, 3). Our findings emphasize the need for a broad and age-sensitive clinical assessment in ADHD prior to treatment initiation, as well as close monitoring for psychiatric problems emerging during treatment, as comorbidities may negatively influence stimulant tolerance and efficacy. Older age at first ADHD diagnosis was also associated with elevated risk of discontinuation and switch, and treatment initiation with nonstimulants was more common in those diagnosed with ADHD after childhood. Similar findings have been reported in previous studies (7, 16, 20, 23), emphasizing that adolescence and early adulthood are high-risk periods for ADHD treatment dropout. Our results suggest that efforts are needed to improve treatment continuity in these age groups.
Finally, we evaluated the feasibility of using EHRs in ADHD pharmacogenomics. GWASs of stimulant initiation and discontinuation did not identify any genome-wide significant loci, and the common variant contribution estimated by h2SNP was not significantly different from zero for any stimulant treatment outcomes. These null findings are unsurprising given our limited sample size and the challenge of defining treatment outcomes that are more proximal to the underlying genetics (54, 55). Nevertheless, it can be noted that despite large standard errors, our h2SNP point estimates are in line with heritabilities reported for other treatment response traits in psychiatry (e.g., antidepressant treatment response [56]). Moreover, (potentially) low h2SNP does not imply that genetic factors are unimportant, as the power to detect genetic variants for a given phenotype depends both on its genomic architecture and GWAS sample size (57). We did identify one genome-wide significant locus associated with switch to nonstimulants, on chromosome 16q23.3. The locus and most proximal gene (AC024590.1) have not been identified in prior GWASs, but the RN7SKP190 and cadherin-13 (CDH13) gene are located within 250 kb of the locus and have been associated with theoretically relevant traits, such as body mass index, blood pressure, and educational attainment (47). However, the potential molecular consequences of the rs58543609 locus for switch require further investigation. Based on the modest h2SNP point estimates and GWAS without any major gene effects, our results suggest that stimulant treatment outcomes (as defined here) likely have a complex genetic architecture and that larger samples will be needed for gene discovery. Given the continuous integration of genetics in large EHR databases globally, and rapid methods development for approximating treatment outcomes (15), we believe our findings illustrate the potential of using prescription data to advance pharmacogenomics in ADHD.
Strengths and Limitations
Our study has several strengths, including the representativeness of the iPSYCH2012 sample and access to longitudinal register data. There are also important limitations. First, pharmacoepidemiological studies rely on modeling treatment outcome proxies from prescriptions, and we do not know how well these proxies map onto actual medication consumption. Further, the reasons for discontinuation or switch are not recorded in the Danish National Prescription Registry. Second, our study was underpowered for GWASs and h2SNP estimation, limiting our ability to draw firm conclusions regarding the common variant contribution to stimulant treatment outcomes. Third, iPSYCH2012 only includes ADHD cases of the combined subtype (F90.0), meaning that the results may not apply in the broader ADHD case group. Fourth, we did not have data on several factors shown to be important for ADHD treatment adherence, such as perceived stigma and patients’ and parents’ attitudes toward treatment (5, 7). Finally, the reliance on a highly homogeneous Danish population sample may limit the findings’ generalizability in more diverse populations.
Conclusions
We present evidence that individuals with ADHD who have a higher genetic liability for mood and/or psychotic disorders, are diagnosed after childhood, or are affected by certain psychiatric comorbidities may be at increased risk of stimulant treatment discontinuation and switch to nonstimulants. Our results also highlight that the majority of evaluated risk factors, and in particular PRSs, have only modest effects on stimulant treatment outcomes. Identifying individuals with ADHD at high risk of suboptimal treatment outcomes will thus likely depend on multifactorial prediction, including both genetic and clinical risk factors. Our GWASs illustrate the potential of utilizing genomics linked to EHRs to identify genetic variants underlying stimulant treatment outcomes in ADHD.
Footnotes
Presented at the 27th World Congress of Psychiatric Genetics, October 26–31, 2019, Los Angeles.
The iPSYCH team was supported by grants from the Lundbeck Foundation (R102-A9118, R155-2014-1724, and R248-2017-2003), the European Union’s FP7 Program (grant 602805, “Aggressotype”), the European Union’s Horizon 2020 Program (grant 667302, “CoCA”), NIMH (grant 1U01MH109514-01 to Dr. Borglum), and the universities and university hospitals of Aarhus and Copenhagen. The Danish National Biobank resource was supported by the Novo Nordisk Foundation. High-performance computer capacity for handling and statistical analysis of iPSYCH data on the GenomeDK HPC facility was provided by the Center for Genomics and Personalized Medicine and the Center for Integrative Sequencing, Aarhus University, Denmark (grant to Dr. Børglum). Dr. Dalsgaard’s research is further supported by Helsefonden (grant 19-8-0260) and the European Union’s Horizon 2020 Program (grant 847879). Dr. Thapar was supported byWellcome Trust.
The authors gratefully acknowledge the research participants and employees of 23andMe for providing the summary statistics used to generate the depression polygenic risk scores.
Supplementary Material
References
1.
Polanczyk GV, Willcutt EG, Salum GA, et al: ADHD prevalence estimates across three decades: an updated systematic review and meta-regression analysis. Int J Epidemiol 2014; 43:434–442
2.
National Institute for Health and Care Excellence (NICE): Attention Deficit Hyperactivity Disorder: Diagnosis and Management (NICE Guideline no 87). London, National Institute for Health and Care Excellence, March 14, 2018 (https://www.nice.org.uk/guidance/ng87)
3.
Caye A, Swanson JM, Coghill D, et al: Treatment strategies for ADHD: an evidence-based guide to select optimal treatment. Mol Psychiatry 2019; 24:390–408
4.
Cortese S, Adamo N, Del Giovane C, et al: Comparative efficacy and tolerability of medications for attention-deficit hyperactivity disorder in children, adolescents, and adults: a systematic review and network meta-analysis. Lancet Psychiatry 2018; 5:727–738
5.
Gajria K, Lu M, Sikirica V, et al: Adherence, persistence, and medication discontinuation in patients with attention-deficit/hyperactivity disorder: a systematic literature review. Neuropsychiatr Dis Treat 2014; 10:1543–1569
6.
Warrer P, Thomsen PH, Dalsgaard S, et al: Switch in therapy from methylphenidate to atomoxetine in children and adolescents with attention-deficit/hyperactivity disorder: an analysis of patient records. J Child Adolesc Psychopharmacol 2016; 26:354–361
7.
Khan MU, Aslani P: A review of factors influencing the three phases of medication adherence in people with attention-deficit/hyperactivity disorder. J Child Adolesc Psychopharmacol 2019; 29:398–418
8.
Chang Z, Ghirardi L, Quinn PD, et al: Risks and benefits of attention-deficit/hyperactivity disorder medication on behavioral and neuropsychiatric outcomes: a qualitative review of pharmacoepidemiology studies using linked prescription databases. Biol Psychiatry 2019: 86:335–343
9.
Myer NM, Boland JR, Faraone SV: Pharmacogenetics predictors of methylphenidate efficacy in childhood ADHD. Mol Psychiatry 2018; 23:1929–1936
10.
Sullivan PF: How good were candidate gene guesses in schizophrenia genetics? Biol Psychiatry 2017; 82:696–697
11.
Demontis D, Walters RK, Martin J, et al: Discovery of the first genome-wide significant risk loci for attention deficit/hyperactivity disorder. Nat Genet 2019; 51:63–75
12.
Pagerols M, Richarte V, Sánchez-Mora C, et al: Integrative genomic analysis of methylphenidate response in attention-deficit/hyperactivity disorder. Sci Rep 2018; 8:1881
13.
Mick E, Neale B, Middleton FA, et al: Genome-wide association study of response to methylphenidate in 187 children with attention-deficit/hyperactivity disorder. Am J Med Genet B Neuropsychiatr Genet 2008; 147B:1412–1418
14.
Murray GK, Lin T, Austin J, et al: Could polygenic risk scores be useful in psychiatry? JAMA Psychiatry 2021; 78:210–219
15.
Smoller JW: The use of electronic health records for psychiatric phenotyping and genomics. Am J Med Genet B Neuropsychiatr Genet 2018; 177:601–612
16.
Pottegård A, Bjerregaard BK, Kortegaard LS, et al: Early discontinuation of attention-deficit/hyperactivity disorder drug treatment: a Danish nationwide drug utilization study. Basic Clin Pharmacol Toxicol 2015; 116:349–353
17.
Perwien A, Hall J, Swensen A, et al: Stimulant treatment patterns and compliance in children and adults with newly treated attention-deficit/hyperactivity disorder. J Manag Care Pharm 2004; 10:122–129
18.
Garbe E, Mikolajczyk RT, Banaschewski T, et al: Drug treatment patterns of attention-deficit/hyperactivity disorder in children and adolescents in Germany: results from a large population-based cohort study. J Child Adolesc Psychopharmacol 2012; 22:452–458
19.
Pottegård A, Bjerregaard BK, Glintborg D, et al: The use of medication against attention deficit/hyperactivity disorder in Denmark: a drug use study from a patient perspective. Eur J Clin Pharmacol 2013; 69:589–598
20.
Zetterqvist J, Asherson P, Halldner L, et al: Stimulant and non-stimulant attention deficit/hyperactivity disorder drug use: total population study of trends and discontinuation patterns, 2006–2009. Acta Psychiatr Scand 2013; 128:70–77
21.
Wang L-J, Yang K-C, Lee S-Y, et al: Initiation and persistence of pharmacotherapy for youths with attention deficit hyperactivity disorder in Taiwan. PLoS One 2016; 11:e0161061
22.
Bhang SY, Kwack YS, Joung YS, et al: Factors that affect the adherence to ADHD medications during a treatment continuation period in children and adolescents: a nationwide retrospective cohort study using Korean health insurance data from 2007 to 2011. Psychiatry Investig 2017; 14:158–165
23.
Biederman J, Fried R, DiSalvo M, et al.: Evidence of low adherence to stimulant medication among children and youths with ADHD: an electronic health records study. Psychiatr Serv 2019; 70:874–880
24.
Zhong Y, Yang B, Su Y, et al.: The association with quantitative response to attention-deficit/hyperactivity disorder medication of the previously identified neurodevelopmental network genes. J Child Adolesc Psychopharmacol 2020; 30:348–354
25.
Robinson EB, St Pourcain B, Anttila V, et al: Genetic risk for autism spectrum disorders and neuropsychiatric variation in the general population. Nat Genet 2016; 48:552–555
26.
Cortese S, Castelnau P, Morcillo C, et al: Psychostimulants for ADHD-like symptoms in individuals with autism spectrum disorders. Expert Rev Neurother 2012; 12:461–473
27.
Crunelle CL, van den Brink W, Moggi F, et al: International consensus statement on screening, diagnosis, and treatment of substance use disorder patients with comorbid attention deficit/hyperactivity disorder. Eur Addict Res 2018; 24:43–51
28.
Cabarkapa S, King JA, Dowling N, et al: Co-morbid obsessive-compulsive disorder and attention deficit hyperactivity disorder: neurobiological commonalities and treatment implications. Front Psychiatry 2019; 10:557
29.
Viktorin A, Rydén E, Thase ME, et al: The risk of treatment-emergent mania with methylphenidate in bipolar disorder. Am J Psychiatry 2017; 174:341–348
30.
Dalsgaard S, Leckman JF, Nielsen HS, et al: Gender and injuries predict stimulant medication use. J Child Adolesc Psychopharmacol 2014; 24:253–259
31.
Owens EB, Hinshaw SP, Kraemer HC, et al: Which treatment for whom for ADHD? Moderators of treatment response in the MTA. J Consult Clin Psychol 2003; 71:540–552
32.
Efron D, Mulraney M, Sciberras E, et al: Patterns of long-term ADHD medication use in Australian children. Arch Dis Child 2020; 105:593–597
33.
Pedersen CB, Bybjerg-Grauholm J, Pedersen MG, et al: The iPSYCH2012 case-cohort sample: new directions for unravelling genetic and environmental architectures of severe mental disorders. Mol Psychiatry 2018; 23:6–14
34.
World Health Organization: WHO ICD-10: Psykiske lidelser og adfærdsmæssige forstyrrelser. 1994
35.
Mors O, Perto GP, Mortensen PB: The Danish Psychiatric Central Research Register. Scand J Public Health 2011; 39(suppl):54–57
36.
Köhler-Forsberg O, Petersen L, Gasse C, et al: A nationwide study in Denmark of the association between treated infections and the subsequent risk of treated mental disorders in children and adolescents. JAMA Psychiatry 2019; 76:271–279
37.
Schork AJ, Won H, Appadurai V, et al: A genome-wide association study of shared risk across psychiatric disorders implicates gene regulation during fetal neurodevelopment. Nat Neurosci 2019; 22:353–361
38.
Patterson N, Price AL, Reich D: Population structure and eigenanalysis. PLoS Genet 2006; 2:e190
39.
Pottegård A, Schmidt SAJ, Wallach-Kildemoes H, et al: Data resource profile: the Danish National Prescription Registry. Int J Epidemiol 2017; 46:798–798f
40.
National Collaborating Centre for Mental Health: Attention Deficit Hyperactivity Disorder: The NICE Guideline on Diagnosis and Management of ADHD in Children, Young People, and Adults. London: British Psychological Society and Royal College of Psychiatrists, 2009
41.
Grove J, Ripke S, Als TD, et al: Identification of common genetic risk variants for autism spectrum disorder. Nat Genet 2019; 51:431–444
42.
Vilhjálmsson BJ, Yang J, Finucane HK, et al: Modeling linkage disequilibrium increases accuracy of polygenic risk scores. Am J Hum Genet 2015; 97:576–592
43.
Albiñana C, Grove J, McGrath JJ, et al: Leveraging both individual-level genetic data and GWAS summary statistics increases polygenic prediction. Am J Hum Genet (Online ahead of print, April 30, 2021)
44.
Loh P-R, Tucker G, Bulik-Sullivan BK, et al: Efficient Bayesian mixed-model analysis increases association power in large cohorts. Nat Genet 2015; 47:284–290
45.
Loh PR, Bhatia G, Gusev A, et al: Contrasting genetic architectures of schizophrenia and other complex diseases using fast variance-components analysis. Nat Genet 2015; 47:1385–1392
46.
Watanabe K, Taskesen E, van Bochoven A, et al: Functional mapping and annotation of genetic associations with FUMA. Nat Commun 2017; 8:1826
47.
Buniello A, MacArthur JAL, Cerezo M, et al: The NHGRI-EBI GWAS catalog of published genome-wide association studies, targeted arrays, and summary statistics 2019. Nucleic Acids Res 2019; 47(D1):D1005–D1012
48.
Watanabe K, Stringer S, Frei O, et al: A global overview of pleiotropy and genetic architecture in complex traits. Nat Genet 2019; 51:1339–1348
49.
Studerus E, Corbisiero S, Mazzariello N, et al: Can neuropsychological testing facilitate differential diagnosis between at-risk mental state (ARMS) for psychosis and adult attention-deficit/hyperactivity disorder (ADHD)? Eur Psychiatry 2018; 52:38–44
50.
Mohr-Jensen C, Vinkel Koch S, Briciet Lauritsen M, et al: The validity and reliability of the diagnosis of hyperkinetic disorders in the Danish Psychiatric Central Research Registry. Eur Psychiatry 2016; 35:16–24
51.
Larsson H, Rydén E, Boman M, et al: Risk of bipolar disorder and schizophrenia in relatives of people with attention-deficit hyperactivity disorder. Br J Psychiatry 2013; 203:103–106
52.
Howe LJ, Dudbridge F, Schmidt AF, et al: Polygenic risk scores for coronary artery disease and subsequent event risk amongst established cases. Hum Mol Genet 2020; 29:1388–1395
53.
Hegvik T-A, Waløen K, Pandey SK, et al: Druggable genome in attention deficit/hyperactivity disorder and its co-morbid conditions: new avenues for treatment. Mol Psychiatry (Online ahead of print, October 18, 2019)
54.
Evans LM, Tahmasbi R, Vrieze SI, et al: Comparison of methods that use whole genome data to estimate the heritability and genetic architecture of complex traits. Nat Genet 2018; 50:737–745
55.
Korte A, Farlow A: The advantages and limitations of trait analysis with GWAS: a review. Plant Methods 2013; 9:29
56.
Pain O, Hodgson K, Trubetskoy V, et al: Antidepressant response in major depressive disorder: a genome-wide association study. medRxiv, December 15, 2020
57.
Holland D, Frei O, Desikan R, et al: Beyond SNP heritability: polygenicity and discoverability of phenotypes estimated with a univariate Gaussian mixture model. PLoS Genet 2020; 16:e1008612
Information & Authors
Information
Published In
History
Received: 4 December 2020
Revision received: 16 March 2021
Revision received: 7 April 2021
Accepted: 19 April 2021
Published online: 22 June 2021
Published in print: September 01, 2021
Keywords
Authors
Competing Interests
Dr. Demontis has received speaking fees from Takeda. The other authors report no financial relationships with commercial interests.
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
There are no citations for this item
View Options
View options
PDF/ePub
View PDF/ePubGet Access
Login options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).