Objective: Community-based specialist behavior therapy teams may be helpful in managing challenging behavior, but evidence of their effectiveness is limited. This study was designed to examine the effectiveness and costs associated with treatment by a specialist behavior therapy team. Method: This was a parallel-group, randomized, single-blind controlled trial carried out in an intellectual disabilities service in England. Participants were 63 male and female service users with mild to severe intellectual disability who presented with challenging behavior. The interventions were standard treatment plus applied behavioral analysis (N=32) and standard treatment only (N=31). The primary outcome measure was challenging behavior, as measured by total and subscale scores on the Aberrant Behavior Checklist 3 and 6 months after randomization. Secondary outcome measures were psychiatric comorbidity assessed at 3 and 6 months using the Psychiatric Assessment Schedule for Adults With a Developmental Disability Checklist (PAS-ADD) and total costs recorded at 6 months. Multilevel modeling was used to compare square root transformations of Aberrant Behavior Checklist scores. Results: Significant differences were found in the transformed total scores on the Aberrant Behavior Checklist (difference=–0.89, 95% CI=–1.74 to –0.04) and transformed lethargy and hyperactivity subscale scores (common intervention effect=–0.56, 95% CI=–0.97 to –0.15). Standard care participants fared worse on the PAS-ADD comorbid organic disorder subscale. There was a clear trend for lower overall costs of the intervention. Conclusions: Use of a specialist behavior therapy team in addition to standard treatment appears to be more effective in improving challenging behavior and may have financial advantages over standard treatment.
Adults with intellectual disabilities may present with challenging behavior, such as aggression, self-injury, and hyperactivity, which may also occur in the context of comorbid mental illness. Prevalence rates for challenging behavior range from 6% to almost 30% (1) . Several U.K. government policies advocate a locally based service model to manage challenging behavior, but progress has been slow (2) . Without robust evidence of effective treatment options, antipsychotic medication has been the approach most often used (3, 4) .
A recent randomized controlled trial (5) failed to demonstrate greater efficacy of two antipsychotic medications (haloperidol and risperidone) for aggressive behavior compared to placebo, which suggests a need for the evaluation of psychological options. So far, most of the available evidence shows a modest positive impact of cognitive-behavioral therapy on aggression and anger management (6) .
Hudson et al. (7) reported pre- and postintervention outcomes of intensive behavior support based on applied behavior analysis (8) . The community-based teams offering the intervention worked intensively with individuals, provided consultation, education, and training, and carried out research. They found that 75% of 134 interventions resulted in improvements in behavior. A case-control study (9) compared outcomes of 14 individuals who received specialist intervention by a “peripatetic” multidisciplinary team with a randomly selected group of 12 individuals who received standard care. The intervention comprised individualized assessment and management plans. Improvements in the intervention group included increased contact with staff and participation in domestic activities.
In this article, we present the clinical and cost outcomes of a randomized single-blind controlled trial that compared individuals with challenging behavior who received standard treatment only with an intervention group that received behavior therapy support by a specialist team in addition to standard treatment. Our objective was to test the hypothesis that use of the specialist behavior therapy team in combination with standard treatment was more effective than standard treatment alone in reducing challenging behavior and costs.
Method
Setting
The trial was carried out from 2005 to 2008 in an area near Greater London (South Essex) with a population of approximately 750,000. Some 1,100 adults with intellectual disability are registered with the local community intellectual disabilities teams.
Participants
Service users age 18 and over with any severity of intellectual disability (10) were included in the study. They were referred to the behavior therapy team by members of the community intellectual disability teams, and the behavior therapy staff decided whether the referral met inclusion criteria: the behavior was severe enough to place the individual or others at risk, or placement breakdown was imminent despite other supports being offered. Service users in whom staff believed the challenging behavior was the direct consequence of a mental disorder were excluded. However, we assumed that patients may well have comorbid ill mental health.
The study received ethical approval from the Essex 1 (formerly West Essex) Research Ethics Committee, and written informed consent or caregiver assent, where necessary, was obtained.
Interventions
Participants were randomly allocated either to the behavior therapy team plus standard treatment or to standard treatment alone. The participants were retained in their randomized arms for 6 months, which was thought to be an acceptable time for service users to wait for specialist behavior treatment based on the team’s previous waiting list record.
Behavior therapy team
The team works across the trust’s intellectual disability services. It comprises one team coordinator, five full-time behavior specialists, two part-time behavior associate practitioners, and one part-time administrator. The behavior specialists, who have nursing backgrounds and a diploma or master’s degree in behavioral analysis and intervention in intellectual disability, coordinate the intervention plans. The behavior associate practitioners work directly with each individual and the caregivers to support the implementation of the plan. They have vocational awards and mostly work under supervision. The specialist behavior team also offers regular support to formal and informal caregivers and education and training to other services.
The team has adopted a multidimensional model including applied behavior analysis and positive behavioral support to address the problem behaviors without resorting to aversive strategies (8, 11, 12) . The service operates during office hours, although there is flexibility to respond outside these hours. New referrals and allocated cases are discussed in weekly meetings. The assessment includes ecological analysis, exploration of caregiver competencies, and gathering of collateral information, such as primary care. In a detailed functional analysis of the presenting problem, records are reviewed, the individual is observed in different settings, and staff who know the individual well complete rating scales assessing the target behaviors and what might motivate them. (For an example of a functional analysis, see the data supplement that accompanies the online edition of this article.) Finally, a comprehensive report is produced based on the functional analysis with recommendations for a multi-element intervention plan, which is discussed with all stakeholders prior to implementation. The progress of the interventions is monitored with the periodic service review (13), which measures performance in implementing the intervention.
Caregivers are expected to employ behavioral strategies, and training is provided to enhance their skills. Any questions about model fidelity were fed back to the trial team and steering group meetings for agreement.
Each participant in the intervention arm received a mean of nine contacts (SD=7) over the study lifetime. The mean duration of the engagement with the behavior therapy team was 6 months, but in several cases the intervention may have continued beyond that time.
Standard treatment
This service consists of five community intellectual disabilities teams, each with a professional skills mix of psychiatrists, community nurses, occupational therapists, speech and language therapists, physiotherapists, and generic psychologists. The teams offer a range of interventions, including pharmacotherapy, nursing, and enhancement of adaptive skills. A 10-bed inpatient unit is available for service users with mental illness or challenging behavior who require admission. Another full-time behavior specialist and behavior practitioner work in the inpatient unit. Social care is provided by Social Services Departments.
Outcome Measures and Instruments
The primary outcome measure was challenging behavior, as measured by the total and subscale scores on the Aberrant Behavior Checklist (14) . The instrument has been widely used for assessment of behavior in people with intellectual disabilities and has demonstrated acceptable reliability and validity. Its five subscales are irritability, lethargy, stereotyped behavior, hyperactivity, and inappropriate speech.
The secondary outcomes measures were psychiatric comorbidity and service use. Psychiatric comorbidity was assessed by the Psychiatric Assessment Schedule for Adults With a Developmental Disability Checklist (PAS-ADD) (15), an informant-administered scale designed to screen for psychiatric disorders in people with intellectual disabilities. It consists of 29 symptom items, rated on a 4-point scale, measuring the possible presence of comorbid affective disorders, organic mental disorders, and psychotic disorders. In addition, we recorded the clinical diagnoses of autism spectrum disorders from the case notes. To record service use by participants in both trial arms over the preceding 6 months, we used the Client Service Receipt Inventory (16) .
Assessments were made at baseline (when the participant enrolled in the trial but before randomization), at 3 months, and at 6 months (end of trial participation).
Sample Size
The sample size calculation was based on previous studies (17) comparing challenging behaviors in community populations using the irritability subscale of the Aberrant Behavior Checklist. To detect a clinically significant mean difference of at least 8 points (0.8 of the standard deviation) between the study and control groups on the irritability domain scores, with a standard deviation of 10, a power of 0.8, and the significance level set at 0.05 (two-tailed), required 28 participants in each trial arm. Allowing for a 10% attrition rate, the total sample sought was 63 individuals.
When data were prepared for analysis, we calculated an inflation factor of 1.6, using an estimated intracluster correlation coefficient of 0.6 and a cluster size of 2 (representing the two follow-up assessments at 3 and 6 months). After adjusting for the inflation factor, a total of 101 observations were needed to achieve the required power for each arm. There were 122 and 124 observations available for the intervention and standard treatment arms, respectively. This sample size was also adequate to estimate a three-parameter (trial arm, baseline Aberrant Behavior Checklist score, and time) repeated-measures analysis model with adequate precision (18) .
Randomization and Concealment of Allocation
Participants were randomly assigned individually to either the intervention or the standard treatment arm. One of the authors (R.B.) developed a computer-driven randomization list that was stratified by clinical catchment area and based on a block size of four. A set of sealed envelopes, each bearing only the name of the area and a number, were held by an independent administrator.
Treatment Costs
The costs associated with treatment by the specialist behavior therapy team plus standard treatment and standard treatment only were estimated from a health and social services system perspective. Cost data were collected under two main categories: treatment and non-treatment costs, such as nonpsychiatric inpatient stays, outpatient appointments, day care, leisure activities, adult education, support for voluntary work, and contact with general practitioners and other professionals, such as community nurse, social worker, and advocate. The costs do not include contact made with the criminal justice system and paid or informal care provided by family. Data on contact with the specialist behavior therapy team were obtained from the team’s records of service users’ contacts.
The unit cost of contact for nursing professionals was estimated using the midpoint of the appropriate annual salary bands, salary on-costs, capital overheads, and indirect overheads weighted by working time. The unit costs of contact with a psychiatrist and inpatient admissions were obtained from a U.K. national compendium of unit costs of health and social care (19, 20) .
Statistical Analysis of Clinical Outcomes
The trial was analyzed following CONSORT guidelines and on an intent-to-treat basis. Because scores on the Aberrant Behavior Checklist and its subscale were highly skewed, we transformed them by taking their square root. However, in approaching normality, such transformations change the relative distances between data points. This makes interpretation of the data complex even though all data points remain in the same rank order as before. Although results can still be understood in terms of increasing scores, care must be used in interpreting results based on transformed data. For this reason we also provide data on changes in the raw total and subscale scores of the Aberrant Behavior Checklist.
We undertook the main analysis using a multilevel (two levels) linear regression model (21) on the square root of the total score on the Aberrant Behavior Checklist. We adjusted for baseline total score and time period. Although randomization was stratified by intellectual disability team catchment area, this was not significantly associated with the outcome, so the variable was not included in the regression model to maintain precision.
In a secondary analysis we used a three-level multivariate-outcome linear regression model with outcomes nested within time periods, which were nested within patients (21) . An advantage of this model is that it allows estimation of the intervention effects for multiple outcomes (all five subscales of the Aberrant Behavior Checklist) simultaneously. In addition, it is possible to use the likelihood ratio test to assess whether parameters are common and, if so, to set them to a common magnitude to increase precision. We adjusted for each baseline subscale score and time period. The effect for time period was assumed to be the same across all outcomes by including a common coefficient for each outcome in the model.
In the analysis of the binary PAS-ADD domains affective disorder, organic mental disorder, and psychotic disorder, we used a generalized estimating equation population-averaged model, assuming binomial family, exchangeable correlation, and robust standard errors. In all analyses, we used random intercepts and the residuals were assumed to be normally distributed. We used Stata, release 10 (StataCorp, College Station, Tex.), and SPSS, version 14 (SPSS, Chicago), as well as MLwiN 2.0 (Centre for Multilevel Modelling, University of Bristol, U.K.) for the multilevel models.
Analysis of Cost Outcomes
All costs to services, including the cost of introducing the intervention, were included in the analysis. Missing service use data were replaced by the median for the group to which the patient was assigned. All costs to agencies were estimated for a period of 6 months to be consistent with the time period over which the intervention took place. Multivariate regression analysis was used to adjust total costs for baseline age, gender, level of intellectual disability, PAS-ADD subscales, pervasive developmental disorder, and total Aberrant Behavior Checklist score at baseline. Data were not collected on baseline service use, so these were not adjusted for in the analysis. Comparison of costs between the groups is presented as mean values; analyses of differences between the groups were assessed by confidence intervals derived from the regression analysis using bootstrapping.
Results
Of 69 eligible participants who were approached, six declined to take part in the study ( Figure 1 ); 63 participants, predominantly male (58.7%) and white (95.2%), took part. Forty-two had mild/moderate and 21 severe/profound intellectual disability. Attrition was low. One participant in each arm died during the trial, and one participant in the intervention arm declined to participate in follow-up assessments. Sociodemographic and clinical details of participants at baseline are summarized in Table 1 .
Figure 1. Participant Flow Through a Study of Behavior Therapy Team Intervention
LOCF=last observation carried forward.
Median total and subscale scores on the Aberrant Behavior Checklist decreased in both trial arms over time, although scores for the inappropriate speech subscale reached a plateau by 3 months ( Table 2 ). Our primary multilevel analysis showed a significantly greater reduction in (transformed) total score in participants in the intervention arm before and after adjustment for total score at baseline (difference=–0.89, 95% CI=–1.74 to –0.04) (see Table S1 in the data supplement that accompanies the online edition of this article).
Initially, a model that allowed for different intervention effects for each Aberrant Behavior Checklist subscale was fitted. However, no significant differences were observed using likelihood ratio tests between the treatment effects for lethargy and hyperactivity or for irritability and inappropriate speech. Hence, a common intervention effect was estimated for each of these two pairs of outcomes. Both subscale scores for hyperactivity and lethargy were significantly reduced in both treatment arms after adjustment, suggesting an overall normalization of activity and inactivity. There was also a nonsignificant tendency for greater improvement in the three other subscales. Although baseline measurement explained more than 50% of between-patient variability, we were able to demonstrate significance in between-patient as well as within-patient variability. Given that randomization in a small trial may not eliminate systematic bias, we also adjusted the model for gender, age, level of disability, and mobility status but found no significant association. Details are shown in Table 3 .
Mental status improved over time, and at 6 months, among participants in the intervention group 18.7% screened positive for comorbid affective disorder (compared with 21.9% in the standard treatment group), 9.3% for psychotic disorder (compared with 6.4%), and 6.25% for organic disorder (compared with 16%). The odds of developing a possible comorbid organic mental condition were 90% higher in the standard treatment group. The other two PAS-ADD conditions were not found to be significantly different at endpoint (see Table S2 in the online data supplement).
Analysis of costs suggests that more service users in the standard treatment only group made use of outpatient and nonpsychiatric inpatient services. One service user was admitted to the inpatient psychiatric unit, and another three were admitted to the general hospital with physical health complaints. In the standard treatment group, eight service users in independent living received daily visits. This is reflected in the higher average cost of community-based services used by the standard treatment group (see Table S3 in the online data supplement).
The total activities-related costs did not differ significantly between the two groups. The multivariate regression analysis did not show significant associations between costs and sociodemographic (age, gender, level of intellectual disability) or clinical variables (total Aberrant Behavior Checklist score at baseline, pervasive developmental disorder, PAS-ADD domains). After adjustment for baseline covariates, total costs including and excluding treatment were not significantly different between the two trial arms. However, there was a trend for lower total costs in the intervention plus standard treatment arm ( Table 4 ).
Discussion
To our knowledge, this is the first randomized trial of the effectiveness of a specialist behavior therapy team in the management of challenging behavior in adults with intellectual disabilities. We found that use of a specialist behavior therapy team in addition to standard treatment leads to significant reductions in overall scores on measures of challenging behaviors and appears to normalize participant activity. This evidence strongly supports clinical observations that specific types of behavioral interventions can have a positive impact on managing challenging behavior and provide us with an effective alternative to medication.
Use of the specialist behavior therapy team in addition to standard treatment appeared to be less costly than standard treatment overall. Although the intervention itself appeared to be more expensive, it was offset by the increased community support costs incurred by participants in the standard treatment only group.
Strengths and Limitations
This trial has a number of strengths. First, it was a pragmatic trial in which we followed as closely as possible the clinical realities of the services involved, and it reflects the treatment needs of a population of community-based adults with intellectual disabilities in a country with National Health Service provision. We recruited all eligible service users who were referred to the specialist behavior therapy team during the study period. Second, the attrition rate was negligible. Third, the model fidelity of the intervention arm was high, given that all members of the specialist behavior therapy team had similar training and adhered to a specified protocol. The author who collected outcome data from participants and their caregivers (D.R.) performed little better than chance at guessing each participant’s trial arm (42/63, 66%).
Ethical constraints meant that patients in the control arm of any trial must receive at least the standard available care. However, a limitation of the trial was the length of time during which we could ensure that participants (particularly those receiving standard treatment only) could remain in their trial arms. We agreed on a 6-month waiting list with the clinical teams as a reasonable period in which they could continue to manage service users in standard treatment. However, this time frame may have been insufficient to reveal the full effects of the intervention. There may also have been an interaction between severity of intellectual disability and outcome, but the study did not have the statistical power to examine that association. Finally, a lack of statistically significant differences in costs is widely reported in cost-effectiveness comparisons of mental health interventions. In our study, it may be due to an insufficient sample size, which was calculated to detect differences in clinical outcome only (22) .
External Validity
This study had broad inclusion criteria and a longer follow-up time than most trials of interventions for any type of challenging behavior (23) . One aspect of the standard service that may differ from other service providers was the low number of clinical psychologists attached to the service, so most participants were unlikely to have had specialist psychological treatment prior to entering the trial.
Psychiatric Comorbidity
The PAS-ADD checklist diagnoses in our sample are comparable to those found in a large administrative sample of adults with intellectual disabilities (24) . The increase in the odds of developing a comorbid organic condition among participants allocated to the standard treatment only group was unexpected. The organic disorder score in the PAS-ADD checklist is partly derived from adding two sets of questions that contain items on activities of daily living, such as self-care on the one hand and mental status (changes in mood and concentration) on the other. Therefore, we cannot be sure whether changes in any of those items reflect frank cognitive decline or the emergence of mental disorder. Given the time course of the trial, the latter possibility is the strongest. The finding may also reflect the ongoing behavior difficulties experienced by those in the control group.
Context of the Results
Our results add to a growing evidence base for noninvasive psychological interventions as an alternative to pharmacological treatments for challenging behavior. Case studies show that intensive community support reduces challenging behavior and increases staff satisfaction (25) . Positive behavior support that requires the implementation and monitoring of behavior plans is associated with a decrease in challenging behavior up to 22 months later (26) . However, previous studies have been uncontrolled evaluations, and therefore the results must be interpreted with caution.
Costs
The costs of challenging behavior are significant (27) . Only one uncontrolled study (28) has included a cost-benefit analysis of a community-based behavior team. The authors reported that the average annual cost of an intensive intervention for challenging behavior was AUS$5,725, compared with a standard service cost of AUS$40,510 (1991–1992 costs). In our study, higher costs for the specialist team were more than offset by higher incremental costs for standard treatment. Furthermore, expenditure for treatment with placebo (29) was comparable to the costs of standard treatment in our study (£3,624 versus £3,271). This finding suggests that while pharmacotherapy is costly and does not lead to better outcomes, psychological treatment, at least in the short term, costs comparatively less and results in significant health benefits.
Patient Perspective.
Footnotes
Received Nov. 29, 2008; revision received April 22, 2009; accepted May 19, 2009 (doi: 10.1176/appi.ajp.2009.08111747). From the Department of Mental Health Sciences, University College London Medical School. Address correspondence and reprint requests to Dr. Hassiotis, Department of Mental Health Sciences, University College London Medical School, Charles Bell House, 67-73 Riding House St., London W1W 7EY, UK; [email protected] (e-mail).
All authors report no competing interests.
Dr. Murad died in May 2009.Funded by the South Essex Partnership University Foundation NHS Trust (grant code GRG3).
The authors thank the members of the specialist behavior therapy team (in particular, Angela Daniels, who has commented on previous drafts of the paper) and the Community Intellectual Disabilities Teams; the participants and their families; and the members of the Trial Steering Committee (Jennifer Secker, John Gould, Helen Blerkom, Angela Daniels, and Carol Lynes).
Current Controlled Trials ID number: ISRCTN62134865 (http://www.controlled-trials.com).
References
1.
Allen DG, Lowe K, Moore K, Brophy S: Predictors, costs, and characteristics of out of area placement for people with intellectual disability and challenging behaviour. J Intellect Disabil Res 2007; 51:409–416
UK Department of Health: Services for People With Learning Disability and Challenging Behaviour or Mental Health Needs, revised ed, 2007. http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_080129
Branford D: A study of the prescribing for people with learning disabilities living in the community and in National Health Service care. J Intellect Disabil Res 1994; 38:577–586
Brylewski J, Duggan L: Antipsychotic medication for challenging behaviour in people with intellectual disability: a systematic review of randomised controlled trials. Cochrane Database Syst Rev 2001; 3:CD000377
Tyrer P, Oliver-Africano PC, Ahmed Z, Bouras N, Cooray S, Deb S, Murphy D, Hare M, Meade M, Reece B, Kramo K, Bhaumik S, Harley D, Regan A, Thomas D, Rao B, North B, Eliahoo J, Karatela S, Soni A, Crawford M: Risperidone, haloperidol, and placebo in the treatment of aggressive challenging behaviour in patients with intellectual disability: a randomised controlled trial. Lancet 2008; 371:57–63
Taylor JL, Novaco RW, Gillmer BT, Robertson A, Thorne I: Individual cognitive-behavioural anger treatment for people with mild-borderline intellectual disabilities and histories of aggression: a controlled trial. Br J Clin Psychol 2005; 44:367–382
Hudson A, Wilken P, Jauering R: Regionally based teams for the treatment of challenging behaviour: a three year outcome study. Behavioural Change 1995; 12:209–215
LaVigna GW, Donnellan AM (eds): Alternatives to Punishment: Solving Behavior Problems With Nonaversive Strategies. Irvington, NY, Institute of Applied Behavior Analysis, 1986
Llewellyn G, Thompson K, Whybrow S, McConnell D: Supporting Families: Family Well-Being and Children With Disabilities. Sydney, Australia, School of Occupation and Leisure Sciences, University of Sydney, 2003
Donnellan AM, LaVigna GW, Zambito J, Thvedt J: A time limited intensive intervention program model to support community placement for persons with severe behavior problems. J Assoc Pers Sev Handicaps 1985; 10:123–131
Aman MG, Singh NN, Stewart AW, Field CJ: The Aberrant Behavior Checklist: a behavior rating scale for the assessment of treatment effects. Am J Ment Defic 1985; 89:485–491
Prosser H, Moss S, Costello H: The Psychiatric Assessment Schedule for Adults With a Developmental Disability (PAS-ADD) Checklist. Manchester, England, University of Manchester, Hester Adrian Research Centre, 1998
Beecham J: Collecting and estimating costs, in The Economic Evaluation of Mental Health Care. Edited by Knapp M. London, Ashgate Publishing, 1995, pp 61–82
Clarke DJ, Kelley S, Thinn K, Corbett JA: Psychotropic drugs and mental retardation: disabilities and the prescription of drugs for behaviour and for epilepsy in three residential settings. J Ment Defic Res 1990; 34:385–395
Hassiotis A, Hall I: Behavioural and cognitive-behavioural interventions for outwardly directed aggressive behaviour in people with learning disabilities (revised review). Cochrane Database Syst Rev 2008; 3:CD003406
Taylor JL, Hatton C, Dixon L, Douglas C: Screening for psychiatric symptoms: PAS-ADD Checklist norms for adults with intellectual disabilities. J Intellect Disabil Res 2004; 48:37–41
Allen D, Lowe K: Providing intensive community support to people with learning disabilities and challenging behaviour: a preliminary analysis of outcomes and costs. J Intellect Disabil Res 1995; 39:67–82
McClean B, Dench C, Grey I, Shanahan S, Fitzsimons E, Hendler J, Corrigan M: Person focused training: a model for delivering positive behavioural supports to people with challenging behaviours. J Intellect Disabil Res 2005; 49:340–352
Hassiotis A, Parkes C, Jones L, Fitzgerald B, Romeo R: Individual characteristics and service expenditure on challenging behaviour for adults with intellectual disabilities. J Appl Res Intellect Disabil 2008; 21:438–445
Hudson A, Jauernig R, Wilken P: Behavioural treatment of challenging behaviour: a cost-benefit analysis of a service delivery model. Behaviour Change 1995; 12:216–226
Tyrer P, Oliver-Africano P, Romeo R, Knapp M, Dickens S, Bouras N, Ahmed Z, Cooray S, Deb S, Murphy D, Hare M, Meade M, Reece B, Kramo K, Bhaumik S, Harley D, Regan A, Thomas D, Rao B, Karatela S, Lenôtre L, Watson J, Soni A, Crawford M, Eliahoo J, North B: Neuroleptics in the treatment of aggressive challenging behaviour for people with intellectual disabilities: a randomised controlled trial (NACHBID). Health Technol Assess 2009; 13:iii–iv, ix–xi, 1–54
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).
If the address matches an existing account you will receive an email with instructions to retrieve your username
Create a new account
Change Password
Password Changed Successfully
Your password has been changed
Login
Reset password
Can't sign in? Forgot your password?
Enter your email address below and we will send you the reset instructions
If the address matches an existing account you will receive an email with instructions to reset your password.
Change Password
Congrats!
Your Phone has been verified
×
As described within the American Psychiatric Association (APA)'s Privacy Policy and Terms of Use, this website utilizes cookies, including for the purpose of offering an optimal online experience and services tailored to your preferences. Please read the entire Privacy Policy and Terms of Use. By closing this message, browsing this website, continuing the navigation, or otherwise continuing to use the APA's websites, you confirm that you understand and accept the terms of the Privacy Policy and Terms of Use, including the utilization of cookies.
