Using Drug Claims Data to Assess the Relationship of Medication Adherence With Hospitalization and Costs
Abstract
OBJECTIVE: This naturalistic study used claims data to examine the relationship of medication nonadherence to hospital use and costs among severely mentally ill clients in Wisconsin. METHODS: Data for 619 clients were obtained from Medicaid drug and hospital claims, county records, and case managers as part of a larger study in eight county-based mental health systems. Study participants were eligible for Medicaid, had a severe and persistent mental illness, were 18 years or older, and were receiving neuroleptics, lithium, or antidepressants. Drug claims were analyzed for a 12-month period to determine how regularly clients obtained their medications. Regression analyses were used to assess the effects of irregular medication use on any hospitalization for psychiatric problems, the number of days hospitalized, and hospital costs. The analyses controlled for several risk factors. RESULTS: Among clients with schizophrenia or schizoaffective disorder, 31 percent used medications irregularly. The rates were 33 percent among those with bipolar disorder and 41 percent among those with other severe mental illnesses. In the total sample, irregular users had significantly higher rates of hospitalization than regular users (42 percent versus 20 percent), more hospital days (16 days versus four days), and higher hospital costs ($3,992 versus $1,048). Irregular medication use was one of the strongest predictors of hospital use and costs even after the analyses controlled for diagnosis, demographic characteristics, baseline functioning, and previous hospitalizations. CONCLUSIONS: The availability of drug claims data and the ability to use them in predictive analyses make them a potentially useful data source in studies of medication adherence among persons with severe mental illness.
Studies have repeatedly demonstrated that inpatient care constitutes the greatest portion of direct medical costs for persons with severe mental illness. For clients with schizophrenia or bipolar disorder, 60 to 70 percent of direct costs are attributable to inpatient services (1,2).
Persons with severe mental illness are also prone to rehospitalization. Twenty-four percent of persons with bipolar disorder are rehospitalized within two years of their initial hospitalization (3). Rehospitalization rates are even higher for persons with schizophrenia. Although medications reduce symptoms in 70 to 85 percent of newly diagnosed persons with schizophrenia, eventually 60 percent will require rehospitalization (4). Some individuals are readmitted so often that they are called revolving-door patients (5).
Although many factors influence rehospitalization, medication nonadherence is a significant contributor. For persons with schizophrenia, nonadherence to neuroleptic drug regimens increases relapse rates (6) and has been linked to more frequent hospitalizations (5,7,8,9) and longer stays (8,9,10). Sporadic medication use has also been linked to hospital use among persons taking lithium (11). However, most published studies have been based on reports of adherence by clients or family members, which introduces substantial bias (12). In addition, many studies have sampled clients from a single institution or clinic (7,9,10,11), making it difficult to generalize to the larger population of persons with severe mental illness.
Few studies have quantified the economic costs associated with nonadherence and rehospitalization. Weiden and Olfson (6) estimated that 40 percent of rehospitalization costs among persons with schizophrenia are attributable to nonadherence. Their estimate probably understates the true burden, because their model included only costs for the first rehospitalization when in actuality 30 percent of patients are rehospitalized more than once in a year (13). In addition, their estimate was based on adherence rates found in clinical trials rather than in everyday practice, where nonadherence is likely to be greater. Clearly, there is a need for more precise accounting of the real-world rates and costs of rehospitalization associated with medication nonadherence (6).
One reason for the scarcity of research in this area is that measuring adherence is very difficult. Patients underreport nonadherence, clinician estimates and pill counts are inaccurate, and laboratory tests can be obtrusive and misleading (12). Epidemiologists have begun to use administrative databases as an efficient source of data for assessing the regularity of drug use (14,15,16). Although it cannot be ascertained whether clients consume all the medication in a single prescription, major gaps in refills suggest nonadherence. Another potential difficulty is that clients may obtain medication from other sources if they change insurers, become ineligible for benefits, or receive medications that are not listed in the database (17). It is therefore important to verify eligibility.
Despite the limitations of using claims data, such data are predictive of outcomes for persons taking antihypertensive drugs and phenytoin (15). We found no studies that used drug claims data to assess nonadherence and outcomes in schizophrenia. However, among persons taking lithium or antidepressants, drug claims data have been linked to hospital use or costs (11,18). Unfortunately, those studies did not control for other factors that can confound the relationship between medication adherence and hospitalization. For example, substance abuse is associated with nonadherence and rehospitalization (5,9,19,20). In such studies, it is also important to control for client demographic characteristics, illness severity, and whether oral or injectable medications are used (9,21,22).
In the study reported here, we assessed adherence by using drug claims data for a 12-month period. Use of these data provided a more objective and comprehensive picture of medication use. Claims data were merged with data from system records and case managers' assessments, enabling us to examine possible confounders not available in claims data.
Methods
Data sources
Data were obtained from a larger study that examined the organization and financing of mental health services for adults with severe mental illness in ten county-based mental health systems in Wisconsin (23). A multistage process was used to identify clients. Researchers and administrators developed a list of provider organizations in each system that included mental health clinics or centers, community support programs, residential treatment programs, and other case management organizations. The larger study focused on organizational issues. Therefore, the organizations included in the study had three or more staff members and had primary responsibility for six or more clients with severe mental illness. To be eligible for the larger study, clients had to be at least 18 years old and to have a severe and persistent mental illness, as defined by Wisconsin law (24). A total of 2,435 clients with severe mental illness were asked to participate in the study, and written consent was obtained from 2,037 of them (83.7 percent).
Data for our study were obtained from system and medical records and from a client assessment questionnaire distributed to case managers. We later obtained drug claims data for Medicaid-eligible clients in eight of the ten participating systems for the same 12-month period in 1989 and 1990. Drug claims contained information about each medication dispensed during the year. Data included the dispensing date, drug code, therapeutic class, and quantity. Claims for each prescription for a neuroleptic, for lithium, and for an antidepressant were reviewed for data entry errors. Each claim was assigned to one of four calendar quarters, according to the dispensing date. We then constructed several drug use measures for each quarter and merged them with other client data.
Case managers returned a client assessment questionnaire for 1,600 clients (78.6 percent). Of these, 731 were enrolled in Medicaid for the full 12 months, and drug data were available. Of the 731 clients, 619 had one or more claims for a neuroleptic, lithium, or an antidepressant during the year. We compared the total sample and the Medicaid subsample and found that persons in the subsample were more likely to have schizophrenia or bipolar disorder. However, persons in both samples had similar baseline functioning, demographic characteristics, previous hospitalizations, and likelihood of hospitalization during the study year. Because we were interested in adherence to the targeted medications, our analysis focused on the subsample of persons for whom these three types of medications were prescribed.
Adherence
Measurement of adherence involved several steps. First, we determined whether clients used each type of medication on a regular or an irregular basis over the year. Regularity of use was determined by counting the number of quarters in which one or more claims had been made for the given drug category (17). Clients taking any oral medication were defined as irregular users if they had one or more quarters without a claim. Clients who took only long-acting injectable (depot) medication were defined as irregular users if they had two or more quarters without a claim. We applied different definitions because a previous validation study showed that a typical depot prescription lasts more than 60 days, whereas a typical oral prescription lasts 30 to 34 days (25).
After determining adherence for each category, we created a summary measure across categories. Clients using medications from one or more prescribed categories on an irregular basis received a score of 1, indicating any irregular use. Those using medications from all of their prescribed categories on a regular basis received a score of 0, indicating no irregular use. Although our approach does not address all types or gradients of nonadherence, it is relatively easy to identify a refill gap of three months or more. A three-month gap also allows for the possibility that a change was made to a different drug in the same category or that extra medication may accumulate when dosages are reduced. This approach also does not require data on days' supply, which tend to be inaccurate for depot medications because of Medicaid reimbursement policies (25).
We lacked detailed prescribing or adherence data from other sources. However, the medical records form and the client assessment questionnaire included several items about psychotropic medication. We compared data sources and found that 98.8 percent of irregular medication users and 99.5 percent of regular users, as measured by claims data, indeed had a prescription for a psychotropic medication according to the medical record audit or case manager survey.
The client assessment questionnaire also asked the case manager about adherence: "Last month, did the client take his/her medication as prescribed: always, quite often, sometimes, very infrequently, or none of the time?" We compared data sources and found that clients whose drug claims indicated irregular use indeed had higher nonadherence according to case managers' reports (kappa=.10; t=2.33, df=593, p<.02), suggesting that the drug claims data reflected true nonadherence.
Other variables
System records and Medicaid claims were used to create three dependent variables: any hospitalization for psychiatric problems, total number of hospital days, and actual hospital expenditures in the 12-month period. Costs due to somatic illness were excluded when possible. To prevent duplication, researchers conducted an item-by-item and client-by-client verification of data on hospital use and costs from system records and Medicaid files. Detailed procedures and definitions have been published elsewhere (23).
We measured several factors that can influence hospital use. The client assessment questionnaire provided information on the client's age, gender (0=male, 1=female), age at the onset of illness, number of previous hospitalizations for psychiatric problems, and score on the Global Assessment Scale (GAS) (26), which measured level of functioning before enrollment in the treatment program (27). Possible GAS scores range from 1 to 100, with higher scores indicating higher functioning.
Primary diagnoses were obtained from DSM-III-R diagnoses reported by the client's case management organization to the county-based system. Researchers collapsed these data into three dichotomous variables indicating the presence or absence of schizophrenia or schizoaffective disorder, bipolar disorder, and other severe mental illnesses (including unipolar depression, organic brain syndrome, and borderline personality disorder). Information about alcohol or other drug use disorders was obtained from system records. Information about whether any oral neuroleptic medication was prescribed was obtained from drug claims data (0=no, 1=yes). Clients with a mix of oral and depot neuroleptics received a score of 1 on this variable.
Statistical analyses
Chi square analyses and t tests were used to examine bivariate associations between adherence and hospital outcomes in the entire sample and in three subgroups: persons with schizophrenia or schizoaffective disorder, those with bipolar disorder, and those with other severe mental illnesses. T test results were based on separate variance estimates and two-tailed tests. Logistic regression techniques were used to analyze the effects of irregular medication use on the odds of any hospitalization, and linear regression techniques were used to analyze the effects of irregular use on the number of hospital days and expenditures. Logarithmic transformations were used to normalize the distribution of hospital days and expenditures in all regression analyses. Client risk factors entered into each model were age, gender, age at onset of illness, alcohol or other drug use disorder, GAS score, number of previous hospitalizations, whether any oral neuroleptics were used, and whether the client had any irregular use of neuroleptics, antidepressants, or lithium.
Results
Descriptive statistics
Of the 619 clients who were prescribed the targeted medications, 575 were white (95 percent). The mean± SD age of the 619 clients was 44±14 years. A total of 424 clients (69 percent) had a primary diagnosis of schizophrenia or schizoaffective disorder, 67 (12 percent) had bipolar disorder, and 128 (10 percent) had other severe mental illnesses. Approximately half of the clients (317, or 51 percent) were women. A total of 378 (65 percent) had a high school education. Ninety-eight (16 percent) had an alcohol or drug use disorder.
The 619 clients had a mean±SD baseline GAS score of 40±14. The mean±SD number of previous hospitalizations was 6±9. During the study year, 168 clients (27 percent) were rehospitalized for a psychiatric problem. The mean±SD number of hospital days for the 619 clients was 8±24, and the mean±SD hospitalization costs were $2,028±$6,021.
Table 1 shows the prevalence of medication use and medication adherence for the 12-month period. Neuroleptics were the most frequently used medication, both in the total sample and in each subgroup. Lithium use was more prevalent among clients with bipolar disorder, and antidepressant use was more prevalent among those with other severe mental illnesses. About half of the clients (N=322) used medications from two or more categories.
Of clients with schizophrenia or schizoaffective disorder, irregular use of medications in one or more categories was found for 31 percent. Of those with bipolar disorder, irregular use was found for 33 percent. Irregular use was found for 41 percent of those with other severe mental illnesses. In the total sample, the rate of irregular use was 33 percent.
Bivariate analysis
Table 2 summarizes data on the prevalence of hospital use and the mean number of hospital days and costs for regular medication users and irregular medication users by diagnostic category. Overall, persons who were nonadherent were more likely to be rehospitalized; 20 percent of regular users were rehospitalized in the study year, compared with 42 percent of irregular users.
Significant differences were also found in hospitalization rates for regular and irregular users in each diagnostic category. As Table 2 shows, the hospitalization rate for clients with schizophrenia or schizoaffective disorder who regularly used medications was 18 percent, compared with 33 percent for those in this diagnostic category who were irregular users. Among those with bipolar disorder, the rates for regular and irregular users were 31 percent and 73 percent, respectively. Among clients with other severe mental illnesses, the respective rates were 17 percent and 52 percent.
It is possible that some clients did not become irregular users until after they were rehospitalized. To check the order of events, we performed a case-by-case analysis to determine the proportion of irregular users who showed evidence of irregular use before rehospitalization. Results showed that most irregular users who were rehospitalized (89 percent) had a refill gap of 30 days or more before they were rehospitalized, suggesting that irregular use contributed to rehospitalization.
Results of t tests revealed significant associations between adherence and the number of hospital days and expenditures. Regular users had a mean of four hospital days, compared with 16 days for irregular users. As shown in Table 2, regular users had significantly fewer hospital days on average than irregular users in all three diagnostic groups: for the group with schizophrenia or schizoaffective disorder, four days versus 14 days; for the group with bipolar disorder, four days versus 37 days; and for the group with other severe mental illnesses, three days versus 12 days.
As Table 2 shows, in the total sample, hospital expenditures were significantly lower for the regular users than for the irregular users—$1,048 versus $3,992. Significant differences also were found between regular and irregular users in the group with schizophrenia or schizoaffective disorder—$1,799 for the regular users versus $3,421 for the irregular users. Hospital costs also varied significantly for regular and irregular users in the bipolar disorder group—$1,657 versus $9,701—and in the group with other severe mental illnesses—$620 versus $3,024.
Multivariate analyses
As shown in Table 3, irregular medication use remained a significant and consistent predictor of rehospitalization after the analysis controlled for other factors. Compared with regular users, irregular users had an adjusted hospitalization rate that was 1.99 times higher among persons with schizophrenia or schizoaffective disorder (Wald statistic=5.47, df=1, p<.05), 8.71 times higher among persons with bipolar disorder (Wald statistic=9.3, df=1, p<.01), 4.91 times higher among those with other severe mental illnesses (Wald statistic=11.2, df=1, p<.001), and 2.97 times higher in the total sample (Wald statistic=25.3, df=1, p<.001).
Clients with schizophrenia or schizoaffective disorder were significantly more likely to be hospitalized if oral neuroleptics had been prescribed for them, if they had an alcohol or other drug use disorder, or if they had more previous hospitalizations. The number of previous hospitalizations was associated with rehospitalization among those with bipolar disorder, but other factors shown in Table 3 were nonsignificant in this group.
Linear regression results, shown in Table 4, indicated that irregular use was a significant predictor of number of hospital days after the analysis controlled for other factors. For all three diagnostic groups and the total group, irregular users were hospitalized for significantly more days than regular users. Irregular use was one of the two strongest predictors of hospital days among persons with schizophrenia or schizoaffective disorder. Irregular use was the strongest predictor among those with bipolar disorder and those with other severe mental illnesses.
Clients with schizophrenia or schizoaffective disorder spent more days in the hospital if they had more previous hospitalizations and if they were taking any oral neuroleptics; however, other risk factors were not significant. The number of previous hospitalizations was associated with the number of days that clients with bipolar disorder spent in the hospital; however, the number of previous hospitalizations was not significantly associated with hospital days for persons with other severe mental illnesses. Age was negatively related to number of hospital days among clients with bipolar disorder, but no age effect was found for the two other diagnostic groups. No significant associations were found for gender, age at illness onset, or GAS score. Client factors explained 12 to 35 percent of the variance in hospital days.
Table 5 shows the results of a logistic regression analysis for hospitalization expenditures. For irregular users, expenditures were significantly greater than for regular users in all three diagnostic groups. For the total group, expenditures for irregular users were significantly greater than those for regular users. In the group of clients with schizophrenia or schizoaffective disorder, a diagnosis of a substance use disorder and more previous hospitalizations also predicted higher costs. Among clients with bipolar disorder, expenditures were greater for younger clients and those with more previous hospitalizations. However, in the bipolar disorder group, other factors were nonsignificant. Among clients with other severe mental illnesses, irregular use was the only significant client factor that predicted higher hospitalization costs.
Discussion
Results of this analysis of drug claims data showed that in this sample of clients with severe mental illness, one-third did not adhere to prescribed drug regimens. A unique contribution of this study is that it relied on existing claims data and a relatively simple measure of irregular drug use among clients with different primary diagnoses and medications. Significant findings were obtained for each of the three diagnostic groups: persons with schizophrenia and schizoaffective disorder, those with bipolar disorder, and those with other severe mental illnesses.
Although use of drug claims data has limitations, it eliminates recall bias and is less obtrusive and less labor-intensive than other methods of assessing adherence. Our measure of irregular use also demonstrated good predictive validity—that is, the ability to predict future performance or events (27). In this case, our measure of irregular use predicted higher hospitalization rates and costs and more hospital days.
Another feature of this study is its naturalistic design, which involved multiple systems and programs. Although such a design makes it difficult to determine causality, it may provide more precise accounting of real-world rates and costs of nonadherence than clinical trials or studies involving a single institution. Our findings also highlight the unexplained variability in hospital use. It was striking that study variables explained only 12 to 35 percent of the variance in number of hospital days, for example.
Our study involved a fairly large sample and extensive data collection from case managers, which enabled us to control for risk factors for which information is not available in most databases. Even when the analyses controlled for these risk factors, nonadherence remained a strong predictor of hospital outcomes. Compared with regular users, irregular users had hospitalization rates nearly two times higher in the group with schizophrenia, nine times higher in the group with bipolar disorder, and five times higher in the group with other severe mental illnesses. The finding that 33 percent of clients had used their medications irregularly was disconcerting, because the study was conducted in a state with a progressive advocacy community, state mental health administration, and community-based programs. Our findings reinforce the magnitude and pervasiveness of nonadherence and the need for innovative methods of screening, supporting, and involving clients in drug therapy (28).
Our study had limitations. First, all clients were Medicaid beneficiaries and our findings cannot be generalized to other populations or states. The level of alcohol and drug abuse was lower than expected; however, the number of previous hospitalizations and the low GAS scores suggest that the sample was as seriously impaired as those in other studies (10). Second, our finding that clients taking oral neuroleptics had higher hospitalization rates is consistent with the literature and suggests the importance of using drug variables in studies of adherence (22). Unfortunately, we were not able to address the difference between patients taking typical and atypical neuroleptics and differences in dosage regimens or adjuvant medications.
Further analysis is needed to identify the predictors of medication nonadherence, possible system or organizational factors associated with differences in adherence, and the impact of nonadherence on the number of contacts with ambulatory care and their costs. Such studies are essential if we are to better understand nonadherence, its impact, and ways to break the vicious cycle that occurs when it goes undetected and unresolved.
Acknowledgments
The authors thank their colleagues on the larger project: Ellen Jane Hollingsworth, M.S., Jeanine K. Mount, Ph.D., Joy P. Newmann, Ph.D., and the late James R. Greenley, Ph.D. The larger study was supported by grant MH-43555 from the National Institute of Mental Health to Dr. Greenley.
The values are standardized beta coefficients controlling for sex, age at onset of illness, Global Assessment Scale score, and the variables in the table
Values are standardized beta coefficients controlling for sex, age at onset of illness, Global Assessment Scale score, and the variables in the table
Footnote
Dr. Svarstad and Ms. Sweeney are affiliated with the School of Pharmacy at the University of Wisconsin-Madison, 425 North Charter Street, Madison, Wisconsin 53706 (e-mail, [email protected]). Dr. Shireman is with the School of Pharmacy at the University of Kansas in Lawrence.
References
1.
Wyatt RJ, Henter I: An economic evaluation of manic-depressive illness, 1991. Social and Psychiatric Epidemiology 30:213-219, 1995
2.
Wyatt RJ, Henter I, Leary MC, et al: An economic evaluation of schizophrenia, 1991. Social Psychiatry and Psychiatric Epidemiology 30:196-205, 1995
3.
Frank RG, Brookmeyer R: Managed mental health care and patterns of inpatient utilization for treatment of affective disorders. Social Psychiatry and Psychiatric Epidemiology 30:220-223, 1995
4.
Health care reform for Americans with severe mental illnesses: report of the National Advisory Mental Health Council. American Journal of Psychiatry 150:1447-1465, 1993
5.
Haywood TW, Kravitz HM, Grossman LS, et al: Predicting the "revolving door" phenomenon among patients with schizophrenic, schizoaffective, and affective disorders. American Journal of Psychiatry 152:856-861, 1995
6.
Weiden PJ, Olfson M: Cost of relapse in schizophrenia. Schizophrenia Bulletin 21:419-429, 1995
7.
Green JH: Frequent rehospitalization and noncompliance with treatment. Hospital and Community Psychiatry 39:963-966, 1988
8.
Caton CLM, Koh SP, Fleiss JL, et al: Rehospitalization in chronic schizophrenia. Journal of Nervous and Mental Disease 173:139-148, 1985
9.
Sullivan G, Wells KB, Morgenstern H, et al: Identifying modifiable risk factors for rehospitalization: a case-control study of seriously mentally ill persons in Mississippi. American Journal of Psychiatry 152:1749-1756, 1995
10.
McEvoy JP, Howe AC, Hogarty GE: Differences in the nature of relapse and subsequent inpatient course between medication-compliant and noncompliant schizophrenic patients. Journal of Nervous and Mental Disease 172:412-416, 1984
11.
Johnson RE, McFarland BH: Lithium use and discontinuation in a health maintenance organization. American Journal of Psychiatry 153:993-1000, 1996
12.
Cramer JA, Rosenheck R: Compliance with medication regimens for mental and physical disorders. Psychiatric Services 49:196-201, 1998
13.
Dickey B, Normand ST, Norton EC, et al: Managing the care of schizophrenia: lessons from a 4-year Massachusetts Medicaid study. Archives of General Psychiatry 53:945-952, 1996
14.
Strom B, Carson JL: Use of automated databases for pharmacoepidemiology research. Epidemiologic Reviews 12:87-107, 1990
15.
Steiner JF, Koepsell TD, Fihn SD, et al: A general method of compliance assessment using centralized pharmacy records. Medical Care 26:814-823, 1988
16.
Ray WA, Griffin MR: Use of Medicaid data for pharmacoepidemiology. American Journal of Epidemiology 129:837-849, 1989
17.
Soumerai SB, McLaughlin TJ, Ross-Degnan D, et al: Effects of limiting Medicaid drug-reimbursement benefits on the use of psychotropic agents and acute mental health services by patients with schizophrenia. New England Journal of Medicine 331:650-655, 1994
18.
McCombs JS, Nichol MB, Stimmel GL, et al: The cost of antidepressant drug therapy failure: a study of antidepressant use patterns in a Medicaid population. Journal of Clinical Psychiatry 51:60-71, 1990
19.
Fenton WS, Blyler CR, Heinssen RK: Determinants of medication compliance in schizophrenia: empirical and clinical findings. Schizophrenia Bulletin 23:637-651, 1997
20.
Kent S, Yellowlees P: Psychiatric and social reasons for frequent rehospitalization. Hospital and Community Psychiatry 45:347-349, 1994
21.
Weiden PJ, Dixon L, Frances A, et al: Neuroleptic noncompliance in schizophrenia, in Advances in Neuropsychiatry and Psychopharmacology, vol 1. Edited by Tamminga CA, Schulz SC. New York, Raven, 1991
22.
Glazer WM, Ereshefsky L: A pharmacoeconomic model of outpatient antipsychotic therapy in "revolving door" schizophrenic patients. Journal of Clinical Psychiatry 57:337-345, 1996
23.
Hollingsworth EJ, Sweeney JK: Mental health expenditures for services for people with severe mental illnesses. Psychiatric Services 48:485-490, 1997
24.
Office of Mental Health, Chapter 51: Wisconsin Mental Health Law. Madison, Wisc, Wisconsin Department of Health and Social Services, 1985
25.
Shireman TI, Svarstad BL, Sweeney JK: Validity of claims for long-acting injectable medications. Journal of Pharmacoepidemiology 7:41-55, 1999
26.
Endicott J, Spitzer RL, Fleiss JL, et al: The Global Assessment Scale: a procedure for measuring overall severity of psychiatric disturbance. Archives of General Psychiatry 33:766-771, 1976
27.
Kerlinger FN, Lee HB: Foundations of Behavioral Research, 4th ed. Ft Worth, Tex, Harcourt, 2000
28.
Kelly GR, Scott JE: Medication compliance and health education among outpatients with chronic mental disorders. Medical Care 28:1181-1197, 1990
Information & Authors
Information
Published In
History
Published online: 1 June 2001
Published in print: June 2001
Authors
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBLogin options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).