Improving Treatment Engagement of Underserved U.S. Racial-Ethnic Groups: A Review of Recent Interventions
Abstract
Objective
Unequal mental health among U.S. underserved racial-ethnic populations has become a prominent national concern. Contributing to this inequity is our limited ability to engage individuals from underserved populations into treatment. To help address this, a systematic literature review was conducted to examine the evidence base for interventions that can improve mental health treatment engagement among underserved racial-ethnic minority populations.
Methods
A MEDLINE search and bibliographic review yielded 1,611 studies that were reviewed according to several inclusion criteria: publication during or after 2001, U.S. adult sample, a randomized design, sufficient (≥50%) representation of underserved racial-ethnic groups, adequate sample size (≥27 participants per condition), explicit focus on mental health treatment engagement, and evaluation of an engagement outcome (for example, adherence or retention).
Results
Ten studies met inclusion criteria. Evidence supported the efficacy of collaborative care for depression as an engagement enhancement intervention among underserved racial-ethnic populations. Several other interventions demonstrated possible efficacy. The effect of the interventions on clinical outcomes, such as symptom improvement and rehospitalization, was mixed.
Conclusions
Collaborative care for depression can be recommended for improving engagement in depression care in primary care among underserved racial-ethnic populations. Future research should continue to examine approaches with initial evidence of efficacy in order to expand the number of engagement enhancement interventions for underserved racial-ethnic adult populations. Additional issues for future engagement research include relative intervention efficacy across racial-ethnic groups, inclusion of other understudied groups (for example, Asian Americans and Native Americans), and greater clarification of the impact of improved engagement on clinical outcomes.
Underserved racial-ethnic groups in the United States experience many disparities in health and health care, including higher risk of certain illnesses, lower access to health care, and lower treatment quality (1). Similar disparities have been found in mental health care, especially in regard to lower engagement by patients from underserved racial-ethnic groups (2). This raises concerns as to whether the mental health needs of underserved racial-ethnic communities are being adequately met. This article focuses on disparities in patient engagement with mental health care, as highlighted in the U.S. Surgeon General’s report on culture, race, and ethnicity (2) and in a 2003 report from the Institute of Medicine (1).
The process of mental health treatment engagement can be seen as occurring on a continuum, beginning with the decision about whether to seek care, followed by ongoing decisions about whether to remain involved in treatment and optimally participate in the various therapeutic components of care. Medication adherence is one such component; it involves maintaining adequate medication dosing and continuity, particularly for mental health conditions that have established guidelines for dosing and treatment periods. Optimal treatment also requires continuity in other aspects of care, such as psychotherapy session attendance and outpatient follow-up after inpatient treatment. Thus treatment engagement is a broad-level process that consists of a series of linked steps: encouraging treatment seeking when there is a need, continuity in various aspects of care (including visit participation), treatment retention, and medication adherence.
Research conducted after publication of the U.S. Surgeon General’s report on culture, race, and ethnicity in mental health in 2001 (2) has consistently shown that our current mental health care system is less effective at engaging members of underserved racial-ethnic groups in services. This is evidenced by a range of engagement indices, including whether formal treatment is sought (3–5), number of visits (6,7), retention in treatment (8,9), follow-up with aftercare subsequent to inpatient discharge (9–11), and psychotropic medication adherence (12–15). To illustrate, a study of a nationally representative sample showed that among individuals with a diagnosis of major depressive disorder, only 31% of Asian Americans, 36% of Latinos, and 41% of African Americans reported seeking depression care in the previous 12 months, compared with 60% of non-Latino whites (3). With respect to antidepressant adherence, another study of a nationally representative sample found that Latino ethnicity and black race were associated with a 42% and 24%, respectively, lower likelihood of continuing an antidepressant for 30 days (15). There is also evidence that these disparities may be worsening. For example, in a nationally representative sample, African Americans’ and Latinos’ use of antidepressants was shown to have remained markedly low over time (1996–2005), despite a trend of increasing use among non-Latino whites during the same period (16).
On the basis of these types of findings, a group of nationally recognized experts on issues of treatment engagement and racial-ethnic disparities convened in a roundtable meeting in 2010 (17). They discussed the problem of low rates of mental health treatment engagement, with a particular focus on racial-ethnic disparities. Meeting participants were researchers, policy makers, consumer advocates, public mental health system leaders, and representatives of the National Institute of Mental Health. The meeting aimed to generate recommendations for a research agenda and for policy initiatives that would address racial-ethnic disparities in treatment engagement (17).
This article focuses on interventions that can improve mental health treatment engagement among underserved racial-ethnic groups. The focus is based on the view that improved engagement is a suitable target for reducing disparities in mental health treatment. If effective treatments are available, not seeking needed care leaves individuals and their families to suffer with the burden of mental illness. Among those who are able to access effective treatments, engagement is critical for optimizing outcomes. For individuals with major depression, adherence to antidepressant treatment helps reduce the risk of relapse and increases the probability of an optimal treatment response (13,18–21). In schizophrenia treatment, medication adherence and outpatient care attendance are associated with a lower likelihood of relapse and rehospitalization (22–28). Similar outcomes (for example, lower rates of relapse and rehospitalization and lower suicide risk) are observed when patients with bipolar disorder achieve optimal medication adherence (23,29–32).
This body of evidence points to the need to improve treatment engagement as a vehicle for reducing mental health disparities. At the roundtable meeting that focused on these engagement problems, a number of attendees proposed that recent advances in engagement intervention efficacy be reviewed and described according to their relative level of evidence for underserved racial-ethnic populations (17). Such information can help identify approaches that have sufficient evidence for implementation, as well as suggest areas for further research. Following this recommendation, we undertook a systematic review of interventions that can improve treatment engagement among patients from underserved racial-ethnic communities who have major mental illnesses. Our review focused on the research generated after publication of the Surgeon General’s report on culture, race, and ethnicity in 2001 (2).
Methods
A systematic literature search began with a MEDLINE search in September 2011. Search terms were entered for engagement (for example, “adherence,” “compliance,” and “engagement”), psychiatric disorders (for example, “psychiatric,” “depression,” “mania,” and “schizophrenia”), and “intervention.” Next, the abstracts of the articles returned from this search were reviewed for relevance. Bibliographies of relevant articles were reviewed to identify additional publications. As noted earlier, we operationalized the broad concept of treatment engagement via multiple indicators of this concept—treatment initiation, retention, number of mental health visits, and medication adherence.
Several inclusion criteria for articles were used: publication during or after 2001; sample of U.S. adults; use of a randomized design; sufficient racial-ethnic representation; minimally adequate sample size (that is, ≥27 participants per condition); an explicitly stated objective for the intervention or approach of improving mental health treatment engagement (for example, treatment entry, retention, and medication adherence); and evaluation of an engagement outcome, such as medication adherence or visit attendance. The minimum sample size was selected on the basis of a review of published studies, which found that a sample size of 27 had minimally acceptable power for the median effect size of the studies reviewed (33). For the final criterion, we included studies whose samples included at least 50% representation of underserved racial-ethnic groups. We also included studies that had less than 50% representation but that had specifically analyzed whether there were racial-ethnic effects for the intervention (that is, moderator effects). We chose a criterion of 50% representation based on review of previous studies, whose criteria ranged from 50% to 75% (34,35). We expected that the available pool of studies meeting criteria would be small, so our criterion of 50% was chosen to balance sufficient inclusion of studies for meaningful review with the need for study results to be sufficiently applicable to underserved racial-ethnic groups.
To assess the level of evidence for engagement interventions, we reviewed previously reported guidelines for evaluating evidence-based psychotherapies (36). The term “possibly efficacious” was used to denote interventions or approaches whose efficacy was supported by one study involving underserved racial-ethnic groups. “Efficacious” was used when at least two studies by different research teams supported the efficacy of the intervention or approach with samples of underserved participants.
Results
Overview of the evidence
The initial database search resulted in 1,522 articles. Bibliographic review identified 89 additional articles, for a total of 1,611. Forty-eight articles met the inclusion criteria with the exception of the criterion for sufficient racial-ethnic representation. This criterion excluded 37 (77%) articles, of which 11 did not report racial-ethnic data, and 26 had less than 50% racial-ethnic minority representation and did not analyze racial-ethnic effects for the intervention. Of the 37 publications that described the sample by race-ethnicity, 18 (47%) reported only that the sample was nonwhite or from minority groups, without specifying a specific group (for example, Latino, African American, or Asian). Eleven published papers met our full criteria. One of these studies (37) reanalyzed previous data (38). We combined these two publications, resulting in a total of ten studies that were the focus of our review.
Table 1 summarizes key characteristics of these ten engagement studies. Table 2 describes their respective interventions, outcomes evaluated, and results. Three studies focused on depression treatment (39–41), three focused on schizophrenia treatment (42–44), and three examined samples with various diagnoses (37,38,45,46). One study focused on patients with suicide-related issues (47). All studies except one characterized participants’ specific racial-ethnic background (for example, African American) (41). Three of the ten studies reported specific sociocultural considerations that were incorporated into the intervention studied (39,40,44). Three of the ten studies reported including Spanish-speaking participants (39,40,44). Finally, some type of fidelity monitoring of the intervention was reported for six studies (39,41–45).
| Intervention and study | Sample | Design | Follow-up period |
|---|---|---|---|
| Collaborative care | |||
| Ell et al., 2010 (39) | Primary care patients with diabetes and major depression; 100% Latino; 84% Spanish speaking; N=387 | Randomized controlled trial (RCT); intervention versus usual care enhanced with a depression pamphlet and community resource list | 18 months |
| Miranda et al., 2003 (40) | Primary care patients in managed care practices with depression; 31% Latino; 7% African American; Spanish speakers enrolled; N=1,269 | RCT, where sites were randomly assigned to receive quality improvements involving the implementation of collaborative care or to provide usual care | 12 months |
| Simon et al., 2004 (41) | Primary care patients in managed care practices with depression; 20% nonwhite; N=600 | RCT; telephone care management versus telephone care management plus telephone psychotherapy versus treatment as usual; fidelity to intervention reported via audiotape review | 6 months |
| Multifamily groups for adherence: Kopelowicz et al., 2012 (44) | Family members of community mental health patients treated for schizophrenia; 100% Mexican American; 100% Spanish speaking; N=174 | RCT; multifamily groups versus multifamily groups with a focus on adherence versus treatment as usual; all intervention sessions were monitored for fidelity by the investigators | 24 months |
| Cognitive adaptive training (CAT): Velligan et al., 2008 (42) | Patients with schizophrenia recently discharged from inpatient treatment; 37% Latino; 21% African American; N=95 | RCT; full-CAT, which included adaptive supports for full community living, versus pharm-CAT, which included adaptive supports targeting only medications, versus treatment as usual; fidelity monitoring via 30% audiotape review and observation of home environment modifications | 18 months |
| Adherence counseling: Hudson et al., 2008 (43) | Veterans with schizophrenia receiving treatment at one of 6 sites; 69% “nonwhite” but described as “mostly African American”; N=349 | Site randomization: 3 of 6 sites randomized within region to implement guideline medication management for schizophrenia versus guideline medication management for schizophrenia plus adherence counseling; fidelity monitored by review of notes | 6 months |
| Brief critical time intervention: Dixon et al., 2009 (45) | Veterans being discharged from inpatient acute psychiatry; 55% African American; 2% Latino; N=135 | RCT; brief critical time intervention versus treatment as usual; fidelity monitored by review of chart notes | 6 months |
| Mobile crisis team: Currier et al., 2010 (47) | Participants discharged from emergency department after being seen for suicidality; 36% African American; 13% Latino; 1% Native American; N=120 | RCT; mobile crisis team versus outpatient appointment scheduled within 5 days | 6 months |
| Meds-Help (pharmacist intervention): Valenstein et al., 2011 (46) | Veterans with serious mental illness receiving antipsychotic medication; 45% African American; 3% Latino; 3% Asian; N=118 | RCT; Meds-Help versus treatment as usual | 12 months |
| Telephone-based motivational interviewing: Zanjani et al., 2008 (38); Zanjani et al., 2010 (37) | Veterans referred from primary care to specialty mental health care; diagnoses were mixed; 69% non-Caucasian; 3% Latino; remainder described as “primarily African American”; N=113 | RCT; telephone-based referral care management versus standard referral; fidelity monitored via audiotape | 6 months |
| Study | Intervention | Sociocultural enhancements | Outcome and measuresa | Results |
|---|---|---|---|---|
| Collaborative care | ||||
| Ell et al., 2010 (39) | Linkage to antidepressant or psychotherapy; monthly telephone monitoring and adherence encouragement; care navigation support; psychiatrist consultative support for clinicians | Addressed treatment misconceptions and stigma; treatment assigned based on preference; open-ended patient support group throughout the 12-month follow-up period | Receipt of antidepressant or psychotherapy assessed via medical record review and self-report; depression assessed with the SCL-20 and PHQ-9 | The intervention improved receipt of depression treatment (antidepressant or psychotherapy) and the likelihood of having a 50% reduction in depressive symptoms and depression remission. |
| Miranda et al., 2003 (40) | Linkage to primary care–based antidepressant or psychotherapy; monthly telephone monitoring and adherence encouragement | Materials reviewed for cultural sensitivity; English and Spanish materials; diverse providers featured in video; addressed barriers among Latinos and African Americans; supervision from investigators from racial-ethnic minority groups | Receipt of appropriate guideline-concordant care for antidepressant and psychotherapy assessed via a survey; diagnosis of a depressive disorder with the CIDI | Sites receiving the quality improvement intervention demonstrated increased rates of appropriate care and lower rates of probable depression. Effects on lowering probable depression were stronger among participants from minority groups. |
| Simon et al., 2004 (41) | Telephone care management (TCM): telephone and mail contacts to assess symptoms and antidepressant use and address concerns by using motivational interviewing; updates provided to primary care physician; self-help materials; TCM plus 8 sessions of telephone-based cognitive-behavioral therapy (CBT) | None reported | Mental health visits (telephone or in person) and medication refills and dosage adequacy assessed via electronic medical record review; depression assessed by the PHQ-9 and SCL-20; 7-point self-rated improvement | TCM was superior to usual treatment in improved antidepressant use. TCM plus CBT was superior to usual treatment in improved antidepressant use but the difference was not significant. Both interventions were superior to usual treatment in improved psychotherapy. Compared with usual treatment, TCM plus CBT improved depression on several outcomes, whereas TCM resulted in greater self-rated improvement but no improvement in mean depression score. |
| Multifamily groups (MFGs) for adherence: Kopelowicz et al., 2012 (44) | MFGs were led to utilize problem solving and address adherence barriers and beliefs about illness and treatment | Assessment of socioculturally based attitudes, norms, and perceived resources toward the treatment of schizophrenia using the Theory of Planned Behavior Inventory | Medication adherence assessed by the Treatment Compliance Interview, which gathers multiple sources of adherence data; hospitalization | Adherence-focused MFGs improved medication adherence and rates of and time to hospitalization, compared with MFGs and usual treatment. |
| Cognitive adaptive training (CAT): Velligan et al., 2008 (42) | Full-CAT: use of environmental supports and structure to provide prompting for overall functioning (such as grooming); pharm-CAT: use of environmental supports and structure to provide prompting for medication adherence | None reported | Medication adherence assessed by unannounced pill counts and review of pharmacy records; symptoms assessed with the BPRS; functioning assessed with the SOFAS | Full-CAT and pharm-CAT were both superior to usual treatment in improving adherence; the two interventions did not differ on adherence. Both increased time to relapse, compared with usual treatment. Full-CAT improved functional outcomes (as measured by the SOFAS) more than pharm-CAT or usual treatment. |
| Adherence counseling: Hudson et al., 2008 (43) | Nurses provided counseling to address 9 barriers to adherence | None reported | Medication adherence assessed via patient self-report and review of clinical notes | Patients at sites providing adherence counseling were more likely to be adherent at 6 months. |
| Brief critical time intervention: Dixon et al., 2009 (45) | Case manager followed patient through postdischarge and developed rapport, individual treatment goals, strategies for managing adherence barriers and maintaining outpatient engagement; outreach visits were used to maintain engagement | None reported | Outpatient visit engagement, visit continuity, and inpatient and emergency care use assessed via electronic medical record review; symptoms and satisfaction with care assessed with the BPRS and QOLI | The intervention improved various indices of outpatient engagement; levels of satisfaction with legal and safety issues; and frequency of social contacts. No group differences were observed in BPRS scores or in use of inpatient or emergency care. |
| Mobile crisis team: Currier et al., 2010 (47) | Clinical assessment within 48 hours of discharge at a location of the patient’s choice (such as home) | None reported | Outpatient visit continuity; symptoms assessed with the HDRS, BPRS, BASIS-32, and Scale for Suicidal Ideation | Patients served by the mobile team were more likely to attend outpatient care but did not have more total visits than those in usual care. No differences in symptom measures by condition were noted. |
| Meds-Help (pharmacist intervention): Valenstein et al., 2011 (46) | Enhanced medication packaging, medication education, refill reminders, clinician notification of nonrefill | None reported | Medication adherence assessed via electronic medical record pharmacy refill data, self-report, and blood concentrations; symptoms assessed with the PANSS and QWB; satisfaction assessed with the CSQ; psychiatric hospitalizations | Meds-Help patients showed improved adherence at 6 and 12 months. Conditions did not differ on PANSS, QWB, or CSQ scores or on hospitalizations. |
| Telephone-based motivational interviewing: Zanjani et al., 2008 (38); Zanjani et al., 2010 (37) | 1 or 2 sessions of telephone-based motivational interviewing | None reported | Outpatient visit engagement assessed via electronic medical record review; symptoms and diagnosis assessed with the PHQ-9, MINI, and MOS-SF-12 | Patients in the intervention were more likely than those in usual care to attend a mental health appointment and to have more appointments. No effects were found on clinical symptom measures. |
a
SCL-20, 20-item Symptom Checklist Depression Scale; PHQ-9, Patient Health Questionnaire–9; CIDI, Composite International Diagnostic Interview; BPRS, Brief Psychiatric Rating Scale; SOFAS, Social and Occupational Functioning Assessment Scale; QOLI, Quality of Life Interview; HDRS, Hamilton Depression Rating Scale; BASIS-32, 32-item Behavior and Symptom Identification Scale; PANSS, Positive and Negative Syndrome Scale; QWB, Quality of Well-Being Scale; CSQ, Client Satisfaction Questionnaire; MINI, MINI Neuropsychiatric Interview; MOS-SF-12, Medical Outcomes Study 12-tem Short-Form Health Survey
Overall, only collaborative care was found to be efficacious for improving engagement among underserved racial-ethnic groups, notably African Americans and Latinos, with support provided by all three studies reviewed here (39–41). Two of these collaborative care studies included Latino participants (38,39), and one study also examined effects on African-American and Latino participants (40). For one study, specific racial-ethnic group (for example, Latino or African American) was not reported (41). Specific sociocultural considerations for the intervention were reported for two studies (39,40).
Other interventions received initial support or demonstrated promise. Three interventions were possibly efficacious for improving engagement among individuals from underserved racial-ethnic groups who were treated for schizophrenia (42–44). Four interventions were possibly efficacious for improving treatment engagement for a variety of diagnoses (37,38,45–47).
In terms of outcomes, all but one study examined a clinical outcome (for example, symptom improvement or relapse) (43). Of the nine studies that examined clinical outcomes, five reported positive effects (39–42,44), three did not find significant effects (37,46,47), and one reported mixed findings (45). Only two studies specifically examined whether the intervention’s effects on clinical outcomes differed by race or ethnicity (40,41). Both of these studies examined collaborative care for depression, and one found intervention effects to be higher among participants from underserved racial-ethnic groups (40), whereas the other study found no differences by race-ethnicity (41).
Description of interventions by target population
Engagement interventions for depression.
Collaborative care for depression was the only approach identified by this review that could be designated as efficacious for engagement. Three studies by different research groups showed that interventions that incorporated principles of collaborative care improved rates of receipt of depression treatment (39–41). Ingredients of this intervention include use of patient preference to choose a primary care–based treatment, use of a depression care manager who monitors symptoms and medication adherence during a follow-up period, and psychiatrist consultation with the primary care physician.
Collaborative care has received ample support in previous research (48). Therefore, the studies identified here, which met the race-ethnicity inclusion criterion, build on previous findings and provide evidence of generalizability to more diverse populations. It is noteworthy that collaborative care was also efficacious for improving clinical outcomes. Furthermore, results by Miranda and colleagues (40) suggest the possibility of an enhanced effect for patients from underserved communities. This is consistent with recent research showing an enhanced clinical effect for African Americans who received collaborative care for depression (49). It should also be noted that the study by Miranda and colleagues (40) utilized a quality improvement design, which suggests that collaborative care programs can be feasibly implemented in real-world settings and remain effective.
Engagement interventions for schizophrenia.
Emphasizing a family-based approach, one study examined multifamily groups (MFGs) that were adapted to address medication adherence among Mexican Americans who had a diagnosis of schizophrenia or schizoaffective disorder (44). The MFGs were also adapted to incorporate social norms regarding the treatment of schizophrenia in this population. Each group included five to eight family members to address problem solving, adherence barriers, and beliefs that lead to lower adherence. This intervention improved medication adherence significantly more than standard multifamily groups and treatment as usual. The same pattern of findings was observed for improving time to rehospitalization, an effect that was partially mediated by medication adherence. This initial trial therefore established this approach as possibly efficacious for improving medication adherence, most likely among Latinos with schizophrenia. The adapted MFGs also appeared to improve clinical outcomes.
Also focusing on patients with schizophrenia, another study examined an intervention that targets environmental cues that promote medication adherence and other adaptive behaviors. Specifically, cognitive adaptation training (CAT) utilizes cognitive compensatory strategies and environmental supports to prompt adaptive behaviors (42), such as placing notes or signs in the home environment to prompt activities of daily living. The trial examined two interventions that incorporated these principles—one broadly targeting overall functioning (full-CAT) and another focusing specifically on medication adherence (pharm-CAT). Both of these interventions were compared with treatment as usual. In a sample in which 37% of participants were Latino and 21% were African American, both CAT interventions were equally effective in improving adherence. The study also examined clinical outcomes and found that both interventions also improved the amount of time to relapse. Although the full-CAT intervention improved functional outcomes significantly more than the pharm-CAT intervention, both were superior to treatment as usual. The use of CAT therefore is possibly efficacious for improving medication adherence. Favorable effects on clinical outcomes were also observed in this trial.
Another study focused on a similar population but pursued an intervention that utilized counseling to help patients to recognize and overcome treatment adherence barriers. Serving a mostly African-American sample (69%), nursing staff in a multisite U.S. Department of Veterans Affairs (VA) study assessed and counseled patients on adherence barriers (for example, medication fears, problems with the medication regimen, and adverse drug reactions) (43). Sessions occurred during each scheduled mental health visit for a period of six months. The study included two sites within three VA networks (Veterans Integrated Service Networks, or VISNs), where each site within a VISN was randomly assigned to implement either the basic guidelines for medication management of schizophrenia or the basic guidelines enhanced with the adherence counseling. The counseling intervention resulted in positive effects on medication adherence and is therefore possibly efficacious. Data are still needed on clinical symptom outcomes.
Engagement interventions for diagnostically mixed populations.
Also utilizing an adherence-counseling approach, a brief critical time intervention addressed outpatient engagement but did so by targeting the critical period of inpatient discharge (45). Case managers identified possible barriers to treatment engagement and established a plan for managing them. A key feature of this intervention is that counseling began before discharge and continued for three months after discharge, thereby utilizing a model that emphasized linkage to outpatient care. In a randomized controlled trial with a predominantly African-American sample, the intervention improved outpatient treatment engagement in terms of reduced time to first outpatient appointment, greater likelihood of an outpatient visit, greater number of visits, and greater likelihood of outpatient continuity of care. The brief critical time intervention appears possibly efficacious for improving outpatient continuity after inpatient discharge among African Americans. Analyses of outcomes showed mixed findings, with improvement in number of social contacts but not symptomatology.
Another study, which also focused on improving outpatient treatment continuity after discharge from an acute care facility, examined the utility of community-based follow-up assessments (47). The aim was to improve outpatient continuity among patients discharged from an emergency department for treatment of suicide risk. Participants from underserved racial-ethnic groups were African American (36% of total sample) and Latino (13%). Mobile crisis teams conducted community-based follow-up assessments that evaluated symptoms and the need for treatment within 48 hours after discharge. As a key feature, the assessments occurred at a location of the patient’s choice (for example, his or her home). Compared with standard referral to outpatient treatment (with an appointment provided within five days of discharge), the use of mobile crisis teams significantly improved the likelihood of attending an initial outpatient appointment (70% versus 30%). This approach appears possibly efficacious for outpatient continuity of care. However, this study reported that the use of mobile crisis teams did not improve symptom or functional outcomes.
Another study that also focused on patients with diverse diagnoses examined a pharmacy-based intervention. Meds-Help was examined for its ability to improve antipsychotic medication adherence in a sample of veterans (46). Meds-Help involved medication packaging enhancements, a medication education session, refill reminders mailed two weeks before refill dates, and notification of treating clinicians when medications remained unfilled. Underserved participants were mostly African American. A randomized design compared treatment as usual to Meds-Help, and the latter was found to improve medication adherence during a 12-month period. Meds-Help therefore appears possibly efficacious for improving medication adherence. Meds-Help was not found to improve patients’ symptoms, quality of life, or satisfaction, but the lack of findings on these clinical outcomes may have been related to insufficient statistical power.
Finally, another study examined the important issue of improving mental health referral follow-up rates. Specifically, the study recruited veterans being seen in primary care who were referred to mental health specialty care. The intervention consisted of brief, telephone-based, motivational interviewing sessions, and the sample consisted of U.S. veterans (37,38). A randomized design compared usual care with telephone-based motivational interviewing (one or two sessions of 15 minutes each). The intervention resulted in improved initial appointment attendance rates and an increase in the total number of appointments attended. Brief telephone-based motivational interviewing therefore appears possibly efficacious for improving session attendance rates among veterans referred to outpatient mental health care. Regarding clinical outcomes, no intervention effects were observed for physical and mental health functioning, depression, and alcohol use (37,38).
Discussion
Our review illustrates some recent gains in engagement research focused on underserved racial-ethnic populations, as well as some persisting problems. Many of the problems highlighted in the U.S. Surgeon General's report on culture, race, and ethnicity remain (2). Samples are often insufficiently characterized according to race and ethnicity. Also, very few studies conduct analyses that examine whether race-ethnicity moderates intervention effects. These persisting limitations curtail our ability to assess the evidence base for improving engagement among the very populations who are disproportionately affected by this issue. Although the studies reviewed had at least 50% racial-ethnic diversity, the infrequent analysis of intervention effects by race-ethnicity leaves open many questions about relative efficacy. This is also true of analyses examining the effects of language (for example, language-compatible services for non–English speakers), which has been shown to be associated with engagement (12,50,51).
Given the multiple areas in which racial-ethnic disparities in treatment engagement have been documented, much work remains to be completed. Missing from this review are interventions addressing engagement of patients with bipolar disorder; such interventions have been shown to be effective in studies that did not meet our inclusion criterion pertaining to representation of underserved racial-ethnic groups (52,53). Also, studies addressing engagement interventions among Asian American and Native American populations are conspicuously lacking (3,54). Also missing are studies focusing on older adults from underserved racial-ethnic groups. Such studies may be relevant given some of the unique issues arising from the intersection of age and minority status. Given the richness of the cross-cultural literature in the last decade, a myriad of questions remain unanswered by this review, and more research is clearly needed.
On the other hand, on the basis of the studies available, we have been able to describe approaches that were either efficacious or possibly efficacious for underserved racial-ethnic populations. One conclusion is that primary care models that incorporate collaborative care principles are likely to improve engagement (that is, receipt of depression care and medication continuity) and clinical outcomes among African-American and Latino patients with depression. The studies of collaborative care reviewed here build on previous empirical studies (48) and therefore have considerable empirical support. Implementation of this model in primary care settings that serve African-American and Latino populations can be recommended.
A number of the interventions reviewed strongly indicate the need for further research in order to establish the needed level of evidence for recommending implementation. Our review identified several interventions that seem possibly efficacious for improving mental health treatment engagement (38,44–47). Other interventions were excluded from the review because they used nonrandomized designs, although they were focused on patients from underserved racial-ethnic groups. These interventions were early in the treatment development process, and additional research with improved methods may indicate efficacy (55,56).
Other interventions have been studied that may prove to be efficacious for improving engagement among underserved racial-ethnic populations (57–63). However, these studies were not included in this review because their racial-ethnic representation was less than 50% or because the race-ethnicity of participants was not reported or was not examined as a moderating variable. Relatively rapid gains can be made to address this knowledge gap through the analysis of existing data sets to explore the role of race-ethnicity in moderating effects of interventions that aim to improve mental health treatment engagement. The feasibility of such analyses is supported by the fact that many studies considered for review had representation of underserved racial-ethnic groups above 20%, although still below the 50% criterion.
Although not a central focus, the impact of engagement interventions on clinical outcomes was considered in our review. As discussed in the treatment engagement roundtable meeting (17), this relationship is not easily discernible. Indeed, our review found mixed results, leaving the question open as to whether targeting engagement leads to improved outcomes. However, understanding these mixed findings also requires sorting out issues of research methodology and treatment context, as well as the nature of the interventions themselves. Regarding research methodology, some of the studies reviewed point to insufficient statistical power as a reason for the lack of observation of effects on clinical outcomes (46). Another methodological issue involves the fact that rehospitalization rates among patients who do not attend postdischarge outpatient care tend to increase with time (24), thereby suggesting that studies with longer assessment windows may be more likely to detect the effects of engagement interventions on clinical outcomes. In this review, some studies examining postdischarge outpatient continuity utilized assessment windows of six months or less (45,47). Thus lack of findings pertaining to outcomes may be related to the brevity of the assessment windows.
The context of care also affects the impact of engagement interventions on clinical outcomes. Different clinical settings will vary in the availability of effective treatments. For this reason, studies examining engagement interventions within naturalistic contexts can expect variability in the degree to which improved engagement translates into improved clinical outcomes. Thus factors that are independent of the effectiveness of the engagement intervention may account for the impact on clinical outcomes. This issue may be of special concern for individuals from underserved racial-ethnic populations, who are more likely to seek care in settings that face quality challenges (64–66). Strategies to improve engagement are likely to have limited impact if the increased engagement is to treatments of poor quality or limited ability to address the unique needs of different racial-ethnic groups (17,67,68).
Yet another factor affecting the relationship between engagement interventions and clinical outcomes pertains to the nature of the engagement interventions themselves. Many interventions have the sole aim of increasing engagement in treatment and have very few active treatment components that would have a direct effect on clinical outcomes. In contrast, interventions such as collaborative care have engagement components (for example, use of patient preference and telephone monitoring of medication adherence), but they also provide direct treatment for depression via guideline-concordant pharmacotherapy and evidence-based brief psychotherapy (40,41). These combined interventions may be more likely to have effects on clinical outcomes. In contrast, the impact of interventions that are more purely engagement oriented will be more dependent on the naturalistic treatment context.
The discussion at the roundtable meeting highlighted the need for engagement research to examine clinical outcomes. The presence of some negative findings in this review does not refute the notion that targeting engagement improves clinical outcomes. Instead, there are methodological and treatment context questions that should guide the interpretation of these findings. Future studies can address these issues by utilizing frameworks that measure indicators of the quality of care and assess their role in moderating the efficacy of engagement interventions on clinical outcomes. Also, research is needed that is sufficiently powered and incorporates assessments that are optimally designed to detect effects on clinical outcomes. Addressing these conceptual and methodological challenges will help the field more precisely evaluate the impact of engagement interventions on clinical outcomes. Finally, as mental health care becomes more patient oriented, engagement research will need to identify outcomes that are a priority for treatment recipients and relevant for diverse populations (69). These outcome measures will allow us to better clarify the impact of engagement interventions.
Finally, as work expands on improving engagement among underserved racial-ethnic groups, conceptual models are needed to determine which interventions to implement. For this review, we conceptualized treatment engagement as a continuum, beginning with treatment seeking and followed by various indicators of remaining engaged with treatment (that is, visit continuity and medication adherence). The interventions reviewed here targeted various points in this continuum, utilizing different ingredients to address engagement. As a common element, however, all interventions appeared to utilize a frame of providing “engagement support.” That is, the approaches did not simply involve the removal of an engagement barrier or the provision of information about illness. Rather, the approaches involved actively reaching out to patients, providing encouragement, and working with them to maintain engagement with various forms of treatment. Three of the interventions reviewed incorporated cultural considerations as part of this process, a feature worth noting given the evidence that shows racial-ethnic variation in factors that affect engagement (for example, stigma, alliance with providers, and illness beliefs) (70–72). At this stage, we therefore conceptualize mental health treatment engagement as requiring active support and encouragement throughout the various stages of treatment. Future studies are needed to determine the degree to which culturally specific elements add to the impact of the intervention.
This review sought to provide an overview of the progress that has been made in engagement interventions across a range of psychiatric disorders and treatments. It was necessary to provide a cross-disorder perspective, because racial-ethnic disparities in treatment engagement span the range of psychiatric difficulties, and a synthesis of interventions that can address this problem was needed. Also, many treatment settings serve a diagnostically diverse population and are thereby in need of approaches that can be used across the range of psychiatric difficulties. However, this transdiagnostic focus has limitations. The factors that contribute to engagement problems are likely to vary by psychiatric disorder. For example, illness insight, stigma, and cognitive symptoms may vary by disorder, and all these factors are likely to have an impact on medication adherence (73–75). The effectiveness of treatment may also vary by disorder. As additional evidence accrues, future work can review engagement interventions for specific psychiatric problems by using frameworks that are guided by the engagement factors that are most relevant in those areas.
Conclusions
Collaborative care for depression appears efficacious for improving treatment engagement among Latino and African-American primary care patients with depression. Other approaches appear possibly efficacious for improving engagement in schizophrenia treatment. In addition, some approaches reviewed showed possible efficacy for improving treatment continuity after psychiatric inpatient discharge, discharge from an emergency department after treatment for suicide risk, and referral to specialty mental health care from primary care. Additional research is needed to study engagement interventions with other underserved racial-ethnic groups (for example, Asian Americans and Native Americans), to examine relative efficacy across racial-ethnic groups, and to better understand the degree to which improved engagement translates to improved outcomes.
Acknowledgments and disclosures
This project was supported by an investigator-initiated grant awarded to Dr. Dixon from Ortho-McNeil Janssen Scientific Affairs. This funding supported a two-day meeting to discuss issues of treatment engagement, which helped generate the concept for this article. The authors are grateful to these meeting participants for their thoughtful comments. Project support and guidance were also provided by Anne Michaels and Marion Crawford Kiley, M.B.A.
The authors report no competing interests.
References
1.
Smedley BD, Stith AY, Nelson AR: Unequal Treatment: Confronting Racial and Ethnic Disparities in Healthcare. Washington, DC, National Academies Press, 2003
2.
Mental Health: Culture, Race, and Ethnicity—A Supplement to Mental Health: A Report of the Surgeon General. Rockville, Md, US Department of Health and Human Services, US Public Health Service, 2001
3.
Alegría M, Chatterji P, Wells K, et al.: Disparity in depression treatment among racial and ethnic minority populations in the United States. Psychiatric Services 59:1264–1272, 2008
4.
Paulose-Ram R, Safran MA, Jonas BS, et al.: Trends in psychotropic medication use among US adults. Pharmacoepidemiology and Drug Safety 16:560–570, 2007
5.
Wang PS, Lane M, Olfson M, et al.: Twelve-month use of mental health services in the United States: results from the National Comorbidity Survey Replication. Archives of General Psychiatry 62:629–640, 2005
6.
Busch AB, Lehman AF, Goldman HH, et al.: Changes over time and disparities in schizophrenia treatment quality. Medical Care 47:199–207, 2009
7.
Kuno E, Rothbard AB: Racial disparities in antipsychotic prescription patterns for patients with schizophrenia. American Journal of Psychiatry 159:567–572, 2002
8.
Fischer EP, McCarthy JF, Ignacio RV, et al.: Longitudinal patterns of health system retention among veterans with schizophrenia or bipolar disorder. Community Mental Health Journal 44:321–330, 2008
9.
Virnig B, Huang Z, Lurie N, et al.: Does Medicare managed care provide equal treatment for mental illness across races? Archives of General Psychiatry 61:201–205, 2004
10.
Stein BD, Kogan JN, Sorbero M: Substance abuse detoxification and residential treatment among Medicaid-enrolled adults: rates and duration of subsequent treatment. Drug and Alcohol Dependence 104:100–106, 2009
11.
Stein BD, Kogan JN, Sorbero MJ, et al.: Predictors of timely follow-up care among Medicaid-enrolled adults after psychiatric hospitalization. Psychiatric Services 58:1563–1569, 2007
12.
Diaz E, Woods SW, Rosenheck RA: Effects of ethnicity on psychotropic medications adherence. Community Mental Health Journal 41:521–537, 2005
13.
Warden D, Rush AJ, Wisniewski SR, et al.: What predicts attrition in second step medication treatments for depression? A STAR*D report. International Journal of Neuropsychopharmacology 12:459–473, 2009
14.
Warden D, Trivedi MH, Wisniewski SR, et al.: Predictors of attrition during initial (citalopram) treatment for depression: a STAR*D report. American Journal of Psychiatry 164:1189–1197, 2007
15.
Olfson M, Marcus SC, Tedeschi M, et al.: Continuity of antidepressant treatment for adults with depression in the United States. American Journal of Psychiatry 163:101–108, 2006
16.
Olfson M, Marcus SC: National patterns in antidepressant medication treatment. Archives of General Psychiatry 66:848–856, 2009
17.
Dixon L, Lewis-Fernández R, Goldman HH, et al.: Adherence disparities in mental health: opportunities and challenges. Journal of Nervous and Mental Disease 199:815–820, 2011
18.
Melartin TK, Rytsälä HJ, Leskelä US, et al.: Continuity is the main challenge in treating major depressive disorder in psychiatric care. Journal of Clinical Psychiatry 66:220–227, 2005
19.
Geddes JR, Carney SM, Davies C, et al.: Relapse prevention with antidepressant drug treatment in depressive disorders: a systematic review. Lancet 361:653–661, 2003
20.
Frank E, Kupfer DJ, Perel JM, et al.: Three-year outcomes for maintenance therapies in recurrent depression. Archives of General Psychiatry 47:1093–1099, 1990
21.
Interian A, Ang A, Gara MA, et al.: The long-term trajectory of depression among Latinos in primary care and its relationship to depression care disparities. General Hospital Psychiatry 33:94–101, 2011
22.
Olfson M, Mechanic D, Hansell S, et al.: Predicting medication noncompliance after hospital discharge among patients with schizophrenia. Psychiatric Services 51:216–222, 2000
23.
Svarstad BL, Shireman TI, Sweeney JK: Using drug claims data to assess the relationship of medication adherence with hospitalization and costs. Psychiatric Services 52:805–811, 2001
24.
Nelson EA, Maruish ME, Axler JL: Effects of discharge planning and compliance with outpatient appointments on readmission rates. Psychiatric Services 51:885–889, 2000
25.
Klinkenberg WD, Calsyn RJ: Predictors of receipt of aftercare and recidivism among persons with severe mental illness: a review. Psychiatric Services 47:487–496, 1996
26.
Schoenbaum SC, Cookson D, Stelovich S: Postdischarge follow-up of psychiatric inpatients and readmission in an HMO setting. Psychiatric Services 46:943–945, 1995
27.
Winston A, Pardes H, Papernik DS, et al.: Aftercare of psychiatric patients and its relation to rehospitalization. Hospital and Community Psychiatry 28:118–121, 1977
28.
Cuffel BJ, Held M, Goldman W: Predictive models and the effectiveness of strategies for improving outpatient follow-up under managed care. Psychiatric Services 53:1438–1443, 2002
29.
Suppes T, Baldessarini RJ, Faedda GL, et al.: Risk of recurrence following discontinuation of lithium treatment in bipolar disorder. Archives of General Psychiatry 48:1082–1088, 1991
30.
Gianfrancesco FD, Sajatovic M, Rajagopalan K, et al.: Antipsychotic treatment adherence and associated mental health care use among individuals with bipolar disorder. Clinical Therapeutics 30:1358–1374, 2008
31.
Müller-Oerlinghausen B, Ahrens B, Volk J, et al.: Reduced mortality of manic-depressive patients in long-term lithium treatment: an international collaborative study by IGSLI. Psychiatry Research 36:329–331, 1991
32.
Revicki DA, Hirschfeld RM, Ahearn EP, et al.: Effectiveness and medical costs of divalproex versus lithium in the treatment of bipolar disorder: results of a naturalistic clinical trial. Journal of Affective Disorders 86:183–193, 2005
33.
Kazdin AE, Bass D: Power to detect differences between alternative treatments in comparative psychotherapy outcome research. Journal of Consulting and Clinical Psychology 57:138–147, 1989
34.
Jackson KF: Building cultural competence: a systematic evaluation of the effectiveness of culturally sensitive interventions with ethnic minority youth. Children and Youth Services Review 31:1192–1198, 2009
35.
Huey SJ, Polo AJ: Evidence-based psychosocial treatments for ethnic minority youth. Journal of Clinical Child and Adolescent Psychology 37:262–301, 2008
36.
Chambless DL, Hollon SD: Defining empirically supported therapies. Journal of Consulting and Clinical Psychology 66:7–18, 1998
37.
Zanjani F, Bush H, Oslin D: Telephone-based psychiatric referral-care management intervention health outcomes. Telemedicine Journal and e-Health 16:543-550, 2010
38.
Zanjani F, Miller B, Turiano N, et al.: Effectiveness of telephone-based referral care management: a brief intervention to improve psychiatric treatment engagement. Psychiatric Services 59:776–781, 2008
39.
Ell K, Katon W, Xie B, et al.: Collaborative care management of major depression among low-income, predominantly Hispanic subjects with diabetes: a randomized controlled trial. Diabetes Care 33:706–713, 2010
40.
Miranda J, Duan N, Sherbourne C, et al.: Improving care for minorities: can quality improvement interventions improve care and outcomes for depressed minorities? Results of a randomized, controlled trial. Health Services Research 38:613–630, 2003
41.
Simon GE, Ludman EJ, Tutty S, et al.: Telephone psychotherapy and telephone care management for primary care patients starting antidepressant treatment: a randomized controlled trial. JAMA 292:935–942, 2004
42.
Velligan DI, Diamond PM, Mintz J, et al.: The use of individually tailored environmental supports to improve medication adherence and outcomes in schizophrenia. Schizophrenia Bulletin 34:483–493, 2008
43.
Hudson TJ, Owen RR, Thrush CR, et al.: Guideline implementation and patient-tailoring strategies to improve medication adherence for schizophrenia. Journal of Clinical Psychiatry 69:74–80, 2008
44.
Kopelowicz A, Zarate R, Wallace CJ, et al.: The ability of multifamily groups to improve treatment adherence in Mexican Americans with schizophrenia. Archives of General Psychiatry 69:265–273, 2012
45.
Dixon L, Goldberg R, Iannone V, et al.: Use of a critical time intervention to promote continuity of care after psychiatric inpatient hospitalization. Psychiatric Services 60:451–458, 2009
46.
Valenstein M, Kavanagh J, Lee T, et al.: Using a pharmacy-based intervention to improve antipsychotic adherence among patients with serious mental illness. Schizophrenia Bulletin 37:727–736, 2011
47.
Currier GW, Fisher SG, Caine ED: Mobile crisis team intervention to enhance linkage of discharged suicidal emergency department patients to outpatient psychiatric services: a randomized controlled trial. Academic Emergency Medicine 17:36–43, 2010
48.
Gilbody S, Bower P, Fletcher J, et al.: Collaborative care for depression: a cumulative meta-analysis and review of longer-term outcomes. Archives of Internal Medicine 166:2314–2321, 2006
49.
Davis TD, Deen T, Bryant-Bedell K, et al.: Does minority racial-ethnic status moderate outcomes of collaborative care for depression? Psychiatric Services 62:1282–1288, 2011
50.
Sleath B, Rubin RH, Huston SA: Hispanic ethnicity, physician-patient communication, and antidepressant adherence. Comprehensive Psychiatry 44:198–204, 2003
51.
Hodgkin D, Volpe-Vartanian J, Alegría M: Discontinuation of antidepressant medication among Latinos in the USA. Journal of Behavioral Health Services and Research 34:329–342, 2007
52.
Depp CA, Lebowitz BD, Patterson TL, et al.: Medication adherence skills training for middle-aged and elderly adults with bipolar disorder: development and pilot study. Bipolar Disorders 9:636–645, 2007
53.
Sajatovic M, Davies MA, Ganocy SJ, et al.: A comparison of the life goals program and treatment as usual for individuals with bipolar disorder. Psychiatric Services 60:1182–1189, 2009
54.
Harris KM, Edlund MJ, Larson S: Racial and ethnic differences in the mental health problems and use of mental health care. Medical Care 43:775–784, 2005
55.
Alegría M, Polo A, Gao S, et al.: Evaluation of a patient activation and empowerment intervention in mental health care. Medical Care 46:247–256, 2008
56.
Post EP, Cruz M, Harman J: Incentive payments for attendance at appointments for depression among low-income African Americans. Psychiatric Services 57:414–416, 2006
57.
Finley PR, Rens HR, Pont JT, et al.: Impact of a collaborative pharmacy practice model on the treatment of depression in primary care. American Journal of Health-System Pharmacy 59:1518–1526, 2002
58.
Bauer MS, McBride L, Williford WO, et al.: Collaborative care for bipolar disorder: part II. impact on clinical outcome, function, and costs. Psychiatric Services 57:937–945, 2006
59.
Patel UB, Ni Q, Clayton C, et al.: An attempt to improve antipsychotic medication adherence by feedback of medication possession ratio scores to prescribers. Population Health Management 13:269–274, 2010
60.
Capoccia KL, Boudreau DM, Blough DK, et al.: Randomized trial of pharmacist interventions to improve depression care and outcomes in primary care. American Journal of Health-System Pharmacy 61:364–372, 2004
61.
Sirey JA, Bruce ML, Kales HC: Improving antidepressant adherence and depression outcomes in primary care: the treatment initiation and participation (TIP) program. American Journal of Geriatric Psychiatry 18:554–562, 2010
62.
Bao Y, Post EP, Ten TR, et al.: Achieving effective antidepressant pharmacotherapy in primary care: the role of depression care management in treating late-life depression. Journal of the American Geriatrics Society 57:895–900, 2009
63.
Cook PF, Emiliozzi S, Waters C, et al.: Effects of telephone counseling on antipsychotic adherence and emergency department utilization. American Journal of Managed Care 14:841–846, 2008
64.
Shi L: Experience of primary care by racial and ethnic groups in the United States. Medical Care 37:1068–1077, 1999
65.
van Ryn M, Burke J: The effect of patient race and socio-economic status on physicians’ perceptions of patients. Social Science and Medicine 50:813–828, 2000
66.
Bauer AM, Alegría M: Impact of patient language proficiency and interpreter service use on the quality of psychiatric care: a systematic review. Psychiatric Services 61:765–773, 2010
67.
Whaley AL, Davis KE: Cultural competence and evidence-based practice in mental health services: a complementary perspective. American Psychologist 62:563–574, 2007
68.
Baker FM, Bell CC: Issues in the psychiatric treatment of African Americans. Psychiatric Services 50:362–368, 1999
69.
Deegan PE: The importance of personal medicine: a qualitative study of resilience in people with psychiatric disabilities. Scandinavian Journal of Public Health 66:29–35, 2005
70.
Johnson RL, Roter D, Powe NR, et al.: Patient race/ethnicity and quality of patient-physician communication during medical visits. American Journal of Public Health 94:2084–2090, 2004
71.
Anglin DM, Link BG, Phelan JC: Racial differences in stigmatizing attitudes toward people with mental illness. Psychiatric Services 57:857–862, 2006
72.
Cooper LA, Gonzales JJ, Gallo JJ, et al.: The acceptability of treatment for depression among African-American, Hispanic, and white primary care patients. Medical Care 41:479–489, 2003
73.
Dell’Osso L, Pini S, Cassano GB, et al.: Insight into illness in patients with mania, mixed mania, bipolar depression and major depression with psychotic features. Bipolar Disorders 4:315–322, 2002
74.
Velligan DI, Lam F, Ereshefsky L, et al.: Psychopharmacology: perspectives on medication adherence and atypical antipsychotic medications. Psychiatric Services 54:665–667, 2003
75.
Link BG, Phelan JC, Bresnahan M, et al.: Public conceptions of mental illness: labels, causes, dangerousness, and social distance. American Journal of Public Health 89:1328–1333, 1999
Information & Authors
Information
Published In
Cover: Mom Alice, by William H. Johnson, circa 1944. Oil on cardboard, 25 × 31 inches. Howard University Gallery of Art, Washington, D.C.
History
Published in print: March 2013
Published online: 15 October 2014
Authors
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBLogin options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).