Peer Support Services for Individuals With Serious Mental Illnesses: Assessing the Evidence
Abstract
Objective
This review assessed the level of evidence and effectiveness of peer support services delivered by individuals in recovery to those with serious mental illnesses or co-occurring mental and substance use disorders.
Methods
Authors searched PubMed, PsycINFO, Applied Social Sciences Index and Abstracts, Sociological Abstracts, Social Services Abstracts, Published International Literature on Traumatic Stress, the Educational Resources Information Center, and the Cumulative Index to Nursing and Allied Health Literature for outcome studies of peer support services from 1995 through 2012. They found 20 studies across three service types: peers added to traditional services, peers in existing clinical roles, and peers delivering structured curricula. Authors judged the methodological quality of the studies using three levels of evidence (high, moderate, and low). They also described the evidence of service effectiveness.
Results
The level of evidence for each type of peer support service was moderate. Many studies had methodological shortcomings, and outcome measures varied. The effectiveness varied by service type. Across the range of methodological rigor, a majority of studies of two service types—peers added and peers delivering curricula—showed some improvement favoring peers. Compared with professional staff, peers were better able to reduce inpatient use and improve a range of recovery outcomes, although one study found a negative impact. Effectiveness of peers in existing clinical roles was mixed.
Conclusions
Peer support services have demonstrated many notable outcomes. However, studies that better differentiate the contributions of the peer role and are conducted with greater specificity, consistency, and rigor would strengthen the evidence.
As stated in the 2003 report of the President’s New Freedom Commission on Mental Health (1), mental health care should be “recovery-oriented”—meaning that services should be collaborative and respectful and should aim to help those with serious mental illnesses achieve a satisfying life even in the presence of symptoms. Peers are individuals with histories of successfully living with serious mental illness who, in turn, support others with serious mental illness. Many terms have been used to describe this group, including peer specialists and consumer-providers. However, they are frequently referred to as “peers,” and we have chosen to use that term here. Peers are believed to be particularly helpful in promoting recovery; therefore, the presence of peers within the continuum of care has expanded considerably for individuals with serious mental illnesses, and, in many cases, peer support services are provided to those with co-occurring substance use disorders (1–3).
This article reports the results of a literature review that was undertaken as part of the Assessing the Evidence Base (AEB) Series (see box on next page). For purposes of this series, the Substance Abuse and Mental Health Services Administration (SAMHSA) has described peer support services as a direct service that is delivered by a person with a serious mental illness to a person with a serious mental disorder (primarily schizophrenia, schizoaffective, or bipolar disorder) or a co-occurring mental and substance use disorder. The peer providers have progressed in recovery (often using treatment services) to the stage where they can manage their illness and pursue fulfilling lives. This specialized assistance offers social support before, during, and after treatment to facilitate long-term recovery in the community in which the recovering person resides.
About the AEB Series
The Assessing the Evidence Base (AEB) Series presents literature reviews for 13 commonly used, recovery-focused mental health and substance use services. Authors evaluated research articles and reviews specific to each service that were published from 1995 through 2012 or 2013. Each AEB Series article presents ratings of the strength of the evidence for the service, descriptions of service effectiveness, and recommendations for future implementation and research. The target audience includes state mental health and substance use program directors and their senior staff, Medicaid staff, other purchasers of health care services (for example, managed care organizations and commercial insurance), leaders in community health organizations, providers, consumers and family members, and others interested in the empirical evidence base for these services. The research was sponsored by the Substance Abuse and Mental Health Services Administration to help inform decisions about which services should be covered in public and commercially funded plans. Details about the research methodology and bases for the conclusions are included in the introduction to the AEB Series (25).
Table 1 presents the definition, goals, targeted populations, and service delivery settings for peer support services. These services are a form of peer support provided within the formal behavioral health services continuum (4). SAMHSA has included peer-based services in its National Registry of Evidence-Based Programs and Practices (5). Although peer2 support services described in this review are often delivered to those with co-occurring mental and substance use disorders, the primary aim of these services has been to address mental illness, and the commonality for individuals receiving these services has been the presence of a mental illness. An emerging type of peer support services is peer recovery support, which involves an individual in recovery from a substance use disorder providing services to others with substance use disorders. These services are addressed in a separate review in this series (6).
| Feature | Description |
|---|---|
| Service definition | Peer support services are delivered to a person with a serious mental illness or co-occurring mental and substance use disorders by a person in recovery. This specialized assistance offers social support before, during, and after treatment to facilitate long-term recovery in the community. |
| Service goals | Assist in the development of coping and problem-solving strategies to facilitate self-management of a person’s mental illness; draw upon lived experiences and empathy to promote hope, insights, and skills; help individuals engage in treatment, access supports in the community, and establish a satisfying life |
| Populations | Individuals with serious mental illnesses or those with co-occurring mental and substance use disorders |
| Settings of servicedelivery | Settings may vary and include inpatient facilities; outpatient facilities, including a range of clinical team types (for example, case management and homeless services); day treatment programs; and psychosocial clubhouses |
Policy makers and other leaders in behavioral health care need information about the effectiveness of peer support services and their value as a benefit covered by insurers. The objectives of this review were to describe peer support services and peer roles, rate the level of evidence of the research (defined here as methodological quality), and describe the effectiveness of the service (defined here as positive, negative, mixed, or null findings). To be useful for a broad audience, the scope of the review is brief and focuses on key findings and an overall assessment of research quality.
Other reviews of peer support services have been conducted. In 2002, Simpson and House (7) reviewed studies on this topic. In 2005, Doughty and Tse’s report (8) for the New Zealand Mental Health Commission used a broader typology that included “service user–run” and “service user–led” mental health services. In 2009, Rogers and colleagues’ report (9) from the Center for Psychiatric Rehabilitation categorized a variety of peer-delivered services that included those added to traditional services, those offered as a one-to-one service, and peer-delivered residential services. In 2011, Repper and Carter (10) reviewed the literature on peer support workers employed in mental health services, and Wright-Berryman and colleagues (11) examined the effects of peers on case management teams. In 2013, Pitt and colleagues (12) published a Cochrane review of peer support services that excluded quasi-experimental trials and studies involving peer-delivered curricula, and they conducted analyses pooling data across peer support services that may have varied. This AEB Series review is more inclusive than the Cochrane review, updates the other reviews, and provides an assessment of three specific types of peer support services delivered in traditional mental health systems.
Description of peer support services
Various forms of peer support have been addressed in the literature and are evident in practice. Historically, peer support began in the form of peer groups, in which participants with similar difficulties met to provide mutual support, discuss their problems, and receive empathy and suggestions from other members on the basis of shared experiences (13). From those origins, other variants of peer support were developed, including the establishment of organizations and programs run by individuals with mental illness.
This review, however, focuses on a particular aspect of peer support: the hiring of a person in recovery from a serious mental illness as an employee to offer services or supports to others with serious mental illnesses (4). Solomon (14) defined peer employees as “individuals who fill designated unique peer positions as well as peers who are hired into traditional mental health positions.” When peers are hired into existing mainstream positions, they typically must self-identify as having a serious mental illness and having received mental health services in the past (14). However, a defining characteristic of the peer as employee or provider is that the relationship between the peer provider and a service recipient is not reciprocal (4). The peer provider and the recipient are not at the same level of skills or degree of recovery, and both parties are not expected to receive mutual benefit. This asymmetrical relationship differs from other forms of peer support in which peers of varying levels of skill and recovery work together and benefit from each other’s experiences.
The literature describes a number of different peer services and supports. They can include services to promote hope, socialization, recovery, self-advocacy, development of natural supports, and maintenance of community living skills (15). They also can be a component in the implementation of peer-run education and advocacy programs, such as Wellness Recovery Action Plans (WRAP) (16). Salzer and colleagues (17) documented a wide range of peer support services and roles through a national survey. They found that the most frequently reported role for peers was to share personal experiences and provide mutual aid. Other roles or services provided by peers included the “encouragement of self-determination and personal responsibility; a focus on health and wellness; addressing hopelessness; assistance in communications with providers; education about illness management; and combating stigma in the community” (17).
Peer support services generally include three types of activities, although they may overlap in practice (18): a distinct set of activities or a curriculum that includes education and the development of coping and problem-solving strategies to facilitate self-management of a person’s mental illness, activities that are delivered as part of a team that may include nonpeers (for example, an assertive community treatment [ACT] team), and traditional activities (for example, forms of case management involving linkage to services) that are delivered in a way that is informed by a peer’s personal recovery experience.
Regardless of the service type, there seems to be agreement that peers as providers “draw upon their lived experiences to share ‘been there’ empathy, insights, and skills . . . serve as role models, inculcate hope, engage patients in treatment, and help patients access supports [in the] community” (19). The use of peers is supported by social modeling theory, which states that other people in similar circumstances might have the most influence on behavior change (20).
Peer support services are becoming professionalized. Organizations such as the International Association of Peer Supporters are developing standards of practice. Peer providers receive training and certification to deliver their services in the field. This training varies but typically involves passing a written examination after completing a 30- to 40-hour week of class instruction that addresses topics in recovery, mental illness, medications, and rehabilitation. This credentialing and certification process allows for reimbursement of services beyond block grant funding. Based on the “Georgia model” of Medicaid-reimbursed peer services (21), several organizations in the United States, including the Veterans Health Administration, provide this type of training. States in which peer support services are Medicaid reimbursable and the Veterans Health Administration require peers to pass the certification exam as a condition of being hired. Many states are including supports offered by certified peer support specialists as Medicaid-reimbursable services (15). The Centers for Medicare & Medicaid Services recognizes peer support services as an evidence-based model of care for mental health and an important component of a state’s effective delivery system (22,23).
Given the growing interest among many in the mental health services field in using peers as providers, policy makers and others have questions about their effectiveness as an intervention. This assessment of the available research will help inform mental health system leaders who are making decisions about whether to provide peer support services or to include them in health insurance plans for Medicaid or benchmark plans.
Methods
Search strategy
We conducted a literature search of outcome studies about peer support services published from 1995 through 2012. We searched the major databases: PubMed (U.S. National Library of Medicine and National Institutes of Health), PsycINFO (American Psychological Association), Applied Social Sciences Index and Abstracts, Sociological Abstracts, Social Services Abstracts, Published International Literature on Traumatic Stress, the Educational Resources Information Center, and the Cumulative Index to Nursing and Allied Health Literature. We also examined bibliographies of major reviews and searched for nonjournal publications, such as government reports. Search terms included combinations of mental health, mental health services, psychotic disorders, mental disorders, psychiatry, peer support, consumer service, consumer run, consumer operated, consumer advocacy, patient advocacy, consumer-provider, psychiatric survivor, and case manager aide.
Inclusion and exclusion criteria
This review was limited to U.S. and international studies in English and included the following types of articles: randomized controlled trials (RCTs), quasi-experimental studies, single-group time-series design studies, and cross-sectional correlational studies; studies that were focused on peer support services for adults with serious mental illnesses only (a DSM diagnosis of a psychotic spectrum disorder or bipolar disorder and persistent impairment in psychosocial functioning); and studies of peer support services for adults with co-occurring substance use disorders (although this population was not the focus of this review). We defined peer support providers as individuals in recovery from serious mental illness who were operating within the formal behavioral health service continuum that included various types of treatment or case management (for example, ACT) within government or private nonprofit treatment facilities. Older reviews were consulted only to ensure that all relevant studies were identified. Given the existence of different types of peer support services, we divided the review of studies into three categories: peers added to traditional services (peers added), peers assuming a regular provider position (peers in existing roles), or peers delivering structured curricula (peers delivering curricula). Within these types, the definition and model of peer support services sometimes differed across studies. Various measures were used to define the effectiveness of these services.
This review did not include peer recovery support services provided to individuals with substance use disorders apart from mental illness. These services are delivered to an individual with a substance use disorder by a provider in recovery from addiction (6,24). Although similarities exist between peer support services and peer-based recovery support services, each has its own extensive and separate body of literature. Because peer support groups and “consumer-operated services” (stand-alone programs run by peers) typically serve as adjuncts to traditional behavioral health services, they were not included in this review. Studies about the effectiveness of online peer support, studies of services for smoking cessation, studies of peer support for individuals with developmental disabilities, and studies that focused on children and adolescents were also excluded.
Strength of the evidence
The methodology used to rate the strength of the evidence is described in detail in the introduction to this series (25). The research designs of the studies that met the inclusion criteria were examined. The series established three levels of evidence (high, moderate, and low) to indicate the overall research quality of the studies. Ratings were based on predefined benchmarks that considered the number of studies and their methodological quality. Each of the three types of peer support services mentioned above (peers added, peers in existing roles, and peers delivering curricula) was rated separately. We discussed the ratings to confirm a consensus opinion.
In general, high ratings indicate confidence in the reported outcomes and are made when there are either three or more RCTs with adequate designs or two RCTs plus two quasi-experimental studies with adequate designs. Moderate ratings indicate that there is some adequate research to judge the service, although it is possible that future research could influence initial conclusions. Moderate ratings are based on the following three options: two or more quasi-experimental studies with adequate design; one quasi-experimental study plus one RCT with adequate design; or at least two RCTs with some methodological weaknesses or at least three quasi-experimental studies with some methodological weaknesses. Low ratings indicate that research for this service is not adequate to draw evidence-based conclusions. Low ratings indicate that studies have nonexperimental designs, there are no RCTs, or there is no more than one adequately designed quasi-experimental study.
We considered other design factors that could increase or decrease the evidence rating, such as sample size; how the service, populations, and interventions were specified; use of statistical methods to account for baseline differences between experimental and comparison groups; identification of moderating or confounding variables with appropriate statistical controls; examination of attrition and follow-up; use of psychometrically sound measures; and indications of potential research bias.
Effectiveness of the service
We described the effectiveness of each of the peer support service types—that is, how well the outcomes of the studies met the service goals. We compiled the findings for separate outcome measures and study populations, summarized the results, and noted differences across investigations. We considered the quality of the research design in our conclusions about the effectiveness of the three service types.
Results
Level of evidence
We were unable to find any meta-analyses on this topic through 2012. The literature search yielded 20 individual studies examining the impact of peer support services (as conceptualized in this review) compared with services without peer support (for example, treatment as usual, treatment teams with nonpeers, and wait-list control groups). There were 11 RCTs published in 15 articles (26–40), six quasi-experimental studies (41–46), and three correlational or descriptive studies (15,47,48). Across the three types of peer support services, there were 13 studies of peers added to traditional services: six RCTs (26,27,36–39), six quasi-experimental designs (41–46), and one correlational study (15). There were three studies of peers assuming a regular provider position: two RCTs published in three articles (28–30) and one correlational study (47). Finally, there were four studies of peers delivering structured curricula: three RCTs published in six articles (31–35,40) and one correlational study (48). These are the only published studies we identified; they may not reflect the total pool of studies, which includes those that were not published because of a bias toward positive results. Summaries of the RCTs are provided in Table 2. Summaries of the quasi-experimental and correlational studies are provided in Table 3.
| Study | Sample descriptionand intervention | Outcomesmeasured | Major findings | Study ratingand explanationb |
|---|---|---|---|---|
| Peers added | ||||
| O’Donnell et al., 1999 (37) | 119 individuals referred for case management and assigned to standard case management versus client-focused case management versus client-focused case management plus peer advocate | Functioning, disability, quality of life, service satisfaction, family burden | No significant between-group differences were found on outcomes at the 12-month follow-up. | Limited. There was a small sample and a high attrition rate and different client loads between conditions. Because of high attrition, the sample may have been less representative of community-based clients with schizophrenia and bipolar disorder. |
| Craig et al., 2004 (38) | 45 individuals assigned to an ACT team with standard case management versus an ACT team with case management plus a peer assistant on the team | Service uptake and engagement, need for care, life skills, social network, service satisfaction | At 12 months postrandomization, participants with peers on their team had lower rates of nonattendance, higher levels of participation in structured social care activities, and fewer unmet needs than those without peers. No significant between-group differences were found on social networks or satisfaction with services. | Limited. The small sample limited generalizability. Most outcome measures were collected from staff who were not blind to study conditions. |
| Davidson et al., 2004 (27) | 260 individuals receiving outpatient services assigned to a peer volunteer versus a nonpeer volunteer versus no volunteer | Depression, other psychiatric symptoms, well-being, self-esteem, functioning, functional impairment, diagnosis, client satisfaction | No significant between-group differences were found on outcomes at the 4- or 9-month follow-up. | Adequate. There was a restricted sample and possible selection bias. |
| Sells et al., 2006 (39) | 137 adults, 70% of whom had a co-occurring substance use disorder, assigned to ACT alone versus ACT plus peer-delivered case management | Therapeutic relationship, frequency and severity of substance use, utilization of various outpatient and day-treatment services, treatment engagement | Participants with peers reported a better therapeutic relationship than those in the control group at the 6-month follow-up. Those who were least engaged with peers had more provider contact than the control group. The therapeutic relationship at 6 months predicted treatment engagement and service use at 12 months, but no between-group differences were found. | Limited. The analysis relied on self-report. The small sample limited the ability to generalize to all individuals with serious mental illness. |
| Rivera et al., 2007 (26) | 203 adult inpatients with ≥2 hospitalizations in the past 2 years assigned to standard care versus case management with nonpeers versus case management with peers | Quality of life, service satisfaction, symptoms | No significant between-group differences were found on outcomes at the 12-month follow-up. | Adequate. It was unclear whether participants were blind to the purpose of the study. |
| Sledge et al., 2011 (36) | 74 patients hospitalized ≥3 times in the past 18 months assigned to usual care versus usual care plus a peer mentor | Number of hospitalizations and hospital days | At the 9-month follow-up, participants with peers had significantly fewer admissions and fewer hospital days than those in usual care. | Limited. The small sample limited the ability to generalize to all psychiatric inpatient admissions. |
| Peers in existing roles | ||||
| Solomon and Draine, 1995 (29); Solomon et al., 1995 (30)c | 96 individuals in a community mental health center at risk for hospitalization assigned to a case management team of peers versus a case management team of nonpeers | Therapeutic alliance, income, social network size, days hospitalized, psychiatric symptoms, attitudes toward medication compliance, quality of life, interpersonal contact, social functioning, treatment satisfaction | No significant between-group differences were found on outcomes 2 years after initiation of services. | Limited. The analysis relied on self-report, and the sample was small. |
| Clarke et al., 2000 (28) | 163 adults assigned to usual care versus ACT without peers versus ACT with peers | Percentage of participants hospitalized and number of days to hospitalization; time to first emergency department visit, arrest, homelessness | Time to first hospitalization was earlier for the ACT nonpeer group than the ACT with peer group, but no significant differences were found between these groups for the first instance of homelessness, first arrest, or first emergency department visit. Compared with the ACT group with peers, more participants in the ACT group without peers had hospitalizations and emergency department visits. | Limited. The sample was small. Participants had less severe symptoms than those in other studies of ACT, limiting generalizability. There was low fidelity to the ACT model. |
| Peers delivering curricula | ||||
| Druss et al., 2010 (40) | 80 individuals with chronic general medical illness assigned to a HARP program versus usual care | Patient activation, primary care visits, physical activity, medication adherence, health-related quality of life | Six months after the intervention, HARP program participants had higher patient activation and higher rates of primary care visits than those with usual care. No between-group differences were found in medication adherence, physical health, quality of life, or physical activity. | Limited. The small sample limited power to detect effects. The analysis used self-reported outcome measures. |
| Cook et al., 2012 (32); Cook et al., 2012 (31); Jonikas et al., 2013 (34)c | 519 outpatients assigned to a WRAP program versus a wait-list control group | Patient self-advocacy, psychiatric symptoms, perceived recovery from mental illness, hopefulness, quality of life | Compared with the control group, WRAP participants reported greater reductions in psychiatric symptoms at 6- and 8-month follow-ups. They also had greater improvements in total and subscale scores for hopefulness and self-advocacy and in subscale scores for quality of life at the 6-month follow-up and for self-perceived recovery at the 8-month follow-up. No significant between-group differences were found for the other measures. | Adequate. The analysis relied primarily on self-report. The sample was restricted to outpatients, and there was a nonactive control group. |
| Cook et al., 2012 (33); Pickett et al., 2012 (35)c | 428 outpatients assigned to a BRIDGES program versus a wait-list control group | Self-perceived recovery from mental illness, hopefulness, empowerment, patient self-advocacy | Compared with the control group at 6-month follow-up, BRIDGES participants reported greater improvements in total and subscale scores for empowerment and recovery and in subscale scores for hopefulness and self-advocacy. After the analysis controlled for depressive symptoms, effects remained for total and subscale scores for recovery and one subscale score for hopefulness. No significant between-group differences were found for the other measures. | Adequate. The analysis relied primarily on self-report. The sample was restricted to outpatients, and there was a nonactive control group. The researchers did not examine other predictors of empowerment and patient self-advocacy. |
a
Articles are in chronological order by the three types of intervention. Abbreviations: ACT, assertive community treatment; BRIDGES, Building Recovery of Individual Dreams and Goals; HARP, Health and Recovery Peer; WRAP, Wellness Recovery Action Planning
b
Various threats to both internal and external validity were considered in each study’s rating of “limited” (study had several methodological limitations) or “adequate” (study had few or minor methodological limitations).
c
Multiple publications based on the same randomized controlled trial are described in the same row.
| Study | Sample descriptionand intervention | Outcomesmeasured | Major findings | Study ratingand explanationb |
|---|---|---|---|---|
| Quasi-experimental | ||||
| Peers added | ||||
| Felton et al., 1995 (41) | 104 participants; case management teams versus case management teams plus nonpeer assistants versus case management teams plus peer specialists | Self-image and outlook, treatment engagement, social support, quality of life, life problems, housing instability, income, family contact | Over the 2-year study, clients of case management teams plus peer specialists reported gains in quality of life indicators, reductions in some major life problems, and more treatment engagement, compared with those in the other two groups. There were no differences in outcomes between teams with nonpeer assistants and those with standard case management. | Limited. Participants were not randomly assigned. The small sample and an overrepresentation of clients in the case management only condition may have limited generalizability. |
| Klein et al., 1998 (42) | 61 participants with co-occurring mental and substance use disorders; intensive case management teams with peers versus without peers | Crisis events (for example, emergency room visits), number of hospital days, social functioning, use of community resources and social integration, quality of life | Participants with peers had fewer inpatient days, better social functioning, and some improvements in quality of life indicators at the end of the intervention. | Limited. Participants were not randomly assigned, and the sample was small, limiting generalizability. The analysis relied on self-report data. |
| Chinman et al., 2001 (43) | 158 participants; peer support services added to standard care versus a matched control group in standard care | Number of hospitalizations and hospital days | No significant between-group differences were found in outcomes 6 months after the service start date. | Limited. Participants were not randomly assigned. |
| Min et al., 2007 (44) | 556 participants with serious mental illness and substance use disorders with a history of hospitalization; teams with case management versus teams with case management plus a peer worker | Days to first hospitalization; percentage hospitalized over 3 years | Participants on teams with peers had more time in the community and less inpatient use. | Limited. Participants were not randomly assigned. There was possible bias from case manager referral of certain participants to the study. |
| Schmidt et al., 2008 (45) | 142 participants with a recent hospitalization; case management team versus case management team plus peer | Client contact, percentage with crisis center visits and number of visits, percentage hospitalized, number of hospitalizations and hospital days, outpatient mental health service use, medication use, substance abuse, housing stability | No significant between-group differences were found in outcomes measured at the 12-month follow-up. | Limited. Participants were not randomly assigned. |
| van Vugt et al., 2012 (46) | 530 participants in 20 ACT teams; teams without peers versus teams with peers | Level of functioning, met and unmet needs, working alliance, number of hospital days, number of homeless days | At 1- and 2-year follow-ups, clients of teams with peers had better psychiatric and social functioning, improvements in met and unmet needs related to their personal recovery, and fewer homeless days than clients of teams without peers. Peer presence was associated with an increased number of hospital days. | Limited. Participants were not randomly assigned to the comparison group. Clients of teams with peers were more severely ill than clients of other teams. Some clients of teams without peers had contact with peers. |
| Correlational or descriptive | ||||
| Peers added | ||||
| Landers et al., 2011 (15) | 35,668 participants with a reimbursed community mental health service; those with a peer support services claim in the past year versus those without | Percentage with a hospitalization or crisis stabilization | Compared with participants without peers, more participants with peers used crisis services, but fewer had a hospitalization. | Limited. The study was restricted to Medicaid enrollees. The research design was cross-sectional. |
| Peers in existing roles | ||||
| Chinman et al., 2000 (47) | 1,203 participants who were homeless; homeless outreach teams versus homeless outreach teams with peers | Quality of life, homelessness days, social support, symptoms and mental health problems, alcohol and drug problems, days worked | No significant between-group differences were found on outcomes over a 12-month period. | Limited. Participants were not randomly assigned. |
| Peers delivering curricula | ||||
| Cook et al., 2010 (48) | 381 consumers of psychiatric services; pretest–posttest comparison of participants who received the WRAP curriculum | Recovery management attitudes and abilities | At the end of the intervention, participants reported significant increased hopefulness for recovery, awareness of early warning signs of decompensation, use of wellness tools, and awareness of symptom triggers. They also reported having a crisis plan in place, a plan to deal with symptoms, a social support system, and the ability to take responsibility for their own wellness. | Limited. The research design was a pretest–posttest comparison with no comparison group and a nonrandom sample. The analysis relied on a self-reported, nonvalidated instrument to measure dependent variables. There was a short follow-up time period. |
a
Articles are in chronological order by research design and type of intervention. Abbreviations: ACT, assertive community treatment; WRAP, Wellness Recovery Action Planning
b
Various threats to both internal and external validity were considered in each study’s rating of “limited” (study had several methodological limitations) or “adequate” (study had few or minor methodological limitations).
The level of evidence (that is, methodological quality) was rated as moderate for all three types of peer support services. This rating was based on two RCTs with adequate designs for peers added to traditional services (26,27), two RCTs with limitations (published in three articles) for peers in existing roles (28–30), and two RCTs with adequate designs (published in five articles) for peers delivering curricula (31–35). There were no discrepancies among the author ratings.
Despite the large number of RCTs we identified, the studies addressed various models of peer support services, and methodological problems and design flaws decreased the research quality rating. For example, sample sizes in various studies often were small, outcome measures with unknown reliability or validity were used, data collectors usually were not blind to the treatment group (raising the issue of possible bias), self-reported data on symptomatology did not have corroborating reports from other sources, and research designs involved wait-list control groups rather than active control groups.
Effectiveness of the service
Effectiveness of peer support services varied across the three service types. There were limitations inherent in the research designs and differences in how effectiveness was defined and measured, making it difficult to draw definitive conclusions. Some study outcomes included clinical measures, such as hospitalization rates, symptomatology, or functioning. Other studies examined process outcomes, such as treatment engagement, retention in treatment, quality of life, or empowerment. One consistent finding across studies was that peers were at least as effective in providing services as nonpeers. The research was less consistent about the extent to which peer support services were more effective than traditional services alone in improving clinical outcomes such as symptomatology and functioning. For example, although reduced inpatient service use was found in two RCTs (28,36) and two quasi-experimental studies (42,44), this result was not found in other RCTs and quasi-experimental trials.
Among the 13 studies in the peers added service type, eight found some positive benefit (15,36,38,39,41,42,44,46). Three of the six RCTs examining the peers added service type documented a benefit to peers, although these three RCTs were judged to have design limitations. One suggested that service users who had involvement from a peer mentor had fewer rehospitalizations and hospital days than those who did not have a peer mentor (36). A second RCT compared patients randomly assigned to an ACT team either with or without peers and found that patients in the team with peers had better treatment engagement six months after entering treatment (39). Although these effects disappeared at 12 months, this enhanced engagement at six months predicted higher levels of self-reported motivation for treatment and more frequent use of Alcoholics Anonymous and Narcotics Anonymous at 12 months. In the third RCT, patients randomly assigned to an ACT team with peers had lower rates of nonattendance at appointments and higher levels of participation in structured social care activities than patients assigned to an ACT team without a peer (38). The remaining three RCTs examining the peers added service type showed no peer-related effects comparing “client-focused” teams with peers versus client-focused teams without peers versus standard care (37), intensive case management with peers versus intensive case management without peers versus standard care (26), and use of a peer volunteer versus a nonpeer volunteer versus no volunteer (27). Of these three RCTs, the first was judged to have design limitations (37), and the other two were judged to have adequate research designs (26,27).
The quasi-experimental and correlational or descriptive studies of the peers added service type generally had more positive outcomes than the RCTs: five showed some positive benefit (15,41,42,44,46), and the remaining two showed no group differences (43,45) For example, Felton and colleagues (41) found that patients served by peers on a case management team had greater treatment engagement, more satisfaction with life situation and finances, and fewer life problems than a comparison group of those served by a team with either a paraprofessional or no additional staff. Klein and colleagues (42) and Min and colleagues (44) found that over time the proportion of clients with inpatient use was lower among those with peer support services than among those without peer support services. Klein and colleagues (42) also reported improved social functioning and quality of life among patients receiving peer support services. Van Vugt and colleagues (46) compared patients from four ACT teams with peers and from 16 ACT teams without peers and found that the presence of a peer was associated with an improvement over time in mental and social functioning, homeless days, and recovery needs. However, the study also found that the presence of a peer was associated with an increase in psychiatric hospitalization days. This was the only study reviewed that documented a negative finding. In the one correlational study of the peers added service type, Landers and Zhou (15) conducted a retrospective review of Medicaid claims data. They found that users of peer support services were less likely to be admitted to a psychiatric hospital compared with nonusers of peer support services with similar diagnoses, but the relationship was statistically significant only if patients did not use crisis stabilization services. There were no peer-related effects in two quasi-experimental studies comparing patients receiving peer support services in addition to standard outpatient care versus standard care alone (43) and comparing patients of case management teams with and without peers (45).
Among the three studies in the service type of peers in existing roles, only one had positive effects. Clarke and colleagues (28) compared patients randomly assigned to standard care, ACT, or ACT with peers and found that patients of peers had significantly more time in the community and significantly less inpatient time than those in the other two conditions. Reflecting the inconsistent findings in this literature, two other studies showed no significant differences between those who received peer support services and those who did not in hospital admission rates, length of stay, hospital readmissions, symptomatology, or a range of outcomes related to functioning (29,30,47). One was an RCT (judged to be limited in design) comparing teams that had all-peer case management versus standard case management (29,30). The other was a correlational study comparing patients of case management teams for homeless individuals that did and did not have case management positions occupied by peers (47).
There was more consistency among the three RCTs (published in six articles) (31–35,40) and one correlational study (48) in the service type of peers delivering curricula. One RCT that was published in three articles (31,32,34) built upon a promising single-group, pre-post treatment study (48). The researchers found that individuals who received WRAP—an eight-session, peer-led, illness self-management program—reported greater reductions in depression and anxiety symptoms and greater increases in perceived recovery, hope, quality of life, and self-advocacy compared with those who received treatment as usual. Similarly, an RCT evaluation of Building Recovery of Individual Dreams and Goals (BRIDGES)—an eight-week class taught by peers that addresses mental illness treatments, recovery, job readiness, communication, and assertiveness—found greater improvement among program participants than among those in the control group in perceived recovery and in some elements of hopefulness, empowerment, and assertiveness with providers (33,35). Finally, Druss and colleagues (40) conducted a small RCT evaluation of the Health and Recovery Peer (HARP) program—a six-session, peer-led, medical self-management intervention that is conducted using a program manual. The authors found greater patient activation and rates of primary care visits at six months postintervention for those in the program compared with those who received usual care. The authors also found notable (but not statistically significant) improvement in medication adherence, quality of life related to physical health, and physical activity. Although all four studies in the service type of peers delivering curricula found a service benefit, the impacts of the specific WRAP, BRIDGES, and HARP programs cannot be separated from their peer delivery in these studies.
Discussion
The purposes of this review were to rate the level of evidence of peer support services using the criteria established by the AEB Series and to describe the effectiveness of peer support services. Conclusions about peer support services depend on the degree to which effectiveness can be judged from studies with moderate evidence. The criteria established by the AEB Series state that moderate evidence has value in contributing to the consideration of effectiveness. On the basis of these criteria, results for the effectiveness of the peers added and the peers delivering curricula types of peer support services are encouraging (but clearly not definitive). These conclusions differ from those in the recent Cochrane review of peer support services, in part because that review excluded quasi-experimental trials and studies involving peer-delivered curricula (12). The Cochrane review found few differences in psychosocial outcomes and in outcomes related to psychiatric symptoms and service use between individuals who received services from peers involved on mental health teams and individuals who received services from professionals employed in similar roles. Our judgment of effectiveness also would be more mixed if only the most rigorous RCTs were considered. Out of the four RCTs judged adequate, two reported null findings (both for the peers added service type) (26,27), and two reported positive findings (in the peers delivering curricula service type) (31–35).
Although the peer support services discussed have demonstrated promising outcomes, research is still needed to show their effectiveness with greater confidence—that is, with a higher level of evidence. Research is needed that has greater specificity (for example, to distinguish various peer support services from each other), consistency (such as in service definitions and outcome measures), and follow-up of outcomes over a longer term. For example, studies of specific recovery programs led by peers (such as WRAP, BRIDGES, and HARP) have not been able to differentiate the contributions of peers from the effects of the overall program, even though a peer’s ability to promote beliefs about hope, recovery, engagement, empowerment, self-efficacy, self-management, and expanded social networks (49,50) is what has been “proposed as the central tenets of recovery” (49). One way to disentangle these effects would be to compare the outcomes of these programs with those obtained when the curricula are delivered by a paraprofessional without a psychiatric illness.
Stakeholders must develop commonly accepted peer support service definitions, types, values, standards, models, manuals, training curricula, and fidelity measures. National standards for certification and licensure of peer providers would create further standardization. This type of formalization of peer support has been questioned for its potential to undercut the informal, mutually supportive nature from which peer support originated. However, it could be possible to create standards and certification for some types of peer support services and not for others that peers and clients would like to keep more informal.
The many variations of peer support delivery could be explored with greater consistency and specificity. It is important to address variables such as setting (for example, traditional case management, psychosocial clubhouses, and outpatient treatment teams), service delivery mode (for example, groups, individual meetings, and drop-in meetings), background of peers (for example, those with serious mental illness versus those with less impairment), functions (for example, having a unique role in a system versus having a role similar to those of nonpeers), and levels of service delivery structure that range from informal support to specific program curricula. Attention also needs to be paid to well-documented implementation challenges, such as ill-defined roles and resistance among staff (19,51). Given the level of evidence to date, the research agenda moving forward should ask not only, “Do peer support services work?” but also, “Under what specific conditions do peer support services work?”
Future research should determine what outcomes are the best indicators of impact and what valid and reliable tools are needed to measure these outcomes. For example, it may be helpful to use illness self-management and other recovery-oriented measures rather than relying only on traditional assessments of symptoms and functioning (45). Engagement might be another effective indicator, because engagement with services is fundamental to the efficacy of evidence-based programming for individuals with co-occurring mental and substance use disorders. Research suggests a valued role for peer providers in this area.
Finally, there is a need to expand the knowledge base of cultural competence in the delivery of peer support services. Given the significance of disparities in the receipt of mental health services, implementing effective culturally responsive care is of critical importance. Most of the studies reviewed did not specifically evaluate the impact of race, ethnicity, or sex on the effectiveness of peer support services. Tondora and colleagues (52) have implemented a clinical trial to examine the effectiveness of a peer-based service that includes cultural modifications for African-American and Latino populations. Forthcoming results may indicate whether these modifications were effective in promoting cultural responsiveness.
Conclusions
On the basis of the evidence standards established for the series, we conclude that each peer support service type (peers added to traditional services, peers in existing roles, and peers delivering curricula) achieved a moderate level of evidence (see box on this page). However, the three types differed in their documented effectiveness. Across the range of experimental rigor (RCT, quasi-experimental, and correlational or descriptive studies), there was more evidence in support of peers added, for which eight of 13 studies found a positive peer impact, and in support of peers delivering curricula, for which four of four studies found similar impact. There was less support for peers in existing roles, for which one of three studies found positive outcomes. Across all studies in this review, only one showed a worsening of one outcome—that of hospitalizations (46). These findings are important, given the stigma often associated with mental illness (4).
Evidence for the effectiveness of three types of peer support services for individuals with serious mental illnesses: moderate
Evidence for the effectiveness of peers added to traditional services and of peers delivering structured curricula was positive, albeit from studies across the range of methodological rigor. The contributions of peers, especially peers delivering curricula, are unclear. Across the service types, improvements have been shown in the following outcomes:
• Reduced inpatient service use
• Improved relationship with providers
• Better engagement with care
• Higher levels of empowerment
• Higher levels of patient activation
• Higher levels of hopefulness for recovery
This review of peer support services has implications for several audiences. For policy makers and insurers, the service types of peers added and the peers delivering curricula appear to be important and emerging interventions in the spectrum of mental health and recovery services. Given that most of these studies show positive outcomes and that there has been only a single negative finding, we recommend that purchasers consider coverage of the peers added and the peers delivering curricula types of peer support services. The proliferation of effective peer support services means that many payers (such as state mental health and substance use directors, managed care companies, and county behavioral health administrators) may want to consider adding peers to covered services. Several states already cover peer support services with Medicaid funding (17).
For consumers, families, and treatment professionals, the increasing availability of peer support services in the traditional mental health system can enhance current services, and we recommend that consumers inquire about these services as part of their care. Within systems that often have too few resources, peer support services place a premium on developing relationships, on guiding patients through fragmented systems to the needed treatments, and on promoting development of a full life beyond illness management. Adding peers to clinical teams can make the teams more successful, and it is recommended that clinical leaders consult the available sources of information about how to do so (51).
Finally, for researchers, it is vital that future studies keep up with the growth of these services for mental and substance use disorders to show with greater confidence whether and how they have an impact. These implications interact, in that as more peer support services are deployed and used by consumers and families, there will be greater need for and enthusiasm about continued research—which, in turn, could fuel more provision of services. Over time, with greater use and research, peer support services have the potential to help mental health services fulfill the promise of recovery for those with serious mental illnesses.
Acknowledgments and disclosures
Development of the Assessing the Evidence Base Series was supported by contracts HHSS283200700029I/HHSS28342002T, HHSS283200700006I/HHSS28342003T, and HHSS2832007000171/HHSS28300001T from 2010 through 2013 from the Substance Abuse and Mental Health Services Administration (SAMHSA). Development of this review was also partly supported by two grants from the U.S. Department of Veterans Affairs: grant RRP 06-147, Evidence-Based Review of Peer Support; and grant IIR 06-227 PEers Enhancing Recovery. The authors acknowledge the contributions of Wilma Townsend, M.S.W., Kevin Malone, B.A., and Suzanne Fields, M.S.W., from SAMHSA; John O’Brien, M.A., from the Centers for Medicare & Medicaid Services; Garrett Moran, Ph.D., from Westat; and John Easterday, Ph.D., Linda Lee, Ph.D., Rosanna Coffey, Ph.D., and Tami Mark, Ph.D., from Truven Health Analytics. The views expressed in this article are those of the authors and do not necessarily represent the views of SAMHSA. The contents do not represent the views of the U.S. Department of Veterans Affairs or the United States government.
The authors report no competing interests.
References
1.
Achieving the Promise: Transforming Mental Health Care in America. Pub no SMA-03-3832. Rockville, Md, Department of Health and Human Services, President’s New Freedom Commission on Mental Health, 2003
2.
Ahmed AO, Doane NJ, Mabe PA, et al.: Peers and peer-led interventions for people with schizophrenia. Psychiatric Clinics of North America 35:699–715, 2012
3.
Goldstrom ID, Campbell J, Rogers JA, et al.: National estimates for mental health mutual support groups, self-help organizations, and consumer-operated services. Administration and Policy in Mental Health and Mental Health Services Research 33:92–103, 2006
4.
Davidson L, Chinman M, Sells D, et al.: Peer support among adults with serious mental illness: a report from the field. Schizophrenia Bulletin 32:443–450, 2006
5.
Wellness Recovery Action Plan (WRAP). SAMHSA's National Registry for Evidence-Based Programs and Practices. Rockville, Md, Substance Abuse and Mental Health Services Administration, 2010. Available at nrepp.samhsa.gov/ViewIntervention.aspx?id=208. Accessed Dec 2, 2013
6.
Reif S, Braude L, Lyman DR, et al.: Peer recovery services for individuals with substance use disorders. Psychiatric Services, in press
7.
Simpson EL, House AO: Involving users in the delivery and evaluation of mental health services: systematic review. British Medical Journal 325:1265–1268, 2002
8.
Doughty C, Tse S: The Effectiveness of Service User–Run or Service User–Led Mental Health Services for People With Mental Illness: A Systematic Literature Review. Wellington, New Zealand, Mental Health Commission, 2005
9.
Rogers ES, Farkas M, Anthony WA, et al.: Systematic Review of Peer Delivered Services Literature, 1989–2009. Boston, Boston University, Center for Psychiatric Rehabilitation, 2009
10.
Repper J, Carter T: A review of the literature on peer support in mental health services. Journal of Mental Health 20:392–411, 2011
11.
Wright-Berryman JL, McGuire AB, Salyers MP: A review of consumer-provided services on assertive community treatment and intensive case management teams: implications for future research and practice. Journal of the American Psychiatric Nurses Association 17:37–44, 2011
12.
Pitt V, Lowe D, Hill S, et al.: Consumer-providers of care for adult clients of statutory mental health services. Cochrane Database of Systematic Reviews 3:CD004807, 2013
13.
Chinman M, Young AS, Hassell J, et al.: Toward the implementation of mental health consumer provider services. Journal of Behavioral Health Services and Research 33:176–195, 2006
14.
Solomon P: Peer support/peer provided services underlying processes, benefits, and critical ingredients. Psychiatric Rehabilitation Journal 27:392–401, 2004
15.
Landers GM, Zhou M: An analysis of relationships among peer support, psychiatric hospitalization, and crisis stabilization. Community Mental Health Journal 47:106–112, 2011
16.
Campbell J, Leaver J: Emerging New Practices in Organized Peer Support, 2003. Alexandria, Va, National Association of State Mental Health Program Directors and National Technical Assistance Center for State Mental Health Planning, 2003. Available at www.consumerstar.org/pubs/Emerging%20New%20Practices%20in%20Oraganized%20Peer%20Support.pdf. Accessed Dec 2, 2013
17.
Salzer MS, Schwenk E, Brusilovskiy E: Certified peer specialist roles and activities: results from a national survey. Psychiatric Services 61:520–523, 2010
18.
Eiken S, Campbell J: Medicaid Coverage of Peer Support for People With Mental Illness: Available Research and State Examples. Baltimore, Center for Medicare and Medicaid Services, 2008. Available at http://www.wicps.org/uploads/1/8/1/4/1814011/peersupport_reuters.pdf. Accessed Dec 2, 2013
19.
Chinman M, Lucksted A, Gresen R, et al.: Early experiences of employing consumer-providers in the VA. Psychiatric Services 59:1315–1321, 2008
20.
Bandura A: Social Learning Theory. Oxford, England, Prentice-Hall, 1977
21.
Sabin JE, Daniels N: Strengthening the consumer voice in managed care: VII. the Georgia peer specialist program. Psychiatric Services 54:497–498, 2003
22.
State Medicaid Director Letter, Peer Support Services. SMDL 07-011. Baltimore, Centers for Medicare and Medicaid Services, 2007. Available at downloads.cms.gov/cmsgov/archived-downloads/SMDL/downloads/SMD081507A.pdf. Accessed Dec 2, 2013
23.
Daniels AS, Cate R, Bergeson S, et al.: Level-of-care criteria for peer support services: a best-practice guide. Psychiatric Services 64:1190–1192, 2013
24.
White W: Peer-Based Addiction Recovery Support, History, Theory, Practice, and Scientific Evaluation. Chicago, Great Lakes Addiction Technology Transfer Center, and Philadelphia, Pennsylvania Department of Behavioral Health and Mental Retardation Services, 2009
25.
Dougherty RH, Lyman DR, George P, et al.: Assessing the evidence base for behavioral health services: introduction to the series. Psychiatric Services 65:11–15, 2014
26.
Rivera JJ, Sullivan AM, Valenti SS: Adding consumer-providers to intensive case management: does it improve outcome? Psychiatric Services 58:802–809, 2007
27.
Davidson L, Shahar G, Stayner D, et al.: Supported socialization for people with psychiatric disabilities: lessons from a randomized controlled trial. Journal of Community Psychology 32:453–477, 2004
28.
Clarke GN, Herinckx HA, Kinney RF, et al.: Psychiatric hospitalizations, arrests, emergency room visits, and homelessness of clients with serious and persistent mental illness: findings from a randomized trial of two ACT programs vs usual care. Mental Health Services Research 2:155–164, 2000
29.
Solomon P, Draine J: The efficacy of a consumer case management team: 2-year outcomes of a randomized trial. Journal of Mental Health Administration 22:135–146, 1995
30.
Solomon P, Draine J, Delaney MA: The working alliance and consumer case management. Journal of Mental Health Administration 22:126–134, 1995
31.
Cook JA, Copeland ME, Floyd CB, et al.: A randomized controlled trial of effects of Wellness Recovery Action Planning on depression, anxiety, and recovery. Psychiatric Services 63:541–547, 2012
32.
Cook JA, Copeland ME, Jonikas JA, et al.: Results of a randomized controlled trial of mental illness self-management using Wellness Recovery Action Planning. Schizophrenia Bulletin 38:881–891, 2012
33.
Cook JA, Steigman P, Pickett S, et al.: Randomized controlled trial of peer-led recovery education using Building Recovery of Individual Dreams and Goals through Education and Support (BRIDGES). Schizophrenia Research 136:36–42, 2012
34.
Jonikas JA, Grey DD, Copeland ME, et al.: Improving propensity for patient self-advocacy through Wellness Recovery Action Planning: results of a randomized controlled trial. Community Mental Health Journal 49:260–269, 2013
35.
Pickett SA, Diehl SM, Steigman PJ, et al.: Consumer empowerment and self-advocacy outcomes in a randomized study of peer-led education. Community Mental Health Journal 48:420–430, 2012
36.
Sledge WH, Lawless M, Sells D, et al.: Effectiveness of peer support in reducing readmissions of persons with multiple psychiatric hospitalizations. Psychiatric Services 62:541–544, 2011
37.
O’Donnell M, Parker G, Proberts M, et al.: A study of client-focused case management and consumer advocacy: the Community and Consumer Service Project. Australian and New Zealand Journal of Psychiatry 33:684–693, 1999
38.
Craig T, Doherty I, Jamieson-Craig R, et al.: The consumer-employee as a member of a Mental Health Assertive Outreach Team: I. clinical and social outcomes. Journal of Mental Health 13:59–69, 2004
39.
Sells D, Davidson L, Jewell C, et al.: The treatment relationship in peer-based and regular case management for clients with severe mental illness. Psychiatric Services 57:1179–1184, 2006
40.
Druss BG, Zhao L, von Esenwein SA, et al.: The Health and Recovery Peer (HARP) Program: a peer-led intervention to improve medical self-management for persons with serious mental illness. Schizophrenia Research 118:264–270, 2010
41.
Felton CJ, Stastny P, Shern DL, et al.: Consumers as peer specialists on intensive case management teams: impact on client outcomes. Psychiatric Services 46:1037–1044, 1995
42.
Klein AR, Cnaan RA, Whitecraft J: Significance of peer social support with dually diagnosed clients: findings from a pilot study. Research on Social Work Practice 8:529–551, 1998
43.
Chinman MJ, Weingarten R, Stayner D, et al.: Chronicity reconsidered: improving person-environment fit through a consumer-run service. Community Mental Health Journal 37:215–229, 2001
44.
Min SY, Whitecraft J, Rothbard AB, et al.: Peer support for persons with co-occurring disorders and community tenure: a survival analysis. Psychiatric Rehabilitation Journal 30:207–213, 2007
45.
Schmidt LT, Gill KJ, Pratt CW, et al.: Comparison of service outcomes of case management teams with and without a consumer provider. American Journal of Psychiatric Rehabilitation 11:310–329, 2008
46.
van Vugt MD, Kroon H, Delespaul PA, et al.: Consumer-providers in assertive community treatment programs: associations with client outcomes. Psychiatric Services 63:477–481, 2012
47.
Chinman MJ, Rosenheck R, Lam JA, et al.: Comparing consumer and nonconsumer provided case management services for homeless persons with serious mental illness. Journal of Nervous and Mental Disease 188:446–453, 2000
48.
Cook JA, Copeland ME, Corey L, et al.: Developing the evidence base for peer-led services: changes among participants following Wellness Recovery Action Planning (WRAP) education in two statewide initiatives. Psychiatric Rehabilitation Journal 34:113–120, 2010
49.
Repper J, Perkins R: Social Inclusion and Recovery: A Model for Mental Health Practice. London, Bailliere Tindall, 2003
50.
Shepard G, Boardman J, Slade M: Making Recovery a Reality. London, Sainsbury Centre for Mental Health, 2008
51.
Chinman M, Shoai R, Cohen A: Using organizational change strategies to guide peer support technician implementation in the Veterans Administration. Psychiatric Rehabilitation Journal 33:269–277, 2010
52.
Tondora J, O’Connell M, Miller R, et al.: A clinical trial of peer-based culturally responsive person-centered care for psychosis for African Americans and Latinos. Clinical Trials 7:368–379, 2010
Information & Authors
Information
Published In
Cover: Marooned, by Howard Pyle, 1909. Oil on canvas. Delaware Art Museum, Museum Purchase, 1912.
History
Published in print: April 2014
Published online: 15 October 2014
Authors
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBLogin options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).