Peer specialists—individuals with mental illness and substance use disorders who are trained to use their experiences to help others with similar disorders—have been used with increasing frequency in mental health services, but research evaluating their impact is mixed (
1,
2). Although several quasi-experimental and randomized trials of peer specialist services have demonstrated positive outcomes, including patient activation, self-efficacy, empowerment, hope, reduced symptom severity, and improved quality of life, other trials and meta-analyses have not yielded positive results (
3–
5). One design issue influencing the results of these trials is the assumption that peer specialist services benefit all participants in the same way. This assumption is consistent with intent-to-treat analyses, when all participants from an intervention group are compared with all participants of a control group, regardless of implementation variations.
However, the use of random assignment and intent-to-treat analyses could be considered an uneasy fit for peer specialist research, because peer specialists provide a unique type of support. Two theories underlying the work of peer specialists are social learning and social comparison theories (
6,
7), which emphasize the importance of the model (the peer specialist) being attended to and valued by the observer (the client). If a client does not, or is not able to, value and attend to the model, then the effectiveness of the peer specialist can be diminished for that client, compared with other clients who are more able to value the model and compared with other providers who do not rely as heavily on modeling and comparison (for example, psychiatrists). Thus the fit between client and peer specialist is uniquely important, compared with other treatment providers. Because this fit can vary, it is possible that within the samples of those receiving peer specialist services across the previous null-result trials, there are consistent subgroups of individuals who did benefit from those services.
As recommended in a peer specialist research agenda published recently in
Psychiatric Services (
8), more research is needed to assess not only whether peer specialists have impact at all but also when and for whom they have measurable impact. In this study, we tested whether there was a subgroup of individuals assigned to receive peer specialist services who exhibited reliably significant improvements on measures of symptoms of mental illness or hope (consistent with recovery) and whether receipt of more peer specialist services was associated with membership in that subgroup.
The AMPS Study: Administering MISSION-Vet Using Peer Support
The study reported here was a secondary analysis within a larger study of a two-site randomized trial. The larger study assessed a modified version of an evidence-based treatment for veterans with dual diagnoses of mental and substance use disorders, called MISSION-Vet (Maintaining Independence and Sobriety through Systems Integration, Outreach, and Networking–Veterans Edition) (
9,
10). MISSION-Vet was originally designed to be led by case managers with assistance from peer specialists. AMPS (Administering MISSION-Vet using Peer Support) used a modified version in which treatment is delivered solely by full-time peer specialists at each site. Peer specialists worked with veterans using the AMPS consumer workbook to provide 20 structured psychoeducation sessions and 20 unstructured visits, over approximately one year. The workbook contains discussion topics and worksheets, such as developing recovery goals, drawn from the MISSION-Vet materials, for peer specialists and veterans to work on together. Peer specialists at two northeastern VA medical centers were hired to provide this intervention to veterans currently enrolled in HUD-VASH (U.S. Department of Housing and Urban Development–Veteran Affairs [VA] Supportive Housing), a VA program that provides subsidized housing and ongoing case management support to veterans who are homeless. Large caseloads and active mental or substance use issues common among HUD-VASH–enrolled veterans (
11–
13) often make it difficult for case managers alone to address these disorders, even after the veterans obtain housing. The study randomly assigned veterans to standard HUD-VASH services or to HUD-VASH plus peer specialist services. The AMPS intent-to-treat analyses did not identify any effects of treatment assignment on measures of substance use and mental health (Ellison M, unpublished manuscript, 2018).
This Study
This study used the Reliable Change Index (RCI) (
12) to attempt to determine whether there was a subgroup of veterans within the larger AMPS trial who may have improved on outcomes. As presented in several previous studies, the RCI is typically used to determine whether the amount of change among persons with mental illnesses who are exposed to a treatment is statistically reliable (
14,
15). The initial RCI formula subtracts the posttest score on an outcome measure from the baseline score and then divides by the standard error of the differences. The updated version of the RCI used here adjusts the pre-post difference score for regression to the mean because of measure unreliability (for RCI formula details, see
15).
The RCI formula was applied to a baseline-to-posttest measure of symptoms of mental illness (Behavior and Symptom Identification Scale [BASIS]) (
16) and a measure of hope (Herth Hope Index [HHI]) (
17) because of their documented high reliability and because previous studies have shown impact on these outcomes (
18–
27). The variable of interest that was assessed for predicting positive change in the intervention group was the extent of the relationship between the peer specialists and individuals assigned to work with them. Social comparison and social learning theories mentioned above and other theories about peer specialist effectiveness—for example, social support theories, experiential knowledge, and the helper-therapy principle (
7,
28–
31)—all assert the importance of a strong relationship. Although multiple qualitative studies (including qualitative data from the larger study [
32]) have supported that peer specialists are highly valued by the individuals with whom they work (
33,
34), significant variation can occur, as observed in the AMPS study (
35) and in other studies of peer specialists working in homelessness services (
36). Such findings suggest that veterans might not always be willing or able to engage with peer specialist services when such services are offered. These theories suggest the hypothesis that among veterans assigned to receive peer specialist services, those with greater engagement would be more likely to experience reliable improvement than those with less engagement.
There are few “as treated” analyses of the effectiveness of peer specialists, and the results are unclear. In one randomized trial whose intent-to-treat analyses showed no differences between peer specialist, community volunteer, and usual care groups (
37), improvement was associated with having at least one contact with a community volunteer but zero contact with the assigned peer specialist. In another randomized trial (
23), the as-treated analysis (at least one peer specialist contact) showed far fewer significantly improved alcohol and drug use outcomes but a few more physical health and functioning outcomes, compared with the intent-to-treat analysis.
This study built on this work and adjusted for sociodemographic and health-related variables (drug and alcohol use history, time homeless, and baseline mental health problems) that may influence the likelihood of positively changing when working with peer specialists. In addition, VA service utilization during the study that may have affected psychiatric symptoms and hope (that is, use of homelessness, substance use, and mental health services) were controlled for in all models to more comprehensively explore the relationship between peer specialist engagement and positive change.
Methods
Measures and Data Collection
The study took place between December 2012 and May 2017. All veterans provided informed consent in the AMPS study; they had already been enrolled in HUD-VASH and had a dual diagnosis (N=166). Eligibility criteria included active or recent history of substance abuse or dependence and a diagnosis of a mental disorder identified through VA records or as clinically judged by a case manager. Participants in the study reported here included only those who had completed study interviews before and after the approximate one-year treatment period (N=140) so that reliable change could be calculated. Those who completed the study did not differ from noncompleters on any demographic or other baseline covariates. All veterans were assessed with the following self-report measures: BASIS (
38), the Addiction Severity Index (ASI) (
39), the HHI (
17), and the Temple University Community Participation Measure (
40). Data on use of VA homelessness, substance use, mental health, and peer specialist services were extracted from the VA Common Data Warehouse (CDW) (
41).
Dependent Variables
As described above, the HHI and BASIS were used to categorize all participants as having either exhibited positive change or no or negative change. The RCI index is sensitive to measurement reliability (internal consistency and Cronbach’s alpha). Thus, to maximize the identification of positive and negative changers, we wanted to use measures with well-documented high reliability in similar populations. Of the measures used in the AMPS study, only the HHI and BASIS had sufficiently high reliability. In addition, these measures address aspects of recovery and symptoms, both of which have shown improvement in previous studies of peer specialists. For example, we chose the HHI because maintaining hope that one can improve is viewed as an important part of recovery (
42), and multiple studies have shown that peer specialists can build hope (
18,
21,
24,
26,
27). The HHI is a 12-item index adapted from the 30-item Herth Hope Scale. It has been shown to be reliable (α=.97) and valid, correlating with the Existential Well-Being Scale (
43) (r=.84) and the Hopelessness Scale (
44) (r=–.73) in a sample of ill adults (
17).
We chose the 24-item BASIS because of its widespread use as a comprehensive measure of psychiatric symptoms and because multiple studies have shown that peer specialists can improve such symptoms (
19,
20,
22,
23,
25,
26). It consists of an overall summary score that includes six domains: depression-functioning, interpersonal relationships, self-harm, emotional lability, psychotic symptoms, and substance abuse. In a national sample of more than 3,200 recipients of mental health or substance abuse services, the BASIS summary score had high reliability (α=.90) (
38). We applied the RCI equation (
15) to compute whether each participant achieved reliable positive change (coded as 1) or negative or no reliable change (coded as 0) on the HHI and BASIS, and the resulting dichotomous variables were used as the dependent variable in logistic regression analyses (see below).
Independent Variables
To test engagement with peer specialist services and still be able to make comparisons to participants in the control group, the engagement variable (number of visits during the study) was dichotomized on the basis of a median split, creating a “low peer specialist engager” group (<12 contacts) and a “high peer specialist engager” group (≥12 contacts). Two dummy variables were created to compare each engagement group with participants in the control group, with one variable representing control group versus low engagers and the second representing control group versus high engagers. We did this because the distribution of peer specialist services was mostly bimodal—that is, some intervention participants saw their assigned peer specialists infrequently while others met with their peer specialist often.
All covariates, except service utilization, were collected at baseline. Participants were asked their age and race (selected among seven different racial categories and recoded into white and nonwhite because of lower frequencies in nonwhite categories). Lifetime homelessness categories (
5) were recoded to one year or less of lifetime homelessness and more than one year on the basis of data distributions. Data on drug and alcohol use and psychological problems were collected with the ASI, a widely used assessment with strong concurrent and discriminant validity among persons with substance use disorders and psychiatric disorders (
39). Participants self-reported years of lifetime alcohol intoxication and years of use for ten illicit or inappropriately used substances—heroin, methadone, other opiates, barbiturates, sedatives, cocaine, amphetamines cannabis, hallucinogens, and inhalants—averaged across the ten variables. Psychological and emotional problems were the number of days participants reported that they had experienced such problems in the past month.
Last, additional variables included as covariates were study site (site 1 or site 2) and use of homelessness, substance use, and mental health services during the study; data on service use were obtained from the VA CDW. Service use contacts were divided by total months enrolled to adjust for differing enrollment lengths across participants (mean±SD=11.49±3.24 months).
Data Analysis
Our primary analyses were conducted to predict positive change on the HHI and BASIS by using logistic regression. Prior to running logistic regression models, we tested for significant differences between participants in the control group, low peer specialist engagers, and high peer specialist engagers on various demographic and other baseline variables (
Table 1). Bivariate correlations were computed to examine the interrelatedness of all study variables and to help assess for multicollinearity between predictors before the regression analyses were conducted. No predictors were correlated at ≥.90, indicating a low likelihood of collinearity problems (
45). Moreover, tolerance and variance inflation factors for each predictor were computed, and all were in acceptable ranges (tolerance values all >.1, and variance inflation factors all <2.5), further indicating low probability of collinearity (
46).
We performed two separate binary logistic regression models assessing the likelihood of positive change on the HHI and BASIS (0, no or negative change; 1, positive change). As stated above, each model included two dummy variables testing peer specialist engagement: control group versus low engagement and control group versus high engagement. Both models adjusted for the following covariates: age; race; study site; history of lifetime homelessness; lifetime alcohol intoxication; lifetime average drug intoxication; frequency of past-30-day emotional or psychological problems; and use of homelessness, substance use, and mental health services during the study. Odds ratios (ORs) and their 95% confidence intervals were computed to determine effect sizes and statistical significance. Analyses were performed with IBM SPSS Statistics 24 and SAS, version 9.4.
Discussion and Conclusions
The findings suggest that there was a subgroup of individuals assigned to work with peer specialists who exhibited reliable positive change after about one year of peer specialist services, even when results of the intent-to-treat analysis did not indicate significant improvement. Over 40% of participants with a high level of engagement with their peer specialist showed reliable positive change in their symptoms (BASIS), compared with only 24% of those with a low level of engagement and 11% of those in the control group. The logistic regression model showed the same result, even after the analysis controlled for several relevant covariates. Having a high level of peer specialist engagement was the only variable associated with a positive change in symptoms, whereas being white was associated with a lower likelihood of having a positive change in symptoms. Other studies have shown that peer specialists can be helpful particularly to those who are African American (
47). However, it is possible that persons of color were more likely to exhibit positive changes in symptoms because traditionally these groups have had difficult accessing services, and the HUD-VASH program provided greater access to services regardless of the presence of a peer specialist. The level of peer specialist engagement was not related to reliable positive change on the HHI, possibly because receiving a housing voucher had already led to improved feelings of hope.
These findings suggest that the use of peer specialist services may be a matter of fit between the client and the peer specialist. This point was suggested in an early review of peer support groups (
48), which documented that initial participation often fell off but that significant benefits accrued for those whose group attendance was higher. It appears that the same may be true for high versus low users of individual peer specialist services. The beneficial effect of greater contact with peer specialists is consistent with qualitative interviews from this study (
32) and with other studies of veterans receiving peer specialist services (
49). In particular, those in this study who were high peer specialist engagers expressed strongly that they benefited from their relationship. These findings could have important policy implications for the use of peer specialists in clinical settings. Although peer specialists are sometimes used in roles that limit relationship building (for example, greeter or driver), the findings of this study suggest that it may be beneficial to allow peer specialists to develop strong, longer-term relationships with those who want that type of support.
Certain limitations should be noted. These results are from a small sample in two VA medical centers. In addition, the use of high and low peer specialist engagement groups in regression analyses did not allow for a determination of causality. Larger-scale studies that can randomly assign individuals to lower and higher amounts of peer specialist services are needed to further explore these results. Qualitative studies that explore reasons for lack of engagement would also be useful. The use of a median split to categorize the level of peer specialist services to assess the impact of engagement is a somewhat rudimentary approach. However, the analysis of peer specialist engagement is in its infancy. For example, the two studies mentioned above (
23,
37) that also analyzed peer specialist engagement used no engagement versus some. Median split—hypothesizing that more contacts than the median would be associated with positive change—is a small advance over that approach. Finally, this study did not control for peer specialist characteristics that may have influenced the relationship between peer engagement frequency and positive change. We hope that this study will stimulate future research (with larger sample sizes) that can begin to address the issue of the exact amount of peer specialist services that are needed to achieve positive outcomes.
Despite these limitations, this study showed that veterans with mental illnesses and substance use disorders who engaged in more frequent interactions with peer specialists benefited. The type of subgroup analysis conducted in this study adds to nascent literature evaluating the impact of variable engagement as envisioned by the peer specialist research agenda mentioned above (
8). More studies are needed to investigate the impact of various patient and peer specialist factors on outcomes, which could shed light on the circumstances under which peer specialists can be most effective.