Enhanced Fidelity to Treatment for Bipolar Disorder: Results From a Randomized Controlled Implementation Trial
Abstract
Objective
The authors compared fidelity to bipolar disorder treatment at community practices that received a standard or enhanced version of a novel implementation intervention called Replicating Effective Programs (REP).
Methods
Five community practices in Michigan and Colorado were assigned at random to receive enhanced (N=3) or standard (N=2) REP to help implement Life Goals Collaborative Care (LGCC), a psychosocial intervention consisting of four self-management support group sessions, ongoing care management contacts by phone, and dissemination of guidelines to providers. Standard REP includes an intervention package consisting of an outline, a treatment manual and implementation guide, a standard training program, and as-needed technical assistance. Enhanced REP added customization of the treatment manual and ongoing, proactive technical assistance from internal and external facilitators. Multiple and logistic regression analyses determined the impact of enhanced versus standard REP on patient-level fidelity.
Results
The participants (N=384) had a mean age of 42 years; 67% were women, and 30% were nonwhite. Participants attended an average of three group sessions and had an average of four care management contacts. After adjustment for patient factors, enhanced REP was associated with 2.6 (p<.001) times more total sessions and contacts than standard REP, which was driven by 2.5 (p<.01) times more care management contacts. Women and participants with a history of homelessness had fewer total sessions and contacts.
Conclusions
Enhanced REP was associated with improved LGCC fidelity, primarily for care management contacts. Additional customization of interventions such as LGCC may be needed to ensure adequate treatment fidelity for vulnerable populations.
Translating effective behavioral treatment models from research to routine practice has been identified by the National Institutes of Health as a public health priority. A number of psychosocial interventions have been shown to improve treatment adherence, quality of care, and health outcomes among persons with mental illnesses (1,2). However, few of these evidence-based practices have been successfully translated into community-based practices.
Even when evidence-based practices are delivered, treatment fidelity is suboptimal, especially in community-based practices, where resources are more limited than in traditional academic settings. Treatment fidelity is a process to ensure that the core treatment components of an intervention are delivered as intended (3,4). Fidelity can falter over time because mental health providers are not given the tools necessary to overcome barriers to implementation or they resist buying in to the new treatment (5–7).
Hence, a growing body of literature focuses on developing implementation interventions that can assist mental health providers in maintaining treatment fidelity. The Centers for Disease Control and Prevention established an implementation strategy called Replicating Effective Programs (REP) to translate effective HIV prevention strategies for community-based organizations (8,9). REP focuses on strategies that maximize fidelity to treatments through the development of treatment manual “packages” that are supported by provider training and limited technical assistance (10,11). Because these strategies mainly focus on the steps required to implement new programs, REP may not fully support the ongoing fidelity or sustainability of these practices, notably by encouraging provider buy-in and organizational support.
Few studies in the literature compare implementation strategies that are designed to enhance treatment fidelity. In one study of a community-academic partnership (12–14), investigators enhanced REP by adding facilitation, a process in which expert consultants and local stakeholders collaborate to implement and sustain new programs (15,16). Facilitation was adapted from the Promoting Action on Research Implementation in Health Services (PARiHS) framework. Facilitators help providers address organizational barriers, such as clinical flow issues and staff turnover, and foster ongoing relationships with organizational leadership to promote buy-in to the program (14,15,17).
This article reports six-month treatment fidelity outcomes from the Recovery-Oriented Collaborative Care study, a randomized controlled trial comparing the effectiveness of enhanced versus standard REP for implementing Life Goals Collaborative Care (LGCC) at community-based practices (15,18). LGCC is an evidence-based, manualized psychosocial intervention for individuals with bipolar disorder (19–23) that is based on the chronic care model (24,25). The intervention has been found in five randomized controlled trials to improve quality of life related to both general medical health and mental health and to reduce symptoms among patients with bipolar disorder (19–23). The core components of LGCC are described in Table 1.
| Fidelity measure | ||||||
|---|---|---|---|---|---|---|
| Protocol and component | Objective | Study design | Training | Treatment | ||
| Delivery | Receipt | Enactment | ||||
| LGCC protocol | Evidence-based, manualized psychosocial intervention for individuals with bipolar disorder | Theory-based model previously tested; a prior plan for number, length, and frequency of contacts; plan to monitor consistency of dose and fidelity to protocol | Hire mental health clinicians from local regions; standardized training curriculum; booster training and supervision based on direct observations of providers | |||
| Self-management support sessions (group sessions lasting 90–120 minutes) | Cover specific focus points and identify a health behavior goal | Use treatment manual as well as session scripts and outline; use Session Fidelity Checklist to directly observe degree to which focus points were covered and occurrence of nonspecific factorsa | Number of self-management sessions completed | Progress on self-management goals measured in care management phase | ||
| Session 1 | Self-management and collaborative care; understanding bipolar disorder, mood monitoring, and stigma; impact of core values and self-efficacy on health habits; bipolar disorder and its impact on health | |||||
| Session 2 | Working through mania; identifying personal triggers of (hypo)mania; responding to manic symptoms—cost-benefit analysis; develop a personal action plan for (hypo)mania | |||||
| Session 3 | Recognizing the symptoms of depression; identifying personal triggers of depression; responding to depressive symptoms—cost-benefit analysis; develop a personal action plan for depression | |||||
| Session 4 | Personal wellness change plan; building and strengthening a collaborative relationship; relapse prevention—community resources | |||||
| All sessions | Choose one health behavior goal from the following areas: nutrition, physical activity, stress reduction, medications. and sleep | |||||
| Care management contacts (monthly phone calls lasting 15–20 minutes) | Encourage health behavior goal progress by using motivational enhancement techniques; symptom self-management; review of upcoming medical and psychiatric appointments and reminding patients to attend; relaying urgent medical or psychiatric concerns to providers | Number of care management contacts made by health specialist to patient and providers | Monitor length of calls, missed calls, and completion of goals in registry | |||
| Provider guideline dissemination | Educational in-service for frontline providers focused on bipolar disorder, including impact on health, followed by dissemination of materials related to the linkage of self-management goals to bipolar disorder treatment guidelines outlined by the American Psychiatric Association (18) | None | None | None | None | None |
The study employed a hybrid effectiveness-implementation model that allowed for the comparison of both clinical and fidelity outcomes at practices that received standard or enhanced REP to support uptake of LGCC (26). We hypothesized that enhanced REP would increase treatment fidelity to LGCC implementation. We also examined patient factors associated with LGCC treatment fidelity.
Methods
The Recovery-Oriented Collaborative Care study has been described elsewhere (15,18). In brief, community-based practices in Michigan or Colorado were enrolled between March 2010 and December 2011 and were assigned at random to receive enhanced versus standard REP to help implement LGCC. This study was reviewed and approved by local institutional review boards.
Study population and treatment
Five community-based clinical practices agreed to participate, two from Colorado and three from Michigan. These practices serve a majority of individuals with mental disorders in their respective regions, and they represent a diverse patient population. To be eligible, practices were required to care for at least 200 unique patients with bipolar disorder and to have available at least one mental health provider, such as a social worker, nurse, or psychologist, to implement LGCC.
Randomization was stratified by state, and practices in each state were assigned at random to receive enhanced (N=3) or standard (N=2) REP. All practices had administrative buy-in to participate in the study before randomization to implementation strategy.
Implementation intervention
Providers at practices using standard REP (18) were provided with three components to support preimplementation and implementation activities: an LGCC manual package that included an outline and a treatment manual and implementation guide that were not customized to the practice; a standard training program; and program support given through as-needed technical assistance and intervention monitoring and feedback reports (Table 2).
| Component | Standard REP | Enhanced REP |
|---|---|---|
| Preimplementation | ||
| Package (manual) | Noncustomized LGCC manual | LGCC manuals customized to practices by using data from organizational needs assessment about population and treatment setting resources |
| Training | Standard LGCC training | Customized training for providers |
| Implementation | ||
| Standard program support | ||
| Technical assistance | As-needed (passive) technical assistance if practice contacts study team | |
| Evaluation and feedback | Monitor LGCC uptake via monthly reporting sheets | |
| Enhanced program support | ||
| External facilitation | 1. Initiation: external facilitator (EF) identifies an internal facilitator (IF) at each practice. | |
| 2. Provider contact: EF works with IF and LGCC providers to set measurable goals in LGCC. | ||
| 3. Active technical assistance: EF makes structured calls to IF and LGCC practice providers, giving specific guidance on implementation LGCC components. | ||
| 4. Ongoing marketing and sustainability: EF supports public recognition of LGCC success stories and helps IF and LGCC provider to summarize progress and develop long-term plans for sustainability. | ||
| Internal facilitation | 1. Initiation: IF meets with EF, LGCC providers, and practice leadership to establish measurable goals in LGCC uptake. | |
| 2. Relationship building: IF identifies practice priorities per leadership input and identifies other LGCC program champions. | ||
| 3. Benchmarking and ongoing rapport: IF measures progress and continues to develop rapport with practice leadership uptake. | ||
| 4. Cultural adaptation: IF uses knowledge of local practice culture to facilitate LGCC, addressing potential barriers and aligning LGCC goals with practice priorities. | ||
| 5. Marketing and sustainability: IF works with EF and LG providers to summarize progress to leadership and develop a business and training to ensure LGCC continuity. |
Enhanced REP included key steps to further adapt the LGCC package to the local practices on the basis of provider and consumer input. In addition, a study staff member provided facilitation for six months to enhance provider buy-in and uptake of LGCC over time (Table 2). We chose to provide facilitation for six months, balancing the potential cost of the implementation intervention with the desire to provide a more intensive intervention than REP (9).
The first step at the practices that received enhanced REP was a needs assessment to identify the practices’ current priorities and initial organizational barriers and facilitators to implementing LGCC. An additional step included customization of the LGCC package and training program for each practice on the basis of feedback from the needs assessment. For example, consumer workbooks were tailored to reflect local community resources, and group meeting frequency was limited to four sessions in both arms on the basis of enhanced REP feedback in step 1 indicating a desire to minimize travel burden.
The next three steps involved facilitation support (Table 2). Two study staff members based in Michigan and Colorado served as external facilitators, providing technical assistance to practices in implementing LGCC. In addition, external facilitation required identifying an internal facilitator at each practice who was an employee with a direct reporting line to leadership. Subsequently, external facilitators worked with internal facilitators to set measureable goals, problem-solve implementation challenges through scheduled telephone consultations, and develop plans to market and sustain the program. Internal facilitators did not provide direct LGCC services; instead, they assisted providers to implement LGCC by addressing organizational barriers unique to each practice and met with practice leadership on a regular basis. As detailed in Table 2, internal facilitators actively worked to ensure the program fit within the work flows, priorities, and long-term plans of their local practice. The internal and external facilitators worked together to identify opportunities to enhance LGCC implementation.
Procedures
Health specialists from each practice were responsible for implementing LGCC. They were jointly hired by research and clinical staff and had a clinical background in mental health care and previous experience in addressing suicidal ideation, severe manic episodes, and other emergent issues. Patients were eligible if they had a diagnosis in their medical record of bipolar disorder (bipolar I, II, or not otherwise specified) or schizoaffective disorder, bipolar subtype; received outpatient care from the participating practices; and lived in the community rather than a nursing home or other institution. Patients were excluded if they were not able to provide informed consent because of serious illness or evidence of intoxication at enrollment.
After identifying eligible patients, the health specialists approached them by mail, by phone, or in person to request that they participate. Patients were enrolled after providing informed consent and completing a baseline assessment survey. Participants were asked to attend weekly group sessions for four weeks, followed by monthly individual care management phone calls for six months. Health specialists recorded patient attendance, clinical status, and length of care management phone contacts in an electronic registry. Participants in both implementation groups received $5 in remuneration for each group session that they attended to cover transportation costs and $20 for each survey assessment completed at baseline, six, 12, and 24 months.
LGCC training, supervision, and fidelity
Health specialist training, supervision, and fidelity monitoring were similar across enhanced and standard REP practices. Training lasted two days and covered the content of the four self-management sessions, registry use, and care management contacts. All health specialists participated in biweekly conference calls with the research team lasting one hour, when recruitment progress and other study issues were discussed.
Study investigators provided additional education and training to the health specialists at the three practices that received enhanced REP. This training was designed to address the specific organizational factors and barriers identified by each practice in its needs assessment. Health specialists at the practices that received enhanced REP also participated in six monthly calls with the external facilitator, who employed a problem-solving approach to discuss implementation barriers and offer support to overcome these barriers. A detailed mixed-methods description related to fidelity to these enhanced REP implementation strategy processes will be published separately.
Measures
Fidelity outcomes.
We used a practical approach for assessing treatment fidelity that was based on the National Institutes of Health’s Behavioral Change Consortium fidelity framework for psychosocial treatments (3,4) and previous LGCC intervention studies (19,27). Fidelity measures were developed a priori on the basis of core LGCC theoretical components (Table 1). Our two primary outcomes for fidelity included number of group self-management sessions completed and number of care management contacts completed.
In addition, study staff observed a random sample of 25% of group self-management sessions and used the LGCC session fidelity checklist to record information about the session (Table 1). The checklist is part of the LGCC package and is used by an independent rater during direct observations of all four self-management sessions (28). Among the multiple group cohorts of participants managed by each health specialist, the rater randomly selected a fourth of these cohorts to observe and rate (29,30). The first section allow raters to record session length, number of participants, location, potential sources of delivery problems, and the overall degree to which the LGCC session’s educational objectives were met. Section 2 rates characteristics of the provider, such as whether he or she was prepared and organized and ensured that participants understood the didactic content, and how well he or she met process goals, such as facilitating participation of discussion and interaction, avoiding judgmental feedback, and displaying empathy. Section 3 rates the degree to which the session covered designated focus points. We used the checklist to collect information about the number of focus points covered and process goals achieved in each session as well as the total amount of time for each session.
Cutpoints for minimum treatment fidelity were based on previously established minimum necessary standards (19,27) and consisted of attending at least three self-management sessions and having at least four care management contacts. For optimal fidelity, a cutpoint of attending all four group sessions and having at least six care management contacts was used (although additional care management contacts could be delivered as needed).
Covariates.
Independent variables that are thought to influence fidelity and also differ between practices that received enhanced or standard REP were ascertained from the patient baseline survey. Variables included demographic characteristics (age, gender, race-ethnicity, education, and living arrangement) and lifetime history of homelessness, defined by a standard question ascertaining whether the patient ever spent at least one night in a shelter, park, abandoned building, or street. Other covariates included clinical indicators at baseline, including depressive symptoms, alcohol use, and comorbid general medical illnesses. Depressive symptoms were ascertained by the Patient Health Questionnaire nine-item survey (PHQ-9) (31). Alcohol use, particularly hazardous drinking, was defined by the question on binge drinking from the Alcohol Use Disorders Identification Test alcohol consumption questions (32,33). The number of comorbid psychiatric and general illnesses was ascertained from patient self-report and was based on the question, “Has the doctor ever told you that you have one or more of the following?” Patients were asked to check boxes for hypertension or high blood pressure, arthritis or chronic pain, angina or coronary heart disease, heart attack or myocardial infarction, depression, posttraumatic stress disorder, diabetes or high blood sugar, or high cholesterol or parents with high cholesterol.
Data analysis.
Multiple regression models were used to compare treatment receipt fidelity (number of group sessions, number of care management contacts, and total number of sessions and contacts) between standard and enhanced REP treatment arms. Logistic regression models were used to compare minimum and optimal treatment receipt fidelity between patients at the practices that received enhanced versus standard REP. All models were adjusted for age, gender, race (white versus nonwhite), college education (yes or no), living arrangement (alone versus with others), lifetime history of homelessness (yes or no), depression symptoms (PHQ-9 score), hazardous drinking (yes or no), and number of comorbid general medical illnesses. For all fidelity models, an alternative analysis was conducted by including individual sites as fixed effects. All analyses were conducted by using SAS, version 9.2.
Results
A total of 2,019 potentially eligible patients were identified from medical record reviews as having a diagnosis of bipolar disorder, of whom 1,158 were found after consultation with providers to be ineligible. Another 477 patients declined to participate or could not be contacted, leaving 384 patients who consented and were enrolled in the study. Between enrollment and the start of LGCC group sessions, 67 patients were excluded from program participation because of dropout (N=51), physical illness (N=5), time conflict (N=4), incarceration (N=4), relocation (N=2), and cognitive impairment (N=1). Thus 317 patients started LGCC group sessions, 140 at the practices that received standard REP and 177 at the practices that received enhanced REP, and were included in the analyses. The patient mean±SD age was 42±11 years, 67% (N=256) were female, and 30% (N=115) were nonwhite, including 64 (17%) African Americans, 34 (9%) Hispanics, eight (2%) Native Americans, seven (2%) persons of multiracial background, and two (1%) Asians. Eighteen (5%) patients declined to divulge their race-ethnicity.
Patients attended 3.0±1.2 self-management group sessions (range 1–4), with 49% attending all four group sessions (Table 3). Group session length averaged 114 minutes (range=112 to 120 minutes). The group sessions conducted by all health specialists covered at least 80% of focus points.
| Total (N=317) | Enhanced REP (N=177) | Standard REP (N=140) | Enhanced vs.standard REP | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| LGCC component | N | % | Range | N | % | Range | N | % | Range | Test statistica | df | p |
| Group sessions (M±SD) | 3.02±1.22 | 0–4 | 3.08±1.12 | 0–4 | 2.94±1.33 | 0–4 | t=1.02 | 270 | .30 | |||
| 0 | 22 | 7 | 7 | 4 | 15 | 11 | χ2=6.09 | 4 | .19 | |||
| 1 | 16 | 5 | 10 | 6 | 6 | 4 | ||||||
| 2 | 51 | 16 | 31 | 17 | 20 | 14 | ||||||
| 3 | 74 | 23 | 43 | 24 | 31 | 22 | ||||||
| 4 | 154 | 49 | 86 | 49 | 68 | 49 | ||||||
| Care management sessions (M±SD) | 3.86±2.45 | 0–15 | 4.93±2.43 | 0–15 | 2.50±1.68 | 0–7 | t=10.51 | 309 | <.001 | |||
| 0 | 30 | 9 | 10 | 6 | 20 | 14 | χ2=97.55 | 7 | <.001 | |||
| 1 | 29 | 9 | 7 | 4 | 22 | 16 | ||||||
| 2 | 43 | 14 | 12 | 7 | 31 | 23 | ||||||
| 3 | 40 | 13 | 15 | 8 | 25 | 18 | ||||||
| 4 | 45 | 14 | 19 | 11 | 26 | 19 | ||||||
| 5 | 46 | 15 | 36 | 20 | 10 | 7 | ||||||
| 6 | 48 | 15 | 44 | 25 | 5 | 4 | ||||||
| ≥7 | 36 | 11 | 35 | 20 | 1 | 1 | ||||||
| Total sessions and contacts (M±SD) | 7.00±2.91 | 0–19 | 8.01±3.04 | 0–19 | 5.44±2.14 | 0–10 | t=8.84 | 1 | <.001 | |||
| Minimum fidelityb | 140 | 44 | 111 | 63 | 29 | 21 | χ2=55.91 | 1 | <.001 | |||
| Optimal fidelityc | 48 | 15 | 46 | 26 | 2 | 1 | χ2=36.69 | 1 | <.001 | |||
a
Chi square tests compared distributions across categories.
b
Minimum fidelity to LGCC was defined as ≥3 group sessions and ≥4 care management sessions.
c
Optimal fidelity to LGCC was defined as 4 group sessions and ≥6 care management sessions.
Participants had 4.0±2.4 care management contacts, and 57% had at least four contacts. Each care management contact lasted 22.6±12.1 minutes (range three to 80 minutes).
The total number of sessions and contacts was higher at practices that received enhanced versus standard REP (8.1±2.9 versus 5.5±2.1, p<.001). Patients at practices that received enhanced REP had significantly more care management contacts (5.0±2.4 versus 2.6±1.7, p<.001), but there was no significant difference in the number of group sessions attended at practices that received standard or enhanced REP.
Logistic regression analyses revealed that patients of practices that received enhanced REP were 7.2 times more likely to achieve minimum treatment receipt fidelity and 22.3 times more likely to achieve optimal treatment receipt fidelity (Table 4).
| Total sessions and contacts | Group sessions | Care management contacts | Minimum fidelityb | Optimal fidelityc | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Characteristic | β | 95% CI | t | β | 95% CI | t | β | 95% CI | t | OR | 95% CI | χ2 | OR | 95% CI | χ2 |
| Enhanced REP (reference: standard REP) | 2.64 | 2.01 to 3.27** | 8.3 | .11 | –.18 to .41 | .7 | 2.51 | 2.00 to 3.01** | 9.8 | 7.15 | 3.96 to 7.89* | 42.7 | 22.27 | 5.09 to 97.37** | 16.9 |
| Age | .03 | .01 to .06* | 2.0 | .01 | –.01 to .03 | 1.8 | .02 | –.01 to .04 | 1.5 | 1.01 | .99 to 1.04 | .8 | 1.02 | .98 to 1.06 | 1.3 |
| Female (reference: male) | –.73 | –1.36 to –.11* | –2.3 | –.30 | –.59 to .01 | –2.0 | –.41 | –.91 to .09 | –1.6 | .61 | .34 to 1.08 | 2.9 | 1.16 | .55 to 2.46 | .1 |
| Nonwhite (reference: white) | –.30 | –.97 to .36 | –.9 | –.05 | –.36 to .26 | –.3 | –.27 | –.80 to .26 | –.9 | .81 | .44 to 1.49 | .5 | .43 | .16 to 1.15 | 2.8 |
| College education (reference: none) | .28 | –.50 to 1.06 | .7 | .07 | –.29 to .44 | .4 | .17 | –.46 to .79 | .5 | 1.39 | .69 to 2.79 | .8 | 1.30 | .59 to 2.89 | .4 |
| Living alone (reference: no) | .72 | .07 to 1.38* | 2.2 | .16 | –.15 to .47 | 1.0 | .48 | –.05 to 1.00 | 1.8 | 1.80 | .99 to 3.29 | 3.7 | 1.39 | .65 to 2.99 | .7 |
| Lifetime history of homelessness (reference: none) | –.66 | –1.26 to –.07* | –2.2 | .03 | –.25 to .32 | .2 | –.72 | –1.20 to –.24* | –2.9 | .82 | .47 to 1.43 | .5 | .93 | .44 to 1.93 | .04 |
| Patient Health Questionnaire–9 score | –.02 | –.07 to .03 | –.9 | –.01 | –.03 to .01 | –1.1 | –.01 | –.04 to .03 | –.3 | .99 | .95 to 1.03 | .3 | .99 | .94 to 1.04 | .2 |
| Hazardous drinking (reference: none) | –.94 | –1.99 to .12 | –1.7 | –.33 | –.84 to .17 | –1.3 | –.64 | –1.48 to .19 | –1.5 | .53 | .20 to 1.39 | 1.7 | .23 | .03 to 1.86 | 1.9 |
| Number of comorbid general medical illnesses | .04 | –.27 to .35 | .2 | –.03 | –.15 to .14 | –.04 | .09 | –.16 to .34 | .7 | 1.07 | .82 to 1.42 | .3 | 1.05 | .72 to 1.53 | .1 |
a
The estimates were obtained from multiple regressions analyses that adjusted for all covariates except standard or enhanced REP (df=1).
b
Minimum fidelity to LGCC was defined as ≥3 group sessions and ≥4 care management sessions.
c
Optimal fidelity to LGCC was defined as 4 group sessions and ≥6 care management sessions.
*p<.05, **p<.001
Regression analyses also revealed that participants with a history of homelessness were less likely to have care management contacts (β=–.72, p<.05). Younger age (β=.03, p<.05) and living alone (β=.72, p<.05) were associated with fewer total sessions and contacts. Additional analyses in which practices were added as fixed effects produced similar results (Table 5).
| Total sessions and contacts | Group sessions | Care management contacts | Minimum fidelityb | Optimal fidelityc | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Characteristic | β | 95% CI | t | β | 95% CI | t | β | 95% CI | t | OR | 95% CI | χ2 | OR | 95% CI | χ2 |
| REP site (reference: standard REP, site 5) | |||||||||||||||
| Enhanced REP | |||||||||||||||
| Site 1 | 2.75 | 1.87 to 3.63** | 6.1 | 53 | .11 to .94* | 2.5 | 2.25 | 1.55 to 2.96** | 6.3 | 6.13 | 2.68 to 14.04** | 18.4 | 12.18 | 2.52 to 58.73* | 9.7 |
| Site 2 | .91 | –.13 to 1.95 | 1.7 | –.16 | –.65 to .34 | –.6 | 1.09 | .26 to 1.93* | 2.6 | 2.11 | .83 to 5.37 | 2.5 | 5.35 | .93 to 30.82 | 3.5 |
| Site 3 | 3.03 | 2.08 to 3.98** | 6.3 | .61 | .16 to 1.07* | 2.6 | 2.38 | 1.62 to 3.14** | 6.2 | 6.35 | 2.61 to 15.42** | 16.7 | 20.54 | 4.25 to 99.43** | 14.1 |
| Standard REP (site 4) | –.45 | –1.36 to .46 | –.9 | .54 | .11 to .97* | 2.5 | –.95 | –1.68 to –.22* | –2.6 | .42 | .16 to 1.09 | 3.2 | .001 | <.001 to >999 | .003 |
| Age | .03 | .0004 to .06* | 1.9 | .01 | –.002 to .03 | 1.7 | 0.02 | –.01 to .04 | 1.5 | 1.01 | .99 to 1.04 | .8 | 1.02 | .98 to 1.06 | .9 |
| Female (reference: male) | –.71 | –1.34 to –.08* | –2.2 | –.17 | –.47 to .13 | –1.1 | –.50 | –1.01 to .004 | –1.9 | .55 | .29 to 1.03 | 3.5 | 1.16 | .54 to 2.53 | .1 |
| Nonwhite (reference: white) | –.26 | –.92 to .40 | –.8 | –.14 | –.46 to .17 | –.9 | –.14 | –.68 to .39 | –.5 | .88 | .46 to 1.67 | .2 | .44 | .16 to 1.22 | 2.5 |
| College education (reference: none) | .24 | –.52 to 1.0 | .6 | .06 | –.30 to .42 | .3 | .13 | –.48 to .74 | .4 | 1.33 | .65 to 2.72 | .6 | 1.33 | .59 to 2.99 | .5 |
| Living alone (reference: no) | .69 | .06 to 1.34* | 2.1 | .16 | –.14 to .46 | 1.0 | .46 | –.06 to .97 | 1.7 | 1.79 | .96 to 3.31 | 3.8 | 1.46 | .66 to 3.21 | .9 |
| Lifetime history of homelessness (reference: none) | –.51 | –1.09 to .08 | –1.7 | .07 | –.21 to .35 | .5 | –.60 | –1.07 to –.13* | –2.5 | .91 | .51 to 1.61 | .1 | 1.08 | .51 to 2.29 | .04 |
| Patient Health Questionnaire–9 score | –.02 | –.07 to .02 | –1.0 | –.01 | –.04 to .01 | –1.2 | –.01 | –.04 to .03 | –.4 | .99 | .94 to 1.03 | .4 | .99 | .93 to 1.04 | .03 |
| Hazardous drinking (reference: none) | –.64 | –1.67 to .39 | –1.2 | –.27 | –.77 to .23 | –1.1 | –.44 | –1.26 to .38 | –1.1 | .63 | .23 to 1.73 | .8 | .28 | .03 to 2.33 | 1.4 |
| Number of comorbid medical illnesses | .09 | –.21 to .39 | .6 | .02 | –.12 to .16 | .3 | .12 | –.12 to .36 | .9 | 1.09 | .83 to 1.46 | .4 | 1.11 | .76 to 1.63 | .3 |
a
Enhanced or standard Replicating Effective Programs (REP) was provided at five sites to support implementation of Life Goals Collaborative Care (LGCC). The estimates were adjusted for all covariates (df=1).
b
Minimum fidelity to LGCC was defined as ≥3 group sessions and ≥4 care management sessions.
c
Optimal fidelity to LGCC was defined as 4 group sessions and ≥6 care management sessions.
*p<.05, **p<.001
Discussion
The Recovery-Oriented Collaborative Care study was one of the first comparative effectiveness trials of two different implementation strategies. One of its goals was to determine whether an enhanced versus standard version of a well-established implementation strategy improved fidelity to a psychosocial treatment for bipolar disorder. There is a growing demand for specific implementation interventions that are theory based and that are proven to enhance and maintain the fidelity of evidence-based practices. However, few implementation frameworks have been rigorously tested in health services trials for use as implementation strategies.
We found that compared with standard REP implementation, enhanced REP improved overall fidelity to LGCC independent of patient factors. The differences in fidelity were most pronounced for the care management component of LGCC. The level of treatment receipt fidelity to LGCC among patients of practices that used enhanced REP was similar to estimates reported in previous randomized controlled trials in more tightly controlled clinical settings (19,27). As in to these previous LGCC studies, we used practical fidelity assessments that were easy to deliver, such as the fidelity rater checklist. We were able to demonstrate good treatment delivery and fidelity of treatment receipt of the LGCC intervention.
Overall, high patient-level treatment receipt fidelity for number of group sessions attended and care management contacts was achieved at all practices. Health specialists across all of the practices demonstrated good fidelity to LGCC focus points and process goals in the group sessions. These findings could be due to the small incentives given to patients to cover transportation costs to group sessions. LGCC group sessions were also limited to four, which may have encouraged participation.
Compared with standard REP, enhanced REP was associated with attendance at significantly more total sessions and contacts and was more likely to be associated with optimal fidelity. The association of enhanced REP and optimal fidelity was due mainly to increased receipt of care management contacts. Often the hardest component to maintain, care management is a crucial component of LGCC and similar chronic care models because it facilitates assessment of clinical status over time and encourages positive changes in health behavior. Enhanced REP might have improved sustainability of care management because the external and internal facilitators were able to help the health specialists to secure the resources required for group sessions and provider follow-up. The added facilitation components, such as leadership engagement and customization, may have contributed to improved fidelity overall by promoting greater overall acceptance of LGCC in the practices. Greater use of care management in the practices that used enhanced REP also suggests that participants were more engaged with the health specialist and more focused on their wellness goals.
Additionally, we examined patient factors that influenced treatment delivery fidelity. Younger participants and those who were living alone participated in fewer group sessions. We also found that women participated in fewer group sessions and that study participants who had a history of homelessness had fewer care management contacts. Younger individuals and persons who live alone may have difficulty attending group sessions because of job constraints or transportation issues. Women may have difficulty attending group sessions because of child or other family caretaking responsibilities and may need different options for group times or child care. Similarly, participants with housing instability might benefit from more flexibility in delivery of care management contacts, such as face-to-face encounters or use of other technologies that enable more cost-efficient communication.
The study had limitations that may affect generalizability to other community settings. We were unable to assess contacts with other providers, either by the health specialists or by participants, or assess whether enthusiasm for LGCC at the sites varied across individual providers or over time. We were also unable to describe the specific components of enhanced REP, including the facilitator’s actions, that led to improved fidelity. We were unable to monitor fidelity to provider guideline support, including use of guidelines by other frontline providers in the practice, or to comprehensively assess patient factors, including co-occurring psychiatric conditions, that may have influenced fidelity. Also, the health specialists who delivered LGCC were all master’s-level clinicians with prior clinical experience, who may not represent the providers available elsewhere, especially in smaller community-based clinics.
Conclusions
Implementation interventions, such as enhanced REP, that include ongoing technical assistance to help providers implement evidence-based practices may improve treatment fidelity. However, future research should consider whether the added costs of facilitation can be absorbed by health care organizations because they contribute to the overall cost-efficiency of delivering effective psychosocial treatments in routine practice. Finally, measures that assess fidelity to the implementation intervention may be useful tools to improve the translation of research into practice. Such fidelity measures should be further developed in order to better assess the uptake of evidence-based practices.
Acknowledgments and disclosures
This work was supported by the National Institute of Mental Health and the Health Services Research and Development Service, Veterans Health Administration, Department of Veterans Affairs (VA). The views expressed in this article are those of the authors and do not necessarily represent the views of the VA.
Dr. Bauer receives royalties from Springer, and he and Dr. Kilbourne receive royalties from New Harbinger, for books related to this research. The other authors report no competing interests.
References
1.
Oxman TE: Collaborative care may improve depression management in older adults. Evidence-Based Mental Health 6:86, 2003
2.
Miklowitz DJ, Price J, Holmes EA, et al.: Facilitated integrated mood management for adults with bipolar disorder. Bipolar Disorders 14:185–197, 2012
3.
Bellg AJ, Borrelli B, Resnick B, et al.: Enhancing treatment fidelity in health behavior change studies: best practices and recommendations from the NIH Behavior Change Consortium. Health Psychology 23:443–451, 2004
4.
Borrelli B: The assessment, monitoring, and enhancement of treatment fidelity in public health clinical trials. Journal of Public Health Dentistry 71(suppl 1):S52–S63, 2011
5.
March JS, Silva SG, Compton S, et al.: The case for practical clinical trials in psychiatry. American Journal of Psychiatry 162:836–846, 2005
6.
Breitenstein SM, Gross D, Garvey CA, et al.: Implementation fidelity in community-based interventions. Research in Nursing and Health 33:164–173, 2010
7.
Glasgow RE, Lichtenstein E, Marcus AC: Why don’t we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. American Journal of Public Health 93:1261–1267, 2003
8.
Kelly JA, Heckman TG, Stevenson LY, et al.: Transfer of research-based HIV prevention interventions to community service providers: fidelity and adaptation. AIDS Education and Prevention 12(suppl):87–98, 2000
9.
Kelly JA, Somlai AM, DiFranceisco WJ, et al.: Bridging the gap between the science and service of HIV prevention: transferring effective research-based HIV prevention interventions to community AIDS service providers. American Journal of Public Health 90:1082–1088, 2000
10.
Adams J, Terry MA, Rebchook GM, et al.: Orientation and training: preparing agency administrators and staff to replicate an HIV prevention intervention. AIDS Education and Prevention 12(suppl):75–86, 2000
11.
O’Donnell L, Scattergood P, Adler M, et al.: The role of technical assistance in the replication of effective HIV interventions. AIDS Education and Prevention 12(suppl):99–111, 2000
12.
Kitson A, Harvey G, McCormack B: Enabling the implementation of evidence based practice: a conceptual framework. Quality in Health Care 7:149–158, 1998
13.
Kitson AL, Rycroft-Malone J, Harvey G, et al.: Evaluating the successful implementation of evidence into practice using the PARiHS framework: theoretical and practical challenges. Implementation Science 3:1, 2008
14.
Stetler CB, Damschroder LJ, Helfrich CD, et al.: A guide for applying a revised version of the PARiHS framework for implementation. Implementation Science 6:99, 2011
15.
Kilbourne AM, Neumann MS, Waxmonsky J, et al.: Evidence-based implementation: the role of sustained community-based practice and research partnerships. Psychiatric Services 63:205–207, 2012
16.
Goodrich DE, Bowersox NW, Abraham KM, et al: Leading from the middle: replication of a re-engagement program for veterans with mental disorders lost to follow-up care. Depression Research and Treatment, 2012; doi 10.1155/2012/325249
17.
Stetler CB, Legro MW, Rycroft-Malone J, et al.: Role of “external facilitation” in implementation of research findings: a qualitative evaluation of facilitation experiences in the Veterans Health Administration. Implementation Science 1:23, 2006
18.
Kilbourne AM, Neumann MS, Pincus HA, et al.: Implementing evidence-based interventions in health care: application of the Replicating Effective Programs framework. Implementation Science 2:42, 2007
19.
Bauer MS, McBride L, Williford WO, et al.: Collaborative care for bipolar disorder: II. impact on clinical outcome, function, and costs. Psychiatric Services 57:937–945, 2006
20.
Simon GE, Ludman EJ, Bauer MS, et al.: Long-term effectiveness and cost of a systematic care program for bipolar disorder. Archives of General Psychiatry 63:500–508, 2006
21.
Kilbourne AM, Post EP, Nossek A, et al.: Improving medical and psychiatric outcomes among individuals with bipolar disorder: a randomized controlled trial. Psychiatric Services 59:760–768, 2008
22.
Kilbourne AM, Goodrich DE, Lai Z, et al.: Life Goals Collaborative Care for patients with bipolar disorder and cardiovascular disease risk. Psychiatric Services 63:1234–1238, 2012
23.
Kilbourne AM, Goodrich DE, Lai Z, et al.: Randomized controlled trial to assess reduction of cardiovascular disease risk in patients with bipolar disorders: the Self-Management Addressing Heart Risk Trial (SMAHRT). Journal of Clinical Psychiatry 74:e655–e662, 2013
24.
Wagner EH, Austin BT, Von Korff M: Organizing care for patients with chronic illness. Milbank Quarterly 74:511–544, 1996
25.
Bodenheimer T, Wagner EH, Grumbach K: Improving primary care for patients with chronic illness. JAMA 288:1775–1779, 2002
26.
Curran GM, Bauer M, Mittman B, et al.: Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Medical Care 50:217–226, 2012
27.
Kilbourne AM, Post EP, Nossek A, et al.: Service delivery in older patients with bipolar disorder: a review and development of a medical care model. Bipolar Disorders 10:672–683, 2008
28.
Bauer MS, McBride L: Structured Group Psychotherapy for Bipolar Disorder: the Life Goals Program, 2nd ed. New York, Springer, 2003
29.
Bauer MS, McBride L, Williford WO, et al.: Collaborative care for bipolar disorder: I. intervention and implementation in a randomized effectiveness trial. Psychiatric Services 57:927–936, 2006
30.
Bauer MS, McBride L, Shea N, et al.: Impact of an easy-access VA clinic-based program for patients with bipolar disorder. Psychiatric Services 48:491–496, 1997
31.
Spitzer RL, Kroenke K, Williams JB: Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. JAMA 282:1737–1744, 1999
32.
Dawson DA, Grant BF, Stinson FS: The AUDIT-C: screening for alcohol use disorders and risk drinking in the presence of other psychiatric disorders. Comprehensive Psychiatry 46:405–416, 2005
33.
Dawson DA, Grant BF, Stinson FS, et al.: Effectiveness of the derived Alcohol Use Disorders Identification Test (AUDIT-C) in screening for alcohol use disorders and risk drinking in the US general population. Alcoholism, Clinical and Experimental Research 29:844–854, 2005
Information & Authors
Information
Published In
Cover: A ‘Bear’ Chance, by Philip Russell Goodwin, 1907. Oil on canvas. Minneapolis Institute of Arts, gift of the National Biscuit Company; the Bridgeman Art Library, New York.
History
Published in print: January 2014
Published online: 15 October 2014
Authors
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBLogin options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).