Standardized Versus Tailored Implementation of Measurement-Based Care for Depression in Community Mental Health Clinics

Twelve community behavioral health clinics across Indiana and Tennessee enrolled in a dynamic cluster-randomized trial conducted from June 2015 to June 2019. MBC was optional for clinics (i.e., not mandated) during this trial. Randomization was conducted similarly to methods described by Chamberlain et al. (7), with a sequential design for clustered observations. Clinics were matched on urbanicity, number of clinicians, and number of patients seeking depression treatment. Eight clinic clusters were randomly created to fill four cohorts (first) and two groups (second) by creating 10,000 random permutations and selecting a clustering with optimal similarity between the two groups. No blinding to group was used. Eligible clinicians at all clinics were offered a 4-hour MBC training, followed by 5 months of active implementation support. Details of the study protocol are described elsewhere (8). All study procedures were approved by the institutional review board at Indiana University.

No usual care comparison group was used; clinics in both groups, that is, standardized MBC implementation and tailored MBC implementation, were offered active MBC implementation support (4). The standardized implementation consisted of empirically informed strategies to enhance MBC use with fidelity to the following clinical guideline: administer, review, and discuss patient-reported outcome measures (PROMs) at each clinical session for adult patients with depression. These strategies included electronic health record (EHR) enhancements for MBC data capture, a needs assessment, interactive workshop training informed by adult learning theory (4), the aforementioned clinical guideline, and triweekly group consultation with an MBC expert during the 5-month active implementation phase. On average, six clinicians per site participated in consultation meetings. Tailored implementation included these strategies, and the workshop training was tailored to clinics’ barriers to MBC delivery informed by a mixed-methods needs assessment at baseline. Clinics could also adapt the MBC guideline (e.g., expand the number of patients receiving MBC) and had 5 months of triweekly implementation team meetings with an MBC implementation expert focused on overcoming contextual barriers to MBC use (e.g., task shifting administration of measures to staff at the front desk). On average, seven staff members per site participated in the meetings, including clinic administrators (N=13), office professionals (N=5), and clinicians (N=22). Across implementation groups, between five and 14 clinicians participated in the workshop training, with an average of 10 clinicians per site. A previous study provides a full description of the implementation groups (4).

Clinicians (N=154) offering individual psychotherapy to depressed adults across 12 clinics participated in this study (recruited in person or via e-mail from June 3, 2015, to October 18, 2016). (See an online supplement to this article for further details.) Clinicians were coded as primarily serving adults (N=107, 69%) versus children and families (N=47, 31%). Clinicians who served primarily children and families were included in this study only if their caseload had a significant portion of adult patients, given the focus of the MBC intervention on adults in this study. Across clinicians, 15,686 sessions yielded MBC-relevant EHR data for adult patients with a depression diagnosis; clinicians had a mean±SD of 101.9±91.6 sessions. For depression severity outcome analyses, clinicians had to have at least one participating patient; 81 clinicians (53%) met this criterion.

Patient data were included in the main clinician-level outcome analyses if the patients were diagnosed as having depression (see the online supplement) and were at least 21 years old at their first session with a participating clinician. Initially, only new patients were included, but eligibility was expanded to include existing patients, given the clinical utility of MBC for terminating psychotherapy cases. A total of 4,025 patients contributed data, across a mean of 3.9±3.4 sessions. For the analyses assessing depression severity outcomes, a subset of these patients were recruited to participate in structured telephone interviews with research specialists if they scored ≥10 on the nine-item Patient Health Questionnaire (PHQ-9) (9–11) after a qualifying encounter (see the online supplement). In total, 228 patients (i.e., the “interview sample”) consented to participate for a 12-week period of data collection that closed with readministration of the PHQ-9.

Our target N of 150 clinicians was informed a priori via a Monte Carlo simulation (power=0.80, two-tailed, α=0.05, intraclass correlation=0.05) to detect effect sizes of Cohen’s d≥0.30. Descriptive statistics were generated for all variables. We investigated the clinician outcome of MBC fidelity with two variables: whether the PHQ-9 was recorded in the EHR and, among encounters for which the PHQ-9 was recorded, whether it was discussed. These outcomes were modeled by using generalized linear mixed models with a logit link function for a binary distribution. Odds ratios (ORs) and ratios of odds ratios (RORs) with 95% confidence intervals (CIs) were derived by exponentiating the parameter estimates and CIs. All models were fit with the lme4 package, version 1.1.23 (17), by using R version 3.6.2 (18). Both outcome models were constructed with an identical sequence and included random intercepts for data from clinicians and patients, and model comparisons were evaluated with the Bayesian information criterion (BIC); the model with the best BIC was selected. Following recommendations from Singer and Willett (19), we fit unconditional growth models for best characterization of time (i.e., linear, quadratic, and log-transformed number of months). After establishing the unconditional growth model, we added fixed effects in the following sequence: one, group main effect, and two, group × time interaction. Finally, we investigated new versus existing patient status as a main effect by using a dummy variable for new patient status and as a patient status × group interaction.

Analyses of depression severity outcomes were modeled with linear mixed models with Satterthwaite degrees of freedom implemented in the R lmerTest package, version 3.1.2 (20). Because 23% (N=52 of 228) of patients who completed a baseline PHQ-9 interview did not complete the interview at week 12, we used multiple imputation to estimate missing values across 20 data sets. Data were imputed with the R Amelia package, version 1.7.6 (21). First, we assessed changes in PHQ-9 scores between intake and week 12 with a mixed model containing patient status as a covariate and random intercepts for patient and clinic. Implementation group was added to the base model described above. To examine the impact of MBC fidelity on patient depression severity, we added to the base model the total number of sessions, the number of sessions in which the PHQ-9 was recorded, and the number of sessions in which the PHQ-9 was discussed. We also examined the interactions between new patient status and the main outcome variables.

The PHQ-9 completion rate was lower in the tailored implementation group at baseline but increased over time, eventually surpassing the rate in the standardized implementation group (Table 3, Figure 1). A statistically significant negative effect on PHQ-9 completion in the tailored implementation group (OR=0.28) indicated that patients in this group had lower PHQ-9 completion rates at baseline (i.e., time=0), but a significant time × tailored interaction effect (ROR=1.60) indicated that the main effect changed over time in the tailored group (Table 3). Differences in estimated values from the models revealed that the odds of PHQ-9 completion were lower in the tailored implementation group at baseline (OR=0.28) but approximately three times greater at the median length of the implementation phase (5.3 months) (OR=3.39, 95% CI=1.00–11.48). In addition, PHQ-9 completion was more likely for new patients than for existing patients in the tailored implementation group (OR=1.77, 95% CI=1.42–2.22) but not in the standardized implementation group (OR=1.05, 95% CI=0.85–1.29).

Information about clinic guidelines for tailored MBC implementation and descriptive statistics of clinicians’ MBC fidelity by implementation group are provided in the online supplement. The set of models examined the impact of implementation group on full MBC fidelity (exploring whether patient scores were discussed in a clinician-patient session after PHQ-9 administration) over time. After excluding sessions in which the PHQ-9 had not been completed, 5,522 sessions held by 126 clinicians for 2,059 patients remained in the sample. After evaluating a variety of models of the change in MBC fidelity, we selected a piecewise model wherein one time segment represented the first 3 months of the implementation phase and another segment the last 2 months of that phase. Full MBC fidelity (i.e., a PHQ-9 result was both recorded and discussed) decreased across time (OR=0.62) for the first 3 months of the earlier phase of implementation (i.e., the first 3 months of implementation) but increased (OR=1.99) during the later implementation phase (i.e., from 3 months to the end of implementation) (Table 3). However, the implementation groups (tailored vs. standardized) did not differ in full MBC fidelity in the final model (Table 3). The addition of the new patients × tailored interaction did not improve model fit.

In both groups, PHQ-9 scores decreased substantially (b=−4.71, 95% CI=−5.53 to −3.89) from baseline (mean=17.6±4.4) to 12 weeks (mean=12.6±5.9). (See Tables S4 and S5 in the online supplement. Neither implementation group nor new patient × group interactions significantly predicted week-12 PHQ-9 scores. Additionally, neither the number of total patient sessions nor the number of sessions with a completed or discussed PHQ-9 predicted week-12 depression severity. Moreover, none of the interactions between these factors and new patient status significantly predicted week-12 depression severity. Finally, neither clinician-level (i.e., MBC attitudes and perceptions of implementation leadership) nor patient-level (i.e., age, gender, number of diagnoses of comorbid conditions, medication type, and new patient status) factors significantly moderated depression severity at week 12.

Exploratory analyses of postsession telephone survey data from patients suggested that in nearly half (47%, N=255 of 541) of the clinical encounters, therapy did not change as a result of PHQ-9 data review, whereas in 27% (N=144 of 541) of encounters a new goal was set and in 24% (N=130 of 541) a new strategy was tried.