Participants and Randomization
Using county lists and partner nominations, we identified agencies offering services identified by community members as relevant to depression (mental health specialty services, primary care, public health services, substance abuse treatment services, social services, faith-based services, park community centers, hair salons, and exercise clubs). Eligible agencies offered services to adults or parents of children and were financially stable—that is, expected to operate for one to three years—and were selected to oversample four subgroups prioritized by participating communities (homeless persons, seniors, African Americans, and persons served in substance abuse treatment programs). Agency and program enrollment occurred from November 6, 2008, through August 17, 2010. In 60 potentially eligible agencies that had 194 programs, 133 programs were confirmed as potentially eligible and randomly assigned (RS, N=65; CEP, N=68). Site visits after randomization to confirm eligibility and finalize enrollment were conducted by staff blinded to assignment; 20 programs were ineligible, and 18 refused. Therefore, 95 programs from 50 consenting agencies enrolled (RS, N=46; CEP, N=49). Zip code–level census tract data indicated that participating and nonparticipating programs were comparable in neighborhood demographic characteristics (p>.10) (
8,
16).
Within programs, staff blinded to the intervention screened clients for eligibility in waiting rooms or at events over two to three days for each program. Eligibility was based on being at least 18 years of age, speaking English or Spanish, providing contact information, and having depression symptoms (as measured by a score of ≥10 on the eight-item Patient Health Questionnaire, modified [PHQ-8]). Between March 25 and November 18, 2010, staff approached 4,649 adults to assess their eligibility; 4,440 (96%) in 93 programs agreed. Of these, 1,322 (30%) were eligible, and 1,246 (94%) of the eligible adults consented. Of these, 981 (79%) completed baseline telephone surveys conducted by staff blinded to the intervention (April 27, 2010, to January 2, 2011). Enrolled participants were invited to complete six- and 12-month follow-ups, except for those whose previous survey status was in one of the following categories: ill or incarcerated, unable to contact, or deceased at previous survey participation. These results were reported elsewhere (
8,
10).
For the extension study, between January 14, 2014 and October 14, 2014, we attempted to contact 1,004 participants from 89 programs who were eligible for three-year surveys—that is, enrolled; completed baseline, six-month, or 12-month follow-ups; and neither refused follow-up nor were reported deceased at prior surveys. Of the 1,004 clients, 600 (60%) participated (RS, N=293; CEP, N=307), 24 (2%) were deceased (RS, N=13; CEP, N=11), ten (1%) refused (RS, N=7, CEP, N=3), three (<1%) were ill or incapable (RS, N=2; CEP, N=1), and 367 (37%) were not reached (RS, N=181, CEP, N=186). [A figure showing study recruitment is included in an online supplement to this article.] The mean postbaseline follow-up interval was 1,321 days (median=1,314).
Interventions
Both CEP and RS interventions encouraged use of depression care QI tool kits, based on collaborative care, including manuals for cognitive-behavioral therapy for depression (CBT), clinician assessment and medication management, care management (that is, depression screening, care coordination, patient education, and outcomes tracking), lay health worker support, and team leadership (
12,
17–
20). Tool kit materials were designed so that all staff with direct client contact (paid, volunteer, licensed, and nonlicensed) could benefit, depending on their role. Tool kits were provided in hard copy, on flash drives, and via a Web site (
http://www.communitypartnersincare.org/community-engagement/cep) and introduced in kickoff conferences prior to participant enrollment. Providers received an orientation and review of modules relevant to their discipline. [A table in the
online supplement presents more information about the interventions and the training.]
RS used a train-the-trainer implementation approach between December 2009 and July 2010, which offered site visits and 12 webinars of 90 to 120 minutes each covering core principles and skills, along with access to all versions of the tool kits. Expert trainers included psychiatrists, a nurse care manager, a CBT trainer, and QI and community engagement specialists with support staff.
CEP used a coalition implementation approach between December 2009 and July 2011, and invited program administrators across all sectors within a given community to attend two-hour meetings biweekly for four months. The coalitions followed a workbook that outlined intervention goals, principles, and potential sessions and provided information on resources. Planning meetings were co-led by academic and community members. Main activities included reviewing and adapting tool kits to community priorities and culture; developing plans for staff training as a network; training local staff to co-lead QI training sessions with experts; and developing a written plan for QI training, implementation, and maintenance. Each CEP council was provided $15,000 and tool kits (equivalent value of RS resources) to support planning. Final plans featured half- or full-day conferences, follow-up training sessions at sites, telephone and webinar supervision for CBT and case management, and innovations such as alternative medicine training, provider self-care, and resiliency psychoeducation classes led by laypersons to introduce CBT concepts. CEP plans as implemented, compared with RS expert training, led to higher rates of participation in depression QI training by eligible staff (
11). Except for one agency with a common waiting room for RS and CEP, lists of enrolled clients were provided to CEP but not RS program administrators for safekeeping in a locked file, consistent with the design for CPIC (
12).
All enrolled CEP and RS clients were instructed that they were free to discuss study participation with their provider. All screened participants were given a health and community services resource guide. Clients were free to access any services or programs they wished, regardless of intervention status.
Data Sources and Outcomes
Staff blinded to randomization status administered surveys by telephone to clients at baseline, six months, 12 months, and three years.
Primary outcomes were poor mental health–related quality of life, defined by a 12-item mental composite score (MCS-12) ≤40 and, according to the original protocol, depression as indicated by a score of ≥10 on the PHQ-8 (
21). The two community-prioritized outcomes, developed under a prespecified community input process, were physical functioning, defined by a 12-item physical composite score (PCS-12) and behavioral health hospitalization nights—that is, hospitalization for a mental or substance use disorder as measured by total nights hospitalized in the prior six months. This measure differs from prior hospitalization measures (any, any with ≥ nights) from six- and 12-month analyses, due to low hospitalization frequency at three years. We developed counts of emergency room visits, outpatient visits to primary care providers (PCPs), outpatient services by PCPs for depression, mental health outpatient visits, outpatient visits to a substance abuse treatment agency or self-help group, social services for depression, calls to a hotline for a substance abuse or mental health problem, and days on which a visit to a self-help group for mental health were made. We also measured in the prior six months any outpatient visits to services for depression in the health care sector (primary care or public health, mental health, or substance abuse treatment program) and in the social and community sector (social services program, faith-based program, park community center, exercise program, hotline calls, and other), as well as visits to faith-based services for depression. We developed indicators of use of any antidepressant, any mood stabilizer, any antipsychotic, and any visit for depression (
8,
12). We developed an indicator of having at least minimally adequate depression treatment, defined as having at least two months of antidepressant use or at least four depression visits across mental health specialty and primary care sectors in six months (
12).
Analyses
Models of longitudinal outcomes models using baseline, six-, and 12-month data were previously published (
10). For this study, we focused the primary analysis on client status at three-year follow-up, or 2.5 years after the end of the active intervention period, adjusting for baseline covariates. This approach permitted use of multiple imputation as well as use of response weights to account for attrition.
We conducted intent-to-treat analyses with intervention status as the main independent variable, using linear regression for continuous variables, logistic regression for binary variables, or Poisson regression for count variables, adjusted for age, sex, three or more chronic general medical conditions, education, race-ethnicity, family income below the federal poverty level, 12-month alcohol abuse or use of illicit drugs, 12-month depressive disorder, community, and baseline status of outcome. We used item- and wave-level imputation for missing data (
22,
23) to adjust findings to the three-year eligible sample (1,004 eligible minus 24 deceased=980). We used weights to account for nonenrollment (
24,
25) and attrition. [Additional information about the analytic procedure is available in the
online supplement.]
All analyses used Taylor series linearization with SUDAAN release 11.1 to estimate variability, accounting for clustering (clients within programs), weighting, and multiple imputation. Significance of comparisons was assessed by using contrasts among regression coefficients. Results of regression models are presented as between-group differences for linear regression, odds ratios (ORs) for logistic regression, and incidence-rate ratios (IRR) for Poisson regression with 95% confidence intervals. We illustrated average results adjusted for covariates by using standardized predictions generated from fitted models (
25). We used two-sided tests with p<.05 for statistical significance. In the proposal, we designed the extension study anticipating a sample of 650 for group differences of ≥11%−12% in comparing proportions and standardized effects of ≥.23– .25 for comparing means, to achieve 80% power with alpha of .05 (two-sided) and intraclass correlation coefficient of .01–.04.
Given multiple secondary outcomes (
26), we built on the false-discovery-rate (FDR) framework (
27) as extended by Yekutieli and Benjamini (
28) and used both standard and FDR-adjusted p values (pFDR) in interpreting results across a large number of regression analyses (
26–
28). Results with pFDR <.05 are viewed as convincing evidence of a difference, and higher pFDR thresholds are considered as suggestive evidence. We separately calculated pFDR for the two primary outcomes, the two community-prioritized outcomes, service use in the health care sector, service use in the social and community sector, medication use, and summary utilization indicators.
We conducted a sensitivity analysis using all waves of data (baseline, six months, 12 months, and three years) without response weights, adjusting for the same set of baseline covariates as in the main analysis. We specified a spline model, with a linear segment between baseline and the first follow-up for initial improvement and another linear segment for the subsequent follow-ups; the two linear segments were specified to join at the first follow-up. In analyzing continuously scaled PCS-12 as the dependent variable, we employed a three-level, mixed-effects regression model by using SAS PROC MIXED. To account for the intraclass correlation due to the multilevel structure, we specified random effects at the clinic level, including random intercepts at the program level and a spatial power covariance structure at the client level to account for the unequal spacing of waves (
29). Initial explorations of three-level, random-effects logistic models by using SAS PROC GLIMMIX for binary outcomes yielded unstable estimates for program-specific random effects. We utilized a generalized estimating equation framework with logistic regression models for binary outcomes and Poisson models for count data by using SAS PROC GENMOD, specifying exchangeable correlation at the program level. From the estimated spline model, we developed a contrast involving a linear combination of coefficients to test intervention effects at each end point (baseline, six months, 12 months, and three years) and tested differences between intervention groups in change from baseline to six months, 12 months, and three years.
To provide context for anticipated dampening of intervention effects over time, we report information from the main implementation periods (baseline, six months, and 12 months) on the extent to which clients were exposed to elements of their assigned intervention as well as to potential cross-intervention contamination (that is, exposure to the intervention not assigned).