Effectiveness in Regular Practice of Collaborative Care for Depression Among Adolescents: A Retrospective Cohort Study

The intervention was a quality improvement project that became standard practice in a community pediatric clinic, with eventual spread to family medicine clinics. The project, termed EMERALD (Early Management and Evidence-based Recognition of Adolescents Living with Depression), was an adaptation of an adult collaborative care model (12). Some patients might have found alternate routes to behavioral health, such as the NNCPAP-affiliated Psychiatric Access Line available in the same state (13). However, EMERALD was designed as a stand-alone set of integrated services.

Pediatric and family medicine patients with a depression diagnosis—a score of ≥10 on the nine-item Patient Health Questionnaire Modified for Adolescents (PHQ-9A) (14)—and without a diagnosis of bipolar disorder were eligible. Beginning in 2014, the PHQ-9A was routinely administered in well-child visits for patients ages 12–17. Prior to that, patients who came in for limited exams with a chief complaint of a mood-related, sleep, or school concern and those who were taking an antidepressant were systematically given the PHQ-9A; it was also administered to other patients at the discretion of the provider or care team. Recent public reporting data indicated a 71% rate of mental health screening at well-child visits among this system’s local sites in 2016 (15); EMERALD sites had rates up to 79%.

A primary care provider (PCP) made the initial diagnosis of depression; if clarification was needed, the patient could see a child and adolescent psychiatrist (CAP) or another member of the integrated behavioral health team. Patients who verbally agreed were then referred, often directly via “warm hand-off,” to a registered nurse care coordinator (RNCC) embedded in a pediatric practice who was trained in depression management and motivational interviewing. Adolescents ages 12–17 required patient and parental consent; those age18 (eligible if in high school) could provide consent themselves.

The RNCC met with the patient and one or both parents (or caregivers), explained the program, answered questions, and scheduled enrollment interviews with both the patient and one or both parents. This initial meeting included a review of confidentiality; the RNCC explained that what the adolescent said was confidential, but because of mandated reporting, questions of safety or abuse would necessitate telling both parents. Addressing this issue in the beginning allowed for questions and was intended to provide reassurance for both adolescents and parents.

Every patient enrolled into EMERALD was administered case-finding tools for depression (PHQ-9A), anxiety (16,17), substance abuse (18), attention-deficit hyperactivity disorder (19), and bipolar disorder (20) to identify comorbidities or exclusions. The RNCC interviewed patients and parents separately and together. Enrollment interviews also covered current stressors; social, family, and medical history; school performance and other activities; and psychiatric history, including medications, counseling, or hospitalization. The RNCC established behavioral activation goals, assisted with internal and external referrals, and started to develop a relapse prevention plan in collaboration with the patient.

In weekly case review meetings with the RNCC, the CAP made recommendations regarding medication, therapy, or a consult with the CAP. The RNCC then passed on this input (for example, drug and dosing recommendations and diagnostic clarifications) to the PCP, who made the final decision. The RNCC then contacted the patient and parents with recommendations.

The RNCC also had weekly or biweekly contact with the adolescent, either by telephone or face to face, and contacted one or both parents biweekly. Fidelity to these contacts was generally good. We lacked data on successful and unsuccessful contacts for this cohort, but unpublished early data on EMERALD indicated that 61% of contacts were successful. The frequency of visits depended on the severity of depression; the RNCC established the time frame with both the adolescent and parents. Motivational interviewing was a core skill of the RNCC, and thus it was used in essentially every interaction. Its use, along with behavioral activation, was key to engaging the adolescent in care coordination, including addressing the willingness of the adolescent to engage and identifying his or her specific concerns and goals for each visit and the next.

Psychotherapy (primarily cognitive-behavioral therapy) and other services and referrals via a licensed clinical social worker were available to all patients and into the primary care setting. Patients who had significant concerns regarding comorbidities or patients who were not improving were seen directly by the CAP or referred on for higher-level services, or both. The CAP was available in the clinic during designated times; providers were encouraged to discuss concerns then. The CAP made medication and treatment recommendations and diagnostic clarifications and saw patients when necessary.

Patient outcomes were tracked via a clinical registry. The primary discharge criterion was sustained depression remission (PHQ-9A score of <5 for three consecutive months). For patients who did not enter EMERALD, other support included usual care with a PCP or provision of a list of community providers (for example, for cognitive-behavioral therapy) to the patient or the PCP. Other referrals could be made to social work, CAPs, and individual and group therapy. PCPs for non-EMERALD patients could also attend a weekly huddle session for pediatric integrated behavioral health or could contact the RNCC to discuss specific cases.

There was no additional unique billing or payment for collaborative care—only what was already present in fee-for-service billing (for example, therapy and PCP visits). Almost 50% of the patients in the practice were self-insured; combined with the promise of a better model of adolescent depression care and existing challenges to accessing the services of a psychiatrist, this was sufficient support to obtain permission from clinic leadership and move forward without additional fees.

Data come from electronic health records for individuals eligible for EMERALD (see below) during the observation period (January 2010 to December 2015) at a large multispecialty practice in the U.S. Midwest. The sample included adolescents (ages 12–17 or age 18 and still in high school) from pediatric and family medicine practices who had a PHQ-9A score of ≥10 and a diagnosis of major depressive disorder or dysthymia (ICD-10 diagnosis of persistent depressive disorder) during the observation period and who were at least six months past baseline at the time of data extraction (this allowed for follow-up assessment). Because eligibility required a PHQ-9A of ≥10, the sample was limited to those with nonmissing baseline scores. Baseline was defined as the date of eligibility—that is, the date with a qualifying PHQ-9A and diagnosis during the period when EMERALD was in place. Individuals were excluded if they elected not to have their clinical data used for research. The final sample was 162 EMERALD and 499 comparison patients (total N=661).

Our two primary outcomes were both based on PHQ-9A scores (the PHQ-9A was rolled out during the observation period, resulting in some regular PHQ-9 scores in the data [see sensitivity analyses below]). Outcomes were six-month remission of depression (six-month score of <5) and six-month treatment response (>50% reduction from baseline). Attrition-related missingness was assumed nonignorable, that is, missing not at random (21), because missing depression scores were likely associated with the missing values themselves or unobserved confounders. Therefore, we used last observation carried forward (LOCF) for those without a score during the follow-up window to render the most conservative comparisons (this was borne out in sensitivity analyses [see below]). We measured two follow-up windows because of changes in quality reporting that happened midstudy: 150–210 days and 120–240 days. We chose the wider window because it reflected quality measurement at the time of writing (see sensitivity analyses).

Our primary independent variable was participation in EMERALD. Comparison patients included those eligible for EMERALD but for whom EMERALD was not yet available at their clinical site (N=360) and those who opted out of EMERALD because of patient, parent, or provider choice for the patient not to participate (N=139). Characteristics were largely similar between different sets of comparison patients. [A table in an online supplement to this article presents data on baseline and follow-up differences for the samples.] We also conducted sensitivity analyses regarding different comparison groups.

Potential confounders added as covariates included demographic characteristics—patient age, sex, race-ethnicity (coded as white non-Hispanic versus other; some groups were very small, creating problems with perfect prediction in models), and use of an interpreter.

We created covariates for clinical adjustments, including baseline PHQ-9A score. In addition, because EMERALD participation potentially entailed more observations or time observed, we controlled for the number of depression score measurements in the extracted data and the number of days from baseline to the latest measure in in the extracted data. Because these variables were based on total data extracted (rather than the six-month follow-up window), the variables could count observations and days beyond the six-month window to offer the broadest possible look at patterns of individual contact. We also measured whether individuals had a depression measure in the six-month follow-up window (versus requiring LOCF for the outcome).

Analyses were conducted with Stata 14 (22). Our main analyses consisted of adjustment by using logistic regression and propensity score matching, both with robust standard errors. First, we used logistic regression models controlling for all covariates to produce marginal adjusted predictions (adjusted rates) of outcomes and contrast tests of those estimates. Next, for propensity score matching, we used Stata’s teffects psmatch command for a 2:1 nearest-neighbor match of comparison with EMERALD patients, using matching with replacement and a caliper (maximum difference in propensity score) of .1, to obtain two or more matches within the caliper. Matching with replacement increases the set of possible matches, minimizing bias, and allows matching for intervention and comparison patients to approximate the population average treatment effect (23). Matching in psmatch also tests for overlap between groups. To ensure balance, we produced box and density plots of propensity score balance [see figure in online supplement]. We found that age and baseline depression score had worse individual balance after matching. After removing them, overall balance improved, and results were slightly more conservative for treatment response; thus the main results omit age and baseline PHQ-9A. We obtained average treatment effects as the difference in potential outcomes between EMERALD and comparisons.

We conducted several sensitivity analyses to assess robustness (not shown). All sensitivity analyses described here produced effect sizes similar to or less conservative than the main results. For simplicity, we used logistic regression to test multiple variations on data and sample, including a narrower follow-up window, using PHQ-9 and PHQ-9A scores, and limiting comparisons to those for whom EMERALD was not available. We also tested inverse probability weighting with regression adjustment and nearest-neighbor Mahalobis distance matching. Finally, other approaches to attrition included complete case analysis; multiple imputation; and a Heckman model (Stata command heckprobit), which incorporates attrition by using a selection model. In addition, we performed pattern mixture models for nonignorable missing data (user-written Stata command rctmiss [24]). Although rctmiss is designed for experimental data, it allows the same logistic regression adjustment as in our main models while also comparing effect sizes across plausible values for missing data. Our pattern-mixture models tested assumptions in which missing EMERALD or comparison patients had remission or response rates that were either similar to their own observed rate or similar to the other group’s observed rate. Pattern-mixture model results were consistent with main results. The most conservative results from the pattern-mixture models—those that assumed that patients with missing values in one group would have had an outcome rate similar to the observed rate in the opposite group—were closest in effect size to the LOCF findings.