Effect of Pediatric Behavioral Health Screening and Colocated Services on Ambulatory and Inpatient Utilization

Our underlying conceptual framework assumed that as screening increases, patients would be identified and enter either specialty behavioral health services or behavioral health services in primary care settings. This would result in increased ambulatory behavioral health services. If early mental health treatment proves efficacious, inappropriate emergency department and inpatient behavioral health admissions should decrease over time. To test our assumptions, we used data from the Cambridge Health Alliance (CHA) pediatric clinics to capitalize on a natural experiment. CHA clinics phased in the use of a validated screening tool during well-child visits from 2004 to 2007. Using data from the CHA data warehouse (16,17), we conducted an interrupted time-series (ITS) analysis of utilization rates in the months pre- and postimplementation of the behavioral health screening and colocation program among a rolling cohort of primary care pediatric patients receiving care. The CHA Institutional Review Board approved the study in 2011.

CHA is a public hospital and clinic network in Cambridge, Massachusetts. At the time of this study, CHA operated three acute care hospitals with three emergency departments, two child mental health units, an inpatient pediatric medical unit, and multiple ambulatory health clinics, as well as a large division of child and adolescent psychiatry. All ambulatory clinics used the same inpatient and outpatient psychiatric resources located less than one mile away. In 2004, CHA began screening children ≥4 years 9 months (fifth-year well-child visit) to <18 years in three of its largest pediatric sites. The Pediatric Symptom Checklist (PSC [18]) was used for children under age 14, and the Youth-PSC (Y-PSC [18]) was used for those age 14 or older. Screening occurred at the annual well-child visit for children in the age range. These sites treated annually almost 16,000 children (from birth to 18 years old) during 2004–2008 (Cambridge Pediatrics, 6,672; Somerville Pediatrics, 7,267; and Union Square, 1,452). Twenty-four providers were available at these sites (seven each at Cambridge and Somerville and ten at Union Square), and none had used behavioral health screens or colocated social workers before project implementation. Sixty-four percent of children were from racial-ethnic minority groups (Cambridge, 61%; Somerville, 55%; and Union Square, 70%), and 43% spoke a language other than English (19). As noted elsewhere, a majority of mental health referrals for primary care patients were made to CHA providers (9).

During the study time frame, many changes occurred nationally in the delivery of child and adolescent mental health, including the black-box warning on the use of antidepressants (20), rising rates of specific disorders (specifically, bipolar disorder) (21), and higher use of antipsychotic medications (22). There was also heightened interest in identifying and treating behavioral health issues in pediatric offices (23).

The behavioral health screening and colocation program (still in existence today) was initially implemented at CHA in phases; Cambridge began screening in December 2003, followed by Somerville in July 2005 and Union Square in April 2007. The PSC and Y-PSC are used as screening tools because they are well validated in diverse urban populations (6,24) and incur no cost to providers. PSC results have compared favorably to the Child Behavior Checklist and the Children’s Global Assessment Scale (25). These 35-item instruments are completed in the waiting room by parent or teen prior to the well-child visit. The physician scores the screen during the visit, and later a medical assistant inputs the information into the electronic medical record (EMR). This process also automates provider reporting of screening rates and is described in depth elsewhere (10). Screening was not reimbursable until January 2008 as part of the Rosie D versus Patrick remedy, which mandated that primary care providers screen for behavioral health issues with validated tools at well-child visits for Medicaid-eligible children (26).

The colocation of behavioral health providers in the clinics happened simultaneously with screening implementation. At each site, a part-time licensed clinical social worker (four total), supervised by the CHA Division of Child and Adolescent Psychiatry, conducted on-site behavioral health services, including initial assessments, therapy, and consultation to pediatricians. This collaborative care model is similar to Kolko and Perrin’s (23) description of on-site interventions with aspects of care coordination. Providers made behavioral health referrals to child psychiatry, and based on patient preference, services were conducted by on-site social workers or at the child psychiatry main office.

Before program implementation, primary care providers and social workers were trained at each site in the use of the PSC and Y-PSC tools and relevant behavioral health diagnosis codes, which were already being reimbursed in Massachusetts. Regular reports on screening were generated and shared monthly with the clinical staff. Within six months of starting the program, each of the sites was able to increase the share of well-child visits with a screen for children in the age range to over 50% per month.

We extracted clinical and demographic data for youths ages ≥4 years 9 months to <18 years 3 months seen between 2003 and 2008. These ages include “screenable” youths and allowed for sufficient time after screening to capture any subsequent utilization. Data were extracted from the CHA data warehouse, which contains all ambulatory encounters within CHA, emergency department encounters, inpatient hospitalizations at a CHA hospital, and demographic data, such as gender, race-ethnicity information, and language preferences (17). We extracted all encounter and inpatient data that included location of service, provider department (pediatrics, family medicine), date of service, CPT4 procedure codes, and ICD-9 diagnosis codes for each visit. To calculate monthly utilization rates, we identified patients for inclusion in the denominator (patient panel) on the basis of location of their well-child and ambulatory care visits (defined by location and CPT code) and on age eligibility for screening. Any child without a primary care visit at the site in the prior year was excluded from the denominator for the given month. Children who had well-child care at multiple CHA sites were likely to have experienced the impact of the intervention before the time when other sites began to screen. Because this would have caused contamination of our data, we excluded these children (N=1,900, 14.7%) from both the pre- and postindex date panels. This group was significantly different from the rest of the sample on all demographic variables (p<.001). They were more likely to be female (55% versus 49%), to speak other languages (2% versus 5%), to be black (16% versus 12%), to be under age seven (19% versus 15%), and to be between 16 and 18 years old (18% versus 12%). They were less likely to be Hispanic (18% versus 21%). Demographic data for children included in the denominator of any study month were generated and aggregated by prepolicy and postpolicy periods for reporting purposes.

The period of patient panel identification and the calendar date of implementation (the index date) were different for each of the three sites. The denominator for the time series was the patient panel counts in each month. The preperiod time frame (18 months) was limited by data availability but allowed for a stable baseline and control for seasonality. For the postperiod, the longest time frame possible (30 months) was used. We initially constructed stratified time-series analysis of monthly utilization using the pre and post time frames for each site. We also conducted a sensitivity analysis with and without data from Union Square, given that it spanned the Rosie D remedy (January 2008). No significant differences in individual site analyses or in our sensitivity analysis were found, and thus data for the three sites were merged and centered on the index date.

We collected utilization data for each month of the observation period. The primary outcomes of interest were defined as follows: specialty behavioral health services were ambulatory psychiatric services delivered by behavioral health clinicians either colocated in the sites or in the department of child and adolescent psychiatry; behavioral health–related primary care visits were well-child visits or other ambulatory visits with an associated mental health–related ICD-9 code delivered by primary care providers; behavioral health–related hospitalizations were inpatient hospitalizations occurring in the CHA inpatient psychiatry unit or under the specialty of inpatient psychiatry; behavioral health–related emergency department visits were visits occurring in the CHA psychiatric emergency department or in any CHA emergency department with an associated mental health–related ICD-9 code.

Demographic variables included sex, age, race-ethnicity (white, black, Hispanic, Asian, other, or unknown), and preferred language of care (English, Portuguese, Spanish, Haitian, other, or unknown). Race and language are self-reported and entered into the medical record by CHA clerical staff. Age and race-ethnicity were dichotomized (under age 13 versus age 13 or older and white, non-Hispanic versus all other races) in order to standardize utilization rates in preparation for interrupted time-series analysis.

We used segmented regression models (27,28) to evaluate the effect of the screening used simultaneously with colocation on population service utilization rates. ITS is the strongest quasi-experimental design for evaluating the effects of natural experiments (29). One threat to the validity of the ITS design is changes in the composition of the population. To account for some change in the clinical demographic characteristics (specifically, age, sex, and race-ethnicity) over time, we standardized (30,31) the distribution of demographic characteristics in each month to that in month 1 at each site. After standardization, we adjusted for seasonality (32). The segmented models included terms for the intercept, prescreening slope, change in level, and change in slope and time, as well as autoregressive terms for up to six-month lags in rates. Because clinics took approximately six months to ramp up screening to a routine procedure, we censored the six months immediately after the behavioral health intervention, as has been done elsewhere (33). All analyses were conducted with SAS version 9.3 (34).

In this local study of a pediatric behavioral health screening and colocation program, rates of behavioral health service utilization, including both specialty and primary care, increased significantly after implementation of the program. Although behavioral health–related inpatient admissions were stable, there was also a significant increase in rates of emergency admissions with behavioral health diagnoses. To our knowledge, this is the first study to report increases in emergency department use associated with screening and colocation for a pediatric population.

As noted in the literature, behavioral health screening in primary care leads to increased rates of referral by primary care providers (8), particularly when mental health services are colocated in the same practice (14). In this study, utilization of specialty behavioral health services increased significantly, suggesting that referrals were completed and more children with behavioral health needs were getting into treatment. However, our data did not include clinical outcome measures; thus we could not determine whether more care was actually better care. It is also important to note that the prepolicy behavioral health care utilization was higher than reported elsewhere for Massachusetts statewide (15% versus 12%) (35). This may reflect access to available child psychiatry. In addition, nationally, outpatient pediatric visits with psychiatric diagnoses increased from 1999 to 2010, but we identified no increases simultaneous to CHA program implementation (36).

The greater increase in CHA’s behavioral health–related primary care visits in comparison to the more limited increase in specialty behavioral health services mimics the national trends seen over the past decade (36). This is encouraging, given pediatricians’ limited knowledge of behavioral health codes, discomfort with behavioral health treatment, and concerns about reimbursement (37,38). It suggests that screening and colocation may influence task shifting of behavioral health treatment from specialty to primary care. Unfortunately, our data do not reveal whether primary care providers were delivering behavioral health treatment or were simply more comfortable than at prepolicy with coding procedures.

Perhaps the most surprising study finding is that emergency department visit rates increased after the intervention (the rate doubled, but the number remained low). We suspect that many of these visits were avoidable (39). There are many possible reasons for this increase. For example, increased screening may identify issues that cannot be adequately triaged or managed on site, requiring emergency visits. Also, providers’ increasing awareness of emergent mental health issues may increase emergency referrals. In the recent study of Oregon Medicaid expansion, both primary care and emergency department utilization increased initially (40). Further, children and families entering treatment may experience behavioral health crises. Therapists themselves could increase utilization by telling patients to go to the emergency department if crises occur in their absence. Unfortunately, we do not know whether crises increased after the intervention. We also recognize that the upward national trend in behavioral health–related emergency visits (41–43) might have contributed to our findings, but this is unlikely to have happened simultaneously to the intervention at each of our sites. To understand the validity of these explanations, additional provider-level data and a longer time frame are required to determine whether this phenomenon is merely a short-lived consequence of improving behavioral health identification.

Despite increased emergency utilization, we did not find a concomitant increase in behavioral health–related inpatient admissions, which is surprising because most would emanate from emergency department visits. But we recognize that CHA patients are not necessarily sent to CHA facilities for inpatient behavioral health admissions. Children are sent to open beds throughout the region. Thus it is possible that inpatient utilization increased without our knowledge.

This study had a number of limitations. First, the study lacked information on behavioral health need and clinical outcomes required to assess clinical improvement. Also, because the study occurred at one delivery system and excluded children who used primary care at multiple CHA sites, it may not be representative of all CHA children or other nonurban populations. However, although this aspect of our study may limit generalizability of our findings, it would not explain the sudden slope changes in outcomes. Second, because data were collected from the CHA data warehouse, our data did not include visits that took place outside CHA. Also, the accuracy of diagnostic codes could not be validated. However, because our data showed increases in utilization, the impact of the program is unlikely to have been underestimated to the degree that outside utilization was not captured. Third, because of a lack of statistical power, we were unable to identify any change in behavioral health–related hospitalization. A claims-based study that captures inpatient utilization more completely would help to clarify these findings. Fourth, the study relied on one validated screening tool (the PSC), and it is possible that results might have differed with other instruments. Fifth, because the study examined the impact of a combined screening and colocation model, we were unable to estimate the independent impact of screening and colocation. We also lacked a concurrent comparison group. Finally, other factors, such as staffing changes, EMR adoption, and coding awareness, may have influenced our findings. However, none took place concurrently with the implementation of screening and colocation at each CHA site and thus would not explain our findings. Further, our ITS design explicitly controlled for national, secular trends affecting child and adolescent mental health delivery (23,36,43,44).