Evaluation of Telehealth in Child Behavioral Health Services Delivery During the COVID-19 Pandemic

We extracted data of behavioral assessments of all children and adolescents (ages 5–18 years) between April 1 and December 31, 2019 (2019 cohort), and between April 1 and December 31, 2020 (2020 cohort), from the Idaho Child and Adolescent Needs and Strengths (ICANS) database, which stores and manages the Child and Adolescent Needs and Strengths (CANS) assessments of the Idaho Department of Behavioral Health (22). The same 9-month period across the 2 years was selected to capture the key periods before and during the pandemic and to facilitate comparisons across equivalent periods. The analysis period was limited to 9 months because March 25, 2020, was the start of Idaho’s statewide stay-at-home order (23), and COVID-19 vaccines became more widely available in January 2021; the study time frame also closely corresponded to increases in nationwide telehealth delivery (24). We note that in order to participate in or receive certain state-funded programs in Idaho, a child or youth must complete a CANS assessment, which is then stored on the ICANS platform. Thus, it was highly unlikely that some children or youths were in state-funded care without having a CANS assessment. The study was conducted with the approval of the University of Kentucky Institutional Review Board.

In total, 4,458 new children and youths entered Idaho state–funded mental health services programs in 2019 versus 2,794 in 2020 (Table 1). Of these patients, 201 had CANS assessments from both years but during different instances of care (i.e., these patients were discharged in 2019 and reentered care in 2020). From 2019 to 2020, measures of turnover, such as admissions and discharges, decreased. Of note, not a single new child or youth who entered care during the 9-month period in 2020 was discharged, whereas 41% were discharged among those admitted during the same period in the 2019 cohort (Table 1). The total number of CANS assessments declined almost twofold in 2020 (6,588 in 2020 vs. 10,229 in 2019). However, the age distribution remained stable (i.e., about 36% of children treated pre- and midpandemic were ages 5–10 years, about 24% were ages 11–13, and about 40% were 14–18), and the decrease in the number of assessments was uniform across all three age groups. However, the average number of per-patient CANS assessments significantly increased in 2020 (2.4 in 2020 vs. 2.3 in 2019, p<0.001).

Table 1 depicts the demographic characteristics of unmatched cohorts. (Table S1 in an online supplement to this article displays pre- and pandemic cohorts after propensity score matching.) Compared with 2019, more girls (48% vs. 50%) and fewer White children (63% vs. 60%, p<0.001) entered Idaho’s behavioral health program in 2020. Children and adolescents exhibited significantly (p<0.05) more complex needs and strengths profiles in 2019 compared with those in 2020 (strengths TAI=4.4 vs. 4.0, life-functioning TAI 2.3 vs. 2.0, and risky behaviors TAI 1.0 vs. 0.9). Finally, in 2020, fewer caregivers reported transportation need as “resolved” (1% in 2020 vs. 2% in 2019).

Table 2 shows the results of the primary matched linear mixed-model analyses. Among the matched children, 129 underwent assessments in both 2019 and 2020 (under different instances of care). The estimated coefficient, β, shows the percentage of areas addressed (i.e., needs resolved or strengths acquired) within each domain associated with every additional 90 days in the system (i.e., between two consecutive CANS assessments) among children whose caregivers never had issues with transportation to or from care. The estimated change, Δ, shows deviations from these percentages of addressed areas among children whose caregivers resolved their transportation issues, Δ_R, or did not resolve transportation issues, Δ_NR. (Table S2 in the online supplement shows the results of the same analyses but on the prepandemic sample without propensity score matching.)

Analyzing changes in rates of strengths building and needs resolution between 2019 and 2020, we observed no statistically significant changes for most children and youths (i.e., those whose caregivers never had issues with transportation access) (Table 2). Results without propensity score matching (Table S2 in the online supplement) were consistent with this finding. Unlike for the 2019 cohort, we found no statistically significant associations between caregivers’ resolution of transportation needs and children’s improvement across all four CANS domains in the 2020 cohort. The magnitudes of change in strengths and behavior-emotional domains in the 2020 cohort were nearly half those observed in the 2019 cohort for Δ_NR.

We note several findings from our study of data in a large statewide public mental and behavioral health system for children and youths. First, new cases significantly declined during the early months of the pandemic. However, children or youths and families who did initiate care stayed in care. Second, as care shifted to telehealth models, we observed clinical outcomes of children and adolescents that were similar to those observed prepandemic. Third, associations between unmet transportation needs and worse clinical outcomes observed before the pandemic were not observed during the pandemic, when most services were delivered via telehealth.

These findings appear to be consistent with results from some other reports of benefits of telehealth among youths (31–34). A recent review of different Web-based interventions to address mental health problems among young adults during the pandemic found that all of them had effectively reduced rates of depression, anxiety, and stress symptoms (34). A nationally representative survey reported that digital interventions were mitigating the negative COVID-19–related psychosocial impact among individuals ages 16–25 years (32). Taken together, these findings suggest that telehealth services may be suitable for lessening behavioral health issues among youths, although questions remain about the wider applicability of telehealth across the whole developmental spectrum.

Importantly, the findings of our study underscore the impact of the COVID-19 pandemic on the relationship between the resolution of caregiver transportation needs and children’s strengths building or needs reduction across multiple functioning domains. We observed a significant association between resolution of caregiver transportation needs and child-level clinical outcomes prepandemic (April–December 2019), such that the resolution of caregiver transportation needs was associated with greater need reduction in the life-functioning domain and strengths-building domain. This represented approximately 5% additional needs resolved and 5% additional strengths developed with the resolution of the caregiver’s transportation needs (see Table 2). This relationship was not statistically significant, however, when the same analyses were conducted with data obtained during the pandemic (April–December 2020). These results suggest that the expansion of telehealth service delivery may have mitigated the impact of caregiver transportation needs on clinical outcomes for children with complex needs and strengths who are served in public mental and behavioral health systems.

The results of this study should be contextualized within its limitations. No precise service delivery data were available that described the exact proportion of services delivered via telehealth versus those delivered in-person during the pandemic. Data from the Idaho Office of the Governor indicate that telehealth sessions increased 40-fold from March to May 2020 (21). Moreover, because we observed similar trends in outcomes for only two of the four domains examined, strengths building and needs resolution, we wonder whether besides the shift to telehealth other changes in service delivery between 2019 and 2020 might have influenced outcomes in these two domains; we also do not know why telehealth outcomes were comparable in these, but not the other, domains. Additionally, because CANS assessments do not distinguish between unmet transportation needs for mental health and those for school, activities, and other aspects of life, it is possible that the 2020 cohort had a reduced negative impact of transportation needs on functional outcomes because the nature of those transportation needs changed because of the pandemic. However, one strength of this study included the size of the database used and the utilization of patient-level data to characterize the impact of the pandemic on mental and behavioral health outcomes of children and youths.

In terms of the pandemic’s impact on behavioral health service activities, substantially fewer children and youths were served in Idaho in 2020 than in 2019. The existing data cannot provide a definite answer as to where these “missing” children and youths went midpandemic. Nonetheless, we note that about 20% of children and youths typically enter care systems through school staff referrals, 15% are referred from social service agencies, and another 15% are directed to the system of care through the courts and correctional institutions (35, 36). We speculate that the decrease in the number of young patients seen during the early pandemic may have been due to shutdowns of several avenues through which children and youths typically enter the system of care. Furthermore, our matched prepandemic and pandemic samples were largely comparable in terms of demographic variables, but the unmatched number of young patients served and number of CANS assessments in April–December 2020 were nearly half those of the unmatched patients during the same period in 2019. This reduction in care delivery is consistent with some documented trends of reduced health care delivery in the United States during the pandemic, because telehealth utilization requires the need for connective technologies (e.g., high-speed Internet, networked devices), which may pose a barrier to both presentation for services and telehealth utilization independent of transportation need or demographic characteristics (37). The intensity and quality of care differences were unknown; however, our findings indicate that children served during the pandemic received more frequent assessments.

The difference in clinical presentation between the two unmatched cohorts, such that the 2019 cohort presented with more actionable and more complex needs, perhaps demands further investigation. Children with the most acute and complex mental and behavioral needs may not have presented to the Idaho Department of Behavioral Health for telehealth services during the pandemic. If that was the case, it would be important to understand the potential barriers to care specific to children and youths with the highest acuity needs.

Our findings suggest that for children with complex needs, mental health services provided primarily via telehealth were comparable to traditionally delivered services in addressing two patient‐level functional outcomes and that telehealth utilization reduced the negative impact of transportation needs on care outcomes. Our work is primarily a descriptive study of clinical and functional changes associated with a major policy shift in service delivery. As clinicians, researchers, and policy makers are advocating for permanent expansion of telehealth services, one such systems-level change suggested by our findings may include continued provision of telehealth for clients with transportation needs as a bridge to support service delivery, while administrators and providers work toward the permanent resolution of transportation issues for these clients. However, for clients without transportation needs, telehealth may be selectively effective (by type of need, level of acuity, or other) or even create new obstacles to accessibility (e.g., technological barriers).