A Help or a Hindrance: The Role of Electronic Health Records in Implementing New Practices in Community Mental Health Clinics

CMHCs included in this study were participants in a larger randomized controlled trial (RCT) examining the implementation and effectiveness of PCCP, a manualized intervention designed to incorporate recovery-oriented principles into the service planning process (12). Consistent with this approach, PCCP provides a structured mechanism for identifying an individual’s strengths, eliciting their unique visions of recovery, and translating them into actionable goals that guide treatment. As part of PCCP, the care planning process is highly individualized to reflect clients’ personal life goals and relies on a collaborative relationship between the client and provider. The use of PCCP of has been associated with several positive outcomes in community mental health settings, including greater continuity of care and adherence to psychiatric medication (13).

In this study, seven CMHC sites across two Northeastern states were assigned to the experimental condition. These clinics provided a range of services, including outpatient therapy, intensive case management, medication management, and residential programming. In the experimental condition, supervisors and direct care providers from participating programs received a 2-day in-person training on PCCP, followed by 12 months of technical assistance led by external expert consultants. Two technical assistance meetings occurred each month; one call supported program supervisors in the implementation of PCCP, and the second focused on enhancing direct care providers’ PCCP-related knowledge and skills. All study activities were approved by the institutional review board at New York University.

Participants in the experimental condition completed a baseline survey (N=273), including measures of organizational capacity for implementation. Fidelity to PCCP was measured through a chart review of service plans at baseline, 12 months, and 18 months. A total of 378 unique patient charts were reviewed (N=126 from each time point). To achieve this sample, 60 charts were abstracted from each CMHC, except for one small organization, where only 18 charts were reviewed because of the clinic’s size. The subsample of service plans was selected randomly from a larger pool of plans completed by trained supervisors or direct care workers at each time point. Abstracted data included clients’ demographic and diagnostic information and PCCP fidelity.

Fidelity to PCCP was measured by the PCCP Assessment Measure (PCCP-AM), a tool developed by experts by using the PCCP manual (12). The PCCP-AM includes 13 items that are given a score of either 0 (not proficient) or 1 (proficient). Scores ranged from 1 to 13, with higher values reflecting higher fidelity to the intervention. To ensure reliable fidelity ratings within the larger RCT, 12 service plans at each of the 14 participating CMHC sites (N=798) were co-coded by two research team members trained in PCCP, yielding an interrater reliability of 80% and a Cronbach’s alpha of 0.70.

CMHCs were stratified by whether they used an EHR system. In the present study, EHRs were considered available if a system was operational at baseline and was used for charting and service planning purposes. With this criterion, four clinics (N=240 service plans) were included in the “EHR” category, and three clinics (N=138 service plans) were included in the “no EHR” category.

Openness to change was included in this measure as a potential confounder affecting the relationship between EHR use and fidelity. This construct was measured with the five-item change subscale of the Organizational Readiness for Change measure (14). Items relate to an organization’s acceptance of change and capacity to implement change. Sample items include, “You are encouraged here to try new and different techniques” and “It is easy to change procedures here to meet new conditions.” Possible scores range from 10 to 50, with higher scores reflecting higher endorsement of items. Individual ratings were aggregated to the program level to establish an organizational estimate of openness to change for each site (14).

Three separate analytic models estimated the association between EHR use and PCCP fidelity at baseline and at 12 and 18 months. Chi-square analyses demonstrated no significant differences in sample demographic characteristics across time points. Organizational openness to change was examined as a covariate in the fully adjusted model. Binary measures of client demographics—including whether the client was a member of a racial-ethnic minority group, female, a young adult (≤35 years old), or diagnosed as having a serious mental illness (schizophrenia, bipolar disorder, or other psychosis)—were present in the final model. To account for the nesting of plans within sites, a mixed-effects linear regression model with a random intercept at the site level was used to analyze the data. Stata, version 16, was used to complete analyses.

Of the client population reflected in the complete pool of sampled service plans (N=378), 35% (N=131) identified as belonging to a nonwhite racial-ethnic minority group or declined to specify race-ethnicity. Within this group, 8% (N=10) were Latinx, 70% (N=92) were black, 15% (N=20) reported other, and 7% (N=9) declined to specify. Additionally, 17% of the client population (N=63) were young adults (≤35 years old), and 55% (N=207) were diagnosed as having a severe mental illness. Organizational estimates suggested moderate openness to change (mean±SD=34.8±2.04) among CMHCs. PCCP fidelity scores ranged from 1 to 12 (baseline, 6.3±2.3; 12 months, 7.3±2.5; 18 months, 7.2±2.5). Regression analyses indicate that the presence of an EHR system was not significantly associated with PCCP fidelity among service plans at baseline (Table 1). EHR use predicted higher PCCP fidelity at both 12 months (β=3.24, p<0.001) and 18 months (β=3.63, p<0.001). Postestimation commands from the fully adjusted models were used to plot predicted PCCP fidelity means among sites with and without EHRs at each time point (Figure 1). Organizational openness to change was negatively associated with fidelity at 12 months (β=−0.88, p<0.001) and 18 months (β=−1.02, p<0.001). Client characteristics did not predict fidelity.

Findings demonstrate no significant baseline differences in PCCP fidelity between service plans completed with an EHR and those completed without one. Postimplementation, the presence of an EHR system was associated with higher PCCP fidelity at 12 and 18 months. Organizational openness to change had a slight negative association with fidelity at 12 and 18 months, although not at baseline.

Nonsignificant baseline differences in PCCP fidelity between plans completed with and without an EHR suggests that these systems did not provide an inherent advantage to the delivery of person-centered practices like PCCP. This finding is consistent with previous work suggesting that typical EHR design may not be aligned with individualized, client-driven approaches to treatment (5). Although the presence of EHRs alone may not be sufficient to support PCCP, these findings suggest that, when used in tandem with deliberate, practice-wide efforts to enact change, these systems can support person-centered practice innovation.

Beyond the effect of EHRs in PCCP, these findings suggest that EHRs may also be an organizational resource that can meaningfully affect practice innovation in CMHCs. Consequently, more deliberate examination of the role of EHRs in implementing a range of emergent evidence-based practices is warranted. The negative association between openness to change and PCCP fidelity was unexpected. It is possible that openness to change may overburden clinicians by encouraging multiple change activities, making them less able to achieve fidelity in one practice (15). Alternatively, change openness may be more reflective of providers’ uptake of a practice (i.e., their decision to implement or not) rather than their fidelity to the model (i.e., adherence to intervention protocol or guidelines).

Several limitations to the present study can inform future work. Primarily, the relatively small number of unique CMHCs in the experimental condition (N=7) limited between-site variation across a number of organizational characteristics relevant to implementation of evidence-based practices, including program size and function and indicators of organizational culture or climate. Consequently, future work in this area would benefit from a more robust examination of these agency characteristics in relation to EHR use and adoption of evidence-based practices. In addition, EHR availability was measured dichotomously; a more nuanced understanding is needed of how diversity in system design and implementation affects fidelity. Although this study indicates that EHRs play a role in implementation, more research is needed to elaborate the nature and extent of this relationship.

This study found that the presence of EHRs was associated with greater fidelity to PCCP over time. These results add to a limited body of knowledge about the effect of EHRs on practice innovation in community mental health. Future work in implementation research may benefit from more explicit examination of EHRs as a unique organizational resource affecting efforts to support practice change.