Rapid Telepsychiatry Implementation During COVID-19: Increased Attendance at the Largest Health System in the United States
Abstract
Objective:
This study aimed to examine differences in completion rates between telepsychiatry and in-person visits during the COVID-19 pandemic and a prior reference period.
Methods:
The authors used electronic medical record data along with chi-squared or t tests to compare patients’ demographic characteristics. Generalized estimating equations for estimating the odds of primary and secondary outcomes were used, controlling for demographic characteristics.
Results:
During COVID-19, the odds of completing a telepsychiatry visit (N=26,715) were 6.68 times the odds of completing an in-person visit (N=11,094). The odds of completing a telepsychiatry visit during COVID-19 were 3.00 times the odds of completing an in-person visit during the pre–COVID-19 reference period (N=40,318).
Conclusions:
In this cross-sectional study, outpatient adult mental health clinic telepsychiatry appointments, largely by telephone, were strongly associated with a higher rate of visit completion compared with in-person visits during and prior to the COVID-19 pandemic. Regulators should consider permanently enabling reimbursement for telephone-only telepsychiatry visits.
HIGHLIGHTS
•
NYC Health + Hospitals implemented the novel use of telepsychiatry during the COVID-19 pandemic.
•
During the pandemic, the odds of completing a telepsychiatry visit were 6.68 times the odds of completing an in-person visit.
•
The odds of completing a telepsychiatry visit during the COVID-19 period were 3.00 times the odds of completing an in-person visit during the pre–COVID-19 reference period.
Research indicates that under normal circumstances, 26% of Americans have a mental illness, but only 58.9% of them attend treatment (1). The unmet need is particularly pronounced for low-income individuals with Medicaid (2). In New York City (NYC), approximately 40% of residents with a serious mental illness did not receive mental health treatment in the past year (3). Left untreated, mental illness is associated with a shorter lifespan, increased number and duration of hospital stays, and increased cost to the individual and community (4). Removing barriers to care is essential to ensuring access, particularly during a period of crisis (5).
Telepsychiatry services are defined as the delivery of mental health services via video-based conferencing (6). The efficacy of telepsychiatry extends to reliability of clinical assessment, treatment outcomes, and cost across many populations and settings, and the services are comparable to in-person care (7, 8). Some studies have demonstrated high levels of patient satisfaction, improved medication adherence, and symptom reduction (9, 10). Although one study found that the number of U.S. facilities offering telepsychiatry has been increasing, these services are most commonly utilized in medically underserved and rural areas (11).
It is estimated that NYC Health + Hospitals (H+H) sees more than one million patients annually, for whom 50% of services are for behavioral health. Outpatient adult mental health clinics at H+H provide a variety of services, including consultation and assessment as well as continuing care psychotherapy, group therapy, and medication management. Traditionally, telepsychiatry was reimbursed by insurance only when rendered via video teleconferencing (12). Because of the upfront cost of technology for telepsychiatry, H+H historically has not offered this service. However, NYC’s COVID-19 crisis (March–April 2020) forced a systemwide and unanticipated shift in services for H+H patients. State and federal regulatory entities recognized the need to reduce barriers associated with delivering services remotely and began allowing clinics to provide and bill for telephone-only and video sessions (13). Dramatically and rapidly expanding telepsychiatry enabled H+H to keep medically high-risk or quarantining patients safely in their homes and to help them avoid potential psychiatric crises. Although some patients needed to attend in-person visits for clinical purposes, such as medication injection, those with access to a phone, video equipment, or both were able to attend telepsychiatry visits.
This study aimed to determine the differences in visit completion rates for visits conducted via telepsychiatry, primarily by telephone, when compared with in-person visits for H+H patients during the COVID-19 pandemic and during a pre–COVID-19 period. A previous study found that telepsychiatry appointments were more likely to be kept and less likely to be canceled or no-shows than were in-person visits in rural clinical settings (14). One recent study reported patient satisfaction to be high but did not examine visit completion rates (15). Other recent telepsychiatry studies during COVID-19 have found increased attendance for telepsychiatry visits; however, the sample size was smaller than in the current study, and the authors did not compare the sample with an in-person control group or control for demographic characteristics (8, 12). This study contributes to the literature on telepsychiatry and is distinguished by a significantly larger sample size than in other published studies with a similar setting and population. Stable or improved visit completion rates would support continuation of this service, including telephone-only visits, during and beyond the COVID-19 crisis, with potential policy implications.
Methods
Telepsychiatry services were rapidly implemented in H+H’s 11 hospital-based and four free-standing clinics during the COVID-19 pandemic. For the purposes of this report, telepsychiatry is defined as telephone-only or audiovisual services. Data for these services were collected as part of standard quality improvement (QI) processes at H+H. The H+H Research Office confirmed that the protocol for this study met QI criteria and did not need review by an institutional review board. Descriptive analyses and reports to leadership are regularly completed for QI purposes to guide clinic processes and modifications.
Data from the electronic medical record (EMR) provided cross-sectional patient-level demographic and clinical data, including age, sex, race, and insurance enrollment. Data were collected on May 19, 2020. The index period for clinic visits was between April 8 and May 8, 2020, representing a period beginning shortly after the onset of the high prevalence of COVID-19 in NYC and H+H’s resultant telepsychiatry implementation. Reliable data from the EMR were not available during April and May 2019, so comparative data were drawn from a pre–COVID-19 period between October 8 and November 8, 2019. This calendar period presents normative utilization data similar to those in the index period in any given year. Data included the type of services rendered, whether the appointment was an in-person visit, and whether the patient did or did not keep the appointment. The analysis excluded two of the four free-standing clinics and one hospital-based clinic because they had not yet transitioned from the old EMR during the comparative period. One additional hospital was excluded because the clinic did not begin tracking telepsychiatry appointments until later in the study period.
Data were grouped into three main categories: telepsychiatry during COVID-19, in-person visits during COVID-19, and in-person visits pre–COVID-19. To compare demographic characteristics among groups, we conducted chi-squared or t tests as appropriate. Because the data contained repeated measures (i.e., patients may have had multiple visits scheduled during the study period), generalized estimating equations were used to estimate the odds of a group attending a scheduled visit and to control for patient demographic characteristics. Visits with missing or unknown demographic characteristics were removed. All analyses were conducted in SAS Enterprise Guide, version 7.15. We considered p<0.001 to be statistically significant.
Results
During the COVID-19 period, 37,809 visits were scheduled, 6% fewer than during the pre–COVID-19 period (N=40,318). The preponderance of visits scheduled for a telepsychiatry visit during COVID-19 were for female patients (65.3%), those who identified their race as “other” (53.5%), and those enrolled in Medicaid (39.3%) or Medicare (31.9%) (Table 1). There were significant differences between the telepsychiatry and in-person patient groups during the COVID-19 period in terms of age (t=12.41, df=21,063, p<0.001), gender (N=37,809, χ2=105.15, df=1, p<0.001), race (N=37,809, χ2=196.53, df=5, p<0.001), and insurance enrollment (N=37,809, χ2 =91.33, df=4, p<0.001). Significant differences also existed between the telepsychiatry during COVID-19 and in-person pre–COVID-19 groups when comparing gender (N=67,033, χ2=39.31, df=1, p<0.001), race (N=67,033, χ2=6,482.66, df=5, p<0.001), and insurance enrollment (N=67,033, χ2=65.35, df=4, p<0.001). However, there was no significant difference between ages when comparing these groups.
| COVID-19 period | Pre–COVID-19 period | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Telepsychiatry (N=26,715) | Face to face (N=11,094) | All visits (N=37,809) | All visits (N=40,318) | |||||||
| Variable | N | % | N | % | N | % | p a | N | % | p b |
| Primary outcome | ||||||||||
| Completed visits | 19,831 | 74.2 | 3,924 | 35.4 | 23,755 | 62.8 | 23,640 | 58.6 | ||
| Secondary outcome | ||||||||||
| Visit cancelations | 3,425 | 12.8 | 4,082 | 36.8 | 7,507 | 19.9 | 8,139 | 20.2 | ||
| Visit no-shows | 3,459 | 13.0 | 3,088 | 27.8 | 6,547 | 17.3 | 8,539 | 21.2 | ||
| Demographic characteristics | ||||||||||
| Age (M±SD) | 49.5±16.0 | 47.3±15.8 | 48.8±16.0 | <.001 | 49.28±15.9 | ns | ||||
| Sex | <.001 | <.001 | ||||||||
| Female | 17,451 | 65.3 | 6,629 | 59.8 | 24,080 | 63.7 | 25,379 | 63.0 | ||
| Male | 9,264 | 34.7 | 4,465 | 40.3 | 13,729 | 36.3 | 14,939 | 37.0 | ||
| Race | <.001 | <.001 | ||||||||
| Other | 14,283 | 53.5 | 5,542 | 50.0 | 19,825 | 52.4 | 13,464 | 33.4 | ||
| Black | 5,801 | 21.7 | 3,117 | 28.1 | 8,918 | 23.6 | 6,735 | 16.7 | ||
| White | 4,075 | 15.3 | 1,608 | 14.5 | 5,683 | 15.0 | 18,052 | 44.8 | ||
| Asian | 2,409 | 9.0 | 773 | 7.0 | 3,182 | 8.4 | 1,950 | 4.8 | ||
| American Indian/Alaskan Native/Native Hawaiian/Pacific Islander | 116 | .4 | 42 | .4 | 158 | .4 | 84 | .2 | ||
| Multiracial | 31 | .1 | 12 | .1 | 43 | .1 | 33 | .1 | ||
| Insurance enrollment | <.001 | <.001 | ||||||||
| Medicaid | 10,493 | 39.3 | 4,675 | 42.1 | 15,168 | 40.1 | 16,478 | 40.9 | ||
| Medicare | 8,525 | 31.9 | 3,011 | 27.1 | 11,536 | 30.5 | 12,709 | 31.5 | ||
| Commercial | 4,979 | 18.6 | 2,291 | 20.7 | 7,270 | 19.2 | 7,680 | 19.1 | ||
| Uninsured/self-pay | 2,641 | 9.9 | 1,082 | 9.8 | 3,723 | 9.9 | 3,309 | 8.2 | ||
| Other | 77 | .3 | 35 | .3 | 112 | .3 | 142 | .4 | ||
a
The p value of chi-squared test (or t test, where applicable) comparing demographic characteristics of patients scheduling telepsychiatry and face-to-face visit during the COVID-19 period.
b
The p value of chi-squared test (or t test, where applicable) comparing demographic characteristics of patients scheduling telepsychiatry visits during the COVID-19 period and those with any visit during the pre–COVID-19 period.
During the COVID-19 period, 70.7% (N=26,715) of visits were scheduled to be conducted via telepsychiatry; of those visits, 74.2% (N=19,831) were completed. The odds of completing a telepsychiatry visit were 6.68 times the odds of completing an in-person visit after adjusting for age, sex, race, and insurance enrollment (odds ratio [OR]=6.68, 95% confidence interval [CI]=6.18–7.21). As a secondary measure, when we examined visits that were not completed (i.e., a patient canceled or was a no-show to his or her appointment), the odds of a no-show telepsychiatry visit were 1.54 times the odds of a no-show in-person visit (OR=1.54, 95% CI=1.40–1.70). The odds that a telepsychiatry visit was canceled were 0.65 times the odds that an in-person visit was canceled (OR=0.65, 95% CI=0.59–0.72). Finally, we found no association between completion of a visit and the virtual platform used (telephone versus audiovisual).
The odds of completing a telepsychiatry visit during the COVID-19 period were 3.00 times the odds of completing an in-person visit during the pre–COVID-19 reference period, after adjusting for age, sex, race, and insurance enrollment (OR=3.00, 95% CI=2.84–3.16). As a secondary measure, when we examined the visits that were not completed, there was no significant difference between the odds of a no-show visit and a canceled visit when scheduled as telepsychiatry or in person.
Discussion
During the COVID-19 pandemic, implementation of telepsychiatry has been essential for continued service delivery to clinic patients. This study supports previous findings that telepsychiatry improves attendance to clinic appointments, likely because of the ease of access to these visits, particularly when patients and providers are quarantining at home. It should be noted that telepsychiatry in itself is no small feat. Both patients and providers require some level of technical support in the earliest stages of implementation, and the acquisition of a video device is not always feasible for patients.
Anecdotally, some clinic staff were concerned that no-show rates would be higher via telepsychiatry and that cancelation rates would be higher still. During COVID-19, the findings suggest that patients who did not complete their appointments canceled telepsychiatry less often than they did in-person visits and instead opted to not show up for telepsychiatry visits. However, there was no difference when comparing with the reference period, suggesting that patients not completing their appointments would likely not show up or cancel in-person and telepsychiatry visits with the same frequency in nonacute pandemic periods.
This study had several strengths. An exhaustive literature review of telepsychiatry found the current sample to be the largest and most diverse study sample, particularly in a municipal health care system setting. The relatively large proportion of Medicare and Medicaid patients in this sample strengthens the case for policy decisions regarding reimbursement of audio-only telepsychiatry services. The study also shows that it is possible to dramatically expand telepsychiatry on a comprehensive scale, even in a crisis, tangibly affecting service delivery.
The results of this study should be considered in the context of its limitations. First, the data include patients only from select H+H facilities and so are not generalizable and are dependent on accurate documentation by staff. Second, although the results suggest that telepsychiatry may improve accessibility to treatment, outcomes of treatment beyond showing up to a scheduled appointment were outside the scope of this report. Other patient variables, such as income, housing status, and diagnosis, were not feasible to report but should be considered important variables. Additionally, future studies should examine a longer period spanning beyond the peak COVID-19 months and the early stages of implementation, because telepsychiatry utilization and regulatory policies may change. Finally, because this was a cross-sectional study, further analysis to examine the impact of in-person visits prior to telepsychiatry implementation was not completed. Patient-provider rapport may affect patients’ comfort with providers, making them more likely than new patients to initiate telepsychiatry treatment. Future studies should explore this relationship.
Conclusions
The COVID-19 crisis presented a very challenging time for H+H patients, many with complex behavioral health issues, and the novel use of telepsychiatry was critical to maintaining treatment while clinics were limiting in-person exposure. We recommend that regulatory bodies consider this study when reexamining telepsychiatry policies in a post–COVID-19 era and encourage ongoing research to determine the most effective ways to support patients in need of mental health treatment. Given the findings and relative ease of access to a telephone rather than a video device for service delivery, it is important for regulatory bodies to consider adopting the modified regulations and allow for reimbursement of audio-only telepsychiatry service as deemed necessary by providers.
Acknowledgments
The authors acknowledge Juan Garcia and Emily Carter, M.S.W., for their assistance and all of the NYC Health + Hospitals employees on the front lines during the COVID-19 pandemic.
References
1.
Kohn R, Ali AA, Puac-Polanco V, et al : Mental health in the Americas: an overview of the treatment gap. Rev Panam Salud Publica 2018 ; 42 : e165
2.
Barnett ML, Huskamp HA : Telemedicine for mental health in the United States: making progress, still a long way to go. Psychiatr Serv 2020 ; 71 : 197 – 198
3.
Norman C, Goldmann E, Staley B : Serious Mental Illness Among New York City Adults. New York, New York City Department of Health and Mental Hygiene, 2015. https://www1.nyc.gov/assets/doh/downloads/pdf/survey/survey-2015serious-mental-illness.pdf
4.
Gomez J : Liaison Psychiatry: Mental Health Problems in the General Hospital. New York, Routledge, 2018
5.
Goldmann E, Galea S : Mental health consequences of disasters. Annu Rev Public Health 2014 ; 35 : 169 – 183
6.
Hubley S, Lynch SB, Schneck C, et al : Review of key telepsychiatry outcomes. World J Psychiatry 2016 ; 6 : 269 – 282
7.
Hilty DM, Ferrer DC, Parish MB, et al : The effectiveness of telemental health: a 2013 review. Telemed J E Health 2013 ; 19 : 444 – 454
8.
Velázquez PP, Gupta G, Gupte G, et al : Rapid implementation of telepsychiatry in a safety-net health system during COVID-19 using Lean. NEJM Catalyst (Epub ahead of print, July 17, 2020 )
9.
Flaum MA : When will telepsychiatry reach its “tipping point”? Psychiatr Serv 2017 ; 68 : 1205
10.
García-Lizana F, Muñoz-Mayorga I : What about telepsychiatry? A systematic review. Prim Care Companion J Clin Psychiatry 2010 ; 12 : PCC.09m00831
11.
Spivak S, Spivak A, Cullen B, et al : Telepsychiatry use in US mental health facilities, 2010–2017. Psychiatr Serv 2020 ; 71 : 121 – 127
12.
Chen JA, Chung W-J, Young SK, et al : COVID-19 and telepsychiatry: early outpatient experiences and implications for the future. Gen Hosp Psychiatry 2020 ; 66 : 89 – 95
13.
Cuomo AM: Executive Order: Continuing Temporary Suspension and Modification of Laws Relating to the Disaster Emergency. Executive order no. 202.1. Albany, NY, Office of Governor of New York State, 2020. https://www.governor.ny.gov/sites/governor.ny.gov/files/atoms/files/EO_202_1.pdf
14.
Leigh H, Cruz H, Mallios R : Telepsychiatry appointments in a continuing care setting: kept, cancelled and no-shows. J Telemed Telecare 2009 ; 15 : 286 – 289
15.
Yellowlees P, Nakagawa K, Pakyurek M, et al : Rapid conversion of an outpatient psychiatric clinic to a 100% virtual telepsychiatry clinic in response to COVID-19. Psychiatr Serv 2020 ; 71 : 749 – 752
Information & Authors
Information
Published In
History
Received: 29 July 2020
Revision received: 27 August 2020
Accepted: 10 September 2020
Published online: 18 March 2021
Published in print: June 2021
Keywords
Authors
Competing Interests
The authors report no financial relationships with commercial interests.
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBLogin options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).