Evaluating the Implementation of Home-Based Videoconferencing for Providing Mental Health Services
Abstract
Objective:
The Veterans Health Administration (VHA) has recently implemented video-to-home (V2H) telehealth as part of a strategy to improve access to mental health treatment. Implementation research of this modality is needed, given that V2H telehealth transforms the traditional face-to-face delivery of mental health services. To address this need, V2H implementation was evaluated by examining barriers and facilitators that were associated with level of staff V2H experience and factors that differentiated facilities with various levels of V2H performance.
Methods:
Semistructured interviews with VHA personnel (N=33) from three facilities were conducted. The facilities were selected by overall number of mental health V2H visits during fiscal year (FY) 2015 as well as by growth in number of visits from FY 2014 through FY 2015. Factors influencing implementation were identified through qualitative analyses that contrasted responses by groups of participants with three different levels of V2H experience (no experience, limited experience, most experience) as well as three facilities that differed in V2H productivity (high visit count, high visit growth, and low visit count and low visit growth).
Results:
Providers seemed to encounter different barriers and facilitators depending on their level of experience with V2H. Site-level analyses illustrated the importance of logistical support, especially for providers who are newly adopting the technology. Other factors that differentiated the facilities were also identified and described.
Conclusions:
Key factors related to implementation of V2H telehealth pertained to provider buy-in and logistical support. Facility-level strategies that address these factors may enhance provider progression from nonuse to sustained use.
Insufficient access to care is one of the most pressing problems in meeting the mental health needs of the U.S. population (1). The Veterans Health Administration (VHA) has addressed the problem, in part, with telehealth that uses videoconferencing to allow a provider at one clinic to provide services to a patient at another clinic that is closer to his or her home (2,3). More recently, the VHA has implemented video-to-home (V2H) telehealth, which allows patients to connect with a provider from their own homes by using a computer Webcam. This technology holds promise for significantly improving treatment access because it completely eliminates geographical and travel barriers.
The VHA implementation of V2H has occurred during a time when evidence supporting the delivery of mental health care through home-based telehealth is expanding rapidly. Earlier evidence indicates that the use of telehealth technology to link a patient in one clinic to a mental health provider in a separate clinic is feasible and acceptable and provides clinical benefit (4–8). More recent studies have examined home-based technologies, such as V2H, and show similarly promising outcomes (9–11).
This new modality will radically transform the delivery of mental health services, which have long emphasized face-to-face visits, and some reports indicate that clinicians are cautious about adopting this new technology (3,12). Research is therefore needed on issues surrounding the implementation of home-based telehealth. Such research can inform ongoing VHA implementation efforts, as well as those of other health care systems. Responding to this need, we conducted a formative evaluation of V2H implementation for mental health services (13). The first aim focused on identifying barriers and facilitators encountered by staff with various levels of experience with V2H technology. The second aim focused on describing barriers and facilitators in implementing V2H services encountered by three facilities with varying numbers of V2H visits and growth in visits.
Methods
Data collection used a concurrent, embedded mixed-methods design (14), purposively sampling three facilities based on facility-level quantitative data. We conducted semistructured interviews with staff at a facility with a high number of V2H visits in fiscal year (FY) 2015 (facility 1), a facility with a large increase in V2H visits from FY 2014 to FY 2015 (facility 2), and a facility that was low on both of these indicators (facility 3). These facilities served urban, suburban, and some rural (<3% of patients) areas in the Northeast. Data were analyzed by using the Consolidated Framework for Implementation Research (CFIR) (15) with previously published methods (16). The CFIR is a consolidation of various implementation science theories and is organized into five major domains (intervention characteristics, outer setting, inner setting, individuals, and implementation process), with each domain comprising several constructs.
Participants
Participants were VHA mental health providers from a wide array of programs who were familiar with or had been assigned to or encouraged by local leadership to deliver V2H services as well as facility staff who assisted with implementing V2H telehealth. Providers who had not yet provided V2H services were not excluded, allowing a greater range of V2H experience to be assessed. A snowball sampling strategy was used, beginning with candidates suggested by regional-level staff, followed by candidates suggested by local telehealth staff, and finally by candidates suggested by the participants themselves. Invitations were e-mailed and included key informed-consent elements (study purpose, procedures, risks, benefits, and rights). All interviews were preceded by verbal consent and permission for audio recording. This study protocol was given exempt status by the local institutional review board. Figure 1 summarizes the timeline of V2H implementation within this VHA region as well as the study timeline. Study interviews were conducted between September 2014 and April 2015.
FIGURE 1. Timeline for implementation of the video-to-home (V2H) telehealth program by the Veterans Health Administration and for the formative evaluation of the programa
aVISN, Veterans Integrated Service Network; FY, fiscal year
Procedures
The CFIR contains 39 constructs across five domains that may be associated with implementation outcomes. To focus on the constructs most relevant to V2H implementation, a questionnaire listing all 39 constructs was completed by implementation staff (N=6) prior to initiating the study. Nineteen constructs spanning all five CFIR domains were rated as most relevant to V2H implementation, and they were included in the semistructured interview. Interviews ranged from 30 to 60 minutes. [The interview guide is available as an online supplement to this article.]
Qualitative Data Coding
Interview recordings were transcribed and imported into a qualitative analysis software program (Atlas.ti version 6.2). The analytic team consisted of the lead investigator (AI) and two individuals (AK and LSH), who identified the CFIR constructs (coding) and evaluated the degree to which the construct affected V2H implementation (rating). The lead investigator had previous experience in qualitative research and trained the other members of the analytic team. All team members had previous experience in qualitative research. The team members independently read each transcript and deductively coded CFIR constructs. A coding guide was developed with definitions for each CFIR construct to guide subsequent coding. When definitions were refined, previously coded transcripts were reviewed again to ensure that consistency remained. After all CFIR coding was completed, a valence rating method was used to quantitatively capture the degree to which CFIR constructs affected V2H implementation by three groups of individuals with various levels of V2H experience and each of the three facilities. Valence ratings indicated whether the implementation impact was strongly negative (−2), negative (−1), neutral or mixed (0), positive (+1), strongly positive (+2), or not ratable because of insufficient information (M). This method was previously published with more extensive description (16). [A more detailed summary of the valence rating method and of aggregate valence ratings for constructs that were rated differently by experience level and facility is available as an online supplement to this article.]
All codes and valence ratings were assigned by consensus among at least two analysts. Initial transcripts were coded and rated by all team members to establish a shared understanding of the constructs. Once that had been established, two analysts continued with the transcripts independently. Agreement was reviewed and consensus was used in instances of disagreement, with a third rater contributing to the consensus process if necessary. In addition, external collaborators (CR and LD) with expertise in this study’s particular methodology and CFIR constructs were occasionally consulted.
Results
A total of 56 individuals were invited to participate in the study, and 33 agreed to participate. Table 1 summarizes the characteristics of the 33 participants. Some CFIR constructs were further subdivided into subconstructs that were specific to V2H telehealth. For example, we used equipment (for example, Webcams), technical support, dedicated time, and implementation staff as subcodes for the CFIR construct for available resources. Also, functionality problems (technical problems) emerged inductively from our analysis.
| Site 1 (N=12) | Site 2 (N=13) | Site 3 (N=8) | Total (N=33) | |||||
|---|---|---|---|---|---|---|---|---|
| Characteristic | N | % | N | % | N | % | N | % |
| Staff type | ||||||||
| Provider | ||||||||
| Psychologist | 5 | 42 | 0 | — | 4 | 50 | 9 | 27 |
| Psychiatrist | 1 | 8 | 2 | 15 | 0 | — | 3 | 9 |
| Social worker | 4 | 33 | 7 | 54 | 1 | 13 | 12 | 36 |
| Nurse | 0 | — | 3 | 23 | 1 | 13 | 4 | 12 |
| Total | 10 | 83 | 12 | 92 | 6 | 75 | 28 | 85 |
| Nonprovider | ||||||||
| Telehealth staffa | 2 | 17 | 1 | 8 | 1 | 13 | 4 | 12 |
| Program analyst | 0 | — | 0 | — | 1 | 13 | 1 | 3 |
| Total | 2 | 17 | 1 | 8 | 2 | 25 | 5 | 15 |
| V2H experience | ||||||||
| No direct experience | 4 | 33 | 9 | 69 | 0 | — | 13 | 39 |
| Limited experience | 3 | 25 | 2 | 15 | 2 | 25 | 7 | 21 |
| Most experienceb | 5 | 42 | 2 | 15 | 6 | 75 | 13 | 39 |
a
Telehealth staff had backgrounds in nursing (N=3) and information technology (N=1).
b
All nonproviders (N=5) were classified as having the most V2H experience.
All CFIR constructs identified as relevant to V2H implementation, from the aim 1 and aim 2 analyses, are reviewed in the sections that follow, with construct names appearing in parentheses. They are also displayed in Table 2, along with strategies for addressing the issues associated with these constructs. Many of these recommendations were derived from expert consensus compilations of implementation strategies (17,18).
| CFIR domain, construct, and subsconstruct | Relevance to V2H implementationa | Recommended implementation strategiesb |
|---|---|---|
| Intervention characteristics | ||
| Complexity | Perceived difficulty of implementation, reflected by duration, intricacy, and number of steps involved | Provide local and ongoing logistical and technical support |
| Functionality problems | Provide local and ongoing logistical and technical support | |
| Intervention source, external | Perception that intervention is externally developed or mandated | Conduct local consensus discussions |
| Outer setting | ||
| Patient needs and resources | ||
| Perceived patient need for V2H | Align V2H performance measure goals with areas of patient need | |
| Access | Patients’ access to needed V2H equipment (computer, Webcam) | Provide Webcams to patients |
| Patient perspectives | Perceived patient preference for and satisfaction with V2H | Provide logistical and technical support directly for patients |
| External policy and incentives | Performance measures that incentive V2H implementation | Conduct local consensus discussions |
| Inner setting | . | |
| Compatibility | Degree of tangible fit between V2H and mental health practice | Conduct educational meeting that addresses compatibility concerns and adaptations to address those concerns |
| Leadership engagement | Commitment, involvement, and accountability of leaders and managers with the implementation | Involve leaders with aligning implementation efforts in areas of greatest need, provision of protected time, identifying resources |
| Available resources | ||
| Implementation staff | Availability of staff to provide logical support, training, and ongoing assistance | Identify and train implementation leaders; develop quality monitoring systems to track key implementation metrics (such as average length of time for provider setup and N of sessions canceled due to technology problems) |
| Dedicated time | Provision of protected time for V2H telehealth | Revise professional roles for providers adopting V2H (discuss productivity adjustments to allow time for V2H adoption) |
| Access to knowledge and information | Access to information about V2H setup and use | Develop educational materials that streamline provider and patient setup |
| Characteristics of individuals | ||
| Self-efficacy | Individuals’ belief in their own capability to set up and utilize V2H | Assign point-of-contact to facilitate provider setup and address ongoing V2H needs and concentrate logistical support during early phase of provider adoption |
| Process | ||
| Engaging patients | Strategies for engaging patients with V2H | Align patient engagement with areas of need, engage in directly marketing V2H to patients, and design system-level tools for identifying appropriate patient candidates (such as assessing need at triage/referral points) |
| Engaging providers | Strategies for engaging staff or providers to adopt V2H or increase its availability | Conduct educational outreach |
| Champion | Dedicated individuals involved with generating V2H interest, engagement, adoptions, and overcoming resistance | Identify and prepare a local champion |
Aim 1: CFIR Constructs and Level of V2H Experience
Each participant’s level of V2H experience was identified based on his or her described experience with V2H telehealth (Table 1). Thirteen participants had no direct experience with providing V2H telehealth or its implementation but were familiar with the technology. Seven participants had limited experience with providing V2H telehealth, ranging from attempting to provide V2H services to having provided generally less than ten sessions, usually to one or two patients. Finally, 13 participants reported being involved with V2H implementation or making regular use of V2H telehealth with three or more patients, generally for more than ten sessions.
No direct V2H experience.
This group was distinguished by the greatest concerns about the fit between a videoconferencing modality and mental health practice (compatibility). Concerns were expressed that V2H services would be less conducive to developing a therapeutic alliance or may constrain a clinician’s ability to perceive nonverbal cues, such as trembling. Concerns were also expressed regarding the suitability of V2H telehealth for patients who are at high risk of suicide or who have psychotic symptoms.
Moreover, because mental health treatment often helps patients to counter avoidance patterns (for example, avoiding activity by staying at home worsens depression), other concerns pertained to using V2H technology in instances in which leaving the home more often is clinically indicated. Although these cautions were described by participants in each of the experience groups, the concern was most consistently described among those with no direct V2H experience. These participants also expressed the greatest need for knowledge about V2H setup and operation (self-efficacy), expressing a desire for step-by-step guidance should they decide to adopt the technology. “If you want people to use ‘telemental’ health, you need to have a very hands-on procedure in place so that someone can come, call, set it up for you, take you through it,” observed a provider with no V2H experience (participant 19, facility 2). Participants with at least limited V2H experience also described having these concerns before they used the technology.
Limited V2H experience.
These participants were notable for initial adoption of V2H services and exposure to logistical barriers, such as the time and number of steps to set up V2H technology (complexity). The process of V2H setup includes training, obtaining the equipment, software installations, and scheduling templates as well as assisting patients with setup at home. In the most problematic instances, participants noted that information for each step was often obtained from different sources (access to knowledge and information), with time lags in between and with relatively limited guidance. As one provider with limited V2H experience recalled,
Also, although providers with no direct V2H experience had not been exposed to functionality problems (technical problems), such difficulties begin to emerge among those with at least limited V2H experience. These included instances of difficulty connecting or of poor sound. Other challenges included finding patients who were interested in telehealth (instead of face-to-face contact) and who had the needed technology in their home (patient needs and resources [patient perspectives and barriers subconstruct]). According to a provider with limited experience,I do remember it being like, the trainings were kind of difficult, there wasn’t a lot of guidance . . . and I had trouble figuring out who was supposed to actually come and see me do that, and then they worked it out, but that was kind of complicated. (participant 9, facility 3)
Altogether, the set of findings for participants with limited V2H experience indicate that once engaged to adopt V2H telehealth, providers will subsequently encounter a number of logistical barriers. Two quotes by a participant with limited V2H experience illustrate how logistical barriers can negatively affect enthusiasm for V2H services. “I love [V2H] for a lot of reasons,” he said. “I feel like . . .we’ve entered a different era of how we deliver services and . . . treatment is more patient centered.” Yet the same participant reported concerns about functionality problems: “I think there’s sort of a sense that beyond the first couple of meetings, . . . the effort and the concern about it working may not make it worth it at this point” (participant 28, facility 1).Actually, we haven't had a lot of [patients] who want to do [V2H]. . . . They want to come in and do face-to-face. . . . We offer this to everybody, but I only have two people . . . who wanted to do [V2H]. There might have been some people that I talked about this with in the past and might have wanted to do it, but then say, “Oh I don't have a Webcam.” (participant 4, facility 1)
Most V2H experience.
Because these participants had accumulated the most V2H experience, they reported more instances of patient satisfaction and being able to provide care to patients who otherwise had difficulty accessing care (patient needs and resources [patient perspectives subconstruct]). However, experienced participants reported the greatest frustration with functionality problems and perceived the availability of local technical support as being insufficient (available resources [technical support subconstruct]). Some of these participants continued to express concerns about the compatibility of delivering mental health services via V2H; however, they did not express these concerns as consistently compared with the group with no V2H experience (compatibility). Finally, participants who were experienced with V2H services were unique in having the greatest amount of protected time for initial setup and related V2H support activities (available resources [dedicated time subconstruct]). As one experienced V2H provider explained,
It was a lot more scattered I remember when I first started. Like what do I do next? It would be weeks in between each step. . . . I didn’t have another job. . . . If you’re already in your job full time, you don’t have time for all those things. (participant 22, facility 2)
Aim 2: Factors Among Different Facilities
Facility 1 produced the largest number of V2H visits during FY 2015 (N=740), along with moderate growth in visits compared with FY 2014 (N=583, an increase of 27%). Facility 2 produced fewer V2H visits during FY 2015 (N=263) compared with facility 1, but showed high growth in visits compared with FY 2014 (N=172, an increase of 53%). Finally, facility 3 had the lowest number of visits in FY 2015 (N=171) and had the lowest growth in V2H visits compared with FY 2014 (N=166, an increase of 3%).
Facility 1.
Facility 1 was notable for having a team of providers who were detailed solely to provide mental health services by using telehealth technology and had protected time to do so; thus the dedicated time subconstruct of the available resources construct was rated as having a strong positive impact on V2H implementation at this facility. However, this team of providers was operating at capacity and other providers at the facility were less likely to use V2H, which limited growth. Facility 1 was also notable for engaging patients by promoting telehealth for mental health treatment at multiple community events and setting up a mental health referral process that prompted referring providers to consider appropriateness for telehealth (engaging patients). These and other efforts resulted in a systemwide process for identifying and engaging appropriate patients for V2H services. According to one provider, “We’re the access center of all mental health. . . . We do the screening, find out what’s going on and if [telehealth] is what they want” (participant 34, facility 1).
Facility 2.
This facility was notable for the presence of an active local champion (available resources [champion subconstruct]). In fact, some providers with no V2H experience reported that they were considering initial V2H adoption because of the efforts of the local champion. According to one provider with no V2H experience,
Several participants at facility 2 described a process to engage providers to adopt V2H (engaging providers). E-mails eliciting interest were sent to providers, and those who were interested were invited to a subsequent V2H in-service training. The training was followed by discussions that focused on areas of need and methods for making V2H available. Moreover, ongoing logistical support was provided by the local champion, in terms of explaining the process, demonstrations of the equipment, compiling resources, and follow-up assistance (available resources [implementation staff support subconstruct]). The availability of this support may have mitigated some of the barriers, such as setup complexity and technology problems, given that these barriers were reported to have the least negative impact at this facility.Sure, I have not done it at all myself. . . . I know more about it from [the champion] really and just the work that she’s been doing. She loves it. She just thinks it’s a great service. . . . So I have been thinking about it. (participant 6, facility 2)
Facility 3.
Participants at this facility reported the greatest difficulty with V2H setup (complexity) and the greatest frustration with functionality (functionality problems). These findings coincided with a perception of fewer supports for V2H telehealth in the areas of training, setup assistance, and technical support (available resources [technical support subconstruct]). Finally, participants from facility 3 acknowledged the potential benefits of V2H services (patient needs and resources [need subconstruct]). However, this facility served an urban, densely populated area, and participants reflected on the relative lack of geographical dispersion in their urban service area and a lower perceived need for V2H telehealth. This coincided with a concern that the implementation impetus was external to the facility (intervention source) and was driven by performance measures (external policy and incentives). Thus participants expressed a misalignment between local need for V2H services and the impetus for implementation. According to one provider with limited V2H experience,
The push for telehealth in general, and numbers in general, I’m not particularly a fan of, as opposed to pushing the use of V2H as it’s needed for [patients]. . . . There’s a high concentration of [outpatient clinics] that have mental health [services]. So most people that I meet prefer to do it in person, and this push for numbers . . . feels arbitrary. (participant 9, facility 3)
Discussion
This formative evaluation identified several issues involved with gaining provider buy-in for adopting V2H telehealth, which is consistent with reports of provider misgivings about the changes that V2H services bring to mental health practice (3,12). Among the concerns reported by providers was reconciling home-based care with treatment goals that encourage leaving the home, goals that are common in treating depression or anxiety (19–21). Our study revealed various approaches to the issue of avoidance, even among providers who were experienced with V2H. Although some providers view avoidance as a V2H contraindication, others see telehealth as an opportunity to establish a critical link to treatment that can work with patients to reduce avoidance (such as participating in activities outside the home). This latter view is consistent with recent studies showing that V2H telehealth is effective for treating disorders where reduced avoidance is a key clinical goal (9–11).
A second issue involving provider buy-in pertained to the perceived local need for implementation. Specifically, responses at the site with the fewest V2H visits and the lowest growth in visits reflected that V2H services were less needed, given the site’s highly urban environment. Staff at this site also perceived that the impetus for implementation of V2H technology was external to the facility and was driven by performance measures. As a result, alignment between providers and leaders, considered a key factor for successful implementation, was lacking (22,23).
In addition to underscoring the importance of gaining provider buy-in, the results pointed strongly to the need for strong logistical support to providers and patients adopting V2H use. A key finding is that by itself, increased provider willingness to adopt the modality was insufficient to improve adoption. Instead, our results point to the need for guidance with setup, training, and ongoing operation of the technology. This is consistent with a previous description of efforts by the VHA to implement telehealth for mental health services, which noted the need for close collaboration with information technology resources (3). The lack of these supports may lead to missed opportunities and failure by providers to progress to sustained use.
In addition to examining the impact of CFIR constructs across sites, we examined constructs that were associated with various levels of V2H experience. This provided a glimpse of how site-level implementation helps providers progress from nonuse to sustained use. This process was particularly illustrated by constructs that differentiated groups with various levels of experience with V2H services and sites with varying numbers of V2H visits. For example, our analyses revealed that individuals with protected time for V2H services were most likely to have regular V2H experience and that the facility providing protected time for V2H services produced the greatest number of V2H visits. Complexity of V2H setup and functionality problems serve as additional examples of constructs that were associated with both provider experience and site-level use of V2H telehealth. Our study showed that barriers related to complexity and functionality are encountered when providers first progress to initial V2H use. Participants from the facility with the fewest V2H visits and the lowest growth in visits perceived the fewest available resources from implementation staff and the greatest impact from issues related to the complexity of setup and functional problems.
The key findings of this study indicate that implementation efforts for V2H services should concentrate on provider engagement and buy-in as well as logistical support. A climate of provider engagement that is driven by local leadership and a local champion helps achieve the necessary visibility for providers to consider V2H use and tailor implementation to local needs. These recommendations align with those of experts in implementation science, particularly those involving identifying and preparing a local champion, conducting educational meetings, and holding local consensus discussions (17,18). The second set of recommendations focuses on helping providers navigate the process of V2H setup and use. For provider setup, this study points to the value of a local point of contact who can provide ongoing, individualized facilitation. Consistent with expert recommendations, providers uniformly expressed a preference for local logistical support. Quality monitoring systems can track how well logistical support is provided by examining various indicators, such as the amount of time between a provider’s request for V2H setup and first use and occurrence of technological difficulties.
This study had limitations that are worth noting. First, compared with similar research (16), this study evaluated a relatively small number of sites, which constrains the degree of inference that can be made about facility-level factors. Thus current results should be interpreted as descriptive and as hypothesis generating. Second, the study did not directly evaluate patients’ perspectives. Third, because our study focused on staff perspectives, it did not measure the degree to which certain issues actually occurred (for example, technical problems). Fourth, our study was conducted in a region that includes some rural areas but serves mostly urban and suburban catchment areas. Thus our findings, especially those pertaining to low perceived need and challenges in finding interested patients, may not generalize to facilities with more rural catchment areas. Finally, our recruitment process focused on staff who were at least familiar with V2H services (even those with no V2H experience). As a result, a large percentage of facility staff was not included in our sampling strategy.
Conclusions
This study outlines provider and organizational considerations regarding the implementation of V2H services for mental health care. Key issues highlighted by the study include the need for provider engagement and buy-in efforts, especially efforts that address clinicians’ concerns about the compatibility between V2H technology and clinical practice. Also, several CFIR constructs emerged that point to the importance of logistical support, especially for providers who are newly adopting the technology. Other key issues involve implementing V2H resources in areas of greatest patient need and creating protected time for providers to adopt the technology.
Acknowledgments
The authors thank the participants of this study for their time and thoughtful reflection as well as Jeffrey Smith, Ph.D., for his review of findings and assistance with formulating recommendations.
Supplementary Material
File (appi.ps.201700004.ds001.pdf)
- View/Download
- 168.51 KB
References
1.
Wang PS, Lane M, Olfson M, et al: Twelve-month use of mental health services in the United States: results from the National Comorbidity Survey Replication. Archives of General Psychiatry 62:629–640, 2005
2.
Godleski L, Darkins A, Peters J: Outcomes of 98,609 US Department of Veterans Affairs patients enrolled in telemental health services, 2006–2010. Psychiatric Services 63:383–385, 2012
3.
Koch EF: The VA Maryland Health Care System’s telemental health program. Psychological Services 9:203–205, 2012
4.
Fortney JC, Pyne JM, Edlund MJ, et al: A randomized trial of telemedicine-based collaborative care for depression. Journal of General Internal Medicine 22:1086–1093, 2007
5.
Fortney JC, Pyne JM, Mouden SB, et al: Practice-based versus telemedicine-based collaborative care for depression in rural federally qualified health centers: a pragmatic randomized comparative effectiveness trial. American Journal of Psychiatry 170:414–425, 2013
6.
Backhaus A, Agha Z, Maglione ML, et al: Videoconferencing psychotherapy: a systematic review. Psychological Services 9:111–131, 2012
7.
Richardson LK, Frueh BC, Grubaugh AL, et al: Current directions in videoconferencing tele-mental health research. Clinical Psychology: Science and Practice 16:323–338, 2009
8.
Ruskin PE, Silver-Aylaian M, Kling MA, et al: Treatment outcomes in depression: comparison of remote treatment through telepsychiatry to in-person treatment. American Journal of Psychiatry 161:1471–1476, 2004
9.
Yuen EK, Gros DF, Price M, et al: Randomized controlled trial of home-based telehealth versus in-person prolonged exposure for combat-related PTSD in veterans: preliminary results. Journal of Clinical Psychology 71:500–512, 2015
10.
Acierno R, Gros DF, Ruggiero KJ, et al: Behavioral activation and therapeutic exposure for posttraumatic stress disorder: a noninferiority trial of treatment delivered in person versus home-based telehealth. Depression and Anxiety 33:415–423, 2016
11.
Egede LE, Acierno R, Knapp RG, et al: Psychotherapy for depression in older veterans via telemedicine: effect on quality of life, satisfaction, treatment credibility, and service delivery perception. Journal of Clinical Psychiatry 77:1704–1711, 2016
12.
Mitchell JE, Crosby RD, Wonderlich SA, et al: A randomized trial comparing the efficacy of cognitive-behavioral therapy for bulimia nervosa delivered via telemedicine versus face-to-face. Behaviour Research and Therapy 46:581–592, 2008
13.
Stetler CB, Legro MW, Wallace CM, et al: The role of formative evaluation in implementation research and the QUERI experience. Journal of General Internal Medicine 21(suppl 2):S1–S8, 2006
14.
Creswell JW: Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Thousand Oaks, CA, Sage, 2009
15.
Damschroder LJ, Aron DC, Keith RE, et al: Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation Science 4:50–65, 2009
16.
Damschroder LJ, Lowery JC: Evaluation of a large-scale weight management program using the consolidated framework for implementation research (CFIR). Implementation Science 8:51, 2013
17.
Powell BJ, McMillen JC, Proctor EK, et al: A compilation of strategies for implementing clinical innovations in health and mental health. Medical Care Research and Review 69:123–157, 2012
18.
Powell BJ, Waltz TJ, Chinman MJ, et al: A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implementation Science 10:21, 2015
19.
Foa EB, Hembree EA, Cahill SP, et al: Randomized trial of prolonged exposure for posttraumatic stress disorder with and without cognitive restructuring: outcome at academic and community clinics. Journal of Consulting and Clinical Psychology 73:953–964, 2005
20.
Dimidjian S, Hollon SD, Dobson KS, et al: Randomized trial of behavioral activation, cognitive therapy, and antidepressant medication in the acute treatment of adults with major depression. Journal of Consulting and Clinical Psychology 74:658–670, 2006
21.
Barlow DH, Gorman JM, Shear MK, et al: Cognitive-behavioral therapy, imipramine, or their combination for panic disorder: a randomized controlled trial. JAMA 283:2529–2536, 2000
22.
Lukas CV, Holmes SK, Cohen AB, et al: Transformational change in health care systems: an organizational model. Health Care Management Review 32:309–320, 2007
23.
Horstman MJ, Naik AD: A ‘just culture’ for performance measures. BMJ Quality and Safety 24:486–489, 2015
Information & Authors
Information
Published In
Cover: Understanding, by Michael Olszewski, 1984. Dyed and pleated silk plain weave with silk embroidery. Philadelphia Museum of Art, gift of Nancy and David Bergman, 2015.
History
Received: 4 January 2017
Revision received: 14 April 2017
Revision received: 8 June 2017
Accepted: 29 June 2017
Published online: 1 September 2017
Published in print: January 01, 2018
Keywords
Keywords
Authors
Competing Interests
The authors report no financial relationships with commercial interests.
Funding Information
Quality Enhancement Research Initiative10.13039/100007181: RRP 12-539
This project was funded by a grant from the Quality Enhancement Research Initiative of the VA (RRP 12–539).This material is the result of work supported with resources of the Veterans Health Administration. The contents do not represent the views of the VA or the United States government.Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBLogin options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).