Using Technology to Train Clinicians in Evidence-Based Treatment: A Systematic Review
Abstract
Objective:
Methods:
Results:
Conclusions:
Methods
Results
Description of Studies
Study | Design | Modality | Demographic characteristics of participants | Provider type | Groups | Outcomes | ||
---|---|---|---|---|---|---|---|---|
Primary outcome | Measures | Findings/effect sizeb | ||||||
Depression and anxiety | ||||||||
Bennett-Levy et al., 2012 (20) | RCT: pre, 12-week post, and 4-week FU | OLT, phone or Skype support | N=49, 82% F | Psychologist, social worker, nurse, counselor, and doctor | PRAXIS CBT+telephone or Skype support (supported), N=24; PRAXIS CBT alone (independent), N=25 | CBT knowledge | Cognitive Behavioral Therapy Questionnaire (CBT-Q) | CBT-Qpost: supported=independent (g=.15). CBT-QFU: supported=independent (g=.09) |
Hubley et al., 2015 (21) | RCT: pre, 90–120 minutes post, and 1-week FU | OLT | N=46, 80% F. White, 78% | Psychiatrist, psychologist, psychiatric nurse practitioner, social worker, mental health counselor or therapist, and student | BA OLT, N= 32; attention control OLT (control OLT), N=14 | Knowledge | BA Knowledge Test (BAKT) | BAKTpost: BA OLT>control OLT (g=1.11). BAKTFU: BA OLT>control OLT (g=1.27) |
Anxiety-related disorder | ||||||||
Chu et al., 2017 (3) | RCT: pre, 10-week post | OLT | N=35, 89% F. Non-Hispanic Caucasian, 69% | Postdegree professional or graduate trainee | OLT+expert streaming (ES), N=13; OLT+peer consultation (PC), N=9; fact sheet self-study (FS), N=13 | CBT knowledge | Knowledge Test | FS=ES (g=.27); FS=PC (g=.15); PC=ES (g=.13) |
Ehrenreich-May et al., 2016 (22) | RCT: pre, 35-day post, and 90-day FU | OLT, social media and phone | N=168, 75% F. White, 70% | Psychologist, social worker, mental health counselor or therapist, student, and other | Text-based treatment manual (TXT), N=51; TXT+OLT, N=44; TXT+OLT+learning community (TXT+OLT+LC), N=45 | Knowledge | Knowledge measure (KM) | KMpost: TXT+OLT+LC=TXT+OLT (g=.09); TXT=TXT+OLT (g=.08); TXT+OLT+LC=TXT (g=0). KMFU: TXT+OLT+LC=TXT+OLT (g=.33); TXT=TXT+OLT (g=.15); TXT+OLT+LC=TXT (g=.45) |
Gega et al., 2007 (24) | RCT: pre, 1-hour post-training1, 1 hour post-training2 | Computer software | N=92 | Mental health nursing student | FearFighter software on computer, N=46; lecture, N=46 | Knowledge gain | Two 10-item multiple choice questionnaires (MCQ1, MCQ2) | MCQ1post1: FearFighter=lecture (g=.09). MCQ2post2: FearFighter=lecture (g=.52) |
Harned et al., 2011 (25) | RCT: pre, post, 1-week FU | OLT, phone | N=46, 83% F. Caucasian, 74% | Psychologist, RN/ARNP, social worker, master’s level counselor, bachelor’s level, high school/associate degree | Exposure therapy (ET) OLT (ET OLT), N=15; ET OLT+motivational interviewing (MI) (ET OLT+MI), N=15; control OLT (control), N=16 | Knowledge | Knowledge measure | Knowledge measurepost: ET OLT>control (g=2.13); ET OLT+MI>control (g=3.10); ET OLT=ET OLT+MI (g=.16). Knowledge measureFU: ET OLT>control (g=1.54); ET OLT+MI>control (g=1.89); ET OLT=ET OLT+MI (g=.10) |
Harned et al., 2014 (26) | RCT: pre, 6-week post, 6- and 12-week FU | OLT, online conferencing | N=181, 71% F. Caucasian, 72% | Psychiatrist, psychologist, psychiatric nurse practitioner, social worker, master’s-level counselor, bachelor's level, and student | OLT, N=60, OLT+motivational enhancement (ME) (OLT+ME), N=60; OLT+ME+learning community (LC) (OLT+ME+LC), N=61 | Knowledge | Knowledge measure | Knowledge measurepost: OLT+ME+LC>OLT (g=1.98); OLT+ME+LC>OLT+ME (g=2.98); OLT=OLT+ME (g=.99). Knowledge measure6wk FU: OLT+ME+LC>OLT (g= 1.98); OLT+ME+LC>OLT+ME (g= 2.98); OLT=OLT+ME (g=.99) |
McDonough and Marks, 2002 (23) | RCT: pre, 90 minutepost | Computer software | N=37, 54% F | Third-year medical student | FearFighter software on computer (computer), N=19; small-group face-to-face teaching (tutorial), N=18 | Knowledge | 15-item multiple choice questions | Tutorial=computer (g=.65) |
Rakovshik et al., 2016 (27) | RCT: pre, midtraining, post | OLT, Skype | N=61, 70% F | Psychologist, psychiatrist, and psychiatrist– psychotherapist | Internet-based training (IBT)+consultation worksheet (IBT-CW), N=19; IBT+Skype (IBT-S), N=22; no-training control (delayed training [DT]), N=20 | CBT skills | Cognitive Therapy Scale (CTS) | IBT-S>IBT-CW (g=.71); IBT-S>DT (g=1.31); IBT-CW=DT (g=.62) |
Substance use disorder | ||||||||
Larson et al., 2013 (30) | RCT: pre, 8-week post, 3-month FU | OLT, phone | N=127, 66% F | Addiction counselor | Web training (Web course), N=62; manual training (control), N=65 | Adequate adherence to CBT delivery | A low pass or greater on at least 1 of 3 core CBT skills—generic skills, client-centered motivational stance, and CBT-specific skills | Counselors passing on 3 core skill outcomespost: Web course=controlc |
Sholomskas and Carroll, 2006 (31) | RCT: pre, 3-week post | CD-ROM | N=28, 64% F. American Indian, 4%; black, 24%; white, 72%d | Social worker, primary clinician, psychiatric nurse/mental health assistant, occupational therapist, and other | Manual plus computer-based training (CD-ROM+manual), N=12; manual only (manual), N=13 | Adherence to and skill in Twelve-Step Facilitation | Videotaped role-plays rated on the Yale Adherence Competence Scale | Role-playspost: CD-ROM+manual>manual (g=.95). Skillpost: CD-ROM+manual>manual (g=.90) |
Sholomskas et al., 2005 (7) | Non-RCT: pre, 4-week post, 3-month FU | OLT, phone supervision | N=78, 54% F. AA, 27%; Hispanic, 8%; Caucasian, 61%; other, 4% | Clinician treating a predominantly substance-using population | Manual, N=27; manual+Web-based training (manual+Web), N=24; manual+didactic training (seminar and supervision) (manual+seminar+supervision), N=27 | Ability to demonstrate key CBT intervention strategies | Structured role-plays rated on the Yale Adherence Competence Scale | Adherencepost: manual+seminar+supervision>manual+Web (g=.38); manual+Web>manual (g=.47); manual+seminar+supervision>manual (g=.86). Skillpost: manual+seminar+supervision>manual+Web (g=.46); manual+Web>manual (g=.38); manual+seminar+supervision>manual (g=.83) |
Weingardt et al., 2006 (28) | RCT: pre, 60-minute post | OLT | N=166, 55% F. AA, 21%; Asian/Pacific Islander, 6%;white, 55%; Latino, 12%; other, 4%; missing, 2% | Substance abuse counselors | Web-based training (WBT), N=52; face-to-face workshop (FTF), N=55; delayed-training control (control), N=59 | Knowledge about the Coping with Craving content | 17 multiple-choice questions | WBT>control; FTF>control; WBT=FTFc |
Weingardt et al., 2009 (29) | RCT: pre, 1-month post | OLT | N=147, 62% F. AA, 20%; Caucasian, 67%; Hispanic, 8%; other, 6%; Asian/Pacific Islander, .7% | Substance use disorder counselor | Strict adherence to OLT course (high fidelity), N=73; flexible adherence to OLT course (low fidelity), N=74 | Knowledge | 73 multiple-choice questions on CBT knowledge | High fidelity=low fidelity (g=.12) |
Substance use disorder and suicidality | ||||||||
Dimeff et al., 2009 (6) | RCT: pre, 2 days post (ILT)/45 days post (OLT/manual), 30-day and 90-day FU | N=150, 70% F. White, 81%; Asian American, 7%; Hispanic/Latino, 3%; AA, 3%; Native American, 2%; other, 5% | Psychiatrist, psychologist, chemical dependency counselor, master’s in social work, master’s level counselor, bachelor’s level counselor, and other | Written treatment manual (manual), N=49; DBT skills OLT (OLT), N=54; instructor-led training (ILT), N=47 | Knowledge | DBT skills knowledge and application (KT) | KTpost: OLT>manual (g=.51); OLT>ILT (g=.34); ILT=manual (g=.20). KTFU: OLT>manual (g=.51); OLT>ILT (g=.41); ILT=manual (g=.12) | |
Dimeff et al., 2011 (32) | RCT: pre; 2.5 hours post; and 2, 7, 11, and 15 weeks FU | CD-ROM | N=132, 74% F. Caucasian, 83%; Hispanic/ Latino, 9%; multiracial, 3%; Native American, 2%; AA, 1%; Asian American, 2%; Middle Eastern, 1% | Psychiatric nurse, psychologist, psychiatric nurse practitioner, chemical dependency counselor, master’s in social work (M.S.W.), mental health counselor/ therapist, mental health counselor/technician, and other | DBT treatment manual (manual), N=43; DBT e-Learning course (e-DBT), N=47; control e-Learning course (e-control), N=42 | Knowledge of DBT distress tolerance skills | DBT Distress Tolerance Skills Knowledge and Application Test (KT) | KTpost: e-DBT>e-control (g=3.4); manual>e-control (g=2.78); e-DBT=manual (g=.24). KTFU: e-DBT>e-control (g=1.86); manual>e-control (g=1.48); e-DBT>manual (g=.35) |
Dimeff et al., 2015 (33) | RCT: pre, 2 days post (ILT), 30 days post (OLT/manual), 60-day and 90-day FU | OLT | N=200, 76% F. Caucasian, 79%; AA, 4%; Asian American, 6%; Hispanic, 4%; Native American, 2%; and other, 5% | Psychologist, psychiatrist, psychiatric nurse practitioner, chemical dependency counselor, social worker (M.S.W.), mental health counselor (M.A./M.S./MFT), mental health counselor (B.A./B.S.), student, and other | OLT, N=66; instructor-led training (ILT), N=67; treatment manual (TM), N=67 | Satisfaction, self-efficacy | Satisfaction measure, adapted Behavioral Anticipation and Confidence Questionnaire (BAQ) | Satisfaction measurepost: ILT>OLT (g=.62); ILT>TM (g=.86); OLT=TM (g=.21). BAQpost: ILT>OLT (g=.78); ILT>TM (g=.94); BAQFU: ILT>OLT (g=.87); ILT>TM (g=.82) |
Posttraumatic stress disorder | ||||||||
Ruzek et al., 2014 (34) | RCT: pre, 1-month post | OLT, phone consultation | N=168, 70% F. white, 74%; AA, 11%; non-white, Hispanic, or other, 18% | VHA mental health clinician with master’s degrees or doctoral-level training in mental health or related disciplines | Web training (Web), N=57; Web training+consultation (Web+consult), N=55; training as usual (control), N=56 | Intervention skills acquisition | Standardized patient evaluation of motivation enhancement and behavioral task assignment | Motivation enhancement: Web>control (g=.70); Web+consult>control (g=1.43); Web+consult>Web (g=.67). Behavioral task assignment: Web>control (g=.34); Web+consult>control (g=.64) |
Bipolar disorder | ||||||||
Stein et al., 2015 (35) | RCT: 180 days post, 365 days and >365 days FU | OLT, phone supervision | N=36 | social worker, licensed professional counselor, and clinical psychologist, nurse | Internet-supported e-learning (e-learning), N=16; in-person training (IPT), N=20 | Extent to which clinicians used interpersonal and social rhythm therapy (IPSRT) techniques | Psychotherapy Practice Scale adapted for IPSRT (PPS-IPSRT) completed by patients | PPS-IPSRTpost: e-learning=IPTc. PPS-IPSRTFU: e-learning=IPTc |
Eating disorder | ||||||||
Cooper et al., 2017 (36) | Pre, 20-week post, 6-month FU | OLT, e-mail, or phone support | N=156, 93.3% F | Mainly clinical psychologists and social workers | OLT alone (independent training,) N=75; OLT+telephone support (supported training), N=81 | Competence | 22 items addressing trainee knowledge and understanding of CBT-E and its implementati-on (i.e., applied knowledge) | Competencepost: independent=supportedc. CompetenceFU: independent=supportedc |
Autism | ||||||||
Granpeesheh et al., 2010 (37) | RCT: pre, 16- hour (in person) or 10-hour (e-learning) post | Computer software | N=88 | Entry-level behavioral therapist | e-learning group, N=33; in-person training group (standard), N=55: | Knowledge/ competence | Written examination consisting of 32 long- and short-answer questions | Standard>e-learningc |
Motivational interviewing | ||||||||
Mullin et al., 2016 (38) | Non-RCT: pre, 5-month post | OLT | N=34 | Psychologist, clinical social worker, medical student, family medicine resident, nurse practitioner, primary care/OB-GYN, physician, research staff, and other | OLT CITMI, N=14; in-person training, N=30 | Motivational interviewing skills | Motivational interviewing treatment integrity code | Online training=in-person trainingc |
General CBT skills | ||||||||
German et al., 2018 (40) | Pre, end of assessment (post1), and competency assessment point (post2) | OLT | N=362, 78% F | Community clinicians | In person, expert-led (IPEL), N= 214; Web-based, trained-peer (WBTP), N=148 | Competency | Cognitive Therapy Rating Scale (CTRS) | CTRSpost1: IPEL=WBTP (g=.09); CTRSpost2: IPEL=WBTP (g=.17) |
Rakovshik et al., 2013 (39) | RCT: pre, 1- month post | OLT | N=63, 91% F | Master’s level student of neuropathology and psychopathology or clinical psychology and psychotherapy | Immediate access to Internet-based CBT training (immediate), N=31; Internet-based CBT training after 1-month wait (delayed), N=32 | Competence | Ratings of performance in the Objective Structured Clinical Exam (OSCE), a 20-minute role-play of a CBT assessment, and assessment of the quality of participant's’ formulation of the OSCE ”patient” | CBT assessment and formulation skills: immediate>delayed (g=.90) |
Methodological Quality
Study | Q1 | Q2 | Q3 | Q4 | Q5 | Q6 | Q7 | Q8 | Q9 | Q10 | Q11 | Q12 | Q13 | Q14 | Rating |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Bennet-Levy et al., 2012 (20) | Yes | NR | NR | No | NA | Yes | Yes | No | Yes | Yes | No | No | Yes | No | Poor |
Chu et al., 2017 (3) | Yes | NR | NR | No | NA | Yes | Yes | Yes | No | NR | No | No | Yes | No | Poor |
Cooper et al., 2017 (36) | Yes | No | No | No | NA | NR | No | Yes | Yes | NR | Yes | NR | Yes | No | Poor |
Dimeff et al., 2009 (6) | Yes | Yes | Yes | No | NR | Yes | Yes | Yes | Yes | NR | No | NR | Yes | Yes | Fair |
Dimeff et al., 2011 (32) | Yes | Yes | Yes | No | Yes | No | Yes | No | NR | NR | No | Yes | Yes | Yes | Poor |
Dimeff et al., 2015 (33) | Yes | Yes | Yes | No | NR | Yes | Yes | No | NR | NR | No | Yes | Yes | No | Poor |
Ehrenreich-May et al., 2016 (22) | Yes | Yes | Yes | No | NR | Yes | No | Yes | NR | NR | No | Yes | Yes | Yes | Poor |
Gega et al., 2007 (24) | Yes | Yes | Yes | No | Yes | Yes | Yes | Yes | NR | NR | No | Yes | Yes | No | Poor |
German et al., 2018 (40) | No | NA | NA | No | NR | Yes | Yes | No | No | NR | Yes | NR | Yes | NA | Poor |
Granpeesheh et al., 2010 (37) | Yes | Yes | No | No | Yes | NR | Yes | NR | NR | NR | No | NR | Yes | No | Poor |
Harned et al., 2011 (25) | Yes | Yes | Yes | No | NA | Yes | Yes | Yes | NR | NR | No | Yes | Yes | Yes | Fair |
Harned et al., 2014 (26) | Yes | Yes | No | No | Yes | Yes | No | No | Yes | NR | No | Yes | Yes | Yes | Poor |
Hubley et al., 2015 (21) | Yes | Yes | Yes | No | NA | Yes | Yes | Yes | NR | NR | Yes | NR | No | No | Poor |
Larson et al., 2013 (30) | Yes | No | NR | No | NR | NR | CD | CD | Yes | NR | No | No | Yes | No | Poor |
McDonough and Marks, 2002 (23) | Yes | Yes | Yes | No | NA | Yes | Yes | Yes | NR | NR | No | NR | Yes | Yes | Fair |
Mullin et al., 2016 (38) | No | NA | NA | NA | Yes | No | Yes | Yes | Yes | NR | Yes | NR | Yes | NA | Poor |
Rakovshik et al., 2013 (39) | Yes | Yes | Yes | No | Yes | Yes | Yes | Yes | NR | Yes | Yes | Yes | Yes | Yes | Good |
Rakovshik et al., 2016 (27) | Yes | Yes | Yes | No | Yes | NR | No | Yes | NR | NR | Yes | Yes | Yes | Yes | Poor |
Ruzek et al., 2014 (34) | Yes | Yes | Yes | No | Yes | Yes | Yes | Yes | No | No | Yes | No | Yes | No | Poor |
Sholomskas and Carroll, 2006 (31) | Yes | NR | NR | No | Yes | Yes | Yes | NR | NR | NR | Yes | NR | Yes | No | Poor |
Sholomskas et al., 2005 (7) | No | No | NA | No | NR | Yes | Yes | No | No | NR | Yes | NR | Yes | No | Poor |
Stein et al., 2015 (35) | Yes | No | NR | No | Yes | NR | Yes | Yes | Yes | NR | No | NR | Yes | Yes | Fair |
Weingardt et al., 2006 (28) | Yes | NR | NR | No | NA | Yes | Yes | Yes | NR | NR | No | NR | Yes | Yes | Fair |
Weingardt et al., 2009 (29) | Yes | NR | NR | No | NA | Yes | No | Yes | NR | NR | No | NR | Yes | No | Poor |
Training Programs
Studies on depression and anxiety.
Studies on anxiety-related disorders.
Studies on substance use disorders.
Studies on substance use and suicidality.
Studies on other disorders.
Studies on supported training.
Discussion
Limitations of the Literature
Limitations of the Study
Conclusions
Supplementary Material
- View/Download
- 55.24 KB
References
Information & Authors
Information
Published In
History
Keywords
Authors
Competing Interests
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBLogin options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).