A Web-Based Platform to Support an Evidence-Based Mental Health Intervention: Lessons From the CBITS Web Site

The Cognitive Behavioral Intervention for Trauma in Schools (CBITS) Web site (cbitsprogram.org) is an online training and implementation support Web site for CBITS, a widely recognized (11), effective treatment for trauma-exposed children (12). The Web site was developed in response to high demand for CBITS training and requests for ongoing implementation support. The site comprises a “public” side that contains general information about CBITS, biographies of the intervention developers, and answers to frequently asked questions and a “private” side that provides registrants with a training course, implementation assistance resources, and a community forum.

To better understand use of the CBITS Web site, we used Google Analytics to examine Web site utilization among all registrants between July 1, 2011, and June 30, 2012 (N=1,406). Average site visit duration, or the average length of time a user spends on the Web site per visit, was used to evaluate Web site engagement. Page views, defined as the total number of pages a registrant viewed during all Web site visits, was the measure used to examine usage of the three sections of the site. Web site user characteristics, including demographic information, training background, clinical experience, purpose for visiting the site, and prior participation in CBITS training, were obtained from the Web site registration survey.

Using SPSS version 21.0, we described registrants’ characteristics and calculated overall Web site engagement and usage of Web site sections. We conducted bivariate analyses, using t tests and one-way analysis of variance (ANOVA), as appropriate, to examine the relationship between user characteristics, Web site engagement, and content usage. The RAND Corporation’s institutional review board (IRB) exempted the study from IRB review.

Registrants were predominantly female (86%; N=1,186 of 1,374) and Caucasian (68%; N=959 of 1,406) and had a master’s degree (68%, N=939 of 1,376) and had not been trained in CBITS (79%; N=1,085 of 1,374). [Further information about user characteristics is available in an online data supplement to this report.] Registrants averaged 13:59 minutes per visit, with registrants not previously trained in CBITS spending significantly longer on the Web site each visit than previously trained registrants (15:11 versus 10:08 minutes, p<.001; Table 1). Registrants with less than four years of CBT experience had significantly longer visits than registrants with four to six years of CBT experience (16:07 versus 11:11 minutes, p<.01), as did registrants with four to seven years of clinical experience compared with those with eight or more years of experience (16:53 versus 11:05 minutes, p<.001). Registrants whose reason for visiting was to work toward becoming a CBITS clinician spent significantly more time per visit than registrants who did not visit the site for this reason (15:29 versus 12:38 minutes, p=.009). We found no other significant differences in visit duration when examining registrants’ purpose for visiting the Web site, including administrators who indicated when registering that they were visiting the site to better understand CBITS.

Registrants’ interaction with Web site content, as assessed by page views, was greatest for the training course (mean of 21.94 pages viewed), followed by the implementation assistance section (6.90) and the online community forum (.71). Registrants without prior training in CBITS viewed significantly more pages in the course section than those with prior training (24.26 versus 14.05 pages, p<.001), as did registrants with less than four years of CBT experience compared with registrants with four to six years of experience (27.28 versus 16.15 pages; p<.001). Clinicians with less than eight years of experience as a mental health clinician viewed significantly more pages compared with those with eight or more years of experience (16.30 versus 25.47 pages; p<.001). Individuals whose purpose for visiting was to become a CBITS clinician viewed significantly more pages in the course section than those who did not indicate this as a reason for visiting (26.79 versus 17.63 pages, p<.001). Registrants whose purpose for visiting was to understand CBITS as an administrator viewed significantly fewer pages in the course section than did those who did not indicate this as a reason for visiting (15.52 versus 23.36 pages, p=.006). In the implementation assistance and online community sections, those previously trained in CBITS viewed more pages in both the implementation assistance section (10.25 versus 6.16 pages; p<.001) and the online community section (1.44 versus .50 pages visited; p<.001).

Evidence-based practices can “languish” 15–20 years before being routinely used (13), but Web sites may facilitate more rapid dissemination. Our examination may begin to inform how Web sites can be used to support training and implementation of evidence-based practices. We found high levels of Web site engagement overall, with the training section getting the most usage across all user types. The substantial use by untrained individuals was unsurprising, given that many clinicians enter the workforce untrained in effective treatments (2). With the need to increase the availability of training in effective interventions (14) and findings showing that distance learning is an effective and cost-efficient approach to enhancing clinical skills (7), initial and repeated exposure to course material through Web sites is one potentially scalable approach to addressing this implementation barrier.

Although not attracting as much traffic as the training course, the implementation support section had substantial usage. The greater use of this section, particularly among individuals previously trained in CBITS, suggests that once trained, clinicians may seek information designed to support implementation. The availability of implementation support materials is likely to result in more successful implementation because ongoing implementation support is essential for establishing evidence-based practices within a system (3).

There was relatively little use of the online community section, likely resulting from a lack of Web site prompts and new material, features that bring individuals back to Web sites on an ongoing basis (15). Research is needed to identify how online communities can enhance implementation and provide real-time feedback regarding implementation barriers or clinical situations as they arise.

Although little is known about how a nonclinical audience interacts with Web-based platforms primarily designed to train clinicians in evidence-based interventions, our findings reveal that administrators had high rates of engagement with the CBITS Web site, with visit duration times comparable with those of clinicians. However, compared with their clinical counterparts, administrators viewed fewer pages in the course section. Implementation is facilitated when administrators are committed to providing evidence-based practices and are knowledgeable about their use (5). These findings may help us understand how Web sites can be used to facilitate administrator support for the intervention. More broadly, research clarifying the needs of specific user groups is necessary to understand how Web-based platforms can more effectively support each user type.

Our study had several limitations. We had no information on whether Web site usage enhanced adoption, implementation, or sustainability of CBITS. Our study did not assess differences in the quality of online versus in-person CBITS training, nor differences in knowledge and skills gained by clinicians trained online versus in person. However, prior research would likely indicate that they are relatively equivalent (7,8,10). Finally, Web analytics data were gathered with a combination of hypertext transfer protocol access logs, sessions, persistent cookies, and Google Analytics user tracking tools. How information obtained through Google Analytics differs from self-reported usage is not known.

Our findings suggest that Web-based platforms have the potential to support the training and implementation of evidence-based practices for clinicians with varying levels of experience and familiarity with the intervention. These findings are particularly relevant at a time when there are greater incentives to disseminate evidence-based practices and a need to train and support a growing mental health workforce. In addition, Web-based platforms may be promising for supporting trauma-related interventions, for which training in or access to implementation assistance should be available rapidly after a traumatic event.

Support for this study was provided by grant SM061270 from the Substance Abuse and Mental Health Services Administration and by grant P30MH082760 from the National Institute of Mental Health.

The authors report no competing interests.