Therapist- and Client-Level Predictors of Use of Therapy Techniques During Implementation in a Large Public Mental Health System
Abstract
Objective:
Many youths receiving community mental health treatment do not receive evidence-based interventions. Research suggests that community mental health therapists use a broad range of therapeutic techniques at low intensities. This study examined the relationship between therapist- and client-level predictors of community-based therapists’ report of cognitive, behavioral, psychodynamic, and family techniques within the context of implementation efforts.
Methods:
A total of 130 therapists participated from 23 organizations in an urban, publicly funded behavioral health system implementing evidence-based practices. Therapist-level predictors included age, gender, clinical experience, licensure status, and participation in evidence-based practice initiatives. Child-level predictors included therapist-reported child primary disorder (externalizing, internalizing, or other) and child age. Therapists completed the Therapist Procedures Checklist–Family Revised, a self-report measure of therapeutic techniques used.
Results:
Unlicensed therapists were more likely than licensed therapists to report using psychodynamic and behavioral techniques. Therapists who did not participate in an evidence-based practice initiative were less likely to report use of cognitive techniques. Those with clients with externalizing disorders were more likely to report use of behavioral and family techniques. Therapists with the youngest clients (ages three to seven years) were most likely to report use of behavioral techniques and less likely to report use of cognitive and psychodynamic techniques.
Conclusions:
Results suggest that both therapist and client factors predict self-reported use of therapy techniques. Participating in an evidence-based practice initiative was associated with increased reports of using cognitive techniques. Therapists reported using behavioral and family techniques more than other techniques when working with youths with externalizing disorders and using fewer cognitive and psychodynamic techniques with young clients.
The past 50 years have resulted in a proliferation of evidence-based practices (EBPs) for treatment of psychiatric disorders for a range of client populations and the patients’ presenting problems (1,2). Despite the development and validation of these treatments, most youths do not have access to EBPs in their communities (3,4). Although the need to translate EBPs from research to practice settings has been identified (5), progress to do so has been slow and hampered by a lack of understanding of community-based usual care (6–8). The limited research to date examining usual care has focused on characterizing the types of treatment techniques used by community providers (9), that is, the procedures used during the intervention process (such as role playing and relaxation training [10]). Although this has helped elucidate usual care, little is known about variables that may influence the use of specific therapeutic techniques delivered by community therapists.
Studies focused on usual care of disruptive behavioral disorders have demonstrated that community therapists report using a wide range of techniques, some evidence based and some not (11). Observational studies have shown that community therapists generally deliver a broad range of techniques at relatively low intensities (12,13) in the treatment of youths with externalizing disorders. Studies that have examined therapist self-report of usual care of youths with internalizing disorders have similarly shown a tendency of therapists to use a variety of therapeutic techniques at low intensities (14,15). Observational studies by McLeod and Weisz (16) and Southam-Gerow and colleagues (17) have demonstrated that community therapists use a variety of techniques in their treatment of youths with internalizing disorders, generally favoring client-centered approaches. Weisz and colleagues (18) found that therapists delivering usual care to youths with depression were observed to use more family and psychodynamic techniques than cognitive-behavioral therapy (CBT) techniques.
Increasing the use of EBPs in usual care is an important objective for mental health services research. The literature suggests that children receiving community-based usual care do not clinically improve at the same magnitude as youths receiving EBPs (19–21). In direct comparisons, EBPs generally outperform usual care. For example, in a series of meta-analyses by Weisz and colleagues (21,22), differences in effect sizes between usual care and EBPs were approximately .29 (Cohen’s d). Youths receiving usual care in publicly funded systems have worse treatment trajectories than youths in managed-care systems (20). Taken together, these data suggest considerable room for improvement in usual community care practices.
To date, few studies have examined contextual predictors of use of evidence-based therapy techniques in usual care. Therapist variables (knowledge and attitudes) and client variables (presenting disorder) may be important predictors of whether a particular therapist uses evidence-based techniques in usual care settings (23,24). In one recent study, therapist factors (knowledge and attitudes) were found to be more predictive of use of psychodynamic techniques, but organizational factors (such as culture and climate) were found to be more predictive of use of CBT and family techniques (25). A limitation of the literature to date is that client factors have been largely ignored as possible predictors of use of therapy techniques. Understanding both therapist and client factors that predict use of therapy techniques is important because it allows for the identification of factors that can be targeted to change therapist behavior.
This study allowed for exploration of predictors of therapist self-report of techniques that fall into cognitive, behavioral, psychodynamic, and family domains within the context of a systemwide effort to improve usual care through implementing EBPs (26). The community mental health therapists in this sample provide care to youths in a large, urban, publicly funded system—circumstances that support the generalizability of our study. In addition, we were able to examine usual care within the unique context of a system in which providers implement multiple EBPs. Specifically, in the publicly funded mental health system in Philadelphia, therapists are receiving training and consultation in a number of EBPs, including cognitive therapy (27), trauma-focused CBT (28), prolonged exposure (29), and dialectical behavior therapy (30). This study examined whether community-based therapists in this system report using cognitive, behavioral, psychodynamic, and family techniques and if therapist- or client-level factors predict use of these techniques. Given the limited data on predictors of usual care practices, this study was exploratory in nature and was not based on any a priori hypotheses.
Methods
Setting
The Department of Behavioral Health and Intellectual disAbility Services (DBHIDS) in the City of Philadelphia (26) has engaged in pilot CBT implementation initiatives in the public mental health system since 2007. Implementation efforts are supported by a full-time city employee who coordinates the initiative. The training and ongoing consultation provided to therapists closely follows the recommendations of the respective treatment developers and includes posttraining consultation. For example, outpatient providers enrolled in the cognitive therapy training participated in 22 hours of didactic workshops followed by six months of weekly group consultation (31). Other initiatives have similarly intensive procedures.
Participants
Purposive sampling was used to recruit the 29 largest agencies within the over 100 community mental health agencies in Philadelphia that provide outpatient services to youths (personal communication, Community Behavioral Health, 2012). These agencies together serve approximately 80% of Philadelphia youths receiving publicly funded mental health care. Of these 29 agencies, 18 (62%) agreed to participate. An additional agency involved in EBP efforts approached us to participate, resulting in a final sample of 19 agencies representing 23 sites, 130 therapists, 36 supervisors, and 22 executive administrators. There were no exclusion criteria for therapist participation, and approximately 60% of therapists from the 23 sites participated. Of participating therapists, 123 (95%) completed all measures on one occasion. Of the agencies enrolled, 16 had participated in DBHIDS-sponsored EBP initiatives (mean±SD years of participation=2.90±2.70); of the therapists enrolled, 54 (41%) had participated in DBHIDS-sponsored EBP initiatives. Table 1 provides demographic information about therapists and clients.
| Therapists (N=130) | Clients (N=125) | |||
|---|---|---|---|---|
| Variable | N | % | N | % |
| Age (M±SD) | 38.09±11.63 | 10.86±3.70 | ||
| Years at current agency (M±SD) | 3.35±4.65 | — | — | |
| Years of clinical experience (M±SD) | 6.89±6.84 | — | — | |
| Current caseload (M±SD) | 28.79±22.05 | — | — | |
| Level of professional burnout (M±SD)a | 4.23±2.58 | — | — | |
| Hours of supervision received each week (M±SD) | 1.32±1.21 | — | — | |
| Genderb | ||||
| Male | 30 | 23 | 69 | 53 |
| Female | 99 | 76 | 56 | 43 |
| Transgender | 1 | 1 | 0 | — |
| Hispanic or Latinob | ||||
| Yes | 26 | 20 | — | — |
| No | 98 | 75 | — | — |
| Ethnicityb | ||||
| Asian | 6 | 5 | 1 | 1 |
| Black or African American | 27 | 21 | 54 | 42 |
| White | 67 | 52 | 12 | 9 |
| Hispanic or Latino | 13 | 10 | 33 | 25 |
| Multiracial | 5 | 4 | 7 | 5 |
| Other | 5 | 4 | 7 | 5 |
| Academic backgroundb | ||||
| Bachelor’s degree | 5 | 4 | — | — |
| Master’s degree | 107 | 82 | — | — |
| Doctoral degree | 12 | 9 | — | — |
| Professional background | ||||
| Master’s-level counselor | 75 | 58 | — | — |
| Social worker | 20 | 15 | — | — |
| Marriage and family therapist | 18 | 14 | — | — |
| Psychologist | 6 | 5 | — | — |
| Psychiatrist | 1 | <1 | — | — |
| Other | 4 | 3 | — | — |
| Licensed | ||||
| Yes | 33 | 25 | — | — |
| No | 97 | 75 | — | — |
| Participation in EBP initiativesc | ||||
| Yes | 54 | 41 | — | — |
| No | 78 | 59 | — | — |
| Primary theoretical orientationb | ||||
| Psychodynamic | 10 | 8 | — | — |
| Behavioral | 6 | 5 | — | — |
| Cognitive | 5 | 4 | — | — |
| Cognitive-behavioral | 50 | 39 | — | — |
| Systemic | 20 | 15 | — | — |
| Object relations | 3 | 2 | — | — |
| Other | 4 | 3 | — | — |
| Eclectic | 26 | 20 | — | — |
| Primary diagnosisb,d | ||||
| Internalizing disorder | — | — | 33 | 27 |
| Posttraumatic stress disorder | — | — | 13 | 11 |
| Major depressive disorder | — | — | 8 | 7 |
| Anxiety disorder NOS | — | — | 6 | 5 |
| Generalized anxiety disorder | — | — | 2 | 2 |
| Depressive disorder NOS | — | — | 2 | 2 |
| Dysthymic disorder | — | — | 1 | 1 |
| Adjustment disorder with anxiety | — | — | 1 | 1 |
| Externalizing disorder | — | — | 71 | 58 |
| ADHD | — | — | 47 | 38 |
| Oppositional defiant disorder | — | — | 7 | 6 |
| ADHD plus oppositional defiant disorder | — | — | 7 | 6 |
| Conduct disorder | — | — | 6 | 5 |
| Disruptive behavior disorder NOS | — | — | 4 | 3 |
| Other disorder | — | — | 19 | 15 |
| Adjustment disorder, unspecified | — | — | 12 | 10 |
| Adjustment disorder with mixed disturbance of emotions and conduct | — | — | 2 | 2 |
| Reactive attachment disorder | — | — | 1 | <1 |
| Autistic disorder | — | — | 1 | <1 |
| Asperger’s disorder | — | — | 1 | <1 |
| Psychotic disorder NOS | — | — | 1 | <1 |
| Unspecified substance use disorder | — | — | 1 | <1 |
a
Responding to an item on the Therapist Background Questionnaire, therapists rated how often they “experienced a feeling of professional burnout” on a scale from zero, never, to 10, constantly.
b
Does not add up to 100% because of missing responses
c
EBP, evidence-based practice
d
NOS, not otherwise specified
Procedure
All procedures were approved by and conducted in compliance with the appropriate institutional review boards. A complete description of the procedures of the larger study within which these data were collected has been published elsewhere (25). Participants provided informed consent and were compensated $50 for participation. The principles outlined in the Declaration of Helsinki were followed. Data were collected between March 1 and July 25, 2013. Therapists were informed that their data would be kept confidential and that only aggregated and deidentified information would be shared with agency leadership and DBHIDS.
Predictor Variables
Therapist characteristics.
Therapist demographic characteristics were assessed with the Therapist Background Questionnaire (32), a 21-item questionnaire that obtains information on personal characteristics, such as age, gender, and licensure status. In addition, therapists indicated on the Evidence-Based Practices Training Survey whether they had participated in any of the four DBHIDS EBP initiatives (cognitive therapy, trauma-focused CBT, prolonged exposure, and dialectical behavior therapy). We asked specifically whether they had received training and a year of consultation to ensure that they formally participated in the city-sponsored initiative.
Client characteristics.
Therapists identified a representative client, defined as a client of their choosing who was most like a typical client on their caseload, and reported on that client’s age, race-ethnicity, and primary diagnosis. Childhood primary diagnoses, when necessary, were coded into DSM-IV disorders (for example, depression was coded as major depressive disorder). To maximize power for statistical analyses, we subsequently coded diagnoses as internalizing, externalizing, and other, given that there is considerable overlap in evidence-based techniques for internalizing disorders as well as for externalizing disorders (33). Therapists then reported on the therapeutic techniques used with the representative client they identified.
Dependent Variable: Therapy Techniques
The Therapist Procedures Checklist–Family Revised (TPC-FR [15,34]) is a 62-item self-report therapist checklist that assesses therapeutic techniques from the following modalities: cognitive, behavioral, family, and psychodynamic. The TPC-FR informed the development of the Therapy Process Observational Coding System (TPOCS [16]), a gold-standard observer-rated measure of therapy fidelity. The TPC-FR is a revised version of the TPC (34) developed to also query about family therapy techniques (15). Therapists selected a recent representative client about whom they reported, on one occasion, on which techniques they had used with that client over the course of therapy to date. The rating for each subscale is a mean of the items that fit within that factor, measured on a continuum from 1 to 5, with 1 indicating rarely; 2, seldom; 3, sometimes; 4, often; and 5, most of the time. Higher scores indicate greater self-reported use of the set of techniques. Good internal consistency has been reported for the TPC-FR (15) and the factor structure has been confirmed (35). In our sample, subscale alpha values ranged from .84 to .94.
Data Analytic Plan
Four mixed-effects linear regression models examined the relationships of the set of variables with self-reported use of specific therapy techniques (specifically, cognitive, behavioral, family, and psychodynamic), as measured by the TPC-FR. Mixed-effects models included random intercepts for organization to account for therapists nested within organizations. Predictor variables included therapist predictors (age; gender, coded as male or female; clinical experience; licensure status, coded as yes or no; EBP initiative participation, coded as yes or no) and childhood predictors (primary diagnosis, coded as externalizing, internalizing, or other; age, coded as early childhood, ages three to seven; middle childhood, coded as eight to 12; or adolescence, ages 13 to 18). These predictor variables were included because treatment recommendations vary somewhat for older versus younger youths and for those with internalizing versus externalizing disorders and because therapists may or may not have participated in training opportunities in EBPs. Missing data were minimal (<10%). Series means were imputed for missing predictor variables. Analyses were conducted with PROC MIXED in SAS 9.0. Effect sizes (Cohen’s f2) were computed with the procedures outlined by Selya and colleagues (36): .02 represents a small effect, .15 a medium effect, and .35 a large effect (37).
Results
Specific therapeutic techniques reported were assessed via the TPC-FR (Table 2 shows model parameters and intraclass correlation coefficients). With regard to therapist-level predictors, therapists who had participated in EBP initiatives were more likely to report use of cognitive (f2=.05) techniques, whereas therapists who were not licensed were more likely to report use of behavioral (f2=.02) and psychodynamic (f2=.07) techniques. With regard to client-level predictors, therapists working with clients with a primary externalizing disorder were more likely than therapists working with youths with primary internalizing diagnoses to report using behavioral (f2=.13) and family (f2=.02) techniques over the course of therapy. Therapists working with early childhood–age youths were more likely than those working with adolescents to report using behavioral (f2=.08) techniques; likewise, therapists working with adolescents were more likely than those working with youths in early childhood to report using cognitive (f2=.05) and psychodynamic (f2=.05) techniques and more likely than those working with youths in middle childhood to report using psychodynamic techniques (f2=.05).
| Variable | Behavioral | Cognitive | Family | Psychodynamic |
|---|---|---|---|---|
| Random effects | ||||
| Organizational variance | .11 | .09 | .19 | .04 |
| Residual variance | .43 | .41 | .68 | .36 |
| Intraclass correlation coefficient | .20 | .18 | .22 | .1 |
| Fixed effects | ||||
| Client primary diagnosis (primary internalizing disorder=0)b | ||||
| Primary externalizing disorder | .66* | .19 | .44* | .16 |
| Primary other disorder | .26 | –.03 | .48 | .08 |
| Client age (adolescence=0)c | ||||
| Early childhood (age 3–7) | .65* | –.52* | –.01 | –.36* |
| Middle childhood (age 8–12) | .22 | –.21 | –.05 | –.31* |
| Therapist factord | ||||
| Age | .01 | .01 | <.01 | .01 |
| Male | –.13 | –.03 | .35 | .15 |
| Clinical experience, years | .02 | –.01 | .03 | –.01 |
| Not licensed | .35* | <–.01 | .36 | .45* |
| No EBP participation | –.15 | –.28* | –.02 | –.07 |
a
N=127 therapists. Each dependent variable is a mean±SD of the items that fit within that factor, measured on a continuum from 1, rarely, to 5, most of the time (behavioral techniques, 2.80±.88; cognitive techniques, 3.64±.75; family techniques, 3.48±.96; psychodynamic techniques, 3.42±.66). Higher scores indicate greater self-reported use of the set of techniques.
b
Client primary diagnosis was dummy coded to indicate whether the primary diagnosis was an externalizing disorder (attention-deficit hyperactivity disorder, oppositional defiant disorder, or conduct disorder), an internalizing disorder (anxiety and depressive disorders or posttraumatic stress disorder), or other disorder (that is, a primary disorder that was not clearly internalizing or externalizing, such as adjustment disorder).
c
Child age was dummy coded to indicate whether the target client was in early childhood, middle childhood, or adolescence (age ≥13).
d
Therapist gender, being licensed, and participating in evidence-based practices (EBPs) were dummy coded. EBP participation indicated number of years the therapist participated in systemwide EBP training initiatives.
*
p<.05
The structure of the analyses required the use of a reference group for dummy-coded variables. Thus report of techniques used could not be compared between externalizing disorders and other disorders in our model. When the reference group was changed from a primary internalizing disorder to a primary other disorder, the results were largely unchanged. Therapists working with clients with a primary externalizing disorder were more likely than therapists working with youths with primary other diagnoses to report using behavioral techniques over the course of therapy (B=.39, p=.04, f2=.13). A significant difference in reported use of family therapy was not observed.
Discussion
This study was the first to examine the relationship between therapist- and client-level predictors of community-based therapists’ self-report of techniques that fall into cognitive, behavioral, psychodynamic, and family domains within the context of implementation efforts. Results indicate that both therapist- and client-level variables predicted self-reported use of therapy techniques. Overall, these findings are largely encouraging and suggest that therapists in this system are self-reporting use of evidence-based usual care practices, such as cognitive and behavioral techniques, in certain circumstances. This may be attributed to the systemwide emphasis on implementation of EBPs; indeed, the finding that therapists participating in initiatives were more likely to report use of cognitive techniques strengthens this interpretation. This study provides preliminary support for the idea that community therapists can self-report ascribing evidence-based techniques to client needs.
Therapist factors, namely therapist licensure status and participation in EBP initiatives, predicted self-reported use of therapy techniques. Specifically, compared with licensed providers, therapists who were unlicensed were more likely to report using behavioral and psychodynamic techniques. Given that we controlled for therapist age and years of experience, it is unlikely that these findings were driven by these factors. Unlicensed providers are supervised by licensed providers, so perhaps supervisors in this system are emphasizing behavioral and psychodynamic techniques with supervisees. Understanding the impact of supervisor support on use of therapy techniques is an important area for future inquiry (38). Therapists who had participated in EBP initiatives were more likely to report use of cognitive techniques. Given that there is a long-standing cognitive therapy training initiative in the system (31), this result is not surprising and suggests a positive impact of this EBP initiative on the system.
That client-level variables also predicted use of therapy techniques is noteworthy. Therapists working with youths with primary externalizing disorders were more likely than those working with youths with primary internalizing diagnoses to report use of behavioral and family techniques. This finding is consistent with the evidence base that suggests that at least some behavioral and family-based techniques are more effective for youths with externalizing disorders (39,40). Also of note, therapists were more likely to report use of cognitive and psychodynamic techniques with adolescents than with younger children (those in early childhood) and more likely to describe using behavioral techniques with this younger group versus adolescents. Cognitive and psychodynamic techniques both involve insight (41), and therapists may be more comfortable using these techniques with older youths.
Although therapists reported using a range of evidence-based cognitive and behavioral therapy techniques, they also reported using non–evidence-based techniques, particularly psychodynamic techniques. This finding is consistent with previous work showing that therapists tend to be eclectic in their approach to therapy and use a variety of therapy techniques (8,42). Deimplementation, or exnovation, refers to an organization divesting itself of a previously adopted innovation (43). This represents an area ripe for future research. Given that unlicensed providers report using psychodynamic techniques more frequently compared with licensed providers, unlicensed providers may be a group worth targeting for exnovation.
These findings are preliminary because we relied on therapist report for two important variables: use of therapy techniques and client’s presenting diagnosis. First, studies on the concordance of therapist self-report with observation are equivocal. For example, a study conducted by Hurlburt and colleagues (44) casts doubt on concordance, but deeper examination of their article suggests that the demand characteristics of the task differed for observers and therapists, calling into question the applicability of these findings to our work. Specifically, therapists reported on their behavior several days after the session, whereas observers scored every minute of treatment sessions. Furthermore, therapists were not trained in how to rate the intensity of their behavior, whereas coders had intensive training and detailed instructions on how to score intensity of behavior. On the other hand, a recent study in which therapists, youths, caregivers, trained raters, and treatment experts rated therapist adherence to a substance abuse treatment protocol found high concordance between self-reported therapist ratings and trained raters (which showed high concordance with treatment expert ratings). In that study, the methodology was more similar to ours in that therapists were asked to reflect on the overall techniques used in treatment over a longer period (one month) (45). Other studies have corroborated this finding (46). Second, therapists reported on their client diagnoses, and we did not independently verify these diagnoses with semistructured interviews; concordance between therapist diagnosis and semistructured interviews is low (47). However, for the particular question at hand, it is more important to capture the diagnosis which therapists report they are targeting in treatment, whether or not the diagnosis is consistent with research criteria. Therefore, if therapists are differentially implementing therapy techniques to client clinical presentation, we would expect to detect that with our methodology.
This study was strengthened by its large and diverse sample of community mental health therapists drawn from a major metropolitan area. Several additional limitations exist. Therapists did not provide information about severity of client’s illness or comorbid conditions, although a recent study of children receiving services though the publicly funded mental health system in Philadelphia suggests that comorbidity was low, as reported in Medicaid claims (specifically, <5%; [48]). Information was obtained about one client on a therapist’s caseload, and treatment duration was not reported. It is not known whether treatment duration was associated with how many intervention techniques therapists used, and future studies would benefit from examining this empirically. Therapists chose which clients to report on. Therapists could have been motivated to report on a client whom they deemed to be a good candidate for EBP techniques; however, we observed a wide range of presenting disorders and ages, suggesting that this may not have been a significant limitation. Finally, data were lacking regarding the TPC-FR’s ability to predict therapist behavior or to distinguish between therapists who use varying levels of specific techniques.
Conclusions
Results suggest that both therapist and client factors predict self-reported use of therapy techniques. Most important, participating in an EBP initiative increased self-reported use of cognitive techniques, suggesting that a systemwide effort to implement EBPs can result in individual therapist report of use of evidence-based techniques. Therapists self-reported increased use of behavioral and family techniques with youths with externalizing disorders and less use of cognitive and psychodynamic techniques with young clients. Future studies would benefit from using observational methods to ascertain the veracity of these findings. In addition, studies examining therapists’ use of EBPs with clients with a range of presenting problems (14) and comorbidity are needed, as are those that examine the role of treatment length and sequencing of EBP techniques in community care.
References
1.
Butler AC, Chapman JE, Forman EM, et al: The empirical status of cognitive-behavioral therapy: a review of meta-analyses. Clinical Psychology Review 26:17–31, 2006
2.
Chorpita B, Daleiden EL, Ebesutani C, et al: Evidence-based treatments for children and adolescents: an updated review of indicators of efficacy and effectiveness. Clinical Psychology: Science and Practice 18:154–172, 2011
3.
New Freedom Commission on Mental Health: Achieving the Promise: Transforming Mental Health Care in America. Final Report. Rockville, Md, US Department of Health and Human Services, 2003
4.
Mental Health: A Report of the Surgeon General. Rockville, Md, US Department of Health and Human Services, US Public Health Service, 1999
5.
Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC, Institute of Medicine, 2001
6.
Stewart RE, Chambless DL: Does psychotherapy research inform treatment decisions in private practice? Journal of Clinical Psychology 63:267–281, 2007
7.
Weissman MM, Verdeli H, Gameroff MJ, et al: National survey of psychotherapy training in psychiatry, psychology, and social work. Archives of General Psychiatry 63:925–934, 2006
8.
Garland AF, Bickman L, Chorpita BF: Change what? Identifying quality improvement targets by investigating usual mental health care. Administration and Policy in Mental Health and Mental Health Services Research 37:15–26, 2010
9.
Garland AF, Haine-Schlagel R, Brookman-Frazee L, et al: Improving community-based mental health care for children: translating knowledge into action. Administration and Policy in Mental Health and Mental Health Services Research 40:6–22, 2013
10.
Goldfried MR: Toward the delineation of therapeutic change principles. American Psychologist 35:991–999, 1980
11.
Brookman-Frazee L, Garland AF, Taylor R, et al: Therapists’ attitudes towards psychotherapeutic strategies in community-based psychotherapy with children with disruptive behavior problems. Administration and Policy in Mental Health and Mental Health Services Research 36:1–12, 2009
12.
Garland AF, Brookman-Frazee L, Hurlburt MS, et al: Mental health care for children with disruptive behavior problems: a view inside therapists’ offices. Psychiatric Services 61:788–795, 2010
13.
Brookman-Frazee L, Haine RA, Baker-Ericzén M, et al: Factors associated with use of evidence-based practice strategies in usual care youth psychotherapy. Administration and Policy in Mental Health and Mental Health Services Research 37:254–269, 2010
14.
Weersing VR, Weisz JR: Community clinic treatment of depressed youth: benchmarking usual care against CBT clinical trials. Journal of Consulting and Clinical Psychology 70:299–310, 2002
15.
Kolko DJ, Cohen JA, Mannarino AP, et al: Community treatment of child sexual abuse: a survey of practitioners in the National Child Traumatic Stress Network. Administration and Policy in Mental Health and Mental Health Services Research 36:37–49, 2009
16.
McLeod BD, Weisz JR: The Therapy Process Observational Coding System for Child Psychotherapy–Strategies Scale. Journal of Clinical Child and Adolescent Psychology 39:436–443, 2010
17.
Southam-Gerow MA, Weisz JR, Chu BC, et al: Does cognitive behavioral therapy for youth anxiety outperform usual care in community clinics? An initial effectiveness test. Journal of the American Academy of Child and Adolescent Psychiatry 49:1043–1052, 2010
18.
Weisz JR, Southam-Gerow MA, Gordis EB, et al: Cognitive-behavioral therapy versus usual clinical care for youth depression: an initial test of transportability to community clinics and clinicians. Journal of Consulting and Clinical Psychology 77:383–396, 2009
19.
Manteuffel B, Stephens RL, Sondheimer DL, et al: Characteristics, service experiences, and outcomes of transition-aged youth in systems of care: programmatic and policy implications. Journal of Behavioral Health Services and Research 35:469–487, 2008
20.
Warren JS, Nelson PL, Mondragon SA, et al: Youth psychotherapy change trajectories and outcomes in usual care: community mental health versus managed care settings. Journal of Consulting and Clinical Psychology 78:144–155, 2010
21.
Weisz JR, Jensen-Doss A, Hawley KM: Evidence-based youth psychotherapies versus usual clinical care: a meta-analysis of direct comparisons. American Psychologist 61:671–689, 2006
22.
Weisz JR, Kuppens S, Eckshtain D, et al: Performance of evidence-based youth psychotherapies compared with usual clinical care: a multilevel meta-analysis. JAMA Psychiatry 70:750–761, 2013
23.
Damschroder LJ, Aron DC, Keith RE, et al: Fostering implementation of health services research findings into practice: a consolodated framework for advancing implementation science. Implementation Science 4:1–15, 2009. Available at
24.
Aarons GA, Hurlburt M, Horwitz SM: Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research 38:4–23, 2011
25.
Beidas RS, Marcus S, Aarons G, et al: Predictors of community therapists’ use of therapy techniques in a large public mental health system. JAMA Pediatrics 169:374–382, 2015
26.
Beidas RS, Aarons G, Barg F, et al: Policy to implementation: evidence-based practice in community mental health—study protocol. Implementation Science 8:38, 2013
27.
Stirman SW, Buchhofer R, McLaulin JB, et al: Public-academic partnerships: the Beck Initiative: a partnership to implement cognitive therapy in a community behavioral health system. Psychiatric Services 60:1302–1304, 2009
28.
Cohen JA, Deblinger E, Mannarino AP, et al: A multisite, randomized controlled trial for children with sexual abuse–related PTSD symptoms. Journal of the American Academy of Child and Adolescent Psychiatry 43:393–402, 2004
29.
Foa EB, McLean CP, Capaldi S, et al: Prolonged exposure vs supportive counseling for sexual abuse-related PTSD in adolescent girls: a randomized clinical trial. JAMA 310:2650–2657, 2013
30.
Linehan MM, Comtois KA, Murray AM, et al: Two-year randomized controlled trial and follow-up of dialectical behavior therapy vs therapy by experts for suicidal behaviors and borderline personality disorder. Archives of General Psychiatry 63:757–766, 2006
31.
Creed TA, Stirman SW, Evans A, et al: A model for implementation of cognitive therapy in community mental health: the Beck Initiative. Behavior Therapist 37:56–64, 2014
32.
Weisz JR: Therapist Background Questionnaire. Los Angeles, University of California Press, 1997
33.
Chorpita BF, Daleiden EL, Weisz JR: Identifying and selecting the common elements of evidence based interventions: a distillation and matching model. Mental Health Services Research 7:5–20, 2005
34.
Weersing VR, Weisz JR, Donenberg GR: Development of the Therapy Procedures Checklist: a therapist-report measure of technique use in child and adolescent treatment. Journal of Clinical Child and Adolescent Psychology 31:168–180, 2002
35.
Do M-C, Warnick E, Weersing VR, et al: Examination of the Psychometric Properties of the Therapy Procedures Checklist–Family Revised. National Harbor, Md, Association for Behavioral and Cognitive Therapies, 2012
36.
Selya AS, Rose JS, Dierker LC, et al: A practical guide to calculating Cohen’s f2, a measure of local effect size, from PROC MIXED. Frontiers in Psychology 3:1–6, 2012
37.
Cohen J: Statistical Power Analysis for the Behavioral Sciences. Hillsdale, NJ, Erlbaum, 1988
38.
Dorsey S, Pullmann MD, Deblinger E, et al: Improving practice in community-based settings: a randomized trial of supervision—study protocol. Implementation Science 8:89, 2013
39.
Chronis AM, Jones HA, Raggi VL: Evidence-based psychosocial treatments for children and adolescents with attention-deficit/hyperactivity disorder. Clinical Psychology Review 26:486–502, 2006
40.
Mattejat F: Evidence-based family therapy: which family-based interventions are empirically supported? [trans from German]. Kindheit und Entwicklung 14:3–11, 2005
41.
Beck AT: Cognitive Therapy and the Emotional Disorders. New York, Meridian, 1979
42.
Beidas RS, Mychailyszyn MP, Edmunds JM, et al: Training school mental health providers to deliver cognitive-behavioral therapy. School Mental Health 4:197–206, 2012
43.
Kimberly JR, Evanisko MJ: Organizational innovation: the influence of individual, organizational, and contextual factors on hospital adoption of technological and administrative innovations. Academy of Management Journal 24:689–713, 1981
44.
Hurlburt MS, Garland AF, Nguyen K, et al: Child and family therapy process: concordance of therapist and observational perspectives. Administration and Policy in Mental Health and Mental Health Services Research 37:230–244, 2010
45.
Chapman JE, McCart MR, Letourneau EJ, et al: Comparison of youth, caregiver, therapist, trained, and treatment expert raters of therapist adherence to a substance abuse treatment protocol. Journal of Consulting and Clinical Psychology 81:674–680, 2013
46.
Hogue A, Dauber S, Henderson CE, et al: Reliability of therapist self-report on treatment targets and focus in family-based intervention. Administration and Policy in Mental Health and Mental Health Services Research 41:697–705, 2014
47.
Lewczyk CM, Garland AF, Hurlburt MS, et al: Comparing DISC-IV and clinician diagnoses among youths receiving public mental health services. Journal of the American Academy of Child and Adolescent Psychiatry 42:349–356, 2003
48.
Kang-Yi CD, Locke J, Marcus SC, et al: School-based behavioral health service use and expenditures for children with autism and children with other disorders. Psychiatric Services 67:101–106, 2016
Information & Authors
Information
Published In
Cover: Assinibone hand drum, by Werner Forman. From the Plains Indian Museum, BBHC, Cody, Wyoming. Photo credit: HIP/Art Resource, New York City.
History
Received: 13 January 2015
Revision received: 28 May 2015
Revision received: 22 July 2015
Accepted: 28 August 2015
Published online: 14 February 2016
Published in print: May 01, 2016
Authors
Funding Information
National Institute of Mental Health10.13039/100000025: MH099179, MH103955
This study was supported by grants MH103955 (Dr. Benjamin Wolk) and MH099179 (Dr. Beidas) from the National Institute of Mental Health. In addition, the preparation of this article was supported in part by the Implementation Research Institute at the George Warren Brown School of Social Work, Washington University in St. Louis, and through award R25 MH080916 from the National Institute of Mental Health and the Quality Enhancement Research Initiative (QUERI), Department of Veterans Affairs contract, Veterans Health Administration, Office of Research and Development, Health Services Research and Development Service. Dr. Beidas was supported through an IRI fellowship (2012–2014). Dr. Benjamin Wolk is a fellow and Dr. Marcus and Dr. Beidas are senior fellows of the Leonard Davis Institute of Health Economics, University of Pennsylvania.Dr. Marcus has received grant support from Alkermes, Forrest, Ortho-McNeil Janssen, Shire, and Sunovion. Dr. Beidas receives royalties from Oxford University Press and has served as a consultant for Kinark Child and Family Services. The other authors report no financial relationships with commercial interests.Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBLogin options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).