Evidence-Based Assessment in Routine Mental Health Services for Youths

All procedures were approved by the University of Missouri’s institutional review board. We provided consent information in a cover letter. Participants consented by returning the survey. For the national survey (2007–2008), we randomly selected 1,000 clinicians each from the American Psychological Association, American Academy of Child and Adolescent Psychiatry, American Counseling Association, National Association of Social Workers, and American Association of Marriage and Family Therapy membership rosters. For the statewide survey (2008), we identified 3,084 clinicians from a public listing of health service clinicians who had billed Medicaid the previous year. Clinicians received up to five mailings. The national survey’s adjusted response rate (undeliverable N=347) was 62% (N=2,863); 1,520 who provided youth assessments were included. The statewide survey’s adjusted response rate (undeliverable N=364) was 50% (N=1,348), and 1,055 were included.

We developed both surveys by using the tailored design method (13). Clinicians reported their professional characteristics: discipline, year of highest degree, percentage of youths and clients with low income and from racial-ethnic minority groups, employment setting, and percentage of Medicaid-reimbursed cases (statewide survey only). Clinicians reported their primary theoretical orientation with a recent, representative case of anxiety, depression, or disruptive behavior among youths. For assessment practices, in both surveys we asked, “How often do you conduct an assessment of problems or strengths during the following points of treatment? At intake or the beginning of treatment, ongoing throughout treatment (e.g., weekly, monthly), at or near the last session, or well after the last session (e.g., weeks, months later)?” In the national survey, we asked, “How often do you use the following specific assessment procedures when treating children and adolescents? Standardized checklists for child/family symptoms or functioning (paper-and-pencil measure) completed by child, parent, teacher, or others with a standardized scoring (e.g., Child Behavior Checklist [CBCL])?” In the statewide survey, we asked, “How often do you use standardized assessment measures with the children/families you treat (e.g., Child Behavior Checklist [CBCL], Connors)?” All responses used a 5-point scale: 1, never or almost never; 2, rarely; 3, sometimes; 4, often; and 5, all or most of the time. The national survey also included the Attitudes Toward Standardized Assessment (ASA [2]), a 22-item measure of attitudes toward standardized measures with three subscales (i.e., benefit over clinical judgment, psychometric quality, or practicality); higher scores on the ASA indicate more positive attitudes.

We first present descriptive information (mean±SD) for assessments at four time points (pretreatment, ongoing treatment, posttreatment, and follow-up). We conducted one-way repeated-measures analysis of variance with a Greenhouse-Geisser correction to compare means across the four time points and a Bonferroni adjustment of p=0.008 for pairwise comparisons. We also present the percentage of clinicians who conducted assessments “often” or “all or most of the time” at each time point and throughout the treatment process (i.e., pretreatment, ongoing, and posttreatment). Second, we present descriptive statistics and the percentage of clinicians who used standardized measures “often” or “all or most of the time” with youths. Third, we conducted a multiple regression analysis predicting standardized measure use by professional discipline (0, not psychology; 1, psychology); percentage of clients from racial-ethnic minorities, with low income, or with Medicaid (statewide survey only); case-mix age (0, primarily [≥50%] youths; 1, primarily adults); private practice setting (0, not private practice; 1, private practice); learning theory orientation (0, other; 1, CBT); and ASA subscale scores (national survey only).

In the national survey (N=1,520), the mean±SD age of clinicians was 52.7±10.0 years; they had 20.4±10.3 years of clinical experience, and most were women (N=958, 63%). The clinicians were Caucasian (N=1,379, 91%), African American (N=39, 5%), Asian/Pacific Islander (N=39, 5%), Latinx (N=43, 3%), other (N=19, 1%), and Native American (N=10, 1%). Most were in private practice (N=922, 61%). A third (N=482, 32%) endorsed learning theory orientation, and more than half had doctorate degrees (N=831, 55%). Their disciplines were psychology (N=482, 32%), counseling (N=357, 23%), marriage and family therapy (N=329, 22%), social work (N=323, 21%), and psychiatry (N=294, 19%). On average, clinicians reported that 32%±28% of their clients belonged to racial-ethnic minority groups and that 35%±32% of the clients had low income. Clinicians reported often conducting assessments pretreatment (4.8±0.6), during treatment (4.2±0.9), and posttreatment (4.0±1.1) and were rarely doing follow-up (2.0±1.2) (F=2,311.44, df=2.60, 3,025.57, p<0.001; all pairwise comparisons p<0.001). Most clinicians conducted pretreatment (N=1,418, 93%), ongoing treatment (N=1,165, 77%), and posttreatment (N=1,032, 68%) assessments often or all or most of the time, and 11% (N=171) used follow-up assessments often or all or most of the time. About half (N=749, 49%) conducted assessments often or all or most of the time throughout the entire treatment process. Clinicians used standardized measures sometimes (3.2±1.4), and 48% (N=726) used them often or all or most of the time. Having more clients from racial-ethnic minority groups (β=0.13, p=0.002) and higher ASA practicality scores (β=0.37, p<0.001) predicted more frequent use of standardized measures, and serving primarily adult clients (β=−0.11, p=0.003) predicted less frequent use of these measures (Table 1).

In the statewide survey (N=1,055), clinician age was 47.5±11.6 years; they had 13.9±9.4 years of clinical experience, and most were women (N=749, 71%). Clinicians were Caucasian (N=944, 90%), African American (N=71, 7%), other (N=11, 1%), Latinx (N=10, 1%), Native American (N=9, 1%), and Asian/Pacific Islander (N=6, 1%). More than a third (N=410, 39%) endorsed learning theory orientation. Most had a master’s degree (N=763, 72%), and most were in private practice (N=651, 59%). Disciplines included counseling (N=533, 51%), social work (N=374, 35%), psychology (N=282, 27%), marriage and family therapy (N=156, 15%), and psychiatry (N=8, 1%). On average, the clinicians reported that of their clients 28%±30% belonged to racial-ethnic minority groups, 32%±24% had low income, and 53%±35% were Medicaid insured. Clinicians conducted pretreatment (4.80±0.55), ongoing treatment (4.1±1.0), and posttreatment (4.1±1.2) assessments often and follow-up (1.9±1.0) never or almost never (F=2,507.99, df=2.75, 2,776.31, p<0.001). Pretreatment assessment was more frequent and follow-up assessment less frequent than assessments at the other two time points (p<0.001). Most clinicians conducted pretreatment (N=1,005, 95%), ongoing treatment (N=811, 77%), and posttreatment (N=768, 73%) assessments “often” or all or most of the time; 7% (N=78) conducted assessments at follow-up. More than half (N=582, 55%) conducted assessments often or all or most the time throughout the treatment period. On average, clinicians used standardized measures sometimes (3.1±1.3), and 38% (N=404) used them often or all or most of the time. Psychology discipline (β=0.27, p<0.001) and having more clients from racial-ethnic minority groups (β=0.12, p<0.001) predicted more frequent standardized measure use, and serving primarily adults (β=−0.11, p=0.001), being in a private practice setting (β=−0.10, p=0.002), and serving more Medicaid clients (β=−0.11, p=0.002) predicted less frequent use of these measures (see Table 1).

Using two large multidisciplinary surveys, we examined when, how, and under what conditions clinicians involved in usual care conduct EBAs with youths. On average, clinicians reported often assessing before, during, and after treatment. Consistent with EBA, approximately half of the clinicians routinely conducted assessments throughout the three treatment stages. However, consistent with previous studies (5, 7–9), the clinicians used standardized measures only sometimes, and less than half routinely used standardized measures. This discrepancy suggests that assessment most often involves informal unstandardized methods, and EBA training initiatives are needed to support the routine use of standardized measures among clinicians.

These results point to two drivers of standardized measure use that may inform future EBA initiatives: practical measures and clinician training. In line with previous studies in which clinicians cited lack of practicality as a primary barrier to standardized measure use (7, 10), ASA practicality was the greatest predictor of standardized measure use in the national survey, suggesting that practical measures may support routine use of standardized measures. In the statewide survey, a greater percentage of Medicaid cases was associated with less frequent standardized measure use. The state survey had no mandated assessment requirements (this finding may not hold if EBA is required for Medicaid billing), and our finding may largely reflect that many clinicians in this survey may work in low-resource settings. As such, EBA initiatives should consider free or low-cost and brief measures that can be easily integrated into universal care.

Training to support standardized measure use throughout the treatment process and with specific populations is also needed. Consistent with previous surveys (9), psychology discipline predicted increased use of standardized measures. If greater assessment training in psychology graduate programs is driving these results, EBA training for all disciplines may yield improved attitudes, comfort with, and use of EBA measures. Private practice setting was also associated with less frequent standardized measure use. Given that clinicians in private practice may have less favorable views of standardized measures (7), training that increases positive attitudes toward standardized measures may increase their use in private practice. Consistent with another survey (8), clinicians serving primarily adult clients used standardized measures less frequently with youths. Having a higher proportion of Medicaid clients was also associated with less frequent use of standardized measures; this was contrary to our hypothesis but consistent with recent findings of a quality improvement project (14). Hence, addressing issues specific to assessments (e.g., multi-informant reports) of youths and increasing familiarity with standardized measures for youths from diverse backgrounds both seem important. Of note, contrary to past research (12), having a greater proportion of youths from racial-ethnic minority groups predicted greater standardized measure use in both samples; perhaps clinicians serving more racial-ethnic minority youths are using EBA more often because of the smaller body of literature on treatment of youths from racial-ethnic minority groups (15).

This study had some limitations. Clinicians may have overreported use of EBAs because of social desirability or recall bias. Clinicians who conduct more EBAs may have also been more likely to respond, although our response rates were comparable to or higher than those in previous surveys (5–9), and we included two samples to increase representativeness (i.e., the Medicaid sample was recruited because guild members may have more resources or be more current on standards than nonguild members). Given the timing of these surveys and the push for greater EBA use in the past decade, standardized measure use may now be higher. A multidisciplinary survey is needed to evaluate current assessment practices. Nevertheless, our results provide a useful data point for understanding assessment practices as they evolve. Our surveys were administered as routine progress monitoring was being recognized as an evidence-based practice and before any large-scale EBA initiatives. As such, our findings may provide a benchmark for gauging the success of EBA initiatives.