Evidence-Based Practice in Child and Adolescent Mental Health Services

First, although to be human is to develop (in the French sense of de-enveloper or un-envelop), children undergo more rapid physiological, neuronal, and psychological changes over a briefer period than adults. The rapidity of this development implies that for evidence-based practice to be meaningful, it has to take into account developmental conditions that affect the durability of the effects of treatment. An evidence-based practice that is effective in the treatment of adolescent depression may well be ineffective or even harmful for children who have not reached puberty.

Attention to developmental increments in the creation of evidence-based practices for children and adolescents means attention not only to age-related changes but also to the complex and dynamic interactions among the child, the family, and the environmental context that accompany maturation (19). In the field of child and adolescent mental health, answers to questions about the evidence base for services are not meaningful if developmental issues have been ignored.

Second, the creation of a treatment for a child is rarely undertaken without consideration of the family context. In fact, some have argued that even the notion of what constitutes a mental illness cannot be ascertained without knowledge of the interaction between the child and his or her family (20,21). Beyond this interaction, parental perceptions of the nature of presenting problems differ substantially from the child's perceptions (22). The nature of the diagnosis itself is contextually bound for children and adolescents to a far greater degree than is true for adults (23). Although involvement of caregivers is important in implementation of evidence-based practices for adults, in child mental health research the family is central not only to the development of the treatment or service but also to the understanding of the diagnosis itself.

Third, evidence-based practices for children differ from those for adults in that the types of services for which an evidence-based practice is developed will necessarily involve substantially different service venues. For example, 70 to 80 percent of the mental health services received by children who have mental health problems are provided by schools (24). The adult analog is likely to be the workplace. Yet an evidence-based practice for treating, say, ADHD in a school setting—for example, classroom management by a teacher—is unlikely to be similar to an evidence-based practice developed for treating inattention in the workplace. Issues related to the context or setting of the service place very different demands on the provider of the treatment and on the recipient.

Because of these contextual differences, a wide range of providers will need to be trained to provide the evidence-based practice. Children who have mental health needs may come to the attention of professionals in schools, primary care offices, welfare systems, or detention facilities. The fragmentation of the mental health service system means that for evidence-based practice to reach those who provide care to children, a range of training curricula, materials, and approaches must be developed and specifically tailored for the providers in these systems.

Until recently, there has been no evidence to summarize, critique, or review in the field of child and adolescent services. In fact, from a historical standpoint it is interesting that the concept of childhood mental illnesses did not arise until the late 19th century. These illnesses were typically not seen as unique to children or distinguishable from adult mental illnesses until the early part of the 20th century. The first English-language text on child psychiatry was published in 1935 (25). The first serious attempts to assess the use of mental health services by children and adolescents were begun in the late 1980s.

Two factors galvanized developments in children's mental health services: recognition that these services were scattered across a vast array of service organizations and systems, including schools, child welfare agencies, pediatric health settings, and juvenile corrections facilities, and recognition that fewer than 20 percent of children who had identified mental health needs received help (24).

The first development was state-level activities to create coordinated points of entry for delivery of mental health services, organized largely under the auspices of the Child and Adolescent Service System Program. This initiative was given principled footing through the creation of a model for revamping children's services. The system-of-care model, developed by Stroul and Friedman (26), articulated a series of values and principles centered on maintaining children in their communities, coordinating services, involving families centrally in delivery and planning of treatments and services, and instantiating attention to the cultural relevance of services. After these principles were developed, the services included in a continuum of care were delineated.

The second development was the creation of a scientific agenda centered on examining the relationship between children's needs for psychiatric care and the availability of such care. The two major efforts in this direction were the Great Smoky Mountains study (27,28) and the study of methods for the epidemiology of children and adolescents (29,30). Both studies found that 4 to 8 percent of children between the ages of nine and 17 years had severe psychiatric disorders and that only about 20 percent of children with the most serious needs were receiving mental health services (24,27,29,31).

During the mid-1990s, the scientific research agenda redirected attention to the quality of the clinical treatments within service systems. The system-of-care studies by Bickman (32,33) showed that system coordination alone improved access to services for children and families and satisfaction with services and also reduced hospitalization and other restrictive forms of care. However, these studies also showed that clinical outcomes for children—for example, alleviation of symptoms, functioning, or reduction of impairments—were the same whether children were receiving coordinated services through systems of care or were receiving usual services.

As a result of these findings, attention was shifted away from general studies of "systemness" to the clinical effectiveness of services within these systems of care and especially to the types, dosages, and intensity of treatments delivered (34,35,36). In particular, the transportability of efficacious clinical treatments into mental health service systems was highlighted as a critical research area (37,38,39). To this end, Burns (40) proposed the creation of a research agenda on clinical interventions for youths that would accomplish four tasks: synthesize, through reviews of the evidence base, the status of science on promising interventions; assess the adequacy of quality indicators to improve standards of clinical practice; evaluate the adequacy of outcome measures; and develop a new research phase model for connecting research to practice. The latter activity was subsequently proposed by Weisz and Weersing (18) as well as by Hoagwood and colleagues (41).

As a result of this new focus on bridging science and policy and strengthening knowledge about the efficacy and effectiveness of mental health services (42), a series of studies has been undertaken by a number of investigators, focusing on questions about the effectiveness of manual-based services for children who have a range of serious psychiatric impairments and the effectiveness of practice strategies for enhancing engagement with services.

Acceptability of engagement and treatment. Treatments that fail to reach those who stand to benefit from them cannot be said to be effective. Unfortunately, inaccessibility of services and termination of treatment are problems that plague the delivery of mental health care. Among children whose families do seek outpatient mental health treatment, 40 to 60 percent may discontinue services before formal completion of treatment (43). Moreover, these families typically do not use outpatient services for very long. Armbruster and Fallon (44) showed that most children who enter outpatient treatment attend for only one or two sessions. There is also evidence that children from especially vulnerable populations—children of single mothers, children living in poverty, and children from minority groups—and children who have serious presenting problems are less likely to stay in treatment beyond the first session and more likely to discontinue treatment prematurely (45,46).

However, successful efforts can be made to enhance a family's service engagement and to decrease rates of premature termination of treatment. A variety of studies have been undertaken to identify and reduce barriers to service engagement and to increase the participation of minority families in services by using brief telephone interviews before service entry (47,48,49).

Empirically supported psychosocial outpatient treatments. Meta-analyses of experimental child psychotherapy intervention trials point to a consistent beneficial effect of treatment compared with no treatment (11,50,51,52). These effects are comparable to those found for adult psychotherapy (10). A similar analysis that summarized a less extensive literature on the treatment effects of therapies conducted in clinical practice settings, as opposed to research settings, found almost no difference between treatment and no treatment (12). In fact, the effect size was negative, falling well below the effects typically found in experimental studies. Consequently, the evidence suggests that psychosocial interventions for children can successfully reduce symptoms associated with childhood mental disorders when conducted in research-based settings; however, the impact of these therapies in clinical practice settings is only now being studied (18).

Family-focused treatments. Because the family plays a major role in the social and emotional development of children, family-focused interventions have long been a part of child and adolescent mental health treatment. Meta-analysis of family-focused treatments shows the general effectiveness of such treatments (19). Controlled trials indicate the effectiveness of family-based interventions for physical child abuse and neglect; conduct problems, including ADHD; emotional disturbance, specifically anxiety, depression, and grief; toileting problems; and psychosomatic concerns (20). Effective family-based treatments are typically short-term, are offered on an outpatient basis, and have cognitive-behavioral, structural, or strategic foundations. Frequently they are combined with individual therapies and medication management.

Integrated community-based treatment. One criticism of empirically supported psychosocial interventions is that their focus on specific diagnosable disorders does not adequately take account of the heterogeneity of the psychiatric problems that a majority of children have when they present at mental health clinics (53). A series of studies on integrated service models for children who have multiple co-occurring disorders has been examining a range of service modalities, including intensive case management, treatment foster care, and home-based services.

Studies of clinically oriented, intensive case management have found that children who have specially trained case managers require fewer restrictive services, such as psychiatric hospitalizations, than children who do not (54,55). Similar reductions in the number of inpatient hospitalizations have been found for youths with substance use problems (55). The use of case managers in community-based interdisciplinary treatment teams has also been found to improve standard-practice foster care through reductions in the number of placement changes and the number of runaway episodes among older youths (56). Multiple uncontrolled studies of case management services have been conducted in the context of wraparound care (57). These studies have shown that case management services can improve children's positive adjustment, decrease negative behaviors, and improve the stability of community living environments.

A series of studies have tested the impact of therapeutic foster care services for children who have multiple comorbid mental disorders. In a therapeutic foster care environment, a child is placed in a home with foster parents who have received specialized training to work with children who have emotional or behavioral problems. Results from these studies have shown decreases in aggressive behavior and increases in positive adjustment at the conclusion of placement (56,57,58). Chamberlain and Reid (59) compared treatment outcomes for youths from a state psychiatric hospital who were placed in either therapeutic foster care or usual community care and found that those in therapeutic foster care had fewer reinstitutionalizations and more rapid behavioral improvement. In addition, youths in the experimental group had less frequent posttreatment incarcerations and criminal referrals and more frequent placements with parents or relatives in the year after treatment (60). In addition, the costs of this service were significantly lower than those of other residential placements (60).

Finally, home-based service models have been developed for children who have serious emotional disturbances. One rigorously studied home-based intervention is multisystemic therapy, the primary goal of which is to develop independent skills among youths who have behavioral problems and their parents to cope with family, peer, school, and neighborhood problems through brief (three to four months) and intense (sometimes daily) treatment (61). Treatment strategies integrate empirically based treatment approaches—for example, behavioral training for parents, cognitive-behavioral therapies, and functional family therapy—to address the problems of children and adolescents across environmental contexts.

Eight randomized trials of multisystemic therapy have been conducted, and the results have been among the strongest found for children's services. Among a group of chronic juvenile offenders, those who received multisystemic therapy had lower rates of recidivism and out-of-home placements 59 weeks after treatment and lower arrest rates more than two years after treatment (62). Similar results were found when multisystemic therapy was compared with individual therapy in a different group of juvenile offenders (63).

A recent study comparing multisystemic therapy with emergency psychiatric hospitalization among children and adolescents with serious psychiatric impairments has found that multisystemic therapy can safely reduce rates of psychiatric hospitalization and improve the functioning of youths and their families (64,65). The effects of multisystemic therapy have been further demonstrated among juvenile sex offenders (66) and abused or neglected children (67). Researchers evaluating multisystemic therapy suggest that adequate supervision, training of therapists, and institutional program support are essential to successful outcomes (65).

School-based interventions. Seventy to 80 percent of children who receive mental health services receive them in school; for many children the school system provides their only form of mental health treatment (24). About 45 studies of school-based interventions for children with emotional or behavioral problems, covering a 15-year period, were recently reviewed (15). Among these studies, a range of effective individual, classroom, and targeted interventions were identified.

The empirically supported treatments for childhood behavioral problems that are effective in school settings include targeted classroom-based contingency management for children with a diagnosis of ADHD (4) and children with other conduct problems (5). Contingency management also appears to successfully reduce aggression when implemented across entire classrooms. The "good-behavior game," a classroom-based behavior-management strategy for first-grade students, has demonstrated long-term benefits in reducing disruptive behaviors in middle school (68,69). Behavioral consultation to teachers to help them accommodate difficult students has been found to reduce the number of special-education referrals and placements and to reduce teachers' reports of students' behavioral problems (70).

School-based preventive interventions designed to target children who are at risk of emotional or behavioral problems have also been shown to alleviate symptoms and to increase the use of positive coping strategies. Cognitive group interventions to modify adolescents' depressive thinking styles have been associated with a reduced risk of the development of full depression (71). Similarly, a group intervention to teach social problem-solving skills to elementary school children with elevated depressive symptoms demonstrated reductions in reported depression, even one year after the intervention (72). There is also evidence that school-based preventive interventions reduce the risk of conduct problems. Successful interventions typically involve multiple components that target classroom, home, and peer environments (73).

Psychopharmacology. About 3.5 million child outpatient physician visits a year result in a prescription for a psychotropic medication (74). Furthermore, prescription rates for psychotropic medications for children are increasing, even among very young children (75). Clinical trials of psychotropic medications are needed for many childhood mental disorders. Despite the widespread use of these medications, surprisingly few randomized controlled studies have been conducted (16). Many medication choices and algorithms continue to be based on the experience of the individual practitioner or on standards of care for adults, with some notable exceptions (76).

In addition, proper medication selection is only one factor in successful pharmacologic treatment. The recent National Institute of Mental Health multimodal treatment study of ADHD found that clinical outcomes for participants who received careful medication management, including systematic titration to the optimal dosage, were superior to clinical outcomes associated with routine prescription of the same medication delivered in the usual manner in the community (77). The differences in outcomes seem to have been a result of management practices associated with participation in the medication treatment group.

Weisz and Jensen (16) recently reviewed evidence of the efficacy of child pharmacotherapy by using criteria established for the International Psychopharmacology Algorithm Project (78). According to these criteria, a drug is considered efficacious if its equivalency or superiority have been demonstrated through random-assignment, control group comparison and the results are replicated in one or more similarly well-controlled studies. In addition, the National Institute of Mental Health recently commissioned six scientific reviews of published studies of the safety and efficacy of psychotropic medications for children (79): psychostimulants (80), mood stabilizers and antimanic agents (81), selective serotonin reuptake inhibitors (SSRIs) (82), tricyclic antidepressants (83), antipsychotic agents (84), and miscellaneous agents (85). These reviews identified several psychotropic medications for which there is empirical support for use in both externalizing and internalizing disorders in childhood, most prominent among which are psychostimulants for children with ADHD.

For depression, the most commonly studied pharmacologic agents have been the tricyclic antidepressants, but they have not been shown to be effective in treating childhood depression. Newer and safer agents, such as SSRIs, are being studied more frequently in the treatment of childhood depression. In the largest study of the use of antidepressants among children, Emslie and colleagues (86) compared fluoxetine with placebo and found that more than half of the fluoxetine group significantly benefited from treatment, whereas only 33 percent of the placebo group showed benefit.

In the only multisite controlled trial of SSRIs for childhood anxiety disorders, fluvoxamine was found to be superior to placebo in treating children with a diagnosis of social phobia, separation anxiety, or generalized anxiety disorder (87). In addition, several pharmacologic agents have been shown to be efficacious in the treatment of children or adolescents with a diagnosis of obsessive-compulsive disorder; these include SSRIs and tricyclic antidepressants (82,83). There is also evidence supporting the usefulness of antipsychotic medications for schizophrenia with onset in childhood or adolescence; however, data on long-term effectiveness and safety are lacking (16).

Unfortunately, despite the fact that in clinical practice it is common for medication treatments to be combined with psychosocial strategies, the literature on the impact of combined treatments for disorders other than ADHD is sparse. A large multisite clinical trial of the efficacy of combined treatments for adolescent depression is under way and is expected to provide knowledge about treatment options in this area.

Potentially ineffective treatments. Recent efforts to identify empirically supported treatments for children have focused largely on the accumulation of supportive findings without an established procedure for dealing with null or even negative results (88). It is, in fact, just as important to identify treatments for which empirical studies consistently fail to show an effect on symptoms or even show worse outcomes for participants in the treatment group. Youth Violence: A Report of the Surgeon General (89) suggests that many of the services provided to delinquent juveniles have little or no evidence base.

Worse yet, a recent study indicated that peer-based, group-based interventions may actually increase behavioral problems among high-risk adolescents (90). For children who have disruptive-behavior disorders, there is no empirical justification for the use of nonbehavioral psychotherapies (91). For example, Pelham and colleagues (4), in their review of effective treatments for ADHD, found no empirical studies that tested the efficacy of many psychosocial treatments commonly used for ADHD, such as individual therapy and play therapy. In addition, although controlled treatment outcome studies have been conducted for cognitive therapy—for example, self-instruction, self-monitoring, and self-reinforcement—to treat children with a diagnosis of ADHD, these studies generally show no clinical or academic benefits from the treatment (92).

A third example of widely used but empirically unjustified services is institutional care—for example, hospitals, residential treatment centers, and group homes (13). Studies have shown that children who are placed in group homes do not maintain improvements once they return to the community (93). Yet group homes continue to be used in community practice.

A general conclusion from this review of evidence-based practices is that the literature on the efficacy of a range of child and adolescent treatments is uneven, although it is gaining strength for particular clinical syndromes. However, the evidence for the effectiveness of either clinical treatments or services within practice settings as opposed to research settings is still weak. Improving the evidence will require attention to service variables that tend to be neglected in most efficacy-based studies.

Although the public health goal of developing a strong research base on effective services is to improve routine care, the implementation of research-based practice is not automatic. Rather, it requires adaptation of research design and methods to practice-related exigencies, as well as accommodation of practice settings to the incorporation of evidence-based practice. Other papers in this series have identified barriers to implementation of evidence-based practice. However, in the child and adolescent field, it is prudent to ask a different and prior set of questions. These questions have to do with whether empirically validated knowledge about treatments and services is ready to be implemented and, if not, why not. To think about these kinds of questions leads us to consider the models that typically guide development of evidence-based treatments and services and the extent to which implementation is justified given these models.

A presupposition of the evidence base is that its development has taken into account the fit between the treatment and the context of delivery. In fact, this fit has been attended to only rarely. One reason that efficacy studies, which constitute a significant portion of the evidence base in children's mental health, have not been readily deployed in service settings may be that the theory, methods, and models used to develop, refine, and test those treatments do not mesh well with the exigencies of clinic-based or community-based care (18).

The culture of psychological science as brought to bear on questions of the efficacy of treatments for children has typically involved conducting studies in controlled and somewhat rarefied environments, such as university laboratories. Over the past 40 years many controlled clinical trials and within-group studies have been published on the impact and efficacy of psychosocial treatments. Specific treatments have been identified for about two dozen clinical conditions in children. These studies have typically been conducted in or in close connection with university laboratories. Studies of conventional treatments delivered in clinics and clinical programs have demonstrated much weaker effects (94).

There has been an implicit assumption that once the laboratory studies of the efficacy of treatments have been completed, the results will be usable and relevant outside the laboratories. However, as has been noted repeatedly (13,18,94), the conditions under which most research is conducted differ in numerous ways from those under which everyday treatment is delivered. Such differences imply that treatments that have been developed through efficacy trials need to be adapted to clinics, schools, or other service settings. However, the differences also imply the opposite, namely, that the service settings or service practices themselves—that is, where, when, and how treatments are delivered—may have to adapt to accommodate delivery of evidence-based practices.

The model that has guided treatment development entails a series of controlled laboratory trials that focus on the efficacy of the treatment. Issues related to the effectiveness, dissemination, implementation, and deployment of treatments take place at the end of the testing process. This model tends to control experimentally the very "nuisance variables" that may need to be understood if treatments are to fit within clinic or community settings (18). These variables—such as comorbidity, parental substance abuse or pathology, life stresses that lead to early termination of treatment, reimbursement structures, service availability, and parental self-efficacy—may make or break the successful adoption of an evidence-based practice in a new practice setting.

Unfortunately, the development of the evidence base has rarely attended to such nuisance variables. Consequently, implementation of many treatments in clinic or community settings may be premature unless such factors are built into the long-term design and cumulative construction of new treatments and services. For example, studies of the development of combined treatments for children with depression who are first seen by primary care physicians should perhaps take into account service linkages with family practice physicians to treat maternal depression. Organizational variables, such as care management, that constrain or facilitate the ability of physicians to communicate with teachers about the impact of treatment on children should be taken into account at the front end of the treatment development cycle.

A second presupposition underlying evidence-based practice is that diffusion of the evidence base will be automatic once the strength of the evidence is ascertained. In fact, diffusion of innovative practice constitutes a researchable set of questions on its own. A major objective of most diffusion studies is to determine whether the practice is adopted as it was designed or adapted, whether the practice is sustained over time, and what factors influence sustainability (95). Diffusion is only beginning to be studied in the context of the mental health treatment of either children or adults. The literature on diffusion identifies individual and contextual factors that are potentially relevant to the effective diffusion of innovation in general, but not the specific factors most likely to predict adoption and implementation of a particular innovation, nor of a particular mental health treatment or service. Thus the evidence base needed to guide successful dissemination of effective treatments has yet to be developed (96).

The central problem is that treatments that have been validated in efficacy studies cannot be assumed to be effective when implemented under routine practice conditions. For example, the use of treatment manuals, special training for clinicians, and continual clinical monitoring to ensure treatment fidelity are characteristics of many research-based interventions but few community-based treatment practices. On the other hand, community treatment is characterized by heterogeneous populations and high caseloads, which are not features of most research-based studies (91). To enable a better fit between evidence-based treatments and community practice contexts will most likely require modifications of both the way in which evidence-based practice is created and modifications in the service delivery settings. The variables at the interface that enable a better match have been described as issues of transportability (96). Features of research protocols and practice contexts that require modification—for example, the training and background of the practitioner, the setting, the organizational culture or context, and financing—and the kinds of modification required are variables that may well influence the portability of evidence-based practices and the adaptability of practice settings (18,41,96,97).

For example, transportability issues have been examined in studies of the promotion of physicians' use of new medications, devices, or procedures. Because most medical technologies must be approved by the FDA before being marketed, the efficacy of the technologies is not in question. However, questions about the fidelity of practitioners' implementation and how it may influence the effectiveness of the intervention cannot be assumed to have been studied. Although some studies of the impact of continuing education, academic detailing, training plus follow-up procedures, and hybrid strategies on physicians' implementation of new technologies have been conducted (98), their application to the dissemination of evidence-based practices in children's services is only now being studied (41), and these practices may well require adaptation.

Attention to the fit will require modification along a variety of dimensions, which differentiate research-based treatments from community practice. The dimensions include the intervention itself; the practitioners who deliver the intervention, including their clinical training, support, and monitoring; the client population (homogeneity or heterogeneity of syndromes); the characteristics of service delivery, such as the setting and the types of services available beyond the intervention; the organizational ethos—for example, the culture or climate in which practitioners provide services, which influences motivation, attitudes, and morale; and the service system, including referral and reimbursement mechanisms and interagency relations (96). The research base that constitutes evidence-based practice for child and adolescent mental health interventions has not typically assessed these dimensions of routine practice.

Finally, factors that predict successful dissemination of evidence-based practices may overlap with those that predict effective services, may be identical to them, or may be entirely different. For example, the organizational climate was found to be a strong predictor of psychosocial outcomes among children receiving casework services in a child welfare agency, and it appears to exert its influence through the motivational attitudes of casework therapists toward their work (99). Is organizational climate an important ingredient of uptake and dissemination? The answer to this question is not yet known. If climate is associated with interpersonal variables that predict the adoption of innovation, then manipulation of climate will be important to the eventual implementation of evidence-based practices. However, if the variables that predict uptake are more closely related to systemic or interpersonal dimensions rather than to organizational dimensions, then climate may not be the active agent for implementation.