Responses of Children With Different Intake Profiles to Mental Health Treatment
Abstract
OBJECTIVE: The study examined the relative change in functioning of children with serious emotional disturbances served in a system of care who were referred from a variety of sources and who had different intake profiles, including diagnoses, demographic characteristics, and levels of risk. METHODS: A total of 203 children who had received services for at least six months from the East Baltimore mental health partnership were included in the study. The children's functioning was assessed using the Child and Adolescent Functioning Assessment Scale (CAFAS) at both baseline and six months. Univariate analyses were used to compare demographic and functioning characteristics at baseline of children referred from different sources. Repeated-measures general linear analysis was conducted to determine differences in changes in functioning by demographic and clinical characteristics as well as referral source. RESULTS: On average, children demonstrated significant functional improvement from baseline to six months, regardless of referral source, presenting problem, risk factors, age, or gender. The amount or proportion of change did not differ significantly as a function of these factors. CONCLUSIONS: The results indicate that children in this system of care referred from different sources with different diagnoses, demographic profiles, and levels of risk had similar rates of improvement during the first six months of services.
Many children with serious emotional disturbances are being served by agencies whose primary responsibilities do not include the provision of mental health services (1). To meet the needs of these children, multidisciplinary, multiagency, child-centered, family-focused, community-based, culturally competent systems of care have been developed. As defined by Stroul and Friedman (2), a system of care is "a comprehensive spectrum of mental health and other necessary services which are organized into a coordinated network to meet the multiple and changing needs of children and adolescents with severe emotional disturbances and their families."
Many communities in the United States have developed extensive collaborations for children with serious emotional disturbances and their families that link mental health services, such as prevention, outpatient treatment, home-based care, emergency services, therapeutic foster care, and inpatient care, with services and support from other agencies and systems, such as education, juvenile justice, social welfare, health care, and substance abuse treatment (3).
In 1993 the child, adolescent, and family branch of the Center for Mental Health Services of the Substance and Mental Health Services Administration began a demonstration program aimed at the development of local systems of care. To date 65 sites have been funded to develop and implement systems of care for children with serious emotional disturbances and their families (4). The East Baltimore mental health partnership, one of the originally funded sites, is a joint community, city, and state effort to develop and implement a system of care (5). In Baltimore, agencies involved in the provision of services to children collaborated to provide a continuum of care to redress the fractured and inconsistent manner in which many children and their families were being served. These agencies incorporated the key elements of systems of care and introduced more intensive services than those traditionally provided, including wraparound services, respite care, and parental support services (2,5,6,7).
Since 1994 the East Baltimore mental health partnership has coordinated a comprehensive multiagency program of services to children and families. The target population includes children and adolescents experiencing a substantial inability to function in their families, schools, or community who are 18 years old or younger, have a DSM-IV diagnosis, are being served by at least two community agencies, and whose disorder has either persisted or can be expected to persist for at least one year.
These children are referred to the partnership from a variety of public service agencies: schools, mental health centers, the departments of juvenile justice and social services, and the Child Advocacy Network. Children may also be referred when services are planned for or provided to their siblings. Children are referred from public service agencies because of behavioral or emotional problems and because they have needs that the agency cannot meet. As a result, compared with more traditional outpatient services, the multiagency coordination of services provided by the partnership is a longer-term, broader-based service plan.
Even though many of the children referred for services are similar with regard to neighborhood, race, and socioeconomic status, variation in the profiles of these children, as well as children referred into other systems of care, has been well documented (8,9,10,11). Specifically, some factors have been shown to vary considerably within the referred population, such as the referring agency, diagnosis, age, gender, history of substance use, history of physical abuse of the child, previous psychiatric hospitalization of the child or parent, history of family mental illness or violence, problem behaviors, and psychosocial functioning.
Little if any research has been published on whether children who enter these collaborative networks with different profiles show equivalent changes in their functioning after referral or whether changes in functioning vary as a function of these profiles. Previous research points out that if we are to determine the clinical effectiveness of systems of care and for whom systems of care work best, this type of subgroup analysis is required (12).
This study determined and compared changes in functioning for children with different presenting problems, referral sources, risk factors, and demographic profiles. We hypothesized that children receiving services from a system of care will show improved functioning over time regardless of variation within these categories. A single-group pretest-posttest design was used to examine this hypothesis.
Methods
Community description
The East Baltimore community is a densely populated, relatively small urban area with approximately 20,000 residents under age 18. Ninety-one percent of these children are African American, and more than half live in households with incomes below the poverty level.
System-of-care services
The components of the East Baltimore mental health partnership system of care include case management, therapeutic foster care, crisis outreach, assessment and evaluation, intensive home-based services, outpatient care, family support services, recreation services, and school-based services. On referral to the partnership, each child and family is assigned a family resource specialist. Specialists are master's-level mental health clinicians who oversee and coordinate the development and implementation of service plans. In addition, they provide direct service to their assigned children and families in the form of individual and family counseling. Children and families also receive recreational instruction, which includes instruction in the arts, music, and culture, and support services, which include family support, respite care, tutoring, and mentoring, among other services. Assessment and crisis services are also available for all referred children.
Participants
Between August 1994 and November 1997, 478 children were referred for intensive community-based mental health services. Of the 478 children, 311 (65 percent) were boys. The mean±SD age at referral for this group was 11.1±2.3 years, and ages ranged from three to 17. A total of 459 (96 percent) were African American. A total of 129 families (27 percent) reported a gross annual household income below $5,000, 186 (39 percent) reported an income between $5,000 and $10,000, and 76 (16 percent) reported an income between $10,000 and $15,000.
Of the 478 children, 124 (26 percent) were referred from Baltimore City public schools, 96 (20 percent) from the Baltimore City Department of Social Services, 81 (17 percent) from the Baltimore City Department of Juvenile Justice, 67 (14 percent) from the Child Advocacy Network, and 57 (12 percent) from the Johns Hopkins Children's Mental Health Center. The remaining 53 children (11 percent) were siblings of referred children.
All 478 children and their families were eligible to participate in an ongoing evaluation. A total of 406 eligible children and their families (85 percent) agreed to participate in the evaluation. Race, gender, and age of the child did not differ between those who participated and those who did not. In addition, no differences were found between the groups in gross household income, the number of child and family risk factors at the time of referral, or the number of child and family services received before referral. In both groups the most common primary presenting problem of the child was physical aggression.
The sample for this study was composed of the 203 children who agreed to participate, who had been receiving services from the partnership for at least six months, and who had functional assessment ratings both at intake and at six months.
Measures
The Child and Adolescent Functional Assessment Scale (CAFAS) (13) is included in the ongoing evaluation protocol of the East Baltimore mental health partnership to provide an assessment of functional impairment at six-month intervals. The CAFAS is a clinician rating scale used to assess child functioning across five life domains: role performance at home, in the community, and at school; behavior toward others; moods and emotion, including self-harmful behavior; substance use and abuse; and thinking. The scale yields five subscale scores as well as a total functional impairment score. The total score ranges from 0 to 150, with higher scores indicating greater functional impairment. This investigation used only the total score. Interrater reliability and validity have been demonstrated in previous studies (15). In addition, for the study reported here, a structured self-training protocol was used, and all raters had to achieve correspondence between their ratings and the established criterion before rating children (16).
The CAFAS is less vulnerable to respondent bias than other commonly used instruments such as axis IV of DSM-IV and the Children's Global Assessment of Functioning (CGAF), because the rater generates scores for life domains rather than global scores. In addition, regression analyses have indicated that the CAFAS total score at intake is a stronger predictor of the cost and number of services received than other measures such as the CGAF and the Child Behavior Checklist (17).
Statistical analyses
For these analyses, we divided subjects into two school-anchored age categories—prekindergarten through elementary and middle school through high school—and five referral sources —Department of Juvenile Justice, Baltimore City public schools, the community mental health center, Department of Social Services, and other, which primarily included children referred from the Child Advocacy Network and siblings of previously referred children. We used two collapsed diagnostic categories: internalizing, which included mood, adjustment, and anxiety disorders, and externalizing, which included attention-deficit hyperactivity disorder, conduct disorder, and oppositional defiant disorder.
We assessed the presence of risk by taking the sum of 14 possible self-reported risk factors that are assessed at intake. The clinician coded these risk factors as present, 1, or absent, 0. This procedure has been used to assess risk in evaluations of other systems of care (8). The risk factors are listed in Table 1, with the number of children and the percentage of families reporting each. We then used a median split to dichotomize the scale into low risk, two or fewer risk factors, and high risk, more than two.
We conducted five independent general linear model repeated-measures analyses of variance (GLM-RM ANOVAs). These analyses assessed the interaction between change in the CAFAS score over time and gender, age, diagnosis, referral source, and level of risk, respectively. Because of small subgroup sizes and empty cells, we chose to avoid the combined analyses of gender, age, diagnosis, and referral source and their relationship with change in the CAFAS score over time.
A priori power analyses indicated sufficient power to detect a 10-point difference in CAFAS scores between groups for each of the five analyses. Power estimates ranged from .58 for the diagnosis and referral source analyses to .87 for the risk analysis.
Results
Sample description
Table 2 presents data on the referral source. The referral pattern for the sample of 203 children was similar to that for the population served by the East Baltimore mental health partnership. The Baltimore City public school system was the single largest referral source (38 percent), followed by the Department of Social Services (22 percent), the community mental health center (16 percent), and the Department of Juvenile Justice (12 percent). Sixty-six percent of the children were boys, and 67 percent were of prekindergarten to elementary school age.
As Table 2 shows, gender and average age of the referred children varied by the referring agency. For example, most referrals from juvenile justice were boys, and they tended to be older than those referred from other sources. Children referred from social services were mostly girls. Those referred from the public school system tended to be younger than those referred from other sources. The average number of risk factors was relatively stable across referral groups, with the exception of children referred from juvenile justice, who tended to report fewer risk factors.
Changes in functioning
Changes in functioning varied considerably from baseline to six months. Changes in the CAFAS score ranged from an 80-point decline to an 80-point improvement over the study period. At six months 31 children (15 percent) had the same CAFAS score as at baseline and 121 children (60 percent) had a lower score. Fifty-one children (25 percent) had a higher score at six months than at baseline.
Table 3 presents the baseline CAFAS score for each group and the mean±SD change in the CAFAS score over six months. On entry into the system of care, boys were more impaired on average than girls. However, the mean change in functioning was similar for both. Older children were more impaired on average than younger children, but the change in functioning was similar for the two groups. At intake children referred from juvenile justice were more impaired on average than children from other referral sources, and children in the "other" category were least impaired. Children referred from both the juvenile justice system and community mental health centers demonstrated the greatest mean change in the CAFAS score.
Children in the high-risk category were more functionally impaired on average than those in the low-risk group, although the mean change in the CAFAS score from baseline to six months was similar for the two groups.
As expected, the within-subjects effect for the change in the CAFAS score from baseline to six months was significant (F=34.61, df=1, 201, p< .001), indicating improvement in functioning. In addition, between-group differences were found for gender (F= 11.07, df=1, 201, p<.001), grade (F= 14.09, df=1, 201, p<.001), referral source (F=4.10, df=4, 198, p<.003), and risk level (F=11.06, 1, 201, p< .001) but not for diagnostic category.
We evaluated the interaction terms of each model to assess the hypothesis of differential improvement as a function of referring agency, gender, age, diagnostic category, and risk level (Table 3). The changes in the interaction between the CAFAS score and the referral source and between the CAFAS score and gender were not significant, indicating that groups from all referral sources as well as boys and girls had statistically equivalent rates of improved functioning over the six months.
The change in the interaction between the CAFAS score and age also was not significant, indicating equivalent rates of improvement for the prekindergarten-elementary group and the middle-high school group. The change in the interaction between the CAFAS score and diagnostic category was not significant, indicating equivalent rates of improved functioning for all diagnostic categories. Finally, the evaluation of the CAFAS score and risk level was not significant, indicating that both risk groups had equivalent rates of improvement.
Discussion and conclusions
Evidence from previous research has suggested that the children referred to and served in systems of care are heterogeneous with regard to their referring agency and their psychosocial and demographic profiles (8,9,10,11,8,18,19). The results of this investigation substantiate those differences at intake and demonstrate that children who were served in a local system of care showed equivalent improvement in functioning during their first six months of receiving services, regardless of referral agency, risk level, diagnosis, age, or gender.
Limitations
Although these findings have important implications, some limitations must be considered. Because of the limited sample size, we chose not to investigate potential higher-order interactions between the psychosocial and demographic variables and change in functioning or to assess the relationship between the reason for referral and change in functioning. The range of a priori power estimates is reasonable; however, some caution may be warranted in interpreting the absence of significant differences for children from various referral sources and in different diagnostic groups. The risk factor scale depended on self-reports, which may have affected its validity and reliability. The only measure of functioning used was a clinician rating, and multiple perspectives of improvement were not available, such as those of teachers, parents, and children. In addition, other contextual factors were not controlled for, such as the treating mental health professional.
Implications
Despite these limitations, the results have important implications for the delivery of behavioral health services. Although it is widely agreed that systems of care are the preferred method of delivering services to children with serious emotional disturbances and their families (2,3), the relative extent to which these systems benefit children referred from mental health, juvenile justice, social service, and education agencies has not been examined. This study provides initial evidence that children with various demographic profiles who are referred from a wide variety of settings demonstrate comparable improvement in functioning over time. We must now focus attention on more detailed assessments of children who do not respond to these services to understand why they do not and to develop alternatives that might be more effective.
Recent outcome and effectiveness studies have required us to investigate who can, does, and will benefit from the comprehensive service delivery model (12,20,21). It is clear that not all children meeting criteria for a psychiatric disorder require a multiagency planning team and an extensive coordination of services to improve. The children included in this study were referred because their referral agencies saw them as being inadequately served within those agencies because of their emotional or behavioral problems. Given the large number of children who meet diagnostic criteria (22,23) and the costs associated with coordinating multiagency planning and service programs, additional research is required to determine the cost-effective thresholds for activating a multiagency service team. Although it is clear that characteristic differences at the start of services have implications for planning and delivery, this preliminary investigation suggests consistent improvement despite subgroup differences.
Future research
This study examined functional outcomes based on gender, age, risk factors, diagnosis, and referral source. On average, children and adolescents in each group showed reduced impairment after six months. However, functional outcome varied considerably, and 25 percent of the children showed a functional decline at six months. It is crucial that we begin to investigate and understand this variability. For example, although the presence of risk factors was assessed in this study, it will also be important to examine risk factors independently and assess their relationship to changes in functioning. Risk factors related to parental functioning such as parental history of mental illness or substance abuse may affect children differently than child-specific risk factors such as the child's previous psychiatric hospitalization or substance use.
Finally, the East Baltimore mental health partnership operates in an impoverished urban minority community that precluded the assessment of differential improvement as a function of race, ethnicity, or income level. As greater attention is paid to the overrepresentation of minorities in the public service sector, this issue must be investigated within and across system of care sites.
Acknowledgments
This research was supported by grant MH-19545-07 from the National Institute of Mental Health and by grant CMHS-5-44SM51392 from the Center for Mental Health Services.
Footnote
Dr. Walrath is a research associate at the Johns Hopkins School of Hygiene and Public Health, where Dr. Leaf is a professor in the department of mental hygiene. Dr. Mandell is a postdoctoral fellow at the Center for Mental Health Policy and Services Research at the University of Pennsylvania in Philadelphia. Send correspondence to Dr. Walrath at the Johns Hopkins School of Hygiene and Public Health, 624 North Broadway, Eighth Floor, Baltimore, Maryland 21205 (e-mail, [email protected]).
References
1.
Winters CA: Learning disabilities, crime, delinquency, and special education placement. Adolescence 32:451-462, 1997
2.
Stroul BA, Friedman RM: A System of Care for Children and Youth With Severe Emotional Disturbances. Washington, DC, Georgetown University Child Development Center, 1986
3.
England MJ, Cole RF: Building systems of care for youth with serious mental illness. Hospital and Community Psychiatry 43:630-633, 1992
4.
Annual Report to Congress on the Evaluation of the Comprehensive Community Mental Health Services for Children and Their Families Program, 1997. Atlanta, Macro International, 1997
5.
Leaf PJ, Bogrov M, Webb MB: The East Baltimore mental health partnership, in Innovative Approaches for Difficult to Treat Populations. Edited by Henggeler SW, Santos AB. Washington, DC, American Psychiatric Press, 1997
6.
Stroul BA: Case management in a system of care, in From Case Management to Service Coordination for Children With Emotional, Behavioral, or Mental Disorders. Edited by Friesen BJ, Poertner J. Baltimore, Brookes, 1995
7.
Lourie IS, Katz-Leavy J: New directions for mental health services for families and children. Journal of Contemporary Human Services, May 1991, pp 277-285
8.
Rosenblatt J, Robertson MB, Wood M, et al: Troubled or troubling? Characteristics of youth referred to a system of care without system-level referral restraints. Journal of Emotional and Behavioral Disorders 6:42-54, 1998
9.
Walrath CM, Nickerson KJ, Crowel RL, et al: Serving children with serious emotional disturbance in a system of care: do mental health and non-mental health agency referrals look the same?. Journal of Emotional and Behavioral Disorders 6:205-213, 1998
10.
Singh NN, Landrum TJ, Donatelli CH, et al: Characteristics of children and adolescents with serious emotional disturbance in systems of care: I. partial hospitalization and inpatient psychiatric services. Journal of Emotional and Behavioral Disorders 2:13-20, 1994
11.
Epstein MH, Cullinan D, Quinn KP, et al: Personal, family, and service use characteristics of young people served by an interagency community-based system of care. Journal of Emotional and Behavioral Disorders 3:55-64, 1995
12.
Bickman L, Summerfelt WT, Noser K: Comparative outcomes of emotionally disturbed children and adolescents in a system of service vs usual care. Psychiatric Services 48:1543-1548, 1997
13.
Hodges K: Child and Adolescent Functional Assessment Scale. Ypsilanti, Mich, Eastern Michigan University, Department of Psychology, 1994
14.
Hodges K: Child and Adolescent Functional Assessment Scale Manual for Training Coordinators. Ypsilanti, Mich, Eastern Michigan University, Department of Psychology, 1995
15.
Hodges K, Wong MM: Psychometric characteristics of a multidimensional measure to assess impairment: the Child and Adolescent Functional Assessment Scale. Journal of Child and Family Studies 5:445-467, 1996
16.
Hodges K: Child and Adolescent Functional Assessment Scale Self-Training Manual. Ypsilanti, Mich, Eastern Michigan University, Department of Psychology, 1995
17.
Hodges K, Gust J: Measures of impairment for children and adolescents. Journal of Mental Health Administration 22:403-413, 1995
18.
Landrum TJ, Singh NN, Nemil MS, et al: Characteristics of children and adolescents with serious emotional disturbance in systems of care: II. community-based services. Journal of Emotional and Behavioral Disorders 3:141-149, 1995
19.
Epstein MH, Cullinan D, Quinn KP, et al: Characteristics of children with emotional and behavioral disorders in community-based programs designed to prevent placement in residential facilities. Journal of Emotional and Behavioral Disorders 2:51-57, 1994
20.
Bickman L: A continuum of care: more is not always better. American Psychologist 51:689-701, 1996
21.
Friedman RM, Burns BJ: The evaluation of the Fort Bragg demonstration project: an alternative interpretation of the findings. Journal of Mental Health Administration 23:128-136, 1996
22.
Brandenburg NA, Friedman RM, Silver SE: The epidemiology of childhood psychiatric disorders: prevalence findings from recent studies. Journal of the American Academy of Child and Adolescent Psychiatry 29:76-83, 1990
23.
Shaffer D, Fisher P, Dulcan MK, et al: The NIMH Diagnostic Interview Schedule for Children, version 2.3 (DISC-2.3): description, acceptability, prevalence rates, and performance in the MECA study. Journal of the American Academy of Child and Adolescent Psychiatry 35:865-877, 1996
Information & Authors
Information
Published In
History
Published online: 1 February 2001
Published in print: February 2001
Authors
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBGet Access
Login options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).