Variations in the Clinical Presentations of Children and Adolescents at Eight Psychiatric Hospitals
Abstract
OBJECTIVE: Clinical presentations of children and adolescents admitted to eight psychiatric hospitals were examined to determine variation in symptoms and severity of illness. METHODS: A prospective design was implemented using the Childhood Severity of Psychiatric Illness (CSPI) scale, a reliable measure of psychiatric severity and factors thought to affect decision making about mental health services. The CSPI was completed by mobile crisis workers after interviews with 875 children and adolescents to assess whether hospitalization was appropriate. All the children were wards of the state in Cook County, Illinois, and all were subsequently hospitalized. RESULTS: A factor analysis of the CSPI revealed three distinct factors: caregiver problems, externalizing symptoms, and internalizing symptoms. Children were classified as having mild, moderate, or severe problems or symptoms in the areas covered by the three factors. Chi square analyses revealed that significant variation existed between hospitals in the proportions of children with problems or symptoms in the three factor areas and in the severity of their problems or symptoms. CONCLUSIONS: The major finding of variation among the eight hospitals on measures of severity of children's psychiatric illness and caregiver problems empirically supports the importance of adjusting for severity before implementing initiatives to manage performance and outcome across a system of hospitals.
Variations in the clinical presentation of service recipients across provider hospitals make it more challenging to manage system performance and outcomes (1,2,3). If variations in patients' clinical presentation affect a hospital's performance, then analyses must control for these variations before fair comparisons can be made between provider hospitals. Studies of medical-surgical hospitals have already demonstrated that controlling for the level of medical severity across hospital types can account for large differences in length of stay and charges (3). Less work has been done on variations across psychiatric hospitals, and still less has been accomplished with children and adolescents.
Obtaining reliable information about patients' clinical characteristics and their variations across hospitals can have several benefits. They include creation of provider-specific outcome expectations, a rationale for resource allocation, identification of children who are being underserved, and creation of an opportunity for research to inform treatment matching or develop clinical pathways (2).
When comparing hospitals, Kiesler and associates (4) found evidence of differences in the type of patient treated—those with a primary diagnosis of mental disorder versus those with primary substance abuse—and in the type of hospital—general versus specialized hospitals. Patterson and colleagues (5) studied the characteristics of a single hospital sample and found a wide range of psychiatric and comorbid nonpsychiatric impairment. Other studies have demonstrated changes in patient characteristics over time (6,7). For example, Wright and coworkers (6) found significant changes between 1970 and 1980 in the level of psychiatric severity and complexity of illness of children and adolescents in their single-hospital population.
Research on patients' characteristics has examined differences at either the level of hospital type (4)—general versus specialized—or the level of the individual hospital (6). Studies show diagnostic variation, which could reflect differential reporting practices; however, no research has provided independent clinical profiles of hospitalized psychiatric patients that reflect potentially divergent treatment needs.
Developing methods to describe between-hospital variations in clinical presentations is particularly important now, given recent guidelines from the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) mandating hospital outcomes management. In addition to outcomes, the JCAHO Oryx system manual suggests that health care organizations provide "risk adjustment/stratification," defined as "a process for reducing, removing, or clarifying the influences of confounding patient factors that differ among comparison groups" (8). Clearly, knowledge about how variations in clinical presentation may influence outcomes is crucial to the reasonable and fair evaluation of hospitals. When awareness of these variations is lacking, organizations are placed in the position of integrating performance differences and potentially making allocation and planning decisions under the untested assumption that all of its providers are serving the same types of clients.
This study examined differences in the clinical presentations of patients across eight psychiatric hospitals serving children and adolescents who were wards of the state in a metropolitan county in a large Midwestern state. A reliable and valid measure of psychiatric severity, the Severity of Psychiatric Illness measure, Child and Adolescent Version (910) was completed by independent hospital screeners before each hospitalization. The measure was then used to create distinct scales measuring severity of children's internalizing and externalizing symptomatology and caregiver issues. The scales were then compared across the eight hospitals.
Methods
Setting
This study was conducted through the screening, assessment, and support service program of the Illinois child welfare system, the Illinois Department of Child and Family Services. The program was implemented in 1992 to provide mobile crisis assessment and treatment services to children in protective custody who are referred for or at risk of hospitalization. Referrals are made to program workers when a ward of the Illinois Department of Child and Family Services demonstrates a risk of self-harm or injury to others and might require hospitalization.
Program workers screen the child to assess for appropriateness of hospitalization. The attending psychiatrist is responsible for the final decision. If a child is not considered appropriate for hospitalization and the psychiatrist agrees, a variety of diversion services can then be implemented. Data from eight hospitals in one large metropolitan county were used. All eight hospitals are under contract with the Illinois Department of Child and Family Services to serve state wards and are reimbursed on a per diem basis through Medicaid.
Sample
The sample included all children and adolescents from the eight largest Cook County hospitals who were screened and hospitalized through the screening, assessment, and support service program from January 1997 through September 1997. All data were collected directly from the program workers at the end of the month in which the screening interview occurred.
The overall sample consisted of 875 children and adolescents. Of these, 370 (42.3 percent) were female and 505 (57.7 percent) were male. The mean±SD age was 13±3.4. A majority of children (718 children, or 82 percent) were African American; an additional 96 (11 percent) were Caucasian, and the remaining 61 (7 percent) were Hispanic or Asian.
Diagnoses were made by the workers in the screening, assessment, and support service program. A total of 333 children (38 percent) were diagnosed as having a primary mood disorder; 140 (16 percent) had anxiety disorders, including posttraumatic stress disorder; 158 (18 percent) had attention-deficit disorder and disruptive behavior disorders; 44 (5 percent) had adjustment disorders; 96 (11 percent) had impulse control disorders; and 53 (6 percent) had psychotic disorders. The remaining 61 children (7 percent) had a variety of other disorders.
Measures
The Childhood Severity of Psychiatric Illness (CSPI) measure, a standardized decision-support tool (2,9,10,11), was used. Completing the CSPI involves making ratings on 4-point scales, with 0 indicating no evidence and 3 severe dysfunction, for a series of 25 items. These items were identified by focus groups of various providers and by a literature review.
The items cover five dimensions: symptoms, risk factors, functioning, comorbid factors, and placement or system characteristics. An example of the rating criteria for one of the 25 variables, emotional disturbance, is presented in Table 1. All workers in the screening, assessment, and support service program attended a three-hour training session on the use and implementation of the CSPI. Interrater reliability was tested during the sessions using clinical vignettes, and it ranged from .70 to .80 using Spearman's rho. Recent reviews of program charts indicated that reliability remained strong after training; the average Spearman's rho of .67 suggested that the measure was used properly in the field.
Results
Nine variables from the CSPI were submitted to a factor analysis. These variables were chosen for their relevance to hospital-based services and a minimum of missing data. A three-factor solution that accounted for 44 percent of the variance in CSPI scores emerged. The variables that constitute the three factors are presented in Table 2. On the basis of factor loadings, the factors were called caregiver problems, externalizing symptoms, and internalizing symptoms.
The factor analysis was used to create three scales. These scales had moderate to high internal consistencies. Cronbach's alphas were .8 for caregiver problems, .7 for externalizing symptoms, and .6 for internalizing symptoms. Correlations between the scales were low to moderate, indicating statistical independence. The caregiver problems scale had a correlation of .33 with the externalizing symptoms scale, and .13 with the internalizing symptoms scale. The correlation between the externalizing symptoms and internalizing symptoms scales was .18.
Because the scales each had three variables, scale scores ranged from 0, indicating an absence of disturbance on all three CSPI variables, to 9, indicating ratings of severe disturbance on all three CSPI variables in the scale.
Scale scores were then grouped into one of three severity categories—mild, moderate, or severe. For each of the three scales, children with scores of 3 or less were categorized as having mild symptoms. Scores of 4 or 5 placed children in the moderate category, and scores greater than 5 placed children in the severe category. For instance, a child would receive a classification of mild on a scale by receiving ratings of 1 on each of the three CSPI variables in the scale, for a total score of 3. Moderate ratings on two of three CSPI variables in a scale would place a child in the moderate severity category.
Notable differences were found on the three scales for the 875 children admitted to the eight Cook County hospitals. Caregiver issues were generally mild—470 cases (68 percent) fell into the lowest severity category on this scale. Only 90 children in the sample (13 percent) had caregiver problems that were considered severe. Overall, 282 children (32 percent) had mild externalizing symptoms, and 353 children (40 percent) had severe externalizing symptoms.
On the internalizing symptoms scale, 334 children (40 percent) had mild symptoms, and 184 (22 percent) had severe symptoms.
To assess variation among hospitals, a set of chi square analyses was performed across the severity categories and the hospital sample. Table 3 shows that 13 percent of the overall sample of children had severe caregiver issues, which can be considered the base rate for this severity category. However, rates at the individual hospitals varied significantly from the base rate. For example, only 3 percent of children from hospital E had severe caregiver issues. Children who stayed at hospital D, on the other hand, were more likely to have severe caregiver issues—23 percent compared with the 13 percent base rate. The overall chi square test across hospitals and severity levels was significant for the caregiver problems scale (χ2=54.7, df=14, p<.01).
The chi square test for the externalizing symptoms scale was not significant overall. However, some variation existed between individual hospitals and the base rate for this scale. For example, as Table 3 shows, only 14 percent of children at hospital F were classified as having severe externalizing symptoms, which is only one-third of the base rate of 40 percent. Twenty-six percent of children and adolescents at hospital G had severe externalizing symptoms, which is also substantially lower than the base rate. None of the hospitals had a significantly higher proportion of children with externalizing symptoms compared with the base rate.
The chi square test for the internalizing symptoms scale across hospitals and severity levels was significant overall (χ2=49.4, df=14, p<.01). Again, several hospitals demonstrated variation from the base rate of 22 percent. Hospital C had a rate of 34 percent of patients with severe internalizing symptoms. Hospital G, with a 14 percent rate, demonstrated a different pattern. Therefore, whereas only one-sixth of the children and adolescents admitted to hospital G had severe internalizing symptoms, one-third of hospital C's caseload was made up of patients with severe internalizing symptomatology.
Discussion
This study used a measure of psychiatric severity and its mitigating factors with children and adolescents presenting to psychiatric hospitals. The study demonstrated significant clinical variation across eight hospitals in one metropolitan county. Comparisons showed that the rate of severe disturbance among children at certain hospitals varied substantially from the base rate.
This variation in clinical presentation has several implications. One important potential use of this information is as an adjustment of outcome expectations. Outcome expectations for providers who deliver services to clients with more challenging symptoms should be lower than for other providers. Previous studies have shown that variations in psychiatric symptoms and severity lead to different rates of improvement (12,13), which further justifies continued research in this area and provides a direction for future studies.
Adjusting for severity in order to control for outcome differences may also limit "creaming"—the practice of serving the easiest-to-treat clients (2). By eliminating incentives for treating less severe and complex patients, populations that are harder to treat are less likely to be underserved.
We can also use information on severity to supplement evaluations of quality and to tailor those evaluations to the unique client populations served by hospitals. For instance, hospital F appears to serve a high proportion of children with both internalizing and externalizing symptoms. Quality assurance staff at this hospital may want to ensure that adequate safety precautions are in place for children with severe internalizing symptoms. Because the correlation between the externalizing symptoms scale and the internalizing symptoms scale was low (.18), we know that patients who were rated high on one scale were generally not rated high on the other, which suggests that there is a group of children with internalizing symptoms who are distinct from, and possibly in need of protection from, a group of children with externalizing symptoms.
About one patient in four admitted to hospital D appeared to come from an unstable caregiver environment, compared with only one in 33 at hospital E. Therefore, quality improvement staff may want to focus on ensuring that hospital D has the capacity to work with high-need caregivers.
It is important to note that data were collected from independent screeners with a limited incentive to portray a biased picture of any particular hospital. Considerable criticism has been directed to the use of psychiatric diagnoses to describe the clinical needs of psychiatric patients in situations where particular diagnoses are required for reimbursement. No such conflict of interest existed in this study. It is possible that different screening, assessment, and support service programs varied in their reporting style. However, because of substantial worker crossover in these programs and in the eight hospitals, such variations in practice patterns were unlikely to influence the results of the study.
Future research should establish a link between variations in the clinical presentation of service recipients and outcomes and utilization. This information can then be used as empirical support for planning decisions and quality improvement efforts. Once this goal has been accomplished, we can begin to explore whether providers whose clients have a predominant profile serve those clients better than other providers with a more diverse case mix. The surgical literature has demonstrated that patients are more likely to survive a procedure when it is performed in a hospital where an above-average number of similar procedures are performed (14). We do not yet know whether a similar finding would be made in psychiatric treatment settings.
Conclusions
This study demonstrated that client characteristics may vary notably across hospitals. This variation has important implications for understanding and improving systems of care. Particularly in the face of performance management initiatives, improving our ability to measure, monitor, and control for clinical variations in the populations served by different hospitals represents a significant challenge to the field. Failure to understand these differences creates the potential for untoward consequences of efforts to manage performance and outcome.
Acknowledgments
The authors thank Jennifer Behr, Sharon Fleishman, Purva Rawal, Katie Schafer, and Courtney West for their assistance. Financial support for this research was provided by the Illinois Department of Child and Family Services.
Footnote
The authors are affiliated with the mental health services and policy program of the department of psychiatry and behavioral sciences at Northwestern University Medical School. Send correspondence to Mr. Leon at the Institute for Health Services Research and Policy Studies, Northwestern University Medical School, 339 East Chicago Avenue, Room 717, Chicago, Illinois 60611 (e-mail, [email protected]).
References
1.
Kiesler CA, Simpkins C: Changes in diagnostic case mix in psychiatric care in general hospitals, 1980-85. General Hospital Psychiatry 14:156-161, 1992
2.
Lyons JS, Howard KI, O'Mahoney MT, et al: Measurement and Management of Clinical Outcomes in Mental Health. New York, Wiley, 1997
3.
Horn SD, Sharkey PD: Measuring severity of illness to predict patient resource use within DRGs. Inquiry 20:314-321, 1983
4.
Kiesler CA, Simpkins CG, Morton TL: Prevalence of dual diagnoses of mental health and substance abuse disorders in general hospitals. Hospital and Community Psychiatry 42:400-403, 1991
5.
Patterson R, Bauer P, McDonald CA, et al: A profile of children and adolescents in a psychiatric unit: multidomain impairment and research implications. Australian and New Zealand Journal of Psychiatry 31:682-690, 1997
6.
Wright HH, Batey SR, Buttefield PT, et al: The changing spectrum of children and adolescents seen in a child and adolescent psychiatry clinic during the 1970s. Psychiatric Forum 15:11-18, 1989
7.
Kiesler CA, Simpkins CA: Changes in psychiatric inpatient treatment of children and youth in general hospitals. Hospital and Community Psychiatry 42:601-604, 1991
8.
Oryx Manual. Oakbrook Terrace, Ill, Joint Commission on Accreditation of Healthcare Organizations, 1997
9.
Leon SL, Uziel-Miller ND, Lyons JS: Psychiatric hospital service utilization of children and adolescents in state custody. Journal of the American Academy of Child and Adolescent Psychiatry 38:305-310, 1999
10.
Lyons JS, Chesler P, Shallcross H, et al: The Childhood Severity of Psychiatric Illness: a tool for needs-based planning and decision support. Family Matters, May 1996, pp 1-10
11.
Lyons JS: The Severity and Acuity of Psychiatric Illness Scales. San Antonio, Psychological Corp, 1998
12.
Blackman M, Eustace J, Chowdhury T: Adolescent residential treatment: a one- to three-year follow-up. Canadian Journal of Psychiatry 36:472-479, 1991
13.
Kolko DJ: Short-term follow-up of child psychiatric hospitalization: clinical description, predictors, and correlates. Journal of the American Academy of Child and Adolescent Psychiatry 31:719-727, 1992
14.
Kelly JV, Hellinger FJ: Physician and hospital factors associated with mortality of surgical patients. Medical Care 24:785-800, 1986
Information & Authors
Information
Published In
History
Published online: 1 June 2000
Published in print: June 2000
Authors
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBGet Access
Login options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).