Identifying Meaningful Subgroups of Adults With Severe Mental Illness
Abstract
OBJECTIVE: The authors describe work conducted to identify, describe, and validate subgroups, or clusters, of adults with severe mental illness. METHODS: Clusters were first identified by using functional assessment ratings and statistical clustering procedures. Seven separate cluster analyses were accomplished, involving agencies in 15 counties in Ohio. Brief prose descriptions of the clusters were developed on the basis of the results of the statistical analyses. These descriptions were then used as stimuli in an extended expert-based validation and enhancement process that included work groups of consumers, family members, and providers. RESULTS: This sequential process produced generalizable, holistic descriptions of five core clusters of individuals with common strengths, problems, treatment histories, and life situations: older people in poor health who have psychiatric symptoms; individuals who have both psychiatric and substance use disorders; chronically mentally ill persons who have long treatment histories and severe disabilities; persons who appear to function well in the community but who are isolated as a result of anxiety, depression, or trauma-related social fears; and mental health consumers who function well in the community and who use personal and professional support systems to manage their mental health problems. CONCLUSIONS: The results support the hypothesis that there is meaningful, but predictable, heterogeneity among persons who have severe mental illness. The clusters identified in this study can be used to plan, manage, and evaluate services.
A key to understanding what works in the treatment of people with severe mental illness is a clear and common understanding of the people themselves. It is generally acknowledged that this population is heterogeneous and that what works in some cases may not be appropriate in others (1). However, efforts to improve the quality of care available to persons with severe mental illness must be based on more precise information about the characteristics and environments of these individuals than is currently available (1).
Persons with severe mental illness have been described in several ways. The most widely recognized classification system is the diagnosis-based DSM-IV (2), in which clients are described in terms of the characteristics of their illness. The DSM-IV system provides a detailed clinical picture of signs and symptoms of various disorders; however, it is not as helpful for predicting the need for or use of a range of therapeutic, rehabilitative, or support services (3,4,5,6,7).
Another approach, the use of diagnosis-related groups (DRGs), is based on illness episodes (8). DRGs have been used to manage inpatient care but are not available for community mental health or community support systems. DRGs have been less helpful in providing clinical pictures (9) or predicting the use and cost of resources (10,11,12). Approaches such as the Level of Need-Care Assessment (13) are based on need profiles and have been used to identify need patterns and gaps in community services.
Although all these approaches are valid for specific purposes, an approach with broader utility is needed. The lack of a more holistic process has been a barrier to the development and management of adequate systems of care, such as recovery-oriented community support systems or capitated managed care systems. It has also limited attempts to assess the effectiveness of mental health services and policy.
For example, in Philadelphia's capitation experiment, although recidivism rates were lower, clients fared no better under capitation in terms of length of stay or hospitalization rate (14). Similarly, in the Rochester, New York, demonstration project, capitation reduced hospital use but did not improve clients' functioning or reduce symptoms (15,16). However, in both studies clients were grouped only by diagnostic category, history of hospitalization, or both (15). Considerable within-group heterogeneity was undoubtedly masked, and the clients were all assessed on the same service outcomes.
In seeking an alternative classification process, the cognitive psychology literature suggests that mental health consumers be described in terms of prototypes (17) based on a multitude of characteristics. The use of prototypes results in a more holistic, person-focused classification that can take into account both the strengths and the weaknesses of members of a group and can consider the whole person within his or her historical, community, and social contexts. This prototype model assumes that those who work with such special populations naturally identify subgroups of people with common problems, strengths, treatment needs, and prospects for recovery (18).
In contrast with more classical categorization approaches that require individual patients to meet necessary and sufficient conditions, prototypes—or clusters—are often characterized by a set of correlated or typical features (17). Prototypes frequently describe common elements and capture the variability among members of the same cluster (19). Thus prototypes of adults with severe mental illness would be expected to encompass a broad array of information, such as information on social and living skills, work skills and history, family role and support, history and effectiveness of treatment, psychiatric symptoms, interference from substance abuse or chronic physical health problems, living environments, personal strengths, and integration into the community (20,21,22,23,24,25,26,27).
Although it has been suggested that it is time to move beyond a "mere description" of heterogeneity and toward the identification and use of evidence-based practices (25), our experience and the literature suggest that mental health professionals are not quite ready for that step. While acknowledging differences within the population of persons with serious mental illness, we continue to describe that heterogeneity in narrow terms. This approach has greatly limited our understanding of what is working and for whom, as well as the resources required. More holistic descriptions of subgroups of these clients are needed. These more useful descriptions will incorporate natural multidimensional categories that can be expressed as clusters of characteristics and life conditions (17,19).
Between 1988 and 1996 we worked with local mental health authorities, publicly funded agencies, and the Ohio Department of Mental Health to identify clusters within the population of persons with serious mental illness. This effort involved seven geographic mental health service regions encompassing 15 of Ohio's 88 counties and several thousand adult consumers.
The data collection and research procedures were generally consistent across the seven regions. About half of the work was conducted as part of research approved and funded by the Ohio Department of Mental Health, which reviewed the procedures under its ethical practice guidelines. The remainder was accomplished under program evaluation of planning consultation contracts with local mental health boards or agencies. We discuss below the systematic process used to identify, describe, and determine the generalizability of clusters of adults with severe mental illness.
Methods
We identified clusters by combining statistical clustering techniques with group planning processes involving consumers, family members, and service providers (20). With some local variation, the steps were repeated in each of the seven service regions.
In each region, case managers assessed their seriously mentally ill clients by using a functional assessment instrument called the Community Living Skills Indicators (COLISK), which we developed. At that time, there were few, if any, relatively comprehensive assessment tools that could be completed by staff within 15 minutes. The COLISK included 32 functional assessment scales and a few demographic and diagnostic items. Most items were presented as 11-point Likert scales.
Interrrater agreement was assessed with data from two urban agencies. A total of 125 randomly selected clients were rated independently by their case manager and a supervisor. Because the same raters were not used to assess each of the clients, rwg (28), a conservative measure, was used to determine the level of agreement between the actual scores on each Likert scale. The rwg controls for the number of response categories and the probability of chance agreement on each item (29). The rwg indexes ranged from .58 to .93, and 84 percent of them exceeded .70. Given our intended use of the data—that is, not for individual classification—these indexes represented an acceptable level of interrater agreement.
Formal assessment of client functioning
Case managers at each of eight organizations were trained and instructed to assess all their clients with the COLISK. They were instructed to base their ratings on clients' functioning over the previous three to six months. Ratings were typically completed for 70 to 80 percent of the agency's case management clients within 45 days of the training. In one urban region, staff assessed about 25 percent of the 2,000 adults who had emergency admissions for possible involuntary commitment (22,27). In another urban region, about 750 consumers from two large mental health centers were assessed through the use of a sample stratified by agency service component. More than 3,600 clients in the seven service regions were assessed.
Cluster analysis
Twenty-three of the functional assessment items were used in the formal cluster analyses. Fifteen of these were grouped into four subscales; alphas ranged from .65 to .90. These four subscales, plus the other eight items and age, were then used in the formal cluster analyses. The 12 items or subscales are listed in Table 1.
For ease of interpretation and to enable us to handle the larger samples, we used SPSS Quick Cluster (30). Quick Cluster enables the user to specify the number of clusters to be created and to repeat the process until a satisfactory solution is reached. Cluster solutions, typically ranging from three to ten clusters, were computed. The sample sizes for the cluster analyses ranged from 346 to 744.
We used several criteria to identify satisfactory solutions. First, the number of clusters would be somewhere between five and ten. The literature on human cognitive categorization processes and expert-based schemas indicates that experts naturally create a moderate number of information-rich prototypes to help understand phenomena (17). Second, no cluster would represent fewer than 5 percent of the clients, to enhance the practical value to agencies in their service planning activities. Third, the matrix of cluster centers had to make clinical sense in terms of our own experience and our knowledge of the literature. Finally, in reviewing solutions, we also examined the computed analyses of variance to determine which variables were contributing the most to the specific solution. Once a final solution was established, frequency distributions were reviewed by cluster for all COLISK items, both functional and demographic.
Preparation of cluster descriptions
In each service region, the quantitative results were translated into preliminary prose descriptions of each cluster. These preliminary descriptions were then validated and enhanced by a panel of experts typically composed of providers, consumers, and family members. Meeting regularly for a year, the members of the panel enhanced the prose descriptions on the basis of their training or personal and professional experiences. The resulting final cluster descriptions incorporated the language, knowledge, and experience of these community experts.
Identifying core outcomes and goals
Using a group process called intention analysis (31), the same panel of experts identified a set of service goals or expected outcomes for each cluster. Behaviorally anchored rating scales were developed for each of these goals.
Assessment of cluster membership and data collection
Case managers and their supervisors were then trained to use the prose cluster descriptions and outcomes to assign each client on their caseload to a cluster. Staff were instructed to use a best-fit strategy, meaning that a client did not need to have all the characteristics outlined in the prose description to be considered a member of a specific cluster. In addition, staff were trained to consider the outcomes linked to each cluster to help identify the best-fitting cluster. Initial outcome ratings were made at this time, and outcome ratings were collected again at six- or 12-month intervals. In addition, service, hospitalization, and cost data were obtained for each client.
Results
Between 1988 and 1996 the independent, multistep cluster development efforts in the seven service regions produced solutions ranging from five to eight clusters. On inspection of all the actual cluster descriptions, it became clear that five core clusters could be described that encompassed all the identified clusters across the seven regions. The titles given to these clusters by each regional work group are listed in Table 2.
As we expected, treatment goals or intended outcomes differed by cluster in all regions. However, treatment goals within a cluster showed considerable similarity. Another important finding can be seen in the case of region 4, where functional assessments were completed only for individuals who were brought for involuntary hospitalization. In this case two core clusters did not emerge. Both clusters described individuals who seldom have emergency admissions.
Core cluster 1
Members of core cluster 1 are generally older and have serious, chronic physical health conditions that interfere greatly in their lives, including heart disease, emphysema, cancer, diabetes, stroke, arthritis, and Alzheimer's disease. Such conditions make it difficult for these individuals to care for themselves or participate in the community. Members of this cluster also have significant problems with psychiatric symptoms, confusion, depression, and anxiety. Often it is difficult to tell whether their problems stem from their physical health problems, their mental conditions, or both. Many live in nursing homes or are cared for by family members. They are often socially isolated and withdrawn and have lost interest in the world around them.
Core cluster 2
Core cluster 2 includes individuals who have both psychiatric symptoms and substance abuse or dependence. Some have less severe mental health problems and, when they are not using drugs, can function in jobs, take care of themselves, and participate in community activities. However, when they are using drugs they are impulsive, engage in high-risk drug-seeking and sexual behaviors, manipulate others, and have great difficulty functioning in work or training settings. Other members of this cluster have more serious mental health symptoms in conjunction with their substance abuse. These individuals have hallucinations, delusions, depression, anxiety, and other symptoms that have persistently interfered with their daily life. They have difficulty managing their time and refuse to take their psychotropic medications. Many feel helpless and have developed a sense of hopelessness as a result of years of living in distress. They have lost many community living skills, and without considerable support their ability to be involved in vocational pursuits is significantly hampered.
Core cluster 3
Persons in core cluster 3 experience significant problems in many life areas. Many have been in the mental health system for a long time. They have been hospitalized for long periods and may have only recently started to benefit from civil rights policies and effective medications. They have hallucinations, delusions, obsessions, or compulsions and often express anger inappropriately. They need help with activities of daily living and assistance taking their medications. However, they often have little support from family members or friends. The members of this cluster often create problems in the community. They lack social skills and have difficulty understanding and maintaining social boundaries. They may be asked to leave stores or restaurants or may become involved with the criminal justice system. For some members of this cluster, the onset of psychiatric illness has been recent. They struggle with the knowledge that they were working or in school only a few years ago, and they have great difficulty recognizing the severity of their condition. They often refuse treatment, act impulsively, and cycle in and out of hospitals and other services.
Core cluster 4
Persons in core cluster 4 struggle with anxiety and depression and often avoid growth opportunities. Many appear to be functioning well at home and in the community. However, they experience considerable interference in their lives as a result of anxiety, depression, or passivity. Some may have histories of abuse or trauma. Many have found a comfort zone within which they like to function and do not want anything to happen to upset that equilibrium. They have the potential to participate in work, education, or vocational training but are afraid to take the associated risks. They have difficulty accepting success and may even sabotage their own efforts. They have a favorable attitude toward treatment as long as they do not feel pushed. Some members of this cluster have chronic health problems that have become the focus of their lives. They often use these health problems to gain attention, avoid uncomfortable situations, and avoid dealing with the causes behind their fear or depression.
Core cluster 5
Members of core cluster 5 are mental health consumers who have functioned well in their communities. They have typically managed their mental health problems well. Many are already working, although they may need some support on the job, and most others could benefit from work or educational opportunities. These individuals have good daily living skills and can use community resources to meet their needs. They have a positive attitude toward treatment and see it as a safety net for times of crisis. They can use information and guidance from case managers and others to deal with their problems and work toward personal goals.
Discussion
Research and practice in Ohio confirm the systematic heterogeneity within the population of adults with severe mental illness. Five core clusters, or subgroups, of this population were identified through a sequential process that included functional assessment ratings and statistical clustering procedures followed by extended multistakeholder, expert-based validation and enhancement processes. Seven separate cluster-development efforts were completed. These processes resulted in prose cluster descriptions incorporating strengths, problems, treatment histories or successes, social and environmental contexts, and life situations of the members of each subgroup. However, the strongest evidence of the validity of these clusters can be found in the way they are being used by agencies to plan, manage, and evaluate services. The following examples highlight this practical utility.
Research funded by the Ohio Department of Mental Health has produced expert-derived best-practice models for each cluster (32). In two urban counties, participatory planning groups comprising providers, consumers, family members, and other experts used the cluster descriptions and outcomes as stimuli to develop preferred-service models for each cluster. Although the models developed by each group converged for similar clusters, the models for various clusters differed considerably. Two agencies are currently engaged in pilot tests of specific services outlined in the models.
In 1998 one of these two agencies decided to reorganize case management services by cluster. Treatment teams now specialize in serving members of one or two core clusters. Staffing patterns, including differential caseload sizes, were determined from past service use data, outcome achievement, expected productivity, and desired best practice. Implementation began in December 1998 and took about a year to complete. About 1,000 clients were transferred, and all staff had the opportunity to bid on their preferred team. Reorganization by cluster is expected to enhance consumer outcomes, focus staff training and development, increase job satisfaction, reduce burnout and turnover, and enhance the agencies' decision making and quality improvement (33).
Finally, in 2001 a new statewide initiative was begun in conjunction with the Ohio Council of Behavioral Healthcare Providers, the statewide provider association. The Ohio Cluster-Based Planning Alliance is an effort to expand the use and utility of the identified clusters throughout Ohio. As part of its effort to disseminate and support the use of best practices, the Ohio Department of Mental Health has designated the alliance as one of its statewide coordinating centers of excellence. The alliance will provide staff training and will analyze service, cost, and outcome data by cluster. Agency-specific analyses will be augmented by comparative data from other members of the alliance.
The cluster-based data will also enable alliance members to assess current practices and develop their own evidence-based practices. For example, members of core cluster 4 often struggle with anxiety and depression that is rooted in past abuse or trauma. Although they appear relatively intact, members of this cluster are actually fragile and require specific clinical interventions to address their needs. Members of this cluster account for 15 to 25 percent of agency caseloads but seldom receive individual or group counseling or referrals to abuse or trauma services. Agency staff who have previously been unaware of the more complicated picture of these individuals have often failed to recognize their needs and have been frustrated by their "failure" to reach their perceived potential in education, work, and social roles.
Use of the clusters is expected to help agencies and their staff identify the clinical interventions and services needed and the types of resources required to provide them to members of each cluster. Through the Ohio cluster-based planning alliance, this information can be developed jointly and shared to increase access to care.
Conclusions
Research has confirmed that natural, systematic, and generalizable subgroups exist within the population of adults with severe mental illness. Characterizing these subgroups by using prose descriptions makes it possible to tap into a vast store of information representing the collective experience of many community experts. Cluster descriptions have been used to identify targeted treatment goals, reorganize services, and identify expert-derived best practice models.
As a tool for service system research, the use of clusters can help clinicians understand the interaction effects that become masked when we focus on measuring all clients on the same outcomes. More critically, the cluster-based approach provides an important framework that can contribute to the recovery of persons who have severe mental illness. Paying attention to clusters can contribute to treatment planning and can enhance the efforts of management to facilitate quality improvement, focus staff training, and support the evaluation of services at all organizational levels.
Acknowledgments
The authors thank Janie Taynor, Ph.D., for her significant contributions over an eight-year period to the conceptualization, testing, and development of the cluster-based planning system. This work was partly supported by the office of program evaluation and research of the Ohio Department of Mental Health.
Footnote
Mr. Rubin is president of Synthesis, Inc., 400 East Town Street, Suite G-60, Columbus, Ohio 43215 (e-mail, [email protected]). Dr. Panzano is president of Decision Support Services, Inc., in Columbus.
References
1.
Caring for People With Severe Mental Disorders: A National Plan of Research to Improve Services. Washington, DC, National Institute of Mental Health, 1991
2.
Gray P: Psychology, 2nd ed. New York, Worth, 1984
3.
Grove WM: The reliability of psychiatric diagnosis, in Issues in Diagnostic Research. Edited by Last CG, Hersen M. New York, Plenum, 1987
4.
Matarazzo JD: The reliability of psychiatric and psychological diagnosis. Clinical Psychology Review 3:103-145, 1983
5.
Spitzer RL, Fleiss JL: A re-analysis of the reliability of psychiatric diagnosis. British Journal of Psychiatry 125:341-347, 1974
6.
Walton M: The Deming Management Method. New York, Perigee, 1986
7.
Wilson M: DSM-III and the transformation of American psychiatry: a history. American Journal of Psychiatry 150:399-410, 1993
8.
Goldman HH, Pincus HH, Taube CA, et al: Prospective payment for psychiatric hospitalization: questions and issues. Hospital and Community Psychiatry 35:460-464, 1984
9.
Holcomb WR, Thompson WA: Medicare prospective reimbursement for mental health services: a literature review. Administration in Mental Health 15:127-138, 1988
10.
English JT, Scharfstein SS, Sherl DJ, et al: DRGs and general hospital psychiatry. American Journal of Psychiatry 141:131-139, 1983
11.
Taube C, Lee ES, Forthoffer RN: DRGs in psychiatry. Medical Care 22:597-610, 1984
12.
English JT, McCarrick RG: DRGs: an overview of the issues. General Hospital Psychiatry 8:359-364, 1986
13.
Uehara ES, Smukler M, Newman FL: Linking resource use to consumer level of need: field test of the Level of Need-Care Assessment (LONCA) method. Journal of Consulting and Clinical Psychology 62:695-709, 1994
14.
Christianson J, Dickey B, Frank R, et al: Early data show mixed results for managed care. Focus on Mental Health Services Research 6(1):3-5, 1993
15.
Cole RE, Reed SK, Babigan HM, et al: A mental health capitation program: I. patient outcomes. Hospital and Community Psychiatry 45:1090-1096, 1994
16.
Sharfstein SS: Capitation versus decapitation in mental health. Hospital and Community Psychiatry 45:1065, 1994
17.
Smith EE, Medin DL: Categories and Concepts. Cambridge, Mass, Harvard University Press, 1981
18.
Fiske ST, Linville: What does the schema concept buy now? Personality and Social Psychology Bulletin 6:543-557, 1980
19.
Holland JH, Holyoak KJ, Nisbett RE, et al: Induction: Processes of Inference, Learning, and Discovery. Cambridge, Mass, MIT Press, 1989
20.
Taynor J, Rubin W: Cluster Analytic Planning for Adults With Severe Mental Disabilities and Children "Most in Need." Report prepared for the Four County Community Mental Health Board. Defiance, Ohio, Jan 1989
21.
Rubin W, Taynor J: Cluster Analytic Planning and Evaluation: Painting Pictures of People With Mental Disabilities. Presented at the First National Case Management Conference, Cincinnati, Ohio, Sept 14-16, 1990
22.
Rubin W, Taynor J, Panzano PC, et al: Community Commitment: Predicting Differential Outcomes. Working Report of the Governor's Study Committee on Mental Health Services. Technical report produced by Synthesis, Inc, under a grant from the Ohio Department of Mental Health, Office of Program Evaluation and Research, Columbus, Ohio, 1992
23.
Rubin W, Taynor J: Cluster Analytic Planning and Evaluation: A Process for Evaluating and Managing Mental Health Care. Presented at the National Conference on State Mental Health Agency Services Research, Annapolis, Md, Oct 2-5, 1993
24.
Herman SE, Mowbray CT: Client typology based on functioning level assessments: utility for service planning and monitoring. Journal of Mental Health Administration 18:101-115, 1991
25.
Luke DA, Mowbray CT, Klump K, et al: Exploring the diversity of dual diagnosis: utility of cluster analysis for program planning. Journal of Mental Health Administration 23:298-316, 1996
26.
Shern D, Wilson N, Coen A, et al: Colorado Treatment Outcome Study: Multi-perspective Outcome Study of Mental Health System Reform. Research report, Colorado Division of Mental Health, Denver, 1990
27.
Rubin W, Snapp MB, Panzano PC, et al: Variation in civil commitment processes across jurisdictions: an approach for monitoring and managing change in mental health systems. Journal of Mental Health Administration 23:375-388, 1996
28.
James L, Demaree RG, Wolf G: rwg: an assessment of within-group agreement. Journal of Applied Psychology 78:306-309, 1993
29.
Koslowski SH, Hattrup K: A disagreement about within-group agreement: disentangling issues of consistency vs consensus. Journal of Applied Psychology 74:368-370, 1992
30.
Norusis MJ: SPSS/PC+ Advanced Statistics, version 2.0. Chicago, SPSS Inc, 1988
31.
Taynor J, Daugherty K, Nelson R: The family intervention scale: the outline of a methodology and a scale to assess treatment outcome in a family service agency: families in society. Journal of Contemporary Human Services, Family Services America 71:202-210, 1990
32.
Rubin W, Panzano PC, Schwartz J, et al: Goodness of Fit in Managed Mental Health Care: Services, Outcomes, and Local Control: Resource Manual. Prepared for the Ohio Department of Mental Health, Office of Program Evaluation and Research, Columbus, Nov 1997
33.
Rubin W, Granger D, Betts J, et al: The "nitty-gritty" of re-organizing agency services around cluster of adults with severe mental disabilities. Presented at the sixth annual Florida Conference on Behavioral Healthcare Evaluation, Orlando, Fla, Dec 8-10, 1999
Information & Authors
Information
Published In
History
Published online: 1 April 2002
Published in print: April 2002
Authors
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBGet Access
Login options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).