It is widely recognized that the major mental illnesses impair daily functioning
(1–
4). Therefore, new psychopharmacologic and psychosocial treatments are often evaluated for their effects on symptoms, as well as functional status
(5–
7). These latter assessments are increasingly important in cost-effectiveness projections.
Official classifications also reflect an increasing interest in functional assessment. The APA’s DSM first included an assessment of functioning on axis V in 1980 (DSM-III) and subsequently added a separate Global Assessment of Functioning scale in 1994 (DSM-IV). The World Health Organization has developed a separate classification system for impairments, disabilities, and social consequences of diseases
(8,
9). A recent report from the National Institute of Mental Health (NIMH) recommended continuing the development of tools to assess functioning as a means of measuring the economic and social burden of mental disorders
(10).
In clinical research, self-administered scales are widely used to survey patients directly about their functional status. Despite the popularity of self-administered scales, surprisingly little attention has been given to the issue of scale selection. Scales vary in breadth and in the variety of covered dimensions, item content within each dimension, and the range of function represented. Some definitions of health implicit in functional scales emphasize the level of impairment or functional disturbance
(11), while others conceive of health not simply as the absence of disease or disability, but as the presence of affirmative capacities and well-being
(12,
13).
Because functional scales vary in coverage and content, it is important to describe how response patterns on individual scales differ from one another. Unfortunately, direct comparisons are rare. To our knowledge, no published study has compared scores on any two of the three functional scales included in the study reported here: the Medical Outcomes Study 36-item Short-Form Health Survey
(14) and the Social Adjustment Scale Self-Report
(15)—both of which have been widely used in clinical trials and outcomes research—and the more recently developed Social Adaptation Self-Evaluation Scale
(12,
13). The latter scale has been used in European clinical trials
(16,
17) and is currently being studied in the United States.
In the study reported here, we first assessed the acceptability of the three scales in an adult primary care population. Then we examined the demographic correlates of scale scores and intercorrelations among the three scales. Finally, we evaluated the ability of each scale to differentiate patients with acute psychiatric symptoms from asymptomatic patients.
Method
Patient Recruitment and Diagnostic Assessment
This study was conducted at the Associates in Internal Medicine at the College of Physicians and Surgeons of Columbia University. The clinic provides approximately 54,000 medical visits annually to 18,000 patients from an ethnically and racially diverse community. Between October 1998 and April 1999, a systematically selected sample of primary care patients with scheduled appointments were approached in the waiting room and invited to participate. To be eligible for the study, patients had to be 1) between 18 and 70 years of age, 2) scheduled for face-to-face contact with a primary care physician, 3) not a first-time visitor to the clinic, 4) able to understand Spanish or English, 5) of a general health status that did not preclude survey completion, and 6) not actively homicidal or suicidal. The latter patients were given appropriate emergency care. The institutional review board of the Departments of Medicine and Psychiatry approved the protocol. All participants provided signed informed consent.
Of the approximately 3,400 patients who were prescreened, 1,266 met eligibility criteria for phase 1 of the study. The most common reasons for exclusion were age (44%) and not having a scheduled visit with a primary care physician (31%). Four patients were excluded because of active suicidal plans. Of the 1,266 who met eligibility criteria, 1,007 (79.5%) consented to participate. Consenters were similar to nonconsenters in gender and race/ethnicity; however, nonconsenters were slightly older (mean=55.5 years, SD=11.3, versus mean=53.2 years, SD=12.2) (t=2.40, df=1, 1188, p=0.02).
Of the 1,007 patients who participated in phase 1, a random subsample (N=271) was invited to participate in the second phase. Those who were selected for phase 2 and those who were not selected did not differ significantly in age, gender or racial/ethnic distribution, educational level, or household income. Of the subsample of 271, consenters (N=223, 82.3%) did not differ from nonconsenters (N=48, 17.7%) in age, gender or racial/ethnic distribution, or household income. However, consenters were somewhat more educated (median attainment of “ninth to 11th grade” versus “eighth grade or below” for nonconsenters, Mann-Whitney U=4024.50, z=2.81, p=0.005). Interview data for 12 subjects were lost because of computer malfunction, resulting in a final phase 2 sample of 211 subjects. These 211 subjects did not differ from the remainder of the original sample eligible for phase 1 (N=1,055) in age, gender, or race/ethnicity.
During phase 1, demographic, screening, treatment utilization, and some social functioning data, described elsewhere
(18), were obtained. The second phase included a diagnostic assessment with the Composite International Diagnostic Interview, version 2.1
(19) (for DSM-IV), conducted by experienced mental health professionals (psychologists or social workers) who had completed training in the use of this instrument. All assessments were translated from English to Spanish and back-translated by a bilingual team of mental health professionals. The Composite International Diagnostic Interview was conducted within a week of screening for 77% of the sample (range=0–39 days, median=4). The Social Adaptation Self-Evaluation Scale was completed during phase 1, and the 36-item Short-Form Health Survey and Social Adjustment Scale Self-Report were completed during phase 2.
Social Functioning Scales
Medical Outcomes Study 36-Item Short-Form Health Survey
The 36-item Short-Form Health Survey is a self-report scale constructed to collect data on health status, functioning, and well-being for the Medical Outcomes Study
(3,
14,
20,
21). The time period covered by most items is 4 weeks. The number of response choices varies across items, from two (yes/no) to six (anchored by “all of the time” and “none of the time”). Eight health concepts are assessed: physical functioning and limitations in physical activities because of health problems; social functioning and limitations in social activities because of physical or emotional problems; role—physical, and limitations in usual role activities because of physical health problems; bodily pain; mental health and psychological distress and well-being; role—emotional, and limitations in usual role activities because of emotional problems; vitality, energy, and fatigue; and perceptions of one’s general health.
In the current study, scores for the eight subscales and the physical and mental component summary scales were calculated by using official Short-Form Health Survey algorithms
(20,
21). The summary scales are computed by aggregating subscale scores according to the factor structure of the subscales in the general U.S. population. Unlike the eight subscales, the two summary scale scores have norms based on the general U.S. population (mean=50, SD=10)
(21).
Social Adjustment Scale Self-Report
The Social Adjustment Scale Self-Report
(15,
22) is a self-report scale with 54 questions that measure instrumental and expressive role performance over the past 2 weeks. It includes questions on work for pay, unpaid work, and work as a student; social and leisure activities, relationships with the extended family, the marital partner, one’s children, and relationships within the family unit; and perception of economic functioning. The questions within each area cover performance at expected tasks, friction with people, finer aspects of interpersonal relationships, and feelings and satisfactions. Each item is scored on a 5-point scale with higher scores indicating poorer functioning. The Social Adjustment Scale Self-Report contains skip-outs, so that nonapplicable items are omitted. Scores for each role area are calculated by averaging the scores for all answered items within that area. The total Social Adjustment Scale Self-Report score is calculated by averaging all applicable items (however, only the work items pertaining to the primary work role are included).
Social Adaptation Self-Evaluation Scale
The Social Adaptation Self-Evaluation Scale is a 21-item self-rating scale developed to detect presumed treatment differences in social motivation and behavior that may not be discernible in psychiatric assessment
(13). The scale was developed by Pharmacia-Upjohn as a tool for determining differential effects of a selective noradrenaline reuptake inhibitor, which was presumed to have an increased effect on drive and motivation toward action, as compared to selective serotonin reuptake inhibitors (SSRIs)
(17,
23). The Social Adaptation Self-Evaluation Scale focuses on the patient’s self-perception and motivation toward action rather than on objective performance. The scale was validated in a large general population survey, and its external and internal validity, test-retest reliability, and sensitivity to change have been described
(13). The time period assessed is “now.” The first two items (interest in one’s occupation and one’s home-related activities) are mutually exclusive for scoring purposes. If the respondent endorses having an occupation, the occupation item is used, otherwise the home-related item is used. A total of 20 items are summed for a possible total score of 0 to 60. Higher scores indicate higher functioning. Only data on total scores have been published.
Statistical Analyses
Statistical analyses were performed with SPSS for Windows, version 9.0 (SPSS, Inc., Chicago). Feasibility of the three scales was assessed by 1) the number of patients who refused to fill out the scales and 2) the number of patients who answered all the items. Skipped items on the Social Adjustment Scale Self-Report were not considered missing if the corresponding role area did not apply to the patient.
Pearson product-moment correlations were used to measure associations among continuous variables, including age and all of the social functioning scores. Student’s t test and analysis of variance were used, where appropriate, to compare functioning scores among groups on the basis of gender, language of choice, race/ethnicity, and diagnostic status. When group variances were significantly different (p<0.05), we used the corrected degrees of freedom (df). In such cases, the df is reported with two decimal places and the nominal df is provided in brackets. All tests were two-tailed with an alpha of 0.05. To help control the rate of type I error, we did multiple group comparisons only when the omnibus F was significant at p<0.05.
Discussion
Psychiatric clinical efficacy, effectiveness, and outcome studies increasingly include measures of functional status. We present the first direct comparison of three scales: the Short-Form Health Survey, the Social Adjustment Scale Self-Report, and the Social Adaptation Self-Evaluation Scale. The Short-Form Health Survey and the Social Adjustment Scale Self-Report have been used together in a large drug trial with dysthymic patients. The results were reported as parallel outcome measures or in comparison to published norms
(5). One other study compared the reading levels of the Short-Form Health Survey and the Social Adjustment Scale Self-Report and showed they were comparable
(24).
In our study all three scales were acceptable to patients, in that patients were able to complete the forms. The Social Adjustment Scale Self-Report and the Short-Form Health Survey were sensitive to gender differences, probably because they include assessments of family and marital functioning. The Short-Form Health Survey was more sensitive to age differences on items assessing physical health. There were few race/ethnicity or language differences on any of the scales.
Correlations among scales were statistically significant but modest. However, the physical component summary scale of the Short-Form Health Survey had little correlation with the other scales or with the mental health-oriented subscales of the same scale. It is no surprise that the Short-Form Health Survey physical and mental components did not correlate, since they were originally created from an orthogonal rotation of factor scores. The fact that the physical component does not correlate with the other scales suggests that this component is measuring a unique domain. All scales were able to differentiate psychiatrically ill from well subjects. The correlations between scales were modest even in the areas covering similar domains. A comparison of the questions used for work assessment shows how the approaches of the three scales differ (
Figure 1). The Short-Form Health Survey focuses on work performance; the Social Adjustment Scale Self-Report differentiates between work roles and assesses the actual number of days lost, affective performance, and interpersonal relations; and the Social Adaptation Self-Evaluation Scale emphasizes interest and motivation for work.
The original purpose for which the scales were designed is well represented in their content. The Short-Form Health Survey was designed to determine outcomes of psychiatric and medical conditions in a medical setting. It is the most widely used of the three scales, and it is the only one of the three that includes measures of physical functioning and physical activities (lifting, climbing, bending, etc.). It also includes assessment of depressive symptoms. There is an effort to separate work limitation due to physical as compared to emotional problems. Work role (e.g., homemaker, student) and actual time lost from work are not differentiated. Two questions deal with social activities with friends and family. Marital and parental roles and perception of economic functioning are not assessed.
The Social Adjustment Scale Self-Report was originally developed as an assessment tool for a clinical trial with drugs and psychotherapy for depressed patients in their child-rearing years. This background is reflected in the item content, in which instrumental and affective functioning in the extended family, with the spouse or partner, and with children are assessed separately. Work roles are differentiated so that a person who does not work for pay is also assessed on work. However, it is the longest of the three scales and, like the Social Adaptation Self-Evaluation Scale, does not assess physical functioning or general health status.
The Social Adaptation Self-Evaluation Scale is the newest of the scales and was designed for testing the efficacy of a new selective noradrenaline reuptake inhibitor, which is hypothesized to have an additional effect on drive and motivation, as compared to an SSRI. Like the Short-Form Health Survey, the Social Adaptation Self-Evaluation Scale does not include work role or actual time lost from work and does not differentiate family roles.
These findings provide general guidelines for instrument selection. For studies that include elderly subjects or that are being conducted in a primary care setting, researchers may find the Short-Form Health Survey’s coverage of physical, as well as mental, functioning useful. Also, the Short-Form Health Survey is now so widely used that it has become a standard, enabling comparison between studies. The Social Adjustment Scale Self-Report covers instrumental role performance and different family roles, which may be useful in economic projections. The Social Adjustment Scale Self-Report may be appropriate for investigations of effects on marital and parental functioning, which may include homemakers and students, for whom a different concept of work role functioning may apply, or may require estimates of actual days lost from work. Studies including noradrenergic drugs with specific hypothesis about target of action might use the Social Adaptation Self-Evaluation Scale
(16).
The limitations of this study include the modest sample size. Although the sample was representative of primary care clinic patients, it was not representative of the U.S. population. Finally, we did not report data on the ability of these scales to differentially predict treatment effectiveness or clinical course.
A recent NIMH workgroup on translating behavioral science into action concluded that impaired functioning cuts across the range of mental illness and creates substantial economic burden
(10). The workgroup noted a need for new standardized assessments, so that health care costs can be accurately estimated. They called for an integration of basic behavioral science theory and methods in the development of scales. Until this is achieved, our data suggest that investigators selecting a functional status scale should carefully review the content of available scales. If functional status is a critical outcome measure, more than one scale should be included in the study.