Reliability and Validity of the Interview and Self-Report Versions of the BASIS-32

A convenience sample of 120 adults was recruited from three psychosocial rehabilitation programs in July and August 1996. Information on demographic characteristics and psychiatric diagnosis was provided by the Missouri Department of Mental Health. For psychiatric diagnosis, the diagnostic formulation of the psychosocial rehabilitation program's psychiatrist was used; only information for the primary diagnosis was available.

A total of 78 participants (65 percent) were Caucasian, and 29 (24 percent) were African American. Sixty-five participants (54 percent) were men. The mean±SD age of the sample was 44±11 years. Eighty-two participants (68 percent) had psychotic disorders; 12 (10 percent) had bipolar disorder; 15 (13 percent) had major depression, recurrent; and five (4 percent) had other diagnoses. For six participants (5 percent), the diagnosis was unknown. Bivariate comparisons revealed no differences between participants assigned to the self-report and interview conditions in gender, race, age, diagnostic distribution, and the presence of substance abuse.

Psychiatric symptoms were assessed for the study with a 43-item derivative of the Hopkins Symptom Checklist (MSCL-43) (4) and the 18-item Brief Psychiatric Rating Scale (BPRS) (5,6). The MSCL-43 has three scales: subjective distress (alpha=.94), anxiety-depression (alpha=.94), and thought disorder-suspiciousness (alpha=.89). Three BPRS scales were used: anxiety-depression, hostility-suspicion, and thought disorder. On both scales, higher scores indicate greater symptomatology.

Case managers completed a two-item substance abuse severity scale—one item each for alcohol and for drug abuse severity. Items were rated using anchors borrowed from the Addiction Severity Index (7). Social functioning was assessed by case managers' ratings of the adequacy of clients' natural support (such as family and friends) and professional support, with higher scores indicating better support. Case managers also rated the amount of conflict in clients' networks, with higher scores indicating more conflict. Finally, case managers indicated the number of leisure activities in which clients were involved and whether clients were involved in vocational activities (0=no, 1=yes).

Participants were randomly assigned to either a self-report or an interview condition. Participants in the self-report condition completed the BASIS-32 and the MSCL-43 and then completed the BASIS-32 again one week later. Interview participants were administered the BASIS-32 and the BPRS and then completed the MSCL-43. Interviews were conducted by social workers at the rehabilitation programs. Two hours of training in the administration of the BASIS-32 and the BPRS were provided. Interviewers were trained to query and assign a rating for each item of the BASIS-32 and BPRS. A related study using identical procedures achieved good interrater reliability (8). Significance levels were set at .05 for all statistical tests, resulting in power just under .80 to detect a medium effect size for the analyses of variance and just under .80 to detect a large effect size for the comparisons of correlations (9).

The main objective of this study was to examine the impact of changing the administration procedures—self-report versus interview—on the psychometric properties of the BASIS-32. Our results suggest that the interview and self-report versions may not have equivalent reliability and validity. In addition, we found the BASIS-32 functional domains to have poorer validity than did previous studies.

Consistent with previous studies, internal consistency coefficients were acceptable for most subscales; however, reliability was uniformly higher for the self-report version. The internal consistency of the impulsive-addictive behaviors subscale was less than desirable. In addition, consistent with other studies (1,2), the interview version of the psychosis subscale had very poor internal consistency. The fact that the psychosis subscale contains only four items undoubtedly affects its reliability (10). However, the low internal consistency found here also raises a concern that the interview version of the psychosis subscale is essentially a group of unrelated items.

In general, the symptom subscales for both the interview and the self-report versions of the BASIS-32 correlated well with other symptom scales, although correlations were higher for the self-report version. Unlike previous studies, we were unable to demonstrate concurrent or discriminant validity for the self-report version of the functional domains. For the interview version, however, clinicians' ratings of conflict in participants' social networks were associated with scores on the relation to self-others subscale of the BASIS-32 and leisure activities.

The results of validity analyses were only slightly better when specific items of the BASIS-32 were examined. Clients who reported greater difficulty with leisure activities were rated by case managers as involved in more leisure activities; no such pattern existed for the interview version. Ratings for severity of alcohol abuse correlated relatively well from all three perspectives—client, interviewer, and case manager. However, no such agreement was found for ratings of drug use; case manager ratings correlated with interviewer ratings but not with self-reports. In fact, all clients who reported difficulty with drug use were rated by case managers as having no drug problem, and all clients who were rated by case managers as having a drug problem reported no difficulty with drug use. Consistent with another study (3), we found a trend for participants who completed the self-report version to report somewhat greater distress on the BASIS-32 subscales than participants who were administered the interview version.

The most significant weakness of this study is that we did not employ a multitrait-multimethod matrix design. Such a design would have added confidence to our differential judgments about concurrent and discriminant validity. Another limitation of the study was that some of the measures of client functioning did not perform as anticipated. For example, we had originally planned to assess the type of housing in which clients resided, but our housing measure had very little variance and was dropped. In addition, concerns about burdening clinical staff with long questionnaires resulted in the selection of brief assessments of client functioning. Future studies should use more detailed assessments of client functioning to assess the validity of the BASIS-32.

Nevertheless, this study raised important questions about the impact that varying the method of administration had on the psychometric properties of the BASIS-32. In addition, our study found the BASIS-32 functional domains to be less valid than other studies found them to be. Future research should continue to examine these critical issues, particularly in light of the increasing popularity of the BASIS-32.

The authors thank the Missouri Department of Mental Health for funding this project. They also thank Robert Calsyn, Ph.D., and Debbie Zand, Ph.D., for reviewing earlier versions of this paper and Michael Trusty, Ph.D., for statistical consultation.

Dr. Klinkenberg is research assistant professor and Dr. Cho is associate professor at the Missouri Institute of Mental Health of the University of Missouri School of Medicine, 5400 Arsenal Street, St. Louis, Missouri 63139-1361. Mr. Vieweg is deputy director of the office of information systems for the Missouri Department of Mental Health.