Subjects
Subjects were recruited on admission to a large, urban Veterans Affairs medical center. Data were drawn from a study, conducted between 1990 and 1995, that compared the efficacy of two experimental treatment programs for cocaine abusers with schizophrenia. One program used case managers to coordinate separate psychiatric and substance abuse services. In the other program, a new treatment unit provided comprehensive integrated services for both disorders. Patients were eligible for the study if the admitting psychiatrist concluded that the patient required psychiatric hospitalization, had a psychotic disorder that might be schizophrenia, and had abused cocaine within the past six months.
The study was approved by the medical center's institutional review board, and all subjects gave informed consent. All patients were hospitalized initially. Each experimental program then attempted to maintain patients in ambulatory care, but success varied. By the 18-month follow-up assessment, patients had received varying durations and intensities of both inpatient and outpatient care.
Altogether, 221 patients were referred to the study by admitting psychiatrists. Forty-nine did not meet study criteria: 13 were too dangerous, ten never had psychotic symptoms, eight never abused stimulants, six had no phone or mailing address, four could not read the consent form, two were too psychotic to consent, two could not participate because it would conflict with current treatment, two planned to move out of the state, one was participating in a conflicting research project, and one had medical problems that were too severe. Seven patients who met study criteria refused to provide consent. Thus a total of 165 subjects completed the initial assessment.
All subjects were males with a mean±SD age of 40±6.4 years. A total of 129 (78 percent) were African American, 28 (17 percent) were Caucasian, and eight (5 percent) were of other races. The mean±SD level of education was 13±1.8 years. Sixty-eight patients (41 percent) had never married, 18 (11 percent) were currently married, and 79 (48 percent) were separated, widowed, or divorced. Seventy-three (44 percent) were homeless.
On average, they had used cocaine regularly for nine years (SD=6.7) and had used cocaine on 13±11.7 days of the 30 days before study entry. A total of 160 patients (97 percent) met DSM-III-R criteria for current cocaine abuse or dependence, 86 (52 percent) for alcohol, 58 (35 percent) for marijuana, 21 (13 percent) for opiates, 17 (10 percent) for sedatives, 12 (7 percent) for amphetamine, and seven (4 percent) for other substances. The mean number of psychiatric hospitalizations for the groups was ten (SD=11).
Thus these patients were severely mentally ill and chronically cocaine dependent individuals. They were characterized by homelessness, many psychiatric hospitalizations, a long duration of cocaine use, and recent extensive use of cocaine as well as other drugs and alcohol.
Diagnostic assessment
All subjects were assessed using the Structured Clinical Interview for DSM-III-R (SCID-R) for axis I disorders (
28), supplemented whenever possible by hospital records and by interviews with family members, friends, and other persons familiar with the patient (collateral interviews). The project was affiliated with the UCLA Clinical Research Center for Schizophrenia and Psychiatric Rehabilitation, which is sponsored by the National Institute of Mental Health. The center maintains reliability of measurement across many projects through standardized training and quality assurance.
Using consensus ratings made by the center's senior diagnosticians as "gold-standard" criteria, five master's-level psychologists were trained to high levels of interrater reliability in the use of the SCID-R (overall kappa of at least .75 for symptoms and 100 percent agreement on diagnosis). Our use of the kappa statistic as a measure of association follows Kraemer's recommendation (
29) that chance-corrected indexes of association are, in general, more interpretable and meaningful. When using gold standards, Kraemer identifies a family of weighted kappa statistics and notes that the conventional, so-called Cohen's kappa (
30) is simply the statistic that derives when false positive and false negative errors are weighted equally.
One statistical feature of our weighted kappa is atypical, namely, that the separate units of observation are not independent subjects but rather items from a scale administered to a single individual. Because separate items within a scale are not independent, conventional significance testing of these weighted kappas is not appropriate. However, we use them only descriptively, and they are not tested for statistical significance. The nonindependence of observations does nothing to compromise the statistic as a descriptive index of the degree of chance-corrected association.
The diagnostic interviewers' initial results were discussed in a weekly case conference involving two senior diagnosticians (the first and third authors). The sources of diagnostic uncertainty were discussed in an effort to identify key questions that may not have been asked or new sources of information. Based on the conference, the diagnostic interviewer collected new information and presented the case again until no further sources of diagnostic information could be identified. This strategy, together with the reassessment described below, is in conformance with the "LEAD standard" proposed by Spitzer (
31). LEAD is an acronym for Longitudinal, Expert and All Data and refers to criterion-based diagnoses made by experts reaching consensus after examining data from a variety of sources and from repeated assessments over considerable time.
Family members or other collaterals were available for interviews in about a quarter of the cases but were rarely able to provide key information, such as whether the patient's psychotic symptoms persisted during abstinence. Medical records were available in nearly three-quarters of cases and occasionally provided important diagnostic information, such as previous incoherence in a patient who reported a nonbizarre delusion as the only psychotic symptom.
The scoring of the SCID-R was modified to allow interviewers to record the source of uncertainty on any item. A list of potential sources of uncertainty was developed. The list included insufficient periods of abstinence, inconsistent responses, poor memory, and disorganized speech, among other sources. If uncertainty on individual items led to uncertainty at decision points in the SCID-R algorithm, interviewers completed questioning along each of the branches from the decision point.
For example, on a number of individual items, the unmodified SCID-R requires that a psychotic symptom be counted toward the diagnosis of schizophrenia only if it is judged to be "not organic." The modified scoring allowed the interviewer to score the symptom in one of three ways: definitely organic, uncertain organicity, or definitely not organic. If organicity was uncertain, the sources of uncertainty were recorded, such as an inconsistent history or insufficient periods of abstinence. Similarly, substance abuse and dependence can raise concern that the mood syndromes, chronicity, and deterioration in functioning may have an organic basis. These criteria were also operationalized to allow for recording uncertainty. (A list of these criteria and their operational characteristics is available from the first author.)
In cases of diagnostic uncertainty, these modifications of SCID-R scoring led to a list of possible diagnoses (differential diagnosis) and a list of sources of uncertainty. The sources of uncertainty were keyed to the criteria for schizophrenia. For example, in
DSM-III-R the organic exclusion criterion for schizophrenia is met if "it cannot be established that an organic factor initiated and maintained the disorder" (
32). For this study, the statement was operationalized with separate criteria for determining whether psychotomimetic substances initiated or maintained the disorder.
For example, a subject reported that persecutory delusions had first started during a period of amphetamine abuse. This temporal relationship suggested that amphetamine might have initiated the delusions, but it did not provide enough information to conclude that amphetamine was an organic etiologic factor. The close temporal relationship might have been coincidental, or the amphetamine might have precipitated a psychotic disorder that then maintained itself. The interviewer then asked about the persistence of the symptom in the absence of amphetamine use. The interviewer would have concluded that amphetamine was etiologic if both of the following had been true: first, that at least twice, persecutory delusions resolved within six weeks of stopping amphetamine use, and, second, that persecutory delusions had never occurred more than six weeks after stopping amphetamine use.
Conversely, the interviewer would have concluded that persecutory delusions were not due to amphetamine if the subject reported that delusions had occurred more than six weeks after stopping amphetamine use. We required two instances of resolution within six weeks to avoid ascribing etiologic significance to chance coincidence of drug use and psychosis. However, in the example discussed here, the subject had been using amphetamine for several years and had never stopped using the drug for more than three weeks. Although persecutory delusions were less prominent during these brief periods of abstinence, they did not resolve.
Thus the interviewer noted that persecutory delusions were present, that it was uncertain whether they were organic, and that the source of the uncertainty was insufficient periods of abstinence. Rating persecutory delusions as present but their organic basis as uncertain resulted in a differential diagnosis that included, among others, amphetamine delusional disorder and schizophrenia. Similar criteria were developed to specify the relationship between abused substances and three other criteria for schizophrenia±_chronicity, deterioration, and mood syndromes.
This project was designed and conducted before publication of
DSM-IV(
33). However, early drafts from the
DSM-IV work group suggested that symptoms should not be considered drug induced if they persisted for more than six weeks after cessation of drug use. Consequently, we used the same time frame in our criteria. In the final version of
DSM-IV this time frame was reduced to four weeks.