Measures and Procedures
All patients were diagnosed according to DSM-IV criteria by using Spanish versions of the PRISM, the SCID for axis I disorders, clinical version
(20), and the SCID for axis II personality disorders (SCID-II)
(21). All patients were also diagnosed by using the LEAD procedure. Diagnoses were described in two time frames: “current” (criteria were met within the past year) and “past” (criteria were met before the previous 12 months).
The Spanish version of the PRISM was obtained after translation of the original version (provided by the authors of PRISM) by a bilingual research translator (whose native language was Spanish) and back-translated by one experienced bilingual research translator (whose native language was English). Following the back-translation, both the Spanish version and the back-translation were sent to the original authors for revision. The final version was the result of corrections for inconsistencies and problems detected by the research team.
The PRISM includes the following disorders: 1) substance use disorders, including substance abuse and dependence for alcohol, cannabis, hallucinogens, licit and illicit opiates, and stimulants; 2) primary affective disorders, including major depression, manic episode (and bipolar I disorder), psychotic mood disorder, hypomanic episode (and bipolar II disorder), dysthymia, and cyclothymic disorder; 3) primary anxiety disorders, including panic, simple phobia, social phobia, agoraphobia, obsessive-compulsive disorder, generalized anxiety disorder, and posttraumatic stress disorder; 4) primary psychotic disorders, including schizophrenia, schizoaffective disorder, schizophreniform disorder, delusional disorder, and psychotic disorder not otherwise specified; 5) eating disorders, including anorexia, bulimia, and binge-eating disorder; 6) personality disorders, including antisocial and borderline personality disorders; and 7) substance-induced disorders, including major depression, mania, dysthymia, psychosis, panic disorder, and generalized anxiety disorder.
The Spanish version of the PRISM was administered in all cases by two research interviewers with a master’s degree and clinical experience with patients with substance abuse or mental disorders (D.S. and M.A.). The interviewers conducted a series of practice interviews together to enhance interrater reliability before beginning their interviews with patients.
We used the Spanish version of the SCID and SCID-II diagnoses
(20,
21). The SCID-II involved a two-stage process in which the patient initially completed a self-report questionnaire followed by a detailed clinician inquiry cued by positive responses. The SCID is the semistructured interview more widely used for psychiatric diagnoses in psychiatric population studies. All of the Spanish versions of the SCID and SCID-II were administered by an independent research interviewer experienced in the use of these instruments for mental disorders (G.P.D.) who was blind in respect to diagnoses obtained by the PRISM instrument.
The LEAD procedure diagnosis was conducted by two clinical experts, both psychiatrists, who were experienced in the diagnosis and treatment of psychoactive substance use and other psychiatric disorders (M.T. and R.M.S.). Each patient’s attending psychiatrist served as the expert for that patient. At the beginning of the study, LEAD experts were informed that at the end of assessment they would give all the DSM-IV diagnosis of each patient in two time frames: current (within the past year) and past (before the previous 12 months). To develop the LEAD diagnoses, the experts used the following material: their own clinical interviews during the admission period (interview at admission day, complete standard clinical interview of the department, and daily clinical contact with the patient), the patient’s complete clinical record (including evaluations conducted by staff nurses, evaluations by medical consultants, laboratory results, and, when available, medical records from previous treatment in both inpatient and outpatient settings), and data provided from a family informant about each patient, when possible. The clinical interview was not structured and provided only general guidelines for assessing all clinical areas (i.e., sociodemographic features, family and personal background, employment, legal status, and toxicological, psychopathological, somatic, and psychiatric examination). At the end of the procedure, the experts had to fill in a questionnaire about the presence or absence of all DSM-IV diagnoses listed (current or past) in each patient. The LEAD experts were blind to diagnoses obtained by the PRISM and SCID.
After a mean of 6 days after admission at the Drug Abuse Treatment Center, patients were assessed with the Spanish version of the PRISM and, a mean of 3.9 days later (SD=4.9), with the Spanish version of SCID and SCID-II. The LEAD procedure was conducted at the end of treatment period, after a mean of 13.8 days of admission (SD=5.8).
Although most psychiatric diagnoses were assessed, analysis was done only when the diagnosis was made by all three systems. These included borderline personality disorder and antisocial personality disorder.
Data Analysis Strategy
For each diagnosis assessed by LEAD as the gold standard, the probability of observing an equal or greater number of patients (or alternatively equal or smaller number of patients) with the diagnosis than obtained by the PRISM or SCID was calculated by the McNemar’s test based on binomial distribution (df=1). The cutoff p value for statistical significance was 0.02 (two-tailed) according to Bonferroni’s method
(22). The agreement between diagnoses in each pair of assessments was calculated by using the kappa statistic and the 95% confidence interval
(23). Values range from 1.00 (perfect agreement) to –1.00 (total disagreement). Kappa values of 0.75 and higher indicate excellent agreement, values from 0.65 to 0.74 indicate good agreement, values of 0.40 to 0.64 indicate fair agreement, and values of 0.39 or lower indicate poor agreement
(24). Kappa values were calculated by using two-by-two comparisons of each diagnosis obtained by the different methods, then pairs of diagnoses obtained with PRISM and LEAD (and SCID and LEAD). Only disorders with a prevalence lower than 10/probability of positive response or 10/(1–probability of positive response) were considered to have a reasonable sample size to calculate kappa values
(25). For concordance in group diagnosis (i.e., affective disorders) when one patient had more than one affective diagnosis, only one diagnosis was taken into account to calculate the kappa index among the three instruments. Subjects could be counted in more than one diagnostic category if multiple diagnoses were given.
The nonparametric bootstrapping with 1,000 replicates to obtain 95% confidence intervals (bias-corrected) was used to calculate the significance of the difference between the kappas (PRISM-LEAD and SCID-LEAD)
(26). The difference was significant at the 5% level if zero was not included in the 95% confidence interval.
McNemar’s test and the kappa statistics with 95% confidence intervals were analyzed with the SPSS 10.0 statistical software package (Chicago, SPSS, Inc.). Bootstrapping was carried out with the STATA 7 program (College Station, Tex., Stata Corp.).