Clinical and Methodological Factors Related to Reliability of the Best-Estimate Diagnostic Procedure
Publication: American Journal of Psychiatry
Abstract
OBJECTIVE: The reliability and accuracy of the best-estimate diagnostic procedure were examined, and factors associated with reliability were determined. METHOD: The subjects were 134 members of large multigenerational pedigrees densely affected by bipolar disorders or schizophrenia. Three best-estimate diagnoses were derived: first, by a research psychiatrist and research assistant unblind to the relatives' diagnoses; second, by two blind independent psychiatrists; third, by a panel of four blind psychiatrists. The subjects were characterized on several clinical and methodological variables, which were used to compare the agreements of two types of best-estimate diagnoses with the disagreements. RESULTS: There was satisfactory agreement between the unblind and blind consensus best-estimate diagnoses and between the two blind independent psychiatrists. Latent class analyses revealed that limited sensitivity was the main source of imperfect reliability. Confusability analyses revealed that the most problematic diagnostic distinctions involved schizoaffective disorder, which was confused with schizophrenia, bipolar I disorder, and schizophreniform disorder. Blindness significantly affected diagnostic outcome in latent class analyses. Moreover, for diagnostic disagreements, unblind diagnoses had greater continuity with the most predominant diagnosis in the pedigree than did blind diagnoses. Diagnostic disagreements were associated with the presence of mixed affective and psychotic symptoms, less diagnostic certainty, and shorter duration of illness. CONCLUSIONS: These results suggest that it is possible to identify cases that are more likely to lead to diagnostic disagreements in family and epidemiological studies and that blind diagnoses may help to prevent false positive diagnoses, which may be particularly detrimental to genetic linkage analyses. (Am J Psychiatry 1997; 154:1726–1733)
Family, twin, and adoption studies strongly support the importance of genetic factors in the etiology of schizophrenia (1–3) and bipolar disorders (4). However, it has been difficult thus far to identify consistent genetic linkages for either disorder, and the imperfect accuracy of diagnostic methods is probably one of the obstacles (5–8). Indeed, linkage analysis can be extremely sensitive to diagnostic misclassification (9, 10). Our review (8) of the diagnostic methods used in genetic linkage studies of schizophrenia and bipolar disorders identified several methodological limitations that may have decreased the studies' power to detect linkage, among which are the following: 1) few studies assessed the reliability of diagnoses; 2) in most studies, diagnoses were made by investigators who were not blind to relatives' diagnoses; 3) few studies used a best-estimate diagnosis procedure, i.e., a diagnosis based on personal interview, family history from family informants, and medical records. Such a best-estimate diagnosis is considered the most valid method for diagnosing psychiatric disorders (5, 11, 12).
Unfortunately, relatively little is known about the reliability of the best-estimate diagnosis procedure, since only a few studies have examined its interrater (8, 13–16) or test-retest (15, 17, 18) reliability. These studies generally indicated satisfactory reliability, although even such a modest level of disagreement may present serious problems for genetic linkage studies. Moreover, these studies relied on kappa statistics, which suffer from several limitations.
First, the kappa is influenced by the sensitivity and specificity of the diagnostic method and by the prevalence of the disorder; poor specificity is probably more worrisome for linkage analyses, owing to the detrimental effect of the false positive diagnoses that result. A group of strategies using latent class analyses has been proposed to address these problems (19, 20). In the absence of a diagnostic gold standard, latent class analyses assume that the true classification of an individual is unknown but that it exists at some unobserved or latent level and that the various diagnostic assessments are imperfect indicators of this latent classification. By using latent class analyses, it is possible to estimate the sensitivity and specificity with which each of these diagnostic assessments reflects this latent classification. We are aware of no previous study using latent class analyses to evaluate the accuracy of best-estimate diagnoses.
Second, the kappa does not allow identification of the source of disagreement, i.e., which disorders are difficult to distinguish from which diagnoses. For example, in a genetic linkage study of bipolar disorder, confusing bipolar I disorder with schizophrenia does not have the same implications as confusing bipolar I disorder with bipolar II disorder, given usual diagnostic hierarchies. This issue can be addressed by using confusability analyses (21), which allow identification of the most problematic diagnostic distinctions.
Third, using kappa statistics does not allow identification of factors associated with the reliability of best-estimate diagnoses. Knowing these factors could help one to identify problematic cases and implement strategies for decreasing their impact on linkage analyses (22). Despite the scarcity of such studies, two potentially important factors have been identified, i.e., a diagnosis of schizoaffective disorder (8, 14, 15, 18) and blindness to the probands' and relatives' diagnoses (8). Indeed, concerning the latter factor, we found that unblind consensus best-estimate diagnoses, compared to consensus best-estimate diagnoses made by investigators who were blind to relatives' diagnoses, were biased toward greater continuity with the most prevalent diagnosis of the pedigree (8).
Therefore, the goals of the present study were 1) to quantify the reliability and accuracy of the best-estimate diagnostic procedure and 2) to identify factors related to the reliability of best-estimate diagnoses, with a particular emphasis on blindness.
METHOD
Subjects
The subjects were drawn from large multigenerational pedigrees densely affected by schizophrenia or bipolar disorders (23). The DSM-III-R diagnoses arrived at by blind consensus best-estimate diagnosis were as follows: schizophrenia, N=43; schizoaffective disorder, N=7; schizophreniform disorder, N=4; bipolar I disorder, N=39; bipolar II disorder, N=10; major depression, single episode and recurrent, N=29; delusional disorder, N=1; psychosis not otherwise specified, N=1. Unaffected subjects were not included in the analyses, since we achieved perfect agreement on them (8). All subjects were personally informed of the objectives and the methods of the study, and they each signed a consent form approved by the ethics committee of our research center. The present study group (N=134; mean age=50 years, SD=17) is a major extension of the group used in a previous study (8).
Consensus Best-Estimate Diagnosis
The diagnostic methods have been previously described (8). Briefly, the diagnosis was based on an audiotaped Structured Clinical Interview for DSM-III-R (SCID) (24) obtained from the subject, medical records, and family history interviews with family respondents chosen for their familiarity with the subject. DSM-III-R criteria were used, with a designation of “possible,” “probable,” or “definite” to indicate the degree of certainty about the diagnosis.
In the first step, the SCID information, medical records, and family history were reviewed by a research psychiatrist (M.M. or M.-A. R.) in close collaboration with an experienced research assistant (psychiatric nurse) (L.C. or M.T.), who had spent on average 25 hours gathering data and performing several clinical and epidemiological ratings on the subject from the data gathered from the three sources. The diagnosis made by this field team was termed “unblind consensus best-estimate diagnosis” because, inevitably, it could not be blind to the relatives' diagnoses.
In a second step, the raw information, including the interview and the audiotape, the medical records, and the family history, was edited for family relationships, unblind consensus best-estimate diagnosis, and previous clinical diagnoses. The edited information was reviewed independently by two blind research psychiatrists (D.C., J.-P.F., N.M., L.N., A. Pirés, H.W., A.-M.P., Y.G., C.D., J.-C.L., and/or A. Potvin). Therefore, besides the details edited to secure blindness, the blind and the unblind psychiatrists were exposed to similar information. These diagnoses were termed “blind independent best-estimate diagnoses.”
In the third step, the edited information was presented to and discussed by a panel of four psychiatrists, including the two first blind psychiatrists. These diagnoses were termed “blind consensus best-estimate diagnoses.”
Reliability and Accuracy
Interrater reliability for the agreement between the unblind and blind consensus best-estimate diagnoses and between the blind independent best-estimate diagnoses was computed by using a weighted kappa (25), for which 1- and 2-point disagreements on certainty had weights of 0.67 and 0.33, respectively.
We also used diagnostic confusability analyses, as described by Übersax (21). Confusability (C) is defined as the number of times (N) one rater diagnoses A and the other rater diagnoses B (NAB) plus the number of times the opposite occurs (NBA), divided by the geometric mean of the frequencies with which each of the two appears in any pair of opinions: C=(NAB+NBA)/(NA×NB)½. (NA and NB are the numbers of times A and B respectively are diagnosed by both diagnosticians.)
Latent class analyses were performed by using the program developed by Szatmari et al. (20), yielding, for each diagnosis, estimates of sensitivity, specificity, and prevalence and a global fit. Three diagnoses were included in the analyses, i.e., the two blind independent best-estimate diagnoses and the unblind consensus best-estimate diagnosis. Because of the assumption of independence of observations that underlies latent class analyses, we could not include the blind consensus best-estimate diagnosis, because it was not independent from the blind independent best-estimate diagnoses.
Prediction of Disagreements
Variables potentially associated with disagreements were chosen on the basis of prior evidence that they may predict diagnostic disagreements (8, 26, 27). These variables were rated by the field researchers, who used the information gathered for diagnosis and were blind to the present research questions and to the concordance between the various best-estimate diagnoses. The independent variables included 1) number of psychiatric hospitalizations; 2) age at onset of the first episode meeting the criteria for diagnosis; 3) Global Assessment Scale (28) score during the premorbid period (5 years preceding onset) and during periods of stabilization between hospitalizations; 4) duration of illness since onset; 5) gender; 6) presence of mixed psychotic and affective symptoms (considered present when either the unblind or the blind consensus best-estimate diagnosis was schizoaffective disorder or affective disorder with mood-incongruent features; repeating the analyses when only the unblind or the blind consensus best-estimate diagnosis was used to define mixed psychotic and affective symptoms led to similar results); 7) level of certainty (definite versus probable/possible) of the unblind field consensus best-estimate diagnosis (using the level of certainty of the blind consensus best-estimate diagnosis led to similar results); 8) quality of information, rated as excellent/very good versus acceptable/poor (scale provided on request); and 9) the predominant diagnosis in the pedigree, either schizophrenia or bipolar disorders.
Three types of disagreements were analyzed. First, 20 disagreements on diagnosis between the unblind and blind consensus best-estimate diagnoses were compared to 86 agreements between the unblind and blind consensus best-estimate diagnoses on diagnosis and level of certainty. Second, 14 diagnostic disagreements between the blind independent best-estimate diagnoses were compared to 100 agreements between the independent best-estimate diagnoses. Third, 28 disagreements on certainty about the same diagnosis between the unblind and blind consensus best-estimate diagnoses were compared to the 86 agreements on certainty. We did not analyze disagreements on certainty between the blind independent best-estimate diagnoses because there were only nine such instances.
We first performed univariate logistic regression on each independent variable (Wilcoxon tests for continuous variables led to similar conclusions). In a second step, we used multivariate stepwise (forward) logistic regression, with a p value of 0.05 as the threshold for inclusion in the final model. All statistical tests reported were two-tailed.
RESULTS
Reliability
The overall weighted kappa for agreement between the blind and the unblind consensus best-estimate diagnoses was 0.69 (95% confidence interval=0.62–0.76). For individual diagnoses (table 1), according to standard guidelines for judging agreement (25), agreement could be considered as very poor for schizoaffective disorder, fair to good for schizophreniform disorder and recurrent major depression, and excellent (e.g., ≥0.75) for the other diagnoses.
The overall weighted kappa for agreement between blind independent best-estimate diagnoses was 0.80 (95% confidence interval=0.72–0.88). Agreement could be considered good for schizoaffective disorder and bipolar II disorder and excellent for the other diagnoses (table 1). It is noteworthy that, qualitatively, for every diagnosis except bipolar II disorder the level of agreement between the blind independent best-estimate diagnoses was better than the agreement between the unblind and blind consensus best-estimate diagnoses, although the confidence intervals overlapped, except for schizoaffective disorder.
Confusability
Table 2 shows the confusability coefficients. For unblind versus blind consensus best-estimate diagnoses, schizoaffective disorder was involved in all of the three situations with the highest confusability coefficients, i.e., schizoaffective disorder versus schizophreniform disorder, bipolar I disorder, and schizophrenia, and the coefficient was greater than 0.1 in all three instances. In decreasing order of diagnostic difficulty, other high confusability coefficients included those for single-episode major depression versus recurrent major depression, schizophrenia versus schizophreniform disorder, bipolar I disorder versus schizophreniform disorder, bipolar I disorder versus bipolar II disorder, and bipolar I disorder versus schizophrenia.
Conversely, among the blind independent best-estimate diagnoses no confusability coefficient reached a level of 0.1. In this situation, the most difficult distinctions were, in decreasing order, schizoaffective disorder versus bipolar I disorder, recurrent major depression versus single-episode major depression, bipolar I disorder versus bipolar II disorder, schizophrenia versus schizophreniform disorder, and bipolar II disorder versus single-episode major depression.
Latent Class Analyses
For each diagnosis, two latent class analysis models were contrasted. First, in the full model, sensitivity and specificity for the independent psychiatrists were constrained to be similar, since these diagnoses were derived by a single group of diagnosticians who were permuted; however, sensitivity and specificity were allowed to differ between the blind independent best-estimate diagnoses and the unblind consensus best-estimate diagnoses. Second, in the restricted model, sensitivity and specificity were constrained to be similar for all three types of diagnoses. Therefore, comparing the chi-square goodness of fit values for the full and the restricted models provides a test (df=2) of the null hypothesis that the degrees of diagnostic accuracy are similar among the diagnostic methods, e.g., that blindness does not influence diagnostic accuracy.
Table 3 shows the results of these latent class analyses. The overall fit of the full models was appropriate (p>0.05) for all diagnoses except schizoaffective disorder (χ2=9.91, df=3, p<0.05). Specificity was very good, the lowest value being 0.89, for bipolar I disorder from unblind consensus best-estimate diagnoses. There were more-severe problems with sensitivity, particularly for schizoaffective disorder derived from unblind consensus best-estimate diagnoses. Therefore, these analyses identify sensitivity as the primary source of imperfect reliability.
For schizophreniform disorder and bipolar II disorder, the full model was not found to be significantly better than the restricted one, meaning that there was no evidence for an effect of blindness. Conversely, for the following disorders the fit was significantly better for the full model: schizophrenia, schizoaffective disorder, bipolar I disorder, recurrent major depression, and single-episode major depression (table 3).
Effect of Blindness
The potential influence of blindness to probands' and relatives' diagnoses was more directly tested by examining the 20 cases of diagnostic disagreement between the unblind and blind consensus best-estimate diagnoses. For each of these 20 cases, we determined which diagnosis, unblind or blind, had greater continuity with the most predominant diagnosis of the pedigree, according to diagnostic hierarchies used in our linkage studies (23), which are shown in table 4. For example, in a predominantly schizophrenia pedigree, a diagnosis of schizophrenia would be considered to have greater continuity than a diagnosis of bipolar disorder. We found greater continuity for the unblind consensus best-estimate diagnosis in 19 cases (95%). If blindness had no effect on the diagnoses, the blind and the unblind diagnoses would each have had a 50/50 chance of having greater continuity. We tested the statistical significance of this departure from a 0.50 proportion by using a binomial test and found the difference to be highly significant (p<0.0001). Since the exact placement of schizoaffective disorder is the subject of considerable controversy, we repeated the analyses by putting schizoaffective disorder at level 2 of the hierarchies for both disorders, which did not affect the results at all.
Predictors of Disagreements
Diagnostic disagreements between unblind and blind consensus diagnoses. Table 5 (first column) shows univariate comparisons of diagnostic disagreements and agreements between unblind and blind consensus diagnoses. The following variables were associated with diagnostic disagreements: mixed psychotic and affective symptoms (χ2=22.24), a predominant pedigree diagnosis of bipolar disorder (χ2=22.24), and a lower level of certainty of the unblind consensus best-estimate diagnosis (χ2=4.88).
We then used stepwise backward multivariate logistic regression to determine the best set of predictors. The final best-fitting model included three variables: 1) mixed psychotic and affective symptoms (χ2=17.53, odds ratio=23.26, p<0.0001); 2) shorter duration of illness (χ2=4.88, odds ratio=0.63 for a 5-year increment, p<0.05); and 3) lower level of certainty of the unblind consensus best-estimate diagnosis (χ2=9.43, odds ratio=12.82, p<0.01). Additional logistic regressions revealed that the effect of the predominant pedigree diagnosis was lost whenever the presence of mixed psychotic and affective symptoms was included in the model, because 29% of the subjects in bipolar disorder pedigrees had a diagnosis of mixed psychotic and affective symptoms and only 11% of the subjects in schizophrenia pedigrees had mixed symptoms. When a predicted probability of diagnostic disagreement of 0.14 was used as a cutoff, this model had sensitivity and specificity of 0.80 and correctly classified 80% of the subjects.
Diagnostic disagreements between blind independent diagnoses. The second column of table 5 shows univariate comparisons of diagnostic disagreements and agreements between two independent psychiatrists. These analyses were performed to disentangle the effect of blindness from the effects of other independent variables in the previous analyses comparing unblind and blind consensus best-estimate diagnoses. It is noteworthy that only three out of 14 disagreements between the blind independent best-estimate diagnoses overlapped the diagnostic disagreements between the unblind and blind consensus best-estimate diagnoses. This limited overlap prevented these analyses from being redundant. These analyses revealed that mixed psychotic and affective symptoms did not predict disagreement between the blind independent best-estimate diagnoses, contrary to what was observed in the comparisons of diagnostic disagreements and agreements between the unblind and blind consensus best-estimate diagnoses. For the other variables the patterns were similar, although they achieved an expected lesser degree of statistical significance because of the smaller number. Multivariate analyses were not used because of the insufficient number of subjects.
Disagreements on certainty between unblind and blind consensus diagnoses. In univariate analyses, a predominant pedigree diagnosis of bipolar disorder and poorer quality of information predicted disagreement on the degree of certainty (table 5, third column). In multivariate analyses, only poorer quality of information (χ2=8.55) still yielded a significant effect.
DISCUSSION
Reliability and Accuracy
We observed in this extended (8) study group a very satisfactory degree of diagnostic agreement, comparable to what was observed in previous studies of the best-estimate diagnosis (13–16). Moreover, we certainly underestimated the kappas by excluding unaffected subjects. While such reliability figures may be satisfactory for other research purposes, this proportion of disagreements nevertheless constitutes a problem for genetic linkage studies because of their sensitivity to phenotypic misclassification (7–10).
Confusability coefficients provide important insights into the source of imperfect reliability. Schizoaffective disorder was particularly problematic, with frequent disagreements with schizophrenia, schizophreniform disorder, and bipolar I disorder. The consequences of such disagreements for linkage analyses can be considerable. For example, if unblind diagnosis were used for linkage analyses of pedigrees with predominantly schizophrenia, a “true” case of bipolar I disorder could end up being confused with schizoaffective disorder. In most linkage studies of schizophrenia, schizoaffective disorder is located at the first or second level of the diagnostic hierarchy, while bipolar I disorder is located at a very low level. Consequently, such a “real” case of bipolar I disorder would end up being included in the restrictive definition of the phenotype. While restricted definitions are aimed at identifying a core phenotype to minimize the risk of false positives, such diagnostic misclassifications pose the risk of introducing false positives.
Latent class analyses revealed that the main source of imperfect reliability was poor sensitivity, a finding similar to previous results by Faraone et al. (29). This result could be reassuring in the context of genetic analyses, since poor specificity, which leads to false positive diagnoses, has a more serious impact on linkage analyses than does poor sensitivity, which leads to false negative diagnoses. To verify this, we computed the positive predictive value (last two columns of table 3) and found less than optimal positive predictive value, especially for schizoaffective disorder. For example, if unblind consensus best-estimate diagnoses were used, almost all of the diagnoses of schizoaffective disorder would in fact be false positives, while if blind independent best-estimate diagnoses were used, a sizable proportion (36%) would be considered false positives. Such rates of false positives are likely to pose serious problems in linkage analyses.
Prediction of Disagreements
Five factors were found to be significantly associated with reliability.
1. Blindness. As previously reported (8), blindness to the most predominant diagnosis in a pedigree multiply affected with either schizophrenia or bipolar disorders had significant effects on diagnostic outcome, as suggested by a) the greater continuity of the unblind consensus best-estimate diagnosis with the most predominant diagnosis in the pedigree in cases of disagreement between the unblind and blind consensus best-estimate diagnoses and b) the significant difference in sensitivity and specificity between the unblind consensus and blind independent best-estimate diagnoses, revealed by the latent class analyses, for five out of seven diagnostic categories. The potential implications of these findings suggest the need for further studies. Indeed, we could locate only one other study addressing the impact of blindness to diagnoses in relatives (30); that study showed no such influence. However, that study used a relatively small number of subjects, used diagnoses only from personal interview, and focused exclusively on affective disorders; these methodological differences render comparison with the present study difficult.
2. Mixed psychotic and affective symptoms. Diagnoses involving mixed symptoms, including schizoaffective disorder and affective disorders with mood-incongruent features, were strongly associated with disagreements between unblind and blind consensus best-estimate diagnoses. However, diagnoses with mixed psychotic and affective symptoms were not associated with disagreements between two blind independent best-estimate diagnoses, suggesting that the effect of mixed psychotic and affective symptoms was linked to that of blindness. This suggests that cases with both psychotic and affective features are particularly difficult to diagnose. On the basis of clinical experience, one can speculate that the observed difficulties in the differential diagnosis of cases with mixed psychotic and affective symptoms according to DSM-III-R criteria include a) retrospective assessment of the temporal dissociation of psychotic and affective symptoms, particularly affecting the distinction between bipolar I disorder and schizoaffective disorder, which confusability analyses revealed as problematic; b) inconsistencies in psychotic episodes across a lifetime; for example, early-onset bipolar disorders often have a predominantly psychotic onset, which may render the distinction between schizophrenia and schizoaffective disorder difficult (31, 32); c) inaccurate assessment of the relative durations of affective and schizophrenic features, which makes it difficult to distinguish between schizophrenia and schizoaffective disorder (33), which was found to be a problem in the confusability analyses. Our data suggest that when such difficulties are encountered, unblind diagnosticians will be more inclined to assign diagnoses that have greater continuity with the most predominant diagnosis of the pedigree.
3. Level of certainty. Cases with lower levels of certainty were more likely to lead to diagnostic disagreements, suggesting that diagnosticians can identify the difficult cases that are more likely to lead to diagnostic disagreements.
4. Duration of illness. A shorter duration of illness predicted disagreement between unblind and blind consensus best-estimate diagnoses, a finding that is consistent with results of studies of agreement between family-history and best-estimate diagnoses (26, 27). Two competing explanations can be offered: a) cases with longer durations of illness yielded more clinical information and b) the clinical picture in cases of mixed psychotic and affective symptoms becomes clearer with time. Indeed, in many cases the onset of bipolar disorder is predominantly psychotic (31, 32), with the affective phenomena often becoming clearer over time. Unfortunately, the present data do not allow us to perform definitive tests of these competing hypotheses.
5. Quality of information. As expected, poorer quality of information was associated with disagreement on certainty. Also, it was more difficult to reach a definitive diagnosis for cases with poorer information, since additional analyses (available on request) revealed an association between quality of information and level of certainty.
Implications
These data have important practical implications for linkage and family studies of psychiatric disorders. First, a crucial issue is whether unblind or blind consensus best-estimate diagnoses should be used for linkage analyses. Examining the potential consequences of unblind versus blind diagnoses may guide a discussion of this issue. Since unblind diagnoses had greater continuity with the most prevalent diagnosis in the pedigree, they might generate more false positive diagnoses for genetic linkage analyses; e.g., the diagnosis assigned to a subject might be at too high a level in the diagnostic hierarchy. Conversely, blind diagnoses could increase the risk of false negative diagnoses, e.g., subjects' being assigned diagnoses at levels lower than should be. Moreover, latent class analyses revealed that the positive predictive value was higher in most instances for blind diagnoses, adding further evidence that unblind diagnoses are more likely to yield false positives than are blind diagnoses. Since linkage analyses are more sensitive to false positive than to false negative diagnoses, the present results suggest using blind diagnoses, as proposed by other investigators (5–7).
Second, our data and those of other groups (13–15, 18) suggest a need for a careful handling of cases with mixed psychotic and affective symptoms in genetic linkage analyses, owing to a) the modest reliability of schizoaffective disorder; b) the strong association of mixed symptoms with diagnostic disagreements and the liability of cases with mixed symptoms to be affected by lack of blindness; c) confusability analyses suggesting that the diagnostic disagreements regarding such cases affect diagnoses at very different levels of the diagnostic hierarchies. These findings suggest that it may be safer not to place schizoaffective disorder at the same level in the diagnostic hierarchy as schizophrenia or bipolar disorders when conducting linkage analyses for either disorder.
Third, the association between duration of illness and reliability suggests that longitudinal follow-up may increase diagnostic reliability.
Fourth, our results suggest that it is possible to identify cases that are more likely to lead to diagnostic disagreements. Indeed, mixed psychotic and affective symptoms, shorter duration of illness, less certainty of diagnosis, and poorer quality of information were associated with poor reliability. These variables could be used to graduate the certainty of diagnosis in linkage analyses, by using logistic regression models (22). Alternatively, probability of belonging to the latent class could be used to weight the diagnostic certainty. The use of such strategies in linkage analyses certainly requires further investigation.
Limitations
In interpreting the present findings, several limitations must be taken into account. First, the present study group included members of families with severe and highly familial psychotic disorders. Therefore, the generalizability of the present findings to families with nonpsychotic disorders or nonfamilial disorders is unknown. Second, the modest size of some subgroups may have decreased our power to detect the effect of some variables associated with reliability. Third, we do not know whether the present conclusions apply to diagnostic criteria other than DSM-III-R.
Footnote
Received June 28, 1996; revision received June 27, 1997; accepted July 15, 1997. From the Centre de recherche Universit<130> Laval Robert-Giffard. Address reprint requests to Dr. Roy, Centre de recherche Universit<130> Laval Robert-Giffard, 2601 de la Canardi<138>re, Beauport, Qu<130>bec G1J 2G3, Canada; [email protected] (e-mail). Supported by grants from the Medical Research Council of Canada, Health and Welfare Canada (National Health Research and Development Program), and Fonds de la recherche en sant<130> du Qu<130>bec. Dr. Roy and Dr. M<130>rette are supported by Scientist Awards from the Fonds de la recherche en sant<130> du Qu<130>bec. Dr. Maziade was supported by an Established Investigator Award from the National Alliance for Research on Schizophrenia and Depression (the Rebecca Simon Gallagher and Patrick Gallagher Award). This research was completed in collaboration with the departments of psychiatry of the Centre hospitalier Robert-Giffard, Sanatorium B<130>gin, H<147>tel-Dieu de L<130>vis, Clinique Roy-Rousseau, Centre hospitalier de l'Archipel, Centre hospitalier de l'Universit<130> Laval, Centre hospitalier Restigouche, H<147>pital de l'Enfant-J<130>sus, H<147>pital Jefferey Hale, H<147>pital St-Fran<135>ois d'Assise, H<147>pital St-Sacrement, H<147>tel-Dieu de Montmagny, H<147>tel-Dieu de Qu<130>bec, H<147>tel-Dieu du Sacr<130>-Coeur, and Institut Roland-Saucier and the group Institut interuniversitaire de recherche sur les populations (IREP).
References
1.
Gottesman II: Schizophrenia Genesis: The Origins of Madness. New York, WH Freeman, 1991
2.
Kendler KS, Diehl SR: The genetics of schizophrenia: a current, genetic-epidemiologic perspective. Schizophr Bull 1993; 19:261–285
3.
Maziade M, Raymond V: The new genetics of schizophrenia, in Contemporary Issues in the Treatment of Schizophrenia. Edited by Shriqui CL, Nasrallah HA. Washington, DC, American Psychiatric Press, 1995, pp 61–79
4.
Tsuang MT, Faraone SV: The Genetics of Mood Disorders. Baltimore, Johns Hopkins University Press, 1990
5.
Merikangas KR, Spence MA, Kupfer DJ: Linkage studies of bipolar disorder: methodologic and analytic issues. Arch Gen Psychiatry 1989; 46:1137–1141
6.
Baron M, Endicott J, Ott J: Genetic linkage in mental illness: limitations and prospects. Br J Psychiatry 1990; 157:645–655
7.
Egeland JA, Sussex JN, Endicott J, Hostetter AM, Offord DR, Schwab JJ, Allen CR, Pauls DL: The impact of diagnoses on genetic linkage study for bipolar affective disorders among the Amish. Psychiatr Genet 1990; 1:5–18
8.
Maziade M, Roy M-A, Fournier J-P, Cliche D, Mérette C, Caron C, Garneau Y, Montgrain N, Shriqui C, Dion C, Nicole L, Potvin A, Lavallée J-C, Pirés A, Raymond V: Reliability of best-estimate diagnosis in genetic linkage studies of major psychoses: results from the Quebec pedigree studies. Am J Psychiatry 1992; 149:1674–1686
9.
Martinez M, Khlat M, Leboyer M, Clerget-Darpoux F: Performance of linkage analysis under misclassification error when the genetic model is unknown. Genet Epidemiol 1989; 6:253–258
10.
Hodge SE, Greenberg DA: Sensitivity of lod scores to changes in diagnostic status. Am J Hum Genet 1992; 50:1053–1060
11.
Leckman JF, Sholomskas D, Thompson D, Bélanger A, Weissman MM: Best estimate of lifetime psychiatric diagnosis. Arch Gen Psychiatry 1982; 39:879–883
12.
Weissman MM, Merikangas K, John K, Wickramaratne P, Prusoff BA, Kidd KK: Family-genetic studies of psychiatric disorders. Arch Gen Psychiatry 1986; 43:1104–1116
13.
Hostetter AM, Egeland JA, Endicott J: Amish study, II: consensus diagnoses and reliability results. Am J Psychiatry 1983; 140:62–66
14.
McGorry PD, Singh BS, Copolov DL, Kaplan I, Dossetor CR, van Riel RJ: Royal Park Multidiagnostic Instrument for Psychosis, part II: development, reliability, and validity. Schizophr Bull 1990; 16:517–536
15.
Leboyer M, Maier W, Teherani M, Lichtermann D, D'Amato T, Franke P, Lépine J-P, Minges J, McGuffin P: The reliability of the SADS-LA in a family study setting. Eur Arch Clin Neurosci 1991; 241:165–169
16.
Kendler KS, McGuire M, Gruenberg AM, O'Hare A, Spellman M, Walsh D: The Roscommon Family Study, I: methods, diagnosis of probands and risk of schizophrenia in relatives. Arch Gen Psychiatry 1993; 50:527–540
17.
Klein DN, Ouimette PC, Kelly HS, Ferro T, Riso LP: Test-retest reliability of team consensus best-estimate diagnoses of axis I and II disorders in a family study. Am J Psychiatry 1994; 151:1043–1047
18.
Fennig S, Kovasznay B, Rich C, Ram R, Pato C, Miller A, Rubinstein J, Carlson G, Schwartz JE, Phelan J, Lavelle J, Craig T, Bromet E: Six-month stability of psychiatric diagnoses in first-admission patients with psychosis. Am J Psychiatry 1994; 151:1200–1208
19.
Faraone SV, Tsuang MT: Measuring diagnostic accuracy in the absence of a gold standard. Am J Psychiatry 1994; 151:650–657
20.
Szatmari P, Volkmar F, Walter S: Evaluation of diagnostic criteria for autism using latent class models. J Am Acad Child Adolesc Psychiatry 1995; 34:216–222
21.
Uuml;bersax JS: Structural analysis of diagnostic disagreements. J Nerv Ment Dis 1983; 171:199–206
22.
Baron M, Hamburger R, Sandkuyl LA, Risch N, Mandel B, Endicott J, Belmaker RH, Ott J: The impact of phenotypic variation on genetic analysis: application to X-linkage in manic-depressive illness. Acta Psychiatr Scand 1990; 82:196–203
23.
Maziade M, Raymond V, Cliche D, Caron C, Garneau Y, Nicole L, Marcotte P, Couture C, Simard C, Boivin R, Rodrigue C, Boutin P, De Braekeleer M, Martinez M, Mérette C: Linkage results on 11Q21-22 in Eastern Québec pedigrees densely affected by schizophrenia. Am J Med Genet Neuropsychiatr Genet 1995; 60:522–528
24.
Spitzer RL, Williams JBW, Gibbon M, First MB: The Structured Clinical Interview for DSM-III-R (SCID), I: history, rationale, and description. Arch Gen Psychiatry 1992; 49:624–629
25.
Fleiss JL: The measurement of interrater agreement, in Statistical Methods for Rates and Proportions. New York, John Wiley & Sons, 1981, pp 212–233
26.
Roy M-A, Walsh D, Prescott CA, Kendler KS: Biases in the diagnosis of alcoholism by the family history method. Alcohol Clin Exp Res 1994; 18:845–851
27.
Roy M-A, Walsh D, Kendler KS: Accuracies and inaccuracies of the family history method: a multivariate investigation. Acta Psychiatr Scand 1996; 93:224–234
28.
Endicott J, Spitzer RL, Fleiss JL, Cohen J: The Global Assessment Scale: a procedure for measuring overall severity of psychiatric disturbance. Arch Gen Psychiatry 1976; 33:766–771
29.
Faraone SV, Blehar M, Pepple J, Moldin SO, Norton J, Nurnberger JI, Malaspina D, Kaufmann CA, Reich T, Cloninger CR, DePaulo JR, Berg K, Gershon ES, Kirch DG, Tsuang MT: Diagnostic accuracy and confusability analyses: an application to the Diagnostic Interview for Genetic Studies. Psychol Med 1996; 26:401–410
30.
Mazure C, Gershon ES: Blindness and reliability in lifetime psychiatric diagnosis. Arch Gen Psychiatry 1979; 36:521–525
31.
Joyce PR: Age of onset in bipolar affective disorder and misdiagnosis as schizophrenia. Psychol Med 1984; 14:145–149
32.
Werry JS, McClellan JM, Chard L: Childhood and adolescent schizophrenic, bipolar, and schizoaffective disorders: a clinical and outcome study. J Am Acad Child Adolesc Psychiatry 1991; 30:457–465
33.
Nurnberger JI, Blehar MC, Kaufmann CA, York-Cooler C, Simpson SG, Harkavy-Friedman J, Severe JB, Malaspina D, Reich T: Diagnostic Interview for Genetic Studies. Arch Gen Psychiatry 1994; 51:849–859
Information & Authors
Information
Published In
History
Published online: 1 December 1997
Published in print: December 1997
Authors
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBGet Access
Login options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).