An Empirical Study of Different Diagnostic Criteria for Delirium Among Elderly Medical Inpatients

This study was a secondary analysis of data collected in two concurrent studies on delirium: a randomized controlled trial of management of delirium and a nonexperimental prospective study of the prognosis of delirium that included nondelirious subjects. The study was conducted at St. Mary's Hospital, a 400-bed primary acute care university-affiliated hospital in Montreal. A study nurse was responsible for patient screening and enrollment in the two studies. Only patients aged 65 years and over who were admitted from the emergency department to the medical services were included in the studies. We excluded patients with a primary diagnosis of stroke, those admitted to the oncology unit, those admitted to the intensive care unit or the cardiac monitoring unit unless they were transferred to a medical ward within 48 hours of admission, and those who did not speak English or French.

The study nurse administered the Confusion Assessment Method (CAM)¹⁰ to subjects whose initial Short Portable Mental Status Questionnaire (SPMSQ)¹¹ score was 3 or more or whose nursing notes indicated symptoms of delirium. The study nurse used various sources to complete the CAM—chart, family, and nursing staff—and assessed the patient at several points in time if necessary. Delirium was diagnosed if the patient met DSM-III-R criteria for delirium. Those whose initial SPMSQ score was less than 3 and those whose initial score was 3 or more but who did not meet DSM-III-R criteria for delirium were rescreened with the SPMSQ daily for the following week. The CAM was readministered if the SPMSQ score increased or there was evidence from the nursing notes of symptoms of delirium. Nondelirious subjects were selected from among the patients who were screened for delirium and were found free of this condition. To balance the distributions of age and prior cognitive impairment among patients with and without delirium, the sampling method for nondelirious patients took into account the patient's age and initial SPMSQ score. Thus, nondelirious subjects were selected from among the patients aged 70 years and over, and only a subsample of patients with SPMSQ scores of less than 3 were included. Prior to enrollment, cognitively intact patients were asked to provide informed consent, and patients with delirium or dementia were asked for their assent to participate in the study and a family member was asked to provide informed consent.

The Delirium Index (DI)¹² and the Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE)¹³ were completed within 24 hours of the diagnosis by a research assistant who had no knowledge of the patient's diagnosis. The research assistant had been trained only to rate the presence and severity of the seven symptoms on the DI at one point in time.

Other measures completed by the study nurse or research assistant included the Clinical Severity of Illness,¹⁴ the Mini-Mental State Examination (MMSE),¹⁵ the Barthel Index,¹⁶ and, for premorbid data, the Instrumental Activities of Daily Living questionnaire from the Older Americans Resources and Services Instrument (OARS).¹⁷

The SPMSQ is a widely used, observer-rated, 10-item questionnaire that evaluates orientation, memory, and concentration. Scores range from 0 (no impairment) to 10 (severe impairment). The test-retest reliability is reported to be 0.8.¹¹ At a cut-off point of three or more errors, the instrument is reported to have a sensitivity of 0.84 and a specificity of 0.89 in identifying medical inpatients with organic brain syndromes.¹⁸

The CAM is a structured instrument that operationalizes the ten symptoms of delirium specified in DSM-III-R: acute onset, fluctuating course, inattention, disorganized thinking, altered level of consciousness, disorientation, memory impairment, perceptual disturbances, psychomotor agitation or retardation, and sleep/wake disturbance. The CAM diagnosis of delirium was previously validated against the clinical judgment of a psychiatrist and found to have a sensitivity of 97% and a specificity of 92%.¹⁰ In our study, the study nurse's diagnosis of delirium had a sensitivity of 0.89 and a specificity of 1.0 compared with a consensus diagnosis.¹⁹ In an earlier study,²⁰ interrater agreement (kappa) for the individual symptoms of delirium ranged from 0.64 to 1.0 (n = 11). In this study, they ranged from 0.28 to 1.0 (n = 14). The earlier study included only severe and typical cases of delirium, whereas this study included a broader range of cases, which accounts for the lower kappa values.

The DI is an instrument developed by our group for the measurement of the severity of symptoms of delirium. It is based solely on observation of the individual patients, without additional information from family members, nursing staff, or the patient's medical chart. The first five questions of the MMSE constitute the minimum basis of observation. Seven of the ten symptoms assessed on the CAM (including disorders of attention, thought, consciousness, orientation, memory, perception, and psychomotor activity but excluding acute onset, fluctuation, and sleep/wake disturbance) are rated on the following impairment scale: 0 = absent, 1 = mild, 2 = moderate, 3 = severe. The total score ranges from 0 (no symptoms) to 21 (maximum severity). Patients who are unresponsive are scored at maximum severity on inattention, disorganized thinking, disorientation, and memory impairment. We established the interrater reliability, concurrent criterion validity, and sensitivity to change of the DI in 30 patients with delirium who were rated simultaneously and independently by one or two research assistants and a geriatric psychiatrist on up to four occasions. The concordance coefficient between DI ratings by psychiatrists and research assistants was 0.88, and between two research assistants it was 0.78. Pearson correlation coefficients were 0.84 between the DI and the Delirium Rating Scale (DRS) and 0.71 between change in the DI and the DRS. The interrater agreement (kappa) for the individual symptoms of delirium in this study ranged from 0.48 to 1.0 (n = 43).

The IQCODE assesses the presence of dementia prior to admission on the basis of the responses of an informant who has known the patient for at least 5 years; the score is an average of the 16 item scores, each rated from 1 (much improved) to 5 (much worse). In the original publication,¹³ in which a cutoff of 3.32 was used to identify patients with dementia, the sensitivity was 0.79 and the specificity was 0.82. In a later validation study of a French-language version of the questionnaire conducted in Quebec,²¹ a cutoff of 3.6 had a sensitivity of 0.75 and a specificity of 0.96. We used a cutoff of 3.51 to identify patients with dementia.

The Clinical Severity of Illness was assessed at baseline by the study nurse; scores range from 1 (minimal) to 9 (most severe). The MMSE measures cognitive function on a scale of 0 (poor) to 30 (excellent). The Barthel Index measures independence in personal care activities; we used a modified scoring system²² that ranges from 0 (dependent) to 100 (independent). The OARS Instrumental Activities of Daily Living questionnaire was administered to an informant to assess premorbid function (prior to current illness but not more than 1 month before admission); the scale ranges from 0 (dependent) to 16 (independent).

Patients were classified into four groups according to their delirium and dementia status: both delirium and dementia, delirium only, dementia only, and neither delirium nor dementia. Descriptive statistics of demographic and clinical characteristics and the frequency of symptoms measured by the CAM and the DI were calculated separately for each of these groups.

The accuracy of each set of diagnostic criteria was estimated by two different approaches. In the first, we assumed arbitrarily that the DSM-III-R was the criterion standard and estimated the sensitivity, specificity, positive predictive value, and negative predictive value of DSM-III, DSM-IV, and ICD-10 criteria with respect to it. We also analyzed the data for criterion A—clouding of consciousness with inattention—in three different ways: clouding of consciousness alone; clouding of consciousness and inattention; or clouding of consciousness or inattention. We excluded from the analysis the criterion requiring a putative medical cause of delirium because this criterion was met by all of our elderly medical inpatients and therefore had no discriminant value.

In the second approach, we used a latent class model^23,24 to simultaneously estimate the prevalence of delirium and the sensitivity and specificity of DSM-III-R and the three options for criterion A in DSM-III, DSM-IV, and ICD-10. Latent class models have been widely used to estimate prevalence and diagnostic accuracy from non–gold standard diagnostic tests. This approach does not require DSM-III-R to be arbitrarily selected as a criterion standard and hence is more realistic, since perfect tests seldom exist in practice.

We analyzed results from pairs of tests—DSM-III-R with each of the other tests—using a latent class random effects model described in Dendukuri and Joseph²⁵ that adjusts for the dependence between tests and uses a Bayesian approach for inference. The Bayesian approach is advantageous because it allows us to explicitly account for prior uncertainty in the accuracy of DSM-III-R in the analysis. Prior information on the sensitivity and specificity of the CAM assessment (using DSM-III-R criteria) for a similar population was available from a previous study by our group.¹⁹ The prior means (and 95% credible intervals) of the sensitivity and specificity of DSM-III-R were 0.83 (0.61, 0.97) and 0.94 (0.79, 0.99), respectively. An exact analytical solution is impossible for this model, so we used a Gibbs sampler, as described in Joseph et al.,²⁶ in which random samples are drawn from the posterior distributions of the parameters by simulation methods. After ensuring convergence, summary statistics of the parameters can be estimated on the basis of these random samples. We ran 10,500 iterations of the Gibbs sampler, the first 500 to ensure convergence and the next 10,000 for inference.