The Predictive Ability of the Classification of Violence Risk (COVR) in a Forensic Psychiatric Hospital

This research was approved by the appropriate governing bodies of each institution. A Certificate of Confidentiality was obtained from the National Institute of Mental Health, which prevented research documents from being subpoenaed. The study was designed as a prospective evaluation of the patients in a 1,200-bed long-term care psychiatric hospital in northern California. Approximately 80% of the patients hospitalized at this facility are under a forensic commitment. Only patients committed as “not guilty by reason of insanity” (NGRI) or as a “mentally disordered offender” (MDO) were eligible to participate. Patients were asked about their willingness to participate in the research on the basis of their commitment status. The MacArthur Competence Assessment Tool for Clinical Research (9) was administered to ensure that the patient had the capacity to provide informed consent. Those deemed capable were then asked to review and sign an approved consent form. A total of 146 patients were administered the COVR between October 1, 2007, and December 31, 2008, as one component of the study.

The COVR (7) is a computer-administered risk assessment developed from the “clinically feasible” iterative classification tree generated from the MacArthur Violence Risk Assessment Study (10). Administration is based on a brief record review and a ten-minute interview. The COVR is intended to be used as an adjunct to clinical judgment to estimate the short-term risk of aggression (less than six months). It assesses patients on 44 risk factors in estimating violence risk, although for any given patient not all factors are relevant. In our study, the COVR scores were not adjusted.

In addition to the COVR, three widely used instruments found to be associated with community violence were employed as violence risk assessments: the Psychopathy Checklist-Revised (PCL-R) (11), the Historical Clinical Risk Management scale (HCR-20) (12), and the Violence Risk Appraisal Guide (VRAG) (13). The PCL-R includes 20 items measuring various traits associated with psychopathy. The HCR-20 contains three subscales: historical, clinical, and risk management items. Scores from these scales are combined to allow the clinician to estimate an overall risk of low, moderate, or high. The VRAG comprises 12 items that combine to produce a total score. In the validation studies, VRAG scores were shown to be statistically related to risk of violent offending at two time points—seven and ten years after administration.

Violence risk assessments were conducted by four doctoral-level psychologists and a master's-level research associate. Group training was conducted until total scores on all risk instruments did not differ between raters by more than 3 points and scores on single items on either the PCL-R or the HCR-20 did not differ by more than 1 point. Interrater reliability varied on the basis of the instrument used (average intraclass correlations varied from a low of .84 for the risk management subscale of the HCR-20 to .97 for the total PCL-R score). Routine recalibrations were performed regularly to prevent rater drift.

We used two methods to track aggression in the hospital: special incident reports, which nursing staff are required to complete after an unusual incident, and nursing logs, which are maintained for each unit in the hospital. Using these two methods, we captured most of the physical aggression exhibited in our facility. We defined physical aggression as any physical contact initiated by a patient wherein the intent was to do physical harm, such as pushing, kicking, or biting or using a weapon to threaten others. Unlike the MacArthur Violence Risk Assessment Study, in our study acts of physical aggression did not necessarily result in physical injury. We recognize that some aggression occurs outside the view of staff and would be documented only if physical injury resulted. For each participant the study period was defined as the 20-week period after completion of the COVR.

Point biserial correlations were performed to examine bivariate associations between the predictor variables and the dichotomized measure of aggression (0=no aggression, 1=one or more aggressive incidents). Because the modal number of aggressive incidents was zero, we supplemented analyses with receiver operating characteristic (ROC) curve analyses. The accuracy of prediction was examined with chi square analyses to determine the percentage of patients who exhibited an aggressive act in three ranges of COVR scores—low (which included low and very low), average, and high (which included high and very high). All analyses were conducted with SPSS, version 18.

Of the 146 patients in the sample, most were male (N=126, 86%) and Caucasian (N=82, 56%). The modal commitment offense (the most serious offense was coded as the commitment offense) was assault or battery or both (N=58, 40%), followed by homicide offenses (N=34, 23%). Most participants in the study were committed under the NGRI statute (N=109, 75%), and the rest were committed as MDOs. Most participants had a diagnosis of a schizophrenia spectrum disorder (N=107, 73%) or bipolar disorder (N=13, 9%). The others had diagnoses of substance use disorders (N=10, 7%), psychotic disorder not otherwise specified (N=7, 5%), and other disorders (N=9, 6%). The mean±SD age of the participants was 46.1±9.4 (range 24.2–85.9 years). The mean length of hospital stay at the time of the completion of the COVR was 7.1±10.1 days. Although COVR scores were normed for individuals between the ages of 18 and 60, removing data for the six participants who were 61 or older when the COVR was administered did not alter the results in any way. Therefore, all data from these participants were retained.

Nineteen (13%) of the participants scored in the very low range on the COVR, 65 (44%) scored in the low range, 42 (29%) in the average range, 14 (10%) in the high range, and six (4%) in the very high range. For the other risk assessment instruments, the mean±SD total PCL-R score was 16.9±8.5 (possible scores range from 0 to 40, with higher scores indicating more characteristics of psychopathy), the mean total HCR-20 score was 26.1±6.6 (possible scores range from 0 to 40), and the mean VRAG score was 8.2±10.7 (possible scores range from −26 to 28, with higher scores indicating an increased likelihood of future violent offending). The COVR score had a correlation of .535 with the PCL-R score, .461 with the total HCR-20 score, and .392 with the VRAG score.

Overall, 22 patients (15%) exhibited 53 acts of physical aggression in the 20-week period after completion of the COVR. The mean±SD length of time between COVR administration and the act of aggression and was 69.0±48.3 days. Twenty-eight percent of the aggressive acts (N=15) were committed within the first month after the assessment, and 40% (N=21) were committed in months 4 and 5.

No differences in demographic characteristics were found between patients who committed aggressive acts and those who did not. In regard to the clinical assessments, the multivariate analysis of variance was statistically significant (Wilk's λ=.867, F=5.39, df=4 and 141, p<.001).

Univariate analyses indicated that scores on all risk assessments differed between patients who did and did not commit an aggressive act. Chi square analyses indicated that COVR scores were strongly associated with overall aggression: only six of 84 (7%) patients who scored in the low or very low categories committed an act of aggression, whereas nine of 20 (45%) who scored in the high or very high range committed an aggressive act (χ²=18.2, df=2, p<.001).

Table 1 presents the means and standard deviations for all risk instruments for aggressive patients, the bivariate correlations between scores on the risk instruments and patient violence, and the results of the ROC analyses.

The most important finding of this study is the strength of the relationship of the COVR to aggression. Our data indicate that the COVR was as effective in predicting aggression as other, lengthier risk assessments. The analyses indicated that all risk measures were associated with overall aggression. No instrument was statistically superior to any other in predicting aggression. The strength of the relationship between the COVR and the standard risk assessments is intriguing. The COVR can be administered in approximately 20 minutes, which includes a brief record review. Scoring the three other risk instruments used in this study requires a much more comprehensive record review, supplemented by a lengthy interview. In long-term care facilities with years of records, this record review can require several hours. These findings are consistent with recent research suggesting that the predictive accuracy of all risk instruments is related to their ability to quantify criminal risk (14). It appears that the COVR is no exception.

In contrast to recent research, our data indicate that standard risk instruments (such as the VRAG) had modest relationships to institutional aggression. All analyses conducted indicated that all measures were modestly associated with institutional aggression; ROC analyses were consistent with results of other published studies of community aggression. These results suggest that aggression seen in institutions may not be substantially different from that found in the community and that any selected risk instrument will likely perform equally well regardless of whether it is applied in the community or in an institution.

Although our data were collected in the United States, the proportion of patients that fell into the five risk categories is strikingly similar to that reported by Snowden and his colleagues in their study in the United Kingdom (8). The majority of our patients scored in the low-risk range, with the next highest percentage in the average-risk range. The reasons for this are fairly clear: the U.K. sample was more comparable to our sample than the sample used to construct the COVR (15) in numerous ways, including type (both were samples of forensic patients rather than civilly committed patients) and gender (both were predominantly male). Although no data were provided for the U.K. sample regarding offense history, presumably most patients were violent felons. In contrast, the construction sample had a much larger percentage of women (41%) and few violent offenders (22%).

Determining which patients are most likely to exhibit aggression in institutions is important for individuals who provide treatment. Our data indicate that the COVR is an effective tool in this regard. The COVR has distinct advantages over other risk instruments: ease of administration with substantially less staff time required and individualized risk estimates for each patient by means of various algorithms. Currently, our hospital is administering the COVR in a real-world setting; all patients admitted are screened for risk, and COVR scores are adjusted on the basis of the clinical judgment of the clinician. We provide immediate feedback to treatment teams when patients score in the high or very high risk category. In this way we are attempting to decrease acts of aggression by identifying individuals at higher risk so that environmental factors can be manipulated. In the future, these estimates may be used to make decisions about unit assignment and specific treatment approaches.

This research was conducted as part of a collaborative effort between Napa State Hospital and the Department of Psychiatry and Behavioral Sciences, University of California, Davis, School of Medicine, funded by the California Department of Mental Health.

The authors report no competing interests.