Effectiveness of Community Treatment Orders? Two Studies With Unusual Outcomes and an Atypical Population

TO THE EDITOR: The two articles by Segal and colleagues on the effects of community treatment orders (CTOs) and based on linked administrative data from the Australian state of Victoria raise more questions than provide answers (1,2). In both, patients placed on a CTO are compared with a similar number of people in a voluntary control group, and the investigators adjusted for potential confounders with multivariate analysis using propensity scores. Highly selected and unusual outcomes are reported in a patient population that is atypical of anywhere else. One example is the use of CTOs in cases of dementia. Clinicians and decision makers should therefore be wary of changing practice on the basis of these findings.

Both studies followed patients from 2000 to 2010, but the reports do not indicate the actual length of follow-up of each person from placement to a specified outcome. Studies of CTO effectiveness usually specify that an outcome occur within a certain period of CTO placement or study entry as a control, such as one or two years. Alternatively, studies take time of follow-up into account by using survival analyses (3).

In the article on the effect of CTOs on subsequent health service use (1), the main, and only, outcome for which adjusted results are presented is the mean number of bed-days per admission over the ten years of the study for CTO patients and those in the control group. The report shows a reduction per post-CTO admission in bed-days compared with the control group. However, unadjusted data—no adjusted results are presented—show that the number of admissions and total bed-days were higher among the patients receiving CTOs compared with the control group. It is unclear why adjusted results for these data were not also presented and given the same prominence as bed-days per admission. This is the outcome most other authors in the field use (4,5). In particular, why were outcomes not reported within a certain period of CTO placement? How can a single, hypothetical CTO placement affect inpatient health service use for an entire decade? Data are presented for average bed-days per CTO, but this is for the admission preceding, not after, CTO placement. The choice of outcome is even more puzzling given that Segal himself reported changes in bed-days before and after CTO placement compared with the control group in an earlier analysis of the effect of CTO placement in Victoria from 1990 to 2000. Interestingly, this earlier study reported that CTO recipients had “more inpatient days, and a longer duration of restrictive care—that is, combined inpatient and conditional release periods” even after analyses adjusted for confounders (6).

The lack of a predefined time for follow-up mentioned previously also means the investigators could not study the effect of CTOs on psychiatric symptoms, which is a pity given that CTOs have been shown in several studies to have no effect on symptoms (4,5,7).

The other article, on mortality, shows that CTOs reduced all-cause mortality compared with the control group (2). However, the sample was somewhat atypical of CTO use in other jurisdictions. Victoria has the highest rate of CTO use worldwide (8), including in cases of dementia (2). This is of particular relevance when considering mortality, given that 20% of CTO cases in the study by Segal and colleagues involved people with dementia and, importantly, these people accounted for 33% of the deaths (2). Where mortality was compared between those receiving CTOs and the control group, the greatest difference between groups was for dementia; in fact, it was the only diagnosis where CTOs had any significant effect on mortality (2, see Table 1). Unfortunately, these are just unadjusted results; only the findings for overall mortality (including those due to dementia) were adjusted with propensity scores. The authors report that they adjusted for baseline diagnosis in these analyses, but this does not completely address the problem. The best way of removing the effect of dementia is a sensitivity analysis of the effect of excluding the patients with dementia from the adjusted analyses. The other issue is that the follow-up period could range from a few days to the full ten years of the length of the study, depending on when the death occurred.

Furthermore, it is important to separate out the direct effect of CTO placement and the influence of changes in community contacts after placement. For instance, in a West Australian (WA) study of the effect of CTOs on one- and two-year mortality rates, any influence vanished after adjustment for post-CTO community-outpatient psychiatric contacts. This finding suggests that increased contact (potentially achievable through other means such as assertive community treatment) rather than the CTO itself was the key factor (9). Whether Segal and colleagues did something similar in their study is unclear. They included a variable termed “treatment days per community care episode,” but because they did not specify a time frame within which the death occurred, values could be from many years previously. The unusual nature of this variable is suggested by the fact that it doesn’t have any influence on mortality, whereas in the WA study community contacts had a very strong influence. It is unclear why the authors did not simply count community contacts for CTO recipients and those in the control group.