Anonymous extracts from administrative mental health and criminal justice databases were analyzed to determine arrest rates for each of the 36 months in the study period. Because the mental health and arrest data sets do not share unique personal identifiers, probabilistic population estimation (
7) was used to determine the number of individuals represented in each mental health data set who were also represented in the corresponding criminal justice data set. Interrupted time-series analysis (
8) was used to measure change in the likelihood of involvement with the criminal justice system associated with the events of September 11.
Statistical analysis
The number of mental health service recipients who were arrested each month was determined by using probabilistic population estimation, a statistical procedure derived from a solution to the classic mathematical "coupon collector problem" presented by Feller (
9). Probabilistic population estimation determines the number of individuals represented in data sets on the basis of distributions of dates of birth rather than unique personal identifiers. The number of individuals necessary to produce the number of birthdays observed in a single birth year cohort would be calculated with use of this formula:
where Pj represents the number of persons and i represents the number of birth dates observed.
To probabilistically determine the number of persons shared across data sets that do not include a common person identifier, the sizes of three populations are determined and the results compared. The number of persons in each of the original data sets constitute the first two populations. The number of individuals in a data set formed by combining the two original data sets constitute the third data set.
The number of persons who are shared by the two data sets is the difference between the sum of the numbers of persons represented in the two original data sets and the number of persons represented in the combined data set. This outcome occurs because the sum of the number of individuals represented in the two original data sets involves a double count of every person who is represented in both data sets. The number of persons represented in the combined data set does not include this duplication. The difference between these two numbers is the size of the duplication between the two original data sets—the size of the caseload overlap. In terms of mathematical set theory, the intersection of two sets is the difference between the sum of the sizes of the two sets (A + B) and the union of the two sets (A ∪ B):
(A ∪ B) = (A + B) - (A ∪ B).
This approach has the major advantage of ensuring the personal privacy of individuals and the confidentiality of medical records (
10). This statistical procedure provides valid and reliable measures of the size and overlap of data sets that do not include personal identifiers (
7). The approach is particularly useful when concerns about the confidentiality of medical records limit the use of personally identifying information.
The relative risk of arrest among recipients of mental health services was measured for each month during the study period. Arrest rates for each month were calculated for "current" mental health clients—those who received at least one mental health service during that month or during the previous two months. The relative risk of arrest is calculated by dividing the arrest rate for service recipients by the arrest rate for the general population. A relative risk of 1 indicates that there is no difference between the two arrest rates. A relative risk of 2 means that service recipients were twice as likely as other residents of Washington, D.C., to be arrested.
Our analysis focused on the relative risk of arrest rather than the simple arrest rates among recipients of mental health services, because we were interested in the degree to which mental health service recipients were treated differently from other residents of Washington, D.C. The relationship between catastrophic events and general social upheaval or disorganization has been observed since as early as the fifth century B.C.E., when Thucydides described "lawless extravagance" in Athens during the period of the Peloponnesian War and plague (
11). The lawlessness that accompanied the 2003 collapse of the Ba'ath regime in Baghdad in 2003 is a more recent example. By examining the relative risk of involvement with the criminal justice system, we statistically controlled for any more general social upheaval that followed the events of September 11, 2001, in Washington, D.C.
Interrupted time-series analysis was used to measure change in the likelihood of involvement with the criminal justice system associated with the events of September 11 (
8). Our method of interrupted time-series analysis was to fit the 36 months of relative risk with two separate linear regressions, using SPSS, version 12.0. The first portion of the time series covered the period October 1999 through August 2001. The second portion covered the period September 2001 through September 2002. This procedure provided information about the direction and statistical significance of changes that occurred before the attacks, at the time of the attacks, and after the attacks.
Durbin-Watson statistics for each of the 12 time-series analyses provided no evidence of autocorrelated residuals. Correlations ranged from -.28 to .26 and averaged .002. This result allows for direct interpretation of levels of significance on the basis of unadjusted data. To summarize the results of this interrupted time-series analysis, we report four relative risk values (
Table 1). These values include the estimated relative risk, based on the time-series analysis, for October 1999, the annual change in relative risk in the two years before September 2001, the estimated relative risk in September 2001, and the annual change in relative risk during the 13 months after September 2001. The statistical significance of changes in relative risk reported in
Table 1 are based on a comparison of the observed values with no change—that is, 0. The significance levels of the relative risks reported in
Table 1 are based on a comparison of the observed value and no elevated risk—that is, 1. All statistical tests were t tests associated with the linear regression.