Data
Admissions. Data on emergency admissions were obtained from the management information system of the San Francisco Department of Mental Health. We defined an emergency admission as the decision to provide treatment for a person aged 18 years or older at one of the department's three psychiatric emergency services during the 180 weeks beginning January 1, 1994, and ending July 6, 1997. (The system for recording unemployment claims fundamentally changed in July 1997.) Admissions did not include follow-up visits or visits to have prescriptions rewritten. Any episode during which a patient admitted to an emergency service was also admitted for inpatient psychiatric care was counted as one admission. Informed consent was not required, because the data included no client-level information.
Approximately 29,010 admissions occurred during the study period. About 64 percent of the patients were men. Approximately 54 percent of admissions were of non-Hispanic whites, 22 percent were of African Americans, 10 percent were of Asians or Pacific Islanders, and 9 percent were of Hispanics. About 53 percent were coerced, meaning that treatment was provided under California's civil commitment authority to treat persons believed to be mentally ill and a threat to themselves or others or gravely disabled (
11).
Reduction of outpatient services. The San Francisco Department of Public Health identified two documented reductions in outpatient services during the study period. First, a clinic was closed for one week for repairs. We anticipated a lag in the effects of this closure on the grounds that several days would probably pass before symptoms worsened to the level that emergency care was sought. Second, budgetary constraints led to the permanent closure of a clinic that had functioned seven days a week from 9 a.m. to 9 p.m. The effect of this closure might also have been lagged. However, the lag may have been longer for those who needed services spaced at relatively long intervals. For example, patients who needed prescriptions rewritten might have suffered an acute episode up to a month or five weeks after the closure.
Efforts to detect acute mental illness. Two initiatives to identify persons in need of emergency psychiatric services were under way during the study. A mobile crisis clinic opened during the study period and functioned throughout the remainder of the study. Police or fire personnel summoned the mobile crisis clinic when they encountered a person who appeared to be mentally ill. We characterized this effort as "reactive." A second, more aggressive program started before and ended during the study. Teams of police, mental health workers, and social workers sought out homeless persons to offer them services and, if necessary, coerce those with mental illness into treatment. We characterized this program as "proactive."
Enhancements of emergency treatment. Emergency service providers began a dual diagnosis unit to reduce relapse among persons with mental illness and a co-occurring substance use disorder. The unit provided brief but intensive case management for clients on their release from the service. We hypothesized that these enhancements would reduce readmissions. Thus we expected that there could be a lag of a week or two in their effect.
We specified several control variables to avoid spurious associations between the system of care and emergency admissions. For example, data from the Epidemiologic Catchment Area survey indicate that job loss increases the risk of antisocial behavior and alcohol abuse (
12,
13). The data further suggest that the fear of layoff inhibits these behaviors among persons who remain employed when unemployment unexpectedly increases (
12). Consistent with these findings, studies of job loss and coerced treatment due to dangerousness to others in San Francisco and Pittsburgh reported that the coerced treatment increased with unemployment until joblessness exceeded levels that would be expected from seasonal patterns. At that point, the association reversed to form an overall inverted U (
11,
14).
To avoid spurious associations between job loss and psychiatric emergencies, we specified two variables used in the published research: the number of initial unemployment compensation claims, and the square of this number (
11,
14). We hypothesized that the coefficient for claims would be positive, because it defines the upward half of the inverted U relationship. We expected the coefficient for the square of claims to be negative, because it defines the downward half.
We also controlled for extreme weather, which impedes access to prophylactic services and social support (
15). San Francisco has mild weather, but heavy rain disrupts the normal functioning of the city. We created an "extreme precipitation" variable by first separating into quartiles the 31 weeks during which there was rain. We then scored all weeks with values less than the fourth quartile (3.1 inches of rain) as 0.
We controlled for holidays, which may increase social contact that could reduce the likelihood of acute illness. Moreover, service organizations often provide shelter and food for homeless persons with mental illness during holiday periods, which may also reduce the incidence of acute illness. We created a binary variable scored 1 for the weeks that included New Year's Day, Easter, Independence Day, Labor Day, Veterans' Day, Thanksgiving, and Christmas to test the holiday hypothesis.
We also controlled for phenomena that could induce cycles and trends in emergency admissions. For example, we know that persons who abuse drugs and alcohol frequently do so after receiving income (
16). This phenomenon, referred to as the check effect, reportedly increases the incidence of psychiatric emergencies in the first week of each month (
17,
18,
19).
Prescriptions often expire after 30 days. Some patients, particularly men, fail to comply with emergency therapy and experience relapse when their prescriptions expire (
20). Thus persons originally admitted because of the check effect and who relapse when prescriptions expire may reinforce the four-week check effect cycle.
Data analysis
The hypothesis that changes in the system of care will affect psychiatric emergencies implies that postchange admissions will differ in the hypothesized direction from those expected under the null hypothesis of no effect. A clinical trial derives values expected under the null hypothesis from a randomly assigned control group. Quasi-experiments such as ours must derive those values through statistical assumptions. The typical assumption is that postchange values will not be outside the 95 percent confidence interval (CI) of the mean of prechange values.
However, observations collected over time often exhibit trends, cycles, and the tendency to remain elevated or depressed after high or low values. These patterns complicate quasi-experiments, because the expected value of a patterned series may not be its mean (
21).
Health services researchers have methods for estimating the expected value and confidence interval for patterned time series (
22). These methods "decompose" a series into its patterned and residual—or unexpected—components. The test then hinges on whether the residual components are consistent with the hypothesis. The residuals would support the hypothesis that an event will increase a phenomenon if they exceed their 95 percent CI at the times specified a priori as those in which the event should have effect.
We used software available from Scientific Computing Associates (
23) to decompose the admissions series into patterned and residual components (
24). We then adjusted the residuals by removing any variance shared with the control variables. We inspected the adjusted residuals to determine whether they were above or below, depending on the hypothesis, their 95 percent CI when the changes in care should have had their effects (
25,
26). If the residuals were outside the CI, we looked for three types of changes in them: permanent increases or decreases, brief increases or decreases that disappeared after one or two weeks, and persistent increases or decreases that decayed slowly to insignificance. A more technically detailed description of the tests is available from the first author.