Sample and procedure
We ascertained 184 participants aged 18 to 65 in treatment facilities in the Bronx and Queens. Seventy-six had been court ordered to assisted outpatient treatment; 108 (outpatient comparison group) had been recently discharged from a psychiatric hospital and were attending the same outpatient facilities as the assisted outpatient treatment group. Participants in the assisted outpatient treatment group had been under court order for at least a month before being interviewed for our study. All participants were screened by a psychiatrist or doctoral-level psychologist, unaffiliated with the study, for capacity to provide informed consent. After a complete description of the study was given to participants, written informed consent was obtained. Institutional review board approval was obtained from the New York State Psychiatric Institute, Bronx Psychiatric Center, Creedmoor Psychiatric Center, Bronx-Lebanon Medical Center, and New York State Office of Mental Health. Interviews were conducted by doctoral- and master's-level psychologists, licensed clinical social workers, and doctoral-level social scientists. Respondents were compensated after each interview ($50 for baseline, $25 for follow-ups, and a $50 bonus for completing all interviews).
Interview data were augmented with clinical charts and information from the New York State Office of Mental Health to verify assignment to assisted outpatient treatment.
Chart diagnoses were as follows: 64 persons (35%) had schizophrenia (assisted outpatient treatment group, 28, 37%; comparison group, 36, 33%), 57 (31%) had schizoaffective disorder (assisted outpatient treatment group, 30, 39%; comparison group 27, 25%), 12 (7%) had major depressive disorder (assisted outpatient treatment group, one, 1%; comparison group, 11, 10%), 34 (18%) had bipolar disorder (assisted outpatient treatment group, 11, 14%; comparison group, 23, 21%), and 17 (9%) had other psychiatric disorders (assisted outpatient treatment group, six, 8%; comparison group, 11, 10%). The mean±SD age was 37.0±11.0 years (35.8±10.2 years for the assisted outpatient treatment group versus 37.8±11.5 years for the comparison group). Most were male (N=109, 59%) (assisted outpatient treatment group, 51, 67%; comparison group, 58, 54%). Most were from racial or ethnic minority groups: 90 (49%) were black (assisted outpatient treatment group, 41, 54%; comparison group, 49, 45%) and 70 (38%) were Hispanic (assisted outpatient treatment group, 27, 36%; comparison group, 43, 40%), 14 (8%) were white (assisted outpatient treatment group, six, 8%; comparison group, eight, 7%), and ten (5%) were from other racial or ethnic groups (assisted outpatient treatment group, two, 3%; comparison group, eight, 7%). The majority of participants had never been married (N=130, 71%) (assisted outpatient treatment group, 59, 78%; comparison group, 71, 66%) and had completed high school (N=111, 60%) (assisted outpatient treatment group, 43, 57%; comparison group, 68, 63%).
We compared the assisted outpatient treatment sample in our study (not including the comparison group) to the population of individuals assigned to assisted outpatient treatment in the Bronx and Queens, using Office of Mental Health data. The study sample overrepresented blacks compared with the state-collected data for the Bronx (Community Outcomes of Assisted Outpatient Treatment data, N=21 of 41, 51%; Office of Mental Health data, 49%) and for Queens (Community Outcomes of Assisted Outpatient Treatment data; N=20 of 35, 57%; Office of Mental Health data, 38%). Whites were underrepresented in our study sample in both boroughs. This discrepancy probably reflects the nature of the clinics from which we recruited. With the exception of race, the study sample was very similar to that of the assisted outpatient treatment population. Age, sex, marital status, employment status, income source, and educational attainment corresponded quite closely to the population of persons assigned to assisted outpatient treatment in the Bronx and Queens. All of these variables were weighted via propensity scores and controlled for in the main analysis.
Interviews were conducted between January 2003 and January 2006. Interviews were attempted at baseline and at three, six, nine, and 12 months. Baseline and 12-month interviews averaged two to three hours, whereas the other interviews averaged 30 to 60 minutes. Follow-up rates were 84% (N=155), 66% (N=121), 62% (N=114), and 72% (N=133), respectively, for three, six, nine, and 12 months. We used baseline data to compare individuals we successfully interviewed at 12 months (N=133) with those we did not (N=51). Persons lost to follow-up were younger (p<.006), more likely to be male (p<.006), and more likely to have never married (p<.02), compared with those interviewed at 12 months. Race, diagnosis, educational level, and length of unemployment in the previous year were unrelated to successful follow-up.
Independent variable: assisted outpatient treatment. Legal criteria for assignment to assisted outpatient treatment include a judgment—based on a history of treatment noncompliance that has resulted in hospitalization, incarceration, or violent acts or threats—that the individual is unlikely to voluntarily adhere to treatment and islikely to benefit from mandated treatment. Individuals appear in court to receive assisted outpatient treatment assignment and are ordered by the court to participate in outpatient treatment. Once assigned to assisted outpatient treatment, individuals who are judged by a physician to be noncompliant with treatment as ordered, who do not respond toefforts to engage them in treatment, and who are judged by a physician to be in possible need of civil commitment can be removed from the community and retained in ahospital for up to 72 hours to determine whether they need inpatient care and treatment. In addition, individuals receiving assisted outpatient treatment have priority access to certain services, including intensive case management, housing, and outpatient clinical services. Thirty-nine percent (N=42) of individuals in the comparison group did not have a case manager.
Dependent variables. We used six multiple-item scales and one binary indicator of serious violence. Scale scores represent the mean of the component items included in each scale.
Psychotic symptoms in the previous three months were assessed with the Structured Clinical Interview for DSM (
9 ). The five-item scale (
α =.62) assesses delusions of reference, persecution, control, grandiosity, and thought broadcasting.
Suicide risk was assessed with a six-item scale (
α =.83) that examines recent self-harm, depression, serious consideration of suicide, suicide plans, suicide attempts, and injuries from attempts requiring treatment (
10 ). Possible scores range from 0, low risk, to 6, high risk. The baseline measure assessed suicide risk in the previous 12 months; follow-ups assessed the previous three months.
Serious violent behavior was measured with the MacArthur Community Violence Interview (
11 ). A score of 1 was given if the participant reported having, in the previous three months, kicked, beaten, or choked anyone; hit anyone with a fist or beaten up anyone; tried to physically force anyone to have sex against his or her will; threatened anyone with a knife, gun, or other weapon; or fired a gun at someone or used a knife or a weapon on him or her. Otherwise, the score was 0.
Illness-related social functioning was measured with a seven-item scale (
α =.80) assessing recent difficulty in completing tasks, such as "make routine decisions," "control the symptoms of your illness," and "deal with day to day stresses" (
12 ). Possible scores on the items range from 0, very difficult, to 1, somewhat difficult, to 2, not difficult. The baseline measure assessed the previous 12 months; follow-ups assessed the previous three months.
Quality of life was measured with a 14-item scale (
α =.91) that was similar to Lehman's (
13 ) more comprehensive scale. Respondents rate how they currently feel (today) about domains such as living conditions, employment, social life, physical health, level of independence, and self-esteem. Possible ratings include 1, poor; 2, fair; 3, good; and 4, excellent.
Perceived coercion was measured with a five-item scale (
14,
15 ) (
α =.86) assessing whether respondents recently felt free to choose outpatient treatment—for example, "It was your idea to get mental health treatment." Possible scores include 0, strongly agree; 1, agree; 2, disagree; and 3, strongly disagree. The baseline measure assesses the previous 12 months; follow-ups assess the previous three months.
Stigma (feeling different) was assessed with a three-item scale ( α =.75) assessing feelings of being set apart or different from other people because of a psychiatric hospitalization—for example, "There is something about the experience of being in a mental hospital that sets you apart from others." Possible scores include 3, strongly agree; 2, agree; 1, disagree; and 0, strongly disagree. Time frame is not specified and is assumed to reflect current states.
Analysis
Propensity score matching. Because we cannot compare the outcomes of individuals assigned to assisted outpatient treatment to what their outcomes would have been without being assigned to such treatment, we compared two groups (one assigned to assisted outpatient treatment and a comparison group that was not) that were as similar as possible on pretreatment characteristics that predict both the probability of assignment to assisted outpatient treatment and subsequent outcomes. Inverse probability of treatment weighting using propensity scores reweights the comparison group so that, with respect to the values of the confounding covariates, it mirrors the assisted outpatient treatment group (
16 ). Assuming the propensity score model is correct, properly adjusting for the propensity score is sufficient to adjust for all the covariates used to create it (
17 ).
Each participant's propensity score is the propensity that he or she was assigned to assisted outpatient treatment, conditional on the confounding covariates measured before treatment assignment. Thus the propensity score approach we employed sought to balance groups on factors occurring before assignment and is not intended to and therefore does not adjust for factors that follow from or are a consequence of assignment. We estimated propensity scores using probit regression and used the estimated scores to implement inverse-probability-of-treatment weighted models. Individuals receiving assisted outpatient treatment were given a weight of 1; individuals not receiving assisted outpatient treatment were given a weight equal to (propensity score)/(1-propensity score). The final model was chosen on the basis of its effectiveness at yielding weighted groups whose means were most similar. A combination of propensity score weighting and regression is generally more effective at reducing bias than either method alone (
5 ). Therefore, we ran a series of weighted regressions in which the outcome variable was regressed on the confounding covariates, interview time point, and the assisted outpatient treatment indicator, with the weights described above incorporated as probability weights.
Main analysis. We employed generalized estimating equations (GEE), which do not restrict analysis to cases with complete data at every time point and which address the problem of correlated error between nonindependent observations in panel data. Because suicide risk, serious violent behavior, illness-related social functioning, and coercion were assessed for the 12 months before the baseline interview, a period that begins before assisted outpatient treatment assignment, we included these variables only for the three-, six-, nine-, and 12-month follow-ups. Because measures of psychotic symptoms, quality of life, and stigma assessed circumstances in the immediate or recent past, we used all time points, including baseline, in analysis of these variables. Consequently, the number of cases and longitudinal assessments varies across the dependent variables.
Using GEE, we conducted logistic regression for serious violent behavior and regression analyses for the other outcomes assessing the effect of assisted outpatient treatment assignment on each of the outcome variables using propensity score weighting. The following covariates were used to construct the propensity score weights. They were also included in the regression equations as an adjustment for any residual differences in balancing: gender, race or ethnicity (black, Hispanic, white, or other), age, marital status, education, psychiatric diagnosis, interview time point (baseline and three-, six-, nine-, and 12-month follow-up), natural log of number of involuntary commitments to psychiatric inpatient treatment, self-reported history of arrest, official history of arrest for violent crime and for nonviolent crime, self-reported antisocial behavior before the age of 15, likelihood of treatment noncompliance as judged by the interviewer, chart-indicated dangerousness, and history of causing bodily harm.
Table 1 shows the unweighted and weighted means for the comparison group and the means for the assisted outpatient treatment group for selected propensity score weighting variables. We report the effects of assisted outpatient treatment assignment with all covariates controlled. Additionally, because all patients in assisted outpatient treatment but not all patients in the comparison group were assigned a case manager, we conducted supplementary analyses that controlled for availability of a case manager.