All English- or Spanish-speaking adults older than 17 years who lived in New York City and had a telephone were potential participants. Sample selection was based on random-digit dialing, with oversampling of persons living in Manhattan. After we obtained verbal consent, one adult per household was randomly selected for an interview. Trained interviewers using a computer-assisted telephone interviewing system conducted the interviews. Both English and Spanish versions of the questionnaire were used. Questionnaires were translated into Spanish and then back-translated by bilingual persons whose native language was English to ensure linguistic and cultural appropriateness. Eight percent of the interviews were conducted in Spanish. A protocol was in place to provide assistance to participants who required counseling. The mean duration of the interview was 35 minutes. The surveys were conducted in January and February 2002.
The institutional review board of the New York Academy of Medicine (NYAM) approved the study protocols. Our surveys were conducted by Schulman, Ronca, and Bucuvalas, Inc. (SRBI), of New York City, a firm experienced in conducting health interviews using telephone surveys of disaster survivors, victims of sexual assault, and combat veterans. All interviewers were trained, supervised, and monitored by senior staff of both SRBI and NYAM.
Independent variables
We collected information about demographic variables that might affect use of mental health services or medications, including age, race or ethnicity, gender, household income, educational level, and marital status. In our analyses, we divided age into four groups, race or ethnicity into five groups, and education into five groups. We also asked respondents whether they had a primary care physician and health insurance coverage. Finally, we asked them about increased alcohol use after the disaster, and we used this variable in our analyses.
We asked participants to report the number of traumatic events to which they had been exposed in their lifetime (
21) and grouped the responses into four categories: no events, one event, two or three events, and four or more events. We also asked about stressful life events, such as the death of a spouse, that the participant experienced in the 12 months before the attack (
18). We divided these responses into three categories: no events, one event, and two or more events. We also assessed ten specific events or experiences associated with the World Trade Center attack, such as whether the respondent had witnessed the attack, had been injured, had lost friends or relatives in the attack, had been displaced, or had lost his or her job (
6,
15,
16). We classified respondents by how far they lived from the disaster site—three miles or more, or less than three miles—and how involved they were in the postdisaster rescue efforts—no, some, or direct involvement.
Our analyses included three mental health measures: PTSD, depression, and a peri-event panic attack—that is, an attack that occurred during the disaster or within a few hours of it. The PTSD measure was used in the National Women's Study (
22) and later modified on the basis of
DSM-IV criteria (
23,
24). This scale used a non-event-specific approach for PTSD assessment (
22,
24). It was designed for administration by trained nonclinical interviewers during a structured telephone survey. The onset of PTSD since the disaster was based on the respondent's meeting diagnostic criteria in the previous four or five months—that is, since the attack. Specifically, a participant was deemed to have PTSD if symptoms were present for the necessary criteria B, C, and D. For our analysis, the measure of onset of PTSD since the disaster did not require the symptoms to be related to the attack. An additional variable was developed for onset of PTSD that was related specifically to the attack (
6) and was used for comparison. We found these two PTSD measures to be highly correlated (kappa=.86). Cronbach's alpha for the symptoms used in this scale was reported to be .90 (
15). In addition, our PTSD scale has been reported to be highly correlated with the clinician-administered Structured Clinical Interview for DSM-III-R (SCID) (kappa=.71 for current PTSD) (
24,
25). Since its implementation, this PTSD scale has been used in several large-scale mental health surveys involving more than 16,000 completed telephone interviews (
22,
26,
27,
28).
To further validate our PTSD measure, we compared our results with those of the PTSD Check List (PCL) (
29), an established screener for PTSD, for a random subsample of 229 participants in our January- February survey. Overall, we found that the PCL had 75 percent sensitivity and 95 percent specificity in detecting PTSD as classified by our PTSD instrument (
30). In a receiver operating characteristic analysis (
31), we found that a PCL cutoff score of 50 or higher, which is the cutoff score recommended for the instrument, also optimally predicted PTSD when our instrument was used (area under the curve=.97).
Analyses performed for another postdisaster community survey conducted with 2,368 adults in New York City (
27) provided additional validity data for our PTSD measure. In the other postdisaster study, we found that a diagnosis of PTSD in the past year was significantly associated with a lower Rosenberg self-esteem score (
32) (odds ratio [OR]=7, p<.001), a clinically low mental health score on the 12-item Self-Report scale (SF-12) (
33) (OR=8.3, p<.001), and lower reported work quality among employed persons (OR=3.4, p<.001). In addition, a PTSD diagnosis in the past year was associated with meeting the clinical case definitions in the past month on the 18-item Brief Symptom Inventory (BSI-18) (
34) for depression (OR=6.8, p<.001), anxiety (OR=7.8, p<.001), and global severity (OR=7.1, p<.001). The correlation coefficients for PTSD symptoms in the past 30 days were .45, .46, and .48 with BSI-18 depression, anxiety, and global severity scales, respectively.
To measure depression we used an adapted version of the SCID's major depressive disorder subscale from the nonpatient version (
25), which has been used in other population studies (
6,
35). Respondents were considered to have major depressive disorder if they reported five or more symptoms listed for the
DSM-IV criteria for at least two weeks. Respondents were also asked to report the last time they experienced these symptoms. The research team has had experience with this scale in previous surveys focusing on the World Trade Center disaster (
6,
14,
15). Cronbach's alpha for the eight symptoms used in this scale was .79 (
15). We also compared the results for depression in the past 30 days obtained by using our depression scale with those obtained by using the BSI-18 (
34) depression scale (
27). Overall, the BSI-18 depression scale had 73 percent sensitivity and 87 percent specificity in detecting depression as classified by our depression instrument (
27). In a receiver operating characteristic analysis (
31), the BSI depression cutoff score of 65 or higher, which is a clinical cutoff for BSI depression, also optimally predicted depression when our instrument was used (area under the curve=.89).
We also found that a diagnosis of depression in the past year as measured by our scale was associated with a lower Rosenberg self-esteem score (OR=8.1, p<.001), a clinically low mental health score on the SF-12 scale (OR=12.8, p<.001), and lower reported work quality among employed persons (OR=4, p<.001). Furthermore, a diagnosis of depression in the past year on our scale was associated with meeting the clinical case definitions on the BSI-18 in the past month for depression (OR= 11.3, p<.001), anxiety (OR=9.7, p<.001), and global severity (OR=13.4, p<.001). The correlation coefficients for depression symptoms in the past 30 days were .52, .52, and .56 with BSI-18 depression, anxiety, and global severity scales, respectively.
The panic attack measure used was a modified version of the Diagnostic Interview Schedule (DIS) subscale for panic (
36), phrased to assess peri-event symptoms—those that occurred during or shortly after the terrorist attack; we have used this measure in other telephone surveys (
6,
15,
16). We asked about panic symptoms specifically in the first few hours after the events of September 11. If a participant reported at least four of the symptoms listed for panic attack in the
DSM-IV, that person was considered to have a diagnosis of a peri-event panic attack (
23).
Statistical analyses
We focused on four outcome variables, all occurring in the past 30 days: visits to mental health professionals, psychiatric medication use, increased mental health visits, and increased psychiatric medication use. We used a pre-post McNemar chi square test to assess the comparison of mental health visits in the 30 days before the attack and in the past 30 days. On the basis of previous research, we identified demographic, mental health, and stress-related variables that might have been associated with postdisaster mental health service use (
19,
37,
38). The unadjusted ORs and 95 percent confidence intervals (CIs) were calculated for these associations. We then used multivariate logistic regression to examine these associations in a predictive model. The models predicted any mental health visits, increased mental health visits, any psychiatric medication use, and increased medication use—all in the past 30 days. In our final multivariate models, only predictor variables with a p value of .05 or less in the initial bivariate analyses (based on the respective overall p values) were selected.
We also tested for interaction effects in the models between age, education, marital status, and race, because these variables have been associated with different utilization rates (
13). We used the survey estimation ("svy") commands in Stata, version 7, to generate our point estimates, p values, CIs, and logistic models (
39). This approach was needed to adjust the data for our sampling design, which included case weights to correct for overrepresentation of persons in households with more telephone lines and oversampling Manhattan residents. Therefore, our McNemar chi square tests were calculated as follows. First, the Stata survey estimates were conducted by using the survey estimations to generate pre-post percent point estimates. These point estimates were then converted into their respective survey frequencies. Next, these frequencies were entered into the "immediate" Stata command for pre versus post design ("mcci" in epitab command set) and used to generate the McNemar chi square tests.
Additional bivariate analyses were conducted to assess factors related to access to services among participants who met criteria for either PTSD or depression, because an earlier study suggested that younger adults, men, and persons without health insurance faced barriers to obtaining care (
40). In addition, we wanted to assess changes in the use of mental health services since our earlier survey (
6,
15,
16). To assess these changes, we used the data from Manhattan, which was the focus of our earlier survey. All p values presented were based on 2-tailed tests.