An Effect-Size Analysis of the Relative Efficacy and Tolerability of Serotonin Selective Reuptake Inhibitors for Panic Disorder
Publication: American Journal of Psychiatry
Abstract
OBJECTIVE: Serotonin selective reuptake inhibitors (SSRIs) are now considered the first-line pharmacotherapy for panic disorder. The preferential use and the presumption of greater tolerability of SSRIs relative to older agents, such as tricyclic antidepressants, occurred without direct comparisons between the two classes of medication. In this study the authors used an effect-size analysis to provide an initial comparison. METHOD: The authors conducted an effect-size analysis of 12 placebo-controlled, efficacy trials of SSRIs for panic disorder and compared these results to findings obtained in a recent meta-analysis of non-SSRI treatments for panic disorder. RESULTS: The mean effect size for acute treatment outcome for SSRIs relative to placebo was 0.55, not significantly different from that for antidepressants in general (0.55) and for imipramine in particular (0.48). More recent studies of SSRIs, and studies using larger samples, were associated with lower effect sizes. No significant differences were found in dropout rates between those taking SSRIs and those taking older agents during acute treatment. CONCLUSIONS: An effect-size analysis of controlled studies of treatments for panic disorder revealed no significant differences between SSRIs and older antidepressants in terms of efficacy or tolerability in short-term trials. An inverse relationship was evident between sample size and effect size for SSRIs. Early studies of small samples may have led to initial overestimations of the efficacy of SSRIs for panic disorder.
Serotonin selective reuptake inhibitors (SSRIs) emerged as first-line agents for the treatment of panic disorder not long after initial treatment trials were published (1–3). Indeed, the rapid acceptance of SSRIs for the treatment of panic disorder occurred without comparative treatment trials with older agents.
To address this issue and provide continuity between past and current efficacy studies of treatments for panic disorder, we conducted an effect-size analysis of recent, controlled trials of SSRIs in treating panic disorder. Specifically, we examined the effect size of SSRI treatments for panic disorder from 12 placebo-controlled efficacy trials and compared these results to findings from Gould and associates’ meta-analysis (4) of non-SSRI treatments for panic disorder.
Method
Double-blind, placebo-controlled studies of SSRIs for panic disorder with or without agoraphobia were identified in literature searches with the use of PsychLIT and MEDLINE, discussions with colleagues, and examination of reference sections of related articles. The following key terms were used alone and in combination in our literature search: panic, medication, treatment outcome, selective serotonin reuptake inhibitor, SSRI, paroxetine, fluoxetine, fluvoxamine, sertraline, and citalopram. We excluded from analysis results from uncontrolled medication trials, studies examining the effects of SSRIs on biological challenge procedures (e.g., CO2 challenges), case reports, long-term or follow-up studies, and articles reviewing several studies that were published separately. Twelve studies met the criteria for inclusion in the present effect-size analysis.
By using methods comparable to those of Gould et al. (4), study effect sizes were calculated for all dependent variables at posttreatment, excluding measures of depression symptoms, e.g., the Hamilton Rating Scale for Depression score (5). A separate panic frequency effect size (for full panic attacks only) was also calculated. Table 1 presents these effect-size measures as well as the number of subjects, dropout rates, and the percentage of subjects who were free of panic at endpoint. To provide the fairest estimate of the efficacy of SSRIs in fixed-dose studies, effect sizes were calculated only for doses identified as effective. For example, Ballenger et al. (13) concluded that daily doses of 10 and 20 mg of paroxetine were not significantly better than placebo, and effect sizes were calculated only for the 40-mg dose. In the case of fluoxetine (15), both 10- and 20-mg daily doses were found to be effective, with slightly higher effect sizes for the 10-mg dose; data from both doses were included in the meta-analysis.
An effect size for a particular measure was computed by subtracting the mean score of the posttreatment comparison group from that of the posttreatment active-treatment group and then dividing by the standard deviation of the posttreatment comparison group (or, alternatively, following a similar computation for change scores if these scores were used for data analysis):
Furthermore, if the results for a particular measure were reported as proportions of individuals who improved or were free of panic (rather than as means, standard deviations, and t scores), then effect sizes were determined by using a table provided by Glass et al. (18, p. 139). Effect sizes were assigned by interpolation from listed values.
Both weighted and unweighted (by sample size) overall effect sizes were calculated. Weighted effect sizes have the advantage of providing stronger representations of studies examining large numbers of subjects and helping overcome the potential bias introduced by the tendency for underpowered studies with smaller effect sizes to be excluded from publication. The extent of this potential bias was examined with a funnel plot that plotted the sample size against the effect size.
Results
Outcome Effect Sizes
We obtained a mean study effect size of d=0.55 and a panic frequency effect size (based on eight studies) of 0.38. Overall effect sizes were significantly associated with the sample size of the study; larger studies were associated with lower effect sizes (R=0.72, F=10.9, df=1, 10, p<0.009). Overall, the mean sample size of the SSRI studies (N=145) tended to be smaller than that of earlier imipramine studies (N=194). When SSRI effect sizes were weighted by sample size, a lower overall effect size and a similar panic frequency effect size estimate were obtained: 0.47 and 0.37, respectively.
These effect-size estimates were compared, by using unpaired t tests, to both the antidepressant (d=0.55) and imipramine effect sizes (nine studies, d=0.48) obtained by Gould et al. (4). Imipramine was selected as a specific comparison drug because of the use of this agent in a recent multicenter collaborative treatment study (19). No significant differences between SSRI effect sizes and these comparison groups were found. The mean study effect size was identical to the mean non-SSRI effect size for antidepressants reported by Gould et al. (4), and, likewise, the weighted effect size was nearly identical to the effect size for imipramine alone.
Study Dropouts
The mean study dropout rate for SSRIs was 19.9% (SD=10.9), an estimate almost identical to that in pharmacotherapy studies (19.8%) reported by Gould et al. (4). In the analysis by Gould et al., antidepressants had a mean dropout rate of 25.4%, and imipramine had a mean dropout rate of 22.4%, based on data from seven studies. No significant differences between SSRIs and these comparison conditions were found by t test analyses. When data were pooled across all studies (thereby giving larger studies a larger contribution to the computation of the overall dropout rate), we found the mean dropout rate for SSRIs was 24.6%.
Associations Between Sample Size and Effect Size
The significant negative association between study sample size and effect size provides evidence for what has been termed a “publication bias” against smaller, lower-powered studies obtaining lower effect sizes for SSRIs. To further examine this hypothesis, a qualitative, funnel plot analysis was conducted (20, 21). When no sample-related bias is evident, a funnel plot (with the effect size on the abscissa and the sample size or standard error on the ordinate) should produce a symmetrical pyramidal shape with the apex at the top. Under conditions of publication bias, typically, the lower left base of the pyramid (reflecting the lower sample size and lower effect-size studies) is missing. This was the case with the current data set, in which the funnel plot was not only asymmetrical but indicative of a strong negative association between sample size and effect size.
In our data, sample size was also confounded with the year of study publication (R=0.76, F=13.4, df=1, 10, p<0.004); more recent studies used larger sample sizes. Accordingly, we examined the independent association between year of publication and effect size. More recently published studies were associated with smaller SSRI effect sizes (R=0.82, F=20.1, df=1, 10, p<0.002), even when the independent influence of sample size was statistically controlled. Furthermore, this effect was not explained by differences in the medication under study (e.g., the influence of year and sample size continued to be significant when the earlier and smaller fluvoxamine studies were eliminated from analysis). As a result of this association, we investigated whether effect sizes may have been influenced by changes in sample responsivity over time. To examine this hypothesis, we used the panic-free rate for patients in placebo treatment as an index of sample responsivity to nonspecific treatment effects. Using a linear regression analysis of the nine studies providing these data, we found no significant association between year of publication and treatment responsivity (R=0.41, F=1.40, df=1, 8, p<0.28). Hence, changes in treatment responsivity, at least as assessed by panic-free rates, do not appear to be the explanation for findings of lower SSRI effect sizes over time in the panic disorder literature. Likewise, we found no obvious associations between the type of outcome variable used or the degree of avoidance between early and later studies of SSRIs for panic disorder.
Discussion
We obtained an overall SSRI effect-size estimate identical to the overall non-SSRI effect size for pharmacotherapy reported by Gould et al. (4), suggesting that results for recent SSRI studies are well within expectations of the kinds of results achieved with older agents. The specific efficacy of SSRIs to reduce panic frequency revealed a smaller effect size, but this estimate was limited to the eight studies that included this outcome variable.
We found a large negative association between the size of the study and the effect-size estimate; larger studies were associated with smaller effect sizes. Consequently, we recalculated effect size by taking into account the greater confidence in an estimate afforded by a larger sample. This smaller, weighted SSRI effect size was nearly identical to the effect size for imipramine treatment alone in the Gould et al. (4) meta-analysis. That is, the present effect-size analysis provides no evidence supporting the hypothesis that SSRIs are more effective than older antidepressants in the treatment of panic disorder. This result agrees in full with analyses of the relative effect sizes of SSRIs and tricyclic antidepressants for major depression (22).
In our limited sample of studies, we did not find that SSRIs were significantly more tolerable than treatment with other antidepressants when dropout rates were examined. However, we were able to analyze only attrition rates as a whole; attrition associated with side effects was not reported consistently enough to allow analysis, as was done for the observed difference in side effect severity between the SSRIs and older antidepressants in depressed samples (23, 24).
We found that, over time, as the investigation of SSRIs intensified and included more agents and larger studies, smaller effect sizes were obtained relative to those of earlier reports. We could not account for this finding by examining the responsivity of the treatment samples, the specific medication under study, differences in sample characteristics, or outcome measures, although we cannot rule out the existence of other cohort effects. Our results are consistent with the notion that the early publication of small but significant studies may have led to an initial overestimation of the effect size of SSRIs for panic disorder; cf., Boyer (25). Moreover, it is clear that our analysis may continue to overestimate the true effect size of SSRIs, given evidence that negative, industry-sponsored trials were not published. For example, Ninan (26) described four negative trials with high rates of placebo response that were not published; in these studies, which found a mean effect size of d=0.16 for categorical outcomes (panic-free rate and Clinical Global Impression improvement score of 2 or more), fluovoxamine did not outperform placebo.
In conclusion, we found no evidence supporting the hypothesis that SSRIs are more effective than older antidepressants for the treatment of panic disorder. One implication of our findings is that earlier estimates of the relative efficacy of cognitive behavior therapy and older antidepressants (4, 10, 27) may well be applicable to SSRIs, as suggested by initial comparisons of open treatment using SSRIs or cognitive behavior therapy (28). Our results are necessarily limited by the biases and challenges encountered with meta-analytic comparisons (29). Direct-comparison (three-arm) outcome studies of SSRIs, older antidepressants, and placebo have the potential of further clarifying the efficacy relationship among these agents.
Footnote
Presented in part at the 33rd convention of the Association for Advancement of Behavior Therapy, Toronto, Nov. 11–14, 1999. Received March 27, 2000; revision received April 20, 2001; accepted May 10, 2001. From Massachusetts General Hospital, Harvard Medical School. Address reprint requests to Dr. Otto, Massachusetts General Hospital, WACC-812, 15 Parkman St., Boston, MA 02114; [email protected] (e-mail).
References
1.
American Psychiatric Association: Practice Guideline for the Treatment of Patients With Panic Disorder. Am J Psychiatry 1998; 155(May suppl)
2.
Coplan JD, Pine DS, Papp LA, Gorman JM: An algorithm-oriented treatment approach for panic disorder. Psychiatr Annals 1996; 26:192-201
3.
Pollack MH, Smoller JW: Pharmacologic approaches to treatment-resistant panic disorder, in Challenges in Clinical Practice: Pharmacologic and Psychosocial Strategies. Edited by Pollack MH, Otto MW, Rosenbaum JF. New York, Guilford Press, 1996, pp 89-112
4.
Gould RA, Otto MW, Pollack MH: A meta-analysis of treatment outcome for panic disorder. Clin Psychol Rev 1995; 15:819-844
5.
Hamilton M: A rating scale for depression. J Neurol Neurosurg Psychiatry 1960; 23:56-62
6.
Den Boer JA, Westenberg HGM: Serotonin function in panic disorder: a double blind placebo controlled study with fluvoxamine and ritanserin. Psychopharmacology (Berl) 1990; 102:85-94
7.
Black DW, Wesner R, Bowers W, Gabel J: A comparison of fluvoxamine, cognitive therapy, and placebo in the treatment of panic disorder. Arch Gen Psychiatry 1993; 50:44-50
8.
Hoenhn-Saric R, McLeod DR, Hipsler PA: Effect of fluvoxamine on panic disorder. J Clin Psychopharmacol 1993; 13:321-326
9.
Oehrberg S, Christiansen PE, Behnke K, Borup AL, Severin B, Soeggard J, Calberg H, Judge R, Ohrstrom JK, Manniche PM: Paroxetine in the treatment of panic disorder: a randomised, double-blind, placebo-controlled study. Br J Psychiatry 1995; 167:374-379
10.
Sharp DM, Power KG, Simpson RJ, Swanson V, Moodie E, Anstee JA, Ashford JJ: Fluvoxamine, placebo, and cognitive behaviour therapy used alone and in combination in the treatment of panic disorder and agoraphobia. J Anxiety Disord 1996; 10:219-242
11.
Lecrubier Y, Bakker A, Dunbar G, Judge R (Collaborative Paroxetine Panic Study Investigators): A comparison of paroxetine, clomipramine and placebo in the treatment of panic disorder. Acta Psychiatr Scand 1997; 95:145-152
12.
Wade AG, Lepola U, Koponne HJ, Pedersen V, Pedersen T: The effect of citalopram in panic disorder. Br J Psychiatry 1997; 170:549-553
13.
Ballenger JC, Wheadon DE, Steiner M, Bushnell W, Gergel IP: Double-blind, fixed-dose, placebo-controlled study of paroxetine in the treatment of panic disorder. Am J Psychiatry 1998; 155:36-42
14.
Londborg PD, Wolkow R, Smith WT, DuBoff E, England D, Ferguson J, Rosenthal M, Weise C: Sertraline in the treatment of panic disorder: a multi-site, double-blind, placebo-controlled, fixed-dose investigation. Br J Psychiatry 1998; 173:54-60
15.
Michelson D, Lydiard RB, Pollack MH, Tamura RN, Hoog SL, Tepner R, Demitrack MA, Tollefson GD (Fluoxetine Panic Disorder Study Group): Outcome assessment and clinical improvement in panic disorder: evidence from a randomized controlled trial of fluoxetine and placebo. Am J Psychiatry 1998; 155:1570-1577
16.
Pohl RB, Wolkow RM, Clary CM: Sertraline in the treatment of panic disorder: a double-blind multicenter trial. Am J Psychiatry 1998; 155:1189-1195
17.
Pollack MH, Otto MW, Worthington JJ, Manfro GG, Wolkow R: Sertraline in the treatment of panic disorder: a flexible-dose multicenter trial. Arch Gen Psychiatry 1998; 55:1010-1016
18.
Glass GV, McGaw B, Smith ML: Meta-Analysis in Social Research. Beverly Hills, Calif, Sage, 1981
19.
Barlow DH, Gorman JM, Shear MK, Woods SW: Cognitive-behavioral therapy, imipramine, or their combination for panic disorder: a randomized controlled trial. JAMA 2000; 283:2529-2536
20.
Begg CB, Berlin JA: Publication bias: a problem in interpreting medical data. J R Stat Soc 1988; 6:11-30
21.
Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR: Publication bias in clinical research. Lancet 1991; 337:867-872
22.
Song F, Freemantle N, Sheldon TA, House A, Watson P, Long A, Mason J: Selective serotonin reuptake inhibitors: meta-analysis of efficacy and acceptability. Br Med J 1993; 306:683-687
23.
Anderson IM, Tomenson BM: Treatment discontinuation with selective serotonin reuptake inhibitors compared with tricyclic antidepressants: a meta-analysis. Br Med J 1995; 310:1433-1438
24.
Montgomery SA, Henry J, McDonald G, Dinan T, Lader M, Hindmarch I, Clare A, Nutt D: Selective serotonin reuptake inhibitors: meta-analysis of discontinuation rates. Int Clin Psychopharmacol 1994; 9:47-53; correction, 9:296
25.
Boyer W: Serotonin uptake inhibitors are superior to imipramine and alprazolam in alleviating panic attacks: a meta-analysis. Int Clin Psychopharmacol 1995; 10:45-49
26.
Ninan PT: Issues in the assessment of treatment response in panic disorder with special reference to fluvoxamine. J Clin Psychiatry 1997; 58:24-31
27.
Clark DM, Salkovskis PM, Hackmann A, Middleton H, Pavlos A, Gelder M: A comparison of cognitive therapy, applied relaxation and imipramine in the treatment of panic disorder. Br J Psychiatry 1994; 164:759-769
28.
Otto MW, Pollack MH, Maki KM: Empirically supported treatment for panic disorder: costs, benefits, and stepped care. J Consult Clin Psychol 2000; 68:556-563
29.
Wilson A, Henry DA: Meta analysis: part 2: assessing the quality of published meta-analyses. Med J Aust 1992; 156:173-187
Information & Authors
Information
Published In
History
Published online: 1 December 2001
Published in print: December 2001
Authors
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBGet Access
Login options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).