A Meta-Analytic Review of Psychosocial Interventions for Substance Use Disorders

We selected randomized, controlled clinical trials for inclusion in this meta-analysis by performing a comprehensive search strategy. First, we conducted a computer-based PsycINFO search of available articles published between 1840 and March of 2005 using the following key terms to conduct title searches (asterisks denote that any characters/letters can follow the last character in the terms): cocaine, crack, opi*, heroin, amphetamine*, methamphetamine*, MDMA, ecstasy, methylenedioxymethamphetamine, cannabis, marijuana, psychedelic, mushroom, glue, inhalant, poly*, substance* abuse, substance* use, addict*, and dependen* singularly and in combination with the following secondary descriptors: treatment*, trial*, outcome*, therapy, random*, intervention*, medication*, psychopharm*, pharm*, buprenorphine, behavior*, counseling, cognitive, meta analys*, contingency, and voucher*. Second, we performed a computer-based MEDLINE search of articles available between 1966 and March 2005, combined with a Cochrane Central Register of Controlled Trials search for the first quarter of 2005 with the following search terms: substance abuse and randomized, or drug abuse and randomized, or drug dependence and randomized, or cocaine and randomized, or heroin and randomized, or opioid and randomized, or opiates and randomized, or methamphetamine and randomized, or amphetamine and randomized, or marijuana and randomized, or polydrug and randomized. These terms were searched as key, title, abstract, name of substance, and MeSH subject heading terms. This search was combined with a MeSH subject heading search in the same database with the following terms: amphetamine-related disorders, cocaine-related disorders, marijuana abuse, opioid-related disorders, phencyclidine abuse, substance abuse, and intravenous. We limited both the PsycINFO and the MEDLINE searches to studies conducted with human participants and published in the English language. All titles or abstracts for the citations produced were screened, and articles were collected for any citation that appeared to meet inclusion criteria.

Studies meeting all of the following inclusion criteria were included in the meta-analysis:

Studies with medication as a backdrop condition were included only if the medication dosage did not vary between the active treatment and control conditions. Controlled trials comparing the efficacy of various treatment intensities (e.g., once per week versus twice per week individual therapy of the same therapy type) were excluded if a clear control group (e.g., one receiving inactive treatment or treatment as usual) was unavailable (seven studies). When multiple control conditions were included in these trials, we employed the “most intensive” treatment condition as the active treatment to compare to the control condition, and any other conditions of varying treatment intensity were excluded. Studies containing follow-up data beyond 1 month posttreatment but not presenting data at posttreatment were excluded (four studies). Likewise, we excluded studies employing control conditions known to be efficacious for substance abuse treatment (e.g., cognitive behavior therapy, five studies) and studies using trials of wrap-around service treatment (one study), therapeutic workplaces/work therapies (one study), hypnotherapy (one study), telecommunication networking (one study), brief motivational enhancement (two studies), supportive/expressive therapy (one study), and acupuncture (seven studies). In total, 30 studies were excluded from consideration.

When available, data on a number of descriptive variables were collected for each study, including the following: sex (data from 91.2% of the studies available), ethnicity (76.5% available), employment status (64.7% available), marital status (61.8% available), average length of substance use (55.9% available), comorbid alcohol abuse/dependence diagnoses (20.6% available), numbers of weeks treatment was administered (97.1% available), number of treatment sessions per week (58.8%), number of participants entered per condition (100% available), treatment retention rates (47.1% available), and numerous outcome variables, as described below.

Treatments were categorized into the following treatment condition “types”: contingency management/vouchers (14 studies), general cognitive behavior therapy interventions (13 studies), relapse prevention (five studies), and cognitive behavior therapy plus contingency management combined (two studies). Descriptions of treatment conditions were used to categorize each condition into its best-fit treatment type.

Given the lack of “gold standard” outcome measures in the substance abuse treatment literature, we reviewed all controlled trials meeting our inclusion criteria, as well as all available substance abuse treatment reviews and meta-analyses, to develop a set of variables that would sufficiently allow for outcome comparison across studies. These variables included a combination of self-report data (data from 58.8% of the studies available) and measures employing toxicology screening procedures (76.5% available). Self-report outcome variables were the following:

Toxicology screen variables were 1) mean number of negative screens throughout treatment, 2) mean percent of negative screens throughout treatment, and 3) percent of sample demonstrating clinically significant abstinence.

With respect to the percent of the sample demonstrating posttreatment and/or clinically significant abstinence, for both self-report and toxicology screen data, studies varied widely in defining this variable. These definitions of posttreatment and/or clinically significant abstinence varied from 4 to 24 weeks of abstinence (or abstinence for the entire length of treatment), as measured by means of the participants’ self-report or toxicology screen results. The 3-week abstinence cutoff appeared to be the most widely employed measure of clinically significant change, as these data were presented by 11 (30%) of the studies included in this meta-analysis. The Addiction Severity Index (used by 16 studies) (25) and the Time Line Follow-Back interview method (used by four studies) (26) were the most widely used interviews for collecting self-report data across studies.

These eight outcome variables were used for the purposes of calculating treatment versus control condition effect size estimates. Effect size was calculated using Cohen’s d (27) . Owing to inconsistency in the outcome variables employed by each study, when studies presented data on two or more outcome variables, we aggregated (averaged) across these variables to yield an aggregate mean effect size for each study. In addition, to determine the differential effect of self-report versus toxicology screen outcome data, we aggregated outcome variables within each study to yield 1) an aggregate mean overall effect size, 2) an aggregate mean self-report effect size, and 3) an aggregate mean toxicology screen effect size. Finally, as a general measure of abstinence rates, we provide the percent of the sample demonstrating posttreatment and/or clinically significant abstinence variable (these data were reported by 16 of the 37 studies), or, if this variable was not reported for a particular study, we calculated this data from the information provided (data calculated for six studies).

Effect sizes were calculated for a total of 34 studies. Across all studies, 2,340 entered the treatment and identified control conditions. The mean age of the participants across all studies was 34.9 years (range of means=20.6 to 43.0, SD=4.5). On average, samples were 62.2% male and 61.0% Caucasian. The majority of the participants were single/unmarried (67.7%), and less than half were employed part-time or full-time (42.5%). The participants reported an average of 10.1 years of substance use (SD=5.1). Of the seven studies reporting specific diagnostic criteria, approximately 50.7% of the participants met criteria for comorbid alcohol abuse or dependence, although alcohol was not the targeted substance in the treatment study.

Table 1 presents a detailed overview of these treatments, including 14 identified as contingency management, two as cognitive behavior therapy/contingency management combination conditions, 13 as general cognitive behavior therapy interventions, and five as relapse prevention. Of these, 13 of the treatments were for polysubstance use, nine for cocaine use, seven for opiate use, and five for cannabis use disorders. Treatment types were not significantly associated with (confounded with) the targeted drug use disorders according to chi-square analyses (contingency management versus all other treatments).

Overall, 43.6% of the studies included samples in which the participants received medication maintenance (e.g., methadone maintenance) in conjunction with both the experimental treatment and control conditions. The mean length of treatment across all conditions was 21 weeks (range=4–52 weeks, SD=14), and the average number of sessions per week averaged 1.8 sessions (range=1–3, SD=0.8). The mean intent-to-treat sample size per treatment condition was 38.23 (SD=32.00), ranging from five to 135 participants across all conditions.

Approximately one-third of the participants across all conditions dropped out before treatment completion (35.4%). Mean dropout among control conditions was 44.6%. Across all substance use groups, cocaine and opiate patients tended to have higher mean dropout rates (42.0% and 37.0%, respectively) than patients treated for cannabis and polysubstance use (27.8% and 31.3%, respectively). Contingency management demonstrated the lowest dropout rates (29.4%), followed by general cognitive behavior therapy (35.3%) and cognitive behavior therapy plus contingency management (44.5%). Only two studies provided relapse prevention dropout rates (57.0%), and these studies were specific to cocaine treatment.

Table 1 presents aggregate effect sizes across all studies. The aggregate effect size across all conditions and all substances was in the moderate range (d=0.45), with a 95% confidence interval (CI) of 0.27 to 0.63. Although there was an apparent difference in effect sizes depending on the outcome measure used, with self-report yielding a high-moderate effect size (d=0.61, 95% CI=0.35 to 1.20) and toxicology screen outcomes yielding a low-moderate effect size (d=0.33, 95% CI=0.17 to 0.49), this difference did not reach significance according to a within-sample t test, which included only the 12 studies that provided both measures (t=1.16, df=11, p<0.28). Urine analysis detection time differs across drugs of abuse. Whereas cocaine and opiates have an approximate window of detection of 1–3 days, the window of detection time for marijuana (cannabis) can extend to weeks to months for individuals who are chronic heavy users. In the current meta-analysis, only one study of marijuana used urine analysis as an outcome variable ( Table 1 ).

Figure 1 represents overall effect sizes for psychosocial treatments (collapsed across treatment type) in terms of the substance use being targeted. Independent-sample t tests revealed that psychosocial treatments had their lowest efficacy for polysubstance use, with a significant difference between outcomes for polysubstance use (d=0.24, 95% CI=0.03 to 0.44) and cannabis use (d=0.81, 95% CI=0.25 to 1.36) disorders (t=2.42, df=17, p<0.03). Treatments targeting cocaine use yielded medium to large effect sizes (d=0.62, 95% CI=0.16 to 1.08), and treatments targeting opiate use yielded small to medium effect sizes (d=0.39, 95% CI=0.18 to 0.60). There were no other significant differences between the substance use disorders treated.

Figure 2 presents effect sizes for treatment outcome in terms of treatment type across all substances. The results indicate that treatments incorporating both cognitive behavior therapy and contingency management had the highest effect sizes (d=1.02); however, this result must be interpreted cautiously as there were few studies in this category (N=2; 95% CI=–0.05 to 2.09). Treatments using contingency management alone produced moderate-high effect sizes (d=0.58, 95% CI=0.25 to 0.90). Cognitive behavior therapy alone and relapse prevention evidenced low moderate effect sizes: d=0.28 (95% CI=0.06 to 0.51) and d=0.32 (95% CI=0.06 to 0.56), respectively.

Across all active treatment conditions, almost one-third of the participants (31%) achieved posttreatment and/or clinically significant abstinence. Alternately, only 13% of all participants in control conditions achieved abstinence. Across drug use groups, rates were similar, with 36.2% of opiate users, 31.7% of cocaine users, and 26.0% of cannabis users achieving abstinence during the study period.

Although the combination of cognitive behavior therapy and contingency management evidenced the largest effect size, this advantage was not evident for the percent of abstinence posttreatment (26.5%). Treatment involving relapse prevention evidenced the largest posttreatment abstinence rates, with 39.0% abstinent. Posttreatment percent abstinent for general cognitive behavior therapy alone was 27.1%, and for contingency management alone, it was 31.0%.

In an attempt to better understand our outcome findings, we examined variables that may potentially moderate the association between aggregate effect size estimates and treatment dropout rates. In terms of sample demographics, we used Pearson’s partial correlations to assess the relationship between age, sex (percent male in each study), ethnicity (percent white), marital status (percent single/unmarried), employment status (percent employed), and average years of substance use in relation to outcome variables (e.g., treatment retention, overall effect size).

A significant negative correlation was found between age and effect size (r=–0.37, p<0.05), suggesting that younger samples were more likely to have larger effect sizes. A significant negative correlation was also found between average years of substance use and treatment dropout rate (r=–0.68, p<0.05), indicating that participants with longer histories of substance use were less likely to drop out of treatment than those with shorter histories of use. No other demographic variables were significantly associated with treatment outcome.

In addition, we also examined a number of treatment variables in relation to key outcome variables. Treatment variables included the number of weeks treatment was administered, the number of treatment sessions per week, and whether or not medication maintenance was employed. The results indicated a significant negative correlation between the number of treatment weeks and effect size (r=–0.34, p<0.05). Number of treatment sessions per week, however, was not significantly related to treatment outcome or treatment retention. Receipt of medication maintenance was negatively associated with dropout rates (r=–0.51, p<0.05); patients in studies in which medication maintenance provided a backdrop to the psychosocial conditions under study were less likely to drop out of treatment than patients in treatment that did not use medication.

An additional issue that may influence the size of treatment effects is the background drug treatment condition. Agonist therapies, stimulating the relevant drug receptor, attenuate withdrawal and drug craving and may provide either a better or worse backdrop for other treatments. In a controlled effect size analysis, this drug condition affects changes in both the experimental and control treatment conditions, and hence, for there to be differential effects evident for comparisons between the psychosocial treatment conditions, the drug must lead to interaction effects. For example, if cravings are attenuated, a subsequent psychosocial treatment may have a differential “foothold” for changing drug use behaviors. Alternatively, the drug benefits applied to both the experimental treatment group and the control condition may make it more difficult to show additive effects by providing all patients in the trial with initial changes in drug use that may approach ceiling effects for short-term treatment.

To assess the potential influence of agonist therapies, we compared studies using this approach (N=13, including both methadone and buprenorphine treatment) to those that offered no drug treatment (N=18) or antagonist therapy (N=3, naltrexone treatment). Agonist treatment was used in seven of 13 polysubstance use studies, three of nine cocaine studies, and three of seven opiate studies. No agonist treatment was offered in studies of cannabis treatment; hence, these studies (N=5) were not considered in the following analysis. Overall, we found no significant difference or tendency in effect sizes for the combined drug groups, excluding marijuana (agonist overall: mean d=0.38, not agonists: d=0.40; t=0.1, df=27, p<0.90). Variability in effects was high, with agonist therapies showing both the highest and lowest effect sizes in the entire sample of studies included in this meta-analysis. No differences between agonist use and the remainder of the sample were significant when they were examined separately in the polysubstance, cocaine, and opiate use groups (all p values >0.14).

Investigators have recognized the potential discrepancy between the number of trials completed and the number of trials published. If studies are not published because the findings are not significant, a meta-analysis of published studies may overestimate effect sizes. This problem has been labeled “the file drawer problem” (62) .

Using a conservative method of addressing this problem, one must assume that the effect sizes of all current or future unpublished studies are equal to 0 and compute the number of studies it would require to reduce the overall effect size to a minimally informative level, in this case, a small effect size (d=0.2). With the guidelines of Orwin (63), more than 42 “file drawer studies” with null results would be required to reduce the overall effect size to a small level, according to Cohen’s standards (27), a result indicating that the findings to date (based on 34 studies) are fairly robust. Moreover, we found no significant relationship between sample size and effect size (r=0.02, p>0.90), and there was no significant association between publication year and effect size (r=–0.09, p>0.50).