Association Between Insight and Outcome of Psychotherapy: Systematic Review and Meta-Analysis

Publications were identified by a search of PubMed, PsycINFO, PsycARTICLES, and PSYNDEX using the search terms (insight OR insightfulness OR self-understanding) AND (psychotherapy OR counseling). Results were restricted to publications in English or German. The last search was conducted on April 6, 2018. In addition, reference lists of relevant publications were examined by hand search, and experts were consulted to identify missing studies.

A flow diagram of the study selection process is presented in Figure 1. Electronic database and hand search yielded a total of 13,849 records. After duplicate removal, the titles and abstracts of 12,780 publications were screened. Next, full texts were inspected. Unpublished dissertations were requested from authors. A total of 261 full-text publications were assessed for eligibility. Publications with unclear inclusion status were discussed between two authors (S.J., U.D.), and ambiguity was resolved by consensus. If two publications were based on the same data, we chose the study with the insight measure that fit our definition of insight best. When insight measures were identical, we selected the study that reported the most comprehensive statistics. We selected Andreoli et al. (29) over Andreoli et al. (30) because the latter only reported associations between insight and outcome for subsamples. Hoffart et al. (31) was chosen over Hoffart and Sexton (32) because only the former reported a zero-order correlation between insight and outcome. We selected Kolden (33) over Kolden (34) because the former reported the purer association between insight and outcome. We chose Luborsky et al. (35) over Grenyer and Luborsky (36) because the insight operationalization in the former fit our definition more closely. In cases of ambiguity about the primary data source, authors were contacted for clarification. This procedure resulted in 22 unique studies (21 peer-reviewed journal articles and one dissertation) representing a k of 23 effect sizes from independent samples that were included in the meta-analysis.

We developed a data extraction template according to the preregistered study protocol, pretested it on five studies, and subsequently refined it. One author (S.J.) extracted the following data: study characteristics, sample characteristics, information on predictor and outcome measures, and effect sizes for the association between insight and outcome. If zero-order correlations could not be extracted or computed from the reported data, authors were contacted for additional information. One randomized controlled trial reported data for two independent treatment subgroups (dynamic therapy and CBT) (37). In this case, each subgroup was treated as a separate study (38). When correlations of several insight scores with outcome (39, 40), of insight with several outcomes (41), or all subscales of an outcome measure (42) were reported, we computed the mean correlation coefficient for that study. For studies reporting only partial information on the insight-outcome association (e.g., only for significant subscales), we imputed missing nonsignificant correlations with a correlation coefficient of zero before calculating a mean score (42, 43).

Two authors (S.J., J.H.) independently assessed the risk of bias in each primary study. For this assessment, we adapted the Systematic Assessment of Quality in Observational Research Scale (44) and included additional items to assess common artifacts in meta-analyses of correlations (45). The modified 16-item risk of bias scale covers four categories: sample, quality of measurements, quality of statistical analyses, and reporting of data (see the online supplement). Each category includes three to five items, which are evaluated with ratings of 0 (no risk of bias), 1 (definite or unclear risk of bias), or not applicable. A mean score of applicable items yields a ratio between 0 and 1. Higher scores represent more risk of bias. Interrater reliability for the risk of bias assessment was good (intraclass correlation coefficient [ICC] of 0.80 using the formula ICC[3,1]; see reference 46), and scores were averaged across raters.

Effect sizes were measured as correlation coefficients. All effects were coded such that positive correlations indicate more insight to be associated with better outcome. All Pearson correlations were transformed with Fisher’s r-to-z transformation prior to any computations and back-transformed for interpretation purposes.

Analyses were performed using the R package metafor, version 1.9-9 (47). Alpha was set at 0.05 for all analyses. First, we estimated the average true population effect size of the insight-outcome association in a random-effects model. This model was chosen a priori to allow for unconditional inferences about the effect size distribution in the population (48). Residual heterogeneity was estimated with the restricted maximum likelihood estimator (49). Study weight was calculated as the inverse of the sum of the study’s standard deviation and the estimated heterogeneity. The amount of heterogeneity was assessed using the Q-test for heterogeneity, the estimated amount of variability in the true effect sizes (tau), and the proportion of observed variability that can be attributed to true heterogeneity (I²) (38). Values for I² of 25%, 50%, and 75% indicate a low, moderate, and high degree of heterogeneity, respectively (50). We performed prospective power analyses to detect different population effect sizes for conditions of low, moderate, and high heterogeneity in SAS Studio, version 3.6 (51), according to the method described by Hedges and Pigott (52). The analyses indicated adequate power to detect small (power=0.70–0.84), medium (power≈1) and large (power≈1) effects for all heterogeneity conditions.

Finally, we assessed the risk of publication bias by visually examining the funnel plot and conducting Egger’s regression test for funnel plot asymmetry (38). We applied Duval and Tweedie’s trim-and-fill procedure (53) as a sensitivity analysis to estimate the unbiased average population effect.

The 22 studies (with a k of 23 independent effect sizes) subsumed a total of 1,112 individuals. The mean age of participants was 35.27 years (SD=6.01, range=23.70–46.40), and 63.84% of the total sample was female. Participants attended a median of 20 sessions (mean=73.27, SD=239.63, range=1–1118). There was considerable variety in psychopathology. Eighteen of the studies did not have major restrictions regarding diagnostic categories and used convenience samples of patients seeking treatment. In these studies, patients were mostly diagnosed as having axis I disorders according to DSM-IV-TR criteria (or criteria from earlier versions of DSM), among them mood, anxiety, somatoform, and eating disorders. There was considerable comorbidity with axis II disorders. The remaining four studies comprised samples of patients with mood, anxiety, eating, or cluster C personality disorders. Eighteen studies provided individual outpatient treatment. Three studies were based on inpatients who received a combination of individual and group psychotherapies. One study employed an outpatient group therapy setting. Theoretical orientations varied among treatments (Table 1). Twelve treatments were identified as psychodynamic and 11 as other treatment types, such as cognitive-behavioral, unspecified counseling, emotion-focused, and mixed forms of humanistic, client-centered, and interpersonal psychotherapy. The 22 studies used 18 different insight measures and 27 different outcome scales. Six studies examined patients’ self-reports of insight, and the remaining 16 studies used clinician-rated insight scales. As outcome measures, 12 studies employed measures of psychopathological symptoms, three studies assessed interpersonal functioning, and one study evaluated quality of life. For 13 studies, other categories, such as occupational functioning and target complaints, were used as outcome measures. Studies employed various designs to associate insight and outcome. Five effect sizes were based on cross-sectional correlations between insight and session outcome, three were correlations of insight and subsequent outcome, five were correlations of insight with change in outcome, and nine represented associations between change in insight and change in outcome.

The random-effects model for the association between insight and outcome estimated an average true population effect size (r) of 0.31 (95% CI=0.22–0.40, p<0.05). Effect sizes and 95% confidence intervals of the studies (27, 29, 31, 33, 35, 37, 39, 41–43, 54–65) are presented in a forest plot in Figure 2. Eleven of the 23 reported individual effect sizes were nonsignificant within the primary study. There was significant heterogeneity in the effect sizes (Q=58.71, p<0.05). The estimated standard deviation of true effects (tau) was 0.19 (95% CI=0.10–0.28), and the ratio of true heterogeneity to total variation in the observed effects (I²) was 60.30% (95% CI=29.08−78.02). The amount of heterogeneity between studies suggested the examination of potential moderating variables. Moderator analyses were attempted for type of treatment (dynamic versus other treatments), patient diagnosis (percentage of personality disorders within study sample), and source of information on insight (observer rating versus self-report). However, the presumed statistical power for the moderator analyses was low (<0.20) (38, 66). Within treatments classified as dynamic, the magnitude of effects (r) ranged from 0.01 to 0.59. Effect sizes (r) for the studies on CBT ranged from −0.08 to 0.34, and those for the unspecified counseling studies varied from 0.24 to 0.68. The associations in the remaining studies on emotion-focused, interpersonal, client-centered, supportive, and mixed types of therapy ranged from 0.19 to 0.53.

The assessment of study quality with the risk of bias scale demonstrated moderate risks across individual studies and risk of bias categories (see Table 2). Risk of bias scores were highest for sample risks, meaning that most studies lacked sufficient sample size or representativeness or did not adequately clarify the source of the given sample. There was a high risk of measurement bias in about a third of the studies, where insight or outcome measures were not operationalized precisely or did not have adequate psychometric properties. In about half the studies, there was a notable risk of biased calculations because of attrition rates ≥5%, insight and outcome ratings provided by the same person, unclear times of measurement, same-session assessment of insight and outcome, or no direct availability of a zero-order correlation. The least concerns were present for risk of reporting bias, as most studies gave explanations for missing data, presented data clearly and accurately, and showed no indicators of selective reporting. Only three studies reported therapist effects, and they controlled for them using a multilevel modeling approach (54, 64, 65). None of the studies had a preregistered study protocol, and none explicitly stated taking preventive actions against possible researcher allegiance effects, such as blind ratings of insight or blind data analysis.

Finally, we examined whether publication bias was a reason for concern. Visual inspection of the funnel plot for the random-effects model suggested minor asymmetry, and Egger’s regression test for funnel plot asymmetry was nonsignificant (z=−0.77, p>0.05). We applied Duval and Tweedie’s trim-and-fill procedure to assess the potential impact of a bias. The method imputed two studies to achieve funnel plot symmetry, leading to an estimated unbiased estimated effect size (r) of 0.35 (95% CI=0.25–0.45, p<0.05). The original and adjusted effect size estimates therefore differed only in the second decimal. Figure 3 presents the funnel plot including the data filled in by the trim-and-fill method.

Although the results of our meta-analysis are encouraging for the relevance of insight as a curative factor in psychotherapy, limitations must be noted. First and most important is that the analyses are based exclusively on correlational data and thus do not allow causal inferences. Not only could insight lead to improved well-being, but it is also possible that symptom reduction enables patients to reflect about themselves more openly and gain new insights (reverse causation). Furthermore, the association between insight and psychotherapy outcome could be caused by a third variable, such as enhanced quality of object relations. The studies discussed in this review vary in the study designs used and the conclusions that can be drawn. Cross-sectional correlations between insight and session outcome have the highest probability of confounding factors or reverse causation. For associations between insight and subsequent outcome or changes in outcome, confounding factors may still be present, but the risk of reverse causation is diminished. About half the studies in this review correlated change in insight with change in outcome and thus tested the insight-outcome association in a more rigorous way. To establish insight as a mediator, that is, a mechanism that brings about change in psychotherapy, several conditions must be met. First, the treatment must be associated with change in outcome. Second, the treatment must be associated with change in insight. Third, change in insight must predict change in outcome. Fourth, when insight is included in the statistical model, the effect of treatment on outcome must be diminished or eliminated. Fifth, it should be demonstrated that change in insight temporally precedes change in outcome (71–73). In order to establish insight as a specific mechanism (as opposed to a common factor), the treatment rationale should target insight, for example, via interventions such as confrontation or interpretation for dynamic therapies. Ulberg and colleagues (71) demonstrated that increased insight during treatment mediated the effect of transference interpretations on improvements in interpersonal functioning. The data available for the present correlational meta-analysis only permitted tests of the association between insight (or change in insight) and outcome (or change in outcome). Further longitudinal research that facilitates all required steps for mediation is necessary to provide evidence for insight as a putative mediator. Nevertheless, demonstrating a significant association between self-understanding and psychotherapy outcome is an important first step to establish insight as mechanism of change (3).

Second, although the topic is relevant for a variety of therapeutic methods and inclusion criteria were defined broadly, relatively few studies were eligible for this meta-analysis. This not only reduced the precision of the population effect size estimate, but it also prevented moderator analyses because of low statistical power. Moderators are pretreatment variables that alter treatment response and thus would have answered the question of which patient characteristics or treatment conditions influence the magnitude of the insight-outcome association (71–73). Further research is necessary to examine potential moderators, such as type of treatment, patient diagnosis, and the source of information on insight. Testing the type of treatment as a moderator will address whether insight should be considered a common factor or whether it is specific to psychodynamic therapy. Patient diagnosis is of interest, because different mechanisms of change may operate for patients with and without personality disorders. For example, patients with personality disorders seem to benefit more from transference interpretations that are assumed to enhance their interpersonal functioning (74, 75). Whether insight is of different importance for various groups of patients is an important topic for future research. Furthermore, the source of information on insight may be influential. Because lower insightfulness can limit patients’ ability to report on their degree of insight, studies with self-reports may yield different results compared with observer ratings.

Finally, the large number of different insight measures may be viewed as a threat to validity. Although only studies using scales that fit our preregistered definition of insight were included, the variety of measures raises the concern that different aspects of the construct were assessed. The different measures may have caused part of the heterogeneity between study effect sizes. Nonetheless, the amount of true heterogeneity in our study was within the usual range of meta-analyses on change mechanisms (such as the alliance [67, 68]) and still led to a considerably precise population effect size estimate that allows conclusions on the insight-outcome association.