Brain Imaging in Psychiatry: Time to Move From Regions of Interest and Interpretive Analyses to Connectomes and Predictive Modeling?

On September 13, 1848, Phineas Gage, a 25-year-old railroad foreman, was injured by an iron rod that destroyed part of his left frontal lobe, leading to profound personality changes (1). This story has fascinated neuroscientists and served as a foundation for a large body of neuroimaging research aiming to identify associations between localized brain regions and psychiatric disorders. Unfortunately, our focus on brain regions of interest did not yield the biological psychiatry revolution the field hoped for. Close to five decades have passed since the first associations of brain imaging measures with psychiatric disorders (2), yet the utility of neuroimaging in psychiatry remains primarily limited to research settings, and neuroimaging is neither part of the standard of care in the clinic nor leading the way in providing precision medicine. One potential reason for the limited progress is that the complexity of human behavior cannot be captured by the function of a single brain region or even a set of brain regions. Just as the genetics literature has largely moved beyond candidate gene analyses toward genome-wide association studies, it may be time for our field to advance new analytic strategies that investigate the brain as a whole rather than as a sum of its parts.

In this issue of the Journal, which focuses on neurodevelopmental psychiatric disorders, Russman Block and colleagues (3) report on an elegant longitudinal study that illustrates both strengths and weaknesses of focusing on a set of regions of interest (ROIs). The authors investigated patterns of brain functional connectivity as predictors of the therapeutic effects of exposure and response prevention (ERP) in adolescents (N=54) and adults (N=62) suffering from obsessive-compulsive disorder (OCD) with childhood onset. The authors computed functional connectivity between 20 ROIs extracted from functional MRI scans obtained before treatment. They then conducted univariate analyses, with appropriate correction for multiple comparisons, to determine the connections that were associated with enhanced response to ERP compared to the response to a control psychotherapeutic treatment. Better responses to ERP were associated with higher connectivity between the ventromedial prefrontal cortex ROI and ROIs in the left caudate and left thalamus. Secondary analyses identified several connections that were associated with improvement in OCD symptoms, but these associations were not specific to treatment modality. Overall, the study has many strengths, including the relatively large sample size, longitudinal interventional design, and comparison of neural predictors of response in adults and adolescents.

A main strength of the ROI approach is that it is optimally suitable for testing specific hypotheses. However, a main weakness of the ROI approach is that it is highly vulnerable to testing indiscriminate associations followed by hypothesizing after the results are known, a problematic research practice described in the literature (4). Another critical limitation specific to functional connectivity studies is that ROI results are regularly interpreted in terms of disturbances in brain intrinsic connectivity networks (ICNs), such as the default mode, central executive, and salience networks. Each ICN contains dozens of brain regions and thousands of connections, and these ROI analyses test only a few brain regions and a limited number of connections within each ICN or between ICNs. Furthermore, the ROI approach tests the strength of connections without assessing ICN subnetwork or graph properties (e.g., efficiency and the integration or segregation of connections). As a field, we need to provide more rigorous assessments of these brain networks, including a more comprehensive testing of brain regions and connections to determine which networks or subnetworks are involved and what the nature of each network disturbance is. It is concerning that the ROI-based connectivity approach followed by inferences about ICNs has restricted the interpretation of findings across most psychiatric disorders to disruptions in the triple-network or tripartite model, that is, a disturbance in the interactions among the central executive, salience, and default mode networks (5). Although these networks appear to play a central role in human behavior, it is important to investigate whether alterations in particular subnetworks or in intermodular communication map better to specific aspects of human behavior and psychiatric symptoms. Network-informed neuroimaging methods to more comprehensively assess the brain connectome and identify symptom-specific networks are increasingly available and should be further developed and adopted in the field (6).

Two other drawbacks in the psychiatric neuroimaging field are the common use of univariate analyses and interpretive models. The use of univariate analyses presents statistical, methodological, and theoretical challenges. A dense connectome could generate up to 1.8 billion unique features, but even vertex- and voxel-wise analyses are still in the range of thousands of features. Therefore, conducting univariate analyses with these data presents a significant challenge for balancing type I and II errors, which may affect reproducibility. Methodologically, assumptions of independence of each univariate association are not supported given that various preprocessing steps (e.g., smoothing and warping) blend colocalized features. Furthermore, human behavior may be related to overall changes in network strength and characteristics and not necessarily limited to changes in connection between two regions within the network; for example, improvements in behavioral symptoms may be predicted by the overall connectivity strength of the default mode network but not by the strength of connectivity between two brain regions within the default mode network. In other words, the behavioral changes may be more closely associated with a pattern of brain changes rather than a single univariate change. The limitation of traditional statistics and interpretive models commonly used in neuroimaging research is that they are based on fitting all available data to make inferences from the model that best fit the tested data. However, the tested samples are substantially smaller than the target population, which leads to overfitting and inflated effect size estimates, with empirical studies showing the need for sample sizes in the thousands to identify reproducible brain-behavior associations using univariate interpretive models (7).

An alternative approach that alleviates these challenges is to conduct multivariate pattern analysis. Multivariate pattern analysis is an analytic technique that makes use of information contained jointly among multiple variables to establish a pattern of changes that are specific to an outcome (e.g., symptom or disease) (8). Multivariate pattern analyses with predictive modeling, regularly used in machine learning, are critical for neuroimaging research and may both enhance reproducibility and identify symptom-specific networks (9). Indeed, the “predictors” mentioned in the Russman Block study are simply correlations. In the true sense of the word, prediction implies a causal connection between a measure and an outcome. Quite a number of imaging studies have demonstrated these correlations, but none of these correlations are used in routine clinical practice in a prospective manner, because none of them seem to have true predictive ability. More routinely incorporating cross-validated modeling approaches (e.g., leave-one-out analyses) in neuroimaging studies would provide this degree of true prospective predictive power.

Notwithstanding the aforementioned limitations of the ROI approach, studies such as Russman Block and colleagues’ may advance our understanding of which nodes (or connections) of the cortico-striato-thalamo-cortical loop are associated with a specific modality of treatment for OCD (10). In the case of ERP, it will be important to learn whether ventromedial prefrontal cortex-subcortical region (i.e., caudate and thalamus) connectivity at baseline predicts which patients with OCD will respond best to ERP. This may translate into improved outcomes if treatment plans can be enhanced by including more mechanistically driven treatment components (e.g., exposure) rather than less relevant components (e.g., cognitive therapy). The authors note that those with greater ventromedial prefrontal cortex-subcortical region connectivity may benefit from relaxation approaches given the stronger affective attachment to compulsions; however, an alternative conclusion is that these patients may need a more robust dose of ERP, particularly given the limited benefits of relaxation observed in this and other OCD studies (11). This more robust dose of ERP may be incorporated through additional sessions that deviate from a one-size-fits-all model, by targeting independent exposure practice (12) or through adjunctive interventions targeting extinction learning (e.g., d-cycloserine) (13).

When presenting neuroimaging findings during grand rounds, a recurrent question often asked by enamored residents and other trainees is “When will these exciting neuroimaging findings be part of psychiatric care in our clinics?” Sadly, our answer is “We’re not there yet.” Outstanding brain imaging research, including that described by Russman Block et al. in this issue, has been conducted for over half a century, and we are still not there yet. Perhaps it is time to forsake traditional statistics and Gage’s “single brain region epiphany” to move toward accepting that the brain is more complex than the sum of its parts and that larger, more concerted efforts are required to unravel its mysteries (7).