Social cognition is critical for human behavior in complex social environments (
1,
2), and deficits in social cognition consistently underlie functional disabilities in individuals suffering from a wide range of psychiatric disorders (
3–
7). Social novelty recognition (the ability to recognize social cues), a key subdomain of social cognition, can be assessed by performance-based measures of social cue (e.g., face or voice perception) in humans (
8,
9) as well as by social novelty recognition/preference paradigms (three-chamber social interaction test and five-trial social novelty recognition test) in rodents (
10,
11).
The insular cortex was one of the least understood brain regions until recently (
12–
14). The advent of functional imaging techniques has unveiled its multiple functions in humans, which include interoception, emotion, cognition, and motivation (
15–
17). These studies, including a recent meta-analysis, have suggested that the insular cortex is a “common core” brain region that is impaired across neurological and psychiatric disorders (
2,
12,
15,
18–
20). The insular cortex can be divided into the anterior insula (AI) and the posterior insula (PI). They differ from each other in cytoarchitecture, connections, and functions, but they are also tightly interconnected. The PI shows a classical six-layer structure, whereas the AI lacks granular layer 4 (
12). The PI receives multiple sensory inputs (
12,
20), whereas the AI makes heavy reciprocal connections with the frontal areas and limbic system and participates in the salience network (
20).
Oxytocin (OT) is a well-appreciated neuropeptide that influences social behavior, including social recognition, pair bonding, mating, and parental care (
21–
23). OT modulates these behaviors by being released into the target brain regions, such as the amygdala, ventral tegmental area, and dorsal raphe, from the axonal fibers of oxytocinergic neurons located in the paraventricular nucleus (PVN) and supraoptic nucleus (SON) (
24,
25). Animal studies for key molecules of the OT signaling pathway, such as OT, the OT receptor (OTR), and CD38 (cluster of differentiation 38, transmembrane glycoprotein with ADP-ribosyl cyclase activity that facilitates OT secretion), consistently support its critical role in social behaviors (
22,
26,
27). When abundant OT release occurs in the brain, for instance during lactation, the AI is one of the most robustly activated brain regions (
28,
29). Nevertheless, quantitative analyses of the distribution of OT fibers in multiple brain regions have found that the OT projections to the AI are negligible in multiple species (
24,
30). In contrast, OT projection and rich expression of OTR binding are found in the PI (
24,
31). Thus, OT signaling may be distinct between the PI and AI. Altogether, the mechanism that underlies the activation of the AI in response to OT in humans remains to be elucidated.
We and others have proposed the importance of using animal models in research to discover novel translatable mechanisms at the molecular and cellular levels in this understudied brain region (
12). Outstanding studies with rodents that explore roles of the insular cortex, in particular focusing on the PI, have indeed emerged in the past several years: at least in rodents, the PI is involved in emotional processing, and the resultant social behavior in response to socially affective stimuli is mediated by OTR-dependent neural activity (
32,
33). Here, we address the aforementioned enigma from human imaging studies and decipher a role of AI in social behavior in association with OT signaling. We used an AI-enriched molecule, Cyp26B1, which is crucial for retinoic acid (RA) degradation and is involved in the pathology of neuropsychiatric conditions, as a molecular lead to decipher the role of AI in social cognition.
Discussion
In the present study, by using mouse models, we report a pivotal role of the AI in social novelty recognition in which proper activity of the layer 5 pyramidal neurons is required. At the molecular and cellular level, AI-mediated social novelty recognition is maintained by proper activity of AI layer 5 pyramidal neurons, for which retinoic acid–mediated gene transcription plays a role. Social novelty recognition is also regulated by 5-HT2C receptor expressed in the AI. Furthermore, we demonstrate that OT influences AI-mediated social cognition not by direct projection of OT neurons, nor by direct diffusion of OT in the AI, but rather by affecting OTR-expressing neurons in the DRN, where serotonergic neurons are projected to the AI. Accordingly, the regulatory mechanism by OT on the AI is distinct from that on the PI, where direct projection of OT neurons plays an important role.
In the brain, OT exerts its control through long-range axonal release, and hence projection of OT fibers and the expression of OTR in the target brain region is crucial. In this context, several brain regions have been postulated to be involved in OT-associated social cognition, including the prefrontal cortex, hippocampus, and olfactory nucleus (
50–
53), and, more recently, the PI (
33). Unlike the PI, projection of OT neurons to the AI is sparse (
33), and the expression of OTR is attenuated (
54). In human functional connectivity studies, there are distinct subdivisions in the insular cortex for processing cognitive information and affective feeling (
13). Recent studies in rodent models have highlighted a role of the PI in social behavior (
32,
33). It is an important future question whether and how the involvement of the AI and PI in social cognition and behavior is similar and different, possibly with distinct regulatory mechanisms for each subregion, including those by OT.
We underscored the significance of the 5-HT
2C receptor expressed in the AI for social novelty recognition. In the past, through systemic administration of a 5-HT
2C receptor antagonist in wild-type animals (
55) or the phenotype of 5-HT
2C receptor genetic knockout (
56), the involvement of this receptor in both social novelty recognition and sociability has been suggested. In contrast to a broad expression of 5-HT
1A and 5-HT
2A receptors in the frontal cortex (
57,
58), the expression of 5-HT
2C receptor is more restricted, with a more prominent expression in the medial prefrontal cortex and AI (
58,
59). Our data in the present study showed that local injection of a 5-HT
2C receptor antagonist elicits the deficits only in social novelty recognition, but not in sociability, which is compatible with our mechanistic proposal. Thus, the impact of 5-HT
2C receptor in sociability may be via the receptor in the medial prefrontal cortex, which is known to be involved in sociability (
60,
61). Genetic manipulations of this receptor in the AI pyramidal neurons can further validate its specific role in social novelty recognition.
The present study has pinned down the mechanistic question for the next step of how OT can influence AI-mediated social novelty recognition in a nonclassic way (i.e., neither OT neuron direct projection, nor direct diffusion of OT to the AI). One of the most tempting working hypotheses may be a disynaptic model in which the OTR-expressing DRN serotonin neurons, receiving input from PVN/SON-originated OT neurons, directly project to the AI pyramidal neurons and exert their effect on AI-dependent social novelty recognition (see Figure S10 in the online supplement). This model is theoretically addressable, although experimental optimization may be very difficult, by depleting OTR specifically from the AI-projecting DRN neurons and further rescuing the AI projection by activating this projection specifically using optogenetic manipulation. Through this experiment, the necessity and sufficiency of the disynaptic mechanism may be shown in future studies to account for the influence of OT on AI-mediated social novelty recognition. Alternatively, the AI may be influenced by OT through an indirect mechanism via a brain area to which OTR-expressing nonserotonergic neurons project from the DRN.
Future investigation may also address local mechanisms to a greater extent, for instance how AI pyramidal neuron activity is regulated for AI-mediated social novelty recognition. These mechanisms may include modulation of neuronal excitability, possibly mediated via G protein–coupled potassium current downstream of 5-HT
2C receptor (
62,
63), and local synaptic regulation via RA signaling. Future studies should also address downstream executive mechanisms whereby the AI pyramidal neurons eventually influence behavior. The functional relationship of the AI with the amygdala and hippocampus, in particular the CA2 (
64) and ventral CA1 (
65), may be topics of future investigations.
Although recent human genetic research has suggested the significance of RA-associated genes in brain disorders (
66,
67), the biological role of the RA-associated cascade in mental conditions has not yet been fully addressed. Mild elevation of RA signaling by RARβ2 expression or by chronic neuronal inactivation has been reported to show a positive effect on neurite regeneration and homeostatic synaptic scaling (
68–
71). In contrast, in our experimental mouse model, we chronically disturbed RA signaling by suppressing Cyp26B1 in the AI layer 5 pyramidal neurons. The present model will be useful in deciphering further mechanisms of how OT and serotonin regulate AI-mediated social cognition. The model will also be useful for studying the pathological impact of chronic disturbance in RA signaling in the AI. Consistent with this notion, in multiple expression studies of postmortem brains from patients with mental disorders (
39), Cyp26B1 has been underscored as a top molecule of interest. Together, the Cyp26B1 models used in the present study will be useful to explore mechanisms of social cognition deficits in mental disorders.