UW Researcher Heads Study Finding Neural Circuit Controls Social Hierarchy Behavior in Mice

A University of Wyoming researcher led a study that used mice to show that social hierarchy involves status-dependent behavioral interactions that are controlled by neurons in the brain.

 

While animals were used in the experiment, the research provides fundamental insights into how neuron-to-neuron communication gives rise to complex behaviors, with applications to neuropsychiatric conditions in humans that are defined by disruptions of social behavior, such as schizophrenia, major depressive disorder and autism spectrum disorder.

 

“The development of social hierarchy reflects an essential function of the brain. Through repeated social interactions, individuals come to adopt certain behaviors that reflect their social rank. A convenient outcome of social rank is that it can reduce interpersonal conflicts within the group,” says Adam Nelson, a principal investigator in the UW Department of Zoology and Physiology. “We know that some key brain regions regulate hierarchy establishment, such as the cortex. But how is this process regulated? At the molecular level, how does neuron-to-neuron communication change as the brain decides to become more dominant or more subordinate?” 

 

Nelson was lead author of a paper, titled “Molecular and neural control of social hierarchy by a forebrain-thalamocortical circuit,” that appears Aug. 11 (today) in Cell. Cell is the flagship journal of Cell Press, a pre-eminent international publisher of cutting-edge biomedical research and reviews.

 

Other contributors to the paper were from the Howard Hughes Medical Institute, Department of Molecular and Cellular Biology, and the Center for Brain Science, all at Harvard University; and Calico Life Sciences LLC in San Francisco.

 

Rat Race

 

The study revealed hierarchy in mice is regulated by a multi-synaptic circuit linking the basal forebrain, orbitofrontal cortex, mediodorsal thalamus (MDT) and the caudal anterior cingulate cortex (cACC). Achieving higher social rank is supported by molecular- and circuit-level changes potentiating MDT-to-cACC inhibition, revealing how coordinated mechanisms underlie competitive behavior, according to the study.

 

To chart the development of social hierarchy among mice that were initially unfamiliar with one another, the study had mice participate in resident-intruder and tube tests daily over a one-week period, according to the paper.

 

During the resident-intruder tests, territorial behavior of mice was measured. By displacing a male mouse from his home cage to the home cage of another male for a short period, this allowed researchers to quantify the aggressive and territorial behaviors that this procedure elicits.

 

The tube test involved placing two mice in a narrow tube and observing their head-to-head interactions. The first mouse to leave the tube was deemed the loser.  

 

Both tests used a round robin-style tournament format, which allowed researchers to tally wins and losses among each animal in the group. Groups consisted of either three or four males. 

 

“We found that interactions within the hierarchy are often decided by the lower-ranking mouse. That is, when a middle-rank mouse engages with a higher-rank mouse, his probability of displaying defensive behavior goes up; when that same animal engages with a lower-rank male, his probability of displaying defensive behavior goes down,” Nelson explains. “What we found was that, over time and with more interactions, the hierarchy became more stable and predictable.”  

 

Researchers associated this behavior with distinctive patterns of activity in two cortical brain areas: the cingulate and the prefrontal cortex. Intriguingly, these areas show opposite activity profiles, Nelson says. During winning, the cingulate in mice is suppressed and the prefrontal cortex is activated, and vice-versa during losing.

 

Brain circuitry
 

The study made two primary discoveries. The first was the identification of a neural circuit “hub” -- the MDT -- that becomes modified in high- and low-rank animals, Nelson says. During social competition, the MDT receives information from two brain areas -- the orbitofrontal cortex and the basal forebrain -- and sends information to the anterior cingulate cortex, or “cingulate.”

 

“We demonstrate that MDT neurons are more excitable in high-rank compared to low-rank animals, owing to increased excitatory neural inputs from the orbitofrontal cortex and decreased inhibitory inputs from the basal forebrain. In turn, enhanced MDT excitability drives feed-forward inhibition of the cingulate,” Nelson says. “By driving a decrease in caudal anterior cingulate activity, MDT activity is associated with higher competitive ability. Using manipulation-based experiments, we demonstrated a causal relationship between activity in this circuit and competitive performance. For example, activation of MDT projections to the cingulate triggered both a reduction in cingulate activity and an increase in the competitive ability of low-rank animals.” 

 

The second major finding was the identification of a channel known as TRPM3 (transient receptor potential cation channel subfamily M member 3) that allows positively charged calcium ions into the cell. TRPM3 is more highly expressed in MDT neurons of high-ranking mice compared to low-rank animals, Nelson says.

 

Using the manipulation-based experiments, the study showed that activation of the TRPM3 channel in the MDT causes both increased excitability of MDT neurons and increased competitive performance, likely by enhancing feedforward inhibition of the cingulate. 

 

“Taken together, our findings provide much deeper insights into the precise changes that occur within neurons to establish flexible, rank-dependent behavioral profiles, which are core building block of social hierarchies,” Nelson says.

 

Future implications for humans?

 

To increase the basic understanding of how human brains give rise to flexible behavioral states, more cause-and-effect experiments are needed, according to Nelson. For example, in neuropsychiatry, the development of therapeutics for diseases, such as schizophrenia and major depressive disorder, tends to move slowly compared to other disease areas. This reflects the limited understanding of how the brain controls behavior.

 

“In humans, social status is highly linked to mental health and psychiatric disease,” Nelson says. “But precisely how status translates into differences in mental health is a mystery.” 

 

The study was funded by the Howard Hughes Medical Institute and National Institutes of Health grants.