Anger and Depression in Middle-Aged Men: Implications for a Clinical Diagnosis of Chronic Traumatic Encephalopathy
Publication: The Journal of Neuropsychiatry and Clinical Neurosciences
Abstract
Objective:
In recent years, it has been proposed that problems with anger control and depression define clinical features of chronic traumatic encephalopathy (CTE). The authors examined anger problems and depression in middle-aged men from the general population and related those findings to the proposed clinical criteria for CTE.
Methods:
A sample of 166 community-dwelling men ages 40–60 was extracted from the normative database of the National Institutes of Health Toolbox. All participants denied prior head injury or traumatic brain injury (TBI). Participants completed scales assessing anger, hostility, aggression, anxiety, and depression.
Results:
In response to the item “I felt angry,” 21.1% of men reported “sometimes,” and 4.8% reported “often.” When asked “If I am provoked enough I may hit another person,” 11.4% endorsed the statement as true. There were moderate correlations between anger and anxiety (Spearman’s ρ=0.61) and between depression and affective anger (ρ=0.51), hostility (ρ=0.56), and perceived hostility (ρ=0.35). Participants were dichotomized into a possible depression group (N=49) and a no-depression group (N=117) on the basis of the question “I feel depressed,” specific to the past 7 days. The possible depression group reported higher anxiety (p<0.001, Cohen’s d=1.51), anger (p<0.001, Cohen’s d=0.96), and hostility (p<0.001, Cohen’s d=0.95).
Conclusions:
Some degree of anger and aggression are reported by a sizable minority of middle-aged men in the general population with no known history of TBI. Anger and hostility are correlated with depression and anxiety, indicating that all tend to co-occur. The base rates and comorbidity of affective dysregulation in men in the general population is important to consider when conceptualizing CTE phenotypes.
There are no consensus-based or validated clinical criteria for the clinical diagnosis of chronic traumatic encephalopathy (CTE), although several sets have been proposed (1–4). During the 20th century, CTE was described as a neurological disorder characterized by dysarthria, gait disturbance, tremor, and cognitive impairment—with some cases remaining mild and not progressing and other cases progressing to advanced Parkinsonism and dementia (5–11). In recent years, the clinical features attributed to the neuropathology of CTE have been expanded greatly, and there is a risk that these purported clinical features and their association with specific types of neuropathology will be woven into a Gordian knot. According to Greek and Roman mythology, a Gordian knot is so intricate and complex that it is nearly impossible to disentangle or untie. A major shift in conceptualizing CTE in recent years has been to assert that a broad range of mental health and psychosocial problems that are present prior to death are clinical features of CTE. Examples of the breadth of mental health problems attributed to CTE include depression and anxiety (12–14); substance abuse (1, 12, 13); personality changes, anger control problems, and violence (12–14); and suicidal thinking and death by suicide (13–19). In regard to death by suicide, several reviews (20–23), one retrospective historical case review study (24), and one epidemiological study (25) have concluded that there is minimal or no scientific evidence to support this assertion in former professional athletes. Authors have attributed psychosocial problems, such as poor financial decisions and bankruptcy (12), gambling (1), and marital problems, separation, and divorce (19), to the neuropathology of CTE. Some authors have even asserted that generalized body aches and pain (12), headaches (12, 14, 26), and insomnia (19) are clinical features of CTE.
It is essential to appreciate that many of the symptoms and problems described above are common in the general population, such as depression (27), anxiety (28), anger (29), financial problems (30), marital problems or divorce (31), headaches (32), bodily pain (33), and insomnia (34). Moreover, people with medical, psychiatric, neurological, and neurodegenerative diseases that are etiologically separate from CTE are also expected to have some of these symptoms and problems. A major gap in the literature is the lack of studies focused on examining the specificity of the proposed clinical features of CTE in the general population and in people with known clinical conditions resembling CTE who have no prior exposure to repetitive neurotrauma. The purpose of this study was to examine two of the assumed core clinical features of CTE, anger control problems and depression (1), in middle-aged men from the general population. We hypothesized that a small minority of men in the general population would report problems with anger and depression and that in those who did there would be a strong association between these two psychological problems.
Methods
Participants
Participants in this study were men from the U.S. general population who participated as subjects in the standardization sample for the National Institutes of Health (NIH) Toolbox for the Assessment of Neurological and Behavioral Function (35, 36). This project sampled a community-dwelling population from 10 sites who were capable of following instructions in English or Spanish, could understand the informed consent process, and had adequate visual, auditory, vestibular, or motor functioning to complete some of the items in the test battery or availability of assistance or assist devices to complete tasks. The overall sample had a distribution of demographic characteristics similar to the 2010 U.S. Census.
The NIH Toolbox Emotion database contained 1,630 rows of data, with 1,445 unique participants at the time 1 assessment (i.e., 185 were assessed a second time). From this, 180 participants were men between the ages of 40 and 60 years old. Ten men were excluded due to prior head injury (i.e., variables ATO_Head, ATOPx_Head, and MedHx_TBI), and four were excluded for not completing the Anger-Affect Questionnaire, leaving a total of 166 men for analysis (mean age, 49.1 years [SD=5.9]).
This sample was predominantly Caucasian (80.7%; African American, 10.2%, American Indian/Native American, 3.6%, Asian, 1.8%, Native Hawaiian/Pacific Islander, 0.6%, unknown, 3.6%). Almost one-half (42.8%) identified as Hispanic or Latino. The assessment was administered in English only (65.1%), Spanish only (23.5%), and bilingually in English and Spanish (English primary=3.6%, Spanish primary=7.8%). Level of education in the sample was as follows: less than high school diploma=16.3%, high school diploma or general equivalency diploma=27.1%, some college or an associate’s degree=20.5%, bachelor’s degree=25.3%, master’s or doctoral degree=5.4% (missing=5.4%).
The week before the assessment, 70.5% of participants reported working (69.2% working for pay; 1.2% were working but not for pay, for example, at a family business), 3.0% were employed but not working, 13.2% were looking for work, and 10.2% were not working or looking for work (3.0% missing or prefer not to say). Of the 29.5% (N=49) who were not working the week before the assessment, the reasons they cited for not working were taking care of house/family (N=5), retired (N=5), vacation (N=2), temporary health reasons (N=1), have job/contract but off-season (N=4), laid off (N=8), disabled (N=7), other (N=12), missing/prefer not to say (N=5). Whether some of these men participated in sports in their youth or whether they served in the military was not recorded in the database.
Measures
NIH Toolbox Anger-Affect Fixed Form (version 2.0).
A five-item questionnaire that instructs participants to rate how frequently they have experienced symptoms associated with anger over the past 7 days (e.g., “I was grouchy”; “I felt annoyed”). Each item is rated on a 5-point Likert-type scale (i.e., never, rarely, sometimes, often, always), with a possible score range of 5–25. Internal consistency for this scale was high in this sample (Cronbach’s α=0.86).
NIH Toolbox Anger-Hostility Fixed Form (version 2.0).
A five-item questionnaire that instructs participants to rate how true statements are for them (e.g., “I have become so mad I have broken things”). Each item is rated on a 7-point Likert scale with the anchors “extremely untrue of me” and “extremely true of me,” with a possible score range of 5–35. Internal consistency for this scale was high in this sample (Cronbach’s α=0.82).
NIH Toolbox Perceived Hostility Fixed Form (version 2.0).
An eight-item questionnaire that asks participants to rate how often other people in their life have engaged in hostile behaviors (e.g., “argue with me,” “criticize the way I do things”) over the past month. Each item is rated on a 5-point Likert-type scale (i.e., never, rarely, sometimes, often, always), with a possible score range of 8–40. Internal consistency for this scale was high in this sample (Cronbach’s α=0.90).
NIH Toolbox Anger-Physical Aggression Fixed Form (version 2.0).
A five-item questionnaire that instructs participants to rate how true statements are for them (e.g., “I have become so mad I have broken things”). Each item is rated on a 7-point Likert-type scale with the anchors “extremely untrue of me” and “extremely true of me,” with a possible score range of 5–35. Internal consistency for this scale was adequate in this sample (Cronbach’s α=0.78).
NIH Toolbox Fear-Affect Fixed Form (version 2.0).
A seven-item questionnaire that instructs participants to rate how frequently they have experienced symptoms associated with anxiety over the past 7 days (e.g., “I felt worried”; “I felt tense”). Each item is rated on a 5-point Likert-type scale (i.e., never, rarely, sometimes, often, always), with a possible score range of 7–35. Internal consistency for this scale was high in this sample (Cronbach’s α=0.89).
Patient-Reported Outcomes Measurement Information System (PROMIS) Depression Short-Form 8b (version 1.0).
An eight-item questionnaire that instructs participants to rate how frequently they have experienced symptoms associated with depression over the past 7 days (e.g., “I felt sad”; “I felt worthless”). Each item is rated on a 5-point Likert-type scale (i.e., never, rarely, sometimes, often, always), with a possible score range of 8–40. Internal consistency for this scale was high in this sample (Cronbach’s α=0.93).
Statistical Analyses
Descriptive statistics were examined at the item level for the self-report questionnaires administered in this study. The total raw scores for each scale were correlated with one another in a correlation matrix. For exploratory purposes, participants were dichotomized based on their response to the item “I feel depressed” over the past 7 days from the PROMIS version 1.0 Depression Short-Form 8b (i.e., “possible depression”=sometimes, often, or always; “no depression”=never, rarely). Independent groups t tests compared anger, physical aggression, hostility, perceived hostility, anxiety, and hopelessness between the possible depression and no-depression groups.
Results
Individual item endorsements on the physical aggression and the hostility scales are presented in Table 1. When asked, “If I am provoked enough I may hit another person,” 11.4% endorsed that item as true, and 4.8% endorsed “I have become so mad that I have broken things” as true. The percentage of men who endorsed one or more of the five physical aggression questions as true (i.e., scoring a single question as ≥5 on a 7-point scale) was 17.5%. For the question “I wonder why sometimes I feel so bitter about things,” 11.4% endorsed that as true.
| Endorsement (%)a | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Extremely untrue of me | Extremely true of me | |||||||||||
| Item | Mean | Median | SD | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 5–7 | 6–7 |
| Physical aggression | ||||||||||||
| If I am provoked enough, I may hit another person. | 2.05 | 1 | 1.62 | 59.6 | 14.5 | 6.0 | 8.4 | 6.0 | 3.0 | 2.4 | 11.4 | 5.4 |
| I get into fights a little more than the average person. | 1.28 | 1 | 0.82 | 84.2 | 9.1 | 4.2 | 1.2 | 0.0 | 0.6 | 0.6 | 1.2 | 1.2 |
| There are people who pushed me so far that we came to blows. | 1.64 | 1 | 1.35 | 71.7 | 14.5 | 4.2 | 3.0 | 3.0 | 1.2 | 2.4 | 6.6 | 3.6 |
| I have threatened people I know. | 1.42 | 1 | 1.05 | 80.6 | 7.3 | 6.7 | 2.4 | 1.8 | 0.0 | 1.2 | 3.0 | 1.2 |
| I have become so mad that I have broken things. | 1.59 | 1 | 1.24 | 71.1 | 15.7 | 6.6 | 1.8 | 1.2 | 1.8 | 1.8 | 4.8 | 3.6 |
| Hostility | ||||||||||||
| I am sometimes eaten up with jealousy. | 1.81 | 1 | 1.27 | 56.6 | 24.7 | 9.0 | 4.2 | 3.0 | 0.6 | 1.8 | 5.4 | 2.4 |
| At times I feel that I have gotten a raw deal out of life. | 2.39 | 2 | 1.60 | 39.4 | 26.7 | 10.9 | 9.1 | 7.9 | 4.2 | 1.8 | 13.9 | 6.0 |
| Other people always seem to get the breaks. | 2.46 | 2 | 1.60 | 36.1 | 25.9 | 15.7 | 9.0 | 6.6 | 4.2 | 2.4 | 13.2 | 6.6 |
| I wonder why sometimes I feel so bitter about things. | 2.29 | 2 | 1.58 | 45.2 | 21.1 | 12.7 | 9.6 | 4.8 | 5.4 | 1.2 | 11.4 | 6.6 |
| I sometimes feel that people are laughing at me behind my back. | 1.93 | 1 | 1.50 | 59.0 | 18.7 | 7.8 | 6.0 | 4.2 | 1.2 | 3.0 | 8.4 | 4.2 |
a
Each item was rated on a 7-point Likert scale.
Individual item endorsements on the anger, perceived hostility, anxiety, and depression scales are presented in Table 2. The men were asked about their experience with anger in the past 7 days. For the item “I was irritated more than people knew,” 26.5% of men reported “sometimes” and 3.6% reported “often.” For the item “I felt angry,” 21.1% of men reported “sometimes” and 4.8% reported “often.” Regarding perceived hostility, the men were asked, “In the past month, please describe how often people in your life argue with me”; 35.0% responded “sometimes” and 3.7% responded “often” or “always.” When asked about anxiety over the past 7 days, 27.1% reported “sometimes” feeling anxious and 5.4% reported “often” feeling anxious. When asked whether they felt hopeless in the past 7 days, 13.9% said “sometimes” and 3.0% said “often” or “always.” When asked whether they felt depressed in the past 7 days, 25.9% said “sometimes” and 3.6% said “often” or “always.”
| Never | Rarely | Sometimes | Often or usually | Always | Often or always | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Item | N | Mean | Median | SD | N | % | N | % | N | % | N | % | N | % | N | % |
| Anger (past 7 days) | ||||||||||||||||
| I was irritated more than people knew. | 166 | 1.97 | 2 | 0.88 | 61 | 36.7 | 55 | 33.1 | 44 | 26.5 | 6 | 3.6 | 0 | 0 | 6 | 3.6 |
| I felt angry. | 166 | 1.95 | 2 | 0.87 | 59 | 35.5 | 64 | 38.6 | 35 | 21.1 | 8 | 4.8 | 0 | 0 | 8 | 4.8 |
| I felt like I was ready to explode. | 166 | 1.60 | 1 | 0.81 | 98 | 59.0 | 40 | 24.1 | 25 | 15.1 | 3 | 1.8 | 0 | 0 | 3 | 1.8 |
| I was grouchy. | 166 | 2.14 | 2 | 0.86 | 42 | 25.3 | 67 | 40.4 | 50 | 30.1 | 6 | 3.6 | 1 | 0.6 | 7 | 4.2 |
| I felt annoyed. | 166 | 2.09 | 2 | 0.89 | 50 | 30.1 | 59 | 35.5 | 49 | 29.5 | 8 | 4.8 | 0 | 0 | 8 | 4.8 |
| Perceived hostility (past month)a | ||||||||||||||||
| Argue with me. | 163 | 2.16 | 2 | 0.90 | 45 | 27.6 | 55 | 33.7 | 57 | 35.0 | 4 | 2.5 | 2 | 1.2 | 6 | 3.7 |
| Act in an angry way toward me. | 163 | 1.90 | 2 | 0.81 | 58 | 35.6 | 67 | 41.1 | 34 | 20.9 | 4 | 2.5 | 0 | 0 | 4 | 2.5 |
| Criticize the way I do things. | 163 | 2.31 | 2 | 0.81 | 26 | 16.0 | 70 | 42.9 | 59 | 36.2 | 7 | 4.3 | 1 | 0.60 | 8 | 4.9 |
| Yell at me. | 163 | 1.69 | 1 | 0.86 | 85 | 52.1 | 49 | 30.1 | 25 | 15.3 | 2 | 1.2 | 2 | 1.2 | 4 | 2.4 |
| Get mad at me. | 163 | 2.15 | 2 | 0.82 | 35 | 21.5 | 75 | 46.0 | 48 | 29.4 | 3 | 1.8 | 2 | 1.2 | 5 | 3.0 |
| Blame me when things go wrong. | 163 | 2.05 | 2 | 0.86 | 47 | 28.8 | 69 | 42.3 | 40 | 24.5 | 6 | 3.7 | 1 | 0.6 | 7 | 4.3 |
| Act nasty toward me. | 163 | 1.75 | 2 | 0.84 | 78 | 47.9 | 51 | 31.3 | 30 | 18.4 | 4 | 2.5 | 0 | 0 | 4 | 2.5 |
| Tease me in a mean way. | 163 | 1.44 | 1 | 0.67 | 107 | 65.6 | 42 | 25.8 | 13 | 8.0 | 1 | 0.6 | 0 | 0 | 1 | 0.6 |
| Anxiety (past 7 days) | ||||||||||||||||
| I felt fearful. | 166 | 1.56 | 1 | 0.80 | 101 | 60.8 | 41 | 24.7 | 20 | 12.0 | 4 | 2.4 | 0 | 0 | 4 | 2.4 |
| I felt anxious. | 166 | 2.04 | 2 | 0.91 | 56 | 33.7 | 56 | 33.7 | 45 | 27.1 | 9 | 5.4 | 0 | 0 | 9 | 5.4 |
| I felt worried. | 166 | 2.23 | 2 | 0.96 | 45 | 27.1 | 53 | 31.9 | 55 | 33.1 | 11 | 6.6 | 2 | 1.2 | 13 | 7.8 |
| I found it hard to focus on anything other than my anxiety. | 166 | 1.73 | 1 | 0.88 | 86 | 51.8 | 46 | 27.7 | 27 | 16.3 | 7 | 4.2 | 0 | 0 | 7 | 4.2 |
| I felt nervous. | 166 | 2.01 | 2 | 0.92 | 57 | 34.3 | 61 | 36.7 | 41 | 24.7 | 4 | 2.4 | 3 | 1.8 | 7 | 4.2 |
| I felt uneasy. | 166 | 2.02 | 2 | 0.88 | 53 | 31.9 | 65 | 39.2 | 42 | 25.3 | 4 | 2.4 | 2 | 1.2 | 6 | 3.6 |
| I felt tense. | 166 | 2.25 | 2 | 0.91 | 41 | 24.7 | 55 | 33.1 | 58 | 34.9 | 12 | 7.2 | 0 | 0 | 12 | 7.2 |
| Depression (past 7 days) | ||||||||||||||||
| I felt worthless. | 166 | 1.49 | 1 | 0.80 | 110 | 66.3 | 35 | 21.1 | 18 | 10.8 | 1 | 0.6 | 2 | 1.2 | 3 | 1.8 |
| I felt that I had nothing to look forward to. | 166 | 1.53 | 1 | 0.81 | 107 | 64.5 | 34 | 20.5 | 22 | 13.3 | 2 | 1.2 | 1 | 0.6 | 3 | 1.8 |
| I felt helpless. | 166 | 1.60 | 1 | 0.82 | 98 | 59.0 | 40 | 24.1 | 24 | 14.5 | 4 | 2.4 | 0 | 0 | 4 | 2.4 |
| I felt sad. | 166 | 2.06 | 2 | 0.89 | 53 | 31.9 | 58 | 34.9 | 47 | 28.3 | 8 | 4.8 | 0 | 0 | 8 | 4.8 |
| I felt like a failure. | 166 | 1.62 | 1 | 0.84 | 98 | 59.0 | 36 | 21.7 | 30 | 18.1 | 1 | 0.6 | 1 | 0.6 | 2 | 1.2 |
| I felt depressed. | 166 | 1.92 | 2 | 0.92 | 70 | 42.2 | 47 | 28.3 | 43 | 25.9 | 5 | 3.0 | 1 | 0.6 | 6 | 3.6 |
| I felt unhappy. | 166 | 2.07 | 2 | 0.90 | 51 | 30.7 | 62 | 37.3 | 45 | 27.1 | 7 | 4.2 | 1 | 0.6 | 8 | 4.8 |
| I felt hopeless. | 166 | 1.61 | 1 | 0.88 | 99 | 59.6 | 39 | 23.5 | 23 | 13.9 | 3 | 1.8 | 2 | 1.2 | 5 | 3.0 |
a
Perceived hostility was measured in response to the question, “In the past month, please describe how often people in your life…”
An intercorrelation matrix for the psychological health measures is presented in Table 3. Spearman’s rho (ρ) values are presented because all variables were nonnormally distributed (Shapiro-Wilk p<0.001 for all six variables). The highest correlations with anger were anxiety (ρ=0.61) and depression (ρ=0.55). Hostility (ρ=0.44) and feeling perceived hostility from others (ρ=0.35) were also significantly correlated with anger. The highest correlation with physical aggression was hostility (ρ=0.37). There were small significant correlations between physical aggression and anger (ρ=0.28), anxiety (ρ=0.28), and depression (ρ=0.27).
| Spearman correlation | |||||
|---|---|---|---|---|---|
| Item | Anger | Physical aggression | Hostility | Perceived hostility | Anxiety |
| Physical aggression | 0.28* | – | |||
| Hostility | 0.44* | 0.37* | – | ||
| Perceived hostility | 0.35* | 0.29* | 0.31* | – | |
| Anxiety | 0.61* | 0.28* | 0.51* | 0.40* | – |
| Depression | 0.51* | 0.27* | 0.56* | 0.35* | 0.78* |
*
p<0.001.
Participants who responded to the question “I feel depressed” with “sometimes” (25.9%), “often” (3.0%), or “always” (0.6%) were included in the possible depression group (N=49 [29.5%]). We did not consider the group to be clinically depressed; rather, we conceptualized them as having some degree of sad mood in their life at the time of the assessment. Those who responded “never” or “rarely” to this question were included in the no-depression group (N=117 [70.5%]). The possible mild depression group endorsed significantly more anger (Cohen’s d=0.96, large effect size), hostility (Cohen’s d=0.95), physical aggression (Cohen’s d=0.48), perceived hostility toward them by others (Cohen’s d=0.38), and anxiety (Cohen’s d=1.51) than the no-depression group. These results are presented in Table 4.
| No depression | Possible depression | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Item | N | Mean | SD | N | Mean | SD | t | p | Cohen’s d |
| Anger | 117 | 8.81 | 2.92 | 49 | 11.98 | 3.64 | –3.17 | <0.001 | 0.96 |
| Physical aggression | 116 | 7.28 | 3.79 | 49 | 9.61 | 5.72 | –2.34 | 0.01 | 0.48 |
| Hostility | 116 | 9.39 | 4.90 | 49 | 14.63 | 6.12 | –5.34 | <0.001 | 0.95 |
| Perceived hostility | 115 | 14.80 | 4.52 | 48 | 17.02 | 5.77 | –2.22 | 0.02 | 0.43 |
| Anxiety | 117 | 12.01 | 3.86 | 49 | 18.16 | 4.30 | –6.15 | <0.001 | 1.51 |
| “I felt hopeless” | 117 | 1.26 | 0.53 | 49 | 2.45 | 0.98 | –10.03 | <0.001 | 1.51 |
a
Individuals who endorsed this item (“I felt depressed”) as a 3, 4, or 5 (sometimes, often, or always, respectively) were included in the possible depression group.
Discussion
This study illustrates that some degree of anger and aggression are reported by a sizable minority of middle-aged men in the general population. Nearly one out of five middle-aged men (17.5%) endorsed one or more of the five physical aggression questions as true (i.e., scoring a single question as ≥5). In addition, symptoms of anxiety and depression were also reported by a minority of men. Anger and hostility were correlated with depression and anxiety, meaning that all tended to co-occur. The subgroup of men who endorsed at least a mild degree of depression also endorsed much greater anger, hostility, and anxiety than men who did not endorse feelings of depression. The comorbidity of affective dysregulation in men in the general population is important to consider when conceptualizing whether former athletes, civilians, or military veterans might have CTE.
Depression
Before 2005, depression was not considered a core or defining clinical feature of CTE. CTE was considered a neurological disorder, and some neurobehavioral characteristics described in former boxers included child-like behavior and volatility. Severe psychiatric illness and severe substance abuse were described in many of the historic case studies of boxers with dementia pugilistica (5, 6, 9, 10, 37), including problems severe enough to require institutionalization. In the past few years, there has been a much greater emphasis on mental health problems, such as depression and anxiety (12–14), as a proposed core clinical feature of CTE. In many of the postmortem case studies (3), the men were reported by a loved one or collateral to have experienced depression at some point before their death; some were depressed immediately before death. The challenge for clinicians and researchers is to determine the extent to which depression is independently related to the neuropathology believed to be specific to CTE versus neuropathology due mostly or entirely to well-established endogenous factors, exogenous factors, or both.
The lifetime prevalence of depression in men born in the United States is approximately 20%−28% (38). Depression is usually conceptualized as multifactorial in causation, and it is believed to arise from the cumulative interaction (39, 40) of genetics (41), adverse events in childhood (42, 43), and ongoing life stressors (44, 45). Former athletes and military veterans may have other health problems that are associated with depression, independent of their risk of neuropathology suggestive of CTE. Numerous psychiatric, medical, and neurological conditions are associated with increased risk for depression, including chronic pain (46, 47); headaches and migraines (48, 49); chronic insomnia (50); sleep apnea (51); posttraumatic stress disorder (PTSD) (52); generalized anxiety disorder (53, 54); substance abuse (55); obesity (56); diabetes (57); hypothyroidism (58); low testosterone (59); cardiovascular, cerebrovascular, and small vessel ischemic disease (60–63); Parkinson’s disease (64); mild cognitive impairment (65); and Alzheimer’s disease (66). Therefore, the challenge of including depression as a defining clinical feature of CTE is related to specificity. There is a risk that attributing depression mostly or entirely to CTE implies that the depression is a manifestation of a progressive neurodegenerative disease, which may or may not be true in any given case. Research is needed to determine if and the extent to which depression or other relatively common psychiatric disorders (e.g., PTSD, generalized anxiety disorder, impulse control syndromes) should be considered a core clinical feature of CTE.
Anger Control Problems
Anger control problems and violence have been described in some of the historical case studies of chronic brain damage in boxers (6, 9, 10, 67). Some authors have questioned the extent to which some of these tendencies might be longstanding and more characterological (11) versus being entirely new-onset and solely neuropsychiatric in origin. Anger control problems have also been described in recent case studies of former football players who had postmortem evidence of CTE pathology (12, 14, 15, 68–71). There are very few studies of anger control problems in former contact sport athletes, such as football players and boxers. In a phone survey, 1,063 retired National Football League (NFL) players were asked if they had ever had attacks of anger in which they lost control and broke or smashed something worth more than a few dollars, hit or tried to hurt someone, or threatened to hit or hurt someone (72). A substantial percentage of retired athletes between the ages of 30 and 49 (30.7%) and those aged ≥50 (29.3%) said yes to this question. Importantly, however, the authors noted that the rate of anger dyscontrol in men in the U.S. general population for these survey questions is greater than that for the retired players (i.e., 54.8% for men between 30 and 49, and 47.2% for men ≥50 years old).
A subgroup of men in the general population have longstanding anger control problems (29). Twin studies suggest that individual differences in anger expression and coping styles can be explained in part by genetic factors, and there are substantial environmental influences (73). Spielberger’s trait anger scale suggests that anger proneness is related to two factors: anger temperament (i.e., experiencing anger without provocation) and anger reaction (e.g., experiencing anger in response to negative events) (74). There is an association between anger control difficulties and adverse childhood events (i.e., abuse), low socioeconomic status and income, and parental mental health and substance use problems (29). Individuals with clinically significant anger difficulties report higher rates of a variety of psychiatric disorders and substance use disorders, and they are more likely to report using tobacco, alcohol, and drugs than individuals without significant anger difficulties (29). There is also an association between impulsivity (as reflected by gambling) and anger (75).
It is important for clinicians and researchers to appreciate that when irritability and anger control problems emerge or clearly worsen over time, these problems might be associated with life stress, marital problems, sleep disturbance, chronic pain, chronic headaches, substance abuse, depression, a neurological problem, a neurodegenerative disease, or a combination of factors. As noted in the DSM-5, people with depression often have considerable problems with irritability (e.g., persistent anger, a tendency to respond to events with angry outbursts or blaming others, and an exaggerated sense of frustration over minor matters), and family members often notice social withdrawal. Men with depression have higher rates of anger attacks or aggression, substance abuse, and risk taking (76). In addition, people in the general population with anger control problems and PTSD are at increased risk for suicidality (77). As with depression, more research is needed to determine the extent to which anger- or other impulse-control problems are related specifically to the neuropathology of CTE.
Limitations
This study has several limitations. The data were obtained from the normative data set for the NIH Toolbox; information relating to lifetime history of participation in contact or collision sports, history of mild concussions, or prior military service was not included in the database. The subjects were evaluated in person, as part of the standardization of the Toolbox, and those who reported a history of head or brain injury were not included in this study. It is possible, perhaps even likely, that some of the men in our case series experienced one or more concussions during the course of their lives that went unreported at the time of their inclusion in this normative data set. Concussions are very common in men in the general population. It is also likely that some of them played contact sports at least briefly during their lives, because sports participation at the high school level is common. Nonetheless, this sample remains a normative one even if such unreported concussive events were experienced by some of the individuals included in this database.
Conclusions
Without question, there are reasons to be concerned about the long-term brain health and neurobehavioral functioning of athletes, civilians, and military service members who are exposed to repetitive neurotrauma. For decades, this concern has been noted for boxers. Imaging studies of retired boxers show evidence of macrostructural and microstructural changes to the brain (78–80). Studies of retired professional football players reveal both functional changes in brain metabolism (81), as measured by functional magnetic resonance imaging, and microstructural differences in white matter (82, 83), as measured by diffusion tensor imaging, in some athletes. Postmortem examinations of the brains of former professional football players have revealed diverse neurodegenerative changes (14, 84). Surveys have revealed that a substantial minority of former NFL players have mental health problems (38, 72, 85); chronic pain and opioid use is relatively common (86, 87), and those with depression and chronic pain also have greater life stress and financial difficulties (85), although the majority of former athletes appear to be functioning well in their daily lives.
Researchers and clinicians are encouraged to be cautious and circumspect when considering the clinical presentation of former athletes and military veterans; they should not uncritically assert or assume that all or nearly all symptoms and problems experienced by athletes and veterans are caused by specific neuropathology. This represents a perspective that is likely over-simplified. Many people classified as having CTE at autopsy have also shown preclinical forms of neuropathology associated with other neurodegenerative diseases, such as Alzheimer’s disease, Parkinson’s disease, and Lewy body disease (26, 88–91); many have met full neuropathological criteria for a separate disease (14) that presumably accounted for some portion of their clinical features. Moreover, in recent years CTE pathology has been identified in a small number of people with no known participation in collision or contact sports and no known exposure to multiple mild injuries to the brain, including individuals with a history of substance abuse (92), temporal lobe epilepsy (93), amyotrophic lateral sclerosis (94), multiple system atrophy (95), and other neurodegenerative diseases (96).
Former athletes and military veterans might experience mental health problems for reasons similar to people in the general population. Transitioning from being a professional athlete to a retiree in one’s 30s or 40s may present unique challenges, such as a loss of identity, worse self-esteem, changes in income, and reductions in positive health behaviors (97, 98). These challenges may be risk factors for the development of mental health problems. These transition difficulties may be more prominent when retirement occurs unexpectedly or in a forced manner (e.g., related to a career-ending injury) (99). Importantly, there are evidence-based and evidence-informed treatment options available for life stress, anxiety, depression, suicidality, chronic pain, sleep problems, headaches, substance abuse, and gambling. Providing these treatments might reduce symptoms and improve functioning in former athletes and military veterans.
Footnote
Supported in part by NIDILRR (grants, 90DP0039-03-00, 90SI5007-02-04,90 D P0060), NIH (grants, 4 U01NS086090-04; 5R24HD082302-02;5U01NS091951-03), and USARMC (grant, W81XWH-112-0210) to Dr. Zafonte.
References
1.
Montenigro PH, Baugh CM, Daneshvar DH, et al: Clinical subtypes of chronic traumatic encephalopathy: literature review and proposed research diagnostic criteria for traumatic encephalopathy syndrome. Alzheimers Res Ther 2014; 6:68
2.
Reams N, Eckner JT, Almeida AA, et al: A clinical approach to the diagnosis of traumatic encephalopathy syndrome: a review. JAMA Neurol 2016; 73:743–749
3.
Victoroff J: Traumatic encephalopathy: review and provisional research diagnostic criteria. NeuroRehabilitation 2013; 32:211–224
4.
Jordan BD: The clinical spectrum of sport-related traumatic brain injury. Nat Rev Neurol 2013; 9:222–230
5.
Martland HS: Punch drunk. J Am Med Assoc 1928; 91:1103–1107
6.
Roberts A: Brain Damage in Boxers: A Study of Prevalence of Traumatic Encephalopathy Among Ex-Professional Boxers. London, Pitman Medical Scientific Publishing Co., 1969
7.
Neubuerger KT, Sinton DW, Denst J: Cerebral atrophy associated with boxing. AMA Arch Neurol Psychiatry 1959; 81:403–408
8.
Mawdsley C, Ferguson FR: Neurological disease in boxers. Lancet 1963; 2:795–801
9.
Payne EE: Brains of boxers. Neurochirurgia (Stuttg) 1968; 11:173–188
10.
Corsellis JA, Bruton CJ, Freeman-Browne D: The aftermath of boxing. Psychol Med 1973; 3:270–303
11.
Jordan BD: Chronic traumatic brain injury associated with boxing. Semin Neurol 2000; 20:179–185
12.
Omalu B, Bailes J, Hamilton RL, et al: Emerging histomorphologic phenotypes of chronic traumatic encephalopathy in American athletes. Neurosurgery 2011; 69:173–183, discussion 183
13.
Baugh CM, Stamm JM, Riley DO, et al: Chronic traumatic encephalopathy: neurodegeneration following repetitive concussive and subconcussive brain trauma. Brain Imaging Behav 2012; 6:244–254
14.
McKee AC, Stern RA, Nowinski CJ, et al: The spectrum of disease in chronic traumatic encephalopathy. Brain 2013; 136:43–64
15.
Stern RA, Daneshvar DH, Baugh CM, et al: Clinical presentation of chronic traumatic encephalopathy. Neurology 2013; 81:1122–1129
16.
Gavett BE, Stern RA, McKee AC: Chronic traumatic encephalopathy: a potential late effect of sport-related concussive and subconcussive head trauma. Clin Sports Med 2011; 30:179–188, xi [xi]
17.
Stern RA, Riley DO, Daneshvar DH, et al: Long-term consequences of repetitive brain trauma: chronic traumatic encephalopathy. PM R 2011; 3(Suppl 2):S460–S467
18.
Omalu BI, Bailes J, Hammers JL, et al: Chronic traumatic encephalopathy, suicides and parasuicides in professional American athletes: the role of the forensic pathologist. Am J Forensic Med Pathol 2010; 31:130–132
19.
Omalu B: Chronic traumatic encephalopathy. Prog Neurol Surg 2014; 28:38–49
20.
Iverson GL: Suicide and chronic traumatic encephalopathy. J Neuropsychiatry Clin Neurosci 2016; 28:9–16
21.
Wortzel HS, Shura RD, Brenner LA: Chronic traumatic encephalopathy and suicide: a systematic review. BioMed Res Int 2013; 2013:424280
22.
Iverson GL: Chronic traumatic encephalopathy and risk of suicide in former athletes. Br J Sports Med 2014; 48:162–165
23.
Maroon JC, Winkelman R, Bost J, et al: Chronic traumatic encephalopathy in contact sports: a systematic review of all reported pathological cases. PLoS One 2015; 10:e0117338
24.
Webner D, Iverson GL: Suicide in professional American football players in the past 95 years. Brain Inj 2016; 30:1718–1721
25.
Lehman EJ, Hein MJ, Gersic CM: Suicide mortality among retired National Football League players who played 5 or more seasons. Am J Sports Med 2016; 44:2486–2491
26.
McKee AC, Cantu RC, Nowinski CJ, et al: Chronic traumatic encephalopathy in athletes: progressive tauopathy after repetitive head injury. J Neuropathol Exp Neurol 2009; 68:709–735
27.
Kessler RC, Bromet EJ: The epidemiology of depression across cultures. Annu Rev Public Health 2013; 34:119–138
28.
Bandelow B, Michaelis S: Epidemiology of anxiety disorders in the 21st century. Dialogues Clin Neurosci 2015; 17:327–335
29.
Okuda M, Picazo J, Olfson M, et al: Prevalence and correlates of anger in the community: results from a national survey. CNS Spectr 2015; 20:130–139
30.
American Psychological Association: Stress in America: Paying With Our Health. Washington, DC, American Psychological Association, 2015
31.
Brown SL, Lin IF: The gray divorce revolution: rising divorce among middle-aged and older adults, 1990-2010. J Gerontol B Psychol Sci Soc Sci 2012; 67:731–741
32.
Smitherman TA, Burch R, Sheikh H, et al: The prevalence, impact, and treatment of migraine and severe headaches in the United States: a review of statistics from national surveillance studies. Headache 2013; 53:427–436
33.
Johannes CB, Le TK, Zhou X, et al: The prevalence of chronic pain in United States adults: results of an Internet-based survey. J Pain 2010; 11:1230–1239
34.
Ohayon MM: Epidemiology of insomnia: what we know and what we still need to learn. Sleep Med Rev 2002; 6:97–111
35.
Gershon RC, Cella D, Fox NA, et al: Assessment of neurological and behavioural function: the NIH Toolbox. Lancet Neurol 2010; 9:138–139
36.
Gershon RC, Wagster MV, Hendrie HC, et al: NIH Toolbox for Assessment of Neurological and Behavioral Function. Neurology 2013; 80(Suppl 3):S2–S6
37.
Critchley M: Medical aspects of boxing, particularly from a neurological standpoint. BMJ 1957; 1:357–362
38.
González HM, Tarraf W, Whitfield KE, et al: The epidemiology of major depression and ethnicity in the United States. J Psychiatr Res 2010; 44:1043–1051
39.
Kendler KS, Thornton LM, Gardner CO: Genetic risk, number of previous depressive episodes, and stressful life events in predicting onset of major depression. Am J Psychiatry 2001; 158:582–586
40.
Monroe SM, Harkness KL: Life stress, the “kindling” hypothesis, and the recurrence of depression: considerations from a life stress perspective. Psychol Rev 2005; 112:417–445
41.
Sullivan PF, Neale MC, Kendler KS: Genetic epidemiology of major depression: review and meta-analysis. Am J Psychiatry 2000; 157:1552–1562
42.
Heim C, Newport DJ, Mletzko T, et al: The link between childhood trauma and depression: insights from HPA axis studies in humans. Psychoneuroendocrinology 2008; 33:693–710
43.
Gatt JM, Nemeroff CB, Dobson-Stone C, et al: Interactions between BDNF Val66Met polymorphism and early life stress predict brain and arousal pathways to syndromal depression and anxiety. Mol Psychiatry 2009; 14:681–695
44.
Friis RH, Wittchen HU, Pfister H, et al: Life events and changes in the course of depression in young adults. Eur Psychiatry 2002; 17:241–253
45.
Kendler KS, Karkowski LM, Prescott CA: Causal relationship between stressful life events and the onset of major depression. Am J Psychiatry 1999; 156:837–841
46.
Campbell LC, Clauw DJ, Keefe FJ: Persistent pain and depression: a biopsychosocial perspective. Biol Psychiatry 2003; 54:399–409
47.
McWilliams LA, Cox BJ, Enns MW: Mood and anxiety disorders associated with chronic pain: an examination in a nationally representative sample. Pain 2003; 106:127–133
48.
Hung CI, Liu CY, Fuh JL, et al: Comorbid migraine is associated with a negative impact on quality of life in patients with major depression. Cephalalgia 2006; 26:26–32
49.
Breslau N, Lipton RB, Stewart WF, et al: Comorbidity of migraine and depression: investigating potential etiology and prognosis. Neurology 2003; 60:1308–1312
50.
Ohayon MM, Lemoine P: Répercussions diurnes de l’insomnie dans la population générale française [Daytime consequences of insomnia complaints in the French general population]. Encephale 2004; 30:222–227
51.
Garbarino S, Bardwell WA, Guglielmi O, et al: Association of anxiety and depression in obstructive sleep apnea patients: a systematic review and meta-analysis. Behav Sleep Med (Epub ahead of print, Nov 19, 2018)
52.
Goldstein RB, Smith SM, Chou SP, et al: The epidemiology of DSM-5 posttraumatic stress disorder in the United States: results from the National Epidemiologic Survey on Alcohol and Related Conditions-III. Soc Psychiatry Psychiatr Epidemiol 2016; 51:1137–1148
53.
Judd LL, Kessler RC, Paulus MP, et al: Comorbidity as a fundamental feature of generalized anxiety disorders: results from the National Comorbidity Study (NCS). Acta Psychiatr Scand Suppl 1998; 393:6–11
54.
Carter RM, Wittchen HU, Pfister H, et al: One-year prevalence of subthreshold and threshold DSM-IV generalized anxiety disorder in a nationally representative sample. Depress Anxiety 2001; 13:78–88
55.
Grant BF, Saha TD, Ruan WJ, et al: Epidemiology of DSM-5 drug use disorder: results from the National Epidemiologic Survey on Alcohol and Related Conditions-III. JAMA Psychiatry 2016; 73:39–47
56.
Luppino FS, de Wit LM, Bouvy PF, et al: Overweight, obesity, and depression: a systematic review and meta-analysis of longitudinal studies. Arch Gen Psychiatry 2010; 67:220–229
57.
Anderson RJ, Freedland KE, Clouse RE, et al: The prevalence of comorbid depression in adults with diabetes: a meta-analysis. Diabetes Care 2001; 24:1069–1078
58.
Medici M, Direk N, Visser WE, et al: Thyroid function within the normal range and the risk of depression: a population-based cohort study. J Clin Endocrinol Metab 2014; 99:1213–1219
59.
Zarrouf FA, Artz S, Griffith J, et al: Testosterone and depression: systematic review and meta-analysis. J Psychiatr Pract 2009; 15:289–305
60.
Ravven S, Bader C, Azar A, et al: Depressive symptoms after CABG surgery: a meta-analysis. Harv Rev Psychiatry 2013; 21:59–69
61.
Ayerbe L, Ayis S, Wolfe CD, et al: Natural history, predictors and outcomes of depression after stroke: systematic review and meta-analysis. Br J Psychiatry 2013; 202:14–21
62.
Meng L, Chen D, Yang Y, et al: Depression increases the risk of hypertension incidence: a meta-analysis of prospective cohort studies. J Hypertens 2012; 30:842–851
63.
Göthe F, Enache D, Wahlund LO, et al: Cerebrovascular diseases and depression: epidemiology, mechanisms and treatment. Panminerva Med 2012; 54:161–170
64.
Djamshidian A, Friedman JH: Anxiety and depression in Parkinson’s disease. Curr Treat Options Neurol 2014; 16:285
65.
Barnes DE, Alexopoulos GS, Lopez OL, et al: Depressive symptoms, vascular disease, and mild cognitive impairment: findings from the Cardiovascular Health Study. Arch Gen Psychiatry 2006; 63:273–279
66.
Raskind MA: Diagnosis and treatment of depression comorbid with neurologic disorders. Am J Med 2008; 121(Suppl 2):S28–S37
67.
Mendez MF: The neuropsychiatric aspects of boxing. Int J Psychiatry Med 1995; 25:249–262
68.
Omalu BI, DeKosky ST, Hamilton RL, et al: Chronic traumatic encephalopathy in a National Football League player: part II. Neurosurgery 2006; 59:1086–1092
69.
Omalu BI, Hamilton RL, Kamboh MI, et al: Chronic traumatic encephalopathy (CTE) in a National Football League player: Case report and emerging medicolegal practice questions. J Forensic Nurs 2010; 6:40–46
70.
Mez J, Solomon TM, Daneshvar DH, et al: Pathologically confirmed chronic traumatic encephalopathy in a 25-year-old former college football player. JAMA Neurol 2016; 73:353–355
71.
Gardner RC, Possin KL, Hess CP, et al: Evaluating and treating neurobehavioral symptoms in professional American football players: Lessons from a case series. Neurol Clin Pract 2015; 5:285–295
72.
Weir DR, Jackson JS, Sonnega A: National Football League Player Care Foundation: Study of Retired NFL Players. Ann Arbor, Mich, University of Michigan, Institute for Social Research, 2009
73.
Wang X, Trivedi R, Treiber F, et al: Genetic and environmental influences on anger expression, John Henryism, and stressful life events: the Georgia Cardiovascular Twin Study. Psychosom Med 2005; 67:16–23
74.
Spielberger CD: STAXI-2: State–Trait Anger Expression Inventory—2, Professional Manual. Odessa, Fla, Psychological Assessment Resources, 1999
75.
Yavuz S, Aydin NE, Celik O, et al: Resveratrol successfully treats experimental endometriosis through modulation of oxidative stress and lipid peroxidation. J Cancer Res Ther 2014; 10:324–329
76.
Martin LA, Neighbors HW, Griffith DM: The experience of symptoms of depression in men vs women: analysis of the National Comorbidity Survey Replication. JAMA Psychiatry 2013; 70:1100–1106
77.
Fanning JR, Lee R, Coccaro EF: Comorbid intermittent explosive disorder and posttraumatic stress disorder: Clinical correlates and relationship to suicidal behavior. Compr Psychiatry 2016; 70:125–133
78.
Orrison WW, Hanson EH, Alamo T, et al: Traumatic brain injury: a review and high-field MRI findings in 100 unarmed combatants using a literature-based checklist approach. J Neurotrauma 2009; 26:689–701
79.
Chappell MH, Uluğ AM, Zhang L, et al: Distribution of microstructural damage in the brains of professional boxers: a diffusion MRI study. J Magn Reson Imaging 2006; 24:537–542
80.
Jordan BD, Jahre C, Hauser WA, et al: CT of 338 active professional boxers. Radiology 1992; 185:509–512
81.
Hampshire A, MacDonald A, Owen AM: Hypoconnectivity and hyperfrontality in retired American football players. Sci Rep 2013; 3:2972
82.
Strain J, Didehbani N, Cullum CM, et al: Depressive symptoms and white matter dysfunction in retired NFL players with concussion history. Neurology 2013; 81:25–32
83.
Hart J Jr, Kraut MA, Womack KB, et al: Neuroimaging of cognitive dysfunction and depression in aging retired National Football League players: a cross-sectional study. JAMA Neurol 2013; 70:326–335
84.
Mez J, Daneshvar DH, Kiernan PT, et al: Clinicopathological evaluation of chronic traumatic encephalopathy in players of American football. JAMA 2017; 318:360–370
85.
Schwenk TL, Gorenflo DW, Dopp RR, et al: Depression and pain in retired professional football players. Med Sci Sports Exerc 2007; 39:599–605
86.
Cottler LB, Ben Abdallah A, Cummings SM, et al: Injury, pain, and prescription opioid use among former National Football League (NFL) players. Drug Alcohol Depend 2011; 116:188–194
87.
Dunne EM, Striley CW, Mannes ZL, et al: Reasons for prescription opioid use while playing in the National Football League as risk factors for current use and misuse among former players. Clin J Sport Med (Epub ahead of print, June 28, 2018)
88.
Hazrati LN, Tartaglia MC, Diamandis P, et al: Absence of chronic traumatic encephalopathy in retired football players with multiple concussions and neurological symptomatology. Front Hum Neurosci 2013; 7:222
89.
Ling H, Kara E, Revesz T, et al: Concomitant progressive supranuclear palsy and chronic traumatic encephalopathy in a boxer. Acta Neuropathol Commun 2014; 2:24
90.
McKee AC, Gavett BE, Stern RA, et al: TDP-43 proteinopathy and motor neuron disease in chronic traumatic encephalopathy. J Neuropathol Exp Neurol 2010; 69:918–929
91.
Ling H, Morris HR, Neal JW, et al: Mixed pathologies including chronic traumatic encephalopathy account for dementia in retired association football (soccer) players. Acta Neuropathol 2017; 133:337–352
92.
Noy S, Krawitz S, Del Bigio MR: Chronic traumatic encephalopathy-like abnormalities in a routine neuropathology service. J Neuropathol Exp Neurol 2016; 75:1145–1154
93.
Puvenna V, Engeler M, Banjara M, et al: Is phosphorylated tau unique to chronic traumatic encephalopathy? Phosphorylated tau in epileptic brain and chronic traumatic encephalopathy. Brain Res 2016; 1630:225–240
94.
Fournier CN, Gearing M, Upadhyayula SR, et al: Head injury does not alter disease progression or neuropathologic outcomes in ALS. Neurology 2015; 84:1788–1795
95.
Koga S, Dickson DW, Bieniek KF: Chronic traumatic encephalopathy pathology in multiple system atrophy. J Neuropathol Exp Neurol 2016; 75:963–970
96.
Ling H, Holton JL, Shaw K, et al: Histological evidence of chronic traumatic encephalopathy in a large series of neurodegenerative diseases. Acta Neuropathol 2015; 130:891–893
97.
Sanders G, Stevinson C: Associations between retirement reasons, chronic pain, athletic identity, and depressive symptoms among former professional footballers. Eur J Sport Sci 2017; 17:1311–1318
98.
Gouttebarge V, Frings-Dresen MH, Sluiter JK: Mental and psychosocial health among current and former professional footballers. Occup Med (Lond) 2015; 65:190–196
99.
Brown JC, Kerkhoffs G, Lambert MI, et al: Forced retirement from Professional Rugby Union is associated with symptoms of Distress. Int J Sports Med 2017; 38:582–587
Information & Authors
Information
Published In
History
Received: 21 November 2018
Revision received: 3 January 2019
Revision received: 6 January 2019
Accepted: 6 January 2019
Published online: 25 April 2019
Published in print: Fall 2019
Keywords
Authors
Funding Information
The authors report no financial relationships with commercial interests.
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
There are no citations for this item
View Options
View options
PDF/ePub
View PDF/ePubGet Access
Login options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).