Site maintenance Wednesday, November 13th, 2024. Please note that access to some content and account information will be unavailable on this date.
Skip to main content
Full access
Priority Data Letter
Published Online: 1 May 2022

Lack of Representation in Psychiatric Research: A Data-Driven Example From Scientific Articles Published in 2019 and 2020 in the American Journal of Psychiatry

Publication: American Journal of Psychiatry

Abstract

Objective:

The authors examined representation and accuracy of descriptions of sociodemographic identities in psychiatric research through quantifying data contained in recently published articles from a high-impact psychiatry journal.

Methods:

Sociodemographic data were aggregated from articles (i.e., studies that provide information on individual samples) published in the American Journal of Psychiatry in 2019 and 2020 (N=125). Articles were coded by two raters for sociodemographic data, acknowledgment of lack of representation as a limitation, and focus on health disparities or inequities.

Results:

While 90% of studies provided the age of participants and 84% provided information about the sex/gender of participants, only 43% presented information about the racial or ethnicity identities of participants. One study reported the sexual identity of participants. Lack of representation relative to 2019 U.S. Census data was found for multiple racial groups, Latino/Hispanic individuals, and women (genetic studies only). Only 25% of studies acknowledged lack of representation as a limitation, and two studies focused on health disparities or inequities.

Conclusions:

These findings highlight a need to increase representation in psychiatric research and improve accuracy of language when describing the sociodemographic characteristics of participants.
Across assigned sex at birth and racial, ethnic, sexual (e.g., bisexual, gay, lesbian, queer), and gender (i.e., one’s internal sense of their gender) identities, there are important differences in prevalence and determinants of psychiatric illness, access to psychiatric care, and outcomes (e.g., 14). However, a growing body of research has shown a lack of representation and missing sociodemographic information regarding racial and ethnic identities in psychiatric research (59). While less research has examined representation of people with minoritized (i.e., experiencing discrimination, prejudice, and reduced power because of historical and current social hierarchies) sexual or gender identities, one review showed a near-absolute lack of description of these identities (10).
While previous reviews convey a clear message that lack of representation has been a persistent issue in psychiatric research, they have focused on specific disorders (e.g., alcohol use disorder) or on specific subpopulations (e.g., women), or have spanned decades. Research spanning psychiatric illnesses, examining multiple sociodemographic characteristics, and focused on recently published science will substantively add information about the state of the field today and provide a benchmark to track progress.
In this study, we analyzed published scientific articles in the American Journal of Psychiatry in 2019 and 2020 (capturing articles accepted for publication prior to the murder of George Floyd and subsequent calls for increased equity in science) to characterize representation across racial, ethnic, gender, and sexual identities. Inaccurate labels and categories, acknowledgment of representation as a limitation, and health inequities or disparities as a focus of the article were also assessed.

Methods

Given the lack of description of sociodemographic data of individual (human) samples, meta-analyses, animal studies, commentaries, editorials, reviews, and letters were excluded, resulting in the identification of 125 scientific articles. Two independent raters (S.L.P., P.P., C.E.V., and K.L.) reviewed all components of each article and supplementary material for information on sociodemographic variables (age, assigned sex at birth, and racial, ethnic, sexual, and gender identities) and description of these questions. Raters also searched the entire article for words (e.g., discrimination, disparities, inequities, race, sexual orientation, generalizability, representation) and topics (e.g., depression in Black adults) related to examination of health inequities and inclusion of lack of generalizability or representation as a limitation. The infrequent discrepancies (e.g., raters disagreed 2.4% [3/125] of the time about whether the article examined health inequity, and 19% [24/125] of the time about representation/generalizability being mentioned) were resolved by a third rater.
Raters coded the type (e.g., genetic, neuroimaging) and the location (e.g., United States, United Kingdom) of the study. Data are described separately for genetic and nongenetic studies, given previous research showing that genetic studies almost exclusively include participants of European ancestry (>78%) (11) and separately for studies conducted within and outside of the United States to allow for comparison with 2019 U.S. Census data population estimates (12). Studies conducted within the United States were assessed for over- or underrepresentation of participants with specific sociodemographic characteristics by comparing to local U.S. Census estimates for articles where a specific location or region was provided (N=19 for race/ethnicity; N=25 for sex/gender). Articles for which a location could not be determined were tested against national U.S. Census estimates. Z-tests of the difference of proportions using the prop.test function in R were run to assess whether the probability of the numbers of female individuals/women and people from specific racial and ethnic groups were higher or lower in the published articles compared to the U.S. Census data (13, 14). The z-tests provide a 95% confidence interval and chi-square with a p value.

Results

Representation Across Racial Groups

While 90% (113/125) of studies reported the age of their participants, only 43% (54/125) provided information about race or ethnicity. Across these 54 studies (Table 1), 94.6% of participants were described as White or Caucasian. When only nongenetic studies were examined (N=39), 71.4% of participants were described as White or Caucasian; the second most prevalent group was described as “other” or “nonwhite” (10.6%), followed by African American, Black, or African (7.6%), and Hispanic or Latino (5.3%). The remaining racial groups comprised <1% of the participants. Table 2 shows the summary results of over- and underrepresentation across race and ethnicity when each study is compared to local or national U.S. Census estimates. Asian/Asian American and Hispanic/Latino participants were underrepresented in 78% of articles. Biracial/multiracial participants were underrepresented in 49% of articles. Black and White participants were underrepresented in 29% and 37% of the articles, respectively.
TABLE 1. Sample sizes in all studies
Measure or Group All StudiesU.S. OnlyAll StudiesU.S. OnlyAll StudiesU.S. OnlyAll StudiesU.S. OnlyAll StudiesU.S. OnlyAll StudiesU.S. Only
  Full sample N across all articlesNumber of articlesNongenetic N across all articlesNumber of articlesGenetic N across all articlesNumber of articles
Total sample size 18,939,676474,9781256215,099,944162,73992493,839,731312,2393313
Total sample size of articles that provided information on sex/gender15,711,826358,7181055714,953,76646,4798044758,060312,2392213
Total sample size of articles that provided information on race or ethnicity1,172,244250,144544148,83445,00539321,123,410205,139159
 U.S. Census dataa (%)Full sample N across all articles% (out of applicable total sample size)Nongenetic N across all articles% (out of applicable total sample size)Genetic N across all articles% (out of applicable total sample size)
Female50.87,976,83099,48450.827.77,662,37222,48251.248.4314,45877,00241.524.7
Male49.27,712,711259,23149.172.37,269,10923,99448.651.6443,602235,23758.575.4
Total100.015,689,541358,71599.9100.014,931,48146,47699.899.9758,060312,239100.0100.0
American Indian or Alaska Native/Native American0.7660.00.0660.00.0000.00.0
Asian/Asian American5.62,5542060.20.15472021.10.42,00740.20.0
Biracial/multiracial/more than one race/multiple2.552500.00.039370.10.113130.00.0
Black/African American/African12.429,78829,3062.511.73,7183,2367.67.226,07026,0702.312.7
Hispanic/Latino18.42,6022,6020.21.02,6022,6025.35.8000.00.0
Native Hawaiian or other Pacific Islander0.2110.00.0110.00.0000.00.0
Other/nonwhite0.35,4274,9130.51.95,1714,65710.610.42562560.00.1
White/Caucasian60.01,109,474211,28194.684.5347,86932,49971.472.21,074,605178,78295.787.2
Total100.01,149,904248,36598.199.346,95324,35396.196.11,102,951205,12598.299.9
a
U.S. Census data are from the 2019 American Community Survey 1-year estimates.
TABLE 2. Summary of significant overrepresentation and underrepresentation in articlesa
 FemaleAmerican Indian or Alaska Native AloneAsian AloneBlack or African American AloneHispanic or Latino (Of Any Race)Native Hawaiian or Other Pacific Islander AloneSome Other Race AloneTwo or More RacesWhite Alone
All studiesN%bN%cN%cN%cN%cN%cN%cN%cN%c
Articles overrepresenting14250000194625002049251537
Articles underrepresenting274710243278122932784102520491537
Articles with NS difference1628317692210247173790194619461127
Nongenetic studies onlyN%dN%eN%eN%eN%eN%eN%eN%eN%e
Articles overrepresenting13300000144426001650261031
Articles underrepresenting2148722237210312372390014441134
Articles with NS difference102325789288257222991165016501134
Genetic studies onlyN%fN%gN%gN%gN%gN%gN%gN%gN%g
Articles overrepresenting180000556000044400556
Articles underrepresenting64633391002229100111222667444
Articles with NS difference646667002220088933333300
a
Articles that exhibited significant overrepresentation or significant underrepresentation were significant at p<0.05.
b
Percentage out of 57 articles.
c
Percentage out of 41 articles.
d
Percentage out of 44 articles.
e
Percentage out of 32 articles
f
Percentage out of 13 articles.
g
Percentage out of nine articles.

Representation Across Sex/Gender

One hundred and five studies (84%) reported on the sex/gender of their participants. Sixty-four studies only presented the sample size for one sex/gender. In these instances, we imputed the other number by assuming the authors restricted the options to binary (male/female, man/woman) responses. Following imputation, 50.8% of the total sample of participants was described as female/women/girls. While this distribution was similar for nongenetic studies, in U.S. genetic samples, only 24.7% are described as female/women/girls (Table 1). Table 2 shows that female participants/women/girls were underrepresented in 47% of articles and overrepresented in 25% of articles.

Inaccurate or Missing Information

One study described the question used to ask participants about their racial or ethnic identity and 43% (26/60) of articles used nonexistent racial categories such as “nonwhite,” “other,” and/or “mixed.” Additionally, no study provided a description of the question asked to establish sex or gender identity, precluding our ability to know if the articles were referring to participants’ assigned sex at birth or gender identity (given this lack of clarity, we used “sex/gender” in the present study). One study presented the sexual identity of participants. Only 25% (31/125) of articles identified lack of representation or generalizability concerns based on sample composition as a study limitation.

Prevalence of Health Equity Research

Two of the 125 studies examined inequities/disparities or psychiatric illness within minoritized populations.

Discussion

We examined representation across a recent 2-year period (2019–2020) in a high-impact psychiatry journal. We found that minoritized racial and ethnic groups, especially Asian American and Latino/Hispanic participants, and female participants/women were underrepresented in these articles. Our results also showed that descriptions of sociodemographic characteristics were frequently missing, use of inaccurate labels was prevalent, and acknowledgment of lack of representation as a limitation was uncommon.
To increase diversity within research samples, investigators can connect with community-engaged researchers at their universities and read the existing literature (e.g., 15) to learn about best practices for partnering with minoritized populations. Researchers should also review the sociodemographic characteristics of participants in an ongoing fashion, like other enrollment criteria, and adjust as needed to ensure adequate representation based on local population estimates. Additionally, it is imperative to collect genetic data from ancestrally diverse populations and interpret these data in culturally informed ways. There are published recommendations to guide researchers (16).
Our results also found missing or inaccurate language about race and ethnicity in the articles. First, less than half of the articles provided the racial composition of the sample, limiting our ability to even evaluate representation. Further, within the studies that provided descriptors for racial identities, minoritized racial and ethnic groups may be being homogenized into an “other/nonwhite” category, as this group was overrepresented compared to U.S. Census data in 49% of the articles (and underrepresented in 5%). This is an example of “white centering” in research (17, 18), which assumes similarity of all minoritized racial and ethnic groups into a single group in relation to White participants. Additionally, “Caucasian,” a label with a racist history (19), was used as a racial category in 45% of articles.
Despite the Institute of Medicine stating that assessing and reporting sexual and gender identities is important (20), only one article reported the sexual identity of participants, no studies clarified if they were reporting on assigned sex or gender identity, and descriptions of samples defaulted to an assumed gender binary. Researchers should review how they are assessing sociodemographic characteristics to ensure that accurate and affirming questions are being used to collect information on sexual and gender identities (as well as racial and ethnic identities). For the 1.2 million people in the United States who have a minoritized gender identity (21), binary response options (e.g., only man or woman) are often inaccurate and hurtful. The PhenXToolkit (22) provides examples of how to assess sociodemographic characteristics.
Despite lack of representation or missing data related to sociodemographic characteristics, there is significant value in utilizing existing data for secondary data analyses. However, when interpreting results and writing manuscripts, directly acknowledging these limitations and framing conclusions accordingly is crucial. This will spur further innovative research and increase rigor and reproducibility. Authors are also called upon to be thoughtful about conducting and interpreting subgroup analyses. It is imperative to place results within a sociohistorical context that recognizes structural inequities experienced by minoritized individuals and communities and to define race, ethnicity, and gender as social constructs. Additionally, if scientists more clearly and accurately describe the sample characteristics and questions used, meta-analyses can be utilized to examine determinants of health inequities by pooling across samples that were not initially powered to examine processes within and across subpopulations.
In addition to individual investigators, scientific journals and funding agencies (e.g., the National Institutes of Health [NIH]) can foster investment in recruiting representative samples and describing sociodemographic characteristics. For example, the Journal states on its website, “The methods section should provide a comprehensive description of the nature of the study group.” However, in the articles reviewed here, this guideline was largely not followed, indicating that expanded guidelines and provision of concrete requirements for characterizing sample sociodemographic characteristics (e.g., sample description must include racial and ethnic identities, gender identities, and age) may be beneficial. Resources can also be added to journal and funding agency websites. For example, the American Psychological Association has published guidelines that are available (23, 24) for reducing bias in scientific language, and there are published recommendations on how to report racial and ethnic identities in medical and scientific journals (25). Journals and funding agencies could also include specialty reviewers to provide critiques focused on inaccurate language, interpretation of results, and representation, ensuring that conclusions are framed within this context. This will also educate scientists and improve future research.
The lack of research (<2% of articles) on health inequities/disparities in 2019 and 2020 in a top-tier psychiatry journal is another urgent issue that can be remedied by journals and funding agencies. Our findings dovetail with a recent review of over 26,000 articles published in top-tier psychology journals that found that around 5% of the articles reviewed focused on outcomes for minoritized racial groups (26). Individuals from minoritized identities are more likely to conduct health equity research, which is subject to publication bias (27), subsequently contributing to NIH funding inequities (28), further hampering the careers of scientists already facing discrimination (28). As NIH pushes to fund more health equity science and scientists call for NIH to address long-standing funding inequities (29), journals could set benchmarks (and track progress over time) for inclusion of health equity research.
While the present study utilized a large sample of 125 recently published articles, several limitations are worth noting. First, we focused on articles only from 2019 and 2020 and one journal. We chose these years because they include recent research that was likely accepted for publication prior to the murder of George Floyd, which occurred in May 2020. We examined one high-impact journal to allow for in-depth review of each article for multiple sociodemographic characteristics. This approach also allows our findings to serve as a “baseline” for the Journal to track progress over time and provides an example that other journals can follow, as these issues are not specific to this journal. We also did not know the specific location of all U.S. studies, which diminished the specificity of the representation results for these studies.
Despite these limitations, the present study significantly advances understanding about representation and scientific language in psychiatric research. Further research that integrates the identities of authors, reviewers, and editorial board members and examines change over time in comparison to this “baseline” is needed. We hope that these results and conclusions encourage action and further discussion across levels of the scientific community (e.g., researchers, community members, institutions, journals, funding agencies).

Acknowledgments

The authors acknowledge Denis McCarthy, Ph.D., for his feedback on the manuscript.

REFERENCES

1.
Maura J, Weisman de Mamani A: Mental health disparities, treatment engagement, and attrition among racial/ethnic minorities with severe mental illness: a review. J Clin Psychol Med Settings 2017; 24:187–210
2.
Mongelli F, Georgakopoulos P, Pato MT: Challenges and opportunities to meet the mental health needs of underserved and disenfranchised populations in the United States. Focus (Am Psychiatr Publ) 2020; 18:16–24
3.
Rice CE, Vasilenko SA, Fish JN, et al: Sexual minority health disparities: an examination of age-related trends across adulthood in a national cross-sectional sample. Ann Epidemiol 2019; 31:20–25
4.
Shim RS: Dismantling structural racism in psychiatry: a path to mental health equity. Am J Psychiatry 2021; 178:592–598
5.
Nilsson JE, Love KM, Taylor KJ, et al: A content and sample analysis of quantitative articles published in the Journal of Counseling and Development between 1991 and 2000. J Couns Dev 2007; 85:357–363
6.
Schick MR, Spillane NS, Hostetler KL: A call to action: a systematic review examining the failure to include females and members of minoritized racial/ethnic groups in clinical trials of pharmacological treatments for alcohol use disorder. Alcohol Clin Exp Res 2020; 44:1933–1951
7.
Geller SE, Koch AR, Roesch P, et al: The more things change, the more they stay the same: a study to evaluate compliance with inclusion and assessment of women and minorities in randomized controlled trials. Acad Med 2018; 93:630–635
8.
Phillips SP, Hamberg K: Doubly blind: a systematic review of gender in randomised controlled trials. Glob Health Action 2016; 9:29597
9.
Fain KM, Nelson JT, Tse T, et al: Race and ethnicity reporting for clinical trials in ClinicalTrials.gov and publications. Contemp Clin Trials 2020; 101:106237
10.
Heck NC, Mirabito LA, LeMaire K, et al: Omitted data in randomized controlled trials for anxiety and depression: a systematic review of the inclusion of sexual orientation and gender identity. J Consult Clin Psychol 2017; 85:72–76
11.
Sirugo G, Williams SM, Tishkoff SA: The missing diversity in human genetic studies. Cell 2019; 177:26–31
12.
United States Census Bureau: Hispanic or Latino origin by race, 2019 American Community Survey 1-year estimates detailed tables. data.census.gov/cedsci/tablle?q=ACS%202019%20race&tid=ACSDT1Y2019.B03002
13.
Thompson LA: R (and S-PLUS) Manual to Accompany Agresti’s Categorical Data Analysis (2002), 2nd ed. 2009. users.stat.ufl.edu/∼aa/cda/Thompson_manual.pdf
14.
Agresti A: An Introduction to Categorical Data Analysis, 2nd ed. Hoboken, NJ, John Wiley & Sons, 2007
15.
Wallerstein N, Duran B, Oetzel J, et al: Community-Based Participatory Research for Health. San Francisco, Jossey-Bass, 2018
16.
Peterson RE, Kuchenbaecker K, Walters RK, et al: Genome-wide association studies in ancestrally diverse populations: opportunities, methods, pitfalls, and recommendations. Cell 2019; 179:589–603
17.
Madowitz J, Boutelle KN: Ethical implications of using the term “non-white” in psychological research. Ethics Behav 2014; 24:306–310
18.
Sharpe RV: Disaggregating data by race allows for more accurate research. Nat Hum Behav 2019; 3:1240
19.
Mukhopadhyay CC: Getting rid of the word “Caucasian”; in Everyday Anti-Racism: Getting Real About Race in School. Edited by Pollock M. New York, New Press, 2008, pp. 12–16
20.
Institute of Medicine: The Health of Lesbian, Gay, Bisexual, and Transgender People: Building a Foundation for Better Understanding. Washington, DC, National Academies Press, 2011
21.
Wilson BDM, Meyer IH: Nonbinary LGBTQ Adults in the United States. Los Angeles, Williams Institute, 2021. williamsinstitute.law.ucla.edu/publications/nonbinar y-lgbtq-adults-us/
22.
Hamilton CM, Strader LC, Pratt JG, et al: The Phenx toolkit: get the most from your measures. Am J Epidemiol 2011; 174:253–260
23.
American Psychological Association: Publication Manual of the American Psychological Association. Washington, DC, American Psychological Association, 2020
24.
American Psychological Association: General Principles for Reducing Bias. Washington, DC, American Psychological Association, 2019. apastyle.apa.org/style-grammar-guidelines/bias-free-language/general-principles
25.
Flanagin A, Frey T, Christiansen SL, et al: The reporting of race and ethnicity in medical and science journals. JAMA 2021; 325:1049
26.
Roberts SO, Bareket-Shavit C, Dollins FA, et al: Racial inequality in psychological research: trends of the past and recommendations for the future. Perspect Psychol Sci 2020; 15:1295–1309
27.
Avery DR, B DK, Dumas TL, et al: Racial biases in the publication process: exploring expressions and solutions. J Manage 2021; 48:7–16
28.
Hoppe TA, Litovitz A, Willis KA, et al: Topic choice contributes to the lower rate of NIH awards to African-American/Black scientists. Sci Adv 2019; 5:eaaw7238
29.
Stevens KR, Masters KS, Imoukhuede PI, et al: Fund Black scientists. Cell 2021; 184:561–565

Information & Authors

Information

Published In

Go to American Journal of Psychiatry
Go to American Journal of Psychiatry
American Journal of Psychiatry
Pages: 388 - 392
PubMed: 35491569

History

Received: 29 July 2021
Revision received: 12 November 2021
Revision received: 16 December 2021
Accepted: 4 January 2022
Published online: 1 May 2022
Published in print: May 2022

Keywords

  1. Minority Issues and Cross-Cultural Psychiatry
  2. Lesbian/Gay/Bisexual/Transgender (LGBT) Issues
  3. Diversity
  4. Health Equity
  5. Research Design and Methods
  6. Sociopolitical Issues

Authors

Details

Sarah L. Pedersen, Ph.D. [email protected]
Department of Psychiatry, University of Pittsburgh, Pittsburgh.
Rachel Lindstrom, Ph.D.
Department of Psychiatry, University of Pittsburgh, Pittsburgh.
Paula M. Powe, M.D.
Department of Psychiatry, University of Pittsburgh, Pittsburgh.
Kelly Louie, B.S.
Department of Psychiatry, University of Pittsburgh, Pittsburgh.
César Escobar-Viera, M.D., Ph.D.
Department of Psychiatry, University of Pittsburgh, Pittsburgh.

Notes

Send correspondence to Dr. Pedersen ([email protected]).

Competing Interests

The authors report no financial relationships with commercial interests.

Metrics & Citations

Metrics

Citations

Export Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

For more information or tips please see 'Downloading to a citation manager' in the Help menu.

Format
Citation style
Style
Copy to clipboard

View Options

View options

PDF/EPUB

View PDF/EPUB

Get Access

Login options

Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.

Personal login Institutional Login Open Athens login

Not a subscriber?

Subscribe Now / Learn More

PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.

Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).

Media

Figures

Other

Tables

Share

Share

Share article link

Share