Evidence for Differential Predictive Performance of the Prime Screen Between Black and White Help-Seeking Youths
Abstract
Objective:
Self-report screening instruments for emerging psychosis have the potential to improve early detection efforts by increasing the number of true positives among persons deemed to be at “clinical high risk” of the disorder, but their practical utility depends on their validity across race. This study sought to examine whether a commonly used self-report screening tool for psychosis risk performed equally among black and white youths in its ability to predict clinical high-risk status.
Methods:
Black (N=58) and white (N=50) help-seeking individuals ages 12–25 (61% female) were assessed with the Prime Screen and the Structured Interview for Psychosis-Risk Syndromes (SIPS). A logistic regression model estimated race differences in the strength of the relation between Prime Screen scores and SIPS-defined risk status.
Results:
Higher Prime Screen scores significantly predicted clinical high-risk status among white (p<.01) but not black participants. Among black youths without clinical high risk, self-reported Prime Screen scores more closely resembled scores for youths (black or white) with clinical high risk than scores of white peers who were also without clinical high risk.
Conclusions:
Results suggest that consideration of race or ethnicity and associated cultural factors is important when screening for clinical high-risk status. Findings support the need to develop culturally valid early psychosis screening tools to promote appropriately tailored early intervention efforts.
HIGHLIGHTS
•
The Prime Screen self-report measure of psychosis-risk syndromes significantly predicted clinician-established risk status for white participants.
•
The Prime Screen did not significantly predict clinician-established psychosis risk status for black participants.
•
Consideration of individual participant characteristics is important when considering results from screening tools designed to detect psychosis risk.
•
Intervention efforts for early psychosis will be augmented by the development of culturally valid psychosis-risk screening tools.
Individuals at “clinical high risk” for psychosis are those experiencing recent attenuated psychotic syndromes or other indicators of susceptibility during adolescence or young adulthood, a key period of risk for first-episode psychosis (1). As only 25% of these individuals develop a formal psychotic illness in the years after identification (2), false-positive identification of psychosis risk syndromes limits the capacity of psychosis prevention efforts (3). Evidence suggests that recent trends toward drawing putatively high-risk research participants from the general, non–help-seeking population contributes to the high rates of false positives (4). In conjunction with the developing consensus that high-risk syndromes warrant clinical attention regardless of eventual psychosis (due to frequently high levels of distress and impairment; 5), these findings raise questions about the most appropriate ways to identify individuals on a path toward worsening prognosis. The use of brief, self-report screening instruments prior to clinical assessment referral may contribute to an efficient and cost-effective solution to this problem (6). Self-report screens can indicate one’s probability of meeting high-risk criteria once fully assessed (7) and have strong validity in the prediction of subsequent psychosis (8).
Given that normative experiences and item interpretation can vary as a function of factors related to race, ethnicity, and culture, validation of instruments designed to capture psychological and behavioral abnormalities requires close examination of an instrument’s performance across different racial and cultural backgrounds (9). Historically, many psychometric instruments lack sensitivity to important cultural factors (9–14), suggesting that the validity of psychosis risk instruments may differ between members of racial-ethnic majority and minority populations (10, 15). This problem can contribute to sociodemographic health disparities by limiting the benefits of screening, including early intervention, for members of racial-ethnic minority groups (16, 17).
Decades of research demonstrate that black individuals are more likely than white individuals to be misdiagnosed as having schizophrenia (18–20), further compounding what may be an actual underlying disparity in prevalence rates (21, 22) and quality of treatment (23). Given the importance of early detection and intervention in curbing the burden of serious psychopathology (24, 25), these findings highlight the need to develop screening tools that can signal emerging psychosis among black, help-seeking youths, who may be at risk of both an eventual misdiagnosis of schizophrenia and—paradoxically—the onset of an actual (not misdiagnosed) psychotic disorder.
This study aimed to determine whether the Prime Screen, a commonly used self-report prescreen for clinical high-risk criteria, performs equivalently across black and white help-seeking youths. Building from literature demonstrating limited cultural sensitivity of many psychometric instruments (9, 10, 14) and extending previous work suggesting a strong predictive relation between Prime Screen scores and clinical high-risk status, we examined whether the relation between the Prime Screen and clinical high-risk status was weaker among black participants relative to white participants. To address the possibility that differences in self-rated symptoms could be explained by group differences in levels of clinician-rated psychopathology, clinician bias, or disparities in socioeconomic status (26), we examined rates of high-risk diagnoses across racial groups, the magnitude of relations between Prime Screen scores and clinician-rated positive symptom severity, and whether family income accounted for any differential relation between Prime Screen scores and risk status. In exploratory analyses, we examined the specific Prime Screen items that may contribute to any observed racial differences.
Methods
Procedures
The study took place within the context of an ongoing longitudinal study of psychosis risk that began in 2010. Participants or their parents (if the child was younger than 18) spoke by phone with a trained researcher, who described study procedures and determined eligibility. Visits took place in a private room within university clinics. After providing informed consent, youths completed self-report measures alone while their parents (when present) were interviewed regarding the youths’ psychiatric history. Subsequently, participants completed psychiatric interviews with the researcher. The study was approved by the universities’ institutional review boards.
Participants
Individuals ages 12 to 25 were recruited from community clinics, hospitals, schools, and private practitioners in Baltimore. Additional inclusion criteria required only that participants were receiving mental health services at the time of enrollment. Participants were typically referred for mental health assessment and diagnosis because of suspected emerging psychosis or other psychiatric concerns (e.g., affective disorders). The participants could be divided into three categories: individuals at clinical high risk, a help-seeking control group made up of individuals with active mental health concerns but without clinical high risk or a diagnosable psychotic disorder (such as schizophrenia), and individuals with a diagnosable psychotic disorder. Given interest in the performance of the Prime Screen in predicting psychosis risk among black relative to white youths, participants with psychosis (N=26) and participants who were neither black nor white (N=27) were excluded from analyses.
Measures
Race.
Race was reported by participants or their parents by using a questionnaire concerning demographic characteristics derived from the National Institutes of Health’s definitions for racial and ethnic categories. The response item corresponding to black race was “Black or African American. A person having origins in any of the black racial groups of Africa. Terms such as ‘Haitian’ or ‘Negro’ can be used in addition to ‘black or African American.’” The item corresponding to white race was “White. A person having origins in any of the original peoples of Europe, the Middle East, or North Africa.” Racial subgroups (e.g., groups corresponding to specific African or European descent) were not identified.
Structured Interview for Psychosis-Risk Syndromes (SIPS).
The SIPS is a semistructured, gold-standard interview for identifying and rating the severity of clinical high-risk syndromes (27). Although no study to our knowledge has directly evaluated the cross-cultural performance of the SIPS, a recent comprehensive review of its reliability and validity across the 31 countries in which it has been used found no evidence of differential performance by culture (28). To meet criteria for a psychosis risk syndrome, participants must have experienced one or more attenuated positive psychotic symptoms at least weekly, an illness episode of psychotic-level intensity that was too brief to meet criteria for formal psychosis, or a recent functional decline of 30% or more in the context of schizotypal personality disorder (SPD) or a family history of psychosis. Given evidence that the degree of risk for transition to psychosis is comparable among adolescents with SPD and those meeting other SIPS criteria (e.g., 21% [29]), we included individuals with SPD but no family history or significant functional decline in the high-risk group. The SIPS items for unusual thought content, suspiciousness, grandiosity, perceptual abnormalities, and disorganized communication are rated on a 7-point scale, from 0 (absent) to 6 (severe), and ratings are summed to create a measure of positive symptom severity. All SIPS raters (nine white, one Asian) were certified following an official 2-day workshop and achieved excellent interrater agreement (intraclass correlation coefficient=.82 for positive symptoms; κ=1 for diagnosis). Raters were blind to participants’ Prime Screen scores.
Kiddie Schedule for Affective Disorders and Schizophrenia (K-SADS).
We used the K-SADS to characterize the sample clinically. The K-SADS is a well-validated, semistructured diagnostic interview used to identify DSM diagnoses among youths (30). K-SADS diagnoses are made on the basis of separate interviews with children and parents. Training included expert instruction, rating of audio-recordings, in vivo interview observation and co-rating, and supervised administration until diagnostic agreement with experienced raters was achieved for at least three participants and approval was given by the principal investigators.
The Prime Screen, revised.
The Prime Screen is a 12-item, self-report questionnaire developed by the SIPS authors as a brief way to estimate the probability of meeting clinical high-risk criteria (31). Items are rated on a 7-point scale, with 0 indicating definitely disagree; 1, somewhat disagree; 2, slightly disagree; 3, not sure; 4, slightly agree; 5, somewhat agree; and 6, definitely agree. Participants who endorse two or more items at the level of a 5 or 6 are considered to screen positive. The sum of positive symptoms is strongly correlated with the sum of SIPS-rated positive symptoms (7) and has been shown to predict subsequent transition to psychosis among those at clinical high risk (8). The average administration time of the instrument is 1 minute, 40 seconds, and the Flesch-Kincaid reading-level estimate is 6:8. Cronbach’s alpha for the present sample was .89.
Statistical Analyses
Preliminary analyses.
Prime Screen scores were computed by totaling the number of items endorsed at the level of 5 or 6 (hereafter referred to as Prime Screen “cutoff” scores), consistent with author recommendations and with previous validity studies (7). Primary analyses were also conducted by using the sum of raw Prime Screen scores; because the results were the same regardless of the scoring method, for simplicity the results obtained using raw scores are not reported here but are available upon request. Black and white participants were compared on demographic variables by using chi-square or t tests.
Primary analyses.
To examine whether the relation between Prime Screen scores and clinical high-risk status was weaker among black relative to white participants, a moderated logistic regression was performed in which dichotomous risk status was regressed on race, Prime Screen cutoff scores, and their cross product (race × Prime Screen cutoff score). In a second linear regression, the sum of SIPS positive symptoms was regressed on these same predictors. In the case of moderation, simple effects were computed to examine the effect of Prime Screen cutoff scores on the outcome variable (i.e., probability of meeting high-risk criteria, SIPS positive symptoms) separately among black and white participants (32). Regression analyses were then repeated controlling for family income and the demographic and clinical variables that differed significantly by race.
Exploratory analyses.
To explore whether specific Prime Screen items were differentially related to clinical high-risk status among black versus white participants, a 2×2 (race × risk status) analysis of covariance (ANCOVA) was conducted for each of the 12 raw Prime Screen item scores. ANCOVAs examined mean differences in scores for each item across risk groups within each race. Because of the exploratory nature of these analyses, no correction for multiple comparisons was applied. Finally, we computed sensitivity and specificity values for the entire sample and for the black and white groups separately.
Results
A total of 108 participants (clinical high-risk group, N=43; help-seeking control group, N=65) were included in the analyses, similar in size to several other psychosis-risk screening studies (6). Of the participants, 58 were black and 50 were white (Table 1). Because of incomplete research procedures, Prime Screen scores were missing for eight participants, the sum of SIPS positive symptoms scores was missing for 2, and family income data were missing for 11. Data were excluded pairwise per analysis. The continuous variables of interest displayed acceptable skew and kurtosis (i.e., <2 [33]) (Table 2). Black participants were on average younger than white participants (t=2.68, df=106, p=.009) and were less likely to have a mood disorder (N=41, 47%, versus N=34, 68%; χ2=5.03, df=1, p=.025). Age (r=–.26, p=.008) and mood disorder (t=−2.18, df=105, p=.032) were related to Prime Screen cutoff scores and were considered covariates. The race groups did not differ on any of the other demographic or clinical variables.
| Black (N=58) | White (N=50) | |||||||
|---|---|---|---|---|---|---|---|---|
| Clinical high risk | Help-seeking control | Clinical high risk | Help-seeking control | |||||
| Characteristic | N | % | N | % | N | % | N | % |
| N of participants | 24 | 41 | 34 | 59 | 19 | 38 | 31 | 62 |
| Female | 16 | 28 | 19 | 33 | 14 | 28 | 17 | 34 |
| Annual family incomea ($) | ||||||||
| <20,000 | 8 | 14 | 14 | 24 | 2 | 4 | 3 | 6 |
| 20,000–39,999 | 7 | 12 | 9 | 16 | 3 | 6 | 2 | 4 |
| 40,000–79,999 | 3 | 5 | 5 | 9 | 4 | 8 | 10 | 20 |
| ≥80,000 | 4 | 7 | 2 | 3 | 8 | 16 | 13 | 26 |
| DSM diagnosisb | ||||||||
| Mood disorder | 15 | 26 | 12 | 21 | 15 | 30 | 19 | 38 |
| Anxiety disorder | 12 | 21 | 11 | 19 | 16 | 32 | 19 | 38 |
| PTSD | 6 | 10 | 7 | 12 | 6 | 12 | 6 | 12 |
| ADHD | 10 | 17 | 17 | 29 | 10 | 20 | 15 | 30 |
| Substance use disorder | 1 | 2 | 0 | — | 2 | 4 | 5 | 10 |
| None | 0 | — | 5 | 9 | 0 | — | 1 | 2 |
| M | SD | M | SD | M | SD | M | SD | |
| Age | 14.88 | 2.01 | 15.75 | 3.14 | 16.70 | 2.87 | 16.97 | 3.12 |
| SIPS positive scorec | 12.46 | 4.74 | 4.87 | 3.42 | 12.16 | 3.22 | 5.13 | 2.85 |
| Prime Screen scored | ||||||||
| Cutoff score | 2.95 | 2.77 | 2.09 | 2.39 | 3.50 | 2.81 | .79 | 1.59 |
| Raw score | 29.00 | 17.42 | 25.23 | 14.72 | 33.56 | 17.26 | 12.52 | 14.70 |
a
Because of small cell sizes, annual family income is presented in 4 categories. For primary analyses involving family income, however, this variable was coded in 6 levels (<$20,000; $20,000–$39,999; $40,000–$59,999; $60,000–$79,999; $80,000–$99,000; and ≥$100,000). Annual family income data were available for 97 participants.
b
More than one diagnosis was common; percentages, therefore, exceed 100%.
c
SIPS, Structured Interview for Psychosis-Risk Syndromes (positive symptom domain). This interview is the gold standard assessment for clinical high-risk criteria. Possible scores range from 0 to 30, with higher scores indicating more severe positive symptoms. SIPS scores were available for 106 participants.
d
The Prime Screen is a 12-item self-report screen for clinical high-risk criteria. Each item ranges from 0 to 6 and assesses the severity of a different attenuated positive symptom. Cutoff scores represent the number of items in which the symptom was rated at the level of a 5 or 6. Raw scores range from 0 to 72, with higher scores indicating more severe positive symptoms. Prime Screen data were available for 100 participants.
| Variable | 1 | 2 | 3 | 4 | M | SD | Skew | Kurtosis |
|---|---|---|---|---|---|---|---|---|
| 1. Risk status | — | |||||||
| 2. Race | .03 | — | ||||||
| 3. SIPS positive scorea | .72* | .03 | — | 7.97 | 5.10 | .51 | –.32 | |
| 4. Prime Screen cutoff scoreb | .34* | .12 | .53** | — | 2.16 | 2.54 | 1.20 | –.62 |
a
SIPS, Structured Interview for Psychosis-Risk Syndromes (positive symptom domain). Scores range from 0 to 30, with higher scores indicating more severe symptoms.
b
Scores range from 0 to 12, equal to the number of items in which the symptom was endorsed at the level of a 5 or 6 on a scale from 0 to 6.
*
p<.01, **p<.001.
As demonstrated in Table 1, the race groups did not significantly differ on rates of high-risk diagnoses or on the severity of Prime Screen cutoff scores or attenuated positive symptoms. Results from a moderated logistic regression, however, revealed a significant interaction between race and Prime Screen cutoff scores in the predicted probability of meeting high-risk criteria (Table 3). Simple effects analyses suggested that higher Prime Screen cutoff scores significantly increased the probability of meeting these criteria for white but not black participants. The effect remained significant when the analyses controlled for household income, age, and mood disorder (b=–.51, Wald χ2=4.66, df=1, p=.031, Exp[B]=.60, 95% confidence interval=.38–.96; see Table S1 in the online supplement ). When participants who met criteria for a formal psychotic disorder were included in the high-risk group, the pattern of findings remained the same (see Table S2 in the online supplement ).
| Analysis and variable | b | Sb | Wald χ2b | p | Exp(B) | 95% CI |
|---|---|---|---|---|---|---|
| Logistic regression model | ||||||
| Race | –.10 | .47 | .05 | .82 | .90 | .36, 2.25 |
| Prime Screen cutoff score | .34 | .11 | 10.14 | .00 | 1.41 | 1.14, 1.74 |
| Race × Prime Screen cutoff score | –.44 | .22 | 4.03 | .05 | .62 | .42, .99 |
| Simple effects | ||||||
| Black | .13 | .11 | 1.43 | .23 | 1.14 | .92, 1.42 |
| White | .58 | .19 | 9.16 | .00 | 1.78 | 1.23, 2.59 |
a
Model terms are centered at zero.
b
df=1.
Table 4 shows means and standard errors of individual Prime Screen item scores, plus results of 2×2 (race × risk status) and within-race ANCOVAs comparing scores on each item across groups (controlling for family income, age, and mood disorder). These analyses sought to determine which Prime Screen items accounted for the differential response pattern among black and white participants, described above. Statistically significant race × risk status interactions were observed for six items (items 1, 2, 5, 6, 8, and 9). For black youths, mean differences between high-risk and help-seeking control groups were substantially smaller (items 1 and 5) or in the opposite direction (items 2, 6, 9, and 12) than was seen among white participants. For these latter items, black participants in the help-seeking control group scored numerically higher than black youths at risk.
| Black | White | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Clinical high risk | Help-seeking control | Clinical high risk | Help-seeking control | Race × risk statusb | |||||||||||
| Prime Screen item | M | SE | M | SE | F | η2 | M | SE | M | SE | F | η2 | F | η2 | |
| 1. | I think that I have felt that there are odd or unusual things going on that I can’t explain. | 3.03 | .49 | 2.79 | .42 | .13 | .00 | 3.55 | .51 | 1.55 | .39 | 9.22** | .20 | 4.74* | .06 |
| 2. | I think that I might be able to predict the future. | 1.05 | .53 | 2.26 | .45 | 2.98 | .07 | 1.67 | .47 | .70 | .36 | 2.52 | .06 | 4.93* | .06 |
| 3. | I may have felt that there could possibly be something interrupting or controlling my thoughts, feelings, or actions. | 2.13 | .50 | 2.20 | .42 | .01 | .00 | 2.27 | .49 | 1.14 | .38 | 3.16 | .08 | 1.96 | .02 |
| 4. | I have had the experience of doing something differently because of my superstitions. | 2.06 | .49 | 2.29 | .42 | .13 | .00 | 2.81 | .52 | 1.31 | .40 | 4.96* | .12 | 3.01 | .04 |
| 5. | I think that I may get confused at times whether something I experience or perceive may be real or may just be part of my imagination or dreams. | 3.77 | .48 | 3.36 | .41 | .42 | .01 | 4.12 | .53 | 1.54 | .41 | 14.25** | .28 | 5.83* | .07 |
| 6. | I have thought that it might be possible that other people can read my mind, or that I can read others’ minds. | .79 | .43 | 1.53 | .37 | 1.64 | .04 | 1.50 | .47 | .69 | .36 | 1.79 | .05 | 4.57* | .05 |
| 7. | I wonder if people may be planning to hurt me or even may be about to hurt me. | 3.23 | .51 | 1.82 | .44 | 4.51* | .10 | 2.63 | .46 | 1.54 | .36 | 3.33 | .08 | .03 | .00 |
| 8. | I believe that I have special natural or supernatural gifts beyond my talents and natural strengths. | 1.17 | .53 | 2.43 | .45 | 3.10 | .07 | 1.31 | .40 | .62 | .31 | 1.77 | .05 | 3.59 | .04 |
| 9. | I think I might feel like my mind is “playing tricks” on me. | 2.19 | .50 | 2.75 | .43 | .70 | .02 | 3.61 | .49 | 1.36 | .38 | 12.32** | .25 | 8.64** | .10 |
| 10. | I have had the experience of hearing faint or clear sounds of people or a person mumbling or talking when there is no one near me. | 2.83 | .50 | 2.20 | .43 | .87 | .02 | 4.18 | .56 | 1.46 | .43 | 14.28** | .28 | 3.47 | .04 |
| 11. | I think that I may hear my own thoughts being said out loud. | 2.01 | .50 | 2.31 | .43 | .12 | .00 | 2.52 | .48 | .91 | .37 | 6.75* | .15 | 3.56 | .04 |
| 12. | I have been concerned that I might be “going crazy.” | 2.01 | .53 | 2.03 | .46 | .00 | .00 | 3.27 | .46 | 1.22 | .36 | 11.54** | .24 | 4.32* | .05 |
a
Item responses range from 0 (definitely disagree) to 6 (definitely agree). Within-race contrasts in Prime Screen item scores were performed by using analyses of covariance (ANCOVAs; black, df=1, 47; white, df=1, 42). Levene’s test estimated unequal variances in Prime Screen scores at the population level for items 4 and 10 for the black group and items 2, 6, and 11 for the white group. Analyses controlled for family income, age, and mood disorder.
b
Race (black versus white) × risk status (clinical high risk versus help-seeking control) interactions were examined by using 2×2 ANCOVAs (df=1, 89). The F statistic and η2 represent the interaction of these two variables.
*
p<.05, **p<.01.
Within-race contrasts explored risk group differences on Prime Screen items separately among black and white participants (Table 4). White participants in the control group consistently scored lower than white youths at high risk, whereas a mixed pattern of results was observed among black youths, with black participants in the control group frequently endorsing items at a level comparable to or even numerically greater than those at risk. In the combined sample, sensitivity and specificity of the Prime Screen were .43 and .90, respectively. Splitting by race, these values were .27 and .90, respectively, for the black group and .61 and .90, respectively, for the white group.
A linear regression predicting the sum of positive symptoms from race, Prime Screen cutoff scores, and their cross product revealed no significant interaction, suggesting that the relation between participant-rated Prime Screen scores and clinician-rated positive symptom severity (irrespective of one’s clinical high-risk status; Table 2) was roughly equal across black and white participants.
Discussion
We found that the Prime Screen, a frequently used self-report assessment of clinical high-risk criteria, did not reliably distinguish between black help-seeking youths who were at risk of psychosis and those who were not, even though it did distinguish these groups among white participants. The findings were not explained by differences in income, age, mood disorder, rates of clinical high-risk diagnosis, or clinician-rated symptom severity. Item-level analyses suggested a differential performance across race for most items, suggesting a relatively widespread versus item-specific effect.
A long history suggests that many psychometric instruments do not perform equivalently across cultures (9–14). Instruments may not measure the same constructs across racial-ethnic groups, may use language that conveys different meaning across these groups, or may concern constructs that are more familiar to some groups than others (34). Questionnaires may be inherently subject to certain of these limitations. The Prime Screen, for example, was designed to convey risk level mental experiences by adding contingency words (e.g., “I think that I have felt…” [italics added]), a convention that may have differentially influenced responses across race. Questionnaires also may restrict the opportunity to provide important contextual information associated with endorsements, such as the degree of associated distress or impairment. By contrast, diagnostic interviews allow clinicians to use age- and culturally appropriate language and to clarify the circumstances surrounding endorsements. The addition of a distress scale to the Prime Screen, as included in a similar measure (the Prodromal Questionnaire–Brief; PQ-B), may partly address this issue.
We found that Prime Screen scores among black participants in the help-seeking control group more closely resembled those of participants at clinical high risk (black or white) than those of white participants in the help-seeking control group. Notably, the frequency of high-risk diagnoses and the severity of clinician-rated positive symptoms did not differ between racial groups. These findings are important because they suggest that the black youths in our sample appear highly symptomatic when only their self-reported Prime Screen scores are considered. Following a structured interview administered by a trained diagnostician, however, it appears that black and white participants in this sample do not differ significantly in their clinical level of psychosis risk. Given the history of misdiagnosis of schizophrenia among black individuals, reduced access to health screening and high-quality treatment (16–19), and generally high levels of discrimination and risk factors for psychosis to which people of color are often exposed (22, 35, 36), these findings highlight the need to carefully consider the most appropriate referral and treatment options for black youths who, based on these and other findings, are at increased risk of inappropriate referral, diagnosis, and intervention.
An alternative explanation for our results is that Prime Screen ratings were a more accurate measure of psychosis risk than the SIPS among black participants in the help-seeking control group, but the SIPS clinicians did not accurately rate psychosis-risk symptoms, potentially because of limitations of the instrument or cultural differences between participants and the majority white clinicians. This possibility is unlikely, however, given that all clinician-measured indices of psychopathology among black participants were either equal to or lower than those of white participants, including SIPS-rated positive symptoms, rates of high-risk diagnosis, and DSM diagnoses; formal psychotic disorders are frequently overdiagnosed among black individuals, in contrast with the roughly equal rates of high-risk diagnoses we observed; and clinicians were blind to participants’ Prime Screen scores during assessment. Therefore, our results point to the screen as the primary source of inaccuracy in assessment.
Two general population studies recently found evidence of measurement invariance across multiple racial-ethnic groups for the PQ-B, another tool designed for psychosis-risk screening (37, 38). Although these results may appear to contrast with ours, a critical distinction between these studies and ours is that only our study assessed participants with both a screening instrument and the gold-standard SIPS. Notably, black and white participants in our pooled sample did not differ on their Prime Screen cutoff scores; only when the clinician-rated risk status was considered did a differential response pattern emerge between racial groups. Given that we observed such a pattern for nearly all Prime Screen items, our results suggest that this instrument may not capture the same constructs across racial or cultural populations. The field would benefit from studies incorporating measurement invariance analysis of multiple psychosis-risk screening instruments with direct comparisons against gold-standard assessments.
A strength of our study was its use of a clinical control group to assess the performance of the Prime Screen, a screening tool used in real-world clinical settings. Relative to “healthy controls,” control groups made up of help-seeking individuals are optimal comparators in studies like ours because they are more clinically representative of the population for which the instrument was designed (39, 40). Nonetheless, the participants in our sample at clinical high risk tended to have more DSM diagnoses than the help-seeking control participants, suggesting greater overall illness severity. Although specificity estimates of the Prime Screen were excellent and our main findings held after adjustment for racial differences in mood disorder, because black participants on average presented with fewer DSM diagnoses than white participants, it remains possible that general illness severity contributed to the differential performance of the Prime Screen.
With federal funding for clinical high-risk intervention programs, large-scale dissemination of screening tools is underway. Findings from this study may inform these efforts, but our relatively small sample may not generalize to larger programs with more inclusive recruitment strategies or a broader range of sociodemographic characteristics. Our requirement that participants had already contacted a mental health care provider, for example, likely distinguishes our sample from individuals whose initial psychosis-risk assessment may be their first lifetime contact with services; it is also possible that referral patterns were differentially distributed across clinical or racial groups in our study. Given that screening thresholds may vary by help-seeking status (41) and referral source (42), identifying interactions between idiographic factors such as these may advance early identification efforts.
It is important to consider that self-reported race is only a crude proxy for numerous cultural, historical, geographic, and socioeconomic factors (among many others [43]) that may influence a person’s mental health status or response to questionnaires. Community studies designed to carefully measure these factors would allow researchers to tease apart their relative influences on psychosis-risk screening in ways that our study could not; they may also have enhanced ability to detect influences on racial-ethnic cultural subgroups (e.g., specific Caribbean, African, or European descent). A valuable approach may be to develop a maximally and cross-culturally effective screening tool based on combinations of items from previously validated psychosis-risk questionnaires. Qualitative interviews with respondents of varying backgrounds may help to promote development of novel screening items.
Conclusions
Mental health screening is a critical juncture in pathways to care. The potentially inadequate performance of psychosis-risk screens among black youths may represent a rupture at this junction, further compounding racial disparities in access to accurate diagnosis and treatment. Greater attention to cultural and contextual influences on clinical assessment may foster more accurate diagnosis and early, targeted intervention.
Supplementary Material
File (appi.ps.201800536.ds001.pdf)
- View/Download
- 31.04 KB
References
1.
McGorry PD, Yung AR, Phillips LJ: The “close-in” or ultra high-risk model: a safe and effective strategy for research and clinical intervention in prepsychotic mental disorder. Schizophr Bull 2003; 29:771–790
2.
Fusar-Poli P, Cappucciati M, Rutigliano G, et al: At risk or not at risk? A meta-analysis of the prognostic accuracy of psychometric interviews for psychosis prediction. World Psychiatry 2015; 14:322–332
3.
Wiltink S, Velthorst E, Nelson B, et al: Declining transition rates to psychosis: the contribution of potential changes in referral pathways to an ultra-high-risk service. Early Interv Psychiatry 2015; 9:200–206
4.
Fusar-Poli P, Schultze-Lutter F, Cappucciati M, et al: The dark side of the moon: meta-analytical impact of recruitment strategies on risk enrichment in the clinical high risk state for psychosis. Schizophr Bull 2016; 42:732–743
5.
Carpenter WT, Schiffman J: Diagnostic concepts in the context of clinical high risk/attenuated psychosis syndrome. Schizophr Bull 2015; 41:1001–1002
6.
Kline E, Schiffman J: Psychosis risk screening: a systematic review. Schizophr Res 2014; 158:11–18
7.
Kline E, Wilson C, Ereshefsky S, et al: Psychosis risk screening in youth: a validation study of three self-report measures of attenuated psychosis symptoms. Schizophr Res 2012; 141:72–77
8.
Kline E, Thompson E, Demro C, et al: Longitudinal validation of psychosis risk screening tools. Schizophr Res 2015; 165:116–122
9.
Sternberg RJ: Cultural explorations of human intelligence around the world. Online Readings in Psychology and Culture 2002; 4(3). https://doi.org/10.9707/2307-0919.1035
10.
Helms JE: Why is there no study of cultural equivalence in standardized cognitive ability testing? Am Psychol 1992; 47:1083
11.
Neighbors HW, Trierweiler SJ, Munday C, et al: Psychiatric diagnosis of African Americans: diagnostic divergence in clinician-structured and semistructured interviewing conditions. J Natl Med Assoc 1999; 91:601–612
12.
Sternberg RJ: Culture and intelligence. Am Psychol 2004; 59:325–338
13.
Strakowski SM, McElroy SL, Keck PE Jr, et al: Racial influence on diagnosis in psychotic mania. J Affect Disord 1996; 39:157–162
14.
Suzuki LA, Valencia RR: Race-ethnicity and measured intelligence: educational implications. Am Psychol 1997; 52:1103
15.
Groth-Marnat G: Handbook of Psychological Assessment. Oxford, England, Wiley & Sons, 2009
16.
Kenik J, Jean-Jacques M, Feinglass J: Explaining racial and ethnic disparities in cholesterol screening. Prev Med 2014; 65:65–69
17.
Coleman Wallace DA, Baltrus PT, Wallace TC, et al: Black white disparities in receiving a physician recommendation for colorectal cancer screening and reasons for not undergoing screening. J Health Care Poor Underserved 2013; 24:1115–1124
18.
Olbert CM, Nagendra A, Buck B: Meta-analysis of black vs white racial disparity in schizophrenia diagnosis in the United States: do structured assessments attenuate racial disparities? J Abnorm Psychol 2018; 127:104–115
19.
Schwartz RC, Blankenship DM: Racial disparities in psychotic disorder diagnosis: a review of empirical literature. World J Psychiatry 2014; 4:133–140
20.
Strakowski SM, Keck PE Jr, Arnold LM, et al: Ethnicity and diagnosis in patients with affective disorders. J Clin Psychiatry 2003; 64:747–754
21.
Bresnahan M, Begg MD, Brown A, et al: Race and risk of schizophrenia in a US birth cohort: another example of health disparity? Int J Epidemiol 2007; 36:751–758
22.
van Os J, Kenis G, Rutten BP: The environment and schizophrenia. Nature 2010; 468:203–212
23.
Kuno E, Rothbard AB: Racial disparities in antipsychotic prescription patterns for patients with schizophrenia. Am J Psychiatry 2002; 159:567–572
24.
McGorry PD: Early intervention in psychosis: obvious, effective, overdue. J Nerv Ment Dis 2015; 203:310–318
25.
Nieman DH, McGorry PD: Detection and treatment of at-risk mental state for developing a first psychosis: making up the balance. Lancet Psychiatry 2015; 2:825–834
26.
Wicks S, Hjern A, Gunnell D, et al: Social adversity in childhood and the risk of developing psychosis: a national cohort study. Am J Psychiatry 2005; 162:1652–1657
27.
McGlashan T, Walsh B, Woods S: The Psychosis-Risk Syndrome: Handbook for Diagnosis and Follow-Up. New York, Oxford University Press, 2010
28.
Woods SW, Walsh B, Powers AR III, et al: Reliability, validity, epidemiology, and cultural variation of the Structured Interview for Psychosis-Risk Syndromes (SIPS) and the Scale of Psychosis-Risk Symptoms (SOPS); in International Handbook of Attenuated Psychosis Syndrome in Youth and Young Adults: Early Identification and Intervention Across Cultures. Edited by Li H, Shapiro D, Seidman LJ. New York, Springer, in press
29.
Woods SW, Addington J, Cadenhead KS, et al: Validity of the prodromal risk syndrome for first psychosis: findings from the North American Prodrome Longitudinal Study. Schizophr Bull 2009; 35:894–908
30.
Kaufman J, Birmaher B, Brent D, et al: Schedule for Affective Disorders and Schizophrenia for school-age children-present and lifetime version (K-SADS-PL): initial reliability and validity data. J Am Acad Child Adolesc Psychiatry 1997; 36:980–988
31.
Miller T: The SIPS-screen: a brief self-report screen to detect the schizophrenia prodrome. Schizophr Res 2004; 70:78
32.
Aiken LS, West SG, Reno RR: Multiple Regression: Testing and Interpreting Interactions. Thousand Oaks, CA, Sage, 1991
33.
Curran PJ, West SG, Finch JF: The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychol Methods 1996; 1:16–29
34.
Lonner WJ: Psychological tests and intercultural counseling; in Counseling Across Cultures. Edited by Pederson P. Honolulu, University Press of Hawaii, 1981
35.
Smedley A, Smedley BD: Race as biology is fiction, racism as a social problem is real: anthropological and historical perspectives on the social construction of race. Am Psychol 2005; 60:16–26
36.
Williams DR, Mohammed SA, Leavell J, et al: Race, socioeconomic status, and health: complexities, ongoing challenges, and research opportunities. Ann N Y Acad Sci 2010; 1186:69–101
37.
Karcher NR, Barch DM, Avenevoli S, et al: Assessment of the Prodromal Questionnaire-Brief Child Version for Measurement of Self-reported Psychoticlike Experiences in Childhood. JAMA Psychiatry 2018; 75:853–861
38.
Cicero DC, Krieg A, Martin EA: Measurement invariance of the Prodromal Questionnaire-Brief among white, Asian, Hispanic, and multiracial populations. Assessment 2019; 26:294–304
39.
Youngstrom E, Meyers O, Youngstrom JK, et al: Comparing the effects of sampling designs on the diagnostic accuracy of eight promising screening algorithms for pediatric bipolar disorder. Biol Psychiatry 2006; 60:1013–1019
40.
Millman ZB, Gold JM, Mittal VA, et al: The critical need for help-seeking controls in clinical high-risk research. Clin Psychol Sci, in press
41.
Savill M, D’Ambrosio J, Cannon TD, et al: Psychosis risk screening in different populations using the Prodromal Questionnaire: a systematic review. Early Interv Psychiatry 2018; 12:3–14
42.
Millman ZB, Schiffman J: False positives and clinical heterogeneity among youth at clinical high-risk for psychosis: clinical and ethical implications for assessment and treatment. J Ethics Ment Health 2018; 10:1–20
43.
Helms JE, Jernigan M, Mascher J: The meaning of race in psychology and how to change it: a methodological perspective. Am Psychol 2005; 60:27–36
Information & Authors
Information
Published In
Cover:
History
Received: 25 November 2018
Revision received: 20 March 2019
Accepted: 9 May 2019
Published online: 16 July 2019
Published in print: October 01, 2019
Keywords
Authors
Competing Interests
This work was funded by National Institute of Mental Health grants R01-MH112612 and R34-MH110506 and the Behavioral Health Administration, Maryland Department of Health and Mental Hygiene, through the Center for Excellence on Early Intervention for Serious Mental Illness (OPASS 14-13717G/M00B4400214) (all to Dr. Schiffman).
Competing Interests
Dr. Woods reports that during the past 36 months he has received sponsor-initiated research funding from Auspex, Teva, Boehringer-Ingelheim, and Amarex. He has consulted to Boehringer-Ingelheim, New England Research Institute, and Takeda. He has been granted U.S. patent 8492418 B2 for a method of treating prodromal schizophrenia with glycine agonists, is an inventor on a patent pending for a method of predicting psychosis risk using blood biomarker analysis, and has received royalties from Oxford University Press. Dr. Walsh is a paid consultant for VeraSci in North Carolina to provide training and supervision of the Structured Interview for Psychosis-Risk Syndromes for a study funded by Boehringer Ingelheim. The other authors report no financial relationships with commercial interests.
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBLogin options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).