Skip to main content

Abstract

Objective:

Practice guidelines emphasize frequent clinical monitoring of patients at high risk for psychosis. No brief instrument assessing attenuated psychotic symptoms has been validated for this purpose. This study examined use of three self-report questionnaires, which were developed as psychosis risk screeners, for monitoring symptom severity in a naturalistic clinical sample of 54 adolescents.

Methods:

Self-report measures (Prime Screen–Revised, Prodromal Questionnaire–Brief Version [PQ-B], and Youth Psychosis At-Risk Questionnaire–Brief) and clinician assessments (Structured Interview for Psychosis Risk Syndromes) were administered to participants at baseline and six months.

Results:

Changes in self-report scores were moderately correlated with changes in clinician ratings. The PQ-B demonstrated slightly better agreement with changes in clinician ratings than the other two measures.

Conclusions:

Questionnaires developed as psychosis risk screeners could be used for symptom monitoring. Further validation of tools to monitor attenuated symptoms will be a valuable step toward developing an evidence-based approach for treating high-risk youths.
Evidence that early intervention in the course of psychotic illness leads to better treatment and quality-of-life outcomes has spurred efforts to identify “prodromal” or “clinical high-risk” (CHR) phases of illness. For patients who present with putatively prodromal symptoms, the American Psychiatric Association recommends “careful assessment and frequent monitoring” until symptoms remit or evolve into a “diagnosable and treatable mental disorder” (1). This “watch and wait” approach is also endorsed by the International Early Psychosis Association, Canadian Psychiatric Association, and German Association for Psychiatry, Psychotherapy, and Neurology (2,3). Until high-quality clinical trials have firmly demonstrated the benefits of further interventions, clinical monitoring of CHR symptoms accompanied by evidence-based interventions for comorbid problems or diagnoses will likely continue to constitute the gold standard of care for this population (3). Furthermore, for patients who are receiving mental health interventions and for whom the focus of treatment is either CHR symptoms or comorbid psychiatric disorders, routine outcomes monitoring may constitute an important component of evidence-based care and a valuable mechanism for patient-provider feedback regarding treatment effectiveness (4).
Despite the apparent consensus that patients with CHR symptoms should be closely monitored to detect worsening of symptoms or transition to psychosis, no self-report tool has been validated for this purpose. Studies have used semistructured interviews, such as the Structured Interview for Prodromal Syndromes (SIPS) and the Comprehensive Assessment of At-Risk Mental States, to track symptom course and progression of illness (5), but readministering lengthy interviews at frequent intervals is impractical in clinical settings. Brief, standardized self-report questionnaires may aid substantially in efforts to offer standardized monitoring of CHR symptoms. To that end, this study examined the effectiveness of self-report questionnaires to detect change in symptoms among CHR youths and a naturalistic comparison group receiving treatment over the course of six months.

Methods

Procedures were reviewed and approved by the Institutional Review Boards of the University of Maryland (UM) School of Medicine and University of Maryland, Baltimore County (UMBC). To achieve the study’s aims of contextualizing youths with psychosis risk symptoms within a naturalistic clinical comparison group, participation was open to any teen or young adult (ages 12–22) receiving mental health services at the time of consent. Participants were recruited through clinician referrals and flyers posted at UM and UMBC. Some, but not all, clinician referrals to the study were intended as psychosis consults. Participants provided informed consent and then completed a form soliciting demographic information and the three self-report tools. After encouraging youths to complete the measures on their own, study staff assisted teens with attention or literacy issues by reading items aloud. Participants were then administered the SIPS by trained study staff. Cases were reviewed weekly in team meetings to ensure reliability. As an additional check on reliability, ten audio-recorded interviews were randomly selected for rereview by the assessment team. Interrater agreement for these ten interviews was high (intraclass correlations of .82 for positive symptom ratings and .84 for all symptom ratings; κ=1.00 for diagnosis). Participants were invited to complete a follow-up assessment after six months, at which the self-report tools and the SIPS were readministered. Data were collected between December 2010 and April 2014.
The self-report measures used in this study were developed as psychosis risk “screeners” and included the Prime Screen–Revised (PS-R), the Youth Psychosis At-Risk Questionnaire-Brief (YPARQ-B), and the Prodromal Questionnaire–Brief Version (PQ-B) (68). Details about validation and scoring have been published elsewhere (9). [A table with further details about these instruments is included in an online supplement to this report.] The clinician-rated SIPS is a semistructured interview targeting interviewees’ experiences of attenuated symptoms and other indicators of psychosis risk (10). SIPS subscales include positive, negative, disorganization, and general symptoms, with positive symptoms figuring most heavily into the conceptualization of risk syndromes. SIPS diagnostic categories include presence of psychosis, presence of psychosis risk syndromes, and no diagnosis. The positive symptom subscale of the clinician-rated Scale of Psychosis Risk Symptoms (P-SOPS) was used as a criterion measure of positive symptoms; this subscale comprises clinician ratings reflecting five positive symptom domains scored on a 7-point scale.
Descriptive data and histogram visualizations identified three participants as likely “random responders”; these individuals were dropped from the sample before any analyses were performed. Potential ordering effects were examined via a series of analyses of variance (ANOVAs) by comparing means for self-report measures administered first, second, and third in the battery. Data were screened for normality, and statistical tests were chosen based on approximately normal distributions.
No clinician-scored data were missing, and 1% of self-report data were missing. Missing cell values were treated as zeros (indicating that the symptom was not present) for the YPARQ-B and PS-R. For the PQ-B distress scale (which uses symptom distress ratings in addition to presence-absence scores), missing data were scored 3, indicating that the symptom, if present, was considered neutral. Some respondents missed either a page of a measure or a measure entirely, and in these cases the respondent was dropped from the analyses using that measure.
To evaluate the degree of agreement between baseline and six-month follow-up self-report scores, correlations were calculated to reflect the relatedness of baseline and follow-up scores for each measure. On the basis of changes in scores on self-report forms and clinician-rated interviews, participants were assigned a change score reflecting increases or decreases in scores on each measure. Change scores for each self-report measure were tested by using t tests to determine whether mean changes on each measure were significant over time. Self-report change scores were then correlated with P-SOPS change scores to examine screeners’ sensitivity to clinician-rated change over time. These analyses were performed for the full sample and repeated in subsamples of only high-risk (CHR) and only low-risk participants. R to Z tests were run to determine whether differences between correlation magnitudes were statistically significant (11,12). Analyses were conducted with SPSS-22.

Results

The analysis sample included 54 participants who completed both baseline and follow-up evaluations. The 54 participants were from an initial pool of 85 participants who completed baseline evaluations and did not meet criteria for a psychotic disorder on the basis of their baseline SIPS score. Of these 85 participants, 56 (66%) completed six-month follow-up assessments. Completers did not significantly differ from noncompleters with regard to sex, age, or baseline SIPS diagnosis (low risk versus high risk).
Most participants were female (N=35, 65%). The sample was racially diverse (African American, N=28, 52%; white, N=17, 31%; more than one category or other, N=7, 13%; and American Indian, N=2, 4%). Mean±SD age at intake was 16.20±3.07. Most participants (N=28, 52%) reported a family income below $40,000 annually. Eleven participants (20%) were recruited from the UMBC campus, and the remaining 43 (80%) were recruited or referred from community clinics. Data distribution was approximately normal (skew and kurtosis ≤1.1) across all measures except for the PS-R at follow-up, and thus Pearson correlations were chosen for analyses.
At baseline, 33 participants (61%) were considered low risk according to SIPS criteria, and 21 (39%) met criteria for one or more CHR diagnoses. At follow-up, 32 participants (59%) were low risk, 18 (33%) met CHR criteria, and four (7%) met SIPS criteria for new-onset psychosis. Of those who were initially low risk, none transitioned to psychosis, 28 (85%) remained in the low-risk group, and five (15%) met SIPS CHR criteria at follow-up. Of the 21 participants who met SIPS CHR criteria at baseline, four (19%) had transitioned to psychosis by follow-up, four (19%) experienced symptom remission and were low risk at follow-up, and 13 (62%) continued to meet CHR criteria at follow-up.
To examine the degree of stability in self-report scores relative to the stability of clinician-observed symptoms, Pearson correlations between baseline and follow-up measures were calculated. Average change in total score was calculated and compared with zero by using a one-sample t test, and change scores for self-report measures were correlated with P-SOPS change scores. Results are presented in Table 1. ANOVAs exploring the possible impact of ordering effect did not detect significant findings.
TABLE 1. Correlations between measures of psychosis risk at baseline and six-month follow-up and between self-report and clinician-rated measures, for full sample and subsamples
MeasureaBaseline and follow-up correlationScore changeCorrelation of score change with change in P-SOPS score
MSD
Full sample (N=54)    
 PS-R.62**−.282.22.27
 PQ-B.78**–6.69b15.14.34*
 YPARQ-B.75**–1.62c4.57.26
 P-SOPS.75**–1.114.28
Clinical-high-risk subsample (N=21)    
 PS-R.53*−.332.83.41
 PQ-B.80**–11.50d16.16.43
 YPARQ-B.71**–1.355.62.28
 P-SOPS.86**–1.333.68
Low-risk subsample (N=33)    
 PS-R.39*−.241.77.18
 PQ-B.51**–1.9413.40.32
 YPARQ-B.50**–1.79e3.89.27
 P-SOPS–.01−.974.67
a
Possible scores on the Prime Screen–Revised (PS-R) (self-report) range from 0 to 12, with higher scores indicating more severe symptoms. Possible scores on the Prodromal Questionnaire–Brief Version (PQ-B) (self-report) range from 0 to 105, with higher scores indicating more severe symptoms. Possible scores on the Youth Psychosis At-Risk Questionnaire–Brief (YPARQ-B) (self-report) range from 0 to28, with higher scores indicating more severe symptoms. Possible scores on the Scale of Psychosis Risk Symptoms (P-SOPS) (clinician rated) range from 0 to 30, with higher scores indicating more severe symptoms.
b
Significant change over time (t=–2.68, df=50, p=.010; d=.25)
c
Significant change over time (t=–2.58, df=53, p=.013; d=.28)
d
Significant change over time (t=–3.31, df=19, p<.01; d=1.52)
e
Significant change over time (t=–2.64, df=32, p=.013; d=.93)
*p<.05, **p<.01

Discussion and Conclusions

Despite the small sample size, the data yielded interesting preliminary findings. Self-report measures demonstrated a high degree of stability over the follow-up interval. Notably, clinician ratings were highly consistent from baseline to follow-up for participants who met CHR criteria but not for participants deemed low risk, suggesting a higher level of symptom stability and persistence among those with more severe symptoms at baseline. Both self-report and P-SOPS scores indicated slight decreases in participants’ average symptom severity from baseline to follow-up. This change may reflect regression toward the mean, positive treatment effects (all participants were receiving mental health services), or a tendency among clinicians to refer participants for evaluation at a time of peak concern. Despite this overall pattern of symptom improvement, a few individuals experienced marked symptom exacerbation between time points, with four transitioning from a CHR state to full-threshold psychosis by follow-up. Most CHR participants continued to meet SIPS CHR criteria at follow-up, indicating some persistence of symptoms over time for most individuals in this group.
Of the three self-report measures tested, the PQ-B change score demonstrated the highest correlation with the P-SOPS change score (r=.34 in the full sample and .43 in the CHR subsample), suggesting that the PQ-B may be most sensitive to clinician-detected changes over time. In other words, over repeated administrations, respondents’ endorsements on the PQ-B were most likely to “track” with the interviewing clinicians’ impressions. Thus this instrument may be the best fit for monitoring symptoms at six-month intervals. This may be due to the PQ-B’s inclusion of the distress scale, which gives respondents an opportunity to report how much symptoms are currently bothering them (rather than simply whether symptoms have occurred), similar to how clinicians conceptualize symptom severity scores. The PQ-B’s potentially greater sensitivity to change may also be attributable to the PQ-B’s instructions that respondents think about their experiences over the past month, whereas the other instruments direct respondents to answer items on the basis of their experiences over the past year (PS-R) or “the kind of person you generally are” (YPARQ-B).
When analyses were repeated only for the CHR subsample, the associations between PQ-B and PS-R change scores and P-SOPS change scores became somewhat stronger (that these correlations were not statistically significant likely reflects the small sample size and lack of adequate statistical power for subsample analyses). The stronger associations could indicate that in the CHR group, changes in self-report scores may be more clinically meaningful than in the low-risk group. Especially in the low-risk group, shifts in psychotic-like symptoms may reflect changes in other conditions, such as mood, anxiety, or substance use disorders.
Although several ongoing clinical trials aim to establish treatment protocols for youths with prodromal symptoms (13,14), practice guidelines for CHR individuals emphasize watchful waiting. This study had several limitations, chiefly the small sample and limited duration of follow-up (larger studies of CHR youths have found that conversion to psychosis may take place several years after initial presentation of attenuated positive symptoms, thus necessitating a potentially longer period of clinical monitoring). The sample was primarily female, which is somewhat unusual for samples of CHR youths and may have influenced self-report or interview responses. Despite these limitations, results indicate that questionnaires developed as psychosis risk screeners could be used for the purpose of symptom monitoring as well as to predict transitions to psychosis (15).
Future directions for research that could lead to improvements in accuracy of symptom monitoring include identifying key items useful for detecting and monitoring symptoms over time, validating monitoring tools over longer periods of follow-up, and investigating more comprehensive self-report inventories as alternative tools for monitoring (for example, the Peters Delusional Inventory, the 92-item Prodromal Questionnaire, and the Community Assessment of Psychotic Experiences). Further validation of self-report tools to track the severity of attenuated symptoms over time will be a valuable step toward developing an evidence-based approach for treating high-risk youths.

Acknowledgments

The authors acknowledge the contributions of study participants and their families.

Supplementary Material

File (appi.ps.201500063.ds001.pdf)

References

1.
Lehman AF, Lieberman JA, Dixon LB, et al: Practice Guideline for the Treatment of Patients With Schizophrenia, 2nd ed. Arlington, Va, American Psychiatric Association, 2010
2.
Addington J, Amminger GP, Barbato A, et al: International clinical practice guidelines for early psychosis. British Journal of Psychiatry 48:s120–s124, 2005
3.
Fusar-Poli P, Borgwardt S, Bechdolf A, et al: The psychosis high-risk state: a comprehensive state-of-the-art review. JAMA Psychiatry 70:107–120, 2013
4.
Lambert MJ, Whipple JL, Hawkins EJ, et al: Is it time for clinicians to routinely track patient outcome? A meta-analysis. Clinical Psychology: Science and Practice 10:288–301, 2003
5.
Addington J, Cadenhead KS, Cornblatt BA, et al: North American Prodrome Longitudinal Study (NAPLS 2): overview and recruitment. Schizophrenia Research 142:77–82, 2012
6.
Miller TJ, Cicchetti D, Markovich PJ, et al: The SIPS screen: a brief self-report screen to detect the schizophrenia prodrome. Schizophrenia Research 70(suppl):78, 2004
7.
Ord LM, Myles-Worsley M, Blailes F, et al: Screening for prodromal adolescents in an isolated high-risk population. Schizophrenia Research 71:507–508, 2004
8.
Loewy RL, Pearson R, Vinogradov S, et al: Psychosis risk screening with the Prodromal Questionnaire–brief version (PQ-B). Schizophrenia Research 129:42–46, 2011
9.
Kline E, Wilson C, Ereshefsky S, et al: Psychosis risk screening in youth: a validation study of three self-report measures of attenuated psychosis symptoms. Schizophrenia Research 141:72–77, 2012
10.
Miller TJ, McGlashan TH, Rosen JL, et al: Prodromal assessment with the Structured Interview for Prodromal Syndromes and the Scale of Prodromal Symptoms: predictive validity, interrater reliability, and training to reliability. Schizophrenia Bulletin 29:703–715, 2003
11.
Dunn OJ, Clark V: Correlation coefficients measured on the same individuals. Journal of the American Statistical Association 64:366–377, 1969
12.
Steiger JH: Tests for comparing elements of a correlation matrix. Psychological Bulletin 87:245–251, 1980
13.
Stafford MR, Jackson H, Mayo-Wilson E, et al: Early interventions to prevent psychosis: systematic review and meta-analysis. British Medical Journal 346:f185–f185, 2013
14.
Okuzawa N, Kline E, Fuertes J, et al: Psychotherapy for adolescents and young adults at high risk for psychosis: a systematic review. Early Intervention in Psychiatry 8:307–322, 2014
15.
Kline E, Schiffman J: Psychosis risk screening: a systematic review. Schizophrenia Research 158:11–18, 2014

Information & Authors

Information

Published In

Go to Psychiatric Services
Go to Psychiatric Services

Cover: Fame Weathervane, by E.G. Washburne and Company, circa 1890. Copper and zinc with gold leaf. American Folk Art Museum, Long Island City, New York. Gift of Ralph Esmerian, accession number 2005.8.62. Photo credit: Gavin Ashworth.

Psychiatric Services
Pages: 456 - 459
PubMed: 26567937

History

Received: 10 February 2015
Revision received: 3 June 2015
Accepted: 1 July 2015
Published online: 16 November 2015
Published in print: April 01, 2016

Authors

Details

Emily Kline, Ph.D.
Dr. Kline is with the Department of Psychiatry, Harvard Medical School, and Beth Israel Deaconess Medical Center, Boston (e-mail: [email protected]). Ms. Thompson, Ms. Demro, and Dr. Schiffman are with the Department of Psychology, University of Maryland, Baltimore County, Baltimore. Ms. Bussell and Dr. Reeves are with the Department of Psychiatry, University of Maryland Medical School, Baltimore.
Elizabeth Thompson, M.A.
Dr. Kline is with the Department of Psychiatry, Harvard Medical School, and Beth Israel Deaconess Medical Center, Boston (e-mail: [email protected]). Ms. Thompson, Ms. Demro, and Dr. Schiffman are with the Department of Psychology, University of Maryland, Baltimore County, Baltimore. Ms. Bussell and Dr. Reeves are with the Department of Psychiatry, University of Maryland Medical School, Baltimore.
Caroline Demro, B.A.
Dr. Kline is with the Department of Psychiatry, Harvard Medical School, and Beth Israel Deaconess Medical Center, Boston (e-mail: [email protected]). Ms. Thompson, Ms. Demro, and Dr. Schiffman are with the Department of Psychology, University of Maryland, Baltimore County, Baltimore. Ms. Bussell and Dr. Reeves are with the Department of Psychiatry, University of Maryland Medical School, Baltimore.
Kristin Bussell, M.S., C.R.N.P.
Dr. Kline is with the Department of Psychiatry, Harvard Medical School, and Beth Israel Deaconess Medical Center, Boston (e-mail: [email protected]). Ms. Thompson, Ms. Demro, and Dr. Schiffman are with the Department of Psychology, University of Maryland, Baltimore County, Baltimore. Ms. Bussell and Dr. Reeves are with the Department of Psychiatry, University of Maryland Medical School, Baltimore.
Gloria Reeves, M.D.
Dr. Kline is with the Department of Psychiatry, Harvard Medical School, and Beth Israel Deaconess Medical Center, Boston (e-mail: [email protected]). Ms. Thompson, Ms. Demro, and Dr. Schiffman are with the Department of Psychology, University of Maryland, Baltimore County, Baltimore. Ms. Bussell and Dr. Reeves are with the Department of Psychiatry, University of Maryland Medical School, Baltimore.
Jason Schiffman, Ph.D.
Dr. Kline is with the Department of Psychiatry, Harvard Medical School, and Beth Israel Deaconess Medical Center, Boston (e-mail: [email protected]). Ms. Thompson, Ms. Demro, and Dr. Schiffman are with the Department of Psychology, University of Maryland, Baltimore County, Baltimore. Ms. Bussell and Dr. Reeves are with the Department of Psychiatry, University of Maryland Medical School, Baltimore.

Notes

An incomplete version of the data and analyses was presented at the annual meeting of the Society for Research in Psychopathology, Oakland, California, on September 19–22, 2013.

Funding Information

Maryland Department of Health and Mental Hygiene10.13039/100006671: OPASS# 14-13717G/M00B4400241
This work was supported in part by funding from the Maryland Department of Health and Mental Hygiene, Mental Hygiene Administration, through the Center for Excellence on Early Intervention for Serious Mental Illness (OPASS no. 14-13717G/M00B4400241).The authors report no financial relationships with commercial interests.

Metrics & Citations

Metrics

Citations

Export Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

For more information or tips please see 'Downloading to a citation manager' in the Help menu.

Format
Citation style
Style
Copy to clipboard

View Options

View options

PDF/EPUB

View PDF/EPUB

Login options

Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.

Personal login Institutional Login Open Athens login
Purchase Options

Purchase this article to access the full text.

PPV Articles - Psychiatric Services

PPV Articles - Psychiatric Services

Not a subscriber?

Subscribe Now / Learn More

PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.

Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).

Media

Figures

Other

Tables

Share

Share

Share article link

Share