Skip to main content
Full access
Letter to the Editor
Published Online: 1 November 1999

Interrater Agreement Among Psychiatrists Regarding Emergency Psychiatric Assessments

Publication: American Journal of Psychiatry
To the Editor: We read with great interest the article by Bruce B. Way, Ph.D., and colleagues (1) regarding poor assessment reliability among psychiatrists in the emergency room setting. In the discussion, the authors suggested that the development of a screening tool with structured clinical interview characteristics that might be used more reliably in this setting would be useful. We have recently published data on just such a brief scale that is designed to be used in emergent psychiatric intake settings to determine medical necessity (2).
This brief, 11-item scale covers most of the items examined in Dr. Way et al.’s study (psychosis, depression, suicidality or homocidality, hostility or aggression, uncooperativeness, treatment noncompliance, substance abuse, physical dysfunction, role dysfunction, and social support), and it is interesting that it rates them on a similar 0–6 Likert scale. Reliabilities were originally reported for only the three subscales that emerged from factor analysis in the published article (0.96, 0.92, 0.79), but we reanalyzed the items individually, and they showed interrater reliabilities that, except for homicidality (0.28), ranged from 0.68 to 0.88. Reliability was facilitated by the use of behavioral descriptor anchor points for each pair of 0–6 ratings (mild=1–2, moderate=3–4, severe=5–6). This type of scaling method, originally used by Bigelow and Berthot (3) with the Psychiatric System Assessment Scale (3), has been used for a larger inpatient scale at our facility (4), developed as a modification of the Psychiatric System Assessment Scale, and has facilitated accurate and reliable ratings in our clinical setting. We also demonstrated that this medical necessity scale was valid on the basis of correlations with subsequent inpatient ratings that were carried out independently by different clinicians, including inpatient length of stay, and on the basis of its ability to discriminate between patients requiring and not requiring hospitalization.
We would like to make an important point regarding implementation of such a scale: while a scale with 11 items may seem rather brief, it has been an onerous task for our emergency staff over the past year of implementation. However, we have been successful and have shown an internal consistency reliability of 0.78 with our first 168 subjects. The authors’ proposal to use “a small number of questions for each dimension” would result in a scale several times larger than ours. While such a scale would certainly be more comprehensive, it would be unlikely to work well and be hard to implement in a busy clinical setting.

References

1.
Way BB, Allen MH, Mumpower JL, Stewart TR, Banks SM: Interrater agreement among psychiatrists in psychiatric emergency assessments. Am J Psychiatry 1998; 155:1423–1428
2.
Roy-Byrne P, Russo J, Rabin L, Fuller K, Jaffe C, Ries R, Dag­adakis C, Avery D: A brief medical necessity scale for mental disorders: reliability, validity, and clinical utility. J Behav Health Serv Res 1998; 25:412–424
3.
Bigelow L, Berthot BD: The Psychiatric Symptom Assessment Scale (PSAS). Psychopharmacol Bull 1989; 25:168–173
4.
Roy-Byrne P, Dagadakis C, Ries R, Decker K, Jones R, Bolte MA, Scher M, Brinkley J, Gallagher M, Patrick DL, Mark H: A psychiatrist-rated battery of measures for assessing the clinical status of psychiatric inpatients. Psychiatr Serv 1995; 46:347–352

Information & Authors

Information

Published In

Go to American Journal of Psychiatry
Go to American Journal of Psychiatry
American Journal of Psychiatry
Pages: 1840-b - 1841

History

Published online: 1 November 1999
Published in print: November 1999

Authors

Details

PETER P. ROY-BYRNE, M.D.
JOAN RUSSO, PH.D.
Seattle, Wash.

Metrics & Citations

Metrics

Citations

Export Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

For more information or tips please see 'Downloading to a citation manager' in the Help menu.

Format
Citation style
Style
Copy to clipboard

View Options

View options

PDF/EPUB

View PDF/EPUB

Get Access

Login options

Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.

Personal login Institutional Login Open Athens login
Purchase Options

Purchase this article to access the full text.

PPV Articles - American Journal of Psychiatry

PPV Articles - American Journal of Psychiatry

Not a subscriber?

Subscribe Now / Learn More

PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.

Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).

Media

Figures

Other

Tables

Share

Share

Share article link

Share