A Brief Scale for Assessing Patients' Satisfaction With Care in Outpatient Psychiatric Services
Abstract
OBJECTIVE: The authors evaluated the reliability and preliminary validity of the Charleston Psychiatric Outpatient Satisfaction Scale, a 15-item measure of patients' satisfaction designed for use in outpatient settings. The instrument uses a 5-point Likert-type response format that minimizes positive response bias and optimizes variability and predictive validity. METHODS: The Charleston Psychiatric Outpatient Satisfaction Scale was administered to 282 patients seen in psychiatric outpatient clinics affiliated with a public-academic psychiatric institution over a one-week period in 1995. RESULTS: The internal reliability of the instrument was high (alpha=.87), and its convergent validity was supported by the significant correlation of all items with anchor items that measured overall satisfaction with care and likelihood of recommending the clinic to others. The best predictors of overall ratings of care were the items measuring patients' satisfaction with helpfulness of the services and with the respect shown for patients' opinions about treatment. The best predictors of recommendation of the program to others were the items measuring satisfaction with matching of the treatment plan to patients' individual needs and with the respect shown for patients' opinions about treatment. Mean scores for all items ranged from 3.6 (satisfaction with parking) to 4.5 (satisfaction with helpfulness of the secretary and with the overall quality of care), indicating that overall satisfaction in this sample was high. CONCLUSIONS: The results provide preliminary support for the reliability and validity of the Charleston Psychiatric Outpatient Satisfaction Scale.
The assessment of patients' satisfaction with medical services has been a rapidly developing area of research for almost 20 years (1,2,3,4,5). However, the assessment of patients' satisfaction with psychiatric clinics has only recently started to come into focus (6,7,8,9). Several specific measures have been developed for evaluating patients' satisfaction in a variety of psychiatric settings, including inpatient (10,11,12) and outpatient (13,14,15) clinics. Furthermore, satisfaction has been shown to be associated with improved clinical outcomes and administrative measures of quality at the individual level (12,16,17), although it is not clear whether patients' satisfaction is a cause or an effect of changes in symptoms.
In either case, psychiatric clinics that have the goal of maintaining or expanding a practice, particularly in a managed care environment, should measure patients' satisfaction to identify opportunities for improvement and to remain competitive. In addition, the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) and other accrediting organizations require information on patients' satisfaction as a performance indicator.
Many patient satisfaction measures described in the literature do not use the item response formats that have been shown to produce more sensitive measures of satisfaction. As a result, many surveys yield highly skewed response distributions with low variability (18). In a study of methods for measuring patients' satisfaction with medical services, Ware and Hays (18) compared two response formats: a 5-point scale offering the response options of excellent, very good, good, fair, and poor (an E5 scale) and a 6-point scale offering the response options of extremely satisfied, very satisfied, somewhat satisfied, neither satisfied nor dissatisfied, somewhat dissatisfied, and very dissatisfied (an S6 scale). Their study was conducted in two types of settings: fee-for-service clinics in rural and urban areas of the Midwest and the West and primary and specialty care clinics in a health maintenance organization in the East.
The E5 format yielded less skewed results with greater variability in both settings. Furthermore, the E5 format better predicted whether patients intended to return to the same doctor for future visits, recommended the doctor to a friend, and complied with their medical regimen. On the basis of their findings, Ware and Hays concluded that the E5 format is preferable to the S6 format in measures of patients' satisfaction with medical services.
The study reported here was conducted to evaluate the reliability and preliminary validity of the Charleston Psychiatric Outpatient Satisfaction Scale, a 15-item measure of patients' satisfaction. This scale was designed to be brief enough to be practical and user-friendly but thorough enough to cover key clinical, administrative, and environmental factors that are important to psychiatric outpatients. It incorporates the E5 response format, which is intended to minimize positive response bias and optimize variability and predictive validity.
Methods
The respondents were 282 male and female adult outpatients treated at 19 clinics of the Institute of Psychiatry, a public-academic psychiatric institution affiliated with the Medical University of South Carolina in Charleston. The clinics include general psychiatric clinics that treat a broad range of disorders as well as specialty clinics, such as treatment programs for substance abuse and eating disorders. Patients in these clinics represent a broad range of socioeconomic statuses. To protect the respondents' anonymity and to encourage honest ratings, data on demographic characteristics such as age and gender were not collected and the surveys were not coded.
The data were collected over a one-week period in 1995. All patients who received services at any outpatient psychiatric clinic affiliated with the Institute of Psychiatry during that week were asked by the receptionist to complete the survey. To standardize survey administration, the receptionists were trained with the following script: "Ms./Mr. ____, the staff here are very interested in your opinions about the services you have received here, and we would like to ask you to complete this brief, anonymous survey. Your honest answers, both positive and negative, will help us improve our services. When you have finished, please drop it off in the hanging file located _____. Thanks so much for your feedback." The survey could be completed in two to five minutes. Anecdotal comments from receptionists suggested that the survey was well received by patients, and the receptionists reported no instances of patients' being unable to complete the survey because of disability or literacy problems.
The 15-item Charleston Psychiatric Outpatient Satisfaction Scale, shown in Table 1, uses a 5-point Likert-type scale with an E5 response format—5, excellent; 4, very good; 3, good; 2, fair; and 1, poor; plus a "does not apply" option—for all but the last item in the scale. The last item, which assesses behavioral intent to recommend the clinic, uses a 4-point response format: 4, yes, definitely; 3, yes, probably; 2, no, probably not; and 1, no, definitely not.
The 15 items were developed and selected in several ways. Some were selected from a large pool of items from an inpatient satisfaction scale previously pilot-tested on the inpatient units at the Institute of Psychiatry. Items that were the strongest predictors of overall ratings of quality of care and intent to recommend the facility were selected for the outpatient scale. Other items were developed to assess administrative and environmental factors unique to the outpatient setting. All of the items were developed on the basis of input from psychiatric clinicians and a comprehensive review of studies that used surveys and focus groups to identify service features that are important to psychiatric patients. These features include staff attitudes and therapeutic components as well as amenities such as parking and the physical environment of the clinic.
Two items measuring overall satisfaction with care (item 8) and likelihood of referring family and friends to the clinic (item 15) were included as anchor items. Anchor items are used to test preliminary validity and to identify items that are the best predictors of general satisfaction (18).
The psychometric properties of the Charleston Psychiatric Outpatient Satisfaction Scale were evaluated through a series of analyses that were planned a priori. Internal reliability, the consistency or preciseness of a measurement tool, was evaluated with Cronbach's alpha. Convergent validity, the extent to which the scale is correlated with similar constructs, was evaluated by using Pearson's product-moment correlation to test the relationship between overall scale scores and scores on the anchor items.
Results
The internal reliability of the Charleston Psychiatric Outpatient Satisfaction Scale, as measured by Cronbach's alpha, was high (alpha=.87). Preliminary convergent validity of the instrument was supported by the finding that scores for all items were significantly correlated with the scores for the two anchor items (r=.27 to .86 for anchor item 8, satisfaction with overall quality of care, and r=.19 to .63 for anchor item 15, likelihood of recommending the clinic to others) (Table 1). In addition, the total scale scores—that is, the sum of the scores for all items except the anchor items—were significantly correlated with the scores for both anchor items.
The best predictors of overall ratings of satisfaction with care were satisfaction with the helpfulness of the services (item 7, r=.86) and satisfaction with respect shown for patients' opinions about treatment (item 5, r=.84). The best predictors of recommendation of the program to others were satisfaction with the matching of the treatment plan to patients' individual needs (item 6, r=.63) and satisfaction with the respect shown for patients' opinions about treatment (item 5, r=.61). The mean scores for all items ranged from 3.6 (item 13, parking) to 4.5 (item 1, helpfulness of secretary, and item 8, overall quality of care), indicating that overall satisfaction with services in this sample was high—that is, very good on the 5-point response scale.
Discussion
Measurement of patients' satisfaction in psychiatric clinics is important because patients' satisfaction has been correlated with improved clinical outcomes and administrative measures of high-quality care—for example, fewer readmissions. In addition, measurement of patients' satisfaction allows organizations to identify areas of service delivery that can be improved. Ongoing improvement of service delivery and clinical outcomes is essential if a psychiatric clinic is to become and remain competitive in the current mental health care environment. Furthermore, JCAHO and other accrediting organizations require measurement of patients' satisfaction as a performance indicator. Thus implementation of a reliable and valid measure of patients' satisfaction can serve several important purposes relevant to the overarching mission of psychiatric clinics.
The patient satisfaction survey was designed specifically for outpatient psychiatric clinics. This 15-item measure represents an advance in the ability to measure patients' satisfaction in two ways. First, its items were developed on the basis of input from outpatient psychiatric clinicians and a comprehensive review of previous studies that have identified areas important to psychiatric patients. Second, it uses a response format—the E5 format—that minimizes positive response bias and optimizes variability and predictive validity. Because use of this response format is supported by the empirical literature, this scale is likely to be a more sensitive measure of psychiatric patients' satisfaction than scales that use a response format with options ranging from satisfied to dissatisfied.
Conclusions
The results of this study provide preliminary support for the reliability and validity of the Charleston Psychiatric Outpatient Satisfaction Scale, a 15-item measure of patients' satisfaction designed for use with adult psychiatric outpatients. However, this scale should be tested further to confirm its advantages and ensure its generalizability to psychiatric outpatients in other settings.
Footnote
Dr. Pellegrin is affiliated with Discovery Alliance International, Inc., in Mobile, Alabama. Dr. Stuart, Dr. Frueh, and Dr. Ballenger are affiliated with the department of psychiatry and behavioral sciences at the Medical University of South Carolina in Charleston. Dr. Stuart is also with the College of Nursing at the university. Dr. Frueh is also with the Veterans Affairs Medical Center in Charleston. Ms. Maree is with Hamilton Medical Center in Dalton, Georgia. Send correspondence to Dr. Frueh at the VA Medical Center, 109 Bee Street, Charleston, South Carolina 29401 (e-mail, fruehbc@ musc.edu).
References
1.
Nelson-Wernick E, Currey HS, Taylor PW, et al: Patient perception of medical care. Health Care Management Review 6:65-72, 1981
2.
Lebow JL: Consumer satisfaction with mental health treatment. Psychological Bulletin 91:244-259, 1982
3.
Weiss GL, Ramsey CA: Regular source of primary medical care and patient satisfaction. Quality Review Bulletin 15:180-184, 1989
4.
Campbell J: Consumerism, outcomes, and satisfaction: a review of the literature, in Mental Health, United States, 1998. Edited by Manderscheid RW, Henderson MJ. Rockville, Md, US Department of Health and Human Services, 1998
5.
Roth TA, Schoolcraft M: Patient satisfaction: the survey says. … Nursing Case Management 3:184-189, 1998
6.
Stevenson JF, Beattie MC, Alves RR, et al: An outcome monitoring system for psychiatric inpatient care. Quality Review Bulletin 14:326-331, 1988
7.
Ruggeri M, Dall'Agnola R: The development and use of the Verona Expectations for Care Scale (VECS) and the Verona Service Satisfaction Scale (VSSS) for measuring expectations and satisfaction with community-based psychiatric services in patients, relatives, and professionals. Psychological Medicine 23:511-523, 1993
8.
Barker DA, Shergill SS, Higgenson I, et al: Patients' views towards care received from psychiatrists. British Journal of Psychiatry 168:641-646, 1996
9.
Etter JF, Perneger TV, Rougemont A: Does sponsorship matter in patient satisfaction surveys? A randomized trial. Medical Care 34:327-335, 1996
10.
Babiker IE, Thorne P: Do psychiatric patients know what is good for them? Journal of the Royal Society of Medicine 86:28-30, 1993
11.
Greenwood N, Key A, Burns T, et al: Satisfaction with in-patient psychiatric services: relationship to patient and treatment factors. British Journal of Psychiatry 174:159-163, 1999
12.
Holcomb WR, Parker JC, Leong GB, et al: Customer satisfaction and self-reported treatment outcomes among psychiatric inpatients. Psychiatric Services 49:929-934, 1998
13.
Glass AP: Identifying issues important to patients on a hospital satisfaction questionnaire. Psychiatric Services 46:83-85, 1995
14.
Parker G, Wright M, Robertson S, et al: The development of a patient satisfaction measure for psychiatric outpatients. Australian and New Zealand Journal of Psychiatry 30:343-349, 1996
15.
Clark CC, Scott EA, Boydell KM, et al: Effects of client interviewers on client-reported satisfaction with mental health services. Psychiatric Services 50:961-963, 1999
16.
Kelstrup A, Lund K, Lauritsen B, et al: Satisfaction with care reported by psychiatric inpatients: relationship to diagnosis and medical treatment. Acta Psychiatrica Scandinavica 87:374-379, 1993
17.
Druss B, Rosenheck RA, Stolar M: Patient satisfaction and administrative measures as indicators of the quality of mental health care. Psychiatric Services 50:1053-1058, 1999
18.
Ware JE Jr, Hays RD: Methods for measuring patient satisfaction with specific medical encounters. Medical Care 26:393-402, 1988
Information & Authors
Information
Published In
History
Published online: 1 June 2001
Published in print: June 2001
Authors
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBLogin options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).