Veterans’ Perceptions of Behavioral Health Care in the Veterans Health Administration: A National Survey
Abstract
Objective
This study provided national estimates of perceptions of behavioral health care services among patients of the Veterans Health Administration (VHA) with a diagnosis of bipolar I disorder, major depression, posttraumatic stress disorder, schizophrenia, or substance use disorder.
Methods
A stratified random sample of 6,190 patients completed telephone interviews from November 2008 through August 2009. Patients (N=5,185) who reported receiving VHA behavioral health care in the prior 12 months were asked about their need for housing and employment services, timeliness and recovery orientation of their care, satisfaction with care, and perceived improvement.
Results
Half of patients reported always receiving routine appointments as soon as requested, and 42% were highly satisfied with their VHA mental health care. Approximately 74% of patients reported being helped by the treatment they received, yet only 32% reported that their symptoms had improved. After controlling for covariates, the analyses showed that patients with a substance use disorder reported lower satisfaction with care and perceived their treatment to be less helpful compared with patients without a substance use disorder.
Conclusions
Although matched sample comparison data were not available, the results showed that overall patient perceptions of VHA mental health care were favorable, but there was significant room for improvement across all areas of assessment. A majority reported being helped by treatment, but few reported symptom improvement. Variations in perceptions among patients with different disorders suggest the potential importance of psychiatric diagnosis, particularly substance use disorder, in assessing patient perceptions of care.
The Institute of Medicine (IOM) (1) outlines six key dimensions of health care quality, including safety, timeliness, effectiveness, efficiency, equity, and patient centeredness. Patients are increasingly asked to report their perceptions of care across a range of dimensions via standardized surveys (2). For some domains, such as patient centeredness, self-reported perceptions may be the primary data source. Because patient centeredness reflects whether care is respectful or responsive to individuals’ needs, patients may be the best source for that information. However, patients can also report on numerous domains of quality, including interpersonal quality, technical quality, timeliness of care, and satisfaction with care. Therefore, patients’ self-reported perceptions may supplement medical record review and administrative data for other IOM domains, such as effectiveness.
Although positive perceptions of care among patients is a worthy objective on its own, prior research has demonstrated that patient perceptions are related to other key health care processes and outcomes, such as patients’ provider choice (3), adherence to treatment regimens (4), complaints (5), and health outcomes (6–11). These associations suggest that efforts to improve patient perceptions may also improve treatment outcomes.
Given the importance of patient perceptions of care, there is a need to investigate factors that contribute to variations in these perceptions. Identifying variations in these perceptions among different patient populations may contribute to strategies to improve care or address disparities. Studies suggest that patient perceptions, such as satisfaction and perceived barriers to care, may vary by psychiatric diagnosis (12–16). To extend this work, we examined perceptions of behavioral health care for a broad range of domains in a national sample of patients in the Veterans Health Administration (VHA) and compared perceptions by patients across selected diagnoses.
The VHA is the largest integrated health care organization in the country. Over 900,000 veterans received care for mental or substance use disorders from the VHA in fiscal year (FY) 2008 (17). These veterans accounted for a disproportionate share of VHA health care utilization and costs (17). The number of veterans receiving behavioral health services has been increasing for over a decade (17,18). The size, growth, and treatment needs of this population suggest that it is important to understand the perceptions of care among veterans receiving behavioral health services.
This article has two objectives. First, it provides national estimates for multiple domains of perceptions of care. Second, it presents an evaluation of whether patients’ perceptions differ by mental health diagnosis (bipolar disorder, depression, posttraumatic stress disorder [PTSD], and schizophrenia) or by whether the patient has a substance use disorder.
Methods
These data were collected as part of a national evaluation of VHA mental health services (17,19), which focused on patients who received care for disorders in one or more of the following five diagnostic groups: schizophrenia, bipolar I disorder, PTSD, major depressive disorder, and substance use disorders.
Participants and recruitment
Study population.
The study population was identified by using existing VHA administrative databases. The population consisted of veterans who received at least one inpatient episode or two outpatient encounters in FY 2007 from, or care paid for by, the VHA for one of the four mental health diagnoses listed above. Patients were assigned to one of four mental health diagnostic cohorts on the basis of the modal frequency of episodes of care. Ties were resolved by using the following rank order: schizophrenia, bipolar disorder, PTSD, and depression. In addition, patients with substance use disorder diagnoses were identified as a separate cohort. Patients with co-occurring substance use and mental disorders could be assigned to the substance use disorder cohort plus one mental health diagnostic cohort.
Given the concern that asking patients to recall the care they received in FY 2007 could result in recall bias, survey questions focused on mental health care during the 12 months prior to the interview date. Therefore, we restricted the study population to patients with any VHA health care utilization in FY 2008 to minimize the number of survey respondents who would report having received no care from the VHA in the 12 months before being contacted for the survey.
Sampling plan.
We stratified the study population by geographic region, mental health diagnostic cohort, and substance use disorder cohort. We randomly sampled 9,619 patients across these strata; 320 deceased patients were subsequently removed from the sample, leaving an eligible sample of 9,299.
Survey procedures
The survey was fielded between November 2008 and August 2009. Prospective participants received a letter and a study brochure one week prior to being contacted by telephone. An additional copy of these materials was sent to the mailing address on file after three weeks of attempting to contact the respondent. Interviewers made an average of four call attempts for each completed interview. After receiving a complete description of the study, the participants gave verbal informed consent. Completed interviews lasted approximately 30 minutes. Participants received a $10 check. The RAND Human Subjects Protection Committee approved all procedures.
Measures
Using an item adapted from the Experience of Care and Health Outcomes (ECHO) survey (20), we asked participants if they received “counseling or treatment” for mental health or substance use problems in the prior 12 months. For patients who endorsed receiving behavioral health care, subsequent survey items assessed patient perceptions across a variety of domains.
Demographics characteristics.
Demographic information was drawn from administrative data and survey data.
Functioning.
We assessed functional status by using the Veterans RAND 12-Item Health Survey (21). The survey has been used extensively with veteran populations and has been shown to effectively identify the incremental impact of several chronic diseases (21). Using this measure, we derived two composite scores reflecting mental and physical health functioning. Possible scores range from 0 to 100, with higher scores indicating better functioning. Both scores are norm-based and can be interpreted in relation to the distribution of scores in the 1990 adult U.S. population (mean±SD=50±10).
Perceptions of VHA behavioral health care.
We assessed patients’ perceptions of timeliness of care, the recovery orientation of care, perceptions of psychosocial services, and overall satisfaction. We assessed timeliness of care by using four items from the ECHO (20). Patients were asked if they needed counseling or treatment in the past 12 months. If so, they were asked to rate how often they received appointments as soon as they wanted on a 4-point scale, from 1, never, to 4, always. Patients were asked separately about routine and urgent care. Timeliness is reported as the percentage who endorsed “always.”
We assessed perceptions of the recovery orientation of VHA staff by using seven items from the Recovery Self-Assessment (22,23). The selected items assess whether staff listened to and respected the patient’s decisions, conveyed high expectations for the patient’s recovery, believed the patient could make his or her own life choices, asked about the patient’s interests, helped develop life goals beyond symptom management, helped to include other important people in treatment planning, and introduced the patient to role models or mentors. Possible responses range from strongly disagree to strongly agree. Because responses on these items are typically highly skewed, we report the percentage who endorsed “strongly agree.”
We assessed perceptions of two aspects of psychosocial services. We assessed need for and receipt of housing services in the past 12 months by using two adapted items from the HIV Cost and Services Utilization Study (24). Patients were asked if they needed help finding a place to live. If they answered yes, they were asked whether they had received help from the VHA. The same items were repeated to assess perceived need for help with employment.
We assessed patients’ overall perception of their VHA counseling or treatment by using one item from the ECHO. Patients were asked to rate their behavioral health care from 0 to 10, with 0 indicating the “worst counseling or treatment possible” and 10 indicating the “best counseling or treatment possible.” Patients who reported a rating of 9 or 10 were considered highly satisfied.
Perceived improvement.
We assessed patients’ perceived improvement by using five items from the ECHO (20). Four items ask about improvement in ability to deal with daily problems and social situations and to accomplish what they would like to do and improvements in symptoms in the past 12 months. There are five possible responses, ranging from much worse to much better. We report the percentage who endorsed “much better” or “a little better.” An additional item asks how much patients were helped by their counseling or treatment overall. Response options include not at all, a little, somewhat, and a lot. We report the percentage who endorsed “a lot” or “somewhat.”
Statistical analyses
A sampling weight was derived to adjust for the stratified sampling design (25). A nonresponse weight reflecting the inverse probability of each sampled veteran completing the survey was derived by using logistic regression analysis of the relationship of survey completion and veteran characteristics (26). To reflect the study population, all estimates were weighted by a final analysis weight that was the product of the sampling and nonresponse weights.
We present demographic and other descriptive characteristics of the sample and national estimates of patient perceptions. We evaluated whether perceptions varied by mental health diagnostic cohort or by substance use disorder cohort. Chi square tests were used to measure significant differences across diagnostic cohorts for the categorical outcomes. Subsequently, logistic regressions were conducted for each diagnostic cohort to test whether diagnosis remained a significant predictor after adjustment for age, race, sex, service-connected disability, mental and physical health functioning, utilization (outpatient visits and inpatient nights), and co-occurring mental or substance use disorder. Given that multiple statistical tests were conducted, p values were compared against an alpha of .01 to determine significance. For these analyses, this threshold corresponds to controlling the false discovery rate (the proportion of rejected null hypotheses that are actually true) to an alpha level of .05 (27).
Results
Sample characteristics
A total of 6,190 patients participated in the telephone interviews (67% response rate). These analyses focused on the 5,185 (84%) participants who reported receiving behavioral health care from the VHA in the past 12 months. Administrative data for the 1,005 patients who denied receiving VHA behavioral health care indicated that 534 (53%) had received such care within the prior 12 months, suggesting some error in recall or understanding of the question.
Demographic and other characteristics of the sample, in weighted percentages, are provided in Table 1. The average age was 58, over 90% were men, and about 63% were non-Hispanic Caucasian. A majority of patients at least had completed some college (59.6%), were not in the labor force (74.3%), and reported annual incomes of $30,000 or less (57.1%). More than half of patients had a service-connected disability for any condition, and approximately 40% of patients received VHA disability compensation for a diagnosis that was the same as the diagnostic cohort to which they were assigned. Approximately 6% of patients served in Operation Enduring Freedom or Operation Iraqi Freedom.
| Characteristic | N | %a | SE |
|---|---|---|---|
| Age | |||
| 18–34 | 249 | 5.4 | .4 |
| 35–44 | 445 | 7.9 | .5 |
| 45–54 | 1,293 | 20.3 | .7 |
| 55–64 | 2,344 | 49.5 | .9 |
| ≥65 | 854 | 16.8 | .7 |
| Male | 4,679 | 92.1 | .4 |
| Race-ethnicity | |||
| Hispanic | 371 | 9.1 | .5 |
| Caucasian | 3,460 | 62.6 | .9 |
| Black | 935 | 19.7 | .7 |
| Otherb | 419 | 8.6 | .5 |
| Married or living as married | 2,192 | 47.5 | .9 |
| Education | |||
| Did not complete high school | 322 | 7.1 | .5 |
| High school graduate or GED | 1,709 | 33.3 | .8 |
| Some college | 1,785 | 33.6 | .8 |
| College graduate or beyond | 1,368 | 26.0 | .8 |
| Missing | 1 | — | — |
| Employment status | |||
| Employed | 944 | 18.4 | .7 |
| Unemployed | 325 | 7.4 | .4 |
| Out of labor force | 3,908 | 74.3 | .8 |
| Annual income | |||
| ≤$15,000 | 1,751 | 30.1 | .8 |
| $15,001–$30,000 | 1,426 | 27.0 | .8 |
| $30,001–$60,000 | 1,293 | 27.3 | .8 |
| >$60,000 | 424 | 9.5 | .5 |
| Missing | 291 | 6.0 | .4 |
| Service-connected disabilityc | 2,784 | 60.4 | .8 |
| VHA compensation for same diagnosis as cohort | 1,715 | 40.1 | .9 |
| Mental health diagnostic cohort | |||
| Bipolar I disorder | 1,215 | 9.2 | .3 |
| Major depressive disorder | 1,231 | 17.8 | .5 |
| PTSD | 1,306 | 49.4 | .9 |
| Schizophrenia | 1,042 | 11.8 | .4 |
| Substance use disorder cohort | 1,581 | 33.0 | .8 |
| Functioningd | |||
| Mental health (M±SE score) | 35.4±.2 | ||
| Physical health (M±SE score) | 31.5±.2 | ||
| VHA outpatient visits (M±SE)e | 14.7±.5 | ||
| VHA inpatient nights (M±SE)f | 29.6±2.4 |
a
Percentages are weighted.
b
Includes Asian, Native Hawaiian or other Pacific Islander, American Indian/Alaskan Native, multiracial, none of these races, and refused or didn’t know
c
A mental or general medical condition that was incurred or aggravated during active duty
d
Measured by the Veterans RAND 12-Item Health Survey. Possible scores range from 0 to 100 for both mental health and physical health functioning, with higher scores indicating better functioning.
e
Among patients with at least one visit
f
Among patients with a stay of at least one night
Respondents’ mental and physical health functioning scores were approximately 1.5 and 1.9 standard deviations below the general population, respectively. Among patients with at least one outpatient visit or one inpatient night in the past year, patients reported a median of 6.0 outpatient visits and 13.0 inpatient nights, respectively.
Perception of VHA behavioral health care
Table 2 summarizes patient perceptions of timeliness of care, staff’s recovery orientation, psychosocial services, and overall satisfaction. Among patients (61.1%) who reported making routine appointments for counseling or treatment in the past 12 months, nearly half reported always receiving appointments as soon as they wanted. Among the 32.9% of patients who reported making urgent appointments, 42.8% reported always receiving appointments as soon as they wanted. Perceptions of timeliness did not differ by mental health diagnostic cohort. Patients who were treated for a substance use disorder were significantly less likely than patients who were not treated for a substance use disorder to endorse always receiving routine and urgent appointments as soon as they wanted.
| Mental health cohort | Substance use disorder cohortb | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Total (N=5,185) | Bipolar I (N=1,215) | Depression (N=1,231) | PTSD (N=1,306) | Schizophrenia (N=1,042) | Yes (N=1,581) | No (N=3,604) | ||||||||||
| Variable | N | % | N | % | N | % | N | % | N | % | p | N | % | N | % | p |
| Timelinessc | ||||||||||||||||
| Routine care | 1,606 | 49.6 | 401 | 50.4 | 378 | 50.2 | 433 | 49.1 | 295 | 54.3 | .333 | 464 | 45.1 | 1,142 | 51.8 | .005 |
| Urgent care | 750 | 42.8 | 173 | 41.5 | 152 | 42.1 | 200 | 42.6 | 180 | 53.4 | .020 | 242 | 36.4 | 508 | 46.6 | <.001 |
| Staff recovery orientation | ||||||||||||||||
| Listened and respected decisions about care | 2,276 | 43.0 | 571 | 46.8 | 548 | 45.1 | 565 | 42.9 | 452 | 44.2 | .306 | 628 | 39.2 | 1,648 | 44.9 | .002 |
| Encouraged hope and high expectations | 2,080 | 39.7 | 539 | 44.6 | 477 | 38.9 | 509 | 39.2 | 408 | 40.3 | .116 | 661 | 41.4 | 1,419 | 38.9 | .180 |
| Believed patient could make own choices | 1,945 | 37.1 | 506 | 42.7 | 483 | 40.1 | 452 | 35.5 | 376 | 37.5 | .004 | 538 | 33.1 | 1,407 | 39.0 | .001 |
| Helped patient to include others in treatment planning | 1,579 | 31.4 | 400 | 33.4 | 348 | 29.7 | 405 | 32.4 | 323 | 31.2 | .365 | 458 | 28.9 | 1,121 | 32.6 | .033 |
| Asked about patient interests | 1,367 | 26.8 | 334 | 27.9 | 329 | 27.8 | 330 | 26.3 | 275 | 26.7 | .782 | 405 | 26.1 | 962 | 27.1 | .583 |
| Helped to develop and plan for life goals beyond symptom management | 1,353 | 26.1 | 349 | 29.3 | 294 | 24.7 | 327 | 25.6 | 290 | 29.0 | .071 | 418 | 26.1 | 935 | 26.1 | .991 |
| Introduced patient to role models or mentors | 651 | 13.9 | 145 | 12.5 | 111 | 10.1 | 168 | 13.9 | 168 | 17.2 | <.001 | 248 | 17.1 | 403 | 12.2 | <.001 |
| Psychosocial services | ||||||||||||||||
| Perceived need for housing help | 701 | 13.7 | 169 | 14.5 | 153 | 12.9 | 118 | 10.0 | 156 | 15.4 | <.001 | 382 | 24.2 | 319 | 8.6 | <.001 |
| Received housing help if needed | 204 | 27.2 | 48 | 27.7 | 42 | 27.7 | 20 | 14.8 | 53 | 35.6 | <.001 | 138 | 35.9 | 66 | 15.2 | <.001 |
| Perceived need for employment help | 554 | 10.5 | 147 | 12.0 | 128 | 11.0 | 102 | 8.0 | 100 | 9.7 | .003 | 263 | 15.8 | 291 | 7.9 | <.001 |
| Received employment help if needed | 149 | 28.4 | 38 | 25.9 | 23 | 19.5 | 27 | 27.0 | 37 | 37.0 | .108 | 76 | 31.7 | 73 | 25.1 | .182 |
| Satisfaction | 2,233 | 42.3 | 513 | 41.5 | 508 | 41.6 | 562 | 42.4 | 516 | 49.6 | .003 | 587 | 36.4 | 1,646 | 45.1 | <.001 |
a
All percentages are weighted.
b
Patients with a substance use disorder could also be assigned to one mental health diagnostic cohort.
c
Patients who tried to make appointments for counseling or treatment reported always receiving them as soon as requested.
For the seven aspects of recovery orientation, the percentage of patients expressing strong agreement was highest for whether staff listened to and respected decisions about treatment (43.0%) and lowest for whether staff introduced patients to role models (13.9%). Patients with bipolar disorder were most likely to strongly agree that VHA staff believed that patients could make their own life choices, and patients with schizophrenia were most likely to strongly agree that VHA staff introduced them to role models or mentors. Significant differences were found in perceptions of patients with and without a substance use disorder for three of the recovery-orientation measures.
Overall, 13.7% of patients reported needing help finding a place to live, and 10.5% of patients reported needing help finding a job. Among patients who reported a housing need, 27.2% reported receiving help from the VHA. Among patients who reported an employment need, 28.4% reported receiving help from the VHA.
There was significant variation by mental health diagnostic cohort in the proportion of patients reporting a need for housing and employment help and receipt of housing help. Patients in the PTSD cohort were the least likely to report both needs and were also the least likely to report receiving housing help. Patients in the substance use disorder cohort were significantly more likely than patients who were not in the cohort to report needs for housing and employment services and were more likely to receive housing, but not employment, services.
Overall, 42.3% of patients reported being highly satisfied with VHA behavioral health care. Among mental health diagnostic cohorts, patients with schizophrenia were most likely to be highly satisfied with their care (p=.003). Patients with a substance use disorder were significantly less likely than patients without a substance use disorder to report being highly satisfied with their care.
Perceived improvement
Nearly three-quarters (73.9%) of patients reported being helped by their counseling or treatment (Table 3). Fewer patients reported improvement in their ability to deal with daily problems (41.4%), deal with social situations (34.3%), and accomplish goals (30.8%) or improvement of their symptoms (31.7%). The modal response was “about the same” for all four of these items. Patients with schizophrenia were most likely to report improvement in all four areas, whereas patients in the PTSD cohort were least likely to report improvement. Patients in the substance use disorder cohort were more likely than patients who were not in the cohort to report improvements in all four areas.
| Mental health cohort | Substance use disorder cohortb | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Total (N=5,185) | Bipolar I (N=1,215) | Depression (N=1,231) | PTSD (N=1,306) | Schizophrenia (N=1,042) | Yes(N=1,581) | No(N=3,604) | ||||||||||
| Area | N | % | N | % | N | % | N | % | N | % | p | N | % | N | % | p |
| Overall helpfulness of care | 3,875 | 73.9 | 942 | 76.8 | 894 | 72.5 | 970 | 74.1 | 800 | 76.4 | .168 | 1,142 | 71.6 | 2,733 | 75.0 | .043 |
| Daily problems | 2,272 | 41.4 | 586 | 48.4 | 501 | 40.7 | 474 | 36.4 | 519 | 50.6 | <.001 | 742 | 46.3 | 1,530 | 39.0 | <.001 |
| Social situations | 1,901 | 34.3 | 482 | 40.3 | 427 | 34.8 | 370 | 28.5 | 450 | 43.2 | <.001 | 648 | 39.8 | 1,253 | 31.5 | <.001 |
| Problems or symptoms | 1,787 | 31.7 | 452 | 38.5 | 407 | 32.9 | 336 | 26.1 | 440 | 42.0 | <.001 | 623 | 37.1 | 1,164 | 29.1 | <.001 |
| Goals | 1,739 | 30.8 | 458 | 38.1 | 364 | 29.3 | 344 | 26.3 | 431 | 41.3 | <.001 | 598 | 36.4 | 1,141 | 28.1 | <.001 |
a
All percentages are weighted.
b
Patients with a substance use disorder could also be assigned to one mental health diagnostic cohort.
Multivariate model results
Tables 4 and 5 provide the results of logistic regression models that evaluated differences in perceptions of VHA behavioral care and perceived improvement by mental health diagnostic cohort and substance use disorder cohort after the analyses controlled for demographic characteristics, mental and physical health functioning, service utilization, and comorbid substance use or mental disorder.
| Mental health diagnostic cohortb | Substance use disorder cohortc | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Depression | PTSD | Schizophrenia | ||||||||||
| Variable | OR | 95% CI | OR | 95% CI | OR | 95% CI | χ2d | p | OR | 95% CI | χ2e | p |
| Timeliness | ||||||||||||
| Routine care | 1.01 | .81–1.27 | 1.00 | .80–1.25 | 1.12 | .87–1.44 | .98 | .805 | .76 | .55–1.04 | 3.02 | .082 |
| Urgent care | 1.04 | .75–1.44 | 1.01 | .74–1.38 | 1.54 | 1.09–2.18 | 7.74 | .052 | .57 | .37–.89 | 6.20 | .013 |
| Staff recovery orientation | ||||||||||||
| Listened and respected decisions about care | 1.02 | .86–1.23 | 1.01 | .84–1.21 | .91 | .75–1.10 | 1.75 | .625 | .66 | .52–.85 | 10.91 | .001 |
| Encouraged hope and high expectations | .89 | .74–1.07 | .96 | .80–1.16 | .79 | .64–.96 | 6.56 | .087 | .85 | .66–1.08 | 1.78 | .183 |
| Believed patient could make own decisions | 1.00 | .83–1.20 | .95 | .78–1.15 | .84 | .69–1.03 | 3.64 | .303 | .74 | .57–.95 | 5.47 | .019 |
| Helped patient include others in treatment planning | .90 | .74–1.10 | 1.13 | .93–1.39 | .83 | .67–1.03 | 9.69 | .022 | .70 | .53–.92 | 6.73 | .009 |
| Asked about patient interests | 1.11 | .91–1.36 | 1.16 | .94–1.44 | .93 | .75–1.15 | 4.99 | .173 | .82 | .62–1.09 | 1.80 | .179 |
| Helped develop and plan for life goals beyond symptom management | .89 | .72–1.09 | 1.05 | .85–1.30 | .92 | .74–1.14 | 3.13 | .372 | .79 | .59–1.05 | 2.69 | .101 |
| Introduced patient to role models or mentors | .88 | .66–1.18 | 1.49 | 1.12–1.99 | 1.41 | 1.06–1.87 | 16.35 | .001 | 1.15 | .81–1.62 | .59 | .444 |
| Psychosocial services | ||||||||||||
| Perceived need for housing help | .84 | .63–1.11 | .75 | .55–1.02 | 1.08 | .79–1.46 | 6.83 | .077 | 3.08 | 2.24–4.23 | 47.92 | <.001 |
| Received housing help if needed | 1.14 | .66–1.97 | .45 | .23–.87 | 1.34 | .74–2.42 | 10.88 | .012 | 2.31 | 1.34–3.99 | 9.02 | .003 |
| Perceived need for employment help | .99 | .74–1.34 | .79 | .57–1.08 | .81 | .59–1.12 | 3.52 | .318 | 2.37 | 1.71–3.28 | 27.11 | <.001 |
| Received employment help if needed | .83 | .43–1.61 | 1.27 | .63–2.58 | 1.98 | 1.02–3.87 | 6.39 | .094 | 1.49 | .73–3.01 | 1.20 | .273 |
| Satisfaction | 1.02 | .85–1.23 | 1.09 | .90–1.31 | 1.35 | 1.11–1.65 | 10.55 | .014 | .64 | .50–.83 | 11.67 | <.001 |
a
The analyses were adjusted for age, race, sex, service-connected disability, mental and physical health functioning, utilization (outpatient visits and inpatient nights), and co-occurring mental or substance use disorder.
b
Patients with bipolar I disorder are the reference group.
c
Patients without a substance use disorder are the reference group.
d
df=3
e
df=1
| Mental health diagnostic cohortb | Substance use disorder cohortc | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Depression | PTSD | Schizophrenia | ||||||||||
| Area | OR | 95% CI | OR | 95% CI | OR | 95% CI | χ2d | p | OR | 95% CI | χ2e | p |
| Overall helpfulness of care | .87 | .71–1.07 | 1.00 | .80–1.24 | .96 | .76–1.21 | 2.39 | .495 | .67 | .51–.88 | 8.23 | .004 |
| Daily problems | .85 | .70–1.03 | .88 | .72–1.07 | .95 | .77–1.18 | 3.24 | .357 | 1.05 | .82–1.35 | .15 | .696 |
| Social situations | .91 | .74–1.11 | .85 | .69–1.05 | 1.01 | .82–1.24 | 3.42 | .331 | 1.19 | .92–1.54 | 1.72 | .190 |
| Problems or symptoms | .94 | .77–1.14 | .86 | .69–1.06 | 1.09 | .88–1.35 | 5.35 | .148 | 1.11 | .86–1.43 | .67 | .413 |
| Goals | .83 | .67–1.02 | .98 | .79–1.21 | 1.02 | .82–1.27 | 4.68 | .197 | .98 | .75–1.27 | .03 | .859 |
a
The analyses were adjusted for age, race, sex, service-connected disability, mental and physical health functioning, utilization (outpatient visits and inpatient nights), and co-occurring mental or substance use disorder.
b
Patients with bipolar I disorder are the reference group.
c
Patients without a substance use disorder are the reference group.
d
df=3
e
df=1
Compared with patients with bipolar disorder, patients in the PTSD and schizophrenia cohorts had the highest odds of any mental health diagnostic cohort of expressing strong agreement that staff introduced patients to role models or mentors (p<.01) (Table 4). There were no other significant differences across mental health cohorts for timeliness of care, other aspects of recovery orientation, psychosocial services, and satisfaction with care.
Patients with a substance use disorder were significantly less likely than patients without a substance use disorder to endorse two of the recovery-orientation items; remained more likely to report a perceived need for housing and employment services and to report receipt of housing help; and remained significantly less likely to report being highly satisfied with the care they received.
Patients with a substance use disorder remained significantly less likely than those without a substance use disorder to report being helped by the care that they received (Table 5). All other differences in perceived improvement across mental health diagnostic and substance use disorder cohorts were reduced to nonsignificance after the analyses controlled for the covariates.
Discussion
Overall, patient perceptions of behavioral health care delivered in the VHA were favorable, but there was room for significant improvement across all domains. The percentages of patients reporting timeliness of care (49.6%, routine care; 42.8%, urgent care) compared favorably with existing data on the percentages of patients receiving mental health care through both public (40.0% and 38.0%, respectively) and private (42.9% and 42.3%, respectively) insurance plans (28). Overall satisfaction with care (42.3%) was slightly lower than satisfaction with private plans (45.7%) and similar to satisfaction with public plans (43.0%) (28). Although most patients reported that treatment was helpful (73.9%), rates of satisfaction in four specific areas were far lower (30.8%−41.4%). These estimates were lower than the percentages of patients in public and private plans who described their treatment as helpful (83.8% and 87.6%, respectively) and who reported improvement (55.2%−69.7% and 58.9%−81.1%, respectively) (28).
Direct comparison of the data from this sample and data from the literature is challenging because comparison data for matched samples were not available. The comparison samples included a higher proportion of patients who were female, younger, and Caucasian than did our sample of veterans. Although results are mixed across the patient perceptions literature, women and younger patients tend to report less positive ratings, and Caucasians tend to report higher ratings (29–31). Nonetheless, the national estimates of patient perceptions of care presented here could be used as benchmarks with which to compare future health system performance.
Our findings suggested that, in this population, there are approximately 70,000 veterans who may need housing assistance and 54,000 veterans who may need employment assistance nationwide. The multivariate analyses suggested that patients with a substance use disorder were three times more likely to report needing help finding a place to live and twice as likely to report needing help with employment compared with patients without a substance use disorder.
Patients with a substance use disorder reported lower satisfaction with care and perceived their treatment to be less helpful than patients without a substance use disorder. What contributed to these differences is unknown, but one possibility is that the diagnosis was associated with variation in expectations for care, types of care received, or interactions between providers and patients. Patients with a substance use disorder may have less favorable perceptions of care if providers interact less warmly or provide lower quality of care, particularly if providers have negative biases toward patients with these diagnoses (32,33). Understanding more about factors that contribute to variations in patient perceptions of care, particularly for patients with substance use disorders, could inform quality improvement efforts.
The differences in recovery-orientation ratings by persons with and without a substance use disorder suggest areas for quality improvement and improvements in overall satisfaction and perceived helpfulness of treatment. Specifically, patients with substance use disorders were significantly less likely than patients without substance use disorders to report that staff listened to and respected their decisions and helped to include other important people in treatment planning. This finding suggests that staff training in a recovery-orientation model or in motivational interviewing (a patient-centered approach that emphasizes active listening) (34) could be helpful. The VHA has recently increased its focus on patient-centered care (35), which could lead to improvements in perceptions of care. To reduce the differences in the extent to which staff introduce patients in different mental health diagnostic cohorts to role models, it may be helpful to increase availability of mutual support groups (36). Unfortunately, reports of patients’ perceptions of care have been criticized for not providing clear direction on how these variables can be improved (37), suggesting that this is an area for future work.
Our study had limitations. Although patient perceptions of care are valuable, they are subject to limitations similar to those of other data gathered through patient self-report (38). For example, patients may have difficulty recalling the care they received, differentially weight the value of positive and negative experiences, differentially weight recent versus distal experiences, or base their perceptions on varying expectations for their care. In addition, because diagnostic cohorts were identified through administrative data, they may not accurately represent diagnoses assigned through standardized clinical assessments.
In this study, we focused on a broad range of patient perception domains. Although this approach allowed us to characterize patient perceptions of VHA behavioral health care, it limited our ability to thoroughly assess each domain. Further, most of the variables we examined do not have benchmarks of acceptable performance, and matched sample comparison data were not available; however, prior work suggests that quality improvement efforts can lead to improvements in patient perceptions of care (39,40). Nonetheless, this study extended prior work by focusing on a large, national sample of veterans who received behavioral health care and examining a broad range of patient perceptions of their behavioral health care.
Conclusions
Patient perceptions of care represent a potentially important assessment of health care quality. This study provided national estimates of patient perceptions of care for mental and substance use disorders among veterans. Further, it highlights the potential importance of diagnosis in assessing patient perceptions of care.
Acknowledgments and disclosures
This project was funded by the U.S. Department of Veterans Affairs (VA) (contract number GS 10 F-0261K, 101-G67214/101-G67215; program evaluation of Veterans Health Administration Mental Health Services; and the VA Substance Use Disorder Quality Enhancement Research Initiative–SUDQ-LIP 1206). Dr. Blonigen was supported by a Career Development Award–2 from the VA Office of Research and Development (Clinical Sciences Research and Development).The authors thank Q. Burkhart, M.S., of the RAND Corporation for conducting study analyses. The opinions expressed here are the authors’ and do not necessarily represent the views of the VA or any other entity of the U.S. government.
Dr. Pincus has received research funding from the Robert Wood Johnson Foundation, the John A. Hartford Foundation, the Heinz Endowments, the Atlantic Philanthropies, the UPMC Health Plan (Community Care Behavioral Health), the Highmark Foundation, the Staunton Farm Foundation, the FISA Foundation, the Eden Hall Foundation, the SCAN Foundation, and the Graham Boeckh Foundation; has served as a consultant or as a member of the advisory board for Value Options, Altarum Institute, and Manila Consulting; and has been paid for presentations by Johnson and Johnson, the Lifespan Health System, and Alberta Innovates Health Solutions. The other authors report no competing interests.
References
1.
Institute of Medicine: Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC, National Academies Press, 2001
2.
Black N: Patient reported outcome measures could help transform healthcare. BMJ 346:f167, 2013
3.
Golin CE, DiMatteo MR, Gelberg L: The role of patient participation in the doctor visit: implications for adherence to diabetes care. Diabetes Care 19:1153–1164, 1996
4.
Jha AK, Orav EJ, Zheng J, et al.: Patients’ perception of hospital care in the United States. New England Journal of Medicine 359:1921–1931, 2008
5.
Taylor DM, Wolfe R, Cameron PA: Complaints from emergency department patients largely result from treatment and communication problems. Emergency Medicine 14:43–49, 2002
6.
Fremont AM, Cleary PD, Hargraves JL, et al.: Patient-centered processes of care and long-term outcomes of myocardial infarction. Journal of General Internal Medicine 16:800–808, 2001
7.
Guldvog B: Can patient satisfaction improve health among patients with angina pectoris? International Journal for Quality in Health Care 11:233–240, 1999
8.
Kane RL, Maciejewski M, Finch M: The relationship of patient satisfaction with care and clinical outcomes. Medical Care 35:714–730, 1997
9.
Maly RC, Bourque LB, Engelhardt RF: A randomized controlled trial of facilitating information giving to patients with chronic medical conditions: effects on outcomes of care. Journal of Family Practice 48:356–363, 1999
10.
Stewart M, Meredith L, Brown JB, et al.: The influence of older patient-physician communication on health and health-related outcomes. Clinics in Geriatric Medicine 16:25–36, 2000
11.
Fenton JJ, Jerant AF, Bertakis KD, et al.: The cost of satisfaction: a national study of patient satisfaction, health care utilization, expenditures, and mortality. Archives of Internal Medicine 172:405–411, 2012
12.
Ouimette P, Wade M, Tirone V, et al.: Perceived barriers to care among Veterans Health Administration patients with posttraumatic stress disorder. Psychological Services 8:212–223, 2011
13.
Rosenheck R, Wilson NJ, Meterko M: Influence of patient and hospital factors on consumer satisfaction with inpatient mental health treatment. Psychiatric Services 48:1553–1561, 1997
14.
Burnett-Zeigler I, Zivin K, Ilgen MA, et al.: Perceptions of quality of health care among veterans with psychiatric disorders. Psychiatric Services 62:1054–1059, 2011
15.
Holcomb WR, Parker JC, Leong GB, et al.: Customer satisfaction and self-reported treatment outcomes among psychiatric inpatients. Psychiatric Services 49:929–934, 1998
16.
Desai RA, Stefanovics EA, Rosenheck RA: The role of psychiatric diagnosis in satisfaction with primary care: data from the Department of Veterans Affairs. Medical Care 43:1208–1216, 2005
17.
Watkins KE, Pincus HA, Paddock S, et al.: Care for veterans with mental and substance use disorders: good performance, but room to improve on many measures. Health Affairs 30:2194–2203, 2011
18.
Rosenheck RA, Fontana AF: Recent trends in VA treatment of post-traumatic stress disorder and other mental disorders. Health Affairs 26:1720–1727, 2007
19.
Watkins KE, Smith B, Paddock SM, et al.: Veterans Health Administration Mental Health Program Evaluation: Capstone Report. Santa Monica, Calif, RAND, 2011
20.
Daniels A, Shaul J, Greenberg P, et al.: The Experience of Care and Health Outcomes (ECHO) survey: a consumer survey to collect ratings of behavioral health care treatment, outcomes and plans; in The Use of Psychological Testing for Treatment Planning and Outcomes Assessment. Edited by, Maruish ME. Mahwah, NJ, Erlbaum, 2004
21.
Iqbal S, Rogers W, Selim A, et al.: The Veterans RAND 12 Item Health Survey (VR-12): What It Is and How It Is Used. Bedford, Mass, Veterans Administration Medical Center, Center for Health Quality, Outcomes and Economic Research, and Boston University School of Public Health, Center for the Assessment of Pharmaceutical Practices, 2007
22.
O’Connell M, Tondora J, Croog G, et al.: From rhetoric to routine: assessing perceptions of recovery-oriented practices in a state mental health and addiction system. Psychiatric Rehabilitation Journal 28:378–386, 2005
23.
National Consensus Statement on Mental Health Recovery. Rockville, Md, US Department of Health and Human Services Substance Abuse and Mental Health Services Administration, 2006
24.
Hays RD, Cunningham WE, Sherbourne CD, et al.: Health-related quality of life in patients with human immunodeficiency virus infection in the United States: results from the HIV Cost and Services Utilization Study. American Journal of Medicine 108:714–722, 2000
25.
Cochran W (ed): Sampling Techniques, 3rd ed. London, Wiley, 1977
26.
Little RJA: Survey Nonresponse Adjustments for Estimates of Means. International Statistical Review 54:139–157, 1986
27.
Benjamini Y, Hochberg Y: Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society 1:289–300, 1995
28.
Shaul J, Eisen S, Clarridge B, et al. Experience of Care and Health Outcomes (ECHO) Survey: Field Test Report and Survey Evaluation. Boston, Harvard Medical School, Department of Health Care Policy, 2001
29.
Elliott MN, Lehrman WG, Beckett MK, et al.: Gender differences in patients’ perceptions of inpatient care. Health Services Research 47:1482–1501, 2012
30.
Eselius LL, Cleary PD, Zaslavsky AM, et al.: Case-mix adjustment of consumer reports about managed behavioral health care and health plans. Health Services Research 43:2014–2032, 2008
31.
Goldstein E, Elliott MN, Lehrman WG, et al.: Racial-ethnic differences in patients’ perceptions of inpatient care using the HCAHPS survey. Medical Care Research and Review 67:74–92, 2010
32.
Amaral-Sabadini MB, Saitz R, Souza-Formigoni ML: Do attitudes about unhealthy alcohol and other drug (AOD) use impact primary care professionals’ readiness to implement AOD-related preventive care? Drug and Alcohol Review 29:655–661, 2010
33.
McCormick KA, Cochran NE, Back AL, et al.: How primary care providers talk to patients about alcohol: a qualitative study. Journal of General Internal Medicine 21:966–972, 2006
34.
Hettema J, Steele J, Miller WR: Motivational interviewing. Annual Review of Clinical Psychology 1:91–111, 2005
35.
Kuehn BM: Veterans health system cited by experts as a model for patient-centered care. JAMA 307:442–443, 2012
36.
Davidson L, Chinman M, Kloos B, et al.: Peer support among individuals with severe mental illness: a review of the evidence. Clinical Psychology: Science and Practice 6:165–187, 1999
37.
Geissler KH, Friedberg MW, Steelfisher GK, et al.: Motivators and barriers to using patient experience reports for performance improvement. Medical Care Research and Review 70:621–635, 2013
38.
Kupfer JM, Bond EU: Patient satisfaction and patient-centered care: necessary but not equal. JAMA 308:139–140, 2012
39.
Edgman-Levitan S, Shaller D: The CAHPS Improvement Guide: Practical Strategies for Improving Patient Care Experience. Boston, Harvard Medical School, 2003
40.
Davies E, Shaller D, Edgman-Levitan S, et al.: Evaluating the use of a modified CAHPS survey to support improvements in patient-centred care: lessons from a quality improvement collaborative. Health Expectations 11:160–176, 2008
Information & Authors
Information
Published In
Cover: Deep Cove Lobster Man, by N. C. Wyeth, ca. 1938. Oil on gessoed board (renaissance panel). Accession number 1939.16. Courtesy of the Pennsylvania Academy of Fine Arts, Philadelphia, Joseph E. Temple Fund.
History
Published in print: August 2014
Published online: 15 October 2014
Authors
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBGet Access
Login options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).