System Transformation Under the California Mental Health Services Act: Implementation of Full-Service Partnerships in L.A. County
Abstract
Objective:
The study evaluated the effect of California’s Mental Health Services Act (MHSA) on the structure, volume, location, and patient centeredness of Los Angeles County public mental health services.
Methods:
This prospective mixed-methods study (2006–2013) was based in five Los Angeles County public mental health clinics, all with usual care and three with full-service partnerships (FSPs). FSPs are MHSA-funded programs designed to “do whatever it takes” to provide intensive, recovery-oriented, team-based, integrated services for clients with severe mental illness. FSPs were compared with usual care on outpatient services received (claims data) and on organizational climate, recovery orientation, and provider-client working alliance (surveys and semistructured interviews), with regression adjustment for client and provider characteristics.
Results:
In the first year after admission, FSP clients (N=174) received significantly more outpatient services than did usual care clients (N=298) (5,238 versus 1,643 minutes, p<.001), and a larger proportion of these services were field based (22% versus 2%, p<.001). Compared with usual care clients, FSP clients reported more recovery-oriented services (p<.001) and a better provider-client working alliance (p=.01). Compared with usual care providers (N=130), FSP providers (N=42) reported more stress (p<.001) and lower morale (p<.001).
Conclusions:
Los Angeles County’s public mental health system was able to transform service delivery in response to well-funded policy mandates. For providers, a structure emphasizing accountability and patient centeredness was associated with greater stress, despite smaller caseloads. For clients, service structure and volume created opportunities to build stronger provider-client relationships and address their needs and goals.
The Affordable Care Act (ACA) (1) is transforming health care delivery throughout the United States, increasing access for previously unserved populations and encouraging health care systems to provide coordinated, patient-centered care for chronic conditions to improve outcomes and reduce costs (http://innovation.cms.gov). For California’s public mental health system, large-scale transformation began earlier, in 2004, when voters passed Proposition 63, the Mental Health Services Act (MHSA) (2). The MHSA aims to address decades of shrinking mental health budgets, overreliance on emergency and hospital services, homelessness, and incarceration. Its 1% tax on personal incomes over $1 million has yielded average allocations of $1.3 billion annually, which represents 26% of funding for California’s public mental health system (fiscal year [FY] 2007–2008 to FY 2012–2013) (3). Funds are restricted to developing and providing new services aligned with recovery principles: recovery from mental illness is possible; services are strengths based, not deficits focused; and services are provided with a patient-centered focus on clients’ treatment goals (4).
Full-service partnerships (FSPs) are the core of MHSA’s recovery-oriented system transformation and receive the plurality of funds (4–6). FSPs use a modified version of assertive community treatment (7). They provide intensive, integrated services for clients with severe mental illness who are unserved, underserved, or inappropriately served by existing services, with the aim of reducing hospitalizations, incarcerations, and homelessness (4).
In Los Angeles (LA) County, which represents 25% of California’s population (8), FSPs must have client-to-staff ratios ≤15:1, as well as treatment teams with a psychiatrist or other prescribing clinician, social workers or other mental health providers, housing and employment specialists, and client advocates; and services must be available 24/7 in the field (home or community) or clinic (9,10). Authorization for FSP enrollment occurs centrally, following community outreach to identify and recruit unserved clients with extensive prior-year homelessness (six months or more), incarceration (two or more episodes of at least 30 days total), inpatient psychiatric treatment (28 or more days of acute care or six months or more in an institution for mental disease [IMD] or a state hospital), or emergency psychiatric treatment (ten or more episodes), or dependent on family and at risk of these outcomes. Transfers from usual care are limited to 20% of FSP slots, with written justification of being underserved or inappropriately served by usual services (11,12).
MHSA seeks to make services more available (volume, location, and immediacy) but also recovery oriented and patient centered. The latter aims, representative of contemporary trends, cannot be evaluated without surveying stakeholders. Our 2006–2013 study of LA County’s MHSA implementation applied a prospective, mixed-methods design to three levels: system-level policies, clinic-level factors, and client outcomes. It evaluated implementation and process questions not addressed by previous evaluations of California’s MHSA or precursor programs (13–18), combining ethnography data, provider survey and interview data, and data from prospectively followed clients (FSP and usual care) with a quasi-experimental design, adjustment for regression to the mean, and administrative data (utilization), survey data (homelessness, incarceration, and symptoms/functioning), and interview data.
System intervention evaluations have found that system-level transformation may not affect client-level outcomes (19–24). One concluded, “Actions at the system level must express themselves through changes in treatment and subsequent clinician-client interactions” (22). Evaluation of intermediate clinic-based factors, although crucial to determining what facilitates or impedes system-to-client impacts, often is lacking. Oversight of MHSA has been no exception. The Little Hoover Commission and the California state auditor both issued reports critiquing the extent to which state agencies and counties required, planned for, and complied with tracking of MHSA implementation goals and outcomes (25,26).
With this study, we filled a gap in this oversight, asking whether the LA County Department of Mental Health (LACDMH), with more than 250,000 clients annually (27) and more than 4,000 adult FSP slots (28), successfully transformed the structure of care, and, if so, how changes had an impact on client and provider experiences. Comparing FSP with usual care, we hypothesized that FSP clients would receive more outpatient and field-based services. Care would be more patient centered, with FSP clients and providers evaluating services as more recovery oriented and FSP clients reporting stronger client-provider working alliances. FSP providers would report greater engagement and higher morale, with stress levels lower (smaller caseloads) or higher (“do whatever it takes”). Subsequent studies will evaluate whether clinic-level changes improved client outcomes.
Methods
Sites and Sample
LACDMH operates 15 adult clinics and has contracts with operators of 51 additional clinics. We employed a quasi-experimental design in five clinics: four operated by LACDMH (two with FSP and usual care and two similar clinics that provided usual care only) and one contracted clinic (with FSP and usual care). We approached all treatment providers for three annual surveys (2007–2010). Forty-two FSP and 130 usual care providers completed one to three surveys (75%−77% response rate). We approached all FSP clients either at study onset or at the time of clinic admission. Of 222 eligible clients, 172 (77%) agreed to participate and 15 (7%) refused; repeated contact and scheduling efforts failed with 35 (16%).
To construct a usual care sample, we screened all preintake assessment forms by using approximate FSP criteria: a primary diagnosis of severe mental illness; Global Assessment of Functioning (GAF) score of ≤55 (29); and prior-12-month history of hospitalization, emergency services, homelessness, incarceration, or family dependency. We approached clients close to admission, excluding those who had completed more than five visits. Of 617 eligible clients, 298 (48%) agreed to participate and 78 (13%) refused; repeated contact and scheduling efforts failed with 241 (39%).
The resulting sample consisted of 174 FSP clients and a nonequivalent comparison group of 298 usual care clients. Admission dates ranged from December 2006 to December 2009. Because recruitment was in person, clients who neither enrolled nor refused tended to be no-shows for clinic appointments; in usual care, many never engaged in treatment. After complete description of the study to potential participants, we obtained written informed consent for the survey and interview and HIPAA authorization for access to clinical data. Per human subjects protocol, clients under conservatorship or too ill to consent were ineligible.
Data Collection and Measures
Clients were followed for three years. Data included clinical and utilization data from LACDMH’s administrative database (2006–2013) (30,31), client and provider surveys and semistructured interviews, and more than 6,000 hours of clinic ethnography data (2007–2011). Institutional review boards at the University of California, Los Angeles, and RAND and LACDMH’s Human Subjects Research Committee approved the study.
LACDMH’s administrative database includes demographic, utilization, billing, and reimbursement data, including publicly funded state or fee-for-service hospitalizations and all mental health care provided or contracted by LACDMH: outpatient, day rehabilitation, emergency department, mobile response team, urgent care, acute inpatient hospitalization, IMDs, residential hospital alternatives (IMD step down and crisis residential), and jail mental health services.
Client surveys were administered at enrollment and every six months for up to three years, in English or Spanish, with in-person or phone follow-up surveys continuing after treatment discontinuation. This yielded 1,700 surveys (727 FSP and 973 usual care), with self-report, multiple-choice instruments and structured interviews assessing homelessness and incarceration. Provider multiple-choice surveys were administered yearly for up to three years, yielding 311 surveys (80 FSP and 231 usual care).
We interviewed 103 clients (41 FSP and 62 usual care) and 108 providers (21 FSP, 63 usual care, four clinic-wide, and 20 other programs), with yearly follow-up. Interviews (252 clients and 232 providers) were recorded, transcribed, and coded, yielding more than 14,000 discretely coded excerpts.
Mental health services utilization.
Utilization was aggregated by type: outpatient or day rehabilitation minutes, emergency or urgent care minutes, acute days (acute inpatient or crisis residential), and long-term days (IMD or IMD step down).
Homelessness and incarceration.
From clients’ retrospective reports of their day-by-day living situations and data on jail mental health services, we calculated days homeless and days incarcerated for six-month periods. We defined homelessness as street, vehicle, or temporary-shelter residence, excluding residence with family or friends or long-term programs (for example, transitional housing).
Organizational climate.
We evaluated providers’ work environments by using individual-level climate measures (engagement, functionality, and stress) and measures of work attitudes (morale) from the Organizational Social Context Measure (reliability alphas, .78–.94). Results are nationally normed T scores (mean±SD=50±10) (32). Provider interviews included probes about MHSA-related clinic changes.
Recovery orientation.
We assessed client- and provider-perceived recovery orientation of services with the Recovery Self-Assessment Scale, Revised (RSA-R) overall and factor scores: life goals, involvement, diversity of treatment options, choice, individually tailored services, and whether services are inviting (33). RSA-R scores derive from client or provider agreement with the presence of recovery-oriented program features on a scale from 1, strongly disagree, to 5, strongly agree. Provider interviews included probes about recovery and MHSA-related work practices.
Client-provider working alliance.
We assessed client-reported working alliance—“extent to which a client and therapist work collaboratively and purposefully and connect emotionally”—with the Working Alliance Inventory, Short (WAI-S) (subscales for goals, tasks, and bond; internal consistency alphas of .90–.92 for the subscales and .98 for the full scale) (34). Scores evaluate client perceptions of the presence of a strong client-provider working alliance on a scale from 1, strongly disagree, to 5, strongly agree. Client and provider interviews included probes about client-provider relationships and treatment.
Analyses
Client analyses adjusted for clinic type, centered admission date (alone and FSP-interacted), and proxies for baseline illness severity, propensity or capacity for service use, and likelihood of arrest. These proxy variables include admission diagnosis, diagnosis of a substance use disorder, and GAF score; prior-year mental health service utilization (outpatient, emergency or urgent, acute inpatient, and long-term inpatient); prior-six-month homelessness and incarceration; and demographic factors (sex, age, race-ethnicity, and language). Survey analyses also adjusted for centered days between admission and survey (alone and FSP-interacted), and excluded surveys completed fewer than 60 days after clinic admission to ensure meaningful service evaluations.
Provider survey analyses adjusted for clinic type, training (M.D. or nurse practitioner, registered nurse or licensed vocational nurse, master’s or doctorate degree, or no advanced degree), and survey date.
All analyses were in Stata 14, with standard error adjustment for within-clinic clustering. Adjusted means for FSP and usual care are predictive margins with delta-method confidence intervals.
Outpatient utilization.
Time in program and program switching during the three-year follow-up were examined by using admission, discharge, and service use data. For service volume, outpatient mental health service use during the first year after clinic admission was regressed on FSP, with covariate adjustment as above, for the full sample and a complete-year subsample. Monthly adjusted utilization was calculated for the subsample to identify service intensity changes over time. Because of adjustment for prior-year values for each utilization category, FSP versus usual care differences are equivalent to difference-in-differences estimates.
Organizational climate, recovery orientation, and working alliance.
We hypothesized that evaluations of organizational climate, recovery orientation, and working alliance would be more positive in FSPs, with differences increasing over time as FSPs matured or, conversely, decreasing as transformations diffused to usual care. Because of clustering (surveys within individual and individuals within clinic), we examined overall FSP versus usual care differences by using random effects (Stata’s mixed), with random intercept for individual and standard error adjustment for within-clinic clustering. We examined within-program change over time by using the same model with FSP-year interactions (for providers, adding FSP-year interactions to existing year covariates; for clients, adding year and FSP-year interactions in lieu of admission date covariates) and calculating within-program year-over-year change by using Stata’s lincom and year-specific FSP versus usual care differences by using Stata’s margins.
Qualitative findings emerged from grounded-theory thematic analysis of ethnographic, interview, and focus group data (35).
Results
Baseline Characteristics
Provider characteristics.
Providers represented a range of training and clinic roles. The proportion of providers with no advanced degree was larger in the FSP sample than in the usual care sample (44% versus 21%, p=.003) (Table 1).
| FSP | Usual care | |||||
|---|---|---|---|---|---|---|
| Characteristic | N | % | N | % | p | |
| Providers | ||||||
| Clinic type | .001 | |||||
| LACDMH directly operated | 20 | 47 | 96 | 74 | ||
| Operated under LACDMH contract | 23 | 53 | 34 | 26 | ||
| Female | 26 | 60 | 78 | 60 | .96 | |
| Training | .02 | |||||
| M.D. or nurse practitioner | 5 | 12 | 25 | 19 | .25 | |
| R.N. or L.V.N. | 4 | 9 | 12 | 9 | .99 | |
| Ph.D., Psy.D., or master’s degree | 15 | 35 | 66 | 51 | .07 | |
| No advanced degree | 19 | 44 | 27 | 21 | .003 | |
| Clients | ||||||
| LACDMH directly-operated clinic | 118 | 68 | 244 | 82 | <.001 | |
| Female | 72 | 41 | 157 | 53 | .02 | |
| Age at baseline (M±SD) | 41±10.1 | 41±9.8 | .62 | |||
| Age group at baseline | .27 | |||||
| 18–29 | 33 | 19 | 55 | 18 | .89 | |
| 30–49 | 98 | 56 | 187 | 63 | .17 | |
| 50–67 | 43 | 25 | 56 | 19 | .13 | |
| Non–English speaking | 18 | 10 | 33 | 11 | .81 | |
| Race-ethnicity | .19 | |||||
| Non-Hispanic white | 77 | 44 | 122 | 41 | .48 | |
| Non-Hispanic black | 45 | 26 | 60 | 20 | .15 | |
| Hispanic | 44 | 25 | 102 | 34 | .04 | |
| Other | 8 | 5 | 14 | 5 | .96 | |
| Primary diagnosis at admission | .03 | |||||
| Bipolar disorder | 53 | 30 | 74 | 25 | .18 | |
| Major depression | 36 | 21 | 76 | 26 | .24 | |
| Other mood or anxiety disorder | 19 | 11 | 46 | 15 | .12 | |
| Schizophrenia | 18 | 10 | 37 | 12 | .50 | |
| Schizoaffective disorder | 33 | 19 | 30 | 10 | .01 | |
| Other psychosis | 15 | 9 | 35 | 12 | .29 | |
| Secondary diagnosis of a substance use disorder at admission | 45 | 26 | 85 | 29 | .53 | |
| Global Assessment of Functioning at admission (M±SD)b | 40±8.4 | 43±7.8 | <.001 | |||
a
LACDMH, Los Angeles County Department of Mental Health; R.N., registered nurse; L.V.N., licensed vocational nurse
b
Possible scores range from 0 to 100, with higher scores indicating better functioning.
Client characteristics.
The FSP and usual care client samples were similar demographically, except that the percentage of women was higher in the usual care sample (53% versus 41%, p=.02) as was the percentage of Hispanics (34% versus 25%, p=.04) (Table 1). The prevalence rates of admission diagnoses and comorbid substance use disorders were similar; the proportion of clients with schizoaffective disorder was larger in the FSP sample (19% versus 10%, p=.01). Differences in functioning at admission were significant but small (GAF=40 for the FSP sample and 43 for the usual care sample, p<.001).
Preadmission homelessness, incarceration, and service utilization.
In the six months before clinic admission, a greater proportion of FSP clients than usual care clients were homeless (40% versus 30%, p=.02) and a greater proportion were incarcerated (24% versus 14%, p=.01) (Table 2). Among those incarcerated, FSP clients spent more days incarcerated (73 versus 47, p=.03). Among those who were homeless, the duration of homelessness did not differ between the samples.
| Clients | Days or minutesa | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Circumstance or utilization category | FSP (N=174) | Usual care (N=298) | FSP | Usual care | ||||||
| N | % | N | % | pb | M | SD | M | SD | pb | |
| Adverse circumstance, prior 6 months | ||||||||||
| Homelessness (days) | 70 | 40 | 88 | 30 | .02 | 92 | 68 | 106 | 71 | .23 |
| Incarceration (days) | 41 | 24 | 41 | 14 | .01 | 73 | 63 | 47 | 42 | .03 |
| Utilization, prior 12 months | ||||||||||
| Outpatient care (minutes) | ||||||||||
| Outpatientc | 90 | 52 | 77 | 26 | <.001 | 1,092 | 1,373 | 975 | 1,944 | .65d |
| Day rehabilitation | 2 | 1 | 0 | — | .14 | 6,360 | 6,958 | na | na | na |
| Emergency or urgent care (minutes) | ||||||||||
| Psychiatric emergency department | 43 | 25 | 54 | 18 | .09 | 1,291 | 1,338 | 1,581 | 1,593 | .34 |
| Psychiatric mobile response | 30 | 17 | 29 | 10 | .02 | 498 | 419 | 459 | 503 | .75 |
| Psychiatric urgent care | 35 | 20 | 66 | 22 | .60 | 504 | 1,026 | 323 | 296 | .19 |
| Inpatient care (days) | ||||||||||
| Acute inpatient | 58 | 33 | 84 | 28 | .24 | 25 | 31 | 12 | 16 | .002 |
| Institution for mental disease (IMD) | 6 | 3 | 9 | 3 | .80 | 58 | 60 | 26 | 52 | .29 |
| IMD step down | 9 | 5 | 0 | — | <.001 | 66 | 121 | na | na | na |
| Crisis residential | 5 | 3 | 0 | — | .007 | 19 | 13 | na | na | na |
a
Among those with any days or minutes
b
From Pearson’s chi-square test, except where Fisher’s exact test was used because of small cell sizes (day rehabilitation, IMD step-down, and crisis residential)
c
Among clients with outpatient utilization, the percentage of visits that were field based averaged 3.8%±15% for FSP and 2.8%±15% for usual care (p=.48).
d
Although mean minutes of utilization did not differ, the median number of minutes was 723 for FSP (interquartile range [IQR]=367–1,349) and 459 for usual care (IQR=129–928).
In the 12 months before clinic admission, a larger percentage of FSP than usual care clients received outpatient services (52% versus 26%, p<.001) and mobile response services (17% versus 10%, p=.02). Among clients who received outpatient services, FSP and usual care clients did not differ in the minutes of services received or the percentage of services that were field based. The percentage hospitalized did not differ between samples, but among those hospitalized, FSP clients had more hospital days (25 versus 12, p=.002). Only FSP clients received IMD step-down services (5%), crisis residential care (3%), and day rehabilitation services (1%).
FSP and Usual Care Outpatient Services
Of 174 FSP clients, 66 (38%) remained in their initial treatment episode for the three-year follow-up, and 81 (47%) received FSP-only services for less than three years, with an initial episode lasting a mean of 493±269 days. The other 27 FSP clients (16%) had an initial episode lasting a mean of 581±294 days, after which they were enrolled in non-FSP services.
Of 298 usual care clients, 89 (30%) remained in their initial treatment episode for the three-year follow-up, and 190 (64%) received non-FSP-only services for less than three years, with an initial episode lasting a mean of 374±234 days. The other 19 usual care clients (6%) had an initial episode lasting a mean of 325±228 days, after which they were enrolled in an FSP.
In the first year after clinic admission, FSP and usual care clients received dramatically different covariate-adjusted volumes of outpatient services (Table 3). FSP clients averaged 5,238 minutes of such services, compared with 1,643 for usual care clients (p<.001); the differences were seen primarily in case management (1,930 versus 231 minutes, p<.001) and rehabilitation (1,695 versus 454 minutes, p=.001), and more visits were recorded as field based (22% versus 2%, p<.001). FSP clients received significantly more medication management, crisis intervention, and case consultation services; however, the differences in minutes of collateral services, no-contact services, diagnostic services, therapy, or day rehabilitation were not significant.
| FSP (N=174) | Usual care (N=298) | Difference (FSP–usual care)b | |||||
|---|---|---|---|---|---|---|---|
| Variablec | Standardized M | 95% CI | Standardized M | 95% CI | Coeff. | 95% CI | p |
| Minutes | |||||||
| Case management | 1,930 | 1,303 to 2,556 | 231 | 45 to 416 | 1,706 | 1,259 to 2,153 | <.001 |
| Rehabilitation | 1,695 | 1,202 to 2,187 | 454 | 225 to 683 | 1,189 | 853 to 1,525 | .001 |
| Medication visit | 449 | 426 to 471 | 301 | 212 to 391 | 146 | 37 to 254 | .02 |
| Crisis intervention | 156 | 109 to 202 | 40 | –7 to 86 | 117 | 35 to 199 | .02 |
| Case consultation | 160 | 108 to 213 | 41 | 15 to 68 | 115 | 39 to 191 | .01 |
| Collateral | 76 | 2 to 150 | 20 | 3 to 37 | 57 | –30 to 145 | .14 |
| No contact | 138 | 76 to 200 | 40 | –5 to 84 | 99 | –10 to 209 | .07 |
| Diagnostic services | 176 | 143 to 210 | 189 | 156 to 222 | –7 | –70 to 56 | .77 |
| Therapy | 345 | 261 to 428 | 261 | 100 to 421 | 85 | –20 to 189 | .09 |
| Day rehabilitation | 105 | –135 to 345 | 73 | –48 to 193 | 24 | –318 to 366 | .85 |
| Total | 5,238 | 4,569 to 5,908 | 1,643 | 1,433 to 1,852 | 3,546 | 2,758 to 4,334 | <.001 |
| Visits | |||||||
| N of field-based visits | 17.8 | 13.3 to 22.3 | 1.1 | 0 to 2.2 | 16.1 | 11.6 to 20.6 | .001 |
| N of all visits | 80.4 | 77.8 to 82.9 | 28.5 | 25.9 to 31.1 | 51.2 | 47.5 to 54.8 | <.001 |
| % of all visitsd | 22 | 18 to 25 | 2 | 0 to 4 | 19 | 17 to 22 | <.001 |
a
Regressions for service minutes are adjusted for prior-12-month outpatient service minutes (in specific outcome category, if applicable, and total of all other outpatient utilization categories) in addition to other covariates. Regressions for number or percentage of visits field based are adjusted for prior-12-month number or percentage of visits in addition to other covariates. Other covariates for all regressions are prior-12-month emergency or urgent minutes, acute inpatient days, and long-term inpatient days; prior-six-month days homeless and days incarcerated; sex; age at baseline; race-ethnicity; whether non–English speaking; Global Assessment of Functioning score, primary psychiatric diagnosis, and comorbid substance use disorder diagnosis at admission; admission date (centered at July 1, 2008), as a main effect and interacted with FSP status; and whether enrolled in a clinic operated by the Los Angeles County Department of Mental Health or a contracted clinic. Confidence intervals are adjusted for within-clinic clustering.
b
Differences are regression coefficients and are equivalent to difference-in-differences estimates due to adjustment for prior-12-month value of the outcome variable. Standardized means are predictive margins (Stata 14 margins command) with delta-method confidence intervals.
c
Rehabilitation includes individual and group services. No-contact services include record writing and record review. Collateral visits are held with individuals from the client’s life, such as family members. Therapy includes family, individual, or group therapy.
d
Mean percentage of visits is not mathematically equivalent to percentage of mean visits.
Sensitivity analysis restricted to clients with an initial episode of at least one year (FSP, N=137, 79%; usual care, N=185, 62%) yielded similar results (not shown). Figure 1 shows adjusted monthly utilization for this complete-year subsample. For both samples, intensive initial services tapered off to maintenance levels. FSP services were more intensive than usual care throughout (month 1, 766 versus 332 minutes, p=.02; average for months 2–12, 442 versus 139 minutes, p<.001).
Organizational Climate
FSP and usual care providers reported similar organizational engagement and functionality (Table 4). FSP providers reported significantly higher stress (score of 55.0 versus 51.3, p<.001) and lower morale (48.1 versus 49.6, p<.001), despite having small caseloads. These differences emerged in FY 2008–2009 and persisted into FY 2009–2010. One provider said, “You can never do enough for these clients. The FSP slogan, ‘Whatever it takes. . .’—that is so innovative and you can really make a difference. . . . But eventually that slogan wears on you. That’s an incredible amount of responsibility to put on us. . . . A caseload of 15 sounds so small, coming from outpatient clinics of 200, but look at what we’re trying to do. . . . In outpatient . . . I would only know half of them. I would have a few that met with me for therapy. Other than that, I would see my clients once a month. If they didn’t show up, I’d give them a call once. If they didn’t answer, oh well. . . . [With FSP] I might have to show up at their door. . . . We’re constantly checking to see if they’ve been hospitalized; checking to see if they’ve been put into jail.”
| Provider assessmenta | Client assessmentb | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| FSP | Usual care | Differencec | FSP | Usual care | Differencec | ||||||||||
| Measure | Standardized M | 95% CI | Standardized M | 95% CI | Coeff. | 95% CI | p | Standardized M | 95% CI | Standardized M | 95% CI | Coeff. | 95% CI | p | |
| Organizational Social Contextd | |||||||||||||||
| Engagement | 48.2 | 42.8 to 53.6 | 50.2 | 47.8 to 52.6 | –2.0 | –7.0 to 3.0 | .43 | — | — | — | — | — | — | — | |
| Functionality | 47.4 | 44.5 to 50.3 | 47.4 | 45.5 to 49.4 | .0 | –3.9 to 3.8 | .99 | — | — | — | — | — | — | — | |
| Stress | 55.0 | 53.2 to 56.7 | 51.3 | 49.8 to 52.9 | 3.6 | 3.1 to 4.1 | <.001 | — | — | — | — | — | — | — | |
| Morale | 48.1 | 46.6 to 49.6 | 49.6 | 47.7 to 51.5 | –1.5 | –2.2 to -.8 | <.001 | — | — | — | — | — | — | — | |
| Job satisfaction | 47.1 | 44.9 to 49.2 | 48.8 | 47.0 to 50.5 | –1.7 | –2.5 to –.9 | <.001 | — | — | — | — | — | — | — | |
| Organizational commitment | 49.2 | 48.2 to 50.2 | 50.4 | 48.7 to 52.2 | –1.2 | –2.6 to .2 | .09 | — | — | — | — | — | — | — | |
| Recovery Self-Assessment Scale, Revisede | |||||||||||||||
| Client choice | 3.8 | 3.7 to 4.0 | 3.8 | 3.7 to 3.9 | .0 | –.1 to .2 | .70 | 3.9 | 3.9 to 4.0 | 3.7 | 3.7 to 3.8 | .2 | .1 to .3 | .001 | |
| Client involvement | 3.3 | 3.2 to 3.3 | 3.2 | 3.2 to 3.3 | .0 | .0 to .1 | .18 | 3.5 | 3.5 to 3.6 | 3.2 | 3.1 to 3.3 | .3 | .2 to .5 | <.001 | |
| Individually tailored | 3.8 | 3.7 to 4.0 | 3.6 | 3.5 to 3.7 | .2 | .0 to .4 | .05 | 3.9 | 3.8 to 4.0 | 3.6 | 3.4 to 3.7 | .3 | .1 to .5 | .005 | |
| Inviting | 3.9 | 3.8 to 4.0 | 3.6 | 3.3 to 3.9 | .3 | .1 to .6 | .01 | 4.1 | 4.0 to 4.2 | 3.9 | 3.8 to 4.0 | .2 | .0 to .4 | .05 | |
| Life goals | 3.9 | 3.8 to 3.9 | 3.8 | 3.7 to 3.8 | .1 | .1 to .1 | <.001 | 3.9 | 3.8 to 3.9 | 3.6 | 3.5 to 3.7 | .3 | .1 to .5 | <.001 | |
| Treatment diversity | 3.6 | 3.5 to 3.8 | 3.4 | 3.4 to 3.5 | .2 | .0 to .4 | .08 | 3.6 | 3.5 to 3.6 | 3.4 | 3.3 to 3.4 | .2 | .1 to .3 | <.001 | |
| Total | 3.7 | 3.7 to 3.8 | 3.6 | 3.6 to 3.7 | .1 | .0 to .2 | .001 | 3.8 | 3.8 to 3.9 | 3.5 | 3.5 to 3.6 | .3 | .1 to .4 | <.001 | |
| Working Alliance Inventory, Shortf | |||||||||||||||
| Task | — | — | — | — | — | — | — | 3.9 | 3.8 to 4.0 | 3.7 | 3.5 to 3.8 | .3 | .0 to .5 | .02 | |
| Bond | — | — | — | — | — | — | — | 3.9 | 3.8 to 4.1 | 3.7 | 3.6 to 3.8 | .2 | .0 to .5 | .04 | |
| Goals | — | — | — | — | — | — | — | 3.7 | 3.6 to 3.7 | 3.4 | 3.4 to 3.5 | .2 | .1 to .3 | .001 | |
| Total | — | — | — | — | — | — | — | 3.8 | 3.8 to 3.9 | 3.6 | 3.5 to 3.7 | .2 | .0 to .4 | .01 | |
a
Provider analyses are random-effects models with random intercept for individual and standard error adjustment for within-clinic clustering. Analyses are adjusted for working at a clinic operated by the Los Angeles County Department of Mental Health (LACDMH) versus a contracted clinic; training (M.D. or nurse practitioner, registered nurse or licensed vocational nurse, master’s or doctorate degree, or no advanced degree [omitted]); and fiscal year of survey completion (uniquely identifies surveys within individuals). Adjusting for provider’s sex did not affect results; the covariate was removed because of small sample size.
b
Client analyses are restricted to surveys completed at least 60 days following clinic admission and are random-effects models with random intercept for individual and standard error adjustment for within-clinic clustering. Analyses are adjusted for date of admission, centered at July 1, 2008, alone and interacted with FSP; number of days between clinic admission and survey completion, centered at the analytic sample mean (465.8 days), alone and interacted with FSP; sex; age at baseline; race-ethnicity; whether non–English speaking; Global Assessment of Functioning score, primary psychiatric diagnosis, and comorbid substance use disorder diagnosis at admission; prior-12-month service utilization (outpatient minutes, emergency or urgent care minutes, acute inpatient days, and long-term inpatient days); prior-six-month days homeless and days incarcerated; and whether enrolled in an LACDMH-operated or contracted clinic.
c
Differences are regression coefficients. Standardized means for FSP and usual care are predictive margins (Stata 14’s margins command) with delta-method confidence intervals.
d
Scores are nationally normed T scores, with a mean of 50 and standard deviation of 10. Results are from 74 FSP and 208 usual care provider surveys. Sample size did not vary across subscales because of formal scoring procedures that excluded entire cases based on missing observations and other filters.
e
Scores derive from client or provider agreement with the presence of recovery-oriented program features on a scale from 1, strongly disagree, to 5, strongly agree. Results are from 79 FSP and 219 usual care provider surveys. Because of missing items, the N for FSP was 78 for total and ranged from 69 to 79 for subscales; the usual care N was 215 for total and ranged from 190 to 217 for subscales. Results are from 574 FSP and 733 usual care client surveys. Because of missing items, the N for FSP was 554 for total and ranged from 468 to 542 for subscales; the usual care N was 664 for total and ranged from 547 to 705 for subscales.
f
Scores evaluate client perceptions of the presence of a strong client-provider working alliance on a scale from 1, strongly disagree, to 5, strongly agree. Results are from 578 FSP and 742 usual care client surveys. Because of missing items, the N for FSP was 578 for total and all subscales; the N for usual care was 736 for total and ranged from 740 to 742 for subscales.
Interviews and ethnographic observations suggested that differences between FSP and usual care went beyond service volume. Providers worked as teams and took seriously the obligation to “do whatever it takes” for their small caseloads.
Program Recovery Orientation and Working Alliance
Providers and clients, on average, rated both programs as having recovery-oriented features (RSA-R) (Table 4). Providers rated FSP programs significantly higher than usual care programs on two of six subscales and overall (3.7 versus 3.6, p=.001), although year-specific estimates (not shown) differed significantly only in FY 2008–2009 (overall and on four subscales). On the RSA-R, clients rated FSP programs higher on five of six subscales and overall (3.8 versus 3.5, p<.001); all differences were present for the FY 2008–2009 and FY 2009–2010 year-specific estimates, and some were present for the FY 2007–2008 and FY2010–2011 estimates (not shown).
Although clients in both programs reported a positive working alliance with providers (WAI-S), FSP ratings were higher on all subscales and overall (3.8 versus 3.6, p<.01); all differences were present for FY 2007–2008, and some were present for FY2008–2009 and FY2009–2010 (not shown).
Client ratings were consistent with data from the semistructured interviews. Usual care clients were less likely to describe feeling close to providers and more likely to express difficulty getting needs met. As one usual care client said, “Not being able to get in touch with your caseworker or psychiatrist when you need them, that’s very difficult. I get to see Dr. [name] when my appointment is, and that’s it.”
A common complaint was that clinicians were overburdened. One usual care client said, “I’m not saying he’s a bad psychiatrist. It’s just that certain things that he should be asking, and since he don’t ask, I don’t feel comfortable volunteering anything. . . . Maybe this place isn’t for that. . . . He should have less caseload. . . . At least try to talk to a person in there for 45 minutes . . . to see what’s going on, and they don’t do it here.”
In contrast, FSP clients reported close, available relationships. One said, “It’s a great relationship. They support me a lot. They’re almost like family to me because of what they try to do.” Another described feeling respected and listened to, in person and through care coordination meetings: “They don’t treat me as someone who’s crazy. They listen to you. They have meetings every morning; they share that kind of stuff. . . . I’ve never felt disrespect. I’ve never felt judged. I’ve always felt, you know, that their goal is really to help us to help ourselves.”
The programs’ structures—FSPs’ small caseloads, daily team meetings, and mandate and resources to “do whatever it takes” versus usual care’s large caseloads and restriction of contact to brief scheduled appointments—shaped not just service volume, but clients’ treatment relationships and experiences.
Discussion
Following a policy mandate, LA County’s public mental health system transformed the structure of care through its new FSPs. Smaller caseloads, greater resources, team-based care, and higher expectations of providers translated into more case management, rehabilitation, and field-based services.
These structural and service-volume differences affected clinicians’ and patients’ experiences of care. Clients and providers rated both programs as recovery oriented, with higher ratings for FSP, and interviews revealed substantial differences in services and client-provider relationships. Although FSPs’ recovery focus likely played a role, our data suggest that FSP clients reported stronger client-provider working relationships and a more patient-centered experience because providers had the resources and mandate to “do whatever it takes,” with small caseloads allowing them to be available to clients, discuss broader issues, plan for clients’ futures, and devote resources to clients between visits via team meetings and follow-up of missing clients. This contributed to FSP clients feeling that they had someone to work with and talk to, whereas usual care clients sometimes felt that treatment scratched the surface and was bounded within scheduled visits.
Although the restructured services were beneficial to clients, providing these services imposed burdens on the system: more resources were required for higher service intensity, and providers experienced stress because they were responsible for providing coordinated, accountable, patient-centered care. These findings anticipate potential issues with ACA’s encouraged shift toward patient-centered, accountable care for patients with complex, chronic needs.
Some limitations should be noted. Although we selected our usual care client sample on the basis of eligibility criteria approximating those of FSP, usual care providers treat the full range of clients meeting DMH medical necessity criteria, all of whom have a Medi-Cal reimbursable mental health diagnosis and impairment in an important area of life functioning but many of whom do not meet FSP eligibility criteria used to identify the most severely ill. Provider responses may reflect case mix in addition to program structure and treatment approaches. However, ethnographic observations and interviews supported the interpretation that doing “whatever it takes” increased FSP providers’ work-related stress.
Conclusions
Our evaluation of MHSA’s impact on clinic-level factors fills a gap in oversight of this large system intervention by showing, at a micro level, how MHSA funds were used to expand and transform services in a core MHSA program in California’s most populous county. These findings are important to evaluating and improving MHSA’s implementation and will inform an upcoming analysis of FSP client-level outcomes over a three-year follow-up: emergency and inpatient utilization and costs; homelessness, incarceration, and employment; and symptoms and functioning.
Acknowledgments
The authors are grateful to the many individuals who contributed to the evaluation of the Mental Health Services Act: Los Angeles Department of Mental Health administrators and clinicians; researchers Thomas Belin, Ph.D., Susan Ettner, Ph.D., Alison Hamilton, Ph.D., Gerald Kominski, Ph.D., Susan Stockdale, Ph.D., Kenneth Wells, M.D., M.P.H., and Alexander Young, M.D., M.H.S.H.; project managers Briana Hedman, M.A., Richell Jose, B.A., Amanda Nelligan, M.A., and Rebecca Wilkinson, M.S.P.H.; and the ethnographers, peer specialists, and survey specialists who enrolled, surveyed, and interviewed clients during the nearly four years in the field, including Jorge Avila, B.A., Rebecca Carrillo, Ph.D., Mayte Diaz-Dryer, M.P.A., Deborah Hastings, Melissa Park, Ph.D., and Deborah Peralta.
Footnote
This project received funding from the National Institute of Mental Health (NIMH) (R-01 MH 080671, 2007–2012; and F-31 MH 085394, 2008–2010), the Los Angeles County Department of Mental Health, and the Robert Wood Johnson Foundation Investigator Award in Health Policy (2010–2014). Methodological support was received from the methods core of the NIMH Partnered Research Center for Quality Care (P30 MH082760) and from a grant (UL1TR000124) from the National Center for Advancing Translation Sciences, National Center for Research Resources, National Institutes of Health, to the UCLA Clinical and Translation Science Institute.
References
1.
Patient Protection and Affordable Care Act, Public Law 111-148., Stat 124 Stat 119 (March 23, 2010)
2.
Mental Health Services Act, 2004. http://www.sccgov.org/sites/mhd/MHSA/Documents/MHSA-Full-Text-English.pdf
3.
Report to the Governor and Legislature. Sacramento, CA, Mental Health Services Oversight and Accountability Commission, Dec 2013
4.
Mental Health Services Act Community Services and Supports: Three-Year Program and Expenditure Plan Requirements, Fiscal Years 2005–06, 2006–07, 2007–08. Sacramento, California Department of Mental Health, 2005
5.
California Code of Regulations, Title 9, Division 1, Chapter 14: Mental Health Services Act. https://govt.westlaw.com/calregs/Browse/Home/California/CaliforniaCodeofRegulations?guid=I689A72A0D45311DEB97CF67CD0B99467
6.
Mayberg SW: Mental Health Services Act Expenditure Report Fiscal Year 2006–2007, Addendum, May 2007. Sacramento, California Department of Mental Health, 2007
7.
Marshall M, Lockwood A: Assertive community treatment for people with severe mental disorders. Cochrane Database of Systematic Reviews 2:CD001089, 2000
8.
State and County Quick Facts, Los Angeles County, California. Washington, DC, US Census Bureau, 2012. http://quickfacts.census.gov/qfd/states/06/06037.html
9.
Los Angeles County's Community Services and Supports Plan Developed According to the Mental Health Services Act: A Framework to Guide Provider Selection for Full Service Partnerships: General and Age-Specific Criteria for Full Service Partnerships. Los Angeles, Los Angeles County, 2005. http://dmh.lacounty.gov/wps/portal/dmh/!ut/p/b0/04_Sj9CPykssy0xPLMnMz0vMAfGjzOJdDQwM3P3dgo3cjd0cDTxdXYxD_AJMDN0DzfQLsh0VAeiT6Tk!/?1dmy&page=dept.lac.dmh.home.mhsa.mhsa_detail.hidden&urile=wcm%3Apath%3A/dmh+content/dmh+site/home/mental+health+services+act/mhsa+detail/css+community+services+and+supports+plan
10.
The Los Angeles County Mental Health System's Community Services and Supports Plan: A Detailed Summary of the Plan Submitted by Los Angeles County to the California Department of Mental Health in Accordance with the Mental Health Services Act. Los Angeles, Los Angeles County, 2005. http://file.lacounty.gov/SDSInter/dmh/158987_CSS_Plan_Summary.pdf
11.
Mental Health Services Act Full Service Partnership (FSP) Guidelines. Version 0.9. Effective: Nov 1, 2006. Rev Issue: May 2007. Los Angeles, Los Angeles County Department of Mental Health, 2007. http://dmhoma.pbworks.com/f/MHSAFSPGUIDELINES_006smaller.pdf
12.
Full Service Partnership (FSP) Frequently Asked Questions. Rev 05/17/2007. Los Angeles, Los Angeles County Department of Mental Health, 2007
13.
Davis G, Johnson G, Mayberg SW: Effectiveness of Integrated Services for Homeless Adults With Serious Mental Illness: A Report to the Legislature as Required by Division 5, Section 5814, of the California Welfare and Institutions Code. Sacramento, California Department of Mental Health, 2002
14.
Gilmer TP, Manning WG, Ettner SL: A cost analysis of San Diego County’s REACH program for homeless persons. Psychiatric Services 60:445–450, 2009
15.
Gilmer TP, Stefancic A, Ettner SL, et al: Effect of full-service partnerships on homelessness, use and costs of mental health services, and quality of life among adults with serious mental illness. Archives of General Psychiatry 67:645–652, 2010
16.
Brown TT, Bruckner TA, Chung J, et al: The Impact of the Mental Health Services Act on Emergency Interventions and Involuntary Hospitalizations. Contract no Petris Report 2010-4. Berkeley, University of California, Berkeley, School of Public Health, Nicholas C Petris Center on Health Care Markets and Consumer Welfare, 2010
17.
Brown TT, Choi S-S, Chung J, et al: A Comparison of Satisfaction, Services Characteristics and Outcomes in the Full Service Partnership Programs Relative to Usual Care. Contract no Petris Report 2010-1. Berkeley, University of California, Berkeley, School of Public Health, Nicholas C Petris Center on Health Care Markets and Consumer Welfare, 2010
18.
Scheffler RM, Felton M, Brown TT, et al: Evidence on the Effectiveness of Full Service Partnership Programs in California’s Public Mental Health System. Berkeley, University of California, Berkeley, School of Public Health, Nicholas C Petris Center on Health Care Markets and Consumer Welfare, 2010
19.
Goldman HH, Morrissey JP, Ridgely MS: Evaluating the Robert Wood Johnson Foundation program on chronic mental illness. Milbank Quarterly 72:37–47, 1994
20.
Bickman L: Evaluating Managed Mental Health Services: The Fort Bragg Experiment. New York, Plenum, 1995
21.
Bickman L, Lambert EW, Andrade AR, et al: The Fort Bragg continuum of care for children and adolescents: mental health outcomes over 5 years. Journal of Consulting and Clinical Psychology 68:710–716, 2000
22.
Bickman L, Noser K, Summerfelt WT: Long-term effects of a system of care on children and adolescents. Journal of Behavioral Health Services and Research 26:185–202, 1999
23.
Randolph F, Blasinsky M, Morrissey JP, et al: Overview of the ACCESS program: access to community care and effective services and supports. Psychiatric Services 53:945–948, 2002
24.
Goldman HH, Morrissey JP, Rosenheck RA, et al: Lessons from the evaluation of the ACCESS program. Psychiatric Services 53:967–969, 2002
25.
Promises Still to Keep: A Decade of the Mental Health Services Act. Report 225. Sacramento, CA, Little Hoover Commission, 2015
26.
Mental Health Services Act: The State’s Oversight Has Provided Little Assurance of the Act’s Effectiveness, and Some Counties Can Improve Measurement of Their Program Performance. Report 2012-122. Sacramento, CA, Office of State Auditor, 2013
27.
About DMH. Los Angeles, Los Angeles County Department of Mental Health, 2014. http://dmh.lacounty.gov/wps/portal/dmh/aboutdmh
28.
FSP Outcomes–Cost Avoidance Analysis. Los Angeles, Los Angeles County Department of Mental Health, 2011
29.
Diagnostic and Statistical Manual of Mental Disorders, 4th ed, Text Revision. Washington, DC, American Psychiatric Association, 2000
30.
Integrated System Project Reporting Repository DB Technical Specification, Version 1.3, Draft. Los Angeles, Los Angeles County Department of Mental Health, 2004
31.
Campbell J: Integrated System Reporting Database and Related Systems Description. Los Angeles, Los Angeles County Department of Mental Health, 2006
32.
Glisson C, Landsverk J, Schoenwald S, et al: Assessing the organizational social context (OSC) of mental health services: implications for research and practice. Administration and Policy in Mental Health and Mental Health Services Research 35:98–113, 2008
33.
O’Connell MJ, Tondora J, Kidd SA, et al: Recovery Self-Assessment, Revised (RSA-R). New Haven, Conn, Yale University, School of Medicine, 2007. http://medicine.yale.edu/psychiatry/prch/tools/rec_selfassessment.aspx
34.
Tracey TJ, Kotovic AM: Factor structure of the Working Alliance Inventory. Journal of Consulting and Clinical Psychology 1:207–210, 1989
35.
Charmaz K: Constructing Grounded Theory, 2nd ed. Thousand Oaks, CA, Sage, 2014
Information & Authors
Information
Published In
Cover: Ripening Pears, by Joseph Decker, circa 1884. Oil on canvas. Gift of Ann and Mark Kington/The Kington Foundation Avalon Fund. National Gallery of Art, Washington, D.C.
History
Received: 9 September 2015
Revision received: 29 February 2016
Accepted: 18 November 2016
Published online: 1 February 2017
Published in print: June 01, 2017
Keywords
Authors
Competing Interests
Dr. Starks, Dr. Meldrum, Dr. Kelly, Dr. Brekke, and Dr. Braslow report receiving funding (2016–2019) from the Los Angeles County Department of Mental Health to evaluate the county's new assisted outpatient treatment program. The other authors report no financial relationships with commercial interests.
Funding Information
Los Angeles County Department of Mental Health:
National Institutes of Health10.13039/100000002: UL1TR000124
National Institute of Mental Health10.13039/100000025: F-31 MH 085394, P30 MH082760, R-01 MH 080671
Robert Wood Johnson Foundation10.13039/100000867:
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBLogin options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).