Use of Administrative Data to Identify Potential Service Gaps for Individuals With Serious Mental Illness
Abstract
Objective:
The New York City Mental Health Care Monitoring Initiative uses Medicaid claims data to identify individuals with serious mental illness who are experiencing or at risk for gaps in services. In this study the authors assessed whether proposed service use algorithms accurately identified such individuals.
Methods:
A random sample of 500 individuals with serious mental illness was identified. Individuals belonged to specific high-need cohorts and met predefined claims-based criteria for potential service gaps. Clinical staff initiated reviews with prior service providers for 230 individuals.
Results:
Over a two-week period staff completed reviews for 188 cases (88%). In 66 cases (35%) the individual was fully engaged in care; 84 (45%) had a recent episode of disengagement that was appropriately addressed, and 38 (20%) were not receiving adequate services.
Conclusions:
The proposed service use algorithms successfully identified high-need individuals with serious mental illness at risk for gaps in services. (Psychiatric Services 62:1094–1097, 2011)
Public mental health leaders have called for a greater emphasis on evidence-based policy and data-driven decision making in policy and program development (1,2). For example, Frazier and Casper (3) demonstrated that clinical events identified from Medicaid claims data predicted future psychiatric hospital readmissions and could be successfully used to guide early intervention strategies. As information technology advances, administrative databases offer the potential to inform policy initiatives and provide real-time clinical decision support.
In 2008, New York City and State government leaders convened a panel of experts to examine incidents of violence involving individuals with serious mental illness. The panel concluded that fragmented public mental health services and limited accountability among providers contributed to gaps in services for individuals with serious mental illness living in the city (4,5). The panel recommended the creation of care monitoring teams in New York City to use Medicaid claims and other state administrative data to examine patterns of service use by high-need individuals at risk of poor outcomes. The panel's recommendations were accepted in June 2008, and the city's public mental health leadership subsequently designed the Care Monitoring Initiative (6). In it, trained clinicians review cases triggering flags of potential clinical concerns with providers who were identified on the basis of Medicaid claims data as having served the individual in the prior 12 months. The reviewers determine whether individuals are receiving adequate and appropriate services, and if not, they work with providers to develop plans for outreach and reengagement and where possible, continue contact with the provider until the individual has reengaged in appropriate services.
The project described in this study was conducted in the spring of 2009, several months before the first care monitoring team started in Brooklyn. The project assessed whether the service use algorithms accurately identified clinically needy individuals who were disengaged from appropriate services.
Methods
Individuals eligible for the Care Monitoring Initiative include residents of New York City who have ever been ordered by a court to receive assisted outpatient treatment (AOT) under New York State's outpatient commitment statute, even if the order had expired (current and expired AOT cohorts), received assertive community treatment (ACT) or case management in the prior 12 months (subdivided into intensive case management [ICM] and blended case management [BCM] cohorts), received mental health services in a forensic psychiatric setting in New York State and were presently living in the community (forensic cohort), or have had two or more psychiatric emergency room or inpatient hospitalizations in the prior 12 months (multi-acute user cohort). The cohorts are ranked by degree of service need, and individuals eligible for more than one cohort are assigned to the group with higher needs. Preliminary data analyses indicated that these groups in total include approximately 20,000 city residents at any point in time.
The study focused on individuals with Brooklyn addresses who belonged to one or more of the high-need cohorts as of September 30, 2008 (N=6,408). Medicaid service claims were analyzed to identify individuals in these cohorts who may have been experiencing gaps in needed services (approximately 85% of individuals in the above cohorts had Medicaid eligibility in the prior year). Patterns of nonuse of services use that triggered notification flags were failure to fill a psychotropic medication prescription in the prior 60 days; not receiving community-based mental health services in the prior 120 days; or having had two or more psychiatric inpatient admissions or emergency visits in the prior 120 days.
The look-back periods (60 and 120 days) for the notification flags were determined based upon review of claims lag data. For example, almost all Medicaid pharmacy claims in New York State are paid at the time that the prescription is filled, yielding very timely pharmacy data. Therefore, when a Medicaid-eligible individual has no pharmacy claims over a two-month period, it is likely that the individual is no longer taking medication. For clinical services (community-based and inpatient or emergency room treatment) the claims lag was significantly greater—on average, 60 days were required to ensure that at least 75% of providers had submitted claims for services. To avoid a high number of false-positive notifications—delays in claims submission as opposed to service gaps—the look-back period was set at 120 days for claims for clinical services.
Of the 6,408 individuals in the overall population, 4,826 met one or more of the notification flags as of October 1, 2008, or November 1, 2008. A random sample of 500 individuals was generated from this group, including at least 50 from each of the high-need cohorts.
A review panel of seven master's level mental health clinicians who had program oversight responsibilities for the New York City or New York State mental health authorities completed reviews. Orientation to the project included a review of the panel report (5) and discussion of case vignettes illustrating levels of engagement in services for high-need individuals. Over a two-week period, the reviewers contacted providers who had been identified based on Medicaid claims as having served the individuals in the prior 12 months. A script created to guide telephone discussions contained specific questions eliciting information about individuals' prior use of services, degree of current engagement in care, and the provider's most recent risk assessment. The goal was to complete reviews for at least 20 individuals from each high-need cohort except the current AOT cohort and to complete reviews for all 58 individuals with current AOT orders. The study was considered a program evaluation activity that did not involve human subjects interviews and did not require institutional review board review.
On the basis of discussions with mental health administrators in New York City and New York State and the work of the review panel described above, the project leadership developed three categories to summarize case reviews. Level 1 cases included those in which the pattern of service use or nonuse was determined not to represent a clinical concern. An example involved individuals receiving medication from clinics' sample supplies without using their Medicaid prescription cards. These individuals triggered the psychotropic medication notification flag but were, in fact, receiving medication.
Level 2 cases involved individuals who were not adequately engaged in care but whose providers had put in place an adequate reengagement plan. An example included an individual receiving ACT services who had had a relapse of psychotic symptoms and who had been hospitalized twice in the prior four months. Because the ACT team was aware of the hospitalizations, had identified the stressors associated with the relapse, and was working with the inpatient treatment team to modify the individual's care plan in an effort to address the stressors and treat the relapse, the case was categorized as level 2.
Finally, level 3 cases involved individuals who were not adequately engaged in care and whose previous providers had not reported having put in place adequate reengagement plans. Examples included individuals who had refused treatment and were lost to care in spite of outreach efforts as well as individuals whose providers had made referrals to inappropriate levels of care that resulted in disengagement and a subsequent lack of follow-up.
Results
Clinicians initiated reviews with providers for 230 of the 500 individuals identified. Reviews were incomplete for 42 (18%) individuals because the clinicians were unable to obtain sufficient follow-up information from providers during the two-week review period. Examples included situations in which providers cited confidentiality concerns or individuals were discharged from care several months previously and no information was available to determine whether they were engaged in services at the time of review. Staff completed reviews for the remaining 188 individuals (82%); each review required an average of three phone contacts (range one to 14) and a time commitment of an average of 15 minutes (range one to 116).
Figure 1 summarizes the number of individuals in the three category levels from each high-need population For the total population, 66 reviews (35%) were categorized as level 1, or of no clinical concern. Eighty-four cases (45%) were labeled as level 2, an indicator that service gaps had been acknowledged and adequately addressed. Thirty-eight cases (20%) were coded as level 3, an indicator of continued need for outreach and reengagement efforts to address a recent service gap.
The percentages of individuals assigned to the various levels of care disruption varied by population. Individuals receiving the most intensive services (AOT and ACT) had lower rates of service gaps. Individuals in the multiacute user cohort (two or more psychiatric inpatient or emergency visits in the prior 12 months) were the most likely to be experiencing service gaps, given that nearly 90% of the gaps were classified as either level 2 or 3. The extent to which individuals had a current community-based provider who was familiar with their care and needs also varied by cohort; rates were highest for the AOT current cohort (90%), followed by ACT (61%), case management (59%), forensic (40%), AOT expired (33%), and multiacute user (15%).
Since this study ended, a care monitoring team was established in Brooklyn. Data from the first six months of care monitoring in Brooklyn yielded somewhat different proportions than those found by the study. The care monitoring team identified 7,242 individuals who triggered a notification; 4,801 reviews were initiated, and 2,545 (53%) were completed within one month (compared with the study rate of 82% for reviews completed in a month). Of the completed reviews, 30% were coded level 1; 18%, level 2; and 52%, level 3, These rates compared with rates of 35%, 45%; and 20%, respectively, for individuals in levels 1–3 reported by the study).
Discussion
In this study, trained clinicians contacted providers and completed timely reviews of high-need individuals with serious mental illness identified via secondary data analyses as possibly experiencing a gap in services. The 188 individuals reviewed were from a random sample of 500 of the 4,826 individuals who met defined notification triggers, and unknown biases related to the relatively small number of reviews and their distribution across the various client cohorts may have influenced the proportion of cases deemed to present higher or lower levels of clinical concern.
Nonetheless, the findings indicate that the service use algorithms were an efficient means of identifying individuals who were in need of treatment but had not become adequately engaged in appropriate services.
We anticipated that the large high-need cohorts defined by the study would yield high numbers of level 1 assignments, given the expectation that claims would lag and the knowledge that Medicaid eligibility and utilization vary significantly over individuals and across time. Such instances were the minority, however, for all populations except for the AOT cohort. These individuals, who were under court orders to participate in community-based care, received extensive monitoring and were expected to have minimal service gaps. Of note, however, is that a substantial number (15%–35%) of individuals in the ACT and case management cohorts also were disengaged from appropriate services. These high-need individuals had been accepted into intensive services in the prior 12 months and had failed to remain engaged in care. Efforts to understand their disengagement from services and what was needed to reengage them in appropriate services should identify opportunities to strengthen the public mental health safety net.
The study illustrated some of the challenges of using secondary data. The 60- to 120-day look-back period for clinical services is necessary, because of claims lag, to limit the numbers of false positive cases. Because of the lengthy look-back period, however, for some individuals a lapse in care was not identified for at least two months, increasing the chances that providers would not be able to contact them after they had been lost to care for so long. Also, the notification definitions relied on Medicaid claims, and individuals who were not eligible for Medicaid were not identified. These are unfortunate limitations, but given the large numbers of high-need individuals with Medicaid who triggered notifications, the initiative offers many opportunities for outreach and engagement of individuals identified as in need of continuing services.
As noted above, after implementation of the Brooklyn care monitoring team by a contracted vendor, more cases were categorized as having level 3 service gaps than were found in this study. There are several potential reasons for this disparity. For example, the vendor staff consisted of newly hired clinicians who were still becoming familiar with the New York City provider system of care and, therefore, were still establishing relationships with providers and educating them about the program. Providers may have been more likely to respond to staff of New York City or New York State who conducted the study because they had already established relationships with them through work than to vendor staff who they didn't know. Finally, the vendor contract required more than identification of gaps in services. Vendor care monitors documented risk factors and other clinical data, followed up cases of individuals disengaged from care, and helped providers develop outreach and reengagement plans. These follow-up requirements and associated activities could have influenced judgments about specific cases. Further experience with the initiative, including case audits, will clarify these issues.
Conclusions
This study confirmed that applying service use algorithms to Medicaid claims data could successfully identify individuals with serious mental illness who are in need of outreach and engagement.
Acknowledgments and disclosures
The authors report no competing interests.
References
1.
Wharam JF, Daniels N: Toward evidence-based policy making and standardized assessment of health policy reform. JAMA298:676–679, 2007
2.
Jewell CJ, Bero LA: “Developing good taste in evidence”: facilitators of and hindrances to evidence-informed health policymaking in state government. Milbank Quarterly 86:177–208, 2008
3.
Frazier RS, Casper ES: A comparative study of clinical events as triggers for psychiatric readmission of multiple recidivists. Psychiatric Services 49:1423–1425, 1998
4.
Smith TE, Sederer LI: Changing the landscape of an urban public mental health system: the 2008 New York State-New York City Mental Health-Criminal Justice Review Panel. Journal of Urban Health 87:129–135, 2010
5.
Hogan MF, Gibbs LI, O'Donnell DE, et al.: New York State-New York City Mental Health-Criminal Justice Panel Report and Recommendations. New York, New York State Office of Mental Health, 2008
6.
Smith TE, Appel A, Donahue SA, et al.: Using Medicaid claims data to identify service gaps for high-need clients: the NYC Mental Health Care Monitoring Initiative. Psychiatric Services 62:9–11, 2011
Figures and Tables
Information & Authors
Information
Published In
Cover: Approaching Thunder Storm, by Martin Johnson Heade, 1859. Oil on canvas, 28 × 44 inches. Gift of Erving Wolf Foundation and Mr. and Mrs. Erving Wolf, 1975. The Metropolitan Museum of Art, New York. Image © The Metropolitan Museum of Art/Art Resource, New York.
History
Published in print: September 2011
Published online: 14 January 2015
Authors
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBLogin options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).