Best Practices: New York's Outpatient Mental Health Clinic Licensing Reform: Using Tracer Methodology to Improve Service Quality
Abstract
This column describes how the New York State Office of Mental Health (OMH) redesigned licensing procedures for outpatient clinics by applying a person-centered focus and clinically relevant standards of care. OMH incorporated a tracer methodology to assess services; a licensing instrument that reflected OMH goals; and a systematic piloting, feedback, and implementation process. Clinic staff reported that the new procedures facilitated communication between OMH and clinics, accurately identified clinics' successes, and provided actionable and programmatic feedback. This initiative represents the first step by a state mental health authority to create a system of accountability that is clinically relevant and supported by stakeholders. (Psychiatric Services 63:418–420, 2012; doi: 10.1176/appi.ps.20120p418)
Professional accountability requires a profession to set and maintain credible and useful standards for its members (1) and to implement best practices (2). New York State oversees mental health services provided by state-licensed treatment providers. In response to several incidents of violence that involved people with serious mental illness, New York State and New York City sponsored a Mental Health-Criminal Justice Panel to improve coordination, oversight, and accountability in mental health treatment services. The panel's report was published in June 2008 (3). Although prompted by the incidents of violence, the New York State Office of Mental Health (OMH) took advantage of the call for better assessment of violence to revise the licensing process to underscore recovery-oriented values, such as treatment responsive to individuals' stated needs and family involvement in treatment, in redesigned clinic licensing procedures.
Mental health care has been moving toward a recovery-focused, person-centered, strengths-based system. The President's New Freedom Commission on Mental Health (4) and others (5–7) have called for increasing consumer- and family-driven services, improving quality and accountability in behavioral health services, and incorporating the perspectives of the people who receive services. Licensing processes have not always kept pace with these efforts to improve service delivery.
Outpatient clinics are the “face of the public mental health system” and are the only level of service used by most people. In New York State, for example, more than 500 licensed clinics offer outpatient mental health services at more than 1,200 sites serving more than 400,000 individuals each year. Though the panel's mandate to apply new standards of care was the initial impetus for change, OMH saw this as an opportunity to promote high-quality outpatient services.
The licensing system as it existed at the time of the panel's report in 2008 was a tiered process focused largely on policies and documentation. This licensing process usually included a discussion with the clinic administrator and a review of medical records, policies, meeting minutes, and safety-related procedures. The panel report included guidelines for standards of care at mental health clinics (3). The guidelines focused on new areas of integrated, coordinated care through “clinical homes” (8); risk assessment; information sharing across providers; services based on individual and family needs; and engagement and clinical supervision. The recommendations challenged OMH to identify licensing procedures that would capture these changing responsibilities. This column describes the first steps in redesigning and implementing the clinic licensing system in New York State.
Redesign of the licensing system
OMH proposed a redesign with three aims: to base the licensing process on the guidelines for standards of care, to focus the process on improving care and emphasizing family involvement, and to model strengths-based interactions to reinforce how OMH would like clinics to interact with people with serious mental illness. To accomplish these goals, OMH created a work group that solicited input from central and field office staff; experts on clinical service, psychometrics, engagement, wellness, and co-occurring disorders; people with serious mental illness and family members; family advocacy groups; service providers and provider associations; and special interest groups. In collaboration with these stakeholders, OMH proposed a tracer methodology (9) to create an interactive process to assess current clinic functioning, evaluate systems for providing care, evaluate the experience of services for people with serious mental illness and their families, and provide clinically relevant and actionable feedback to clinics (10).
Tracer methodology, as promoted by the Joint Commission (9), is a tool to assess a facility's compliance with selected standards and to evaluate systems for providing care and services. To use the tracer methodology, individuals with high service needs are selected and, using the medical record as a “road map,” each service contact is traced to evaluate whether program components were compliant with standards of care. Discussion of findings with organizations can also provide relevant feedback to guide mental health service improvements (11).
Throughout the development process, OMH openly shared presentations and data on the OMH Web site, sought feedback from stakeholders, and pilot-tested survey instruments and procedures.
Tools for implementing redesign
OMH developed three tools to support the redesign process.
Site visit protocols
Protocols to select tracer cases directed OMH survey teams to choose individuals with high service needs, such as individuals needing complex care (for example, those with co-occurring medical or substance abuse problems) or with recent use of intensive services, such as hospitalization. To pilot the protocols, OMH survey teams included two central office staff and two or three field office staff from regional offices who conducted site visits and surveys; currently survey teams include two field office staff. OMH developed spreadsheets that listed clients who had used a particular clinic in the past year and who met criteria for having high needs. On the morning of the visit, clinic staff were asked to recommend the most high-need individuals on the appointment list for that day, and the OMH team then compared the clinic's list with names on their spreadsheets and, when possible, selected the matching individuals to trace. Using the medical record as a “road map,” surveyors interviewed people receiving services, their significant others, program clinical and administrative staff, and other service providers to evaluate whether service contacts were compliant with standards of care. After the initial interview, clinic staff and state surveyors reviewed cases together, compared information, compared scoring, and discussed each item to determine areas of consensus, clarity, and ambiguity.
Licensing instrument
The team developed care standards and integrated them into a licensing instrument that would maintain a strengths-based approach, promote person and family centeredness, reflect the current evidence base involving treatment engagement, and include a quality improvement perspective that would continuously push clinics to increase their quality of care. OMH worked with clinicians and administrators to translate standards of care into 31 measurable criteria and examples and to assign them to three levels of competency: exemplary, adequate, and needs improvement. OMH incorporated evidence-based practice in the exemplary and adequate levels. [The criteria are available in an online data supplement to this article.]
Licensing criteria directly reflected standards of care. Criteria were classified into three categories: exemplary, adequate, and needs improvement. The criteria enumerated in the exemplary category enabled OMH to communicate a clear vision to clinics and to provide an aspirational goal. The final licensing instrument included a decision matrix for determining the length of the operating license (which ranges from 0 to 36 months). [A table in the online data supplement presents an example of how a standard of care (violence assessment) was classified into the three categories.]
Implementation toolkit
OMH developed a toolkit to aid providers (12). The toolkit includes documents associated with the licensing process; video presentations on topics related to the standards of care and on implementing best practices in clinics; a questions-and-answers page addressing topics raised by providers and other stakeholders; and a dedicated e-mail address for questions, suggestions, issues, and comments.
Pilot testing and site visits
Central office and field office staff field-tested the licensing instrument and then pilot tested the redesigned process with 11 clinics, soliciting feedback from programs. These activities identified strengths and potential shortcomings of the instrument as a licensing tool, determined the viability of tracer methodology, assessed the validity of the scoring protocol, and provided opportunity for field office and central office staff to work together to ensure consistency in implementation.
Two or three clinics from each region of the state took part in the pilot testing of the new licensing standards, scoring protocol, and tracer methodology. These 11 clinics were promised a minimum of an 18-month operating license to encourage participation. The clinics were a mix of not-for-profit (N=4), county (N=3), hospital-based (N=3), and state-operated (N=1) clinics. The clinics served a variety of populations, including adults (N=4), children and adolescents (N=1), and both adults and children (N=6). Clinics also were a mix of those with satellite offices (N=4) and those without (N=7). Licensing staff reviewed materials and methods with each provider before the site visit and throughout the process.
The OMH survey team conducted pilot visits, spending two days at each of the 11 clinics. At each site, staff examined at least four open records (range 4–8) and the records of two individuals no longer engaged in services (range 2–6) to identify individuals to trace. Interviews were generally conducted by one field office and one central office staff member; some interviews, such as those with administrators, were conducted by the entire OMH survey team. During the visit, staff interviewed two or more individuals receiving services at each site (N=26 interviews). Staff also conducted in-person and telephone interviews with 12 family members or significant others, ten collateral service providers, and 86 clinic staff, including clinicians, administrators, nursing staff, and quality assurance staff. The OMH survey team concluded visits with an exit conference to discuss survey findings and invited clinics to complete an online survey about the visit.
Outcomes of the new process
Under the new process, five of the 11 clinics (45%) received an operating license for fewer months of operation than in the previous licensing process, and six clinics (55%) received a license for more months of operation. The current process was comparable to the previous process, in which most clinics received a license for the same period of operation as previously. A common challenge identified by the tracer methodology, which might not have been identified by the former licensing procedures, was that providers were reluctant to share information about service users with other providers. The new licensing procedures prompted discussion between clinic and OMH staff about confidentiality and privacy laws that allow communication for treatment purposes.
Clinic staff members' responses to the online survey indicated that the new process was more focused and clinically relevant, that the revised standards were reasonable and appropriate, and that the survey team's contact with people receiving services and their family members increased OMH's ability to assess the quality and effectiveness of the treatment offered. (The survey results are available from the authors.) Clinic staff reported that they felt more involved in the process and that the process was more collaborative, affirming, and clinically relevant. They also said that there was more consensus and clarity on clinic functioning than in the previous licensing processes and that the new process was more likely than the previous system to identify clinics' exemplary activities, which was more affirming. One provider wrote, “In my 20+ years of experience, our 2010 recertification process was by far the most patient-centered, clinic-enhancing state process I have overseen.” Implementation of the redesigned licensing procedures began statewide in April 2010.
Conclusions
We are not aware of other state mental health authorities' efforts to radically redesign licensing procedures, aimed at increasing the person-centeredness of licensing processes and evaluating clinically relevant standards of care. The previous process emphasized policies and procedures, minutes of meetings, and adherence to medical record documentation, whereas the tracer methodology allows for more accurate assessment and follow-up of clinical issues. Key collaborations contributed to successful development of the tracer procedures, with all stakeholders in agreement that they would like every clinic to achieve exemplary status. Both the OMH survey team and clinic staff reported a more positive licensing process focused on quality enhancement and clinical issues, rather than the previous focus on paperwork.
There are still challenges in continuously incorporating feedback from clients and providers, ensuring consistency among the OMH survey team, and maintaining consensus. OMH staff anticipate that as research-informed best practices, stakeholder perspectives, OMH priorities, and clinic performance change, the licensing processes will require continued updating. Future efforts will include applying the redesigned procedures to other types of programs and assisting other states that want to adopt similar procedures.
Supplementary Material
File (appi.ps.20120p418.ds001.pdf)
- View/Download
- 145.69 KB
References
1.
Benson JA: Certification and recertification: one approach to professional accountability. Annals of Internal Medicine 114:238–252, 1991
2.
Drake RE, Goldman HH, Leff HS, et al.: Implementing evidence-based practices in routine mental health service settings. Psychiatric Services 52:179–182, 2001
3.
Hogan MF, Gibbs LI, O'Donnell DE, et al.: New York State/New York City Mental Health-Criminal Justice Panel Report and Recommendations. June 2008. Available at www.nyc.gov/html/om/pdf/2008/pr221-08_report.pdf
4.
Achieving the Promise: Transforming Mental Health Care in America. Pub no SMA-03-3832. Rockville, Md, Department of Health and Human Services, President's New Freedom Commission on Mental Health, 2003
5.
The Breakthrough Series: IHI's Collaborative Model for Achieving Breakthrough Improvement. IHI Innovation Series white paper. Boston, Institute for Healthcare Improvement, 2003. Available at www.IHI.org
6.
Institute of Medicine: Improving the Quality of Health Care for Mental and Substance-Use Conditions: Quality Chasm Series. Washington, DC, National Academies Press, 2005
7.
Infusing Recovery-Based Principles Into Mental Health Services: A White Paper by People Who Are New York State Consumers, Survivors, Patients, and Ex-Patients. Albany, New York State Office of Mental Health, Sept 2004
8.
Smith TE, Sederer LI: A new kind of homelessness for individuals with serious mental illness? The need for a “mental health home.” Psychiatric Services 60:528–533, 2009
9.
Mock Tracer Workbook. Washington, DC, Joint Commission International, 2009
10.
Greenfield D, Travaglia J, Braithwaite J, et al.: Unannounced Surveys and Tracer Methodology: Literature Review. New South Wales, Australia, Centre for Clinical Governance Research in Health, University of New South Wales, 2007
11.
Smith GR, Fisher EP, Nordquist CR, et al.: Implementing outcomes management systems in mental health settings; in Outcomes Assessment in Mental Health Treatment: A Compendium of Articles. Washington, DC, American Psychiatric Association, 1998
12.
Clinic Licensing Toolkit. Albany, New York State Office of Mental Health, Bureau of Inspection and Certification, 2011. Available at www.omh.ny.gov/omhweb/clinic_standards/toolkit.html
Information & Authors
Information
Published In
History
Published online: 1 May 2012
Published in print: May 2012
Authors
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBGet Access
Login options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).