Interventions for depression management that are successful in randomized trials are not readily adopted and disseminated into general practice (
1,
2). The use of quality improvement (QI) learning collaboratives, a well-established approach for implementing improvement in health care systems and disseminating evidence-based medicine, has gained momentum in recent years; evidence suggests gains achieved through learning collaboratives can be sustained (
3,
4).
This brief report describes self-reports among psychiatric practices of the sustainability of key depression QI interventions that were implemented as a part of a yearlong learning collaborative of the National Depression Management Leadership Initiative (NDMLI), a joint project of the American Psychiatric Association, the American Academy of Family Physicians, and the American College of Physicians. Twelve months after the NDMLI concluded, key elements of the intervention among psychiatric practices were examined—specifically, continued use of the nine-item depression scale of the Patient Health Questionnaire (PHQ-9) to monitor symptom severity in routine care of patients with depressive disorders and incorporation of other key elements of the QI model. We hypothesized that a majority of practices would report success in sustaining most QI interventions. Additionally, we examined whether project psychiatrists engaged in efforts to disseminate the intervention to other clinicians within and outside their practice.
Seventeen psychiatric practices participated in the original, yearlong NDMLI, which concluded by May 2006. Practices were diverse: six multispecialty group practices, five mental health group practices, three academic or departmental practices, one outpatient public clinic, and two solo private practitioners.
Two representatives from each practice, a lead psychiatrist and a nonphysician coleader (psychologist, nurse, social worker, or office manager) participated in three “learning sessions” during the 12-month project. The learning sessions introduced strategies for improving depression care through application of the chronic care model (
5). Practices used “plan-do-study-act” cycles to test small changes in a rapid fashion in an effort to implement incremental improvements in clinical work flows. Learning sessions emphasized the use of the PHQ-9 to facilitate depression severity monitoring, implementing follow-up processes for patients, and systematically using and documenting self-management activity, a form of behavioral activation that is collaboratively agreed upon by the patient and the clinician.
PHQ-9 scores for patients receiving depression care, treatment changes, and documentation of self-management were collected monthly from project psychiatrists. Study results for 1,763 patients and 6,363 visits indicated that psychiatrists and their coleaders found the PHQ-9 extremely helpful at both baseline and follow-up visits for confirming or changing treatment plans. Treatment changes commonly reported were changing dose and augmenting or switching antidepressant medications when patients were not in partial or full remission (
6). Response and remission rates at 24 weeks were 45% and 18%, respectively (
7). More detailed descriptions of the study methods and results can be found elsewhere (
6,
7).
Methods
The 17 psychiatrists and their coleaders were contacted 12 months after the completion of the original NDMLI. Follow-up data were collected between May and November 2007. The follow-up study involved data collection at the practice level only. A $20 gift certificate was provided along with the study questionnaires to each project psychiatrist and coleader.
Psychiatrists were asked to complete two separate surveys 12 months after the NDMLI concluded. The first survey was the 21-item Assessment of Clinician Depression Management (ACDM) in Psychiatry, primarily derived from the 34-item Assessment of Chronic Illness Care, a validated scale with scores shown to be linked to patient outcomes (
8,
9). The ACDM, modified to highlight depression care in psychiatry, obtained information about practices’ capacity to provide depression care. It was completed by the project’s lead psychiatrist at baseline, at project completion 12 months later, and at 24 months after baseline (one year after the NDMLI concluded). Prior to study initiation, a pilot study of the ACDM was conducted with a sample of psychiatrists to examine their comprehension of the survey and its relevance. [A copy of the ACDM for Psychiatry is available online as a
data supplement to this report.]
Project coleaders, typically other mental health clinicians or office managers who were paired with the project psychiatrist from each practice site, were also asked to complete a briefer, ten-item version of this survey to provide their assessment of practices’ depression care during the same time intervals. Coleaders were sent this questionnaire separately to preserve confidentiality of their responses apart from the practice psychiatrist.
The other survey was the Sustainability and Spread Activities Questionnaire, developed by the study team, which captured data about practices’ continued use of the PHQ-9 for routine care of patients with depression, progress in spreading the use of the PHQ-9 to other practice staff as well as to external unaffiliated practices, barriers and successes in adopting PHQ-9 by other practice staff, and integration of self-management as a component of patient care.
For qualitative open-ended questions, two bachelors’-level research staff reviewed the responses and independently coded common themes into specific response categories. Then a senior staff member reviewed the two lists, addressed discrepancies, and finalized coding. Frequencies for each response category cited by the respondents when answering open-ended questions were subsequently enumerated.
Basic frequencies, cross-tabulations, and their respective chi square tests were performed by using SAS 9.1.3 software. For ACDM subscales and total scores, a paired t test was used.
The American Psychiatric Institute for Research and Education Institutional Review Board gave approval for the project.
Results
We were able to collect 24-month follow-up data from 15 lead psychiatrists and eight coleaders from the 17 practices that completed the original study. Fourteen psychiatrists’ surveys were available for analyses across all time points.
Prior to project initiation, a majority reported using depression assessment tools for screening and diagnostic purposes, but only 29% used these tools during treatment to monitor change in depression severity. Types of standardized depression assessment tools used at baseline included the PHQ-9, the Quick Inventory of Depressive Symptomatology (self-report), the Beck Depression Inventory, the Center for Epidemiologic Studies Depression Scale, and the Zung Self-Rating Depression Scale. At 12 and 24 months, all 14 psychiatrists who reported data for each time point stated that they usually or always used the PHQ-9 for screening, with an overwhelming majority also using it for diagnosis and monitoring of symptom severity during treatment. [A table describing depression assessment by project psychiatrists at baseline, 12 months, and 24 months is available online as a
data supplement to this report.]
Table 1 displays the results of the ACDM at baseline, 12 months, and 24 months. Survey items were grouped into six domains of the collaborative care model that psychiatrists adopted to foster changes in depression care. Psychiatrists reported substantial and statistically significant gains in all elements of the model at 12 months and at 24 months compared with baseline.
T tests comparing results from the 24-month follow-up for identical items from the ACDM and the brief survey completed by coleaders for the eight practices with both sets of data found no statistically significant differences for all the domains.
One year after the conclusion of the project, 73% (N=11) of project psychiatrists reported that at least 85% to 100% of their clinical staff utilized the PHQ-9 as a routine part of depression care. Specifically, 80% (N=12) of project psychiatrists reported that all psychiatrists at their practice site had adopted use of the PHQ-9 (range 1–23 psychiatrists). Of the nine psychiatrists who worked at practice sites that had nonphysician mental health providers (including psychologists and social workers), 56% (N=5) reported 100% adoption of the PHQ-9 among nonphysician providers (range 1–18 mental health clinicians).
A major reason cited for successful implementation of the PHQ-9 at project sites was making its use part of the standard work flow (N=9). For example, this could include placing a clinical flow sheet for PHQ-9 scores in the chart after the clinician reviewed the self-report items and scores to facilitate tracking symptom severity over time. During open-ended questions, psychiatrists were asked to identify a major barrier that was systematically addressed, and eight psychiatrists identified staff or organizational resistance to change. Only two psychiatrists indicated that time constraints was a major barrier.
Notably, 15 psychiatrists reported dissemination of the PHQ-9 for screening, diagnostic, and monitoring purposes to at least 26 other distinct general medical or behavioral health practices beyond their own, of which 50% (N=13) of these targeted external sites adopted use of the PHQ-9, and another 23% of the practices were testing implementation of the PHQ-9. One factor reportedly contributing to the success of external dissemination was engaging primary care clinicians (N=6). Among these external sites, the most commonly cited barriers were time constraints (N=7) and staff and organizational resistance to change (N=5).
Discussion
Previous studies have primarily examined the impact of QI learning collaboratives for mental health care in primary care settings (
2,
3,
10), whereas this project used an approach modified for psychiatrists working mostly in behavioral health settings. In this one-year follow-up after the original NDMLI concluded, we found that most improvements were sustained, according to psychiatrists’ self-report. In particular, practices reported success in using the PHQ-9 as part of routine depression care. These specific gains are no small task. Randomized controlled trials have found that screening, diagnostic confirmation, and measurement of symptom severity are key facets of evidence-based care for depression (
11,
12).
At both the NDMLI end point and at one-year follow-up, the two key domains of using the PHQ-9 for monitoring and self-management support, respectively, were the most improved practice changes, according to psychiatrists’ report. These results appear to support feasibility of practice changes in routine psychiatric practice for depression. While participating in NDLMI, practices implemented improvements without hiring any additional staff and were encouraged to adopt the practice changes by incrementally changing their daily clinical work flows. Many of the learning sessions specifically focused on sharing best practice changes, given that funding to support improvement changes was not available.
Many psychiatrists stated that they were able to disseminate the use of the PHQ-9 to other clinicians in their practices as well as to external practices that were generally within their professional networks. Because psychiatrists are often seen as local and regional mental health experts, their ability to influence system change when supported by a structured QI training approach may be noteworthy.
Psychiatrists also reported sustaining integration of self-management as a component of depression care. Use of self-management was one of the largest reported changes from baseline to 12-month and 24-month follow-ups. Self-management goal setting (structured depression education with handouts, medication adherence, and exercise and pleasurable activities) during treatment and formal documentation of these goals were highlighted during the NDMLI learning sessions (
6,
7). Structured self-management activities as part of an overall treatment plan may play an important role in improving depression outcomes (
7,
13,
14).
The major limitation of this study was the sole reliance on self-report surveys of the project psychiatrists. Although separate confirmatory reports among the project coleaders were found, we cannot discount the possibility of a strong positive bias. As discussed in our previous articles (
6,
7), the project goal from inception was to foster real-world practice change for depression management. Despite the fact that practices were given no funding resources to make changes, they were able to collect and report individual PHQ-9 data on over 6,000 visits by over 1,700 patients. We believe this high level of motivation was a result of the peer-based learning approach, which emphasizes making intentional changes in clinical work flows, and of the psychiatrists’ appreciation of the clinical value of the data for treatment planning and monitoring (
9).
Nonetheless, this study would have been strengthened by site visits and random chart audits from a sample of the project psychiatrists. In addition, patient confirmation of PHQ-9 monitoring and self-management goal setting might have offered a perspective on how patients view these practice changes. Future sustainability studies should incorporate such study methods to provide further support of this approach to improving outcomes.
Moreover, project psychiatrists came from a range of settings, and many were viewed as leaders in their local and regional community; therefore, the results may not be generalizable to all psychiatric practices. However, from a systems change perspective, the project psychiatrists may be the ideal candidates with whom to start significant QI efforts. They have the potential to function as change agents because of their status and ability to influence peers within and outside their practices. Conway and Clancy (
15) and others have cited the importance of using peer networks to foster change at the “front lines.” It is notable that prior to project initiation, a majority of the psychiatrists had familiarity in the use of standardized depression tools for screening and diagnosis. We speculate that future QI improvement efforts using peer-based learning approaches will benefit from the influence of champions who have certain readiness factors in place.
Conclusions
Reports by psychiatrists of the sustainability of QI interventions and, in particular, the use of the PHQ-9 for depression treatment are encouraging. The learning collaborative model was designed to maximize opportunities for practice change and sustainability. Further study and replication of this approach adding external validation, such as chart auditing and patient confirmations, will be necessary to support the utility of these methods for QI in depression care.
Acknowledgments and disclosures
Funding for this grant has been made possible by unrestricted educational grants to the American Psychiatric Foundation from Forest Laboratories, Inc., and Wyeth Pharmaceuticals. The authors acknowledge the significant contributions of the 17 psychiatric practices that participated in the original NDMLI and the subsequent follow-up. The authors also thank the consultants for their contributions to the study and staff for assisting with the project implementation.
Dr. Chung has served as a consultant to Takeda Pharmaceuticals and Lundbeck. Dr. Katzelnick is a principal shareholder of stock in Healthcare Technology Systems. Dr. Trivedi is or has been an advisor or consultant to Alkermes; AstraZeneca; Bristol-Myers Squibb Company; Cephalon, Inc.; Eli Lilly & Company; Evotec; Forest Pharmaceuticals; GlaxoSmithKline; Janssen Pharmaceutica Products, LP; Lundbeck; MedAvante; Neuronetics; Otsuka Pharmaceuticals; Pamlab; Pfizer, Inc.; Rexahn Pharmaceuticals; Sepracor; SHIRE Development; Takeda; and Tal Medical/Puretech Venture. He has received research support from Valient. The other authors report no competing interests.