Reliability and Feasibility of the First-Episode Psychosis Services Fidelity Scale–Revised for Remote Assessment
Abstract
Objective:
The authors sought to evaluate the interrater reliability and feasibility of the First-Episode Psychosis Services Fidelity Scale–Revised (FEPS-FS-R) for remote assessment of first-episode psychosis programs according to the coordinated specialty care model.
Methods:
The authors used the FEPS-FS-R to assess the fidelity of 36 first-episode psychosis program sites in the United States with information from three sources: administrative data, health record review, and phone interviews with staff. Four raters independently conducted fidelity assessments of five program sites by listening to each of the staff interviews and independently rating the two other data sources from each site. To calculate interrater reliability, the authors used intraclass correlation coefficients (ICCs) for each of the five sites and across the total scores for each site.
Results:
Total interrater reliability was in the good to excellent range, with a mean ICC of 0.91 (95% confidence interval = 0.72–0.99, p<0.001). Two first-episode psychosis program sites (6%) achieved excellent fidelity, 25 (69%) good fidelity, and nine (25%) fair fidelity. Of the 32 distinct items on the FEPS-FS-R, 23 (72%) were used with good or excellent fidelity. Most sites achieved high fidelity on most items, but five items received ratings indicating low-fidelity use at most sites. The fidelity assessment proved feasible, and sites required on average 10.5 hours for preparing and conducting the fidelity review.
Conclusions:
The FEPS-FS-R has high interrater reliability and can differentiate high-, moderate-, and low-fidelity sites. Most sites had good overall fidelity, but the FEPS-FS-R identified some services that were challenging to implement at many sites.
HIGHLIGHTS
•
The First-Episode Psychosis Services Fidelity Scale–Revised (FEPS-FS-R) can be used for reliable remote fidelity assessments and is suitable for assessing fidelity according to the coordinated specialty care model.
•
The FEPS-FS-R proved to be feasible, with a site burden average of 10.5 hours required for fidelity planning, data collection, and interviews.
•
The FEPS-FS-R differentiated among high-, moderate-, and low-fidelity sites and identified components that were challenging for many sites, which may reflect broader system difficulties delivering certain components.
•
The surveyed sites generally adhered with high fidelity to standards of services provided to clients with first-episode psychosis.
Systematic reviews have summarized extensive evidence supporting the superiority of intensive, team-based care for treating a first episode of a psychotic disorder (1). In the United States, a large-cluster randomized controlled trial demonstrated the feasibility of delivering such programs in community mental health services while retaining their clinical superiority (2).
In 2014, the U.S. Congress approved legislation that mandated states to direct 5% of funding (increased to 10% in 2016) received through the Mental Health Block Grant (MHBG) to early intervention programs for people with first-episode psychosis (3). This 10% set-aside fund has contributed to the establishment of 236 coordinated specialty care (CSC) programs across the United States (4). A self-report survey of CSC program leaders identified 107 measures to assess fidelity to CSC components in first-episode psychosis services (4). Most measures were not formal fidelity scales, suggesting the need for a reliable, valid, and feasible fidelity scale.
Fidelity scales have been developed in other mental health services to assess implementation and the degree to which programs adhere to evidence-based services (5). An international review identified several fidelity scales for first-episode psychosis services (FEPSs) (6); however, most research studies on the effectiveness of FEPSs have lacked assessments of fidelity that would support successful implementation. In the Recovery After an Initial Schizophrenia Episode study, Mueser et al. (7) comprehensively assessed the fidelity for treatment components by using a set of diverse assessment procedures, including direct observation of the clinician’s practice through videos and adapting assessment methods used in other psychosocial interventions for individuals with severe mental illness. Such labor-intensive methods are feasible in research studies but not for widescale assessment of fidelity in routine practice. We sought to develop a scale that would be useful for both research and quality-improvement purposes.
This study is drawn from a national prospective study of first-episode programs. The research team invited 250 CSC programs across the United States receiving MHBG 10% set-aside funds to participate and ultimately enrolled 36 sites. The sites were selected to ensure geographic diversity, rural representation, and diverse service models. Participating programs received a payment to cover the costs of study participation. One component of this study was assessing fidelity. The fidelity measure that the research team selected was the First-Episode Psychosis Services Fidelity Scale (FEPS-FS) (8). Unlike fidelity scales based on specific program models (9, 10), the FEPS-FS was systematically developed with a standardized methodology for developing fidelity scales (11). This process involved a systematic review to identify program components, an assessment of the level of evidence supporting these components, and an international expert consensus process to identify the essential components (12). The scale has been tested for face validity, interrater reliability, and feasibility of programs in Canada and the United States (12, 13). It has not yet been tested for predictive validity.
During the time when Addington et al. (8, 12) were developing and testing the FEPS-FS, Heinssen and colleagues (14) proposed the CSC model for first-episode psychosis programs. This model includes 12 key components composed of six key roles and clinical services and six core functions. This model has provided a shared framework for the emerging expansion of first-episode programs in the United States in response to the MHBG 10% set-aside funding. If validated, this framework promises to promote communication among both researchers and program managers, permitting standardized comparisons among programs. A conceptual framework is not sufficient for operationalizing a program model; however, a fidelity scale is needed to provide a roadmap for program leaders and to measure the components of the model.
In this article, we examined the interrater reliability of a revised fidelity scale, the FEPS-FS–Revised (FEPS-FS-R), and evaluated the feasibility of its application in a remote-assessment process. We also describe the variability of fidelity across the sample and the degree to which the sites adhered to the CSC components. We found that clusters of programs received training and support from different groups, and we explored potential differences in fidelity among these programs.
Methods
Overview
As part of a 3-year project evaluating a national sample of first-episode psychosis programs funded through the MHBG 10% set-aside, we conducted a psychometric study of the FEPS-FS-R. During the first project year, the fidelity research team modified the FEPS-FS, revised the fidelity assessment manual, and trained the primary fidelity rater on the fidelity assessment procedures. The fidelity rater underwent a 2-day training program. The first day comprised a review of the scale and the manual, and the second day involved rating a real program. Follow-up training comprised joint interviews of site staff and discussions about individual study site ratings.
We made minor modifications to the scale content on the basis of the CSC framework and feedback from our funders and research collaborators. Major modifications concerned the use of remote-assessment procedures. Addington et al. (8) had worked with experienced first-episode psychosis clinicians as assessors to develop and validate onsite observational procedures for conducting the fidelity assessment. For the current project, we redesigned the procedures to accommodate large-scale fidelity and conduct assessments without a site visit. We developed a structured interview that could be conducted by phone or in person, procedures for abstracting health record data that could be uploaded to a secure website, and remote training for onsite health record abstractors.
During the second project year, we piloted the new assessment procedures at the study sites. We further improved the scale during this phase on the basis of feedback from multiple sources, including regular discussions among the fidelity research team after individual and joint fidelity assessments, the results of an interrater reliability study, and findings from another study of the FEPS-FS (13). During the third project year, we assessed the 36 study sites using the revised scale and assessment procedures.
Study Sites
Thirty-six sites were selected from 250 sites receiving MHBG 10% set-aside funds. The sites were selected to include all U.S. regions, representing rural, urban, and suburban areas.
Data Collection Procedures
For each site, a trained fidelity assessor conducted a fidelity review and completed the fidelity ratings. Three researchers served as fidelity assessors completing 26, six, and four reviews, respectively. The data used to rate fidelity included staff interviews, a review of health records, and program documents. We interviewed at least four staff members, including team leader, prescriber, case manager or care coordinator, and supported employment or education specialist. Each site identified a local health record abstractor to complete the health record review. To enhance efficiency of data collection and ensure that the fidelity assessor obtained the best available data, we developed procedures to educate the site’s team leader and health record abstractor.
First, a research assistant contacted the site’s primary contact (typically the team leader) to schedule a team leader orientation, consisting of a 30-minute, one-on-one webinar with the study coordinator, outlining fidelity review procedures and requirements. The fidelity review team also provided the team leader with a fidelity scale guide for site leaders. Next, each site’s health record abstractor participated in a 30-minute webinar with the study coordinator, explaining how to complete the health record checklist. The fidelity review team also provided a health record abstractor guide (included in the site leader guide). The health record abstractor then recorded deidentified client information extracted from 10 randomly selected client charts. Each site also identified a staff member to upload required documents for the fidelity review and the completed health record checklist. These materials were uploaded to a secure document transfer portal and retrieved by the fidelity research team.
The administrative data and the health record review included information that required transcribing factual information, such as staffing levels as well as prescribed medications and dosages for individual clients, directly from the site’s records. Apart from clerical errors, the accuracy of these data may be presumed high. Other data were cross-checked during the interview. Some health record review data, such as the number of psychoeducation sessions attended, did require record reviewer judgment. The FEPS-FS-R review manual further details data collection and fidelity item rating procedures.
Fidelity Measure
The FEPS-FS-R consists of 33 distinct items; supported employment and supported education were labeled 27A and 27B. Each item is rated on a 5-point behaviorally anchored continuum, with a rating of 5 indicating full adherence to the evidence base (or best practice guidelines) and a rating of 1 representing substantial lack of model adherence. Ratings of 4, 3, and 2 represent equally spaced increments between these two extremes. To improve the reliability of the ratings, we designed the fidelity scale to consist of items that require little clinician inference.
Following the scoring conventions used with standardized evidence-based practice fidelity scales (15), we summed item ratings to obtain a total score with values ranging from 33 to 165. A total score of ≥149 (i.e., ≥90% of the maximum) is considered “excellent” fidelity, between 132 and 148 (i.e., between 80% and 89%) “good” fidelity, between 116 and 131 (i.e., between 70% and 79%) “fair” fidelity, and of <116 (i.e., <70%) “poor” fidelity.
Reliability Substudy
During the final project year, four independent raters used the 33-item FEPS-FS-R to rate a convenience sample of five sites, independently reviewing the site’s administrative and health record data. Each rater also attended the phone interviews with the four staff members at each site. These interviews were conducted by an interviewer who was also one of the raters. The raters each independently rated fidelity on the basis of data from the three sources of information.
Data Analysis
For the interrater reliability substudy, we used intraclass correlation coefficients (ICCs) for each of the five sites and across the total scores to assess reliability. We selected a two-way, mixed-effects model (single-rater type and absolute agreement) (16) and for analysis with IBM SPSS Statistics, version 25. We used the single-rater type because the standard fidelity assessment protocol calls for fidelity assessment rating by one person. We used the absolute agreement option for estimating reliability because the fidelity scale ultimately will be used to establish benchmarks, which depend on specific cutoff scores. We assessed feasibility by asking these sites about the time required for the data collection and submission, and we recorded the time required for fidelity interviews.
Next, we calculated M and SD values; distributions of item scores across the sites for full (rating=5), adequate (rating=4), and poor (rating=1–3) fidelity; and site scores at 18 months. We also tested potential predictors of fidelity, including urbanicity of the community served, adherence to specific CSC models, and program size measured by number of clients served and full-time equivalent staffing, using analysis of variance and Pearson correlations.
Following HIPAA requirements, the research team did not receive any data with personal identifying information. Local ethics review boards reviewed and approved the research study protocol. The administrative data and those abstracted from health records were uploaded to a secure website.
Results
All 36 (100%) of the MHBG study sites completed the fidelity reviews. The number of staff interviewed at each site varied, and primary interviews were supplemented with information from other interviewees when the designated interviewee was unable to provide critical information. As shown in Table 1, site-level interrater reliability ranged from good to excellent (ICC of ≥0.80) for all five sites. The ICC value for the total scores across each site was 0.91 (95% confidence interval=0.72–0.99, p<0.001), also suggesting excellent reliability. The mean range in total scores across the four raters was 5.2 (N=5 sites); excluding the least experienced fidelity rater (who generally gave lower ratings), the mean score range was 3.4, further suggesting convergence in ratings. We examined item-level ratings for the five sites and found no obvious pattern in rating discrepancies by item.
| Study site | Total scoreb | ICC | 95% CI |
|---|---|---|---|
| Site A | 136 | .86 | .82–.89 |
| Site B | 128 | .80 | .69–.88 |
| Site C | 123 | .90 | .84–.90 |
| Site D | 140 | .98 | .96–.99 |
| Site E | 149 | .81 | .71–.89 |
a
FEPS-FS-R, First-Episode Psychosis Services Fidelity Scale–Revised; ICC, intraclass correlation coefficient.
b
Mean score across four raters.
The mean time the CSC team required for collecting and submitting data and for completing staff interviews was 10.5 hours (Table 2). In Table 3, we present descriptive statistics (M, SD, and frequency distributions) for the 32 items of the FEPS-FS-R in the sample of the 36 sites. Most sites achieved high fidelity on a substantial majority of items. The mean item rating was 4.16, and 23 items (72%) were used with good or excellent fidelity. Two sites (6%) achieved excellent fidelity, 25 (69%) achieved good fidelity, and nine (25%) achieved fair fidelity. No site scored below the minimum for fair fidelity.
| Fidelity review element | Time (hours) |
|---|---|
| Team leader interview | 1.5 |
| Employment or education specialist | .5 |
| Psychiatrist interview | .5 |
| Clinician or therapist interview | 1.0 |
| Team leader training | .5 |
| Health record review training | .5 |
| Compile and upload required documents | 3.0 |
| Health record audit | 3.0 |
| All | 10.5 |
| Site fidelity | ||||||||
|---|---|---|---|---|---|---|---|---|
| Item score | Low (rating=1–3) | Good (rating=4) | Excellent (rating=5) | |||||
| FEPS-FS-R item no. and description | M | SD | N | % | N | % | N | % |
| 1. Practicing team leader | 4.89 | .32 | 0 | 0 | 4 | 11 | 32 | 89 |
| 2. Client-clinical staff ratio | 4.94 | .32 | 0 | 0 | 2 | 6 | 34 | 94 |
| 3. Range of services delivered by team | 4.74 | .51 | 1 | 3 | 7 | 19 | 28 | 78 |
| 4. Assigned case manager | 5.00 | .00 | 0 | 0 | 0 | 0 | 36 | 100 |
| 5. Assigned prescriber | 4.34 | 1.19 | 6 | 17 | 5 | 14 | 25 | 69 |
| 6. Prescriber role on team | 4.80 | .41 | 0 | 0 | 8 | 22 | 28 | 78 |
| 7. Weekly multidisciplinary team meetings | 4.89 | .68 | 2 | 6 | 0 | 0 | 34 | 94 |
| 8. Explicit admission criteria | 4.94 | .24 | 0 | 0 | 2 | 6 | 34 | 94 |
| 9. Population served | 1.69 | 1.16 | 33 | 92 | 1 | 3 | 2 | 6 |
| 10. Duration of FEP program | 2.80 | .93 | 32 | 89 | 3 | 8 | 1 | 3 |
| 11. Community group outreach | 3.86 | 1.31 | 12 | 33 | 7 | 19 | 17 | 47 |
| 12. Early intervention | 2.17 | 1.25 | 30 | 83 | 4 | 11 | 2 | 6 |
| 13. Timely contact with referred individual | 4.26 | 1.01 | 8 | 22 | 9 | 25 | 19 | 53 |
| 14. Family involvement in initial assessment | 4.60 | .81 | 2 | 6 | 8 | 22 | 26 | 72 |
| 15. Comprehensive clinical assessment | 4.94 | .24 | 0 | 0 | 2 | 6 | 34 | 94 |
| 16. Comprehensive psychosocial needs assessment | 4.89 | .47 | 2 | 6 | 0 | 0 | 34 | 94 |
| 17. Treatment or care plan | 4.77 | .60 | 2 | 6 | 5 | 14 | 29 | 81 |
| 18. Antipsychotic prescription after diagnosis | 4.94 | .24 | 0 | 0 | 2 | 6 | 34 | 94 |
| 19. Antipsychotic dosing within guidelines | 4.23 | .65 | 5 | 14 | 19 | 53 | 12 | 33 |
| 20. Clozapine for medication-resistant symptoms | 3.51 | 1.92 | 14 | 39 | 1 | 3 | 21 | 58 |
| 21. Client psychoeducation | 4.51 | .92 | 3 | 8 | 8 | 22 | 25 | 69 |
| 22. Family psychoeducation | 3.11 | 1.53 | 21 | 58 | 3 | 8 | 12 | 33 |
| 23. Individual or group CBT | 4.26 | 1.20 | 6 | 17 | 8 | 22 | 22 | 61 |
| 24. Supporting health management | 4.26 | 1.04 | 6 | 17 | 12 | 33 | 18 | 50 |
| 25. Annual formal comprehensive assessment | 4.74 | .70 | 3 | 8 | 2 | 6 | 31 | 86 |
| 26. Services for clients with substance use disorders | 4.57 | .61 | 2 | 6 | 11 | 31 | 23 | 64 |
| 27A. Supported employment | 2.80 | 1.59 | 24 | 67 | 4 | 11 | 8 | 22 |
| 27B. Supported education | 3.69 | 1.88 | 12 | 33 | 0 | 0 | 24 | 67 |
| 28. Client engagement out of office | 4.00 | 1.33 | 12 | 33 | 3 | 8 | 21 | 58 |
| 29. Client retention | 3.74 | 1.12 | 14 | 39 | 12 | 33 | 10 | 28 |
| 30. Crisis intervention services | 4.34 | .64 | 2 | 6 | 20 | 56 | 14 | 39 |
| 31. Communication between FEPS and inpatient | 3.40 | 1.56 | 17 | 47 | 7 | 19 | 12 | 33 |
| 32. Timely contact after hospital discharge | 4.51 | 1.22 | 4 | 11 | 2 | 6 | 30 | 83 |
a
CBT, cognitive-behavioral therapy; FEP, first-episode psychosis; FEPS, first-episode psychosis services; FEPS-FS-R, First-Episode Psychosis Services Fidelity Scale–Revised; MHBG, Mental Health Block Grant.
Five items received low-fidelity ratings (median rating <4) at most sites: community served (<60% of incident cases from the catchment area were admitted to the program), duration of first-episode psychosis program (program served patients for ≤2 years), early intervention (≥40% of patients had a psychiatric hospitalization before CSC admission), family psychoeducation (<70% of families received an evidence-based family education program), and supported employment (employment services met no more than five of seven evidence-based elements of supported employment).
The results for the fidelity ratings of 12 CSC components (14) are presented in Table 4. Two components, transition of care and assuring fidelity to the early intervention treatment model, were not assessed. All but one site achieved good fidelity to the first three components (team-based approach, team leadership, and case management). One-third of the sites delivered good fidelity to supported employment, and two-thirds provided supported education with good fidelity. More than four-fifths of sites provided good fidelity to evidence-based psychotherapy, pharmacotherapy, and health management. Regarding family education and support, families were consistently involved in the initial assessment at all but two sites; however, a minority of sites delivered good fidelity to ongoing family education and support.
| Sites meeting fidelity standards | |||
|---|---|---|---|
| Key CSC roles and clinical serviceb | FEPS-FS-R components | N | % |
| Team leadership | Practicing team leader | 36 | 100 |
| Case management | Assigned case manager | 36 | 100 |
| Supported employment | Supported employment | 12 | 33 |
| Supported education | Supported education | 24 | 67 |
| Psychotherapy | Cognitive behavioral therapy | 30 | 83 |
| Family education and support | Family education and support | 15 | 42 |
| Pharmacotherapy | Antipsychotic prescribed | 36 | 100 |
| Primary care coordination | Supporting health management | 30 | 83 |
| Community outreach | Targeted education to health/social service community groups | 24 | 67 |
| Client and family engagement | Family involvement in assessment | 34 | 94 |
| Mobile outreach | Active engagement and retention | 24 | 67 |
| Crisis intervention services | Crisis intervention services | 34 | 94 |
| Team-based approach | Services delivered by team | 35 | 97 |
| Team-based approach | Weekly team meeting | 34 | 94 |
| Transition of care | Not addressed by FEPS-FS-R | NA | NA |
| Fidelity to CSC model | Not addressed by FEPS-FS-R | NA | NA |
a
MHBG, Mental Health Block Grant; NA, not applicable.
b
Coordinated specialty care (CSC) components are matched to First-Episode Psychosis Services Fidelity Scale–Revised (FEPS-FS-R) items (see Table 3). Sites scoring 4 or 5 on the FEPS-FS-R were considered as meeting fidelity standards. Staff training and supervision, as part of specialized training in first-episode psychosis care, are not identified as a separate FEPS-FS-R component. They are incorporated into the service delivery ratings for supported employment and education, psychotherapy, and family education and support.
We examined three potential predictors of fidelity: urbanicity of community served, adherence to specific CSC models, and program size. Urban, suburban, and rural sites did not differ significantly in fidelity. Sites also did not differ in fidelity according to the primary model followed according to five different CSC models: The Early Assessment and Support Alliance (EASA), Coordinated Specialty Care for First Episode Psychosis (FIRST), NAVIGATE Coordinated Specialty Care, OnTrackNY Coordinated Specialty Care, and Portland Identification and Early Referral program (PIER). The overall F test for differences among groups was not statistically significant. Both of the measures regarding program size did not correlate with fidelity (number of clients served: r=0.21, p=0.23; full-time equivalent staffing: r=0.18, p=0.30).
Discussion
In this study, we evaluated the feasibility and reliability of the FEPS-FS-R procedures for remote assessment of fidelity of first-episode psychosis services to the CSC model, finding that these procedures can be conducted with good to excellent interrater reliability. The average time commitment for programs previously completing an initial review was 10.5 hours, suggesting that the process is both feasible from a program perspective and sustainable from a system perspective. We identified five CSC components that most programs found challenging to implement with fidelity. Supported employment received low ratings at some sites because of the lack of training or awareness of evidence-based principles. The early intervention item also received low ratings, and we note that this fidelity item rating is measured by using the percentage of caseloads with a psychiatric hospitalization before program enrollment. Our findings that 60%–80% of patients had received inpatient care before program enrollment are consistent with those from the Recovery After an Initial Schizophrenia Episode study, which found that 78% of patients had undergone a previous hospitalization (17). Most sites reported that CSC enrollment was usually limited to 2 years. Increasingly, research supports the benefits of providing services for longer periods, that is, >2 years (18–21). Population served is an item comparing caseload size for a site with the estimated incidence of first-episode psychosis in communities served by the site. Most sites fell short of this standard, which estimates the demand for services for all those who might benefit.
Fidelity of evidence-based practices is commonly assessed by computing a total score based on the sum of individual items without weighting the items (6). A unit-weighting scheme has been shown to be statistically justified, but a simple scoring system is more readily adopted for quality improvement in routine practice. Nonetheless, we recommend that the total score not be used in isolation but interpreted in light of the individual item scores. In addition, further research should determine whether some items should be removed for either psychometric or substantive reasons. Unlike some other first-episode psychosis fidelity scales, the FEPS-FS-R does not categorize items into essential and secondary components (9), which would lead to reporting of subscale scores.
The small sample size suggests caution in interpreting a lack of statistically significant predictors of fidelity, and we note nonsignificant differences in fidelity across teams adopting one of several well-known CSC models, such as NAVIGATE. Thus, we found no evidence that the FEPS-FS-R is inappropriate for a specific “brand name” model. This finding may reflect the fact that FEPS-FS-R was developed after a systematic review of the literature on FEPS support (8, 12) rather than from a single effective program model as done previously (9, 22). We also did not find that the type of community served, adherence to specific CSC models, or program size correlate with fidelity. The FEPS-FS-R items gave good coverage of most components identified in the CSC model, suggesting that this scale is an appropriate measure for assessing fidelity to the CSC model.
The limitations of this study included a small and nonrandomly selected sample, both of which limit the generalizability of the findings. The correlational analyses were limited by small sample size and a restricted range. Some items exhibited little variation, suggesting the need for recalibration of the item rating scale. Some fidelity item ratings were based exclusively on the administrative data and the health record data provided by the site, the accuracy of which was not independently verified. Staff interviews may have introduced self-report bias. Finally, future research is needed to assess the generalizability of the interrater reliability found here to other fidelity reviewers.
Conclusions
The findings of this study support the conclusion that remote fidelity assessment with the FEPS-FS-R has face validity and can be both reliable and feasible. Importantly, the scale distinguished between sites rated as having excellent, good, or poor fidelity, suggesting that it is sensitive to differences among sites due to specific practice patterns. Finally, it indicates that the scale assesses the components of the CSC model.
The main finding regarding fidelity to CSC in the United States is that most sites achieved high fidelity on most fidelity items. Further research with larger samples is needed to evaluate whether the CSC model is robust and feasible to implement in a range of settings. These results support the use of the FEPS-FS-R scale as an outcome measure for implementation studies and as a practical tool for quality assurance by health care providers or funders. Further research is required to test the predictive validity of the scale.
References
1.
Fusar-Poli P, McGorry PD, Kane JM: Improving outcomes of first-episode psychosis: an overview. World Psychiatry 2017; 16:251–265
2.
Kane JM, Robinson DG, Schooler NR, et al: Comprehensive versus usual community care for first-episode psychosis: 2-year outcomes from the NIMH RAISE early treatment program. Am J Psychiatry 2016; 173:362–372
3.
Azrin ST, Goldstein AB, Heinssen RK: Expansion of coordinated specialty care for first-episode psychosis in the US. Focal Point 2016; 30:3
4.
Lutterman T, Kazandjian M, Urff J: Snapshot of State Plans for Using the Community Mental Health Block Grant Ten Percent Set-Aside to Address First Episode Psychosis. Rockville, MD, Substance Abuse and Mental Health Services Administration, 2018., www.nasmhpd.org/sites/default/files/Snapshot_of_State_Plans.pdf, Accessed June 11, 2020
5.
Bond GR, Drake RE: Assessing the fidelity of evidence-based practices: history and current status of a standardized measurement methodology. Adm Policy Ment Health Ment Health Serv Res 2019.
6.
Addington D, Birchwood M, Jones P, et al: Fidelity scales and performance measures to support implementation and quality assurance for first episode psychosis services. Early Interv Psychiatry 2018; 12:1235–1242
7.
Mueser KT, Meyer-Kalos PS, Glynn SM, et al: Implementation and fidelity assessment of the NAVIGATE treatment program for first episode psychosis in a multi-site study. Schizophr Res 2019; 204:271–281
8.
Addington DE, Norman R, Bond GR, et al: Development and testing of the First-Episode Psychosis Services Fidelity Scale. Psychiatr Serv 2016; 67:1023–1025
9.
Melau M, Albert N, Nordentoft M: Development of a fidelity scale for Danish specialized early interventions service. Early Interv Psychiatry 2019; 13:568–573
10.
Killackey E, van der El K, Stavely H: A fidelity tool for the Australian early psychosis system. Schizophr Bull 2018; 44:S321
11.
Bond G, Williams J, Evans L, et al: Psychiatric Rehabilitation Fidelity Toolkit. Cambridge, MA, Human Services Research Institute, 2000., www.hsri.org/publication/psychiatric_rehabilitation_fidelity_toolkit, Accessed June 11, 2020
12.
Addington DE, McKenzie E, Norman R, et al: Essential evidence-based components of first-episode psychosis services. Psychiatr Serv 2013; 64:452–457
13.
Durbin J, Selick A, Langill G, et al: Using fidelity measurement to assess quality of early psychosis intervention services in Ontario. Psychiatr Serv 2019; 70:840–844
14.
Heinssen R, Goldstein AB, Azrin ST: Evidence-Based Treatments for First Episode Psychosis: Components of Coordinated Care. Bethesda, MD, National Institute of Mental Health, 2014., www.nimh.nih.gov/health/topics/schizophrenia/raise/evidence-based-treatments-for-first-episode-psychosis-components-of-coordinated-specialty-care.shtml, Accessed June 11, 2020
15.
McHugo GJ, Drake RE, Whitley R, et al: Fidelity outcomes in the National Implementing Evidence-Based Practices Project. Psychiatr Serv 2007; 58:1279–1284
16.
Koo TK, Li MY: A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016; 15:155–163
17.
Robinson DG, Schooler NR, Rosenheck RA, et al: Predictors of hospitalization of individuals with first-episode psychosis: data from a 2-year follow-up of the RAISE-ETP. Psychiatr Serv 2019; 70:569–577
18.
Lutgens D, Iyer S, Joober R, et al: A five-year randomized parallel and blinded clinical trial of an extended specialized early intervention vs. regular care in the early phase of psychotic disorders: study protocol. BMC Psychiatry 2015; 15:22
19.
Malla A, Joober R, Iyer S, et al: Comparing three-year extension of early intervention service to regular care following two years of early intervention service in first-episode psychosis: a randomized single blind clinical trial. World Psychiatry 2017; 16:278–286
20.
Albert N, Melau M, Jensen H, et al: Five years of specialised early intervention versus two years of specialised early intervention followed by three years of standard treatment for patients with a first episode psychosis: randomised, superiority, parallel group trial in Denmark (OPUS II). BMJ 2017; 356:i6681
21.
Chang WC, Chan GH, Jim OT, et al: Optimal duration of an early intervention programme for first-episode psychosis: randomised controlled trial. Br J Psychiatry 2015; 206:492–500
22.
Killackey E: The EPPIC Model Integrity Tool—Application Guide. Melbourne, Orygen, 2016. www.orygen.org.au/Training/Resources/Psychosis/Guidelines/EPPIC-Model-Integrity-Tool-%e2%80%93-Application-Guide. Accessed June 11, 2020
Information & Authors
Information
Published In
History
Received: 2 February 2020
Revision received: 21 May 2020
Accepted: 27 May 2020
Published online: 27 October 2020
Published in print: December 01, 2020
Keywords
Authors
Competing Interests
The authors report no financial relationships with commercial interests.
Funding Information
This study was funded by a grant from the Substance Abuse and Mental Health Services Administration, in collaboration with the National Institute of Mental Health and the Office of the Assistant Secretary for Planning and Evaluation (Mental Health Block Grant 10% set-aside evaluation HHSS283201200011I/HHSS28342008T, RFTOP No. 280-16-0482).
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBLogin options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).