The Danish OPUS Early Intervention Services for First-Episode Psychosis: A Phase 4 Prospective Cohort Study With Comparison of Randomized Trial and Real-World Data
Publication: American Journal of Psychiatry
Abstract
Objective:
The Danish OPUS trial showed significant efficacy of early intervention services for first-episode schizophrenia spectrum disorders compared with standard treatment, leading to implementation of the OPUS intervention in clinical practice. The authors sought to determine whether the effectiveness of OPUS treatment in real-world clinical practice is comparable to the efficacy seen in the trial.
Methods:
The study compared patients who received OPUS treatment as part of the original randomized trial to those who received standard treatment in the trial (the control group) and those who received OPUS treatment after it was implemented in Denmark. The authors investigated whether the three groups differed on register-based outcomes, such as use of secondary health care, functional outcomes, and death. Analyses were adjusted for relevant confounders.
Results:
Compared with trial study participants, patients who received OPUS treatment after implementation (N=3,328) had a tendency toward lower mortality (hazard ratio=0.60, 95% CI=0.33, 1.09), fewer and shorter psychiatric admissions, and possibly fewer filled prescriptions of antipsychotics and other psycholeptics after 4 or 5 years. While at first less likely to be working or studying, patients who received postimplementation OPUS treatment eventually had higher odds of working than did those in the OPUS trial (after 5 years, odds ratio=1.49, 95% CI=1.07, 2.09). The odds of being in a couple relationship were also higher among patients in the postimplementation group than those in the trial. Other outcomes showed less clear associations with treatment group. Generally, the control group in the trial fared worse than both of the OPUS treatment groups.
Conclusions:
Not only did OPUS treatment maintain its efficacy after it was implemented as a standard treatment, it paralleled or surpassed many of the effects observed when the OPUS intervention was delivered in a randomized trial. The study results provide further evidence in support of implementation and funding of early intervention services worldwide.
A recent systematic review concluded that early intervention services for early-phase psychosis are superior to treatment as usual (1). These robust findings were also seen in the Danish OPUS trial, which was one of the first trials in the world to investigate early intervention services for first-episode psychosis in the schizophrenia spectrum. Treatment in OPUS consists of modified assertive treatment including family involvement and social skills training provided by a multidisciplinary team of psychiatrists, psychologists, nurses, social workers, physiotherapists, and vocational therapists (2, 3). OPUS treatment showed significant effects in a range of areas, including psychotic and negative symptoms, everyday functioning, and substance abuse, among other measures, compared with treatment as usual (3). These findings led to the implementation of integrated services such as OPUS treatment in many countries around the world (4). Implementation in Denmark was facilitated by training courses and treatment manuals and by national grants specifically allocated for regional implementation (5). A high level of support is continuously provided through a national network of OPUS centers, as well as annual meetings (5).
It has been debated whether the consistent significant advantage of early intervention services at the end of a trial can be generalized to patients in daily, real-world routine practice conditions not captured in randomized trials (1). Patients in phase 3 trials may have been carefully selected and randomized to avoid the influence of confounding variables, and patients with certain characteristics may have been excluded (6). Moreover, in trials there is a rigorous attention to program fidelity, such as maintaining a low caseload, and the patients and personnel may be more motivated during a trial compared with those in real-world daily clinical practice (6, 7).
A.L. Cochrane pioneered the design of modern randomized trials, and he understood well the limitations of this methodology, highlighting that the relevance of the results of trials depends on external validity and generalizability to patients in daily clinical practice settings (8, 9). However, historically, the potential gap between randomized trials and routine practice has been neglected, both in medical science and in the field of psychiatric psychosocial interventions (9). Today in medical science, there is an increasing number of publications of real-world studies of pharmaceutics, referred to as phase 4 studies or postimplementation studies, where the efficacy of the intervention in the trial is compared with the effectiveness of the intervention in a real-life setting (10). This approach has several benefits, including detection of rare side effects and the possibility of testing whether a medication is still effective when provided outside the constraints of a randomized trial. For instance, for safety or other reasons, many trials include only a fraction of the types of patients who would eventually receive the medication (11).
Unfortunately, we do not see the same trend within the field of psychosocial interventions, and, to our knowledge, no one has investigated whether the effectiveness of early intervention services in a real-world daily routine clinical practice is comparable to the efficacy seen in randomized trials. Early invention services for psychosis are, by definition, a complex psychosocial intervention, and, once implemented, they may not be delivered with the same rigorous attention to program fidelity, low caseload, motivation, and so on, as they had in the controlled environment of a randomized trial (12). For these reasons, early intervention services may not be as efficient in daily routine practice as in the randomized trials. Establishing whether the effects of early intervention services such as OPUS are still effective once implemented could help identify the need to tighten treatment protocols or to develop these protocols even further. In times of health care cost cuts around the world, it is also of utmost importance to conduct phase 4 or real-world studies of early intervention services for psychosis and to document effectiveness in routine practice settings (1).
Our aim in this study was to compare the prognosis of patients who received OPUS treatment after it was implemented nationally in Denmark as first-choice treatment for first-episode psychosis with that of patients who participated in the original randomized trial and received either the OPUS intervention or the usual-treatment control condition. For all outcomes, we hypothesized a priori that the OPUS intervention in the trial would be superior to the OPUS intervention as implemented in the real world, and that the control group in the trial would have the poorest prognosis of the three groups.
Methods
We combined data from the original Danish OPUS trial with data from the nationwide Danish registers (3).
Populations
We included the 547 original participants in the OPUS trial (1998–2001), of whom 275 were originally randomized to OPUS treatment (henceforth “OPUS-RCT”) and 272 to standard treatment (henceforth “control-RCT”). We further included 3,328 individuals identified in the Psychiatric Central Research Register as having received OPUS treatment (henceforth “OPUS-real-world”) in one of 17 centers after OPUS was implemented nationally in 2003 and until December 31, 2014 (13). This time frame was chosen to ensure that the population would both have time to finish OPUS treatment and have at least 2 years of follow-up in the registers. The caseload was about 1:10 in the OPUS-RCT arm, and between 1:20 and 1:30 in the control-RCT arm (3). Caseload in OPUS-real-world is more difficult to establish, since it has not been part of a trial and likely has some variability over both time and across treatment sites. The most accurate information regarding caseload in this group is that between 2009 and 2012, it varied between 1:12 and 1:15 (2). The duration of all three treatments was 2 years.
Outcomes
Using the Psychiatric Central Register, the following outcomes were estimated: time to first psychiatric hospitalization following end of treatment (defined as 2 × 365 days after start of treatment for all individuals), number of psychiatric hospitalizations per year after start of treatment, number of psychiatric bed days per year after start of treatment, and number of psychiatric outpatient visits per year after start of treatment. Using the National Patient Register (14), we also estimated number of nonpsychiatric hospitalizations per year and number of nonpsychiatric bed days per year after start of treatment. Use of antipsychotic or other psychotropic medication (ATC codes N05 [psycholeptics] and N06 [psychoanaleptics]), summarized as defined daily doses per year after start of treatment, was estimated using the National Prescription Register (15). Registered alcohol or substance use disorder per year after start of treatment was identified through a combination of registers. Within the Psychiatric Central Register and the National Patient Register, this was identified as contacts with the secondary treatment sector (inpatient or outpatient, during the OPUS trial or during postimplementation OPUS treatment) in which an ICD-10 diagnosis of F1x.x was given, or by referral to dedicated alcohol or substance use disorder treatment clinics, as registered in the National Alcohol Treatment Register or the National Substance Abuse Register. Time to death of any cause was estimated using the Danish Civil Registration System, which was updated for mortality through December 31, 2018 (16). Time to death by suicide was estimated using the Cause of Death register, which was updated through December 31, 2017 (17). Annual employment status, living in a couple relationship, and living with children per year after start of treatment were estimated using population registers from Statistics Denmark (18).
Confounders
The following variables were constructed and used for confounder control, using the same registers and information as provided above: number of psychiatric bed days 2 years before start of treatment, number of nonpsychiatric bed days 2 years before start of treatment, number of redeemed prescriptions for antipsychotic or other psychotropic medication 2 years before start of treatment, registered alcohol or substance use disorder 2 years before start of treatment, income 2 years before start of treatment (standardized to 1996 levels using an annual inflation rate of 2.2%), employment status 2 years before start of treatment, and sex and age at start of treatment as identified through the Danish Civil Registration System. Diagnosis was used as a potential confounding variable, obtained either from clinical interviews in the original OPUS trial or from diagnoses listed in the Psychiatric Central Research Register. Because for the two groups in the trial there were some differences in diagnosis registered in the trial and in the registers, we reran the analyses using the register diagnoses for these groups, which had virtually no influence on the results (data not shown).
Statistical Analysis
Time-to-event data were analyzed using Cox proportional hazards regression, estimating hazard ratios, except for time to suicide, which was analyzed using Fine and Gray’s method for competing risk regression, estimating subhazard ratios (19). For the time-to-death analyses, individuals were entered into the analyses at start of treatment (or randomization for the OPUS trial groups). For the time-to-psychiatric-admission analyses, we performed separate analyses in which individuals were entered into the analyses either at the start of treatment or 2 years after this date (since increased hospitalization the first 2 years could have been facilitated by OPUS case managers). In both cases, people were followed until the outcome or until censoring due to migration, end of registers (December 31, 2018, for psychiatric admission and all-cause mortality, and December 31, 2017, for suicide), death (for the analyses on psychiatric admission), or death from other causes (for suicide). Count data (number of bed days, hospitalizations, and prescriptions) were analyzed using negative binomial regression with the natural logarithm of number of days under observation as offset. Negative binomial regression was chosen because, after initially using Poisson regression, it was clear that the data were overdispersed. Dichotomous data (working or studying, living in a couple relationship, having children living at home, registered alcohol or substance use disorder) were analyzed using binary logistic regression. Both count data and dichotomous data were analyzed for each of the first 5 years after start of treatment (or randomization for the OPUS trial groups). We chose 5 years of follow-up because the original OPUS trial showed positive results after 5 years but no further (20, 21). For all analyses, we estimated two models: one without any confounder adjustment, and one adjusted for all of the covariates described above. A two-sided p value <0.05 was considered statistically significant. For psychiatric admissions and antipsychotic medication, we conducted sensitivity analyses for the period ending in 2008 to assess confounding by trends unrelated to implementation of OPUS. We also conducted sensitivity analyses including time from first admission with schizophrenia to start in treatment as an indicator of duration of untreated psychosis. As this had virtually no influence on results, and because this was a suboptimal indicator, these data are not presented. All analyses were conducted in Stata/MP, version 15.1.
Ethical Considerations
Register-based studies do not require ethical approval according to Danish law. The study was approved by the Danish Data Protection Agency (VD-2018–398, I-Suite 6648). Analyses were conducted on pseudo-anonymized data, using encrypted personal identification numbers, on servers hosted at Statistics Denmark.
Results
Of the 547 people who took part in the original OPUS randomized trial, two (one in each group) had changed their personal identification number and could not be tracked in the registers. Of the remaining 545 participants, 274 (50.3%) were originally assigned to the OPUS intervention (“OPUS-RCT”), and 271 (49.7%) were assigned to what was then considered standard treatment (“control-RCT”). From the registers, we identified 3,328 people who received OPUS treatment after it was implemented (“OPUS-real-world”).
Table 1 summarizes the baseline characteristics of the three groups, indicating statistically significant differences between groups on all variables except sex and presence of alcohol or substance use disorder 2 years before start of treatment.
| Characteristic | OPUS-RCT Group | Control-RCT Group | OPUS-Real-World Group | Overall p | |||
|---|---|---|---|---|---|---|---|
| N | % | N | % | N | % | ||
| Male | 158 | 57.7 | 164 | 60.5 | 1,830 | 55.0 | 0.16 |
| Diagnosis | <0.001 | ||||||
| Schizophrenia | 184 | 67.2 | 176 | 64.9 | 2,284 | 68.6 | |
| Schizotypal personality disorder | 42 | 15.3 | 37 | 13.7 | 697 | 20.9 | |
| Other | 48 | 17.5 | 58 | 21.4 | 347 | 10.4 | |
| Mean | SD | Mean | SD | Mean | SD | ||
| Age (years) | 27.1 | 6.3 | 27.0 | 6.4 | 23.9 | 4.8 | <0.001 |
| Psychiatric bed days 2 years before treatment | 20.0 | 36.2 | 23.7 | 36.7 | 37.5 | 70.3 | <0.001 |
| Nonpsychiatric bed days 2 years before treatment | 6.9 | 8.8 | 5.6 | 8.5 | 4.5 | 7.5 | 0.05 |
| Redeemed prescriptions for psychotropic medicationsb 2 years before treatment | 3.4 | 7.8 | 4.2 | 9.5 | 7.7 | 11.4 | <0.001 |
| Income 2 years before treatment (in Danish kroner)c | 131,381 | 71,158 | 136,321 | 70,762 | 108,349 | 77,926 | 0.001 |
| N | % | N | % | N | % | ||
| Substance abuse 2 years before treatment | 51 | 18.6 | 64 | 23.6 | 729 | 21.9 | 0.34 |
| Employment status 2 years before treatment | <0.001 | ||||||
| Employed | 95 | 34.7 | 100 | 36.9 | 844 | 25.4 | |
| Unemployed, retired, or disability pension | 18 | 6.6 | 26 | 9.6 | 652 | 16.6 | |
| Student | 68 | 24.8 | 63 | 23.2 | 1,168 | 35.1 | |
| Otherd | 93 | 33.9 | 82 | 30.3 | 664 | 20.0 | |
a
Control-RCT=standard treatment in the OPUS trial; OPUS-RCT=OPUS intervention in the OPUS trial; OPUS-real-world=postimplementation OPUS treatment.
b
ATC codes N05 and N06.
c
Adjusted to 1996 levels using an annual inflation factor of 2.2%.
d
Includes children under age 15 and the category “unknown.”
Table 2 presents the main results of the postimplementation study. Time to first psychiatric admission for the OPUS-RCT and control-RCT groups were identical when the analyses were started 2 years after start of treatment, but not when the analyses were started concurrently with enrollment in treatment. Compared with the OPUS-RCT group, the OPUS-real-world group had a lower probability of being psychiatrically hospitalized, regardless of time scale chosen. The absolute hospitalization rates are shown in Figure 1.
FIGURE 1. Cumulative incidence of psychiatric admission after end of OPUS treatment or standard treatment and overall mortality and suicide after start of OPUS treatment or standard treatmenta
a In the upper left panel, the hospitalization data for the two groups in the OPUS trial (Control-RCT and OPUS-RCT) are identical. Control-RCT=standard treatment in the OPUS trial; OPUS-RCT=OPUS intervention in the OPUS trial; OPUS-real-world=postimplementation OPUS treatment.
| OPUS-Real-World Group | Control-RCT Group | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Measure | Unadjusted Analyses | Adjusted Analyses | Unadjusted Analyses | Adjusted Analyses | ||||||||
| HR | 95% CI | p | HR | 95% CI | p | HR | 95% CI | p | HR | 95% CI | p | |
| First admission at least 2 years after start of treatment | 0.79 | 0.67, 0.94 | 0.006 | 0.73 | 0.61, 0.87 | <0.001 | 1.01 | 0.81, 1.25 | 0.96 | 1.00 | 0.80, 1.24 | 0.99 |
| First admission at any time after start of treatment | 0.76 | 0.66, 0.89 | <0.001 | 0.68 | 0.58, 0.80 | <0.001 | 1.23 | 1.02, 1.50 | 0.04 | 1.17 | 0.95, 1.43 | 0.13 |
| All-cause mortality | 0.48 | 0.27, 0.84 | 0.01 | 0.60 | 0.33, 1.09 | 0.09 | 1.35 | 0.82, 2.23 | 0.25 | 1.33 | 0.80, 2.23 | 0.27 |
| Suicide | 1.15 | 0.31, 4.28 | 0.83 | 1.35 | 0.30, 6.19 | 0.70 | 1.44 | 0.46, 4.52 | 0.54 | 1.43 | 0.43, 4.72 | 0.56 |
| IRR | 95% CI | p | IRR | 95% CI | p | IRR | 95% CI | p | IRR | 95% CI | p | |
| Psychiatric bed days | ||||||||||||
| Year 1 | 0.25 | 0.17, 0.38 | <0.001 | 0.16 | 0.11, 0.24 | <0.001 | 1.30 | 0.75, 2.24 | 0.35 | 1.16 | 0.69, 1.95 | 0.57 |
| Year 2 | 0.47 | 0.27, 0.80 | 0.006 | 0.28 | 0.17, 0.49 | <0.001 | 1.32 | 0.63, 2.76 | 0.46 | 1.07 | 0.52, 2.18 | 0.85 |
| Year 3 | 0.59 | 0.33, 1.07 | 0.09 | 0.31 | 0.17, 0.58 | <0.001 | 1.67 | 0.74, 3.76 | 0.22 | 1.12 | 0.51, 2.46 | 0.78 |
| Year 4 | 0.55 | 0.30, 1.01 | 0.05 | 0.30 | 0.16, 0.56 | <0.001 | 1.17 | 0.50, 2.70 | 0.72 | 0.78 | 0.34, 1.78 | 0.56 |
| Year 5 | 0.62 | 0.31, 1.22 | 0.17 | 0.42 | 0.21, 0.83 | 0.01 | 0.88 | 0.35, 2.25 | 0.80 | 0.78 | 0.31, 1.95 | 0.59 |
| Psychiatric admissions | ||||||||||||
| Year 1 | 0.87 | 0.67, 1.14 | 0.32 | 0.68 | 0.52, 0.87 | 0.003 | 1.21 | 0.85, 1.72 | 0.29 | 1.35 | 0.97, 1.87 | 0.08 |
| Year 2 | 1.01 | 0.73, 1.40 | 0.94 | 0.67 | 0.49, 0.92 | 0.01 | 1.48 | 0.96, 2.29 | 0.08 | 1.49 | 0.99, 2.23 | 0.05 |
| Year 3 | 0.98 | 0.69, 1.41 | 0.93 | 0.65 | 0.46, 0.93 | 0.02 | 1.37 | 0.84, 2.23 | 0.20 | 1.11 | 0.71, 1.74 | 0.65 |
| Year 4 | 0.69 | 0.48, 0.99 | 0.04 | 0.52 | 0.37, 0.74 | <0.001 | 0.95 | 0.57, 1.56 | 0.83 | 0.79 | 0.50, 1.26 | 0.32 |
| Year 5 | 0.85 | 0.56, 1.29 | 0.45 | 0.67 | 0.44, 1.02 | 0.06 | 1.05 | 0.60, 1.85 | 0.86 | 1.12 | 0.66, 1.91 | 0.67 |
| Psychiatric outpatient visits | ||||||||||||
| Year 1 | 0.86 | 0.78, 0.95 | 0.002 | 0.86 | 0.78, 0.95 | 0.002 | 0.43 | 0.38, 0.49 | <0.001 | 0.44 | 0.38, 0.49 | <0.001 |
| Year 2 | 0.83 | 0.73, 0.94 | 0.003 | 0.79 | 0.70, 0.90 | <0.001 | 0.47 | 0.40, 0.56 | <0.001 | 0.48 | 0.40, 0.57 | <0.001 |
| Year 3 | 0.85 | 0.73, 0.99 | 0.03 | 0.83 | 0.71, 0.97 | 0.02 | 0.69 | 0.56, 0.85 | 0.001 | 0.70 | 0.57, 0.87 | 0.001 |
| Year 4 | 1.00 | 0.82, 1.20 | 0.97 | 0.93 | 0.77, 1.13 | 0.50 | 0.96 | 0.74, 1.24 | 0.73 | 0.94 | 0.72, 1.22 | 0.64 |
| Year 5 | 1.00 | 0.81, 1.24 | 0.99 | 0.92 | 0.74, 1.15 | 0.48 | 1.06 | 0.79, 1.41 | 0.70 | 1.05 | 0.79, 1.41 | 0.73 |
| Nonpsychiatric bed days | ||||||||||||
| Year 1 | 1.44 | 0.82, 2.53 | 0.20 | 1.23 | 0.72, 2.12 | 0.45 | 1.36 | 0.64, 2.91 | 0.43 | 1.79 | 0.87, 3.69 | 0.11 |
| Year 2 | 0.64 | 0.39, 1.06 | 0.09 | 0.48 | 0.29, 0.79 | 0.004 | 1.28 | 0.64, 2.56 | 0.48 | 1.42 | 0.74, 2.74 | 0.29 |
| Year 3 | 1.26 | 0.72, 2.21 | 0.41 | 0.88 | 0.50, 1.53 | 0.64 | 1.46 | 0.68, 3.13 | 0.33 | 1.03 | 0.50, 2.13 | 0.93 |
| Year 4 | 0.49 | 0.29, 0.83 | 0.008 | 0.50 | 0.29, 0.85 | 0.01 | 1.30 | 0.64, 2.65 | 0.48 | 1.04 | 0.53, 2.04 | 0.91 |
| Year 5 | 0.98 | 0.57, 1.67 | 0.93 | 0.76 | 0.44, 1.31 | 0.32 | 1.74 | 0.84, 3.61 | 0.14 | 1.28 | 0.63, 2.62 | 0.50 |
| Nonpsychiatric admissions | ||||||||||||
| Year 1 | 1.41 | 0.90, 2.19 | 0.13 | 1.00 | 0.64, 1.56 | 1.00 | 1.38 | 0.77, 2.47 | 0.29 | 1.43 | 0.81, 2.52 | 0.22 |
| Year 2 | 1.13 | 0.76, 1.68 | 0.54 | 0.92 | 0.62, 1.36 | 0.67 | 1.12 | 0.66, 1.92 | 0.68 | 1.05 | 0.63, 1.76 | 0.86 |
| Year 3 | 1.90 | 1.18, 3.07 | 0.008 | 1.68 | 1.04, 2.70 | 0.03 | 1.97 | 1.06, 3.66 | 0.03 | 1.65 | 0.90, 3.01 | 0.10 |
| Year 4 | 0.99 | 0.66, 1.50 | 0.98 | 0.89 | 0.59, 1.35 | 0.59 | 1.18 | 0.68, 2.04 | 0.57 | 1.06 | 0.62, 1.81 | 0.84 |
| Year 5 | 1.09 | 0.72, 1.66 | 0.68 | 0.91 | 0.60, 1.40 | 0.68 | 1.21 | 0.69, 2.13 | 0.51 | 1.04 | 0.60, 1.82 | 0.89 |
| Redeemed defined daily doses of antipsychotics | ||||||||||||
| Year 1 | 1.03 | 0.65, 1.61 | 0.91 | 0.76 | 0.48, 1.21 | 0.25 | 1.08 | 0.58, 1.99 | 0.81 | 1.13 | 0.62, 2.06 | 0.69 |
| Year 2 | 0.98 | 0.60, 1.60 | 0.93 | 0.80 | 0.48, 1.34 | 0.40 | 1.22 | 0.62, 2.40 | 0.56 | 1.12 | 0.58, 2.18 | 0.73 |
| Year 3 | 1.15 | 0.80, 1.64 | 0.46 | 0.96 | 0.66, 1.40 | 0.84 | 1.12 | 0.68, 1.82 | 0.66 | 0.97 | 0.60, 1.59 | 0.91 |
| Year 4 | 0.90 | 0.62, 1.29 | 0.56 | 0.74 | 0.51, 1.08 | 0.12 | 0.90 | 0.55, 1.48 | 0.67 | 0.79 | 0.48, 1.30 | 0.35 |
| Year 5 | 0.78 | 0.53, 1.14 | 0.20 | 0.63 | 0.42, 0.93 | 0.02 | 0.85 | 0.50, 1.45 | 0.56 | 0.81 | 0.48, 1.37 | 0.43 |
| Redeemed defined daily doses of other psycholeptics | ||||||||||||
| Year 1 | 1.36 | 0.85, 2.17 | 0.19 | 1.26 | 0.79, 2.03 | 0.34 | 1.25 | 0.66, 2.36 | 0.49 | 1.47 | 0.79, 2.75 | 0.22 |
| Year 2 | 1.08 | 0.64, 1.82 | 0.77 | 1.03 | 0.60, 1.77 | 0.91 | 1.27 | 0.62, 2.58 | 0.52 | 1.50 | 0.74, 3.03 | 0.26 |
| Year 3 | 0.95 | 0.58, 1.54 | 0.82 | 0.77 | 0.46, 1.28 | 0.31 | 1.22 | 0.62, 2.39 | 0.56 | 1.35 | 0.70, 2.62 | 0.37 |
| Year 4 | 0.73 | 0.44, 1.20 | 0.22 | 0.52 | 0.31, 0.88 | 0.02 | 1.12 | 0.56, 2.23 | 0.75 | 1.40 | 0.71, 2.78 | 0.34 |
| Year 5 | 0.69 | 0.41, 1.16 | 0.16 | 0.49 | 0.29, 0.85 | 0.01 | 1.13 | 0.55, 2.30 | 0.74 | 1.37 | 0.67, 2.77 | 0.39 |
| OR | 95% CI | p | OR | 95% CI | p | OR | 95% CI | p | OR | 95% CI | p | |
| Working or studying | ||||||||||||
| Year 1 | 0.52 | 0.40, 0.66 | <0.001 | 0.52 | 0.39, 0.69 | <0.001 | 0.88 | 0.63, 1.23 | 0.46 | 0.89 | 0.61, 1.29 | 0.54 |
| Year 2 | 0.55 | 0.43, 0.71 | <0.001 | 0.58 | 0.44, 0.77 | <0.001 | 1.06 | 0.75, 1.50 | 0.73 | 1.09 | 0.75, 1.58 | 0.65 |
| Year 3 | 0.70 | 0.53, 0.91 | 0.008 | 0.78 | 0.58, 1.04 | 0.09 | 0.98 | 0.68, 1.40 | 0.89 | 0.98 | 0.67, 1.44 | 0.92 |
| Year 4 | 1.05 | 0.79, 1.39 | 0.74 | 1.20 | 0.88, 1.64 | 0.24 | 1.13 | 0.77, 1.66 | 0.53 | 1.18 | 0.79, 1.77 | 0.43 |
| Year 5 | 1.25 | 0.92, 1.70 | 0.15 | 1.49 | 1.07, 2.09 | 0.02 | 1.16 | 0.77, 1.75 | 0.48 | 1.17 | 0.76, 1.81 | 0.47 |
| Living in a couple relationship | ||||||||||||
| Year 1 | 1.28 | 0.96, 1.71 | 0.10 | 1.02 | 0.75, 1.39 | 0.88 | 1.01 | 0.68, 1.50 | 0.96 | 1.03 | 0.69, 1.56 | 0.88 |
| Year 2 | 1.78 | 1.27, 2.50 | 0.001 | 1.57 | 1.10, 2.23 | 0.01 | 1.56 | 1.01, 2.42 | 0.05 | 1.63 | 1.04, 2.55 | 0.03 |
| Year 3 | 1.88 | 1.33, 2.66 | <0.001 | 1.69 | 1.18, 2.42 | 0.004 | 1.54 | 0.97, 2.42 | 0.06 | 1.62 | 1.02, 2.57 | 0.04 |
| Year 4 | 1.91 | 1.35, 2.71 | <0.001 | 1.69 | 1.18, 2.43 | 0.004 | 1.47 | 0.93, 2.32 | 0.10 | 1.46 | 0.92, 2.34 | 0.11 |
| Year 5 | 1.58 | 1.13, 2.21 | 0.007 | 1.44 | 1.02, 2.04 | 0.04 | 1.24 | 0.79, 1.94 | 0.34 | 1.44 | 0.80, 1.98 | 0.33 |
| Having children living at home | ||||||||||||
| Year 1 | 0.58 | 0.40, 0.84 | 0.004 | 0.87 | 0.56, 1.33 | 0.52 | 0.94 | 0.57, 1.56 | 0.82 | 0.95 | 0.55, 1.64 | 0.85 |
| Year 2 | 0.77 | 0.51, 1.16 | 0.21 | 1.39 | 0.87, 2.23 | 0.17 | 0.75 | 0.41, 1.35 | 0.34 | 0.74 | 0.39, 1.40 | 0.36 |
| Year 3 | 0.81 | 0.54, 1.22 | 0.31 | 1.24 | 0.77, 1.97 | 0.37 | 0.69 | 0.38, 1.27 | 0.23 | 0.65 | 0.34, 1.25 | 0.20 |
| Year 4 | 1.00 | 0.66, 1.53 | 0.98 | 1.40 | 0.88, 2.22 | 0.15 | 1.01 | 0.57, 1.80 | 0.97 | 1.04 | 0.57, 1.90 | 0.91 |
| Year 5 | 0.86 | 0.58, 1.26 | 0.44 | 1.14 | 0.74, 1.74 | 0.55 | 0.91 | 0.53, 1.56 | 0.73 | 0.94 | 0.53, 1.65 | 0.82 |
| Registered alcohol or substance use disorder | ||||||||||||
| Year 1 | 1.18 | 0.87, 1.62 | 0.29 | 1.06 | 0.73, 1.54 | 0.74 | 0.92 | 0.60, 1.42 | 0.70 | 0.73 | 0.44, 1.20 | 0.21 |
| Year 2 | 1.24 | 0.80, 1.91 | 0.34 | 1.01 | 0.63, 1.63 | 0.97 | 1.49 | 0.86, 2.60 | 0.15 | 1.43 | 0.79, 2.59 | 0.24 |
| Year 3 | 1.06 | 0.72, 1.56 | 0.78 | 0.91 | 0.59, 1.40 | 0.67 | 0.98 | 0.57, 1.66 | 0.93 | 0.89 | 0.50, 1.58 | 0.69 |
| Year 4 | 0.90 | 0.60, 1.35 | 0.61 | 0.76 | 0.48, 1.18 | 0.23 | 1.05 | 0.61, 1.82 | 0.85 | 0.97 | 0.54, 1.74 | 0.91 |
| Year 5 | 0.69 | 0.47, 1.02 | 0.07 | 0.59 | 0.38, 0.91 | 0.02 | 0.80 | 0.46, 1.39 | 0.43 | 0.74 | 0.41, 1.33 | 0.31 |
a
The reference group in the analyses is the group that received the OPUS intervention in the OPUS trial. Adjusted analyses were adjusted for number of psychiatric bed days 2 years before treatment, number of nonpsychiatric bed days 2 years before treatment, number of redeemed prescriptions for antipsychotic or other psychotropic medication 2 years before treatment, registered alcohol or substance use disorder 2 years before treatment, income 2 years before treatment (standardized to 1996 levels using an annual inflation rate of 2.2%), employment status 2 years before treatment, sex, age at start of treatment, and diagnosis. Control-RCT=standard treatment in the OPUS trial; OPUS-real-world=postimplementation OPUS treatment; HR=hazard ratio; IRR=incidence rate ratio; OR=odds ratio.
All-cause mortality was lower in the OPUS-real-world group than in the OPUS-RCT group, although the difference was statistically significant only in the unadjusted model, and dipped below the threshold for statistical significance in the fully adjusted model (p=0.09). There were no indications of between-group differences in the rate of suicide. The cumulative incidence rates of both all-cause mortality and suicide are shown in Figure 1.
Adjusted incidence rate ratios for use of the secondary health sector are shown in Figure 2, and the exact numbers as well as unadjusted estimates are listed in Table 2. Number of psychiatric bed days per year for each of the first 5 years after start of treatment was nearly identical in the OPUS-RCT and control-RCT groups. The incidence rates for psychiatric bed days, however, were much lower for the OPUS-real-world group than for the OPUS-RCT group, with adjusted incidence rate ratios usually well below 0.5. Sensitivity analyses for the period ending in 2008 showed the same results, except that the difference was not statistically significant in year 3 (data not shown). The absolute incidence rates of psychiatric bed days are shown in Figure S1 in the online supplement . There was no consistent picture regarding nonpsychiatric bed days, with the OPUS-real-world group showing fewer bed days at some time points but not at others. The absolute incidence rates of nonpsychiatric bed days are shown in Figure S2 in the online supplement .
FIGURE 2. Incidence rate ratios of use of the secondary health care sector among patients who received standard treatment in the OPUS trial or who received the OPUS intervention after real-world implementation compared with patients who received the OPUS intervention in the OPUS triala
a The reference group for incidence rate ratios is the group that received the OPUS intervention in the OPUS trial. All estimates are adjusted for prior psychiatric and nonpsychiatric bed days, diagnosis, prior income, prior psycholeptic prescriptions, sex, age at onset, prior alcohol or substance use disorder, and prior work ability. Error bars indicate 95% confidence interval. Control-RCT=standard treatment in the OPUS trial; OPUS-real-world=postimplementation OPUS treatment.
The OPUS-real-world group also had significantly fewer psychiatric admissions compared with the OPUS-RCT group in the fully adjusted model. There were no indications of differences between the groups regarding nonpsychiatric admissions, except for more admissions in both the OPUS-real-world and control-RCT groups than in the OPUS-RCT group the first year after end of treatment (year 3).
For the first 3 years, including during the 2 years of treatment, the OPUS-RCT group had the most outpatient visits, followed by the OPUS-real-world group, with even fewer in the control-RCT group. For the final 2 years of the 5-year follow-up, there was no difference in outpatient visits between groups. Figure S3 in the online supplement shows the absolute yearly incidence rates of psychiatric outpatient visits for the three groups. For the first 2 to 3 years, the two OPUS treatment groups were at higher levels of outpatient visits, whereas for the final 3 to 5 years, all three groups stabilized at the level already reached by the control-RCT group from the beginning of follow-up.
Adjusted incidence rate ratios and odds ratios for the remaining analyses are shown in Figure 3, again with exact numbers and unadjusted estimates listed in Table 2. The OPUS-real-world group used significantly fewer defined daily doses of antipsychotics 5 years after start of treatment compared with the OPUS-RCT group. Sensitivity analyses ending in 2008, however, showed that the OPUS-real-world group used more antipsychotic medication than the OPUS-RCT group during the first 3 years (data not shown). The same tendency was observed for defined daily doses of other psycholeptic medication both 4 and 5 years after start of treatment, at least in the fully adjusted model.
FIGURE 3. Incidence rate ratios and odds ratios of other outcome measures among patients who received standard treatment in the OPUS randomized trial or who received the OPUS intervention after real-world implementation compared with patients who received the OPUS intervention in the OPUS triala
a The reference group for incidence rate ratios and odds ratios is the group that received the OPUS intervention in the OPUS trial. All estimates are adjusted for prior psychiatric and nonpsychiatric bed days, diagnosis, prior income, prior psycholeptic prescriptions, sex, age at onset, prior alcohol or substance use disorder, and prior work ability. Medication data were in defined daily doses. Error bars indicate 95% confidence interval. Control-RCT=standard treatment in the OPUS trial; OPUS-real-world=postimplementation OPUS treatment.
For the first 2 to 3 years, the OPUS-real-world group had lower odds of working or studying compared with the OPUS-RCT group, and the control-RCT group was situated in between. Five years after start of the 2-year treatment, however, the OPUS-real-world group had higher odds of working or studying compared with the OPUS-RCT group.
Except for the first year in treatment, the OPUS-real-world group had significantly higher odds of being in a couple relationship than the OPUS-RCT group. For much of the period, even the control-RCT group had higher odds of being in a couple relationship than the OPUS-RCT group.
There were no differences between groups in the odds of having children living at home, once confounding variables such as age were included in the fully adjusted model.
Overall, there were no differences between groups in the odds of being registered with an alcohol or substance use disorder. However, after 5 years, the odds of being registered with an alcohol or substance disorder were significantly lower in the OPUS-real-world group compared with the OPUS-RCT group after adjusting for potential confounders.
Discussion
In this phase 4 prospective cohort study, we investigated whether the effectiveness of OPUS treatment in a real-world daily clinical setting is comparable to the efficacy seen in the OPUS randomized trial. We hypothesized a priori that OPUS treatment delivered in the OPUS randomized trial (OPUS-RCT) would be superior to that delivered after implementation (OPUS-real-world), for the reasons discussed in the introduction. We were surprised by the fact that many of our analyses actually identified a better prognosis for the OPUS-real-world group than the OPUS-RCT group, with the control group in the OPUS trial (control-RCT) faring worst in most (but not all) analyses.
For instance, the OPUS-real-world group had both fewer and shorter psychiatric admissions than the OPUS-RCT group. This tendency was strongest in the adjusted models, because the OPUS-real-world group had significantly more psychiatric bed days prior to starting treatment than the two groups in the OPUS trial. It may be difficult to establish whether this is a causal effect or is due to a shift in the way psychiatric inpatient stays have been used over the period of approximately 20 years from which the study data were drawn (22, 23). Indeed, the fact that there were also fewer nonpsychiatric bed days in this group may indicate a general tendency toward shorter admissions. However, it is reassuring that the departure from the constraints of a randomized trial, as well as an increase in caseload, has not led to an apparent increase in psychiatric admissions.
Similarly, although not statistically significant, there was an indication of lower mortality in the OPUS-real-world group. As this was not the case for suicides, this probably indicates a shift toward better pharmacological treatment of schizophrenia or better somatic medical care of patients. While a mortality gap between people with schizophrenia and the general population is apparent and may even be increasing, this is not caused by a decrease in life expectancy in schizophrenia, but rather a slower increase in life expectancy among patients with schizophrenia than in the general population (24).
The OPUS-real-world group received fewer defined daily doses of both antipsychotic medication and other psycholeptic medication than the OPUS-RCT group. However, when the analyses were restricted to the earliest period (ending in 2008), the OPUS-real-world group actually used more antipsychotic medication than the OPUS-RCT group. Consequently, it may be that time trends regarding antipsychotic prescription render results regarding antipsychotic medication too uncertain to compare OPUS treatment before and after implementation. However, we cannot exclude the possibility that these and other findings may be caused by residual confounding that we were unable to account for. This could occur, for instance, if the OPUS-real-world group was less ill than the OPUS-RCT group at the start of treatment. The younger age could, for instance, reflect a shorter duration of untreated psychosis, although it has been shown that, in the 2009–2012 period, the duration of untreated psychosis in the OPUS-real-world group was comparable to that in the groups in the original OPUS trial (2). Beyond the variables we adjusted for, we do not have information on illness severity. Previous studies have shown that after 1.5 to 2 years of OPUS treatment, severity of psychotic symptoms as measured by the Scale for the Assessment of Positive Symptoms was higher in the 2009–2012 period than in the 1998–2000 period (2, 3, 25). While not representative of the full duration of follow-up in the present study, this does not seem consistent with the presence of such residual confounding.
Interestingly, while the OPUS-real-world group had lower odds of working or studying during the first years of follow-up compared with the OPUS-RCT group, this shifted over time, with the OPUS-real-world group eventually overtaking the OPUS-RCT group on this important functional outcome (albeit only after 5 years, and in the fully adjusted model). Such vocational measures are increasingly being perceived as more important than symptom outcomes (26 –28). Furthermore, since the OPUS-real-world group started off worse (after onset of psychosis) than the OPUS-RCT group, this is likely not caused by residual confounding. This is particularly interesting since, from 2012 to 2017, a randomized trial of individual placement and support was conducted in many psychiatric settings in Denmark, including in OPUS treatment settings (28). However, we cannot exclude the possibility that OPUS treatment has become slower at getting people into work compared with during the randomized trial. While social workers have always been a component of OPUS, no specific work rehabilitation focus has been applied in OPUS except the above-mentioned individual placement and support trial, although rapid return to work has always been encouraged.
Strangely, both the OPUS-real-world group and the control-RCT group consistently had higher odds of being in a couple relationship, except during the first year, compared with the OPUS-RCT group. We have previously shown that singleness in schizophrenia is associated with the polygenic risk score for schizophrenia (29). This is difficult to explain but is likely not due to differences over time.
Finally, while starting off similarly, the odds of alcohol or substance use disorder being registered also decreased over time in the OPUS-real-world group compared with the OPUS-RCT group, again indicating that not only is OPUS treatment still effective, it could even be effective for longer than it was while being investigated as part of the original randomized trial. This is of utmost importance given that comorbid alcohol or substance use is associated with a range of poor prognostic outcomes in schizophrenia (30).
Generally, differences between the OPUS-real-world group and the two groups in the randomized trial could be a mixture of positive implementation of OPUS treatment and of improved post-OPUS services over time. Positive effects beyond those observed in the randomized trial may be due to more flexible setups, improved experience with delivering the intervention, or other factors. We accounted for potential differences in patient characteristics in the analyses, which suggests that this is not likely to have been a confounder in our results.
While postimplementation studies (often called phase 4 trials or real-world-evidence studies) are finding increasing use in pharmacology, this is not the case for psychosocial interventions (31, 32). While we do not argue that psychosocial interventions should be implemented without proper testing in randomized trials, it may be worthwhile testing whether such therapies remain effective once they are implemented. In particular, it is interesting that we find indications that a complex psychosocial intervention would be more effective in real-world settings than in the controlled confines of a randomized trial. The Recovery After an Initial Schizophrenia Episode (RAISE) trial in the United States may be considered intermediary to individually randomized trials and a real-world study, given its cluster-randomized nature, and this trial also showed the effectiveness of the specialized early intervention (33).
Strengths and Limitations
A number of strengths in our study design should be highlighted. First, the inclusion rates in the original OPUS trial were very high (3). Regarding the OPUS-real-world group, our use of nationwide unselected registers means that there was no selection bias into this group. Similarly, the original randomized trial is considered unselected and highly generalizable (3). The register-based outcomes mean that we have complete follow-up data, since we did not rely on participation in research interviews or surveys.
However, the use of register-based information also has certain limitations, notably in the depth of information available on the participants in the study. Indeed, many of the positive effects identified in the original OPUS trial were not replicable in this study, simply because information regarding, for example, severity of psychotic and negative symptoms is not directly available in the registers. Hence, many of the outcomes we do measure should be seen as proxies for these outcomes.
Other limitations exist as well. For instance, since the OPUS trial groups are separated in time from the real-world group, we cannot exclude the possibility that time is a confounder. This is not something we can address statistically. However, as discussed above, time does not appear to be a relevant confounder in many of our analyses. Since OPUS is implemented nationwide as first-choice treatment, a contemporary non-OPUS control group would not be feasible and would provide results that would likely be overestimates of the true effect of postimplementation OPUS treatment. Unfortunately, data were not available regarding adaptations from the OPUS trial setup in the real-world implementation of OPUS, but program fidelity has been shown to be rather high (34). Finally, we did not adjust for multiple comparisons, so findings near the threshold for statistical significance should be (and were) interpreted cautiously.
Implications and Conclusions
Our study holds a number of implications. Most importantly, it highlights that OPUS treatment and similar integrated psychosocial treatments for first-episode psychosis are at least as effective in real-world settings as within the controlled environments of randomized trials in spite of the fact that the real-world treatment settings have higher caseloads. One possible explanation for the finding that OPUS treatment in real-world settings appeared even better than in the OPUS trial could be that the contents of the intervention may have been improved upon, even if caseloads have been increased.
The implication of this study is that places that have already implemented these interventions should continue doing so, and that places that have not implemented such treatments should consider doing so. However, our results merit replication in other real-world cohorts in order to establish whether they can be generalized to other settings and countries.
Supplementary Material
File (appi.ajp.2020.20111596.ds001.pdf)
- View/Download
- 118.36 KB
References
1.
Correll CU, Galling B, Pawar A, et al : Comparison of early intervention services vs treatment as usual for early-phase psychosis: a systematic review, meta-analysis, and meta-regression. JAMA Psychiatry 2018 ; 75 : 555 – 565
2.
Albert N, Melau M, Jensen H, et al : Five years of specialised early intervention versus two years of specialised early intervention followed by three years of standard treatment for patients with a first episode psychosis: randomised, superiority, parallel group trial in Denmark (OPUS II). BMJ 2017 ; 356 : i6681
3.
Petersen L, Jeppesen P, Thorup A, et al : A randomised multicentre trial of integrated versus standard treatment for patients with a first episode of psychotic illness. BMJ 2005 ; 331 : 602
4.
McGorry PD : Early intervention in psychosis: obvious, effective, overdue. J Nerv Ment Dis 2015 ; 203 : 310 – 318
5.
Nordentoft M, Melau M, Iversen T, et al : From research to practice: how OPUS treatment was accepted and implemented throughout Denmark. Early Interv Psychiatry 2015 ; 9 : 156 – 162
6.
Pinnock H, Epiphaniou E, Taylor SJC : Phase IV implementation studies: the forgotten finale to the complex intervention methodology framework. Ann Am Thorac Soc 2014 ; 11 ( suppl 2 ): S118 – S122
7.
Essock SM, Drake RE, Frank RG, et al : Randomized controlled trials in evidence-based mental health care: getting the right answer to the right question. Schizophr Bull 2003 ; 29 : 115 – 123
8.
Cochrane AL : Effectiveness and Efficiency: Random Reflections on Health Services. London, Nuffield Provincial Hospitals Trust, 1972
9.
Rothwell PM : External validity of randomised controlled trials: “to whom do the results of this trial apply?” Lancet 2005 ; 365 : 82 – 93
10.
Smith PG, Morrow RH, Ross DA (eds): Field Trials of Health Interventions: A Toolbox, 3rd ed. Oxford, UK, London School of Hygiene and Tropical Medicine, 2015
11.
Humphreys K : A review of the impact of exclusion criteria on the generalizability of schizophrenia treatment research. Clin Schizophr Relat Psychoses 2017 ; 11 : 49 – 57
12.
Gastaldon C, Mosler F, Toner S, et al : Are trials of psychological and psychosocial interventions for schizophrenia and psychosis included in the NICE guidelines pragmatic? A systematic review. PLoS One 2019 ; 14 : e0222891
13.
Mors O, Perto GP, Mortensen PB : The Danish Psychiatric Central Research Register. Scand J Public Health 2011 ; 39 ( suppl ): 54 – 57
14.
Lynge E, Sandegaard JL, Rebolj M : The Danish National Patient Register. Scand J Public Health 2011 ; 39 ( suppl ): 30 – 33
15.
Kildemoes HW, Sørensen HT, Hallas J : The Danish National Prescription Registry. Scand J Public Health 2011 ; 39 ( suppl ): 38 – 41
16.
Pedersen CB : The Danish Civil Registration System. Scand J Public Health 2011 ; 39 ( suppl ): 22 – 25
17.
Helweg-Larsen K : The Danish Register of Causes of Death. Scand J Public Health 2011 ; 39 ( suppl ): 26 – 29
18.
Petersson F, Baadsgaard M, Thygesen LC : Danish registers on personal labour market affiliation. Scand J Public Health 2011 ; 39 ( suppl ): 95 – 98
19.
Fine JP, Gray RJ : A proportional hazards model for the subdistribution of a competing risk. J Am Stat Assoc 1999 ; 94 : 496 – 509
20.
Bertelsen M, Jeppesen P, Petersen L, et al : Five-year follow-up of a randomized multicenter trial of intensive early intervention vs standard treatment for patients with a first episode of psychotic illness: the OPUS trial. Arch Gen Psychiatry 2008 ; 65 : 762 – 771
21.
Secher RG, Hjorthøj CR, Austin SF, et al : Ten-year follow-up of the OPUS specialized early intervention trial for patients with a first episode of psychosis. Schizophr Bull 2014 ; 41 : 617 – 626
22.
Babalola O, Gormez V, Alwan NA, et al : Length of hospitalisation for people with severe mental illness. Cochrane Database Syst Rev 2014 ; 2014 : CD000384
23.
Tulloch AD, Fearon P, David AS : Length of stay of general psychiatric inpatients in the United States: systematic review. Adm Policy Ment Health Ment Health Serv Res 2011 ; 38 : 155 – 168
24.
Hjorthøj C, Stürup AE, McGrath JJ, et al : Years of potential life lost and life expectancy in schizophrenia: a systematic review and meta-analysis. Lancet Psychiatry 2017 ; 4 : 295 – 301
25.
Andreasen NC : The Scale for the Assessment of Positive Symptoms (SAPS). Iowa City, University of Iowa, 1984
26.
Nuechterlein KH, Subotnik KL, Ventura J, et al : Enhancing return to work or school after a first episode of schizophrenia: the UCLA RCT of individual placement and support and workplace fundamentals module training. Psychol Med 2020 ; 50 : 20 – 28
27.
Nuechterlein KH, Subotnik KL, Turner LR, et al : Individual placement and support for individuals with recent-onset schizophrenia: integrating supported education and supported employment. Psychiatr Rehabil J 2008 ; 31 : 340 – 349
28.
Christensen TN, Wallstrøm IG, Stenager E, et al : Effects of individual placement and support supplemented with cognitive remediation and work-focused social skills training for people with severe mental illness: a randomized clinical trial. JAMA Psychiatry 2019 ; 76 : 1232 – 1240
29.
Hjorthøj C, Uddin MJ, Hougaard DM, et al : Polygenic risk for psychiatric disorder and singleness in patients with severe mental illness and controls. J Psychiatr Res 2019 ; 119 : 60 – 66
30.
Large M, Mullin K, Gupta P, et al : Systematic meta-analysis of outcomes associated with psychosis and co-morbid substance use. Aust N Z J Psychiatry 2014 ; 48 : 418 – 432
31.
Corrigan-Curay J, Sacks L, Woodcock J : Real-world evidence and real-world data for evaluating drug safety and effectiveness. JAMA 2018 ; 320 : 867 – 868
32.
Breckenridge AM, Breckenridge RA, Peck CC : Report on the current status of the use of real-world data (RWD) and real-world evidence (RWE) in drug development and regulation. Br J Clin Pharmacol 2019 ; 85 : 1874 – 1877
33.
Kane JM, Robinson DG, Schooler NR, et al : Comprehensive versus usual community care for first-episode psychosis: 2-year outcomes from the NIMH RAISE early treatment program. Am J Psychiatry 2016 ; 173 : 362 – 372
34.
Melau M, Albert N, Nordentoft M : Programme fidelity of specialized early intervention in Denmark. Early Interv Psychiatry 2019 ; 13 : 627 – 632
Information & Authors
Information
Published In
History
Received: 12 November 2020
Revision received: 3 February 2021
Accepted: 1 June 2021
Published online: 28 July 2021
Published in print: October 01, 2021
Keywords
Authors
Competing Interests
The authors report no financial relationships with commercial interests.
Funding Information
This study was funded by the Lundbeck Foundation.
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBLogin options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).