Effect of Mobile Health on In-person Service Use Among People With Serious Mental Illness

Data came from an assessor-blind randomized comparative effectiveness trial that took place between June 2015 and September 2017. The study was conducted in partnership with a large multisite agency that provides services to people with serious mental illness in Chicago. Full descriptions of the trial—including patient clinical outcomes, treatment engagement rates, and satisfaction ratings—were recently published in this journal (10). The study was approved by the institutional review boards of the University of Washington and Dartmouth College. All study participants provided informed consent. The study consisted of eight treatment cycles; in each cycle, a cohort of participants was randomly assigned into one of two treatments: a smartphone-delivered mHealth intervention called FOCUS or a clinic-based group intervention called Wellness Recovery Action Plan (WRAP). Interventions lasted for a period of 3 months. Intervention cycles that occurred during holiday periods may have been affected by the availability of services; as a result, these cycles were extended to ensure all participants had comparable treatment periods and access to care. Participants were compensated $30 for completing assessments. Briefly, both interventions produced comparably positive patient satisfaction ratings, clinical outcomes, and recovery gains. FOCUS demonstrated superior treatment commencement rates and patient engagement (10).

The study enrolled 163 participants, who were on average 49 years old; 59% (N=96) were male, 65% (N=106) were African American, and 27% (N=44) were white. The sample included 49% (N=80) who were diagnosed as having schizophrenia or schizoaffective disorder, 28% (N=46) diagnosed as having bipolar disorder, and 23% (N=37) diagnosed as having major depressive disorder.

Participants identified as using the electronic health record were recruited by 20 clinical teams at three centers. Inclusion criteria included chart diagnosis of schizophrenia, schizoaffective disorder, bipolar disorder, or major depressive disorder; age 18 years or older; a rating of 3 or lower on one of three items constituting the domination-by-symptoms factor from the Recovery Assessment Scale (RAS [11]). Exclusion criteria were hearing, vision, or motor impairment affecting operation of a smartphone (determined by using a demonstration device); less than fifth-grade English reading ability (determined by using the reading section of the Wide Range Achievement Test-4 [12]); and exposure to WRAP or FOCUS in the past 3 years. Participants who enrolled and received the study interventions continued to be eligible for all other services provided by Thresholds, a large provider of community services in Chicago, including crisis intervention, assertive community treatment, supported employment, psychiatric evaluation, medication monitoring, psychosocial rehabilitation, and case management. In-person services occurred in the community or at one of the agency’s multiple locations. Participant recovery was assessed by using the RAS, a 24-item measure evaluating five recovery factors with a 5-point Likert scale: personal confidence and hope, willingness to ask for help, goal and success orientation, reliance on others, and domination by symptoms (11). The RAS was administered by trained assessors at baseline (0 months), posttrial (3 months), and follow-up (6 months). Service use was recorded in Thresholds’ electronic health record. Agency staff provided our research team with a complete log of each participant’s service use minutes in the 90-day period preceding the day the individual began FOCUS and in two subsequent 90-day periods: the intervention and follow-up periods.

We calculated response to treatment as a dichotomous variable to facilitate comparisons between treatment responders and nonresponders. Treatment responders included 45 (31%) participants who experienced at least a small increase in RAS scores postintervention that was maintained at 6-month follow up (within-person change equivalent to Cohen’s d≥.2, consistent with effects seen in psychosocial intervention trials for people with serious mental illness [13]). Treatment non-responders included 101 (69%) individuals who did not meet these criteria. Seventeen participants were excluded in this analysis because they did not complete the RAS posttreatment or at follow-up. Mixed-effects models with a factorial design and random effects for time were used to test whether a three-way interaction (treatment response × treatment condition × time [day]) predicted service use.

FOCUS participants used a mean±SD total of 1,746±1,672 service minutes in the 90 days preceding the study intervention, compared with 1,587±1,491 minutes during the intervention period (9% decrease) and 1,554±1,284 minutes in the follow-up period (additional 2% decrease). WRAP participants used 2,114±1,541 minutes in the 3 months preceding intervention, compared with 1,825±1,366 minutes during the 3-month intervention period (14% decrease) and 1,613±1,198 minutes in the 3-month follow-up period (12% decrease).

The mixed-effects model revealed a significant three-way interaction of treatment response × treatment condition × time (B=–0.05, SE=0.02, p=0.02). To probe this interaction, we tested separate models for each treatment condition. In the FOCUS condition, a reduced model (service use minutes predicted by time only) demonstrated a significant main effect for time (B=–0.01, SE=0.01, p=0.04), and in the full model, accounting for treatment response, there was a significant main effect for time (B=–0.03, SE =0.01, p=0.01) and a significant treatment response × time interaction (B =0.02, SE=0.01, p=0.04). This reflected a greater decrease in service use minutes among treatment responders, given that FOCUS responders experienced a decrease of 256 minutes (12%) during the intervention period and of an additional 217 minutes (11%) in the follow-up period. FOCUS nonresponders experienced an average decrease of 146 minutes (10%) during the intervention period and a 115-minute (8%) increase during the follow-up period. Figure 1 shows these patterns. In the WRAP condition, reduced models demonstrated a significant main effect for time (B=–0.03, SE=0.01, p=0.001), but in a full model accounting for treatment response and the interaction, there were no significant main or interaction effects.

People with serious mental illness who received the smartphone FOCUS intervention used less in-person services in the months during (9% reduction) and after (additional 2% reduction) treatment than in the months preceding it. A similar pattern was evident and more pronounced among people in the clinic-based WRAP condition in the months during (14%) and after (additional 12%) treatment. These findings suggest that the introduction of an evidence-based illness management intervention, whether via mHealth or more traditional clinic-based care, may reduce individuals’ need for other services. Notably, not all mHealth interventions are designed to accomplish the same objectives (1, 2). Various service models may leverage mHealth resources differently, with some serving as standalone device-delivered resources and others as treatment augmenters in the context of blended care. Previous studies involving mHealth approaches developed to improve adherence to in-person treatment have demonstrated that these can effectively increase the number of contacts clients have with their providers (14, 15). For people who struggle with treatment engagement, mHealth may therefore also be useful in augmentation and facilitation of person-delivered care.

Reduction in service use during the intervention period was slightly greater among FOCUS treatment responders than among nonresponders (12% versus 10%). More prominent differences emerged after treatment ended; FOCUS treatment responders’ service use continued to drop postintervention (additional 11%), whereas nonresponders actually increased service use by 8% to a level that was still slightly lower than their preintervention status. This finding suggests that treatment responders internalized FOCUS skills and were perhaps able to continue to use these self-management strategies postintervention. FOCUS nonresponders might have had more difficulty retaining self-management techniques once the scaffolding of the intervention was no longer available. Such individuals may need ongoing access to mHealth technology and multimedia resources.

This study had several limitations. Associations between recovery and service use were correlational. Although it seems more likely that participants’ improved recovery led to reduction in their service use rather than the other way around, we cannot rule out the possible effects of an unmeasured third variable on both. We did not conduct a formal cost analysis to evaluate the economic impact of mHealth. Future research should evaluate the costs associated with implementing mHealth and evaluate the potential savings associated with changes in the volume of person-delivered services.

mHealth continues to demonstrate promise in providing accessible, scalable, and effective mental health interventions. This study provides preliminary support for the notion that individuals who benefit from mHealth interventions may also experience reduced need for other face-to-face delivered services. Studies should examine whether mHealth can serve as an economical offering in stepped-care models by using less resource-intensive treatment options first. As the use of mHealth interventions grows, researchers can explore the integration of these interventions into clinical workflow and evaluate their impact on mental health services models and workforce.

The authors gratefully acknowledge the contributions of staff and service recipients at Thresholds in Chicago. The views and statements in this brief report are solely the responsibility of the authors and do not necessarily represent the views of PCORI or its board of governors or methodology committee.