Enhancing Problem-Solving Therapy With Smartphone Technology: A Pilot Randomized Controlled Trial

We recruited 30 men and four women (N=34) with a current diagnosis of anxiety or depression from a VA primary care clinic (one participant was excluded after the assessment). Eligible veterans were over 18 years old and on a stable medication regimen (for at least 4 weeks). Diagnostic inclusionary criteria were confirmed by using the Mini International Neuropsychiatric Interview (7). Exclusionary criteria included current suicidal intent, inability to provide informed consent, and diagnosed substance dependence or psychosis. Informed consent documents and all procedures were approved by the local VA institutional review board.

Participants completed assessments at baseline and 6- and 12-week follow-ups, which included the Depression Anxiety and Stress Scale (DASS) (8), the Short-Form 12-Item Health Survey for veterans to measure mental health quality of life (9), the Social Problem-Solving Inventory–Revised (10) subscales (positive problem solving orientation, negative problem solving orientation, rational problem solving orientation, impulsivity, carelessness style, and social problem solving), and the Perceptions of Computerized Therapy Questionnaire−Patient Version (11) subscales (compatibility [fit between innovation and values], complexity [ease/difficulty of use], observability [observation of others using the intervention], trialability [ability to “try it out”], and future use intent [likelihood of use]). At the 6- and 12-week follow-ups, participants completed the Client Satisfaction Questionnaire–8 (CSQ-8) (12) and the Problem Solving Skills Use (PSSU) Checklist, which was designed for this study to measure the frequency that PST skills were used over the previous 2 weeks. Veterans randomly assigned to receive the app completed a semistructured qualitative interview.

Computer randomization stratified eligible participants by current smartphone ownership into two groups. Each participant received weekly sessions of PST for 6 weeks with either a workbook (PST-only group) or the Moving Forward app (app group). Session 1 took place in person. Sessions 2–6 took place by telephone. All veterans received an iPhone 4S equipped with basic apps (social media, news, and sports). Veterans randomly assigned to receive Moving Forward were instructed on how to access and use the app.

Because of the small sample size, we did not anticipate that our analysis would yield statistically significant group differences (13); instead, all hypotheses were specified as absolute effect sizes. We compared the Moving Forward condition to an active control group by using intent-to-treat analysis. We expected that the two treatments would be similar in terms of their overall impact on symptoms but that the augmentation of the app would yield a small but clinically meaningful benefit. Therefore, we hypothesized that we would observe a small between-group effect size, defined as a Cohen’s d ≥0.30 or odds ratio (OR) ≥1.70. Missing data were imputed by using a multiple imputation (14). A trained research assistant conducted 20- to 45-minute audio-recorded qualitative interviews. Interviews were transcribed and analyzed by using Atlas.ti. The goal was to describe and understand how participants felt, thought, and behaved. A doctorate-level qualitative expert read through the first few transcripts to identify inductive themes (i.e., words, phrases, or concepts repeated across interviews). A preliminary codebook containing code names and definitions was developed from these themes. Additional themes were added to the codebook, and initial codes were refined as analysis progressed.

There were no substantial baseline differences between groups. Both groups reported reductions in depression at 6 and 12 weeks. Both groups reported reductions in anxiety and stress at 6 weeks, with symptoms increasing slightly by 12 weeks for those using the app. Mental health quality of life improved throughout treatment for both groups from baseline to 6 and 12 weeks. As shown in Table 1, the app group exhibited above-threshold effect size differences (Cohen’s d≥0.30, OR≥1.70) for observability (d=0.34) and trialability (d=0.48) at 6 weeks, and worksheet completion at 12 weeks. The PST-only group exhibited higher than threshold scores for treatment satisfaction at 6 weeks (d=0.39), all scales of the DASS (d=0.30–0.48), positive problem solving (d=0.80), rational problem solving (d=0.54), relative advantage of technology (d=0.76), and future use intention (d=0.31) at 12 weeks. A majority of veterans in both groups reported completing breathing practice at 6 and 12 weeks. Above-threshold group differences were reported for completing problem-solving worksheets at 12 weeks (OR=1.86).

Five themes emerged via qualitative analysis (N=14): convenience/accessibility, empowerment, engagement in healthy behaviors, stigma and shame reduction, and extra support offered by the app (see the online supplement for more details). Users described the added value of the app’s user-friendly, easily accessible self-management tools that could be utilized privately at times when they would be most valuable. Nearly all participants who used the app (N=14) stated that they would recommend the app to others, and about half reported that they had already done so (N=7).

The Moving Forward app was well received and satisfactory to app users. Veterans who received the app reported high rates of skills use at 6 weeks, which were sustained at the 12-week follow-up. Therapy developers assert that out-of-session practice is required in order to adopt a more effective problem-solving style (4), suggesting that homework has a linear and causal effect on treatment outcome (15). A large systematic review of digital technologies concluded that improvement in treatment engagement, particularly out-of-session practice, was one of the main benefits of digital access (16).

The app group also reported larger increases in trialability and observability at 6 weeks, suggesting positive interest, knowledge, and experience with app-based care (11). These findings were consistent with our qualitative data, which suggested that patients in our study were largely satisfied with the app and found it easy to use, thought that it supported adaptive coping, and used the app when needed on a regular basis. App users also suggested that the app can reduce barriers to care because of the immediacy, anonymity, and stigma reduction associated with app-based care. Both groups reported high levels of satisfaction on the CSQ-8; however, the PST-only group reported higher levels of satisfaction with care relative to the app group.

Although differences in anxiety, depression, and stress symptoms indicated greater improvements in the PST-only group at 12 weeks, both groups remained in the mild-to-moderate range for each subscale. This pilot study expands knowledge of patient perspectives about smartphone technology. Consistent with the SOTA model (1), this study demonstrated that digital technology has the potential to increase access to mental health care by providing an alternative avenue to connect with care.

This pilot study had several limitations. Because of the small sample size, we urge caution in interpreting effect sizes. Additionally, the therapy occurred at one site with one therapist (13). All app-use data were self-reported, which could have introduced bias, and there was only one qualitative data coder.

This study suggests that the Moving Forward app is useful and well received by patients. Because of the flexibility of the app and because it as available at no cost, Moving Forward may provide a useful option for mental health care clinics. Providers should discuss apps with patients because this option might not be equally suitable or preferable for everyone.

The authors gratefully acknowledge the patients and staff in the Primary Care Mental Health Integration Clinic at the Central Arkansas VA Healthcare System, including Laura Donnelly, Ph.D., and Patricia Gibson, M.D. They also acknowledge the important contributions of project staff, including Amanda Lunsford, Katherine “Libby” Frith, George Cliff Hudson, Debbie Hodges, Liya Lu, Michael McCarther, and Raphael Suarez.