Smartphone-Assisted Delivery of Cognitive-Behavioral Guided Self-Help for Binge Eating: Cautionary Musings of Implications Given the Importance of Comparison Groups

In this issue of the Journal, Hildebrandt and colleagues (1) report on a randomized controlled trial testing a smartphone-assisted delivery of cognitive-behavioral therapy (CBT) via guided self-help (CBT-GSH) for adults with binge-eating disorder (N=150) or bulimia nervosa (N=75). This randomized controlled trial represents an innovative collaboration between academia (Mount Sinai), industry (Noom, Inc.), and an integrated health care system in the Pacific Northwest of the United States (Kaiser Permanente). This is an important study for several reasons, including the focus on whether evidence-based scalable treatments (CBT-GSH) for binge eating can be effectively delivered by nonexpert health coaches assisted with a smartphone app in real-world clinical practice settings.

Binge-eating disorder and bulimia nervosa are prevalent eating disorders (2) associated with substantially elevated risks for psychiatric and medical comorbidities (3), psychosocial impairments, and suicide attempts (4). Despite the availability of effective treatments (5–7), including specific specialist psychological treatments (e.g., CBT, interpersonal psychotherapy) and medications approved by the Food and Drug Administration (e.g., lisdexamfetamine for binge-eating disorder and fluoxetine for bulimia nervosa), research has documented low rates of help-seeking for eating disorders, and rates of obtaining specific evidence-based treatments are likely even much lower (8). The limited availability of clinicians trained in the delivery of specialist CBT treatments coupled with the need to reach more patients and to overcome associated barriers to treatment (e.g., time, costs) stimulated the development of scalable or disseminable versions of empirically supported CBT (5). Research has generally supported the effectiveness of various CBT-GSH methods (9), including their treatment specificity relative to other credible active treatments and control for attention comparison groups (10).

Recent years have witnessed a proliferation of technology-assisted methods (e.g., web-based Internet, smartphone app, wearable sensors) for increasing dissemination of health-related information and interventions for nearly every medical area. It has been estimated that more than 10,000 apps are available for health-related topics; however, few have been investigated, and many serious concerns exist about their safety, let alone their utility (11). Nonetheless, technology-assisted methods do have great potential to address important clinical gaps and barriers to treatment, and emerging research targeting eating disorders is encouraging in suggesting that they can facilitate broader dissemination of evidence-based guided self-help treatments such as CBT-GSH (12). Importantly, although the rigorous INTERBED randomized controlled trial (13) found that Internet-based CBT-GSH was inferior to traditional individual face-to-face CBT, Internet-based CBT-GSH clearly represents a viable low-threshold treatment for binge-eating disorder.

Hildebrandt and colleagues have contributed two noteworthy randomized controlled trials examining the utility of a smartphone app (Noom Monitor) for delivering CBT-GSH to adults with binge eating (1, 14). In the first study (14), comprising 66 adults treated at a medical-school-based specialty program, CBT-GSH and CBT-GSH with a smartphone app differed little on outcomes at posttreatment and 6-month follow-up assessments, although the treatment condition with the smartphone app was associated with some advantages in meal adherence, which mediated binge-eating outcomes. This randomized controlled trial, which unlike the INTERBED study (13) was not powered as a noninferiority study, nonetheless suggested that the smartphone app might have utility for enhancing adherence to CBT-GSH and dissemination. In their second study, published in this issue, Hildebrandt and colleagues compared smartphone-assisted CBT-GSH, delivered by nonexpert health coaches, with what they termed as standard care in a real-world integrated health care system. The smartphone-assisted treatment condition (CBT-GSH plus Noom Monitor) was associated with significantly better outcomes than the control condition, including superior remission rates (56.7% compared with 30%; number needed to treat=3.74) and reductions in eating-disorder psychopathology and clinical impairment. The authors concluded that CBT-GSH plus Noom Monitor delivered by health coaches is superior to standard care in a nonacademic health care system.

I offer here cautionary comments and a broader context from the relevant research literature for interpreting the findings reported in this study. First, the selection of control or comparison groups is critically important for randomized controlled trials, as they can influence the results as much as the experimental (or hypothesized active) treatments (15–17). The use of the term standard care in this study is inappropriate and potentially misleading. Essentially, the comparison condition used in this randomized controlled trial was a no-treatment control, which, at best, could perhaps generously be labeled as usual care or treatment as usual (15, 16). Standard care generally implies that there exists a standard treatment and that the standard treatment is actually given. In this trial, participants assigned to the comparison condition were described as having “unrestricted access to clinical resources within the health plan,” but unlike the experimental treatment condition, they were not assigned to anything nor to any clinicians. It was noted that there were no services “specifically structured for the treatment of binge-eating disorder,” although there are providers “whose scope of practice includes treatment for both bulimia nervosa and binge-eating disorder” available within the health care network. The authors go on to report that no participants in what they termed standard care received any form of eating disorder-related treatment, and only 15 received any psychiatric services during the intervention period. Thus, what was termed standard care was essentially no treatment, and this reflects one of the weakest possible controls (i.e., not even any control for attention beyond the act of being in a study and completing assessments). It is important to emphasize that this is neither merely semantics nor methodological technicality. Methodologically, it has long been established that such controls will typically result in strong effect sizes for experimental conditions (15). Indeed, critical reviews and meta-analyses of randomized controlled trials have clearly shown that effect sizes of superiority of CBT (or any effective treatment) are substantially greater for comparisons with no treatment, waitlist, control for attention, minimal treatment, and nonspecific usual care treatments than for credible controls or other active treatment conditions for eating disorders (6), much like what is documented for other disorders (18).

Second, the role of the smartphone app for assisting the delivery of CBT-GSH for binge eating remains unclear. Although Hildebrandt and colleagues found CBT-GSH plus Noom Monitor to be superior to standard care (which I argue above is actually a no-treatment control without even control for attention), their earlier study (14) demonstrated that it differed little from CBT-GSH (without assistance from a smartphone app). While it is highly encouraging that the results observed for CBT-GSH plus Noom Monitor were achieved by nonexpert health coaches in a real-world health care setting, two previous randomized controlled trials performed by one of the investigators in the same health care delivery system reported strikingly similar findings for CBT-GSH that was also delivered by nonexpert clinicians treating binge eating (19, 20). For example, DeBar and colleagues (20) reported a number needed to treat of 5 for CBT-GSH delivered by nonexperts without the smartphone assistance. Thus, it remains unclear whether the delivery method of CBT-GSH matters much or whether it can be enhanced at all by innovative technology in general or by this smartphone app specifically.

I conclude with a few broader cautions and implications for future research on the dissemination of evidence-based treatments for eating disorders. While I see much potential for technology and for innovative uses of technology to either facilitate (i.e., scale or disseminate) or enhance (i.e., augment) treatments, I emphasize that the treatments themselves must work for the technology to do either thing. As a behavioral scientist, I offer the parallel “pill” example that if a specific medicine is not effective for a condition, then a smartphone reminder to take the pill matters not. While I share the established potential of GSH-CBT as an effective and scalable treatment (9) and have, in fact, reported specificity of treatment effectiveness for CBT-GSH (10), I have been humbled and therefore urge caution when research suggests that there are limits to how low minimal guidance can be for self-help or pharmacological treatments in real-world settings (21). These caveats are offered with the view of encouraging further rigorous research in this area and urging caution in uncritically adopting the promise of technology as a means for disseminating care.

Comer and Barlow (22) offered astute observations in support of retaining a role for specialist psychological treatments for addressing psychiatric conditions. Cost-effectiveness models have the potential to be applied unchecked to achieve economic versus public-health needs. Nonetheless, there are examples of how cost-effectiveness models can promote greater adoption of evidence-based psychological treatments (23). There are also examples of broadened training approaches that could facilitate greater uptake of psychological interventions (24) and examples of stepped-care models based on patient response rather than ease or cost (25) and implementation science models for testing how technologies can benefit patients and delivery networks (26).

Finally, I close with a concern about unwanted or unexpected outcomes. Differences in access and in willingness to use emerging technologies for health-care-related needs must be considered. Indeed, research performed at Kaiser Permanente Health System (27) and elsewhere (e.g., Germany as just one example [28]) has suggested particularly low use and interest in technology for health care among certain groups (minority groups, older persons, persons with lower education). This must be recognized in order to avoid contributing inadvertently to the existing substantial disparities in help-seeking by persons with eating disorders (8).