Dichotomies in the Development and Implementation of Digital Mental Health Tools

Even though there are more than 10,000 mental health–related apps available for download in the Apple and Android marketplaces (1), recent meta-analyses have identified only 22 apps for depression symptoms (2) and nine for anxiety disorders (3) that have been assessed in randomized controlled trials. Given the time and expense of conducting randomized controlled trials, compounded by the rapidly evolving nature of technologies such as smartphone apps, experts in the field, such as David Mohr and his colleagues (4), have argued for new study methodologies, such as “trials of intervention principles,” and Heckler and colleagues (5) have argued for an iterative evaluation framework, called “agile science.” Nevertheless, large-scale funders, regulatory bodies such as the Food and drug Administration (FDA), and payers such as insurance companies continue to ask for randomized controlled studies. There is good reason for these demands, because many studies of the use of digital tools in mental health and many studies of app interventions report high effect sizes that quickly vanish when an active control group is considered (2), as seen in a recent study of the popular mindfulness app Headspace (6), perhaps because of a strong digital placebo effect (7).

The potential of technology and smartphone apps is so vast that these tools can be used for both personal medicine and population health. Efforts such as the National Institute of Health’s (NIH’s) All of Us Research Program (a major component of the Precision Medicine Initiative) use smartphones and wearable sensors to reveal unique and personal differences between individuals. With the sensors in today’s smartphones and smartwatches, it is feasible to quantify critical health behaviors, such as physical activity and sleep, and to capture behaviors that were previously challenging to assess, such as sociability, on the basis of smartphone communication logs. However, most large app-based research efforts continue to report results on a population level, such as the recent 8,000-participant Asthma Mobile App study (8) and the 9,500-participant Parkinson’s disease mPower study (9). Delivering personalized insights requires a strong understanding of the digital signature of smartphone data in relationship to each unique individual, which is a challenge given the still-evolving science behind digital phenotyping methods and analyses. It is theoretically possible to design a mobile health platform to offer both precision medicine and population health, but the dichotomous scope and clinical targets often create a difficult choice for app developers and researchers.

At the time of this writing, only one mental health–related smartphone app has received FDA marketing approval of the more than 10,000 smartphone apps available in commercial marketplaces. Lack of FDA oversight is partly a result of the sheer challenge of regulating these apps, which often update on a monthly basis and are constantly changing in functionality. Although the FDA is piloting a new precertification program to regulate smartphone apps, the vast majority of smartphone apps remain unregulated—and marketer of many apps make bold and likely misleading medical claims. A 2016 lawsuit by the Federal Trade Commission (FTC) against Lumosity for deceptive marketing of its brain-training program highlights the potential for real-world harm. However, asking every app developer to submit premarket quality data to the FDA and postmarket quality data to the FTC could stifle innovation, delay implementation, and prevent smaller app developers and research teams from competing. The currently regulatory landscape has led some companies, such as Pear Therapeutics and Akili, to pursue the formal regulatory pathway, but many others have chosen the dichotomous path of labeling their apps as general wellness tools rather than clinical devices.

As a result of regulatory issues, most smartphone apps today market themselves as health and wellness tools aimed toward consumers rather than as clinical devices directed toward health care markets. The marketing, features, and designs of these direct-to-consumer apps naturally force a focus on commercialization, independent of the effectiveness of the underlying intervention. Recent reviews of the quality and efficacy of apps on the marketplaces for disorders such as bipolar disorder (10), substance abuse (11), and mindfulness (12) among others have found that evidence of the effectiveness of interventions in direct-to-consumer apps is often lacking and that the apps do not follow clinical best practices. Because of the current lack of clinical reimbursement related to app use, direct-to-consumer sales are often the only viable pathway to sustain app-related efforts. Although efforts such as the NIH Small Business Innovation Research program offer pathways to transition apps from consumer tools to clinical devices, the dichotomy between consumer versus clinical apps still remains stark.

Much of the potential of smartphone apps is attributable to their ability to passively collect a wealth of real-time sensor data to enable longitudinal behavioral monitoring. Early research has suggested that patterns of geolocation data automatically (or passively) collected from smartphones may be correlated with severity of depression (13) and relapse in schizophrenia (14). However, the same data can be easily misused or mishandled. For example, a privacy breach could enable hackers to know where a patient sleeps at night. The recent breach of over 150 million accounts of users of the fitness app MyFitnessPal underscores how big data and privacy can clash, with unfortunate consequences. The recent Cambridge Analytica scandal, which resulted in unauthorized access to more than 87 million Facebook accounts because of breaches in research ethics, also highlights a darker side of scalability of digital technologies. A subtler but equally important dichotomy lies in the marketing of individuals’ health data gathered from many direct-to-consumer apps. As noted above, many health apps exist outside federal regulations, including the privacy-oriented HIPAA statute, which means that the app developers can legally share, sell, and market users’ personal data. Protecting privacy and gathering vast amounts of data from apps need not be dichotomous, but in today’s landscape they often are.

The potential of digital mental health and smartphone apps is also driven by scalability, with the notion that what works on one smartphone has the potential to work on billions of others already in use across the world. However, this scalability is hindered by proprietary software that limits open and reproducible science. Currently, many groups are developing smartphone app–related tools and data analysis methods but restricting access to source code used to create their smartphone app or resulting data sets. Only a handful of smartphone app–related studies in mental health have ever been reproduced, and these studies often yield contradictory results (15). Commercial funders will likely not favor open software efforts in which they are asked to share code and algorithms, and grant-supported efforts may not sustain open source tools and data repositories when funding expires. This dichotomy between open and closed platforms hampers the field by limiting reproducible science during the critical development phase of the digital health space.

The six dichotomies in the uptake and adoption of digital health tools presented above each embody unique paradoxes and conflicting demands. Together, dichotomies 1, 2, and 3 represent a wealth of new data and hypotheses, driven by the potential of digital phenotyping. However, this potential is juxtaposed with a paucity of rigorous or replicable findings—and an evidence base of such findings is the hallmark of a mature field. Dichotomies 4, 5, and 6 represent the dynamic pace of technology change and the need to support rapid innovation juxtaposed with the need for safety, privacy, and transparency.

Given how these current dichotomies, either alone or in sum, are fragmenting the digital mental health field, limiting collaboration, and impairing reproducible science, there is an urgent need for change. Awareness of these dichotomies is growing within federal funding agencies (16) and foundations, but a rapid solution is not readily apparent. Although there will always be custom solutions to each dichotomy, the tensions driving these dichotomies are not easily resolved. Rather, here we reframe these dichotomies as a need to acknowledge some degree of inconsistency and to realize that successful digital mental health efforts will often have to pivot between competing positions. Rather than viewing this inconsistency as a weakness, the ability to adapt a “both/and” instead of an “either/or” mindset may be more productive (17). Understanding and evaluating digital psychiatry tools by using this lens may offer a more variable but perhaps more valid understanding of the opportunities and challenges ahead.