Three Problems With Current Digital Mental Health Research . . . and Three Things We Can Do About Them

The research literature typically describes and evaluates mental health technologies as if they were products. Doing so assumes that the technology is the primary agent of change. The technology itself is thoroughly described and evaluated, but there is little description or evaluation of the ecosystem around that technology (such as human support and organizational factors). However, the relatively consistent finding has been that substantive mental health benefits are more consistently achieved in the context of human support (1), suggesting that mental health technologies are not products—rather, they are technology-enabled services (TESs). Treating mental health technologies as products has meant that we have largely developed the tools without understanding how they can fit in the context of mental health services.

Improvement in mental health conditions usually requires sustained behavior change over many weeks or months. Currently available mental health technologies require a patient’s time, attention, and motivation—all things that are in short supply when psychiatric disorders sap motivation and are accompanied by hopelessness and helplessness. Furthermore, mental health technologies are mainly didactic or informational, which might not be ideal for promoting sustained engagement and behavior change for many people. Although improved design and technology may make mental health technologies easier and more engaging to use in the future, many of today’s mental health technologies require some human support from a coach or therapist to sustain engagement and obtain substantive, reliable outcomes (1). Thus the goals, methods, and provider qualifications for this support must be established. For example, should support target only engagement, or should it also help patients understand what steps to take, gain insight, and succeed at implementing behavior change in their lives (4)? What are the most effective methods of supporting these goals? Effectiveness encompasses when to use which medium (such as phone calls or various types of messaging), frequency of providing support (continuously, weekly, or as needed), types of communications (supportive, positive reinforcement, clarifying, instructive, and so on), and how to adapt each of these elements to individual patient preferences and needs. What role, if any, should support have in connecting patients to other services? Who should these supporters be—mental health professionals or paraprofessionals?

Reconceptualization of mental health technologies as TESs would highlight these interventions as services that are supported by technologies rather than as human-supported technologies. The implications of this reconceptualization are that the goals and strategies of the service, the role of the provider, and the technology must all be designed and evaluated simultaneously as an integrated service.

Randomized controlled efficacy trials have consistently demonstrated that Web-based and mobile interventions can treat a range of psychiatric disorders, particularly when coupled with low-intensity support provided via brief phone calls or messaging (1,2). Although the large-scale feasibility of such interventions has been demonstrated (5), emerging evidence suggests that the benefits seen in these trials are not realized in practice settings. Gilbody and colleagues (3) found no significant effect in recent large pragmatic trial of two widely used coached mental health technologies for depression. Patients did not engage with the Web-based treatments and stopped answering the telephone support calls. This is consistent with reports we have heard from health care organizations in the United States that have tried these interventions. This research-to-practice gap suggests that there is much to learn about how to implement these interventions.

The design of mental health technologies has been largely top down. We as clinical researchers design tools for patients to support behavioral strategies that we believe are important for successful treatment, likely incorporating some of our own biases about how we like to receive and interact with information. We have typically not done a good job of getting input from patients about their goals, needs, or preferences.

Trials often bear little resemblance to clinical settings, having largely emphasized internal validity over real-world issues, such as the technological environment and implementation and sustainment. The rapid pace of technology development relative to the slow pace of research methods often results in mental health technologies that are outdated and obsolete by the time they are validated (6). Recruitment challenges, common in clinical research, are often addressed by increasing the recruitment pool (expanding the number of venues, refining social media strategies, and so on). Thus, recruitment favors people who are interested in using and likely to adhere to mental health technologies, which is likely an infinitesimal portion of the larger population served by care systems. Essentially, clinical researchers have designed tools to try to get people to do what we want them to do and how we want them to do it—and then searched for and found people who were interested in or willing to use these tools in our trials. Thus, we should not be surprised that these products and services are not appealing to the general population.

For TESs that are intended to be delivered in a larger health care context, design must include input from key stakeholders, including patients, providers, administrators, and information technology managers. Mental health technologies must be designed for the people who will use them. Tools must fit into and leverage people’s daily behavioral patterns, and behavioral science can aid in nudging users to useful actions. For providers (care managers, physicians, and mental health providers), a new TES must fit into their workflows and offer some meaningful benefit rather than just adding another task to their work days. And rather than expecting implementation and sustainment to emerge after a trial, these processes should be built into the TES design from the beginning. User-centered design can be employed from the earliest exploratory stages to help understand and design for the needs, goals, limitations, capabilities, and preferences of all stakeholders (7).

Once a TES is designed, its evaluation should be conducted within the intended setting and examine both its effectiveness and implementation. This can be achieved through hybrid trial designs that integrate simultaneous testing of treatment effects and implementation models (8). Such trials give preference to pragmatic approaches that emphasize usefulness, applicability, and feasibility of new technologies and evaluate them with patients and measurements common to routine treatment settings (9). Initial deployments of intervention technologies, services, and implementation strategies are likely to require adjustments during the trial to address unforeseen and changing circumstances. Rather than locking down TESs during trials, we must employ methods that harness knowledge acquired throughout the trial to optimize interventions, care models, and implementation strategies (10).

Evaluation within a treatment context requires new approaches to recruitment challenges. Most researchers (authors included), when confronted with recruitment challenges, expand the sources of recruitment. This has had the unintended consequence of searching until we find people who are willing to use the tools, thereby damaging the generalizability of the findings. Instead, clinical researchers should view recruitment challenges as indications of failures in the design of the TES, the implementation strategy, or both.

Although the use of technologies for behavior change outside of mental health has tended to view these tools as new forms of interventions (such as activity monitoring or diet management), the perception has persisted in some mental health circles that mental health technologies are a new method of delivering evidence-based psychotherapy via the Web or mobile phone. Leveraging principles from evidence-based treatments has brought us far in a short time, and this will likely continue to be a productive approach. However, we believe this subtle framing is also having a stifling effect on the potential for transformative change in the use of technology for mental health. The notion that mental health technologies should mimic existing evidence-based treatments has become a skeuomorph, limiting our vision of what is possible by maintaining a frame based on past conceptualizations (11).

Most Web-based intervention technologies provide primarily didactic information and some interactive tools—essentially more sophisticated, digital versions of self-help manuals. New technologies and methods, such as artificial intelligence (12), natural language processing (13), and virtual reality (14), open fundamentally new intervention paradigms. A true paradigm shift cannot be achieved by clinging to old models, however; innovation will require new models of behavior change that move away from traditional psychotherapy models and incorporate more granular understanding of the lives of patients and applications of these technologies. We need close collaborations across multiple disciplines, including engineering, computer science, ethics, anthropology, business, psychology, and medicine not only for the expertise they bring but also to challenge mental health experts’ most dearly held assumptions, ideas, and models.

The use of technology for mental health has come a long way in a few years. Past successes have shown us what is possible, but the gaps in our knowledge and approaches have also been revealed. We have hit a bump in the road, which gives us an opportunity to take stock of our achievements and failures and to reconceptualize the needs and direction of this field of research. The ultimate success of the mental health technology revolution requires strengthening the path toward implementation and accelerating innovation by challenging our traditional ways of thinking.