Open Scientific Practices Are the Way Forward for Internet Gaming Disorder Research: Response to Yao et al.

To the Editor: We appreciate the comments by Yao and colleagues regarding our recent study on Internet gaming disorder, an important yet little-understood phenomenon. In these comments, the authors highlight three aspects of Internet gaming disorder research that we believe merit careful consideration as this area of research develops.

First, the letter highlighted that sampling strategies are important, and we agree. Current estimates of Internet gaming disorder prevalence vary widely as a function of the methodology employed and range from 0.2% of the general population (1) to nearly half of self-selecting gamers (2). Such tremendous variation can be addressed with approaches following best practices in health research (3), as was done in the present work. The issue of sampling methodology should be considered carefully because nonrepresentative samples likely yield biased estimates and lead readers to draw faulty conclusions about the effects of gaming. Survey design, in addition to sampling, is essential. For example, Yao et al. cite an estimate based on a study sample using a Likert-based assessment that deviates both from the proposed DSM-5 criteria and from alternative theoretically rigorous approaches (4). As a result, the 12-month period prevalence estimates range from a modest 1.2% (95% CI=1.0–1.4), in line with our findings, to an implausibly high 6.3% (95% CI=5.8–6.7). Such discrepancies within single studies underline the need to minimize researcher degrees of freedom by distinguishing between exploratory and confirmatory research reporting (5).

Second, as the authors of the letter suggest, there is a dearth of high-quality data investigating Internet gaming disorder in non-Western contexts, but we cannot at present conclude that Internet gaming disorder has any greater or lesser impact on the health or psychosocial functioning of individuals in Asian milieus. Although we acknowledge that there may indeed be unobserved cultural mediators, we believe such factors need to be investigated through a confirmatory lens (5) and without unduly stigmatizing specific cultures or recreational activities (6).

Finally, a strength of our work is the open-science approach we adopted. Because we shared our study materials on the Open Science Framework, there is no need to speculate about how low indicator incidence might have affected sensitivity analyses or how different thresholds would relate to the health outcomes we observed. Indeed, we invite others to draw their own conclusions from dives into the data. We strongly advise researchers investigating this phenomenon to follow our example and share their raw data and analysis code and register their hypotheses before they collect their data. This approach will produce robust empirical insights and improve our understanding of the links between Internet gaming disorder and a wider range of correlates, including personal and clinical factors.