Use of Predictive Analytics to Identify Unhealthy Opioid Use and Guide Intervention

The proprietary Relias Opioid Risk Stratification Tool stratifies individuals into low, medium, or high risk for problematic opioid use. The tool uses an algorithm, developed with logistic regression, that contains 96 data points, such as chronicity of use, multiple opioid prescriptions, use of multiple pharmacies, concurrent opioid use with a diagnosis of opioid use disorder or other substance use disorder, and use of opioids at a high dose without a supporting diagnosis. “Problematic opioid use” is identified when any one of the following occurs in the outcome period: the opioid misuse indicator—signaling aberrant behaviors around drug acquisition and abuse at the end of the outcomes period (e.g., multiple prescribers or pharmacies, malingering or somatization, or chronic or high opioid use without supporting diagnosis)—is triggered; a specific opioid use disorder is diagnosed; some other pattern of misuse emerges that is not a quality indicator (e.g., multiple early refills or for small amounts); or an assessment indicates opioid misuse, addiction, or use disorder. The first three items are extracted from claims data (e.g., pharmacy claims and diagnosis codes), and the fourth is extracted from scores from the Daily Living Assessment–20 for alcohol and drug use (12).

The tiers of problematic use were calculated by using raw risk scores on the basis of a classification model. The algorithm includes 96 variables that contribute to the raw risk score. The resulting score is a value ranging from 0.000 to 1.000, with higher scores indicating high probability of opioid misuse in the 3-month follow-up period. Finally, to determine the risk tiers, the raw risk score is converted to a percentile ranking on the basis of the report period and population. This tool helps inform decisions at the population level and helps focus interventions to the patient or prescriber level.

To validate the risk stratification model, a receiver operating characteristic curve was used to assess the model’s performance in terms of sensitivity and specificity. The model was built with one data set (N=1.4 million monthly patient records) and was validated with another (N=453,000 monthly patient records), such that none of the same patients were in both the training and validation samples. Both samples comprised records from behavioral health organizations in the Midwest or Eastern U.S. regions that serve Medicaid-insured adults and children—similar to the organizations of the intervention and control groups in this study. For the final predictive risk model on the patient set used for validation, the area under the curve was 0.89, suggesting robust performance of the model.

For this study, we included all members of a local managed care organization (MCO) in the Mid-Atlantic region of the United States who were at moderate to high risk for opioid dependence, according to the Opioid Risk Stratification Tool, in January through March 2019. The control group was a convenience sample derived from a different MCO in the same region and included members who were at moderate to high risk for opioid dependence in January through March 2019. All participants in both groups were eligible for Medicaid during the entire study period. The Center for Outcomes Analysis Institutional Review Board approved this study, which involved deidentified data.

Data for analysis were collected from Medicaid claims from Mid-Atlantic providers between November 2018 and May 2019 for the preintervention period and between November 2019 and May 2020 for the postintervention period. The same period was compared across years to avoid any potential seasonality effects in health care use. In addition, we also present results for claims where the postintervention period was truncated to end in March 2020 to account for the onset of the COVID-19 pandemic.

We compared the change in several measures of opioid use between the intervention and control groups during the preintervention period before letters were sent to any prescribers and patients and during the postintervention period.

To analyze receipt of multiple opioid prescriptions, having multiple prescribers of opioid prescriptions, and taking both opioids and benzodiazepines, we used logistic regression, conducted within the generalized estimating equations statistical framework to account for multiple observations (GENMOD procedure in SAS, version 9.4), in order to estimate change in the odds of the outcome at postintervention relative to preintervention. For prescription of any opioids, we used a conventional multivariate binary logistic regression (LOGISTIC procedure). Because taking any opioids was an inclusion criterion, all participants were taking opioids preintervention, eliminating the need for repeated-measures analysis. We used the same logistic regression for MME reduction, where the variable reflected MME reduction from pre- to postintervention versus no reduction. Age and comorbid general medical conditions were controlled for in all analyses because descriptive analyses revealed significant group differences in these two variables. The results were interpreted as ORs, with an OR <1.00 indicating reduced risk and an OR >1.00 indicating increased risk. ORs were considered statistically significant at a two-tailed p<0.05, when the lower and upper limits of the 95% CI did not span 1.00. No data were missing.

As determined by the inclusion criteria, all participants were receiving opioids at preintervention. At postintervention, the odds of having any opioid prescription were significantly lower in the intervention group compared with the control group (OR=0.55, 95% CI=0.39–0.78, p<0.001). Sixty participants (46%) in the intervention group and 120 participants (64%) in the control group were still receiving opioids postintervention. For the sample with a truncated postintervention period, the odds of receiving any opioid prescription at postintervention were lower for the intervention group compared with the control group (OR=0.50, 95% CI=0.35–0.70, p<0.001); 57 participants (44%) in the intervention group (data for one participant were not available in the truncated period) and 120 (64%) in the control group were still receiving any opioids postintervention. No significant differences were detected in the reduction in MME from pre- to postintervention by group (OR=0.99, 95% CI=0.69–1.42).

At baseline, 165 (88%) of those in the control group and 103 (79%) of those in the intervention group were receiving multiple opioid prescriptions, 77 (41%) versus 42 (32%) were receiving opioids from multiple prescribers, and 24 (13%) versus 33 (25%) were receiving concurrent opioid and benzodiazepine prescriptions. The difference between the two groups in multiple prescriptions of opioids was statistically significant at baseline (p=0.021) and postintervention (p=0.003).

Results for changes in the odds of these outcomes are presented in Table 2, revealing significant reductions in the odds of having multiple prescriptions (OR=0.17, p<0.001) and multiple prescribers (OR=0.31, p<0.001) between the pre- and postintervention time points overall, but these reductions did not differ statistically significantly between the two groups. The intervention group had significantly higher odds of concurrent prescription of opioids and benzodiazepines (OR=2.75, p=0.001); we observed a significant reduction in the odds of having concurrent opioid and benzodiazepine prescriptions between pre- and postintervention for both groups (OR=0.58, p=0.019), and this reduction was greater in the intervention compared with the control group (OR=0.49, p=0.042; Figure 1). The results for the sample with a truncated postintervention period (Table 2) were essentially identical to those of the main sample, but the reduction in the concurrent prescription of opioids and benzodiazepines by group was no longer statistically significant.

No opioid-related overdoses or deaths were reported in the claims data for either group.

These results should be interpreted in the context of several methodological limitations. First, participants were not randomly assigned to the intervention or control conditions. Therefore, we could not account for possible group differences that affected outcomes. However, the two organizations comprising the intervention and control groups were similar in size and covered similar populations in the same geographic location. Second, this study was based on claims data, which may contain inaccurate coding and incomplete claims or may not capture deaths. Third, the study contained a relatively short 6-month follow-up period; a longer study period may yield greater differences between groups by allowing for further advancement through the stages of change. Fourth, overall health care resource utilization, including use of providers, emergency department use, and hospital visits, was not evaluated for changes or between-group differences. Fifth, although feedback was requested from providers in the form of an anonymous prestamped postcard, this approach was not successful, limiting our ability to understand provider engagement and to ascertain whether the in-person assessment was conducted.

This study sets the stage for further investigation. Increasing the follow-up time to 12 months would allow for a more extended period of change. The impact of offering patients an anonymous contact line to confidentially ask questions to a provider could be a valuable addition for individuals who may be reluctant to address difficult issues face to face. Finally, other interventions that may affect opioid prescribing practices should be studied.