Assessing Substance Use Disorder Among Social Security Administration Disability Applicants
Abstract
Objective:
Co-occurrence of substance use disorder and mental illness complicates treatment and is associated with increased disability. However, identification of substance use disorder in populations recently engaged in treatment can be challenging. This study aimed to examine traditional screening tools for substance use disorder and proxy characteristics (i.e., demographic characteristics, behaviors, and diagnoses) as predictors of clinician-observed substance use disorder.
Methods:
The Supported Employment Demonstration, funded by the Social Security Administration, studied 2,960 adults whose applications for disability benefits because of mental illness were recently denied. In a subsample (N=1,354) for whom substance use disorder was determined by community-based teams during follow-up, the authors used logistic regression to identify baseline predictors of observed substance use disorder and compared the sensitivity and specificity of detection by using standardized screening tools (Alcohol Use Disorder Identification Test [AUDIT], 10-item Drug Abuse Screening Test [DAST-10]) with detection via decision trees based on proxy characteristics.
Results:
Using decision trees based on a combination of personal characteristics (sensitivity=0.47, specificity=0.83, area under the curve [AUC]=0.71) or personal characteristics including diagnostic data (sensitivity=0.54, specificity=0.81, AUC=0.72) provided more accurate identification of substance use disorder than using a combination of the AUDIT and DAST-10 (sensitivity=0.34, specificity=0.88, AUC=0.61). Adding the combined AUDIT and DAST-10 substance use disorder indicator to either tree negligibly improved accuracy.
Conclusions:
In populations recently engaged in treatment, for whom standardized substance use disorder screening tools are less accurate than expected, consideration of personal characteristics could improve the detection of substance use disorder essential for treatment planning.
HIGHLIGHTS
•
Traditional screening tests for substance use disorder can be inaccurate in populations that tend to inaccurately self-report drug and alcohol use.
•
Other methods of substance use disorder identification, such as time-line follow-back interviews, are resource intensive.
•
This study compared the accuracy of using personal characteristics and behaviors, such as sex and smoking, to that of using the Alcohol Use Disorder Identification Test and 10-item Drug Abuse Screening Test in identifying substance use disorder.
•
Personal characteristics yielded more accurate identification of substance use disorder compared with traditional screening tools, indicating that the use of such readily available information could improve routine substance use disorder detection essential for treatment planning.
Among people with serious mental illness and other mental health conditions, the presence of co-occurring substance use disorder complicates and adversely affects illness trajectories that can be effectively managed only if the disorder is identified (1, 2). Accurate assessment is relatively straightforward among those who are already in treatment for and who self-report substance use, but biased self-reporting can make identification difficult among those recently engaged by treatment providers. This bias has several sources (3). People with mental illness develop difficulties with relatively small amounts of substance use, confuse the effects of substances with mental illness, and, compared with those who use substances and who do not have mental illness, experience different consequences of use because of reduced personal resources and support. Admitting to having a substance use disorder and related problems can also lead to loss of both child custody and access to benefits such as housing, disability payments, employment, and welfare. These consequences provide ample motive for dissembling. When patients are given enough time to interact with care managers and other community teams, correct identification of substance use disorder is likely (4–6), but failure to identify it at initial engagement represents an opportunity cost associated with delayed treatment. In this study, we examined the potential use of self-reported personal characteristics to substitute for or augment established measures for substance use detection in a population at high risk for disability and substance use disorder.
The Supported Employment Demonstration (SED) was a study of people whose first applications for disability benefits (Supplemental Security Income or Social Security Disability Insurance), based on an alleged mental impairment in the past 30–60 days, were denied (https://www.ssa.gov/disabilityresearch/supported_employment.html) (7). To help enrollees find employment and reduce rates of reapplication and eventual award of disability benefits, the SED randomly assigned participants to one of two treatment arms (basic or full) or a treatment-as-usual control group (N=2,960). After a 1-year enrollment period, the SED conducted follow-up of each participant for 3 years. Participants in the basic- and full-treatment arms were offered team-based care management, behavioral health care, and individual placement and support (7), an effective, evidence-based practice that helps people with serious mental illness find employment (8). Participants in the full-treatment arm were also offered medication management support and coordination of medical services from a nurse care coordinator (7). Trained interviewers assessed baseline substance use disorder status in person for all enrollees by using the Alcohol Use Disorder Identification Test (AUDIT) and the 10-item Drug Abuse Screening Test (DAST-10). These screening measures have some utility for individuals with either known substance use disorder or a recent history of treatment for mental illness (9–12). Participation in the SED was not predicated on meeting either of these criteria. SED participants reported a low level of substance use disorder on the AUDIT and DAST-10, a self-assessment that conflicted with early qualitative data, including clinical observations and interviews of providers with study enrollees, and a high prevalence of baseline characteristics known to be associated with substance use disorder (13). Examples of such prevalent characteristics (13, 14) included back pain (63%) (15, 16), smoking (46%) (17, 18), obesity (47%) (19, 20), not being employed (81%) (21, 22), homelessness (5%) (23–25), a history of arrest (54%) (26), and high school or less as the highest educational level attained (49%) (27, 28). Baseline diagnostic interviews that used the Composite International Diagnostic Interview with 1,842 SED participants indicated a high prevalence of anxiety disorders (71%) and antisocial personality disorders (46%) among these participants (14, 29).
SED participants experienced clinical difficulties, such as those related to legal problems, family conflicts, or medical conditions related to substance use disorder, which reinforced the importance of reliable substance use disorder identification (30). Projects like the SED that rely on baseline measures to determine substance use disorder status may not include more reliable approaches, such as the use of collateral information from friends and family (31), urine drug screens (6), time-line follow-back interviews (32, 33), extensive research interviews (34), or the collection of basic demographic, behavioral, and diagnostic characteristics associated with substance use disorder (5). The SED enabled evaluation of this last option by providing a large sample of baseline data and a long period of follow-up during which treatment teams, including employment specialists, case managers, and clinicians, evaluated participants’ substance use (6). The objective of this study was to examine the sensitivity and specificity of the baseline AUDIT and DAST-10 scores for identification of substance use disorder, to determine the associations of known enrollee characteristics with observed substance use disorder, and to use classification tree analysis (CTA) to demonstrate practical means of using combinations of personal characteristics to identify participants with substance use disorder.
Methods
Participants
This study assessed detection of potential substance use disorder based on combined AUDIT and DAST-10 scores and baseline characteristics in a subsample (N=1,354) of the SED cohort for whom observed substance use disorder status was known and in a subset of that group for whom the presence or absence of posttraumatic stress disorder (PTSD), antisocial personality disorder, anxiety disorder, and mood disorder diagnoses was known (N=706). We chose not to define smoking as a substance use disorder; instead we considered it an observable behavior, because smoking had the potential to indicate alcohol abuse or drug use that the AUDIT and DAST-10 may have failed to detect. Standard cutoff scores for the AUDIT and DAST-10 were used to determine sensitivity and specificity, with observed substance use disorder status as the gold standard for detection of alcohol abuse or drug use. We constructed five series of decision trees that were based on combinations of the baseline AUDIT and DAST-10 substance use disorder indicator and on baseline personal and diagnostic characteristics, and we determined the mean sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) for each.
Measures
Determination of substance use disorder.
Observed and instrument-based measures of substance use disorder were expressed as indicator variables.
Observed behavior.
At the end of follow-up, SED treatment teams classified each participant in both intervention arms as having or not having a substance use disorder or as not having been observed fully enough to make a determination; these data were collected in Microsoft Excel. Not all participants in either treatment arm participated in services. Despite extensive and ongoing attempts to engage enrolled participants, many never engaged with their treatment teams (largely because of inaccurate contact information and lack of response to outreach) (35) or declined any participation in services (36). During discussions, treatment teams applied DSM-V criteria (37) on the basis of team members’ interactions with study enrollees to determine substance use disorder status. This measure (i.e., observed substance use disorder by community-based teams) constituted the gold standard for detection of substance use disorder in this study. Subsequent analyses omitted individuals whose substance use disorder status was not observed.
Instrument-based measures.
Every SED participant completed the AUDIT and DAST-10 at baseline. Meeting threshold scores of 7 for the AUDIT or 6 for the DAST-10 indicated presence of a substance use disorder.
The AUDIT is a 10-item screening test designed to detect high-risk drinking. Reliability and construct and criterion validity are acceptable. In populations with a range of alcohol use patterns, sensitivity and specificity of the AUDIT have generally ranged from 0.7 to 1.0 (38). In standard usage, scores range from 0 to 40, with higher scores indicating more harmful alcohol use and a cutoff score of 8 indicating harmful alcohol use and potential alcohol dependence (39). Cutoff sores of 7 or 8 were identified as optimal for people with severe mental illness (9).
The DAST-10 is a 10-item version of the original 28-item Drug Abuse Screening Test (DAST) that is scored on a scale of 0–10, with higher scores indicating more harmful drug use. It has been reported to have moderate to high validity, sensitivity, and specificity. Most studies of the psychometric properties of the DAST and DAST-10 have been conducted in populations of individuals known to use drugs or with patients with an established psychiatric history (12). Scores of 6 and 9 represent standard cutoffs, indicating substantial and severe substance use, respectively, and lower cutoff scores optimize detection of problematic drug use among those with serious mental illness (9–11). The SED sample contained individuals with a range of mental illness severities.
Diagnoses.
Diagnoses of PTSD, antisocial personality disorder, anxiety disorder, and mood disorder were based on the results of the Composite International Diagnostic Interview administered by trained interviewers either in person or by telephone to 1,842 SED enrollees soon after enrollment.
Baseline characteristics.
Baseline demographic, behavioral, and general medical measures included age (<35 or ≥35 years), sex (male or female), race (White, Black, Asian, American Indian or Alaska Native, or two or more races), ethnicity (Hispanic or non-Hispanic), highest level of education achieved (less than high school, high school or general equivalency diploma, some college or technical school, or completed bachelor’s degree), current housing status (homeless or in a shelter, multiple apartment or house addresses, or one apartment or house address), any employment in the past 2 years (yes or no), current smoker (yes or no), any history of arrest (yes or no), obesity at baseline (yes or no), and back pain at baseline (yes or no).
Statistical Analysis
Data preparation.
This analysis used a combination of SED baseline data and observed substance use disorder status. To assess the generalizability of results to the SED sample, we compared the distribution of the chosen baseline characteristics in two groups: participants with observed substance use disorder status and participants with no observed status.
Detection of substance use disorder.
We first determined the sensitivity and specificity of the combined AUDIT and DAST-10 instruments by using the full sample of individuals whose substance use disorder status had been observed. Two multivariable logistic regressions were used to describe the associations of positive substance use disorder detection (on the basis of the combined baseline indicator) and of baseline characteristics, including or not including diagnostic data, with observed substance use disorder status. The model that included diagnostic data was restricted to the subset for whom these data were available. To account for type I error inflation associated with multiple hypothesis tests, we selected an α=0.003.
We used CTA to evaluate discrimination of substance use disorder status, with five sets of predictors to be used as potential nodes; each tree was trained and validated in the sample for which predictors were available. CTA hierarchically maximizes correct classification without assuming multivariate normal distributions and can illuminate patterns of interaction in the data and predictive values relevant to practical identification of substance use disorder (40). We computed sensitivity and specificity with confusion matrices and reported AUCs for CTAs. Higher AUCs (those between 0.5 and 1.0) indicate that a model discriminates between outcomes increasingly well compared with pure chance. Predictor sets included the baseline substance use disorder indicator, baseline characteristics of SED enrollees, and baseline characteristics and psychiatric diagnoses. To the latter two groups, we then added the baseline substance use disorder indicator. Performance measures represent means based on 10 CTAs for each predictor set; each CTA used 70% of the sample for training and 30% for validation. We used pruning with the C4.5 algorithm (41) to avoid overfitting the data and improve generalizability to other data sets; we limited the tree to a depth of four, binary branching, and a minimum of 20 enrollees per leaf. For purposes of illustration, we created a classification tree by using the entire sample for training and based on the four baseline characteristics most frequently utilized across 40 decision trees. Analyses were conducted in SAS, version 9.4 (42), with proc logistic and proc hpsplit.
All procedures contributing to this work complied with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.
Results
Treatment teams provided observed substance use disorder status for 1,354 enrollees. Although largely representative of the SED cohort, they were slightly older, had higher educational attainment, were less likely to smoke at baseline, and had higher rates of anxiety and mood disorders than those for whom observed substance use disorder status was unavailable (Table 1).
| Observed (N=1,354) | Not observed (N=1,674)a | ||||||
|---|---|---|---|---|---|---|---|
| Characteristic | N | % | N | % | χ2 | df | p |
| Back pain | |||||||
| No | 488 | 36 | 618 | 37 | .61 | 1 | .435 |
| Yes | 864 | 64 | 1,031 | 63 | |||
| Sex | |||||||
| Female | 781 | 58 | 917 | 56 | 1.41 | 1 | .236 |
| Male | 571 | 42 | 732 | 44 | |||
| Age in years | |||||||
| 18–34 | 543 | 40 | 732 | 44 | 5.43 | 1 | .020 |
| ≥35 | 809 | 60 | 917 | 56 | |||
| Arrest history | |||||||
| No | 612 | 46 | 776 | 47 | .73 | 1 | .394 |
| Yes | 729 | 54 | 868 | 53 | |||
| Educational attainment | |||||||
| High school or general equivalency diploma | 374 | 28 | 532 | 32 | 8.25 | 3 | .041 |
| Completed bachelor’s | 236 | 17 | 267 | 16 | |||
| Some high school | 253 | 19 | 309 | 19 | |||
| Some college or technical school | 488 | 36 | 541 | 33 | |||
| Ethnicity | |||||||
| Hispanic | 169 | 13 | 208 | 13 | .01 | 1 | .943 |
| Non-Hispanic | 1,167 | 87 | 1,425 | 87 | |||
| Housing | |||||||
| 1 apartment or house address | 1,168 | 86 | 1,435 | 87 | 1.23 | 2 | .541 |
| >1 apartment or house address | 106 | 8 | 132 | 8 | |||
| Homeless or shelter | 77 | 6 | 79 | 5 | |||
| Obesity | |||||||
| No | 687 | 51 | 871 | 53 | 1.26 | 1 | .263 |
| Yes | 652 | 49 | 761 | 47 | |||
| Race | |||||||
| ≥2 | 138 | 11 | 141 | 9 | 5.44 | 4 | .245 |
| American Indian or Alaska Native | 23 | 2 | 40 | 3 | |||
| Asian | 8 | 1 | 15 | 1 | |||
| Black | 397 | 31 | 499 | 32 | |||
| White | 733 | 56 | 868 | 56 | |||
| Employed in past 2 years | |||||||
| No | 442 | 34 | 550 | 35 | .35 | 1 | .553 |
| Yes | 874 | 66 | 1,038 | 65 | |||
| Current smoker | |||||||
| No | 651 | 48 | 701 | 43 | 10.03 | 1 | .002 |
| Yes | 693 | 52 | 943 | 57 | |||
| Antisocial personality disorder | |||||||
| No | 462 | 54 | 554 | 55 | .27 | 1 | .602 |
| Yes | 394 | 46 | 450 | 45 | |||
| Anxiety disorder | |||||||
| No | 227 | 26 | 315 | 31 | 5.15 | 1 | .023 |
| Yes | 636 | 74 | 699 | 69 | |||
| Mood disorder | |||||||
| No | 312 | 36 | 412 | 41 | 3.95 | 1 | .047 |
| Yes | 551 | 64 | 602 | 59 | |||
| PTSD | |||||||
| No | 419 | 54 | 482 | 53 | .18 | 1 | .673 |
| Yes | 357 | 46 | 428 | 47 | |||
a
Represents the remainder of all participants initially enrolled at baseline for whom observed substance abuse status was not available by the end of the follow-up period.
The 2,959 and 2,946 SED enrollees who completed baseline AUDIT and DAST-10 assessments, respectively, had a computed baseline substance use disorder prevalence of 19% (N=544 of 2,922). The observed substance use disorder prevalence during follow-up was 32% (N=433 of 1,354). Assuming that observed substance use disorder status constituted a gold standard, we found that the baseline indicator detected substance use disorder with a sensitivity of 0.34, specificity of 0.88, and positive and negative predictive values of 0.57 and 0.74, respectively. The latter values represent the probabilities that positive or negative test results are correct.
Table 2 shows the results of both logistic regressions, including the ORs associated with each parameter. In the model that used only baseline characteristics as predictors (N=1,222), smoking versus nonsmoking (OR=3.32, p<0.001), having versus not having an arrest history (OR=2.28, p<0.001), and being homeless or living in a shelter versus living at a single fixed address (OR=2.83, p<0.001) were significantly associated with substance use disorder. In the model that also included baseline diagnoses (N=706), smoking versus nonsmoking (OR=2.94, p<0.001), having versus not having an arrest history (OR=2.38, p<0.001), and having versus not having a diagnosis of antisocial personality disorder (OR=1.87, p=0.001) were associated with substance use disorder.
| Observed substance use disorder status (N=1,222) | Diagnostic data also available (N=706) | |||||
|---|---|---|---|---|---|---|
| Characteristic | OR | 95% CI | p | OR | 95% CI | p |
| Ages ≥35 years (reference: <35 years) | .85 | 0.64–1.13 | .267 | .78 | 0.53–1.15 | .209 |
| Female (reference: male) | .88 | 0.67–1.17 | .379 | .95 | 0.64–1.40 | .789 |
| Back pain (reference: no) | 1.30 | 0.97–1.74 | .074 | 1.38 | 0.92–2.07 | .116 |
| Obesity (reference: no) | .67 | 0.51–0.89 | .005 | .60 | 0.41–0.87 | .007 |
| Current smoker (reference: no) | 3.32 | 2.48–4.45 | <.001 | 2.94 | 1.96–4.43 | <.001 |
| Educational attainment (reference: less than high school) | ||||||
| High school or general equivalency diploma | .91 | 0.62–1.35 | .648 | .64 | 0.37–1.09 | .098 |
| Completed bachelor’s | .54 | 0.34–0.88 | .014 | .36 | 0.18–0.72 | .004 |
| Some college or technical school | .72 | 0.50–1.05 | .088 | .52 | 0.32–0.86 | .012 |
| Housing status (reference: 1 address) | ||||||
| >1 apartment or house address | 1.24 | 0.76–2.02 | .380 | .88 | 0.43–1.79 | .729 |
| Homeless or shelter | 2.83 | 1.58–5.05 | <.001 | 2.85 | 1.23–6.62 | .015 |
| History of arrest, yes (reference: no) | 2.28 | 1.68–3.08 | <.001 | 2.38 | 1.56–3.63 | <.001 |
| Race (reference: White) | ||||||
| ≥2 | .81 | 0.52–1.25 | .339 | 1.18 | 0.65–2.16 | .580 |
| American Indian or Alaska Native | 1.01 | 0.37–2.74 | .988 | 1.22 | 0.30–5.02 | .783 |
| Asian | .53 | 0.06–4.76 | .570 | .00 | — | .984 |
| Black | .83 | 0.61–1.12 | .227 | .78 | 0.51–1.20 | .266 |
| Hispanic ethnicity (reference: non-Hispanic) | .90 | 0.56–1.44 | .649 | 1.10 | 0.60–2.01 | .763 |
| Employed in past 2 years, no (reference: yes) | 1.12 | 0.84–1.49 | .440 | 1.12 | 0.76–1.66 | .567 |
| PTSD (reference: no) | .84 | 0.55–1.28 | .420 | |||
| Antisocial personality disorder (reference: no) | 1.87 | 1.27–2.73 | .001 | |||
| Anxiety disorder (reference: no) | 1.13 | 0.65–1.94 | .670 | |||
| Mood disorder (reference: no) | .98 | 0.65–1.47 | .908 | |||
a
Results of multivariable logistic regressions in which baseline personal characteristics were used to predict observed substance use disorder status among individuals with observed substance use disorder status and among those with both observed substance use disorder status and diagnostic information. Models excluded observations with any missing predictors.
The results of the CTAs, which were based on random sample partitions and displayed a range of performance characteristics, are shown in Table 3. The trees created by using baseline characteristics had a mean AUC of 0.71, sensitivity of 0.47, and specificity of 0.83; those that included diagnostic status had a mean AUC of 0.72, sensitivity of 0.54, and specificity of 0.81. Addition of the baseline substance use disorder indicator to these predictor groups only negligibly improved the AUC at the cost of decreased sensitivity. Figure 1 shows the classification tree that was trained by using the entire sample and was based on the four baseline nondiagnostic characteristics most frequently selected for the prior decision trees (smoking, arrest history, education, and housing). Only 16% (N=90 of 577) of nonsmokers, compared with 47% (N=300 of 645) of smokers, and 32% (N=60 of 189) of smokers without arrest histories, compared with 53% (N=240 of 456) of smokers with arrest histories, had observed substance use disorder. Of SED enrollees who smoked, had an arrest history, and were homeless at baseline, 76% (N=31 of 41) were observed to have a substance use disorder.
| Sensitivity | Specificity | Area under the curve | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Predictors | Training | Validation | Meanb | Training | Validation | Meanb | Training | Validation | Meanb |
| AUDIT and DAST-10 only (N=1,315 participants) | .33 | .36 | .34 | .88 | .88 | .88 | .61 | .62 | .61 |
| Baseline characteristics only (N=1,222 participants) | .48 | .44 | .47 | .83 | .83 | .83 | .71 | .72 | .71 |
| Baseline characteristics and diagnoses (N=706 participants) | .55 | .52 | .54 | .80 | .82 | .81 | .73 | .71 | .72 |
| AUDIT and DAST-10 and baseline characteristics (N=1,187 participants) | .37 | .35 | .36 | .89 | .88 | .89 | .72 | .71 | .72 |
| AUDIT and DAST-10, baseline characteristics, and diagnoses (N=685 participants) | .39 | .35 | .37 | .90 | .89 | .89 | .74 | .72 | .73 |
a
Each row describes the mean performances, across 10 decision trees based on the specified combinations of predictors, in identifying observed substance use disorder. Training sets used 70% of available sample, and validation sets used the remaining 30%. AUDIT, Alcohol Use Disorder Identification Test; DAST-10, 10-item Drug Abuse Screening Test.
b
Means of training and validation samples, weighted by relative sizes of samples.
FIGURE 1. Decision tree for observed substance use disorder (SUD)a
aTrained on the basis of the entire sample (N=1,354) and on the basis of the four baseline characteristics most commonly selected across 40 validated trees (sensitivity=0.58, specificity=0.78, area under the curve=0.72).
Discussion
Several key points emerged from this study of the detection of substance use disorder among individuals whose first application for disability benefits because of a mental illness was denied. First, screening for substance use disorder status based on a few personal characteristics commonly known at intake was more accurate than AUDIT and DAST-10 scores. Second, combining these characteristics with the results of the AUDIT and DAST-10 instruments did not substantially improve detection of substance use disorder. Third, further work is necessary to understand the practical value of personal characteristics to screening and to combine these findings with existing techniques in order to develop a more efficient screening process.
The approach used here resembles that of a previous study of substance use among hospitalized psychiatric patients with severe mental illness, which also used hierarchical regression tree analysis to identify characteristics associated with alcohol, cocaine, or cannabis use (5). The themes of incarceration, antisocial personality disorder, and housing identified in that study echo those identified in our SED sample, but the former study omitted other background characteristics associated with substance use disorder, including tobacco use and arrest history. Previous research has reported mixed success in using existing screening instruments to detect substance use disorder. Although some studies have indicated that measures such as the AUDIT, DAST, and DAST-10 are useful for detecting substance use among recently hospitalized psychiatric patients (9, 11), others have concluded that these measures are generally of low value (6). Some of this variance may be due to the sensitivity of standardized instruments to both the skill of the interviewer and the characteristics of the enrolled participants (43). These difficulties are compounded in populations, such as that sampled for the SED, with histories of arrest and of seeking disability benefits (3). The results of this study reinforce the inaccuracy of established measures for screening for substance use disorder applied in isolation to disability applicants with mental illness and highlight the importance of confirming test performance under challenging circumstances. Subgroup-specific prediction of substance use disorder represents an effective and easily implemented augmentation of existing practices.
Basic demographic, behavioral, and diagnostic data that correlate with substance use disorder are commonly available when clinicians initially engage patients with mental illness. The AUDIT and DAST-10 accurately screen for possible substance use disorder when responses are unbiased, a condition that appears to be consistently unmet in this population. Research is necessary to confirm whether other established instruments are subject to the same shortcoming and to explore whether recall bias can be reduced. However, in populations prone to biased recall, detection of possible substance use disorder based on specific background characteristics appears to be more accurate than the use of simple screening instruments. This finding is of crucial importance when more resource-intensive techniques are impractical. Given larger samples and the measurement and evaluation of additional personal traits, such as the characteristics of relatives (44), subgroup-specific detection of substance use disorder could constitute a first level of screening. This screening would be of specific value in clinical environments treating populations that are prone to biased recall and lacking the resources for universal application of more rigorous procedures. The process could be implemented by individual providers (e.g., case managers) or through automated application to electronic health records. Additional methods, such as time-line follow-back interviews (32), could then be applied selectively to individuals in high-risk subgroups. Development of tiered substance use disorder detection based on high-risk subgroup identification that guides deployment of more accurate methods would enhance the convergent validity long known to compensate for shortcomings in any one method of detection (45).
Use of observed substance use disorder status as a gold standard was a primary limitation of this study. Evidence suggests that this information is of high accuracy, but in this study, it represented a period prevalence (i.e., the number of cases during a specified period divided by the total number of people in the population) of substance use disorder. The AUDIT and DAST-10 scores yielded an estimate of the baseline substance use disorder as a measure of point prevalence. Although the emergence of many new substance use disorders during the study was unlikely, this difference between period prevalence and point prevalence may have caused an upward bias in the relative estimated risk for substance use disorder by including substance use disorders that developed during the follow-up period. Also, the group with observed substance use disorder status was not entirely representative of the SED sample, being older, more educated, and less likely to smoke and having higher rates of anxiety and mood disorders, thereby limiting generalizability.
Conclusions
Traditional substance use disorder screening instruments are inaccurate, especially for people who are under surveillance or applying for various benefits and whose drug use may have perceived negative legal or financial consequences. Biased responses to items in tests such as the AUDIT and DAST-10 can cause underestimation of substance use disorder prevalence and lead to misdiagnosis, inaccurate treatment planning, and suboptimal care. Combining personal characteristics correlated with substance use disorder with other detection methods could increase the reliability of substance use disorder identification at the point of initial engagement.
Acknowledgments
The authors thank Mustafa Karakus for his comments on the manuscript.
REFERENCES
1.
Drake RE, O’Neal EL, Wallach MA: A systematic review of psychosocial research on psychosocial interventions for people with co-occurring severe mental and substance use disorders. J Subst Abuse Treat 2008; 34:123–138
2.
Mueser K, Noordsy D, Drake R, et al: Integrated Treatment for Dual Disorders: A Guide to Effective Practice. New York, Guilford, 2003
3.
Mueser KT, Drake RE, Clark RE, et al: Evaluating Substance Abuse in Persons With Severe Mental Illness. Cambridge, MA, Human Services Research Institute, 1995
4.
Drake RE, Osher FC, Noordsy DL: Diagnosis of alcohol use disorders in schizophrenia. Schizophr Bull 1990; 16:57–67
5.
Mueser KT, Yarnold PR, Rosenberg SD: Substance use disorder in hospitalized severely mentally ill psychiatric patients: prevalence, correlates, and subgroups. Schizophr Bull 2000; 26:179–192
6.
Wolford G, Rosenberg S, Drake R, et al: Evaluation of methods for detecting substance use disorder in persons with severe mental illness. Psychol Addict Behaviors 1999; 13:313–326
7.
Riley J, Drake RE, Frey W, et al: Helping people denied disability benefits for an alleged mental health impairment: the Supported Employment Demonstration. Psychiatric Serv 2021; 72:1434–1440
8.
Drake RE, Wallach MA: Employment is a critical mental health intervention. Epidemiol Psychiatr Sci 2020; 29:e1782020
9.
Maisto SA, Carey MP, Carey KB: Use of the AUDIT and the DAST-10 to identify alcohol and drug use disorders among adults with a severe and persistent mental illness. Psychol Assess 2000; 12:186–192
10.
Cocco K, Carey K: Psychometric properties of the Drug Abuse Screening Test in psychiatric outpatients. Addict Behav 1990; 15:257–264
11.
Staley D, el-Guebaly N: Psychometric properties of the Drug Abuse Screening Test in a psychiatric patient population. Addict Behav 1990; 15:257–264
12.
Yudko E, Lozhkina O, Fouts A: A comprehensive review of the psychometric properties of the Drug Abuse Screening Test. J Subst Abuse Treat 2007; 32:189–198
13.
Supported Employment Demonstration: Early Assessment Report. Woodlawn, MD, Social Security Administration, 2019
14.
Borger C, Morrow J, Drake R, et al: Characteristics of enrollees in the Supported Employment Demonstration. Psychiatric Serv 2021; 72:1400–1406
15.
Dale AM, Buckner-Petty S, Evanoff BA, et al: Predictors of long-term opioid use and opioid use disorder among construction workers: analysis of claims data. Am J Ind Med 2021; 64:48–57
16.
Pensa MA, Galusha DH, Cantley LF: Patterns of opioid prescribing and predictors of chronic opioid use in an industrial cohort, 2003 to 2013. J Occup Environ Med 2018; 60:457–461
17.
McHugh RK, Janes AC, Griffin ML, et al: Clinical correlates of smoking status in men and women with opioid use disorder. Subst Use Misuse 2020; 55:1054–1058
18.
Pettey D, Aubry T: Tobacco use and smoking behaviors of individuals with a serious mental illness. Psychiatr Rehabil J 2018; 41:356–360
19.
Stokes A, Berry KM, Collins JM, et al: The contribution of obesity to prescription opioid use in the United States. Pain 2019; 160:2255–2262
20.
Yusufov M, Kopeski LM, Silverman AL: Associations of body weight and waist circumference with psychopathology, substance use, and well-being in an adult transdiagnostic sample. J Affect Disord 2021; 281:279–288
21.
Compton WM, Gfroerer J, Conway KP, et al: Unemployment and substance outcomes in the United States 2002–2010. Drug Alcohol Depend 2014; 142:350–353
22.
Henkel D: Unemployment and substance use: a review of the literature (1990–2010). Curr Drug Abuse Rev 2011; 4:4–27
23.
Caton CLM, Dominguez B, Schanzer B, et al: Risk factors for long-term homelessness: findings from a longitudinal study of first-time homeless single adults. Am J Public Health 2005; 95:1753–1759
24.
Schanzer B, Dominguez B, Shrout PE, et al: Homelessness, health status, and health care use. Am J Public Health 2007; 97:464–469
25.
Stringfellow EJ, Kim TW, Gordon AJ, et al: Substance use among persons with homeless experience in primary care. Subst Abus 2016; 37:534–541
26.
Prince JD, Wald C: Risk of criminal justice system involvement among people with co-occurring severe mental illness and substance use disorder. Int J L Psychiatry 2018; 58:1–8
27.
Duan WR, Hand DJ: Association between opioid overdose death rates and educational attainment—United States, 2010–2019. Prev Med 2021; 153:106785
28.
Results From 2020 National Survey on Drug Use and Health. Table 5.5B—Substance Use Disorder in Past Year: Among People Aged 12 or Older; by Age Group and Demographic Characteristics, Percentages, 2019 and 2020. Rockville, MD, Substance Abuse and Mental Health Services Administration, 2021
29.
Grant BF, Stinson FS, Dawson DA, et al: Prevalence and co-occurrence of substance use disorders and independent mood and anxiety disorders: results from the National Epidemiologic Survey on Alcohol and Related Conditions. Arch Gen Psychiatry 2004; 61:807–816
30.
Smith T, Bury D, Hendrick D, et al: Barriers to client engagement and strategies to improve participation in mental health and supported employment services. Psychiatr Serv (Epub Sep 6, 2022)
31.
Carey KB, Simons J: Utility of collateral information in assessing substance use among psychiatric outpatients. J Subst Abuse 2000; 11:139–147
32.
Carey KB, Carey MP, Maisto SA: Temporal stability of the timeline followback interview for alcohol and drug use with psychiatric outpatients. J Stud Alcohol 2004; 65:774–781
33.
Sacks JY, Drake RE, Williams VF: Utility of the Time-Line Follow-Back to assess substance use among homeless adults. J Nerv Ment Dis 2003; 191:145–153
34.
Hasin DS, Trautman KD, Miele GM: Psychiatric Research Interview for Substance and Mental Disorders (PRISM): reliability for substance abusers. Am J Psychiatry 1996; 153:1195–1201
35.
Metcalfe JD, Drake RE: Participation in individual placement and support in the Supported Employment Demonstration. Adm Policy Ment Health 2021; 49:521–529
36.
Bury D, Hendrick D, Smith T, et al: The psychiatric nurse care coordinator on a multi-disciplinary community mental health treatment team. Community Ment Health J 2022; 58:1354–1360
37.
Diagnostic and Statistical Manual of Mental Disorders, 5th ed. Washington, DC, American Psychiatric Association, 2022
38.
Reinert DF, Allen JP: The Alcohol Use Disorders Identification Test: an update of research findings. Alcohol Clin Exp Res 2007; 31:185–199
39.
Babor T, Higgins-Biddle J, Saunders J, et al: The Alcohol Use Disorders Identification Test: Guidelines for Use in Primary Care. Geneva, World Health Organization, 2001
40.
Breiman L, Friedman J, Stone C, et al: Classification and Regression Trees. Belmont, CA, Wadsworth, 1984
41.
Quinlan JR: C4.5: Programs for Machine Learning. San Francisco, Morgan Kaufmann, 1993
42.
SAS 9.4 Output Delivery System User’s Guide. Cary, NC, SAS Institute, 2013
43.
Babor TF, Brown J, del Boca FK: Validity of self-reports in applied research on addictive behaviors: fact or fiction? Behav Assess 1990; 12:5–31
44.
Mueser KT, Glynn SM, Cather C, et al: Family intervention for co-occurring substance use and severe psychiatric disorders: participant characteristics and correlates of initial engagement and more extended exposure in a randomized controlled trial. Addict Behav 2009; 34:867–877
45.
Sobell L, Sobell M: Convergent validity: an approach to increasing confidence in treatment outcome conclusions with alcohol and drug abusers; in Evaluating Alcohol and Drug Abuse Treatment Effectiveness: Recent Advances. Edited by Sobell L, Sobell M, Ward E. New York, Pergamon, 1980
Information & Authors
Information
Published In
History
Received: 29 June 2022
Revision received: 21 September 2022
Accepted: 8 November 2022
Published online: 15 February 2023
Published in print: August 01, 2023
Keywords
Authors
Competing Interests
The authors report no financial relationships with commercial interests.
Funding Information
Financial support for the Supported Employment Demonstration was provided by the Social Security Administration (SSA) under contract SS00-16-60014. ClinicalTrials.gov identifier: NCT03682263.Data were collected under contract SS00-16-60014 and are under the auspices of the SSA. The SSA reviewed and approved the use of these data for this article.
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBLogin options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).