Skip to main content

Abstract

Objective:

Among people with severe mental illnesses, neuropsychological abilities may contribute to vocational outcomes, such as job attainment, job tenure, and wages earned. The current study aimed to determine the strongest neuropsychological and other modifiable predictors of work outcomes in 153 people with severe mental illness (schizophrenia, 38%; bipolar disorder, 24%; and major depression, 38%) who participated in a 2-year supported employment study.

Methods:

Assessments of neuropsychological performance, functional capacity, social skills, and psychiatric symptom severity were administered at baseline; work outcomes (job attainment, weeks worked, and wages earned) were collected weekly for 2 years.

Results:

Independent of education, diagnosis, and estimated intellectual functioning, more recent work history and less severe negative symptoms significantly predicted job attainment during the 2-year study. Among the 47% who obtained jobs, better global neuropsychological performance (i.e., lower global deficit score) was a significant predictor of greater weeks worked. Both global neuropsychological performance and more recent work history predicted higher wages earned.

Conclusions:

Modifiable predictors of supported employment outcomes included cognitive functioning and negative symptom severity; thus, interventions to improve these factors may improve work outcomes and decrease the loss of productivity associated with severe mental illness.

HIGHLIGHTS

Understanding the relationship between neuropsychological abilities and various vocational outcomes provides a means for targeted cognitive training and remediation among people with severe mental illness.
The results showed that the varied cognitive impairments seen within severe mental illness provide additional predictive utility in explaining vocational outcomes and that more recent work history predicts better work outcomes, regardless of diagnosis.
Unemployment is common among individuals with severe mental illnesses and is associated with high economic costs, the largest being lost productivity (15). Evidence-based supported employment, also known as individual placement and support (IPS), is an evidence-based practice to assist people with severe mental illness in returning to work; multiple studies and meta-analyses have shown IPS to be more effective than conventional vocational rehabilitation in improving work outcomes such as job acquisition, job tenure, and wages (69). IPS principles include eligibility based on client choice (zero exclusion); attention to client preferences; competitive employment as the goal; rapid job search; integration of mental health treatment and supported employment; individualized, time-unlimited job support; systematic job development; and benefits counseling (10).
A growing body of research has addressed predictors of work outcomes in people with severe mental illness, both in general and among individuals receiving supported employment. Predictors of better work outcomes in general include higher education (11, 12), stronger or more recent work history (1, 1316), and absence of psychosis (17). The relationship between age and work outcomes remains uncertain (6, 7, 10, 17, 18). Similarly, the link between ethnicity and work outcome remains inconclusive; some studies have found an association between Hispanic ethnicity and better work outcomes (15, 18), whereas others have not (19).
Ascertaining individual predictors of work outcomes in the context of IPS may help providers in addressing modifiable client factors for IPS service users. A landmark meta-analysis of four large IPS trials found that the effects of IPS on work outcomes were robust even when accounting for most demographic, clinical, and employment characteristics (7). Another study of 2,055 Social Security Disability Insurance beneficiaries showed that work history was the strongest predictor of supported employment outcomes (15). These and other studies have concluded that IPS should be offered to anyone with severe mental illness who wants to work (9, 20). However, the search for modifiable predictors of work outcomes continues to improve the typical job acquisition rate in IPS programs (61%) (10).
Many studies have suggested that neurocognitive and functional abilities may be strong predictors of work outcome (11, 21, 22), but relatively few IPS studies have included comprehensive neuropsychological and functional assessments. Better performance on measures of verbal learning (13, 18), working memory (23), executive functioning (11, 13), and processing speed (16) have been linked to better vocational outcomes. Additionally, greater baseline functional capacity (14) and better social skills (12) have been associated with better work outcomes.
Much of the prior research on the relationship between neuropsychological ability and work outcomes has been limited by small sample size, short follow-up periods, reliance on samples of individuals with the same diagnosis or the same type of disability benefits, limited neurocognitive test batteries, or the inclusion of clients not enrolled in supported employment. As such, this study aimed to fill some of the gaps in the published literature by determining the strongest neuropsychological and other modifiable predictors of work outcomes in a large sample of IPS service users with varying diagnoses. Participants received IPS for up to 2 years and were assessed by using a comprehensive neuropsychological battery and performance-based measures of functional and social skills, in addition to standard clinical measures.

Methods

Participants

The study was registered as a clinical trial (NCT00895258) and data were collected from June 2008 to February 2014. Study procedures were approved by the University of California, San Diego, Institutional Review Board, and all participants provided written informed consent. One hundred fifty-three unemployed outpatients with severe mental illness (schizophrenia or schizoaffective disorder, N=58; bipolar disorder, N=37; and major depressive disorder, N=58) enrolled in the trial. Inclusion criteria were age 18 and older; literate and fluent in English; DSM-IV diagnosis of schizophrenia, schizoaffective disorder, bipolar disorder, or major depressive disorder confirmed via Structured Clinical Interview for DSM-IV (24) or Mini International Neuropsychiatric Interview (25); and being unemployed for at least 30 days and being interested in working. All participants received IPS for up to 2 years, based on their preference, and were randomly assigned to also receive either compensatory cognitive training (N=77) or additional supported employment sessions (enhanced supported employment; N=76) for the first 12 weeks of the trial (see Twamley et al. [26] for further details). Each group had its own employment specialist. Fidelity to supported employment was rated as “fair” during the study period. Because work outcomes associated with compensatory cognitive training and enhanced supported employment did not differ (26), the groups were collapsed for all analyses. Table 1 provides the demographic and clinical characteristics of the sample.
TABLE 1. Association between various characteristics and diagnosis among 153 participants in individual placement and support
 Total sample (N=153)Major depression (MD) (N=58)Bipolar disorder (BD) (N=37)Schizophrenia spectrum disorder (SS) (N=58)Test  Pairwise
CharacteristicN%N%N%N%statisticdfpcomparisons
Demographic and clinical            
 Age (M±SD)43.70±11.69 45.05±11.75 44.78±11.20 41.66±11.84 F=1.442, 150.240 
 Education (M±SD years)13.46±2.78 13.84±2.77 14.22±2.42 12.60±2.83 F=4.912, 150.009MD, BD>SS
 Female6643345915411729χ2=10.292.006MD>SS
 Race-ethnicity        χ2=22.056.001MD, BD>SS
  White12481529035953764    
  Black21144701729    
  Asian53121335    
  Native American32121312    
  Hispanic31201424381424χ2=4.462.107 
 Duration of illness (M±SD years)24.37±14.16 24.78±14.38 28.95±14.01 21.03±13.3 F=3.692, 150.027BD>SS
 Work history (months since last employment) (M±SD)35.27±53.46 35.28±63.70 20.84±21.68 44.63±55.40 F=2.262, 148.108 
Work outcome            
 Attained competitive work7247315323621831χ2=10.322.006MD, BD>SS
 Weeks worked in 2 years (M±SD)21.75±33.38 22.41±31.09 36.59±41.42 11.62±25.94 F=6.832, 150.001MD, BD>SS
 Wages earned in 2 years (M±SD$)7,640.76±15,005.65 7,880.94±12,209.82 14,460.25±22,882.26 3,050.22±8,304.99 F=7.062, 150.001MD, BD>SS
Symptom severity (M±SD score)            
 PANSS positivea12.53±5.03 11.33±3.51 11.86±5.08 14.16±5.86 F=5.302, 150.006SS>MD
 PANSS negativea13.20±5.00 12.07±3.95 11.89±4.45 15.16±5.67 F=7.842, 150.001SS>MD, BD
 HAM-Db12.94±6.82 15.16±6.82 13.14±7.07 10.52±5.89 F=7.152, 148.001MD>SS
Neuropsychological functioningc            
 Premorbid IQ103.00±9.66 104.62±8.36 106.51±7.97 99.14±10.62 F=8.702, 150<.001MD, BD>SS
 GDSd.64±.67 .45±.53 .50±.65 .91±.73 F=8.422, 150<.001SS>MD, BD
 Processing speed            
  TMT-A42.31±11.33 44.91±9.80 42.32±10.76 39.71±12.61 F=3.152, 150.046MD>SS
  BACS-SC38.98±10.94 41.55±10.61 42.41±10.32 34.22±10.11 F=9.942, 150<.001MD, BD>SS
  Category fluency44.81±9.30 46.67±9.61 46.81±8.57 41.67±8.71 F=5.642, 150.004MD, BD>SS
 Sustained attention            
  CPT-IP42.79±12.80 46.11±10.77 45.78±12.34 37.78±13.41 F=7.972, 146.001MD, BD>SS
 Working memory            
  WMS-III SS46.75±9.57 47.41±9.64 47.30±10.06 45.72±9.26 F=.532, 150.590 
  LNS44.37±10.59 47.10±9.13 48.03±10.49 39.29±10.19 F=12.432, 150<.001MD, BD>SS
 Verbal learning            
  HVLT-R46.51±10.85 48.69±10.55 49.24±10.42 42.59±10.42 F=6.602, 150.002MD, BD>SS
 Visual learning            
  BVMT-R44.44±10.91 44.81±11.74 47.51±10.47 42.10±9.93 F=2.902, 150.058 
 Executive function            
  NAB Mazes44.56±10.16 45.31±10.52 43.78±9.15 44.31±10.52 F=.282, 150.755 
  TMT-B44.05±11.38 44.52±10.67 43.86±10.63 43.70±12.66 F=.082, 148.924 
  WCST–6444.24±10.84 46.79±10.83 44.46±10.84 41.55±10.39 F=3.512, 150.032MD>SS
 Prospective memory            
  MIST (percentile)44.90±31.88 49.84±32.57 50.86±32.32 36.16±29.38 F=3.652, 150.028MD, BD>SS
 Letter fluency            
  FAS46.41±9.98 49.38±8.53 46.43±10.34 43.41±10.34 F=5.492, 150.005MD>SS
Functional capacity            
 UPSA-Be78.43±10.73 82.49±7.74 83.24±8.30 71.31±10.96 F=27.962, 150<.001MD, BD>SS
 SSPAf4.18±.65 4.38±.58 4.27±.44 3.91±.76 F=8.722, 146<.001MD, BD>SS
a
PANSS, Positive and Negative Syndrome Scale. Possible scores range from 7 to 49, with higher scores indicating greater symptom severity.
b
HAM-D, Hamilton Depression Rating Scale. Possible scores range from 0 to 52, with higher scores indicating greater depression severity.
c
The following measures of neuropsychological functioning are expressed as t test scores (M±SD=50±10, range 20–80), with the exception of premorbid IQ (Wide Range Achievement Test-III–Reading subtest), which is expressed as a standard score (M±SD=100±15): TMT-A, Trail Making Test, Part A; BACS-SC, Brief Assessment of Cognition in Schizophrenia, Symbol-Coding; category fluency; CPT-IP, Continuous Performance Test–Identical Pairs; WMS-III SS, Wechsler Memory Scale-III Spatial Span; LNS, Letter-Number Span; HVLT-R, Hopkins Verbal Learning Test–Revised; BVMT-R, Brief Visual Memory Test–Revised; NAB, Neuropsychological Assessment Battery, Mazes; TMT-B, Trail Making Test, Part B; WCST–64, Wisconsin Card Sorting Test–64-card version; letter fluency; MIST, Memory for Intentions Screening Test. Higher scores signify better performance.
d
GDS, Global Deficit Score. Scores range from 0 to 5 (0, no impairment; 1, mild; 2, mild to moderate; 3, moderate; 4, moderate to severe; and 5, severe).
e
UPSA-B, University of California, San Diego, Performance-Based Skills Assessment–Brief. Possible scores range from 0 to 100, with higher scores indicating better functioning.
f
SSPA, Social Skills Performance Assessment. Possible scores range from 1 to 5, with higher scores indicating better functioning.
Baseline neuropsychological, clinical, and functional assessments were used in the current analyses. Data from these participants have been used in prior publications (2630); however, the analyses presented in this article have not been published previously.

Measures

The following assessments were administered at baseline, prior to randomization. All raters were trained to a high degree of interrater reliability (intraclass correlation coefficient ≥.90).

Neuropsychological Measures

Premorbid intellectual ability was estimated with the reading subtest of the Wide Range Achievement Test-III (31). The cognitive subtests of the Measurement and Treatment Research to Improve Cognition in Schizophrenia Consensus Cognitive Battery (32) assessed neuropsychological functioning in the domains of processing speed (Trail Making Test, Part A [TMT-A], Brief Assessment of Cognition in Schizophrenia Symbol-Coding [BACS-SC], and category fluency), sustained attention (Continuous Performance Test–Identical Pairs), working memory (Wechsler Memory Scale-III Spatial Span and University of Maryland Letter-Number Span [LNS]), verbal learning (Hopkins Verbal Learning Test–Revised [HVLT-R]), visual learning (Brief Visual Memory Test–Revised [BVMT-R]), and executive functioning (Neuropsychological Assessment Battery [NAB] Mazes). All t test scores were corrected for age and education. Additional tests of executive functioning measured set shifting (Trail Making Test, Part B [TMT-B] [33]), letter fluency using the letters F, A, and S (FAS) (33), and reasoning and set shifting (Wisconsin Card Sorting Test–64-card version [WCST-64] [34] total errors). Additionally, prospective memory ability was measured by using the Memory for Intentions Screening Test (35). We calculated a global deficit score (GDS) by transforming individual neuropsychological test scores (i.e., t test scores) to deficit scores ranging from 0, no impairment, to 5, severe impairment, which were then averaged across all tests (36).

Functional Skills and Symptom Severity

The University of California San Diego Performance-Based Skills Assessment–Brief (37) assesses performance-based functional capacity in the domains of financial management and communication. The Social Skills Performance Assessment (38) measures social skills relevant to neutral and adversarial situations. Positive and negative symptom severity and general psychopathology were measured with the Positive and Negative Syndrome Scale (39), and depressive symptom severity was measured by using the Hamilton Depression Rating Scale (40).

Work Outcomes

Work outcomes (competitive job attainment, total weeks of competitive employment, and wages earned) were gathered weekly during the 2-year study by the employment specialist (if engaged in weekly contact with the participant) or a research assistant; work participation and earnings were corroborated with paystubs. Competitive work was defined as employment paying at least minimum wage and not set aside for a person with a disability. Participants who dropped out of the study prior to obtaining a job were assumed not to have worked.

Statistical Analyses

One-way analysis of variance and chi-square tests were conducted to examine differences in characteristics between diagnostic groups. Prior to analyses, model assumptions were checked, including screening for outliers and evaluating for multicollinearity, with tolerance values of <.40 and variance inflation factor values of >2.5 suggestive of multicollinearity (41). Job attainment during the 2-year study period (0=no; 1=yes) was analyzed by using a logistic regression model. Examination of variable distributions for competitive weeks worked and wages earned showed positively skewed distributions, with excess zeroes. Thus these variables were log-transformed before being included in all analyses and modeled by using a zero-altered count regression approach, known as a hurdle model. Hurdle models are two-part models, in which all the zeroes are modeled with a probit regression, and nonzero counts are modeled by a truncated count regression (i.e., truncated because it does not include zero) (42). Our hurdle models reflected the two-stage process resulting in the observed distributions of competitive weeks worked and wages earned. That is, participants first had to attain a job (i.e., pass the “hurdle”) to report weeks of employment and wages earned. Thus the probit regression component of the hurdle model examined the predictors of job attainment, whereas the count regression in the hurdle models examined weeks worked and wages earned for those who attained a job (N=72). Analyses were conducted by using SPSS. version 24.0, except for analyses of hurdle models, which were conducted by using STATA/IC, version 15.0.
Bivariate Pearson and point-biserial correlations between individual tests of neuropsychological functioning (i.e., neuropsychological measures, including premorbid intellectual ability), GDS, psychiatric symptom severity, performance-based functional capacity and social skills, demographic variables, and work outcomes, were conducted. Bivariate-significant correlates of job attainment (p<0.05) were entered as predictors of job attainment in the logistic regression model as well as the probit component of the hurdle models. Similarly, for the subset of participants who attained a job, bivariate-significant variables were entered as predictors of weeks of competitive work and wages earned for the count regression models. (Although work history was not significantly associated with weeks worked, it was entered in the model because of computational requirements of hurdle model, i.e., algorithm limitations. Including work history did not affect model estimates. Our model excluding work history converged using an alternative linear maximum likelihood model [i.e., zero-inflated negative binomial model], and had similar results, thereby bolstering the robustness of the hurdle models.) There were significant demographic differences by diagnostic group in years of education and premorbid IQ estimate (p<0.05), which were controlled for in subsequent analyses.

Results

Demographic characteristics and outcomes for the 153 participants are summarized in Table 1. Table 2 summarizes characteristics of the 72 participants who attained work during the 2-year study. Bivariate correlations (N=153) determined significant associations between several participant characteristics and job attainment, including education, racial-ethnic minority status, work history, diagnosis, and psychiatric symptom severity (Table 3). Furthermore, better functional capacity, greater estimated intellectual functioning, and better performance on BACS-SC and LNS were associated with job attainment. As such, these variables were entered in the logistic regression model and the probit regression component of the hurdle models.
TABLE 2. Association between various characteristics and diagnosis among 72 participants in individual placement and support who attained work during the 2-year study period
 Total sample (N=153)Major depression (MD) (N=58)Bipolar disorder (BD) (N=37)Schizophrenia spectrum disorder (SS) (N=58)Test  Pairwise
CharacteristicN%N%N%N%statisticdfpcomparisons
Demographic and clinical            
 Age (M±SD)42.50±11.48 44.81±11.69 42.57±11.94 38.44±9.91 F=1.792, 69.175 
 Education (M±SD years)13.99±2.62 14.16±3.01 14.48±1.86 13.06±2.65 F=1.632, 69.203 
 Female33461858939633χ2=3.422.181 
 Race-ethnicity            
  White6388289121921478χ2=3.676.721 
  Black341300211    
  Asian34131415.5    
  Native American34131415.5    
  Hispanic1217619313317χ2=.382.828 
 Duration of illness (M±SD years)24.60±13.74 25.84±13.59 27.17±13.96 19.17±12.94 F=1.992, 69.144 
Work history (months since last employment)16.18±18.73 12.50±13.43 22.2±21.72 14.56±21.20 F=1.912, 68.156 
Work outcome            
 Weeks worked in 2 years (M±SD)46.22±35.19 41.94±31.48 58.87±37.90 37.44±35.11 F=2.362, 69.102 
 Wages earned in 2 years (M±SD $)16,236.62±18,453.18 14,744.98±13,366.67 23,262.15±25,353.17 9,828.49±12,677.64 F=3.022, 69.056 
Symptom severity            
 PANSS positivea11.61±4.53 11.68±3.81 10.83±4.13 12.50±6.04 F=.6882, 69.506 
 PANSS negativea12.06±4.49 11.74±3.89 11.17±4.34 13.72±5.39 F=1.802, 69.174 
 HAM-Db12.68±6.57 14.81±6.13 11.70±7.11 10.12±5.63 F=3.392, 68.040MD>SS
Neuropsychological functioningc            
 Premorbid IQ104.99±9.39 107.29±6.95 106.74±7.62 98.78±12.38 F=6.012, 69.004MD, BD>SS
 GDSd.54±.58 .46±.61 .39±.48 .87±.54 F=4.292, 69.017SS>MD, BD
 Processing speed            
  TMT-A42.42±11.05 43.48±9.00 43.74±10.35 38.89±14.52 F=1.242, 69.297 
  BACS-SC41.15±10.62 42.29±10.17 45.13±10.05 34.11±9.02 F=6.672, 69.002MD, BD>SS
  Category fluency45.06±10.06 46.45±10.73 46.74±9.33 40.50±8.79 F=2.582, 69.083 
 Sustained attention            
  CPT-IP44.66±11.89 45.30±12.26 47.95±8.98 39.56±13.24 F=2.672, 67.077 
 Working memory            
  WMS-III SS47.40±8.75 47.48±9.33 48.87±8.13 45.39±8.58 F=.7962, 69.455 
  LNS46.93±9.83 47.23±9.88 48.48±8.88 44.44±10.90 F=.8722, 69.423 
 Verbal learning            
  HVLT-R47.57±10.83 48.32±11.53 49.48±9.15 43.83±11.24 F=1.532, 69.225 
 Visual learning            
  BVMT-R45.32±10.77 45.19±11.70 49.04±8.34 40.78±10.58 F=3.162, 69.049BD>SS
 Executive function            
  NAB Mazes45.89±9.67 45.77±10.73 46.52±8.81 45.28±9.26 F=.092, 69.918 
  TMT-B43.43±10.20 43.74±10.79 43.65±10.23 42.61±9.62 F=.082, 69.927 
  WCST–6445.50±10.83 46.68±12.43 44.96±10.69 44.17±8.02 F=2.952, 69.059 
 Prospective memory            
  MIST (percentile)49.17±32.90 55.39±33.07 48.04±32.08 39.89±33.11 F=1.292, 69.281 
 Letter fluency            
  FAS45.94±10.21 49.10±9.88 45.65±9.98 40.89±9.42 F=4.012, 69.022MD>SS
Functional capacity            
 UPSA-Be80.77±9.32 82.52±8.42 83.49±7.38 74.30±10.34 F=6.832, 69.002MD, BD>SS
 SSPAf4.29±.58 4.40±.58 4.32±.42 4.04±.70 F=2.262, 68.112 
a
PANSS, Positive and Negative Syndrome Scale. Possible scores range from 7 to 49, with higher scores indicating greater symptom severity.
b
HAM-D, Hamilton Depression Rating Scale. Possible scores range from 0 to 52, with higher scores indicating greater depression severity.
c
The following measures of neuropsychological functioning are expressed as t test scores (M±SD=50±10, range 20–80), with the exception of premorbid IQ (Wide Range Achievement Test-III–Reading subtest), which is expressed as a standard score (M±SD=100±15): TMT-A, Trail Making Test, Part A; BACS-SC, Brief Assessment of Cognition in Schizophrenia, Symbol-Coding; category fluency; CPT-IP, Continuous Performance Test–Identical Pairs; WMS-III SS, Wechsler Memory Scale-III Spatial Span; LNS, Letter-Number Span; HVLT-R, Hopkins Verbal Learning Test–Revised; BVMT-R, Brief Visual Memory Test–Revised; NAB, Neuropsychological Assessment Battery, Mazes; TMT-B, Trail Making Test, Part B; WCST–64, Wisconsin Card Sorting Test–64-card version; Letter fluency; MIST, Memory for Intentions Screening Test. Higher scores signify better performance.
d
GDS, Global Deficit Score. Scores range from 0 to 5 (0, no impairment; 1, mild; 2, mild to moderate; 3, moderate; 4, moderate to severe; and 5, severe).
e
UPSA-B, University of California, San Diego, Performance-Based Skills Assessment–Brief. Possible scores range from 0 to 100, with higher scores indicating better functioning.
f
SSPA, Social Skills Performance Assessment. Possible scores range from 1 to 5, with higher scores indicating better functioning.;
TABLE 3. Correlations between work outcomes and characteristics of 153 participants in individual placement and supporta
CharacteristicJob attainmentCompetitive weeksCompetitive wages
Demographic and clinical   
 Age–.097–.066.016
 Education (years).178*–.100–.063
Racial-ethnic minority status–.197*.017.013
 Gender–.051–.192–.100
 Illness duration (years).015.012.013
 Work history (months since last employment–.337**–.215–.310**
 Diagnosis–.196*–.084–.142
Symptom severity   
 PANSS positive–.173*–.259*–.275*
 PANSS negative–.216**–.139–.193
 HAM-D–.037–.039–.011
Neuropsychological functioning   
 Premorbid IQ.195*–.049.046
 GDS–.134–.306**–.356**
 TMT-A.009.171.160
 BACS-SC.188*.228.159
 Category Fluency.025.105.017
 CPT-IP.138.042.174
 WMS-III SS.065.139.103
 LNS.229**–.078–.027
 HVLT-R.092.136.191
 BVMT-R.076.188.216
 NAB Mazes.124–.009–.008
 TMT-B–.052.226.221
 WCST–64.151.071.074
 MIST.127.181.178
 FAS–.044.044.172
Functional capacity   
 UPSA-B.206**.042.062
 SSPA.160.173.194
a
Weeks of competitive work and competitive wages earned were assessed for 72 participants who found competitive employment. Abbreviations: BACS-SC, Brief Assessment of Cognition in Schizophrenia, Symbol-Coding; BVMT-R, Brief Visual Memory Test–Revised; CPT-IP, Continuous Performance Test–Identical Pairs; FAS, letter fluency test using the letters F, A, and S; GDS, Global Deficit Score; HAM-D, Hamilton Depression Rating Scale; HVLT-R, Hopkins Verbal Learning Test–Revised; LNS, Letter Number Span; MIST, Memory for Intentions Screening Test; NAB, Neuropsychological Assessment Battery; PANSS, Positive and Negative Syndrome Scale; SSPA, Social Skills Performance Assessment; TMT-A, Trail Making Test, Part A; TMT-B, Trail Making Test, Part B; UPSA-B, University of California, San Diego, Performance-Based Skills Assessment–Brief; WMS-III SS, Wechsler Memory Scale–III Spatial Span; WCST–64, Wisconsin Card Sorting Test–64 card version. Racial-ethnic minority status coded as 0=no and 1=yes. Diagnosis coded as 1=major depression; 2=bipolar disorder; 3=schizophrenia spectrum disorders.
*
p<.05, **p<.01.
Forward entry likelihood ratio (LR) stepwise analysis found work history and negative symptom severity to be significant predictors of job attainment during the 2-year study period. Jointly, these variables improved model fit by 26% (χ2=32.92, N=151, df=2, p<0.001, Nagelkerke R2=0.26), with less severe negative symptoms (odds ratio [OR]=.910, Wald z=5.90, df=1, p=0.015, 95% confidence interval [CI]=0.843–0.982) and more recent work history (OR=.971, Wald z=13.25, df=1, p<0.001, 95% CI=0.956–0.987) associated with increased odds of obtaining employment.
For competitive weeks worked, variables significantly associated with job attainment were entered into the probit regression component of the hurdle model. Simultaneously, variables significantly associated with competitive weeks worked (i.e., positive symptom severity and GDS; Table 3) for the subset of participants who attained a job (N=72) were entered in the count regression component of the hurdle model, along with work history. (Note that the probit and count regression components of the hurdle model are separate, so including GDS in the count regression and individual test scores in the probit regression, respectively, did not introduce multicollinearity.) The hurdle model found more recent work history to be a significant predictor of job attainment, whereas lower levels of neuropsychological impairment (GDS) emerged as a significant predictor of greater competitive weeks worked (p<0.05).
Backward selection elimination of predictor variables with p>0.05 in the hurdle models was performed to determine the robustness of the findings and determine additional significant predictors that may not have been detected because of redundancies. This resulted in a more parsimonious final model, reported here, with additional significant predictors for job attainment. Specifically, consistent with the logistic regression model reported above, lower negative symptom severity emerged as an additional significant predictor of job attainment along with more recent work history (Table 4). GDS remained the only significant predictor of weeks of competitive work, with the overall hurdle model improving model fit by 14% (LR χ2=42.33, N=151, df=2, p<0.001, pseudo-R2=0.14). Postestimation analyses determined that higher GDS was related to fewer weeks worked; a participant with a GDS of 0 worked 33.67 weeks on average during the 2-year study duration, whereas a participant with a GDS of 2.5, indicating moderate impairment, worked an average of 9.23 weeks throughout the study duration, approximately four times less.
TABLE 4. Significant predictors of work outcomes among 151 participants in individual placement and supporta
PredictorCoefficientSEzdfpOR95% CI
Logistic regression       
 Job attainmentb       
  PANSS negativec–.094.0395.901.015.91.84, .98
  Work history (months since last employment)–.029.00813.251<.001.97.96, .99
Hurdle model       
 Job attainment (probit regression)       
  PANSS negativec–.058.023–2.461.014–.10, –.01
  Work history (months since last employment)–.018.005–3.781<.001–.03, –.01
 Weeks of competitive work       
  Work history (months since last employment)–.004.003–1.491.135–.01, .001
  Global Deficit Score–.222.090–2.481.013–.40, –.05
 Wages earned       
  Work history (months since last employment)–.009.004–2.381.017–.02, –.002
  Global Deficit Score–.371.122–3.041.002–.61, –.13
a
Data for two participants were not included in the logistic regression and hurdle models.
b
Job attainment was coded as 0=no job obtained, 1=job obtained.
c
PANSS, Positive and Negative Syndrome Scale.
For wages earned, the results of the probit regression model were identical to the hurdle model reported above. For the count regression, variables significantly associated with wages earned (i.e., work history, positive symptom severity, and GDS; Table 3) for the subset of participants who obtained a job were entered into this component of the hurdle model. GDS emerged as a significant predictor of wages earned (p<0.05). Backward selection elimination identified work history as an additional significant predictor of wages earned, with the overall hurdle model improving model fit by 14% (LR χ2=48.89, N=151, df=2, p<0.001, pseudo-R2=0.14; Table 4). Further examination of these associations revealed that less neuropsychological impairment (GDS) was related to higher wages; on average, a participant with a GDS of 0 earned twice as much as a participant with a GDS of 1 ($7,942.28 vs. $3,387.44). Furthermore, more recent work history was related to higher wages; a participant who was unemployed for 1 month at study entry earned, on average, $9,331.54 over the 2-year study compared with $3,234.94 for participants who were unemployed for 1 year.
Additional exploratory analyses were conducted to examine associations between cognitive functioning, psychiatric symptom severity, and work history. Better working memory (LNS), executive functioning (NAB Mazes, WCST-64), and visual learning (BVMT-R) and less severe positive symptoms were associated with more recent work history (p<0.05 for all). There were no differences between the psychiatric diagnostic groups on work history.

Discussion

This study examined the strongest neuropsychological and other modifiable predictors of vocational outcomes in a sample of individuals with severe mental illness. Considering the transdiagnostic presentation of cognitive impairments in psychiatric disorders and poor associated work outcomes, understanding the differential relationship of neuropsychological abilities with vocational outcomes may reveal targets for cognitive training or remediation among people with severe mental illness.
Indeed, we found several neuropsychological abilities that were associated with work outcomes at the bivariate level, including processing speed (BACS-SC), working memory (LNS), premorbid IQ (Wide Range Achievement Test-III), and global neuropsychological performance (GDS). However, our hurdle models showed that only overall neuropsychological ability (GDS) remained a significant predictor of weeks worked and wages earned. Significant predictors of job attainment included more recent work history and less severe negative symptoms. Among those who obtained jobs, better global neuropsychological functioning (i.e., lower GDS) predicted greater weeks worked, above and beyond demographic and clinical characteristics and work history. After the analyses were controlled for demographic and clinical characteristics, better global neuropsychological functioning also predicted greater wages earned over the 2 years, along with more recent work history. These findings are consistent with previous studies demonstrating cognitive functioning as a significant predictor of work outcomes even after controlling for work history, itself a robust predictor of work outcomes (18). In the general population, cognitive ability is a strong predictor of work outcomes, and this association is partly mediated by the fact that better cognitive ability predicts better learning and job knowledge (43, 44). Our results suggest that, regardless of diagnosis, the cognitive impairments seen within severe mental illness uniquely predict vocational outcomes.
Consistent with previous findings (15), diagnosis did not emerge as a significant predictor of work outcomes. Our results highlight the importance of negative symptoms in predicting employment outcomes, a significance which may be explained through transdiagnostic models of negative symptom phenomenology ascribing a stronger role to these clinical symptoms compared with diagnosis (45). These findings underscore the importance of the independent examination of clinical phenotypes as discrete from diagnostic entities (46).
The findings may hold practical significance for mental health clinicians and employment specialists who work with clients with severe mental illness; targeting negative symptoms and providing cognitive training or remediation may improve work outcomes in supported employment service users. Although cognitive training programs are associated with improvements in varied cognitive domains (26, 47, 48) as well as negative symptoms (26, 49, 50), further research is needed to investigate the efficacy of such programs in improving work outcomes via cognition.
Given the significance of work history in predicting work outcomes and given that better cognitive performance was associated with more recent work, the inclusion of work history in our models may have served as a proxy for cognitive functioning. Thus, inclusion of individual cognitive tests as predictors may have had an insignificant effect on improving model fit when added along with work history. Despite these intercorrelations, our findings highlight the significance of overall cognitive performance for work success.
There were limitations to the current study that must be acknowledged. Given that our participants were community-dwelling, unemployed individuals with severe mental illness who received supported employment, the sample lacked a control group that did not receive supported employment and the results may be limited in generalizability to other samples. Also, the use of the Positive and Negative Syndrome Scale to assess negative symptoms precluded the examination of the role of primary versus secondary negative symptoms (e.g., negative symptoms secondary to depression) in predicting work outcomes. Future research should incorporate these distinctions in their investigations. The Measurement and Treatment Research to Improve Cognition in Schizophrenia Consensus Cognitive Battery includes limited assessment of attention, verbal learning, and visual learning (one test per domain); future studies should consider using additional measures of these constructs. Additionally, differential psychometric properties, such as sensitivity to deficits, may explain the lack of significance across all measures for domains assessed through multiple tests (51).

Conclusions

This study generated evidence for a transdiagnostic consideration of the role of neurocognitive deficits and negative symptom severity in predicting work outcomes in supported employment service users. The results suggest that improving negative symptom severity and cognitive performance may improve supported employment outcomes in people with severe mental illness

Acknowledgments

The authors gratefully acknowledge the contributions of the participants in this study.

Footnote

The funding source had no other role in the work described in this article.

References

1.
Marwaha S, Johnson S: Schizophrenia and employment: a review. Soc Psychiatry Psychiatr Epidemiol 2004; 39:337–349. doi
2.
Waghorn G, Saha S, Harvey C, et al: “Earning and learning” in those with psychotic disorders: the second Australian national survey of psychosis. Aust N Z J Psychiatry 2012; 46:774–785. doi
3.
Miller S, Dell’Osso B, Ketter TA: The prevalence and burden of bipolar depression. J Affect Disord 2014; 169(suppl 1):S3–S11. doi
4.
Greenberg PE, Fournier A-A, Sisitsky T, et al: The economic burden of adults with major depressive disorder in the United States (2005 and 2010). J Clin Psychiatry 2015; 76:155–162. doi
5.
Cloutier M, Aigbogun MS, Guerin A, et al: The economic burden of schizophrenia in the United States in 2013. J Clin Psychiatry 2016; 77:764–771. doi
6.
Twamley EW, Jeste DV, Lehman AF: Vocational rehabilitation in schizophrenia and other psychotic disorders: a literature review and meta-analysis of randomized controlled trials. J Nerv Ment Dis 2003; 191:515–523. doi
7.
Campbell K, Bond GR, Drake RE: Who benefits from supported employment: a meta-analytic study. Schizophr Bull 2011; 37:370–380. doi
8.
Modini M, Tan L, Brinchmann B, et al: Supported employment for people with severe mental illness: systematic review and meta-analysis of the international evidence. Br J Psychiatry 2016; 209:14–22. doi
9.
Metcalfe JD, Drake RE, Bond GR: Economic, labor, and regulatory moderators of the effect of individual placement and support among people with severe mental illness: a systematic review and meta-analysis. Schizophr Bull 2018; 44:22–31. doi
10.
Bond GR, Drake RE, Becker DR: An update on randomized controlled trials of evidence-based supported employment. Psychiatr Rehabil J 2008; 31:280–290. doi
11.
McGurk SR, Meltzer HY: The role of cognition in vocational functioning in schizophrenia. Schizophr Res 2000; 45:175–184. doi
12.
Tsang HWH, Leung AY, Chung RCK, et al: Review on vocational predictors: a systematic review of predictors of vocational outcomes among individuals with schizophrenia: an update since 1998. Aust N Z J Psychiatry 2010; 44:495–504. doi
13.
McGurk SR, Mueser KT, Harvey PD, et al: Cognitive and symptom predictors of work outcomes for clients with schizophrenia in supported employment. Psychiatr Serv 2003; 54:1129–1135. doi
14.
Twamley EW, Vella L, Burton CZ, et al: The efficacy of supported employment for middle-aged and older people with schizophrenia. Schizophr Res 2012; 135:100–104. doi
15.
Metcalfe JD, Drake RE, Bond GR: Predicting employment in the mental health treatment study: do client factors matter? Adm Policy Ment Health Ment Health Serv Res 2017; 44:345–353. doi
16.
Corbière M, Lecomte T, Reinharz D, et al: Predictors of acquisition of competitive employment for people enrolled in supported employment programs. J Nerv Ment Dis 2017; 205:275–282. doi
17.
Wewiorski NJ, Fabian ES: Association between demographic and diagnostic factors and employment outcomes for people with psychiatric disabilities: a synthesis of recent research. Ment Health Serv Res 2004; 6:9–21. doi
18.
McGurk SR, Drake RE, Xie H, et al: Cognitive predictors of work among Social Security Disability Insurance beneficiaries with psychiatric disorders enrolled in IPS supported employment. Schizophr Bull 2018; 44:32–37. doi
19.
Mueser KT, Bond GR, Essock SM, et al: The effects of supported employment in Latino consumers with severe mental illness. Psychiatr Rehabil J 2014; 37:113–122. doi
20.
Becker DR, Drake RE. A Working Life for People With Severe Mental Illness. Oxford, United Kingdom, Oxford University Press, 2003. doi.
21.
Ikebuchi E, Sato S, Yamaguchi S, et al: Does improvement of cognitive functioning by cognitive remediation therapy effect work outcomes in severe mental illness? A secondary analysis of a randomized controlled trial. Psychiatry Clin Neurosci 2017; 71:301–308. doi
22.
Metcalfe JD, Riley J, McGurk S, et al: Comparing predictors of employment in individual placement and support: a longitudinal analysis. Psychiatry Res 2018; 264:85–90. doi
23.
Joseph J, Kremen WS, Franz CE, et al: Predictors of current functioning and functional decline in schizophrenia. Schizophr Res 2017; 188:158–164. doi
24.
First M, Spitzer R, Gibbon M, et al: Structured Clinical Interview for DSM-IV-TR Axis I Disorders, Research Version, Patient Edition (SCID-I/P). New York, New York State Psychiatric Institute, 2002
25.
Sheehan DV, Lecrubier Y, Sheehan KH, et al: The validity of the Mini International Neuropsychiatric Interview (MINI) according to the SCID-P and its reliability. Eur Psychiatry 1997; 12:232–241. doi
26.
Twamley EW, Thomas KR, Burton CZ, et al: Compensatory cognitive training for people with severe mental illnesses in supported employment: a randomized controlled trial. Schizophr Res 2019; 204:397–400
27.
Burton CZ, Vella L, Twamley EW: Prospective memory, level of disability, and return to work in severe mental illness. Clin Neuropsychol 2018;4046:1–12
28.
Thomas KR, Olga P, Twamley EW: Age as a moderator of change following compensatory cognitive training in individuals with severe mental illnesses. Psychiatr Rehabil J 2017; 40:70–78
29.
Mahmood Z, Burton CZ, Vella L, et al: Neuropsychological predictors of performance-based measures of functional capacity and social skills in individuals with severe mental illness. J Psychiatr Res 2018;102:201–206
30.
Puig O, Thomas KR, Twamley EW: Age and improved attention predict work attainment in combined compensatory cognitive training and supported employment for people with severe mental illness. J Nerv Ment Dis 2016; 204:869–872
31.
Wilkinson G: WRAT-3: Wide Range Achievement Test Manual. San Antonio, TX, Psychological Corporation, 1993
32.
Nuechterlein KH, Green MF, Kern RS, et al: The MATRICS Consensus Cognitive Battery, part 1. test selection, reliability, and validity. Am J Psychiatry 2008; 165:203–213. doi
33.
Heaton R, Miller S, Taylor M, et al: Revised Comprehensive Norms for an Expanded Halstead- Reitan Battery: Demographically Adjusted Neuropsychological Norms for African American and Caucasian Adults. Lutz, FL, Psychological Assessment Resources, 2004
34.
Kongs S, Thompson L, Iverson G, et al: Wisconsin Card Sorting Test-64 Card Version (WCST-64). Odessa, FL, Psychological Assessment Resources, 2000
35.
Raskin S: Memory for Intentions Screening Test: psychometric properties and clinical evidence. Brain Impair 2009; 10:23–33. doi
36.
Blackstone K, Moore DJ, Franklin DR, et al: Defining neurocognitive impairment in HIV: deficit scores versus clinical ratings. Clin Neuropsychol 2012; 26:894–908. doi
37.
Mausbach BT, Harvey PD, Goldman SR, et al: Development of a brief scale of everyday functioning in persons with serious mental illness. Schizophr Bull 2007; 33:1364–1372. doi
38.
Patterson TL, Moscona S, McKibbin CL, et al: Social skills performance assessment among older patients with schizophrenia. Schizophr Res 2001; 48:351–360. doi
39.
Kay SR, Fiszbein A, Opler LA: The Positive and Negative Syndrome Scale (PANSS) for schizophrenia. Schizophr Bull 1987; 13:261–276. doi
40.
Hamilton M: Development of a rating scale for primary depressive illness. Br J Soc Clin Psychol 1967; 6:278–296
41.
Allison P: When Can You Safely Ignore Multicollinearity? [blog post]. 2012. https://statisticalhorizons.com/multicollinearity
42.
Atkins DC, Baldwin SA, Zheng C, et al: A tutorial on count regression and zero-altered count models for longitudinal substance use data. Psychol Addict Behav 2013; 27:166–177. doi
43.
Hunter JE: Cognitive ability, cognitive aptitudes, job knowledge, and job performance. J Vocat Behav 1986; 29:340–362. doi
44.
Morris TW, Levinson EM: Relationship between intelligence and occupational adjustment and functioning: a literature review. J Couns Dev 1995; 73:503–514. doi
45.
Strauss GP, Cohen AS: A transdiagnostic review of negative symptom phenomenology and etiology. Schizophr Bull 2017; 43:712–719. doi
46.
Insel T, Cuthbert B, Garvey M, et al: Research domain criteria (RDoC): toward a new classification framework for research on mental disorders. Am J Psychiatry 2010; 167:748–751. doi
47.
Wykes T, Huddy V, Cellard C, et al: A meta-analysis of cognitive remediation for schizophrenia: methodology and effect sizes. Am J Psychiatry 2011; 168:472–485. doi
48.
Vella L, Patterson TL, Harvey PD, et al: Exploratory analysis of normative performance on the UCSD Performance-Based Skills Assessment-Brief. Psychiatry Res 2017; 256:150–155. doi
49.
Ventura J, Subotnik KL, Gretchen-Doorly D, et al: Cognitive remediation can improve negative symptoms and social functioning in first-episode schizophrenia: a randomized controlled trial. Schizophr Res 2019; 203: 24–31 doi
50.
Mahmood Z, Clark JMR, Twamley EW: Compensatory cognitive training for psychosis: effects on negative symptom subdomains. Schizophr Res 2018; 204:397–400. doi
51.
Gold JM, Dickinson D: “Generalized cognitive deficit” in schizophrenia: overused or underappreciated? Schizophr Bull 2013; 39:263–265. doi

Information & Authors

Information

Published In

Go to Psychiatric Services
Go to Psychiatric Services
Psychiatric Services
Pages: 782 - 792
PubMed: 31185856

History

Received: 10 December 2018
Revision received: 9 March 2019
Accepted: 11 April 2019
Published online: 12 June 2019
Published in print: September 01, 2019

Keywords

  1. schizophrenia
  2. bipolar disorder
  3. major depressive disorder
  4. cognition
  5. work
  6. vocational outcomes
  7. unemployment
  8. cognition

Authors

Details

Zanjbeel Mahmood, B.A.
San Diego State University and University of California, San Diego (SDSU/UC San Diego), Joint Doctoral Program in Clinical Psychology, San Diego (Mahmood); Research Service (Mahmood, Keller) and Center of Excellence for Stress and Mental Health (Twamley), U.S. Department of Veterans Affairs San Diego Healthcare System, San Diego; Department of Psychiatry, University of California, San Diego (Twamley); Department of Psychiatry, University of Michigan, Ann Arbor (Burton), Department of Quality, University of California, San Francisco Health, San Francisco (Vella); Department of Psychology, San Diego State University, San Diego (Matt); Department of Occupational Therapy and Psychological and Brain Sciences, Center for Psychiatric Rehabilitation, Boston University, Boston (McGurk). Drs. Burton and Vella were with the SDSU/UC San Diego Joint Doctoral Program in Clinical Psychology at the time of this research.
Amber V. Keller, B.A.
San Diego State University and University of California, San Diego (SDSU/UC San Diego), Joint Doctoral Program in Clinical Psychology, San Diego (Mahmood); Research Service (Mahmood, Keller) and Center of Excellence for Stress and Mental Health (Twamley), U.S. Department of Veterans Affairs San Diego Healthcare System, San Diego; Department of Psychiatry, University of California, San Diego (Twamley); Department of Psychiatry, University of Michigan, Ann Arbor (Burton), Department of Quality, University of California, San Francisco Health, San Francisco (Vella); Department of Psychology, San Diego State University, San Diego (Matt); Department of Occupational Therapy and Psychological and Brain Sciences, Center for Psychiatric Rehabilitation, Boston University, Boston (McGurk). Drs. Burton and Vella were with the SDSU/UC San Diego Joint Doctoral Program in Clinical Psychology at the time of this research.
Cynthia Z. Burton, Ph.D.
San Diego State University and University of California, San Diego (SDSU/UC San Diego), Joint Doctoral Program in Clinical Psychology, San Diego (Mahmood); Research Service (Mahmood, Keller) and Center of Excellence for Stress and Mental Health (Twamley), U.S. Department of Veterans Affairs San Diego Healthcare System, San Diego; Department of Psychiatry, University of California, San Diego (Twamley); Department of Psychiatry, University of Michigan, Ann Arbor (Burton), Department of Quality, University of California, San Francisco Health, San Francisco (Vella); Department of Psychology, San Diego State University, San Diego (Matt); Department of Occupational Therapy and Psychological and Brain Sciences, Center for Psychiatric Rehabilitation, Boston University, Boston (McGurk). Drs. Burton and Vella were with the SDSU/UC San Diego Joint Doctoral Program in Clinical Psychology at the time of this research.
Lea Vella, Ph.D.
San Diego State University and University of California, San Diego (SDSU/UC San Diego), Joint Doctoral Program in Clinical Psychology, San Diego (Mahmood); Research Service (Mahmood, Keller) and Center of Excellence for Stress and Mental Health (Twamley), U.S. Department of Veterans Affairs San Diego Healthcare System, San Diego; Department of Psychiatry, University of California, San Diego (Twamley); Department of Psychiatry, University of Michigan, Ann Arbor (Burton), Department of Quality, University of California, San Francisco Health, San Francisco (Vella); Department of Psychology, San Diego State University, San Diego (Matt); Department of Occupational Therapy and Psychological and Brain Sciences, Center for Psychiatric Rehabilitation, Boston University, Boston (McGurk). Drs. Burton and Vella were with the SDSU/UC San Diego Joint Doctoral Program in Clinical Psychology at the time of this research.
Georg E. Matt, Ph.D.
San Diego State University and University of California, San Diego (SDSU/UC San Diego), Joint Doctoral Program in Clinical Psychology, San Diego (Mahmood); Research Service (Mahmood, Keller) and Center of Excellence for Stress and Mental Health (Twamley), U.S. Department of Veterans Affairs San Diego Healthcare System, San Diego; Department of Psychiatry, University of California, San Diego (Twamley); Department of Psychiatry, University of Michigan, Ann Arbor (Burton), Department of Quality, University of California, San Francisco Health, San Francisco (Vella); Department of Psychology, San Diego State University, San Diego (Matt); Department of Occupational Therapy and Psychological and Brain Sciences, Center for Psychiatric Rehabilitation, Boston University, Boston (McGurk). Drs. Burton and Vella were with the SDSU/UC San Diego Joint Doctoral Program in Clinical Psychology at the time of this research.
Susan R. McGurk, Ph.D.
San Diego State University and University of California, San Diego (SDSU/UC San Diego), Joint Doctoral Program in Clinical Psychology, San Diego (Mahmood); Research Service (Mahmood, Keller) and Center of Excellence for Stress and Mental Health (Twamley), U.S. Department of Veterans Affairs San Diego Healthcare System, San Diego; Department of Psychiatry, University of California, San Diego (Twamley); Department of Psychiatry, University of Michigan, Ann Arbor (Burton), Department of Quality, University of California, San Francisco Health, San Francisco (Vella); Department of Psychology, San Diego State University, San Diego (Matt); Department of Occupational Therapy and Psychological and Brain Sciences, Center for Psychiatric Rehabilitation, Boston University, Boston (McGurk). Drs. Burton and Vella were with the SDSU/UC San Diego Joint Doctoral Program in Clinical Psychology at the time of this research.
Elizabeth W. Twamley, Ph.D. [email protected]
San Diego State University and University of California, San Diego (SDSU/UC San Diego), Joint Doctoral Program in Clinical Psychology, San Diego (Mahmood); Research Service (Mahmood, Keller) and Center of Excellence for Stress and Mental Health (Twamley), U.S. Department of Veterans Affairs San Diego Healthcare System, San Diego; Department of Psychiatry, University of California, San Diego (Twamley); Department of Psychiatry, University of Michigan, Ann Arbor (Burton), Department of Quality, University of California, San Francisco Health, San Francisco (Vella); Department of Psychology, San Diego State University, San Diego (Matt); Department of Occupational Therapy and Psychological and Brain Sciences, Center for Psychiatric Rehabilitation, Boston University, Boston (McGurk). Drs. Burton and Vella were with the SDSU/UC San Diego Joint Doctoral Program in Clinical Psychology at the time of this research.

Notes

Send correspondence to Dr. Twamley ([email protected]).

Competing Interests

The authors report no financial relationships with commercial interests.

Funding Information

National Institute of Mental Healthhttp://dx.doi.org/10.13039/100000025: R01MH080150, T32MH019934
This work was funded by the National Institutes of Health (R01MH080150; T32MH019934).

Metrics & Citations

Metrics

Citations

Export Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

For more information or tips please see 'Downloading to a citation manager' in the Help menu.

Format
Citation style
Style
Copy to clipboard

View Options

View options

PDF/EPUB

View PDF/EPUB

Login options

Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.

Personal login Institutional Login Open Athens login
Purchase Options

Purchase this article to access the full text.

PPV Articles - Psychiatric Services

PPV Articles - Psychiatric Services

Not a subscriber?

Subscribe Now / Learn More

PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.

Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).

Media

Figures

Other

Tables

Share

Share

Share article link

Share