An Evaluation of SOAR: Implementation and Outcomes of an Effort to Improve Access to SSI and SSDI
Abstract
Objective:
Accessing Supplemental Security Income (SSI) and Social Security Disability Insurance (SSDI) is challenging for people experiencing or at risk of homelessness. The federally funded SSI/SSDI Outreach, Access and Recovery (SOAR) project aims to improve access to SSI and SSDI through technical assistance (TA) to communities for promoting collaboration among agencies that serve this population and training professionals to assist people with applications. This study examined how SOAR TA has influenced community implementation efforts and the outcomes of these efforts.
Methods:
Data were collected through site visits, a social network survey, and a management information system (MIS) in 13 states that began receiving TA in 2010 or 2011 and from Social Security Administration (SSA) administrative files on all applications (N=2,438,944) nationwide that were filed initially in fiscal year 2010.
Results:
About 13% of trainees completed an application with the SOAR process. These trainees achieved a 73% approval rate for initial applications (including those approved after reconsideration), according to the MIS. Initial applications were approved almost twice as often among SOAR participants than among all homeless applicants (50% versus 28%), according to SSA. The odds of approval of the initial application were significantly higher (odds ratio=2.30) for SOAR participants compared with other homeless applicants, when other variables were held constant.
Conclusions:
Clients achieved better application outcomes when practitioners provided application assistance by using concepts from SOAR. However, training alone was not sufficient for successful SOAR implementation. Careful systems planning and dedicated staff resources are critical for improved outcomes.
Supplemental Security Income (SSI) and Social Security Disability Insurance (SSDI), administered by the Social Security Administration (SSA), provide financial assistance, access to health care, and employment assistance for eligible individuals with qualifying disabilities. A substantial portion of people who are homeless or at risk of homelessness have impairments that would qualify them for SSI or SSDI.
Research suggests that at least 46% of individuals who are homeless or at risk of homelessness have one or more chronic general medical conditions (1–3), and as many as 67% have received a psychiatric diagnosis during their lifetimes (4–6). Yet accessing SSI or SSDI is challenging for this population. Their precarious living situations, the combination of disabilities they face (which often include mental health or substance abuse problems), and their tendency to lack social and familial supports make it difficult for them to file a complete and high-quality application. In addition, SSA staff and disability examiners charged with assessing an applicant’s medical condition may have trouble contacting applicants without a stable address and phone number, making it difficult to collect the information necessary to determine eligibility. They may also have trouble obtaining details of the applicant’s medical history, which makes it difficult to develop evidence to support the applicant’s claim.
The SSI/SSDI Outreach, Access and Recovery (SOAR) project, funded by the Substance Abuse and Mental Health Services Administration (SAMHSA), aims to improve access to SSI and SSDI for people who are homeless or at risk of becoming homeless, with a particular focus on people with mental illness. SAMHSA’s SOAR Technical Assistance (TA) Center provides states and local communities with free TA to help them create systems-level change by promoting collaboration among agencies that serve this population and to train professionals who work with the target population to provide assistance throughout the SSI and SSDI application process.
The SOAR TA Center uses SAMHSA’s Stepping Stones to Recovery curriculum to train service providers in assisting people through the SSI and SSDI application process. The curriculum emphasizes ten critical components for improving the quality of SSI and SSDI applications. First, it emphasizes the need to conduct extensive outreach to establish rapport with potential applicants, track their whereabouts, and engage them on an ongoing basis. Other components include becoming an applicant’s authorized representative (which allows the practitioner to provide and receive information on the applicant’s behalf), collecting medical evidence from treatment providers to submit with the application (collaborating with physicians and psychologists for assessments, if necessary), writing and submitting a report that summarizes how the impairment affects the applicant’s daily functioning, and obtaining a quality review of the application from a SOAR expert (such as an agency supervisor or director, an in-state SOAR trainer, or the SOAR TA Center) before submission. Together, these practices are intended to avoid the need for an applicant’s personal treatment provider or an SSA-contracted provider to conduct assessments of the applicant’s general medical or mental health (consultative exams). Disability examiners request a consultative exam if they do not have enough evidence to determine whether an applicant has a qualifying disability.
Until recently, communities that wished to implement SOAR were required to send one or more professionals to an in-person train-the-trainer program, support these professionals in training others to provide application assistance, and facilitate relationships among various entities that support the application process. A key short-term goal of these implementation efforts, for which communities received no direct funding, was to increase the SSI and SSDI application approval rate. The purpose of this study was to examine how SOAR TA has influenced community efforts to implement SOAR and the outcomes of these efforts on approval rates for SSI and SSDI applications. In 2014, the SOAR TA Center replaced the train-the-trainer model with online training to reduce the implementation costs for both the TA Center and communities.
Methods
The study relied on four key sources of data. First, data on SOAR processes were collected from stakeholder interviews (in person and by telephone) and researcher observations in the most active SOAR community in each of 13 states that began receiving TA in fiscal year (FY) 2010 or 2011. Data collection continued in each community for up to three years after implementation of SOAR. Stakeholders included individuals leading SOAR efforts within the community, SOAR practitioners and their supervisors, SOAR trainers, SSA field office staff, staff at state Disability Determination Services (DDS) offices responsible for determining whether an applicant has a qualifying disability, and health care providers.
Second, stakeholders in these communities completed a brief survey about the nature of their relationships and amount of their communication with other stakeholders before and after their participation in SOAR. The sample consisted of 102 stakeholders across all communities, and the response rate was 90% (N=92). Of those who did not respond to the survey, most were unreachable because they had left their positions.
We calculated network density using survey data by creating binary variables indicating whether each stakeholder in the network had communication at least every month or two with the other entities in the network (0=no, 1=yes). We then divided that number by the number of scores of 1 that were possible at each site—for example, at a site with seven stakeholders, each stakeholder could assign a score of 1 for communication with each of the other stakeholders, for a total possible score of 42. Density may be used as a measure of group cohesion or the degree to which the group is unified or tight-knit. We calculated density twice—once before SOAR implementation and once after.
Third, SOAR practitioners in these communities used a management information system (MIS) developed by the SOAR TA Center to track data on their efforts and the outcomes of applications submitted with their help. The study used data entered during a two- to three-year period after implementation of SOAR. Data collection instruments for each of these three study components were cleared by the Office of Management and Budget.
Finally, SSA provided administrative data from its Structured Data Repository and Systems of Records for all individuals who filed SSI and SSDI applications nationally in FY 2010 (N=2,438,944). These data, which contain variables identifying SOAR participants, were used to compare application outcomes for SOAR participants and other SSI and SSDI applicants and to conduct fixed-effects logistic regressions (with robust standard errors calculated using the Huber-White sandwich estimator) to assess the relationship between SOAR participation and the SSI and SSDI application decision. The dependent variable was whether the application was approved (0=no, 1=yes), and the dummy variable was whether the application was submitted through the SOAR process (0=no, 1=yes). Control variables were applicant’s age, primary diagnosis, and state of residence and whether the applicant had submitted an application before. Although many other factors may influence the application decision, we used these control variables because they were the only ones available in the administrative data files for analysis.
The SSA data were also used to predict the probability of application approval associated with use of various SOAR practices emphasized in the training. Because the response variable for this analysis was binary (approval or disapproval), it was necessary to specify the regression model in such a fashion that the probability of the outcome was bound between 0 and 1 for each observation in the data. Therefore, although the outcome of a logistic regression is either 0 or 1, the predicted value for each observation was the probability of an outcome being a success (or a failure).
Results
Training
Over the course of the study, 563 individuals in the 13 communities that were the focus of the process analysis were trained in SOAR. Only 75 (13%), however, completed an application by using the SOAR process. Qualitative data suggested that many trained individuals did not complete applications by using the SOAR process because they had no direct interaction with the target population (administrators and others who do not necessarily work directly with people who are homeless or at risk of homelessness often attend training), they did not have time because of competing obligations (a single application can take between 20 and 40 hours to complete), they left their job before having a chance to provide application assistance (the homeless services workforce generally has high staff turnover), or they did not feel prepared to apply their skills in practice.
Trainees who completed applications by using the SOAR process generally complied with the model’s critical components. According to the SOAR MIS, SOAR trainees helped submit 696 applications that received an initial decision. Almost all of those 696 applications used authorized representatives (N=690, 99%) and included copies of applicants’ medical records (N=663, 95%). Five of the 13 communities developed formal agreements with medical providers—such as state and regional hospital systems, community clinics, correctional agencies, and the state veterans administration—to provide medical records in support of SSI or SSDI applications for free and on an expedited basis. Just over 80% of applications (N=565) were submitted with a report summarizing the applicant’s functioning, and 61% (N=427) were submitted after a quality review. Two-thirds (N=457, 66%) did not require a consultative exam.
The three components analyzed with SSA administrative data—an authorized representative, submission of medical evidence with the application, and avoidance of a consultative exam—used alone or in combination, were predictive of higher approval rates for initial applications compared with applications with none of these components (Figure 1). Submission of medical evidence with the application and avoidance of a consultative exam are likely related, however; the more evidence that is submitted with an application, the less likely it is that the disability examiner will need to gather additional evidence with an exam. In addition, the chances of the application’s approval increase as more of these practices are used.
FIGURE 1. Likelihood (percentage) of approval of initial applications for Supplemental Security Income (SSI) and Social Security Disability Insurance (SSDI), by use of three SOAR componentsa
aSource: Administrative data on all initial adult SSI and SSDI applications (N=2,438,944) filed in all states in fiscal year 2010 from the Social Security Administration’s Structured Data Repository (60–0320) and Systems of Records. SOAR, SSI/SSDI Outreach, Access and Recovery; AR, authorized representative; ME, medical evidence; CE, consultative exam
Regardless of participation in SOAR or an applicant’s living situation, the average likelihood that an initial SSI or SSDI application would be approved was 66% if all three practices were used, compared with 26% if none of them were used. Using only one of the practices raised the likelihood of approval to between 29% and 40%; using two increased the likelihood of approval to between 41% and 56%. Data on use of the other seven components were not available in the SSA files obtained for this study.
Collaboration
Communication among stakeholders increased after the introduction of SOAR in all 13 communities, as measured by network density. On average, across all 13 communities, network density increased by one-third, from eight of 30 scores of 1 at baseline (before SOAR training) to 18 of 30 scores of 1 at the end of the study period, with 30 as the average number of possible scores of 1 across the 13 sites. The increase reflected both new lines of communication and increases in preexisting communication. Across the 13 communities, network density increased over time from 12% (from 19 to 24 of 42 possible scores of 1) to 80% (from 6 to 30 of 30 possible scores of 1).
Stakeholders in all 13 communities reported increased communication with SSA field office and DDS staff. A majority of stakeholders in each community found that SSA and DDS were helpful in assisting people who are homeless or at risk of homelessness to access SSI or SSDI benefits (Figure 2). Few respondents indicated that SSA or DDS had not been helpful to them.
FIGURE 2. Ratings by stakeholders of helpfulness of SSA and DDS in carrying out SOAR, by SOAR sitea
aSource: Social network survey of SSI/SSDI Outreach, Access and Recovery (SOAR) stakeholders, including individuals leading SOAR efforts within the community, SOAR practitioners and their supervisors, SOAR trainers, Social Security Administration (SSA) field office staff, staff at state Disability Determination Services (DDS) offices responsible for determining whether an applicant has a qualifying disability, and health care providers. The number of stakeholders in each site varied. Stakeholders in site 11 were not asked about DDS because it was not an active stakeholder at that site at the time of the survey.
Application Outcomes
SOAR practitioners in the 13 states reported in the MIS that 73% (N=506) of the 696 applications that had received a decision were approved. Two-thirds (N=465, 67%) were approved at initial application, and the rest (N=41, 6%) were approved after a reconsideration of the initial decision. The approval rate among all initial applications in the MIS (N=833), including 137 that did not have an initial decision, was 56%. The number of approved applications across the 13 communities ranged from three to 111 (approval rates of 20% to 100%).
Results from analyses of SSA administrative data indicate that SSI and SSDI applications submitted through the SOAR process were approved at much higher rates than other applications. In FY 2010, SOAR participants were approved for SSI or SSDI at the initial application at almost double the rate for all homeless applicants—50% compared with 28%—and at a substantially higher rate than applicants who were not homeless (35%). Relatively more SOAR participants whose initial application was denied went on to receive reconsideration—46% compared with about 40% of all homeless applicants and applicants who were not homeless. In addition, relatively more SOAR participants (26%) were approved after reconsideration compared with approval rates among all SSI or SSDI applicants who were homeless (14%) or not homeless (12%). Altogether, SSI or SSDI applications were approved for 32% of applicants who were homeless and for 38% of applicants who were not homeless.
Logistic regressions indicated that SOAR participants had higher approval rates relative to non-SOAR participants even after the analyses controlled for other individual and state factors that may influence the application decision (Table 1). Among homeless applicants, the odds of approval of the initial application were significantly higher (odds ratio=2.30) for individuals who went through the SOAR process compared with those who did not.
| Variable | Log odds estimate | SE | p | OR estimate |
|---|---|---|---|---|
| Homeless | .83 | .13 | <.001 | 2.30 |
| Age | .05 | .00 | <.001 | 1.05 |
| Prior application | ||||
| First indicator | –.03 | .03 | .19 | .97 |
| Second indicator | .70 | .89 | .42 | 2.02 |
| Intercept | –4.16 | .15 | <.001 | na |
a
Source: Administrative data on all states from the Social Security Administration’s Structured Data Repository (60–0320) and Systems of Records. Model statistic: R2=.171; intraclass correlation (ICC)=.020. States are also included as control variables, and the ICC is the variation in the outcome variable due to state. Primary diagnosis codes are not shown, but log odds estimates for all primary diagnosis codes are significant at the .05 level. SOAR, SSI/SSDI Outreach, Access and Recovery
Discussion
Results suggest that when practitioners provide application assistance by using concepts from SOAR, they help clients achieve better application outcomes. However, training alone is not an efficient use of resources, given that most practitioners who were trained in SOAR never applied SOAR in practice. Communities with relatively high levels of success obtaining application approvals each had practitioners dedicated solely to conducting SOAR application assistance. Communities that expected practitioners to take on SOAR duties along with their other job responsibilities struggled more because staff found it difficult to carve out time for the effort amid competing demands and high caseloads, even when benefit access theoretically was an important component of their work. These findings suggest that communities would benefit from more targeted training efforts, from creating full- or part-time SOAR positions within stakeholder agencies, and from maintaining caseloads that are manageable.
The TA Center estimates that one full-time-equivalent practitioner can complete 50 applications per year. Some communities have secured funds for dedicated SOAR staff through SSA’s Interim Assistance Reimbursement program (which reimburses states for general assistance provided to individuals after the SSI protective filing date and before the approval date), through reimbursement from the Centers for Medicare and Medicaid Services for uncompensated medical care provided between 90 days before the SSI or SSDI protective filing date and the approval date, through grants from federal funding streams, or through state general funds.
The organizational networking facilitated by SOAR communities and supported by the SOAR TA Center successfully increased communication among key stakeholders. The biggest payoff was in new collaborations between SOAR providers and SSA field offices and DDS staff. As the entities responsible for processing applications and making determinations, SSA and DDS have large roles to play in SOAR. Ideally, SSA and DDS staff understand the challenges SOAR applicants face, accommodate those difficulties to the extent possible, and develop relationships with SOAR practitioners, who can provide additional information about applicants. These agencies can facilitate their roles—and have done so in many communities—by designating a point of contact to handle SOAR-related issues and applications and by flagging and expediting applications submitted through SOAR.
Among applicants with similar impairments, applications submitted by using the SOAR process were approved at higher rates compared with applications that did not use SOAR, probably because SOAR helps applicants provide more information to SSA and disability examiners. The more information that is included with an application, the less likely it is that disability examiners will have to find additional evidence to support an applicant’s claim. When disability examiners do not have enough evidence to support a claim, they may ask the applicant to undergo a consultative exam. Consultative exams are particularly problematic for individuals who are homeless or at risk of homelessness. Even if disability examiners have contact information to notify the applicant of the appointment and even if the applicant keeps the appointment, the consultative exam provider may not be able to obtain a complete picture of the individual’s condition and functioning in one short session. The need for a consultative exam, as well as problems collecting or remedying other missing or inadequate information in the application, can delay the claim adjudication process or result in a denial.
This study provides the first independent assessment of SOAR’s outcomes and the mechanisms associated with SOAR’s success. The study had several limitations, however. The communities included in the interviews and observations about process, the social network survey, and MIS data collection and analysis may not be representative of SOAR communities nationally, and the outcomes in the MIS data are based on self-reports from SOAR practitioners and were not independently verified. In addition, SSI and SSDI applicants who were homeless and who submitted applications through the SOAR process were not always identifiable in the SSA administrative data, and these data contained limited variables to use as controls in regression analyses.
Despite this study’s contributions and given its limitations, several questions remain that could be addressed in future studies. For instance, how would the results change if all applicants and SOAR participants who are homeless were properly identified in SSA administrative data? Are better short-term application outcomes for SOAR participants associated with positive long-term outcomes (with respect to income, housing, health, and general well-being)? What would an experimental evaluation of SOAR’s effects show? Planned expansion of SOAR training into new communities through the U.S. Department of Housing and Urban Development’s Continuum of Care program or with the U.S. Department of Veterans Affairs and to new populations (such as staff at hospitals, jails, or prisons) may provide a valuable context in which to embed small randomized control trials.
Conclusions
Findings suggest that providing application assistance by using the SOAR model shows substantial promise for helping individuals who are experiencing or who are at risk of homelessness to access SSI or SSDI and that support from the SOAR TA Center promotes systems-level collaboration toward the goal of improving program access. Applications for SSI and SSDI submitted through the SOAR process were approved at much higher rates than applications that were not associated with SOAR. Promulgating training to provide this application assistance through a train-the-trainer model, however, uses resources inefficiently. The online training that the TA Center began offering in 2014 may prove more cost-efficient and may appeal to individuals most likely to use SOAR in practice.
Acknowledgments
Although not a party to the contract, the Social Security Administration provided administrative data without which much of this evaluation would not have been possible.
Footnote
This study was conducted by Mathematica Policy Research for the U.S. Department of Health and Human Services, Substance Abuse and Mental Health Services Administration (contract HHSS283200700036I/HHSS28342001T) under subcontract to Policy Research Associates, Inc.
References
1.
Burt MR, Aron LY, Douglas T, et al: Homelessness: Programs and the People They Serve: Findings of the National Survey of Homeless Assistance Providers and Clients. Washington, DC, Urban Institute, 1999
2.
O’Toole TP, Conde-Martel A, Gibbon JL, et al: Where do people go when they first become homeless? A survey of homeless adults in the USA. Health and Social Care in the Community 15:446–453, 2007
3.
Zlotnick C, Zerger S: Survey findings on characteristics and health status of clients treated by the federally funded (US) Health Care for the Homeless Programs. Health and Social Care in the Community 17:18–26, 2009
4.
A Status Report on Hunger and Homelessness in America’s Cities: A 27-City Survey. Washington, DC, United States Conference of Mayors–Sodexho USA, Dec 2004. Available at www.usmayors.org/hungersurvey/2004/onlinereport/HungerAndHomelessnessReport2004.pdf
5.
Goering P, Tolomiczenko G, Sheldon T, et al: Characteristics of persons who are homeless for the first time. Psychiatric Services 53:1472–1474, 2002
6.
North CS, Eyrich KM, Pollio DE, et al: Are rates of psychiatric disorders in the homeless population changing? American Journal of Public Health 94:103–108, 2004
Information & Authors
Information
Published In
Cover: Tea infuser and strainer, by Marianne Brandt, circa 1924. Silver and ebony. The Beatrice G. Warren and Leila W. Redstone Fund, 2000, The Metropolitan Museum of Art, New York City. Image copyright © The Metropolitan Museum of Art; image source: Art Resource, New York City.
History
Received: 26 June 2015
Revision received: 21 December 2015
Accepted: 1 February 2016
Published online: 2 May 2016
Published in print: October 01, 2016
Authors
Competing Interests
The authors report no financial relationships with commercial interests.
Funding Information
Substance Abuse and Mental Health Services Administration10.13039/100000058: HHSS283200700036I/HHSS28342001T
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBLogin options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).