Stable housing is a vital component of and a prerequisite for effective treatment of persons with serious mental illness who are living in the community ( 1, 2, 3, 4 ). Numerous studies have demonstrated that regardless of specific housing approaches, high-quality housing coupled with support services is associated with positive consumer outcomes ( 5, 6, 7, 8 ). Supported housing, which is independent living coupled with the provision of community-based mental health services, has been considered a preferred and viable housing model that offers an opportunity for community integration through its normalized housing setting, consumer-empowerment perspective, and provision of flexible and individualized services ( 3, 4 ).
Several studies have demonstrated the effectiveness of supported housing compared with other housing alternatives. Positive outcomes in favor of supported housing include longer housing tenure ( 4, 9, 10 ), better quality of life ( 11 ), more meaningful activities and work ( 12 ), greater housing satisfaction ( 13, 14, 15 ), lower rate of psychiatric hospitalization ( 16 ), and lower cost ( 17, 18 ). These positive findings notwithstanding, studies have also found that not all consumers of supported housing sustain long-term tenancy, suggesting that maintaining stable housing may be a primary life challenge for some, because consumers with mental health problems face multiple demands as a result of their medical and psychological conditions ( 19, 20, 21 ).
Research has documented the incidence of departure from supported housing, but it has not distinguished between different types of departure. Indeed, not all departures from supported housing should be considered a negative outcome. Moving to market-rate rental housing or to subsidized housing without residential support services indicates a higher level of autonomy than that associated with supported housing and may be perceived as a positive exit. Therefore, for effective planning of policies and programs that address the needs of supported housing residents, it is critical not only to address how well persons with serious mental illness maintain housing but also to distinguish between different types of departure and to understand their associations with various individual and housing factors.
This study examined personal and housing characteristics as potential predictors of different positive and negative departures from supported independent living programs in Philadelphia. Positive departures denote a higher level of autonomy than is provided in these programs, and negative departures indicate a contrary outcome. The supported independent living programs in this study have features that are consistent with core principles and operational characteristics of the supported housing model, as well as features that deviate from those of the model ( 22 ). A multisite evaluation of supported housing funded by the Center for Mental Health Services similarly documented that housing programs represent a continuum of supported housing and that they do not fit one ideal model ( 23 ). The study reported here extends previous research on housing tenure that aggregated different departure types into a single category, which may produce misleading results and questionable policy and practice implications.
Methods
Study site and overview of programs
Supported independent living programs in Philadelphia began in 1987 when the city participated in the Robert Wood Johnson (RWJ) Foundation Program on Chronic Mental Illness. The initial allotment included 125 Section 8 housing vouchers to subsidize rents for persons with serious mental illness. From 1987 to 2002 the capacity of the supported independent living programs rose to 632 units through funding from federal, state, and local sources. In 2002 a total of 27 residential support teams provided services to residents to maintain independent living through skills development and by linking residents to community-based mental health services. Admissions were processed through a single-point-of-entry referral and placement unit, which conducted assessments and placements for all applicants to community residential programs ( 22 ).
A variety of housing units were available for residents, including single-room-occupancy units, one- to three-bedroom apartments, and single-family homes. A majority (60%) of supported independent living residents in 2002 lived in rental units that were either owned by or leased through a quasi-government housing development agency established at the time of the RWJ program. The role of the housing agency was that of a regular property owner for persons with psychiatric and other disabilities; the agency was not involved in providing support services for its tenants. An equal proportion of residents lived in rental units that were owned privately and leased through residential support teams (20%) and in buildings that were owned by mental health agencies that also operated residential support teams (20%) ( 22 ).
Although supported independent living programs provided long-term subsidized housing with support for independent living, access to and maintenance of housing was contingent upon sobriety, making these programs resemble the "treatment first" approach ( 24 ). A majority (85%) of residential support teams instituted drug and alcohol testing for residents. Program rules and regulations were used to exert behavioral control over residents and facilitate treatment compliance. For example, more than one-third of the 27 teams required resident participation in house and community meetings, and one in five implemented a mandatory escrow policy for residents to put aside savings for future use. Despite these program requirements, residential support staff discouraged noncompliant and problematic behaviors, such as treatment noncompliance and potentially self-destructive behavior, but staff did not regard these behaviors as warranting immediate discharge from housing. A protracted process for working with "noncompliant" residents was generally put in place to avoid undesirable housing discharges.
Participants
The study presented here was a secondary analysis of data from a cross-sectional study of supported independent living residents in Philadelphia. The protocol of the parent study, conducted in 2002–2003, examined the extent to which housing and service characteristics were associated with community integration of a sample of 252 residents. The sample size of 252 in the parent study was determined by conducting power analyses to arrive at an appropriate number of study participants ( 25 ). The study reported here tracked 237 individuals in the original sample, because data on stays in and discharges from supported independent living were available for them. Study participants were tracked for 30 months from the date of the 2002–2003 interview to determine whether they stayed in or left the supported independent living program. Participation in the original study was based on the following eligibility criteria: residence at the current supported independent living housing unit for six months or more, a primary diagnosis of a serious mental illness (schizophrenia or a major mood disorder) ascertained by a psychiatric evaluation conducted by the agency, age 18 years or older, and voluntary agreement to participate in the research as demonstrated by a signed consent form.
Data sources
Data were collected from structured self-report interviews by trained interviewers with study participants. The interviews, which were conducted in 2002 and 2003, lasted an average of two hours. The interview protocol included questions on basic sociodemographic and clinical characteristics, income, social support, perceived relationship with residential support staff, and housing preference. The residential outcome data, drawn from the administrative database of the supported independent living program, consisted of dates of admission and discharge and destinations of discharge. Data on environmental characteristics of the housing were compiled by using home addresses of 237 residents. Block-group-level data on neighborhood sociodemographic characteristics and crime were drawn from the 2000 U.S. Census data file and the Philadelphia police crime database, respectively. This research was approved by the University of Pennsylvania Institutional Review Board.
Measures
Demographic and clinical characteristics and control variables. Age was determined at the time of the interview. Gender, race (African American or other), psychiatric diagnosis (schizophrenia or other), and housing preference (yes or no) were all binary coded. Previous length of stay in housing provided by the supported independent living program was measured by number of days between the admission date and the interview date. Substance abuse history was measured by a one-item yes-or-no question in the original interview: "In your lifetime, have you considered your use of any substances on the list a major problem?" Substances on the list included alcohol, heroin, methadone, other opiates and analgesics, barbiturates, sedative-hypnotics and tranquilizers, cocaine and crack cocaine, amphetamines, cannabis and marijuana, hallucinogens, inhalants, and other substances. Severity of psychiatric symptoms was measured by 14 items of the Colorado Symptom Index (CSI), a self-report measure that has been found to be reliable and valid ( 26, 27 ). Possible scores range from 0 to 70, with higher scores indicating more severe psychiatric symptoms. Self-reported health status was measured by the 12-item Short-Form Health Survey (SF-12), which is known to be a reliable and valid instrument for measuring functional health status ( 28, 29 ). Possible scores range from 0 to 100, with higher scores indicating better health. These instruments were administered as part of the interview protocol in the parent study.
Personal resources. Income was measured as the total amount of money received from all sources during the past month. We included two aspects of social support—number of social network members as an objective dimension and satisfaction with social relations as a subjective dimension. Categories of relationship in the social network included spouse, partner, or significant other; family member; housemate or living companion; work, volunteer, or school associate; friend or acquaintance; group or organization member; service professional; and peer consumer. The number of social network members was dichotomized, with 1 indicating ten or more members and 0 fewer than ten members. Satisfaction with social relations was measured by a ten-item subscale from Lehman's Quality of Life interview, which has satisfactory reliability and validity ( 30, 31 ). Possible scores range from 0 to 70, with higher scores indicating more satisfaction with social relations (Cronbach's α =.89).
Supportiveness of the program's residential staff as perceived by participants was measured by an adapted version (29-item) of the Community Oriented Programs Environment Scale developed by Moos ( 32 ) to indicate the extent to which the consumer-staff relationship is considered to be respectful, spontaneous, consumer-oriented, and proactive on the part of the staff. Possible scores range from 0 to 29, with higher scores indicating a more supportive relationship with staff as perceived by the consumer ( α =.88).
Housing environmental characteristics. To construct the measures of neighborhood distress and crime level, we extracted data from the 2000 Census data file on demographic, housing, and socioeconomic characteristics of 1,816 block groups in Philadelphia. A neighborhood distress score was derived by principal-components analysis reflecting the percentages in the block group with female-headed households with children under age six, households below 75% of the poverty line, vacant housing units, the unemployment rate for the block group, and households with public assistance income. From the same principal-components analysis, a crime factor score was derived, reflecting crimes against persons per 1,000 population and crimes against properties per 1,000 population.
Housing outcomes. On the basis of the discharge codes reported in the administrative data, two types of housing departure were distinguished. Positive departure referred to moving to a higher level of independence, including living alone or living with a spouse in other subsidized housing, living in unsubsidized market-rate rental housing, and living in privately owned housing. Negative departure referred to leaving supported independent living for less independent settings, including those requiring higher intensity of care and institutionalized settings, or for unstable living arrangements, including nursing homes, psychiatric hospitals, congregate mental health residential programs, drug or alcohol treatment facilities, jails, or return to the streets or homeless shelters. To qualify as having departed, study participants had to have left supported independent living for 90 days or longer. This definition precluded temporary moves to other housing settings that were followed by a return to supported independent living.
Creating the housing outcome variable required quantifying the length of time each participant stayed in supported independent living. For persons who departed, length of stay was computed as the difference between date of admission to supported independent living and date of discharge. For those who remained in supported independent living through the end of the study, length of stay was computed as the difference between date of admission and date at the end of the study, or 30 months after the interview date. To capture the three housing outcomes—two types of departure and remaining in the program—an indicator variable was constructed for each participant, coded 1 for a positive departure, 2 for a negative departure, and 0 if the participant remained in supported independent living throughout the study period.
Analysis
Data analysis included computation of descriptive statistics for all study variables. A Cox proportional hazards model was used to analyze the effects of predictor variables on the probability of departure, with control for sociodemographic and clinical characteristics, length of stay in the supported independent living program, and housing preference. Different entry time was handled by including length of stay in the program before the interview as a covariate in the regression model. Because the study participants were nested within census block groups, robust sandwich estimation for the covariance matrix in SAS 9.1 was used to account for intrablock group correlation of outcomes. This particular option corrected for nonindependence within clusters (in this case, block groups) by specifying the COVSANDWICH (aggregate) statement in the procedure statement and then specifying the block group identifier variable as the level of aggregate in the model ( 33, 34 ).
Results
Participant characteristics
As Table 1 shows, the mean age of study participants was 45. Women accounted for more than half of the sample (58%), and 60% of participants were African American. Most participants (71%) had a diagnosis of schizophrenia. The mean CSI and SF-12 scores were, respectively, 30.8 and 56.5. Nearly half (54%) reported a lifetime history of substance abuse. The mean income was $796.80 per month. About two-thirds of the sample (68%) listed ten persons or more in their social network. The average perceived satisfaction with social relations score was 49.9. The average score of perceived supportiveness of residential staff was 20.7. Of the 237 participants in the sample, 17% experienced a negative departure and 14% experienced a positive departure. Sixty-nine percent stayed in supported independent living continuously.
Table 1 Characteristics of 237 participants in a supported independent living program
The mean neighborhood distress score was -.03, and the mean crime score was -.37. Because the factor scores were standardized with a mean of 0 and a standard deviation of 1, reflecting the city average, a score of less than zero indicates lower than average distress or crime level, and a score of greater than zero indicates higher than average distress or crime level.
Factors associated with departure
Table 2 displays results of the Cox proportional hazards model, indicated by conditional hazards ratios associated with predictor variables included in the regression model. Study participants who reported more income were more likely to experience a positive departure. The risk ratio of 2.60 for income indicates that for each unit increase in monthly income (logged), the probability of a positive departure increased by an estimated 160%. Neighborhood distress was inversely associated with a positive departure. For each unit increase in neighborhood social distress, the probability of a positive departure decreased by an estimated 56%. Substance abuse and perceived supportiveness of supported independent living staff were significant predictors of a negative departure. Specifically, the risk of negative departure for those who reported a past substance abuse problem was nearly twice that for those without a history. A greater level of perceived supportiveness of program staff was associated with a lower risk of negative departure.
Table 2 Cox proportional hazards ratios for predictors of departure type among 73 persons who left supported independent living
Discussion
The overall rate of housing retention found in this study was high. However, it should be noted that the study sample comprised individuals who had long stays; most had participated in the supported independent living program for an average of four years before the interview. Only 12% of participants had been in the program for a year or less. Residents who left the program within six months of their admission were not included in the study. The findings also show that maintaining a stable and independent housing situation was a challenge for some residents. These residents could not sustain tenancy in supported independent living and eventually made a transition to homeless shelters, institutions, or community residential settings that required a higher level of supervision and care. Nonetheless, one in eight participants moved to more independent housing.
The study identified several characteristics that predicted departure from supported independent living. Self-report of a past substance abuse problem was associated with an elevated risk of leaving the program for more dependent and potentially less stable living arrangements. Given the fact that all program applicants are admitted to housing only after they attain sobriety for six to 12 months, it is possible that a portion of residents who had past substance abuse problems had a recurrence at some point during their stay. Although our data did not allow us to ascertain which participants departed housing because of recurrent substance use problems, our finding highlights the importance of continuity of care in substance use treatment for residents with a history of substance abuse problems in order to avoid a negative housing outcome.
There has been much debate on whether abstinence from alcohol and illicit drugs should be mandated in housing programs for persons with serious mental illness. Proponents of the sobriety mandate believe that providing immediate access to independent housing without requiring sobriety enables residents to continue substance use and that residents with a past substance abuse problem may be motivated to stop using substances if continued access to housing is contingent on abstinence ( 35 ). The "housing first" approach, on the other hand, is based on the premise that provision of housing before treatment may facilitate recovery from substance abuse and lead to housing stability. The housing-first model adopts a "harm reduction approach" to substance use and provides permanent independent apartments without requiring treatment compliance ( 10 ). Research findings have not been conclusive regarding the benefits and risks of the two housing approaches. Although several studies have suggested that substance abuse is a strong predictor of negative housing outcomes among persons with serious mental illness ( 2, 4, 6, 36, 37, 38 ), others have found that persons with co-occurring disorders served by the housing-first model achieve superior residential outcomes without increased substance use ( 4, 10, 24, 39 ). Few studies, however, have examined the mechanisms through which these two contrasting housing approaches enable residents with mental disorders to achieve residential stability. Although it is unclear which approach works best for whom ( 40 ), housing programs need to respond with individualized support services to meet the heterogeneous needs of consumers as they progress toward recovery from substance abuse.
An inverse relationship was found between a consumer's perceived supportiveness of residential staff and the likelihood of a negative departure. It is plausible that a trustful relationship with support staff may facilitate housing stability by ensuring continuous provision of supports, which is especially important in times of crisis and emergencies.
Having more income was associated with an increased likelihood of a positive departure. Conversely, limited income may act as a barrier for residents to move on to living situations that require higher levels of financial independence, as in the case of market-rate housing. Given the fact that persons with serious mental illness have generally low and often fixed incomes, the significance of financial resources is not a surprising finding. Services that may augment personal income by facilitating access to education and employment opportunities may provide avenues for consumers to procure additional financial resources and reduce future financial difficulties, thereby increasing their opportunities to access more independent housing options.
The inverse relationship between neighborhood distress and a positive departure signals the importance of environmental factors in the likelihood of successful housing outcomes. Distressed conditions in the neighborhood surrounding a supported independent living residence may affect mental health outcomes of residents by exacerbating psychiatric symptoms, fostering a sense of social isolation, and increasing exposure to illicit drug use ( 41, 42, 43 ). The principal implication for policy makers and service providers is that the environment of housing programs needs to be considered in service planning and development.
This study had several limitations that need to be addressed in future research. First, data on variables pertaining to personal resources were based on self-report, which is susceptible to bias. Second, although admission to supported independent living was based on a common set of eligibility criteria, there may be selection bias that was not captured in the analysis. For example, selection bias might exist if supported independent living residents with similar characteristics were more likely to be placed in the same housing environment. However, potential effects of differential selection were controlled for statistically by including sociodemographic and clinical characteristics and by using statistical modeling that corrected for clustering of residents in the same census block group. Third, the study focused on supported independent living programs in Philadelphia, which has a unique service context that may differ from those of similar housing programs in other communities. This includes a centralized gatekeeping system for assessing and placing program applicants and the availability of a variety of support from federal, state, and local governments in providing housing subsidies and support services to persons with psychiatric disabilities ( 44 ).
Moreover, our findings cannot be generalized to populations served by the housing-first model or independent housing programs that do not require sobriety before admission, given that extended sobriety is an eligibility criterion for supported independent living. Finally, it should be noted that findings were based on a sample of residents who consented to participate in the study. Thus caution is needed in generalizing the findings to the entire population of supported independent living residents. In addition, although inclusion in the study required residents to have a continuous six-month stay in supported independent living, a large portion of study participants had much longer stays. Although the analysis adjusted for length of stay in supported independent living before the interview, interpretation of our findings, strictly considered, should be limited to residents who had achieved residential stability for at least six months.
Conclusions
Because housing resources for persons with serious mental illness are limited, it is important for services planning and resource allocation to be able to predict who will be successful or unsuccessful in subsidized independent housing with support. This study makes a contribution to the knowledge base on housing for persons with psychiatric disabilities by identifying personal and environmental factors that may be modified to promote positive housing outcomes and reduce negative outcomes.
Acknowledgments and disclosures
The authors thank the Philadelphia Department of Behavioral Health and Mental Retardation for providing data on supported independent living programs. They also thank Robert Spencer, Ph.D., and Raj Phatak, M.A., for assistance with data analysis.
The authors report no competing interests.
Footnote
Dr. Lee is affiliated with the Department of Social Welfare, Pusan National University, 30 Jangjeon-dong, Geumjung-gu, Busan, Korea 609-750 (e-mail: [email protected]).Dr. Wong is affiliated with the School of Social Policy and Practice and Dr. Rothbard is with the Department of Psychiatry, University of Pennsylvania, Philadelphia.
References
1.
Bigelow D: Supportive homes for life versus treatment way-stations: an introduction to TAPS project 41. Community Mental Health Journal 34:403–405, 1998
Hurlburt MS, Wood PA, Hough RL: Providing independent housing for the homeless mentally ill: a novel approach to evaluating long-term longitudinal housing patterns. Journal of Community Psychology 24:291–310, 1996
Kasprow WJ, Rosenheck RA, Frisman L, et al: Referral and housing processes in a long-term supported housing program for homeless veterans. Psychiatric Services 51: 1017–1023, 2000
Lipton FR, Nutt S, Sabatini A: Housing the homeless mentally ill: a longitudinal study of a treatment approach. Hospital and Community Psychiatry 39:40–45, 1988
Parkinson S, Nelson G, Horgan S: From housing to homes: a review of the literature on housing approaches for psychiatric consumer/survivors. Canadian Journal of Community Mental Health 18:145–164, 1999
Tsemberis S: From streets to homes: an innovative approach to supported housing for homeless adults with psychiatric disabilities. Journal of Community Psychology 27: 225–241, 1999
Tsemberis S, Gulcur L, Nakae M: Housing first, consumer choice, and harm reduction for homeless individuals with a dual diagnosis. American Journal of Public Health 94:651–656, 2004
Tsemberis S, Rogers ES, Rodis E, et al: Housing satisfaction for persons with psychiatric disabilities. Journal of Community Psychology 31:581–590, 2003
Siegel CE, Samuels J, Tang DI, et al: Tenant outcomes in supported housing and community residences in New York City. Psychiatric Services 57:982–991, 2006
Browne G, Courtney M, Meehan T: Type of housing predicts rate of readmission to hospital but not length of stay in people with schizophrenia on the Gold Coast in Queensland. Australian Health Review 27:65–72, 2004
Culhane DP, Metraux S, Hadley T: The impact of supportive housing for homeless people with severe mental illness on the utilization of the public health, corrections, and emergency shelter systems: the New York-New York Initiative. Housing Policy Debate 13:107–163, 2002
Dickey B, Latimer E, Powers K, et al: Housing costs for adults who are mentally ill and formerly homeless. Journal of Mental Health Administration 24:291–305, 1997
Brown MA, Ridgway P, Anthony WA, et al: Comparison of outcomes for clients seeking and assigned to supported housing services. Hospital and Community Psychiatry 42: 1150–1153, 1991
Newman SJ, Reschovsky JD, Kaneda K, et al: The effects of independent living on persons with chronic mental illness: an assessment of the Section 8 certificate program. Milbank Quarterly 72:171–198, 1994
Wong YLI, Filoromo M, Tennille J: From principles to practice: a study of implementation of supported housing for psychiatric consumers. Administration and Policy in Mental Health and Mental Health Services Research 34:13–28, 2007
Rog DJ, Randolph FL: A multisite evaluation of supported housing: lessons learned from cross-site collaboration. New Directions for Evaluation 94:61–72, 2002
Padgett DK, Gulcur L, Tsemberis S: Housing first services for people who are homeless with co-occurring serious mental illness and substance abuse. Research on Social Work Practice 16:74–83, 2006
Wong YLI, Nath SB, Solomon P: Group and organizational involvement among persons with psychiatric disabilities in supported housing. Journal of Behavioral Health Services and Research 34:151–167, 2007
Conrad KJ, Yagelka JR, Matters MD, et al: Reliability and validity of a modified Colorado Symptom Index in a national homeless sample. Mental Health Services Research 3:141–153, 2001
Ware JE, Kosinski M, Keller SD: A 12-item Short Form Health Survey: construction of scales and preliminary tests of reliability and validity. Medical Care 34:220–233, 1996
Salyers MP, Bosworth HB, Swanson JW, et al: Reliability and validity of the SF-12 Health Survey among people with severe mental illness. Medical Care 38:1141–1150, 2000
Wood PA, Hurlburt MS, Hough RL, et al: Longitudinal assessment of family support among homeless mentally ill participants in a supported housing program. Journal of Community Psychology 26:327–344, 1998
Lee EW, Wei LJ, Amato DA: Cox-type regression analysis for large number of small groups of correlated failure time observations, in Survival Analysis: State of the Art. Edited by Klein JP, Goel PK. Dordrecht, Netherlands, Kluwer Academic, 1992
VanZandt PA, Mopper S: Delayed and carryover effects of salinity on flowering in Iris hexagona (Iridaceae). American Journal of Botany 89:1847–1851, 2002
Mares AS, Kasprow W, Rosenheck RA: Outcomes of supported housing for homeless veterans with psychiatric and substance abuse problems. Mental Health Services Research 6:199–211, 2004
Clark CC, Rich AR: Outcomes of homeless adults with mental illness in a housing program and in case management only. Psychiatric Services 54:78–83, 2003
O'Connell MJ, Kasprow W, Rosenheck RA: Rates and risk factors for homelessness after successful housing in a sample of formerly homeless veterans. Psychiatric Services 59:268–275, 2008
Martinez TE, Burt MR: Impact of permanent supportive housing on the use of acute care health services by homeless adults. Psychiatric Services 57:992–999, 2006
Mares AS, Desai RA, Rosenheck RA: Association between community and client characteristics and subjective measures of the quality of housing. Psychiatric Services 56:315–319, 2005
Steptoe A, Feldman PJ: Neighborhood problems as sources of chronic stress: development of a measure of neighborhood problems, and associations with socioeconomic status and health. Annals of Behavioral Medicine 23:177–185, 2001
Wong YLI, Hadley TR, Culhane D, et al: Predicting Staying in or Leaving Permanent Supportive Housing That Serves Homeless People With Serious Mental Illness. Washington, DC, US Department of Housing and Urban Development, Office of Policy Development and Research, 2006
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).
If the address matches an existing account you will receive an email with instructions to retrieve your username
Create a new account
Change Password
Password Changed Successfully
Your password has been changed
Login
Reset password
Can't sign in? Forgot your password?
Enter your email address below and we will send you the reset instructions
If the address matches an existing account you will receive an email with instructions to reset your password.
Change Password
Congrats!
Your Phone has been verified
×
As described within the American Psychiatric Association (APA)'s Privacy Policy and Terms of Use, this website utilizes cookies, including for the purpose of offering an optimal online experience and services tailored to your preferences. Please read the entire Privacy Policy and Terms of Use. By closing this message, browsing this website, continuing the navigation, or otherwise continuing to use the APA's websites, you confirm that you understand and accept the terms of the Privacy Policy and Terms of Use, including the utilization of cookies.
