Objective: Adults with mental retardation have histories of cognitive and adaptive deficits posing unique ethical challenges for research consent assessment. This study examined the capacity of persons with mental retardation to consent to participate in randomized clinical trials. Method: A total of 150 adults (50 each with mild and moderate mental retardation and 50 comparison subjects without mental retardation) responded to a set of consent questions for a hypothetical randomized clinical trial testing a medication for aggressive disorders. Intelligence, adaptive behavior, medical treatment history, and consent history were evaluated. Univariate and multivariate methods were used to compare performance across and within groups. Results: Comparison subjects scored significantly higher on measures of consent capacity than participants with mild mental retardation, who scored higher than those with moderate mental retardation. Most subjects with mental retardation were able to make a participation choice, and many understood research methods and appreciated the protagonist’s disorder and the consequences of participation. Almost half of those with mild mental retardation understood human subject protections. Performance was weakest on understanding the purpose of research and reasoning about whether to participate, suggesting vulnerability to the therapeutic misconception. Psychiatric and experiential factors did not predict consent capacity. Conclusions: While adults with mental retardation as a group showed consent deficits, many attained consent capacity scores comparable to those of comparison subjects. Investigators should consider individual differences and a consent format suited to deficits in language, memory, and attention before restricting consent opportunities for persons with mental retardation.
Mental retardation affects a range of cognitive and adaptive abilities that may impair the capacity to provide informed consent. Many adults with mental retardation have limited abstract reasoning skills, deficits in basic knowledge and communication ability, and limited opportunity for, and experience of, autonomous decision making; all of these limitations can impair the ability to make and communicate independent and reasoned choices in research settings. Persons with mental retardation are also vulnerable to a variety of psychiatric disorders, in which they have prevalence rates three to four times higher than the general population (1) . A dual diagnosis of mental retardation and mental illness can exacerbate difficulties in information processing, perception, empathic communication, and social coping (2) . Thus, among adults with mental retardation, those most likely to be recruited for psychiatric research may be those most likely to have a diminished decision-making capacity.
A few studies have shown strong relationships between mental retardation classifications and global indices of consent, reasoning about consent choices, and appreciation of the consequences of consent decisions (3 – 5) . Most studies of consent capacity have examined persons with schizophrenia, depression, HIV, or Alzheimer’s disease—populations that may suffer from intermittent episodes of distorted thinking and perceptions, impaired information processing and concentration, or progressively declining cognitive abilities (6 – 9) . In contrast to the lifelong history of cognitive deficits among persons with mental retardation, these patient populations had the capacity for fully developed reasoning and language skills before the onset of their illness.
In this study, we examined the capacity of adults with mental retardation to consent to participate in a hypothetical randomized clinical trial of a pharmacological treatment for aggressive disorders, which constitute the most common category of comorbid psychiatric diagnoses reported for the population with mental retardation and the category of disorders most commonly treated with pharmacotherapy (1, 2) . We were interested in exploring several questions in particular. In a sample of adults with mental retardation, what proportion can communicate a choice on whether to participate in a randomized clinical trial? Will this population have a better understanding of concrete aspects of research procedures than of the more abstract purposes of research in general or the use of placebo? Can adults with mental retardation appreciate the situation of being a patient and the consequences of the choice of whether or not to participate in a randomized clinical trial? Do intelligence, psychiatric profile, and experience with consent predict capacity to consent to participate in such trials?
Method
Participants
English-speaking adults with a documented diagnosis of mental retardation were recruited from community residences and adult day programs of not-for-profit agencies in New York City and Rochester, New York. Persons with dementia, autism spectrum disorders, active schizophrenia, and uncorrected vision or hearing problems were excluded. Participants included 50 adults with mild mental retardation (mean age=43.8 years, SD=10.4, range=25–69) and 50 with moderate mental retardation (mean age=41.2 years, SD=11.5, range=21–66). Overall, 54% were female, and 70% were Caucasian. For some participants the etiology of the mental retardation was a genetic chromosomal disorder or other prenatal or postnatal disorders, although for the majority the etiology was diagnosed as unknown. During the previous year, 76% of those with mild mental retardation and 64% of those with moderate mental retardation had been on a behavior management plan; 78% and 60%, respectively, were taking medication for behavioral disorders; and 48% and 49%, respectively, had diagnoses of psychiatric disorders. Most participants with mild (82%) and about half of those with moderate (48%) mental retardation had experience in consenting for medication or treatment, and 50% and 54%, respectively, had a guardian whose prior consent was required.
Selecting an appropriate comparison group for a study of adults with mental retardation is problematic because, unlike with adults who have average intelligence, in those with mental retardation the chronological and intellectual ages do not coincide, and their education and experience with autonomous decision making do not increase linearly with age. For our comparison group, we recruited freshman college students, a group generally in the age range at which individuals are first recognized as legally competent to give consent. Comparison subjects (mean age=19.5 years, SD=1.1, range=18–22) were recruited through campus announcements, and all provided written informed consent.
The research protocol was approved by institutional review boards at Fordham University and the University of Rochester School of Medicine and Dentistry. Permission was obtained from legal guardians or family members who were regularly involved in participants’ decisions. Written informed consent or assent was obtained in the presence of a staff member after the study procedures were explained and any questions the participant had were answered. All groups were informed that the purpose of the study was to understand how adults with developmental disabilities make decisions. Comparison subjects received $15 for their participation, and participants with mental retardation received the equivalent in the form of gift certificates to a popular fast-food chain.
Assessment Instruments
To assess intelligence, the Kaufman Brief Intelligence Test (11) was administered, and residence supervisors independently completed the Vineland Adaptive Behavior Scales (12) for participants with mental retardation.
The Assessment of Consent Capacity—Randomized Clinical Trials (ACC-RCT) was used to assess consent capacity. In this interview instrument, a vignette is presented of a hypothetical protagonist being given information about a randomized clinical trial being conducted to test the efficacy of a hypothetical new drug to treat aggressive behavior. The development of the ACC-RCT drew on the four-abilities model and format developed by Grisso and Appelbaum for the MacArthur Competence Assessment Tool for Clinical Research (10, 13) . In previous work we found that populations with mental retardation lacked the communication, memory, and attentional skills necessary to provide measurable responses to the semistructured interview format of the MacArthur instrument (3) . The ACC-RCT uses a single-unit disclosure format to present 19 paragraphs (1–4 lines each) of informed consent disclosure information written in a story-like sequence using brief sentences and simple concrete terms. Questions follow the reading of each paragraph. To minimize the possibility of rote answers, questions are phrased to avoid mimicking terms used in the disclosure, and the entire vignette is summarized before the questions related to reasoning and participation choice are presented. Administration of the Kaufman Brief Intelligence Test and the ACC-RCT took approximately 45 minutes altogether.
After being presented a summary of the elements of consent, participants were asked to communicate a choice of whether or not to participate (one item) for the hypothetical protagonist. The reasoning category (three items) included reasons for and against participating and a reason for their participation choice. Except for the item on communicating a choice, which was rated as pass/fail (coded 1 or 0), all questions were scored on a 3-point scale (0=no credit, 1=partial credit, 2=full credit). For partial credit, a response had to include some of the essential information. For example, the item on the purpose of the research described the researcher asking the protagonist “to be in a study to help test whether a new medicine called Paygo can help people control their behavior problems.” Full credit was given if the respondent was able to describe the purpose as “testing the medicine” or “seeing if Paygo works” and to indicate that the medicine could help control behavior problems or anger. Partial credit was given if one of these two concepts was mentioned. Standardized prompts (“Tell me more” or a repetition of the disclosure information and question) were used to probe no-credit or partial-credit responses. Scores increased after initial prompting for 27% of participants with mild and 52% with moderate mental retardation. All ACC-RCT interviews were audiotaped, transcribed, and independently scored by two trained raters (kappa coefficients for each item ranged from 0.68 to 0.94, and percent agreement ranged from 79% to 99%).
Statistical Analysis
Univariate and multivariate statistical methods were used to analyze the data, including correlation tests, t tests, analysis of variance (ANOVA), regression analysis, and multivariate analysis of variance (MANOVA). SPSS (version 12) (Chicago, SPSS) was used for all analyses.
Results
On the Kaufman Brief Intelligence Test, the mean standard score was 60.3 (SD=12.7, range=40–97) for participants with mild mental retardation, 48 (SD=10.5, range=40–84) for those with moderate mental retardation, and 106 (SD=7.4, range=86–126) for comparison subjects. Eighty-six percent with mild and 96% with moderate mental retardation scored in the low range of the Vineland Adaptive Behavior Scales composite score.
For analysis, the 19 items of the ACC-RCT were grouped into the four consent capacity categories. Table 1 presents the mean scores for each item on the ACC-RCT for each group, along with the numbers and percentages of participants in each group whose responses received full or partial credit. Significant effects of intellectual functioning were found for each item, at p<0.001 except for the item on communicating a choice, which was significant at p<0.05. Following a significant MANOVA, contrast tests assuming unequal variance indicated that the scores for the comparison subjects were significantly higher than those for participants with mild mental retardation, and these in turn were higher than scores for participants with moderate mental retardation (p<0.05), with two exceptions: participants with mild mental retardation and those with moderate mental retardation did not differ in their ability to provide reasons against participation, and the three groups did not differ in communicating a participation choice. All but three participants with mental retardation communicated a participation choice. In the end, 12% of those with mild mental retardation, 18% of those with moderate mental retardation, and 6% of comparison subjects decided that the protagonist should not participate in the hypothetical study.
A three-by-six MANOVA on the mean summary scores for the six subclasses of ability categories indicated differences in intellectual functioning across all scores (p<0.001) ( Table 2 ). Contrast tests for all subclasses yielded significant differences (p<0.001) between the mild mental retardation group and the comparison group and between the mild and moderate mental retardation groups. Most participants with mild mental retardation and 32% to 58% of those with moderate mental retardation scored within the range of the comparison group’s scores in the categories of research procedures, appreciation of the disorder, and appreciation of the consequences of participation. Almost half of subjects with mild mental retardation scored within the range of the comparison group in the category of human subject protections.
Within-Group Differences in Consent Capacity
As Table 2 shows, reasoning and understanding the purpose of research were more difficult than other ability categories for all three groups (for the comparison group, t=61.26, df=49, p<0.001; for the mild mental retardation group, t=26.38, df=49, p<0.001; for the moderate mental retardation group, t=13.07, df=49, p<0.001). Following a nonsignificant interaction between item difficulty and mental retardation, correlated t tests found similar patterns of items across the comparison group and the combined mental retardation groups. Participants found it easier to understand societal research benefits than the research purpose (within groups, t=4.58, df=49, p<0.01; between groups, t=2.66, df=99, p<0.01), easier to understand voluntary participation than confidentiality (within groups, t=2.82, df=49, p<0.01; between groups, t=9.58, df=99, p<0.01), easier to understand reasons for participation than reasons against participation (within groups, t=3.35, df=49, p<0.01; between groups, t=3.68, df=99, p<0.01), and easier to understand reasons for participation than giving a reasoned explanation for participation choice (within groups, t=5.07, df=49, p<0.01; between groups, t=2.68, df=99, p<0.01).
Participants with mental retardation found it easier to understand treatment procedures than assessment procedures (t=10.04, df=99, p<0.001), easier to understand the purpose of research than the purpose of a placebo (t=2.5, df=99, p<0.05), easier to appreciate research risks than research benefits (t=2.01, df=99, p<0.05), and easier to appreciate the protagonist’s disorder and the vignette’s description of a behavioral intervention than appreciating that a psychosocial and a pharmacological treatment had not reduced the symptoms of the protagonist’s behavior problem (t=3.04, df=99, p<0.01). Only the group with moderate mental retardation found it more difficult to appreciate the consequences of random assignment to treatment or placebo than to appreciate either treatment risks or benefits (t=2.15, df=49, p<0.05).
Table 2 provides full-scale scores based on the combined average of the 18 items reflecting understanding, appreciation, and reasoning that are presumed to reflect cognitive skills associated with consent comprehension (the “communicating a choice” item was not included). In separate analyses for the mild and moderate mental retardation groups, we examined correlations between the ACC-RCT full-scale score and intelligence scores (the verbal and matrices scales of the Kaufman Brief Intelligence Test; the Vineland adaptive behavior subscales [communication, daily life, socialization, and motor functioning]; medical history [aggressive behavior, psychopharmacological medications, and number of comorbid psychiatric diagnoses]; and consent experiences [for medical treatment, medication, and nonmedical decision making]).
For the mild mental retardation group, only the Kaufman Brief Intelligence Test verbal score correlated with the full-scale score (r=0.47, df=48, p<0.001). By contrast, the moderate mental retardation group’s full-scale score was correlated with the Kaufman verbal score (r=0.37, df=48, p<0.01), the Kaufman matrices score (r=0.32, df=47, p<0.05), the Vineland daily living subscale score (r=0.41, df=47, p<0.01), and the Vineland socialization subscale score (r=0.33, df=47, p<0.05).
To further examine the influence of these variables on the moderate mental retardation group’s full-scale ACC-RCT scores, regression models with a constant included in the equation were built with Vineland daily living subscale score, the Kaufman verbal score, the Kaufman matrices score, and the Vineland socialization subscale score regressed onto the full-scale score. Model 2, using the Vineland daily living subscale score and the Kaufman verbal score, produced the best fit (adjusted R 2 =0.25; R 2 change=113, F change=7.201, df=1, 46, p=0.01).
For both mental retardation groups, summary scores reflecting understanding of research procedures, human subject protections, appreciation, and reasoning were significantly correlated with one another, suggesting that common intellectual processes underlie different aspects of consent comprehension (r’s ranged from 0.46 to 0.83, df=48, p<0.01). Not surprisingly, a history of aggressive behavior, psychopharmacotherapy, and psychiatric diagnoses were also significantly correlated with one another (r’s ranged from 0.37 to 0.64, df=48, p<0.01). However, none of these mental health and treatment indices were related to participants’ history of consenting to medical treatments, except for a significant negative correlation between aggressive behavior and consent experience in the moderate mental retardation group (r=–0.32, df=48, p<0.05). Counter to expectations based on an experiential model of consent capacity, no associations were observed between medical or consent histories and any of the consent capacity summary scores.
Discussion
This study examined the ability of adults with mild and moderate mental retardation to comprehend essential elements of informed consent for randomized clinical trials. As would be expected given past findings involving adults with other cognitive disorders (9), adults with mental retardation were strongest in communicating a participation choice and weakest in providing reasons for or against participation. Also as expected, adults with mental retardation as compared to those without, and adults with moderate mental retardation as compared to those with mild mental retardation, had greater difficulty understanding, appreciating, and reasoning about a hypothetical randomized clinical trial, with one exception: adults in the mild and moderate groups had similar difficulty providing reasons against participation. This latter finding may be a function of the easy-to-understand potential benefits of, and the absence of serious risk in, this particular hypothetical randomized clinical trial. The possible benefits of the experimental medication were described in the vignette as reduced feelings of anger, shouting, and fighting behavior and the possibility that the protagonist could return to the workshop setting from which he or she had been suspended; risks included dizziness, sleepiness, dry mouth, stomachache, frequent urination, limb stiffness, and tremors.
What was unexpected was the proportion of adults with mental retardation whose performance on certain consent categories was comparable to that of comparison subjects with average intelligence. On questions reflecting understanding of research procedures, appreciating the nature of the protagonist’s problem, and appreciating the consequences of research participation, nearly all participants with mild mental retardation had scores within the range of comparison subjects’ scores, and about half scored within the range of the comparison group on understanding human subject protections. Participants with moderate mental retardation also performed better than anticipated: one-third to one-half scored within the range of comparison subjects on understanding of research procedures, appreciating the nature of the protagonist’s problem, and appreciating the consequences of participation in the research. One possible explanation for these high levels of performance is the grade-school-level language, the single-unit disclosure format, and the repetition of information in the ACC-RCT, all specifically incorporated into the instrument to compensate for the deficits in language comprehension, memory, and attention associated with mental retardation.
Unlike the typical patterns seen with the MacArthur Competence Assessment Tool for Clinical Research, appreciation scores on the ACC-RCT were not lower than understanding scores. One possible explanation for this pattern is that the nature and consequences of the hypothetical protagonist’s behavior problems and the behavioral and pharmacological treatments described in the vignette were personally or vicariously familiar to adults with mental retardation living in community residences. Another possible explanation is that, unlike in the MacArthur instrument, the ACC-RCT appreciation criteria did not require respondents to generate novel participation consequences; rather, full-credit scores could be attained by recalling the consequences described in the vignette. The importance of the mental retardation groups’ ability to appreciate the nature of situational background information and participation consequences should not go unnoticed. In actual research settings, reviewing with prospective subjects the history of the presenting problem and past attempts at treating it may facilitate comprehension of the rationale and nature of the clinical trial and positively contribute to the reasoning and decision-making process (14) . The data thus suggest that consent capacity may be enhanced when disclosures and consent assessment for randomized clinical trials are individualized for adults with mental retardation, and even when the capacity to give fully informed consent is questionable, obtaining meaningful assent from adults with mild to moderate mental retardation is feasible in nearly all cases.
The expectation that adults with mental retardation living in community residential settings would find the concept of voluntarism difficult (15) met with mixed results. The majority of participants with mild mental retardation and half of those with moderate mental retardation understood that the protagonist had the right to refuse research participation, whereas less than half understood the right to withdraw once the study had started. A diagnosis of mental retardation is characterized by both low standard intelligence scores and an impaired capacity to make adaptive decisions in daily life. Thus adults with mental retardation are more vulnerable than others to acquiescing to requests to please the investigator. Accordingly, participants with mental retardation should be reminded throughout a study of their right to discontinue participation at any time without penalty.
Confidentiality was also a difficult concept. A possible explanation for this finding is that adults with mental retardation living in community residences interact with a relatively limited number of adults without mental retardation, in a relatively isolated community, who are all involved in their treatment decisions (residence supervisors, physicians, and family caregivers). Of practical interest is the difficulty people with mental retardation had in understanding the purpose of research in general, randomization to placebo, and reasons not to participate, even though many were able to list the negative side effects of the medication and verbalize an understanding that assignment to the placebo group might not alleviate the aggressive behavior. These difficulties suggest that there is a strong influence of a “therapeutic misconception” among persons with mental retardation in the decision of whether to participate in a randomized clinical trial (16) .
Finally, as anticipated, within each mental retardation group, intelligence score predicted total score on the ACC-RCT. However, an unanticipated finding was the lack of association between the total consent comprehension score and participants’ consent experience, history of aggressive disorders in particular, and psychiatric symptom severity in general. While these findings can be explained in part by our exclusion of participants with schizophrenia and autism spectrum disorders, it underscores the major role of general intelligence in the comprehension of consent information. Moreover, continued poor comprehension for many participants with mental retardation despite prompting by repeating disclosure information suggests that short-term memory deficits are not exclusively responsible for vulnerabilities in consent capacity.
Regarding the generalizability of findings, disclosure information was presented in the form of a hypothetical treatment vignette about another individual. It might be argued that in actual research contexts, where information is personally relevant and investigators have an opportunity to respond to questions more readily and to observe prospective participants for nonverbal cues signaling confusion, consent abilities may be raised to a higher level. Moreover, investigators should be careful to explain to participants and their caretakers that difficulty in understanding information about a randomized clinical trial does not imply deficits in the capacity to make other treatment decisions or decisions about daily life.
Psychiatric treatments validated through randomized clinical trials hold great promise for advancing the quality of life of persons with mental retardation and comorbid psychiatric disorders and their ability to live and work in community settings. However, such research challenges investigators to balance the right of persons in this population to make autonomous participation decisions with the obligation to protect them from impaired decision making that may jeopardize their welfare (17) . While there are no agreed-on criteria for consent competence, comprehension of informed consent disclosure information is often expected to be at the level of a reasonable person presented with similar information. The results of this study suggest that many adults with mild mental retardation and some with moderate mental retardation are at least minimally able to grasp information of this sort when the consent format is tailored to accommodate general deficits in language, memory, and attention. The results further support previous recommendations that consent capacity should be evaluated on a sliding scale of risks (12, 18) . In many research contexts, the ability of adults with mental retardation to communicate a participation choice and to understand research and human subject protection information would be acceptable when risks are minimal and reversible. In studies in which risks are high and benefits unlikely, the inability of many of our study subjects to appreciate the voluntary nature of the study and to identify reasons against participation would make reliance on their consent ethically unjustified.
Footnote
Received March 19, 2005; revision received May 24, 2005; accepted July 12, 2005. From the Department of Psychology, Fordham University, and the Fordham University Center for Ethics Education, Bronx, New York; and the Strong Center for Developmental Disabilities, University of Rochester School of Medicine, Rochester, New York. Address correspondence and reprint requests to Dr. Fisher, Fordham University Center for Ethics Education, Department of Psychology, Dealy Hall, Bronx, NY 10458; [email protected] (e-mail).Supported by the National Institute for Child Health and Human Development, grant HD 39332-02.All authors report no competing interests.
References
1.
Lewis MH, Aman MG, Gadow KD, Schroeder SR, Thompson T: Psychopharmacology, in Manual of Diagnosis and Professional Practice in Mental Retardation. Edited by Jacobson JW, Mulick JA. Washington, DC, American Psychological Corporation, 1996, pp 323–340
Davidson PW, Cain NN, Sloane-Reeves JE, Van Speybroech A, Segel J, Gutkin J, Quijano LE, Kraner BM, Porter B, Shoham I, Goldstein E: Characteristics of community-based individuals with mental retardation and aggressive behavioral disorders. Am J Ment Retard 1994; 98:704–716
Morris DC, Niederbuhl JM, Mahr JM: Determining the capability of individuals with mental retardation to give informed consent. Am J Ment Retard 1993; 98:263–272
Arscott K, Dagnan D, Kroese BS: Consent to psychological research by people with an intellectual disability. Journal of Applied Research in Intellectual Disabilities 1998; 11:77–83
Moser DJ, Schultz SK, Arndt S, Benjamin ML, Fleming FW, Brems CS, Paulsen JS, Appelbaum PS, Andreasen NC: Capacity to provide informed consent for participation in schizophrenia and HIV research. Am J Psychiatry 2002; 159:1201–1207
Kim SYH, Caine ED, Currier GW, Leibovici A, Ryan JM: Assessing the competence of persons with Alzheimer’s disease in providing informed consent for participation in research. Am J Psychiatry 2001; 158:712–717
Kim SYH, Cox C, Caine ED: Impaired decision-making ability in subjects with Alzheimer’s disease and willingness to participate in research. Am J Psychiatry 2002; 159:797–802
Grisso TH, Appelbaum PS: Assessing Patients’ Capacities to Consent to Treatment: A Guide for Healthcare Professionals. New York, Oxford University Press, 1998
Fisher CB: A goodness-of-fit ethic for informed consent to research involving persons with mental retardation and developmental disabilities. Ment Retard Dev Disabil Res Rev 2003; 9:27–31
Roberts LW, Warner TD, Brody JL, Roberts B, Lauriello J, Lyketsos C: Patient and psychiatrist ratings of hypothetical schizophrenia research protocols: assessment of harm potential and factors influencing participation decisions. Am J Psychiatry 2002; 159:573–584
Appelbaum PS, Roth LH, Lidz CW, Benson P, Winslade W: False hopes and best data: consent to research and the therapeutic misconception. Hastings Cent Rep 1987; 17:20–24
Fisher CB: Relational ethics and research with vulnerable populations. Reports on Research Involving Persons With Mental Disorders That May Affect Decision-Making Capacity, vol II: Commissioned Papers by the National Bioethics Advisory Commission, Paper 3. Rockville, Md, 1999:29–49
Delano SJ: Research involving adults with decisional impairment, in Institutional Review Boards: Management and Function. Edited by Amdur RJ, Bankert EA. Sudbury, Mass, Jones & Bartlett, 2002, pp 389–893
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).
If the address matches an existing account you will receive an email with instructions to retrieve your username
Create a new account
Change Password
Password Changed Successfully
Your password has been changed
Login
Reset password
Can't sign in? Forgot your password?
Enter your email address below and we will send you the reset instructions
If the address matches an existing account you will receive an email with instructions to reset your password.
Change Password
Congrats!
Your Phone has been verified
×
As described within the American Psychiatric Association (APA)'s Privacy Policy and Terms of Use, this website utilizes cookies, including for the purpose of offering an optimal online experience and services tailored to your preferences. Please read the entire Privacy Policy and Terms of Use. By closing this message, browsing this website, continuing the navigation, or otherwise continuing to use the APA's websites, you confirm that you understand and accept the terms of the Privacy Policy and Terms of Use, including the utilization of cookies.
