Consent Form Readability and Educational Levels of Potential Participants in Mental Health Research

The study was approved by the Central Office Research Review Committee and by the institutional review board of the University of Massachusetts Medical School.

Table 1 shows the range and average readability of the informed consent forms as calculated by each readability formula. The mean FRES for 154 tested forms was 47.6, which indicated a "difficult" reading level, with a corresponding mean FKGL score at a 12th-grade reading level. The mean Fog Index readability for 154 forms was a 14.5th-grade reading level. Ten informed consent forms fell beyond the 16th grade level on the Fry Graph; the mean score for the remaining 144 informed consent forms was a 13.9th-grade reading level.

The readability scores were assigned a ranking and then grouped by risk level. Of the 154 forms, 68 were for risk level 1 studies, 74 were for risk level 2 studies, and 12 were for risk level 3 studies. Table 2 shows the mean rank of readability scores by risk level for each formula. For each formula tested, the mean rank of readability scores increased with increasing risk level, indicating that the consent forms used in studies with higher risk were more difficult to read. The chi square analysis showed that the difference in ranking was statistically significant between the stratified risk levels.

Table 3 shows the maximum attained educational level for the 12,848 adults receiving services from the Massachusetts Department of Mental Health. Thirty-five percent did not graduate from high school (included in this category are 1,075 persons who received a "certificate of completion" in lieu of a diploma). Thirty-seven percent graduated from high school or obtained their GED but had no further education, and 28% had some education beyond the 12th grade.

This study examined 154 consent forms approved by the institutional review board of the Massachusetts Department of Mental Health between 1993 and 2004 and found that the average readability ranged from grade 12 to 14.5. Furthermore, consent forms became more difficult to read as the risk level of the study increased. The significance of this finding is emphasized when placed in context with the educational data for adults eligible to participate in the studies. Even by the most conservative estimate (that is, the FKGL, which yielded the lowest overall mean readability score), approximately 35% lacked the educational level required to read the average informed consent form. Considering that persons with mental illness tend to read three to five grade levels below their maximum level of education, the discrepancy we found between the readability of consent forms and the reading ability of potential study participants may actually be underestimated. This discrepancy calls into question the utility of informed consent forms in conveying to participants the information that they need to make informed judgments about whether to enter a study.

This study has several limitations. First, the educational levels of clients that were taken from the Massachusetts Department of Mental Health database may differ from those of the actual participants in the research studies approved by the department. It is possible that individuals who consented and participated in the research studies had higher reading abilities and therefore were more likely to comprehend the consent forms. In fact, some reports involving non-mentally ill adults suggest that individuals with lower levels of education are less likely to consent to participate in research studies ( 19, 20 ). If this were also true of those with mental illness, our findings would overestimate the percentage of study participants who are unable to read the average consent form. In addition, we note the temporal discrepancy between the data on educational levels gathered from the Massachusetts Department of Mental Health in September 2005 and the informed consent forms of studies conducted between 1993 and 2004. These educational levels may not represent the individuals who were eligible to participate in earlier studies; however, we are not aware of any significant change in the educational characteristics of the population receiving services over time. Furthermore, the educational attainments of this population do not necessarily represent those of patients with mental illness in general.

Another important drawback of our study lies with the use of readability formulas. Although the formulas we chose have been used extensively in medical studies looking at informed consent forms, health-related education materials intended for patient consumption, and even articles published in medical journals, they clearly have limitations. Most formulas (including those used in our study) do not take into account the difficulty of vocabulary used in text; instead they focus on "surface" characteristics. However, we found that nearly all of the forms reviewed included complex medical vocabulary that is not likely to be well understood by lay persons, much less those with serious mental illness.

We also note that readability formulas were originally designed to assess elementary and secondary school texts. Thus scores falling beyond the 12th grade level may be considered to have less practical validity than those at or below the 12th grade level. Furthermore, our calculations do not take into account the length of informed consent forms. Many contained more than ten single-spaced, typed pages, especially studies at risk level 3. Recent research on the readability of text has highlighted the importance of coherence, the logical structure of text, and cohesion—that is, how well different parts of the text relate to one another ( 21 ). Although coherence and cohesion are not captured by the readability formulas currently available, at least one group is seeking to change that ( 22 ). As noted previously, a number of the consent forms we studied provided tables that were not included in our analyses. Forms using such organizational devices, presumably designed to improve comprehension of study participants, may "score higher" on coherence and cohesion than others. In fact, some commentators have recommended the use of similar devices to supplement traditional text-based consent forms ( 23 ).

Finally, we are aware that use of any form is generally considered but one part of a larger informed consent process. Many of the consent forms we examined included statements that either encouraged prospective participants to ask questions or asked them to sign a statement indicating that all questions had been answered. Although verbal interaction in concert with reading an informed consent form is likely to improve comprehension, it seems reasonable to ask that forms be readable, especially for the target population.

On the other hand, we believe that it would be unfortunate for institutional review boards simply to mandate that informed consent forms meet certain readability criteria. Notwithstanding the mandates many institutional review boards have already imposed, it is not clear whether forms can be written at a sufficiently low level to match the educational level of prospective participants. Nor is it clear whether this would guarantee comprehension. At least one study suggests that patients with schizophrenia have poorer reading comprehension, compared with IQ-matched controls ( 24 ). Kovnick and others ( 25 ) examined 27 patients with schizophrenia and found that negative symptoms correlated strongly with poor understanding and appreciation (two of the four components in decision-making capacity); however, at least a third of the patients studied performed as well as the control group. Two studies found that the use of simplified consent forms for participants in nonpsychiatric studies did not improve comprehension or recall testing. However, participants did report that using simplified forms lowered anxiety and increased satisfaction ( 16, 26 ). These data suggest that comprehension during the informed consent process may need to be assessed on an individual basis.

In fact, efforts have been made by a number of groups to develop instruments to measure the comprehension of individual study participants. Some instruments assess multiple aspects of participants' decisional capacity—for example, the MacArthur Competence Assessment Tool for Clinical Research ( 27 ). Other instruments provide more rapid assessments of understanding of a more limited range of disclosed information—for example, the Evaluation of Signed Consent ( 28 ). In such cases, participants who do not meet the threshold scores could undergo educational training on the details of research studies in general and those specific to their respective study. If on repeat testing they still fail to demonstrate the capacity to understand relevant information, they would be excluded from participating in the study. Although data are not available to indicate how commonly these practices are used in mental health research today, the authors are aware of a number of studies, including the multisite Clinical Antipsychotic Trials in Intervention Effectiveness study of antipsychotic medications, that have employed these approaches ( 29 ).

The precise means to ensure adequate informed consent procedures for those with mental illness remain elusive. Extreme approaches have been suggested at both ends of the continuum. One could eliminate high-risk research involving persons with serious mental illness, but this seems unreasonable because testing indicates that many persons with mental illness have the requisite capacity to consent ( 30 ). At the other end of the spectrum at least one philosopher has suggested that it is our current understanding of the informed consent process that must change ( 31 ). A good-faith effort on the part of institutional review boards and researchers to inform study participants should suffice for what is unattainable: the truly informed study participant. This philosopher's argument is partly utilitarian: to continue on our present course "threatens to bring enrollment in many ethical clinical trials to a near standstill." At this point, however, in the absence of sufficient data to determine the extent to which participants' capacities can be improved by educational interventions, this conclusion seems decidedly premature. But it does seem clear that without some change in the informed consent process, a large number of persons participating in mental health research are likely to continue to be at risk of being uninformed.

The authors report no competing interests.

Dr. Christopher is affiliated with the Department of Psychiatry, Brown Medical School, Butler Hospital, 345 Blackstone Blvd., Providence, RI 02906 (e-mail: [email protected]). Dr. Foti is with the Massachusetts Department of Mental Health, Boston. Ms. Roy-Bujnowski is with the Department of Psychiatry, University of Massachusetts Medical School, Worcester, Massachusetts. Dr. Appelbaum is with the Department of Psychiatry, Columbia University College of Physicians and Surgeons, New York City.