Impact of Mental Health First Aid on Confidence Related to Mental Health Literacy: A National Study With a Focus on Race-Ethnicity

This study used a cross-sectional design with secondary data from the National Council for Behavioral Health’s MHFA Database, which includes course-specific feedback from individuals who completed the 12-hour, in-person MHFA training. All training sessions were conducted by certified MHFA instructors. Feedback forms were either completed and returned to the MHFA instructors for data entry upon completion of the training or completed online at a later date. The feedback form included 30 questions, of which 15 were relevant to our study.

Three questions related to demographic characteristics (age, gender, and race-ethnicity). Participants identified their race-ethnicity by selecting all that applied among six categories, including American Indian/Alaska native (AI/AN), Asian, black or African American, Hispanic or Latino, native Hawaiian and other Pacific islander, and white.

Three questions related to the competency of the MHFA instructor, specifically the extent to which the instructor’s presentation skills were engaging and approachable, he or she demonstrated knowledge of the material presented, and he or she facilitated activities and discussion in a clear and effective manner. Nine questions assessed perceived confidence with respect to the practical application of skills and knowledge related to MHL. [A list of the nine questions is available as an online supplement to this report.]

The 12 items related to the perceived ability of the MHFA instructor and perceived MHL confidence were rated on a 5-point Likert scale. Perceived ability of the instructor was determined by summing responses to the three questions. If there were two instructors, two sets of responses were obtained, and the average score for each item was calculated. The average scores were summed to provide a total indicator of perceived ability of the MHFA instructor. An MHL confidence score was calculated by summing responses to the nine items related to MHL, for a possible score of 9 to 45.

Between July 2010 and September 2012, a total of 59,006 individuals completed the 12-hour MHFA training in 46 states and the District of Columbia. The largest percentage of participants were from California (N=8,070, 17.4%), followed by Missouri (N=4,091, 9.3%). Delaware had the fewest participants (<.1%). Feedback forms were completed by 46,289 participants, a 78% response rate. A majority of the sample was female (N=33,740, 72.9%), the greatest number were between the ages of 25–44 (N=19,771, 42.7%) and most were white (N=25,050, 54.1%). Multiple racial-ethnic categories were selected by 740 (1.6%) participants, too few to constitute a separate category; therefore, these records were recoded as missing. [A table summarizing the characteristics of the study participants is available in the online supplement.]

MHL confidence scores were calculated by gender, age, and race-ethnicity, and F tests and t tests were used to determine whether the variables were associated with significant differences in scores. Multiple regression was used to examine differences in total MHL confidence scores among racial-ethnic groups after the analysis controlled for age, gender, and perceived ability of the MHFA instructor. Given the large sample size, effect sizes were calculated using Cohen’s d (13). MHL confidence scores were calculated for 44,997 (97.2%) participants who completed the feedback form. Because of missing data on the covariates, which ranged from .3% for perceived ability of the MHFA instructor to 17.5% for race-ethnicity, the multiple regression analysis included 36,263 participants. Institutional review board approval was obtained by the Human Research Protections Office of the University of New Mexico.

The mean±SD MHL confidence score was 41.06±4.94. The distribution was skewed, with scores ranging from 9 to 45 (median=43). Mean MHL confidence scores were slightly, but significantly, higher for females compared with males (41.20 versus 40.61, p<.001 [t test]). Significant differences in total MHL confidence scores were found between different age groups (p<.001) and racial and ethnic categories (p<.001), according to F tests in one-way analyses of variance [see online supplement].

Multiple regression was used to examine differences in total MHL confidence score by race-ethnicity, while controlling for age, gender, and perceived ability of the MHFA instructor. An F test identified significant overall differences in MHL confidence scores by race-ethnicity (p=.003). The normality of residual error assumption in regression analysis, which may not be strictly met because of the skewed distribution of MHL-confidence scores, is less of an issue in our analysis given the sufficiently large sample size and the central limit theorem.

After the analyses corrected for multiple comparisons, pairwise comparisons of total MHL confidence scores by racial and ethnic category identified significant group differences between AI/ANs and Hispanics or Latinos, blacks or African Americans, and whites (p<.05, Bonferroni corrected) (Table 1). For all comparisons, AI/ANs were significantly more likely to have lower MHL confidence scores. Cohen’s d was used to calculate the effect size for each significant pairwise comparison of total MHL confidence scores. All of the effect sizes were trivial to small. The largest difference in total MHL confidence score was between Hispanic and AI/AN samples (Cohen’s d=.22) and was still within the small range.

This is the first national study of MHFA in the United States. According to a large number of trainees, MHFA increased one’s confidence in being able to apply various skills and knowledge related to MHL, including the ability to recognize someone who may be dealing with a mental health problem or crisis and to actively and compassionately listen to someone in distress. Although AI/ANs were more likely to have lower MHL confidence scores compared with Hispanics or Latinos, blacks or African Americans, and whites, the differences were found to be trivial. Our findings are consistent with evaluations conducted elsewhere and add to the small but growing evidence base for MHFA in this country. Substantiation of the impact of MHFA on MHL (or MHL confidence, as in this study) in the United States is necessary to support the widespread dissemination of the program and recent financial investments in the program by federal agencies (4–6).

The strengths of this study included a large sample size, a national scope, and racial and ethnic diversity among participants. However, several limitations merit discussion. Missing data can be a major problem when working with large secondary data sets, and this was the case with the National Council for Behavioral Health’s MHFA Database (14). An examination of the trends in missing data showed that records with higher MHL confidence scores were more likely to be selected for inclusion in the multiple regression analysis. The self-perceived impact of the MHFA training may be overestimated given that participants with higher scores were more likely to be included in the final sample. The findings may also have been influenced by use of a convenience and voluntary response sample, which included participants who completed the feedback form.

The lack of representativeness of our sample may limit the generalizability of our results. For example, participants from 20 states made up 88% of the sample, and 26 states were each represented by less than 1% of participants. The self-report nature of the data and the lack of comparison data on MHL confidence before training were additional limitations of the methodology. The extent to which MHFA increased MHL cannot be measured without pretraining MHL confidence scores. The use of secondary data also limited the examination of covariates to those for which data were collected on the feedback form. The impact of MHFA on MHL confidence is likely influenced by many variables, including education, literacy, and experience with mental illness, either as a person with a mental illness or a caregiver. Also, the time elapsed between the course and the receipt of the feedback form and the psychometric properties of the items used to assess MHL confidence are unknown. Finally, because MHFA is limited to a Western model of MHL and because culture influences the ways in which mental illnesses are defined, understood, and treated, conclusions regarding the impact of MHFA on MHL must be interpreted with caution (2).

The National Council recently shortened the 12-hour course to eight hours and has introduced a module overlay for the adult course for both military members and veterans and public safety workers. A separate curriculum for persons working with youths has also been released. Future evaluations of the impact of these and other adaptations on MHL are necessary.

The small to trivial differences in MHL confidence, determined by effect sizes rather than significance testing, between racial and ethnic groups—contrary to hypotheses—may be influenced by the small sample size among some of the groups (for example, AI/ANs) and the reliance on quantitative responses to items on a structured survey. Focus groups may better identify the impact of MHFA on MHL among various cultures. Focus groups could also advance our understanding of MHL among different racial and ethnic groups by asking individuals from various cultures to identify culturally appropriate items to include in pre-post assessments of interventions that aim to improve how people respond to psychosocial distress. Finally, the psychometric properties of the items used to measure MHL should be assessed and a standard pretest assessment of MHL should be implemented in all MHFA training sessions so that the impact of the course on changing MHL can be determined.

Our findings suggest that MHFA improves confidence about MHL. Although research in health promotion indicates that self-efficacy beliefs are related to health behavior outcomes, we do not know at present whether knowledge and skills acquired from MHFA actually translate into action (15). Future evaluations of MHFA with better outcome measures and a better research design should examine whether and how persons who complete the training respond when crises surface and the outcomes of those encounters.

The authors acknowledge the mentorship provided by A. Kriechman, M.D., in preparing the manuscript.