Biostatistics a Mystery to Many Residents

Many medical residents do not have sufficient education and training to help them fully understand and objectively evaluate the mountain of research information with which they are regularly confronted, according to a study in the September 5 Journal of the American Medical Association (JAMA).

Donna Windish, M.D., M.P.H., an assistant professor of medicine at Yale University School of Medicine, and colleagues surveyed residents in 11 internal medicine residency programs in Connecticut. The survey contained questions about the residents' attitude toward statistics and their confidence in interpreting statistical concepts, as well as 20 multiple-choice questions designed to test their knowledge of statistical methods, study design, and interpretation of study results most commonly represented in recent articles in major general medical journals, such as JAMA and Annals of Internal Medicine.

The 277 residents who completed the survey did not do very well on the statistics knowledge assessment. The mean knowledge score was 41.1 percent, indicating an average of eight correct answers for the 20 test questions. While over 80 percent of the residents were able to recognize double-blind studies and interpret relative risk correctly, only about 1 in 10 interpreted the results of a Kaplan-Meier analysis correctly. Only 1 in 3 understood how to interpret an adjusted odds ratio from a multivariate regression analysis. Slightly more than half could give the meaning of a P value.

The lack of knowledge in statistics is consistent with the residents' self-reported lack of confidence and attitudes about interpreting medical literature. Three-quarters of the respondents said they did not understand all the statistics they encountered in the literature, and 15 percent felt they did not trust statistics “because it is easy to lie.” The mean score indicating their understanding of statistical concepts was 11.4 on a scale of 0 to 20.

Several statistically significant factors were identified in the article as associated with the residents' performance on the knowledge test. Being male, having an advanced degree, having had biostatistics training, and enrollment in a university-based rather than a community-based training program were linked to higher scores. The duration since graduating from medical school was inversely associated with the score, indicating that new graduates did better on the test.

The authors urged improvements in teaching and reinforcing statistics in medical schools and residency programs, especially more advanced statistical methods beyond Student's t test and chi-square test, as more complex statistics such as multiple logistic regression and Kaplan-Meier analysis are used increasingly in medical research. Physicians who do not understand the statistics used in medical literature will have difficulty in critically evaluating the validity of published evidence and conclusions, which in turn will affect their ability to use the information appropriately in patient care.

“Understanding and being able to examine the current research is [also] critical in psychiatric practice,” Molly McVoy, M.D., chair of the APA Committee of Residents and Fellows, commented to Psychiatric News. “There is so much to be learned in this specialty and so much new information daily that can affect our patients.”

McVoy agreed that training on biostatistics and evaluation of medical literature seems to be inconsistent in programs across the country. She believes that adequate education in biostatistics should “start in medical school, where the foundation for medical practice is laid” and that “medical schools need to do a better job of helping future physicians understand and navigate current research.”

She pointed out that the ability to critically evaluate research information in the literature is the key to physicians' lifelong learning once they leave residency.

An abstract of “Medicine Residents' Understanding of the Biostatistics and Results in Medical Literature” is posted at<http://jama.ama-assn.org/cgi/content/abstract/298/9/1010>.▪