New ABPN Executive Sees Big Changes for Board Exam

The psychiatry board certification examination is likely to evolve over time toward a fully computerized test, replacing the oral exam with a vignette-based format testing specific competencies.

So said Larry Faulkner, M.D., incoming executive vice president and chief executive officer of the American Board of Psychiatry and Neurology (ABPN).

Faulkner, who is currently vice president for medical affairs and dean of the University of South Carolina School of Medicine, will replace Stephen Scheiber, M.D., on June 30.

The ABPN is the independent, nonprofit organization that certifies doctors practicing psychiatry and neurology, as well as associated subspecialties.

In an interview with Psychiatric News, Faulkner described what he foresees as an evolution of the board certification and re-certification exams to become more standardized and reliable, reflecting the broad movement across American medicine toward holding physicians accountable for demonstrating specific and discrete competencies.

Standardization of the test is likely to mean, in time, elimination of the entire oral examination, a costly and logistically difficult aspect of the board exam. The oral examination has its detractors among some educators, while also proving difficult for some psychiatrists to pass. Neurology has already moved to eliminate the oral exam, Faulkner said.

But he cautioned that psychiatry has yet to make a formal decision to drop the exam, and that any such move would likely take several years before becoming a reality.

“If I had to get out my imperfect crystal ball, I would say we will probably follow the path of neurology and institute a fully computerized written exam with videotaped vignettes,” he said. “But that is going to be several years down the road.”

Faulkner said that for the Part II Board examination this month, which is being given in Philadelphia, the exam's audiovisual section will be replaced for the first time with a series of three written vignettes and one videotape of an interview with a psychiatric patient.

Candidates will rotate among the four stations and be examined by a different examiner at each station. In a written vignette station, the candidate will receive a copy of the vignette, and the examiner will read the vignette aloud. In a video-clip station, the candidate will view a four-to-five-minute video clip on a DVD player. The examiner will then ask the candidate a series of questions based on the case.

“The vignettes are designed to test specific competencies that the board believes are important,” Faulkner said. “If that's wildly successful, there is talk of modifying the oral part of the examination. That is clearly a direction [in which] we are moving. We are trying to make the exams fair and reliable and as valid as possible, and there is a sense that standardized vignettes will ultimately be more viable.”

Cost of the psychiatry board exam could also be reduced by elimination of the oral exam, he said.

“The oral examination is problematic and getting logistically more and more difficult and very expensive,” he said. “But if we eliminate that, we also have to figure out how to maintain our emphasis on testing competencies in the doctor-patient relationship, interviewing, case presentation, and clinical skills. So it's costly to plan for changes while at the same time continuing to do what we have been doing. You don't want to jump into something new without carefully thinking through all the possibilities. But ultimately, one key to having a more cost-friendly exam is to eliminate the oral part of the examination.”

Faulkner added that a goal toward which medical education is moving is to test candidates on competencies currently evaluated by the oral examination within the residency program itself. He cited a pilot program at Wake Forest University School of Medicine designed to assess the ability of a training program to evaluate residents on clinical skills during their training.

“There are going to be specific requirements for residency programs to document trainees' ability to do an interview,” he told Psychiatric News. “Training directors are going to have to certify whether residents can meet those competencies, and it will become another element of credentialing.

“An advantage to that is that if a resident has problems, they can be identified early with remedial attention given during the training period,” he added. “We won't have individuals finishing a residency who can't do those things.

“As it is now, candidates take the board exam and fail, but they have no framework for re-education,” he said.

Faulkner said the relatively low pass rate on the board exam for psychiatry, compared with that for other specialties, is a complex issue and is being addressed by APA's diagnostic examination of candidates who have failed multiple times (see facing page)

“We do not have a uniform population who sit for the examination,” he said. “If you look at young American medical graduates who take the exam right from residency, their scores are very similar to scores in other specialties. But we have a diverse population, and some candidates have difficulty passing the boards. When you add their experience, it looks like our pass rate is lower than average.

“Some of it may be explained by cultural or language barriers,” Faulkner said of those who fail the board. “But we suspect that a predominant reason is the crippling effect of anxiety.”

Certainly one of the most difficult and potentially controversial aspects of future ABPN testing is the implementation of “performance in practice” measures for maintenance of certification.

Faulkner explained that the American Board of Medical Specialties has mandated that all medical specialties require physicians to demonstrate competence in four categories for maintenance of certification: professional standing, self-assessment and lifelong learning, cognitive expertise, and performance in practice.

The latter refers to a requirement that physicians build into their routine practice the capacity to assess their performance continually against guidelines for best practice and make improvements to meet those guidelines.

“The performance in practice issue is a controversial and difficult one to develop,” he said. “It boils down to a quality improvement process. In general, physicians are going to have to demonstrate that they have looked at their practices and identified issues that need to be improved.

“The specifics of how to do that have not formally been decided, and we are looking at a number of options,” he said. “For psychiatry and neurology, this aspect of Maintenance of Certification is not scheduled to be implemented until 2010.

“We are trying to make it as user friendly as possible,” he said. “The challenge for the board and for APA is to come up with some ideas for how psychiatrists can look at their cohort of depressed patients, for instance, and make sure they are providing state-of-the-art treatment and make improvements where they are necessary. But it's definitely going to require some serious collaboration with APA and other organizations.”▪