Test Successfully Assesses Psychotherapy Competency

The test consists of case vignettes, drawn from clinical practice and disguised to protect patient confidentiality. Background information about the patient is followed by detailed descriptions of individual psychotherapy sessions. For each session, a multiple-choice question is presented that tests the resident's understanding of the dynamics of the session and the kinds of interventions that the therapist should pursue.

Ronald Rieder, M.D., director of psychiatric residency training at Columbia and one of the report's authors, said the test can be used to compare the skills of residents across a program, but also to compare overall program results with those from other residencies around the country.

“If your residents are doing poorly, program directors will know they have a job on their hands,” Rieder told Psychiatric News.

David Goldberg, M.D., president of the American Association of Directors of Psychiatric Residency Training, said the competency test is part of an effort among psychiatric educators to reinvigorate the teaching of psychotherapy in the context of an evolving—and sometimes contentious—debate about what the competencies of a psychiatrist should be.

“There has been a gradual shift in psychiatry that has been going for 20 years now toward pharmacology and diagnosis as the core of the field,” Goldberg told Psychiatric News. “But there continues to be a debate about where psychiatry should be heading. In the midst of all the advances in psychopharmacology, psychotherapy has become less and less an area that every training program puts a lot of training and time into. Educators and people in practice began to sense that psychiatry was losing its rigor and expertise incorporating psychotherapy and psychosocial dimensions in clinical work.”

Nearly seven years ago, Goldberg initiated a task force of educators to look at psychotherapy training. Surveys of training directors confirmed what Goldberg called a “deepening marginalization” of psychotherapy in residencies.

One outcome of that focus was a change in training requirements calling for every psychiatry resident in the country to demonstrate competency in five modalities: psychodynamic psychotherapy, cognitive-behavioral therapy, supportive psychotherapy, combined psychotherapy with medication, and brief psychotherapy.

But to teach something is one thing; to measure it is another. “This is where the test comes in,” Goldberg said. “How do you measure competency?”

He believes that the proven validity of the test and its growing acceptance among residency programs across the country is an indication of success.“ This is truly a pioneering step, proving that it can be done,” Goldberg said. “It is the only test of its kind in psychiatry.... My residents love it. They think it's an interesting, relevant test, and it has spurred them to want to learn more about this area.”

But whether it is really possible to test how future physicians will perform in the “awesome now” of a clinical encounter with a patient—and in an area as deeply dependent as psychotherapy is on the individual skills of empathy and intuition—remains controversial. And the psychotherapy competency test is not without critics who say that a paper-and-pencil exam can't take the place of supervision of actual clinical experience.

“My response is that you don't take the exam as an overall measure of whether someone can practice,” Goldberg said. “But it does give you a lot of information that can be used in conjunction with a supervisory report, giving you a much more three-dimensional picture of what people know.”

Goldberg said that supervision alone can also be a misleading indicator of real clinical skill. “Under supervision, the resident can have the right emotional tone, but may not know that much technically,” he said.“ It's possible to cover that up. But because the competency test is so case-based, it takes different points in the therapy and asks questions about what residents would do in certain situations, testing whether they understand technically what is going on.”

And Rieder believes the test can serve as a model for measuring other modalities. “We are not only developing a specific test, we are testing a way of making tests,” he said. “There is no reason you couldn't do this for testing interpersonal psychotherapy.”

Residency directors interested in using the test can contact Mullen by e-mail at [email protected].▪