Academic Performance of Psychiatrists Compared to Other Specialists Before, During, and After Medical School
Publication: American Journal of Psychiatry
Abstract
OBJECTIVE: This study was designed to compare psychiatrists with other physicians on measures of academic performance before, during, and after medical school. METHOD: More than three decades of data for graduates of Jefferson Medical College (N=5,701) were analyzed. Those who pursued psychiatry were compared to physicians in seven other specialties on 18 performance measures. Analysis of covariance was used to control for gender effect. RESULTS: Compared to other physicians, psychiatrists scored higher on measures of verbal ability and general information before medical school and on evaluations of knowledge and skills in behavioral sciences during medical school, but they scored lower on United States Medical Licensing Examinations step 3. CONCLUSIONS: The results generally confirmed the authors’ expectations about psychiatrists’ academic performance. More attention should be paid to the general medical education of psychiatrists.
Physicians’ specialty choices can be influenced by market forces, personality, personal experiences, preferences, and skills (1–7). More than any other specialty—including family medicine, in which laboratory testing and “minor” procedures play a considerable role—psychiatry focuses on interviewing, human behavior, and the doctor-patient relationship and employs a biopsychosocial, as opposed to biomedical, paradigm of health and illness (8, 9). A recent study of 704 physicians demonstrated that compared to other specialists, psychiatrists scored higher on levels of empathy (10). In part because the classic psychiatric disorders have no pathognomonic laboratory tests, interviewing and precise observation (the mental status examination) are the core of psychiatric diagnosis.
Although psychiatry is becoming more neuropsychiatrically oriented and third-party payers and psychiatrists’ employers require more productivity and limit time and numbers of visits with patients (11), psychiatrists employ psychotherapy—changing behaviors through verbal interchange (12)—much more than do other medical specialists. A 1987 national study (11, 13) noted that psychiatrists provided five times as many psychotherapy visits as did nonpsychiatrist physicians, and a 1999 survey (14) revealed that the average psychiatrist spent more time (39 minutes/visit) with his or her patients than did other physicians. The next highest average was for internists, who spent 20.7 minutes per visit.
Those who become psychiatrists may come to medical school with predispositions and skills that are different from others who become procedure-oriented specialists. For the reasons mentioned, psychiatrists must be verbally and interpersonally skillful. Consistent with this notion, medical students interested in psychiatry have scored higher than those interested in other disciplines in the verbal section (15, 16) of the Medical College Admission Test.
In addition, studies (1) indicate that compared with other students and physicians, medical students preferring or choosing psychiatry are more apt to have majored or be more interested in humanities or social sciences than natural sciences (17–20), to be politically liberal (21–23), to have attended a politically liberal college (24), to tolerate ambiguity (25–27), and to be more interested in interpersonal than technological matters (15, 28).
Psychiatrists must have knowledge and skill in general medicine sufficient to address—by screening and then treatment or referral—the general health of their patients and to consult for other specialists. But there is no intuitive reason to expect future psychiatrists to perform better than others on tests of scientific or quantitative attainment before medical school or to outscore others on measures of basic medical sciences and in medical licensing examinations. In studies of future psychiatrists (16, 17, 19), their mean science subtest scores on the Medical College Admission Test ranged broadly, from below to above the national average.
Based on these perspectives, this study was designed to test the following hypotheses: psychiatrists, compared to other physicians, would
1.
Score higher on verbal ability tests but not on science or quantitative tests taken before medical school
2.
Perform better in behavioral science examinations during medical school and in the relevant discipline in the medical licensing examinations but not on tests of general medical knowledge
3.
Be rated higher in clinical competence in psychiatry clerkships than in other clerkships
Method
Data for this study were retrieved from the database of the Jefferson Medical College’s longitudinal study of undergraduate medical education (29), which is approved by Jefferson’s institutional review board.
Participants
Study participants included 5,701 physicians (25% women) who graduated from Jefferson Medical College between 1970 and 2001. This sample represents 84% of all graduates with known specialties during the study period. There were 253 psychiatrists (32% women); 986 family physicians (30% women); 1,290 internists (21% women); 420 pediatricians (51% women); 367 obstetrician-gynecologists (39% women); 374 emergency physicians (22% women); 766 hospital-based specialists (23% women) who were defined as anesthesiologists (N=274), pathologists (N=147), and radiologists (N=345), and 1,245 in surgery and surgical subspecialties (11% women). For recent graduates still in residency, the areas of residency were used as the most likely area of future practice (30).
Measures
Physicians’ area of practice in the aforementioned specialties was the independent variable. For the dependent variables, physicians were compared on 18 variables in the following categories: 1) before medical school, 2) during medical school, and 3) medical licensing examinations during and after medical school.
For measurements of performance before medical school, the variables included scores on the Scholastic Aptitude Test’s verbal and quantitative sections taken during high school and the Medical College Admission Test. For the pre-1978 Medical College Admission Test version, we used general information, science, and verbal scores. For the 1978–1991 version, we used problem-solving and verbal scores, and for the post-1991 version, we used biological sciences and verbal reasoning scores.
For measurements of performance during medical school, the variables included grade point averages in the first and second years of medical school and grades on objective examinations in six core clerkships in the third year of medical school (family medicine, medicine, obstetrics-gynecology, pediatrics, psychiatry, and surgery). The grade point averages are based on objective examinations that have reliability coefficients that are usually in the 0.70s. Also, physicians were compared on the medical school faculty’s global ratings of clinical competence in two core third-year clerkships (medicine and psychiatry). Data support the psychometrics (e.g., predictive validity) of these clerkship ratings (31).
For measurements on medical licensing examinations during and after medical school, the variables included scores on steps 1, 2, and 3 of the United States Medical Licensing Examinations—formerly parts 1, 2, and 3 of the National Board of Medical Examiners (NBME) examinations—and scores on the psychiatry discipline of NBME part 2. We did not ascertain failure rates on the United States Medical Licensing Examinations (32), and the NBME (G.F. Dillon, NBME, personal communication) does not ascertain United States Medical Licensing Examinations failure rates by specialty.
Statistical Analysis
All dependent variables were transformed to a distribution with a mean of 100 and a standard deviation of 10 to facilitate comparisons of the magnitudes of differences. Because of differences in medical school performance (33), we controlled for the confounding effect of gender by using analysis of covariance in which gender was a covariate.
Results
The adjusted means and the summary results of statistical analyses are reported in Table 1. Before medical school, psychiatrists reached a significantly higher mean verbal score on the Scholastic Aptitude Test. Similarly, on the Medical College Admission Test’s verbal section, psychiatrists outscored their counterparts in family medicine, internal medicine, emergency medicine, obstetrics-gynecology, and surgery. On the Medical College Admission Test reading section, psychiatrists scored significantly higher than family physicians and obstetrician-gynecologists. On the Medical College Admission Test’s general information section, psychiatrists outscored physicians in family, internal, and emergency medicine; obstetrics-gynecology, and surgery. No statistically significant difference was observed between the psychiatrists and others on the Scholastic Aptitude Test’s quantitative section and on the Medical College Admission Test’s problem-solving, verbal reasoning, and biological sciences sections.
During the first year of medical school, psychiatrists did not perform as well as others (Table 1). Performance comparisons in the second year of medical school and the third-year objective examinations showed that while psychiatrists performed as well as family physicians, these two groups scored lower than their cohorts. Comparisons on global ratings of clinical competence in third-year clerkships revealed that psychiatrists obtained the highest and the lowest clinical competence ratings in psychiatry and medicine clerkships, respectively.
Regarding medical licensing examinations, in the examination of sciences basic to medicine (step 1, formerly part 1, and step 2, formerly part 2), while psychiatrists scored as well as family physicians and obstetrician-gynecologists, their mean score was lower than that for other physicians. Psychiatrists outscored all other physicians on the psychiatry and behavioral science discipline of the licensing examinations (for examinees who took the NBME part 2).
On step 3 (formerly part 3), psychiatrists’ scores fell below their step 1 and 2 scores. Also on step 3, they scored lower than other physicians, including family physicians, whose scores rose dramatically from their step 2 scores, and obstetrician-gynecologists, whose scores were identical with their step 1 and 2 scores.
Similar patterns of results were obtained in additional statistical analyses in which the findings were compared for the three decades of study, indicating that the patterns of results are relatively unchanged in different time periods. Illustrative examples include psychiatrists’ Scholastic Aptitude Test verbal scores and United States Medical Licensing Examinations step 3 scores. In 1970–1979, psychiatrists and pediatricians obtained the highest mean score on the Scholastic Aptitude Test’s verbal section (mean=106, SD=10), significantly higher than internists (mean=103, SD=10), family physicians (mean=101, SD=10), obstetrician-gynecologists (mean=101, SD=9), and surgeons (mean=100, SD=11). From 1980 to 1989, psychiatrists obtained the highest mean score (mean=104, SD=10), which differed significantly from the other specialist groups (mean scores ranged from 101 [SD=10] for pediatrics to 97 [SD=10] for obstetrics-gynecology). From 1990 to 2001, psychiatrists scored the highest (mean=104, SD=9), significantly higher than physicians in any other specialty; mean scores ranged from 100 (SD=9) for pediatricians to 97 (SD=9) for family physicians.
For United States Medical Licensing Examinations step 3, we found that from 1970 to 1979, psychiatrists scored significantly lower (mean=93, SD=10) than internists (mean=105, SD=11), emergency physicians (mean=102, SD=10), family physicians (mean=101, SD=10), pediatricians (mean=99, SD=10), surgeons (mean=99, SD=9), hospital-based specialties (mean=98, SD=10), and obstetrician-gynecologists (mean=97, SD=10).
From 1980 to 1989, psychiatrists scored significantly lower (mean=95, SD=10) than internists, family physicians (mean=102, SD=10), emergency physicians (mean=101, SD=10), pediatricians (mean=100, SD=9), and obstetrician-gynecologists (mean=99, SD=10), but the differences between psychiatrists and hospital-based specialists (mean=97, SD=11) and surgeons (mean=97, SD=9) were not significant. From 1990 to 2001, psychiatrists scored significantly lower (mean=95, SD=9) than emergency physicians (mean=102, SD=8), family physicians (mean=101, SD=10), pediatricians (mean=100, SD=9), and obstetrician-gynecologists (mean=99, SD=10), but not significantly lower than surgeons (mean=97, SD=9) and hospital-based specialists (mean=96, SD=9).
Discussion
Our findings support our first hypothesis about higher performance of psychiatrists in verbal ability and essentially equal performance in science and quantitative tests before medical school. Our second hypothesis was also confirmed, as psychiatrists performed better on psychiatry and behavioral science examinations in medical school and in medical licensing examinations. Our third hypothesis about better ratings in psychiatry clerkships was also confirmed.
Our finding that psychiatrists scored lower than other physicians on step 3 of the medical licensing examination and that family physicians, internists, and emergency physicians scored best is consistent with those reported by Gonnella and colleagues (34, 35) and Dillon and colleagues (36). Early career specialization provides less exposure to a wide variety of clinical situations (34, 35). Those who pursue residencies (e.g., family medicine, internal medicine, and emergency medicine) that expose them to a broad spectrum of general medical conditions during their first postgraduate year are expected to score higher on step 3, which assesses “delivering general medicine care to patients” (37, p. 3). Psychiatrists’ low scores on United States Medical Licensing Examinations step 3, after having performed similarly to family physicians and obstetricians on steps 1 and 2, could be partially explained on this basis.
Our findings raise several questions. What do our data about psychiatrists’ and other specialists’ academic performance mean in terms of the eventual quality of care and patient health outcomes they eventually provide? Do psychiatrists’ comparatively better performances on measures of verbal skill in their psychiatry clerkships and on the psychiatry section of step 2 of the NBME translate into better mental and general health outcomes for their patients? Also, since general medical conditions are common among psychiatric patients (38–42), do psychiatrists’ relatively low scores on an examination in general medicine (i.e., step 3) predict poorer general or mental health outcomes for their patients?
The literature on the predictive validity of standardized written medical licensing examinations in terms of the eventual quality of patient care and patient health outcomes provides very limited information (43, 44). Tamblyn (43), who wrote in 1994 that “research on the predictive validity of credentialing examinations for medical practice and patient outcome is nonexistent,” recently demonstrated among 912 family physicians in Quebec (44) the predictive validity of the Medical Council of Canada Qualifying Examination, a written examination typically taken in the final year of medical school. The physicians who scored highest on the Medical Council of Canada Qualifying Examination scored higher during their first 4–7 years of practice than their family physician colleagues on six clinical performance indicators (e.g., mammography screening rate, continuity of care, and disease-specific as opposed to symptom-specific prescription rates) known to be associated with clinical outcomes and costs of care. Of course, findings from a study of Canadian generalists do not necessarily generalize to American psychiatrists.
Our results strongly suggest a need for studies of the extent to which psychiatrists’ scores on licensing examinations are eventually associated with the outcomes—general medical as well as psychiatric—of their patient care. Without such validity studies, it is difficult to make definitive recommendations for curricula for third- and early fourth-year medical students planning psychiatric careers or for postgraduate first-year psychiatry residents over and above the recommendations (45) and guidelines (46) that already exist. It would also be valuable for the National Board of Medical Examiners to ascertain failure rates of the United States Medical Licensing Examinations step 3 by specialty. This, too, could facilitate curriculum planning.
Based on our findings, it seems reasonable to recommend that in providing career counseling to medical students planning psychiatric careers and career counseling and program planning for first-year residents that faculty advisors and program directors should be familiar with each of their trainees’ knowledge (including United States Medical Licensing Examinations data, when possible) and skills in general medicine. This familiarity should guide the planning of rotations, coursework, and study programs related to the general medical care of psychiatric patients.
A limitation of this study is that the results are from one school only and, hence, cannot automatically be generalized to all schools nationally. Nevertheless, the past 30-year track record of the study shows a consistent record of replication by the Jefferson research team and by researchers from other medical schools (47, 48).
In summary, our findings generally confirmed our expectations about psychiatrists’ performances before, during, and after medical school. More attention should be paid to the general medical education of psychiatrists.
Footnote
Received July 10, 2002; revisions received Dec. 3, 2002, and July 15, 2003; accepted Oct. 10, 2003. From the Department of Psychiatry and Behavioral Sciences, Rosalind Franklin University of Medicine and Science/Chicago Medical School; and the Department of Psychiatry and Human Behavior, the Center for Research in Medical Education and Health Care, Jefferson Medical College, Thomas Jefferson University, Philadelphia. Address reprint requests to Dr. Sierles, Rosalind Franklin University, 3333 Green Bay Rd., North Chicago, IL 60064; [email protected] (e-mail). The authors thank Nutan Atre-Vaidya, M.D., and Michael Alan Taylor, M.D., for their suggestions.
References
1.
Sierles FS, Taylor MA: Decline of US medical student career choice of psychiatry and what to do about it. Am J Psychiatry 1995; 152:1416–1426
2.
Feifel D, Moutiera CY, Swerdlow NR: Attitudes toward psychiatry as a prospective career among students entering medical school. Am J Psychiatry 1999; 156:1397–1402
3.
Kassebaum DG, Szenas PL: Factors influencing the specialty choice of 1993 medical school graduates. Acad Med 1994; 69:164–170
4.
Bland CJ, Meurer LN, Maldonado G: Determinants of primary care specialty choice: a non-statistical meta-analysis of the literature. Acad Med 1995; 70:620–641
5.
Reed VA, Jernstedt C, Reber ES: Understanding and improving medical student specialty choice: a synthesis of the literature using decision theory as a referent. Teach Learn Med 2001; 13:117–129
6.
Christodoulou GN, Lykousras LP, Mountaokalakis T, Voulgari A, Stefanis CN: Personalities of psychiatric versus other medical trainees. J Nerv Ment Dis 1995; 183:330–340
7.
Weissman SH, Haynes RA, Killian CD, Robinowitz C: A model to determine the influence of medical school on students’ career choices: psychiatry, a case study. Acad Med 1994; 69:58–59
8.
Engel GL: The clinical application of the biopsychosocial model. Am J Psychiatry 1980; 137:535–544
9.
Hojat M, Samuel S, Thompson TL: Searching for the lost key under the light of biomedicine: a triangular biopsychosocial paradigm may cast additional light on medical education, research and patient care, in Medicine and Health Care Into the Twenty-First Century. Edited by Majumdar SK, Rosenfeld LM, Nash DB, Audet AM. Easton, Pa, Pennsylvania Academy of Science, 1995, pp 310–325
10.
Hojat M, Gonnella JS, Nasca TJ, Mangione S, Vergare M, Magee M: Physician empathy: definition, components, measurement, and relationship to gender and specialty. Am J Psychiatry 2002; 159:1563–1569
11.
Olfson M, Marcus SC, Druss B, Pincus HA: National trends in the use of outpatient psychotherapy. Am J Psychiatry 2002; 159:1914–1920
12.
Ursano RJ, Silberman EK: Psychoanalysis, psychoanalytic psychotherapy and supportive therapy, in American Psychiatric Press Textbook of Psychiatry. Edited by Hales RE, Yudofsky SC, Talbott JA. Washington, DC, American Psychiatric Press, 1994, pp 1035–1060
13.
Olfson M, Pincus HA: Outpatient psychotherapy in the United States, I: volume, costs, and user characteristics. Am J Psychiatry 1994; 151:1281–1288
14.
National Ambulatory Medical Care Survey, 1999: Vital signs: average time spent with physicians during office visits. Clin Psychiatry News, September 2002, p 1
15.
Paiva REA, Haley HB: Intellectual, personality and environmental factors in career specialty preferences. J Med Educ 1971; 46:281–288
16.
Peterson OL, Lyden FJ, Geiger J, Colton T: Appraisal of medical students’ abilities as related to training and careers after graduation. N Engl J Med 1963; 269:1174–1182
17.
Gough HG: Some predictive implications of premedical scientific competence and preferences. J Med Educ 1978; 53:291–300
18.
Weissman S, Bashook PG: An analysis of changing patterns of American medical student career choice of psychiatry. J Psychiatr Educ 1980; 4:225–234
19.
Perry S, Cella DF, Klar H, Simring S, Oldham J, Tardiff K: Medical students selecting psychiatric careers—a distinct breed. J Psychiatr Educ 1988; 12:170–181
20.
Zeleznik C, Hojat M, Veloski JJ: Baccalaureate preparation for medical school: does type of degree make a difference? J Med Educ 1977; 52:548–554
21.
Schumacher CF: Interest and personality factors as related to choice of medical school career. J Med Educ 1963; 38:932–942
22.
Sharaf MR, Schneider P, Kantor D: Psychiatric interest and its correlates among medical students. Psychiatry 1968; 31:150–160
23.
Rogow AA: The Psychiatrists. New York, GP Putnam’s Sons, 1970
24.
Funkenstein DH: The problem of increasing the number of psychiatrists. Am J Psychiatry 1965; 121:852–863
25.
Matteson MT, Smith SV: Selection of medical specialties: preferences versus choices. J Med Educ 1968; 2:195–199
26.
Davies BM, Mowbray RM: Medical students’ personality and academic achievement. Br J Med Educ 1968; 2:195–199
27.
Walton HJ: Personality correlates of a career interest in psychiatry. Br J Psychiatry 1969; 115:211–219
28.
Weissman SH, Bashook PG: A view of the prospective child psychiatrist. Am J Psychiatry 1986; 143:722–727
29.
Hojat M, Gonnella JS, Veloski JJ, Erdmann JB: Jefferson Medical College Longitudinal Study: a prototype for evaluation of changes. Educ Health 1996; 9:99–113
30.
Rabinowitz HK, Hojat M, Veloski JJ, Rattner SL, Robeson MR, Xu G: Who is a generalist? an analysis of whether physicians trained as generalists practice as generalists. Eval Health Prof 1999; 22:497–502
31.
Callahan C, Erdmann JB, Hojat M, Veloski JJ, Rattner S, Nasca TJH, Gonnella JS: Validity of faculty ratings of students’ clinical competence in core clerkships in relation to scores on licensing examinations and supervisors’ ratings in residency. Acad Med 2000; 75(suppl 10):S71-S73
32.
Verhulst SJ, Distlehorst CH: Examination of post-response bias in major residency followup study, in Assessment Measures in Medical School, Residency and Practice. Edited by Gonnella JS. New York, Springer, 1993, pp 121–127
33.
Hojat M, Robeson MR, Veloski JJ, Blacklow RS, Xu G, Gonnella JS: Gender comparisons prior to, during and after medical school: using two decades of longitudinal data at Jefferson Medical College. Eval Health Prof 1994; 170:290–306
34.
Gonnella JS, Hojat M, Erdmann JB, Veloski JJ: The impact of early career specialization on licensing requirements and related educational implications. Adv Heal Sci Edu 1997; 1:125–139
35.
Gonnella JS, Veloski JJ: The impact of early specialization on clinical competence of residents. N Engl J Med 1982; 306:275–277
36.
Dillon GF, Henzel TR, Walsh WP: The impact of postgraduate training on an examination for medical licensure, in Advances in Medical Education. Edited by Scherpbier AJJA, van der Vleuten CPM, Rethans JJ, van der Steeg AFW. Dordrecht, the Netherlands, Kluwer Academic, 1997, pp 146–148
37.
National Board of Medical Examiners: Federation of State Boards of the United States and the National Board of Medical Examiners, Step 3: General Instructions, Content Description, and Sample Items. Philadelphia, NBME, 1995
38.
Koran LM, Sox HC Jr, Marton KI, Moltzen S, Sox CH, Kraemer HC, Imai K, Kelsey TG, Rose TG Jr, Levin LDC, Chandra S: Medical evaluations of psychiatric patients: results in a state mental health system. Arch Gen Psychiatry 1989; 46:733–740
39.
Druss BG, Rosenheck RA: Use of medical services by patients with psychiatric disorders. Psychosomatics 1997; 38:451–458
40.
Cradock-O’Leary J, Young AS, Yano E, Wang M, Lee MC: Use of general medical services by patients with psychiatric disorders. Psychiatr Serv 2002; 53:874–878
41.
Hall RCW, Popkin MK, Devaul RA, Faillace LA, Stickney SK: Physical illness presenting as psychiatric disease. Arch Gen Psychiatry 1978; 35:1315–1320
42.
Hall RCW, Gardner ER, Stickney SK, Lecann AF, Popkin MK: Physical illness presenting as psychiatric disease, II: analysis of state hospital inpatient population. Arch Gen Psychiatry 1980; 37:989–995
43.
Tamblyn R: Is the public being protected? prevention of suboptimal medical practice through training programs and credentialing examinations. Eval Health Prof 1994; 17:198–241
44.
Tamblyn R, Abrahamowicz M, Dauphinee WD, Hanley JA, Norcini J, Girard N, Grand’Maison P, Brailovsky C. Association between licensure examination scores and practice in primary care. JAMA 2002; 23:3019–3026
45.
American Psychiatric Association Committee on Medical Student Education and Committee on Graduate Education: Careers in Psychiatry. Arlington, Va, American Psychiatric Association, Oct 2000, pp 3, 4
46.
Accreditation Council for Graduate Medical Education: Program Requirements for Residency Training in Psychiatry. Washington, DC, Association of American Medical Colleges, Jan 2001
47.
Gonnella JS, Hojat M, Erdmann JB, Veloski JJ: Assessment Measures in Medical School, Residency and Practice: The Connections. New York, Springer, 1993
48.
Gonnella JS, Hojat M, Erdmann JB, Veloski JJ: What have we learned and where do we go from here? Acad Med 1993; 68(suppl 2):S79-S87
Information & Authors
Information
Published In
History
Published online: 1 August 2004
Published in print: August 2004
Authors
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBGet Access
Login options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).