Childhood Videotaped Social and Neuromotor Precursors of Schizophrenia: A Prospective Investigation
Publication: American Journal of Psychiatry
Abstract
OBJECTIVE: The authors examined videotaped behaviors of children who developed schizophrenia as adults and of comparison subjects to disclose possible social and neuromotor deficits foreshadowing later development of schizophrenia. METHOD: In 1972, a sample of 265 11–13-year-old Danish children were filmed under standardized conditions while they were eating lunch. The examination was part of a larger study investigating early signs of schizophrenia spectrum disorders. Many of the subjects had a parent with schizophrenia, leaving them at high risk for developing a schizophrenia spectrum disorder. In 1991, adult psychiatric outcome data were obtained for 91.3% (N=242). This study systematically analyzed the videotapes to determine whether the children who developed schizophrenia as adults evidenced greater social and/or neuromotor deficits than children who did not develop a psychiatric disorder and children who developed other psychiatric disorders. RESULTS: The findings from this study suggest that the brief videotaped footage of children eating lunch was able to discriminate between the individuals who later developed schizophrenia and those who did not. Specifically, the preschizophrenia children evidenced differences on measures of sociability and general neuromotor functioning (among boys) from the children who developed other psychiatric disorders and the children who did not develop a psychiatric disorder. CONCLUSIONS: Social and neuromotor deficits specific to children who develop schizophrenia in adulthood provide further support for a neurodevelopmental hypothesis of schizophrenia.
The majority of individuals with schizophrenia manifest the illness in the second or third decade of life (1), yet subclinical signs of neuropathology are already evident during adolescence (2) and possibly as early as birth and infancy (3–5). Although some research suggests that the early signs of schizophrenia are nonspecific and that no one sign predicts schizophrenia uniquely and effectively (6), other studies indicate that both social and neuromotor abnormalities in childhood are reliable developmental precursors of the disorder.
Social deficits exhibited before the onset of schizophrenia suggest that interpersonal difficulties precede recognizable psychotic symptoms. A unique study by Walker and colleagues (7) evaluated childhood home movies of schizophrenia patients and comparison subjects. The authors reported that girls who later developed schizophrenia showed fewer expressions of joy than did same-sex comparison subjects from infancy through adolescence; preschizophrenia boys showed nonsignificantly more negative expressions in preadolescence and early adolescence.
Longitudinal study of “high-risk” individuals (those having at least one parent with schizophrenia) provides an alternative opportunity to view the developmental course of schizophrenia. Several investigations have followed high-risk subjects through the age of risk, providing information regarding premorbid functioning and adult diagnostic outcome. In an investigation in the Copenhagen High-Risk Study (8), teachers noted more negative-type behavior (e.g., having difficulty making friends or being passive, socially unresponsive to peers, or unresponsive to praise or punishment from the teacher) among high-risk subjects who later succumbed to predominately negative-symptom schizophrenia than among those who developed predominately positive-symptom schizophrenia. In a study from the New York High-Risk Project incorporating adult diagnostic information, Amminger and colleagues (9) found more behavior problems in childhood among adults who developed a schizophrenia-related psychosis than among adults who developed anxiety or affective disorders, adults who developed substance abuse, or adults with no disorders. Although limited in number, these studies following high-risk subjects through adulthood provide compelling evidence for the developmental nature of social deficits in schizophrenia.
In addition to social deficits, researchers have also investigated neuromotor deficits in childhood preceding the development of schizophrenia. Walker et al. (10) evaluated the neuromotor functioning of preschizophrenia patients through the naturalistic observation of home movies filmed by parents during the patients’ childhoods. The results suggested more neurological soft signs (abnormalities without a known area of localization in the brain) and poorer motor skills than in the comparison groups. In a separate study of adolescents diagnosed with schizotypal personality disorder, Walker and colleagues (11) found a higher rate of involuntary movements than in adolescents with other diagnoses and normal comparison subjects.
Several studies investigating neuromotor functioning have followed subjects from childhood through the age of risk. Investigating a large birth cohort, Rosso and colleagues (12) found neuromotor deficits in children ages 4 and 7 who later developed schizophrenia. More recent follow-up studies from the New York High-Risk Project indicated that neuromotor deficits predicted schizophrenia spectrum disorders in adulthood (13).
Collectively, the studies reviewed suggest that schizophrenia is a longitudinal syndrome, with premorbid signs of social and neuromotor deficits already present during childhood. The current study was intended to replicate and extend earlier findings by examining observable behavior from childhood, using a sample of children ages 11–13 who were observed in 1972. Videotape records, the primary source of data in this study, were uniformly and systematically obtained in 1972 during preadolescence, before any of the subjects developed a psychiatric disorder. We now have DSM-III-R adult outcome data for these subjects based on structured psychiatric interviews and psychiatric hospital records obtained in 1992.
Given the literature supporting differences in neuromotor and social functioning between children who do and do not develop schizophrenia in adulthood, it was hypothesized that children who later developed schizophrenia would manifest more social and neuromotor deficits than 1) children who did not develop a mental illness and 2) children who developed a nonpsychotic disorder.
Method
The current study was part of a larger longitudinal high-risk project investigating the precursors of schizophrenia. We have described the design of the study, the subject characteristics, and the premorbid and follow-up diagnoses in greater detail elsewhere (14).
Diagnostic Methods: The Parents
The present study investigated high-risk children in a major perinatal cohort of 9,182 deliveries in Copenhagen during 1959–1961. In order to identify high-risk children and comparison subjects, in 1969 the lifetime record of psychiatric admissions was checked through the Danish psychiatric record for the 18,012 parents of the birth cohort.
Subjects were drawn from a Danish birth cohort consisting of all children born between Sept. 1, 1959, and Dec. 31, 1961, at Rigshospitalet in Copenhagen (15). In 1972, a sample of 265 11–13-year-old children from this cohort was intensively examined (16). Of this sample, 242, falling into three groups, were available for follow-up examinations at ages 31–33. All children with a mother or father who had a psychiatric hospital diagnosis of schizophrenia comprised the first group (N=81). One group of comparison subjects consisted of 87 children who had a parent with a psychiatric record of hospitalization for a disorder other than schizophrenia. The remaining 74 subjects were comparison subjects drawn from the original cohort with no parental records of psychiatric hospitalization. An effort was made to match these subjects on the basis of gender, social class, parent’s age, and mother’s marital status.
Diagnostic Methods: The Offspring
In 1992, when the subjects were between 31 and 33 years of age, their psychiatric status was ascertained. A psychiatrist administered two structured clinical psychiatric interviews, the Structured Clinical Interview for DSM-III-R (SCID) (17) and the psychosis section of the Present State Examination (18). In addition, Danish psychiatric hospital records of the subjects were examined. A detailed coding scheme yielded DSM-III-R diagnoses. On the basis of the interviews and/or hospital records, we obtained adult diagnostic outcomes for 242 of the 265 subjects (91.3% successful follow-up after 20 years; 23 subjects had neither an interview nor hospital records) (Table 1). After complete description of the study to the subjects, written informed consent was obtained.
1972 Videotaping Procedures
In 1972, when the subjects were 11–13 years of age, they came to a laboratory at Kommunehospitalet in Copenhagen for a 1-day assessment. At noon on the day of the assessment, the researchers provided the subjects with a lunchtime meal in a specially designated room. The videotaping took place during lunch under highly standard conditions, by camera operators blind to psychiatric risk status and, obviously, blind to diagnostic outcome 20 years later.
Generally, two randomly paired children were examined each day. In the case of 41 subjects, lunch partners were not available. When two children were present, they sat on the two sides of the corner of a table facing the tripod-mounted video camera. The children, who were of the same age level, did not know each other before entering this situation. Some pairs of children began a conversation during their meal. The meal consisted of Danish open-face sandwiches. These sandwiches were constructed in layers and required some motor skills.
The camera was an Akai VT-100R black-and-white, 0.25-inch, reel-to-reel video and audio recorder new to the market that year, 1972 (Akai, Tokyo). The lighting, distance of the camera from the subjects, and sound recording level were not altered during the course of the study. During each recording, the camera angle was changed from 1) an initial focus on both children (2 minutes) to 2) a close-up of the subject sitting to the left (1 minute), 3) a close-up of the subject sitting to the right (1 minute), and a final focus on both children (1 minute). These changes were done with the same timing for all of the subject dyads. The camera operator made an effort to ensure that variations in the subjects’ videotaped behavior were not partly ascribable to variations in situational factors. These videotapes were transcribed onto videocassettes.
Training in Coding Procedures
We established a coding scheme for the videotapes sensitive enough to detect interindividual variability among the subjects. The coding scheme yielded 10 continuously distributed variables similar to variables established in previous video studies investigating early signs of schizophrenia (7, 10). The number of instances of each behavior was noted. Figure 1 contains the coding form and coding instructions.
A research assistant was trained in the coding system. Together, this research assistant and one of us (J.S.) trained another research assistant. Training involved specific instructions followed by evaluation and discussion of independent work. Questions from the raters regarding the independent coding were addressed in regular training meetings to avoid drift; however, all coding decisions were made independently. All raters, including the principal investigator, were completely blind to adult diagnostic outcome.
Statistical Analysis
Coding of the videotapes resulted in 10 continuous measures of behavior. We analyzed the dimensionality of the 10 coding scores by using a principal components factor analysis. The factor analysis yielded three factors with an eigenvalue greater than 1. Consequently, we rotated the three factors using a varimax rotation procedure. The rotated solution yielded three interpretable factors: sociability, involuntary hand movements, and general neuromotor signs. The sociability factor accounted for 26.8% of the item variance, the involuntary hand movements factor accounted for 18.1%, and the general neuromotor factor accounted for 15.7%. One item, involuntary facial movements, loaded on both the involuntary hand movements factor and the general neuromotor factor at a value above 0.400 but did not differ by more than 0.100 in magnitude between factors. It did, however, load more strongly on the general neuromotor factor. Another item, other abnormal movements, loaded on both the involuntary hand movements factor and the general neuromotor factor at a value above 0.400 but loaded significantly higher on the general neuromotor factor. As a result, we assigned involuntary facial movements and other abnormal movements to the general neuromotor factor, leaving the second factor purely involuntary hand movements (Table 2).
To create scales representing the three factors, we first standardized the scores for the behavioral variables, setting them to equal means and standard deviations. We then summed the scores for all of the variables that loaded on each factor. To account for missing data, we divided the summed scores by the number of variables available per subject (missing data were rare).
To determine the level of interrater agreement between coders, a subset of subjects (N=50, 32.7% of the 153 subjects included in the analyses) was evaluated separately by the two raters. Two-way random, absolute agreement intraclass correlation coefficients (ICCs) for the three scales were 0.91 for sociability, 0.65 for involuntary hand movements, and 0.65 for general neuromotor signs. The value of Cronbach’s alpha for each scale was 0.83 for sociability, 0.73 for involuntary hand movements, and 0.52 for general neuromotor signs.
Given the limited number of schizophrenia subjects and the uniqueness of these data, we were concerned about type II error (not detecting a significant difference when a significant difference exists). Consequently, rather than employing omnibus tests to assess for overall group differences and correcting significance levels to reflect multiple tests, we used uncorrected-planned-comparison t tests as the primary statistical analyses assessing group differences on the scales for sociability, involuntary hand movements, and general neuromotor signs. While these practices minimized the chances of type II error, they increased the risk for type I error (detecting a significant difference when one does not exist). All statistical tests were two-tailed, and alpha levels were set at p=0.05.
Results
Missing Data
Of the 265 original subjects videotaped in 1972, 23 either died or refused to participate. Additionally, 48 subjects did not have videotapes. As mentioned previously, the camcorder used was among the first personal video recording devices, using reel-to-reel technology. When the reel-to-reel tapes were converted to modern standards, a reel from some may have been missing, accounting for the 48 subjects without footage. The subjects without videotapes, however, did not systematically differ from the subjects with videotapes in terms of diagnostic outcome, sex, or psychiatric risk status (Table 3). Because of equipment failure, one subject who was given ratings for raised elbows, nystagmus-like eye movements, involuntary facial movements, involuntary hand movements, and other abnormal movements was not rated for smiles, laughs, vocalization responses, and initiated vocalizations.
As discussed in the Method section, every effort was made to standardize the conditions of the videotaping. Not every subject, however, had a lunch partner (41 subjects were without a partner). Although two subjects were invited for testing every day, occasionally only one came. The subjects without partners did not systematically differ from the subjects with partners in terms of diagnostic outcome, psychiatric risk status, or sex (Table 3). However, t tests revealed significant differences between those with and without partners on all three scales (Table 4). Because of these differences between partnered and nonpartnered subjects and the limited number of nonpartnered subjects, we excluded the nonpartnered subjects from further analyses. In total, we had 153 subjects with complete scores on the scales for general neuromotor signs and involuntary hand movements and 152 subjects with complete sociability scale scores (Table 3). In unreported analyses, the results for the partnered and nonpartnered subjects combined resembled the results for the partnered-only subjects.
We tested for potential confounding effects of socioeconomic status and sex on all three scales. Socioeconomic status at rearing did not significantly correlate with the scores on any of the three scales. Similarly, males and females did not significantly differ on the scales for sociability and involuntary hand movements. The males did, however, show higher scores on the general neuromotor scale than the females (males, N=81: mean=–0.02, SD=0.46; females, N=72: mean=–0.22, SD=0.40) (t=2.81, df=151, p<0.01).
Relation of Adult Psychiatric Outcome to Videotaped Childhood Behaviors
The mean scale scores by diagnostic group are presented in Table 5. Comparisons of mean scores on the sociability scale revealed that the children who later developed schizophrenia had lower scores than did the children who did not develop mental illness (t=4.48, df=87.8, p<0.001). These two groups did not significantly differ from each other on the scale for involuntary hand movements (t=–0.74, df=99, p=0.47) or the general neuromotor scale (t=1.94, df=99, p=0.06). When males and females were analyzed separately, the scores for general neuromotor signs did not significantly differentiate between an outcome of schizophrenia and an outcome of no mental illness (males: t=1.93, df=49, p=0.06; females: t=0.85, df=48, p=0.41). Although the findings were not significant at the p=0.05 level, the difference for male subjects approached significance.
In addition to the main hypotheses regarding the behavior of the preschizophrenia subjects in relation to that of the normal comparison subjects, we were also interested in the specificity of these effects to children who would later develop schizophrenia as compared to children who would later develop other psychopathology. We repeated the preceding analyses but compared subjects with an outcome of schizophrenia to those with an outcome of some other psychopathology on all three scales. Compared to the subjects who developed nonschizophrenia psychopathology, the schizophrenia group had significantly lower sociability scale scores (t=2.51, df=51.9, p=0.02). A higher general neuromotor scale score did not significantly differentiate an outcome of schizophrenia from other psychopathology (t=2.05, df=10.3, p=0.07). Among males, the score on the general neuromotor scale significantly differed between the schizophrenia group and the group with other psychopathology (t=3.36, df=30, p=0.002). Among females, however, the general neuromotor scale failed to significantly differentiate an outcome of schizophrenia from other psychopathology (t=1.19, df=20, p=0.25). The involuntary hand movements scale did not significantly differentiate the schizophrenia group from the other psychopathology group (t=–0.93, df=52, p=0.36).
Discussion
Overall, the observations from these standardized videotapes suggest differences between children who do and who do not develop schizophrenia in adulthood. Specifically, the schizophrenia patients showed premorbid social deficits, and male schizophrenia patients showed a tendency toward premorbid neuromotor deficits (as measured by the general neuromotor scale but not the scale for involuntary hand movements) in relation to comparison subjects with no psychiatric diagnosis in adulthood. Furthermore, the findings from this study suggest impairments unique to schizophrenia. Differences were observed between the subjects with a schizophrenia outcome and those with no mental illness and between those with a schizophrenia outcome and those with an outcome of other psychopathology. We did not detect significant differences between outcome groups on the involuntary hand movements scale.
Overall, boys showed higher scores on the scale for general neuromotor signs than did girls. Similarly, Rieder and Nichols (19) and Marcus and colleagues (20) found more neuromotor impairment among high-risk boys than high-risk girls. Differences between boys and girls on the neuromotor scale may suggest differences in vulnerability to disruption of the early neural development responsible for neuromotor functioning.
The current investigation has several notable strengths. All measures for this study were prospective. The data were gathered 20 to 33 years before diagnosis, thus reducing the likelihood that the adult clinical picture influenced measurement of early childhood behaviors. We assessed the subjects at ages 11–13, before any overt signs of mental illness. In 1972, when the tapes were created, the researchers were blind to psychiatric risk status and, obviously, blind to diagnostic outcome. Additionally, all of the current research team members (raters, data entry assistants, statisticians) were blind to the adult diagnostic outcomes until after the coding of the videotapes.
The high-risk subjects from this study were selected from a Danish birth cohort consisting of all children born between Sept. 1, 1959, and Dec. 31, 1961, at Rigshospitalet in Copenhagen. Every child having a parent diagnosed with schizophrenia was included in our sample. Thus, rather than being selected through various means, the 81 identified high-risk subjects were representative of high-risk children in Denmark. Therefore, the 16 patients with schizophrenia in this sample are representative of genetically at-risk Danish children born around 1960 who later developed schizophrenia.
Previous work discerning social and neuromotor deficits in premorbid schizophrenia suffers from lack of standardized conditions (7, 10). The protocol for this project, however, dictated strict standardization. The attention to a uniform and systematic protocol reduced measurement error.
This study also suffered from notable limitations. The 10 variables measuring the observed behaviors from the videotapes yielded three scales. Agreement between raters ranged from high for the sociability scale (ICC=0.91) to moderate for the involuntary hand movements scale (ICC=0.65) and the general neuromotor scale (ICC=0.65). The two neuromotor scales contained items that were more ambiguous. Subtle distinctions between behaviors likely resulted in less agreement between raters.
Measurement of internal consistency varied across the scales as well. The scale for involuntary hand movements showed moderate internal consistency (alpha=0.73). The general neuromotor scale showed only modest internal consistency (alpha=0.52). Low internal consistency may result from the low number of items in the scales and low behavioral frequencies of each item. Poorer reliability suggests that a scale might not measure a single factor and might contribute to a failure to reject the null hypothesis, particularly when the sample size is small. Despite these limitations, however, we still found the hypothesized significant effects on scores on the general neuromotor scale among boys.
This study suffers from sizable numbers of missing data. Of the 242 subjects with adult follow-up information, 48 subjects did not have videotapes. The subjects with and without tapes did not seem to systematically differ in terms of adult diagnostic outcome, sex, or psychiatric risk status. We therefore assume that the subjects we did have were representative of the original study sample. Furthermore, we also excluded 41 subjects who did not have partners. While the partnered and nonpartnered subjects differed on the scales, they did not significantly differ in terms of demographic characteristics. Most important, the findings from analyses of only the subjects with partners were similar to the results of tests on the combined group of subjects with and without partners.
We based our results on a relatively small number of patients with schizophrenia (N=10). Additionally, because the scores on the general neuromotor scale differed by sex, we analyzed boys and girls separately, further reducing our sample of patients. Our analyses may have lacked the power to detect significant differences and reduce our confidence in interpreting null findings as evidence that significant differences do not exist. Furthermore, significant differences observed between groups with small sample sizes should be viewed with caution.
The results from this study suggest that brief videotaped footage of children eating lunch can discriminate between individuals who later develop schizophrenia and individuals who do not. In particular, the children who developed schizophrenia in adulthood showed deficits on measures of sociability relative to children with an outcome of no mental illness. Boys who later developed schizophrenia showed nonsignificantly greater impairment on a measure of general neuromotor functioning than did boys who had an outcome of no mental illness. Additionally, the findings appear specific to schizophrenia, as children who later developed schizophrenia showed deficits in sociability and differences on a measure of neuromotor functioning (among boys) compared to children who developed other, nonpsychotic, psychopathology.
Figure 1. Form and Instructions for Coding Possible Precursors of Schizophrenia in Videotapes of Children in Naturalistic Settingsa
aThe form with instructions (in blue) was used for training. The instructions did not appear on the form used for coding of study subjects.
Footnote
Received Oct. 7, 2003; accepted March 4, 2004. From the Department of Psychology, University of Hawaii at Manoa; the Department of Psychology, Emory University, Atlanta; the Institute of Preventive Medicine, Copenhagen University Hospital; and the Social Science Research Institute, University of Southern California, Los Angeles. Address reprint requests to Dr. Schiffman, Department of Psychology, University of Hawaii at Manoa, Gartley Hall 110, 2430 Campus Rd., Honolulu, HI 96822-2216; [email protected] (e-mail). Supported by NIMH Research Scientist Award 5KO5 MH-00619 to Dr. Mednick and by NIMH grant MH-37692.
References
1.
Torrey EF: The epidemiology of schizophrenia: questions needing answers, in Schizophrenia: Scientific Progress. Edited by Schulz S, Tamminga C. New York, Oxford University Press, 1989, pp 45–51
2.
Schulz SC, Koller MM, Kishore PR, Hamer RR, Gehl JJ, Friedel RO: Ventricular enlargement in teenage patients with schizophrenia spectrum disorder. Am J Psychiatry 1983; 140:1592–1595
3.
Cannon TD, Mednick SA, Parnas J: Genetic and perinatal determinants of structural brain deficits in schizophrenia. Arch Gen Psychiatry 1989; 46:883–889
4.
Dykes KL, Mednick SA, Machon R, Praestholm J, Parnas J: Adult third ventricle width and infant behavioral arousal in groups at high and low risk for schizophrenia. Schizophr Res 1992; 7:13–18
5.
Walker EF: Developmentally moderated expressions of the neuropathology underlying schizophrenia. Schizophr Bull 1994; 20:453–480
6.
Jones PB, Tarrant CJ: Developmental precursors and biological markers for schizophrenia and affective disorders: specificity and public health implications. Eur Arch Psychiatry Clin Neurosci 2000; 250:286–291
7.
Walker EF, Grimes KE, Davis DM, Smith AJ: Childhood precursors of schizophrenia: facial expressions of emotion. Am J Psychiatry 1993; 150:1654–1660
8.
Cannon TD, Mednick SA, Parnas J: Antecedents of predominantly negative- and predominantly positive-symptom schizophrenia in a high-risk population. Arch Gen Psychiatry 1990; 47:622–632
9.
Amminger GP, Pape S, Rock D, Roberts SA, Ott SL, Squires-Wheeler E, Kestenbaum C, Erlenmeyer-Kimling L: Relationship between childhood behavioral disturbance and later schizophrenia in the New York High-Risk Project. Am J Psychiatry 1999; 156:525–530
10.
Walker EF, Savoie T, Davis D: Neuromotor precursors of schizophrenia. Schizophr Bull 1994; 20:441–451
11.
Walker EF, Lewis N, Loewy R, Palyo S: Motor dysfunction and risk for schizophrenia. Dev Psychopathol 1999; 11:509–523
12.
Rosso IM, Bearden CE, Hollister JM, Gasperoni TL, Sanchez LE, Hadley T, Cannon TD: Childhood neuromotor dysfunction in schizophrenia patients and their unaffected siblings: a prospective cohort study. Schizophr Bull 2000; 26:367–378
13.
Erlenmeyer-Kimling L, Rock D, Roberts SA, Janal M, Kestenbaum C, Cornblatt B, Adamo UH, Gottesman II: Attention, memory, and motor skills as childhood predictors of schizophrenia-related psychoses: the New York High-Risk Project. Am J Psychiatry 2000; 157:1416–1422
14.
Schiffman J, Ekstrom M, LaBrie J, Schulsinger F, Sorensen H, Mednick S: Minor physical anomalies and schizophrenia spectrum disorders: a prospective investigation. Am J Psychiatry 2002; 159:238–243
15.
Zachau-Chistiansen B, Ross EM: Babies: Human Development During the First Year. New York, John Wiley & Sons, 1975
16.
Mednick SA, Mura E, Schulsinger F, Mednick B: Perinatal conditions and infant development in children with schizophrenic parents. Soc Biol 1971; 18(Sept suppl):S103-S113
17.
Spitzer RL, Williams JBW, Gibbon M, First MB: User’s Guide for the Structured Clinical Interview for DSM-III-R (SCID). Washington, DC, American Psychiatric Press, 1990
18.
Wing JK, Cooper JE, Sartorius N: The Measurement and Classification of Psychiatric Symptoms: An Instruction Manual for the PSE and CATEGO Program. New York, Cambridge University Press, 1974
19.
Rieder RO, Nichols PL: Offspring of schizophrenics, III: hyperactivity and neurological soft signs. Arch Gen Psychiatry 1979; 36:665–674
20.
Marcus J, Hans SL, Auerbach J, Auerbach AG: Children at risk of schizophrenia: the Jerusalem Infant Development Study, II: neurobehavioral deficits at school age. Arch Gen Psychiatry 1993; 50:797–809
Information & Authors
Information
Published In
History
Published in print: November 2004
Published online: 28 January 2015
Authors
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBGet Access
Login options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).