Three-Year Predictive Validity of DSM-IV Attention Deficit Hyperactivity Disorder in Children Diagnosed at 4–6 Years of Age
Publication: American Journal of Psychiatry
Abstract
OBJECTIVE: Predictive validity is a fundamental consideration in evaluating the DSM-IV diagnostic criteria for attention deficit hyperactivity disorder (ADHD), particularly for younger children. METHOD: The authors conducted four annual assessments of ADHD and functional impairment using multiple informants in 255 probands and matched comparison children who were 4–6 years old in wave 1. RESULTS: Nearly all children who met full criteria for ADHD in wave 1 met full criteria for ADHD over the next 3 years and continued to display marked functional impairment relative to comparison children, even when intelligence, co-occurring psychopathology, and demographic characteristics were controlled. CONCLUSIONS: These findings support the validity of the DSM-IV diagnosis of ADHD in younger children by demonstrating that the symptoms and associated impairment are likely to persist well into elementary school.
DSM-IV criteria for attention deficit hyperactivity disorder (ADHD) were based on reviews of research (1, 2) and a field trial (3, 4), but the paucity of preschool children in research studies raises concerns about the validity of the diagnosis at younger ages. It is particularly important to study the validity of ADHD in younger children because of recent increases in the use of psychoactive medication among preschoolers (5, 6). Because all treatments confer some risk, it is essential to know if ADHD is associated with significant and persisting functional impairment in younger children to determine if the risks inherent in treating are less than the risks inherent in not treating (7).
In 1995, we initiated a study of the validity of ADHD in children first diagnosed at 4–6 years of age (8). The diagnosis showed strong concurrent validity: children who met DSM-IV symptom criteria for ADHD exhibited greater social and academic impairment, more placements in special education, and more unintentional injuries in wave 1 than comparison children, even when demographic factors, intelligence, and co-occurring conduct and internalizing problems were controlled (8). A similar study (9) found that preschool children who met symptom criteria for ADHD exhibited more concurrent social skills problems than comparison children.
It is clear that the diagnosis of ADHD in elementary school predicts continuing symptoms and impairment (10–13). There are currently no published data on the persistence of impairment in younger children diagnosed with ADHD, but two longitudinal studies have shown that preschoolers rated as highly hyperactive showed persistent symptoms over time (14, 15). In the present study, we determined if ADHD has predictive validity by measuring if 4–6-year-old children who meet full diagnostic criteria for ADHD continued to exhibit symptoms and impairment over time. To avoid circularity, the impairment measures assessed in the three waves after the initial diagnosis of ADHD were not used in our operational definition of cross-situational impairment in making the initial diagnosis.
Method
Participants
Two cohorts of 3.8–7.0-year-old children were recruited in consecutive years at two sites. In Chicago, child psychiatry clinic attendees with complaints of ADHD were recruited as probands. In Pittsburgh, 42.4% of ADHD probands were recruited from a child psychiatry clinic, with the remainder recruited through advertisements. Probands recruited from clinics or advertisements did not differ significantly on demographic or impairment measures (8).
All participants lived with their biological mother and were enrolled in structured educational programs: 36% in preschool, 43% in kindergarten, 21% in first grade, and 1% in second grade. Five (two in Pittsburgh, three in Chicago) potential probands were excluded because they had previously received clinical diagnoses of pervasive developmental disorder, mental retardation, or seizure disorder. In addition, comparison children were recruited from the same schools as probands or from schools in similar neighborhoods. Comparison children had never been referred for mental health problems but were not excluded if they met criteria for a mental disorder other than ADHD. Comparison children were selected to approximately match probands in terms of gender, ethnicity, and age. Of the 310 eligible participants, 259 parents (120 in Chicago, 139 in Pittsburgh) gave written informed consent after the study was explained; all children gave oral assent.
Psychoactive medications taken by the children were grouped into three classes: stimulants, antihypertensives (clonidine or guanfacine), and mood medications (selective serotonin reuptake inhibitors, imipramine, divalproex, or risperidone). The percentages of children meeting wave 1 symptom criteria for ADHD who were taking medications during the year preceding each assessment were 17.0%, 41.1%, 44.6%, and 48.4% for stimulants; 0.0%, 2.4%, 5.0%, and 5.7% for antihypertensives; and 0.0%, 3.2%, 5.8%, and 11.5% for mood medications. The corresponding percentages for comparison children were 0.0%, 0.8%, 1.7%, and 3.4% for stimulants; 0.0% in all waves for antihypertensives; and 0.0%, 0.0%, 0.0%, and 0.8% for mood medications, respectively. Parents were asked to discontinue medications on the day of the assessment if approved by the child’s physician, and informants were asked to rate children when they were not taking medication.
Measures
Two lay interviewers concurrently conducted independent interviews of the mothers and children. Intelligence was estimated by averaging Stanford-Binet Intelligence Scale Short Form (16) scores from waves 1 and 2. Four children were excluded because their average intelligence scores were <70. The Diagnostic Interview Schedule for Children (17) was administered to the parent in each wave. Information was obtained on DSM-III-R diagnostic criteria for ADHD, oppositional defiant disorder, conduct disorder, anxiety disorders (simple phobia, social phobia, agoraphobia, separation anxiety, and panic, avoidant, and overanxious disorders), major depression, and dysthymia during the last 6 months. In addition, questions used in the DSM-IV field trials (4) to assess DSM-IV symptoms of disruptive behavior disorders not in DSM-III-R were asked. The test-retest reliability of the subtypes of ADHD with this version of the Diagnostic Interview Schedule for Children is kappa=0.51–0.64 (4).
Information on symptoms of the disruptive behavior disorders was obtained from teachers by mail using the DSM-IV version of the Disruptive Behavior Disorder Rating Scale (18). As in previous studies (18), symptoms rated “pretty much” or “very much” were scored as present. As in the DSM-IV field trials, symptoms were considered present if reported by either parent or teacher (19). In wave 1, impairment was assessed in two ways to make the diagnosis of ADHD. First, the parent was asked in the Diagnostic Interview Schedule for Children if the child’s ADHD symptoms had caused problems 1) at home or with friends or 2) at school. Second, parents and teachers completed the Impairment Rating Scale (unpublished measure by Fabiano GA et al.), in which the child’s need for treatment in a variety of areas is rated using 7-point subscales ranging from “no problem, definitely does not need treatment” (score=0) to “extreme problem, definitely needs treatment” (score=6). Parents rated the child’s need for treatment because of problems in relations with peers, siblings, and parents; academic progress at school; self-esteem; and impact on the family. Teachers rated the child’s need for treatment because of problems in relations with classmates and teachers, academic progress, classroom disruption, and self-esteem. Both respondents rated the child’s overall need for treatment. In two groups, Impairment Rating Scale ratings of ≥3 on at least one subscale optimally differentiated clinic and nonclinic children for both parents and teachers (unpublished, by Fabiano GA et al.). Test-retest stability (different teachers, 1 year apart) for the six Impairment Rating Scale subscales is r=0.39–0.63 (p<0.001) (unpublished, by Fabiano GA et al.).
Diagnoses
Children met full diagnostic criteria for ADHD (full ADHD) if they met DSM-IV symptom criteria and exhibited cross-situational impairment. Cross-situational impairment was defined as 1) parent report of problems at home or with peers in the Diagnostic Interview Schedule for Children or a parent rating of ≥3 on any Impairment Rating Scale subscale (excluding the school setting) and 2) parent report of school problems in the Diagnostic Interview Schedule for Children or a teacher rating of ≥3 on any Impairment Rating Scale subscale. All children met the age-at-onset criterion. In wave 1, 96 of 129 children who met symptom criteria for ADHD also exhibited cross-situational impairment. However, 29 children met symptom criteria for ADHD but displayed impairment in only one setting during wave 1. Therefore, we created a situational ADHD group to evaluate the validity of the cross-situational impairment criterion. The remaining 126 comparison children did not meet symptom criteria for ADHD in wave 1. Four children who met symptom criteria for ADHD in wave 1 but did not display impairment in any setting were conservatively classified in the comparison group (N=130).
Although 29.4% of the situational ADHD group met symptom criteria for the combined type, 76.0% of the full ADHD group met symptom criteria for the combined type (χ2=14.99, df=2, p<0.001, N=129). Therefore, differences between the full and situational ADHD groups necessarily also reflect differences in subtypes.
Longitudinal Assessments
Of the 255 children assessed in wave 1, 91.4% were reassessed in all four annual assessments, with no significant differences in retention among the three groups (full ADHD, situational ADHD, and comparison groups). Because only 10.2% had fewer than two waves of teacher reports and our data analyses allowed for missing data, there was little opportunity for bias due to attrition. The percents of interviews of parents conducted by interviewers who were unaware of previous assessments across waves 2–4 were 70.9%, 65.8%, and 60.0%, respectively. The corresponding percentages of youth assessments conducted by blind interviewers were 92.5%, 85.3%, and 74.1%. We controlled for nonblind interviews in all analyses of parent and youth reports. This was not relevant to teacher reports because different teachers completed questionnaires each year.
Child functioning during waves 2–4 was assessed by using impairment measures not used in the wave 1 diagnosis. Ratings of the child’s lowest level of overall functioning during the past 6 months were obtained on a 1–100 thermometer-like scale independently from the parent and the interviewer who administered the Diagnostic Interview Schedule for Children using the Nonclinician Version of the Children’s Global Assessment Scale (20). In each wave, the parent was also asked if the child had suffered an injury more serious than a scratch, bruise, or bump on the head that the parent attributed to the child’s carelessness, impulsiveness, or poor judgment.
The child’s regular education teacher used 5-point scales to estimate the proportions of classmates who liked and disliked the child (21). The widely used and well-validated social preference score (22, 23) was created by subtracting the “dislike” rating from the “like” rating. The sum of two items (each rated 0–2) from the Social Skills Rating Scale (24) quantified the teacher’s perception of classroom academic functioning: “produces correct schoolwork” and “finishes class assignments within time limits.” Findings based on this measure of classroom performance should be interpreted cautiously because the wording of some symptoms of ADHD (i.e., “makes careless mistakes in schoolwork” and “fails to finish schoolwork”) themselves imply impaired classroom functioning. However, if children who met criteria for ADHD were not impaired on this measure over time, it would raise serious questions about the predictive validity of ADHD. A child self-report scale validated against sociometrics in this age range (25, 26) assessed friendship difficulties from the child’s perspective in each wave (2-week test-retest reliability: r=0.77–0.79) (26). The 15 items cover making friends, keeping friends, and engaging in activities with friends.
Data Analysis
The 12-month stability of impairment measures without published reliability data was assessed from the present group with wave-to-wave correlations for continuous measures and Cohen’s kappa for nominal measures (27). We assessed the association of each class of medication with ADHD symptoms by conducting separate Poisson regressions in general estimating equations (28), treating each class of medication as a time-varying covariate (0=no, 1=yes). Continuous Children’s Global Assessment Scale ratings were reversed to approximate Poisson distributions, and planned comparisons among the three groups were conducted in general estimating equations. Differences in the rates of meeting full criteria for ADHD during waves 2, 3, and 4 were assessed by using binomial longitudinal regression. All longitudinal analyses specified unstructured correlations and used the z statistic (df=1) and robust standard errors. Interactions between time (wave) and groups were tested but were never significant. Cumulative response variables defined by one or more occurrence (unintentional injury and special education placement) were assessed by using logistic regression. Ordinal measures of impairment (teacher ratings of classroom work and social preferences) were analyzed by using mixed-effects ordinal regression in MIXOR (29). All analyses controlled for age, sex, family income (logged), intelligence, total number of oppositional defiant disorder and conduct disorder symptoms, total number of anxiety and depression symptoms, and site. In longitudinal analyses, time (waves) and the presence of blind interviews in each wave also were controlled.
Results
Stability across successive annual assessments of parent Diagnostic Interview Schedule for Children ratings of the sum anxiety and depression symptoms was r=0.79, 0.71, and 0.68, and the 12-month stability of Diagnostic Interview Schedule for Children impairment in home or peer relations was kappa=0.71, 0.70, and 0.56, and kappa=0.60, 0.55, and 0.52 for school impairment. Annual stability of teacher ratings was r=0.52, 0.56, and 0.61 for academic problems, and r=0.54, 0.49, and 0.47 for social preference. Annual stability of Children’s Global Assessment Scale ratings was r=0.67, 0.70, and 0.70 for parents, and r=0.72, 0.76, and 0.79 for interviewers. All correlations were p<0.0001.
In wave 1, the groups did not differ in sex, race/ethnicity, age, or family income (Table 1). Both ADHD groups had lower mean intelligence scores than the comparison group but did not differ significantly from one another. The full ADHD group exhibited more internalizing and conduct symptoms than the situational group in wave 1, which was reported to exhibit more such symptoms than the comparison children.
Similar to findings with older children (30), taking stimulants (z=3.76, df=1, p<0.001) or mood medications (z=2.42, df=1, p<0.02) but not antihypertensives (z=1.13, df=1, p=0.26) was associated with more ADHD symptoms during the same wave, with control for time, site, and blinding status. We interpret this as indicating that children with more ADHD symptoms were more likely to be given prescription medication.
As shown in Figure 1, both the full ADHD group (β=3.19, z=8.87, p<0.0001) and the situational ADHD group (β=1.72, z=4.46, p<0.0001) were more likely to meet full diagnostic criteria for ADHD during waves 2–4 than the comparison children. The full ADHD diagnosis in wave 1 predicted the full diagnosis in wave 4, with a positive predictive value of 0.75 and a negative predictive value of 0.86. The situational group was significantly less likely to meet full diagnostic criteria for ADHD during waves 2–4 than the full ADHD group (β=–1.46, z=–4.28, p<0.0001). Cumulatively, 3.1% of the comparison group met full diagnostic criteria for ADHD at least twice during waves 2, 3, and 4, compared to 79.2% of the children in the full ADHD group (odds ratio=119.7, 95% confidence interval [CI]=39.4–363.4) and 34.5% of the situational group (odds ratio=16.6, 95% CI=4.7–58.2).
The full ADHD group exhibited significantly greater global, academic, and social impairment during waves 2–4 than the comparison children (Table 2 and Figure 2, Figure 3, Figure 4, Figure 5, and Figure 6). The situational ADHD group was also more impaired on the Children’s Global Assessment Scale and was reported to have greater academic and social impairment over time than the comparison children. Similarly, the odds of at least one unintentional injury during waves 2–4 was 7.1 times greater for the full ADHD group (95% CI=2.3–21.8) and 5.1 greater for the situational group (95% CI=1.5–17.8) than for the comparison group. The odds of placement in special education during waves 2–4 for the full ADHD group was 3.1 times greater than for the comparison children (95% CI=1.3–7.5), but the children in the situational group were not more likely to be placed in special education than the comparison children (odds ratio=2.0, 95% CI=0.7–5.3) when other variables were controlled. The full and situational ADHD groups differed significantly on global Children’s Global Assessment Scale ratings but did not differ significantly on any measure of specific aspects of functional impairment.
Discussion
These findings strongly support the predictive validity of DSM-IV criteria for ADHD by showing that 4–6-year-old children who met full diagnostic criteria were likely to continue to meet criteria for ADHD and to experience greater functional impairment than comparison children over 3 years. These enduring differences in impairment were found across parent, teacher, interviewer, and child reports. It is particularly impressive that strong evidence of persisting functional impairment was found in teacher reports because different teachers provided data each year and were blind to diagnoses.
The 3 years covered in this study are developmentally important. Because the present study followed the children from preschool, kindergarten, or first grade well into elementary school, these findings suggest that the great majority of children given the diagnosis of ADHD at 4–6 years of age continue to exhibit ADHD symptoms and impairment into the age range during which ADHD has been extensively studied (4). It will be essential to follow these youth much longer, however, to determine if they continue to be symptomatic and impaired.
Thirty-four percent of situational ADHD children met full diagnostic criteria for ADHD at least twice in waves 2–4 compared to 3.1% of comparison children, and they were more impaired than comparison children on several measures. This suggests that a substantial minority of children who meet symptom criteria for ADHD but do not meet the cross-situational impairment requirement in their initial assessment will later meet full diagnostic criteria for ADHD. This could be limited to younger children because classroom demands on children’s attention and behavior at 4–6 years may not yet reveal classroom impairment that will emerge later, but the validity of the cross-situational impairment requirement has not been tested in children diagnosed at older ages. It is hoped that the present findings will stimulate research on this requirement, which was added in DSM-IV. Indeed, it would be useful to reconsider the appropriateness of this criterion on logical grounds. It is unclear why children who meet all diagnostic criteria for ADHD but display impairment in only one setting (e.g., seriously impaired academic functioning) should not be given the diagnosis of ADHD. Functional impairment must be a diagnostic criterion for all mental disorders, but it may be inappropriate to require impairment in multiple settings only for ADHD.
The present findings showed that children who met criteria for ADHD continued to display impairment over time. We stringently controlled age, sex, income, intelligence, co-occurring psychopathology, site, and nonblind assessments to ensure that impairment was related to ADHD, per se, and not to factors that co-occur with it (8). If ADHD had not accounted for unique variance in predicting future impairment after controlling these variables, there might be no need for the construct of ADHD. For example, Werry et al. (31) hypothesized that lower intelligence might fully explain the impairment associated with ADHD. If this hypothesis had been confirmed, one could dispense with the diagnosis of ADHD. None of the many types of functional impairment associated with ADHD could be fully explained by lower intelligence, concurrent psychopathology, or other factors, however. Indeed, we may have “overcontrolled” these characteristics in our analyses. For example, children who meet criteria for ADHD may score lower on intelligence tests than comparison children wholly or partly because their inattention and activity interfere with testing. If so, controlling for intelligence would artificially reduce the amount of variance in impairment explained by ADHD symptoms. Impressively, however, we were able to demonstrate that DSM-IV ADHD was associated with persisting impairment, even when intelligence was controlled.
Nonetheless, the present study was limited by the impairment measures used. To date, more effort has gone into the study of diagnostic criteria than functional impairment. Because the accurate measurement of functional impairment is at least as important to the diagnostic process as the measurement of symptoms and even more important to treatment planning, this is an area that sorely requires additional research. Similarly, because only one of many possible operational definitions of “cross-situational impairment” was used in this study, further research is need on that criterion. Finally, because this study used a structured diagnostic protocol, it is not clear that the findings of validity generalize to the kinds of unstructured assessments of ADHD often used in clinical practice.
Figure 2. Parent and Interviewer Ratings on the Nonclinician Version of the Children’s Global Assessment Scale Among Children With Full Attention Deficit Hyperactivity Disorder (ADHD), Children With Situational ADHD, and Nonreferred Comparison Children in Wave 1a
aHigher scores on the Children’s Global Assessment Scale indicate more adaptive functioning.
Figure 4. Teacher Ratings of the Completion of Accurate Classroom Academic Work on Time Among Children With Full Attention Deficit Hyperactivity Disorder (ADHD), Children With Situational ADHD, and Nonreferred Comparison Childrena
aHigher scores indicate better performance.
Figure 5. Peer Social Preference Score Among Children With Full Attention Deficit Hyperactivity Disorder (ADHD), Children With Situational ADHD, and Nonreferred Comparison Childrena
aScore was the teacher rating for the proportion of classmates who liked the child minus the teacher rating for the proportion of classmates who disliked the child.
Figure 6. Child Self-Rated Friendship Dissatisfaction Among Children With Full Attention Deficit Hyperactivity Disorder (ADHD), Children With Situational ADHD, and Nonreferred Comparison Childrena
aHigher scores indicate greater difficulty making and keeping friends.
Footnote
Received Oct. 9, 2003; revision received Jan. 15, 2004; accepted March 17, 2004. From the Department of Psychiatry, University of Chicago. Address reprint requests to Dr. Lahey, Department of Psychiatry (MC 3077), University of Chicago, 5811 South Maryland Ave., Chicago, IL 60637; [email protected] (e-mail). Supported by NIMH grant R01-MH-53554.
References
1.
Lahey BB, Carlson CL, Frick PJ: Attention deficit disorder without hyperactivity, in DSM-IV Sourcebook, vol 3. Edited by Widiger TA, Frances AJ, Pincus HA, Ross R, First MB, Davis W. Washington, DC, American Psychiatric Press, 1997, pp 163–188
2.
McBurnett K: Attention deficit/hyperactivity disorder: a review of diagnostic criteria. Ibid, pp 111–144
3.
Frick PJ, Lahey BB, Applegate B, Kerdyck L, Ollendick T, Hynd GW, Garfinkel B, Greenhill L, Biederman J, Barkley RA, et al: DSM-IV field trials for the disruptive behavior disorders: symptom utility estimates. J Am Acad Child Adolesc Psychiatry 1994; 33:529–539
4.
Lahey BB, Applegate B, McBurnett K, Biederman J, Greenhill L, Hynd GW, Barkley RA, Newcorn J, Jensen P, Richters J: DSM-IV field trials for attention deficit hyperactivity disorder in children and adolescents. Am J Psychiatry 1994; 151:1673–1685
5.
Olfson M, Gameroff MJ, Marcus SC, Jensen PS: National trends in the treatment of attention deficit hyperactivity disorder. Am J Psychiatry 2003; 160:1071–1077
6.
Zito JM, Safer DJ, dos Reis S, Gardner JF, Boles M, Lynch F: Trends in the prescribing of psychotropic medications to preschoolers. JAMA 2000; 283:1025–1030
7.
Greenhill LL, Jensen PS, Abikoff H, Blumer JL, DeVeaugh-Geiss J, Fisher C, Hoagwood K, Kratochvil CJ, Lahey BB, Laughren T, Leckman J, Petti TA, Pope K, Shaffer D, Vitiello B, Zeanah C: Developing strategies for psychopharmacological studies in preschool children. J Am Acad Child Adolesc Psychiatry 2003; 42:406–414
8.
Lahey BB, Pelham WE, Stein MA, Loney J, Trapani C, Nugent K, Kipp H, Schmidt E, Lee S, Cale M, Gold E, Hartung CM, Willcutt E, Baumann B: Validity of DSM-IV attention-deficit/hyperactivity disorder for younger children. J Am Acad Child Adolesc Psychiatry 1998; 37:695–702
9.
DuPaul GJ, McGoey KE, Eckert TL, VanBrakle J: Preschool children with attention-deficit/hyperactivity disorder: impairments in behavioral, social, and school functioning. J Am Acad Child Adolesc Psychiatry 2001; 40:508–515
10.
Fischer M, Barkley RA, Fletcher KE, Smallish L: The adolescent outcome of hyperactive children: predictors of psychiatric, academic, social, and emotional adjustment. J Am Acad Child Adolesc Psychiatry 1993; 32:324–332
11.
Mannuzza S, Klein RG, Bonagura N, Malloy P, Giampino TL, Addalli KA: Hyperactive boys almost grown up: replication of psychiatric status. Arch Gen Psychiatry 1991; 48:77–83
12.
Mannuzza S, Klein RG, Bessler A, Malloy P, LaPadula M: Adult outcome of hyperactive boys, I: educational achievement, occupational rank, and psychiatric status. Arch Gen Psychiatry 1993; 50:565–576
13.
Taylor E, Chadwick O, Heptinstall E, Danckaerts M: Hyperactivity and conduct problems as risk factors for adolescent development. J Am Acad Child Adolesc Psychiatry 1996; 35:1213–1226
14.
Campbell SB, Ewing LJ, Breaux AM, Szumowski EK: Parent-referred problem three-year-olds: follow-up at school entry. J Child Psychol Psychiatry 1986; 27:473–488
15.
McGee R, Partridge F, Williams S, Silva PA: A twelve-year follow-up of preschool hyperactive children. J Am Acad Child Adolesc Psychiatry 1991; 30:224–232
16.
Thorndike RL, Hagen EP, Sattler JM: Stanford-Binet Intelligence Scale. Chicago, Riverside, 1986
17.
Shaffer D, Fisher P, Piacentini J, Schwab-Stone M, Wicks J: Diagnostic Interview Schedule for Children. New York, Columbia University, 1993
18.
Pelham WE, Gnagy EM, Greenslade KE, Milich R: Teacher ratings of DSM-III-R symptoms for the disruptive behavior disorders. J Am Acad Child Adolesc Psychiatry 1992; 31:210–218
19.
Piacentini JC, Cohen P, Cohen J: Combining discrepant diagnostic information from multiple sources: are complex algorithms better than simple ones? J Abnorm Child Psychol 1992; 20:51–63
20.
Setterberg S, Bird H, Gould M: Parent and Interviewer Version of the Children’s Global Assessment Scale. New York, Columbia University, 1992
21.
Dishion T: The peer context of troublesome child and adolescent behavior, in Understanding Troubled and Troubling Youth. Edited by Leone PE. Newbury Park, Calif, Sage Publications, 1990, pp 128–153
22.
Coie JD, Dodge KA, Coppotelli H: Dimensions and types of social status: a cross-age perspective. Dev Psychol 1982; 18:557–570
23.
Sandstrom MJ, Cillessen AHN: Sociometric status and children’s peer experiences: use of the daily diary method. Merrill-Palmer Q 2003; 49:427–452
24.
Gresham FM, Elliott SN: Social Skills Rating System: Preschool Level. Circle Pines, Minn, American Guidance Service, 1990
25.
Cassidy J, Asher SR: Loneliness and peer relations in young children. Child Dev 1992; 63:350–365
26.
Sanderson JA, Siegal M: Loneliness and stable friendship in rejected and nonrejected preschoolers. J Appl Dev Psychol 1995; 16:555–567
27.
Cohen J: A coefficient of agreement for nominal scales. Educational and Psychol Measurement 1960; 20:37–46
28.
Zeger SL, Liang K-Y: Longitudinal data analysis for discrete and continuous outcomes. Biometrics 1986; 42:121–130
29.
Hedeker D, Gibbons RD: MIXOR: A Computer Program From Mixed-Effects Ordinal Regression Analysis. Chicago, University of Illinois, 1996
30.
Hart EL, Lahey BB, Loeber R, Applegate B, Green SM, Frick PJ: Developmental change in attention-deficit hyperactivity disorder in boys: a four-year longitudinal study. J Abnorm Child Psychol 1995; 23:729–749
31.
Werry JS, Elkind GS, Reeves JC: Attention deficit, conduct, oppositional, and anxiety disorders in children, III: laboratory differences. J Abnorm Child Psychol 1987; 15:409–428
Information & Authors
Information
Published In
History
Published in print: November 2004
Published online: 28 January 2015
Authors
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBGet Access
Login options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).