Post Hoc, Ergo Propter Hoc

“Post hoc, ergo propter hoc” (“After this, therefore because of it”) is a very common way of thinking. Sometimes it is true, but often it is a logical fallacy. There is abundant evidence that victimized children have poorer cognitive functioning than other children, and not surprisingly plenty of research papers link the two causally. What’s wrong with that? Child victimization is wrong, and suboptimal cognitive functioning is a waste of human resources, so why not go ahead and work to reduce both? Well, for one thing, it is not good science. The most valuable, and practical, scientific explanation of a causal process is the one that is still standing when all the others have been knocked down.

In an article this issue, Danese and colleagues (1) do a magnificent job of testing the causal link between child victimization and cognitive deficit. They do this by the use of study designs that get as close as possible to our best methods for getting beyond “post hoc…” thinking and actually testing causal hypotheses. The standard method is of course the randomized controlled trial. But there are many areas of human experience where randomized controlled trials are morally or practically impossible. We cannot randomly assign children to be victimized. There are, however, next-best methods (2), and several of them are used in this study.

First, we can make sure that the sample resembles the general population as closely as possible (a quality known as “generalizability”). This means that there is limited value in studying restricted samples of children—clinical samples, for example, or samples of volunteers responding to an advertisement. In this study, Danese et al. analyzed data from two large samples, one in which participants were all the children born in a town in New Zealand over the course of 1 year in 1972 and 1973 (the Dunedin study), and another in which participants were pairs of twins recruited from a birth register of twins born in the United Kingdom in 1994 and 1995 (the E-Risk study). In the latter, certain groups were oversampled (children of mothers under 18) or undersampled (children of older mothers), but the sampling fractions were known, and therefore the final sample could be weighted to represent the population. Increasing generalizability even more, the same statistical tests were run on these two large samples from different countries and decades and showed remarkably similar results.

The basic statistical method used was, first, to run a set of ordinary least squares regressions testing the links between each definition of victimization and each measure of cognitive function, nearly all of which were significant, as the literature predicts. Next, the authors reran the analyses including other potentially explanatory variables. In the case of the Dunedin study, these were maternal IQ, child IQ at age 3, and family socioeconomic status. For the E-Risk study, they were IQ at age 5 and family socioeconomic status. In all cases, the associations between victimization and cognitive function were markedly reduced once these factors were taken into account.

Second, both studies were prospective and longitudinal. This is tremendously important because it enables the researchers to date, or at least to order, events—in this case, to be sure that child victimization occurred at specific ages, rather than being recalled retrospectively in adulthood. Widom, who has explored the value of retrospective recall, showed (3) that among individuals with a documented history of childhood sexual abuse, 29% did not report it in either of two adult follow-ups, 49% reported it at both follow-ups, and 22% were inconsistent. This is important because another study of the same sample showed that adults who recalled childhood maltreatment but who were not adjudicated cases as children were more likely than a control group to suffer from drug abuse disorders as adults, whereas adjudicated cases of maltreatment were not (4). The phenomenon of “seeking after meaning” by ascribing current problems to past misfortune is not uncommon (5).

Another strength of the longitudinal design is the ability to test whether at least some of the associations between victimization and cognitive functioning predated the victimization, in which case the observed association might be accounted for by “preexisting vulnerabilities and nonspecific effects of socioeconomic disadvantage” (1, Supplementary Methods, p. 10). For example, the authors reran the E-Risk analyses excluding children who were victimized before the first IQ test at age 5, and found no differences in the results.

Third, one of the samples consists of twins, who provide another test, examining whether differences in cognitive function within twin pairs were associated with differences in victimization in pairs growing up in the same family environment and sharing some (in dizygotic pairs) or all (in monozygotic pairs) of their genetic makeup. With a single exception, the authors found no differences within twin pairs in the associations between victimization and cognitive functioning.

Fourth, both studies included several tests of cognitive function, covering a wide range of areas and repeated at different ages. The tests for links between victimization and cognitive function were largely significant across the range of bivariate models, and nonsignificant when adjusted for IQ and socioeconomic status.

If we know why one thing leads to another, we are one step closer to being able to change things for the better. On the other hand, if our causal explanations are wrong, we may be wasting our efforts or even doing damage. Much of the history of medicine consists of mistaken and often painful “treatments” that did no good and often did harm: think of the way psychiatric patients were beaten and starved to cure their mental illness. When we accept the argument that childhood trauma causes poorer cognitive functioning in children and young adults, we accept the corollary that removing childhood trauma will improve cognitive functioning. There are many excellent reasons for removing child maltreatment from the world, but this study suggests that this alone is unlikely to improve cognitive functioning in most children.

This is an important study, with detailed supplementary material that is well worth reading. The study is valuable both for its content and as an example of how to use epidemiologic research to guide clinical treatment and policy.