Assessing Outcomes of Prevention Interventions

In Reply: I appreciate the interest of Drs. Hawkins and Catalano in my paper and their suggestion for further reading. In turn, I would urge them to read Karl Popper's Conjectures and Refutations (1). The first chapter discusses the limitations of theories that have such broad explanatory power that they can explain nearly everything that happens in the fields to which they refer. Popper observes that it is easy to obtain empirical observations that verify one's theory but that a good scientific theory is one that has survived genuine tests aimed at refuting it.

I am not at all surprised that Drs. Hawkins and Catalano can point to data that verify their belief that their intervention program works. Recent evaluations of their Seattle Social Development Project (SSDP) do indeed indicate that the program has some positive effects. However, the preponderance of evidence reported in these studies shows no difference in criminal behavior and substance use between persons who received the intervention and those in the comparison group (2). I take this as refutation of the hypothesis that the SSDP has long-term effects on these behaviors. I would be interested to know what Drs. Hawkins and Catalano would count as a genuine critical test of their intervention—that is, exactly what empirical evidence would it take for them to give up the idea that the SSDP is effective?

As for Drs. Hawkins and Catalano's contention that risky behaviors covary and have a common developmental path, empirical evidence certainly exists to support this view. However, there is also evidence that such behaviors cluster into more than a single underlying factor. Indeed, Drs. Hawkins and Catalano conclude one of their own investigations in this area by stating that "the results of this study suggest that the 'common factor' thesis in which different types of problem behaviors are hypothesized to reflect a single underlying proneness to deviance does not hold for preadolescent youths" (3). Like many other researchers, I would argue that the population of "at risk" youths is composed of various subgroups, among these being "generalists" and "specialists." If this is the case, it would help explain why universal interventions that assume a homogeneous target population with a common causal pathway have such limited effects.