Understanding and Evaluating Research in Applied and Clinical Settings

This book is a reworking and expansion of a very well received series of short articles originally published as a series in the Journal of the American Academy of Child and Adolescent Psychiatry (which was, in turn, based on an earlier version of this book). In the preface, the authors claim that they provide “an integrated approach to quantitative research methods, data analysis, and article evaluation for consumers of research, especially those involved in practical or clinical applications.” They succeed in this goal.

The first five chapters provide introductory materials, including one chapter discussing quantitative and qualitative research, a chapter discussing principles of human research, and one related to ethical issues in publication and reviewing. The next five chapters discuss variables and their measurement, followed by six chapters on research approaches and design and three chapters on validity. Then follow 14 chapters on statistics and interpretation and a single chapter each on levels of evidence for evidence-based practice, meta-analysis, and effect sizes. The book ends with a chapter on how to evaluate the research validity of a study and a chapter on how to evaluate sample articles.

Reflecting their most recent prior incarnation as a series of separate articles, the chapters are each relatively short (typically 5–8 pages each) and largely self contained. The quite substantial advantage to this approach is that the chapters may be read separately, and most are sufficiently self-contained so that the teacher may pick and choose chapters and assign them in arbitrary order as they may fit into a particular course.

This book is clearly, as the authors tell us, targeted at consumers of research and not per se at researchers or post docs. The former will find this on target, scholarly, clear, and concise. The latter will find a book such as Clinical Trials: A Methodologic Perspective by Steven Piantadosi (1) much more comprehensive, detailed, and modern.

How would I use this book? Teaching psychiatry residents or other clinical trainees enough methodology and statistics to evaluate the current literature is a difficult task, but one of great importance. The clinicians we train today will practice for perhaps four or more decades. No matter how pessimistic one may occasionally become about the slow rate of progress, much new will be learned. Today’s trainee therefore must be prepared to evaluate the scientific literature far into the future. The chapters in this book could, combined with astute choice of current research articles, provide the basis for such a course.

What is best about this book? Chapter 36 entitled “Effect Sizes and Clinical Significance” clearly teaches the importance of looking at effect sizes and confidence intervals and covers various effect size measures, including area under the curve and number needed to treat. I would start my course here, and I would revisit this chapter repeatedly. The mischief of trainees (and of us all) of concentrating on p values rather than effect sizes and confidence intervals is great. Also commendable are the chapters on measurement validity and ethical issues.

For the intended purpose of the book, there are only a few areas of weakness. Perhaps the greatest is that two important analytic approaches in the modern treatment literature are not really covered: life table analyses and random effects regression. The former will be the method of choice in most studies of relapse, recovery, or remission, and the latter is becoming the preferred approach for acute treatment studies replacing “last observation carried forward” approaches. One small disadvantage of the approach the authors have taken is that because the chapters are designed to stand on their own reasonably well (although, as one gets to the statistical chapters, one may need to take a few chapters as a block rather than a single chapter), there is also some repetition of concepts and even sentences that can be distracting if the book is read through linearly.

In summary, the authors lay out their goals and achieve them. For the intended task and target, this book can be quite highly recommended.