‘Common Statistical Pitfalls’ Found in Basic Science Research.

Sample size and power, and careful attention to control groups and confounding factors, are all critical elements of statistical analyses of basic science studies, according to a research paper led by School of Public Health faculty members.

Writing in the Journal of the American Heart Association, Lisa Sullivan and Janice Weinberg, professors of biostatistics, and colleagues identified a number of common “statistical pitfalls” in basic science research. Among them are study design flaws, such as sample sizes that are too small to “robustly detect or exclude meaningful effects,” and a lack of attention to control groups, which can lead to bias and confounding.

The authors caution that researchers should not cast a statistical net too widely, but instead should conduct analyses only on factors of scientific interest. “Each time a statistical test is performed, it is possible that the statistical test will be significant by chance alone, when, in fact, there is no effect,” they write. “Because each test carries (some) probability of incorrectly claiming significance (i.e., a finite false-positive rate), performing more tests only increases this potential error.”

Sullivan and co-authors said basic science studies often are more complex than clinical studies because they may span several scientific disciplines, involving biochemistry, cell culture, model animal systems, and even selected clinical samples. Summarizing evidence and drawing conclusions based on the data are “particularly challenging” because of the complexity of study designs, small sample size,s and novel outcome measures.

“Careful attention to the research question, outcomes of interest, relevant comparisons (experimental condition vs. an appropriate control), and unit of analysis (to determine sample size) is critical for determining appropriate statistical tests to support precise inferences,” the authors say.

John Keaney Jr. of the division of cardiovascular medicine, University of Massachusetts Medical School, was a co-author on the report.

—Lisa Chedekel