When can we claim that higher scores are higher?

Comment on “Increasing chemistry students’ knowledge, confidence, and conceptual understanding of pH using a collaborative computer pH simulation”

One of my publications is:

Taber, K. S. (2020). Comment on “Increasing chemistry students’ knowledge, confidence, and conceptual understanding of pH using a collaborative computer pH simulation” by S. W. Watson, A. V. Dubrovskiy and M. L. Peters, Chem. Educ. Res. Pract., 2020, 21, 528. Chemistry Education Research and Practice. doi:10.1039/D0RP00131G

Abstract:

This comment discusses some issues about the use and reporting of experimental studies in education, illustrated by a recently published study that claimed (i) that an educational innovation was effective despite outcomes not reaching statistical significance, and (ii) that this refuted the findings of an earlier study. The two key issues raised concern how the research community should understand the concept of refutation when comparing across studies, and whether the adoption of inferential statistics in a study should bind researchers to accept the inferences such tests suggest.

When can the findings of one study refute the findings of another?

[Download a full-size copy of this image]

Contents:

  • Introduction
  • Implications of adopting statistical tests
  • Reporting context-directed research
  • What do we mean by refutation?
  • Recommendations

This article is Open Access, and can be accessed for free: here

Direct link to downloadable pdf: here