Replication studies

A topic in research methodology

In the natural sciences, there is supposed to be a tradition of replication, with results only being accepted once reproduced by other researchers.

Actually, there is some scholarly debate about the extent to which replication actually occurs, even in natural science (Taber, 2019).

Is replication even meaningful in social sciences such as education?

There are clearly major challenges in trying to 'replicate' educational studies in new settings. (See, for example, Is reproducibility a realistic norm for scientific research into teaching?)

"Although the use of controlled experiments has sometimes been seen as an ideal in science education, this is clearly not a realistic approach (leaving aside the ethics of treating human learners as experimental subjects) when dealing with learners studying in very different educational contexts where many "variables" are likely to be pertinent, interacting, and even shifting. This complicates, but does not negate the value of exploring students' ideas to inform teaching [and many other topics!]

It is clearly the case that we are dealing with phenomena that are too complex to ever describe and understand fully, but there are–nonetheless–practical ways of developing useful knowledge, that fall within a generally postpositivist (i.e., "scientific," [2]) approach to producing new knowledge."

Taber, 2012: 7

Often in education there are international comparisons where researchers suggest that there are differences (for example in the responses to a questionnaire – sometimes requiring translation of items) between populations in different national contexts – although often the samples used are not strictly representative of the national context.

"However, differences between populations can only be considered significant where surveys are based upon
(i) methodology that is comparable;
(ii) sampling methods that can be considered to give representative findings."

Taber, 2012: 8

Incremental generalisation

The assumption behind the notion of incremental generalisation is that social contexts (for example, teaching and learning contexts) vary across a wide range of variables, and this makes it very difficult to reach general conclusions. It is clearly not possible to generalise widely form a single context to other different contexts. Nor is it feasible to set up research to check every possible context. Incremental generalisation offers guidance for maximising the vlaue of 'replication studies' across different contexts.

Read about incremental generalisation

Sources cited:

My introduction to educational research:

Taber, K. S. (2013). Classroom-based Research and Evidence-based Practice: An introduction (2nd ed.). London: Sage.

facebookShare on Facebook
TwitterTweet