A topic in research methodology
One methodology (general research strategy) is the experiment: "…by definition, experiment is the kind of scientific experience in which some change is deliberately provoked, and its outcome observed, recorded and interpreted with a cognitive aim" (Bunge, 1967/1988).
A choice of research strategy should fit the background assumptions underpinning a particular study (see research paradigms):
Experimental research is interventionist (as opposed to naturalistic), that is it deliberately intervenes in the natural state of affairs to see what will happen.
"In experimental research…control is attained through the manipulation of experimental arrangements. The conditions of the experiment provide a context of production for the phenomenon to be studied. The experiment is designed to 'produce' the phenomena under conditions suitable for observation: the direct intervention of control standardises the conditions of production so that measurements (observations) will be comparable from condition to condition and study to study."
Kemmis, 1980: 110
"And the control of stimuli may consist just in recording or measuring them, or in a deliberate variation of kind or intensity…In the first case we may speak of passive control, in the second of active control".
Bunge, 1967/1988
The aim is to find the effect of a changing variable (the independent variable) on a variable that is being observed/measured (the dependent variable) whilst controlling (keeping constant) and other factors that might affect the observation/measurement (control or controlled variables – any potentially confounding variables).
Read about confounding variables
Experiments are difficult even in laboratories, which are places designed to allow us to control conditions…
"…physical experiments are, of course, only as reliable as the least reliable phase of their design and execution…
Norwood Russell Hanson
"…experiments are designed to be as chain-like as possible. It takes long training to master this technique of design. To bring together a cluster of theoretical considerations in a single, tersely-expressed hypothesis; to torture it in an experiment, each phase of which keeps everything constant except one set of factors; to insure that when these vary to a certain degree they initiate another phase of the experiment, where again everything is constant save one factor; to have arranged that these in turn play roles in further phases of the demonstration – not just anyone, not every physicist, can juggle this into an efficient laboratory operation."
…and experiments in social settings are that much more challenging! (Taber, 2019)
In a true experiment the investigators randomly assign the 'units of analysis' to the different conditions. When randomisation is not possible (e.g., learners cannot be shuffled into new classes), the design is referred to as a quasi-experiment.
Control conditions
When setting up an educational experiment, the experimental condition needs to be compared with a control condition. Ideally, the only difference between the two conditions is related to the independent variable, but in education this is not feasible (the confounds referred to above).
Researchers relying on working with an educational institution such as a school may find that the control condition is imposed on them as the only comparison available. However, there may be a choice of the comparison.
If this is, it is important to remember that educational research relies on the goodwill of others, and the researcher has responsibilities to them.
Sometimes researchers set up comparison conditions likely to give poor outcomes, so favouring the experimental condition ('rhetorical experiments').
Unfortunately there has been a tradition in some research into science teaching to deliberately set up control conditions that fall well short of best practice (Taber, 2019) – presumably to give the experimental condition the best chance of being found to outperform the control. This is both questionable on methodological grounds (leading to 'rhetorical research') and ethical grounds (unfairly diminishing the educational experience of learners).
Read about 'Rhetorical experiments'
There are different 'levels' at which control conditions can be set up, and it is important to design the control so it is both ethical and provides the right comparison to answer the research questions.
Read about control conditions in experimental research
Designs counterbalancing different conditions
One way to minimise some of the between confound that can undermine experimental design is to build in counterbalancing. The figure shows the design of a study exploring the use of 'DARTs' (directed activities related to text) as a teaching approach in a girls' school in Zambia (Magawa & Kalebaila, 2020). Even though both classes selected for the experiment scored very similarly on a pre-test, we might wonder if differences between the teachers, or between the students groups (or the interaction between the two – some groups 'get on' better with some teachers rather than others), contributed to the experimental class outperforming the control class on the first post-test. However, the researchers then swapped the treatments, and we see now it is the other class which performs better:
So, in this small scale study the greater measured learning takes place in the 'DARTs' condition, regardless of which of the classes gets this treatment.
Natural experiments
Where manipulation is not possible, it may sometimes be possible to use a 'natural experiment' where the existing situation provides contrasting conditions that cam be studied.
Read about 'Natural experiments'
Read about testing for equivalence between groups
Educational experiments may be very difficult to fully replicate.
Read about replication studies
A detailed review of the nature of, and challenges in undertaking and interpreting, educational experiments may be found in Taber (2019).
Sources cited:
- Bunge, M. (2017/1998). Philosophy of Science. Volume 2: From explanation to justification (Revised ed.). Routledge. (1967)
- Hanson, N. R, (1958) Patterns of Discovery. An inquiry into the conceptual foundations of science. Cambridge: Cambridge University Press.
- Kemmis, S. (1980). The Imagination of the Case and the Invention of the Study. In H. Simons (Ed.), Towards a Science of the Singular: Essays about Case Study in Educational Research and Evaluation (pp. 96-142). Norwich: Centre for Applied Research in Education, University of East Anglia.
- Magawa, P., & Kalebaila, K. K. (2020). The effect of integrating DARTs on learners academic performance in rates of chemical reaction. International Journal of Chemistry Education Research, 4(2), 67-74.
- Taber, K. S. (2014). Methodological issues in science education research: a perspective from the philosophy of science. In M. R. Matthews (Ed.), International Handbook of Research in History, Philosophy and Science Teaching (Vol. 3, pp. 1839-1893): Springer Netherlands.) [Download chapter]
- Taber, K. S. (2019). Experimental research into teaching innovations: responding to methodological and ethical challenges. Studies in Science Education. doi:10.1080/03057267.2019.1658058 [Download the paper]
Further reading:
Read: Not a great experiment…What was wrong with The Loneliness Experiment?
My introduction to educational research:
Taber, K. S. (2013). Classroom-based Research and Evidence-based Practice: An introduction (2nd ed.). London: Sage.
