Rhetorical experiments - Science-Education-Research

A topic in research methodology

'Rhetorical research' is research which is set up – either deliberately or inadvertently – in such a way that they seem designed to produce particular results.

Read about rhetorical research

A common feature of research set up to test the effectiveness of some teaching innovation (e.g., enquiry learning, flipped learning, jigsaw learning, etc.) is that the experimental condition is compared to a control or comparison condition that seems designed to offer weak pedagogy. Often this condition is based on the teacher talking and giving notes and/or exercises to be undertaken independently, even when the authors of the study report have set out how effective teaching requires active engagement of learners and constructive dialogue. That is, even if this is not the deliberate intention, the comparison condition is set up to fail compared with the experimental treatment.

Read about an example: 'A case of hybrid research design?'

"…some of the experimental studies reported in the literature are rhetorical in the … sense that the researchers clearly expect to demonstrate a well- established effect, albeit in a specific context where it has not previously been demonstrated. The general form of the question 'will this much-tested teaching approach also work here' is clearly set up expecting the answer 'yes'. Indeed, control conditions may be chosen to give the experiment the best possible chance of producing a positive outcome for the experimental treatment…What is noteworthy is that as part of the conceptual framework justifying the research readers are told fairly unequivocally that the teaching approach to be tested has already been shown to be clearly superior to (what is sometimes termed) 'traditional' teaching, yet the researchers then seek to test this in a specific context where they set up a control treatment that reflects the very traditional conditions that they have already told readers are ineffective for achieving learning objectives" (Taber, 2019, p.108).

A rhetorical experiment is not designed to produce substantially new knowledge: rather to create the conditions for a 'positive' result (Figure 8 from Taber, 2019).

This is not only a poor test of the innovation being studied, but is arguably ethically inappropriate are learners are deliberately being subjected to ineffective teaching for the sake of the research.

Read about research ethics

Read about unethical control conditions

Three criteria for rhetorical experiments:

The researchers claim the experimental condition should be effective based on well-established and widely accepted theory;
The researchers report that the experimental condition has been found to be effective in a range of previous studies in various contexts;
The researchers set up restrictive control conditions, to ensure the learners in the control condition are not taught as well as they should be.