Rhetorical research

A topic in research methodology

The philosopher of science Karl Popper considered making 'bold' conjectures (hypotheses) a virtue – rather than playing safe by looking for expected results.

The purpose of research is to generate new knolwedge. Sometimes we see studies that have been designed (deliberately or otherwise) such that they are not bold, but so cautious that is seems their outcomes were never really in doubt: they employ meek conjectures.

'Rhetorical research', then, is research which is set up – either deliberately or inadvertently – in such a way that studies seem designed to produce particular results. These studies have weak research designs better suited for supporting the researchers' expectations than for answering well-motivated research questions.

Rhetorical experiments

A common feature of research set up to test the effectiveness of some teaching innovation (e.g., enquiry learning, flipped learning, jigsaw learning, etc.) is that the experimental condition is compared to a control or comparison condition that seems designed to offer weak pedagogy. Often this condition is based on the teacher talking and giving notes and/or exercises to be undertaken independently, even when the authors of the study report have set out how effective teaching requires active engagement of learners and constructive dialogue. That is, even if this is not the deliberate intention, the comparison condition is set up to fail compared with the experimental treatment.

(Read about 'Rhetorical experiments')

This is not only a poor test of the innovation being studied, but is arguably ethically inappropriate are learners are deliberately being subjected to ineffective teaching for the sake of the research.

(Read about 'Research ethics')

Finding what the researchers are looking for

Another kind of study will set out a conceptual framework which leads to research questions along the lines:

can this teaching and learning episode be understood in terms of [a theoretical perspective]

where the perspective might be Piagetian stage theory, cultural-historical activity theory, social and cultural capital, restricted and elaborated language codes – or whatever.

An analytical framework is then developed based around indicators relating to the theory that has been adopted.

The rich data is collected by observations, interviews, accessing documents, etcetera, and the data is interrogated according to the coding framework to see if there are data that can be interpreted as evidence of the relevance of the theory.

As far as it goes this is fine, but it can only show that some aspects of the teaching and learning activity being researched can be understood in terms of the theory. That is only helpful if there is a genuine question about the relevance of the theory to that context (based on a bold conjecture). So, for example, it would be a genuine surprise perhaps if learning in a post-graduate seminar was found to be impeded by the Piagetian level of some of the post-graduate students!

Of itself, finding a well-established theory that has been widely applied to educational contexts can make sense of the data from some specific context does not tell us much: not for instance whether that is the most productive perspective for making sense of the context.

If a lot of data is collected, it may be found that there is much potential evidence to support the applicability of the theory, but unless alternative theories are compared and/or negative examples are deliberately sought, this is often not especially insightful.

Did the activity engage learners?

One study which implemented an innovative learnig activity with students asked the research question:

"Romeo and Juliet: A Love out of the Shell": Using Storytelling to Address Students' Misconceptions and Promote Modeling Competencies in Science

• (RQ2) to what extent did the TLS [Teaching-Learning Sequence] engage students?

The authors describe how:

"To address the second research question (RQ2), namely, 'To what extent did the narrative engage students?', we analyzed the following:

–  answers to a satisfaction questionnaire administered to students at the end of the activity, to gauge how the students themselves experienced the entire activity;

–  the observations conducted by the teachers participating in the activity (the instructor who conducted the activity and two chemistry teachers who took part in the final discussion), to understand if the teacher's perception was corroborated or not.

In the analysis of the supervising teacher's diary and in the audio recordings of the observations made by the chemistry teachers, phrases or expressions indicating student engagement, such as 'I noticed that the students participated/were involved', were sought. Similarly, in the written responses of the students, phrases such as 'I felt involved', 'I enjoyed it', or 'It was nice' were searched for."

Aquilina, G.; Dello Iacono, U.; Gabelli, L.; Picariello, L.; Scettri, G.; Termini, G. "Romeo and Juliet: A Love out of the Shell": Using Storytelling to Address Students' Misconceptions and Promote Modeling Competencies in Science. Educ.Sci.2024,14,239. https://doi.org/10.3390/educsci14030239

This seems to suggest that the reesearchers actively searched the data for positive examples that would show their innovation was successful, rather than seeking any evidence for levels of engagement (including possible disengagement). Of course, the authors may have phrased their description of the process this way because they only found positive instances in the data – but a reader would have more confidence in their findings if they had reported they were interested in all relevant evidence.

An example of Rhetorical research

In one (supposedly peer-reviewed) study, published in an Education journal, it was claimed that when students' grades suffer because they are suffering episodes of hallucinations, this can be addressed by implementing a soul purification module that removes jinn, devils and other subtle beings that cause the hallucinations.

The paper presented evidence of how the grades of some students had decreased during periods of hallucinations and then recovered later. Absolutely no evidence was presented to show this recovery was due to the intervention. Presumably the researcher was so convinced that hallucinations are due to possession, and that exorcism will cure hallucinations, that it was assumed that the soul purification module must have been the reason for the recovery in academic grades. Sadly, the journal's peer reviewers (if there really were any) and editor saw no reason to decline publication!

---

(You can read more about this study, here.)

---