Unacceptable control conditions in educational experiments
A topic in research methodology
There should be an ethical imperative in educational research: researchers should never ask learners to participate in a study condition they have good reason to expect will damage their opportunities to learn.
"Experiments are set up to test specific hypotheses. In a 'true' experiment the researcher controls variables, so that only the factor which is hypothesised to have an effect differs between the experimental and control treatments" (Taber, 2013, p.83).
This means there will be two (or sometimes more) conditions or treatments:
- an experimental condition – that is being tested
- a control condition – which acts as the benchmark for comparison
In general we might look to compare some educational innovation with no input at all, with a standard educational input, or with something recognised as good educational practice. The choice of control condition in a study will reflect the research purposes/questions, but also needs to meet ethical standards.
Read about the choosing the appropriate level of control
An ethically question alternative
The three levels of control in educational experiments might be summarised as:
- Does it have any educational value? (level 1);
- Is it better than standard educational provision? (level 2);
- How does it compare to what is already recognised as good practice? (level 3)" (Taber, 2019, p.88).
But sometimes, some research studies read as if the question informing the design of control conditions is
4. How does an innovation compare with currently recognised poor practice?
This question can only be answered by asking some learners, those in the control condition, to experience such poor practice.
Do no harm
There should be an ethical imperative in educational research:
- researchers should never ask learners to participate in a study condition they have good reason to expect will damage their opportunities to learn.
"Education has values at its core, and educational researchers should always pay particular attention to research ethics: the potential consequences that their actions could have for others. Participants (and suitable gatekeepers, when participants are children) in educational research studies should always give voluntary, informed, consent – but researchers retain a major responsibility for the ethics of experiments as participants cannot be assumed to fully understand the background and nature of the research in the way the researchers do. Teachers and educational researchers should in particular seek to avoid doing anything that is likely to harm those they are working with. In most educational research experiments…potential harm is likely to be limited to subjecting students (and teachers) to conditions where teaching may be less effective, and perhaps demotivating. This may happen in experimental treatments with genuine innovations (given the nature of research). It can also potentially occur in control conditions if students are subjected to teaching inputs of low effectiveness when better alternatives were available. This may be judged only a modest level of harm, but – given that the whole purpose of experiments to test teaching innovations is to facilitate improvements in teaching effectiveness – this possibility should be taken seriously" (Taber, 2019, pp.93-94).
If researchers are testing an innovation they suspect may prove educationally beneficial, especially an innovation which as already proved its worth in many previous studies across a range of other contexts (different topic, different age groups, different countries…) there seems little point is setting up an experiment that only compares it with a restrictive educational input which does not reflect what it recognised as good practice. The only obvious reason to do this, is to ensure the experiment gives a positive outcome. That is, the 'experiment' is not meant as a genuine scientific enquiry to test an uncertain hypothesis, but is intended to be 'rhetorical': a demonstration of what thee researchers already think they know.
I have suggested three criteria for rhetorical experiments, one of which is a sub-optimal educational input for a control condition:
- The researchers claim the experimental condition should be effective based on well-established and widely accepted theory;
- The researchers report that the experimental condition has been found to be effective in a range of previous studies in various contexts;
- The researchers set up restrictive control conditions, to ensure the learners in the control condition are not taught as well as they should be.
Read about rhetorical experiments
Examples of ethically questionable control conditions used as comparisons for innovative teaching
| Control condition | Source | Notes |
| "In the control group, a teacher directed strategy representing the traditional approach was used… where students are completely passive… The teacher used direct teaching and question and answer methods… In this group, the teacher provided instruction through lecture and discussion methods to teach the concepts. The teacher… wrote notes on the chalkboard about the definition of concepts, and passed out worksheets for students to complete. The primary underlying principle was that knowledge takes the form of information that is transmitted to students…" | Abdi, A. (2014). The effect of inquiry-based learning method on students' academic achievement in science course. Universal Journal of Educational Research, 2, 37-41. | • 5th grade in primary schools • topics: hidden strangles; nervous system and sense organs; human and environment. • Comparing 5E learning cycle (a structured inquiry learning approach) vs. traditional method |
| "The control group was taught with a teacher-centered traditional didactic lecture format. Teaching strategies were dependent on teacher expression without consideration for student misconceptions. . . . students were required to use their textbooks; students were passive participants and rarely asked questions; they did not benefit from the library or internet sources; activities such as computer animations or brainstorming were not used; generally the teacher wrote the concepts on the board and then explained them; students listened and took notes as the teacher lectured on the content." | Acar, B., & Tarhan, L. (2007). Effect of cooperative learning strategies on students' understanding of concepts in electrochemistry. International Journal of Science and Mathematics Education, 5(2), 349-373 | • 11th grade students (17 years of age) • understanding of electrochemistry •Comparing: cooperative learning based on a constructivist approach vs, ' 'traditional instruction' |
| "The control group was taught using traditional, teacher-centered lecture-style pedagogy." [High school students, aged from 15 to 18 years.] | Barthlow, M. J., & Watson, S. B. (2014). The Effectiveness of Process-Oriented Guided Inquiry Learning to Reduce Alternative Conceptions in Secondary Chemistry. School Science and Mathematics, 114(5), 246-255. https://doi.org/https://doi.org/10.1111/ssm.12076 | • college preparatory chemistry students enrolled in large suburban high schools (15-18 year olds) ª• secondary chemistry students' alternate conceptions in physical and chemical changes in matter • active, student-centred POGIL vs. traditional teacher-centred, lecture-style chemistry pedagogy |
| "Students in CG [control group] were instructed by lecturing method, discussion and sometimes students performed the laboratory activities in that students were passive listeners and teacher's role was to transmit the facts and concepts to the students. . . . Teacher did not give emphasis on students' misconceptions. Students were passive listeners and they were taking notes. In the laboratory activity section, students were required to do experiment by using the handout. . . . like 'cookbook', described the all steps of the experiment" | Çam, A., & Geban, O. (2011). Effectiveness of case-based learning instruction on epistemological beliefs and attitudes toward chemistry. Journal of Science Education and Technology, 20(1), 26-32. | • eleventh grade students (16 years old) • solubility equilibrium topic • to investigate the effectiveness of case-based learning instruction over [sic] traditionally designed chemistry instruction on eleventh grade students' epistemological belie |
| ". . .was taught using the lesson plan based on the conventional teaching method. . .which was commonly practised in that school . . . in which the teacher dominants [sic], whereas the learners remain passive" | Gidena, A., & Gebeyehu, D. (2017). The effectiveness of advance organiser model on students' academic achievement in learning work and energy. International Journal of Science Education, 39(16), 2226-2242. | •preparatory school, grade-11 • Concept of work and energy • This study was designed to investigate the effectiveness of AOM on students' academic achievement in learning work and energy |
| "the teacher mainly used lecture and discussion methods…The chemistry textbook was the primary source of knowledge in this group… During the transmission of knowledge, the teacher and frequently used the board to write chemical formula[e] and equations and draw some figures…. the teacher summarized the concepts under consideration and prompted students to take notes…some algorithmic problems on the board and asked students to solve those problems individually…….nature of their laboratory activities was traditional … to verify what students learned in the classroom…follow the step-by-step instructions for the experiment… students were asked to record their observations and data. They were not required to reason about the data in a deeper manner…" | Kingir, S., Geban, O., & Gunel, M. (2013). Using the Science Writing Heuristic Approach to Enhance Student Understanding in Chemical Change and Mixture. Research in Science Education, 43(4), 1645-1663. https://doi.org/10.1007/s11165-012-9326-x | • grade 9 students • understanding of chemical change and mixture concepts • "Students in the treatment group were instructed by the SWH approach, while those in the comparison group were instructed with traditionally designed chemistry instruction." |
| "One experimental group (EG; N = 30) and one comparison group (CG; N = 35) were used in the study. While EG was taught with simulation-based instruction, the CG was exposed to traditional instruction." [No additional details provided in the published account, but the wording here suggests students were passive observers.] | Naseriazar, A., & Özmen, H. (2012). Effectiveness of simulations on university students' understanding of chemical equilibrium. Information Technology & Computer Science, 2, 285-290. | • university students who enrolled in the general chemistry course – two classes of the same teacher • understanding of chemical equilibrium • the effectiveness of simulation-based instruction over [sic] traditional instruction |
| ". . .teacher-centred instruction, [where] learning focuses on the mastery of content, with little development of the skills and attitudes necessary for scientific inquiry. The teacher transmits information to students, who receive and memorize it. . . . The curriculum is loaded with many facts and a large number of vocabulary words, which encourages a lecture format of teaching. . .the control group were instructed via teacher-centred didactic lecture format . . . The students were instructed with regular chemistry textbooks. They listened to the teacher carefully, took notes and solved algorithmic problems" | Sesen, B.A., & Tarhan, L. (2011). Active-learning versus teacher-centred instruction for learning acids and bases. Research in Science & Technological Education, 29(2), 205-226. | • high-school students (average age 17 years) • to investigate the effectiveness of active-learning implementation on students' understanding of 'acids and bases'. |
| "The traditional class was taught the same subjects as were the experimental group using a teacher-centred lecture format. Students were passive participants and rarely asked questions. They did not benefit from library or Internet sources." "The same teacher, experienced in active learning and PBL techniques, did the facilitating in the experimental group, and the lecturing in the control group." | Tarhan, L., & Acar, B. (2007). Problem‐based learning in an eleventh grade chemistry class: 'factors affecting cell potential'. Research in Science & Technological Education, 25(3), 351-369 | • the effectiveness of problem-based learning (PBL) on eleventh grade students' understanding of 'The effects of temperature, concentration and pressure on cell potential' and also their social skills. • high schools third year (11th grade) |
| "…the control group was taught the same topics as the experimental group using a teacher-centred traditional didactic lecture format. Teaching strategies were dependent on teacher expression and question-answer format. However, students were passive participants during the lessons and they only listened and took notes as the teacher lectured on the content." "Both of the groups were taught by the same chemistry teacher, who was experienced in active learning and PBL." | Tarhan, L., Ayar-Kayali, H., Urek, R. O., & Acar, B. (2008). Problem-Based Learning in 9th Grade Chemistry Class: 'Intermolecular Forces'. Research in Science Education, 38(3), 285-300 | • 9th Grade • topic – intermolecular forces dipole- dipole forces, London dispersion forces and hydrogen bonding) • to examine the effectiveness of a problem-based learning (PBL) on students' understanding |
| "…traditional instruction which relied on instructors' explanations with no consideration of the students' misconceptions. The instructor used overhead projector to show the definitions of concepts, explained the facts, solved the questions, meanwhile students took notes through the lessons" | Taşlidere, E. (2013). The effect of concept cartoon worksheets on students' conceptual under- standings of geometrical optics. Education & Science/Egitim Ve Bilim, 38, 167. | • pre-service science teachers • geometric optics • implementation of concept cartoon worksheets vs. traditional instruction. |
Justifying suboptimal control conditions
"This raises an ethical issue in such studies that, given the current state of knowledge prior to the research, the researchers employ a control treatment that is considered to be of limited educational value. Students in the control condition are expected to be disadvantaged compared to those in the experimental condition. Authors often justify this by reporting that the suboptimal conditions set up for the control are just what these students would experience anyway, and so they are not disadvantaged compared to not being in the study. That is only so if authors are correct that 'traditional' teaching, with no elements of more 'progressive' approaches, is endemic in the local context. Whilst studies may present traditional and progressive teaching as being a dichotomy, actual observations of teachers' classroom practice suggest actual practice is more nuanced and often reflects a blend of these two extremes…" (Taber, 2019, p.108).
Sources cited:
- Taber, K. S. (2013). Classroom-based Research and Evidence-based Practice: An introduction (2nd ed.). London: Sage.
- Taber, K. S. (2019). Experimental research into teaching innovations: responding to methodological and ethical challenges. Studies in Science Education. doi:10.1080/03057267.2019.1658058 [Download this article]
My introduction to educational research:
Taber, K. S. (2013). Classroom-based Research and Evidence-based Practice: An introduction (2nd ed.). London: Sage.
