Confounding variables

Page contents

One methodology (general research strategy) is the experiment. The aim is to find the effect of a changing variable (the independent variable) on a variable that is being observed/measured (the dependent variable) whilst controlling (keeping constant) and other factors that might affect the observation / measurement (control or controlled variables). Potentially relevant factors that are not controlled are known as confounding variables.

Read about experiments

Confounding variables: the authors of the study thought they could draw clear conclusions (read 'Shock result: more study time leads to higher test scores')

Can the authors of this study draw general conclusions about the differences between pre-service chemistry teachers and pre-service physics teachers? (Read 'Are physics teachers unaware of the applications of physics to other sciences?')

"Research in educational contexts often employs quasi-experiments or natural experiments rather than true experiments, and these types of designs raise additional questions about the equivalence between experimental and control groups and the potential influence of confounding variables."
Taber, 2019: 69

A confounding variable is anything that distinguishes the experimental and control conditions which is not accounted for. Clearly we could imagine a near-infinite range of factors that could vary between conditions, and we can never be sure that seemingly unlikely variables have no effect. So we are often left using our best understanding of current theories, and our practical experiences, to make judgements that exclude most such factors from (probably) being relevant.

"Experiments are set up to test specific hypotheses. In a 'true' experiment the researcher controls variables, so that only the factor which is hypothesised to have an effect differs between the experimental and control treatments. In reality, such control is rarely (if ever) possible in enquiries into teaching and learning – even if the range of potentially significant variables can be identified.

However, it is sometimes possible to make comparisons between situations that approximate to the conditions needed for an experiment. For example, an experimental procedure might require students to be assigned to one of two classes randomly, but the researcher may have to work with existing classes. However, if it can be shown that the two groups are sufficiently similar (on whatever measures are considered relevant), it may be possible to continue as if there was an experimental set-up. In such a situation, although it is not possible to randomly assign individual learners to the two classes, there should be a random assignment (e.g., a coin toss) to determine which class will experience the 'experimental' treatment, and which will act as the control.

In making a judgement about which factors may be relevant, we will exclude from consideration many potential factors that we have, in effect, decided in advance are not important in the study. The range of potential factors we could potentially take an interest in when comparing classes is immense (see Box 4.1 for some examples)" Taber, 2013, pp.83-84.

Consider an experimental study in a school to evaluate some pedagogy or new learning resource; or a comparison between examination results from two neighbouring schools. Which of these factors can we be really sure will not make any difference?
From Taber, 2013: 83.

One such variable might be that those involved (as investigators, and/or participants) have different expectations about outcomes in the different conditions – as such expectations can influence outcomes. This is not just a hypothetical risk, but something that has been often demonstrated.

Read about expectancy effects