Unit of analysis - Science-Education-Research

A topic in research methodology

Unit of analysis is a term used in experimental research, and refers to how the data will be conceptualised and grouped during analysis. For example, if research investigates school learning, the unit of analysis might be the lesson, or the learner, or the teacher, or the curriculum subject? This will depend upon the research question being investigated, and should be established at the outset of the research before data collection commences.

"An important term used in discussing experimental research is 'unit of analysis'. An experiment may, for example, be comparing outcomes between different learners, different classes, different year groups, or different schools… It is important at the outset of an experimental study to clarify what the unit of analysis is, and this should be explicit in research reports so that readers are aware what is being compared" (Taber, 2019, p.72)

"The unit of analysis refers to the types of 'things' that will be characterised and perhaps compared in a study. In educational research the unit of analysis could be a student, a lesson, a class, a teacher, a school, a group within a class, a question asked, an explanation given, a conversational exchange, a test script, a scheme of work, a lesson plan, etc. That is, we might characterise and compare different students; we might characterise and compare different lessons; we might characterise and compare different classes, etc.

So, in a study looking at teacher beliefs about pedagogy, the unit of analysis is likely to be the teacher.
In a study of the relationship between school ethos and exclusion rates, the unit of analysis is likely to be the school.
In a study of student understanding of creation myths in different cultures, the unit of analysis is likely to be the student.
In a study on the effect of gender on school science group work, the unit of analysis is likely to be the group (although a group does not have a gender, and so the gender composition of the group will need to be seen as the 'independent' (or input) variable" (Taber, 2013, p.254).

Units of analysis in experiments

In experimental research, in a 'true' experiment, the units of analysis must be randomly assigned to conditions:

In experimental research there must be randomisation of the units of analysis to the different conditions (figure after Taber, 2019, p.72)

So, for example, if a researcher has to compare two existing classes, then the unit if analysis should be the class, not the individual learners (which has consequences for the ability to use statistics to test for statistically different outcomes). "If the units of analysis are schools, it may be difficult to enrol a large enough number of schools into the sample for the statistical methods to be used – especially in those national contexts that rely on schools responding to invitations to volunteer (this is less of a problem when research access is granted at regional/district or state level)" (Taber, 2019, p.74).

"So one might consider 50 students who were to be part of a study where it was intended to use individual student test results as a measure of learning to explore whether some teaching approach brought about greater learning than some other teaching approach. If it is possible to randomly assign the 50 students into two groups of 25, then there are 25 'units of analysis' [n=25] in each group. However, if the researchers are required to work with existing classes then the most randomisation that is possible is to assign whole classes to the two conditions. This would mean the units of analysis were whole classes (one in each condition). To consider this a true experiment (meeting the requirement of randomisation, see Figure) there would need to be one measure of learning from each class, but it would be difficult to use statistics to infer anything useful when comparing just two values" (Taber, 2019, p.84)

"A random control trial (R.C.T.) is an experiment where the units of analysis are randomly assigned to different conditions, and statistical methods are used to determine whether any overall difference in the measured outcomes in those conditions is (probably) due to the intervention….A R.C.T. is referred to as a 'true experiment' because there is randomisation of the 'units of analysis' (people, classes, schools, etc.) to conditions" (Taber, 2019, p.73).