What was wrong with The Loneliness Experiment?
Keith S. Taber
The loneliness experiment, a.k.a. The BBC Loneliness Experiment was a study publicised through the BBC (British public service broadcaster), and in particular through it's radio programme All in the Mind, ("which covers psychology, neuroscience & mental health" according to presenter, Claudia Hammond's website.)1 It was launched back in February 2018 – pre-COVD.2
Claudia Hammond describes herself as an "award-winning broadcaster, author and psychology lecturer". In particular "She is Visiting Professor of the Public Understanding of Psychology at the University of Sussex" where according to the University of Sussex "the post has been specially created for Claudia, who studied applied psychology at the University in the 1990s", so she is very well qualified for her presenting role. (I think she is very good at this role: she has a good voice for the radio and manages to balance the dual role of being expert enough to exude authority, whilst knowing how to ask necessarily naive questions of guests on behalf of non-specialist listeners.)
A serious research project
The study was a funded project based on a collaboration between academics from a number of universities, led by Prof Pamela Qualter, Professor of Education at the Manchester Institute of Education at the University of Manchester. Moreoever, "55,000 people from around the world chose to take part in the BBC Loneliness Experiment, making it the world's largest ever study on loneliness" (https://claudiahammond.com/bbc-loneliness-experiment/)
Loneliness is a serious matter that affects many people, and is not be made light of. So this was a serious study, on an important topic – yet every time I heard this mentioned on the radio (and it was publicised a good deal at the time) I felt myself mentally (and sometimes physically) cringe. Even without hearing precise details of the research design, I could tell this was simply not a good experiment.
This was not due to any great insight on my behalf, but was obvious from the way the work was being described. Readers may wish to see if they can spot for themselves what so irked me.
What is the problem with this research design?
This is how the BBC described the study at its launch:
The Loneliness Experiment, devised by Professor Pamela Qualter and colleagues, aims to look at causes and possible solutions to loneliness. And we want as many people as possible to fill in our survey, even if they've never felt lonely, because we want to know what stops people feeling lonely, so that more of us can feel connected.
This is how Prof. Hammond described the research in retrospect:
55,000 people from around the world chose to take part in the BBC Loneliness Experiment, making it the world's largest ever study on loneliness. Researchers from the universities of Manchester, Brunel and Exeter, led by Professor Pamela Qualter and funded by the Wellcome Trust, developed a questionnaire asking people what they thought loneliness was, when they felt lonely and for how long.
And this is how the work is described on the University of Manchester's pages:
The Loneliness Experiment was a study conducted by BBC Radio 4's All in the Mind….
The study asked respondents to give their opinions and record their experiences of loneliness and related topics, including friendship, relationships, and the use of technology – as well as recording lifestyle and background information. Respondents also engaged in a number of experiments.
The survey was developed by Professor Pamela Qualter, from The University of Manchester's Manchester Institute of Education (MIE), with colleagues from Brunel University London, and the University of Exeter. The work was funded by a grant from The Wellcome Trust.
https://www.seed.manchester.ac.uk/education/research/impact/bbc-loneliness-experiment/
When is an experiment not an experiment?
These descriptions make it obvious that the The Loneliness Experiment was not an experiment. Experiment is a specific kind of research – a methodology where the researchers randomly assign participants randomly to conditions, intervene in the experimental condition,and take measurements to see what effect the intervention has by comparing with measurements in a control condition. True experiments are extremely difficult to do in the social sciences (Taber, 2019), and often quasi-experiments or natural experiments are used which do not meet all the expectations for true experiments. BUT, to be an experiment there has to be something that can be measured as changing over time in relation to specified different conditions.
Experiment is not the only methodology used in research – there are also case studies, there is action research and grounded theory, for example – and non-experimental research may be entirely appropriate in certain situations, and can be of very high quality. One alternative methodology is the survey which collects data form a sample of a population at some particular time. Although surveys can be carried out in various ways (for example, through a series of observations), especially common in social science is the survey (a methodology) carried out by using participant self-responses to a questionnaire (a research instrument).
it is clear from the descriptions given by the BBC, Professor Hammond and the University of Manchester that the The Loneliness Experiment was not actually an experiment at all, but basically a survey (even if, tantalisingly, the Manchester website suggests that "Respondents also [sic] engaged in a number of experiments". )
The answer to the question 'when is an experiment not an experiment?' might simply be: when it is something other than an experiment
What's in a name: does it really matter?
Okay, so I am being pedantic again.
But I do think this matters.
I think it is safe to assume that Prof. Hammond, Prof. Qualter and colleagues know the difference between an experiment and a survey. Presumably someone decided that labelling the research as the loneliness study or the loneliness survey would not be accessible (or perhaps not as impressive) to a general audience and so decided to incorrectly use the label experiment as if experiment was synonymous with study/research.
As a former research methods lecturer, that clearly irks as part of my job was to teach new researchers about key research concepts. But I would hope that people actually doing research or learning to do research are not going to be confused by this mislabelling.
But, as a former school science teacher, I know that there is widespread public misunderstanding of key nature of science terms such as theory and experiment. School age students do need to learn what is meant by the word experiment, and what counts as an experiment, and the BBC is being unhelpful in presenting research that is not experimental as an experiment – as this will simply reinforce common misconceptions of what the term experiment is actually used to denote in research .
So, in summary, I'll score The BBC Loneliness Experiment
- motivation – excellent;
- reach – impressive;
- presentation – unfortunate and misleading
Further reading:
Work cited:
Taber, K. S. (2019). Experimental research into teaching innovations: responding to methodological and ethical challenges. Studies in Science Education, 55(1), 69-119. doi:10.1080/03057267.2019.1658058 [Download manuscript version]
Note:
1: Websites cited accessed on 28th August, 2021.
2: It would have been interesting to repeat when so many people around the world were in 'lock-down'. (A comparison between pre-COVID and pandemic conditions might have offered something of a natural experiment.)