Keynote:
Student Research Symposium
Tennessee Association of Science Department Chairs
Research Symposium and Competition
(for Community College Undergraduates in Tennessee, USA)
April 2020
Download the talk text as a pdf file
Abstract
In the English-speaking world, the word 'science' (when used without qualification) generally means the natural sciences, although in some other contexts it can refer to STEM more widely, or even include the social sciences. The social sciences are often considered to be modelled upon, or even to aspire to be like, the natural sciences. Yet, those trained in the natural sciences may perceive the social sciences to be imprecise, vague, verbose, and inconclusive. Studies may seem to lack validity or robustness: and the greater challenge of reaching consensus in the social sciences (when compared to the natural sciences) may appear to offer evidence of such a view. Yet when natural scientists venture into the social sciences they often show a bias towards inappropriate methodology that is not fit for purpose. It is suggested that scientific training often lacks a level of meta-thinking (thinking outside the paradigm, so to speak) that blinkers natural scientists and channels a kind of tunnel-vision. This produces highly effective researchers within well-established, stable ('mature') fields – but poor preparation for appreciating more diverse areas of scholarship and research. This lacuna in scientific training is unhelpful when scientists venture into research focused on social phenomena (such as in science education). Moreover, this blind-spot in scientific preparation supports a short-circuiting of the logic of research that could impede major developments within the natural sciences themselves.
The conference proceedings may be accessed here.
Text of the talk
Introduction
One should not over-generalise, but I want to start with some provocation:
- Natural scientists are biased.
- Natural scientists who move into education are not well prepared to become researchers.
I am sure there are many exceptions to these 'rules', but there is enough truth in these accusations for me to be prepared to make the claims.
You may well, and reasonably, ask – what is my evidence? So, let me give you a little background. I consider myself a scientist – I am a Chartered Scientist – and a champion of science, so have no wish to do any 'scientist-bashing' for its own sake. I loved science at school, and did a degree in chemistry (in England, where it is normal to take an undergraduate degree primarily in one discipline). I then trained as a secondary school teacher of chemistry and physics, before teaching science in schools and a further education college for 17 years. Although I was teaching science, rather than doing science, I certainly identified as a scientist as well as a teacher. However, during this period I became involved in research.
I undertook an MSc degree where I explored the factors which led to most girls in a school where I was teaching dropping physics at age fourteen (as the curriculum then allowed). This work was undertaken as a case study. In the way it is sometimes loosely used, the term 'case study' can imply little more than an example, but in research in the social sciences this implies much more than this: a particular kind of research strategy or methodology that involves exploring one instance of some phenomenon in some depth. For my doctorate I explored my college students' developing understandings about the topic of chemical bonding. So I was a scientist, and I became a researcher: but not a science researcher, rather an education researcher.
For the past 20 years I have been teaching at the University of Cambridge. For much of the first decade I was heavily involved in teacher preparation – in particular working with science graduates preparing for school teaching at either secondary or middle school levels. This was for a 'postgraduate' qualification, so students were expected to engage with the research literature and complete a small-scale research project related to the supervised teaching they undertook on school placements. I therefore had much experience of graduates arriving with very good science degrees, and often higher degrees, and having their first taste of research into science classrooms. Indeed, recognising the cultural jump they were having to make motivated me to write an introductory textbook on educational research in classrooms. So, I was teaching science graduates who were doing research. But they were doing educational research, not science research.
You may notice my phrasing there. I say they were not doing science research, but avoided saying that they were not doing scientific research. I think 'science research' implies research into natural phenomena, where perhaps research outside the natural sciences can also be scientific. That is clearly a big and much debated question, so I am simply going to offer a quick 'pass notes' version based on my own perspective. I am going to suggest that
research is scientific if it is carried out systematically, and it involves empirical work which is both informed by, and intended to potentially develop, theoretical understandings.
For the past decade the core of my work has been in teaching research methodology, in particular in a one year course in Educational Research either leading to an MPhil [or MEd] degree, or acting as the training component of a doctorate. The course members were undertaking research projects across a wide range of educational topics – and most were not working in science education. They were graduates in a wide array of disciplines. Some of them were natural scientists, but most were not.
So, I now have a rather manifold, though I hope not confused, professional identity. I am a scientist who does not actually do science, and a teacher who no longer works in schools teaching science yet still thinks of himself as a science teacher. I think I can justify this, as I do teach students how to do research, and research, surely, is synonymous with science. Or at least, research done properly, rigorous research, robust research, is science, isn't it?
What does research training involve?
I want to now offer two stories, two narratives of how new researchers become inducted into the work of research. These are fictitious stories, but I believe they reflect common experiences – so although they cannot be said to be factual, I think they do represent some kind of 'truth'. I am contrasting two traditions, and I am aware that any claim that all new researchers' experiences will match one or other approach would be (another) gross over-generalisation. And over-generalisation is antithetical to careful scientific work.
But, this is just a model: a pedagogic model to help make a point – and like all models it is strictly false, yet can still be useful. There is an idea in education that in order to make the unfamiliar, familiar one should aim for what might be called an 'optimum level of simplification', where one simplifies a complex situation sufficiently to introduce it to others, whilst seeking to retain the essence of what is being explained. A famous American educationalist, Jerome Bruner, used the term 'intellectually honest' for simplifications that avoid distorting that essence. So, here are two caricatures, or perhaps we might call them vignettes, and I hope presenting them offers an intellectually honest simplification.
Researching education
Graduate Adam decided to do research in education. He made a proposal for a topic he really wanted to research, and then identified a potential supervisor who was working in related areas and who he could interest in his topic. After admission to the postgraduate programme, Adam was advised to read widely around his topic and to develop a conceptual framework for his research. This involved considering a wide range of ways of thinking about his topic, and how the key concepts might be understood, and how they might be related.
Adam's supervisor encouraged him to put his own stamp on the project, and to argue for which ideas and concepts he thought were most important, and how they should be best juxtaposed to address the issues Adam was interested in. Most importantly, Adam had to take a view about how much he wanted to structure his data ahead of carrying out any empirical work.
- Would the project be largely deductive, with a tightly defined research question that was framed through a very specific theoretical perspective drawn from the literature (perhaps needing an early judgement about which of a number of theorists to follow), with the consequence that it was predetermined how the data should be understood, and how results would arise from the analysis.
- Or, alternatively, was a more exploratory approach indicated, where the way data were to be understood would emerge later from a more open-ended process with aspects of conceptualisation deferred until later in the process in the light of the ongoing analysis.
Whichever approach was taken there was a need to select an overall methodology for the study, and build a research design drawing upon different data collection and analytical tools – although the extent to which this was a rigid design, or a more fluid, evolving design, would reflect the earlier decision about whether the project was primarily deductive or inductive in nature. All these decisions would need to be discussed and justified in the eventual thesis. Moreover, great consideration had to be given to ethical issues, in particular where ethical considerations might seem to conflict with the methodological choices needed to ensure a rigorous chain of logic in drawing conclusions form the study.
Induction in scientific research practice
Our other fictitious, but let's say archetypical, Graduate, Eve decided to do research in an area of the natural sciences. She had some topics and themes she was interested in, but pragmatically needed to decide which field or research programmes she might work in, and then see which projects were being offered by potential lab heads. Once she started work, she was inducted into a team of several academics, with their post-docs and graduate students at various stages in their projects – all of which were designed to be addressing different aspects of the same core research questions that external funding had facilitated the lab to become equipped to tackle.
Through all this she would be helped by the fortnightly lab meetings where everyone updated on their sub-project, and by working in close proximity to the handful of more experienced researchers already established in the lab, who were further ahead in their induction (or indoctrination, if you prefer).
Eve was told to read extensively, but focusing on all the past work that had been done by the lab, and other competing labs, working in this specific programme of research. She needed to understand the core concepts used in the field, and precisely how the specialist technical vocabulary was employed. She also needed to develop first competence, and then expertise, in using the specific techniques and apparatus used in the lab, and had to learn to 'see' the relevant artefacts in the data that count as providing results in this field.
She would also learn, partly through informal conversations, which papers in the field were trusted, and which had results that were not considered so reliable; which researchers elsewhere were held in high esteem; and which journals and conferences were suitable for reporting research: that is, those outputs where the concepts, laboratory techniques, analytical methods, vocabulary, inherent assumptions, turns of phrase, and ways of presenting results, that were adopted as norms in this laboratory would be widely understood, appreciated, and, in particular, shared, with other researchers in the field.
Two different forms of life?
Now, at this juncture, I want to make an important point about my vignettes. I am certainly not trying to imply that one of these traditions, these ways of working, or perhaps 'forms of life', is inherently superior to the other. Things are done differently for reasons, and although sometimes that could just mean 'custom and practice', there can be both pragmatic and principled reasons why different approaches tend to be adopted in different contexts.
One obvious difference is that cutting edge science often involves specialist, and sometimes extremely costly, equipment, so a lab is set up for one kind of work, and cannot readily shift to a very different mode of working. A researcher who wishes to work on a dissimilar project using different techniques would not usually have that option unless they can themselves attract substantial funding, so must compromise on the project or move to another lab – and perhaps that means moving some geographical distance. In terms of funding, it is very common for grants to be awarded to science groups to work on particular projects, and then for the group to advertise for PhD students to take on those projects. This certainly can also happen in the social sciences, but it is more common that doctoral students attract funding individually based on a project they are proposing, and so a supervisor may have a choice of whether to take on a student or project (and can certainly help shape it), but not usually the power to define the project for the student.
However, moving away from pragmatic concerns, there is a more fundamental difference between Adam and Eve's experience. Eve's training is primarily in the research programme, in the specialist sub-field working within what has been termed a paradigm or disciplinary matrix. It is intended to be highly focused and produce a specialist researcher in that field.
To refer to a person as being, say, a PhD chemist is a bit like referring to someone as a musical composer – it tells you something about their work, but not whether they are going to produce something like the Art of Fugue, Finlandia, Koyaanisqatsi, or For No One. That is not a question of quality (I have deliberately cited examples I hold in high regard) but just that whilst to the layperson 'chemistry' or 'physics' or other science disciplines may seem specialist areas of interest, to someone within the discipline they show as much diversity as when comparing Titian, Canaletto, van Gogh, and Pollock – a discipline is like love, covering a multitude of sins.
Scientific training tends to be narrow
Science proceeds, largely at least, through the work of specialists not polymaths, and so scientific training needs to be narrow and deep. This is a necessity, and so a virtue in scientific training – albeit that one might suggest that 'narrow and deep' should never exhaust any scholar's range. Eve was not asked to consider if she thought her research question was best addressed by a deductive approach involving a well-defined research design or a more exploratory approach where conceptualisation of the project was best deferred till the data were in and analysis was underway.
Eve is doing science, and research questions are asked by setting up tightly designed and carefully controlled experiments that allow one to deduce conclusions only because you decide in advance exactly what outcomes will be considered to tend to support or refute a hypothesis. Of course, there is a creative element in posing the hypothesis and designing the experiment, but one is not meant to be creative in finding ways to best interpret the data 'after the event': that would seem to be cheating – to be scientific malpractice.
What about Adam? Does research training in the social sciences then require generalists – trained to be, perhaps, broad and shallow? That does not seem right. Anyone being awarded a PhD, and so formally welcomed into the Academy, should at that moment be the world's greatest expert on their particular research focus, which will likely be as specifically narrow in any field of scholarship – whether natural science, social science, or humanities.
But Adam wanted to work in education, which is a very broad field. It is not a fundamental area of scholarship like biochemistry or geology but an area that draws upon other foundations (traditionally philosophy, sociology, history and psychology – but increasingly economics and other areas).
Educational scholarship is promiscuous
So educational theory is informed by a wide range of theorists from different areas of scholarship. Perhaps in his project Adam would choose as his theoretical framework the ideas of a sociological thinker like Foucault or Bourdieu or a developmental psychologist such as Piaget, or perhaps a more specifically educational psychologist like Bruner or … Such a choice will be important for framing the project as different theories highlight different features of interest in educational phenomena, or conceptualise the same phenomenon in different ways: sometimes subtly different ways, but sometimes radically different ways.
Adam's supervisor has her own ideas about how to best understand the research focus, but is open to Adam making different choices. Eve, in contrast, is probably not going to readily persuade her supervisor to commit lab resources to work that is not clearly aligned with the other research going on there, and for which funding was granted. It is not for the apprentice to suggest to the master how the work should be undertaken.
Educational research uses a diverse toolkit
Research into aspects of education has called upon methods developed in the natural sciences, but also in other areas such as sociology, psychology and anthropology. Adam could do some kind of experiment, but more likely might be using a questionnaire, doing observational work, interviews, documentary analysis or some combination of such techniques. Adam is aware that drawing conclusions from the data generated by such techniques often requires a good deal of careful interpretation, opening his work up to admitting a degree of subjectivity that would likely be seen as a fatal flaw if it was judged to be present in Eve's thesis. Yet, Adam will argue that because of the nature of his research focus, and the inherent limitations to his methods, it is only possible to build in some checks and balances to limit subjectivity – and then be explicit about not being able to eliminate it completely. Adam's supervisor tells him that all research is inevitably subject to caveats, and a good thesis does the work of highlighting and discussing these rather than ignoring or downplaying them.
Eve avoids overt suspicions of subjectivity, by using objective measuring instruments and carrying out a controlled experiment. She can do that because she is working in laboratory conditions where she can isolate specimens and identify, measure, and control, the relevant variables. As she is working with natural phenomena, these are instances of so-called 'natural kinds' which means they have inherent properties, and are well-defined. A particular strain of bacterium, a particular alloy, a particular concentration of a solution of a specific oxidising agent, can be clearly defined in a thesis or journal paper, so that other scientists can (in principle) replicate and check her work.
Eve tells Adam that the reason he seems to be spending the first half of his PhD making choices about what to do, rather than getting on with the work, is because he is working in a less mature field. The natural sciences are mature, with well-established research programmes, so we know how to make progress. We know how to understand phenomena. We know what variables matter. We know how to define, identify, and measure. For Eve the challenge is getting the experiments she needs to undertake to actually 'work'. Despite all this maturity, nature has to be enticed to give up her secrets. The apprentice needs to learn the master's secrets in order to coax the precise instruments to work, and to get those natural phenomena to reliably behave naturally when she is observing – but once she has developed the laboratory scientist's green fingers, it is then just a matter of collecting her data, and writing up. She teases Adam, that, by contrast, he has got half-way through his PhD before he really knows what he is actually looking for!
Adam, however, knows that it is not just a matter of the maturity of the field. He knows that he is working with people rather than natural objects, and that makes his interaction with his 'subjects' (or participants, as he would say, pointing out the ethical implications of treating fellow human beings as data-fodder) more problematic. Unlike the specimens in Eve's lab, the people Adam has to collect data from behave differently with different people. People being observed may respond differently depending on the age or dress of the observer. Replace one interviewer with another, and you get different rapport which leads to different interview responses.
There are no scientific instruments that can tease out people's ideas, views, opinions, or beliefs, if they do not wish to share them, and unlike laboratory specimens, people may present with the impression they want to give. It is as if a copper rod was found to be made of a material with element number 79, because it aspired to be gold, or it was embarrassed to be considered a base metal. Natural scientists can completely discard such ridiculous possibilities. Experimental subjects in science labs rarely care about the outcome of the research, but in educational experiments we know that the expectations of the participants can substantially change outcomes.
Researching complex systems
Educational phenomena are highly complex – teaching is a complex system. It is not a matter of, say, whether a classroom should always be observed from a psychological or sociological perspective – as both individual psychology, social psychology, and sociology, are highly relevant to what is going on there. The question is which perspective may be most valuable when considering certain research questions.
Any particular teaching episode is heavily embedded in a context which matters. If a researcher is observing a group of students working on some science task in a classroom – it would likely change what was observed if that group were moved to another class, another teacher, another school, another curriculum context, another language context, and so forth. Observing one biology class of 14 year old students is not the same as observing another biology class of 14 year old students. Interviewing one teacher about their classroom experiences is not equivalent to interviewing a different teacher. Testing one students' understanding of Newton's laws of motion is not equivalent to testing another student's understanding.
The instances studied in education are not interchangeable in the way specimens often are in the natural sciences. A specimen of lead at 293K will have a certain density, and a certain crystal structure, and a certain Young's modulus. A specimen of science learner at grade 12 will have an uncertain attitude to science, an uncertain confidence in their science, and an uncertain level of knowledge and understanding. This may not only change from student to student, but may appear to be different if you repeat your measurements a day or two later (or, possibly, even if you wait till after lunch).
Even if education is relatively immature compared with physics, no amount of leaving education to mature further will overcome any of these inherent complexities and challenges for researchers. This is the context for my initial suggestions that
- Natural scientists are biased.
- Natural scientists who move into education are not well prepared to become researchers.
Scientific training may not readily transfer to social contexts
Scientific training is necessary and appropriate for those working in a scientific field, but it is a very limited and impoverished preparation for research in education. Those in science who claim that educational research is not scientific because it uses such imprecise and subjective approaches, and that it could be made more scientific by adopting the gold standard of experimental random control trials (and I include the English govt. Department of Education among such claimants) are simply ignorant about that of which they talk. There are many experiments in the science education literature, and often those studies reflect bad science as they not only fail to address all the variables at work, but they often do not even offer relevant caveats and acknowledge the possible confounds that undermine their results.
Having worked with science graduates entering teaching, and science teachers in practice, for many years, there is clear tendency for them to bring the scientific paradigm of the experiment into their thinking about classroom research. So, often, they wish to undertake experiments with meaningless controls and inadequate sample sizes because they think experiments are scientific and offer definite results.
Often they are best advised to deliberately select a less definite approach, but one that (unlike an experiment) they have a reasonable chance of employing in a robust and rigorous way. Their scientific training often biases them to see all research problems as best treated as akin to laboratory work, and they are blinkered to readily appreciate why adopting what seems the most scientific approach in a context where its basic assumptions do not apply is actually a most un-scientific way of proceeding. If they want to do useful educational enquiry, they often need to swap their natural science goggles for some social science goggles, because only by overcoming their scientific bias, can they proceed to do educational research that deserves to be considered scientific.
Download the talk text as a pdf file
