What is wrong with 'practice' papers

One of my  publications is

This editorial is free access – anyone can access and download articles in the journal by registering (for free) with the Royal Society of Chemistry site.

Contents:

• Here's a good idea we had earlier
• Our students seem to like it
• It must be research, because it is action research
• It must be research, we did an experiment
• It works here too
• But it's a case study
• Research and practice papers

Practice papers

This editorial discusses the notion of 'practice' papers "meaning articles that report on aspects of the authors' own practices as chemistry educators" (p.639).

The editorial makes the point that the journal does not distinguish between 'research' and 'practice' papers as two separate categories, but rather as a research journal has criteria for publication that apply to submissions whether or not authors are discussing their own teaching practices.

"[Practice] articles are certainly welcome under the 'paper' category – as long as they meet the journal's criteria for what counts as research – so they may be 'practice papers', but as with all papers in CERP, they must also fit in a research journal"

p.639

When is teaching practice also research?

The article explains under what circumstances a 'practice' paper would be considered research. In simple terms, a straight description of practice, even some innovative practice, is not research as it lacks the critical and evaluative features that are needed to provide the kind of pubic knowledge that can usefully inform the practice (or research) of others.

'We did this', or even 'we did this, and we thought it went well' is insufficient. Even 'we did this, and we thought it went well, and the students seemed to like it' is of itself anecdote rather than research,

"Simply sharing teaching ideas and activities, important as it is, does not comprise research."

p.640

Yet accounts of innovative practice that explain the motivation for making changes and offer some robust evaluation of educational outcomes may count as research,

"…to make a case for the effectiveness of any innovation, there has to be systematic evaluation. That means data collection has to be planned carefully, using appropriate (valid, reliable) instrumentation, with careful (and explicitly explained) analysis, that allows robust conclusions to be drawn."

p.640

Context-directed research?

A particular issue is the distinction between context-directed research and theory-directed research – that is inquiry which is primarily about finding out what work in a specific situation as opposed to looking for more generalisable knowledge.

A study might take an idea which is reported in the literature as being useful in teaching, perhaps indeed one that has been found to be useful in a wide range of contexts, and test out whether it proves useful in some particular school, university, classroom where it had not been tried before.

"The literature suggests that pupils at secondary schools often fail to think enough about the ideas that practical work is meant to illustrate whilst they are doing the practicals… Yet we know that what is true in one context may not be true elsewhere. So we teachers at the Theresa May New Grammar School in Poortown have undertaken some research, and we have shown it is true here with our students doing their chemistry practical work as well."

p.642

At one level this is very useful in the actual research context as not all innovations are found to work as well everywhere, but there would be little point in the journals being full of thousands of 'replication' studies that did not go beyond reporting 'no, not here' or 'yes, here also'.¹ That said, such 'replications' studies are useful beyond the original context when they help determine the range of application of an innovation (which needs the study to give a detailed account of the teaching and learning context) or go beyond simple testing to examine mechanisms for why something does or does not work (see – Experimental research into teaching innovations: responding to methodological and ethical challenges).

"There is much potential for this kind of research in chemistry education and indeed more widely in science education…where the studies are principled because they explore practice in relation to identifiable variations (i.e. abstracted from particular contexts, so theoretical) rather than just because no one has tried it just here before."

p.643

High quality (practice) papers are published

The editorial concluded that, although some articles written as practice papers do not offer new knowledge to the community and so are not suitable for publication,

"In summary, CERP welcomes papers reporting high quality chemistry education research relevant to practitioners, and this certainly includes those where manuscripts report studies carried out into the researchers' own practice. … As long as the work meets the journal's usual criteria for publication, in principle, there is nothing wrong with practice papers.

p.644

Note:

¹ Strictly replication is usually taken to mean repeating the same thing , as closely to the original conditions, as possible. Strictly this is not possible in education (it would mean repeating the test with the same teachers and learners meeting the innovation, again, for the first time – a logical contradiction). Replication is more feasible in science -although studies into the history of science suggests that actually even in science 'replication' usually means 'repeating' the original work but with some at least minor innovative changes.