Tag: acids

POEsing assessment questions…

…but not fattening the cow

Keith S. Taber

A well-known Palestinian proverb reminds us that we do not fatten the cow simply by repeatedly weighing it. But, sadly, teachers and others working in education commonly get so fixated on assessment that it seems to become an end in itself.

Images by Clker-Free-Vector-Images from PixabayOpenClipart-Vectors and Deedster from Pixabay

A research study using P-O-E

I was reading a report of a study that adopted the predict-observe-explain, P-O-E, technique as a means to elicit "high school students' conceptions about acids and bases" (Kala, Yaman & Ayas, 2013, p.555). As the name suggests, P-O-E asks learners to make a prediction before observing some phenomenon, and then to explain their observations (something that can be specially valuable when the predictions are based on strongly held intuitions which are contrary to what actually happens).

Read about Predict-Observe-Explain

The article on the publisher website

Kala and colleagues begin the introduction to their paper by stating that

"In any teaching or learning approach enlightened by constructivism, it is important to infer the students' ideas of what is already known"

Kala, Yaman & Ayas, 2013, p.555
Constructivism?

Constructivism is a perspective on learning that is informed by research into how people learn and a great many studies into student thinking and learning in science. A key point is how a learner's current knowledge and understanding influences how they make sense of teaching and what they go on to learn. Research shows it is very common for students to have 'alternative conceptions' of science topics, and often these conceptions either survive teaching or distort how it is understood.

The key point is that teachers who teach the science without regard to student thinking will often find that students retain their alternative ways of thinking, so constructivist teaching is teaching that takes into account and responds to the ideas about science topics that students bring to class.

Read about constructivism

Read about constructivist pedagogy

Assessment: summative, formative and diagnostic

If teachers are to take into account, engage with, and try to reshape, learners ideas about science topics, then they need to know what those ideas are. Now there is a vast literature reporting alternative conceptions in a wide range of science topics, spread across thousands or research reports – but no teacher could possibly find time to study them all. There are books which discuss many examples and highlight some of the most common alternative conceptions (including one of my own, Taber, 2014)

A book that describes and discusses a good many alternative conceptions – but by no means all!

However, in any class studying some particular topic there will nearly always be a spread of different alternative conceptions across the students – including some so idiosyncratic that they have never been reported in any literature. So, although reading about common misconceptions is certainly useful to prime teachers for what to look out for, teachers need to undertake diagnostic assessment to find out about the thinking of their own particular students.

There are many resources available to support teachers in diagnostic assessment, and some activities (such as using concept cartoons) that are especially useful at revealing student thinking.

Read about diagnostic assessment

Diagnostic assessment, assessment to inform teaching, is carried out at the start of a topic, before the teaching, to allow teachers to judge the learners' starting points and any alternative conceptions ('misconceptions') they may have. It can therefore be considered aligned to formative assessment ('assessment for learning') which is carried out as part of the learning process, rather than summative assessment (assessment of leaning) which is used after studying to check, score, grade and certify learning.

P-O-E as a learning activity…

P-O-E can best support learning in topics where it is known learners tend to have strongly held, but unhelpful, intuitions. The predict stage elicits students' expectations – which, when contrary to the scientific account, can be confounded by the observe step. The 'cognitive conflict' generated by seeing something unexpected (made more salient by having been asked to make a formal prediction) is thought to help students concentrate on that actual phenomena, and to provide 'epistemic relevance' (Taber, 2015).

Epistemic relevance refers to the idea that students are learning about things they are actually curious about, whereas for many students following a conventional science course must be experienced as being presented with the answers to a seemingly never-ending series questions that had never occurred to them in the first place.

Read about the Predict-Observe-Explain technique

Students are asked to provide an explanation for what they have observed which requires deeper engagement than just recording an observation. Developing explanations is a core scientific practice (and one which is needed before another core scientific practice – testing explanations – is possible).

Read about teaching about scientific explanations

To be most effective, P-O-E is carried out in small groups, as this encourages the sharing, challenging and justifying of ideas: the kind of dialogic activity thought to be powerful in supporting learners in developing their thinking, as well as practicing their skills in scientific argumentation. As part of dialogic teaching such an open-forum for learners' ideas is not an end in itself, but a preparatory stage for the teacher to marshal the different contributions and develop a convincing argument for how the best account of the phenomenon is the scientific account reflected in the curriculum.

Constructivist teaching is informed by learners' ideas, and therefore relies on their elicitation, but that elicitation is never the end in itself but is a precursor to a customised presentation of the canonical account.

Read about dialogic teaching and learning

…and as a diagnostic activity

Group work also has another function – if the activity is intended to support diagnostic assessment, then the teacher can move around the room listening in to the various discussions and so collecting valuable information on what students think and understand. When assessment is intended to inform teaching it does not need to be about students completing tests and teachers marking them – a key principle of formative assessment is that it occurs as a natural part of the teaching process. It can be based on productive learning activities, and does not need marks or grades – indeed as the point is to help students move on in their thinking, any kind of formal grading whilst learning is in progress would be inappropriate as well as a misuse of teacher time.

Probing students' understandings about acid-base chemistry

The constructivist model of learning applies to us all: students, teachers, professors, researchers. Given what I have written above about P-O-E, about diagnostic assessment, and dialogic approaches to learning, I approached Kala and colleagues' paper with expectations about how they would have carried out their project.

These authors do report that they were able to diagnose aspects of student thinking about acids and bases, and found some learning difficulties and alternative conceptions,

"it was observed that eight of the 27 students had the idea that the "pH of strong acids is the lowest every time," while two of the 27 students had the idea that "strong acids have a high pH." Furthermore, four of the 27 students wrote the idea that the "substance is strong to the extent to which it is burning," while one of the 27 students mentioned the idea that "different acids which have equal concentration have equal pH."

Kala, Yaman & Ayas, 2013, pp.562-3

The key feature seems to be that, as reported in previous research, students conflate acid concentration and acid strength (when it is possible to have a high concentration solution of a weak acid or a very dilute solution of a strong acid).

Yet some aspects of this study seemed out of alignment with the use of P-O-E.

The best research style?

One feature was the adoption of a positivistic approach to the analysis,

Although there has been no reported analyzing procedure for the POE, in this study, a different [sic] analyzing approach was offered taking into account students' level of understanding… Data gathered from the written responses to the POE tasks were analyzed and divided into six groups. In this context, while students' prediction were divided into two categories as being correct or wrong, reasons for predictions were divided into three categories as being correct, partially correct, or wrong.

Kala, Yaman & Ayas, 2013, pp.560

GroupPredictionReasons
❀correctcorrect
❁correctpartially correct
✯correctwrong
✿wrongcorrect
❂wrongpartially correct
✤wrongwrong
"the written responses to the POE tasks were analyzed and divided into six groups"

There is nothing inherently wrong with doing this, but it aligns the research with an approach that seems at odds with the thinking behind constructivist studies that are intended to interpret a learner's thinking in its own terms, rather than simply compare it with some standard. (I have explored this issue in some detail in a comparison of two research studies into students' conceptions of forces – see Taber, 2013, pp.58-66.)

In terms of research methodology we might say it seem to be conceptualised within the 'wrong' paradigm for this kind of work. It seems positivist (assuming data can be unambiguously fitted into clear categories), nomothetic (tied to 'norms' and canonical answers) and confirmatory (testing thinking as matching model responses or not), rather than interpretivist (seeking to understand student thinking in its own terms rather than just classifying it as right or wrong), idiographic (acknowledging that every learner's thinking is to some extent unique to them) and discovery (exploring nuances and sophistication, rather than simply deciding if something is acceptable or not).

Read about paradigms in educational research

The approach used seemed more suitable for investigating something in the science laboratory, than the complex, interactive, contextualised, and ongoing life of classroom teaching. Kala and colleagues describe their methodology as case study,

"The present study used a case study because it enables the giving of permission to make a searching investigation of an event, a fact, a situation, and an individual or a group…"

Kala, Yaman & Ayas, 2013, pp.558
A case study?

Case study is a naturalistc methodology (rather than involving an intervention, such as an experiment), and is idiographic, reflecting the value of studying the individual case. The case is one from among many instances of its kind (one lesson, one school, one examination paper, etc.), and is considered as a somewhat self contained entity yet one that is embedded in a context in which it is to some extent entangled (for example, what happens in a particular lesson is inevitably somewhat influenced by

  • the earlier sequence of lessons that teacher taught that class {the history of that teacher with that class},
  • the lessons the teacher and student came from immediately before this focal lesson,
  • the school in which it takes place,
  • the curriculum set out to be followed…)

Although a lesson can be understood as a bounded case (taking place in a particular room over a particular period of time involving a specified group of people) it cannot be isolated from the embedding context.

Read about case study methodology

Case study – study of one instance from among many

As case study is idiographic, and does not attempt to offer direct generalisation to other situations beyond that case, a case study should be reported with 'thick description' so a reader has a good mental image of the case (and can think about what makes it special – and so what makes it similar to, or different from, other instances the reader may be interested in). But that is lacking in Kala and colleagues' study, as they only tell readers,

"The sample in the present study consisted of 27 high school students who were enrolled in the science and mathematics track in an Anatolian high school in Trabzon, Turkey. The selected sample first studied the acid and base subject in the middle school (grades 6 – 8) in the eighth year. Later, the acid and base topic was studied in high school. The present study was implemented, based on the sample that completed the normal instruction on the acid and base topic."

Kala, Yaman & Ayas, 2013, pp.558-559

The reference to a sample can be understood as something of a 'reveal' of their natural sympathies – 'sample' is the language of positivist studies that assume a suitably chosen sample reflects a wider population of interest. In case study, a single case is selected and described rather than a population sampled. A reader is left to rather guess what population being sampled here, and indeed precisely what the 'case' is.

Clearly, Kala and colleagues elicited some useful information that could inform teaching, but I sensed that their approach would not have made optimal use of a learning activity (P-O-E) that can give insight into the richness, and, sometimes, subtlety of different students' ideas.

Individual work

Even more surprising was the researchers' choice to ask students to work individually without group discussion.

"The treatment was carried out individually with the sample by using worksheets."

Kala, Yaman & Ayas, 2013, p.559

This is a choice which would surely have compromised the potential of the teaching approach to allow learners to explore, and reveal, their thinking?

I wondered why the researchers had made this choice. As they were undertaking research, perhaps they thought it was a better way to collect data that they could readily analyse – but that seems to be choosing limited data that can be easily characterised over the richer data that engagement in dialogue would surely reveal?

Assessment habits

All became clear near the end of the study when, in the final paragraph, the reader is told,

"In the present study, the data collection instruments were used as an assessment method because the study was done at the end of the instruction/ [sic] on the acid and base topics."

Kala, Yaman & Ayas, 2013, p.571

So, it appears that the P-O-E activity, which is an effective way of generating the kind of rich but complex data that helps a teacher hone their teaching for a particular group, was being adopted, instead, as means of a summative assessment. This is presumably why the analysis focused on the degree of match to the canonical science, rather than engaging in interpreting the different ways of thinking in the class. Again presumably, this is why the highly valuable group aspect of the approach was dropped in favour of individual working – summative assessment needs to not only grade against norms, but do this on the basis of each individual's unaided work.

An activity which offers great potential for formative assessment (as it is a learning activity as well as a way of exploring student thinking); and that offers an authentic reflection of scientific practice (where ideas are presented, challenged, justified, and developed in response to criticism); and that is generally enjoyed by students because it is interactive and the predictions are 'low stakes' making for a fun learning session, was here re-purposed to be a means of assessing individual students once their study of a topic was completed.

Kala and colleagues certainly did identify some learning difficulties and alternative conceptions this way, and this allowed them to evaluate student learning. But I cannot help thinking an opportunity was lost here to explore how P-O-E can be used in a formative assessment mode to inform teaching:

Yes, I agree that "in any teaching or learning approach enlightened by constructivism, it is important to infer the students' ideas of what is already known", but the point of that is to inform the teaching and so support student learning. What were Kala and colleagues going to do with their inferences about students ideas when they used the technique as "an assessment method … at the end of the instruction".

As the Palestinian adage goes, you do not fatten up the cow by weighing it, just as you do not facilitate learning simply by testing students. To mix my farmyard allusions, this seems to be a study of closing the barn door after the horse has already bolted.

Work cited

Not me, I'm just an ugly chemist

Keith S. Taber

Actress Francesca Tu playing an 'ugly chemist', apparently.

The 1969 film 'The Chairman' (apparently released in the UK as 'The Most Dangerous Man in the World') was just shown on the TV. I had not seen it before, but when I noticed it was on I vaguely recalled having heard something about it suggesting it was a film worth watching, so thought I would give it a try. And it had "that nice Gregory Peck" in it, which I seem to recall was the justification given for one of my late wife's sweet little Aunties going to see 'The Omen' (wasn't that also about the The Most Dangerous Man in the World?).

Nobel prize winner AND man of action

Dr John Hathaway (played by Gregory Peck): scientist and international man of mystery

Peck plays a Nobel laureate chemist, so I got interested. He had received a letter from a Chinese scientist, an old mentor who had worked with him at Princeton, warning him not to go to visit him in China, which (a) piqued his interest as (i) he had had no contact with the colleague for a decade, and (ii) he had no plans to go to China, and (b) told us viewers he would be off to China.

Peck's character, Hathaway, is an American who is currently a visiting professor at the University of in London. He contacts his embassy, suspecting there must be something of international significance in the message.

Hathaway's love interest (played by Anne Heywood) is seen teaching in the biophysics department

It transpires that this Nobel prize winning chemist had some kind of background in "the game" – intelligence work (of course! Well, at least this gets away from the stuffy stereotype of the scientist who never leaves the lab.), but had reached an epiphany three years earlier when his wife had been killed in a road accident while he was driving, and the experience of being with her as she died had led to him deciding that every life was unique and precious (as he later explained to Mao Zedong, the eponymous Chairman of the title) and he would no longer take on a job that would oblige him to kill. (Later in the film Hathaway seemed to have forgotten his high principles when he accepted a pistol as he made an escape in a stolen armoured car.) The intelligence communities had become aware that China had identified a natural product that could be extracted in tiny quantities, an enzyme which allowed any crop to be grown under any conditions.

The film seemed to be intended to make some serious points about detente, the cold war, the cultural revolution and the cult of Mao, and political and moral imperatives.

It is the responsibility of all to cultivate themselves, and study Marxism-Leninism deeply. / [Thinks: Sure, as soon as we've finished cultivating this rice.]
The allies argue that China will keep the new discovery to itself and use it to bring developing countries with food shortages into its sphere of influence, and Hathaway seems motivated to ensure all of humanity should share the benefits, thus he accepts the mission to go to China; later Mao agrees to provide a written promise that if Hathaway helps in the research then he can leave China at any time he likes and take with him whatever information he wishes to share with the world.

For the rest of the film to make any sense, Hathaway and the viewer have to assume that the promise and document will not be honoured (and it seems to be assumed that a character simply suggesting this is all Hathaway, or indeed any of us, need to be convinced of this). Yet, (SPOILER ALERT) when Hathaway is safely back in London, and has decoded the structure, he is told that the Western authorities have decided not to share the discovery.

I was not sure what a young audience who do not remember the context might make of some aspects of the film. We are told that the operation to obtain the enzyme, operation Minotaur *, has according to the US officer in charge cost half a billion federal dollars (which seems a lot for 1969, even allowing for some exaggeration) and was supported by the UK with a contribution a British intelligent officer suggests was likely "two pounds ten" (i.e., £2.50).

I wondered whether Chinese agents actually operated so easily in moving into and out of Hong Kong as is suggested, and there was some interesting brief news footage  playing on a hotel television suggesting (British) Hong Kong police were responding to civil unrest in a way that does not seem so different from contemporary reports under the already notorious 2020 Hong Kong national security law.

Anyway, I will try and avoid too many plot spoilers, but suffice to say I was interested and intrigued in how matters would pan out for the first three quarters of the film (until people started firing guns and throwing grenades, at which point I lost any investment I'd had in what would happen.)

Science in the media in 1969

The science in the film was far-fetched, but perhaps not too far fetched for a general audience in 1969. 1969 was after all, a different age. (In 1969 the Beatles were still together, 'In the Court of the Crimson King' was released, and NASA's landing on the moon showed just what the USA could achieve when a President believed in, and encouraged, and resourced, the work of scientists and engineers.)

A transmitter made of undetectable plastic parts, suppposedly

Hathaway was bugged (through a sinus implant) such that his US /UK handlers (and USSR observer) could hear everything he said and everything said to him from half a world away through a bespoke satellite that the Chinese had not noticed recently appearing over their territory. The Americans initially had serious trouble with signal:noise and just made out the odd consonant, and so could not understand any speech, but a UK intelligence officer suggested simply filling in the gaps with uniform white noise, which, amazingly, and (even more amazingly) immediately at first attempt, gave a much cleaner sound than I can get on FaceTime or Zoom or Skype today (Implied message: the British may be the poor relatives, but have the best ideas?)

High stakes communication

What Hathaway did not know (but perhaps he should have been paying more attention when he was told the implanted transmitter was a 'remedy' in case the Chinese would not let him leave the country?) was that the implanted transmitter also had an explosive device that could be used if he needed to be terminated.

Indeed there was supposedly enough plastic explosive that when Hathaway was invited to meet Chairman Mao (was he meant to be 'the most dangerous man in the world'?) it raised the issue of whether the device should be used to remove the Chairman as he played table tennis with Hathaway (asking us to believe that democratic governments might sanction the violent summary execution of perceived enemies, without due legal process, in foreign lands) *.

Is it stretching credibility to believe that democratic governments would sanction the violent summary execution of perceived enemies, without due legal process, on foreign soil?

The command code to explode the device was stored on magnetic tape that took over thirty seconds to execute the instructions (something that seems ridiculous even for 1969, and was presumably only necessary to provide faux tension at the point where the clock counts down and the audience are supposed to wonder if the British and Americans are going to have to kill the film's star off before the movie is over).

Equally ridiculous, the implant supposedly had the same density as human tissue so that it would not show up on  X-rays. (A wise precaution: when in  Hong Kong, Hathaway is lured to some kind of decadent, Western, casino-cum-brothel where Chinese agents manage to covertly X-ray him from the next room as he enjoys a bowl of plain rice with a Chinese intelligence officer – quite a technical feat).

Of course, human tissue is not all of one 'density' (in the sense of opaqueness to X-rays), or else there would be little point in using X-rays in medical diagnosis – actually a sinus should show up on an X-ray as an empty cavity!

Would blocked sinuses show on an X-ray?

Highly technical information appeared on screens at the listening post as displays little more complex than sine waves – not even the Lissajous figures so popular with 1970s sci-fi programmes.

I think it's just the carrier wave, sir

At one point Hathaway broke into a room through a thick solid metal floor by using just a few millilitres of nitrohydrochloride acid (aqua regia) that was apparently a standard bench reagent in the Chinese biochemistry laboratory (these enzymes must be pretty robust, or perhaps Professor Soong had a side project that involved dissolving gold), and which Hathaway was quite happy to carry with him in a small glass bottle in his jacket pocket. The RSC's Education in Chemistry magazine warns us that "because its components are so volatile, [aqua regia] is usually only mixed immediately prior to use". Risk assessment has come on a lot since Dr Hathaway earned his Nobel.

Laboratory safety glasses: check. Bench mat: check. Gloves: check. Lab coat: check. Fume cupboard: check.

The focal enzyme was initially handled rather well – the molecular models looked convincing enough, and the technical problem of scaling up by synthesising it seemed realistic. The Chinese scientist could not produce the enzyme in quantity and hoped Hathaway could help with the synthesis – a comparison was made with how producing insulin originally involved the sacrifice of many animals to produce modest amounts, but now could be readily made at scale. I seem to recall from my natural products chemistry that before synthetic routes were available, sex hormones were obtained by collecting vast amounts of 'material' from slaughterhouses and painstakingly abstracting tiny quantities – think the Curies, but working with with tonnes of gonads rather than tonnes of pitchblende.

Before Hathaway had set out on his mission he had pointed out that the complexity of an enzyme molecule was such that he could never memorise the molecular structure as it would contain anything from 3000 to 400 000 atoms. So, the plot rather fell apart at the end (SPOILER ALERT) as he brings back a copy of Mao's little red book, in which his mentor had hidden the vital information – as the codes for three amino acids.

Ser – Tyr – Pro

Hm.

Beauty and the chemist

You are beautiful, just like your mother – but OBVIOUSLY not as clever as your dad.

But, what sparked me to wrote something about this film, was some dialogue which brought home to me just how long ago 1969 was (I was still in short trousers – well, to be honest, for about half the year I am still in short trousers, but then it was all year round). Hathaway is flown to China from Hong Kong, and on arrival is met by the daughter of his old mentor:

Soong Chu (Francesca Tu): I am Professor Soong's daughter

Dr. John Hathaway (Peck): You look a great deal like your beautiful mother.

Soong Chu: Not I. I am just an ugly chemist

Hathaway: I read your recent paper on peptides. I thought it was brilliant – for a woman.

Soong Chu: Oh, I agree, but my father helped a great deal.

Working in the dark to avoid any more comments on her looks?

I was taken aback by the reference to just being an ugly chemist, and had to go back and check that I'd heard that correctly. Was the implication that one could not be beautiful, and a chemist? Nothing more was said on the topic, but that seemed to be the implication. And what is meant by being 'just' a chemist?

Hathaway's comment that Soong Chu's paper had been brilliant, was followed by a pause. Then came "…for a woman". Did he really say that?

Not bad for a girl

I was waiting for the follow-up comment which would resolve this moment of tension. This surely had to be some kind of set up for a punch line: "It would have been beyond brilliant for a man", perhaps.

But no, Soong Chu just agreed. There did not seem to be intended to be any tension or controversy or social critique or irony or satire there. So much for Soong Chu's membership of the Red Guard and all the waving of the thoughts of the Chairman (she would have known that "Women represent a great productive force in China, and equality among the sexes is one of the goals of communism").

"The red armband is the most treasured prize in China…[representing] responsibility…[as] a leader of our revolution"
Soong Chu had needed the help of her father to prepare her paper, but he had presumably declined to be a co-author, not because his input did not amount to a substantial intellectual contribution (the ethics of authorship have also come on a bit since then), but because his daughter was a woman and so not able to stand on her own two feet as a scientist.

This dialogue is not followed up later in the film.

So, this is not planting a seed for something that will later turn out to be of significance for character development or plot, or that will be challenged by subsequent scenes. It is not later revealed that Soong Chu has a parallel career as Miss People's Republic of China (just as Hathaway is a chemist and also a kind of James Bond figure). Nor does it transpire that Professor Soong had been senile for many years and all of his work was actually being undertaken for him by his even more brilliant daughter.

Sadly, no, it just seems to be the kind of polite conversation that the screenwriters assumed would be entirely acceptable to an audience that was presumably well aware that females cannot be both beautiful and scientists; and that women need help from men if they are to be successful in science.

Times have changed … I hope.

 

 

* Interestingly, I've now found a poster for the film which seems to suggest that the whole purpose of the operation was not to acquire the enzyme structure at all, but to get Hathaway close enough to Mao to assassinate him.

Getting viewers to watch the film under false pretences

This seems to describe a very different cut to one I watched – where the audience with Mao seems to have surprised everyone, and the senior intelligence officers contacted their governments to alert them of this unexpected opportunity!