Comparisons – Page 16 – Science-Education-Research

dummy header2

An example of an analogy drawing upon a chemical concept to explain another idea (from research methods):

"In practice, many studies rely on testing for a statistically significant difference, although this is a very dubious criterion for equivalence…. This becomes clear if we consider how inferential tests are interpreted when comparing the final outcome measures in a study. At the end of a true experiment, statistical tests may be used to infer that a difference in final outcomes was unlikely enough that we can confidently assume it is not due to random variations but is due to a systematic difference (i.e., the difference between the experimental treatment and the control condition) and so can be assumed to (probably) apply more generally in the population and not just to this specific permutation of learners. So, a statistically significant difference means a very unlikely one (in practice, normally one with a probability value, p<0.05).

Often a similar approach is used in studies to evaluate the differences found at pre-test. The results are analysed to see if there is a very unlikely difference between the scores in the different conditions. If a statistically significant difference is found, then this clearly suggests the groups cannot be considered equivalent. That is reasonable.

Unfortunately the reverse does not apply: if the differences do not reach significance, we cannot assume that implies equivalence. Say p=0.08 (which means that the differences were unlikely enough to only occur by chance about once in 12 times, rather than once in twenty times as when p=0.05), this still shows there was a difference that was unlikely to be down to random factors. There is a logical difference between what we are seeking to do in these two situations. In one case (comparing post-test results), we are trying to exclude all but those outcomes that are most unlikely to be chance events, and in the other (comparing pre-test scores), we are trying to show that any difference is small enough to be insignificant in affecting later outcomes. So, in the first case, we are trying to show something is very improbable, but in the other case, we are trying to show we have a very probable outcome. So, using the same kind of inferential test as a test of equivalence means (sensibly) excluding cases with very different pre-test outcomes across treatments from being labelled equivalent: but still (dubiously) admitting other substantially different pre-test outcomes across treatments as being equivalent.

If this seems a little abstract, consider this analogy. Consider Table 2.2 which presents two questions that might be posed to a learner and her hypothetical responses. The two questions are looking at two different ends of a spread (that is a continuum from combinations of elements with very different electronegativities to combinations of elements with the same electronegatiity) and a suitable criterion that works for one extreme cannot be simply reversed to be used at the other extreme (which would be like saying anyone who is not over 2 m tall should be considered short). That is, if we agree that an electronegativity difference of >2.5 is a good criterion to identify highly ionic compounds, then it is inappropriate to use the same cut-off as the basis for a criterion (<2.5) to identify highly covalent (i.e., non- polar) compounds. Perhaps we should instead look for an electronegative difference <1.0 or <0.5? The precise choices are open to opinion (you might actually suggest >3.0 for the most ionic compounds): but the invalidity of using the same cut-off to identify both sets of extreme cases is not. If we decide the most covalent compounds are those where the electronegative difference is <0.7, we should not then class any where the difference is >0.7 as ionic.

dummy header

An example of an analogy drawing upon a chemical concept to explain another idea (from research methods):

"In practice, many studies rely on testing for a statistically significant difference, although this is a very dubious criterion for equivalence…. This becomes clear if we consider how inferential tests are interpreted when comparing the final outcome measures in a study. At the end of a true experiment, statistical tests may be used to infer that a difference in final outcomes was unlikely enough that we can confidently assume it is not due to random variations but is due to a systematic difference (i.e., the difference between the experimental treatment and the control condition) and so can be assumed to (probably) apply more generally in the population and not just to this specific permutation of learners. So, a statistically significant difference means a very unlikely one (in practice, normally one with a probability value, p<0.05).

Often a similar approach is used in studies to evaluate the differences found at pre-test. The results are analysed to see if there is a very unlikely difference between the scores in the different conditions. If a statistically significant difference is found, then this clearly suggests the groups cannot be considered equivalent. That is reasonable.

Unfortunately the reverse does not apply: if the differences do not reach significance, we cannot assume that implies equivalence. Say p=0.08 (which means that the differences were unlikely enough to only occur by chance about once in 12 times, rather than once in twenty times as when p=0.05), this still shows there was a difference that was unlikely to be down to random factors. There is a logical difference between what we are seeking to do in these two situations. In one case (comparing post-test results), we are trying to exclude all but those outcomes that are most unlikely to be chance events, and in the other (comparing pre-test scores), we are trying to show that any difference is small enough to be insignificant in affecting later outcomes. So, in the first case, we are trying to show something is very improbable, but in the other case, we are trying to show we have a very probable outcome. So, using the same kind of inferential test as a test of equivalence means (sensibly) excluding cases with very different pre-test outcomes across treatments from being labelled equivalent: but still (dubiously) admitting other substantially different pre-test outcomes across treatments as being equivalent.

If this seems a little abstract, consider this analogy. Consider Table 2.2 which presents two questions that might be posed to a learner and her hypothetical responses. The two questions are looking at two different ends of a spread (that is a continuum from combinations of elements with very different electronegativities to combinations of elements with the same electronegatiity) and a suitable criterion that works for one extreme cannot be simply reversed to be used at the other extreme (which would be like saying anyone who is not over 2 m tall should be considered short). That is, if we agree that an electronegativity difference of >2.5 is a good criterion to identify highly ionic compounds, then it is inappropriate to use the same cut-off as the basis for a criterion (<2.5) to identify highly covalent (i.e., non- polar) compounds. Perhaps we should instead look for an electronegative difference <1.0 or <0.5? The precise choices are open to opinion (you might actually suggest >3.0 for the most ionic compounds): but the invalidity of using the same cut-off to identify both sets of extreme cases is not. If we decide the most covalent compounds are those where the electronegative difference is <0.7, we should not then class any where the difference is >0.7 as ionic.

ignoring modest initial differences in experimental studies is like considering polar compounds as highly covalent

An example of an analogy drawing upon a chemical concept to explain another idea (from research methods, 'testing for initial equivalence'):

Table from book — From 'Chemical Pedagogy: : Instructional Approaches and Teaching Techniques in Chemistry'

"…many studies rely on testing for a statistically significant difference, although this is a very dubious criterion for equivalence….

The results are analysed to see if there is a very unlikely difference between the scores in the different conditions. If a statistically significant difference is found, then this clearly suggests the groups cannot be considered equivalent. That is reasonable.

Unfortunately the reverse does not apply: if the differences do not reach significance, we cannot assume that implies equivalence. Say p=0.08 (which means that the differences were unlikely enough to only occur by chance about once in 12 times, rather than once in twenty times as when p=0.05), this still shows there was a difference that was unlikely to be down to random factors. There is a logical difference between what we are seeking to do in these two situations. In one case (comparing post-test results), we are trying to exclude all but those outcomes that are most unlikely to be chance events, and in the other (comparing pre-test scores), we are trying to show that any difference is small enough to be insignificant in affecting later outcomes. So, in the first case, we are trying to show something is very improbable, but in the other case, we are trying to show we have a very probable outcome. So, using the same kind of inferential test as a test of equivalence means (sensibly) excluding cases with very different pre-test outcomes across treatments from being labelled equivalent: but still (dubiously) admitting other substantially different pre-test outcomes across treatments as being equivalent.

Read about analogy in science

Read examples of scientific analogies

Many examples of science analogies are listed in 'Creative comparisons: Making science familiar through language. An illustrative catalogue of figurative comparisons and analogies for science concepts'. Free Download.

material chunked in memory is like a complex ion

An example of an analogy drawing on a scientific concept:

"Despite all normal adult human beings having a similar severe limit on working memory, an expert in a field is able to deal with much more complex and extensive information than a novice. This effect is explained by something called 'chunking' which refers to how material represented in memory can be 'chunked' into more complex structures that can be accessed from memory as a single unit. We might think, by way of analogy, of how a complex ion might comprise a central cation linked to a number (perhaps 4 or 6) of ligands. Not only can we treat a species such as [Cu(H₂O)₆]²⁺ as a unitary chemical object for purposes of chemical discourse, but also the expert can mentipulate their concept of this species as a single mental object."

Taber (2024) Chemical Pedagogy. Instructional approaches and teaching techniques in chemistry . Royal Society of Chemistry. [Download Chapter 1]

Mea culpa: the aim here was to explain an idea to readers who were likely to know about chemistry (as they were reading a book about chemistry teaching) by using a chemical context to give an analogy for a idea from the learning sciences.

Read about analogy in science

Read examples of scientific analogies

school students behave like the phases of matter

An example of a teaching analogy:

"Students in a classroom are analogous to the particles of a solid, since they have a regular arrangement and a limited freedom of movement. Students shifting or turning in their seats represent the vibrational motion of solid particles.

During breaks between classes, students have a wider range of motion. They now also have limited translational motion which allows them to move among one another to the doorway and through the halls, but are still confined to the volume of the school. This is analogous to the behaviour of liquid particles.

At the end of the day, students are like gas particles since they have unrestricted and primarily translational motion which causes them to escape from their school building and diffuse throughout the community."

Previously posted at scienceanalogies.com by retired science teacher Murray Hart – original source: Licata, Kenneth P. Chemistry Is Like A … The Science Teacher 1988, 55(8), 42.

Read about analogy in science

Read examples of scientific analogies

mean free path is like riding a bumper car

An example of a teaching analogy:

"The mean free path is the average distance which a molecule travels between collisions with neighbouring molecules. This is like riding in the bumper cars at a carnival: you only can travel a short distance, on the average, before being involved in a collision with another bumper car."

Source: Murray Hart, retired science teacher, previously posted at scienceanalogies.com.

Read about analogy in science

Read examples of scientific analogies

Graham's law of diffusion is like running a race

An example of a teaching analogy:

"According to Graham's Law, the velocity or rate of diffusion of a gas is inversely related to the square root of the molecular mass. Thus, molecules of small mass travel more rapidly and molecules of larger mass travel more slowly. This idea can be easily remembered by considering the following analogy:

Consider 10 runners who were so closely matched that they had essentially equal times for running a 1000 metre race. Now suppose half of the runners were required to carry an extra 10 kg mass that was attached to them by a belt or backpack. Likely the 5 who had to carry the extra mass would now lag behind the rest of the runners."

Previously posted at scienceanalogies.com by retired science teacher Murray Hart – original source: Based on Toon, E.R. and Ellis, G.L. Foundations of Chemistry New York: Holt, Rinehart and Winston, 1973 p. 136

Read about analogy in science

Read examples of scientific analogies

ignoring non-significant initial differences is like considering polar compounds as highly covalent

An example of an analogy drawing upon a chemical concept to explain another idea (here an idea from research methods):

"In practice, many studies rely on testing for a statistically significant difference, although this is a very dubious criterion for equivalence…. This becomes clear if we consider how inferential tests are interpreted when comparing the final outcome measures in a study. At the end of a true experiment, statistical tests may be used to infer that a difference in final outcomes was unlikely enough that we can confidently assume it is not due to random variations but is due to a systematic difference (i.e., the difference between the experimental treatment and the control condition) and so can be assumed to (probably) apply more generally in the population and not just to this specific permutation of learners. So, a statistically significant difference means a very unlikely one (in practice, normally one with a probability value, p<0.05).

Often a similar approach is used in studies to evaluate the differences found at pre-test. The results are analysed to see if there is a very unlikely difference between the scores in the different conditions. If a statistically significant difference is found, then this clearly suggests the groups cannot be considered equivalent. That is reasonable.

Unfortunately the reverse does not apply: if the differences do not reach significance, we cannot assume that implies equivalence. Say p=0.08 (which means that the differences were unlikely enough to only occur by chance about once in 12 times, rather than once in twenty times as when p=0.05), this still shows there was a difference that was unlikely to be down to random factors. There is a logical difference between what we are seeking to do in these two situations. In one case (comparing post-test results), we are trying to exclude all but those outcomes that are most unlikely to be chance events, and in the other (comparing pre-test scores), we are trying to show that any difference is small enough to be insignificant in affecting later outcomes. So, in the first case, we are trying to show something is very improbable, but in the other case, we are trying to show we have a very probable outcome. So, using the same kind of inferential test as a test of equivalence means (sensibly) excluding cases with very different pre-test outcomes across treatments from being labelled equivalent: but still (dubiously) admitting other substantially different pre-test outcomes across treatments as being equivalent.

If this seems a little abstract, consider this analogy. Consider Table 2.2 which presents two questions that might be posed to a learner and her hypothetical responses. The two questions are looking at two different ends of a spread (that is a continuum from combinations of elements with very different electronegativities to combinations of elements with the same electronegatiity) and a suitable criterion that works for one extreme cannot be simply reversed to be used at the other extreme (which would be like saying anyone who is not over 2 m tall should be considered short). That is, if we agree that an electronegativity difference of >2.5 is a good criterion to identify highly ionic compounds, then it is inappropriate to use the same cut-off as the basis for a criterion (<2.5) to identify highly covalent (i.e., non- polar) compounds. Perhaps we should instead look for an electronegative difference <1.0 or <0.5? The precise choices are open to opinion (you might actually suggest >3.0 for the most ionic compounds): but the invalidity of using the same cut-off to identify both sets of extreme cases is not. If we decide the most covalent compounds are those where the electronegative difference is <0.7, we should not then class any where the difference is >0.7 as ionic.

By the same logic, whilst it makes sense to exclude pre-test differences which reach statistical significance from being considered equivalent (like excluding KF from our list of highly covalent compounds) that is not sufficient to judge equivalence, and something more is needed. One rule of thumb that has been suggested is that rather than using p≥.05 as the critical value here, it should be p≥0.5 (i.e., only admitting as equivalent groups where the pre-test differences are more likely to occur by chance than not), but there are more sophisticated approaches…"

Mea culpa: the aim here was to explain an idea to readers who were likely to know about chemistry (as they were reading a book about chemistry teaching) by using a chemical context to give an analogy for how a research technique is often misused.

Treating differences that do not reach statistical significance as equivalent does violence to what most people understand 'equivalent' to mean.

Necessary but not sufficient: Testing for equivalence is a widely used technique in experimental research in education and the social sciences.Using the conventional cut-off of p<0.05 is such studies is the like equating…

Read about testing for initial equivalence in research studies

entropy is like shaking a box of ordered marbles

An example of a teaching analogy:

"Entropy is an important factor in discussing equilibrium. Either the forward or reverse reaction will be favoured by the entropy factor … that is, allowing atoms and molecules to move from a state of lesser to greater disorder.

Imagine taking a small box and into it packing marbles so that all the marbles in each horizontal layer are the same color, or show some other definite pattern. When the box is shaken for a while then the contents examined, you will see that the marbles are now quite randomly* arranged … they have spontaneously moved to a state of greater disorder (higher entropy)."

Previously posted at scienceanalogies.com by retired science teacher Murray Hart – original source: Smoot, R.C., Price, J. and Smith, R.G. Chemistry A Modern Course Don Mills: Maxwell Macmillan Canada, 1987, p.386

Read about analogy in science

Read examples of scientific analogies

* Arguably, the marbles only seem to be randomly arranged as in principle they have moved according to known laws due to the actions of specific forces – perhaps 'arbitarily arranged' might be better?

chemical equilibrium is like substituting players in a sports game

An example of a teaching analogy:

"Equilibrium is a situation attained when the rates of forward and reverse reaction are equal, so that there is no change in the concentrations of the reactants and products. Equilibrium can be reached at various points; the concentrations of reactants and products need not be equal, only their rate of exchange. The equilibrium point also shifts in response to stresses placed upon the system.

In a sports game like soccer or basketball, for every new player substituted onto the field, an old player must leave – thus the rate of these opposing reactions are equal. There is no change in the number of players on the field, even though their identities are different. There is no requirement that the number of players on the field and on the bench be equal (and usually they are not equal); the only thing that must be equal is the rate of exchange between these two groups.

In a game like hockey, a penalty would be like a stress … it increases the number of players leaving the ice, compared to the number of players going onto the ice.

At this new equilibrium point the rate of player exchange is again equal for the duration of the penalty. When the penalty is over, the equilibrium point shifts back to the original position."

Previously posted at scienceanalogies.com by retired science teacher Murray Hart – original source: Licata, Kenneth P. Chemistry Is Like A … The Science Teacher 1988, 55(8), 43.

Read about analogy in science

Read examples of scientific analogies

hydronium ion concentration and hydroxide ion concentration can be imagined on opposite ends of a seesaw

"Imagine a seesaw or teeter totter with hydronium ion concentration at one end and hydroxide ion concentration located at the other end. As the seesaw operates, it will show the required inverse relationship between these factors…. as the hydronium ion concentration goes up, the hydroxide ion concentration goes down, and vice versa.

Similarly, a seesaw with pH at one end and hydronium ion concentration at the other, shows the relationship these factors have…. as hydronium ion concentration goes up, the pH goes down, and vice versa."

Previously posted at scienceanalogies.com by retired science teacher Murray Hart – original source: Fortman, John J. Pictorial Analogies X: Solutions of Electrolytes Journal of Chemical Education January 1994, 71(1), 27

[This analogy needs to be interpreted qualitatively.]

Read about analogy in science

Read examples of scientific analogies

electrolytes are like nuts and bolts

An example of an teaching analogy:

"If a bolt represents a positive ion, a nut analogous to a negative ion and assembled units represent undissociated molecules of the electrolyte, these units can be used to represent various types of electrolytes:

A strong electrolyte would be represented by an equal no. of separate nuts and bolts in a box (complete dissociation).

A weak electrolyte would be represented by assemblying most nuts and bolts into pairs, and having only a few disconnected (small per cent ionization).

A non-electrolyte would be represented by assemblying all the nuts and bolts into pairs so that none were disconnected (no ionization).

A more concentrated solution of a weak electrolyte would be represented by showing a greater number of assembled nuts and bolts, but a smaller per cent of them dissociated into separate pieces."

Read about analogy in science

Read examples of scientific analogies