Natural experiments

A topic in research methodology

Experiment is basically an interventionist methodology. However, in education, it is often not practical to control all the variables necessary for a true experiment. Sometimes the conditions of an existing situation may provide the opportunity for a 'natural' experiment:.

One example:

"Using an arbitrary cutoff date, school districts regulate which children will begin school. This 'natural experiment' was used to examine effects of age- and schooling-related influences on memory …and 3 levels of phonological segmentation in children who just made vs. missed the cutoff."

Morrison, Smith & Dow-Ehrensberger, 1995

A unusually large educational experiment:

"Planned as a large-scale teaching program for disadvantaged children during the Johnson administration of 1967, it was transformed into a series of "planned variations" of education. When deep cuts in the educational budget made it impossible to implement the teaching program broadly, the Office of Education turned to the concept of comparing about seventeen models of schooling for disadvantaged children…To evaluate Follow Through on this count, the study included close comparisons between the outcomes resulting from a Follow Through model and the outcomes from non-Follow Through education programs in the same schools and communities. To insure that the models would be judged fairly, the outcomes of Follow Through and non-Follow Through classes in a given community were equated in terms of entry levels with analysis of covariance procedures." p.240

"A battery of tests was assembled to encompass basic skills, cognitive/conceptual development, and affective factors." p241

"We should be alerted to the fact that no program was successful everywhere it was tried. The results are for average effects across several locations. All of the programs were successful in at least one location on at least one class of outcome, indicating that local effects are extremely important." p.242

Guthrie, 1977

Read about testing for equivalence between groups

Sources cited:
  • Guthrie, J. T. (1977). Research Views: Follow through: A Compensatory Education Experiment. The Reading Teacher, 31(2), 240-244.
  • Morrison, F. J., Smith, L., & Dow-Ehrensberger, M. (1995). Education and cognitive development: A natural experiment. Developmental Psychology, 31(5), 789-799. doi: 10.1037/0012-1649.31.5.789

My introduction to educational research:

Taber, K. S. (2013). Classroom-based Research and Evidence-based Practice: An introduction (2nd ed.). London: Sage.