A topic in research methodology
Research in one of the major traditions, or paradigms, of research is often referred to as confirmatory.
Confirmatory research is often contrasted with discovery research or exploratory research.
Biddle and Anderson (1986: 231) characterise confirmatory research as studies that "presumes to establish objective information about social behaviour that can be generalized".
"Confirmatory research sets out to test a specific hypothesis to the exclusion of other considerations; whereas discovery research seeks to find out what might be important in understanding a research context, presenting findings as conjectural (e.g., 'suggestive', 'indicative') rather than definite"
Taber, 2013: 45
The choice of confirmatory research has consequences for research design.
Study design in confirmatory research
The degree to which a research design needs to be followed precisely is quite different in different forms of research. In other forms of discovery research, designs may be flexible and even emergent.
"However, anyone planning to use an 'experimental' design to test a hypothesis must plan the research in detail at the start, and so the techniques of data collection and analysis need to be firmly established before any data are collected. Research claiming to use this type of methodology, which involved substantive changes (in such matters as how 'subjects' are assigned to groups, or which statistical tests were to be applied) once the research was underway, is open to being challenged as failing to follow accepted procedures and so is potentially invalid."
Taber, 2013: 79
"…give stress to careful research design, to reliable measurement of variables, to statistical manipulation of data, and to the detailed examination of evidence. Hypotheses are stated to indicate knowledge claims, and these are judged to be confirmed if they are supported by inferential statistics that reach arbitrary levels of significance. Confirmed hypotheses ('findings') are presumed to generalise to populations or contexts similar to the one studied."
Biddle & Anderson, 1986: 231.
Statistical tests
Statistical tests deal with probabilities. Statistical tests are often used in experimental research in the social sciences as there are usually many variables at work, most of which cannot be controlled, even if they can be measured,
"Statistical methods are, in a way, makeshifts. They are devices for handling instances in which numerous conditions are simultaneously operating. They do not show how all these conditions are related, and by themselves they throw no light upon the nature of the conditions."
Bartlett, 1932/1995
Statistical tests can be vey useful, but they only show us how likely something is or how well something fits a pattern – we still need to interpret the results.
Statistical tests will output results when data is entered – even if the test is inappropriate for that data set.
Care is therefore needed in choosing, applying and interprting any statistical tests used in research.
Sources cited:
- Bartlett, F. C. (1932/1995) Remembering. A study in experimental and social psychology. Cambridge; Cambridge University Press.
- Biddle, B. J., & Anderson, D. S. (1986). Theory, methods, knowledge and research on teaching. In M. C. Wittrock (Ed.), Handbook of Research on Teaching (3rd ed.), pp. 230-252. New York: Macmillan.
- Taber, K. S. (2013). Classroom-based Research and Evidence-based Practice: An introduction (2nd ed.). London: Sage.
My introduction to educational research:
Taber, K. S. (2013). Classroom-based Research and Evidence-based Practice: An introduction (2nd ed.). London: Sage.
