A topic in research methodology
Once data has been collected, it needs to be analysed. Analysis is the process by which data is interpreted in order to answer research questions. The 'unit of analysis' (analysis by teacher? by class? by school? etc.) should have been decided before data collection commenced.
Read about the unit of analysis
The nature of data analysis is different in interpretivist studies working with qualitative data, and confirmatory research working with quantitative data.
Two basic approaches to data analysis involve:
- adopting a framework for analysis prior to collecting data
- developing a framework for analysis in the light of inspection of the data collected
In practice hybrid approaches may be used which combine elements of both. However, the decision on whether an analytical framework should be constructed before collecting data is a highly principled one (related to whether the study is understood as confirmatory or exploratory), in that in some forms of research the wrong choice completely undermines the study.
For example, experiments and (quantitative) surveys ('forms of confirmatory research') are designed according to strict prior assumptions about the nature of what is being studied, and the categories useful for analysis, and so such research only produces valid outcomes when the analysis follows those same assumptions so the way data is analysed matches the way it was identified and categorised during data collection,
"The execution of an experiment is the realization [sic] of a plan conceived on the strength of certain assumptions, and what is yields is a ciphered message that cannot be decoded outside a body of knowledge."
Bunge, 1967/1988
Many kinds of research collect qualitative data of different kinds, and there are various approaches to qualitative data analysis depending upon research purposes and the nature of the data.
In more exploratory studies, analytical pluralism may be appropriate: that is, analysing the data from multiple perspectives – applying several analytical 'lenses' to the same data set.
The first stage in data analysis, is usually coding, but how that is undertaken, and the subsequent steps, tend to be quite different in these two forms of research.
Often researchers make a key distinction between the analysis of quantitative data ('quantitative data analysis') and the analysis of texts or artefacts ('qualitative data analysis'). However, qualitative data may be analysed in different modes, including to support a quantitative analysis.
Read about 'Approaches to qualitative data analysis'.
Researcher bias
All researchers are biased, as inevitably their background, training and wider education and experiences has predisposed a person to notice certain patterns rather than others. So, even when a researcher is not prejudiced (actively seeking a certain outcome, rather than being open to what the data suggest), she will have biases, which by their very nature are tacit – for example, we cannot be aware that we are not considering approaches or perspectives we have never met. In particular, humans tend to show 'confirmation bias' – finding it easier to spot evidence that supports prior assumptions.
- scholars have a kind of magnet in the mind which tends to draw out data that confirms assumptions (Herbert Butterfield)
Researchers can never be bias-free, but they can be aware of the potential for such biases to influence their work, and seek to employ techniques that ensure rigorous analytical processes (such as deliberately checking for negative instances of provision fidnings, and undertaking 'member checking' with informants.).
Analysis involves data reduction
Whatever form of analysis is undertaken, the process involves moving for a large set of many data, to an account reporting identified patterns. The analytical process involves selecting (and so self-selecting) data, and organising it. There is usually data reduction (for example putting aside some data deemed less pertinent to our research purposes; reporting averages and spread rather than specific data points). Such processes are irreversible in the sense that in being selective much information is lost.
An analogy my be useful in terms of audio files. If someone buys a music CD and rips it into their computer as mp3 files, then although the tracks are just as long when played, the detail in the mp3 files is less than in the original. The algorithms that convert to the smaller file size are designed to represent the music in as much detail as possible, but will inevitably lose some detail. If the mp3 files are used to burn a back-up CD in the original form, the files produced will be as large as those on the original, but will not be able to replace any lost details – in that sense they will be more like a low resolution digital photograph that can be 'blown-up' to any size we like, but will simply look very blurred when magnified.
"The raw datum may well contain any information, but the refined [ital] datum must convey only relevant and universally useful information; this is achieved both by pruning the raw datum and by refining it to a standard (conventional) condition, such as 15 ˚C and 76 cm of mercury….whenever statistical data such as averages or spreads around averages are sought – as in the case of quantitative measurement – the raw data are subject to reduction [ital]: the final information is actually a construction out of the raw data. Some information is always lost in the process of critical scrutiny, standardization [sic] and reduction of raw data…an irreversible process…we are interested in systematizing [sic, ital] data with the aim of disclosing patterns in them, and this cannot be done unless the 'noise' is eliminated by the pruning process…"
Bunge, 1967/1988
Is this a problem? Does it matter? Data reduction is essential to produce meaningful results anyone will read and make sense of, BUT we should consider whether to retain the full, original, data set in case we, or a later researcher wanting to check our work, wish to review the analysis, as this will allow different choices about which data to select and focus on. (This may not mean we have have done a bad job – but perhaps as the research field moves on, different choices become more sensible.)
This raises issues about ethics and data protection, We should only use data in the ways permitted under the permissions we originally sought (which may stop sharing with other researchers) and must only retain data in ways that are secure – but given this, the most ethical approach is always to retain data so it can re-analysed if indicated.
Collecting data in another language
Sometimes researchers are working in a context whether the local language is different to the language in which they expect to write-up their research. This introduces particular considerations about choice of language for data collection and analysis, and the use of translation.
Read about collecting and analysing data in another language
Work cited:
- Bunge, M. (1998). Philosophy of Science. Volume 2: From explanation to justification (Revised ed.). Routledge. (1967)
My introduction to educational research:
Taber, K. S. (2013). Classroom-based Research and Evidence-based Practice: An introduction (2nd ed.). London: Sage.
