CRESTe: Critical Reading of Empirical Studies Tool evaluation
Important notice: this invitation has now closed.
(The page is kept live as a record of the process of inviting participants to contribute.)
You can:
- read about critical reading of research here
- read about the Critical Reading of Empirical Studies Tool, and download the tool for your own use, here
- read about the Critical Reading of Empirical Studies Tool evaluation outcomes here
Volunteers sought for a project on critically reading research studies
Prof. Keith Taber is seeking some volunteers for a small scale project (CRESTe: Critical Reading of Empirical Studies Tool evaluation) evaluating a simple tool intended for use when reading empirical research.
Who can volunteer?
Any student at university level* who will be reading empirical studies for their course during the term. The nature of the tool is such that it is relevant to empirical research – studies that involve collecting and analysing data. The tool is intended for use when reading original research papers (rather than accounts in secondary literature such as textbooks, reviews, etc.).
* The tool is designed primarily for higher degrees students – however if you are a senior secondary school student, or undertaking a professional programme, and following a course that requires you to read primary literature (such as research studies reported in journals), and would like to involved, please let me know.
What is involved?
The tool is a one page sheet highlighting some different aspects of research reports that a reader might consider whilst reading and evaluating a research paper. The tool asks the user to give an impressionist evaluation of how convincing they found aspects of the paper being read. (This simply involves selecting a point on a scale for each feature relevant to the paper being read.)
Students who volunteer to participate will be sent two documents by email: A copy of the tool; and a document explaining the rationale behind it, and how to use it, along with instructions for returning feedback (by email) after you have engaged with the tool when reading research papers.
Volunteers are asked to use and complete a least one copy of the tool, and to return the completed sheets (as email attachments), with their responses to the very brief feedback questionnaire. Volunteers are not being asked to take on any additional reading, but simply to use the tool when they are reading research reports as a normal part of their course work or research.
Can I withdraw from the study after volunteering?
If you volunteer, and then change your mind for any reason, you can withdraw from the study, and do not need to explain your decision. You may withdraw either by notifying the investigator or simply by not making a return. No record of those volunteering, completing or withdrawing, will be shared with anyone else.
How will the data be used?
It is intended to share outcomes in a way, accessible to all who may be interested. The results will be published on this website.
It is also possible there will be further dissemination (e.g., in one or more articles or book chapters or blogs if the project raises points of wider interest. In this case, electronic copies will be made available to any participant who would like to be kept informed.)
No participants will be identified in any report: all outcomes will be reported as collated results and/or anonymous responses (only distinguishing level of course, i.e., undergraduate, PGCE, masters or doctoral).
What form will the outcomes take?
• Information on whether the tool was found useful (in which case it will be made generally available – possibly after modification should feedback indicate changes are needed).
• Information on the general level of ratings by participants on different aspects of studies read. (In this study participants will not be asked to specify the papers what have read, so this will just give a general impression of how convincing readers found published research.)
Why should I volunteer?
Many students will at some point be asking others to participate in enquiry to support their course requirements – such research is only possible if others offer us the 'gift' of data. Someone who asks others to contribute to their research should consider volunteering to participate in some** studies themselves. (** That does not necessarily imply this one, of course.)
The extra time and effort required is minimal – you are only asked to use the tool when you are reading a research report for your studies, and it has been designed to be very simple to complete.
The tool is intended to help a reader focus on some of the different aspects of the design and execution of a study – as a kind of aide-mémoire – so may actually support your critical evaluation of the research that you read. (Whether it is indeed useful in this regard, is what the evaluation is intended to find out.)
Is this research ethical?
The project has been considered against the guidelines of the British Educational Research Association (see Figure), and has been subject to the University of Cambridge Faculty of Education ethical clearance procedures.
If you have any concerns about this research, you may contact the investigator, Prof. Taber (at kst24@cam.ac.uk), or, if you prefer, or if you wish to make a complaint, contact the Faculty of Education Research Office (research-office@educ.cam.ac.uk) at the University of Cambridge.
Participation in the study is not based on any payment or other inducement, and offers no course credit. Non-participation or withdrawal from the study will not lead to any negative consequences beyond not benefiting from any inherent value that using the tool may bring.
How will the evaluation be reported?
Assuming sufficient data is collected for a viable evaluation, a short report will be made publicly available, via this website.
If the evaluation suggests that the tool is found valuable by a fair proportion of students, then the tool will also be made freely available at the same place.
If the evaluation leads to any findings which are thought to deserve wider circulation (to researchers, teachers or learners) it is possible there could be other appropriate forms of dissemination: e.g., material could be referred to in articles, chapters, talks or similar.
How do I volunteer?
To participate in the study, please email the CRESTe project (at creste@science-education-research.com) with the subject line 'volunteer'.
Please include (copy and paste) the following text in your email.
I would like to volunteer to take part in the evaluation of the Critical Reading of Empirical Studies Tool. I understand I will be sent a copy of the tool, and instructions for its use. I have been informed of the purpose of the evaluation, and how the information collected will be used. I understand that no information identifying individual participants will be included in any report, or shared in any other way. I understand that, having indicated I would like to take part, I can later decide not to try out the tool, or not to provide a return, for any reason I may have – without needing to offer any explanation, and with no adverse consequences. I have been told that the evaluation has been subject to the Faculty of Education ethical clearance procedures and that if I have any concerns I can contact the University of Cambridge Faculty of Education Research Office about those concerns. I give consent for any data I provide to be analysed for the evaluation and used anonymously in any subsequent report of the evaluation.
[Your name]
Thank you for reading this invitation.
