Critical appraisal can be defined as "the process of carefully and systematically examining research to judge its trustworthiness, and its value and relevance in a particular context" (Burls 2009 p. 1).
Critical appraisal skills are often discussed in the context of evidence-based practice in the fields of nursing, midwifery, health sciences, pharmacy and medicine. This involves finding and critically appraising relevant research for its validity, relevance and reliability before it is combined with the expertise of the health practitioner (and the patients preferences) in treating patients.
This systematic approach can be applied in assessing the quality of research studies in other fields.
It is often assumed that if an article is published in a journal it must be of quality; this is not always the case.
It has been argued that most, "if not in fact 99%, of published articles belong in the bin, and should certainly not be used to inform practice" (Greenhalgh 2014 p. 28). They may use "inappropriate designs, unrepresentative samples, small samples, incorrect methods of analysis, and faulty interpretation" (Altman 1994 p. 283).
When reading journal articles and studies it is important to ask yourself if the research is methodologically sound, relevant, is of value to the subject area and patients, has minimized the effects of potential biases and is reported in a clear way.
Consider some fundamental questions when you begin your appraisal (Greenhalgh 2014):
What was the research question and why was the study needed?
|PROBLEM||Is there a focused question?|
|INTERVENTION||Is there a clear intervention being examined?|
|COMPARISON||Is there a suitable control or alternative?|
|OUTCOMES||Have results and outcomes been properly gathered and assessed?|
|What was the research design?||
|Was the design appropriate to the question?||
|Did the study meet expected standards of ethics and governance?||
You may also want to use the structure of the article as a framework for your appraisal, which will logically lead you to ask the right questions (Ingham-Broomfield 2014):
|Abstracts and Introduction||
If you are an undergraduate student you will probably find these resources sufficient to allow you to successfully complete your critical appraisal assignment. Postgraduate research students and staff working at a higher level may also wish to consult our additional resources page for further materials.
There is considerable overlap in the checklists for the same types of research from different organisations. The CASP checklists are possibly the most widely used but you may find you prefer the style of one organisation's lists to another. If in doubt you may wish to consult your lecturer.
This new tool takes you through a series of questions to assess the credibility of a piece of research. Designed to be accessible to all types of users, including members of the public seeking guidance on research on their own health circumstances, it is a practical way of grasping the basic concepts involved in critical appraisal.
The useful information section of the website also includes helpful explanations of concepts such as sampling methods, common sources of bias, correlation and causation.
CASP provides a set of 8 checklists for qualitative studies, randomised controlled trials (RCTs), cohort studies, case control studies, diagnostic studies, systematic reviews and economic evaluations.
Each checklist contains 10 questions to facilitate your critical appraisal.
Go to CASP
These notes and checklists cover systematic reviews and meta-analyses, randomised controlled trials, cohort studies, case-control studies, diagnostic studies and economic studies.
The British Medical Journal's How to read a paper collection of free journal articles (mostly by Trish Greenhalgh) provides excellent advice on questions to ask when reading different types of research.
Although the papers do not contain checklists themselves the book, How to read a paper: the basics of evidence-based practice (Greenhalgh 2014), includes summaries of the points in the papers, together with additional information in similarly titled chapters and additional chapters, as checklists.
Based in Oxford, the Centre' website provides critical appraisal worksheets covering randomised controlled trials, systematic reviews, diagnostics and prognosis.
Coverage includes checklists for randomised controlled trials and qualitative research.
Although not strictly a checklist, the 'risk of bias' tool provided by the Cochrane Collaboration in the Cochrane Handbook for Systematic Reviews of Interventions (2011) provides highly useful advice on checking for bias in research.
Go To 'Risk of Bias' Tool
The tools comprise checklists of questions for use in assessing 13 types of research study (including qualitative research and randomised controlled trials).
The following may facilitate the appraisal of more complicated studies and assist postgraduate research students and members of staff working in greater depth and/or seeking to critically appraise studies for possible inclusion or exclusion from systematic reviews.
An extensive lists of checklists and tools for all types of research design.
Some internally produced checklists, plus links to additional sources, including coverage of research studies focusing on therapy, diagnosis, prognosis and harm.
Critical appraisal tutorial from a team of occupational therapists from two major Australian universities.
Go To OT Seeker
The Physiotherapy Evidence Database (PEDro) contains details of over 33,000 randomised trials, systematic reviews and clinical practice guidelines in physiotherapy. The PEDro scale provides a structure for critically appraising trials only.
Go To PEDro Scale
AMSTAR stands for A MeaSurement Tool to Assess systematic Reviews. The checklist allows you to logically appraise the quality of a systematic review.
Go To AMSTAR
While the PRISMA statement gives guidance on writing up and reporting systematic reviews it can be useful to use the checklist to appraise a particular completed review against the standards set by PRISMA.
Go To CONSORT
The Consort statement provides a set of recommendations for reporting randomized trials. The checklist provided for authors of studies writing up trials also provides a handy list of points to use when judging the quality of trials.
Go To CONSORT
The Agree II instrument provides guidance on assessing the quality of guidelines.
Go To AGREE II