Critical Appraisals

Library Skills

Critical Appraisals

Critical appraisal can be defined as "the process of carefully and systematically examining research to judge its trustworthiness, and its value and relevance in a particular context" (Burls 2009 p. 1).


Why Critically Appraise Literature?

Critical appraisal skills are often discussed in the context of evidence-based practice in the fields of nursing, midwifery, health sciences, pharmacy and medicine. This involves finding and critically appraising relevant research for its validity, relevance and reliability before it is combined with the expertise of the health practitioner (and the patients preferences) in treating patients.

This systematic approach can be applied in assessing the quality of research studies in other fields.

It is often assumed that if an article is published in a journal it must be of quality; this is not always the case.

It has been argued that most, "if not in fact 99%, of published articles belong in the bin, and should certainly not be used to inform practice" (Greenhalgh 2014 p. 28). They may use "inappropriate designs, unrepresentative samples, small samples, incorrect methods of analysis, and faulty interpretation" (Altman 1994 p. 283).

When reading journal articles and studies it is important to ask yourself if the research is methodologically sound, relevant, is of value to the subject area and patients, has minimized the effects of potential biases and is reported in a clear way.


How Do I Start?

Consider some fundamental questions when you begin your appraisal (Greenhalgh 2014):

What was the research question and why was the study needed?

  • The question / hypothesis should be clearly stated in the study.
  • Does the study add something to knowledge in the area?
  • Consider the PICO method (MacInnes and Lamont 2014):
    • PROBLEM: Is there a focused question?
    • INTERVENTION: Is there a clear intervention being examined?
    • COMPARISON: Is there a suitable control or alternative?
    • OUTCOMES: Have results and outcomes been properly gathered and assessed?

What was the research design?

  • Quantitative or qualitative research? Combination?
  • Primary or secondary research?
  • What's the methodology?
  • This is crucial to your in-depth appraisal (will determine the appropriate checklist to apply)
  • The design used should be made clear. Unsure? Check out University of Minnesota's Understanding Research Study Designs guide, for a reminder of some of the types of health research.

Was the design appropriate to the question?

  • Different questions lend themselves to different sorts of research design.

Did the study meet expected standards of ethics and governance?

  • Were ethical issues given proper consideration?

You may also want to use the structure of the article as a framework for your appraisal, which will logically lead you to ask the right questions (Ingham-Broomfield 2014):

  • Has the research question been clearly defined?
  • Does the abstract make clear the methods used, results and conclusions?
  • Is there a clear description of the problem being studied?
  • Has current and relevant literature been critically reviewed - rather than just described?
  • Is the choice of research method justified?
  • Are there issues of bias?
  • Was any sample population appropriately selected to adequately represent the population being targeted by any particular treatment or drug?
  • Were any issues of consent, confidentiality etc. properly addressed?
  • Are the results set out clearly?
  • Have they been analysed properly?
  • Was the choice of particular statistical methods justified?
  • Are any conclusions set out clearly?
  • Do the results support the conclusion?

Key Tools

If you are an undergraduate student you will probably find these resources sufficient to allow you to successfully complete your critical appraisal assignment.

Postgraduate research students and staff working at a higher level may also wish to consult our additional resources page for further materials.

There is considerable overlap in the checklists for the same types of research from different organisations. The CASP checklists are possibly the most widely used but you may find you prefer the style of one organisation's lists to another. If in doubt you may wish to consult your lecturer.

UNDERSTANDING HEALTH RESEARCH

CRITICAL APPRAISAL SKILLS PROGRAMME (CASP)

SCOTTISH INTERCOLLEGIATE GUIDELINES NETWORK (SIGN)

HOW TO READ A PAPER: MEDICAL JOURNAL

HOW TO READ A PAPER: THE BASICS OF EVIDENCE-BASED MEDICINE

CENTRE FOR EVIDENCE-BASED MEDICINE (CEBM): CRITICAL APPRAISAL TOOLS

BESTBETS: CRITICAL APPRAISAL WORKSHEETS

RISK OF BIAS TOOL

JOANNA BRIGGS INSTITUTE CRITICAL APPRAISAL TOOL

Additional Resouces

The following may facilitate the appraisal of more complicated studies and assist postgraduate research students and members of staff working in greater depth and/or seeking to critically appraise studies for possible inclusion or exclusion from systematic reviews.

INTERNATIONAL CENTRE FOR ALLIED HEALTH EVIDENCE: CRITICAL APPRAISAL TOOLS

DUKE UNIVERSITY MEDICAL CENTRE UNIVERSITY LIBRARY & ARCHIVES: EVIDENCE-BASED PRACTICE

OT SEEKER

PEDRO SCALE

AMSTAR CHECKLIST

PREFERRED REPORTING ITEMS FOR SYSTEMATIC REVIEWS AND META-ANALYSIS (PRISMA)

CONSORT

AGREE II