Critical Appraisal: Managing your Literature

An overview of critical appraisal, with emphasis on process and key critical appraisal tools

Reference Management Tools

Reference management software can be a lifesaver for organizing your reference list and putting citations into the correct format. Currently, AHS permits the following three options which can be requested through the IT Service Hub:

Different tools have different advantages – the right one for you will depend on the complexity of your project, how you plan to use the tool, and whether you need to collaborate and share. Check out the Citation Management FAQ for more information.

How do I Keep Track of All My Literature?

Keeping track of your literature searches and literature are key to the critical appraisal process.  Tracking allows you to easily update your literature searches, supports transparency and enables others to assess the trustworthiness of your findings.

  • You can use specific reference management use tools to organize your references and help with sorting as you critically appraise materials
  • Sometimes, a simple spreadsheet or word table is sufficient to keep track of your searches and literature you appraise 
  • You can use PRISMA guidelines and tools to ensure you are keeping track of important information that people will expect to see in any reports, for instance see the PRISMA-S checklist.

For more about how to document searches, see this example.

How do I Screen the Results of My Literature Searches?

There are THREE levels of screening to carry out. 

Level 1 - High level screening
Focus: screening for the trustworthiness and quality of your literature search resources. 
  • From a high level view, is the study relevant to your study question? 
  • Did the evidence come from a reputable source? 
  • Is the date of publication current? (i.e. within the past 5 years) 
  • Are conflicts of interest declared? 
  • Are funding sources identified? 

 

Level 2 - Mid-level screening

Focus: screening the literature for its applicability to your research question.  This screening is based on your INCLUSION and EXCLUSION criteria which determine the boundaries for your critical review.  It is important to record what you have chosen to exclude and why.  

The development of your criteria are guided by the purpose of your review and the nature of your research question.  Common criteria consider areas of primary interest and relevance such as: 

  • TYPE of publication
  • DATE of publication 
  • LANGUAGE
  • GEOGRAPHIC LOCATION of study 
  • STUDY POPULATION
  • STUDY SETTING
  • STUDY DESIGN 
  • STUDY PARTICIPANTS/AGES 
  • EXPOSURE OF INTEREST (i.e. condition, interventions) 
  • REPORTED OUTCOMES of interest 
  • PEER REVIEW 

 

Level 3 - Deep-level screening
Focus: Carrying out the actual critical review of the selected literature (See next section: Appraising the Literature).  There are critical appraisal tools to support a review process that is both methodical and standardized.  The one you choose needs to align with the characteristics of the literature you have selected for critical appraisal.