Health Subject Guide — Systematic Reviews

Before proceeding to Step 1:  

Has a review on your topic been published?  See PubMed > Limit by Article Type for instructions on searching PubMed for systematic reviews and meta-analysis.

Is a review on your topic registered? Check PROSPERO: International Prospective Register of Systematic Reviews for pending reviews. Submit a protocol to PROSPERO for your own review prior to commencing work!

Step 1: Define a specific research question

Systematic reviews begin with a question that clearly states the problem(s) to be addressed by the review.  Many clinical reviews adhere to the PICO format when defining the scope of the research question. PICO requires that reviewers frame a question by defining: Population, Intervention, Comparison, and Outcome. The use of PICO is not required, but all reviewers must clearly define what they will and will not include in their review prior to commencement. For clinical questions about risk factors and causes consider using the PEO question framework (Patient/Population, Exposure, Outcome).

SPIDER is designed to search for qualitative and mixed-methods studies 

Step 2: Locate all relevant evidence

Extensive searching is required to locate all relevant studies. Multiple research databases and grey literature are searched using all possible keywords (and controlled vocabulary is used when available). Several supplemental search methods ensure comprehensiveness: manual perusal (hand searching) is done for journals most likely to publish articles on the topic, bibliographies of relevant articles are scanned, and it is permissible to contact researchers to inquire about in press or pending publications.  After the evidence is compiled, any reason for excluding a source must be recorded. Rationale for exclusion will pertain to broad categories (ex. presence of a co-morbidity), rather than item-specific evaluation. The PRISMA statement is a useful tool for reporting systematic reviews. The diagram presented by PRISMA, or similarly constructed diagrams, are very commonly used in published systematic reviews to represent the identification and inclusion/exclusion of studies for review.

Step 3: Assess the quality of studies

Criteria are employed to evaluate the quality of studies without bias. A variety of grading scales have been developed for use in appraisal (ex. The CEBM: Centre for Evidence Based Medicine.) Care must be given in determining how studies will be graded for a particular question. Studies of the highest quality for one question (ex. what is the best treatment?) will be weighted differently for another type of questions (ex. what is the best diagnostic tool?). 

Step 4: Summarize the evidence

Data tables are created to summarize the important characteristics of studies and the grade assigned to them in Step 3. At this point, the larger topic may be divided into subgroups.

Step 5: Interpret the findings

After sorting through dozens, hundreds, or even thousands of citations, a reviewer must now determine and briefly state what the evidence concludes about the original research question and whether the strength of the evidence is strong enough to make a clinical recommendation. It is not uncommon for systematic reviews to conclude that there is evidence, but it is not strong enough to make a recommendation for clinical practice. When it comes to the interpretation, it's all about the evidence!

All databases that could contain articles on the topic should be searched. See key databases. There is a high amount of redundancy/duplication between many databases; however, each database has unique content. Citation management tools, such as RefWorks are useful for deleting duplicate references. In addition to published research studies, a comprehensive search includes grey literature.

RefWorks is a citation management tool.  It is ideal for systematic reviews because it:

• can delete duplicate citations found across databases
• organize and save citations
• lets you attach notes to citations
• automatically creates bibliographies using saved citations