Evidence Syntheses & Systematic Reviews: Steps in a Systematic Review

1. Check for existing reviews/protocols and complete an initial search of the literature.

Has a review on your topic already been completed? Has a protocol been registered? A librarian can help with an initial search which will answer this question and give you an idea of the literature on your topic.

2. Formulate your question.

A clear and defined research question is an essential component of a systematic review. Formulating your research question is one of the most important steps in this process. This questions typically follow a framework. You can learn more about different types of frameworks here.

3. Assemble a team.

Systematic reviews are dependent on teamwork. Most standards recommend, and even require, multiple reviewers to screen and assess bias. Also, studies have shown that the inclusion of librarians increase the quality of the systematic review. You also might need a statistician, particularly if you plan on conducting a meta-analysis as well.

4. Develop a protocol. Think about registering it.

It is important to plan your research in advance of beginning your systematic search. A pre-established protocol should include research question, scope, extent of the review, screening process (including exclusion and inclusion criteria), data extraction, quality appraisal, and synthesis. Many protocols are published and establish rigor and transparency.

5. Search for studies

The goal of a systematic review is to identify all relevant studies on your research question. ​​Work with a librarian to design a comprehensive search strategy across multiple databases (at minimum three). Determine if you are going to use grey literature and then develop a methodical search for it. Choose a citation manager or other program to help with deduplication.

6. Screening and selection of studies

Use the inclusion and exclusion criteria defined in your protocol, to determine which studies will be included in the appraisal. This process will remove studies that do not meet your criteria. The screening is performed twice; first with just title/abstracts, then to the full-text. This process is methodical and well-documented; each study at both stages should be screened by at least two members of the review team to minimize bias. A third reviewer is used to resolve conflicts.

7. Quality appraisal.

At least two members will appraise the quality of the research in the included studies. This evaluation will include risk of bias using a standardized tool which can be adapted if need be.

8. Extract data.

You will also extract the data from the relevant studies. You should use a spreadsheet or systematic review software. It is recommended to pilot your extraction tool with a small subset of your studies to make sure if fields are missing or need to be modified.

9. Synthesize your results.

Your collected data must be combined into a coherent whole and accompanied by an analysis that conveys a deeper understanding of the body of evidence. All reviews will include a qualitative synthesis, and may or may not include a quantitative synthesis (also known as a meta-analysis).

10. Disseminate your report.

Share your research findings in a clear and comprehensive manner using the appropriate medium.

Clark, W. R., Clark, L. A., Raffo, D. M., & Williams, R. I. (2021). Extending Fisch and Block’s (2018) tips for a systematic review in management and business literature. Management Review Quarterly, 71(1), 215–231. https://doi.org/10.1007/s11301-020-00184-8

Committee on Standards for Systematic Reviews of Comparative Effectiveness Research, Institute of Medicine (U.S.). (2011). Finding what works in health care: Standards for systematic reviews. National Academies Press. https://nap.nationalacademies.org/catalog/13059/finding-what-works-in-health-care-standards-for-systematic-reviews

Fisch, C., & Block, J. (2018). Six tips for your (systematic) literature review in business and management research. Management Review Quarterly, 68(2), 103–106. https://doi.org/10.1007/s11301-018-0142-x

Mohamed Shaffril, H. A., Samsuddin, S. F., & Abu Samah, A. (2021). The ABC of systematic literature review: The basic methodological guidance for beginners. Quality & Quantity, 55(4), 1319–1346. https://doi.org/10.1007/s11135-020-01059-6

Okoli, C. (2015). A guide to conducting a standalone systematic literature review. Communications of the Association for Information Systems, 37(1). https://doi.org/10.17705/1CAIS.03743

Siddaway, A. P., Wood, A. M., & Hedges, L. V. (2019). How to do a systematic review: A best practice guide for conducting and reporting narrative reviews, meta-analyses, and meta-syntheses. Annual Review of Psychology, 70, 747–770. https://doi.org/10.1146/annurev-psych-010418-102803

Zawacki-Richter, O., Kerres, M., Bedenlier, S., Bond, M., & Buntins, K. (2020). Systematic reviews in educational research: Methodology, perspectives and application. Springer VS. https://proxy.lib.miamioh.edu/login?url=https://search.ebscohost.com/login.aspx?direct=true&AuthType=cookie,ip,url,uid,cpid&custid=s9002934&db=cat00344a&AN=mucat.b4716649&site=eds-live&scope=site&profile=eds_cat