a case study systematic review

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Systematic Review | Definition, Example, & Guide

Systematic Review | Definition, Example & Guide

Published on June 15, 2022 by Shaun Turney . Revised on November 20, 2023.

A systematic review is a type of review that uses repeatable methods to find, select, and synthesize all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer.

They answered the question “What is the effectiveness of probiotics in reducing eczema symptoms and improving quality of life in patients with eczema?”

In this context, a probiotic is a health product that contains live microorganisms and is taken by mouth. Eczema is a common skin condition that causes red, itchy skin.

Table of contents

What is a systematic review, systematic review vs. meta-analysis, systematic review vs. literature review, systematic review vs. scoping review, when to conduct a systematic review, pros and cons of systematic reviews, step-by-step example of a systematic review, other interesting articles, frequently asked questions about systematic reviews.

A review is an overview of the research that’s already been completed on a topic.

What makes a systematic review different from other types of reviews is that the research methods are designed to reduce bias . The methods are repeatable, and the approach is formal and systematic:

• Formulate a research question
• Develop a protocol
• Search for all relevant studies
• Apply the selection criteria
• Extract the data
• Synthesize the data
• Write and publish a report

Although multiple sets of guidelines exist, the Cochrane Handbook for Systematic Reviews is among the most widely used. It provides detailed guidelines on how to complete each step of the systematic review process.

Systematic reviews are most commonly used in medical and public health research, but they can also be found in other disciplines.

Systematic reviews typically answer their research question by synthesizing all available evidence and evaluating the quality of the evidence. Synthesizing means bringing together different information to tell a single, cohesive story. The synthesis can be narrative ( qualitative ), quantitative , or both.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Systematic reviews often quantitatively synthesize the evidence using a meta-analysis . A meta-analysis is a statistical analysis, not a type of review.

A meta-analysis is a technique to synthesize results from multiple studies. It’s a statistical analysis that combines the results of two or more studies, usually to estimate an effect size .

A literature review is a type of review that uses a less systematic and formal approach than a systematic review. Typically, an expert in a topic will qualitatively summarize and evaluate previous work, without using a formal, explicit method.

Although literature reviews are often less time-consuming and can be insightful or helpful, they have a higher risk of bias and are less transparent than systematic reviews.

Similar to a systematic review, a scoping review is a type of review that tries to minimize bias by using transparent and repeatable methods.

However, a scoping review isn’t a type of systematic review. The most important difference is the goal: rather than answering a specific question, a scoping review explores a topic. The researcher tries to identify the main concepts, theories, and evidence, as well as gaps in the current research.

Sometimes scoping reviews are an exploratory preparation step for a systematic review, and sometimes they are a standalone project.

A systematic review is a good choice of review if you want to answer a question about the effectiveness of an intervention , such as a medical treatment.

To conduct a systematic review, you’ll need the following:

• A precise question , usually about the effectiveness of an intervention. The question needs to be about a topic that’s previously been studied by multiple researchers. If there’s no previous research, there’s nothing to review.
• If you’re doing a systematic review on your own (e.g., for a research paper or thesis ), you should take appropriate measures to ensure the validity and reliability of your research.
• Access to databases and journal archives. Often, your educational institution provides you with access.
• Time. A professional systematic review is a time-consuming process: it will take the lead author about six months of full-time work. If you’re a student, you should narrow the scope of your systematic review and stick to a tight schedule.
• Bibliographic, word-processing, spreadsheet, and statistical software . For example, you could use EndNote, Microsoft Word, Excel, and SPSS.

A systematic review has many pros .

• They minimize research bias by considering all available evidence and evaluating each study for bias.
• Their methods are transparent , so they can be scrutinized by others.
• They’re thorough : they summarize all available evidence.
• They can be replicated and updated by others.

Systematic reviews also have a few cons .

• They’re time-consuming .
• They’re narrow in scope : they only answer the precise research question.

The 7 steps for conducting a systematic review are explained with an example.

Step 1: Formulate a research question

Formulating the research question is probably the most important step of a systematic review. A clear research question will:

• Allow you to more effectively communicate your research to other researchers and practitioners
• Guide your decisions as you plan and conduct your systematic review

A good research question for a systematic review has four components, which you can remember with the acronym PICO :

• Population(s) or problem(s)
• Intervention(s)
• Comparison(s)

You can rearrange these four components to write your research question:

• What is the effectiveness of I versus C for O in P ?

Sometimes, you may want to include a fifth component, the type of study design . In this case, the acronym is PICOT .

• Type of study design(s)
• The population of patients with eczema
• The intervention of probiotics
• In comparison to no treatment, placebo , or non-probiotic treatment
• The outcome of changes in participant-, parent-, and doctor-rated symptoms of eczema and quality of life
• Randomized control trials, a type of study design

Their research question was:

• What is the effectiveness of probiotics versus no treatment, a placebo, or a non-probiotic treatment for reducing eczema symptoms and improving quality of life in patients with eczema?

Step 2: Develop a protocol

A protocol is a document that contains your research plan for the systematic review. This is an important step because having a plan allows you to work more efficiently and reduces bias.

Your protocol should include the following components:

• Background information : Provide the context of the research question, including why it’s important.
• Research objective (s) : Rephrase your research question as an objective.
• Selection criteria: State how you’ll decide which studies to include or exclude from your review.
• Search strategy: Discuss your plan for finding studies.
• Analysis: Explain what information you’ll collect from the studies and how you’ll synthesize the data.

If you’re a professional seeking to publish your review, it’s a good idea to bring together an advisory committee . This is a group of about six people who have experience in the topic you’re researching. They can help you make decisions about your protocol.

It’s highly recommended to register your protocol. Registering your protocol means submitting it to a database such as PROSPERO or ClinicalTrials.gov .

Step 3: Search for all relevant studies

Searching for relevant studies is the most time-consuming step of a systematic review.

To reduce bias, it’s important to search for relevant studies very thoroughly. Your strategy will depend on your field and your research question, but sources generally fall into these four categories:

• Databases: Search multiple databases of peer-reviewed literature, such as PubMed or Scopus . Think carefully about how to phrase your search terms and include multiple synonyms of each word. Use Boolean operators if relevant.
• Handsearching: In addition to searching the primary sources using databases, you’ll also need to search manually. One strategy is to scan relevant journals or conference proceedings. Another strategy is to scan the reference lists of relevant studies.
• Gray literature: Gray literature includes documents produced by governments, universities, and other institutions that aren’t published by traditional publishers. Graduate student theses are an important type of gray literature, which you can search using the Networked Digital Library of Theses and Dissertations (NDLTD) . In medicine, clinical trial registries are another important type of gray literature.
• Experts: Contact experts in the field to ask if they have unpublished studies that should be included in your review.

At this stage of your review, you won’t read the articles yet. Simply save any potentially relevant citations using bibliographic software, such as Scribbr’s APA or MLA Generator .

• Databases: EMBASE, PsycINFO, AMED, LILACS, and ISI Web of Science
• Handsearch: Conference proceedings and reference lists of articles
• Gray literature: The Cochrane Library, the metaRegister of Controlled Trials, and the Ongoing Skin Trials Register
• Experts: Authors of unpublished registered trials, pharmaceutical companies, and manufacturers of probiotics

Step 4: Apply the selection criteria

Applying the selection criteria is a three-person job. Two of you will independently read the studies and decide which to include in your review based on the selection criteria you established in your protocol . The third person’s job is to break any ties.

To increase inter-rater reliability , ensure that everyone thoroughly understands the selection criteria before you begin.

If you’re writing a systematic review as a student for an assignment, you might not have a team. In this case, you’ll have to apply the selection criteria on your own; you can mention this as a limitation in your paper’s discussion.

You should apply the selection criteria in two phases:

• Based on the titles and abstracts : Decide whether each article potentially meets the selection criteria based on the information provided in the abstracts.
• Based on the full texts: Download the articles that weren’t excluded during the first phase. If an article isn’t available online or through your library, you may need to contact the authors to ask for a copy. Read the articles and decide which articles meet the selection criteria.

It’s very important to keep a meticulous record of why you included or excluded each article. When the selection process is complete, you can summarize what you did using a PRISMA flow diagram .

Next, Boyle and colleagues found the full texts for each of the remaining studies. Boyle and Tang read through the articles to decide if any more studies needed to be excluded based on the selection criteria.

When Boyle and Tang disagreed about whether a study should be excluded, they discussed it with Varigos until the three researchers came to an agreement.

Step 5: Extract the data

Extracting the data means collecting information from the selected studies in a systematic way. There are two types of information you need to collect from each study:

• Information about the study’s methods and results . The exact information will depend on your research question, but it might include the year, study design , sample size, context, research findings , and conclusions. If any data are missing, you’ll need to contact the study’s authors.
• Your judgment of the quality of the evidence, including risk of bias .

You should collect this information using forms. You can find sample forms in The Registry of Methods and Tools for Evidence-Informed Decision Making and the Grading of Recommendations, Assessment, Development and Evaluations Working Group .

Extracting the data is also a three-person job. Two people should do this step independently, and the third person will resolve any disagreements.

They also collected data about possible sources of bias, such as how the study participants were randomized into the control and treatment groups.

Step 6: Synthesize the data

Synthesizing the data means bringing together the information you collected into a single, cohesive story. There are two main approaches to synthesizing the data:

• Narrative ( qualitative ): Summarize the information in words. You’ll need to discuss the studies and assess their overall quality.
• Quantitative : Use statistical methods to summarize and compare data from different studies. The most common quantitative approach is a meta-analysis , which allows you to combine results from multiple studies into a summary result.

Generally, you should use both approaches together whenever possible. If you don’t have enough data, or the data from different studies aren’t comparable, then you can take just a narrative approach. However, you should justify why a quantitative approach wasn’t possible.

Boyle and colleagues also divided the studies into subgroups, such as studies about babies, children, and adults, and analyzed the effect sizes within each group.

Step 7: Write and publish a report

The purpose of writing a systematic review article is to share the answer to your research question and explain how you arrived at this answer.

Your article should include the following sections:

• Abstract : A summary of the review
• Introduction : Including the rationale and objectives
• Methods : Including the selection criteria, search method, data extraction method, and synthesis method
• Results : Including results of the search and selection process, study characteristics, risk of bias in the studies, and synthesis results
• Discussion : Including interpretation of the results and limitations of the review
• Conclusion : The answer to your research question and implications for practice, policy, or research

To verify that your report includes everything it needs, you can use the PRISMA checklist .

Once your report is written, you can publish it in a systematic review database, such as the Cochrane Database of Systematic Reviews , and/or in a peer-reviewed journal.

In their report, Boyle and colleagues concluded that probiotics cannot be recommended for reducing eczema symptoms or improving quality of life in patients with eczema. Note Generative AI tools like ChatGPT can be useful at various stages of the writing and research process and can help you to write your systematic review. However, we strongly advise against trying to pass AI-generated text off as your own work.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Student’s  t -distribution
• Normal distribution
• Null and Alternative Hypotheses
• Chi square tests
• Confidence interval
• Quartiles & Quantiles
• Cluster sampling
• Stratified sampling
• Data cleansing
• Reproducibility vs Replicability
• Peer review
• Prospective cohort study

Research bias

• Implicit bias
• Cognitive bias
• Placebo effect
• Hawthorne effect
• Hindsight bias
• Affect heuristic
• Social desirability bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

A systematic review is secondary research because it uses existing research. You don’t collect new data yourself.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Turney, S. (2023, November 20). Systematic Review | Definition, Example & Guide. Scribbr. Retrieved April 9, 2024, from https://www.scribbr.com/methodology/systematic-review/

Is this article helpful?

Shaun Turney

Other students also liked, how to write a literature review | guide, examples, & templates, how to write a research proposal | examples & templates, what is critical thinking | definition & examples, what is your plagiarism score.

Introduction to Systematic Reviews

• Reference work entry
• First Online: 20 July 2022
• pp 2159–2177
• Cite this reference work entry

• Tianjing Li 3 ,
• Ian J. Saldanha 4 &
• Karen A. Robinson 5  

200 Accesses

A systematic review identifies and synthesizes all relevant studies that fit prespecified criteria to answer a research question. Systematic review methods can be used to answer many types of research questions. The type of question most relevant to trialists is the effects of treatments and is thus the focus of this chapter. We discuss the motivation for and importance of performing systematic reviews and their relevance to trialists. We introduce the key steps in completing a systematic review, including framing the question, searching for and selecting studies, collecting data, assessing risk of bias in included studies, conducting a qualitative synthesis and a quantitative synthesis (i.e., meta-analysis), grading the certainty of evidence, and writing the systematic review report. We also describe how to identify systematic reviews and how to assess their methodological rigor. We discuss the challenges and criticisms of systematic reviews, and how technology and innovations, combined with a closer partnership between trialists and systematic reviewers, can help identify effective and safe evidence-based practices more quickly.

• Systematic review
• Meta-analysis
• Research synthesis
• Evidence-based
• Risk of bias

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

AHRQ (2015) Methods guide for effectiveness and comparative effectiveness reviews. Available from https://effectivehealthcare.ahrq.gov/products/cer-methods-guide/overview . Accessed on 27 Oct 2019

Andersen MZ, Gülen S, Fonnes S, Andresen K, Rosenberg J (2020) Half of Cochrane reviews were published more than two years after the protocol. J Clin Epidemiol 124:85–93. https://doi.org/10.1016/j.jclinepi.2020.05.011

Article   Google Scholar  

Berkman ND, Lohr KN, Ansari MT, Balk EM, Kane R, McDonagh M, Morton SC, Viswanathan M, Bass EB, Butler M, Gartlehner G, Hartling L, McPheeters M, Morgan LC, Reston J, Sista P, Whitlock E, Chang S (2015) Grading the strength of a body of evidence when assessing health care interventions: an EPC update. J Clin Epidemiol 68(11):1312–1324

Borah R, Brown AW, Capers PL, Kaiser KA (2017) Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open 7(2):e012545. https://doi.org/10.1136/bmjopen-2016-012545

Chalmers I, Bracken MB, Djulbegovic B, Garattini S, Grant J, Gülmezoglu AM, Howells DW, Ioannidis JP, Oliver S (2014) How to increase value and reduce waste when research priorities are set. Lancet 383(9912):156–165. https://doi.org/10.1016/S0140-6736(13)62229-1

Clarke M, Chalmers I (1998) Discussion sections in reports of controlled trials published in general medical journals: islands in search of continents? JAMA 280(3):280–282

Cooper NJ, Jones DR, Sutton AJ (2005) The use of systematic reviews when designing studies. Clin Trials 2(3):260–264

Djulbegovic B, Kumar A, Magazin A, Schroen AT, Soares H, Hozo I, Clarke M, Sargent D, Schell MJ (2011) Optimism bias leads to inconclusive results-an empirical study. J Clin Epidemiol 64(6):583–593. https://doi.org/10.1016/j.jclinepi.2010.09.007

Elliott JH, Synnot A, Turner T, Simmonds M, Akl EA, McDonald S, Salanti G, Meerpohl J, MacLehose H, Hilton J, Tovey D, Shemilt I, Thomas J (2017) Living systematic review network. Living systematic review: 1. Introduction-the why, what, when, and how. J Clin Epidemiol 91:23–30

Equator Network. Reporting guidelines for systematic reviews. Available from https://www.equator-network.org/?post_type=eq_guidelines&eq_guidelines_study_design=systematic-reviews-and-meta-analyses&eq_guidelines_clinical_specialty=0&eq_guidelines_report_section=0&s=+ . Accessed 9 Mar 2020

Garner P, Hopewell S, Chandler J, MacLehose H, Schünemann HJ, Akl EA, Beyene J, Chang S, Churchill R, Dearness K, Guyatt G, Lefebvre C, Liles B, Marshall R, Martínez García L, Mavergames C, Nasser M, Qaseem A, Sampson M, Soares-Weiser K, Takwoingi Y, Thabane L, Trivella M, Tugwell P, Welsh E, Wilson EC, Schünemann HJ (2016) Panel for updating guidance for systematic reviews (PUGs). When and how to update systematic reviews: consensus and checklist. BMJ 354:i3507. https://doi.org/10.1136/bmj.i3507 . Erratum in: BMJ 2016 Sep 06 354:i4853

Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, Norris S, Falck-Ytter Y, Glasziou P, DeBeer H, Jaeschke R, Rind D, Meerpohl J, Dahm P, Schünemann HJ (2011) GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol 64(4):383–394

Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (eds) (2019a) Cochrane handbook for systematic reviews of interventions, 2nd edn. Wiley, Chichester

Google Scholar  

Higgins JPT, Lasserson T, Chandler J, Tovey D, Thomas J, Flemyng E, Churchill R (2019b) Standards for the conduct of new Cochrane intervention reviews. In: JPT H, Lasserson T, Chandler J, Tovey D, Thomas J, Flemyng E, Churchill R (eds) Methodological expectations of Cochrane intervention reviews. Cochrane, London

IOM (2011) Committee on standards for systematic reviews of comparative effectiveness research, board on health care services. In: Eden J, Levit L, Berg A, Morton S (eds) Finding what works in health care: standards for systematic reviews. National Academies Press, Washington, DC

Jonnalagadda SR, Goyal P, Huffman MD (2015) Automating data extraction in systematic reviews: a systematic review. Syst Rev 4:78

Krnic Martinic M, Pieper D, Glatt A, Puljak L (2019) Definition of a systematic review used in overviews of systematic reviews, meta-epidemiological studies and textbooks. BMC Med Res Methodol 19(1):203. Published 4 Nov 2019. https://doi.org/10.1186/s12874-019-0855-0

Lasserson TJ, Thomas J, Higgins JPT (2019) Chapter 1: Starting a review. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (eds) Cochrane handbook for systematic reviews of interventions version 6.0 (updated July 2019). Cochrane. Available from www.training.cochrane.org/handbook

Lau J, Antman EM, Jimenez-Silva J, Kupelnick B, Mosteller F, Chalmers TC (1992) Cumulative meta-analysis of therapeutic trials for myocardial infarction. N Engl J Med 327(4):248–254

Lau J (2019) Editorial: systematic review automation thematic series. Syst Rev 8(1):70. Published 11 Mar 2019. https://doi.org/10.1186/s13643-019-0974-z

Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D (2009) The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med 6(7):e1000100. https://doi.org/10.1371/journal.pmed.1000100

Lund H, Brunnhuber K, Juhl C, Robinson K, Leenaars M, Dorch BF, Jamtvedt G, Nortvedt MW, Christensen R, Chalmers I (2016) Towards evidence based research. BMJ 355:i5440. https://doi.org/10.1136/bmj.i5440

Marshall IJ, Noel-Storr A, Kuiper J, Thomas J, Wallace BC (2018) Machine learning for identifying randomized controlled trials: an evaluation and practitioner’s guide. Res Synth Methods 9(4):602–614. https://doi.org/10.1002/jrsm.1287

Michelson M, Reuter K (2019) The significant cost of systematic reviews and meta-analyses: a call for greater involvement of machine learning to assess the promise of clinical trials. Contemp Clin Trials Commun 16:100443. https://doi.org/10.1016/j.conctc.2019.100443 . Erratum in: Contemp Clin Trials Commun 2019 16:100450

Moher D, Liberati A, Tetzlaff J (2009) Altman DG; PRISMA group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med 151(4):264–269. W64

Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA, PRISMA-P Group (2015) Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev 4(1):1. https://doi.org/10.1186/2046-4053-4-1

NIHR HTA Stage 1 guidance notes. Available from https://www.nihr.ac.uk/documents/hta-stage-1-guidance-notes/11743 ; Accessed 10 Mar 2020

Page MJ, Shamseer L, Altman DG, Tetzlaff J, Sampson M, Tricco AC, Catalá-López F, Li L, Reid EK, Sarkis-Onofre R, Moher D (2016) Epidemiology and reporting characteristics of systematic reviews of biomedical research: a cross-sectional study. PLoS Med 13(5):e1002028. https://doi.org/10.1371/journal.pmed.1002028

Page MJ, Higgins JPT, Sterne JAC (2019) Chapter 13: assessing risk of bias due to missing results in a synthesis. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ et al (eds) Cochrane handbook for systematic reviews of interventions, 2nd edn. Wiley, Chichester, pp 349–374

Chapter   Google Scholar  

Robinson KA (2009) Use of prior research in the justification and interpretation of clinical trials. Johns Hopkins University

Robinson KA, Goodman SN (2011) A systematic examination of the citation of prior research in reports of randomized, controlled trials. Ann Intern Med 154(1):50–55. https://doi.org/10.7326/0003-4819-154-1-201101040-00007

Rouse B, Cipriani A, Shi Q, Coleman AL, Dickersin K, Li T (2016) Network meta-analysis for clinical practice guidelines – a case study on first-line medical therapies for primary open-angle glaucoma. Ann Intern Med 164(10):674–682. https://doi.org/10.7326/M15-2367

Saldanha IJ, Lindsley K, Do DV et al (2017) Comparison of clinical trial and systematic review outcomes for the 4 most prevalent eye diseases. JAMA Ophthalmol 135(9):933–940. https://doi.org/10.1001/jamaophthalmol.2017.2583

Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, Porter AC, Tugwell P, Moher D, Bouter LM (2007) Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol 7:10

Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, Moher D, Tugwell P, Welch V, Kristjansson E, Henry DA (2017) AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ 358:j4008. https://doi.org/10.1136/bmj.j4008

Shojania KG, Sampson M, Ansari MT, Ji J, Doucette S, Moher D (2007) How quickly do systematic reviews go out of date? A survival analysis. Ann Intern Med 147(4):224–233

Sterne JA, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, Henry D, Altman DG, Ansari MT, Boutron I, Carpenter JR, Chan AW, Churchill R, Deeks JJ, Hróbjartsson A, Kirkham J, Jüni P, Loke YK, Pigott TD, Ramsay CR, Regidor D, Rothstein HR, Sandhu L, Santaguida PL, Schünemann HJ, Shea B, Shrier I, Tugwell P, Turner L, Valentine JC, Waddington H, Waters E, Wells GA, Whiting PF, Higgins JP (2016) ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ 355:i4919. https://doi.org/10.1136/bmj.i4919

Sterne JAC, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, Cates CJ, Cheng HY, Corbett MS, Eldridge SM, Emberson JR, Hernán MA, Hopewell S, Hróbjartsson A, Junqueira DR, Jüni P, Kirkham JJ, Lasserson T, Li T, McAleenan A, Reeves BC, Shepperd S, Shrier I, Stewart LA, Tilling K, White IR, Whiting PF, Higgins JPT (2019) RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ 366:l4898. https://doi.org/10.1136/bmj.l4898

Thomas J, Kneale D, McKenzie JE, Brennan SE, Bhaumik S (2019) Chapter 2: determining the scope of the review and the questions it will address. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (eds) Cochrane handbook for systematic reviews of interventions version 6.0 (updated July 2019). Cochrane. Available from www.training.cochrane.org/handbook

USPSTF U.S. Preventive Services Task Force Procedure Manual (2017). Available from: https://www.uspreventiveservicestaskforce.org/uspstf/sites/default/files/inline-files/procedure-manual2017_update.pdf . Accessed 21 May 2020

Whitaker (2015) UCSF guides: systematic review: when will i be finished? https://guides.ucsf.edu/c.php?g=375744&p=3041343 , Accessed 13 May 2020

Whiting P, Savović J, Higgins JP, Caldwell DM, Reeves BC, Shea B, Davies P, Kleijnen J (2016) Churchill R; ROBIS group. ROBIS: a new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol 69:225–234. https://doi.org/10.1016/j.jclinepi.2015.06.005

Download references

Author information

Authors and affiliations.

Department of Ophthalmology, University of Colorado Anschutz Medical Campus, Aurora, CO, USA

Tianjing Li

Department of Health Services, Policy, and Practice and Department of Epidemiology, Brown University School of Public Health, Providence, RI, USA

Ian J. Saldanha

Department of Medicine, Johns Hopkins University, Baltimore, MD, USA

Karen A. Robinson

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Tianjing Li .

Editor information

Editors and affiliations.

Department of Surgery, Division of Surgical Oncology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA

Steven Piantadosi

Department of Epidemiology, School of Public Health, Johns Hopkins University, Baltimore, MD, USA

Curtis L. Meinert

Section Editor information

Department of Epidemiology, University of Colorado Denver Anschutz Medical Campus, Aurora, CO, USA

The Johns Hopkins Center for Clinical Trials and Evidence Synthesis, Johns Hopkins University, Baltimore, MD, USA

Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this entry

Cite this entry.

Li, T., Saldanha, I.J., Robinson, K.A. (2022). Introduction to Systematic Reviews. In: Piantadosi, S., Meinert, C.L. (eds) Principles and Practice of Clinical Trials. Springer, Cham. https://doi.org/10.1007/978-3-319-52636-2_194

Download citation

DOI : https://doi.org/10.1007/978-3-319-52636-2_194

Published : 20 July 2022

Publisher Name : Springer, Cham

Print ISBN : 978-3-319-52635-5

Online ISBN : 978-3-319-52636-2

eBook Packages : Mathematics and Statistics Reference Module Computer Science and Engineering

Share this entry

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Systematic Reviews and Meta Analysis

• Getting Started
• Guides and Standards
• Review Protocols
• Databases and Sources
• Randomized Controlled Trials
• Controlled Clinical Trials
• Observational Designs
• Tests of Diagnostic Accuracy
• Software and Tools
• Where do I get all those articles?
• Collaborations
• EPI 233/528
• Countway Mediated Search
• Risk of Bias (RoB)

Systematic review Q & A

What is a systematic review.

A systematic review is guided filtering and synthesis of all available evidence addressing a specific, focused research question, generally about a specific intervention or exposure. The use of standardized, systematic methods and pre-selected eligibility criteria reduce the risk of bias in identifying, selecting and analyzing relevant studies. A well-designed systematic review includes clear objectives, pre-selected criteria for identifying eligible studies, an explicit methodology, a thorough and reproducible search of the literature, an assessment of the validity or risk of bias of each included study, and a systematic synthesis, analysis and presentation of the findings of the included studies. A systematic review may include a meta-analysis.

For details about carrying out systematic reviews, see the Guides and Standards section of this guide.

Is my research topic appropriate for systematic review methods?

A systematic review is best deployed to test a specific hypothesis about a healthcare or public health intervention or exposure. By focusing on a single intervention or a few specific interventions for a particular condition, the investigator can ensure a manageable results set. Moreover, examining a single or small set of related interventions, exposures, or outcomes, will simplify the assessment of studies and the synthesis of the findings.

Systematic reviews are poor tools for hypothesis generation: for instance, to determine what interventions have been used to increase the awareness and acceptability of a vaccine or to investigate the ways that predictive analytics have been used in health care management. In the first case, we don't know what interventions to search for and so have to screen all the articles about awareness and acceptability. In the second, there is no agreed on set of methods that make up predictive analytics, and health care management is far too broad. The search will necessarily be incomplete, vague and very large all at the same time. In most cases, reviews without clearly and exactly specified populations, interventions, exposures, and outcomes will produce results sets that quickly outstrip the resources of a small team and offer no consistent way to assess and synthesize findings from the studies that are identified.

If not a systematic review, then what?

You might consider performing a scoping review . This framework allows iterative searching over a reduced number of data sources and no requirement to assess individual studies for risk of bias. The framework includes built-in mechanisms to adjust the analysis as the work progresses and more is learned about the topic. A scoping review won't help you limit the number of records you'll need to screen (broad questions lead to large results sets) but may give you means of dealing with a large set of results.

This tool can help you decide what kind of review is right for your question.

Can my student complete a systematic review during her summer project?

Probably not. Systematic reviews are a lot of work. Including creating the protocol, building and running a quality search, collecting all the papers, evaluating the studies that meet the inclusion criteria and extracting and analyzing the summary data, a well done review can require dozens to hundreds of hours of work that can span several months. Moreover, a systematic review requires subject expertise, statistical support and a librarian to help design and run the search. Be aware that librarians sometimes have queues for their search time. It may take several weeks to complete and run a search. Moreover, all guidelines for carrying out systematic reviews recommend that at least two subject experts screen the studies identified in the search. The first round of screening can consume 1 hour per screener for every 100-200 records. A systematic review is a labor-intensive team effort.

How can I know if my topic has been been reviewed already?

Before starting out on a systematic review, check to see if someone has done it already. In PubMed you can use the systematic review subset to limit to a broad group of papers that is enriched for systematic reviews. You can invoke the subset by selecting if from the Article Types filters to the left of your PubMed results, or you can append AND systematic[sb] to your search. For example:

"neoadjuvant chemotherapy" AND systematic[sb]

The systematic review subset is very noisy, however. To quickly focus on systematic reviews (knowing that you may be missing some), simply search for the word systematic in the title:

"neoadjuvant chemotherapy" AND systematic[ti]

Any PRISMA-compliant systematic review will be captured by this method since including the words "systematic review" in the title is a requirement of the PRISMA checklist. Cochrane systematic reviews do not include 'systematic' in the title, however. It's worth checking the Cochrane Database of Systematic Reviews independently.

You can also search for protocols that will indicate that another group has set out on a similar project. Many investigators will register their protocols in PROSPERO , a registry of review protocols. Other published protocols as well as Cochrane Review protocols appear in the Cochrane Methodology Register, a part of the Cochrane Library .

• Next: Guides and Standards >>
• Last Updated: Feb 26, 2024 3:17 PM
• URL: https://guides.library.harvard.edu/meta-analysis

A Protocol for the Use of Case Reports/Studies and Case Series in Systematic Reviews for Clinical Toxicology

Affiliations.

• 1 Univ Angers, CHU Angers, Univ Rennes, INSERM, EHESP, Institut de Recherche en Santé, Environnement et Travail-UMR_S 1085, Angers, France.
• 2 Department of Occupational Medicine, Epidemiology and Prevention, Donald and Barbara Zucker School of Medicine, Northwell Health, Feinstein Institutes for Medical Research, Hofstra University, Great Neck, NY, United States.
• 3 Department of Health Sciences, University of California, San Francisco and California State University, Hayward, CA, United States.
• 4 Program on Reproductive Health and the Environment, University of California, San Francisco, San Francisco, CA, United States.
• 5 Cesare Maltoni Cancer Research Center, Ramazzini Institute, Bologna, Italy.
• 6 Department of Research and Public Health, Reims Teaching Hospitals, Robert Debré Hospital, Reims, France.
• 7 CHU Angers, Univ Angers, Poisoning Control Center, Clinical Data Center, Angers, France.
• PMID: 34552944
• PMCID: PMC8450367
• DOI: 10.3389/fmed.2021.708380

Introduction: Systematic reviews are routinely used to synthesize current science and evaluate the evidential strength and quality of resulting recommendations. For specific events, such as rare acute poisonings or preliminary reports of new drugs, we posit that case reports/studies and case series (human subjects research with no control group) may provide important evidence for systematic reviews. Our aim, therefore, is to present a protocol that uses rigorous selection criteria, to distinguish high quality case reports/studies and case series for inclusion in systematic reviews. Methods: This protocol will adapt the existing Navigation Guide methodology for specific inclusion of case studies. The usual procedure for systematic reviews will be followed. Case reports/studies and case series will be specified in the search strategy and included in separate sections. Data from these sources will be extracted and where possible, quantitatively synthesized. Criteria for integrating cases reports/studies and case series into the overall body of evidence are that these studies will need to be well-documented, scientifically rigorous, and follow ethical practices. The instructions and standards for evaluating risk of bias will be based on the Navigation Guide. The risk of bias, quality of evidence and the strength of recommendations will be assessed by two independent review teams that are blinded to each other. Conclusion: This is a protocol specified for systematic reviews that use case reports/studies and case series to evaluate the quality of evidence and strength of recommendations in disciplines like clinical toxicology, where case reports/studies are the norm.

Keywords: case reports/studies; case series; epidemiology; protocol; public health; systematic review; toxicology.

Copyright © 2021 Nambiema, Sembajwe, Lam, Woodruff, Mandrioli, Chartres, Fadel, Le Guillou, Valter, Deguigne, Legeay, Bruneau, Le Roux and Descatha.

• UNC Libraries
• HSL Academic Process
• Systematic Reviews

Systematic Reviews: Home

Created by health science librarians.

• Systematic review resources

What is a Systematic Review?

A simplified process map, how can the library help, publications by hsl librarians, systematic reviews in non-health disciplines, resources for performing systematic reviews.

• Step 1: Complete Pre-Review Tasks
• Step 2: Develop a Protocol
• Step 3: Conduct Literature Searches
• Step 4: Manage Citations
• Step 5: Screen Citations
• Step 6: Assess Quality of Included Studies
• Step 7: Extract Data from Included Studies
• Step 8: Write the Review

  Check our FAQ's

   Email us

  Chat with us (during business hours)

   Call (919) 962-0800

   Make an appointment with a librarian

  Request a systematic or scoping review consultation

A systematic review is a literature review that gathers all of the available evidence matching pre-specified eligibility criteria to answer a specific research question. It uses explicit, systematic methods, documented in a protocol, to minimize bias , provide reliable findings , and inform decision-making.  ¹  

There are many types of literature reviews.

Before beginning a systematic review, consider whether it is the best type of review for your question, goals, and resources. The table below compares a few different types of reviews to help you decide which is best for you. 

• Scoping Review Guide For more information about scoping reviews, refer to the UNC HSL Scoping Review Guide.

• UNC HSL's Simplified, Step-by-Step Process Map A PDF file of the HSL's Systematic Review Process Map.
• Text-Only: UNC HSL's Systematic Reviews - A Simplified, Step-by-Step Process A text-only PDF file of HSL's Systematic Review Process Map.

The average systematic review takes 1,168 hours to complete. ¹   A librarian can help you speed up the process.

Systematic reviews follow established guidelines and best practices to produce high-quality research. Librarian involvement in systematic reviews is based on two levels. In Tier 1, your research team can consult with the librarian as needed. The librarian will answer questions and give you recommendations for tools to use. In Tier 2, the librarian will be an active member of your research team and co-author on your review. Roles and expectations of librarians vary based on the level of involvement desired. Examples of these differences are outlined in the table below.

• Request a systematic or scoping review consultation

The following are systematic and scoping reviews co-authored by HSL librarians.

Only the most recent 15 results are listed. Click the website link at the bottom of the list to see all reviews co-authored by HSL librarians in PubMed

Researchers conduct systematic reviews in a variety of disciplines.  If your focus is on a topic outside of the health sciences, you may want to also consult the resources below to learn how systematic reviews may vary in your field.  You can also contact a librarian for your discipline with questions.

• EPPI-Centre methods for conducting systematic reviews The EPPI-Centre develops methods and tools for conducting systematic reviews, including reviews for education, public and social policy.

Environmental Topics

• Collaboration for Environmental Evidence (CEE) CEE seeks to promote and deliver evidence syntheses on issues of greatest concern to environmental policy and practice as a public service

Social Sciences

• Siddaway AP, Wood AM, Hedges LV. How to Do a Systematic Review: A Best Practice Guide for Conducting and Reporting Narrative Reviews, Meta-Analyses, and Meta-Syntheses. Annu Rev Psychol. 2019 Jan 4;70:747-770. doi: 10.1146/annurev-psych-010418-102803. A resource for psychology systematic reviews, which also covers qualitative meta-syntheses or meta-ethnographies
• The Campbell Collaboration

Social Work

Software engineering

• Guidelines for Performing Systematic Literature Reviews in Software Engineering The objective of this report is to propose comprehensive guidelines for systematic literature reviews appropriate for software engineering researchers, including PhD students.

Sport, Exercise, & Nutrition

• Application of systematic review methodology to the field of nutrition by Tufts Evidence-based Practice Center Publication Date: 2009
• Systematic Reviews and Meta-Analysis — Open & Free (Open Learning Initiative) The course follows guidelines and standards developed by the Campbell Collaboration, based on empirical evidence about how to produce the most comprehensive and accurate reviews of research

• Systematic Reviews by David Gough, Sandy Oliver & James Thomas Publication Date: 2020

Updating reviews

• Updating systematic reviews by University of Ottawa Evidence-based Practice Center Publication Date: 2007

Looking for our previous Systematic Review guide?

Our legacy guide was used June 2020 to August 2022

• Systematic Review Legacy Guide
• Next: Step 1: Complete Pre-Review Tasks >>
• Last Updated: Mar 28, 2024 9:43 AM
• URL: https://guides.lib.unc.edu/systematic-reviews

Search & Find

• E-Research by Discipline
• More Search & Find

Places & Spaces

• Places to Study
• Book a Study Room
• Printers, Scanners, & Computers
• More Places & Spaces
• Borrowing & Circulation
• Request a Title for Purchase
• Schedule Instruction Session
• More Services

Support & Guides

• Course Reserves
• Research Guides
• Citing & Writing
• More Support & Guides
• Mission Statement
• Diversity Statement
• Staff Directory
• Job Opportunities
• Give to the Libraries
• News & Exhibits
• Reckoning Initiative
• More About Us

• Search This Site
• Privacy Policy
• Accessibility
• Give Us Your Feedback
• 208 Raleigh Street CB #3916
• Chapel Hill, NC 27515-8890
• 919-962-1053

Study Design 101

• Helpful formulas
• Finding specific study types
• Systematic Review
• Meta- Analysis
• Practice Guideline
• Randomized Controlled Trial
• Cohort Study
• Case Control Study
• Case Reports

A document often written by a panel that provides a comprehensive review of all relevant studies on a particular clinical or health-related topic/question. The systematic review is created after reviewing and combining all the information from both published and unpublished studies (focusing on clinical trials of similar treatments) and then summarizing the findings.

• Exhaustive review of the current literature and other sources (unpublished studies, ongoing research)
• Less costly to review prior studies than to create a new study
• Less time required than conducting a new study
• Results can be generalized and extrapolated into the general population more broadly than individual studies
• More reliable and accurate than individual studies
• Considered an evidence-based resource

Disadvantages

• Very time-consuming
• May not be easy to combine studies

Design pitfalls to look out for

Studies included in systematic reviews may be of varying study designs, but should collectively be studying the same outcome.

Is each study included in the review studying the same variables?

Some reviews may group and analyze studies by variables such as age and gender; factors that were not allocated to participants.

Do the analyses in the systematic review fit the variables being studied in the original studies?

Fictitious Example

Does the regular wearing of ultraviolet-blocking sunscreen prevent melanoma? An exhaustive literature search was conducted, resulting in 54 studies on sunscreen and melanoma. Each study was then evaluated to determine whether the study focused specifically on ultraviolet-blocking sunscreen and melanoma prevention; 30 of the 54 studies were retained. The thirty studies were reviewed and showed a strong positive relationship between daily wearing of sunscreen and a reduced diagnosis of melanoma.

Real-life Examples

Yang, J., Chen, J., Yang, M., Yu, S., Ying, L., Liu, G., ... Liang, F. (2018). Acupuncture for hypertension. The Cochrane Database of Systematic Reviews, 11 (11), CD008821. https://doi.org/10.1002/14651858.CD008821.pub2

This systematic review analyzed twenty-two randomized controlled trials to determine whether acupuncture is a safe and effective way to lower blood pressure in adults with primary hypertension. Due to the low quality of evidence in these studies and lack of blinding, it is not possible to link any short-term decrease in blood pressure to the use of acupuncture. Additional research is needed to determine if there is an effect due to acupuncture that lasts at least seven days.

Parker, H.W. and Vadiveloo, M.K. (2019). Diet quality of vegetarian diets compared with nonvegetarian diets: a systematic review. Nutrition Reviews , https://doi.org/10.1093/nutrit/nuy067

This systematic review was interested in comparing the diet quality of vegetarian and non-vegetarian diets. Twelve studies were included. Vegetarians more closely met recommendations for total fruit, whole grains, seafood and plant protein, and sodium intake. In nine of the twelve studies, vegetarians had higher overall diet quality compared to non-vegetarians. These findings may explain better health outcomes in vegetarians, but additional research is needed to remove any possible confounding variables.

Related Terms

Cochrane Database of Systematic Reviews

A highly-regarded database of systematic reviews prepared by The Cochrane Collaboration , an international group of individuals and institutions who review and analyze the published literature.

Exclusion Criteria

The set of conditions that characterize some individuals which result in being excluded in the study (i.e. other health conditions, taking specific medications, etc.). Since systematic reviews seek to include all relevant studies, exclusion criteria are not generally utilized in this situation.

Inclusion Criteria

The set of conditions that studies must meet to be included in the review (or for individual studies - the set of conditions that participants must meet to be included in the study; often comprises age, gender, disease type and status, etc.).

Now test yourself!

1. Systematic Reviews are similar to Meta-Analyses, except they do not include a statistical analysis quantitatively combining all the studies.

a) True b) False

2. The panels writing Systematic Reviews may include which of the following publication types in their review?

a) Published studies b) Unpublished studies c) Cohort studies d) Randomized Controlled Trials e) All of the above

← Previous Next →

© 2011-2019, The Himmelfarb Health Sciences Library Questions? Ask us .

• Himmelfarb Intranet
• Privacy Notice
• Terms of Use
• GW is committed to digital accessibility. If you experience a barrier that affects your ability to access content on this page, let us know via the Accessibility Feedback Form .

• Open access
• Published: 13 November 2023

Title-plus-abstract versus title-only first-level screening approach: a case study using a systematic review of dietary patterns and sarcopenia risk to compare screening performance

• Lynn Teo   ORCID: orcid.org/0000-0002-5372-362X 1 ,
• Mary E. Van Elswyk   ORCID: orcid.org/0000-0002-4579-0072 2 ,
• Clara S. Lau   ORCID: orcid.org/0000-0002-3667-4539 3 &
• Christopher J. Shanahan   ORCID: orcid.org/0000-0002-2902-6808 4  

Systematic Reviews volume  12 , Article number:  211 ( 2023 ) Cite this article

771 Accesses

9 Altmetric

Metrics details

Conducting a systematic review is a time- and resource-intensive multi-step process. Enhancing efficiency without sacrificing accuracy and rigor during the screening phase of a systematic review is of interest among the scientific community.

This case study compares the screening performance of a title-only (Ti/O) screening approach to the more conventional title-plus-abstract (Ti + Ab) screening approach. Both Ti/O and Ti + Ab screening approaches were performed simultaneously during first-level screening of a systematic review investigating the relationship between dietary patterns and risk factors and incidence of sarcopenia. The qualitative and quantitative performance of each screening approach was compared against the final results of studies included in the systematic review, published elsewhere, which used the standard Ti + Ab approach. A statistical analysis was conducted, and contingency tables were used to compare each screening approach in terms of false inclusions and false exclusions and subsequent sensitivity, specificity, accuracy, and positive predictive power.

Thirty-eight citations were included in the final analysis, published elsewhere. The current case study found that the Ti/O first-level screening approach correctly identified 22 citations and falsely excluded 16 citations, most often due to titles lacking a clear indicator of study design or outcomes relevant to the systematic review eligibility criteria. The Ti + Ab approach correctly identified 36 citations and falsely excluded 2 citations due to limited population and intervention descriptions in the abstract. Our analysis revealed that the performance of the Ti + Ab first-level screening was statistically different compared to the average performance of both approaches (Chi-squared: 5.21, p value 0.0225) while the Ti/O approach was not (chi-squared: 2.92, p value 0.0874). The predictive power of the first-level screening was 14.3% and 25.5% for the Ti/O and Ti + Ab approaches, respectively. In terms of sensitivity, 57.9% of studies were correctly identified at the first-level screening stage using the Ti/O approach versus 94.7% by the Ti + Ab approach.

Conclusions

In the current case study comparing two screening approaches, the Ti + Ab screening approach captured more relevant studies compared to the Ti/O approach by including a higher number of accurately eligible citations. Ti/O screening may increase the likelihood of missing evidence leading to evidence selection bias.

Systematic review registration

PROSPERO Protocol Number: CRD42020172655.

Peer Review reports

Systematic reviews and meta-analyses are a necessary foundation of evidence-based public health recommendations as they help synthesize vast amounts of research to aid in evidence-based healthcare decisions and policy making [ 1 , 2 ]. Systematic reviews are rigorous and time- and resource-intensive processes involving many steps and may often take years to complete [ 3 , 4 ].

A key step in the systematic review process is during the screening phase when one needs to make a judgement on whether a study should be included or excluded based on the pre-determined eligibility criteria. While conventional practice is to screen both the title and abstract (i.e., title-plus-abstract (Ti + Ab)) at the initial screening phase [ 2 , 5 , 6 ], there has been interest in using a title-only (Ti/O) based screening approach to expedite the process [ 7 , 8 , 9 ]. The aim of this case study is to compare the relative screening performance of a title-only (Ti/O) screening approach to the more conventional title-plus-abstract (Ti + Ab) screening approach during first-level screening of citations using a systematic review protocol designed to investigate the relationship between dietary patterns and risk of sarcopenia using disease endpoints and risk factors [ 10 ].

The main outcomes of the current study were designed to quantitatively compare the relative accuracy, sensitivity, specificity, and positive predictive power of the Ti/O versus Ti + Ab approaches; as well as to investigate qualitative reasons for incorrect exclusions of citations of each screening approach. To test the screening performance of the two screening approaches for eligibility for systematic review inclusion, one investigator followed the Ti/O screening approach, and two other investigators separately followed the Ti + Ab screening approach.

Screening and study selection

The same pre-defined eligibility criteria, according to population, intervention, comparator, outcome, and study design (PICOS) were used for both Ti/O and Ti + Ab screening approaches and are reported in Additional file  1 .

During the first-level screening, two investigators with systematic review expertise (L.T. and M.V.E.) independently followed the Ti + Ab approach and a third investigator with subject matter expertise (C.L.) independently followed the Ti/O approach. For training purposes, each screener followed their assigned screening approach on a subset of 1,998 citations (out of a total of 8,526 citations) until an inter-rater agreement reached over 90%. After this training period, C.L. screened all remaining citations using the Ti/O approach, and L.T. and M.V.E. divided the same remaining citations and screened using the Ti + Ab approach. Endnote was used to track the screening approach for each individual screener. At the conclusion of first-level screening, the full team used a Ti + Ab approach to re-screen all studies which passed the first-level screening, regardless of the screening approach, according to the eligibility criteria for the systematic review [ 10 ].

Statistical analysis

To test the relative performance of the two first-level study screening approaches, contingency tables were developed for each screening approach (Table  1 ) and compared against each other using the R statistical analysis package and Microsoft Excel [ 11 ]. A contingency table is a simple 2 × 2 matrix that maps the performance of a given screening approach where one of the axes indicates a study’s eligibility qualification status (e.g., whether a study should have been included/excluded) and the other axis indicates the study’s actual eligibility status (e.g., whether a study has actually been included/excluded) as determined by the reviewer [ 12 ].

Eligible studies (i.e., trues) that either passed or did not pass the first-level screening were defined as “included trues” and “excluded trues”, respectively. Eligible studies were studies ultimately included in the final analysis of the systematic review [ 10 ]. Ineligible studies (i.e., falses) that either passed or did not pass the first-level screening were defined as “included falses” and “excluded falses”, respectively. Ineligible studies were studies ultimately not included in the final analysis of the systematic review [ 10 ]. Table  1 provides a graphical representation of a contingency table used to structure and assess each screening approach’s predictive power.

Contingency tables were used to determine which screening approach is relatively more effective at maximizing the number of correct judgments and/or minimizing the number of incorrect judgements. The relative accuracy of a given screening approach (i.e., ability to correctly identify and classify both qualified or unqualified studies) is a function of the relative sensitivity of the approach or maximizing the number of correct judgments (e.g., “included trues” and “excluded falses”) and the relative specificity of the approach or minimizing the number of incorrect judgments (e.g., “excluded trues” and “included falses”). The sensitivity of the screening approach is the percentage of the citations ultimately included in the systematic review that were correctly identified at the first-level screening stage using the specific screening approach. The specificity reflects the screening approach’s ability to minimize incorrect judgments allowing for the correct identification and exclusion of unqualified citations. The predictive power is the percentage of included studies correctly predicted to be eligible.

Mathematically, the accuracy of a specific screening approach \(i\) was deduced using the following equation:

where \({A}_{i}\) is the accuracy fraction of screening approach \(i\) , \(Z\) is the total potential number of citations included at the initial screening level by one or more reviewers, \({T}_{i}\) is the total number of “trues” included by the reviewer given the study screening approach, \({Q}_{i}\) is the total number of “trues” in the study set \(Z\) , \({F}_{i}\) is the total number of “falses” included by the reviewer given the study screening approach, and \({E}_{i}\) is the total number of “falses” in the study set \(Z\) . The fraction \(\frac{{T}_{i}}{{Q}_{i}}\) is the sensitivity of the screening approach and the fraction \(\frac{{{E}_{i}-F}_{i}}{{E}_{i}}\) is the screening approach’s specificity fraction. In addition, the predictive power \({P}_{i}\) ratio of each screening approach was determined from the contingency table and is calculated as follows:

\({P}_{i}\) describes how well the first-level screening approach was at correctly predicting the qualified studies included or “included trues.”

To test whether a given screening approach was statistically more effective in terms of sensitivity and specificity, and thus relative accuracy and predictive power, each screening approach was compared to a composite or average contingency table of both screening approaches to calculate the differences in sensitivity and specificity and then Chi-squared test statistics and associated p-values for each comparison were calculated. If a Chi-squared test of a given screening approach’s sensitivity and specificity measurements are found to be statistically significant and higher in value compared to the other screening approach, then this implies that the given screening approach is likely more effective at obtaining an accurate outcome..

Quantitative results

When comparing the results of the first-level screening to the final analysis, the Ti/O screening approach resulted in missing 16 of the 38 citations that were eventually included in the final systematic review (Fig.  1 ), resulting in an excluded “trues” rate of 42.1% (Table  2 ). The Ti/O screening approach’s sensitivity was 57.9%; with 22 of the 38 studies ultimately included in the systematic review correctly identified using the Ti/O approach. The specificity of the Ti/O approach was 98.4%, allowing the correct identification and exclusion of 8356 unqualified citations out of a total of 8526 citations, and an accuracy score of 98.3%. With a predictive power of 14.3%, the Ti/O screening approach correctly predicted only 14.3% of the included studies versus the average predictive power of 19.7%. The overall screening performance of the Ti/O approach was not statistically significantly different from the average performance as reported in Table  2 (chi-squared statistic = 2.92; p value = 0.874; Table  2 C).

Systematic review flow chart comparing screening results between a title-only (Ti/O) to a title-plus-abstract (Ti + Ab) screening approach at the first level. 1st level: first-level screening where two investigators independently followed a Ti + Ab screening approach, and a 3rd investigator followed a Ti/O screening approach. 2nd level: second-level screening where the full team rescreened all studies which passed the first-level screen using a Ti + Ab screening approach. Full-text level: full-text screening where the full team screened all studies which passed the second-level screen

1 Population; 2 Intervention; 3 Comparison; 4 Outcome; 5 Study design

The performance of the Ti + Ab first-level screening approach resulted in missing 2 of the 38 citations that were eventually included in the final systematic review (Fig.  1 ), resulting in an “excluded trues” rate of 5.3% (Table  2 ). With 36 of the 38 qualified studies correctly identified using the Ti + Ab approach at the first-level screening stage, the sensitivity of 94.7% was higher than that of the Ti/O approach (Table  2 and Fig.  1 ). The specificity of the Ti + Ab approach was 98.8% and an accuracy score of 98.75%. At 25.5%, the predictive power of the Ti + Ab approach was higher than the average screening performance of 19.7%. Overall, the performance of the Ti + Ab first-level screening approach is significantly different from the average performance (chi-squared statistic = 5.21; p value = 0.0225; Table  2 C and Fig.  1 ).

Qualitative results

The Ti/O approach mistakenly excluded 16 qualified (“excluded trues”) citations that were included by the Ti + Ab approach and ultimately included in the final systematic review. A closer retrospective look at the “excluded trues” by the Ti/O approach revealed that 14 titles were lacking a clear study design element. Further, 13 study titles made no mention of an outcome measure directly related to the systematic review’s eligibility criteria (see Additional file  2 ).

Of the 141 citations passed by the Ti + Ab approach at first-level screening, 36 citations were ultimately included for the systematic review. Two citations were falsely excluded where the description of the population was not clear in one abstract [ 14 ] and the description of protein quartiles was not clearly described as being outside the acceptable macronutrient distribution range (AMDR) in the other abstract [ 15 ].

This case study found that Ti/O screening performed in accordance with statistical expectations while Ti + Ab performed significantly better than was expected. It should be noted that the Chi-squared statistic only tests whether the screening approach’s performance is statistically different from a composite or average level of performance. It does not indicate how or why there is a difference in performance. Only a comparison of the summary performance statistics from the test and average contingency tables (i.e., specificity, sensitivity, accuracy, and predictive power) can address the questions of how and why there is a relative difference in performance. This case study found the Ti + Ab screening approach at the first-level screening phase had higher predictive power due to being more effective at finding qualified studies to include in the next stage of evaluation compared to the Ti/O screening approach. This suggests that reviewing both the title and abstract during first-level review stage resulted in a more effective screening performance.

Our case study demonstrated that both the Ti/O and Ti + Ab first-level screening approaches achieved high specificity (98.4% and 98.8%, respectively), due to the small base of qualified studies within the total number of citations included at the initial screening level. In our case, high specificity was easy to achieve as the actual prevalence of eligible studies in the initial search results is relatively small (36/8526 = 0.4%) allowing for a high rate of correct identification and exclusion of unqualified studies by both approaches. Likewise, both the Ti/O and Ti + Ab first-level approaches also achieved high accuracy (98.3% and 98.8%).

The Ti + Ab approach’s predictive power of 25.5% was observed to be higher than the Ti/O approach’s power of 14.3%. This suggests that more information assessed at the first-level screening will likely increase the number of studies correctly predicted to be eligible; thus less time will be required for the next stage of evaluation. This also suggests that despite the Ti/O approach including comparably more studies for the next stage evaluation compared to the Ti + Ab approach (154 citations versus 141 citations, respectively, Table  2 ), the predictive power of the Ti/O screening approach is still lower and thus less reliable for identifying qualified studies. Further, compared to the Ti/O approach, the Ti + Ab approach had a much lower excluded “trues” rate (5.3% vs. 41.1%) and higher sensitivity (57.9% vs. 94.7%) highlighting that the latter approach is more likely to capture studies of interest and less likely to pass unqualified studies to the full-text level, potentially saving both time and financial resources. Methodologies that systematically lead to missing relevant articles result in evidence selection bias, a bias that occurs when all available data on a topic has not been identified [ 16 ]. This can impact the synthesis of evidence and bias the resulting conclusions, in a direction inconsistent with the true association [ 2 ].

Our findings parallel a previous study which compared the differences between Ti/O and Ti + Ab screening approaches where Mateen et al. [ 8 ], found that Ti + Ab screening achieved higher precision compared to the Ti/O approach and required the review of fewer full-text articles compared to the Ti/O screening. However, the authors also suggest that Ti/O screening was possibly more efficient than Ti + Ab screening because of the expected time saved from not reading the abstracts of unqualified studies, even though more time was required during the full-text review stage. The authors do admit that they did not measure relative screening times and thus could not provide substantiation of their time-savings hypothesis.

Choosing a screening approach may be based on the nature of the eligibility criteria. Titles alone may not have enough information to make predefined PICOS eligibility judgments. More detailed eligibility criteria may increase the likelihood that titles alone will not be enough to reveal if a study qualifies. Intermediate markers of sarcopenia risk (i.e., skeletal muscle mass, muscle strength, muscle performance) may not have been explicitly mentioned in titles but only relayed in the abstract or the full text. For example, in this case study, four titles that mentioned “frailty” were screened out using Ti/O because this was not an outcome of interest but the Ti + Ab screening provided the opportunity to review more detailed methodology and information regarding measurement of secondary outcomes that were consistent with the eligibility criteria. Information in the abstracts revealed that frailty can be determined by a variety of outcomes including gait speed and hand grip which were also eligible measurements of muscle strength and muscle performance in our protocol (see Additional file  2 ). Zhu et al.’s “A Prospective Investigation of Dietary Intake and Functional Impairments Among the Elderly [ 17 ]” does not mention investigating impairment in walking capability, a measure of muscle performance, in its title but this information is available in the abstract. To capture these outcomes of interest, Ti + Ab screening would become a necessary next step of Ti/O screening, and erring on the side of inclusion to screen more full texts may be needed for eligibility determination. There are situations when the title or sometimes even the abstract of a paper does not contain all the PICOS information necessary to predict whether the citation should be included. This is particularly true for studies published previous to reporting guidelines: the CONSORT Statement [ 18 ], the reporting guidelines for randomized controlled trials was developed in 1996; and QUORUM, the predecessor to the PRISMA Statement, the reporting guidelines for systematic reviews and meta-analyses was developed in 1999 [ 19 ]. In the current review, the information needed to assess whether a dietary pattern based on a macronutrient distribution, where at least one macronutrient had to be outside of AMDR, would often not be present in the title alone (e.g., “Adult macronutrient intake and physical capability in the MRC National Survey of Health and Development [ 20 ]”) nor sufficiently specified in the abstract. In this case, erring on the side of inclusion for full-text review during screening would be a viable strategy to avoid missing relevant citations.

The Ti + Ab approach to first-level screening in this study mistakenly excluded two citations, one per Ti + Ab reviewer, that were ultimately included in the final analysis, despite having access to the abstract. Due to the complexity of the eligibility criteria in this case study, it can be debated that each citation should have been reviewed in tandem per screening approach to avoid excluded trues. It is common for teams to debate whether some studies fit the eligibility criteria, even at the full-text level, as in the case of these two studies (see Additional file  2 ).

Should Ti/O prove effective and time-saving as a screening approach, reporting guidelines should consider recommending study titles to include all PICOS elements, as currently the abstract is used for this information. Implementing revised reporting guidelines would further require reconsideration by journals of title character limitations which currently prove a challenge when attempting to include all PICOS elements. Currently, both the current CONSORT Statement [ 21 ] for randomized controlled trials, and the PRISMA Statement [ 22 ] for systematic reviews and meta-analyses recommend that the study design is explicit in the title. CONSORT explicitly requires PICO to be included in the journal and conference abstracts [ 23 ] and the PRISMA Statement recommends that the study design is explicit in the title and that the eligibility criteria (which usually reflects PICO) are reported in the abstract [ 22 ]. Additionally, the Journal of the American Medical Association as well as the Annals of Internal Medicine requires the type of study design (i.e., clinical trials, meta-analyses, and systematic reviews) as part of the publication’s title [ 24 , 25 ]. While not an exhaustive review of the topic, these examples suggest that standard reporting guidelines and journal requirements for titles may not readily support the use of the Ti/O screening approach.

This case study has limitations. First, it would have been more methodologically optimal to have two investigators conducting Ti/O first-level screening in tandem for consistency with the Ti + Ab first-level screening. Further, there were differences in backgrounds in investigating teams for the two approaches: the Ti/O approach was performed by an investigator with subject matter expertise while Ti + Ab approach was performed by two investigators with systematic review expertise. Second, the average time to conduct the first-level screening using either approach was not measured. This would have been useful information as time taken is essential to reflect both efficiency as well as cost. Time was not tracked as this case study was a secondary focus to the original systematic review. This being said, we would recommend tracking time in future similar investigations. Lastly, this case study is based on the manual screening of citations and may not be generalizable to screening approaches using artificial intelligence (A.I.) or automated screening technology (e.g., DistillerSR or Covidence) to lessen the workload attributed to screening. On the other hand, this case study has strengths. First, this case study utilizes eligibility criteria developed by (although endorsement by not implied) the United States Department of Agriculture’s Nutrition Evidence Systematic Review team (NESR) [ 10 , 26 , 27 ], government experts that specialize in conducting food- and nutrition-related systematic reviews. Most recently, NESR implemented a Ti/O approach at first-level screening for 33 original systematic reviews conducted to support the 2020 Dietary Guidelines Advisory Committee, which informed the development of the 2020–2025 Dietary Guidelines for Americans [ 28 , 29 ]. Second, the results of this case study are supported by two systematic reviews that have both been recently peer-reviewed and published [ 10 , 26 ]. Lastly, this case study adds to the limited knowledge base of evaluating the difference in Ti/O compared to Ti + Ab screening approaches when conducting systematic reviews.

In summary, this case study demonstrated that using the conventional Ti + Ab screening approach had better screening performance than the Ti/O approach. If the final systematic review had relied only on Ti/O screening approach, 16 citations may have been erroneously excluded. Conducting an effective systematic review requires researchers to balance both researcher efficiency with a screening approach that maximizes eligible study inclusion to help reduce the risk of evidence selection bias and ensure a comprehensive evidence base. Avoidance of evidence selection bias stemming from missing relevant evidence is important to consider when conclusions of a systematic review are often used by researchers, clinicians, and key stakeholders to inform the development of clinical guidelines and public health recommendations.

Availability of data and materials

Data described in the manuscript will be made available upon request from Dr. Van Elswyk (e-mail: [email protected]).

Abbreviations

Acceptable Macronutrient Distribution Range

United States Department of Agriculture’s Nutrition Evidence Systematic Review team

Population, intervention, comparator, outcome, and study design

Randomized controlled trial

Title-plus-abstract

Wang Z, Asi N, Elraiyah TA, Abu Dabrh AM, Undavalli C, Glasziou P, et al. Dual computer monitors to increase efficiency of conducting systematic reviews. J Clin Epidemiol. 2014;67(12):1353–7.

Article   PubMed   Google Scholar  

Higgins JPT TJ, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated February 2022). Cochrane, 2022. www.training.cochrane.org/handbook . Accessed 2023 Mar 10.

Borah R, Brown AW, Capers PL, Kaiser KA. Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open. 2017;7(2):e012545.

Article   PubMed   PubMed Central   Google Scholar  

Tsertsvadze A, Chen YF, Moher D, Sutcliffe P, McCarthy N. How to conduct systematic reviews more expeditiously? Syst Rev. 2015;4:160.

Dekkers OM, Vandenbroucke JP, Cevallos M, Renehan AG, Altman DG, Egger M. COSMOS-E: Guidance on conducting systematic reviews and meta-analyses of observational studies of etiology. PLoS Med. 2019;16(2):e1002742.

Institute of Medicine. Finding what works in health care: standards for systematic reviews. Eden J, Levit L, Berg A, Morton S, editors. Washington, DC: The National Academies Press; 2011.

Devane D CM, McCarthy B, Casey D. Citation screening in systematic reviews: two approaches, two authors and time taken (SWAR-1 (Study Within A Review 1)). Evidence-Informed Public Health: Opportunities and Challenges Abstracts of the 22nd Cochrane Colloquium; 2014; Hyderabad, India: John Wiley & Sons.

Mateen FJ, Oh J, Tergas AI, Bhayani NH, Kamdar BB. Titles versus titles and abstracts for initial screening of articles for systematic reviews. Clin Epidemiol. 2013;5:89–95.

Rathbone J, Albarqouni L, Bakhit M, Beller E, Byambasuren O, Hoffmann T, et al. Expediting citation screening using PICo-based title-only screening for identifying studies in scoping searches and rapid reviews. Syst Rev. 2017;6(1):233.

Van Elswyk ME, Teo L, Lau CS, Shanahan CJ. Dietary Patterns and the Risk of Sarcopenia: A Systematic Review and Meta-Analysis. Curr Dev Nutr. 2022;6(5):nzac001.

The R Foundation. What is R? https://www.r-project.org/about.html . Accessed 2022 Apr 29.

Wang Z, Nayfeh T, Tetzlaff J, O’Blenis P, Murad MH. Error rates of human reviewers during abstract screening in systematic reviews. PLoS ONE. 2020;15(1):e0227742.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Trevethan R. Sensitivity, specificity, and predictive values: foundations, pliabilities, and pitfalls in research and practice. Front Public Health. 2017;307(5):1–7.

Pilis K, Pilis A, Stec K, Pilis W, Langfort J, Letkiewicz S, et al. Three-year chronic consumption of low-carbohydrate diet impairs exercise performance and has a small unfavorable effect on lipid profile in middle-aged men. Nutrients. 2018;10(1914):1-17.

Mangano KM, Sahni S, Kiel DP, Tucker KL, Dufour AB, Hannan MT. Dietary protein is associated with musculoskeletal health independently of dietary pattern: The Framingham Third Generation Study. Am J Clin Nutr. 2017;105(3):714–22.

Drucker AM, Fleming P, Chan AW. Research Techniques Made Simple: Assessing Risk of Bias in Systematic Reviews. J Invest Dermatol. 2016;136(11):e109–14.

Article   CAS   PubMed   Google Scholar  

Zhu J, Xiang YB, Cai H, Li H, Gao YT, Zheng W, et al. A prospective investigation of dietary intake and functional impairments among the elderly. Am J Epidemiol. 2018;187(11):2372–86.

PubMed   PubMed Central   Google Scholar  

Cuschieri S. The CONSORT statement. Saudi J Anaesth. 2019;13(Suppl 1):S27-s30.

History & Development of PRISMA 2023. http://prisma-statement.org/PRISMAStatement/HistoryandDevelopment . Accessed 2023 Aug 15.

Mulla UZ, Cooper R, Mishra GD, Kuh D, Stephen AM. Adult macronutrient intake and physical capability in the MRC National Survey of Health and Development. Age Ageing. 2013;42(1):81–7.

Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, et al. CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c869.

Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097.

Article   Google Scholar  

Hopewell S, Clarke M, Moher D, Wager E, Middleton P, Altman DG, et al. CONSORT for Reporting Randomized Controlled Trials in Journal and Conference Abstracts: Explanation and Elaboration. PLoS Med. 2008;5(1):e20.

Journal of the American Medical Association (JAMA). Instructions for Authors. https://jamanetwork.com/journals/jama/pages/instructions-for-authors#SecTitle . Accessed 2023 Mar 10.

Annals of Internal Medicine. Information for Authors - Preparing Manuscripts for Submission. https://www.acpjournals.org/journal/aim/authors2 . Accessed 2023 Mar 10.

Boushey C, Ard J, Bazzano L, Heymsfield S, Mayer-Davis E, Sabaté J, et al. Dietary Patterns and Sarcopenia: A Systematic Review. July 2020. U.S. Department of Agriculture, Food and Nutrition Service, Center for Nutrition Policy and Promotion, Nutrition Evidence Systematic Review. https://doi.org/10.52570/NESR.DGAC2020.SR0107 . Accessed 2023 Mar 28.

2020 Dietary Guidelines Advisory Committee: Meeting 2. Day 1: July 10, 2019. Dietary Patterns Subcommittee Presentation. [Timestamp 6:29]. https://www.youtube.com/watch?v=iqitbzrmiTM . Accessed 2023 Mar 28.

USDA Nutrition Evidence Systematic Review Branch. 2020 Dietary Guidelines Advisory Committee: Nutrition Evidence Systematic Review (NESR) Process for Conducting Systematic Reviews. U.S. Department of Agriculture, Food and Nutrition Service, Center for Nutrition Policy and Promotion, Nutrition Evidence Systematic Review. https://nesr.usda.gov/sites/default/files/2020-07/NESR%20Systematic%20Review%20Methodology%20for%20the%202020%20Advisory%20Committee_0.pdf . Accessed 2023 Mar 10.

USDA Nutrition Evidence Systematic Review Branch. Systematic Reviews for the 2020 Dietary Guidelines Advisory Committee. https://nesr.usda.gov/2020-dietary-guidelines-advisory-committee-systematic-reviews#systematicreview . Accessed 28 Mar 2023.

Download references

Acknowledgements

We thank Kristy Hancock of W.K. Kellogg Health Sciences Library, Dalhousie University for her assistance in implementing the search strategy for this review.

Funding was provided by the Beef Checkoff.

Author information

Authors and affiliations.

Teo Research Consulting, Portland, ME, USA

Van Elswyk Consulting, Inc, Clark, CO, USA

Mary E. Van Elswyk

National Cattlemen’s Beef Association, a contractor to the Beef Checkoff, 1275 Pennsylvania Avenue NW, Suite 801, Washington, D.C, 20004, USA

Clara S. Lau

Unleash Creatives, LLC, Reynoldsburg, OH, USA

Christopher J. Shanahan

You can also search for this author in PubMed   Google Scholar

Contributions

The authors’ contributions were as follows: L.T. and C.S. analyzed data; L.T., M.V.E., and C.L. wrote and edited the paper. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Clara S. Lau .

Ethics declarations

Ethics approval and consent to participate.

We did not require ethics approval to conduct this study.

Consent for publication

Not applicable.

Competing interests

C.L. is currently employed by the National Cattlemen's Beef Association (NCBA), a contractor to the Beef Checkoff, as the Senior Director of Human Nutrition Research. M.V.E., C.S., and L.T. are consultants and have been paid by NCBA, a contractor to the Beef Checkoff, for work related to this manuscript.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1..

Description of population, intervention, comparator, outcome, and study design (PICOS) criteria for the research question, “What is the relationship between dietary patterns and risk of sarcopenia?”

Additional file 2.

Studies included in final systematic review (Van Elswyk et al., 2022) [ 10 ] as determined by each screening approach, passed at first-level screening.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Teo, L., Van Elswyk, M.E., Lau, C.S. et al. Title-plus-abstract versus title-only first-level screening approach: a case study using a systematic review of dietary patterns and sarcopenia risk to compare screening performance. Syst Rev 12 , 211 (2023). https://doi.org/10.1186/s13643-023-02374-3

Download citation

Received : 28 March 2023

Accepted : 24 October 2023

Published : 13 November 2023

DOI : https://doi.org/10.1186/s13643-023-02374-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Systematic review
• Contingency tables
• Methodology
• Dietary pattern
• Citation screening

Systematic Reviews

ISSN: 2046-4053

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

SYSTEMATIC REVIEW article

Game-based learning in early childhood education: a systematic review and meta-analysis.

• Department of Kindergarten, College of Education, Najran University, Najran, Saudi Arabia

Game-based learning has gained popularity in recent years as a tool for enhancing learning outcomes in children. This approach uses games to teach various subjects and skills, promoting engagement, motivation, and fun. In early childhood education, game-based learning has the potential to promote cognitive, social, and emotional development. This systematic review and meta-analysis aim to summarize the existing literature on the effectiveness of game-based learning in early childhood education This systematic review and meta-analysis examine the effectiveness of game-based learning in early childhood education. The results show that game-based learning has a moderate to large effect on cognitive, social, emotional, motivation, and engagement outcomes. The findings suggest that game-based learning can be a promising tool for early childhood educators to promote children’s learning and development. However, further research is needed to address the remaining gaps in the literature. The study’s findings have implications for educators, policymakers, and game developers who aim to promote positive child development and enhance learning outcomes in early childhood education.

1 Introduction

Game-based learning in early childhood education has evolved over time, driven by advancements in technology, educational research, and changing pedagogical approaches. Digital game-based learning refers to the use of digital technology, such as computers or mobile devices, to deliver educational content through interactive games ( Behnamnia et al., 2020 ). Game-based learning, on the other hand, is a broader term that encompasses both digital and non-digital games as tools for educational purposes. In the early years, educational games were primarily non-digital, consisting of board games, puzzles, and manipulatives designed to teach basic concepts and skills ( Pivec, 2007 ). These games often focused on early literacy, numeracy, and problem-solving. With the advent of computers and educational software, digital games emerged as a new medium for learning in the late 20th century. Early educational computer games, such as “Reader Rabbit” and “Math Blaster,” aimed to engage young learners through interactive gameplay while reinforcing educational content. As technology continued to advance, game-based learning expanded beyond standalone software to web-based platforms, mobile apps, and immersive virtual environments ( Shamir et al., 2019 ). The introduction of touchscreen devices, such as tablets and smartphones, made educational games more accessible and interactive for young children. These advancements allowed for greater customization, adaptive learning experiences, and real-time feedback, tailoring the games to meet the individual needs and abilities of young learners.

Researchers and educators recognized the potential of game-based learning to enhance engagement, motivation, and learning outcomes in early childhood education. Studies began to explore the cognitive, social, emotional, and behavioral effects of game-based learning, highlighting its effectiveness in promoting critical thinking, problem-solving, collaboration, creativity, and digital literacy skills ( Park and Park, 2021 ).

In early childhood education, online educational game-based learning has gained popularity as a tool to promote cognitive, social, and emotional development in young children ( Anastasiadis et al., 2018 ). Online educational games are interactive digital games specifically designed to educate and teach children a wide range of skills and concepts. These games utilize engaging and interactive elements to promote learning in areas such as literacy, numeracy, problem-solving, and critical thinking ( Papanastasiou et al., 2022 ). These games are typically played on digital devices such as computers, tablets, and smartphones, and they offer a variety of engaging and interactive learning experiences for young children. Young children are naturally curious and have a strong desire to explore and learn about their environment ( Gurholt and Sanderud, 2016 ). Online educational game-based learning taps into this natural curiosity and provides children with opportunities to engage in meaningful and engaging learning experiences. These games can be tailored to meet the unique needs and abilities of young children, and they can be adapted to suit different learning styles and preferences ( Qian and Clark, 2016 ).

One of the key benefits of online educational game-based learning in early childhood education is its ability to promote cognitive development ( Ferreira et al., 2016 ). Online games can help children develop their problem-solving skills, memory, attention, and processing speed. For example, puzzle games can help children develop their spatial reasoning and problem-solving skills, while memory games can help them improve their memory and concentration ( Suhana, 2017 ).

In addition to promoting cognitive development, online educational game-based learning can also enhance social development in young children. Online games provide children with opportunities to interact with their peers and develop important social skills such as cooperation, communication, and empathy. Children can learn to work together, take turns, and share resources, which are essential skills for building positive relationships and succeeding in life ( Lamrani and Abdelwahed, 2020 ).

Moreover, online educational game-based learning can promote emotional development in young children ( Peterson et al., 2016 ). Online games can help children develop their emotional regulation skills, self-awareness, and self-confidence ( Simion and Bănuț, 2020 ). Games that involve role-playing can help children develop their emotional intelligence and understand different perspectives, while games that require children to take risks and try new things can help them build resilience and confidence ( Huynh et al., 2020 ).

This distinction is further exemplified in studies using online educational game-based learning in early childhood education for is its ability to increase children’s motivation and engagement in learning ( Hwa, 2018 ). Traditional teaching methods can sometimes be dry and one-dimensional, leading to disengagement and boredom in children ( Fotaris et al., 2016 ). Online educational games, on the other hand, provide a fun and interactive way to learn, which can increase children’s motivation and engagement in learning ( Nieto-Escamez and Roldán-Tapia, 2021 ). Children are more likely to be engaged in learning when they are having fun and enjoying the process ( Iten and Petko, 2016 ). Furthermore, online educational game-based learning can be tailored to meet the individual needs and abilities of young children ( Ke, 2014 ). Online games can be adapted to suit different learning styles and preferences, ensuring that all children can benefit from this approach to learning. This is certainly true in the case of games that involve movement and physical activity can be used to promote learning in children who have a kinesthetic learning style, while games that involve visual aids can be used to promote learning in children who have a visual learning style ( Hayati et al., 2017 ).

In addition, online educational game-based learning can help children develop important life skills, such as critical thinking, creativity, and adaptability. Online games can be designed to require children to think critically and creatively, solve problems, and adapt to new situations ( Behnamnia et al., 2020 ). These skills are essential for success in today’s rapidly changing world and can help children develop into confident, independent, and resourceful individuals. Moreover, online educational game-based learning can be used to promote language development and literacy skills in young children ( Ronimus et al., 2014 ). Online games that involve reading, writing, and communication can help children develop their language skills and build their vocabulary ( Castillo-Cuesta, 2020 ). Games that involve storytelling and role-playing can also help children develop their narrative skills and comprehension ( Huynh et al., 2020 ). Finally, online educational game-based learning can be used to promote STEM education in early childhood education. Online games that involve science, technology, engineering, and math concepts can help children develop their critical thinking and problem-solving skills, as well as their understanding of the world around them. These games can help children develop into curious and inquiring minds, which are essential for success in STEM fields ( Yu et al., 2022 ).

Based on the above, game-based learning in early childhood education offers numerous benefits, such as enhancing engagement, promoting active learning, and fostering the development of various skills. However, it is essential to acknowledge and address potential drawbacks or challenges associated with this approach to ensure its effective implementation. One notable challenge is the need for careful game selection. Not all educational games are created equally, and some may lack appropriate content, fail to align with specific learning objectives, or not adequately support the developmental needs of young learners ( Domoff et al., 2019 ). It is crucial to critically evaluate the quality, educational value, and appropriateness of games before incorporating them into early childhood education settings ( Derevensky et al., 2019 ). Another challenge is the limited generalizability of skills acquired through games. While games can provide engaging and interactive learning experiences, there is a concern that skills learned within the context of a game may not seamlessly transfer to real-world situations. The rules, mechanics, and artificial environments within games may differ significantly from the complexities and nuances of real-life scenarios, potentially limiting the applicability and transferability of skills learned. It is important for educators to provide explicit connections and opportunities for children to apply their game-based learning experiences to real-life contexts ( All et al., 2021 ).

Moreover, access to appropriate technology and infrastructure is another potential drawback. Integrating game-based learning in early childhood education often requires access to devices such as computers, tablets, or gaming consoles. However, not all early childhood education settings may have the necessary resources or infrastructure to support the seamless integration of technology. Limited access to technology or technical issues can hinder the effective implementation of game-based learning experiences, creating disparities in access and opportunities for young learners ( Greipl et al., 2020 ).

Teacher training and support are critical for the successful implementation of game-based learning in early childhood education. Educators need to be equipped with the necessary knowledge, skills, and pedagogical approaches to effectively integrate games into the curriculum and facilitate meaningful learning experiences. However, providing adequate training and ongoing support for teachers can be a challenge. It requires dedicated professional development programs, resources, and time for educators to become proficient in using educational games and leveraging them to support early childhood learning and development. Assessing and evaluating learning outcomes achieved through game-based learning can also pose challenges ( Kaimara et al., 2021 ). Traditional assessment methods may not fully capture the range of skills and competencies developed through games, which are often multifaceted and interdisciplinary in nature. Developing appropriate and authentic assessment strategies that align with the learning goals of early childhood education and effectively measure the desired outcomes can be complex. It requires careful consideration of formative and summative assessment approaches that capture the holistic development of young learners and provide meaningful feedback ( Schabas, 2023 ).

Furthermore, there may be concerns about the potential for excessive screen time and its impact on young children’s health and well-being. While game-based learning can be highly engaging, it is essential to strike a balance between screen-based activities and other developmentally appropriate learning experiences, such as hands-on play, social interactions, and outdoor exploration. Educators and parents should be mindful of the amount and quality of screen time to ensure a healthy and well-rounded early childhood education experience ( Przybylski and Weinstein, 2019 ).

Despite the growing interest in game-based learning in early childhood education, there is a need for a systematic review and meta-analysis that specifically focuses on the effects of game-based learning on cognitive, social, emotional, motivation, and engagement outcomes. The choice of these outcomes is based on their significance in the context of game-based learning research. Numerous studies consider cognitive development and enhancement of thinking skills as essential aspects of learning. Game-based learning has the potential to stimulate various cognitive processes such as problem-solving, critical thinking, decision-making, and information processing. Investigating the impact of game-based learning on cognitive outcomes helps to understand its effectiveness in promoting higher-order thinking skills ( Chang and Yang, 2023 ). Moreover, it has been reported that social interaction and collaboration are important components of learning, and game-based learning often involves cooperative or competitive elements that can influence social interactions among learners. Exploring the impact of game-based learning on social outcomes can shed light on how it affects teamwork, communication, and social skills development ( Sun et al., 2022 ). Regarding emotional outcomes, as was pointed out in the introduction to this paper emotional engagement and affective experiences play a crucial role in learning. Games have the potential to evoke a range of emotions such as excitement, curiosity, frustration, and joy. Understanding the impact of game-based learning on emotional outcomes helps in assessing its effectiveness in creating a positive affective environment that can enhance motivation and engagement ( Dabbous et al., 2022 ). Recent research has suggested that examining the impact of game-based learning on motivational outcomes can explore aspects such as intrinsic motivation, self-efficacy, persistence, and enjoyment, which are crucial for effective learning experiences especially for kids in kindergarten ( Yu and Tsuei, 2022 ). Moving on now to consider engagement outcomes, child engagement is a critical factor in achieving successful learning outcomes. Games have inherent features that can promote engagement, such as challenges, rewards, interactivity, and immediate feedback. Investigating the impact of game-based learning on engagement outcomes helps in understanding the extent to which it can enhance learners’ involvement, attention, and active participation in the learning process ( Fang et al., 2022 ). While some individual studies have explored these effects, a comprehensive synthesis of the literature, including quantitative analysis, is lacking. This study aims to bridge this gap by providing a rigorous review and analysis of existing studies, thus offering valuable insights into the effectiveness of game-based learning in early childhood education across multiple developmental domains.

This systematic review and meta-analysis aim to summarize the existing literature on the effectiveness of online game-based learning in early childhood education. Specifically, we will examine the impact of game-based learning on children’s cognitive, social, and emotional development, as well as their motivation and engagement in learning. The primary objective of this study is to investigate the effect of game-based learning on cognitive, social, emotional, motivation, and engagement outcomes in early childhood education. Specifically, the study aims to answer the following questions:

1. What is the effect of game-based learning on cognitive development in early childhood education?

2. What is the effect of game-based learning on social development in early childhood education?

3. What is the effect of game-based learning on emotional development in early childhood education?

4. What is the effect of game-based learning on motivation in early childhood education?

5. What is the effect of game-based learning on engagement in early childhood education?

2 Materials and methods

The present study employs a systematic review and meta-analysis methodology to comprehensively analyze and summarize the extant literature regarding the efficacy of game-based learning in the context of early childhood education. Specifically, the study aims to investigate the effects of game-based learning on various facets of children’s development, including cognitive, social, and emotional domains, as well as their motivation and engagement levels in the learning process.

Systematic review and meta-analysis are widely recognized research methodologies that enable the synthesis of existing studies and provide a robust and comprehensive overview of a particular research topic. By systematically searching, selecting, and critically evaluating relevant empirical studies, the researchers ensure the inclusion of high-quality evidence in the analysis. Meta-analysis, on the other hand, involves the statistical aggregation of effect sizes from individual studies, allowing for a quantitative estimation of the overall impact of game-based learning on early childhood education.

2.1 A systematic review

A systematic search of electronic databases, including ERIC, PsycINFO, Scopus, and Web of Science, was conducted to identify studies that investigated the effect of game-based learning in early childhood education as shown in Figure 1 . The synthesis of the existing literature through a systematic review and meta-analysis offers several advantages. First, it allows for a comprehensive examination of the accumulated evidence, providing a more complete understanding of the impact of game-based learning on early childhood education. Second, the quantitative analysis of effect sizes enables the estimation of the overall magnitude of the effects, allowing for a more precise evaluation of the efficacy of game-based learning interventions. Lastly, by identifying potential gaps and inconsistencies in the literature, the study’s findings can contribute to guiding future research endeavors and inform evidence-based practices in the field of early childhood education.

Figure 1 . Systematic review process ( Robson et al., 2019 ).

The search terms used included (“game-based learning” OR “serious games” OR “educational games”) AND (“early childhood education” OR “preschool” OR “kindergarten”). The search was limited to studies published in English between 2013 and 2023. Studies that met the following criteria were included in the review:

• Focused on children aged 3–8 years old.

• Included a control group or baseline measure.

• Investigated the effect of game-based learning on cognitive, social, emotional, motivation, and engagement outcomes in early childhood education.

• Published in English.

• Used a quantitative study design (experimental or quasi-experimental).

Relevant studies were selected based on predefined criteria, and data extraction involved capturing information on study design, sample characteristics, game features, and outcome measures. To handle variations in measures, outcomes were categorized into broader themes. Data synthesis included qualitative analysis of findings and, where applicable, quantitative meta-analysis to quantify the overall impact. Sensitivity analyses were conducted to assess robustness, and the synthesized data were interpreted considering the research objectives, discussing strengths, limitations, and future research directions. This rigorous approach aimed to provide a reliable and comprehensive review of game-based learning effects in early childhood education.

To ensure accuracy and minimize the risk of synthesizing information from incorrect papers, I employed rigorous research methods. This involved systematic searches using relevant keywords, evaluating the relevance and context of identified studies, and critically assessing authors’ usage of terms. Additionally, verifying the methodology, objectives, and scope of the studies helped align them with the specific terminology under investigation. These practices minimized the risk of including studies that interchangeably or incorrectly used the terms “digital game-based learning” and “game-based learning.” Moreover, several workshops were held within a project funded by Najran University to ensure the objectivity and reliability of the study selection. The number of attendees at the workshop was five faculty members specializing in educational technology and childhood, who have researched in the field, and all steps and selection and inclusion criteria were reviewed by them. Data was extracted from each study using a standardized form. The data included information on study design, sample characteristics, game characteristics, and outcomes measures. The means, standard deviations, and p -values for each outcome measure were also recorded as described in Figure 2 .

Figure 2 . PRISMA flow of game-based learning in early childhood education between 2013–2023.

2.2 A meta-analytic approach

A meta-analytic approach was used to synthesize the data. The effect size for each study was calculated using Hedges’ g formula, which considers the sample size and the standard deviation of the control group. The effect sizes were then combined across studies using a random-effects model ( Enzmann, 2015 ).

2.3 Sensitivity analyses and risk of bias assessment

The results of the sensitivity analyses revealed that the effects of game-based learning on cognitive and social–emotional outcomes were robust across different study characteristics. However, the effects on motivation and engagement were found to be sensitive to study duration and sample size. Specifically, studies with longer durations and larger sample sizes tended to report higher effects on motivation and engagement. Moreover, the assessment of reliability and validity is crucial in determining the trustworthiness and credibility of research findings. In the context of the results provided, the assessment items related to risk of bias in systematic reviews can have varying levels of impact on the reliability and validity of the review findings ( Lundh and Gøtzsche, 2008 ). For this regard, the revised Cochrane risk of bias tool for randomized trials (RoB 2) was used for studies reviewed ( n  = 136). Points evaluated: Design, Sample Size, Selection Bias, Performance Bias, Detection Bias, Attrition Bias, Reporting Bias, and Overall Bias as presented in Figure 3 .

Figure 3 . Summary of risk of bias assessment for studies reviewed ( n  = 136). Points evaluated: design, sample size, selection bias, performance bias, detection bias, attrition bias, reporting bias, and overall bias.

Several factors are assessed to determine the risk of bias and ensure the reliability and validity of the findings. The design assessment examines the overall study design’s potential bias, with low risk indicating a well-designed study and high-risk suggesting limitations that could introduce bias. Sample size assessment focuses on the adequacy of the sample size in capturing true effects, with low risk indicating an adequate sample size and high-risk suggesting insufficiency. Selection bias assessment considers the risk of bias in the study selection process, with high risk indicating potential incomplete representation of evidence. Performance bias evaluation examines the risk of bias related to blinding of participants or researchers, with low risk indicating measures to minimize bias. Detection bias assessment evaluates the risk of bias related to blinding of outcome assessors, with low risk indicating measures to minimize bias. Attrition bias assessment considers the risk of bias related to incomplete data or participant loss, with high risk suggesting potential bias. Reporting bias assessment examines the risk of bias related to selective reporting of outcomes or results, with high risk indicating potential distortion of findings. Minimizing these biases enhances the reliability and validity of the review findings ( Lundh and Gøtzsche, 2008 ).

3 Results and discussions

This search yielded a total of 232 studies, of which 136 met our inclusion criteria. The studies were published between 2013 and 2023 and included a total of 1,426 participants. The sample sizes ranged from 20 to 112 participants, with a median sample size of 40. Ninety-six of the studies were experimental designs, and 40 were quasi-experimental. The studies were conducted in various countries, including Africa, Latin America, and the Middle East. In addition to North America, i.e., United States, Canada. Followed by Australia, and the United Kingdom as presented in Figure 4 .

Figure 4 . Distributed studies based on locations.

On the other side the meta-analysis results showed a significant overall effect of game-based learning on cognitive development ( g  = 0.46, p  < 0.001), social development ( g  = 0.38, p  < 0.001), emotional development ( g  = 0.35, p  < 0.001), motivation ( g  = 0.40, p  < 0.001), and engagement ( g  = 0.44, p  < 0.001). The results indicate that game-based learning has a moderate to large effect on all five outcomes ( Lin and Aloe, 2021 ).

3.1 Moderator analysis

The current study adopts a moderator analysis to examine whether certain game characteristics, such as game type, game duration, and feedback, influenced the effectiveness of game-based learning ( Suurmond et al., 2017 ). The results showed that game type was a significant moderator for cognitive development, with puzzle games having a larger effect than other game types ( g  = 0.63 vs. g  = 0.31). Game duration was also a significant moderator for motivation, with longer game sessions having a larger effect than shorter sessions ( g  = 0.50 vs. g  = 0.26). Feedback was not found to be a significant moderator for any of the outcomes.

4 Discussion

Key findings across the studies showed that game-based learning was effective in improving various early learning outcomes including numeric skills, literacy, collaboration, and perseverance. Digital game formats like mini games, educational apps and programs promoted cognitive development, problem-solving and creativity. Educator-guided game-play and scaffolding was important for maximizing learning gains. Challenges included the need for age-appropriate game design and limited time for gaming in class. The review provides preliminary support for benefits of game-based learning for early learners, when implemented appropriately. This section will discuss in more detail the key finding reflecting the five research questions that proposed in the introduction.

4.1 Cognitive development

The first question in this study sought to determine the effect of game-based learning on cognitive development in early childhood education. Numerous studies have been conducted in this line of research. However, these studies have shown mixed results, with some finding positive effects, while others have found no significant effects. Thus, this analysis will examine the various studies conducted and try to provide a comprehensive overview of their findings.

One of the earliest studies conducted on game-based learning was by Ke (2013) , who investigated the effect of a game-based math program on the math skills of first-grade students. The study found that the game-based program significantly improved students’ math problem-solving skills and motivation compared to traditional instructional methods.

Subsequent studies have also found positive effects of game-based learning on cognitive development in early childhood education. For example, a study by Lin et al. (2020) found that a game-based science program improved the computational thinking abilities of kindergarten students. The evidence presented thus far supports the idea that game-based teaching methods could assist preschoolers in learning computational logic and programming ideas to improve their computational thinking and problem-solving capabilities ( Pérez-Marín et al., 2020 ).

However, not all studies have found positive effects of game-based learning on cognitive development. This is certainly true in the case study by Brezovszky et al. (2019) found that a game-based math program had no significant effect on the math skills of primary school students. Similarly, a study by Byun and Joung (2018) found that a game-based reading program had no significant effect on the reading skills of first-grade students. Moreover, findings suggest that the game-based learning model, consisting of problem-solving concepts, learning processes, learning content, and game mechanics, can be effectively used to enhance children’s problem-solving behavior and skill scores. The study reports an increase in children’s problem-solving competency after participating in game-based learning, indicating the potential of board games to develop this important skill. Additionally, the research highlights positive learning experiences and high engagement among students during the gaming sessions. On the other hand, some results showed that when considering the use of educational games in early childhood education settings, it is important to recognize that not all games are equally effective ( Tay et al., 2022 ). Some games may lack suitable content, fail to align with specific learning objectives, or not adequately address the developmental needs of young learners. Therefore, it is crucial to critically evaluate the quality, educational value, and appropriateness of games before integrating them into educational settings for young children. Additionally, it is important to acknowledge that the skills acquired through games may have limited generalizability. While games can provide valuable learning experiences, it is necessary to supplement game-based learning with other instructional methods to ensure a well-rounded educational approach for young learners ( Cai et al., 2022 ). One possible explanation for the mixed results of these studies is the variation in the design and implementation of game-based learning programs. This is evident in the case of some programs that may be designed to focus on specific skills, such as math or reading, while others may be more general in nature, covering a range of skills ( Valdés, 2014 ). Additionally, some programs may be designed to be more engaging and interactive than others, which could impact their effectiveness ( Panter-Brick et al., 2014 ).

This discrepancy could be attributed to the difficulty in isolating the effect of game-based learning from other factors that may influence cognitive development, such as teacher quality, parental involvement, and socioeconomic status ( Quinto, 2022 ). Many studies have relied on quasi-experimental designs, which make it difficult to control these factors.

Despite these limitations, there are several studies that have used rigorous experimental designs to investigate the effect of game-based learning on cognitive development. For example, a study by Di Tore et al. (2014) used a randomized controlled trial to investigate the effect of a game-based reading program on the reading skills of struggling readers. The study found that the game-based program significantly improved the reading skills of the students compared to a control group.

A similar study by Thai et al. (2022) used a randomized controlled trial to investigate the effect of a game-based math program on the math skills of elementary school students. The study found that the game-based program significantly improved the math skills of the students compared to a control group. Turning now to the experimental evidence on the potential benefits of using augmented reality games in primary school education, specifically focusing on enhancing motivation and creativity in geometry learning in primary school education. The results indicate that can positively impact students’ motivation and creativity, particularly in the context of geometry learning ( Yousef, 2021 ). Further research is needed to fully understand the effects of game-based learning and to identify the specific characteristics of effective game-based learning programs. Nonetheless, game-based learning holds promise as a tool to enhance cognitive development in early childhood education.

4.2 Social development

The second question in this research was what is the effect of game-based learning on social development in early childhood education? Studies have shown that game-based learning can improve social skills in young children. A study conducted by Craig et al. (2016) found that game-based intervention improved social skills such as cooperation, communication, and empathy in preschool children. Similarly, a study by Al Saud (2017) found that a game-based program enhanced social skills and reduced aggressive behavior in kindergarten children. Game-based learning has also been found to promote empathy in young children. A study by Mukund et al. (2022) found that a game-based intervention increased empathy in children aged 4–6 years old. Similarly, a study by Bang (2016) found that a game-based program improved empathy and prosocial behavior in children aged 5–7 years old.

Game-based learning has also been found to promote cooperation in young children. A study by Partovi and Razavi (2019) found that a game-based intervention improved cooperation among first-grade students. Similarly, a study by Craig et al. (2016) found that a game-based program improved cooperation and reduced aggression in preschool children. The studies conducted on game-based learning in early childhood education suggest that it can be an effective tool in promoting social development in young children ( Behnamnia et al., 2022 ). Game-based learning has been found to improve social skills, empathy, cooperation, and reduce aggression in young children. Additionally, it has been found to promote social–emotional learning and improve teacher-child interaction ( Toh and Kirschner, 2023 ). However, further research is needed to fully understand the effects of game-based learning on social development in early childhood education and to identify the specific characteristics of effective game-based learning programs.

4.3 Emotional development

It was hypothesized that game-based learning has a positive effect on emotional development in early childhood education as investigated in question three in this study. Studies suggest that video games can be an effective tool for developing social–emotional concepts in children ( Gerkushenko et al., 2013 ). Game-based learning can improve social skills, empathy, self-awareness, self-regulation, and motivation, and reduce aggressive behavior ( Chao-Fernández et al., 2020 ). Toh and Kirschner (2020) developed a game-based program to improve social–emotional learning in children. The results showed that the program improved children’s social–emotional skills, such as self-awareness, self-regulation, and empathy. Hausknecht et al. (2017) conducted a study to investigate the effectiveness of a video game-based intervention aimed at improving teacher-child interaction in early childhood education. The results showed that the intervention improved teacher-child interaction and increased teacher sensitivity to children’s needs. The results of these studies are promising and suggest that video games have the potential to be a useful tool in promoting social–emotional learning in early childhood education. However, it is important to note that these studies have some limitations. Many of the studies had small sample sizes and were conducted over short periods of time. Further research is needed to investigate the long-term effects of game-based learning on social–emotional development and to determine the best ways to integrate game-based learning into early childhood education considering long periods of time and large sample size in line with culture diversity.

4.4 Motivation development

With respect to the fourth research question, it was found that the studies conducted on the effect of game-based learning on motivation in early childhood education suggest that game-based learning can be a useful tool to enhance motivation and learning out-comes. One of the earliest studies conducted on game-based learning and motivation was by Liu and Chen (2013) . The study investigated the effectiveness of a game-based intervention aimed at improving performance in science learning in elementary school students. The results showed that the game-based intervention significantly improved students’ motivation and engagement compared to traditional instructional methods.

Ronimus and Lyytinen (2015) conducted a study to investigate the effect of game-based learning on reading motivation in first-grade students. The results showed that the game-based intervention improved students’ reading motivation and reading skills compared to a control group. Similar to this, a study by Brennan et al. (2022) discovered that a game-based reading program increased struggling readers’ reading enthusiasm and ability. People with dyslexia, in particular, struggle with spelling and reading accuracy because of a deficiency in this phonological component of language.

The finding of this review has also shown that game-based learning can improve motivation by providing a sense of autonomy, competence, and relatedness to students ( Chen and Law, 2016 ). Eseryel et al. (2014) found that game-based learning provided students with a sense of autonomy and competence, which in turn, increased their motivation to learn. Similarly, a study by Anastasiadis et al. (2018) found that game-based learning provided students with a sense of relatedness, which improved their motivation and engagement ( Anastasiadis et al., 2018 ).

Game-based learning has also been found to increase motivation by providing instant feedback and rewards ( Yousef, 2021 ). A study by Hung et al. (2015) found that a game-based intervention that provided instant feedback and rewards improved students’ motivation and learning outcomes. Similarly, a study by Zabala-Vargas et al. (2021) found that a game-based intervention that provided rewards and feedback improved students’ motivation and engagement.

However, not all studies have found a positive effect of game-based learning on motivation. A study by Xu et al. (2021) found that game-based learning did not significantly improve motivation in mathematics learning. A systematic review by Hussein et al. (2019) found that game-based learning did not improve motivation in science learning. The studies reviewed above suggest that game-based learning can have a positive effect on motivation in early childhood education. Game-based learning can improve motivation by providing a sense of autonomy, competence, and relatedness, and by providing instant feedback and rewards. However, it is important to note that the effectiveness of game-based learning on motivation may depend on various factors, such as the type of game, the student’s prior knowledge and skills, and the learning objectives.

4.5 Engagement development

Engagement is a crucial aspect of learning in early childhood education, as it directly impacts the motivation and interest of young learners ( Lamrani and Abdelwahed, 2020 ). Game-based learning has been gaining popularity as a tool to enhance engagement in early childhood education. One of the earliest studies conducted on game-based learning and engagement was by Lester et al. (2013) . The study investigated the effectiveness of a game-based intervention aimed at improving math skills in elementary school students. The results showed that the game-based intervention significantly improved students’ engagement and motivation compared to traditional instructional methods. Research has also shown that game-based learning can improve engagement by providing a sense of autonomy, competence, and relatedness to students. Mekler et al. (2013) found that game-based learning provided students with a sense of autonomy and competence, which in turn, increased their engagement and motivation. Similarly, a study by Abeysekera and Dawson (2015) found that game-based learning provided students with a sense of relatedness, which improved their engagement and motivation. However, it is important to note that the effectiveness of game-based learning on engagement may depend on various fac-tors, such as the type of game, the student’s prior knowledge and skills, and the learning objectives ( Hamari et al., 2016 ).

5 Conclusion

In early childhood education, game-based learning has the potential to promote cognitive, social, and emotional development. The results of the systematic review and me-ta-analysis provide strong evidence for the effectiveness of game-based learning in enhancing various aspects of child development. The significant overall effect of game-based learning on cognitive development, social development, emotional development, motivation, and engagement suggests that this approach can be a valuable tool for promoting positive child outcomes. The effect size for cognitive development ( g  = 0.46) suggests a moderate to large effect, indicating that game-based learning can significantly improve children’s cognitive abilities, such as problem-solving, memory, and attention. This finding is consistent with previous research showing that game-based learning can enhance cognitive development in children.

The effect size for social development ( g  = 0.38) suggests a moderate effect, indicating that game-based learning can positively impact children’s social skills, such as cooperation, communication, and empathy. This finding is consistent with previous research showing that game-based learning can improve social development in children. The effect size for emotional development ( g  = 0.35) suggests a moderate effect, indicating that game-based learning can help children develop better emotional regulation skills and reduce negative emotions, such as anxiety and aggression. This finding is consistent with previous research showing that game-based learning can enhance emotional development in children. The effect size for motivation ( g  = 0.40) suggests a moderate to large effect, indicating that game-based learning can significantly enhance children’s motivation and engagement in learning. The effect size for engagement ( g  = 0.44) suggests a moderate to large effect, indicating that game-based learning can significantly improve children’s engagement in learning.

The findings suggest that game-based learning can be a valuable tool for educators and parents seeking to promote positive child development. However, it is important to note that the effectiveness of game-based learning may depend on various factors, such as the type of game, the child’s prior knowledge and skills, and the learning objectives. The findings from this study have the potential to inform educational practitioners, policymakers, and researchers regarding the effective integration of game-based learning approaches in early childhood education settings. Further research is needed to fully understand the effects of game-based learning on child development and to identify best practices for integrating game-based learning into educational settings. Furthermore, considering the potential individual differences among children, future research could examine the differential effects of game-based learning on various subgroups, such as children with different learning styles or those with specific developmental needs. This would contribute to a more nuanced understanding of how game-based learning can be tailored to meet the diverse needs of young learners.

Data availability statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

Author contributions

MA: Writing – review & editing, Writing – original draft, Visualization, Validation, Resources, Project administration, Methodology, Funding acquisition, Formal analysis, Data curation, Conceptualization.

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. Deanship of Scientific Research at Najran University for funding this work under the General Research Funding program grant code (NU/DRP/SEHRC/12/5).

Acknowledgments

The author is thankful to the Deanship of Scientific Research at Najran University for funding this work under the General Research Funding program grant code (NU/DRP/SEHRC/12/5).

Conflict of interest

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Abeysekera, L., and Dawson, P. (2015). Motivation and cognitive load in the flipped classroom: definition, rationale and a call for research. Higher Educ. Res. Develop. 34, 1–14. doi: 10.1080/07294360.2014.934336

Crossref Full Text | Google Scholar

Al Saud, A. F. (2017). Educational video games enrich underprivileged children’s social skills in Saudi Arabia. Eur. J. Educ. Sci. 4, 32–47. doi: 10.19044/ejes.v4no2a3

All, A., Castellar, E. N. P., and Van Looy, J. (2021). Digital game-based learning effectiveness assessment: reflections on study design. Comput. Educ. 167:104160. doi: 10.1016/j.compedu.2021.104160

Anastasiadis, T., Lampropoulos, G., and Siakas, K. (2018). Digital game-based learning and serious games in education. Int. J. Adv. Sci. Res. Engin. 4, 139–144. doi: 10.31695/IJASRE.2018.33016

Bang, A. H. (2016). The restorative and transformative power of the arts in conflict resolution. J. Transform. Educ. 14, 355–376. doi: 10.1177/1541344616655886

Behnamnia, N., Kamsin, A., and Ismail, M. A. B. (2020). The landscape of research on the use of digital game-based learning apps to nurture creativity among young children: a review. Think. Skills Creat. 37:100666. doi: 10.1016/j.tsc.2020.100666

Behnamnia, N., Kamsin, A., Ismail, M. A. B., and Hayati, S. A. (2022). A review of using digital game-based learning for preschoolers. J. Comput. Educ. 10, 603–636. doi: 10.1007/s40692-022-00240-0

Brennan, A., McDonagh, T., Dempsey, M., and McAvoy, J. (2022). Cosmic sounds: a game to support phonological awareness skills for children with dyslexia. IEEE Trans. Learn. Technol. 15, 301–310. doi: 10.1109/TLT.2022.3170231

Brezovszky, B., McMullen, J., Veermans, K., Hannula-Sormunen, M. M., Rodríguez-Aflecht, G., Pongsakdi, N., et al. (2019). Effects of a mathematics game-based learning environment on primary school students' adaptive number knowledge. Comput. Educ. 128, 63–74. doi: 10.1016/j.compedu.2018.09.011

Byun, J., and Joung, E. (2018). Digital game-based learning for K–12 mathematics education: a meta-analysis. Sch. Sci. Math. 118, 113–126. doi: 10.1111/ssm.12271

Cai, Z., Mao, P., Wang, D., He, J., Chen, X., and Fan, X. (2022). Effects of scaffolding in digital game-based learning on student’s achievement: a three-level meta-analysis. Educ. Psychol. Rev. 34, 537–574. doi: 10.1007/s10648-021-09655-0

Castillo-Cuesta, L. (2020). Using digital games for enhancing EFL grammar and vocabulary in higher education. Int. J. Emerg. Technol. Learn. 15, 116–129. doi: 10.3991/ijet.v15i20.16159

Chang, C. C., and Yang, S. T. (2023). Learners’ positive and negative emotion, various cognitive processing, and cognitive effectiveness and efficiency in situated task-centered digital game-based learning with different scaffolds. Interact. Learn. Environ. , 1–20. doi: 10.1080/10494820.2023.2209600

Chao-Fernández, R., Gisbert-Caudeli, V., and Vázquez-Sánchez, R. (2020). Emotional training and modification of disruptive behaviors through computer-game-based music therapy in secondary education. Appl. Sci. 10:1796. doi: 10.3390/app10051796

Chen, C. H., and Law, V. (2016). Scaffolding individual and collaborative game-based learning in learning performance and intrinsic motivation. Comput. Hum. Behav. 55, 1201–1212. doi: 10.1016/j.chb.2015.03.010

Craig, A. B., Brown, E. R., Upright, J., and DeRosier, M. E. (2016). Enhancing children’s social emotional functioning through virtual game-based delivery of social skills training. J. Child Fam. Stud. 25, 959–968. doi: 10.1007/s10826-015-0274-8

Dabbous, M., Kawtharani, A., Fahs, I., Hallal, Z., Shouman, D., Akel, M., et al. (2022). The role of game-based learning in experiential education: tool validation, motivation assessment, and outcomes evaluation among a sample of pharmacy students. Educ. Sci. 12:434. doi: 10.3390/educsci12070434

Derevensky, J. L., Hayman, V., and Gilbeau, L. (2019). Behavioral addictions: excessive gambling, gaming, internet, and smartphone use among children and adolescents. Pediatr. Clin. 66, 1163–1182. doi: 10.1016/j.pcl.2019.08.008

PubMed Abstract | Crossref Full Text | Google Scholar

Di Tore, P. A., Di Tore, S., Ludovico, L. A., and Mangione, G. R. (2014). Madrigale: a multimedia application for dyslexia and reading improvement gamifying learning experience. In 2014 International Conference on Intelligent Networking and Collaborative Systems 486–491. IEEE

Google Scholar

Domoff, S. E., Harrison, K., Gearhardt, A. N., Gentile, D. A., Lumeng, J. C., and Miller, A. L. (2019). Development and validation of the problematic media use measure: a parent report measure of screen media “addiction” in children. Psychol. Pop. Media Cult. 8, 2–11. doi: 10.1037/ppm0000163

Enzmann, D. (2015). Notes on effect size measures for the difference of means from two independent groups: the case of Cohen’s d and Hedges’g. Tech. Rep . 12. doi: 10.13140/2.1.1578.2725

Eseryel, D., Law, V., Ifenthaler, D., Ge, X., and Miller, R. (2014). An investigation of the interrelationships between motivation, engagement, and complex problem solving in game-based learning. J. Educ. Technol. Soc. 17, 42–53.

Fang, M., Tapalova, O., Zhiyenbayeva, N., and Kozlovskaya, S. (2022). Impact of digital game-based learning on the social competence and behavior of preschoolers. Educ. Inf. Technol. 27, 3065–3078. doi: 10.1007/s10639-021-10737-3

Ferreira, S. M., Gouin-Vallerand, C., and Hotte, R. (2016). Game based learning: a case study on designing an educational game for children in developing countries. In 2016 8th international conference on games and virtual worlds for serious applications (VS-GAMES) 1–8. IEEE

Fotaris, P., Mastoras, T., Leinfellner, R., and Rosunally, Y. (2016). Climbing up the leaderboard: an empirical study of applying gamification techniques to a computer programming class. Electron. J. E Learn. 14, 94–110.

Gerkushenko, G. G., Sokolova, S. V., Meshcheryakova, E. V., and Meshcheryakova, J. V. (2013). The influence of computer games on Children’s play activity development. World Appl. Sci. J. 24, 177–182. doi: 10.5829/idosi.wasj.2013.24.itmies.80036

Greipl, S., Moeller, K., and Ninaus, M. (2020). Potential and limits of game-based learning. Int. J. Technol. Enhanc. Learn. 12, 363–389. doi: 10.1504/IJTEL.2020.110047

Gurholt, K. P., and Sanderud, J. R. (2016). Curious play: Children’s exploration of nature. J. Adv. Educ. Outdoor Learn. 16, 318–329. doi: 10.1080/14729679.2016.1162183

Hamari, J., Shernoff, D. J., Rowe, E., Coller, B., Asbell-Clarke, J., and Edwards, T. (2016). Challenging games help students learn: an empirical study on engagement, flow and immersion in game-based learning. Comput. Hum. Behav. 54, 170–179. doi: 10.1016/j.chb.2015.07.045

Hausknecht, S., Neustaedter, C., and Kaufman, D. (2017). Blurring the lines of age: intergenerational collaboration in alternate reality games. In M. Romero, K. Sawchuk, J. Blat, S. Sayago, and H. Ouellet (Eds.) Game-based learning across the lifespan: Cross-generational and age-oriented topics , New York: Springer 47–64

Hayati, H. S., Myrnawati, C., and Asmawi, M. (2017). Effect of traditional games, learning motivation and learning style on childhoods gross motor skills. Int. J. Educ. Res. 5, 53–66.

Hung, C. Y., Sun, J. C. Y., and Yu, P. T. (2015). The benefits of a challenge: student motivation and flow experience in tablet-PC-game-based learning. Interact. Learn. Environ. 23, 172–190. doi: 10.1080/10494820.2014.997248

Hussein, M. H., Ow, S. H., Cheong, L. S., Thong, M. K., and Ebrahim, N. A. (2019). Effects of digital game-based learning on elementary science learning: a systematic review. IEEE Access 7, 62465–62478. doi: 10.1109/ACCESS.2019.2916324

Huynh, E., Nyhout, A., Ganea, P., and Chevalier, F. (2020). Designing narrative-focused role-playing games for visualization literacy in young children. IEEE Trans. Vis. Comput. Graph. 27, 924–934. doi: 10.1109/TVCG.2020.3030464

Hwa, S. P. (2018). Pedagogical change in mathematics learning: harnessing the power of digital game-based learning. J. Educ. Technol. Soc. 21, 259–276.

Iten, N., and Petko, D. (2016). Learning with serious games: is fun playing the game a predictor of learning success? Br. J. Educ. Technol. 47, 151–163. doi: 10.1111/bjet.12226

Kaimara, P., Fokides, E., Oikonomou, A., and Deliyannis, I. (2021). Potential barriers to the implementation of digital game-based learning in the classroom: pre-service teachers’ views. Technol. Knowl. Learn. 26, 825–844. doi: 10.1007/s10758-021-09512-7

Ke, F. (2013). Computer-game-based tutoring of mathematics. Comput. Educ. 60, 448–457. doi: 10.1016/j.compedu.2012.08.012

Ke, F. (2014). An implementation of design-based learning through creating educational computer games: a case study on mathematics learning during design and computing. Comput. Educ. 73, 26–39. doi: 10.1016/j.compedu.2013.12.010

Lamrani, R., and Abdelwahed, E. H. (2020). Game-based learning and gamification to improve skills in early years education. Comput. Sci. Inf. Syst. 17, 339–356. doi: 10.2298/CSIS190511043L

Lester, J. C., Ha, E. Y., Lee, S. Y., Mott, B. W., Rowe, J. P., and Sabourin, J. L. (2013). Serious games get smart: intelligent game-based learning environments. AI Mag. 34, 31–45. doi: 10.1609/aimag.v34i4.2488

Lin, L., and Aloe, A. M. (2021). Evaluation of various estimators for standardized mean difference in meta-analysis. Stat. Med. 40, 403–426. doi: 10.1002/sim.8781

Lin, S. Y., Chien, S. Y., Hsiao, C. L., Hsia, C. H., and Chao, K. M. (2020). Enhancing computational thinking capability of preschool children by game-based smart toys. Electron. Commer. Res. Appl. 44:101011. doi: 10.1016/j.elerap.2020.101011

Liu, E. Z. F., and Chen, P. K. (2013). The effect of game-based learning on students’ learning performance in science learning–a case of “conveyance go”. Procedia Soc. Behav. Sci. 103, 1044–1051. doi: 10.1016/j.sbspro.2013.10.430

Lundh, A., and Gøtzsche, P. C. (2008). Recommendations by Cochrane review groups for assessment of the risk of bias in studies. BMC Med. Res. Methodol. 8, 1–9. doi: 10.1186/1471-2288-8-22

Mekler, E. D., Brühlmann, F., Opwis, K., and Tuch, A. N. (2013). Do points, levels and leaderboards harm intrinsic motivation? An empirical analysis of common gamification elements. In Proceedings of the First International Conference on gameful design, research, and applications 66–73.

Mukund, V., Sharma, M., Srivastva, A., Sharma, R., Farber, M., and Chatterjee Singh, N. (2022). Effects of a digital game-based course in building adolescents’ knowledge and social-emotional competencies. Games Health J. 11, 18–29. doi: 10.1089/g4h.2021.0138

Nieto-Escamez, F. A., and Roldán-Tapia, M. D. (2021). Gamification as online teaching strategy during COVID-19: a mini-review. Front. Psychol. 12:648552. doi: 10.3389/fpsyg.2021.648552

Panter-Brick, C., Burgess, A., Eggerman, M., McAllister, F., Pruett, K., and Leckman, J. F. (2014). Practitioner review: engaging fathers–recommendations for a game change in parenting interventions based on a systematic review of the global evidence. J. Child Psychol. Psychiatry 55, 1187–1212. doi: 10.1111/jcpp.12280

Papanastasiou, G., Drigas, A., and Skianis, C. (2022). Serious games in pre-K and K-6 education. Tech. Educ. Human. 2, 1–18. doi: 10.47577/teh.v2i3.7365

Park, J. H., and Park, M. (2021). Smartphone use patterns and problematic smartphone use among preschool children. PLoS One 16:e0244276. doi: 10.1371/journal.pone.0244276

Partovi, T., and Razavi, M. R. (2019). The effect of game-based learning on academic achievement motivation of elementary school students. Learn. Motiv. 68:101592. doi: 10.1016/j.lmot.2019.101592

Pérez-Marín, D., Hijón-Neira, R., Bacelo, A., and Pizarro, C. (2020). Can computational thinking be improved by using a methodology based on metaphors and scratch to teach computer programming to children? Comput. Hum. Behav. 105:105849. doi: 10.1016/j.chb.2018.12.027

Peterson, C., Slaughter, V., Moore, C., and Wellman, H. M. (2016). Peer social skills and theory of mind in children with autism, deafness, or typical development. Dev. Psychol. 52, 46–57. doi: 10.1037/a0039833

Pivec, M. (2007). Play and learn: potentials of game-based learning. Br. J. Educ. Technol. 38, 387–393. doi: 10.1111/j.1467-8535.2007.00722.x

Przybylski, A. K., and Weinstein, N. (2019). Digital screen time limits and young children's psychological well-being: evidence from a population-based study. Child Dev. 90, e56–e65. doi: 10.1111/cdev.13007

Qian, M., and Clark, K. R. (2016). Game-based learning and 21st century skills: a review of recent research. Comput. Hum. Behav. 63, 50–58. doi: 10.1016/j.chb.2016.05.023

Quinto, J. D. G. (2022). Development and validation of survey instrument on game-based learning approach (SIGBLA). Int. J. Emerg. Technol. Learn. 17, 233–242. doi: 10.3991/ijet.v17i15.33267

Robson, R. C., Hwee, J., Thomas, S. M., Rios, P., Page, M. J., and Tricco, A. C. (2019). Few studies exist examining methods for selecting studies, abstracting data, and appraising quality in a systematic review. J. Clin. Epidemiol. 106, 121–135. doi: 10.1016/j.jclinepi.2018.10.003

Ronimus, M., Kujala, J., Tolvanen, A., and Lyytinen, H. (2014). Children's engagement during digital game-based learning of reading: the effects of time, rewards, and challenge. Comput. Educ. 71, 237–246. doi: 10.1016/j.compedu.2013.10.008

Ronimus, M., and Lyytinen, H. (2015). Is school a better environment than home for digital game-based learning?: the case of GraphoGame. Hum. Technol. 11, 123–147. doi: 10.17011/ht/urn.201511113637

Schabas, A. (2023). Game-based science learning: what are the problems with teachers practicing it in class? Assyfa Learn. J. 1, 91–105. doi: 10.61650/alj.v1i2.128

Shamir, H., Pocklington, D., Feehan, K., and Yoder, E. (2019). Game-based learning for young learners. Int. J. Learn. Teach. 5, 206–212. doi: 10.18178/ijlt.5.3.206-212

Simion, A., and Bănuț, M. (2020). Digital technology dimensions from the perspective of socio-emotional development at school children. Eur. Proceed. Soc. Behav. Sci. 104. doi: 10.15405/epsbs.2021.03.02.34

Suhana, M. (2017). “Influence of gadget usage on Children's social-emotional development” in International conference of early childhood education (ICECE 2017) (Atlantis Press), 224–227.

Sun, C., Shute, V. J., Stewart, A. E., Beck-White, Q., Reinhardt, C. R., Zhou, G., et al. (2022). The relationship between collaborative problem solving behaviors and solution outcomes in a game-based learning environment. Comput. Hum. Behav. 128:107120. doi: 10.1016/j.chb.2021.107120

Suurmond, R., van Rhee, H., and Hak, T. (2017). Introduction, comparison, and validation of meta-essentials: a free and simple tool for meta-analysis. Res. Synth. Methods 8, 537–553. doi: 10.1002/jrsm.1260

Tay, J., Goh, Y. M., Safiena, S., and Bound, H. (2022). Designing digital game-based learning for professional upskilling: a systematic literature review. Comput. Educ. 184:104518. doi: 10.1016/j.compedu.2022.104518

Thai, K. P., Bang, H. J., and Li, L. (2022). Accelerating early math learning with research-based personalized learning games: a cluster randomized controlled trial. J. Res. Educ. Effect. 15, 28–51. doi: 10.1080/19345747.2021.1969710

Toh, W., and Kirschner, D. (2020). Self-directed learning in video games, affordances and pedagogical implications for teaching and learning. Comput. Educ. 154:103912. doi: 10.1016/j.compedu.2020.103912

Toh, W., and Kirschner, D. (2023). Developing social-emotional concepts for learning with video games. Comput. Educ. 194:104708. doi: 10.1016/j.compedu.2022.104708

Valdés, G. (2014). Expanding definitions of giftedness: the case of young interpreters from immigrant communities Routledge.

Xu, J., Lio, A., Dhaliwal, H., Andrei, S., Balakrishnan, S., Nagani, U., et al. (2021). Psychological interventions of virtual gamification within academic intrinsic motivation: a systematic review. J. Affect. Disord. 293, 444–465. doi: 10.1016/j.jad.2021.06.070

Yousef, A. M. F. (2021). Augmented reality assisted learning achievement, motivation, and creativity for children of low-grade in primary school. J. Comput. Assist. Learn. 37, 966–977. doi: 10.1111/jcal.12536

Yu, J., Denham, A. R., and Searight, E. (2022). A systematic review of augmented reality game-based learning in STEM education. Educ. Technol. Res. Dev. 70, 1169–1194. doi: 10.1007/s11423-022-10122-y

Yu, Y. T., and Tsuei, M. (2022). The effects of digital game-based learning on children’s Chinese language learning, attention and self-efficacy. Interact. Learn. Environ. 31, 6113–6132. doi: 10.1080/10494820.2022.2028855

Zabala-Vargas, S., de-Benito, B., Darder-Mesquida, A., Arciniegas-Hernandez, E., Reina-Medrano, J., and García-Mora, L. (2021). Strengthening motivation in the mathematical engineering teaching processes-a proposal from gamification and game-based learning. Int. J. Emerg. Technol. Learn. 16, 4–10.

Keywords: game-based learning, early childhood, cognitive outcomes, social engagement, emotional development

Citation: Alotaibi MS (2024) Game-based learning in early childhood education: a systematic review and meta-analysis. Front. Psychol . 15:1307881. doi: 10.3389/fpsyg.2024.1307881

Received: 05 October 2023; Accepted: 20 March 2024; Published: 02 April 2024.

Reviewed by:

Copyright © 2024 Alotaibi. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Manar S. Alotaibi, [email protected]

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

• Open access
• Published: 01 April 2024

The impact of housing prices on residents’ health: a systematic review

• Ashmita Grewal 1 ,
• Kirk J. Hepburn 1 ,
• Scott A. Lear 1 ,
• Marina Adshade 2 &
• Kiffer G. Card 1  

BMC Public Health volume  24 , Article number:  931 ( 2024 ) Cite this article

430 Accesses

19 Altmetric

Metrics details

Rising housing prices are becoming a top public health priority and are an emerging concern for policy makers and community leaders. This report reviews and synthesizes evidence examining the association between changes in housing price and health outcomes.

We conducted a systematic literature review by searching the SCOPUS and PubMed databases for keywords related to housing price and health. Articles were screened by two reviewers for eligibility, which restricted inclusion to original research articles measuring changes in housing prices and health outcomes, published prior to June 31st, 2022.

Among 23 eligible studies, we found that changes in housing prices were heterogeneously associated with physical and mental health outcomes, with multiple mechanisms contributing to both positive and negative health outcomes. Income-level and home-ownership status were identified as key moderators, with lower-income individuals and renters experience negative health consequences from rising housing prices. This may have resulted from increased stress and financial strain among these groups. Meanwhile, the economic benefits of rising housing prices were seen to support health for higher-income individuals and homeowners – potentially due to increased wealth or perception of wealth.

Conclusions

Based on the associations identified in this review, it appears that potential gains to health associated with rising housing prices are inequitably distributed. Housing policies should consider the health inequities born by renters and low-income individuals. Further research should explore mechanisms and interventions to reduce uneven economic impacts on health.

Peer Review reports

Introduction

In contemporary society, the structures we live in, as well as our legal relationships to these structures, are intertwined with our fundamental senses of self and belonging [ 1 , 2 , 3 ]. For decades, homeownership has been recognized as a core measure of success [ 4 , 5 ]. Recognizing the importance of housing, studies have variously examined the effects of wide-ranging housing-related factors on health, including housing quality, overcrowding, neighbourhood deprivation, social cohesion, housing density, housing suitability or sufficiency, and neighbourhood socioeconomic status [ 6 , 7 ]. While these effects continue to be explored, it is generally agreed that housing is a fundamental determinant of health [ 7 ], which broadly exerts impacts on health through a variety of mechanisms.

Indeed, housing-related health effects arise from specific housing conditions, as well as the legal conditions that define our relationships to these spaces, and our emotional attachments to these various factors. For example, living and owning a home can create access to opportunities that can further bolster health [ 8 ]. Similarly, housing related factors—such as indebtedness, mortgage stress, and credit problems—can cause severe mental health problems, depression, and suicide ideation [ 9 , 10 ]. With these factors in mind, people in most countries face numerous barriers to securing their right to a home [ 5 , 11 ], and a wide array of policies have been proposed and implemented to address these barriers [ 12 , 13 , 14 ]. In addition to these factors, the location of a home, the quality of a building, or the neighbourhood context in which a home exists are also hugely influential to health [ 7 , 15 , 16 ].

In conceptualizing these varied mechanisms, it is important to consider both direct and indirect mechanisms through which the relationship between housing and health manifests. Direct effects predominantly emerge from psycho-physiological stress responses. Elevated housing costs can induce chronic stress, leading to mental health conditions, like anxiety and depression, and other health problems [ 17 ]. Indirectly, escalating housing prices exert economic pressures that limit individuals' capacity to allocate resources towards health-promoting activities and necessities. This economic strain can result in compromised nutrition, reduced access to healthcare services, and diminished ability to manage chronic conditions, therefore, exacerbating health disparities. Moreover, the financial burden can lead to other lifestyle changes that further impair physical and mental well-being, such as increased substance use or reduced physical activity.

Despite these effects being documented in previous studies, there are no systematic reviews on the impact of rising housing prices on health. The present review aims to examine the effect of housing price on health by considering whether changes in housing market price impact the health of residents living in an area. To accomplish this aim, we conduct a systematic review. This review is especially timely since housing prices have risen in the past five years at an alarming rate.

Article search

The first step in our multi-stage systematic literature review was to manually identify relevant articles through a rudimentary search on SCOPUS and PubMed ( Appendix B ). We then created a list of keywords to use for our search. Keywords aimed to identify articles that measured changes in housing prices and health impacts, Appendix A outlines how we identified keywords and provides a complete list of selected keywords. After conducting the keyword search in PubMed and SCOPUS, duplicates were removed and the remaining articles were then uploaded to Rayyan, an online software that aids in systematic reviews [ 18 ]. To assess whether our search is comprehensive, AG confirmed that the articles identified in the rudimentary initial search, mentioned earlier, were also included in this search. For the purposes of this literature review, we define health using the language provided by the World Health Organization (1948): “health is a complete state of mental, physical, and social well-being, and not merely the absence of disease.” As such, no additional inclusion or exclusion criteria were used to exclude or include specific health conditions. We felt this was appropriate given that this is the first literature review on this topic and because after a review of included articles, it was apparent that a wide variety of health outcomes have been considered. Furthermore, the biopsychosocial models of health that we engage to inform our view that housing prices have direct and indirect effects on health underscore that diverse and nuanced pathways across various mental and physical domains of health are likely important to consider. Using Rayyan, AG and LW reviewed the titles of each manuscript to remove articles that were clearly not relevant to this review [ 18 ]. The application of inclusion and exclusion criteria resulted in 21 articles that were directly relevant to this review. AG and LW also searched the reference lists for these 21 articles to identify any additional articles. These missed articles were added to our final inclusion list, creating a total of 23 included articles.

Data extraction

Data were extracted by AG and LW from each of the identified and included articles and AG re-reviewed the data extraction to verify accuracy. Extracted variables included: first author name, year of publication, years of data collection, sample size, location(s) of study, study design (e.g., case control, cohort, cross-sectional, serial cross-sectional study), analysis type (e.g., regression), outcome, explanatory factor, confounders/mediators/moderators, and a summary of primary findings (including effect size measures). This data extraction is provided as Table  1 .

Risk of bias assessment

During the data extraction process, we conducted an assessment based on the Joanna Briggs Institute Critical Appraisal Tools [ 42 ]. Each study was classified according to its study design and rated using the appropriate tool designed for each study. However, despite varying methodological quality, no studies were excluded based on risk of bias assessment, as there were no clear sources of systematic bias with sufficient likelihood of challenging the conclusions of the source studies.

Narrative synthesis

During the data extraction and risk of bias assessment phases, AG and LW recorded general notes on each of the studies. These notes, along with the extracted information, were used to construct a narrative synthesis of the evidence. This process was guided by Popay et al.’s [ 43 ] Guidance for Narrative Synthesis in Systematic Reviews. A narrative approach was selected to allow for an examination of the potential complexity inherent in the synthesis of findings across contexts, time periods, and populations to provide a nuanced discussion of what roles housing and rental markets might play in shaping health, with attention to both outcomes and potential mechanisms. Findings within study classes were reviewed to determine potential mediation and moderation. These explorations informed the development of a list of key points used to organize the presentation of our results. We then integrated and contextualized these findings with those from other relevant (though excluded) studies identified through our review process and from the texts of the included articles.

Included studies

Our keyword search returned 6,180 articles. Of these, 5,590 were removed based on review of the abstract and title as they were not directly related to our review topic (i.e., they did not measure changes in housing price and/or health outcomes). The remaining articles were reviewed based on their full-texts and a final list of 26 articles were considered for inclusion. However, five articles were not able to be retrieved (even after emailing the original authors), leaving us with 21 articles. The reference lists and bibliographies for these 21 included articles were then screened and two additional articles were thus included in our review resulting in a final sample of 23 articles. Figure  1 shows the flow diagram for included studies and these studies are listed in Appendix B .

PRISMA systematic review flow diagram

Dates and locations of studies

A full description of studies is included in Table  1 . Studies were published from 2013–2022. Ten studies were from East Asia, eight from the United States, three in Europe, one from Australia, and one included nine countries (France, Japan, Netherlands, Spain, Switzerland, Sweden, United Kingdom, USA).

Study design

Of the included studies, ten had a longitudinal study design, and thirteen studies were serial cross-sectional studies. Studies examined the effect of housing prices on health over time by repeatedly surveying a specific geographical area or population. One study included both qualitative and quantitative data collection.

Outcome variable measurement

Most studies compared multiple outcomes. Seven studies focused on mental health as the outcome variable—utilizing various measures, including self-rated mental health, standardized scales for depression or anxiety, and receipt of pharmaceutical prescriptions [ 22 , 32 , 33 , 35 , 40 , 41 ]. Nine studies analyzed the impact of housing prices on physical health—utilizing various measures of physical health, including objective assessments of physical health (e.g., body mass), self-rated physical health assessments, reports of specific health conditions (e.g., COVID-19), reported health behaviours (e.g., alcohol use, smoking), and mortality ( [ 24 , 28 , 29 , 37 , 44 , 30 , 36 , 38 , 39 ]). Seven studies included both physical and mental health measures as their outcome variable [ 19 , 20 , 23 , 26 , 27 , 34 , 45 ].

Explanatory variable measurement

Housing prices were measured using many different types of data, including house price index, self-reported housing price (extracted from surveys), and average market price. Many studies used house price index as a measure of housing prices [ 19 , 21 , 22 , 30 , 38 , 39 , 41 ]. Zhang & Zhang [ 33 ] included self-reported housing price. Alternatively, many studies examined housing market prices using existing survey data [ 20 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 31 , 32 , 34 , 35 , 36 , 37 , 40 ].

Key findings

Studies included in our review highlighted a plurality of results when testing the relationships between housing prices and health. As shown in Tables  2 (physical health) and Table  3 (mental health), the included articles reported mixed findings across the outcomes explored. Given the heterogeneity of findings regarding the associations between housing prices and health outcomes, several authors examined potential moderators and mediators in attempt to understand the mechanisms at play. These included studies examine the role of wealth effects (by comparing effects on homeowners and renters), socioeconomic status (e.g., income level), and broader economic forces (e.g., area-level improvements). While keeping these pathways in mind, there are likely other alternative explanations beyond those explored. However, these appear to be the most dominant frameworks used to understand the effects in our included studies.

Wealth effects

The first major pathway has been described as a “wealth effect” – which produces different effects for homeowners and renters [ 19 , 20 , 23 , 25 , 27 , 33 , 35 , 37 , 45 ]. For example, Hamoudi & Dowd [ 37 ] report that homeowners living in areas with steep price increases, perceive this as an increase in their overall wealth, resulting in positive health outcomes (not observed for renters). Similarly, Zhang & Zhang [ 33 ] show that increases in house prices has a positive effect on homeowner’s subjective well-being. De & Segura [ 30 ] specifically notes that price depreciation causes homeowners to experience feelings of a loss of wealth, leading to increases in alcohol consumption. Among studies that fail to show a wealth effect, Daysal et al. [ 29 ] shows that rising prices in Denmark do not impact households due to the buffering effects of government supports. Conversely, when examining the effects among renters, Wang & Liang [ 31 ] argue that rising housing prices have detrimental "strain" effect, which is also observed in several studies included in our review [ 25 , 27 , 38 , 39 , 45 ].

Income level

In addition to the wealth and strain effects illustrated through studies among homeowners and renters, many studies also examined the mediating effects of income [ 19 , 20 , 22 , 28 , 29 , 30 , 32 , 33 , 35 , 38 , 39 , 40 ]. Several of these studies show that housing unaffordability constrains spending and that low-income individuals are particularly impacted [ 22 , 24 , 33 , 38 ]. For example, Wong et al. [ 39 ] show that housing prices lead to reduced fruit consumption. However, results also show positive impacts for low-income homeowners – as exemplified by work showing that low-income homeowners are more sensitive to housing price gains [ 38 , 40 ].

Broader economic forces

In considering both mechanisms described above, authors of included studies have also considered whether housing prices are merely an indicator of broader economic trends merits consideration. The most common strategy for accounting for this has been to include other indicators that might capture area level improvements. Indeed, most studies controlled for both individual characteristics or variables, such as age, gender, marital status, years of education, race/ethnicity, and employment status [ 20 , 23 , 24 , 25 , 26 , 27 , 29 , 30 , 32 , 34 , 35 , 37 , 39 , 40 , 41 , 45 ], and a variety of economic factors, including individual income, country-level median income, and local area characteristics [ 19 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 30 , 32 , 36 , 37 , 38 , 39 , 40 , 44 , 45 ]. These factors are important to control for because rising housing prices can indicate a growing economy in which there are substantial improvements to neighbourhoods and communities [ 27 , 29 , 33 , 38 , 40 ]. As such, the observed improvements in health could simply arise from broader economic benefits (rather than being specifically attributable to housing prices) [ 31 , 33 ]. However, generally speaking studies showed that there were independent effects of housing price or value, even after controlling for local area level improvements, and wider economic conditions [ 19 , 27 , 38 ].

Strength of effects

Given heterogeneity in the direction of effects, the lack of standardization in the reporting of effect sizes from study to study, differences in the measurement of exposure and outcome variables, and variation in the inclusion of mediators, moderators, and confounders, we did not conduct a meta-analysis to describe the effect size of housing price on health. However, housing prices appear to exert influence on health and wellbeing with statistically significant effects across various health-related outcomes (See Table  1 for range of effect measures). The effects generally appear to be smaller when considering specific health conditions and greater when considering more subjective and more broad definitions of health (e.g., self-rated health). Of course, at a population-level, even relatively small effect sizes may pose a considerable challenge. For example, Xu & Wang [ 36 ] report that a 10% increase in housing prices is associated with a 6.5% increase in probability of reporting a chronic disease – a relatively small increase on a person-level, but when scaled could easily pose a considerable burden to the health system. In summary, further careful measurement and methodological refinement is needed to quantify the effects of housing prices on various health conditions. For any given health condition, this will require multiple well-designed studies across place and time. Such replication is particularly important given the observed sensitivity of findings to the inclusion of confounders, moderators, and mediators.

Primary findings

While examining whether changes in housing prices are associated with changes in health, we recognized it is difficult to establish a directional and causal relationship between housing price and health. This is particularly true given that health may increase opportunities for home ownership and economic success [ 3 , 46 , 47 , 48 , 49 ]. Nevertheless, given the wealth of literature highlighting housing affordability as a key determinant of health [ 7 , 9 , 10 ], it is reasonably anticipated that rising housing prices would be associated with worse health outcomes among individuals who do not own housing. However, based on analyses of the studies included in this review, the relationship between housing price and health is complex and nuanced, with a significant degree of heterogeneity across outcomes and populations.

First, this review illuminates that changing housing prices impacted different people differently, depending on for example, income level, gender and/or homeownership status [ 19 , 22 , 25 ]. The negative impact of housing price on health for renters and low-income individuals could be due to the existential angst from being excluded from home ownership, which is often considered an important indicator of social class and success [ 8 ]. However, this could also be due to the cost effect that is created from the rising house prices, subsequently raising low-income owners and renters’ cost of living [ 31 ]. Additionally, renting may be associated with lower neighbourhood tenure, especially when individuals are priced out of a neighbourhood [ 8 ]. As a result, they may experience deleterious health effects associated with loneliness, social isolation, lack of neighbourhood cohesion, and community disconnectedness [ 8 ]. Likewise, the positive effect observed among homeowners and high-income individuals may be explained by increases in psychological safety leading to changes in health behaviors, for example, knowing they have invested in a home that will support them or their heirs financially, people may be better able to focus on their well-being. As well, homeowners may be able to directly leverage the value of their home to gain access to additional capital and investment opportunities – which could support increased financial wellbeing [ 8 ].

The effects of housing on health can be conceptualized as arising from two sources. It appears that rising “cost effects” (the increased costs of houses and the costs passed on to tenants) are inversely correlated with health while “wealth effects” (the contributions of housing price to person wealth) contribute positively to health (for example, for homeowners and investors whose wealth increases due to the rising cost of housing). The balance of these effects differs depending on their unique impact on individuals – with lower income people and renters more strongly impacted by cost effects, and higher income people and homeowners more strongly impacted by wealth effects.

In considering these effects, we note that there is likely considerable geographic, temporal, and contextual variation in the health effects of rising housing prices. For example, rising housing prices may occur alongside neighbourhood improvements (or degradation) and economic booms (or recessions) [ 45 ], which themselves are associated with improvements (or deterioration) to health [ 8 ]. As such, the presence of these factors may obscure or interact with the gains to health. Similarly, variations across cultures and countries may change how individuals internalize the rising housing prices [ 50 ], causing them to experience greater or lesser distress in reaction to rising prices.

Limitations of included studies and directions for future research

Given these two primary factors, research highlights several opportunities for improving this literature. First, future studies should give more careful attention to how moderators and mediators are conceptualized. For example, “home-owners” are hardly a homogenous class of individuals: some own their homes outright and others are paying mortgages that offer varying levels of security (e.g., fixed vs. variable mortgages, 5-year vs. 30-year mortgages) [ 8 ]. Second, a broader range of effect moderators should be explored. For example, few studies specifically examined the health effects of rising home prices on vulnerable populations, including young adults and first time home buyers who may be especially disadvantaged by rapidly growing housing prices [ 50 ]. Similarly, isolating effects as arising from economic, legal, environmental, and social pathways can help identify strategies for mitigating health harms. For example, it may be important to understand whether changes to neighbourhood environments drive health harms as opposed to changes in personal financial status. Third, more within-person studies are needed to understand the potential mechanisms and situational factors that promote or mitigate the health effects of rising housing prices. Along with use of appropriate, theoretically informed moderators, isolating the within-person effects can help us better quantify the effects of interest to inform policies and prevention strategies. Fourth, longer follow-up times may allow for better understanding regarding the time-horizons of the effects explored. Indeed, it is possible that rising housing prices could have differential effects on the health of a population in the short versus long term. This is particularly important given the interaction between housing prices (which may act as a price signal for investments) and other economic factors with strong potential to increase health [ 8 ]. Fifth, the studies used a variety of measures for housing price and health outcomes – which varied in quality. For example, health outcomes were primarily measured using self-reported measures [ 19 , 20 , 23 , 25 , 26 , 27 , 30 , 31 , 33 , 34 , 35 , 37 , 38 , 39 , 40 , 41 ] – which may be highly sensitive to bias due to the likelihood that individuals might report worse health when they are unhappy with economic factors. Improving measurement of outcomes can be done by leveraging administrative and other data sources. Sixth, it can be difficult to link area-level and individual-level factors, particularly in the context of limited cross-sectional studies or in longitudinal studies with only a few follow-up points. Likewise, many cohort-based studies have limited geographic coverage or insufficient temporal scope. As such, longer, larger, and wider studies are needed to fully ascertain the relationships under consideration.

Implications of findings

Although further research is required to overcome the limitations mentioned, existing evidence indicates that increases in housing prices may significantly influence health outcomes. Future studies should aim to exclude alternative explanations, examine the effects over longer periods, and establish consistent measurement methods to predict the impact of housing prices more accurately on health. The findings of those students will aid policymakers in creating strategies that address the health implications of rising housing prices. Policy makers should develop frameworks that respond to the impacts of rising housing prices on health. Such approaches could be facilitated through frameworks such as the WHO’s Health in All Policies (HiAP) policy, which advocates for the inclusion of health and social impacts among other criteria used throughout decision making processes [ 51 ]. Many studies in this review support this view and describe their work as having important implications for housing and health policy [ 19 , 20 , 21 , 24 , 26 , 27 , 28 , 29 , 31 , 34 , 35 , 37 , 38 , 39 ]. For example, Yuan et al. [ 26 ] notes the importance of directing government support and housing subsidies towards vulnerable groups – though these should be packaged with other policies [ 26 ]. Such supports can apparently buffer against the negative effects of rising housing prices by creating a saftey net that reduces the psychosocial and cognitive effects associated with economic changes in one's personal circumnstance. Arcaya et al. [ 21 ] also recommends governments investigate establishing more mental health facilities in areas where housing price fluctuations impact people's mental health but warns economic development that allows for greater investment in health infrastructure can also lead to increases in housing prices.

Of course, other types of interventions may also be warranted, including broader financial interventions (e.g., direct loans; [ 52 ], those which promote community, neighborhood, and social cohesion among residents [ 53 ], or those that aim to change how people value home ownership [ 26 ]. With respect to this final option, communities should consider whether renting may in fact be a desirable outcome for some individuals and therefore promote a culture in which individuals realize the variety of investment opportunities available to them rather than being overly-focused on a traditional model of investment [ 26 , 32 ]. For example, Zhang & Zhang [ 33 ] writes that homeowners should be provided financial and economic knowledge to better manage wealth gains, however, this could be taken one step further and include the importance of educating people on the dangers on the commodification of housing, to prevent an over reliance on the importance of housing wealth gains.

Limitations of our review

In addition to the limitations specific to studies included in our review, our review itself also has several limitations. First, while we trained two reviewers to conduct article screening, assessed inter-rater reliability as greater than 80%, and adjudicated conflicts with the help of a third reviewer, it is possible some articles that could have been included were excluded due to the many different forms of outcome and explanatory variable measurement. Second, while we have searched multiple databases, used comprehensive key words for our search, and conducted manual searches of the reference lists, it is possible there were studies that we missed and failed to include in this study. That said, it is unlikely the exclusion of these articles would change our conclusion that the literature is currently mixed and that there is a need to flesh out the mechanisms and moderators that link housing prices to health. Third, we were not able to conduct a meta-analysis, and our numeric reports of the number of studies with each characteristic should not be treated as a meta-analysis. Rather, these findings and analyses of these studies should be interpreted as a descriptive analysis that highlights significant heterogeneity of findings and critical inconsistencies in the mechanisms, mediators, and moderators that give rise to these associations. Fourth, we did not exclude any studies based on article quality because we did not find there to be sufficient heterogeneity in the quality of the observational studies to merit exclusion according to variable inclusion, study design, or sampling method. In other words, we sought to avoid introducing bias by arbitrarily excluding articles – particularly given that the number of articles captured here was already relatively low (at least given the diversity of methods, measures, and populations captured). However, future reviews might consider more narrowed inclusion and exclusion criteria when a sufficient body of literature is available for a given outcome. For example, limiting analyses to only well-designed cohort studies might support a more careful selection of articles. Finally, we note that we included studies across a wide variety of health outcomes. While this was done to maximize inclusion (given the wide heterogeneity of measures used), we acknowledge that future research might be strengthened by studying specific pathways linking housing prices to specific health and social outcomes. Such detailed research is greatly needed so as to not only highlight the relevance of housing prices to health but identify strategies for mitigating potential harms of rising housing prices.

Our review shows that there are complex relationships between housing prices and health – with studies arriving to mixed conclusions across a wide-variety of health outcomes and populations. Yet, there is insufficient evidence for a causal relationship, but it appears that if such a relationship exists it likely differs according to homeownership status, income-level, and as a factor of the broader economic and structural forces in play, including the level of economic supports provided by governments for low income individuals. Future research should explore these pathways, moderators, and confounders using long-term, geographically diverse, cohort studies that account for a broad diversity of causal or alternative mechanisms. Such future research will allow for a more nuanced understanding of health and health inequities related to rising housing prices.

Availability of data and materials

All data generated or analysed during this study are included in this published article and its supplementary information files.

Saunders P. The meaning of ‘home’ in contemporary english culture. Hous Stud. 1989;4:177–92.

Article   Google Scholar  

Shields M. Community belonging and self-perceived health. Health Rep. 2008;19:51–60.

PubMed   Google Scholar  

Smith A. Exploring the interrelationship between the meanings of homeownership and identity management in a liquid society: a case study approach. [Doctoral dissertation, Keele University] Keele Business School. 2017.

Després C. The meaning of home: literature review and directions for future research and theoretical development. Locke Sci Publ Co Inc. 1991;8:96–115.

Goodman LS, Mayer C. Homeownership and the American Dream. J Econ Perspect. 2018;32:31–58.

Kawachi I. Social Capital and Community Effects on Population and Individual Health. Ann N Y Acad Sci. 1999;896:120–30.

Article   CAS   PubMed   Google Scholar  

Rolfe S, Garnham L, Godwin J, Anderson I, Seaman P, Donaldson C. Housing as a social determinant of health and wellbeing: developing an empirically-informed realist theoretical framework. BMC Public Health. 2020;20:1138.

Article   PubMed   PubMed Central   Google Scholar  

Rohe WM, Van Zandt S, McCarthy G. Home Ownership and Access to Opportunity. Hous Stud. 2002;17:51–61.

Turunen E, Hiilamo H. Health effects of indebtedness: a systematic review. BMC Public Health. 2014;14:489.

Waldron R, Redmond D. “We’re just existing, not living!” Mortgage stress and the concealed costs of coping with crisis. Hous Stud. 2017;32:584–612.

Lemphers B. Barriers & Opportunities; A Meta-Analysis of Affordable Rental Housing In Canada. Dalhousie University; 2017.  https://cdn.dal.ca/content/dam/dalhousie/pdf/faculty/architecture-planning/school-of-planning/pdfs/studentwork/MPlanIndependentStudy/Lemphers-Barriers%20and%20Opportunity-A%20Meta-Analysis%20of%20Affordable%20Rental%20Housing%20in%20Canada-2017.pdf .

Canadian Mortgage and Housing Association. Demand or Supply Side Housing Assistance?. 2019. https://www.cmhc-schl.gc.ca/professionals/housing-markets-data-and-research/housing-research/research-reports/accelerate-supply/research-insight-updating-debate-between-demand-supply-side-housing-assistance .

Glaeser EL. Rethinking the Federal Bias Toward Homeownership. SSRN Electron J. 2011. https://doi.org/10.2139/ssrn.1914468 .

Pomeroy S. Optimizing Demand and Supply Side Housing Assistance Programs. 2016. Carleton University Centre for Urban Research and Education (CURE). https://www.focus-consult.com/wp-content/uploads/Policy-Brief_Optimizing-Supply-and-Demand-Measures-2016.pdf .

Ige-Elegbede J, Pilkington P, Orme J, Williams B, Prestwood E, Black D, et al. Designing healthier neighbourhoods: a systematic review of the impact of the neighbourhood design on health and wellbeing. Cities Health. 2022;6:1004–19.

Article   PubMed   Google Scholar  

Pérez E, Braën C, Boyer G, Mercille G, Rehany É, Deslauriers V, et al. Neighbourhood community life and health: A systematic review of reviews. Health Place. 2020;61: 102238.

Çalıyurt O. The Mental Health Consequences of the Global Housing Crisis. Alpha Psychiatry. 2022;23:264–5.

Rayyan. 2022. https://www.rayyan.ai

Yue D, Ponce NA. Booms and Busts in Housing Market and Health Outcomes for Older Americans. Innov Aging. 2021;5:igab012.

Chen N, Shen Y, Liang H, Guo R. Housing and Adult Health: Evidence from Chinese General Social Survey (CGSS). Int J Environ Res Public Health. 2021;18:916.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Arcaya MC, Nidam Y, Binet A, Gibson R, Gavin V. Rising home values and Covid-19 case rates in Massachusetts. Soc Sci Med. 2020;265: 113290.

Lee CY, Chen PH, Lin YK. An Exploratory Study of the Association between Housing Price Trends and Antidepressant Use in Taiwan: A 10-Year Population-Based Study. Int J Environ Res Public Health. 2021;18:4839.

Fichera E, Gathergood J. Do Wealth Shocks Affect Health? New Evidence from the Housing Boom. Health Econ. 2016;25:57–69.

Kim I. Spatial distribution of neighborhood-level housing prices and its association with all-cause mortality in Seoul, Korea (2013–2018): A spatial panel data analysis. SSM - Popul Health. 2021;16: 100963.

Hamoudi A, Dowd JB. Housing Wealth, Psychological Well-being, and Cognitive Functioning of Older Americans. J Gerontol Ser B. 2014;69:253–62.

Yuan W, Gong S, Han Y. How does rising housing price affect the health of middle-aged and elderly people? The complementary mediators of social status seeking and competitive saving motive. PLoS ONE. 2020;15: e0243982.

Atalay K, Edwards R, Liu BYJ. Effects of house prices on health: New evidence from Australia. Soc Sci Med. 2017;192:36–48.

Bao W, Tao R, Afzal A, Dördüncü H. Real Estate Prices, Inflation, and Health Outcomes: Evidence From Developed Economies. Front Public Health. 2022;10: 851388.

Daysal NM, Lovenheim M, Siersbæk N, Wasser DN. Home prices, fertility, and early-life health outcomes. J Public Econ. 2021;198:104366.

De PK, Segura-Escano R. Drinking during downturn: New evidence from the housing market fluctuations in the United States during the Great Recession. Econ Hum Biol. 2021;43: 101070.

Wang H-Q, Liang L-Q. How Do Housing Prices Affect Residents’ Health? New Evidence From China. Front Public Health. 2022;9: 816372.

Wei G, Zhu H, Han S, Chen J, Shi L. Impact of house price growth on mental health: Evidence from China. SSM - Popul Health. 2021;13: 100696.

Zhang C, Zhang F. Effects of housing wealth on subjective well-being in urban China. J Hous Built Environ. 2019;34:965–85.

Feng Y, Nie C. The effect of housing price on health: new evidence from China. Appl Econ Lett. 2022;29:1439–46.

Chun H. Do housing price changes affect mental health in South Korea? Ethiop J Health Dev. 2020;34:48–59.

Google Scholar  

Xu Y, Wang F. The health consequence of rising housing prices in China. J Econ Behav Organ. 2022;200:114–37.

Hamoudi A, Dowd JB. Physical health effects of the housing boom: quasi-experimental evidence from the health and retirement study. Am J Public Health. 2013;103:1039–45.

Sung J, Qiu Q. The Impact of Housing Prices on Health in the United States Before, During, and After the Great Recession. South Econ J. 2020;86:910–40.

Wong ES, Oddo VM, Jones-Smith JC. Are Housing Prices Associated with Food Consumption? Int J Environ Res Public Health. 2020;17:3882.

Ratcliffe A. Wealth Effects, Local Area Attributes, and Economic Prospects: On the Relationship between House Prices and Mental Wellbeing. Rev Income Wealth. 2015;61:75–92.

Joshi NK. Local house prices and mental health. Int J Health Econ Manag. 2016;16:89–102.

Joanna Briggs Institute. Critical Appraisal Tools. 2022. https://jbi.global/critical-appraisal-tools

Popay J, Roberts H, Sowden AJ, Petticrew M, Arai L, Rodgers ME., Britton, N. Guidance on the Conduct of Narrative Synthesis in Systematic Reviews. 2006. https://www.lancaster.ac.uk/media/lancaster-university/content-assets/documents/fhm/dhr/chir/NSsynthesisguidanceVersion1-April2006.pdf .

Arcaya MC, Nidam Y, Binet A, Gibson R, Gavin V. Rising home values and Covid-19 case rates in Massachusetts. Soc Sci Med. 1982;2020(265): 113290.

Wang H-Q, Liang L-Q. How Do Housing Prices Affect Residents’ Health? New Evidence From China. Front Public Health. 2022;9:1–11.

Shlay AB. Low-income Homeownership: American Dream or Delusion? Urban Stud. 2006;43:511–31.

Boehm TP, Schlottmann AM. The dynamics of race, income, and homeownership. J Urban Econ. 2004;55:113–30.

Fazel S, Khosla V, Doll H, Geddes J. The Prevalence of Mental Disorders among the Homeless in Western Countries: Systematic Review and Meta-Regression Analysis. PLOS Med. 2008;5: e225.

McConnell ED. Who has Housing Affordability Problems? Disparities in Housing Cost Burden by Race, Nativity, and Legal Status in Los Angeles. Race Soc Probl. 2013;5:173–90.

Ding Y, Chin L, Lu M, Deng P. Do high housing prices crowd out young professionals?—Micro-evidence from China. Econ Res-Ekon Istraživanja. 2022;36(2):1–19.

CAS   Google Scholar  

World Health Organization. 2023. https://www.who.int/about/accountability/governance/constitution .

Fernald LC, Hamad R, Karlan D, Ozer EJ, Zinman J. Small individual loans and mental health: a randomized controlled trial among South African adults. BMC Public Health. 2008;8:409.

Silverstein M, Cong Z, Li S. Intergenerational Transfers and Living Arrangements of Older People in Rural China: Consequences for Psychological Well-Being. J Gerontol Ser B. 2006;61:S256–66.

Download references

Acknowledgements

We would like to acknowledge the support of Logan White for his support in conducting the literature review.

KGC is supported by a Michael Smith Health Research BC Scholar Award. This project was funded by grants from The Canadian Institutes for Health Research and the GenWell Project.

Author information

Authors and affiliations.

Faculty of Health Sciences, Simon Fraser University, Blusson Hall, 8888 University Dr. , Burnaby, BC, V5A 1S6, Canada

Ashmita Grewal, Kirk J. Hepburn, Scott A. Lear & Kiffer G. Card

Vancouver School of Economics, University of British Columbia, Vancouver, BC, Canada

Marina Adshade

You can also search for this author in PubMed   Google Scholar

Contributions

KGC & AG conceptualized the study design. KGC, LW, and AG conducted the literature review, search, and data extraction. KGC & AG drafted the initial manuscript. All authors provided substantive intellectual and editorial revisions and approved the final manuscript.

Corresponding author

Correspondence to Ashmita Grewal .

Ethics declarations

Ethics approval and consent to participate.

This study does not require ethics approval as it is a literature review.

Consent for publication

There are no figures or materials presented in this manuscript that require consent.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Grewal, A., Hepburn, K.J., Lear, S.A. et al. The impact of housing prices on residents’ health: a systematic review. BMC Public Health 24 , 931 (2024). https://doi.org/10.1186/s12889-024-18360-w

Download citation

Received : 04 August 2023

Accepted : 14 March 2024

Published : 01 April 2024

DOI : https://doi.org/10.1186/s12889-024-18360-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Housing price
• Economic stress
• Social determinants of health

BMC Public Health

ISSN: 1471-2458

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Open access
• Published: 29 January 2022

Gut microbiota in patients with obesity and metabolic disorders — a systematic review

• Zhilu XU 1 , 2 , 3   na1 ,
• Wei JIANG 1   na1 ,
• Wenli HUANG 1 , 2 , 3 ,
• Yu LIN 1 , 2 , 3 ,
• Francis K.L. CHAN 1 , 2 , 3 &
• Siew C. NG 1 , 2 , 3  

Genes & Nutrition volume  17 , Article number:  2 ( 2022 ) Cite this article

11k Accesses

63 Citations

35 Altmetric

Metrics details

Previous observational studies have demonstrated inconsistent and inconclusive results of changes in the intestinal microbiota in patients with obesity and metabolic disorders. We performed a systematic review to explore evidence for this association across different geography and populations.

We performed a systematic search of MEDLINE (OvidSP) and Embase (OvidSP) of articles published from Sept 1, 2010, to July 10, 2021, for case–control studies comparing intestinal microbiome of individuals with obesity and metabolic disorders with the microbiome of non-obese, metabolically healthy individuals (controls). The primary outcome was bacterial taxonomic changes in patients with obesity and metabolic disorders as compared to controls. Taxa were defined as “lean-associated” if they were depleted in patients with obesity and metabolic disorders or negatively associated with abnormal metabolic parameters. Taxa were defined as “obesity-associated” if they were enriched in patients with obesity and metabolic disorders or positively associated with abnormal metabolic parameters.

Among 2390 reports screened, we identified 110 full-text articles and 60 studies were included. Proteobacteria was the most consistently reported obesity-associated phylum. Thirteen, nine, and ten studies, respectively, reported Faecalibacterium , Akkermansia , and Alistipes as lean-associated genera. Prevotella and Ruminococcus were obesity-associated genera in studies from the West but lean-associated in the East. Roseburia and Bifidobacterium were lean-associated genera only in the East, whereas Lactobacillus was an obesity-associated genus in the West.

Conclusions

We identified specific bacteria associated with obesity and metabolic disorders in western and eastern populations. Mechanistic studies are required to determine whether these microbes are a cause or product of obesity and metabolic disorders.

Introduction

Obesity-related metabolic disorders, including type 2 diabetes (T2DM), cardiovascular diseases, and non-alcoholic fatty liver disease (NAFLD), affect 13% of the population and result in 2.8 million deaths each year [ 1 , 2 ], and are a significant socioeconomic burden to society. Pathophysiology of obesity and metabolic disorders is multi-factorial, and currently, therapies are limited. The role of intestinal microbiota in patients with obesity and metabolic disorders have been extensively studied in the past decade. Humanized mouse models showed that the microbiome in obese subjects appeared to be more efficient in harvesting energy from the diet and may thereby contribute to the pathogenesis of obesity [ 3 , 4 ]. However, observational studies reported inconsistent and inconclusive changes of intestinal microbiota in patients with obesity and metabolic disorders [ 5 ]. For instance, the Firmicutes and Bacteriodetes ratio (F/B ratio) is not a reproducible marker across human cohorts [ 6 ].

Microbial-based therapies such as probiotics aiming to reshape the gut microbial ecosystem have been increasingly explored in the treatment of obesity-related metabolic disorders [ 7 , 8 ]. Traditional probiotics, primarily consisting of Lactobacillus and Bifidobacterium have been shown to elicit weight loss in subjects with obesity yet the effect sizes were small with large variations of efficacy among different studies [ 9 ]. Emerging evidence showed that Akkermansia muciniphila was depleted in patients with obesity-related metabolic disorders. These results have led to mechanistic studies and clinical trials to test its efficacy in the management of obesity and metabolic disorders [ 10 ].

Age, geography, and dietary patterns largely affect the gut microbiome [ 11 , 12 , 13 ]. The gut microbiota of vegetarians was dominated by Clostridium species [ 14 ] whereas subjects who mainly consumed fish and meat had high level of F. prausnitzii [ 15 ]. In recent years, the prevalence of childhood obesity has increased sharply. However, only limited data has issued the function and structure of gut microbiota in children and adolescents with obesity [ 16 ].

We have therefore conducted a systematic review of case–control studies evaluating the microbiota in patients with obesity and metabolic disorders compared to lean, healthy controls to summarize the current evidence in the relationship between individual members of the microbiota and obesity. We aimed to identify novel candidates as live biotherapeutics to facilitate the treatment of obesity and metabolic disorders.

Materials and methods

Search strategy.

This systematic review was performed in accordance with the PRISMA 2009 guidelines [ 17 ]. We performed a systematic search of MEDLINE (OvidSP) and Embase (OvidSP) of articles published from Sept 1, 2010 to July 10, 2021 to identify case-control studies comparing gut microbiota in patients with obesity and metabolic disorder and non-obese, metabolically healthy controls. Search strategy is shown in the Appendix .

Study selection and patient population

Studies were included if they were (1) case–control studies comparing gut microbiota in patients with obesity and metabolic disorders and non-obese, metabolically healthy individuals (controls); (2) intestinal microbiota was assessed by next-generation sequencing (NGS; 16s rRNA amplicon or shotgun metagenomic sequencing); and (3) obesity was defined based on body mass index (BMI) ≥ 30kg/m 2 and metabolic disorders including type 2 diabetes mellitus, non-alcoholic fatty liver disease, cardiovascular disease, and metabolic syndrome were diagnosed according to respective guidelines (Table 1 ). Studies from all age groups were included. Studies were excluded if they were (1) case reports, reviews, meta-analyses, re-analysis of public datasets, or conference abstracts, (2) without data for individual bacterial groups, (3) not in English, and (4) not a case–control design. Studies of genetic-associated obesity such as Prader–Willi syndrome were also excluded.

Study outcomes

The primary outcome was the bacterial taxonomic changes in patients with obesity and metabolic disorders compared to non-obese, metabolically healthy controls. Secondary outcomes included the changes in bacteria diversity and F/B ratio, subgroup analysis of microbiota changes in adults and children with obesity and metabolic disorders, and in Eastern and Western populations. Data on microbiota community composition were extracted from each study. Taxa were defined as “lean-associated” if they were depleted in patients with obesity and metabolic disorders or negatively associated with abnormal metabolic parameters such as high body mass index (BMI), elevated fasting plasma glucose and elevated serum cholesterol. Taxa were defined as “obesity-associated” if they were enriched in patients with obesity and metabolic disorders or positively associated with abnormal metabolic parameters. Taxon at each level (phylum, class, order, family, genus) was only counted once for each study (i.e., if a genus was both depleted in obesity and negatively associate with fat mass in the same study, it was only counted once).

Eligibility assessment and data extraction

Two authors (JW, HW) independently reviewed studies and excluded based on titles, abstracts, or both to lessen the selection bias and then reviewed selected studies with full text for complete analysis. JW extracted data from studies and entered it into a designated spreadsheet. HW checked the accuracy of this process. The data were re-checked when there was a discrepancy. XZ arbitrated if the discrepancy cannot be resolved by consensus and discussion. The data collected included the following: participant characteristics, including age group, country, types of metabolic disorders, number of patients; types of specimens, microbiota assessment method, microbiome diversity, and Firmicutes/Bacteroides ratio.

Quality assessment

The Newcastle-Ottawa Scale was applied to assess the quality of included studies. The Newcastle-Ottawa Scale consists of 3 domains (maximum 9 stars); selection (is the case definition adequate, representativeness of the cases, selection of controls, definition of controls); comparability (comparability of baseline characteristics); and exposure (ascertainment of exposure, same method of ascertainment for cases and controls, attrition rate).

Study characteristics

Overall, 2390 citations were retrieved; 2280 were excluded based on title, abstract, and the availability of full text; 110 articles were subsequently fully reviewed. After further review, 50 full-text articles were rejected (Fig. 1 ). The final analysis included 60 studies (Table 1 ). Of these, 44 studies assessed the gut microbiota in adults and 16 in infants, children, and adolescents. Ethnicity of subjects consisted of Asian, Black, Caucasian, Hispanic, or Latino. Fifty-eight out of 60 (96.7%) studies evaluated intestinal microbiota in stool samples and two studies assessed the microbiota in duodenal biopsies. Thirty-two studies involved patients with obesity [ 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 76 ], ten involved patients with T2DM [ 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 ], eleven involved patients with NAFLD or non-alcoholic steatohepatitis (NASH) [ 59 , 60 , 61 , 62 , 63 , 64 , 65 , 66 , 67 , 68 , 75 ], and seven involved patients with metabolic syndrome [ 69 , 70 , 71 , 72 , 73 , 74 , 77 ]. General characteristics and diagnostic criteria for obesity and metabolic disorders in each study were summarized in Table 1 .

Flowchart of study selection

Microbiome assessment methods

Of the 58 studies assessing stool microbiome, 50 studies assessed the gut microbiota by using 16S ribosomal RNA (rRNA) gene sequencing, six used shotgun metagenomic sequencing and two studies applied both 16s rRNA and shotgun metagenomic sequencing. Both studies assessing biopsy microbiome applied 16S rRNA sequencing.

Primary outcomes

At the phylum level, significant changes of phyla Firmicutes, Bacteroidetes, and Proteobacteria were most reported in obese, metabolic diseased subjects compared with controls. Among 60 studies included, 22 studies reported significant changes in Firmicutes with 15 studies showing phylum Firmicutes were obesity-associated and 7 showing it was lean-associated [ 18 , 21 , 23 , 28 , 29 , 32 , 34 , 42 , 43 , 45 , 46 ,, 46 , 48 50 , 53 , 54 , 55 , 59 , 62 , 63 , 68 , 69 , 71 ]; 20 studies reported significant changes in Bacteroidetes with 8 studies showing it was obesity-associated and 12 showing it was lean-associated [ 20 , 23 , 29 , 31 , 32 , 35 , 37 , 43 , 46 , 55 , 57 , 59 , 61 , 62 , 63 , 68 , 68 ,, 69 , 71 , 74 , 75 ]. Fifteen studies reported significant change in Proteobacteria with 13 studies showing it was obesity-associated and 2 showing it was lean-associated [ 19 , 20 , 22 , 29 , 31 , 32 , 45 , 46 , 55 , 59 , 61 , 65 , 68 , 69 , 71 ]. Studies consistently reported that Fusobacteria as obesity-associated taxa ( n = 5) [ 18 , 20 , 22 , 32 , 61 ], Actinobacteria was a lean-associated taxa ( n = 7) [ 20 , 23 , 32 , 45 , 62 , 68 , 69 ] and Tenericutes was lean-associated ( n = 4) [ 20 , 22 , 48 , 77 ] (Table 2 ). The details on the differential levels of taxon in each eligible study are shown in Supplementary table 1 .

At lower taxonomic levels, studies consistently reported the class Bacilli, Gammaproteobacteria and family Coriobacteriaceae to be obesity-associated. Controversial results were reported for class Clostridia, family Lachnospiraceae, Rikenellaceae, and Ruminococcaceae ( Supplementary table 2 ). At the genus level, Alistipes , Akkermansia , Bifidobacterium , Desulfovibrio , and genera in the Clostridium cluster IV ( Faecalibacterium , Eubacterium , Oscillospira , Odoribacter ) were the most reported lean-associated genera, while Prevotella , Lactobacillus , Blautia , Escherichia , Succinivibrio , and Fusobacterium were the most reported obesity-associated genera. Significant change in genera Ruminococcus , Coprococcus , Dialister , Bacteroides , Clostridium and Roseburia were reported but results were controversial (Table 3 ).

Secondary outcomes

Forty (67%) studies provided alpha diversity of the gut microbiota. Among them, 18 reported significant reduction in diversity while four reported significant increase of alpha diversity in obesity and metabolic disorders compared with controls. The remaining studies ( n = 18) found no significant difference in alpha diversity between both groups. In addition, 11 studies demonstrated significant difference in β-diversity [ 20 , 23 , 27 , 28 , 32 , 40 , 47 , 55 , 58 , 66 , 69 ], while 10 studies showed no significant difference in β-diversity between patients with obesity and metabolic disorders and controls [ 24 , 26 , 38 , 49 , 50 , 57 , 65 , 70 , 74 , 79 ]. Twenty-two (37%) studies reported Firmicutes/Bacteroidetes (F/B) ratio [ 51 , 52 , 53 , 54 , 56 , 57 , 58 , 59 , 60 , 61 , 62 , 63 , 64 , 65 , 66 , 67 , 68 , 71 , 72 , 73 , 74 , 75 ]. Among them, eight studies reported significant increase [ 34 , 35 , 36 , 39 , 48 , 52 , 59 , 75 ] and three studies reported a significant decreased in F/B ratio [ 33 , 41 , 44 ]. Eleven studies reported no significant change in F/B ratio in patients with obesity and metabolic disorders compared with controls (Supplementary Table 3 ) [ 37 , 42 , 46 , 53 , 54 , 60 , 61 , 62 , 63 , 67 , 68 ].

Difference of microbiota between adult and childhood obesity

The trend for most microbial changes in adult and childhood obesity were consistent. Studies reported Actinobacteria as lean-associated, while Proteobacteria and Firmicutes as obesity-associated in both adults and childhood obesity. However, discrepancies were observed for several genera. Three studies in adults consistently reported that Fusobacterium was obesity-associated, but controversial results were found in children [ 18 , 20 , 22 , 32 , 61 , 77 ]. Moreover, more studies reported that Dorea [ 39 , 46 , 49 , 77 ] and Ruminococcus [ 39 , 44 , 49 , 69 ] were obesity-associated in adults, while more studies reported them to be lean-associated in children [ 19 , 68 ]. Three studies consistently reported that Turicibacter was lean-associated in adults [ 44 , 66 , 69 ], but one study reported it to be obesity-associated in children [ 20 ]. Notably, three studies in adults reported that the genus Bifidobacterium was lean-associated [ 22 , 57 , 58 ], while controversial results were found in children (3 lean-associated and 2 obesity-associated) [ 19 , 20 , 21 , 38 , 68 ]. These findings suggested that microbiota in childhood obesity and metabolic disorders were more heterogeneous compared with adults.

Difference of microbiota between the East and the West

Large discrepancies in gut microbiome in obesity and metabolic disorders were observed in studies from the East and the West. Four studies exclusively consisting of populations in the West reported that the Family Coriobacteriaceae was obesity-associated [ 27 , 38 , 53 , 71 ] whereas none in the East reported significant change of this bacterial family between obese subjects and controls. Four studies in the East reported that the family Ruminococcaceae was lean-associated [ 22 , 60 , 61 , 63 ], but conflicting results were found in studies from the West (2 lean-associated and 2 obesity-associated) [ 27 , 36 , 43 , 68 ]. At the genus level, four studies reported that Prevotella was lean-associated in the East (3 lean-associated and 1 obesity-associated) [ 19 , 20 , 26 , 61 ], while other studies from the West have reported it to be obesity-associated (2 lean-associated and 5 obesity-associated) [ 38 , 55 , 67 , 68 , 72 , 73 , 75 ]. Three studies reported that Ruminococcus was lean-associated in the East [ 20 , 63 , 67 ], but most studies reported it to be obesity-associated in the West (1 lean-associated and 5 obesity-associated) [ 23 , 39 , 44 , 49 , 62 , 69 ]. Similar findings were observed for Roseburia (3 lean-associated in the east [ 30 , 63 , 66 ], 1 lean-associated and 2 obesity-associated in the west [ 53 , 68 , 69 ]). Notably, the common genus Lactobacillus was repeatedly reported to be obesity-associated in the West (1 lean-associated and 4 obesity-associated) [ 19 , 38 , 44 , 46 , 57 ]. Controversial results for Lactobacillus were also reported in the East (2 lean-associated and 2 obesity-associated) [ 21 , 59 , 60 , 63 ].

Quality of the evidence

The Newcastle Ottawa Scale showed that all 60 studies provided an adequate explanation in the definition and selection method for patients with obesity and metabolic disorders (Table 4 ). Fifty-five (91.7%) of 60 studies did the same process for controls. Twenty (33.3%) and 27 (45%) studies demonstrated comparable data of sex and age in patients with obesity / metabolic disorders and controls.

To our knowledge, this is the most comprehensive systematic review in microbiota and obesity and metabolic disorders, as we extracted the data of each available bacterial group using the lowest taxonomic level based on NGS of each included study. We believe that the findings reflect the best available current evidence demonstrating the relationship between individual bacterial taxa and obesity or metabolic disorders.

Proteobacteria was the most consistently reported obesity-associated phylum. Several members of Proteobacteria, such as Proteus mirabilis and E. coli , were potential drivers of inflammation in the gastrointestinal tract [ 7 , 80 , 81 ]. Low-grade inflammation is a risk factor for developing metabolic diseases including atherosclerosis, insulin resistance, and diabetes mellitus [ 82 ]. Besides stool microbiota, obese subjects with T2DM also showed a high bacterial load with an increase in Enterobacteriaceae in plasma, liver, and omental adipose tissue microbiota [ 83 ].

Lactobacillus was reported to be an obesity-associated taxon and abundance was higher in the stool of patients with obesity and metabolic diseases. This food-derived probiotic genus showed relative low prevalence and abundance in the commensal gut microbiota [ 52 ]. Previous clinical trials of Lactobacillus , alone or in combination with Bifidobacterium, showed variable efficacy in weight loss in patients with obesity [ 9 ]. These inconsistent results indicated that the underlying mechanisms of Lactobacillus (at least some of its species) in the treatment of metabolic disorders warrant further investigation. Other commensal bacteria such as Bifidobacterium spp., Alistipes spp., and Akkermansia that constitute a large proportion of the gut microbiota were frequently observed to be higher in healthy individuals than obese, metabolically affected subjects. These species might therefore exert a more durable beneficial effect for the consideration in managing obesity compared with Lactobacillus .

Akkermansia muciniphila (Actinobacteria phylum), a species identified by NGS, was one of the most commonly reported lean-associated bacteria in obesity and metabolic diseases. A. muciniphila was reported to help modulate the gut lining which could promote gut barrier function and prevent inflammation caused by the “leaky” gut [ 84 ]. A clinical trial demonstrated that supplementation with A. muciniphila could reduce body weight and decrease the level of blood markers for liver dysfunction and inflammation in obese insulin-resistant volunteers [ 10 ]. Another proof-of-concept study showed that supplementation with five strains including A. muciniphila was safe and associated with improved postprandial glucose control [ 85 ]. These findings highlight the potential of specific live biotherapeutics in weight control in subjects with obesity and metabolic diseases.

Other genera that were consistently reported to be more abundant in lean healthy individuals than obese subjects were Alistipes (Bacteroidetes phylum) and Faecalibacterium (Firmicutes phylum). Alistipes could produce small amounts of short-chain fatty acids (SCFA, acetic, isobutyric, isovaleric, and propionic acid) [ 86 ] while Faecalibacterium is one of the major butyrate producers in the human gut [ 87 , 88 ]. SCFA have anti-inflammatory properties [ 89 ] and may promote weight loss through the release of glucagon-like peptide 1 that promotes satiety and the activation of brown adipose tissue via the gut–brain neural circuit [ 90 , 91 ]. Butyrate could activate the GPR43-AKT-GSK3 signaling pathway to increase glucose metabolism by liver cells and improve glucose control in diabetes mice [ 92 ]. They could also inhibit the expression of PPARγ, increase fat oxidation in skeletal muscle mitochondria, and reduce lipogenesis in high-fat diet (HFD) mouse model [ 93 ].

We have identified several genera, including Bifidobacterium , Roseburia , Prevotella , and Ruminococcus, that were consistently reported to be lean-associated exclusively in subjects from the East. Bifidobacterium spp. are widely used probiotics proven to be safe and well-tolerated and exhibited a significant effect in lowering serum total cholesterol both in mice and in humans [ 94 ]. Roseburia is another major butyrate-producing genus of the human gut [ 95 ]. R. intestinalis could maintain the gut barrier function through upregulation of the tight junction protein [ 96 ]. Supplementation of R. intestinalis and R. hominis could ameliorate alcoholic fatty liver disease in mice [ 97 ]. Ruminococcus bromii is a keystone species for the degradation of resistant starch in the human colon [ 98 ]. Prevotella copri (Bacteroidetes phylum) was found to improve aberrant glucose tolerance syndromes and enhance hepatic glycogen storage in animals via the production of succinate [ 99 ]. However, a recent study also showed that the prevalence of P. copri exacerbated glucose tolerance and enhanced insulin resistance which occur before the development of ischemic cardiovascular disease and type 2 diabetes [ 100 ].

Only limited human studies in the current review reported an increased ratio of F/B in obesity. An increased ratio of F/B was shown in studies of the high-fat diet mouse model [ 6 ]. No taxon distinction was found to be specific for any type of metabolic disease. This was in line with a recent study that showed obesity, but not type 2 diabetes, was associated with notable alterations in microbiome composition [ 58 ].

The strength of this study is that we applied a robust method of grouping various types of disease-microbiome associations into “lean, metabolically healthy state” or “obese, metabolically diseased state.” Despite various metabolic disorders may affect the gut microbiota in different manners, the inter-study variation often supersedes the intra-study variation between disease and control groups [ 101 ]. Overall, the most striking observation is the lack of consistency in results between studies. This probably relates to the limitations of the studies included in this review. Also, it relies on the striking stability and individuality of adult microbiota, changing over time. Heterogeneity between studies is often a problem in systematic reviews. Several different methods were used to assess the microbiota, which makes it difficult to compare results between studies and likely contributes to the differences in results. While the standardization of study protocol (sample storage, DNA extraction, sequencing, analysis methods, and stringent subject recruitment criteria) could potentially result in comparable data between studies, this remains a big challenge across different regions. Moreover, we excluded studies that used species- or group-specific primers for microbiota assessment because such methods could only capture certain bacterial groups. This limits the total number of studies included. For robust microbiota results that are comparable among studies, there need to be efforts for standardization of sample storage, DNA extraction, sequencing, and analysis methods among groups undertaking gut microbiota studies. Finally, longitudinal studies would allow for a more robust association of changes in the microbiota to changes in obesity and metabolic disorders.

This systematic review identified consistent evidence for several lean-associated genera that may have therapeutic potential for obesity and metabolic diseases. Besides A. muciniphila , species from genera Faecalibacterium , Alistipes , and Roseburia might also harbor therapeutic potentials against obesity and metabolic diseases. These results provided a guide for the future development of certain bacteria into live biotherapeutics that may be helpful for the management of obesity and metabolic disorders. Further in-vitro and in-vivo research are needed to elucidate their role in the management of obesity and metabolic diseases.

Availability of data and materials

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.

Abbreviations

Type 2 diabetes

Non-alcoholic fatty liver disease

Next-generation sequencing

Body mass index

Firmicutes/Bacteroidetes

Non-alcoholic steatohepatitis

Obese with metabolic syndrome

Short-chain fatty acids

High-fat diet

Banack HR, Kaufman JS. The obesity paradox: understanding the effect of obesity on mortality among individuals with cardiovascular disease. Prev Med. 2014;62:96–102. https://doi.org/10.1016/j.ypmed.2014.02.003 .

Article   PubMed   Google Scholar  

WHO. Obesity and overweight, World Health Organization. 2017. http://www.who.int/en/news-room/fact-sheets/detail/obesity-and-overweight . Accessed 12 Jan 2018.

Turnbaugh PJ, Ley RE, Mahowald MA, Magrini V, Mardis ER, Gordon JI. An obesity-associated gut microbiome with increased capacity for energy harvest. Nature. 2006;444(7122):1027–31. https://doi.org/10.1038/nature05414 .

Turnbaugh PJ, Hamady M, Yatsunenko T, Cantarel BL, Duncan A, Ley RE, et al. A core gut microbiome in obese and lean twins. Nature. 2009;457(7228):480–4. https://doi.org/10.1038/nature07540 .

Article   CAS   PubMed   Google Scholar  

Castaner O, Goday A, Park YM, Lee SH, Magkos F, Shiow SATE, et al. The gut microbiome profile in obesity: a systematic review. Int J Endocrinol. 2018;2018:4095789. https://doi.org/10.1155/2018/4095789 .

Article   PubMed   PubMed Central   Google Scholar  

Bisanz JE, Upadhyay V, Turnbaugh JA, Ly K, Turnbaugh PJ. Meta-analysis reveals reproducible gut microbiome alterations in response to a high-fat diet. Cell Host Microbe. 2019;26(2):265–72.e4.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Allegretti JR, Kassam Z, Mullish BH, Chiang A, Carrellas M, Hurtado J, et al. Effects of Fecal Microbiota transplantation with oral capsules in obese patients. Clin Gastroenterol Hepatol. 2020;18(4):855–63.e2. https://doi.org/10.1016/j.cgh.2019.07.006 .

Yu EW, Gao L, Stastka P, Cheney MC, Mahabamunuge J, Torres Soto M, et al. Fecal microbiota transplantation for the improvement of metabolism in obesity: the FMT-TRIM double-blind placebo-controlled pilot trial. PLoS Med. 2020;17(3):e1003051. https://doi.org/10.1371/journal.pmed.1003051 .

Borgeraas H, Johnson LK, Skattebu J, Hertel JK, Hjelmesaeth J. Effects of probiotics on body weight, body mass index, fat mass and fat percentage in subjects with overweight or obesity: a systematic review and meta-analysis of randomized controlled trials. Obes Rev. 2018;19(2):219–32. https://doi.org/10.1111/obr.12626 .

Depommier C, Everard A, Druart C, Plovier H, van Hul M, Vieira-Silva S, et al. Supplementation with Akkermansia muciniphila in overweight and obese human volunteers: a proof-of-concept exploratory study. Nat Med. 2019;25(7):1096–103. https://doi.org/10.1038/s41591-019-0495-2 .

Nam YD, Jung MJ, Roh SW, Kim MS, Bae JW. Comparative analysis of Korean human gut microbiota by barcoded pyrosequencing. PLoS One. 2011;6(7):e22109. https://doi.org/10.1371/journal.pone.0022109 .

Deschasaux M, Bouter KE, Prodan A, et al. Depicting the composition of gut microbiota in a population with varied ethnic origins but shared geography. Nat Med. 2018;24(10):1526.

Yatsunenko T, Rey FE, Manary MJ, et al. Human gut microbiome viewed across age and geography. Nature. 2012;486(7402):222.

Hayashi H, Sakamoto M, Benno Y. Fecal microbial diversity in a strict vegetarian as determined by molecular analysis and cultivation. Microbiol Immunol. 2002;46(12):819–31. https://doi.org/10.1111/j.1348-0421.2002.tb02769.x .

Mueller S, Saunier K, Hanisch C, Norin E, Alm L, Midtvedt T, et al. Differences in fecal microbiota in different European study populations in relation to age, gender, and country: a cross-sectional study. Appl Environ Microbiol. 2006;72(2):1027–33. https://doi.org/10.1128/AEM.72.2.1027-1033.2006 .

Hollister EB, Riehle K, Luna RA, Weidler EM, Rubio-Gonzales M, Mistretta TA, et al. Structure and function of the healthy pre-adolescent pediatric gut microbiome. Microbiome. 2015;3(1):36. https://doi.org/10.1186/s40168-015-0101-x .

Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Bmj. 2009;339(1):b2535. https://doi.org/10.1136/bmj.b2535 .

Andoh A, Nishida A, Takahashi K, Inatomi O, Imaeda H, Bamba S, et al. Comparison of the gut microbial community between obese and lean peoples using 16S gene sequencing in a Japanese population. J Clin Biochem Nutr. 2016;59(1):65–70. https://doi.org/10.3164/jcbn.15-152 .

Bai J, Hu Y, Bruner DW. Composition of gut microbiota and its association with body mass index and lifestyle factors in a cohort of 7-18 years old children from the American Gut Project. Pediatr Obes. 2019;14(4):e12480. https://doi.org/10.1111/ijpo.12480 .

Chen X, Sun H, Jiang F, Shen Y, Li X, Hu X, et al. Alteration of the gut microbiota associated with childhood obesity by 16S rRNA gene sequencing. PeerJ. 2020;2020(1):8317. https://doi.org/10.7717/peerj.8317 .

Article   Google Scholar  

Da Silva CC, Monteil MA, Davis EM. Overweight and obesity in children are associated with an abundance of Firmicutes and Reduction of Bifidobacterium in their gastrointestinal microbiota. Child Obes (Print). 2020;16(3):204–10. https://doi.org/10.1089/chi.2019.0280 .

Gao R, Zhu C, Li H, Yin M, Pan C, Huang L, et al. Dysbiosis signatures of gut microbiota along the sequence from healthy, young patients to those with overweight and obesity. Obesity. 2018;26(2):351–61. https://doi.org/10.1002/oby.22088 .

Gao X, Jia R, Xie L, Kuang L, Feng L, Wan C. A study of the correlation between obesity and intestinal flora in school-age children. Sci Rep. 2018;8(1):14511. https://doi.org/10.1038/s41598-018-32730-6 .

Haro C, Rangel-Zuniga OA, Alcala-Diaz JF, et al. Intestinal microbiota is influenced by gender and body mass index. PLoS One. 2016;11(5):e0154090. https://doi.org/10.1371/journal.pone.0154090 .

Houttu N, Mokkala K, Laitinen K. Overweight and obesity status in pregnant women are related to intestinal microbiota and serum metabolic and inflammatory profiles. Clin Nutr. 2018;37(6):1955–66. https://doi.org/10.1016/j.clnu.2017.12.013 .

Hu H-J, Park S-G, Jang HB, Choi MG, Park KH, Kang JH, et al. Obesity alters the microbial community profile in Korean adolescents. PloS One. 2015;10(7):e0134333. https://doi.org/10.1371/journal.pone.0134333 .

Kaplan RC, Wang Z, Usyk M, Sotres-Alvarez D, Daviglus ML, Schneiderman N, et al. Gut microbiome composition in the Hispanic Community Health Study/Study of Latinos is shaped by geographic relocation, environmental factors, and obesity. Genome Biol. 2019;20(1):219. https://doi.org/10.1186/s13059-019-1831-z .

Liu R, Hong J, Feng Q, et al. Gut microbiome and serum metabolome alterations in obesity and after weight-loss intervention. Nat Med. 2017;23(7):859–68. https://doi.org/10.1038/nm.4358 .

Lopez-Contreras BE, Moran-Ramos S, Villarruel-Vazquez R, et al. Composition of gut microbiota in obese and normal-weight Mexican school-age children and its association with metabolic traits. Pediatr Obes. 2018;13(6):381–8. https://doi.org/10.1111/ijpo.12262 .

Lv Y, Qin X, Jia H, Chen S, Sun W, Wang X. The association between gut microbiota composition and BMI in Chinese male college students, as analysed by next-generation sequencing. Br J Nutr. 2019;122(9):986–95. https://doi.org/10.1017/S0007114519001909 .

Mendez-Salazar EO, Ortiz-Lopez MG, Granados-Silvestre MDLA, Palacios-Gonzalez B, Menjivar M. Altered gut microbiota and compositional changes in firmicutes and proteobacteria in Mexican undernourished and obese children. Front Microbiol. 2018;9:2494.

Nardelli C, Granata I, D’ Argenio V, et al. Characterization of the Duodenal mucosal microbiome in obese adult subjects by 16S rRNA sequencing. Microorganisms 2020; 8, 4, 485, https://doi.org/10.3390/microorganisms8040485 .

Blasco G, Moreno-Navarrete JM, Rivero M, Pérez-Brocal V, Garre-Olmo J, Puig J, et al. The gut metagenome changes in parallel to waist circumference, brain iron deposition, and cognitive function. J Clin Endocrinol Metab. 2017;102(8):2962–73. https://doi.org/10.1210/jc.2017-00133 .

Davis SC, Yadav JS, Barrow SD, Robertson BK. Gut microbiome diversity influenced more by the Westernized dietary regime than the body mass index as assessed using effect size statistic. MicrobiologyOpen. 2017;6(4):e00476. https://doi.org/10.1002/mbo3.476 .

Article   PubMed Central   Google Scholar  

Dominianni C, Sinha R, Goedert JJ, Pei Z, Yang L, Hayes RB, et al. Sex, body mass index, and dietary fiber intake influence the human gut microbiome. PLoS One. 2015;10(4):e0124599. https://doi.org/10.1371/journal.pone.0124599 .

Escobar JS, Klotz B, Valdes BE, Agudelo GM. The gut microbiota of Colombians differs from that of Americans, Europeans and Asians. BMC Microbiol. 2015;14(1):311. https://doi.org/10.1186/s12866-014-0311-6 .

Kasai C, Sugimoto K, Moritani I, Tanaka J, Oya Y, Inoue H, et al. Comparison of the gut microbiota composition between obese and non-obese individuals in a Japanese population, as analyzed by terminal restriction fragment length polymorphism and next-generation sequencing. BMC Gastroenterol. 2015;15(1):100. https://doi.org/10.1186/s12876-015-0330-2 .

Nirmalkar K, Murugesan S, Pizano-Zarate ML, et al. Gut microbiota and endothelial dysfunction markers in obese Mexican children and adolescents. Nutrients. 2018;10(12):2009. https://doi.org/10.3390/nu10122009 .

Article   CAS   PubMed Central   Google Scholar  

Ottosson F, Brunkwall L, Ericson U, Nilsson PM, Almgren P, Fernandez C, et al. Connection between BMI-related plasma metabolite profile and gut microbiota. J Clin Endocrinol Metab. 2018;103(4):1491–501. https://doi.org/10.1210/jc.2017-02114 .

Peters BA, Shapiro JA, Church TR, Miller G, Trinh-Shevrin C, Yuen E, et al. A taxonomic signature of obesity in a large study of American adults. Sci Rep. 2018;8(1):9749. https://doi.org/10.1038/s41598-018-28126-1 .

Ppatil D, Pdhotre D, Gchavan S, et al. Molecular analysis of gut microbiota in obesity among Indian individuals. J Biosci. 2012;37(4):647–57. https://doi.org/10.1007/s12038-012-9244-0 .

Article   CAS   Google Scholar  

Rahat-Rozenbloom S, Fernandes J, Gloor GB, Wolever TMS. Evidence for greater production of colonic short-chain fatty acids in overweight than lean humans. Int J Obes. 2014;38(12):1525–31. https://doi.org/10.1038/ijo.2014.46 .

Riva A, Borgo F, Lassandro C, Verduci E, Morace G, Borghi E, et al. Pediatric obesity is associated with an altered gut microbiota and discordant shifts in Firmicutes populations. Environ Microbiol. 2017;19(1):95–105. https://doi.org/10.1111/1462-2920.13463 .

Vieira-Silva S, Falony G, Belda E, et al. Statin therapy is associated with lower prevalence of gut microbiota dysbiosis. Nature. 2020;581(7808):310–5. https://doi.org/10.1038/s41586-020-2269-x .

Ville A, Levine E, Zhi D, Lararia B, Wojcicki JM. Alterations in the gut microbiome at 6 months of age in obese Latino infants. J Am Coll Nutr. 2020;39(1):47–53. https://doi.org/10.1080/07315724.2019.1606744 .

Yasir M, Angelakis E, Bibi F, Azhar EI, Bachar D, Lagier JC, et al. Comparison of the gut microbiota of people in France and Saudi Arabia. Nutr Diabetes. 2015;5(4):e153. https://doi.org/10.1038/nutd.2015.3 .

Yun Y, Kim HN, Kim SE, Heo SG, Chang Y, Ryu S, et al. Comparative analysis of gut microbiota associated with body mass index in a large Korean cohort. BMC Microbiol. 2017;17(1):151. https://doi.org/10.1186/s12866-017-1052-0 .

Zacarias MF, Collado MC, Gomez-Gallego C, et al. Pregestational overweight and obesity are associated with differences in gut microbiota composition and systemic inflammation in the third trimester. PloS One. 2018;13(7):e0200305. https://doi.org/10.1371/journal.pone.0200305 .

Allin KH, Tremaroli V, Caesar R, et al. Aberrant intestinal microbiota in individuals with prediabetes. Diabetologia. 2018;61(4):810–20. https://doi.org/10.1007/s00125-018-4550-1 .

Barengolts E, Green SJ, Eisenberg Y, et al. Gut microbiota varies by opioid use, circulating leptin and oxytocin in African American men with diabetes and high burden of chronic disease. PLoS One. 2018;13(3):e0194171.

Leite AZ, Rodrigues NC, Gonzaga MI, et al. Detection of increased plasma interleukin-6 levels and prevalence of Prevotella copri and Bacteroides vulgatus in the feces of type 2 diabetes patients. Front Immunol. 2017;8:1107.

Qin J, Li Y, Cai Z, Li S, Zhu J, Zhang F, et al. A metagenome-wide association study of gut microbiota in type 2 diabetes. Nature. 2012;490(7418):55–60. https://doi.org/10.1038/nature11450 .

Karlsson FH, Tremaroli V, Nookaew I, Bergström G, Behre CJ, Fagerberg B, et al. Gut metagenome in European women with normal, impaired and diabetic glucose control. Nature. 2013;498(7452):99–103. https://doi.org/10.1038/nature12198 .

Larsen N, Vogensen FK, van den Berg FW, et al. Gut microbiota in human adults with type 2 diabetes differs from non-diabetic adults. PLoS One. 2010;5(2):e9085. https://doi.org/10.1371/journal.pone.0009085 .

Ahmad A, Yang W, Chen G, Shafiq M, Javed S, Ali Zaidi SS, et al. Analysis of gut microbiota of obese individuals with type 2 diabetes and healthy individuals. PLoS One. 2019;14(12):e0226372. https://doi.org/10.1371/journal.pone.0226372 .

Koo SH, Chu CW, Khoo JJC, Cheong M, Soon GH, Ho EXP, et al. A pilot study to examine the association between human gut microbiota and the hostʼs central obesity. JGH Open. 2019;3(6):480–7. https://doi.org/10.1002/jgh3.12184 .

Sroka-oleksiak A, Mlodzinska A, Bulanda M, et al. Metagenomic analysis of duodenal microbiota reveals a potential biomarker of dysbiosis in the course of obesity and type 2 diabetes: a pilot study. J Clin Med. 2020;9(2):369. https://doi.org/10.3390/jcm9020369 .

Thingholm LB, Ruhlemann MC, Koch M, et al. Obese individuals with and without type 2 diabetes show different gut microbial functional capacity and composition. Cell Host Microbe. 2019;26(2):252–64.e10.

Zhao Y, Zhou J, Liu J, Wang Z, Chen M, Zhou S. Metagenome of gut microbiota of children with nonalcoholic fatty liver disease. Front Pediatr. 2019;7:518. https://doi.org/10.3389/fped.2019.00518 .

Jiang W, Wu N, Wang X, Chi Y, Zhang Y, Qiu X, et al. Dysbiosis gut microbiota associated with inflammation and impaired mucosal immune function in intestine of humans with non-alcoholic fatty liver disease. Sci Rep. 2015;5(1):8096. https://doi.org/10.1038/srep08096 .

Shen F, Zheng R-D, Sun X-Q, Ding W-J, Wang X-Y, Fan J-G. Gut microbiota dysbiosis in patients with non-alcoholic fatty liver disease. Hepatobiliary Pancreat Dis Int. 2017;16(4):375–81. https://doi.org/10.1016/S1499-3872(17)60019-5 .

Sobhonslidsuk A, Chanprasertyothin S, Pongrujikorn T, Kaewduang P, Promson K, Petraksa S, et al. The association of gut microbiota with nonalcoholic steatohepatitis in Thais. Biomed Res Int. 2018;2018:9340316–8. https://doi.org/10.1155/2018/9340316 .

Wang B, Jiang X, Cao M, Ge J, Bao Q, Tang L, et al. Altered fecal microbiota correlates with liver biochemistry in nonobese patients with non-alcoholic fatty liver disease. Sci Rep. 2016;6(1):32002. https://doi.org/10.1038/srep32002 .

Li F, Sun G, Wang Z, Wu W, Guo H, Peng L, et al. Characteristics of fecal microbiota in non-alcoholic fatty liver disease patients. Sci China Life Sci. 2018;61(7):770–8. https://doi.org/10.1007/s11427-017-9303-9 .

Nistal E, Saenz de Miera LE, Ballesteros PM, et al. An altered fecal microbiota profile in patients with non-alcoholic fatty liver disease (NAFLD) associated with obesity. Rev Esp Enferm Dig. 2019;111(4):275–82. https://doi.org/10.17235/reed.2019.6068/2018 .

Yun Y, Kim HN, Lee EJ, Ryu S, Chang Y, Shin H, et al. Fecal and blood microbiota profiles and presence of nonalcoholic fatty liver disease in obese versus lean subjects. PLoS One. 2019;14(3):e0213692. https://doi.org/10.1371/journal.pone.0213692 .

Michail S, Lin M, Frey MR, Fanter R, Paliy O, Hilbush B, et al. Altered gut microbial energy and metabolism in children with non-alcoholic fatty liver disease. FEMS Microbiol Ecol. 2015;91(2):1–9. https://doi.org/10.1093/femsec/fiu002 .

Zhu L, Baker SS, Gill C, Liu W, Alkhouri R, Baker RD, et al. Characterization of gut microbiomes in nonalcoholic steatohepatitis (NASH) patients: a connection between endogenous alcohol and NASH. Hepatology. 2013;57(2):601–9. https://doi.org/10.1002/hep.26093 .

Chavez-Carbajal A, Nirmalkar K, Perez-Lizaur A, et al. Gut microbiota and predicted metabolic pathways in a sample of Mexican women affected by obesity and obesity plus metabolic syndrome. Int J Mol Sci. 2019;20(2):438. https://doi.org/10.3390/ijms20020438 .

De La Cuesta-Zuluaga J, Corrales-Agudelo V, Carmona JA, Abad JM, Escobar JS. Body size phenotypes comprehensively assess cardiometabolic risk and refine the association between obesity and gut microbiota. Int J Obes. 2018;42(3):424–32. https://doi.org/10.1038/ijo.2017.281 .

Gallardo-Becerra L, Cornejo-Granados F, Garcia-Lopez R, et al. Metatranscriptomic analysis to define the Secrebiome, and 16S rRNA profiling of the gut microbiome in obesity and metabolic syndrome of Mexican children. Microb Cell Factories. 2020;19(1):61. https://doi.org/10.1186/s12934-020-01319-y .

Gozd-Barszczewska A, Koziol-Montewka M, Barszczewski P, Mlodzinska A, Huminska K. Gut microbiome as a biomarker of cardiometabolic disorders. Ann Agric Environ Med. 2017;24(3):416–22. https://doi.org/10.26444/aaem/75456 .

Kashtanova DA, Tkacheva ON, Doudinskaya EN, Strazhesko I, Kotovskaya Y, Popenko A, et al. Gut microbiota in patients with different metabolic statuses: Moscow study. Microorganisms. 2018;6(4):98. https://doi.org/10.3390/microorganisms6040098 .

Lippert K, Kedenko L, Antonielli L, Kedenko I, Gemeier C, Leitner M, et al. Gut microbiota dysbiosis associated with glucose metabolism disorders and the metabolic syndrome in older adults. Benefic Microbes. 2017;8(4):545–56. https://doi.org/10.3920/BM2016.0184 .

Feinn R, Kravetz AM, Galuppo B, et al. Effect of gut microbiota and PNPLA3 rs738409 variant on nonalcoholic fatty liver disease (NAFLD) in obese youth. J Clin Endocrinol Metab. 2020;105(10):e3585. https://doi.org/10.1210/clinem/dgaa382 .

Li R, Liang X, Su M, Lai KP, Chen J, Huang X. Integrated omics analysis reveals the alteration of gut microbe-metabolites in obese adults. Brief Bioinforma. 2021;22(3):bbaa165. https://doi.org/10.1093/bib/bbaa165 .

Yuan X, Chen R, Zhang Y, Lin X, Yang X, McCormick KL. The role of the gut microbiota on the metabolic status of obese children. Microb Cell Factories. 2021;20(1):53. https://doi.org/10.1186/s12934-021-01548-9 .

Gomez-Acebo I, Dierssen-Sotos T, de Pedro M, et al. Epidemiology of non-steroidal anti-inflammatory drugs consumption in Spain. The MCC-Spain study. BMC Public Health. 2018;18(1):1134.

Manosalva AGL, Yntema T, Chen L, Garmaeva S, Hu S, Koster M, et al. Omeprazole-induced dysbiosis impacts bile acid metabolism in mice and humans. Atherosclerosis. 2019;287:e120. https://doi.org/10.1016/j.atherosclerosis.2019.06.349 .

Rhodes JM. The role of Escherichia coli in inflammatory bowel disease. Gut. 2007;56(5):610–2. https://doi.org/10.1136/gut.2006.111872 .

Zhang J, Hoedt EC, Liu Q, et al. Elucidation of Proteus mirabilis as a key bacterium in Crohnʼs disease inflammation. Gastroenterology. 2021;160(1):317–30.e11.

Ellulu MS, Patimah I, Khazaʼai H, Rahmat A, Abed Y. Obesity and inflammation: the linking mechanism and the complications. Arch Med Sci. 2017;13(4):851–63. https://doi.org/10.5114/aoms.2016.58928 .

Anhe FF, Jensen BAH, Varin TV, et al. Type 2 diabetes influences bacterial tissue compartmentalisation in human obesity. Nat Metab. 2020;2(3):233–42. https://doi.org/10.1038/s42255-020-0178-9 .

Ansaldo E, Slayden LC, Ching KL, Koch MA, Wolf NK, Plichta DR, et al. Akkermansia muciniphila induces intestinal adaptive immune responses during homeostasis. Science. 2019;364(6446):1179–84. https://doi.org/10.1126/science.aaw7479 .

Perraudeau F, McMurdie P, Bullard J, Cheng A, Cutcliffe C, Deo A, et al. Improvements to postprandial glucose control in subjects with type 2 diabetes: a multicenter, double blind, randomized placebo-controlled trial of a novel probiotic formulation. Bmj Open Diab Res Ca. 2020;8(1):e001319. https://doi.org/10.1136/bmjdrc-2020-001319 .

Parker BJ, Wearsch PA, Veloo ACM, Rodriguez-Palacios A. The genus Alistipes: gut bacteria with emerging implications to inflammation, cancer, and mental health. Front Immunol. 2020;11:906. https://doi.org/10.3389/fimmu.2020.00906 .

Walker AW, Duncan SH, Louis P, Flint HJ. Phylogeny, culturing, and metagenomics of the human gut microbiota. Trends Microbiol. 2014;22(5):267–74. https://doi.org/10.1016/j.tim.2014.03.001 .

De Vuyst L, Leroy F. Cross-feeding between bifidobacteria and butyrate-producing colon bacteria explains bifdobacterial competitiveness, butyrate production, and gas production. Int J Food Microbiol. 2011;149(1):73–80. https://doi.org/10.1016/j.ijfoodmicro.2011.03.003 .

Oliphant K, Allen-Vercoe E. Macronutrient metabolism by the human gut microbiome: Major fermentation by-products and their impact on host health. Microbiome. 2019;7(1):91. https://doi.org/10.1186/s40168-019-0704-8 .

Canfora EE, Jocken JW, Blaak EE. Short-chain fatty acids in control of body weight and insulin sensitivity. Nat Rev Endocrinol. 2015;11(10):577–91. https://doi.org/10.1038/nrendo.2015.128 .

Li Z, Yi CX, Katiraei S, Kooijman S, Zhou E, Chung CK, et al. Butyrate reduces appetite and activates brown adipose tissue via the gut-brain neural circuit. Gut. 2018;67(7):1269–79. https://doi.org/10.1136/gutjnl-2017-314050 .

Zhang WQ, Zhao TT, Gui DK, Gao CL, Gu JL, Gan WJ, et al. Sodium butyrate improves liver glycogen metabolism in type 2 diabetes mellitus. J Agric Food Chem. 2019;67(27):7694–705. https://doi.org/10.1021/acs.jafc.9b02083 .

den Besten G, Bleeker A, Gerding A, van Eunen K, Havinga R, van Dijk TH, et al. Short-chain fatty acids protect against high-fat diet-induced obesity via a PPARgamma-dependent switch from lipogenesis to fat oxidation. Diabetes. 2015;64(7):2398–408. https://doi.org/10.2337/db14-1213 .

Xiao JZ, Kondo S, Takahashi N, Miyaji K, Oshida K, Hiramatsu A, et al. Effects of milk products fermented by Bifidobacterium longum on blood lipids in rats and healthy adult male volunteers. J Dairy Sci. 2003;86(7):2452–61. https://doi.org/10.3168/jds.S0022-0302(03)73839-9 .

Duncan SH, Hold GL, Barcenilla A, Stewart CS, Flint HJ. Roseburia intestinalis sp. nov., a novel saccharolytic, butyrate-producing bacterium from human faeces. Int J Syst Evol Microbiol. 2002;52(Pt 5):1615–20. https://doi.org/10.1099/00207713-52-5-1615 .

Hiippala K, Jouhten H, Ronkainen A, et al. The potential of gut commensals in reinforcing intestinal barrier function and alleviating inflammation. Nutrients. 2018;10(8):988. https://doi.org/10.3390/nu10080988 .

Seo B, Jeon K, Moon S, et al. Roseburia spp. abundance associates with alcohol consumption in humans and its administration ameliorates alcoholic fatty liver in mice. Cell Host Microbe. 2020;27(1):25.

Ze X, Duncan SH, Louis P, Flint HJ. Ruminococcus bromii is a keystone species for the degradation of resistant starch in the human colon. Isme j. 2012;6(8):1535–43. https://doi.org/10.1038/ismej.2012.4 .

De Vadder F, Kovatcheva-Datchary P, Zitoun C, Duchampt A, Bäckhed F, Mithieux G. Microbiota-produced succinate improves glucose homeostasis via intestinal gluconeogenesis. Cell Metab. 2016;24(1):151–7. https://doi.org/10.1016/j.cmet.2016.06.013 .

Pedersen HK, Gudmundsdottir V, Nielsen HB, Hyotylainen T, Nielsen T, Jensen BAH, et al. Human gut microbes impact host serum metabolome and insulin sensitivity. Nature. 2016;535(7612):376–81. https://doi.org/10.1038/nature18646 .

Sze MA, Schloss PD. Looking for a signal in the noise: revisiting obesity and the microbiome. MBio. 2016;7(4):e01018-16.

Download references

Acknowledgements

Not applicable.

It was funded by InnoHK, The Government of Hong Kong, Special Administrative Region of the People’s Republic of China.

Author information

Zhilu XU and Wei JIANG contributed equally to this work.

Authors and Affiliations

Department of Medicine and Therapeutics, Institute of Digestive Disease, State Key Laboratory of Digestive Diseases, LKS Institute of Health Science, The Chinese University of Hong Kong, Hong Kong, China

Zhilu XU, Wei JIANG, Wenli HUANG, Yu LIN, Francis K.L. CHAN & Siew C. NG

Center for Gut microbiota research, Faculty of Medicine, The Chinese University of Hong Kong, Hong Kong, China

Zhilu XU, Wenli HUANG, Yu LIN, Francis K.L. CHAN & Siew C. NG

Microbiota Innovation Centre (MagIC Centre), Hong Kong, China

You can also search for this author in PubMed   Google Scholar

Contributions

All authors contributed to the analysis and interpretation of data, the writing and critical revision of manuscript at all stages of development. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Siew C. NG .

Ethics declarations

Ethics approval and consent to participate.

Not applicable

Consent for publication

Competing interests.

Francis Chan and Siew Ng are co-founders and in the board of GenieBiome Ltd. Siew Ng has served as an advisory board member for Pfizer, Ferring, Janssen, and Abbvie and a speaker for Ferring, Tillotts, Menarini, Janssen, Abbvie, and Takeda. She has received research grants from Olympus, Ferring, and Abbvie. Francis Chan has served as an advisor and lecture speaker for Eisai Co. Ltd., AstraZeneca, Pfizer Inc., Takeda Pharmaceutical Co., and Takeda (China) Holdings Co. Ltd. XU Zhilu is an employee of GenieBiome Ltd. All other authors declare that there are no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1..

Supplementary Table 1. Differentially abundant taxa at each taxonomic level in patients with obesity and metabolic diseases reported in individual studies. Supplementary Table 2. Differentially abundant taxa at class, order, and family level in obesity / metabolic diseases. Supplementary Table 3. Microbiota diversity and F/B Ratio in Obesity / metabolic diseases.

Appendix. Searching strategy

1 obese.mp. [mp=ti, ab, hw, tn, ot, dm, mf, dv, kw, fx, dq, nm, kf, ox, px, rx, an, ui, sy]

2 obesity.mp. [mp=ti, ab, hw, tn, ot, dm, mf, dv, kw, fx, dq, nm, kf, ox, px, rx, an, ui, sy]

3 overweight.mp. [mp=ti, ab, hw, tn, ot, dm, mf, dv, kw, fx, dq, nm, kf, ox, px, rx, an, ui, sy]

4 microbiota.mp. [mp=ti, ab, hw, tn, ot, dm, mf, dv, kw, fx, dq, nm, kf, ox, px, rx, an, ui, sy]

5 microbiome.mp. [mp=ti, ab, hw, tn, ot, dm, mf, dv, kw, fx, dq, nm, kf, ox, px, rx, an, ui, sy]

6 fecal.mp. [mp=ti, ab, hw, tn, ot, dm, mf, dv, kw, fx, dq, nm, kf, ox, px, rx, an, ui, sy]

7 faecal.mp. [mp=ti, ab, hw, tn, ot, dm, mf, dv, kw, fx, dq, nm, kf, ox, px, rx, an, ui, sy]

8 gut.mp. [mp=ti, ab, hw, tn, ot, dm, mf, dv, kw, fx, dq, nm, kf, ox, px, rx, an, ui, sy]

9 intestinal.mp. [mp=ti, ab, hw, tn, ot, dm, mf, dv, kw, fx, dq, nm, kf, ox, px, rx, an, ui, sy]

10 1 or 2 or 3

12 6 or 7 or 8 or 9

13 10 and 11 and 12

14 metagenomics.mp. [mp=ti, ab, hw, tn, ot, dm, mf, dv, kw, fx, dq, nm, kf, ox, px, rx, an, ui, sy]

15 metagenomic.mp. [mp=ti, ab, hw, tn, ot, dm, mf, dv, kw, fx, dq, nm, kf, ox, px, rx, an, ui, sy]

16 16s.mp. [mp=ti, ab, hw, tn, ot, dm, mf, dv, kw, fx, dq, nm, kf, ox, px, rx, an, ui, sy]

17 14 or 15 or 16

18 13 and 17

19 remove duplicates from 18

20 metabolic disease.mp. [mp=ti, ab, hw, tn, ot, dm, mf, dv, kw, fx, dq, nm, kf, ox, px, rx, an, ui, sy]

21 10 or 20

22 11 and 12 and 21

23 17 and 22

24 remove duplicates from 23

25 limit 24 to full text

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

XU, Z., JIANG, W., HUANG, W. et al. Gut microbiota in patients with obesity and metabolic disorders — a systematic review. Genes Nutr 17 , 2 (2022). https://doi.org/10.1186/s12263-021-00703-6

Download citation

Received : 23 August 2021

Accepted : 23 November 2021

Published : 29 January 2022

DOI : https://doi.org/10.1186/s12263-021-00703-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Metabolic disorder

Genes & Nutrition

ISSN: 1865-3499

• General enquiries: [email protected]

• Open access
• Published: 23 August 2022

Prognostic risk factors for moderate-to-severe exacerbations in patients with chronic obstructive pulmonary disease: a systematic literature review

• John R. Hurst 1 ,
• MeiLan K. Han 2 ,
• Barinder Singh 3 ,
• Sakshi Sharma 4 ,
• Gagandeep Kaur 3 ,
• Enrico de Nigris 5 ,
• Ulf Holmgren 6 &
• Mohd Kashif Siddiqui 3  

Respiratory Research volume  23 , Article number:  213 ( 2022 ) Cite this article

6342 Accesses

20 Citations

33 Altmetric

Metrics details

Chronic obstructive pulmonary disease (COPD) is a leading cause of morbidity and mortality worldwide. COPD exacerbations are associated with a worsening of lung function, increased disease burden, and mortality, and, therefore, preventing their occurrence is an important goal of COPD management. This review was conducted to identify the evidence base regarding risk factors and predictors of moderate-to-severe exacerbations in patients with COPD.

A literature review was performed in Embase, MEDLINE, MEDLINE In-Process, and the Cochrane Central Register of Controlled Trials (CENTRAL). Searches were conducted from January 2015 to July 2019. Eligible publications were peer-reviewed journal articles, published in English, that reported risk factors or predictors for the occurrence of moderate-to-severe exacerbations in adults age ≥ 40 years with a diagnosis of COPD.

The literature review identified 5112 references, of which 113 publications (reporting results for 76 studies) met the eligibility criteria and were included in the review. Among the 76 studies included, 61 were observational and 15 were randomized controlled clinical trials. Exacerbation history was the strongest predictor of future exacerbations, with 34 studies reporting a significant association between history of exacerbations and risk of future moderate or severe exacerbations. Other significant risk factors identified in multiple studies included disease severity or bronchodilator reversibility (39 studies), comorbidities (34 studies), higher symptom burden (17 studies), and higher blood eosinophil count (16 studies).

Conclusions

This systematic literature review identified several demographic and clinical characteristics that predict the future risk of COPD exacerbations. Prior exacerbation history was confirmed as the most important predictor of future exacerbations. These prognostic factors may help clinicians identify patients at high risk of exacerbations, which are a major driver of the global burden of COPD, including morbidity and mortality.

Chronic obstructive pulmonary disease (COPD) is the third leading cause of death worldwide [ 1 ]. Based upon disability-adjusted life-years, COPD ranked sixth out of 369 causes of global disease burden in 2019 [ 2 ]. COPD exacerbations are associated with a worsening of lung function, and increased disease burden and mortality (of those patients hospitalized for the first time with an exacerbation, > 20% die within 1 year of being discharged) [ 3 ]. Furthermore, patients with COPD consider exacerbations or hospitalization due to exacerbations to be the most important disease outcome, having a large impact on their lives [ 4 ]. Therefore, reducing the future risk of COPD exacerbations is a key goal of COPD management [ 5 ].

Being able to predict the level of risk for each patient allows clinicians to adapt treatment and patients to adjust their lifestyle (e.g., through a smoking cessation program) to prevent exacerbations [ 3 ]. As such, identifying high-risk patients using measurable risk factors and predictors that correlate with exacerbations is critical to reduce the burden of disease and prevent a cycle of decline encompassing irreversible lung damage, worsening quality of life (QoL), increasing disease burden, high healthcare costs, and early death.

Prior history of exacerbations is generally thought to be the best predictor of future exacerbations; however, there is a growing body of evidence suggesting other demographic and clinical characteristics, including symptom burden, airflow obstruction, comorbidities, and inflammatory biomarkers, also influence risk [ 6 , 7 , 8 , 9 ]. For example, in the prospective ECLIPSE observational study, the likelihood of patients experiencing an exacerbation within 1 year of follow-up increased significantly depending upon several factors, including prior exacerbation history, forced expiratory volume in 1 s (FEV 1 ), St. George’s Respiratory Questionnaire (SGRQ) score, gastroesophageal reflux, and white blood cell count [ 9 ].

Many studies have assessed predictors of COPD exacerbations across a variety of countries and patient populations. This systematic literature review (SLR) was conducted to identify and compile the evidence base regarding risk factors and predictors of moderate-to-severe exacerbations in patients with COPD.

• Systematic literature review

A comprehensive search strategy was designed to identify English-language studies published in peer-reviewed journals providing data on risk factors or predictors of moderate or severe exacerbations in adults aged ≥ 40 years with a diagnosis of COPD (sample size ≥ 100). The protocol is summarized in Table 1 and the search strategy is listed in Additional file 1 : Table S1. Key biomedical electronic literature databases were searched from January 2015 until July 2019. Other sources were identified via bibliographic searching of relevant systematic reviews.

Study selection process

Implementation and reporting followed the recommendations and standards of the Preferred Reporting Items for Systematic reviews and Meta-analyses (PRISMA) statement [ 10 ]. An independent reviewer conducted the first screening based on titles and abstracts, and a second reviewer performed a quality check of the excluded evidence. A single independent reviewer also conducted the second screening based on full-text articles, with a quality check of excluded evidence performed by a second reviewer. Likewise, data tables of the included studies were generated by one reviewer, and another reviewer performed a quality check of extracted data. Where more than one publication was identified describing a single study or trial, data were compiled into a single entry in the data-extraction table to avoid double counting of patients and studies. One publication was designated as the ‘primary publication’ for the purposes of the SLR, based on the following criteria: most recently published evidence and/or the article that presented the majority of data (e.g., journal articles were preferred over conference abstracts; articles that reported results for the full population were preferred over later articles providing results of subpopulations). Other publications reporting results from the same study were designated as ‘linked publications’; any additional data in the linked publications that were not included in the primary publication were captured in the SLR. Conference abstracts were excluded from the SLR unless they were a ‘linked publication.’

Included studies

A total of 5112 references (Fig.  1 ) were identified from the database searches. In total, 76 studies from 113 publications were included in the review. Primary publications and ‘linked publications’ for each study are detailed in Additional file 1 : Table S2, and study characteristics are shown in Additional file 1 : Table S3. The studies included clinical trials, registry studies, cross-sectional studies, cohort studies, database studies, and case–control studies. All 76 included studies were published in peer-reviewed journals. Regarding study design, 61 of the studies were observational (34 retrospective observational studies, 19 prospective observational studies, four cross-sectional studies, two studies with both retrospective and prospective cohort data, one case–control study, and one with cross-sectional and longitudinal data) and 15 were randomized controlled clinical trials.

PRISMA flow diagram of studies through the systematic review process. CA conference abstract, CENTRAL Cochrane Central Register of Controlled Trials, PRISMA  Preferred Reporting Items for Systematic Reviews and Meta-Analyses

Of the 76 studies, 16 were conducted in North America (13 studies in the USA, two in Canada, and one in Mexico); 26 were conducted in Europe (seven studies in Spain, four in the UK, three in Denmark, two studies each in Bulgaria, the Netherlands, and Switzerland, and one study each in Sweden, Serbia, Portugal, Greece, Germany, and France) and 17 were conducted in Asia (six studies in South Korea, four in China, three in Taiwan, two in Japan, and one study each in Singapore and Israel). One study each was conducted in Turkey and Australia. Fifteen studies were conducted across multiple countries.

The majority of the studies (n = 54) were conducted in a multicenter setting, while 22 studies were conducted in a single-center setting. The sample size among the included studies varied from 118 to 339,389 patients.

Patient characteristics

A total of 75 studies reported patient characteristics (Additional file 1 : Table S4). The mean age was reported in 65 studies and ranged from 58.0 to 75.2 years. The proportion of male patients ranged from 39.7 to 97.6%. The majority of included studies (85.3%) had a higher proportion of males than females.

Exacerbation history (as defined per each study) was reported in 18 of 76 included studies. The proportion of patients with no prior exacerbation was reported in ten studies (range, 0.1–79.5% of patients), one or fewer prior exacerbation in ten studies (range, 46–100%), one or more prior exacerbation in eight studies (range, 18.4–100%), and two or more prior exacerbations in 12 studies (range, 6.1–55.0%).

Prognostic factors of exacerbations

A summary of the risk factors and predictors reported across the included studies is provided in Tables 2 and 3 . The overall findings of the SLR are summarized in Figs. 2 and 3 .

Risk factors for moderate-to-severe exacerbations in patients with COPD. Factors with > 30 supporting studies shown as large circles; factors with ≤ 30 supporting studies shown as small circles and should be interpreted cautiously. BDR bronchodilator reversibility, BMI body mass index, COPD chronic obstructive pulmonary disease, EOS eosinophil, QoL quality of life

Summary of risk factors for exacerbation events. a Treatment impact studies removed. BDR bronchodilator reversibility, BMI body mass index, COPD chronic obstructive pulmonary disease, EOS eosinophil, QoL quality of life

Exacerbation history within the past 12 months was the strongest predictor of future exacerbations. Across the studies assessing this predictor, 34 out of 35 studies (97.1%) reported a significant association between history of exacerbations and risk of future moderate-to-severe exacerbations (Table 3 ). Specifically, two or more exacerbations in the previous year or at least one hospitalization for COPD in the previous year were identified as reliable predictors of future moderate or severe exacerbations. Even one moderate exacerbation increased the risk of a future exacerbation, with the risk increasing further with each subsequent exacerbation (Fig.  4 ). A severe exacerbation was also found to increase the risk of subsequent exacerbation and hospitalization (Fig.  5 ). Patients experiencing one or more severe exacerbations were more likely to experience further severe exacerbations than moderate exacerbations [ 11 , 12 ]. In contrast, patients with a history of one or more moderate exacerbations were more likely to experience further moderate exacerbations than severe exacerbations [ 11 , 12 ].

Exacerbation history as a risk factor for moderate-to-severe exacerbations. Yun 2018 included two studies; the study from which data were extracted (COPDGene or ECLIPSE) is listed in parentheses. CI confidence interval, ES effect size

Exacerbation history as a risk factor for severe exacerbations. Where data have been extracted from a linked publication rather than the primary publication, the linked publication is listed in parentheses. CI confidence interval, ES , effect size

Overall, 35 studies assessed the association of comorbidities with the risk of exacerbation. All studies except one (97.1%) reported a positive association between comorbidities and the occurrence of moderate-to-severe exacerbations (Table 3 ). In addition to the presence of any comorbidity, specific comorbidities that were found to significantly increase the risk of moderate-to-severe exacerbations included anxiety and depression, cardiovascular comorbidities, gastroesophageal reflux disease/dyspepsia, and respiratory comorbidities (Fig.  6 ). Comorbidities that were significant risk factors for severe exacerbations included cardiovascular, musculoskeletal, and respiratory comorbidities, diabetes, and malignancy (Fig.  7 ). Overall, the strongest association between comorbidities and COPD readmissions in the emergency department was with cardiovascular disease. The degree of risk for both moderate-to-severe and severe exacerbations also increased with the number of comorbidities. A Dutch cohort study found that 88% of patients with COPD had at least one comorbidity, with hypertension (35%) and coronary heart disease (19%) being the most prevalent. In this cohort, the comorbidities with the greatest risk of frequent exacerbations were pulmonary cancer (odds ratio [OR] 1.85) and heart failure (OR 1.72) [ 7 ].

Comorbidities as risk factors for moderate-to-severe exacerbations. Yun 2018 included two studies; the study from which data were extracted (COPDGene or ECLIPSE) is listed in parentheses. Where data have been extracted from a linked publication rather than the primary publication, the linked publication is listed in parentheses. CI confidence interval, ES effect size, GERD gastroesophageal disease

Comorbidities as risk factors for severe exacerbations. Where data have been extracted from a linked publication rather than the primary publication, the linked publication is listed in parentheses. CI confidence interval, CKD , chronic kidney disease, ES effect size

The majority of studies assessing disease severity or bronchodilator reversibility (39/41; 95.1%) indicated a significant positive relation between risk of future exacerbations and greater disease severity, as assessed by greater lung function impairment (in terms of lower FEV 1 , FEV 1 /forced vital capacity ratio, or forced expiratory flow [25–75]/forced vital capacity ratio) or more severe Global Initiative for Chronic Obstructive Lung Disease (GOLD) class A − D, and a positive relationship between risk of future exacerbations and lack of bronchodilator reversibility (Table 3 , Figs. 8 and 9 ).

Disease severity as a risk factor for moderate-to-severe exacerbations. Yun 2018 included two studies; the study from which data were extracted (COPDGene or ECLIPSE) is listed in parentheses. Where data have been extracted from a linked publication rather than the primary publication, the linked publication is listed in parentheses. CI confidence interval, ES effect size, FEV 1 f orced expiratory volume in 1 s, FVC , forced vital capacity, GOLD Global Initiative for Obstructive Lung Disease, HR hazard ratio, OR odds ratio

Disease severity and BDR as risk factors for severe exacerbations. ACCP American College of Chest Physicians, ACOS Asthma-COPD overlap syndrome, ATS  American Thoracic Society, BDR bronchodilator reversibility, CI confidence interval, ERS  European Respiratory Society, ES effect size, FEV 1 forced expiratory volume in 1 s, FVC  forced vital capacity, GINA Global Initiative for Asthma, GOLD Global Initiative for Obstructive Lung Disease

Of 21 studies assessing the relationship between blood eosinophil count and exacerbations (Table 3 ), 16 reported estimates for the risk of moderate or severe exacerbations by eosinophil count. A positive association was observed between higher eosinophil count and a higher risk of moderate or severe exacerbations, particularly in patients not treated with an inhaled corticosteroid (ICS); however, five studies reported a significant positive association irrespective of intervention effects. The risk of moderate-to-severe exacerbations was observed to be positively associated with various definitions of higher eosinophil levels (absolute counts: ≥ 200, ≥ 300, ≥ 340, ≥ 400, and ≥ 500 cells/mm 3 ; % of blood eosinophil count: ≥ 2%, ≥ 3%, ≥ 4%, and ≥ 5%). Of note, one study found reduced efficacy of ICS in lowering moderate-to-severe exacerbation rates for current smokers versus former smokers at all eosinophil levels [ 13 ].

Of 12 studies assessing QoL scales, 11 (91.7%) studies reported a significant association between the worsening of QoL scores and the risk of future exacerbations (Table 3 ). Baseline SGRQ [ 14 , 15 ], Center for Epidemiologic Studies Depression Scale (for which increased scores may indicate impaired QoL) [ 16 ], and Clinical COPD Questionnaire [ 17 , 18 ] scores were found to be associated with future risk of moderate and/or severe COPD exacerbations. For symptom scores, six out of eight studies assessing the association between moderate-to-severe or severe exacerbations with COPD Assessment Test (CAT) scores reported a significant and positive relationship. Furthermore, the risk of moderate-to-severe exacerbations was found to be significantly higher in patients with higher CAT scores (≥ 10) [ 15 , 19 , 20 , 21 ], with one study demonstrating that a CAT score of 15 increased predictive ability for exacerbations compared with a score of 10 or more [ 18 ]. Among 15 studies that assessed the association of modified Medical Research Council (mMRC) scores with the risk of moderate-to-severe or severe exacerbation, 11 found that the risk of moderate-to-severe or severe exacerbations was significantly associated with higher mMRC scores (≥ 2) versus lower scores. Furthermore, morning and night symptoms (measured by Clinical COPD Questionnaire) were associated with poor health status and predicted future exacerbations [ 17 ].

Of 36 studies reporting the relationship between smoking status and moderate-to-severe or severe exacerbations, 22 studies (61.1%) reported a significant positive association (Table 3 ). Passive smoking was also significantly associated with an increased risk of severe exacerbations (OR 1.49) [ 20 ]. Of note, three studies reported a significantly lower rate of moderate-to-severe exacerbations in current smokers compared with former smokers [ 22 , 23 , 24 ].

A total of 14 studies assessed the association of body mass index (BMI) with the occurrence of frequent moderate-to-severe exacerbations in patients with COPD. Six out of 14 studies (42.9%) reported a significant negative association between exacerbations and BMI (Table 3 ). The risk of moderate and/or severe COPD exacerbations was highest among underweight patients compared with normal and overweight patients [ 23 , 25 , 26 , 27 , 28 ].

In the 29 studies reporting an association between age and moderate or severe exacerbations, more than half found an association of older age with an increased risk of moderate-to-severe exacerbations (58.6%; Table 3 ). Four of these studies noted a significant increase in the risk of moderate-to-severe or severe exacerbations for every 10-year increase in age [ 25 , 26 , 29 , 30 ]. However, 12 studies reported no significant association between age and moderate-to-severe or severe exacerbation risk.

Sixteen out of 33 studies investigating the impact of sex on exacerbation risk found a significant association (48.5%; Table 3 ). Among these, ten studies reported that female sex was associated with an increased risk of moderate-to-severe exacerbations, while six studies showed a higher exacerbation risk in males compared with females. There was some variation in findings by geographic location and exacerbation severity (Additional file 2 : Figs. S1 and S2). Notably, when assessing the risk of severe exacerbations, more studies found an association with male sex compared with female sex (6/13 studies vs 1/13 studies, respectively).

Both studies evaluating associations between exacerbations and environmental factors reported that colder temperature and exposure to major air pollution (NO 2 , O 3 , CO, and/or particulate matter ≤ 10 μm in diameter) increased hospital admissions due to severe exacerbations and moderate-to-severe exacerbation rates [ 31 , 32 ].

Four studies assessed the association of 6-min walk distance with the occurrence of frequent moderate-to-severe exacerbations (Table 3 ). One study (25.0%) found that shorter 6-min walk distance (representing low physical activity) was significantly associated with a shortened time to severe exacerbation, but the effect size was small (hazard ratio 0.99) [ 33 ].

Five out of six studies assessing the relationship between race or ethnicity and exacerbation risk reported significant associations (Table 3 ). Additionally, one study reported an association between geographic location in the US and exacerbations, with living in the Northeast region being the strongest predictor of severe COPD exacerbations versus living in the Midwest and South regions [ 34 ].

Overall, seven studies assessed the association of biomarkers with risk of future exacerbations (Table 3 ), with the majority identifying significant associations between inflammatory biomarkers and increased exacerbation risk, including higher C-reactive protein levels [ 8 , 35 ], fibrinogen levels [ 8 , 30 ], and white blood cell count [ 8 , 15 , 16 ].

This SLR has identified several demographic and clinical characteristics that predict the future risk of COPD exacerbations. Key factors associated with an increased risk of future moderate-to-severe exacerbations included a history of prior exacerbations, worse disease severity and bronchodilator reversibility, the presence of comorbidities, a higher eosinophil count, and older age (Fig.  2 ). These prognostic factors may help clinicians identify patients at high risk of exacerbations, which are a major driver of the burden of COPD, including morbidity and mortality [ 36 ].

Findings from this review summarize the existing evidence, validating the previously published literature [ 6 , 9 , 23 ] and suggesting that the best predictor of future exacerbations is a history of exacerbations in the prior year [ 8 , 11 , 12 , 13 , 14 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 26 , 29 , 34 , 35 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 , 60 ]. In addition, the effect size generally increased with the number of prior exacerbations, with a stronger effect observed with prior severe versus moderate exacerbations. This effect was observed across regions, including in Europe and North America, and in several global studies. This relationship represents a vicious circle, whereby one exacerbation predisposes a patient to experience future exacerbations and leading to an ever-increasing disease burden, and emphasizes the importance of preventing the first exacerbation event through early, proactive exacerbation prevention. The finding that prior exacerbations tended to be associated with future exacerbations of the same severity suggests that the severity of the underlying disease may influence exacerbation severity. However, the validity of the traditional classification of exacerbation severity has recently been challenged [ 61 ], and further work is required to understand relationships with objective assessments of exacerbation severity.

In addition to exacerbation history, disease severity and bronchodilator reversibility were also strong predictors for future exacerbations [ 8 , 14 , 16 , 18 , 19 , 20 , 22 , 23 , 24 , 26 , 28 , 29 , 33 , 37 , 40 , 43 , 44 , 45 , 46 , 48 , 50 , 51 , 52 , 56 , 59 , 62 , 63 , 64 , 65 , 66 , 67 , 68 , 69 , 70 , 71 , 72 , 73 , 74 , 75 , 76 , 77 , 78 ]. The association with disease severity was noted in studies that used GOLD disease stages 1–4 and those that used FEV 1 percent predicted and other lung function assessments as continuous variables. Again, this risk factor is self-perpetuating, as evidence shows that even a single moderate or severe exacerbation may almost double the rate of lung function decline [ 79 ]. Accordingly, disease severity and exacerbation history may be correlated. Margüello et al. concluded that the severity of COPD could be associated with a higher risk of exacerbations, but this effect was partly determined by the exacerbations suffered in the previous year [ 23 ]. It should be noted that FEV 1 is not recommended by GOLD for use as a predictor of exacerbation risk or mortality alone due to insufficient precision when used at the individual patient level [ 5 ].

Another factor that should be considered when assessing individual exacerbation risk is the presence of comorbidities [ 7 , 14 , 16 , 18 , 19 , 20 , 21 , 22 , 24 , 25 , 26 , 27 , 28 , 30 , 33 , 34 , 35 , 40 , 41 , 44 , 45 , 46 , 47 , 48 , 51 , 52 , 53 , 54 , 56 , 58 , 59 , 63 , 64 , 73 , 74 , 76 , 77 , 80 , 81 , 82 , 83 , 84 , 85 ]. Comorbidities are common in COPD, in part due to common risk factors (e.g., age, smoking, lifestyle factors) that also increase the risk of other chronic diseases [ 7 ]. Significant associations were observed between exacerbation risk and comorbidities, such as anxiety and depression, cardiovascular disease, diabetes, and respiratory comorbidities. As with prior exacerbations, the strength of the association increased with the number of comorbidities. Some comorbidities that were found to be associated with COPD exacerbations share a common biological mechanism of systemic inflammation, such as cardiovascular disease, diabetes, and depression [ 86 ]. Furthermore, other respiratory comorbidities, including asthma and bronchiectasis, involve inflammation of the airways [ 87 ]. In these patients, optimal management of comorbidities may reduce the risk of future COPD exacerbations (and improve QoL), although further research is needed to confirm the efficacy of this approach to exacerbation prevention. As cardiovascular conditions, including hypertension and coronary heart disease, are the most common comorbidities in people with COPD [ 7 ], reducing cardiovascular risk may be a key goal in reducing the occurrence of exacerbations. For other comorbidities, the mechanism for the association with exacerbation risk may be related to non-biological factors. For example, in depression, it has been suggested that the mechanism may relate to greater sensitivity to symptom changes or more frequent physician visits [ 88 ].

There is now a growing body of evidence reporting the relationship between blood eosinophil count and exacerbation risk [ 8 , 13 , 14 , 20 , 37 , 48 , 52 , 56 , 59 , 60 , 62 , 89 , 90 , 91 , 92 , 93 , 94 , 95 , 96 , 97 , 98 , 99 ]. Data from many large clinical trials (SUNSET [ 89 ], FLAME [ 96 ], WISDOM [ 98 ], IMPACT [ 13 ], TRISTAN [ 99 ], INSPIRE [ 99 ], KRONOS [ 91 ], TRIBUTE [ 48 ], TRILOGY [ 52 ], TRINITY [ 56 ]) have also shown relationships between treatment, eosinophil count, and exacerbation rates. Evidence shows that eosinophil count, along with other effect modifiers (e.g., exacerbation history), can be used to predict reductions in exacerbations with ICS treatment. Identifying patients most likely to respond to ICS should contribute to personalized medicine approaches to treat COPD. One challenge in drawing a strong conclusion from eosinophil counts is the choice of a cut-off value, with a variety of absolute and percentage values observed to be positively associated with the risk of moderate-to-severe exacerbations. The use of absolute counts may be more practical, as these are not affected by variations in other immune cell numbers; however, there is a lack of consensus on this point [ 100 ].

Across the studies examined, associations between sex and the risk of moderate and/or severe exacerbations were variable [ 14 , 16 , 18 , 20 , 21 , 22 , 23 , 24 , 26 , 27 , 28 , 29 , 37 , 40 , 42 , 44 , 45 , 46 , 47 , 48 , 51 , 52 , 56 , 58 , 59 , 63 , 73 , 74 , 77 , 80 , 83 , 84 , 85 ]. A greater number of studies showed an increased risk of exacerbations in females compared with males. In contrast, some studies failed to detect a relationship, suggesting that country-specific or cultural factors may play a role. A majority of the included studies evaluated more male patients than female patients; to further elucidate the relationship between sex and exacerbations, more studies in female patients are warranted. Over half of the studies that assessed the relationship between age and exacerbation risk found an association between increasing age and increasing risk of moderate-to-severe COPD exacerbations [ 14 , 16 , 18 , 20 , 21 , 22 , 23 , 24 , 26 , 27 , 28 , 29 , 33 , 40 , 42 , 44 , 45 , 47 , 51 , 52 , 54 , 56 , 63 , 73 , 74 , 77 , 80 , 83 , 85 ].

Our findings also suggested that patients with low BMI have greater risk of moderate and/or severe exacerbations. The mechanism underlying this increased risk in underweight patients is poorly understood; however, loss of lean body mass in patients with COPD may be related to ongoing systemic inflammation that impacts skeletal muscle mass [ 101 , 102 , 103 ].

A limitation of this SLR, that may have resulted in some studies with valid results being missed, was the exclusion of non-English-language studies and the limitation by date; however, the search strategy was otherwise broad, resulting in the review of a large number of studies. The majority of studies captured in this SLR were from Europe, North America, and Asia. The findings may therefore be less generalizable to patients in other regions, such as Africa or South America. Given that one study reported an association between geographic location within different regions of the US and exacerbations [ 34 ], it is plausible that risk of exacerbations may be impacted by global location. As no formal meta-analysis was planned, the assessments are based on a qualitative synthesis of studies. A majority of the included studies looked at exposures of certain factors (e.g., history of exacerbations) at baseline; however, some of these factors change over time, calling into question whether a more sophisticated statistical analysis should have been conducted in some cases to consider time-varying covariates. Our results can only inform on associations, not causation, and there are likely bidirectional relationships between many factors and exacerbation risk (e.g., health status). Finally, while our review of the literature captured a large number of prognostic factors, other variables such as genetic factors, lung microbiome composition, and changes in therapy over time have not been widely studied to date, but might also influence exacerbation frequency [ 104 ]. Further research is needed to assess the contribution of these factors to exacerbation risk.

This SLR captured publications up to July 2019. However, further studies have since been published that further support the prognostic factors identified here. For example, recent studies have reported an increased risk of exacerbations in patients with a history of exacerbations [ 105 ], comorbidities [ 106 ], poorer lung function (GOLD stage) [ 105 ], higher symptomatic burden [ 107 ], female sex [ 105 ], and lower BMI [ 106 , 108 ].

In summary, the literature assessing risk factors for moderate-to-severe COPD exacerbations shows that there are associations between several demographic and disease characteristics with COPD exacerbations, potentially allowing clinicians to identify patients most at risk of future exacerbations. Exacerbation history, comorbidities, and disease severity or bronchodilator reversibility were the factors most strongly associated with exacerbation risk, and should be considered in future research efforts to develop prognostic tools to estimate the likelihood of exacerbation occurrence. Importantly, many prognostic factors for exacerbations, such as symptom burden, QoL, and comorbidities, are modifiable with optimal pharmacologic and non-pharmacologic treatments or lifestyle modifications. Overall, the evidence suggests that, taken together, predicting and reducing exacerbation risk is an achievable goal in COPD.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Body mass index

COPD Assessment Test

Chronic obstructive pulmonary disease

Forced expiratory volume in 1 s

Global Initiative for Chronic Obstructive Lung Disease

Inhaled corticosteroid

Modified Medical Research Council

Quality of life

St. George’s Respiratory Questionnaire

World Health Organization. The top 10 causes of death. 2018. https://www.who.int/news-room/fact-sheets/detail/the-top-10-causes-of-death . Accessed 22 Jul 2020.

GBD 2019 Diseases and Injuries Collaborators. Global burden of 369 diseases and injuries in 204 countries and territories, 1990–2019: a systematic analysis for the Global Burden of Disease Study 2019. The Lancet. 2020;396:1204–22.

Article   Google Scholar  

Hurst JR, Skolnik N, Hansen GJ, Anzueto A, Donaldson GC, Dransfield MT, Varghese P. Understanding the impact of chronic obstructive pulmonary disease exacerbations on patient health and quality of life. Eur J Intern Med. 2020;73:1–6.

Article   PubMed   Google Scholar  

Zhang Y, Morgan RL, Alonso-Coello P, Wiercioch W, Bała MM, Jaeschke RR, Styczeń K, Pardo-Hernandez H, Selva A, Ara Begum H, et al. A systematic review of how patients value COPD outcomes. Eur Respir J. 2018;52:1800222.

Global Initiative for Chronic Obstructive Lung Disease. 2022 GOLD Report. Global strategy for the diagnosis, management and prevention of COPD. 2022. https://goldcopd.org/2022-gold-reports-2/ . Accessed 02 Feb 2022.

Müllerová H, Shukla A, Hawkins A, Quint J. Risk factors for acute exacerbations of COPD in a primary care population: a retrospective observational cohort study. BMJ Open. 2014;4: e006171.

Article   PubMed   PubMed Central   Google Scholar  

Westerik JAM, Metting EI, van Boven JFM, Tiersma W, Kocks JWH, Schermer TR. Associations between chronic comorbidity and exacerbation risk in primary care patients with COPD. Respir Res. 2017;18:31.

Vedel-Krogh S, Nielsen SF, Lange P, Vestbo J, Nordestgaard BG. Blood eosinophils and exacerbations in chronic obstructive pulmonary disease. The Copenhagen General Population Study. Am J Respir Crit Care Med. 2016;193:965–74.

Hurst JR, Vestbo J, Anzueto A, Locantore N, Müllerová H, Tal-Singer R, Miller B, Lomas DA, Agusti A, Macnee W, et al. Susceptibility to exacerbation in chronic obstructive pulmonary disease. N Engl J Med. 2010;363:1128–38.

Article   CAS   PubMed   Google Scholar  

Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6:e1000097.

Çolak Y, Afzal S, Marott JL, Nordestgaard BG, Vestbo J, Ingebrigtsen TS, Lange P. Prognosis of COPD depends on severity of exacerbation history: a population-based analysis. Respir Med. 2019;155:141–7.

Rothnie KJ, Müllerová H, Smeeth L, Quint JK. Natural history of chronic obstructive pulmonary disease exacerbations in a general practice-based population with chronic obstructive pulmonary disease. Am J Respir Crit Care Med. 2018;198:464–71.

Pascoe S, Barnes N, Brusselle G, Compton C, Criner GJ, Dransfield MT, Halpin DMG, Han MK, Hartley B, Lange P, et al. Blood eosinophils and treatment response with triple and dual combination therapy in chronic obstructive pulmonary disease: analysis of the IMPACT trial. Lancet Respir Med. 2019;7:745–56.

Yun JH, Lamb A, Chase R, Singh D, Parker MM, Saferali A, Vestbo J, Tal-Singer R, Castaldi PJ, Silverman EK, et al. Blood eosinophil count thresholds and exacerbations in patients with chronic obstructive pulmonary disease. J Allergy Clin Immunol. 2018;141:2037-2047.e10.

Yoon HY, Park SY, Lee CH, Byun MK, Na JO, Lee JS, Lee WY, Yoo KH, Jung KS, Lee JH. Prediction of first acute exacerbation using COPD subtypes identified by cluster analysis. Int J Chron Obstruct Pulmon Dis. 2019;14:1389–97.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Yohannes AM, Mulerova H, Lavoie K, Vestbo J, Rennard SI, Wouters E, Hanania NA. The association of depressive symptoms with rates of acute exacerbations in patients with COPD: results from a 3-year longitudinal follow-up of the ECLIPSE cohort. J Am Med Dir Assoc. 2017;18:955-959.e6.

Tsiligianni I, Metting E, van der Molen T, Chavannes N, Kocks J. Morning and night symptoms in primary care COPD patients: a cross-sectional and longitudinal study. An UNLOCK study from the IPCRG. NPJ Prim Care Respir Med. 2016;26:16040.

Jo YS, Yoon HI, Kim DK, Yoo CG, Lee CH. Comparison of COPD Assessment Test and Clinical COPD Questionnaire to predict the risk of exacerbation. Int J Chron Obstruct Pulmon Dis. 2018;13:101–7.

Marçôa R, Rodrigues DM, Dias M, Ladeira I, Vaz AP, Lima R, Guimarães M. Classification of Chronic Obstructive Pulmonary Disease (COPD) according to the new Global Initiative for Chronic Obstructive Lung Disease (GOLD) 2017: comparison with GOLD 2011. COPD. 2018;15:21–6.

Han MK, Quibrera PM, Carretta EE, Barr RG, Bleecker ER, Bowler RP, Cooper CB, Comellas A, Couper DJ, Curtis JL, et al. Frequency of exacerbations in patients with chronic obstructive pulmonary disease: an analysis of the SPIROMICS cohort. Lancet Respir Med. 2017;5:619–26.

Yii ACA, Loh CH, Tiew PY, Xu H, Taha AAM, Koh J, Tan J, Lapperre TS, Anzueto A, Tee AKH. A clinical prediction model for hospitalized COPD exacerbations based on “treatable traits.” Int J Chron Obstruct Pulmon Dis. 2019;14:719–28.

McGarvey L, Lee AJ, Roberts J, Gruffydd-Jones K, McKnight E, Haughney J. Characterisation of the frequent exacerbator phenotype in COPD patients in a large UK primary care population. Respir Med. 2015;109:228–37.

Margüello MS, Garrastazu R, Ruiz-Nuñez M, Helguera JM, Arenal S, Bonnardeux C, León C, Miravitlles M, García-Rivero JL. Independent effect of prior exacerbation frequency and disease severity on the risk of future exacerbations of COPD: a retrospective cohort study. NPJ Prim Care Respir Med. 2016;26:16046.

Engel B, Schindler C, Leuppi JD, Rutishauser J. Predictors of re-exacerbation after an index exacerbation of chronic obstructive pulmonary disease in the REDUCE randomised clinical trial. Swiss Med Wkly. 2017;147: w14439.

PubMed   Google Scholar  

Benson VS, Müllerová H, Vestbo J, Wedzicha JA, Patel A, Hurst JR. Evaluation of COPD longitudinally to identify predictive surrogate endpoints (ECLIPSE) investigators. Associations between gastro-oesophageal reflux, its management and exacerbations of chronic obstructive pulmonary disease. Respir Med. 2015;109:1147–54.

Santibáñez M, Garrastazu R, Ruiz-Nuñez M, Helguera JM, Arenal S, Bonnardeux C, León C, García-Rivero JL. Predictors of hospitalized exacerbations and mortality in chronic obstructive pulmonary disease. PLoS ONE. 2016;11: e0158727.

Article   PubMed   PubMed Central   CAS   Google Scholar  

Jo YS, Kim YH, Lee JY, Kim K, Jung KS, Yoo KH, Rhee CK. Impact of BMI on exacerbation and medical care expenses in subjects with mild to moderate airflow obstruction. Int J Chron Obstruct Pulmon Dis. 2018;13:2261–9.

Alexopoulos EC, Malli F, Mitsiki E, Bania EG, Varounis C, Gourgoulianis KI. Frequency and risk factors of COPD exacerbations and hospitalizations: a nationwide study in Greece (Greek Obstructive Lung Disease Epidemiology and health ecoNomics: GOLDEN study). Int J Chron Obstruct Pulmon Dis. 2015;10:2665–74.

PubMed   PubMed Central   Google Scholar  

Liu D, Peng SH, Zhang J, Bai SH, Liu HX, Qu JM. Prediction of short term re-exacerbation in patients with acute exacerbation of chronic obstructive pulmonary disease. Int J Chron Obstruct Pulmon Dis. 2015;10:1265–73.

Müllerová H, Maselli DJ, Locantore N, Vestbo J, Hurst JR, Wedzicha JA, Bakke P, Agusti A, Anzueto A. Hospitalized exacerbations of COPD: risk factors and outcomes in the ECLIPSE cohort. Chest. 2015;147:999–1007.

de Miguel-Díez J, Hernández-Vázquez J, López-de-Andrés A, Álvaro-Meca A, Hernández-Barrera V, Jiménez-García R. Analysis of environmental risk factors for chronic obstructive pulmonary disease exacerbation: a case-crossover study (2004–2013). PLoS ONE. 2019;14: e0217143.

Krachunov II, Kyuchukov NH, Ivanova ZI, Yanev NA, Hristova PA, Borisova ED, Popova TP, Pavlov PS, Nikolova PT, Ivanov YY. Impact of air pollution and outdoor temperature on the rate of chronic obstructive pulmonary disease exacerbations. Folia Med (Plovdiv). 2017;59:423–9.

Article   CAS   Google Scholar  

Baumeler L, Papakonstantinou E, Milenkovic B, Lacoma A, Louis R, Aerts JG, Welte T, Kostikas K, Blasi F, Boersma W, et al. Therapy with proton-pump inhibitors for gastroesophageal reflux disease does not reduce the risk for severe exacerbations in COPD. Respirology. 2016;21:883–90.

Annavarapu S, Goldfarb S, Gelb M, Moretz C, Renda A, Kaila S. Development and validation of a predictive model to identify patients at risk of severe COPD exacerbations using administrative claims data. Int J Chron Obstruct Pulmon Dis. 2018;13:2121–30.

Crisafulli E, Torres A, Huerta A, Méndez R, Guerrero M, Martinez R, Liapikou A, Soler N, Sethi S, Menéndez R. C-reactive protein at discharge, diabetes mellitus and ≥1 hospitalization during previous year predict early readmission in patients with acute exacerbation of chronic obstructive pulmonary disease. COPD. 2015;12:311–20.

Bollmeier SG, Hartmann AP. Management of chronic obstructive pulmonary disease: a review focusing on exacerbations. Am J Health Syst Pharm. 2020;77:259–68.

Bafadhel M, Peterson S, De Blas MA, Calverley PM, Rennard SI, Richter K, Fagerås M. Predictors of exacerbation risk and response to budesonide in patients with chronic obstructive pulmonary disease: a post-hoc analysis of three randomised trials. Lancet Respir Med. 2018;6:117–26.

Calverley PM, Anzueto AR, Dusser D, Mueller A, Metzdorf N, Wise RA. Treatment of exacerbations as a predictor of subsequent outcomes in patients with COPD. Int J Chron Obstruct Pulmon Dis. 2018;13:1297–308.

Calverley PM, Tetzlaff K, Dusser D, Wise RA, Mueller A, Metzdorf N, Anzueto A. Determinants of exacerbation risk in patients with COPD in the TIOSPIR study. Int J Chron Obstruct Pulmon Dis. 2017;12:3391–405.

Eklöf J, Sørensen R, Ingebrigtsen TS, Sivapalan P, Achir I, Boel JB, Bangsborg J, Ostergaard C, Dessau RB, Jensen US, et al. Pseudomonas aeruginosa and risk of death and exacerbations in patients with chronic obstructive pulmonary disease: an observational cohort study of 22 053 patients. Clin Microbiol Infect. 2020;26:227–34.

Estirado C, Ceccato A, Guerrero M, Huerta A, Cilloniz C, Vilaró O, Gabarrús A, Gea J, Crisafulli E, Soler N, Torres A. Microorganisms resistant to conventional antimicrobials in acute exacerbations of chronic obstructive pulmonary disease. Respir Res. 2018;19:119.

Fuhrman C, Moutengou E, Roche N, Delmas MC. Prognostic factors after hospitalization for COPD exacerbation. Rev Mal Respir. 2017;34:1–18.

Krachunov I, Kyuchukov N, Ivanova Z, Yanev NA, Hristova PA, Pavlov P, Glogovska P, Popova T, Ivanov YY. Stability of frequent exacerbator phenotype in patients with chronic obstructive pulmonary disease. Folia Med (Plovdiv). 2018;60:536–45.

Make BJ, Eriksson G, Calverley PM, Jenkins CR, Postma DS, Peterson S, Östlund O, Anzueto A. A score to predict short-term risk of COPD exacerbations (SCOPEX). Int J Chron Obstruct Pulmon Dis. 2015;10:201–9.

Montserrat-Capdevila J, Godoy P, Marsal JR, Barbé F. Predictive model of hospital admission for COPD exacerbation. Respir Care. 2015;60:1288–94.

Montserrat-Capdevila J, Godoy P, Marsal JR, Barbé F, Galván L. Risk factors for exacerbation in chronic obstructive pulmonary disease: a prospective study. Int J Tuberc Lung Dis. 2016;20:389–95.

Orea-Tejeda A, Navarrete-Peñaloza AG, Verdeja-Vendrell L, Jiménez-Cepeda A, González-Islas DG, Hernández-Zenteno R, Keirns-Davis C, Sánchez-Santillán R, Velazquez-Montero A, Puentes RG. Right heart failure as a risk factor for severe exacerbation in patients with chronic obstructive pulmonary disease: prospective cohort study. Clin Respir J. 2018;12:2635–41.

Papi A, Vestbo J, Fabbri L, Corradi M, Prunier H, Cohuet G, Guasconi A, Montagna I, Vezzoli S, Petruzzelli S, et al. Extrafine inhaled triple therapy versus dual bronchodilator therapy in chronic obstructive pulmonary disease (TRIBUTE): a double-blind, parallel group, randomised controlled trial. Lancet. 2018;391:1076–84.

Lipson DA, Barnhart F, Brealey N, Brooks J, Criner GJ, Day NC, Dransfield MT, Halpin DMG, Han MK, Jones CE, et al. Once-daily single-inhaler triple versus dual therapy in patients with COPD. N Engl J Med. 2018;378:1671–80.

Pasquale MK, Xu Y, Baker CL, Zou KH, Teeter JG, Renda AM, Davis CC, Lee TC, Bobula J. COPD exacerbations associated with the modified Medical Research Council scale and COPD assessment test among Humana Medicare members. Int J Chron Obstruct Pulmon Dis. 2016;11:111–21.

Schuler M, Wittmann M, Faller H, Schultz K. Including changes in dyspnea after inpatient rehabilitation improves prediction models of exacerbations in COPD. Respir Med. 2018;141:87–93.

Singh D, Papi A, Corradi M, Pavlišová I, Montagna I, Francisco C, Cohuet G, Vezzoli S, Scuri M, Vestbo J. Single inhaler triple therapy versus inhaled corticosteroid plus long-acting β 2 -agonist therapy for chronic obstructive pulmonary disease (TRILOGY): a double-blind, parallel group, randomised controlled trial. Lancet. 2016;388:963–73.

Søgaard M, Madsen M, Løkke A, Hilberg O, Sørensen HT, Thomsen RW. Incidence and outcomes of patients hospitalized with COPD exacerbation with and without pneumonia. Int J Chron Obstruct Pulmon Dis. 2016;11:455–65.

Stanford RH, Nag A, Mapel DW, Lee TA, Rosiello R, Schatz M, Vekeman F, Gauthier-Loiselle M, Merrigan JFP, Duh MS. Claims-based risk model for first severe COPD exacerbation. Am J Manag Care. 2018;24:e45–53.

Stanford RH, Lau MS, Li Y, Stemkowski S. External validation of a COPD risk measure in a commercial and medicare population: the COPD treatment ratio. J Manag Care Spec Pharm. 2019;25:58–69.

Vestbo J, Papi A, Corradi M, Blazhko V, Montagna I, Francisco C, Cohuet G, Vezzoli S, Scuri M, Singh D. Single inhaler extrafine triple therapy versus long-acting muscarinic antagonist therapy for chronic obstructive pulmonary disease (TRINITY): a double-blind, parallel group, randomised controlled trial. Lancet. 2017;389:1919–29.

Wei X, Ma Z, Yu N, Ren J, Jin C, Mi J, Shi M, Tian L, Gao Y, Guo Y. Risk factors predict frequent hospitalization in patients with acute exacerbation of COPD. Int J Chron Obstruct Pulmon Dis. 2018;13:121–9.

Whalley D, Svedsater H, Doward L, Crawford R, Leather D, Lay-Flurrie J, Bosanquet N. Follow-up interviews from The Salford Lung Study (COPD) and analyses per treatment and exacerbations. NPJ Prim Care Respir Med. 2019;29:20.

Zeiger RS, Tran TN, Butler RK, Schatz M, Li Q, Khatry DB, Martin U, Kawatkar AA, Chen W. Relationship of blood eosinophil count to exacerbations in chronic obstructive pulmonary disease. J Allergy Clin Immunol Pract. 2018;6:944-954.e945.

Vogelmeier CF, Kostikas K, Fang J, Tian H, Jones B, Morgan CL, Fogel R, Gutzwiller FS, Cao H. Evaluation of exacerbations and blood eosinophils in UK and US COPD populations. Respir Res. 2019;20:178.

Celli BR, Fabbri LM, Aaron SD, Agusti A, Brook R, Criner GJ, Franssen FME, Humbert M, Hurst JR, O’Donnell D, et al. An updated definition and severity classification of COPD exacerbations: the Rome proposal. Am J Respir Crit Care Med. 2021;204:1251–8.

Adir Y, Hakrush O, Shteinberg M, Schneer S, Agusti A. Circulating eosinophil levels do not predict severe exacerbations in COPD: a retrospective study. ERJ Open Research. 2018;4:00022–2018.

Bartels W, Adamson S, Leung L, Sin DD, van Eeden SF. Emergency department management of acute exacerbations of chronic obstructive pulmonary disease: factors predicting readmission. Int J Chron Obstruct Pulmon Dis. 2018;13:1647–54.

Kim V, Zhao H, Regan E, Han MK, Make BJ, Crapo JD, Jones PW, Curtis JL, Silverman EK, Criner GJ, COPDGene Investigators. The St. George’s Respiratory Questionnaire definition of chronic bronchitis may be a better predictor of COPD exacerbations compared with the classic definition. Chest. 2019;156:685–95.

Abston E, Comellas A, Reed RM, Kim V, Wise RA, Brower R, Fortis S, Beichel R, Bhatt S, Zabner J, et al. Higher BMI is associated with higher expiratory airflow normalised for lung volume (FEF25-75/FVC) in COPD. BMJ Open Respir Res. 2017;4: e000231.

Emura I, Usuda H, Satou K. Appearance of large scavenger receptor A-positive cells in peripheral blood: a potential risk factor for severe exacerbation of chronic obstructive pulmonary disease. Pathol Int. 2019;69:187–92.

Erol S, Sen E, Gizem Kilic Y, Yousif A, Akkoca Yildiz O, Acican T, Saryal S. Does the 2017 revision improve the ability of GOLD to predict risk of future moderate and severe exacerbation? Clin Respir J. 2018;12:2354–60.

Han MZ, Hsiue TR, Tsai SH, Huang TH, Liao XM, Chen CZ. Validation of the GOLD 2017 and new 16 subgroups (1A–4D) classifications in predicting exacerbation and mortality in COPD patients. Int J Chron Obstruct Pulmon Dis. 2018;13:3425–33.

Huang TH, Hsiue TR, Lin SH, Liao XM, Su PL, Chen CZ. Comparison of different staging methods for COPD in predicting outcomes. Eur Resp J. 2018;51:1700577.

Jung YH, Lee DY, Kim DW, Park SS, Heo EY, Chung HS, Kim DK. Clinical significance of laryngopharyngeal reflux in patients with chronic obstructive pulmonary disease. Int J Chron Obstruct Pulmon Dis. 2015;10:1343–51.

CAS   PubMed   PubMed Central   Google Scholar  

Kim J, Kim WJ, Lee CH, Lee SH, Lee MG, Shin KC, Yoo KH, Lee JH, Lim SY, Na JO, et al. Which bronchodilator reversibility criteria can predict severe acute exacerbation in chronic obstructive pulmonary disease patients? Respir Res. 2017;18:107.

Kobayashi S, Hanagama M, Ishida M, Sato H, Ono M, Yamanda S, Yamada M, Aizawa H, Yanai M. Clinical characteristics and outcomes in Japanese patients with COPD according to the 2017 GOLD classification: the Ishinomaki COPD Network Registry. Int J Chron Obstruct Pulmon Dis. 2018;13:3947–55.

Lee SH, Lee JH, Yoon HI, Park HY, Kim TH, Yoo KH, Oh YM, Jung KS, Lee SD, Lee SW. Change in inhaled corticosteroid treatment and COPD exacerbations: an analysis of real-world data from the KOLD/KOCOSS cohorts. Respir Res. 2019;20:62.

Pavlovic R, Stefanovic S, Lazic Z, Jankovic S. Factors associated with the rate of COPD exacerbations that require hospitalization. Turk J Med Sci. 2017;47:134–41.

Song JH, Lee CH, Um SJ, Park YB, Yoo KH, Jung KS, Lee SD, Oh YM, Lee JH, Kim EK, Kim DK. Clinical impacts of the classification by 2017 GOLD guideline comparing previous ones on outcomes of COPD in real-world cohorts. Int J Chron Obstruct Pulmon Dis. 2018;13:3473–84.

Sundh J, Johansson G, Larsson K, Lindén A, Löfdahl CG, Sandström T, Janson C. The phenotype of concurrent chronic bronchitis and frequent exacerbations in patients with severe COPD attending Swedish secondary care units. Int J Chron Obstruct Pulmon Dis. 2015;10:2327–34.

Urwyler P, Hussein NA, Bridevaux PO, Chhajed PN, Geiser T, Grendelmeier P, Zellweger LJ, Kohler M, Maier S, Miedinger D, et al. Predictive factors for exacerbation and reexacerbation in chronic obstructive pulmonary disease: an extension of the Cox model to analyze data from the Swiss COPD cohort. Multidiscip Respir Med. 2019;14:7.

Wallace AE, Kaila S, Bayer V, Shaikh A, Shinde MU, Willey VJ, Napier MB, Singer JR. Health care resource utilization and exacerbation rates in patients with COPD stratified by disease severity in a commercially insured population. J Manag Care Spec Pharm. 2019;25:205–17.

Halpin DMG, Decramer M, Celli BR, Mueller A, Metzdorf N, Tashkin DP. Effect of a single exacerbation on decline in lung function in COPD. Respir Med. 2017;128:85–91.

Bade BC, DeRycke EC, Ramsey C, Skanderson M, Crothers K, Haskell S, Bean-Mayberry B, Brandt C, Bastian LA, Akgün KM. Sex differences in veterans admitted to the hospital for chronic obstructive pulmonary disease exacerbation. Ann Am Thorac Soc. 2019;16:707–14.

Iyer AS, Bhatt SP, Dransfield M, Kinney G, Holm K, Wamboldt FS, Hanania N, Martinez C, Regan E, Foreman MG, et al. Psychological distress prospectively predicts severe exacerbations in smokers with and without airflow limitation—a longitudinal follow-up study of the COPDGene cohort [abstract]. Am J Respir Crit Care Med. 2017. https://doi.org/10.1164/ajrccm-conference.2017.195.1_MeetingAbstracts.A4709 .

Diamond M, Zhao H, Armstrong HF, Morrison M, Bailey KL, Carretta EE, Criner GJ, Han MK, Bleeker E, Cooper CB, et al. Anxiety and depression, either alone or in combination, are associated with respiratory exacerbations in smokers with and without COPD [abstract]. Am J Respir Crit Care Med. 2017;195:1615–31.

Google Scholar  

Lau CS, Siracuse BL, Chamberlain RS. Readmission After COPD Exacerbation Scale: determining 30-day readmission risk for COPD patients. Int J Chron Obstruct Pulmon Dis. 2017;12:1891–902.

Pikoula M, Quint JK, Nissen F, Hemingway H, Smeeth L, Denaxas S. Identifying clinically important COPD sub-types using data-driven approaches in primary care population based electronic health records. BMC Med Inform Decis Mak. 2019;19:86.

Wei YF, Tsai YH, Wang CC, Kuo PH. Impact of overweight and obesity on acute exacerbations of COPD—subgroup analysis of the Taiwan Obstructive Lung Disease cohort. Int J Chron Obstruct Pulmon Dis. 2017;12:2723–9.

Barnes PJ, Celli BR. Systemic manifestations and comorbidities of COPD. Eur Resp J. 2009;33:1165–85.

Polverino E, Dimakou K, Hurst J, Martinez-Garcia MA, Miravitlles M, Paggiaro P, Shteinberg M, Aliberti S, Chalmers JD. The overlap between bronchiectasis and chronic airway diseases: state of the art and future directions. Eur Respir J. 2018;52:1800328.

Xu W, Collet JP, Shapiro S, Lin Y, Yang T, Platt RW, Wang C, Bourbeau J. Independent effect of depression and anxiety on chronic obstructive pulmonary disease exacerbations and hospitalizations. Am J Respir Crit Care Med. 2008;178:913–20.

Chapman KR, Hurst JR, Frent SM, Larbig M, Fogel R, Guerin T, Banerji D, Patalano F, Goyal P, Pfister P, et al. Long-term triple therapy de-escalation to indacaterol/glycopyrronium in patients with chronic obstructive pulmonary disease (SUNSET): a randomized, double-blind, triple-dummy clinical trial. Am J Respir Crit Care Med. 2018;198:329–39.

Couillard S, Larivée P, Courteau J, Vanasse A. Eosinophils in COPD exacerbations are associated with increased readmissions. Chest. 2017;151:366–73.

Ferguson GT, Rabe KF, Martinez FJ, Fabbri LM, Wang C, Ichinose M, Bourne E, Ballal S, Darken P, DeAngelis K, et al. Triple therapy with budesonide/glycopyrrolate/formoterol fumarate with co-suspension delivery technology versus dual therapies in chronic obstructive pulmonary disease (KRONOS): a double-blind, parallel-group, multicentre, phase 3 randomised controlled trial. Lancet Respir Med. 2018;6:747–58.

Ko FWS, Chan KP, Ngai J, Ng SS, Yip WH, Ip A, Chan TO, Hui DSC. Blood eosinophil count as a predictor of hospital length of stay in COPD exacerbations. Respirology. 2019;25:259–66.

MacDonald MI, Osadnik CR, Bulfin L, Hamza K, Leong P, Wong A, King PT, Bardin PG. Low and high blood eosinophil counts as biomarkers in hospitalized acute exacerbations of COPD. Chest. 2019;156:92–100.

Müllerová H, Hahn B, Simard EP, Mu G, Hatipoğlu U. Exacerbations and health care resource use among patients with COPD in relation to blood eosinophil counts. Int J Chron Obstruct Pulmon Dis. 2019;14:683–92.

Bafadhel M, Greening NJ, Harvey-Dunstan TC, Williams JEA, Morgan MD, Brightling CE, Hussain SF, Pavord ID, Singh SJ, Steiner MC. Blood eosinophils and outcomes in severe hospitalised exacerbations of COPD. Chest. 2016;150:320–8.

Roche N, Chapman KR, Vogelmeier CF, Herth FJF, Thach C, Fogel R, Olsson P, Patalano F, Banerji D, Wedzicha JA. Blood eosinophils and response to maintenance chronic obstructive pulmonary disease treatment. Data from the FLAME trial. Am J Respir Crit Care Med. 2017;195:1189–97.

Vestbo J, Vogelmeier CF, Small M, Siddall J, Fogel R, Kostikas K. Inhaled corticosteroid use by exacerbations and eosinophils: a real-world COPD population. Int J Chron Obstruct Pulmon Dis. 2019;14:853–61.

Watz H, Tetzlaff K, Wouters EFM, Kirsten A, Magnussen H, Rodriguez-Roisin R, Vogelmeier C, Fabbri LM, Chanez P, Dahl R, et al. Blood eosinophil count and exacerbations in severe chronic obstructive pulmonary disease after withdrawal of inhaled corticosteroids: a post-hoc analysis of the WISDOM trial. Lancet Respir Med. 2016;4:390–8.

Pavord ID, Lettis S, Locantore N, Pascoe S, Jones PW, Wedzicha JA, Barnes NC. Blood eosinophils and inhaled corticosteroid/long-acting beta-2 agonist efficacy in COPD. Thorax. 2016;71:118–25.

Singh D. Predicting corticosteroid response in chronic obstructive pulmonary disease. Blood eosinophils gain momentum. Am J Respir Crit Care Med. 2017;196:1098–100.

Vestbo J, Prescott E, Almdal T, Dahl M, Nordestgaard BG, Andersen T, Sørensen TIA, Lange P. Body mass, fat-free body mass, and prognosis in patients with chronic obstructive pulmonary disease from a random population sample: findings from the Copenhagen City Heart Study. Am J Respir Crit Care Med. 2006;173:79–83.

Agustí AGN, Noguera A, Sauleda J, Sala E, Pons J, Busquets X. Systemic effects of chronic obstructive pulmonary disease. Eur Respir J. 2003;21:347–60.

Agustí AGN, Sauleda J, Miralles C, Gomez C, Togores B, Sala E, Batle S, Busquets X. Skeletal muscle apoptosis and weight loss in chronic obstructive pulmonary disease. Am J Respir Crit Care Med. 2002;166:485–9.

Labaki WW, Martinez FJ. Time to understand the infrequency of the frequent exacerbator phenotype in COPD. Chest. 2018;153:1087–8.

Hartley BF, Barnes NC, Lettis S, Compton CH, Papi A, Jones P. Risk factors for exacerbations and pneumonia in patients with chronic obstructive pulmonary disease: a pooled analysis. Respir Res. 2020;21:5.

Kim Y, Kim YJ, Kang YM, Cho WK. Exploring the impact of number and type of comorbidities on the risk of severe COPD exacerbations in Korean Population: a Nationwide Cohort Study. BMC Pulm Med. 2021;21:151.

Mackay AJ, Kostikas K, Roche N, Frent SM, Olsson P, Pfister P, Gupta P, Patalano F, Banerji D, Wedzicha JA. Impact of baseline symptoms and health status on COPD exacerbations in the FLAME study. Respir Res. 2020;21:93.

Smulders L, van der Aalst A, Neuhaus EDET, Polman S, Franssen FME, van Vliet M, de Kruif MD. Decreased risk of COPD exacerbations in obese patients. COPD. 2020;17:485–91.

Battisti WP, Wager E, Baltzer L, Bridges D, Cairns A, Carswell CI, Citrome L, Gurr JA, Mooney LA, Moore BJ, et al. Good publication practice for communicating company-sponsored medical research: GPP3. Ann Intern Med. 2015;163:461–4.

Putcha N, Barr RG, Han M, Woodruff PG, Bleecker ER, Kanner RE, Martinez FJ, Tashkin DP, Rennard SI, Breysse P, et al. Understanding the impact of passive smoke exposure on outcomes in COPD [abstract]. Am J Respir Crit Care Med. 2015;191:411–20.

Wu Z, Yang D, Ge Z, Yan M, Wu N, Liu Y. Body mass index of patients with chronic obstructive pulmonary disease is associated with pulmonary function and exacerbations: a retrospective real world research. J Thorac Dis. 2018;10:5086–99.

Download references

Acknowledgements

Medical writing support, under the direction of the authors, was provided by Julia King, PhD, and Sarah Piggott, MChem, CMC Connect, McCann Health Medical Communications, funded by AstraZeneca in accordance with Good Publication Practice (GPP3) guidelines [ 109 ].

This study was supported by AstraZeneca.

Author information

Authors and affiliations.

UCL Respiratory, University College London, London, WC1E 6BT, UK

John R. Hurst

Division of Pulmonary and Critical Care, University of Michigan, Ann Arbor, MI, USA

MeiLan K. Han

Formerly of Parexel International, Mohali, India

Barinder Singh, Gagandeep Kaur & Mohd Kashif Siddiqui

Parexel International, Mohali, India

Sakshi Sharma

Formerly of AstraZeneca, Cambridge, UK

Enrico de Nigris

AstraZeneca, Gothenburg, Sweden

Ulf Holmgren

You can also search for this author in PubMed   Google Scholar

Contributions

The authors have made the following declaration about their contributions. JRH and MKH made substantial contributions to the interpretation of data; BS, SS, GK, and MKS made substantial contributions to the acquisition, analysis, and interpretation of data; EdN and UH made substantial contributions to the conception and design of the work and the interpretation of data. All authors contributed to drafting or critically revising the article, have approved the submitted version, and agree to be personally accountable for their own contributions and to ensure that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature. All authors read and approved the final manuscript.

Corresponding author

Correspondence to John R. Hurst .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

JRH reports consulting fees from AstraZeneca; speaker fees from AstraZeneca, Chiesi, Pfizer, and Takeda; and travel support from GlaxoSmithKline and AstraZeneca. MKH reports assistance with conduction of this research and publication from AstraZeneca; personal fees from Aerogen, Altesa Biopharma, AstraZeneca, Boehringer Ingelheim, Chiesi, Cipla, DevPro, GlaxoSmithKline, Integrity, Medscape, Merck, Mylan, NACE, Novartis, Polarean, Pulmonx, Regeneron, Sanofi, Teva, Verona, United Therapeutics, and UpToDate; either in kind research support or funds paid to the institution from the American Lung Association, AstraZeneca, Biodesix, Boehringer Ingelheim, the COPD Foundation, Gala Therapeutics, the NIH, Novartis, Nuvaira, Sanofi, and Sunovion; participation in Data Safety Monitoring Boards for Novartis and Medtronic with funds paid to the institution; and stock options from Altesa Biopharma and Meissa Vaccines. BS, GK, and MKS are former employees of Parexel International. SS is an employee of Parexel International, which was funded by AstraZeneca to conduct this analysis. EdN is a former employee of AstraZeneca and previously held stock and/or stock options in the company. UH is an employee of AstraZeneca and holds stock and/or stock options in the company.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file1: table s1..

Search strategies. Table S2. List of included studies with linked publications. Table S3. Study characteristics across the 76 included studies. Table S4. Clinical characteristics of the patients assessed across the included studies.

Additional file 2: Fig. S1.

Sex (male vs female) as a risk factor for moderate-to-severe exacerbations. Fig. S2. Sex (male vs female) as a risk factor for severe exacerbations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Hurst, J.R., Han, M.K., Singh, B. et al. Prognostic risk factors for moderate-to-severe exacerbations in patients with chronic obstructive pulmonary disease: a systematic literature review. Respir Res 23 , 213 (2022). https://doi.org/10.1186/s12931-022-02123-5

Download citation

Received : 02 March 2022

Accepted : 20 July 2022

Published : 23 August 2022

DOI : https://doi.org/10.1186/s12931-022-02123-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Exacerbations
• Comorbidities
• Hospitalization

Respiratory Research

ISSN: 1465-993X

• General enquiries: [email protected]

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• For authors
• Browse by collection
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Volume 14, Issue 4
• Chronic loneliness and chronic social isolation among older adults: a study protocol for a systematic review, meta-analysis and meta-regression
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0002-6886-2745 André Hajek ,
• Giuliana Posi ,
• http://orcid.org/0000-0001-5711-6862 Hans-Helmut König
• Department of Health Economics and Health Services Research , University Medical Center Hamburg-Eppendorf , Hamburg , Germany
• Correspondence to Dr André Hajek; a.hajek{at}uke.de

Introduction There are around 20 studies identifying the prevalence of chronic loneliness and chronic social isolation in older adults. However, there is an absence of a systematic review, meta-analysis and meta-regression that consolidates the available observational studies. Therefore, our objective was to address this knowledge gap. Here, we present the study protocol for this upcoming work. Such knowledge can help in addressing older individuals at risk for chronic loneliness and chronic social isolation.

Methods and analysis Established electronic databases will be searched. Observational studies reporting the prevalence of chronic loneliness and chronic social isolation among individuals aged 60 years and over will be included. Disease-specific samples will be excluded. The focus of data extraction will be on methods, sample characteristics and key findings. The Joanna Briggs Institute (JBI) standardised critical appraisal instrument for prevalence studies will be used for assessing the quality of the studies. Two reviewers will be responsible for carrying out the study selection, data extraction and assessment of study quality. The results will be presented through the use of figures, tables, narrative summaries and a meta-analysis and meta-regression.

Ethics and dissemination No primary data will be collected. Thus, there is no need for approval from an ethics committee. We intend to share our results through publication in a peer-reviewed journal.

• Social Interaction
• Social Support

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:  http://creativecommons.org/licenses/by-nc/4.0/ .

https://doi.org/10.1136/bmjopen-2023-080399

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

STRENGTHS AND LIMITATIONS OF THIS STUDY

First work: aimed to identify the prevalence and factors associated with chronic loneliness and chronic social isolation in older adults.

Study quality will be evaluated.

Key stages (such as study selection, data extraction and assessment of study quality) will be undertaken by two reviewers.

Meta-analysis and meta-regression are planned.

Search focused on peer-reviewed articles.

Introduction

In late life, individuals often encounter various challenges. Among these challenges, chronic loneliness and chronic social isolation emerge as significant concerns. With the progression of age, individuals may experience a reduction in their social relationships, influenced by factors such as retirement, the loss of close friends and family members, or physical limitations. 1 These transformations can contribute to (chronic) loneliness and social isolation 1 which can have detrimental impacts on mental as well as physical health and longevity among older adults. 2 More precisely, a previous review and meta-analysis showed medium-to-large effects of loneliness on different health outcomes such as physical health, general health, sleep, cognition or mental health—whereby the largest effects of loneliness were found for mental health outcomes. 3 Another systematic review of systematic reviews (ie, a systematic overview) also revealed (1) an association between social isolation and cardiovascular diseases and (2) an association between social isolation and all-cause mortality. 4 Moreover, particularly such chronic feelings (compared with temporary feelings of loneliness) can have harmful effects for health. 5 6

Chronic loneliness among older adults is more than a temporary feeling of solitude; it reflects an enduring and distressing emotional state of dissatisfaction with one’s own social connection. 7 It can be characterised as a deficiency in meaningful social interactions, companionship in their lives or emotional support. 8 Factors such as living alone, loss of spouse or friends, restricted access to transportation or a decreased social engagement can contribute to chronic loneliness. 9

Chronic social isolation, although connected, establishes itself as a distinct concept apart from chronic loneliness. It indicates a situation in which older adults have restricted interactions with social networks and sustain a persistent lack of involvement in social activities. 10 A previous study also showed that chronic social isolation is associated with higher subsequent depression scores. 11

Thus far, numerous studies have examined (temporary) loneliness and social isolation in old age (as an overview, see Refs. 1 2 ). Considerably fewer studies have investigated the prevalence and determinants of chronic loneliness and social isolation in old age. 12–15 To date, a systematic review, meta-analysis and meta-regression is missing exploring the prevalence of chronic loneliness and chronic social isolation—and the factors associated with them. Identifying the prevalence of chronic loneliness and chronic social isolation is of great importance, especially in light of the ongoing rise in the population of individuals aged 60 and above. Furthermore, our upcoming work aimed to examine the factors associated with chronic loneliness and chronic social isolation in this specific age group. This can assist in addressing individuals at risk for chronic loneliness and chronic social isolation. This in turn can help to sustain health and can contribute to successful ageing. 16 17 Overall, addressing chronic loneliness and chronic social isolation is important for policy-makers, healthcare providers and society as a whole. Additionally, this future work has the potential to identify research gaps and thus to guide future research in this area.

Methods and analysis

The methodology for this review adheres to the guidelines outlined in the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols. 18 Additionally, it is registered by the International Prospective Register of Systematic Reviews (PROSPERO, ID: CRD42023467646). This work began on mid-November 2023 (search) and we anticipate that it will be completed by 30 April 2024.

Eligibility criteria

Prior to establishing the final eligibility criteria, a preliminary examination was carried out involving the screening of 100 titles/abstracts. No adjustments to the criteria were made following this pretest. Detailed inclusion and exclusion criteria are shown in the subsequent sections.

Inclusion criteria

Final inclusion criteria were as follows:

Cross-sectional and longitudinal observational studies centred on chronic loneliness or chronic social isolation prevalence within individuals aged 60 years and above.

Validated instruments for evaluating loneliness/social isolation.

Studies accessible in either English or German and released in peer-reviewed scientific journals.

Exclusion criteria

The final exclusion criteria were as follows:

Studies solely focused on samples with particular conditions, such as samples only including individuals with cognitive or mental disorders.

Studies involving such disease-specific samples were excluded as it is unclear to what extent they can be generalised to the older population in general. However, it should be emphasised that samples are not excluded if they include people with diseases (as long as these studies are not purely disease-specific).

We focused on observational studies (and therefore excluded other designs such as randomised controlled trials so that respondents would not be influenced by any interventions). Moreover, it is important to highlight that the appropriateness of the instruments follows closely the criteria outlined in the COSMIN (COnsensus-based Standards for the selection of health Measurement INstruments) guidelines. 19 With regard to chronicity of loneliness (and social isolation), it will be based on the definition in the papers. We assume that the great majority of studies assume chronicity when loneliness (or social isolation) exists for several consecutive waves.

The electronic databases include PubMed, PsycInfo, CINAHL and Web of Science. The final search strategy for PubMed is displayed in table 1 (for the other databases: see online supplemental file 1 ). No restrictions will be applied in terms of time or location. Furthermore, two reviewers will manually explore the reference lists of studies that meet our ultimate inclusion criteria.

Supplemental material

• View inline

Search strategy (PubMed)

Data management

We will use Endnote V.20, developed by Clarivate Analytics (Philadelphia, Pennsylvania, USA), for importing the data. Additionally, Stata V.18.0 (StataCorp) will be employed to conduct a potential meta-analysis and meta-regression if feasible (ie, when the studies are not too different in their design, methods or sample to be combined statistically).

Study selection process

On completing the search, two reviewers (AH and GP) will assess the titles/abstracts to determine their potential inclusion based on the eligibility criteria. Following this, the full texts will be evaluated by these aforementioned two reviewers. If differences of opinions are present, discussions will be held to reach a consensus. If an agreement cannot be reached, a third party (H-HK) will be consulted.

Data collection process and data items

Data extraction will be carried out by two reviewers (AH and GP). The first reviewer (GP) will initially extract the data, and then the second reviewer (AH) will cross-verify it. In instances where clarification is needed, a third party (H-HK) will be engaged. Additionally, if necessary, communication with study authors via email will be initiated. Data extraction will encompass various elements such as study design, definition/assessment of key variables (ie, chronic loneliness and chronic social isolation), sample characteristics (if reported: sample size, mean age and proportion of female individuals), statistical analysis and key findings (prevalence of chronic loneliness and prevalence of chronic social isolation; also stratified by sex, if reported; correlates of chronic loneliness or chronic social isolation).

Regarding meta-analysis, random-effect models will be used to pool proportion across studies included in this upcoming work (since we assume heterogeneity across studies). We will use forest plots to display aggregated estimates and illustrate the degree of variation among the included studies. The Higgin’s I² statistic will be employed to evaluate the heterogeneity among the studies using the following categorisation: (1) 25%–50%, indicating low heterogeneity; (2) 50%–75%, and (3) 75%–100%, representing high heterogeneity. 20 If possible, subgroup analysis will be conducted based on, for example, sex or living arrangement (community-dwelling vs institutionalised settings). We will use funnel plots and conduct the Egger test to investigate the presence of publication bias. If possible, we will conduct meta-regressions to investigate the origins of heterogeneity (eg, tool used to quantify chronic loneliness, proportion of women, average age or country of origin).

Assessment of study quality/risk of bias

The Joanna Briggs Institute standardised critical appraisal instrument for prevalence studies 21 will be used for assessing the quality of the studies. Two independent reviewers (AH and GP) will individually evaluate the study quality. If necessary, discussions will be conducted until a consensus is achieved. In cases where consensus remains elusive, a third party (H-HK) will be consulted.

Data synthesis

On completion of the screening process, a Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram will be generated to illustrate the study selection procedure. In a narrative synthesis, we will present the most important findings. Similar to recent reviews and books, 1 2 our intention is to categorise the correlates into: socioeconomic factors (eg, sex, age, education or income), lifestyle-related factors (eg, physical activity, smoking, alcohol intake) and health-related factors (eg, self-rated health). If possible, we will conduct a meta-analysis and meta-regression.

Patient and public involvement statement

The present review protocol did not involve individual patients or public agencies.

A bulk of studies examined loneliness and social isolation in old age. However, there are far less studies focusing on chronic loneliness and chronic social isolation. Thus, the aim of our upcoming systematic review, meta-analysis and meta-regression will be to give an overview of observational studies examining the prevalence and (ideally) the correlates of chronic loneliness and chronic social isolation. In addition, we will assess the quality of the studies included. Our upcoming work could enhance discussions surrounding loneliness and social isolation in old age (and in other age groups). This may contribute to maintaining general health in later life and successful ageing.

Our upcoming systematic review, meta-analysis and meta-regression have the potential to uncover gaps in research. For example, it may be the case that more studies exist focusing on the prevalence of chronic loneliness rather than chronic social isolation among older adults. It may also be the case that the existing longitudinal studies only use data from only a few years (rather than decades). Additionally, we assume that there is an imbalance between the countries studied so far. For example, many studies could come from North America, Europe and Asia. The prevalence could also depend on variables such as the proportion of women, the tool used to quantify chronic loneliness/chronic social isolation or the geographical region.

Strengths and limitations

This will be the first systematic review, meta-analysis and meta-regression regarding the prevalence and correlates of chronic loneliness and chronic isolation among older adults. The upcoming work engages two reviewers in various tasks, such as study selection and quality assessment. It is intended to do a meta-analysis and meta-regression. It should be noted that our work is restricted to peer-reviewed studies published in English or German language which may exclude potential relevant articles. Moreover, the exclusive focus on peer-reviewed articles may exclude some studies which may be relevant. However, this choice ensures a certain quality of included studies.

Ethics and dissemination

No primary data will be collected. Therefore, approval by an ethics committee is not required. Our findings are planned to be published in a peer-reviewed journal.

Ethics statements

Patient consent for publication.

Not applicable.

• Ishii K , et al
• Riedel-Heller SG ,
• Gill H , et al
• Leigh-Hunt N ,
• Bagguley D ,
• Bash K , et al
• Martín-María N ,
• Caballero FF ,
• Miret M , et al
• Lara E , et al
• Heinrich LM ,
• Bessaha M ,
• Persaud U ,
• Asfe R , et al
• Newall NEG ,
• Chipperfield JG ,
• Xin Z , et al
• Manera KE ,
• Owen KB , et al
• Matovic S ,
• Vasiliadis H-M , et al
• Panes Lundmark V ,
• Josefsson M ,
• Rieckmann A
• Jones RN , et al
• Shamseer L ,
• Clarke M , et al
• Prinsen CAC ,
• Mokkink LB ,
• Bouter LM , et al
• Higgins JPT ,
• Thompson SG ,
• Deeks JJ , et al
• Tufanaru C , et al

Supplementary materials

Supplementary data.

This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

• Data supplement 1

Contributors The study concept was developed by AH and H-HK. The manuscript of the protocol was drafted by AH and critically revised by GP and H-HK. The search strategy was developed by AH and H-HK. Study selection, data extraction and quality assessment will be performed by AH and GP, with HH-K as a third party in case of disagreements. All authors have approved the final version of the manuscript.

Funding We acknowledge financial support from the Open Access Publication Fund of UKE - Universitätsklinikum Hamburg-Eppendorf.

Competing interests None declared.

Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Read the full text or download the PDF:

• Open access
• Published: 07 October 2022

Advances in e-learning in undergraduate clinical medicine: a systematic review

• T. Delungahawatta 1 ,
• S. S. Dunne 1 ,
• S. Hyde 1 ,
• L. Halpenny 1 ,
• D. McGrath 1 , 2 ,
• A. O’Regan 1 &
• C. P. Dunne 1 , 2  

BMC Medical Education volume  22 , Article number:  711 ( 2022 ) Cite this article

4701 Accesses

10 Citations

3 Altmetric

Metrics details

E-learning is recognised as a useful educational tool and is becoming more common in undergraduate medical education. This review aims to examine the scope and impact of e-learning interventions on medical student learning in clinical medicine, in order to aid medical educators when implementing e-learning strategies in programme curricula.

A systematic review compliant with PRISMA guidelines that appraises study design, setting and population, context and type of evaluations. Specific search terms were used to locate articles across nine databases: MEDLINE/PubMed, ScienceDirect, EMBASE, Cochrane Library, ERIC, Academic Search Complete, CINAHL, Scopus and Google Scholar. Only studies evaluating e-learning interventions in undergraduate clinical medical education between January 1990 and August 2021 were selected. Of the 4,829 papers identified by the search, 42 studies met the inclusion criteria.

The 42 studies included varied in scope, cognitive domain, subject matter, design, quality and evaluation. The most popular approaches involved multimedia platforms (33%) and case-based approaches (26%), were interactive (83%), asynchronous (71%) and accessible from home (83%). Twelve studies (29%) evaluated usability, all of which reported positive feedback. Competence in use of technology, high motivation and an open attitude were key characteristics of successful students and preceptors.

Conclusions

Medical education is evolving consistently to accommodate rapid changes in therapies and procedures. In today’s technologically adept world, e-learning is an effective and convenient pedagogical approach for the teaching of undergraduate clinical medicine.

Peer Review reports

E-learning, a pedagogical approach supported by the principles of connectivism learning theory, involves the use of technology and electronic media in knowledge transfer [ 1 , 2 ]. Connectivism views knowledge as a fluid entity circulated through technology enabled networks that foster interactions between individuals, organizations, and societies at large [ 2 ]. Based on this conceptual framework, medical curricula can potentially benefit from enhanced communication and knowledge exchange using technology.

Common e-learning instructional designs in clinical medicine include “online and offline computer-based programmes, massive open online courses, virtual reality environments, virtual patients, mobile learning, digital game-based learning and psychomotor skills trainers”[ 1 ]. To maximize the potential for e-learning, it seems rational that the roles and needs of the e-learner, e-teacher and host institution should be defined and appreciated. According to the Association for Medical Education in Europe (AMEE), an e-learner is any individual taught in an online learning environment [ 1 ]. As the role of the e-learner is central to the learning process, effective e-learning strategies should consider potential learning challenges encountered by the e-learner. Employing skilled e-teachers and providing them with sufficient supports are also important considerations. Furthermore, institutional management of the content versus process elements of educational technology use should best align with the objectives of the program [ 1 ]. For example, if the intent is to provide student access to digital content, then managing sound or video files, podcasts, and online access to research papers, clinical protocols, or reference materials, should be prioritized. On the other hand, if the focus is on student participation in digital activities, then managing processes such as discussion boards and test-taking should take precedence. Accounting for the role of the e-learner, e-teacher, and host institution in this manner, can result in successful implementation of an e-learning system. In fact, e-learning has been shown to be at least as effective as, and can serve as an adjunct to, face-to-face teaching and learning methods [ 3 , 4 , 5 ].

An institution may choose to employ educational technologies for the entirety of the course or provide a combination of online and in-class interactions, with the latter approach referred to as ‘blended learning’ [ 1 ]. Incorporation of e-learning into the curriculum allows for new avenues of interactive knowledge and skill transfer between teachers and students and amongst students. Interactions are not limited to face-to-face conversations but can involve text, audio, images, or video, thereby enriching the learning experience. Giving access to a greater breadth of learning resources further develops lifelong learning skills in students as they are required to independently evaluate and extract the pertinent information [ 1 ]. E-learning interventions can also be accessed at any time from almost any location, which facilitates a student-centred approach through self-directed and flexible learning [ 6 ]. As such, e-learning is an attractive instructional undergraduate health education approach [ 7 ].

To date, e-learning interventions in the sciences, particularly anatomy [ 8 ] and physiology [ 9 ], and postgraduate medical training [ 3 , 4 ] have been described. However, their use has not been reviewed systematically in the specific context of augmenting, enhancing or supporting student learning in undergraduate clinical medicine [ 10 ], or replacing face-to-face learning with online learning in the case of COVID-19 emergency remote teaching. In 2014, survey responses from senior medical students in Illinois, reported use of online collaborative authoring, multimedia, social-networking, and communication tools as point of opportunity study resources during clinical rotations [ 11 ]. Additionally, the COVID-19 pandemic has necessitated stepping away from traditional classroom and bedside teaching, and development of more flexible course delivery. A recent survey by Barton et al. collected 1,626 responses from medical students across 41 medical schools in the United Kingdom during the COVID lockdown. Results of study resources accessed daily showed that 41.6% of students used information provided by university (PowerPoint lecture slides, personal notes), 29.6% accessed free websites and question banks, and 18.4% accessed paid websites and question banks [ 12 ]. The work therefore suggests a strong tendency for students to supplement university materials with online resources [ 12 , 13 ]. The popularity of online learning platforms seems to stem from an association with achieving higher exam scores [ 14 , 15 ], ability to self-monitor knowledge gaps [ 16 ], improved knowledge retention from repeat exposure [ 17 , 18 ], and to practice exam technique [ 16 ].

Medical school educators are, therefore, called to evaluate e-learning approaches and to consider incorporation of suitable strategies into current curricula to ensure equitable access and student success. Thus, we aimed to systematically review the scope and impact of e-learning interventions published regarding undergraduate clinical medicine, and to inform medical educators of the effectiveness and character of various online learning environments.

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines are used for the reporting of this systematic review [ 19 ]. The PRISMA checklist is included as Additional File 1 .

Search methods

The early 1990s marked the commercial availability of computer-based learning multimedia [ 20 ] as well as the emergence of online education programs [ 21 ]. Thus, medical subject headings (MeSH), key words and specific database headings were used to locate articles published between January 1990 and August 2021: ‘e-learning’ or ‘digital resources’ or ‘internet learning resources’ AND ‘medical education’ AND ‘undergraduate’ AND ‘techniques’ or ‘programmes’ or ‘interventions’. The search was piloted on PubMed and adapted subsequently for the databases. A total of nine databases were searched: MEDLINE/PubMed, ScienceDirect, EMBASE, Cochrane Library, ERIC, Academic Search Complete, CINAHL, Scopus, Google Scholar and grey literature. The bibliographies of each selected paper were searched manually for further studies. Websites of medical education organisations were searched for position statements and guidelines, including the Association for the Study of Medical Education, AMEE and the British Medical Journal.

Inclusion and exclusion criteria

Only studies in the English language that evaluated an e-learning intervention in subjects related to clinical medicine were selected. These included: family medicine, surgery, internal medicine, radiology, psychiatry, dermatology, paediatrics and obstetrics. Studies that did not involve undergraduate medical students, were based on pre-clinical sciences or were not focussed on an e-learning intervention were excluded. Studies that focussed on the use of internet for assessment and course administration only were not included. Additionally, studies that described interventions but not their evaluation were excluded. Of the 4,829 papers identified by the search, 42 studies were deemed eligible for inclusion in this review.

Data extraction and analysis

AMEE guidelines on e-learning interventions [ 1 ] were used to modify a previous data extraction tool that had been used in a systematic evaluation of effectiveness of medical education interventions [ 22 ]. This was subsequently piloted and refined by three of the authors until consensus was achieved to form the data extraction tool (see Additional File 2 ). With application of connectivism, individual elements of e-learning were identified to infer and appreciate their collective effects on the learning process. More specifically, data was extracted by examining two central questions: how and when to use e-learning in undergraduate clinical medical education. The primary outcomes relating to how to use e-learning were: instructional features that made the e-learning intervention effective; usability features; assessment of effectiveness and quality of the intervention. Primary outcomes relating to when were: the context, and the learner and preceptor characteristics. In addition to the outcomes measured, descriptive data was also extracted to summarise the studies including: the study design, setting and population; context and discipline; type of evaluations. All selected papers were filed in an Endnote library and the data extraction tool for each was stored in an Excel file, a summary of which is provided as Additional File 2 and Additional File 3 .

Guidelines for evaluating papers on medical education interventions from the Education Group for Guidelines on Evaluation were used as a framework to assign a global score for the strength of each paper [ 23 ]. Among these guidelines, significant value is placed on development of strong intervention rationale and intervention evaluation methods [ 23 ]. The impact of the evaluation was also measured using Kirkpatrick’s levels, a recognised system of understanding the effect of interventions [ 24 ]. The first level, reaction, is a measure of learner satisfaction with the intervention [ 24 ]. The second Kirkpatrick level, learning, is a measure of change in knowledge, skills, or experience. The third Kirkpatrick level of behaviour is a measure of behavioural change. The final level, results, is a measure of overall impact on the organization (i.e., improved quality of work, reduction in time wasted, better patient care).

Search results

A total of 4,829 papers were retrieved from database and manual searches, and this number was reduced to 42 after removal of duplicates and application of inclusion/exclusion criteria at set stages (see Fig.  1 for the PRISMA flow diagram). Two papers were retrieved from manual searches of bibliographies [ 25 , 26 ]. The main reasons for excluding studies were a lack of focus on undergraduate medical students (112 studies) or absence of an e-learning intervention (34 studies).

PRISMA flow diagram

Design of included studies

The year of publication ranged from 2003 to 2021, with most conducted within the past ten years (31 studies). Interventions were conducted in nine different countries, mainly the United States (13 studies) and Germany (9 studies). More than half of the studies were conducted in the European Union (21 studies). Several research designs were described, including 17 observational studies [ 25 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 ], 13 randomised control trials [ 26 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 ], three non-randomised control trials [ 55 , 56 , 57 ], eight qualitative studies [ 58 , 59 , 60 , 61 , 62 , 63 , 64 , 65 ], and one mixed methods study [ 66 ]. Thirteen of the total studies included data collection both pre- and post- intervention [ 25 , 27 , 31 , 34 , 36 , 38 , 39 , 45 , 48 , 52 , 53 , 54 , 61 ]. Six studies had follow-up data (collected weeks to months after intervention) [ 34 , 45 , 49 , 52 , 54 , 56 ] and twelve papers reported ethical approval [ 28 , 29 , 30 , 31 , 33 , 34 , 39 , 40 , 42 , 46 , 49 , 54 ]. Furthermore, eight studies described learning theories in the development or evaluation of medical curricula [ 29 , 30 , 33 , 49 , 51 , 52 , 56 , 58 ]. Of these studies, five referenced constructivism [ 29 , 49 , 51 , 52 , 58 ] three studies highlighted cognitivism [ 30 , 56 , 59 ], and one study evaluated behaviourist learning theory [ 33 ].

Study population

Students in the third year of medical school experiencing clinical exposure were the most commonly studied (sixteen studies), with fourteen studies involving multiple cohorts of students (see Additional File 3 ). Sample sizes ranged from 10 to 42,190 individuals. The most common disciplines investigated were interdisciplinary (13 studies), surgery (8 studies), radiology (7 studies), and dermatology (4 studies) (see Fig.  2 Intervention Discipline).

Intervention discipline

Intervention characteristics

Twelve types of intervention were described and the most commonly used were multimedia platforms (fourteen studies) and case-based learning (eleven studies), as per Additional File 2 and Fig.  3 . In terms of cognitive domain, 27 interventions were in the domain of knowledge [ 25 , 26 , 27 , 29 , 30 , 32 , 33 , 34 , 35 , 39 , 40 , 42 , 43 , 47 , 48 , 50 , 52 , 53 , 54 , 57 , 60 , 61 , 62 , 63 , 64 , 66 , 67 ]; eight were in the domain of skills [ 9 , 30 , 31 , 36 , 37 , 46 , 49 , 51 ] and seven in combined knowledge and skills [ 38 , 41 , 44 , 45 , 56 , 59 , 65 ]. The interventions ranged in duration from a single session to a complete academic year. Thirteen of the interventions were synchronous, where users log on at a given time [ 8 , 26 , 27 , 31 , 33 , 34 , 37 , 43 , 47 , 51 , 52 , 58 , 66 ], and the remaining 29 used an asynchronous platform (users logging on independently in their own time). Seven were accessible in a classroom setting only [ 26 , 27 , 36 , 47 , 52 , 58 , 66 ] while the others could be accessed from home (Fig. 4 ).

Intervention type

Reported roles for e-learning within the curriculum included a revision aid for examinations [ 58 ]; the flipped classroom concept [ 44 , 57 ], whereby lectures held after an e-lecture become an interactive session; to facilitate an online community where knowledge could be discussed/ shared [ 25 ]; and, enabling just-in-time learning through timely access to facts [ 30 , 31 , 37 ]. Seven (17%) of the 42 interventions were didactic in approach [ 27 , 30 , 37 , 55 , 57 , 63 , 65 ], while the others were interactive. Twelve studies described a collaborative approach, whereby students discussed cases and problems with one another and engaged in role-plays [ 25 , 26 , 36 , 38 , 40 , 41 , 42 , 46 , 52 , 59 , 61 , 66 ]. The context of e-learning in relation to the curriculum was not stated in ten of the studies but another thirteen studies used the terms “adjunct”, “complement”, “supplement”,”hybrid” and “blended” to illustrate the common theme of integrating e-learning with traditional learning [ 25 , 29 , 30 , 32 , 44 , 45 , 46 , 47 , 50 , 56 , 57 , 58 , 62 , 63 ]. Seven studies describe temporary replacement of traditional curricula with e-learning platforms in response to COVID-19 [ 33 , 40 , 41 , 42 , 61 , 62 , 64 ]. Eight studies described a pilot phase or the inclusion of students in the development of the intervention [ 33 , 37 , 44 , 45 , 48 , 49 , 53 , 66 ]. Nineteen of the interventions had a built-in assessment, with multiple choice questions being used in most cases, to evaluate whether an improvement in learning had taken place [ 25 , 27 , 31 , 34 , 37 , 39 , 43 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 54 , 55 , 59 , 66 ]. Justification for the chosen assessment strategy or a statement on its suitability was included in two studies [ 50 , 66 ]. Kourdioukova et al. reported an improvement in knowledge and skills with computer supported collaborative case-based approach as judged by in-built multiple-choice questions (MCQ), suggesting the importance of content-specific scripting [ 66 ]. Schneider et al. used a combination of MCQ and survey, and justified their use by demonstrating that learning improved with the intervention compared to the control [ 50 ]. Five of the interventions used end of module assessments as the marker of quality [ 26 , 29 , 53 , 56 , 57 ], with one stating that this was not a suitable mechanism due to its inability to assess the students’ ability to take a patient history or perform a clinical examination [ 53 ].

Intervention evaluation

Each study was given a global rating from 1–5 based on guideline criteria from the Education Group for Guidelines on Evaluation, including whether learning outcomes and curricular context were outlined and the power and rigor of the studies [ 23 ] (Additional File 2 ). Accordingly, eleven studies scored 4/5; two scored 3.5/5; twelve studies scored 3/5; twelve studies scored 2.5/5; and five scored 2/5 (σ = 0.138).

Intervention effectiveness and acceptability

Nine studies described an impact matching a Kirkpatrick level 1, where the student reaction to e-learning intervention was evaluated using student surveys or questionnaires [ 32 , 35 , 44 , 58 , 60 , 61 , 62 , 64 , 65 ]. All these studies report that most students were satisfied with the addition of an e-learning intervention. For instance, Orton et al. note that over 91% of survey responses either ‘strongly agreed’ or ‘agreed’ that use of computer-based virtual patients enabled learning [ 35 ].

Twenty-one (50%) of the 42 studies evaluated acceptability [ 26 , 30 , 32 , 33 , 36 , 37 , 40 , 41 , 42 , 44 , 48 , 53 , 54 , 55 , 56 , 57 , 58 , 63 , 65 , 66 , 68 ]. Of these, 17 reported that the intervention was acceptable. A neutral attitude was reported to a radiology e-learning intervention that involved peer collaboration and was found to be time consuming[ 66 ]. Attitude in another study was much more favourable in junior years than in senior years, with the authors commenting on the conflict between completing assignments and preparing for high stakes examinations [ 55 ]. Another study that focussed on acceptability, with positive outcomes, found that perceived utility and ease of use were the key factors [ 30 ]. Twelve (57%) of the 21 studies further evaluated usability [ 30 , 36 , 37 , 40 , 41 , 42 , 44 , 53 , 56 , 57 , 58 , 65 ], all with positive outcomes, but only one used a formal usability assessment tool [ 58 ]. In that study, Farrimond et al. found that a usable intervention should be: simple and intuitive to use and, from a learner perspective, interactive and enjoyable [ 58 ]. In the development of virtual lectures, ease of navigation, audio-visual quality and accessibility were the key usability features [ 57 ]. Wahlgren et al. concluded that as well as navigation, interactivity is a priority for e-learning development [ 53 ]. Regarding mobile learning, the display should be adaptable to varying screen sizes, termed ‘chunking’, and it should be suitable for a number of platforms [ 30 ].

Twenty-nine (69%) of the 42 studies described an impact matching a Kirkpatrick level 2, where evaluation of whether learning took place was assessed through post intervention scores [ 25 , 27 , 31 , 36 , 38 , 39 , 47 , 48 , 50 , 52 , 53 , 54 , 56 , 57 , 61 ], final exam results [ 26 , 29 , 45 , 66 ], direct observation [ 28 , 31 , 33 , 43 , 46 , 51 , 55 ] and student survey [ 25 , 26 , 30 , 37 , 38 , 39 , 40 , 41 , 42 , 45 , 48 , 49 , 53 , 54 , 56 , 65 , 66 ]. Among these studies, two studies had included both pre- and post- intervention evaluations but neither had a control group nor longer term follow-up [ 25 , 27 ]. One randomised control trial showed a statistically significant improvement in factual knowledge acquisition after participation in an online module as judged based on performance in end of year assessments, compared to a traditional teaching control group (84.8% ± 1.3 vs. 79.5% ± 1.4, p  = 0.006, effect size 0.67) [ 26 ]. Likewise, Davis et al., found that the use of a procedural animation video on mobile device resulted in higher medical student scores on skills checklist (9.33 ± 2.65 vs. 4.52 ± 3.64, p  < 0.001, effect size 1.5) [ 30 ]. Similarly, in Sijstermans et al., mean students’ self-evaluation of their skills using five-point Likert scale questionnaire, before and after two patient stimulations showed improvement (3.91 ± 0.28 vs 3.56 ± 0.34, P  < 0.0001, effect size 1.12). Furthermore, in one study employing a problem-based e-learning approach, the number of first-class honours awarded were found to be significantly improved when compared to control group [ 29 ]. However, in another study using a problem-based e-learning intervention, no significant difference was found between control and intervention groups in subsequent examinations ( p  = 0.11) [ 53 ]. In contrast, Al Zahrani et al. found that delivery of new e-learning platforms (Blackboard Collaborate, ZOOM) in response to COVID-19 was poorly accepted by students, whereby 59.2% did not feel adequately educated on learning outcomes, 30% felt no educational difference between e-learning and traditional curriculums, and 56.1% felt e-learning is insufficient as an educational tool for the health sciences [ 40 ].

Four studies demonstrated a change in student behaviour in line with Kirkpatrick level 3 [ 50 , 52 , 59 , 63 ]. In de Villiers et al., it was found that students were using podcasts to learn course content and the classroom teaching setting to strengthen their understanding, inadvertently accepting the flipped classroom approach [ 63 ]. In Sward et al., students who were assigned to a gaming intervention were more willing to engage in answer creating and answer generating as well as independent study of subject materials prior to session time [ 52 ]. Similarly, in Schneider et al., students in the computer case-based intervention group were found to invest more time into studying course subjects (38.5 min vs 15.9 min) which resulted in significantly higher test scores [ 50 ]. Finally, in Moriates et al., following the integration of value-based modules, students have reported increased awareness of patient needs and discussions with peers regarding value-based decision-making during clerkship [ 59 ].

Learner and preceptor characteristics

Learner characteristics identified to enable successful e-learning include: good digital skills, less resistance to change [ 32 ] and a willingness to collaborate with peers [ 66 ]. Preceptor characteristics were not described in most of the studies, but the role involved guiding students through their learning [ 33 , 46 , 61 , 66 ], selection of topics of broad interest to students [ 60 ], technical support [ 54 ], student evaluation[ 28 , 31 , 37 , 40 , 42 , 45 , 46 , 49 , 51 ], content development and management [ 32 , 41 , 42 , 46 , 54 , 62 ] and providing feedback and clear instruction on what is expected of the learners [ 28 , 37 , 40 , 42 , 51 , 54 , 60 ].

The COVID-19 pandemic resulted in global university closures during periods of lockdown, necessitating educators to quickly adopt alternate pedagogical approaches. As a result, there has been a substantial increase in the use of e-learning, by which teaching and learning activities occur at a distance on online platforms [ 69 ].

In enabling a shift in the control of knowledge acquisition and distribution from the teacher to the student, e-learning facilitates the learning process. Learners filter the available information, develop new perspectives, log into networks to share their understanding, and repeat the cycle [ 2 ]. This view of learning as a fluid and dynamic process is the basis of the learning theory of connectivism and highlights the benefit of this instructional design in medical education – a field amenable to rapid changes in therapies and procedures. In fact, educational theorists have significantly influenced the development of medical curricula throughout history. Amongst the 25 higher impact studies (achieving a global score greater or equal to 3), only 7 studies (28%) were found to have described theoretical underpinnings [ 30 , 33 , 49 , 51 , 52 , 58 , 59 ]. Initially, the behaviourist perspective supported pedological practices [ 70 ]. Behaviourism described learning as largely deriving from responses to external stimuli and led to curricula aimed to influence behaviour through reward and positive and negative reinforcement. In one study reviewed, the lack of direct observation of non-verbal communication by instructors was seen as a significant learning challenge in the virtual environment [ 33 ]. A shift from behaviourism to cognitivism later ensued with the belief that the brain is much more than a ‘black box’ and learning rather involved mental processing and organization of knowledge, and memory functions [ 70 ]. With the recognition of individual differences in the learning process, online systems attempted to introduce interventions that suited multiple learning strategies. For example, learning from auditory narration with animation was found to be more effective than use of text with animation [ 71 ]. This review further highlighted the impact of repetition [ 30 ] and clinical reasoning [ 56 , 59 ] on the learning process. More recently, constructivist learning theory and the perception that learners incorporate new information into pre-existing knowledge schemas has greatly contributed to reformation of medical education [ 70 ]. Incorporating real world connections [ 29 , 49 , 58 ], building on motivations [ 52 ], application of feedback [ 51 ] and continuous reflection [ 49 ] has been noted in this review as important factors in knowledge handling and retention. Presently, e-learning interventions often utilize aspects of more than one theoretical perspective. For instance, problem-based learning interventions have emphasised the critical thinking processes of cognitivism and the self-direction of constructivism [ 29 ]. While primary studies have increased the reporting of underlying theory over time, there is still a significant lack of discussion – future work should reference theoretical principles to objectively frame and assess online education.

In addition to recognizing the needs of the e-learner, identifying required skills of e-teachers and developing content that appropriately supplement the curriculum are vital to ensuring successful implementation of an e-learning system [ 1 ]. Therefore, this study involved review of studies published between 1990 and 2021, assessing the effectiveness and character of various online learning environments in undergraduate clinical medical education. Specifically, these studies involved medical students pursuing medicine as a primary degree and those enrolled with prior degrees.

Intervention design

Critical appraisal of the collected studies using EGGE criteria, identified seventeen studies (40%) meeting a global rating of less than 3. The EGGE criteria encompass a standardized framework by which quality indicators can be recognized. Lower ratings of included studies suggests that conducting and reporting of e-learning interventions is largely lacking in methodological rigour and therefore limits transferability of study results. This finding is consistent with conclusions from a review by Kim et al., describing how most of the existing literature on e-learning interventions have little quantitative data, evaluate a limited range of outcomes and have significant gaps in study designs [ 72 ]. Additionally, only 13 (31%) randomized control trials (RCTs) were included in the review [ 26 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 ]. Amongst these studies, five reported pre and post test scores [ 45 , 48 , 52 , 53 , 54 ], three of which report long term follow up [ 45 , 52 , 54 ]. Interestingly, all the RCTs report no significant differences in knowledge mastery between control and intervention groups. However, in the immediate short term, e-learning interventions were associated with greater learner satisfaction. For example, in Lee et al., mobile learning with interactive multimedia had higher satisfaction scores compared with conventional Microsoft PowerPoint Show content, despite non-significant differences in knowledge gain [ 48 ]. Similarly, in the study by Wahlgren et al., the majority of students in the intervention group reported that the interactive computerised cases enabled better understanding of disease diagnosis and management, particularly referencing the user-friendliness and feedback [ 53 ]. Yet, knowledge gain as assessed by post-intervention examination scores did not show statistically significant differences between the two groups. Systematic reviews examining the effect of e-learning on nursing education have also demonstrated no differences between e-learning and traditional teaching modalities but report high satisfaction rates with the former [ 73 , 74 ]. While these studies suggest that e-learning is as effective as traditional educational methods, higher student satisfaction levels are indicative of more effective learning programs [ 75 ]. Therefore, the lack of longitudinal data may limit our ability to accurately evaluate the impact of e-learning technologies.

Many of the studies in this review used virtual patient and case-based pedagogical methods reflecting an educational trend towards more critical thinking [ 76 ]. Thirty-five of the interventions under review used an interactive approach, encouraging a style in which students collaborated and discussed ideas with their peers and tutors, the importance of which has been recognised [ 77 ]. Two studies of mobile learning identified wasted time for students as a concern that could be addressed by allowing immediate access to information that would soon be required [ 30 , 55 ]. This ‘just in time learning’, defined as a “brief educational experience targeting a specific need or clinical question” [ 78 ], can be facilitated through e-learning. Ten of the included studies concluded that an integrated approach works best, whereby educators do not seek to replace traditional methods but rather supplement them. This has previously been described as a ‘blended-learning’ style [ 77 ]. A recent study suggests that students thrive in blended- versus self-directed virtual reality environments due to face-to-face teacher support [ 79 ].

Despite variability in methodological design, several studies of e-learning across domains of education, politics, business, and military training have shown knowledge gains assessed by pre- versus post-intervention tests [ 80 ]. Similarly, subjects within the studies we have reviewed have reported e-learning interventions to be conducive to learning [ 32 , 35 , 36 , 44 , 58 , 60 , 61 , 62 , 64 , 65 ], have demonstrated improvements in learning [ 25 , 26 , 27 , 29 , 30 , 31 , 34 , 36 , 37 , 38 , 39 , 43 , 46 , 48 , 49 , 54 , 55 , 56 , 57 , 66 ] and modified learning strategies [ 50 , 52 , 63 ]. The specific features of e-learning strategies most likely to enhance the learning experience may include: peer-to-peer learning [ 52 ], making use of wasted time [ 30 , 40 , 41 , 42 , 81 ], feedback from clinicians and ongoing technical support [ 32 , 82 ], consolidation of information and skill through repetition [ 52 , 82 , 83 ], and convenience of online content access [ 25 , 30 , 40 , 41 , 42 ]. Usability of the intervention has specifically featured strongly in this review. Vital features of e-learning interventions facilitating its use may include: interactive software, active learning promotion (built-in quizzes following cases), asynchronous use, multimedia platforms (i.e., slideshows, videos, images), ease of use and adaptability [ 76 , 81 , 84 ]. Unsurprisingly, students are more engaged with educational material after the typical 9-to-5 work hours [ 25 , 35 ]. Whereas traditional learning opportunities may be restricted to these hours, the flexibility of being able to access online resources outside of this timeframe, may better facilitate achievement of learning objectives [ 25 , 35 ]. Additionally, the use of discussion boards [ 78 ] and games [ 77 ] may facilitate active learning and feedback to be sought and received in a timely manner. Furthermore, quality assurance is recognized as a critical factor, and if considered at the planning stage of an intervention and built into e-learning interventions, may lead to more favourable outcomes [ 23 ]. Engagement with students in this manner is in keeping with the AMEE recommended goals of e-learning [ 1 ]. Several studies also highlight how online learning might provide an encouraging environment for the development of knowledge and skills, relatively easily tailored to individual learning preferences and prior knowledge, and with the possibility of compensating for a lack of accessibility of patients or teachers [ 35 , 36 , 38 , 63 , 85 ]. Furthermore, the ability to access an extensive network of additional resources may allow students to take control of their learning and regulate the volume of information studied [ 36 ].

While our review found improved learning outcomes, other systematic reviews assessing the effectiveness of technology and electronic media in health education, report equivocal findings [ 77 , 86 ]. Proposed factors that may limit learning capacity include: hesitancy to adopt changes by students and teachers, poor technical or financial support, limited technological skills, and the lack of direct and personalized teacher communication [ 25 , 32 , 82 , 87 ]. For example, Davies et al. suggests that an open outlook on mobile device usage was required by students and clinicians, to limit non-use and acquire potential benefits [ 30 ]. In another study conducted by Alsoufi et al., online medical education programs implemented in Libya in response to COVID-19 were found to be negatively received by respondents [ 87 ]. Financial and technical barriers and the lack of hands-on bedside teaching were stated by respondents as limitations to acceptance of e-learning. The shift to online medical learning in the Philippines during the COVID-19 pandemic also identified lack of access to computers and the internet as a significant barrier [ 82 ]. Of course, with these later interventions, the rapid onset of the pandemic required development of e-learning platforms with relatively little training and preparation. As such, the logistics of e-learning curricula as it pertains to specific communities may not have been foreseen. Another reason for such discrepancies may be the underlying discipline in which the intervention is being evaluated [ 47 ]. For instance, the use of only e-learning materials when teaching new skills may not be sufficient, as the direct observation and guidance of an expert is valuable [ 88 ]. A blended-learning environment may be more appropriate in these circumstances [ 47 ]. Indeed, viewing e-learning as a complement rather than replacement of traditional approaches is already well accepted amongst students [ 80 ].

Learner, preceptor and institution characteristics

The twenty-first century learners are known to be avid consumers of various digital platforms. However, studies have shown an incongruence between their ability to use technology for entertainment and ability to use it for educational purposes [ 89 ]. Most students require guidance to synthesize information and create new understanding. In fact, students in middle school through undergraduate level studies have consistently demonstrated poor digital research skills [ 90 , 91 ]. Furthermore, students may require adjustment of learning practices to best engage with the presented e-learning platform. For example, use of PowerPoint presentations or handouts in replacement of in-class teaching can cause visual and auditory learners to require more time to comprehend the information [ 82 ]. Therefore, in addition to carrying an acceptant attitude and a willingness to collaborate with peers, the ability to engage with and extract relevant content from online resources, is a characteristic linked to success in e-learning [ 32 , 66 ].

Nevertheless, recognition of the need for continued mentoring and support in the online learning environment, requires appreciation of the role of the e-teacher. Preceptors’ roles involve development and delivery of the intervention and acting as a resource person for the duration of the module [ 68 ]. In our previous discussion of e-learning strategy effectiveness, two further roles of the e-teacher can be recognized. Firstly, the e-teacher is instrumental in providing timely feedback, one of the main features associated with improved e-learning outcomes [ 32 ]. E-teachers should actively monitor student activity and provide feedback or support where needed [ 92 ]. Secondly, success of e-learning is also strongly related to the motivation of the students and indirectly the motivation demonstrated by the e-teacher [ 30 , 92 ]. The ARCS motivational model highlights four components needed to create a highly motivational e-learning system: maintain student attention, content relevance, student confidence, student satisfaction [ 93 ]. If e-teachers can convey subject material through strategies which encompass use of interactive multimedia, humour, and inquiry for instance, they can satisfy the first component of attention [ 92 ]. Generating activities that best illustrate main ideas, tailoring to the learner knowledge level and providing positive feedback are examples of methods to instil content relevance, student confidence and student satisfaction, accordingly. In Gradl-Dietsch et al., combination of video-based learning, team-based learning and peer-teaching, along with practical skills teaching in point of care ultrasound, feedback from peer teachers, and positive instructor-learner interactions, collectively fulfil the components of the ARCS model [ 54 ]. In Sox et al., the use of a web-based module to teach oral case presentation skills satisfied student attention and content relevance [ 51 ]. However, poor adherence to module largely due to time constraints, can be suggestive of poor student satisfaction. As a result, student confidence and the quality of oral case presentations did not differ from controls (faculty-led feedback sessions). As suggested by the authors, a combination of web module with direct faculty feedback may better instil student confidence and satisfaction with module content, and thereby improve student performance [ 51 ]. Recent studies have shown that the digital literacy skills of most instructors are inadequate [ 90 , 91 ]. Therefore, institutions need to invest into the provision of training programs and supports to allow e-teachers to develop and strengthen competencies needed to sufficiently handle educational technologies [ 92 , 94 , 95 ]. For example, the use of offline tablet-based materials was shown to improve medical education in Zambia, but reported usage amongst healthcare workers was low [ 95 ]. Authors suggest that a lack of training in tablet use was the underlying reason. Taken together, while the role of the teacher has changed compared to traditional pedological approaches, their actions can still heavily influence student learning outcomes.

Limitations and future directions

In a field where technology is changing faster than studies can be completed and interventions are evolving rapidly, medical education research has become a challenging topic of debate. Research can “provide the evidence to prove—and improve—the quality and effectiveness of teaching” and therefore advise the restructuring of curricula to respond to advances in science and technology [ 96 ]. In this review, 29 studies received a global score of 3 or less out of 5, highlighting a lack of transparency and rigour in most of the studies. This justifies a need for a standardised approach for reporting medical education interventions. Pre- and post-intervention testing is informative, but follow-up months later would be an important measure of knowledge retention and therefore intervention effectiveness. Moreover, most of the studies in this review examined knowledge or skill development but few examined higher Kirkpatrick levels. The inclination towards focus on the lower levels of the Kirkpatrick model may stem from difficulty following students in the field to evaluate long-term results of the educational intervention on student behaviours (level three) and the organization at large (level four) [ 97 ]. Future work on the evaluation of associated changes in behaviour, professional practice or patient outcomes would be valuable. Other e-learning characteristics that can be evaluated in future work (Fig. 4 ) may include the capacity for adaptivity (to accommodate changing student needs and performance) and collaboration [ 98 ]. Including descriptions of curricula context can also facilitate the exploration of which e-learning strategies are best suited for specific medicine disciplines and socioeconomic settings. The use of internet resources by both students and patients alike, and the exponential growth in social media influence may also provide a platform for future e-learning interventions [ 99 ].

Future intervention design recommendations

Over the past twenty years and with the recent advent of the COVID-19 pandemic, there has been a substantial increase in the use of e-learning. This review found that e-learning interventions are positively perceived by students and associated with improvements in learning. Improved learning outcomes are closely correlated with interactive, asynchronous, easily accessible and usable interventions, and those involving students and preceptors with digital skills, high motivation and receptive attitudes. While further exploration of the strengths and weaknesses of e-learning technologies is warranted, use of online platforms is a creditable educational tool for undergraduate clinical medicine.

Abbreviations

Association for Medical Education in Europe

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

Ellaway R, Masters K. AMEE Guide 32: e-Learning In Medical Education - Part 1: Learning. Teaching And Assessment Med Teach. 2008;30(5):455–73.

Google Scholar  

Goldie JG. Connectivism: A Knowledge Learning Theory For The Digital Age? Med Teach. 2016;38(10):1064–9.

Article   Google Scholar  

Maertens H, Madani A, Landry T, Vermassen F, Van Herzeele I, Aggarwal R. Systematic Review Of E-Learning For Surgical Training. Br J Surg. 2016;103(11):1428–37.

Tarpada SP, Morris MT, Burton DA. E-Learning In Orthopedic Surgery Training: A Systematic Review. J Orthop. 2016;13(4):425–30.

Feng J-Y, Chang Y-T, Chang H-Y, Erdley WS, Lin C-H, Chang Y-J. Systematic Review of Effectiveness of Situated E-Learning on Medical and Nursing Education. Worldviews Evid Based Nurs. 2013;10(3):174–83.

Price Kerfoot B, Masser BA, Hafler JP. Influence Of New Educational Technology On Problem-Based Learning At Harvard Medical School. Med Educ. 2005;39(4):380–7.

Guarino S, Leopardi E, Sorrenti S, De Antoni E, Catania A, Alagaratnam S. Internet-Based Versus Traditional Teaching And Learning Methods. Clin Teach. 2014;11(6):449–53.

Trelease RB. From Chalkboard, Slides, And Paper To E-Learning: How Computing Technologies Have Transformed Anatomical Sciences Education. Anat Sci Educ. 2016;9(6):583–602.

Felder E, Fauler M, Geiler S. Introducing E-Learning/Teaching In A Physiology Course For Medical Students: Acceptance By Students And Subjective Effect On Learning. Adv Physiol Educ. 2013;37(4):337–42.

Martin EA. Concise medical dictionary. 9th ed. Oxford: Oxford University Press; 2015.

Book   Google Scholar  

Han H, Nelson E, Wetter N. Medical students’ online learning technology needs. Clin Teach. 2014;11(1):15–9.

Barton J, Rallis KS, Corrigan AE, Hubbard E, Round A, Portone G, et al. Medical students’ pattern of self-directed learning prior to and during the coronavirus disease 2019 pandemic period and its implications for Free Open Access Meducation within the United Kingdom. J Educ Eval Health Prof. 2021;18:5.

O’Hanlon R, Laynor G. Responding to a new generation of proprietary study resources in medical education. J Med Libr Assoc. 2019;107(2):251–7.

Zhang J, Peterson RF, Ozolins IZ. Student approaches for learning in medicine: What does it tell us about the informal curriculum? BMC Med Educ. 2011;11(1):87.

Cooper AL, Elnicki DM. Resource utilisation patterns of third-year medical students. Clin Teach. 2011;8(1):43–7.

Wynter L, Burgess A, Kalman E, Heron JE, Bleasel J. Medical students: what educational resources are they using? BMC Med Educ. 2019;19(1):36.

Larsen DP, Butler AC, Roediger HL 3rd. Test-enhanced learning in medical education. Med Educ. 2008;42(10):959–66.

Augustin M. How to learn effectively in medical school: test yourself, learn actively, and repeat in intervals. Yale J Biol Med. 2014;87(2):207–12.

Moher D, Liberati A, Tetzlaff J, Altman DG. Reprint—Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. Phys Ther. 2009;89(9):873–80.

Hammond M, editor Learning From Experience: Approaching The Research Of CD-ROM In Schools. WCCE; 1995.

Kentnor HE. Distance Education and the Evolution of Online Learning in the United States. Curric Teach. 2015;17:21–34.

Barry Issenberg S, McGaghie WC, Petrusa ER, Lee Gordon D, Scalese RJ. Features And Uses Of High-Fidelity Medical Simulations That Lead To Effective Learning: A BEME Systematic Review. Med Teach. 2005;27(1):10–28.

Evaluation. EGfGo. Guidelines for Evaluating Papers on Educational Interventions. Education Group for Guidelines on Evaluation ed: BMJ; 1999. p. 1265–7.

Kirkpatrick D, Kirkpatrick J. Transferring Learning to Behavior : Using the Four Levels to Improve Performance. Oakland, United States: Berrett-Koehler Publishers, Incorporated; 2005.

Bernardo V, Ramos MP, Plapler H, De Figueiredo LF, Nader HB, Ancao MS, et al. Web-Based Learning In Undergraduate Medical Education: Development And Assessment Of An Online Course On Experimental Surgery. Int J Med Inform. 2004;73(9–10):731–42.

Raupach T, Munscher C, Pukrop T, Anders S, Harendza S. Significant Increase In Factual Knowledge With Web-Assisted Problem-Based Learning As Part Of An Undergraduate Cardio-Respiratory Curriculum. Adv Health Sci Educ Theory Pract. 2010;15(3):349–56.

Casillas JM, Gremeaux V. Evaluation Of Medical Students’ Expectations For Multimedia Teaching Materials: Illustration By An Original Method Using The Evaluation Of A Web Site On Cardiovascular Rehabilitation. Ann Phys Rehabil Med. 2012;55(1):25–37.

Cevik AA, Shaban S, El Zubeir M, Abu-Zidan FM. The Role Of Emergency Medicine Clerkship E-Portfolio To Monitor The Learning Experience Of Students In Different Settings: A Prospective Cohort Study. Int J Emerg Med. 2018;11(1):24.

Corrigan M, Reardon M, Shields C, Redmond H. “SURGENT” – Student E-Learning For Reality: The Application Of Interactive Visual Images To Problem-Based Learning In Undergraduate Surgery. J Surg Educ. 2008;65(2):120–5.

Davies BS, Rafique J, Vincent TR, Fairclough J, Packer MH, Vincent R, et al. Mobile Medical Education (Momed) - How Mobile Information Resources Contribute To Learning For Undergraduate Clinical Students - A Mixed Methods Study. BMC Med Educ. 2012;12:1.

de Sena DP, Fabricio DD, Lopes MH, da Silva VD. Computer-Assisted Teaching Of Skin Flap Surgery: Validation Of A Mobile Platform Software For Medical Students. PLoS ONE. 2013;8(7):e65833.

Howlett D, Vincent T, Gainsborough N, Fairclough J, Taylor N, Cohen J, et al. Integration Of A Case-Based Online Module Into An Undergraduate Curriculum: What Is Involved And Is It Effective? E-Learn. 2009;6(4):372–84.

Khalil R, Mansour AE, Fadda WA, Almisnid K, Aldamegh M, Al-Nafeesah A, et al. The Sudden Transition To Synchronized Online Learning During The COVID-19 Pandemic In Saudi Arabia: A Qualitative Study Exploring Medical Students’ Perspectives. BMC Med Educ. 2020;20(1):285.

Ogura A, Hayashi N, Negishi T, Watanabe H. Effectiveness of an e-Learning Platform for Image Interpretation Education of Medical Staff and Students. J Digit Imaging. 2018;31(5):622–7.

Orton E, Mulhausen P. E-Learning Virtual Patients For Geriatric Education. Gerontol Geriatr Educ. 2008;28(3):73–88.

Sijstermans R, Jaspers MW, Bloemendaal PM, Schoonderwaldt EM. Training Inter-Physician Communication Using The Dynamic Patient Simulator. Int J Med Inform. 2007;76(5–6):336–43.

Tews M, Brennan K, Begaz T, Treat R. Medical Student Case Presentation Performance And Perception When Using Mobile Learning Technology In The Emergency Department. Med Educ Online. 2011;16. https://doi.org/10.3402/meo.v16i0.7327 .

Wunschel M, Leichtle U, Wulker N, Kluba T. Using A Web-Based Orthopaedic Clinic In The Curricular Teaching Of A German University Hospital: Analysis Of Learning Effect, Student Usage And Reception. Int J Med Inform. 2010;79(10):716–21.

Zayed MA, Lilo EA, Lee JT. Impact of an Interactive Vascular Surgery Web-Based Educational Curriculum on Surgical Trainee Knowledge and Interest. J Surg Educ. 2017;74(2):251–7.

Al Zahrani EM, Al Naam YA, AlRabeeah SM, Aldossary DN, Al-Jamea LH, Woodman A, et al. E- Learning experience of the medical profession’s college students during COVID-19 pandemic in Saudi Arabia. BMC Med Educ. 2021;21(1):443.

Dost S, Hossain A, Shehab M, Abdelwahed A, Al-Nusair L. Perceptions of medical students towards online teaching during the COVID-19 pandemic: a national cross-sectional survey of 2721 UK medical students. BMJ Open. 2020;10(11): e042378.

Coffey CS, MacDonald BV, Shahrvini B, Baxter SL, Lander L. Student Perspectives on Remote Medical Education in Clinical Core Clerkships During the COVID-19 Pandemic. Medical Science Educator. 2020;30(4):1577–84.

Diekhoff T, Kainberger F, Oleaga L, Dewey M, Zimmermann E. Effectiveness Of The Clinical Decision Support Tool ESR Eguide For Teaching Medical Students The Appropriate Selection Of Imaging Tests: Randomized Cross-Over Evaluation. Eur Radiol. 2020;30(10):5684–9.

Dombrowski T, Wrobel C, Dazert S, Volkenstein S. Flipped Classroom Frameworks Improve Efficacy In Undergraduate Practical Courses – A Quasi-Randomized Pilot Study In Otorhinolaryngology. BMC Med Educ. 2018;18(1):294.

Hari R, Kälin K, Harris M, Walter R, Serra A. Comparing Blended Learning With Faculty-Led Ultrasound Training: Protocol For A Randomised Controlled Trial (The SIGNATURE Trial). Praxis (Bern 1994). 2020;109(8):636–40.

Herrmann-Werner A, Weber H, Loda T, Keifenheim KE, Erschens R, Mölbert SC, et al. “But Dr Google Said…” - Training Medical Students How To Communicate With E-Patients. Med Teach. 2019;41(12):1434–40.

Jenkins S, Goel R, Morrell DS. Computer-Assisted Instruction Versus Traditional Lecture For Medical Student Teaching Of Dermatology Morphology: A Randomized Control Trial. J Am Acad Dermatol. 2008;59(2):255–9.

Lee LA, Chao YP, Huang CG, Fang JT, Wang SL, Chuang CK, et al. Cognitive Style and Mobile E-Learning in Emergent Otorhinolaryngology-Head and Neck Surgery Disorders for Millennial Undergraduate Medical Students: Randomized Controlled Trial. J Med Internet Res. 2018;20(2): e56.

Plackett R, Kassianos AP, Kambouri M, Kay N, Mylan S, Hopwood J, et al. Online Patient Simulation Training To Improve Clinical Reasoning: A Feasibility Randomised Controlled Trial. BMC Med Educ. 2020;20(1):245.

Schneider AT, Albers P, Muller-Mattheis V. E-Learning in Urology: Implementation of the Learning and Teaching Platform CASUS(R) - Do Virtual Patients Lead to Improved Learning Outcomes? A Randomized Study among Students. Urol Int. 2015;94(4):412–8.

Sox CM, Tenney-Soeiro R, Lewin LO, Ronan J, Brown M, King M, et al. Efficacy of a Web-Based Oral Case Presentation Instruction Module: Multicenter Randomized Controlled Trial. Acad Pediatr. 2018;18(5):535–41.

Sward KARNP, Richardson SRNP, Kendrick JMD, Maloney CMDP. Use of a Web-Based Game to Teach Pediatric Content to Medical Students. Acad Pediatr. 2008;8(6):354–9.

Wahlgren CF, Edelbring S, Fors U, Hindbeck H, Stahle M. Evaluation Of An Interactive Case Simulation System In Dermatology And Venereology For Medical Students. BMC Med Educ. 2006;6:40.

Gradl-Dietsch G, Menon AK, Gürsel A, Götzenich A, Hatam N, Aljalloud A, et al. Basic Echocardiography For Undergraduate Students: A Comparison Of Different Peer-Teaching Approaches. Eur J Trauma Emerg Surg. 2018;44(1):143–52.

Davis JS, Garcia GD, Wyckoff MM, Alsafran S, Graygo JM, Withum KF, et al. Use Of Mobile Learning Module Improves Skills In Chest Tube Insertion. J Surg Res. 2012;177(1):21–6.

Roesch A, Gruber H, Hawelka B, Hamm H, Arnold N, Popal H, et al. Computer Assisted Learning In Medicine: A Long-Term Evaluation Of The “Practical Training Programme Dermatology 2000.” Med Inform Internet Med. 2003;28(3):147–59.

Sendra-Portero F, Torales-Chaparro OE, Ruiz-Gomez MJ, Martinez-Morillo M. A Pilot Study To Evaluate The Use Of Virtual Lectures For Undergraduate Radiology Teaching. Eur J Radiol. 2013;82(5):888–93.

Farrimond H, Dornan TL, Cockcroft A, Rhodes LE. Development And Evaluation Of An E-Learning Package For Teaching Skin Examination. Action Research Br J Dermatol. 2006;155(3):592–9.

Moriates C, Valencia V, Stamets S, Joo J, MacClements J, Wilkerson L, et al. Using Interactive Learning Modules to Teach Value-Based Health Care to Health Professions Trainees Across the United States. Acad Med. 2019;94(9):1332–6.

Naeger DM, Straus CM, Phelps A, Courtier J, Webb EM. Student-Created Independent Learning Modules: An Easy High-Value Addition To Radiology Clerkships. Acad Radiol. 2014;21(7):879–87.

Smith E, Boscak A. A Virtual Emergency: Learning Lessons From Remote Medical Student Education During The COVID-19 Pandemic. Emerg Radiol. 2021;28(3):445–52.

Taurines R, Radtke F, Romanos M, König S. Using Real Patients In E-Learning: Case-Based Online Training In Child And Adolescent Psychiatry. GMS J Med Educ. 2020;37(7):Doc96-Doc.

De Villiers M, Walsh S. How Podcasts Influence Medical Students’ Learning – A Descriptive Qualitative Study. Afr J Health Prof Educ. 2015;7(1):130–3.

Wagner-Menghin M, Szenes V, Scharitzer M, Pokieser P. Designing Virtual Patient Based Self-Study Quizzes Covering Learning Goals In Clinical Diagnostic Sciences For Undergraduate Medical Students - The Radiology Example. GMS J Med. 2020;37(7):Doc91.

Nelson TM. Preparing for Practice: Strengthening Third-Year Medical Students’ Awareness of Point-of-Care Resources. Med Ref Serv Q. 2018;37(3):312–8.

Kourdioukova EV, Verstraete KL, Valcke M. The Quality And Impact Of Computer Supported Collaborative Learning (CSCL) In Radiology Case-Based Learning. Eur J Radiol. 2011;78(3):353–62.

Clark RC, Mayer RE. E-Learning and the Science of Instruction: Proven Guidelines for Consumers and Designers of Multimedia Learning. Hoboken: Center for Creative Leadership; 2016.

Gruner D, Pottie K, Archibald D, Allison J, Sabourin V, Belcaid I, et al. Introducing global health into the undergraduate medical school curriculum using an e-learning program: a mixed method pilot study. BMC Med Educ. 2015;15(1):142.

UNESCO. National education responses to COVID-19: summary report of UNESCO's online survey. United Nations Educational, Scientific and Cultural Organization.

Mann KV. Theoretical Perspectives In Medical Education: Past Experience And Future Possibilities. Med Ed. 2011;45(1):60–8.

Moreno R, Mayer RE. Cognitive Principles Of Multimedia Learning: The Role Of Modality And Contiguity. J Educ Psychol. 1999;91(2):358–68.

Kim S. The Future of e-Learning in Medical Education: Current Trend and Future Opportunity. J Educ Eval Health Prof. 2006;3:3.

Du S, Liu Z, Liu S, Yin H, Xu G, Zhang H, et al. Web-based distance learning for nurse education: a systematic review. Int Nurs Rev. 2013;60(2):167–77.

Lahti M, Hätönen H, Välimäki M. Impact of e-learning on nurses’ and student nurses knowledge, skills, and satisfaction: A systematic review and meta-analysis. Int J Nurs Stud. 2014;51(1):136–49.

Kirkpatrick DL. The Four Levels of Evaluation. In: Brown SM, Seidner CJ, editors. Evaluating Corporate Training: Models and Issues. Dordrecht: Springer, Netherlands; 1998. p. 95–112.

Chapter   Google Scholar  

Fuks A, Boudreau JD, Cassell EJ. Teaching Clinical Thinking To First-Year Medical Students. Med Teach. 2009;31(2):105–11.

Chumley-Jones HS, Dobbie A, Alford CL. Web-Based Learning: Sound Educational Method Or Hype? A Review Of The Evaluation Literature. Acad Med. 2002;77(10 Suppl):S86-93.

Kahn CE Jr, Ehlers KC, Wood BP. Radiologists’ Preferences for Just-in-Time Learning. J Digit Imaging. 2006;19(3):202–6.

Fairén M, Moyés J, Insa E. VR4Health: Personalized teaching and learning anatomy using VR. J Med Syst. 2020;44(5):94.

Ruiz JG, Mintzer MJ, Leipzig RM. The impact of E-learning in medical education. Acad Med. 2006;81(3):207–12.

Klímová B. Mobile Learning in Medical Education. J Med Syst. 2018;42(10):194.

Baticulon RE, Sy JJ, Alberto NRI, Baron MBC, Mabulay REC, Rizada LGT, et al. Barriers to Online Learning in the Time of COVID-19: A National Survey of Medical Students in the Philippines. Medical Science Educator. 2021;31(2):615–26.

Mehrpour SR, Aghamirsalim M, Motamedi SM, Ardeshir Larijani F, Sorbi R. A supplemental video teaching tool enhances splinting skills. Clin Orthop Relat Res. 2013;471(2):649–54.

Larvin M. E-Learning In Surgical Education And Training. ANZ J Surg. 2009;79(3):133–7.

Lane J, Slavin S. Simulation in Medical Education: A Review. Simulation & Gaming - Simulat Gaming. 2001;32.

Bell DS, Fonarow GC, Hays RD, Mangione CM. elf-study from web-based and printed guideline materials. A randomized, controlled trial among resident physicians. Annals of internal medicine. 2000;132(12):938–46.

Alsoufi A, Alsuyihili A, Msherghi A, Elhadi A, Atiyah H, Ashini A, et al. Impact of the COVID-19 pandemic on medical education: Medical students’ knowledge, attitudes, and practices regarding electronic learning. PLoS ONE. 2020;15(11):e0242905.

Rogers DA, Regehr G, Yeh KA, Howdieshell TR. Computer-assisted Learning versus a Lecture and Feedback Seminar for Teaching a Basic Surgical Technical Skill. The American journal of surgery. 1998;175(6):508–10.

Guri RS. E-Teaching In Higher Education: An Essential Prerequisite For E-Learning. J New Approaches Educ Res. 2018;7(2):93–7.

Alexander B, Adams-Becker S, Cummins M, Hall-Giesinger C. Digital Literacy in Higher Education, Part II: An NMC Horizon Project Strategic Brief. . Austin, Texas; 2017.

Wineburg S, McGrew S, Breakstone J, Ortega T. Evaluating Information: The Cornerstone of Civic Online Reasoning. Stanford Digital Repository; 2016.

Yengin İ, Karahoca D, Karahoca A, Yücel A. Roles Of Teachers In E-Learning: How To Engage Students & How To Get Free E-Learning And The Future. Procedia Soc. 2010;2(2):5775–87.

Keller J, Suzuki K. Learner Motivation and E-learning Design: A Multinationally Validated Process. Learn Media Technol. 2004;29:229–39.

Howe DL, Heitner KL, Dozier A, Silas S. Health Professions Faculty Experiences Teaching Online During the COVID-19 Pandemic. ABNF J. 2021;32(1):6–11.

Barteit S, Neuhann F, Bärnighausen T, Bowa A, Lüders S, Malunga G, et al. Perspectives of Nonphysician Clinical Students and Medical Lecturers on Tablet-Based Health Care Practice Support for Medical Education in Zambia, Africa: Qualitative Study. JMIR Mhealth Uhealth. 2019;7(1):e12637.

Easton G. Primary care education research-time to raise our game? Educ Prim Care. 2014;25(6):304–7.

Cahapay M. Kirkpatrick Model: Its Limitations As Used in Higher Education Evaluation. IJATE. 2021;8:135–44.

Byrnes KG, Kiely PA, Dunne CP, McDermott KW, Coffey JC. Communication, collaboration and contagion: “Virtualisation” of anatomy during COVID-19. Clin Anat. 2021;34(1):82–9.

Dunne S, Cummins NM, Hannigan A, Shannon, Dunne C, Cullen W. A method for the design and development of medical or health care information websites to optimize search engine results page rankings on Google. Journal of Medical Internet Research. 2013;15(8):e2632.

Download references

Acknowledgements

Not applicable

No funding received.

Author information

Authors and affiliations.

School of Medicine, University of Limerick, Limerick, Ireland

T. Delungahawatta, S. S. Dunne, S. Hyde, L. Halpenny, D. McGrath, A. O’Regan & C. P. Dunne

Centre for Interventions in Infection, Inflammation & Immunity (4I), University of Limerick, Limerick, Ireland

D. McGrath & C. P. Dunne

You can also search for this author in PubMed   Google Scholar

Contributions

CPD, AOR, SH and DM conceived the study and its design. LH and AOR performed initial analysis. TD and SSD drafted and revised the manuscript. All authors co-wrote and approved the final manuscript.

A uthors’ information

Not applicable.

Corresponding author

Correspondence to C. P. Dunne .

Ethics declarations

Ethics approval and consent to participate, availability of data and material.

All data generated or analysed during this study are included in this published article [and its supplementary information files].

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1:.

PRISMA Checklist.

Additional file 2:  

Summary of Review Results.

Additional file 3:  

Intervention Characteristics.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Delungahawatta, T., Dunne, S.S., Hyde, S. et al. Advances in e-learning in undergraduate clinical medicine: a systematic review. BMC Med Educ 22 , 711 (2022). https://doi.org/10.1186/s12909-022-03773-1

Download citation

Received : 26 February 2022

Accepted : 26 September 2022

Published : 07 October 2022

DOI : https://doi.org/10.1186/s12909-022-03773-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Online learning
• Distance learning
• Medical education
• Medical students
• Clinical medicine
• Systematic review

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Wiley-Blackwell Online Open

An overview of methodological approaches in systematic reviews

Prabhakar veginadu.

1 Department of Rural Clinical Sciences, La Trobe Rural Health School, La Trobe University, Bendigo Victoria, Australia

Hanny Calache

2 Lincoln International Institute for Rural Health, University of Lincoln, Brayford Pool, Lincoln UK

Akshaya Pandian

3 Department of Orthodontics, Saveetha Dental College, Chennai Tamil Nadu, India

Mohd Masood

Associated data.

APPENDIX B: List of excluded studies with detailed reasons for exclusion

APPENDIX C: Quality assessment of included reviews using AMSTAR 2

The aim of this overview is to identify and collate evidence from existing published systematic review (SR) articles evaluating various methodological approaches used at each stage of an SR.

The search was conducted in five electronic databases from inception to November 2020 and updated in February 2022: MEDLINE, Embase, Web of Science Core Collection, Cochrane Database of Systematic Reviews, and APA PsycINFO. Title and abstract screening were performed in two stages by one reviewer, supported by a second reviewer. Full‐text screening, data extraction, and quality appraisal were performed by two reviewers independently. The quality of the included SRs was assessed using the AMSTAR 2 checklist.

The search retrieved 41,556 unique citations, of which 9 SRs were deemed eligible for inclusion in final synthesis. Included SRs evaluated 24 unique methodological approaches used for defining the review scope and eligibility, literature search, screening, data extraction, and quality appraisal in the SR process. Limited evidence supports the following (a) searching multiple resources (electronic databases, handsearching, and reference lists) to identify relevant literature; (b) excluding non‐English, gray, and unpublished literature, and (c) use of text‐mining approaches during title and abstract screening.

The overview identified limited SR‐level evidence on various methodological approaches currently employed during five of the seven fundamental steps in the SR process, as well as some methodological modifications currently used in expedited SRs. Overall, findings of this overview highlight the dearth of published SRs focused on SR methodologies and this warrants future work in this area.

1. INTRODUCTION

Evidence synthesis is a prerequisite for knowledge translation. 1 A well conducted systematic review (SR), often in conjunction with meta‐analyses (MA) when appropriate, is considered the “gold standard” of methods for synthesizing evidence related to a topic of interest. 2 The central strength of an SR is the transparency of the methods used to systematically search, appraise, and synthesize the available evidence. 3 Several guidelines, developed by various organizations, are available for the conduct of an SR; 4 , 5 , 6 , 7 among these, Cochrane is considered a pioneer in developing rigorous and highly structured methodology for the conduct of SRs. 8 The guidelines developed by these organizations outline seven fundamental steps required in SR process: defining the scope of the review and eligibility criteria, literature searching and retrieval, selecting eligible studies, extracting relevant data, assessing risk of bias (RoB) in included studies, synthesizing results, and assessing certainty of evidence (CoE) and presenting findings. 4 , 5 , 6 , 7

The methodological rigor involved in an SR can require a significant amount of time and resource, which may not always be available. 9 As a result, there has been a proliferation of modifications made to the traditional SR process, such as refining, shortening, bypassing, or omitting one or more steps, 10 , 11 for example, limits on the number and type of databases searched, limits on publication date, language, and types of studies included, and limiting to one reviewer for screening and selection of studies, as opposed to two or more reviewers. 10 , 11 These methodological modifications are made to accommodate the needs of and resource constraints of the reviewers and stakeholders (e.g., organizations, policymakers, health care professionals, and other knowledge users). While such modifications are considered time and resource efficient, they may introduce bias in the review process reducing their usefulness. 5

Substantial research has been conducted examining various approaches used in the standardized SR methodology and their impact on the validity of SR results. There are a number of published reviews examining the approaches or modifications corresponding to single 12 , 13 or multiple steps 14 involved in an SR. However, there is yet to be a comprehensive summary of the SR‐level evidence for all the seven fundamental steps in an SR. Such a holistic evidence synthesis will provide an empirical basis to confirm the validity of current accepted practices in the conduct of SRs. Furthermore, sometimes there is a balance that needs to be achieved between the resource availability and the need to synthesize the evidence in the best way possible, given the constraints. This evidence base will also inform the choice of modifications to be made to the SR methods, as well as the potential impact of these modifications on the SR results. An overview is considered the choice of approach for summarizing existing evidence on a broad topic, directing the reader to evidence, or highlighting the gaps in evidence, where the evidence is derived exclusively from SRs. 15 Therefore, for this review, an overview approach was used to (a) identify and collate evidence from existing published SR articles evaluating various methodological approaches employed in each of the seven fundamental steps of an SR and (b) highlight both the gaps in the current research and the potential areas for future research on the methods employed in SRs.

An a priori protocol was developed for this overview but was not registered with the International Prospective Register of Systematic Reviews (PROSPERO), as the review was primarily methodological in nature and did not meet PROSPERO eligibility criteria for registration. The protocol is available from the corresponding author upon reasonable request. This overview was conducted based on the guidelines for the conduct of overviews as outlined in The Cochrane Handbook. 15 Reporting followed the Preferred Reporting Items for Systematic reviews and Meta‐analyses (PRISMA) statement. 3

2.1. Eligibility criteria

Only published SRs, with or without associated MA, were included in this overview. We adopted the defining characteristics of SRs from The Cochrane Handbook. 5 According to The Cochrane Handbook, a review was considered systematic if it satisfied the following criteria: (a) clearly states the objectives and eligibility criteria for study inclusion; (b) provides reproducible methodology; (c) includes a systematic search to identify all eligible studies; (d) reports assessment of validity of findings of included studies (e.g., RoB assessment of the included studies); (e) systematically presents all the characteristics or findings of the included studies. 5 Reviews that did not meet all of the above criteria were not considered a SR for this study and were excluded. MA‐only articles were included if it was mentioned that the MA was based on an SR.

SRs and/or MA of primary studies evaluating methodological approaches used in defining review scope and study eligibility, literature search, study selection, data extraction, RoB assessment, data synthesis, and CoE assessment and reporting were included. The methodological approaches examined in these SRs and/or MA can also be related to the substeps or elements of these steps; for example, applying limits on date or type of publication are the elements of literature search. Included SRs examined or compared various aspects of a method or methods, and the associated factors, including but not limited to: precision or effectiveness; accuracy or reliability; impact on the SR and/or MA results; reproducibility of an SR steps or bias occurred; time and/or resource efficiency. SRs assessing the methodological quality of SRs (e.g., adherence to reporting guidelines), evaluating techniques for building search strategies or the use of specific database filters (e.g., use of Boolean operators or search filters for randomized controlled trials), examining various tools used for RoB or CoE assessment (e.g., ROBINS vs. Cochrane RoB tool), or evaluating statistical techniques used in meta‐analyses were excluded. 14

2.2. Search

The search for published SRs was performed on the following scientific databases initially from inception to third week of November 2020 and updated in the last week of February 2022: MEDLINE (via Ovid), Embase (via Ovid), Web of Science Core Collection, Cochrane Database of Systematic Reviews, and American Psychological Association (APA) PsycINFO. Search was restricted to English language publications. Following the objectives of this study, study design filters within databases were used to restrict the search to SRs and MA, where available. The reference lists of included SRs were also searched for potentially relevant publications.

The search terms included keywords, truncations, and subject headings for the key concepts in the review question: SRs and/or MA, methods, and evaluation. Some of the terms were adopted from the search strategy used in a previous review by Robson et al., which reviewed primary studies on methodological approaches used in study selection, data extraction, and quality appraisal steps of SR process. 14 Individual search strategies were developed for respective databases by combining the search terms using appropriate proximity and Boolean operators, along with the related subject headings in order to identify SRs and/or MA. 16 , 17 A senior librarian was consulted in the design of the search terms and strategy. Appendix A presents the detailed search strategies for all five databases.

2.3. Study selection and data extraction

Title and abstract screening of references were performed in three steps. First, one reviewer (PV) screened all the titles and excluded obviously irrelevant citations, for example, articles on topics not related to SRs, non‐SR publications (such as randomized controlled trials, observational studies, scoping reviews, etc.). Next, from the remaining citations, a random sample of 200 titles and abstracts were screened against the predefined eligibility criteria by two reviewers (PV and MM), independently, in duplicate. Discrepancies were discussed and resolved by consensus. This step ensured that the responses of the two reviewers were calibrated for consistency in the application of the eligibility criteria in the screening process. Finally, all the remaining titles and abstracts were reviewed by a single “calibrated” reviewer (PV) to identify potential full‐text records. Full‐text screening was performed by at least two authors independently (PV screened all the records, and duplicate assessment was conducted by MM, HC, or MG), with discrepancies resolved via discussions or by consulting a third reviewer.

Data related to review characteristics, results, key findings, and conclusions were extracted by at least two reviewers independently (PV performed data extraction for all the reviews and duplicate extraction was performed by AP, HC, or MG).

2.4. Quality assessment of included reviews

The quality assessment of the included SRs was performed using the AMSTAR 2 (A MeaSurement Tool to Assess systematic Reviews). The tool consists of a 16‐item checklist addressing critical and noncritical domains. 18 For the purpose of this study, the domain related to MA was reclassified from critical to noncritical, as SRs with and without MA were included. The other six critical domains were used according to the tool guidelines. 18 Two reviewers (PV and AP) independently responded to each of the 16 items in the checklist with either “yes,” “partial yes,” or “no.” Based on the interpretations of the critical and noncritical domains, the overall quality of the review was rated as high, moderate, low, or critically low. 18 Disagreements were resolved through discussion or by consulting a third reviewer.

2.5. Data synthesis

To provide an understandable summary of existing evidence syntheses, characteristics of the methods evaluated in the included SRs were examined and key findings were categorized and presented based on the corresponding step in the SR process. The categories of key elements within each step were discussed and agreed by the authors. Results of the included reviews were tabulated and summarized descriptively, along with a discussion on any overlap in the primary studies. 15 No quantitative analyses of the data were performed.

From 41,556 unique citations identified through literature search, 50 full‐text records were reviewed, and nine systematic reviews 14 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 were deemed eligible for inclusion. The flow of studies through the screening process is presented in Figure  1 . A list of excluded studies with reasons can be found in Appendix B .

Study selection flowchart

3.1. Characteristics of included reviews

Table  1 summarizes the characteristics of included SRs. The majority of the included reviews (six of nine) were published after 2010. 14 , 22 , 23 , 24 , 25 , 26 Four of the nine included SRs were Cochrane reviews. 20 , 21 , 22 , 23 The number of databases searched in the reviews ranged from 2 to 14, 2 reviews searched gray literature sources, 24 , 25 and 7 reviews included a supplementary search strategy to identify relevant literature. 14 , 19 , 20 , 21 , 22 , 23 , 26 Three of the included SRs (all Cochrane reviews) included an integrated MA. 20 , 21 , 23

Characteristics of included studies

SR = systematic review; MA = meta‐analysis; RCT = randomized controlled trial; CCT = controlled clinical trial; N/R = not reported.

The included SRs evaluated 24 unique methodological approaches (26 in total) used across five steps in the SR process; 8 SRs evaluated 6 approaches, 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 while 1 review evaluated 18 approaches. 14 Exclusion of gray or unpublished literature 21 , 26 and blinding of reviewers for RoB assessment 14 , 23 were evaluated in two reviews each. Included SRs evaluated methods used in five different steps in the SR process, including methods used in defining the scope of review ( n  = 3), literature search ( n  = 3), study selection ( n  = 2), data extraction ( n  = 1), and RoB assessment ( n  = 2) (Table  2 ).

Summary of findings from review evaluating systematic review methods

There was some overlap in the primary studies evaluated in the included SRs on the same topics: Schmucker et al. 26 and Hopewell et al. 21 ( n  = 4), Hopewell et al. 20 and Crumley et al. 19 ( n  = 30), and Robson et al. 14 and Morissette et al. 23 ( n  = 4). There were no conflicting results between any of the identified SRs on the same topic.

3.2. Methodological quality of included reviews

Overall, the quality of the included reviews was assessed as moderate at best (Table  2 ). The most common critical weakness in the reviews was failure to provide justification for excluding individual studies (four reviews). Detailed quality assessment is provided in Appendix C .

3.3. Evidence on systematic review methods

3.3.1. methods for defining review scope and eligibility.

Two SRs investigated the effect of excluding data obtained from gray or unpublished sources on the pooled effect estimates of MA. 21 , 26 Hopewell et al. 21 reviewed five studies that compared the impact of gray literature on the results of a cohort of MA of RCTs in health care interventions. Gray literature was defined as information published in “print or electronic sources not controlled by commercial or academic publishers.” Findings showed an overall greater treatment effect for published trials than trials reported in gray literature. In a more recent review, Schmucker et al. 26 addressed similar objectives, by investigating gray and unpublished data in medicine. In addition to gray literature, defined similar to the previous review by Hopewell et al., the authors also evaluated unpublished data—defined as “supplemental unpublished data related to published trials, data obtained from the Food and Drug Administration  or other regulatory websites or postmarketing analyses hidden from the public.” The review found that in majority of the MA, excluding gray literature had little or no effect on the pooled effect estimates. The evidence was limited to conclude if the data from gray and unpublished literature had an impact on the conclusions of MA. 26

Morrison et al. 24 examined five studies measuring the effect of excluding non‐English language RCTs on the summary treatment effects of SR‐based MA in various fields of conventional medicine. Although none of the included studies reported major difference in the treatment effect estimates between English only and non‐English inclusive MA, the review found inconsistent evidence regarding the methodological and reporting quality of English and non‐English trials. 24 As such, there might be a risk of introducing “language bias” when excluding non‐English language RCTs. The authors also noted that the numbers of non‐English trials vary across medical specialties, as does the impact of these trials on MA results. Based on these findings, Morrison et al. 24 conclude that literature searches must include non‐English studies when resources and time are available to minimize the risk of introducing “language bias.”

3.3.2. Methods for searching studies

Crumley et al. 19 analyzed recall (also referred to as “sensitivity” by some researchers; defined as “percentage of relevant studies identified by the search”) and precision (defined as “percentage of studies identified by the search that were relevant”) when searching a single resource to identify randomized controlled trials and controlled clinical trials, as opposed to searching multiple resources. The studies included in their review frequently compared a MEDLINE only search with the search involving a combination of other resources. The review found low median recall estimates (median values between 24% and 92%) and very low median precisions (median values between 0% and 49%) for most of the electronic databases when searched singularly. 19 A between‐database comparison, based on the type of search strategy used, showed better recall and precision for complex and Cochrane Highly Sensitive search strategies (CHSSS). In conclusion, the authors emphasize that literature searches for trials in SRs must include multiple sources. 19

In an SR comparing handsearching and electronic database searching, Hopewell et al. 20 found that handsearching retrieved more relevant RCTs (retrieval rate of 92%−100%) than searching in a single electronic database (retrieval rates of 67% for PsycINFO/PsycLIT, 55% for MEDLINE, and 49% for Embase). The retrieval rates varied depending on the quality of handsearching, type of electronic search strategy used (e.g., simple, complex or CHSSS), and type of trial reports searched (e.g., full reports, conference abstracts, etc.). The authors concluded that handsearching was particularly important in identifying full trials published in nonindexed journals and in languages other than English, as well as those published as abstracts and letters. 20

The effectiveness of checking reference lists to retrieve additional relevant studies for an SR was investigated by Horsley et al. 22 The review reported that checking reference lists yielded 2.5%–40% more studies depending on the quality and comprehensiveness of the electronic search used. The authors conclude that there is some evidence, although from poor quality studies, to support use of checking reference lists to supplement database searching. 22

3.3.3. Methods for selecting studies

Three approaches relevant to reviewer characteristics, including number, experience, and blinding of reviewers involved in the screening process were highlighted in an SR by Robson et al. 14 Based on the retrieved evidence, the authors recommended that two independent, experienced, and unblinded reviewers be involved in study selection. 14 A modified approach has also been suggested by the review authors, where one reviewer screens and the other reviewer verifies the list of excluded studies, when the resources are limited. It should be noted however this suggestion is likely based on the authors’ opinion, as there was no evidence related to this from the studies included in the review.

Robson et al. 14 also reported two methods describing the use of technology for screening studies: use of Google Translate for translating languages (for example, German language articles to English) to facilitate screening was considered a viable method, while using two computer monitors for screening did not increase the screening efficiency in SR. Title‐first screening was found to be more efficient than simultaneous screening of titles and abstracts, although the gain in time with the former method was lesser than the latter. Therefore, considering that the search results are routinely exported as titles and abstracts, Robson et al. 14 recommend screening titles and abstracts simultaneously. However, the authors note that these conclusions were based on very limited number (in most instances one study per method) of low‐quality studies. 14

3.3.4. Methods for data extraction

Robson et al. 14 examined three approaches for data extraction relevant to reviewer characteristics, including number, experience, and blinding of reviewers (similar to the study selection step). Although based on limited evidence from a small number of studies, the authors recommended use of two experienced and unblinded reviewers for data extraction. The experience of the reviewers was suggested to be especially important when extracting continuous outcomes (or quantitative) data. However, when the resources are limited, data extraction by one reviewer and a verification of the outcomes data by a second reviewer was recommended.

As for the methods involving use of technology, Robson et al. 14 identified limited evidence on the use of two monitors to improve the data extraction efficiency and computer‐assisted programs for graphical data extraction. However, use of Google Translate for data extraction in non‐English articles was not considered to be viable. 14 In the same review, Robson et al. 14 identified evidence supporting contacting authors for obtaining additional relevant data.

3.3.5. Methods for RoB assessment

Two SRs examined the impact of blinding of reviewers for RoB assessments. 14 , 23 Morissette et al. 23 investigated the mean differences between the blinded and unblinded RoB assessment scores and found inconsistent differences among the included studies providing no definitive conclusions. Similar conclusions were drawn in a more recent review by Robson et al., 14 which included four studies on reviewer blinding for RoB assessment that completely overlapped with Morissette et al. 23

Use of experienced reviewers and provision of additional guidance for RoB assessment were examined by Robson et al. 14 The review concluded that providing intensive training and guidance on assessing studies reporting insufficient data to the reviewers improves RoB assessments. 14 Obtaining additional data related to quality assessment by contacting study authors was also found to help the RoB assessments, although based on limited evidence. When assessing the qualitative or mixed method reviews, Robson et al. 14 recommends the use of a structured RoB tool as opposed to an unstructured tool. No SRs were identified on data synthesis and CoE assessment and reporting steps.

4. DISCUSSION

4.1. summary of findings.

Nine SRs examining 24 unique methods used across five steps in the SR process were identified in this overview. The collective evidence supports some current traditional and modified SR practices, while challenging other approaches. However, the quality of the included reviews was assessed to be moderate at best and in the majority of the included SRs, evidence related to the evaluated methods was obtained from very limited numbers of primary studies. As such, the interpretations from these SRs should be made cautiously.

The evidence gathered from the included SRs corroborate a few current SR approaches. 5 For example, it is important to search multiple resources for identifying relevant trials (RCTs and/or CCTs). The resources must include a combination of electronic database searching, handsearching, and reference lists of retrieved articles. 5 However, no SRs have been identified that evaluated the impact of the number of electronic databases searched. A recent study by Halladay et al. 27 found that articles on therapeutic intervention, retrieved by searching databases other than PubMed (including Embase), contributed only a small amount of information to the MA and also had a minimal impact on the MA results. The authors concluded that when the resources are limited and when large number of studies are expected to be retrieved for the SR or MA, PubMed‐only search can yield reliable results. 27

Findings from the included SRs also reiterate some methodological modifications currently employed to “expedite” the SR process. 10 , 11 For example, excluding non‐English language trials and gray/unpublished trials from MA have been shown to have minimal or no impact on the results of MA. 24 , 26 However, the efficiency of these SR methods, in terms of time and the resources used, have not been evaluated in the included SRs. 24 , 26 Of the SRs included, only two have focused on the aspect of efficiency 14 , 25 ; O'Mara‐Eves et al. 25 report some evidence to support the use of text‐mining approaches for title and abstract screening in order to increase the rate of screening. Moreover, only one included SR 14 considered primary studies that evaluated reliability (inter‐ or intra‐reviewer consistency) and accuracy (validity when compared against a “gold standard” method) of the SR methods. This can be attributed to the limited number of primary studies that evaluated these outcomes when evaluating the SR methods. 14 Lack of outcome measures related to reliability, accuracy, and efficiency precludes making definitive recommendations on the use of these methods/modifications. Future research studies must focus on these outcomes.

Some evaluated methods may be relevant to multiple steps; for example, exclusions based on publication status (gray/unpublished literature) and language of publication (non‐English language studies) can be outlined in the a priori eligibility criteria or can be incorporated as search limits in the search strategy. SRs included in this overview focused on the effect of study exclusions on pooled treatment effect estimates or MA conclusions. Excluding studies from the search results, after conducting a comprehensive search, based on different eligibility criteria may yield different results when compared to the results obtained when limiting the search itself. 28 Further studies are required to examine this aspect.

Although we acknowledge the lack of standardized quality assessment tools for methodological study designs, we adhered to the Cochrane criteria for identifying SRs in this overview. This was done to ensure consistency in the quality of the included evidence. As a result, we excluded three reviews that did not provide any form of discussion on the quality of the included studies. The methods investigated in these reviews concern supplementary search, 29 data extraction, 12 and screening. 13 However, methods reported in two of these three reviews, by Mathes et al. 12 and Waffenschmidt et al., 13 have also been examined in the SR by Robson et al., 14 which was included in this overview; in most instances (with the exception of one study included in Mathes et al. 12 and Waffenschmidt et al. 13 each), the studies examined in these excluded reviews overlapped with those in the SR by Robson et al. 14

One of the key gaps in the knowledge observed in this overview was the dearth of SRs on the methods used in the data synthesis component of SR. Narrative and quantitative syntheses are the two most commonly used approaches for synthesizing data in evidence synthesis. 5 There are some published studies on the proposed indications and implications of these two approaches. 30 , 31 These studies found that both data synthesis methods produced comparable results and have their own advantages, suggesting that the choice of the method must be based on the purpose of the review. 31 With increasing number of “expedited” SR approaches (so called “rapid reviews”) avoiding MA, 10 , 11 further research studies are warranted in this area to determine the impact of the type of data synthesis on the results of the SR.

4.2. Implications for future research

The findings of this overview highlight several areas of paucity in primary research and evidence synthesis on SR methods. First, no SRs were identified on methods used in two important components of the SR process, including data synthesis and CoE and reporting. As for the included SRs, a limited number of evaluation studies have been identified for several methods. This indicates that further research is required to corroborate many of the methods recommended in current SR guidelines. 4 , 5 , 6 , 7 Second, some SRs evaluated the impact of methods on the results of quantitative synthesis and MA conclusions. Future research studies must also focus on the interpretations of SR results. 28 , 32 Finally, most of the included SRs were conducted on specific topics related to the field of health care, limiting the generalizability of the findings to other areas. It is important that future research studies evaluating evidence syntheses broaden the objectives and include studies on different topics within the field of health care.

4.3. Strengths and limitations

To our knowledge, this is the first overview summarizing current evidence from SRs and MA on different methodological approaches used in several fundamental steps in SR conduct. The overview methodology followed well established guidelines and strict criteria defined for the inclusion of SRs.

There are several limitations related to the nature of the included reviews. Evidence for most of the methods investigated in the included reviews was derived from a limited number of primary studies. Also, the majority of the included SRs may be considered outdated as they were published (or last updated) more than 5 years ago 33 ; only three of the nine SRs have been published in the last 5 years. 14 , 25 , 26 Therefore, important and recent evidence related to these topics may not have been included. Substantial numbers of included SRs were conducted in the field of health, which may limit the generalizability of the findings. Some method evaluations in the included SRs focused on quantitative analyses components and MA conclusions only. As such, the applicability of these findings to SR more broadly is still unclear. 28 Considering the methodological nature of our overview, limiting the inclusion of SRs according to the Cochrane criteria might have resulted in missing some relevant evidence from those reviews without a quality assessment component. 12 , 13 , 29 Although the included SRs performed some form of quality appraisal of the included studies, most of them did not use a standardized RoB tool, which may impact the confidence in their conclusions. Due to the type of outcome measures used for the method evaluations in the primary studies and the included SRs, some of the identified methods have not been validated against a reference standard.

Some limitations in the overview process must be noted. While our literature search was exhaustive covering five bibliographic databases and supplementary search of reference lists, no gray sources or other evidence resources were searched. Also, the search was primarily conducted in health databases, which might have resulted in missing SRs published in other fields. Moreover, only English language SRs were included for feasibility. As the literature search retrieved large number of citations (i.e., 41,556), the title and abstract screening was performed by a single reviewer, calibrated for consistency in the screening process by another reviewer, owing to time and resource limitations. These might have potentially resulted in some errors when retrieving and selecting relevant SRs. The SR methods were grouped based on key elements of each recommended SR step, as agreed by the authors. This categorization pertains to the identified set of methods and should be considered subjective.

5. CONCLUSIONS

This overview identified limited SR‐level evidence on various methodological approaches currently employed during five of the seven fundamental steps in the SR process. Limited evidence was also identified on some methodological modifications currently used to expedite the SR process. Overall, findings highlight the dearth of SRs on SR methodologies, warranting further work to confirm several current recommendations on conventional and expedited SR processes.

CONFLICT OF INTEREST

The authors declare no conflicts of interest.

Supporting information

APPENDIX A: Detailed search strategies

ACKNOWLEDGMENTS

The first author is supported by a La Trobe University Full Fee Research Scholarship and a Graduate Research Scholarship.

Open Access Funding provided by La Trobe University.

Veginadu P, Calache H, Gussy M, Pandian A, Masood M. An overview of methodological approaches in systematic reviews . J Evid Based Med . 2022; 15 :39–54. 10.1111/jebm.12468 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]