scholarly articles literature review

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

• An introductory paragraph that explains what your working topic and thesis is
• A forecast of key topics or texts that will appear in the review
• Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
• Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically Evaluate: Mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

• Summarize the key findings you have taken from the literature and emphasize their significance
• Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

• Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
• Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
• Qualitative versus quantitative research
• Empirical versus theoretical scholarship
• Divide the research by sociological, historical, or cultural sources
• Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

• It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
• Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
• Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
• Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• CAREER FEATURE
• 04 December 2020
• Correction 09 December 2020

How to write a superb literature review

Andy Tay is a freelance writer based in Singapore.

You can also search for this author in PubMed   Google Scholar

Literature reviews are important resources for scientists. They provide historical context for a field while offering opinions on its future trajectory. Creating them can provide inspiration for one’s own research, as well as some practice in writing. But few scientists are trained in how to write a review — or in what constitutes an excellent one. Even picking the appropriate software to use can be an involved decision (see ‘Tools and techniques’). So Nature asked editors and working scientists with well-cited reviews for their tips.

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 51 print issues and online access

185,98 € per year

only 3,65 € per issue

Rent or buy this article

Prices vary by article type

Prices may be subject to local taxes which are calculated during checkout

doi: https://doi.org/10.1038/d41586-020-03422-x

Interviews have been edited for length and clarity.

Updates & Corrections

Correction 09 December 2020 : An earlier version of the tables in this article included some incorrect details about the programs Zotero, Endnote and Manubot. These have now been corrected.

Hsing, I.-M., Xu, Y. & Zhao, W. Electroanalysis 19 , 755–768 (2007).

Article   Google Scholar  

Ledesma, H. A. et al. Nature Nanotechnol. 14 , 645–657 (2019).

Article   PubMed   Google Scholar  

Brahlek, M., Koirala, N., Bansal, N. & Oh, S. Solid State Commun. 215–216 , 54–62 (2015).

Choi, Y. & Lee, S. Y. Nature Rev. Chem . https://doi.org/10.1038/s41570-020-00221-w (2020).

Download references

Related Articles

• Research management

I had my white colleagues walk in a Black student’s shoes for a day

Career Q&A 28 MAY 24

Changemakers — Nature’s new series celebrates champions of inclusion in science

Editorial 28 MAY 24

Researcher parents are paying a high price for conference travel — here’s how to fix it

Career Column 27 MAY 24

How researchers in remote regions handle the isolation

Career Feature 24 MAY 24

Guidelines for academics aim to lessen ethical pitfalls in generative-AI use

Nature Index 22 MAY 24

Who will make AlphaFold3 open source? Scientists race to crack AI model

News 23 MAY 24

Egypt is building a $1-billion mega-museum. Will it bring Egyptology home?

News Feature 22 MAY 24

Pay researchers to spot errors in published papers

World View 21 MAY 24

Zhejiang Provincial Hospital of Chinese Medicine on Open Recruitment of Medical Talents and Postdocs

Director of Clinical Department, Professor, Researcher, Post-doctor

Hangzhou, Zhejiang, China

The First Affiliated Hospital of Zhejiang Chinese Medical University

Postdoc in Biomechanical Engineering (m/f/d)

The Muskuloskelettales Universitätszentrum München (MUM) on the Campus Großhadern invites applications for the department Orthopädie

Munich (Stadt), Bayern (DE)

PWG-LMU Klinikum

Assistant, Associate or Full Professor

The McLaughlin Research Institute and Touro University – Montana campus invite applications for open rank faculty positions.

McLaughlin Research Institute

Postdoctoral Associate- Neuroscience

Houston, Texas (US)

Baylor College of Medicine (BCM)

Call for applications- junior and senior scientists

The BORDEAUX INSTITUTE OF ONCOLOGY (BRIC U1312, https://www.bricbordeaux.com/) is seeking to recruit new junior and senior researchers

Bordeaux (Ville), Gironde (FR)

INSERM - U1312 BRIC

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Dissertation
• What is a Literature Review? | Guide, Template, & Examples

What is a Literature Review? | Guide, Template, & Examples

Published on 22 February 2022 by Shona McCombes . Revised on 7 June 2022.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research.

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarise sources – it analyses, synthesises, and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Be assured that you'll submit flawless writing. Upload your document to correct all your mistakes.

Table of contents

Why write a literature review, examples of literature reviews, step 1: search for relevant literature, step 2: evaluate and select sources, step 3: identify themes, debates and gaps, step 4: outline your literature review’s structure, step 5: write your literature review, frequently asked questions about literature reviews, introduction.

• Quick Run-through
• Step 1 & 2

When you write a dissertation or thesis, you will have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and scholarly context
• Develop a theoretical framework and methodology for your research
• Position yourself in relation to other researchers and theorists
• Show how your dissertation addresses a gap or contributes to a debate

You might also have to write a literature review as a stand-alone assignment. In this case, the purpose is to evaluate the current state of research and demonstrate your knowledge of scholarly debates around a topic.

The content will look slightly different in each case, but the process of conducting a literature review follows the same steps. We’ve written a step-by-step guide that you can follow below.

The only proofreading tool specialized in correcting academic writing

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Correct my document today

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research objectives and questions .

If you are writing a literature review as a stand-alone assignment, you will have to choose a focus and develop a central question to direct your search. Unlike a dissertation research question, this question has to be answerable without collecting original data. You should be able to answer it based only on a review of existing publications.

Make a list of keywords

Start by creating a list of keywords related to your research topic. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list if you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can use boolean operators to help narrow down your search:

Read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

To identify the most important publications on your topic, take note of recurring citations. If the same authors, books or articles keep appearing in your reading, make sure to seek them out.

You probably won’t be able to read absolutely everything that has been written on the topic – you’ll have to evaluate which sources are most relevant to your questions.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models and methods? Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• How does the publication contribute to your understanding of the topic? What are its key insights and arguments?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible, and make sure you read any landmark studies and major theories in your field of research.

You can find out how many times an article has been cited on Google Scholar – a high citation count means the article has been influential in the field, and should certainly be included in your literature review.

The scope of your review will depend on your topic and discipline: in the sciences you usually only review recent literature, but in the humanities you might take a long historical perspective (for example, to trace how a concept has changed in meaning over time).

Remember that you can use our template to summarise and evaluate sources you’re thinking about using!

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It’s important to keep track of your sources with references to avoid plagiarism . It can be helpful to make an annotated bibliography, where you compile full reference information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

You can use our free APA Reference Generator for quick, correct, consistent citations.

Prevent plagiarism, run a free check.

To begin organising your literature review’s argument and structure, you need to understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly-visual platforms like Instagram and Snapchat – this is a gap that you could address in your own research.

There are various approaches to organising the body of a literature review. You should have a rough idea of your strategy before you start writing.

Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarising sources in order.

Try to analyse patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organise your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text, your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

If you are writing the literature review as part of your dissertation or thesis, reiterate your central problem or research question and give a brief summary of the scholarly context. You can emphasise the timeliness of the topic (“many recent studies have focused on the problem of x”) or highlight a gap in the literature (“while there has been much research on x, few researchers have taken y into consideration”).

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, make sure to follow these tips:

• Summarise and synthesise: give an overview of the main points of each source and combine them into a coherent whole.
• Analyse and interpret: don’t just paraphrase other researchers – add your own interpretations, discussing the significance of findings in relation to the literature as a whole.
• Critically evaluate: mention the strengths and weaknesses of your sources.
• Write in well-structured paragraphs: use transitions and topic sentences to draw connections, comparisons and contrasts.

In the conclusion, you should summarise the key findings you have taken from the literature and emphasise their significance.

If the literature review is part of your dissertation or thesis, reiterate how your research addresses gaps and contributes new knowledge, or discuss how you have drawn on existing theories and methods to build a framework for your research. This can lead directly into your methodology section.

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a dissertation , thesis, research paper , or proposal .

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarise yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your  dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

McCombes, S. (2022, June 07). What is a Literature Review? | Guide, Template, & Examples. Scribbr. Retrieved 27 May 2024, from https://www.scribbr.co.uk/thesis-dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, how to write a dissertation proposal | a step-by-step guide, what is a theoretical framework | a step-by-step guide, what is a research methodology | steps & tips.

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 5. The Literature Review
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

• Give a new interpretation of old material or combine new with old interpretations,
• Trace the intellectual progression of the field, including major debates,
• Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
• Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

• Place each work in the context of its contribution to understanding the research problem being studied.
• Describe the relationship of each work to the others under consideration.
• Identify new ways to interpret prior research.
• Reveal any gaps that exist in the literature.
• Resolve conflicts amongst seemingly contradictory previous studies.
• Identify areas of prior scholarship to prevent duplication of effort.
• Point the way in fulfilling a need for additional research.
• Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE: Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

• An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
• Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
• An explanation of how each work is similar to and how it varies from the others,
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

• Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
• Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
• Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
• Persuasiveness -- which of the author's theses are most convincing or least convincing?
• Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

• Current Situation : Information necessary to understand the current topic or focus of the literature review.
• Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
• History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
• Standards : Description of the way in which you present your information.
• Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

• Sources in your literature review do not clearly relate to the research problem;
• You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
• Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
• Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
• Does not describe the search procedures that were used in identifying the literature to review;
• Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
• Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

• How are they organizing their ideas?
• What methods have they used to study the problem?
• What theories have been used to explain, predict, or understand their research problem?
• What sources have they cited to support their conclusions?
• How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

• Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
• Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
• Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

• << Previous: Theoretical Framework
• Next: Citation Tracking >>
• Last Updated: May 25, 2024 4:09 PM
• URL: https://libguides.usc.edu/writingguide

Libraries | Research Guides

Literature reviews, what is a literature review, learning more about how to do a literature review.

• Planning the Review
• The Research Question
• Choosing Where to Search
• Organizing the Review
• Writing the Review

A literature review is a review and synthesis of existing research on a topic or research question. A literature review is meant to analyze the scholarly literature, make connections across writings and identify strengths, weaknesses, trends, and missing conversations. A literature review should address different aspects of a topic as it relates to your research question. A literature review goes beyond a description or summary of the literature you have read. 

• Sage Research Methods Core Collection This link opens in a new window SAGE Research Methods supports research at all levels by providing material to guide users through every step of the research process. SAGE Research Methods is the ultimate methods library with more than 1000 books, reference works, journal articles, and instructional videos by world-leading academics from across the social sciences, including the largest collection of qualitative methods books available online from any scholarly publisher. – Publisher

• Next: Planning the Review >>
• Last Updated: May 2, 2024 10:39 AM
• URL: https://libguides.northwestern.edu/literaturereviews

Get science-backed answers as you write with Paperpal's Research feature

What is a Literature Review? How to Write It (with Examples)

A literature review is a critical analysis and synthesis of existing research on a particular topic. It provides an overview of the current state of knowledge, identifies gaps, and highlights key findings in the literature. 1 The purpose of a literature review is to situate your own research within the context of existing scholarship, demonstrating your understanding of the topic and showing how your work contributes to the ongoing conversation in the field. Learning how to write a literature review is a critical tool for successful research. Your ability to summarize and synthesize prior research pertaining to a certain topic demonstrates your grasp on the topic of study, and assists in the learning process. 

Table of Contents

• What is the purpose of literature review? 
• a. Habitat Loss and Species Extinction: 
• b. Range Shifts and Phenological Changes: 
• c. Ocean Acidification and Coral Reefs: 
• d. Adaptive Strategies and Conservation Efforts: 

How to write a good literature review 

• Choose a Topic and Define the Research Question: 
• Decide on the Scope of Your Review: 
• Select Databases for Searches: 
• Conduct Searches and Keep Track: 
• Review the Literature: 
• Organize and Write Your Literature Review: 
• How to write a literature review faster with Paperpal? 
• Frequently asked questions 

What is a literature review?

A well-conducted literature review demonstrates the researcher’s familiarity with the existing literature, establishes the context for their own research, and contributes to scholarly conversations on the topic. One of the purposes of a literature review is also to help researchers avoid duplicating previous work and ensure that their research is informed by and builds upon the existing body of knowledge.

What is the purpose of literature review?

A literature review serves several important purposes within academic and research contexts. Here are some key objectives and functions of a literature review: 2  

1. Contextualizing the Research Problem: The literature review provides a background and context for the research problem under investigation. It helps to situate the study within the existing body of knowledge. 

2. Identifying Gaps in Knowledge: By identifying gaps, contradictions, or areas requiring further research, the researcher can shape the research question and justify the significance of the study. This is crucial for ensuring that the new research contributes something novel to the field. 

Find academic papers related to your research topic faster. Try Research on Paperpal  

3. Understanding Theoretical and Conceptual Frameworks: Literature reviews help researchers gain an understanding of the theoretical and conceptual frameworks used in previous studies. This aids in the development of a theoretical framework for the current research. 

4. Providing Methodological Insights: Another purpose of literature reviews is that it allows researchers to learn about the methodologies employed in previous studies. This can help in choosing appropriate research methods for the current study and avoiding pitfalls that others may have encountered. 

5. Establishing Credibility: A well-conducted literature review demonstrates the researcher’s familiarity with existing scholarship, establishing their credibility and expertise in the field. It also helps in building a solid foundation for the new research. 

6. Informing Hypotheses or Research Questions: The literature review guides the formulation of hypotheses or research questions by highlighting relevant findings and areas of uncertainty in existing literature. 

Literature review example

Let’s delve deeper with a literature review example: Let’s say your literature review is about the impact of climate change on biodiversity. You might format your literature review into sections such as the effects of climate change on habitat loss and species extinction, phenological changes, and marine biodiversity. Each section would then summarize and analyze relevant studies in those areas, highlighting key findings and identifying gaps in the research. The review would conclude by emphasizing the need for further research on specific aspects of the relationship between climate change and biodiversity. The following literature review template provides a glimpse into the recommended literature review structure and content, demonstrating how research findings are organized around specific themes within a broader topic. 

Literature Review on Climate Change Impacts on Biodiversity:

Climate change is a global phenomenon with far-reaching consequences, including significant impacts on biodiversity. This literature review synthesizes key findings from various studies: 

a. Habitat Loss and Species Extinction:

Climate change-induced alterations in temperature and precipitation patterns contribute to habitat loss, affecting numerous species (Thomas et al., 2004). The review discusses how these changes increase the risk of extinction, particularly for species with specific habitat requirements. 

b. Range Shifts and Phenological Changes:

Observations of range shifts and changes in the timing of biological events (phenology) are documented in response to changing climatic conditions (Parmesan & Yohe, 2003). These shifts affect ecosystems and may lead to mismatches between species and their resources. 

c. Ocean Acidification and Coral Reefs:

The review explores the impact of climate change on marine biodiversity, emphasizing ocean acidification’s threat to coral reefs (Hoegh-Guldberg et al., 2007). Changes in pH levels negatively affect coral calcification, disrupting the delicate balance of marine ecosystems. 

d. Adaptive Strategies and Conservation Efforts:

Recognizing the urgency of the situation, the literature review discusses various adaptive strategies adopted by species and conservation efforts aimed at mitigating the impacts of climate change on biodiversity (Hannah et al., 2007). It emphasizes the importance of interdisciplinary approaches for effective conservation planning. 

Strengthen your literature review with factual insights. Try Research on Paperpal for free!    

Writing a literature review involves summarizing and synthesizing existing research on a particular topic. A good literature review format should include the following elements. 

Introduction: The introduction sets the stage for your literature review, providing context and introducing the main focus of your review. 

• Opening Statement: Begin with a general statement about the broader topic and its significance in the field. 
• Scope and Purpose: Clearly define the scope of your literature review. Explain the specific research question or objective you aim to address. 
• Organizational Framework: Briefly outline the structure of your literature review, indicating how you will categorize and discuss the existing research. 
• Significance of the Study: Highlight why your literature review is important and how it contributes to the understanding of the chosen topic. 
• Thesis Statement: Conclude the introduction with a concise thesis statement that outlines the main argument or perspective you will develop in the body of the literature review. 

Body: The body of the literature review is where you provide a comprehensive analysis of existing literature, grouping studies based on themes, methodologies, or other relevant criteria. 

• Organize by Theme or Concept: Group studies that share common themes, concepts, or methodologies. Discuss each theme or concept in detail, summarizing key findings and identifying gaps or areas of disagreement. 
• Critical Analysis: Evaluate the strengths and weaknesses of each study. Discuss the methodologies used, the quality of evidence, and the overall contribution of each work to the understanding of the topic. 
• Synthesis of Findings: Synthesize the information from different studies to highlight trends, patterns, or areas of consensus in the literature. 
• Identification of Gaps: Discuss any gaps or limitations in the existing research and explain how your review contributes to filling these gaps. 
• Transition between Sections: Provide smooth transitions between different themes or concepts to maintain the flow of your literature review. 

Write and Cite as you go with Paperpal Research. Start now for free.   

Conclusion: The conclusion of your literature review should summarize the main findings, highlight the contributions of the review, and suggest avenues for future research. 

• Summary of Key Findings: Recap the main findings from the literature and restate how they contribute to your research question or objective. 
• Contributions to the Field: Discuss the overall contribution of your literature review to the existing knowledge in the field. 
• Implications and Applications: Explore the practical implications of the findings and suggest how they might impact future research or practice. 
• Recommendations for Future Research: Identify areas that require further investigation and propose potential directions for future research in the field. 
• Final Thoughts: Conclude with a final reflection on the importance of your literature review and its relevance to the broader academic community. 

Conducting a literature review

Conducting a literature review is an essential step in research that involves reviewing and analyzing existing literature on a specific topic. It’s important to know how to do a literature review effectively, so here are the steps to follow: 1  

Choose a Topic and Define the Research Question:

• Select a topic that is relevant to your field of study. 
• Clearly define your research question or objective. Determine what specific aspect of the topic do you want to explore? 

Decide on the Scope of Your Review:

• Determine the timeframe for your literature review. Are you focusing on recent developments, or do you want a historical overview? 
• Consider the geographical scope. Is your review global, or are you focusing on a specific region? 
• Define the inclusion and exclusion criteria. What types of sources will you include? Are there specific types of studies or publications you will exclude? 

Select Databases for Searches:

• Identify relevant databases for your field. Examples include PubMed, IEEE Xplore, Scopus, Web of Science, and Google Scholar. 
• Consider searching in library catalogs, institutional repositories, and specialized databases related to your topic. 

Conduct Searches and Keep Track:

• Develop a systematic search strategy using keywords, Boolean operators (AND, OR, NOT), and other search techniques. 
• Record and document your search strategy for transparency and replicability. 
• Keep track of the articles, including publication details, abstracts, and links. Use citation management tools like EndNote, Zotero, or Mendeley to organize your references. 

Review the Literature:

• Evaluate the relevance and quality of each source. Consider the methodology, sample size, and results of studies. 
• Organize the literature by themes or key concepts. Identify patterns, trends, and gaps in the existing research. 
• Summarize key findings and arguments from each source. Compare and contrast different perspectives. 
• Identify areas where there is a consensus in the literature and where there are conflicting opinions. 
• Provide critical analysis and synthesis of the literature. What are the strengths and weaknesses of existing research? 

Organize and Write Your Literature Review:

• Literature review outline should be based on themes, chronological order, or methodological approaches. 
• Write a clear and coherent narrative that synthesizes the information gathered. 
• Use proper citations for each source and ensure consistency in your citation style (APA, MLA, Chicago, etc.). 
• Conclude your literature review by summarizing key findings, identifying gaps, and suggesting areas for future research. 

Whether you’re exploring a new research field or finding new angles to develop an existing topic, sifting through hundreds of papers can take more time than you have to spare. But what if you could find science-backed insights with verified citations in seconds? That’s the power of Paperpal’s new Research feature!  

How to write a literature review faster with Paperpal?

Paperpal, an AI writing assistant, integrates powerful academic search capabilities within its writing platform. With the Research feature, you get 100% factual insights, with citations backed by 250M+ verified research articles, directly within your writing interface with the option to save relevant references in your Citation Library. By eliminating the need to switch tabs to find answers to all your research questions, Paperpal saves time and helps you stay focused on your writing.   

Here’s how to use the Research feature:  

• Ask a question: Get started with a new document on paperpal.com. Click on the “Research” feature and type your question in plain English. Paperpal will scour over 250 million research articles, including conference papers and preprints, to provide you with accurate insights and citations. 
• Review and Save: Paperpal summarizes the information, while citing sources and listing relevant reads. You can quickly scan the results to identify relevant references and save these directly to your built-in citations library for later access. 
• Cite with Confidence: Paperpal makes it easy to incorporate relevant citations and references into your writing, ensuring your arguments are well-supported by credible sources. This translates to a polished, well-researched literature review. 

The literature review sample and detailed advice on writing and conducting a review will help you produce a well-structured report. But remember that a good literature review is an ongoing process, and it may be necessary to revisit and update it as your research progresses. By combining effortless research with an easy citation process, Paperpal Research streamlines the literature review process and empowers you to write faster and with more confidence. Try Paperpal Research now and see for yourself.  

Frequently asked questions

A literature review is a critical and comprehensive analysis of existing literature (published and unpublished works) on a specific topic or research question and provides a synthesis of the current state of knowledge in a particular field. A well-conducted literature review is crucial for researchers to build upon existing knowledge, avoid duplication of efforts, and contribute to the advancement of their field. It also helps researchers situate their work within a broader context and facilitates the development of a sound theoretical and conceptual framework for their studies.

Literature review is a crucial component of research writing, providing a solid background for a research paper’s investigation. The aim is to keep professionals up to date by providing an understanding of ongoing developments within a specific field, including research methods, and experimental techniques used in that field, and present that knowledge in the form of a written report. Also, the depth and breadth of the literature review emphasizes the credibility of the scholar in his or her field.  

Before writing a literature review, it’s essential to undertake several preparatory steps to ensure that your review is well-researched, organized, and focused. This includes choosing a topic of general interest to you and doing exploratory research on that topic, writing an annotated bibliography, and noting major points, especially those that relate to the position you have taken on the topic. 

Literature reviews and academic research papers are essential components of scholarly work but serve different purposes within the academic realm. 3 A literature review aims to provide a foundation for understanding the current state of research on a particular topic, identify gaps or controversies, and lay the groundwork for future research. Therefore, it draws heavily from existing academic sources, including books, journal articles, and other scholarly publications. In contrast, an academic research paper aims to present new knowledge, contribute to the academic discourse, and advance the understanding of a specific research question. Therefore, it involves a mix of existing literature (in the introduction and literature review sections) and original data or findings obtained through research methods. 

Literature reviews are essential components of academic and research papers, and various strategies can be employed to conduct them effectively. If you want to know how to write a literature review for a research paper, here are four common approaches that are often used by researchers.  Chronological Review: This strategy involves organizing the literature based on the chronological order of publication. It helps to trace the development of a topic over time, showing how ideas, theories, and research have evolved.  Thematic Review: Thematic reviews focus on identifying and analyzing themes or topics that cut across different studies. Instead of organizing the literature chronologically, it is grouped by key themes or concepts, allowing for a comprehensive exploration of various aspects of the topic.  Methodological Review: This strategy involves organizing the literature based on the research methods employed in different studies. It helps to highlight the strengths and weaknesses of various methodologies and allows the reader to evaluate the reliability and validity of the research findings.  Theoretical Review: A theoretical review examines the literature based on the theoretical frameworks used in different studies. This approach helps to identify the key theories that have been applied to the topic and assess their contributions to the understanding of the subject.  It’s important to note that these strategies are not mutually exclusive, and a literature review may combine elements of more than one approach. The choice of strategy depends on the research question, the nature of the literature available, and the goals of the review. Additionally, other strategies, such as integrative reviews or systematic reviews, may be employed depending on the specific requirements of the research.

The literature review format can vary depending on the specific publication guidelines. However, there are some common elements and structures that are often followed. Here is a general guideline for the format of a literature review:  Introduction:   Provide an overview of the topic.  Define the scope and purpose of the literature review.  State the research question or objective.  Body:   Organize the literature by themes, concepts, or chronology.  Critically analyze and evaluate each source.  Discuss the strengths and weaknesses of the studies.  Highlight any methodological limitations or biases.  Identify patterns, connections, or contradictions in the existing research.  Conclusion:   Summarize the key points discussed in the literature review.  Highlight the research gap.  Address the research question or objective stated in the introduction.  Highlight the contributions of the review and suggest directions for future research.

Both annotated bibliographies and literature reviews involve the examination of scholarly sources. While annotated bibliographies focus on individual sources with brief annotations, literature reviews provide a more in-depth, integrated, and comprehensive analysis of existing literature on a specific topic. The key differences are as follows: 

References 

• Denney, A. S., & Tewksbury, R. (2013). How to write a literature review.  Journal of criminal justice education ,  24 (2), 218-234. 
• Pan, M. L. (2016).  Preparing literature reviews: Qualitative and quantitative approaches . Taylor & Francis. 
• Cantero, C. (2019). How to write a literature review.  San José State University Writing Center . 

Paperpal is an AI writing assistant that help academics write better, faster with real-time suggestions for in-depth language and grammar correction. Trained on millions of research manuscripts enhanced by professional academic editors, Paperpal delivers human precision at machine speed.  

Try it for free or upgrade to  Paperpal Prime , which unlocks unlimited access to premium features like academic translation, paraphrasing, contextual synonyms, consistency checks and more. It’s like always having a professional academic editor by your side! Go beyond limitations and experience the future of academic writing.  Get Paperpal Prime now at just US$19 a month!

Related Reads:

• Empirical Research: A Comprehensive Guide for Academics 
• How to Write a Scientific Paper in 10 Steps 
• How Long Should a Chapter Be?
• How to Use Paperpal to Generate Emails & Cover Letters?

6 Tips for Post-Doc Researchers to Take Their Career to the Next Level

Self-plagiarism in research: what it is and how to avoid it, you may also like, mla works cited page: format, template & examples, how to ace grant writing for research funding..., powerful academic phrases to improve your essay writing , how to write a high-quality conference paper, how paperpal’s research feature helps you develop and..., how paperpal is enhancing academic productivity and accelerating..., how to write a successful book chapter for..., academic editing: how to self-edit academic text with..., 4 ways paperpal encourages responsible writing with ai, what are scholarly sources and where can you....

• Interlibrary Loan and Scan & Deliver
• Course Reserves
• Purchase Request
• Collection Development & Maintenance
• Current Negotiations
• Ask a Librarian
• Instructor Support
• Library How-To
• Research Guides
• Research Support
• Study Rooms
• Research Rooms
• Partner Spaces
• Loanable Equipment
• Print, Scan, Copy
• 3D Printers
• Poster Printing
• OSULP Leadership
• Strategic Plan

Scholarly Articles: How can I tell?

• Journal Information

Literature Review

• Author and affiliation
• Introduction
• Specialized Vocabulary
• Methodology
• Research sponsors
• Peer-review

The literature review section of an article is a summary or analysis of all the research the author read before doing his/her own research. This section may be part of the introduction or in a section called Background. It provides the background on who has done related research, what that research has or has not uncovered and how the current research contributes to the conversation on the topic. When you read the lit review ask:

• Does the review of the literature logically lead up to the research questions?
• Do the authors review articles relevant to their research study?
• Do the authors show where there are gaps in the literature?

The lit review is also a good place to find other sources you may want to read on this topic to help you get the bigger picture.

• << Previous: Journal Information
• Next: Author and affiliation >>
• Last Updated: Apr 15, 2024 3:26 PM
• URL: https://guides.library.oregonstate.edu/ScholarlyArticle

Contact Info

121 The Valley Library Corvallis OR 97331–4501

Phone: 541-737-3331

Services for Persons with Disabilities

In the Valley Library

• Oregon State University Press
• Special Collections and Archives Research Center
• Undergrad Research & Writing Studio
• Graduate Student Commons
• Tutoring Services
• Northwest Art Collection

Digital Projects

• Oregon Explorer
• Oregon Digital
• ScholarsArchive@OSU
• Digital Publishing Initiatives
• Atlas of the Pacific Northwest
• Marilyn Potts Guin Library  
• Cascades Campus Library
• McDowell Library of Vet Medicine

University Library

Write a literature review.

• Examples and Further Information

1. Introduction

Not to be confused with a book review, a literature review surveys scholarly articles, books and other sources (e.g. dissertations, conference proceedings) relevant to a particular issue, area of research, or theory, providing a description, summary, and critical evaluation of each work. The purpose is to offer an overview of significant literature published on a topic.

2. Components

Similar to primary research, development of the literature review requires four stages:

• Problem formulation—which topic or field is being examined and what are its component issues?
• Literature search—finding materials relevant to the subject being explored
• Data evaluation—determining which literature makes a significant contribution to the understanding of the topic
• Analysis and interpretation—discussing the findings and conclusions of pertinent literature

Literature reviews should comprise the following elements:

• An overview of the subject, issue or theory under consideration, along with the objectives of the literature review
• Division of works under review into categories (e.g. those in support of a particular position, those against, and those offering alternative theses entirely)
• Explanation of how each work is similar to and how it varies from the others
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research

In assessing each piece, consideration should be given to:

• Provenance—What are the author's credentials? Are the author's arguments supported by evidence (e.g. primary historical material, case studies, narratives, statistics, recent scientific findings)?
• Objectivity—Is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
• Persuasiveness—Which of the author's theses are most/least convincing?
• Value—Are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

3. Definition and Use/Purpose

A literature review may constitute an essential chapter of a thesis or dissertation, or may be a self-contained review of writings on a subject. In either case, its purpose is to:

• Place each work in the context of its contribution to the understanding of the subject under review
• Describe the relationship of each work to the others under consideration
• Identify new ways to interpret, and shed light on any gaps in, previous research
• Resolve conflicts amongst seemingly contradictory previous studies
• Identify areas of prior scholarship to prevent duplication of effort
• Point the way forward for further research
• Place one's original work (in the case of theses or dissertations) in the context of existing literature

The literature review itself, however, does not present new primary scholarship.

• Next: Examples and Further Information >>

Creative Commons Attribution 3.0 License except where otherwise noted.

Land Acknowledgement

The land on which we gather is the unceded territory of the Awaswas-speaking Uypi Tribe. The Amah Mutsun Tribal Band, comprised of the descendants of indigenous people taken to missions Santa Cruz and San Juan Bautista during Spanish colonization of the Central Coast, is today working hard to restore traditional stewardship practices on these lands and heal from historical trauma.

The land acknowledgement used at UC Santa Cruz was developed in partnership with the Amah Mutsun Tribal Band Chairman and the Amah Mutsun Relearning Program at the UCSC Arboretum .

• Skip to search box
• Skip to main content
• Princeton University Library
• Research Guides
• Writing Seminars
• Writing Seminar 163/164 And the Rest is Drag
• What is a Literature Review?

Writing Seminar 163/164 And the Rest is Drag: What is a Literature Review?

• Finding Books
• Finding Articles
• Specialized Databases
• Academic Integrity at Princeton This link opens in a new window

Finding Examples

It may be useful to look at other reviews to learn how researchers in the field "summarize and synthesize" the literature. Most research article or dissertation in the sciences will include a section which reviews the literature. Though the section may not be labeled as such, you will quickly recognize it by the number of citations and the discussion of the literature. Another option is to look for Review Articles, which are literature reviews as a stand alone article. Here are some resources where you can find Research Articles, Review Articles and Dissertations:

• Articles+ - Due to the interdisciplinary nature of gender & sexuality studies Articles+ can be a great place to start your research. Please make use of the filters on the left-hand side of the screen to help refine your searches. 
• Gender Studies Database  & LGBT Thought and Culture - Gender Studies Database & LGBT Though and Culture have a large corpus of reviews and research articles. As with Articles+ make sure to take advantage of the filters (type of publication, publication date) to help refine your searches. 
• Google Scholar   - Using the Cited By feature, hyperlinked below the search results, you can trace the scholarly conversation moving forward. 
• Dissertations @ Princeton - Provides access to many Princeton dissertations, full text is available for most published after 1996.
• Purdue OWL - The Purdue OWL site provides tips and examples of literature reviews and is a great source for reviewing citation styles 

*** Note about using Review Articles in your research - while they are useful in helping you to locate articles on your topic, remember that you must go to and use the original source if you intend to include a study mentioned in the review. The only time you would cite a review article is if they have made an original insight in their work that you talk about in your paper. Going to the original research paper allows you to verify the information about that study and determine whether the points made in the review are valid and accurate.

What is a literature review?

A literature review surveys scholarly articles, books and other sources relevant to a particular issue, area of research, or theory. The purpose is to offer an overview of significant literature published on a topic.

A literature review may constitute an essential chapter of a thesis or dissertation, or may be a self-contained review of writings on a subject. In either case, its purpose is to:

• Place each work in the context of its contribution to the understanding of the subject under review
• Describe the relationship of each work to the others under consideration
• Identify new ways to interpret, and shed light on any gaps in, previous research
• Resolve conflicts amongst seemingly contradictory previous studies
• Identify areas of prior scholarship to prevent duplication of effort
• Point the way forward for further research
• Place one's original work (in the case of theses or dissertations) in the context of existing literature

A literature review can be just a simple summary of the sources, but it usually has an organizational pattern and combines both summary and synthesis. A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information. It might give a new interpretation of old material or combine new with old interpretations. Or it might trace the intellectual progression of the field, including major debates. And depending on the situation, the literature review may evaluate the sources and advise the reader on the most pertinent or relevant.

Similar to primary research, development of the literature review requires four stages:

• Problem formulation—which topic or field is being examined and what are its component issues?
• Literature search—finding materials relevant to the subject being explored
• Data evaluation—determining which literature makes a significant contribution to the understanding of the topic
• Analysis and interpretation—discussing the findings and conclusions of pertinent literature

Remember, this is a process and not necessarily a linear one. As you search and evaluate the literature, you may refine your topic or head in a different direction which will take you back to the search stage. In fact, it is useful to evaluate as you go along so you don't spend hours researching one aspect of your topic only to find yourself more interested in another.

The main focus of an academic research paper is to develop a new argument, and a research paper will contain a literature review as one of its parts. In a research paper, you use the literature as a foundation and as support for a new insight that you contribute. The focus of a literature review, however, is to summarize and synthesize the arguments and ideas of others without adding new contributions.

• << Previous: Specialized Databases
• Next: Academic Integrity at Princeton >>
• Subjects: Writing Program
• Last Updated: Jan 30, 2024 2:56 PM
• URL: https://libguides.princeton.edu/andtherestisdrag

Article Contents

Introduction, supplementary material, authorship contributions, acknowledgements, data availability.

• < Previous

Unleashing the potential of Health Promotion in primary care—a scoping literature review

• Article contents
• Figures & tables
• Supplementary Data

Adela Bisak, Martin Stafström, Unleashing the potential of Health Promotion in primary care—a scoping literature review, Health Promotion International , Volume 39, Issue 3, June 2024, daae044, https://doi.org/10.1093/heapro/daae044

• Permissions Icon Permissions

The purpose of this study is to gain a better understanding of the role and extent of health promotion lifestyle interventions targeting adults in primary care, and especially those who are considered overall healthy, i.e. to study the outcomes of research applying salutogenesis. We performed a literature review, with three specific aims. First, to identify studies that have targeted the healthy population in intervention within the primary health care field with health promotion activities. Second, to describe these interventions in terms of which health problems they have targeted and what the interventions have entailed. Third, to assess what these programs have resulted in, in terms of health outcomes. This scoping review of 42 studies, that applied salutogenesis in primary care interventions shows that health promotion targeting healthy individuals is relevant and effective. The PRISMA-ScR guidelines for reporting on scoping review were used. Most interventions were successful in reducing disease-related risks including CVD, CVD mortality, all-cause mortality, but even more importantly success in behavioural change, sustained at follow-up. Additionally, this review shows that health promotion lifestyle interventions can improve mental health, even when having different aims.

This article describes the importance of including healthy individuals in health promotion activities, applying salutogenesis, as there are significant positive health outcomes effects if they participate in health interventions.

The study amplifies that the prevention paradox should always be considered when designing health promotion interventions.

This article shows that the greatest effects when targeting healthy individuals are found in lower all-cause mortality and CVD risks, mainly because these programs manage to lead to long-lasting lifestyle changes.

Health Promotion is, according to Nutbeam and Muscat (2021 , p. 1580), ‘[…] the process of enabling people to increase control over, and to improve their health’. This process entails a comprehensive approach to change on all levels, from structures to individuals, improving health mainly through addressing the social determinants of health.

Whereas the most overarching processes are initiated on a structural level through global and national health policies ( Cross et al ., 2020 ), health promotion strategies are also widely employed in health interventions targeting individuals. It could entail smoking cessation programs, weight loss programs and adolescent alcohol use, just to mention some common health outcome target areas ( Green et al. , 2019 ). Even when deploying health promotion strategies at a national policy level, it is not uncommon that the programs designed to target individuals and groups are more inspired by pathogenesis, rather than salutogenesis ( Nutbeam and Muscat, 2021 ).

A widespread strategy in the latter programs is that individuals are screened for a need to receive an intervention, so-called secondary or indicated prevention programs, where those who report a riskier lifestyle, or test worse on psychometric or biometric indicators are eligible for receiving the intervention, and those not having the same risks are excluded from the program on the premise that they are, based on study protocol definitions, healthy individuals.

Based on the principles of salutogenesis, this is a somewhat inappropriate approach. Within the strategy of health promotion, it is assumed that all people, no matter their level of risk, would find feedback on their health valuable. Those in need of change should receive the necessary resources and tools to change, whereas those who do not have to change should have their lifestyles positively reinforced. In addition, the prevention paradox ( Rose, 1981 ) postulates that it is important to address the majority, as there will be plenty of adverse health outcomes stemming from them. In conventional indicated prevention programs, attention to those who are non-eligible for interventions is, thus, often completely disregarded.

One common arena for such programs is primary health care. There is a wide range of evidence-based programs that have shown efficacy in reducing the health risks among those who have the riskiest lifestyles in relation to, e.g. alcohol use ( Beyer et al ., 2019 ), smoking ( Cantera et al ., 2015 ), depression ( Bortolotti et al ., 2008 ), diabetes ( Galaviz et al ., 2018 ) and cardiovascular diseases ( Álvarez-Bueno et al. , 2015 ). The at-risk groups vary across the different diseases, but a vast majority of patients targeted in the above studies were identified after screening as non-eligible to participate in the intervention in question. From this follows that a large number of individuals do not receive any substantial health information, nor are their health outcomes measured as they are not included in the intervention. From a health promotion perspective, this seems like a lost opportunity. Additionally, this raises the question of whether a healthy population is systematically disadvantaged compared to those individuals at high risk, which might point to some less-known health inequities or disparities ( Braveman, 2006 ) present in primary care.

In order to gain a better understanding of the effects of health promotion as an overall approach, and to understand the implications of the prevention paradox, it would be pertinent to include the non-eligible group in both the feedback loop—mainly offering them structured positive reinforcement—and to subsequently measure their health and attributed lifestyles.

The purpose of this study is to gain a better understanding of the role and extent of health promotion lifestyle interventions targeting adults in primary care, especially those who are considered overall healthy. More precisely we aim to assess to what extent health promotion practices in primary care address healthy individuals, not only those who need to undergo a lifestyle change. In order to do so, we performed a literature review, with three specific aims. First, to identify studies that have only targeted the healthy population, or healthy population in addition to high-risk group in intervention within the primary health care field with health promotion activities. Second, to describe these interventions in terms of which health problems they have targeted and what the interventions have entailed. Third, to assess what the initiatives published in the research literature have resulted in terms of health outcomes.

Due to the width of the topic and study designs we chose to perform a scoping review, with the aim of summarizing and disseminating previous research and identifying research gaps in the literature ( Arksey and O’Malley, 2005 ). The search process was iterative and non-linear, reflecting upon the results from the literature search at each stage and then repeating steps where necessary to cover the literature more comprehensively ( Arksey and O’Malley, 2005 ).

A few terms demand some further definition within the scope of this review . Healthy individual is a fluid term varying across different studies and contexts, yet it is a key concept in this particular study. The term involves those without chronic disease, who are indicated as not being of an elevated risk of developing a disease linked to the health outcome they have been screened for, but they could very well be at risk for diseases beyond the scope of the study they have been examined within. Primary health care may in this review indicates different types of settings from the most common one relating to general practitioners and family doctors to occupational medicine or periodical work-related health check-ups but also dental health care. Health promotion interventions in this study are understood as interventions that aim to keep people healthy longer, by providing positive feedback in relation to current and new health behaviours, rather than controlling health status by medication use.

Search strategy

The search was done across two databases PubMed and Embase, by combining different strings related to keywords ‘health promotion’ and ‘primary care’, while the rest of the strings varied, more specific search queries are available in Supplementary Appendix A . The search was conducted during June and July 2023 and consisted of publications dated between July 2008 and July 2023 (i.e. the last 15 years). Additional studies were identified manually from references of the included articles and by ‘See all similar articles’ option in PubMed and ‘similar records’ in Embase. The article titles were scanned from databases, followed by screening titles and abstracts through the Covidence software, and then finally the full articles were read. Results were filtered for adult humans, defined as age 18–75, abstracts being available and the studies were authored in English.

Articles were included if (i) the population consisted of working-age adults, (ii) the population included those screened as healthy within a whole sample followed by an intervention or interventions ideally at follow-up, (iii) the study focused on primary prevention (iv) the study focused on lifestyle interventions, (v) the study examined lifestyle-related behaviours. Exclusion criteria for papers were (i) focused on children—below the age of 18 or elderly, (ii) addiction behaviours, (iii) excluding healthy individuals from intervention after screening or using them exclusively in the control group, (iv) using only high-risk population as healthy, (v) promoting only mental health, (vi) secondary prevention, (vii) screening is the only intervention, (viii) reviews and study protocols.

After full-text screening, the data charting process for reviewing, sorting and documenting information ( Arksey and O’Malley, 2005 ) was done using Covidence, Data Extraction version 2 recommended for scoping reviews. The Data Extraction Template included columns for article title, author, country in which the study was conducted, methods (aim, design, population description, inclusion and exclusion criteria) intervention description, outcome measures, relevant results, follow-up (yes/no), study setting (primary care, worksite/occupational, population-based), study category (lifestyle, physical activity and diet, cardiovascular disease, alcohol consumption) and a field for additional notes where needed.

Due to great inconsistencies between studies in the design, populations and outcomes, critical appraisal of individual sources of evidence—an optional step in PRISMA-ScR ( Tricco et al ., 2018 ) guideline list was not done, although concerning research aim it would be useful for assessing the quality of evidence. Although exclusion/inclusion criteria were respected, what was considered as ‘healthy’, ‘middle-’ or ‘high-risk population’ differed significantly in studies, due to differences in definition of terms. Moreover, this decision was made as the AMSTAR tool would not be an adequate choice due to the inclusion of a non-randomized design, and although the AMSTAR 2 tool could potentially be used, this review also included several economic evaluations and follow-ups ( Supplementary Table S1 for more details), or indicators differing highly across studies.

For the synthesis of results ( Tricco et al ., 2018 ), the studies were grouped by the type of the outcome—disease, i.e. CVD or lifestyle/behaviour: physical activity and diet or alcohol consumption. Furthermore, the studies were summarized by setting, risk group and follow-up. None of the systematic reviews with similar research aims were detected during the search.

The selection of sources of evidence ( Tricco et al ., 2018 ) was done as described: 353 references were imported for screening, 72 duplicates were removed, 268 studies were screened against title and abstract during which 198 studies were excluded while 69 studies were assessed for full-text eligibility, when 27 studies were excluded: 12 for wrong intervention, 8 for wrong patient population, 4 for wrong study design 1 was not in English, 1 for wrong indication and 1 for wrong setting, after which 42 studies were included. PRISMA of full screening is found in Figure 1 .

PRISMA of full screening strategy.

Lifestyle interventions

A summary of the study setting, samples and the main outcomes of the 42 studies analysed in this scoping review is presented in Supplementary Table S1 .

In general, the intervention studies analysed here had different main strategies, including: individually tailored programs ( Doumas and Hannah, 2008 ; Gram et al ., 2012 ; Watson et al ., 2015 ) risk-based, group-based ( Recio-Rodriguez et al ., 2016 ) or mixed variants ( Matano et al ., 2007 ; Matzer et al ., 2018 ).

Cardiovascular health

We found several different lifestyle interventions targeting CVD risk. There were a set of programs that addressed physical activity in the workplace, which significantly reduced the CVD risk in healthy participants adhering to the program ( Gram et al ., 2012 ; Dalager et al ., 2016 ; Eng et al ., 2016 ; Biffi et al ., 2018 ). In primary care, an observational study by Journath et al . (2020) , showed an association between healthy participant participation in a CVD prevention programme promoting physical activity and a healthy lifestyle with lower risk of CV events (12%), CV mortality (21%) and all-cause mortality (17%) after 20 years of follow-up.

Similarly, we found interventions in primary care settings that led to changes in physical activity and dietary patterns among all participants—not only those at high risk of CVD morbidity and mortality. These studies described generally decreased CVD risks ( Richardson et al ., 2008 ; Buckland et al ., 2009 ; Nguyen et al ., 2012 ; Gibson et al ., 2014 ; Bo et al ., 2016 ; Lidin et al ., 2018 ; Lingfors and Persson, 2019 ), CVD-related mortality ( Blomstedt et al ., 2011 ; Persson et al ., 2015 ; Jeong et al ., 2019 ) and all-cause mortality ( Blomstedt et al ., 2015 ; Bo et al ., 2016 ; Bonaccio et al ., 2019 ).

In a prospective observational study on healthy individuals and those with CVD conducted by Lidin et al . (2018) , the prevalence within the sample at risk of CVD decreased significantly at 12-month follow-up by 15%. In several studies, the changes in health behaviours among the participants showed to be sustained in follow-ups conducted after intervention discontinuation ( Buckland et al ., 2009 ; Gibson et al ., 2014 ; Baumann et al ., 2015 ; Blomstedt et al ., 2015 ; Lidin et al ., 2018 ), while some cardiovascular risk factors, such as salty diets and smoking, showed evidence of significant decrease in a relatively short period ( Nguyen et al ., 2012 ).

Physical activity and diet

In interventions addressing physical activity and diet, it was evident that healthy individuals were more likely to adhere to physical activity interventions ( Dalager et al ., 2016 ; Biffi et al ., 2018 ; Jeong et al ., 2019 ) compared to those with a disease. One community-based walking intervention ( Yang and Kim, 2022 ) affected not only the level of physical activity significantly but also a positive overall change towards a health-promoting lifestyle and decreased perceived stress. Similarly, several mental health measures including general mental health ( Oude Hengel et al ., 2014 ), anxiety and depression ( Gibson et al ., 2014 ) and stress ( Lingfors et al ., 2009 ; Matzer et al ., 2018 ) in participants improved during interventions and at follow-up when targeting physical activity and diet.

Additionally, concerning physical activity and diet outcomes, there were a higher feasibility of uptake among participants in health promotion programs compared to those only receiving standard care in primary care ( Lingfors et al ., 2009 ; Zabaleta-Del-Olmo et al ., 2021 ). Anokye et al. (2014) argued that brief advice intervention was more effective—leading to 466 QALYs gained, compared to standard care—implying greater cost-effectiveness.

Healthier lifestyles were also maintained at the follow-up. Reduction in risk factors was found to be sustained in follow-ups at 12 months ( Gibson et al ., 2014 ) or improvements in dietary outcomes over 5 years ( Baumann et al ., 2015 ), and sustained lower blood pressure over 6 years ( Eng et al ., 2016 ).

Several interventions promoting physical activity in primary care settings showed significant results in increasing it in all patients, not only in those with chronic disease diagnosis ( Robroek et al ., 2010 ; Gram et al ., 2012 ; Hardcastle et al ., 2012 ; Viester et al ., 2015 ; Byrne et al ., 2016 ; Dalager et al ., 2016 ; Eng et al ., 2016 ; Recio-Rodriguez et al ., 2016 ; Biffi et al ., 2018 ; Matzer et al ., 2018 ; Yang and Kim, 2022 ), and similar patterns were also found concerning a change towards a healthier diet ( Lingfors et al ., 2009 ; Wendel-Vos et al ., 2009 ; Robroek et al ., 2010 ; Baumann et al ., 2015 ; Viester et al ., 2015 ; Bo et al ., 2016 ; Byrne et al ., 2016 ; Kosendiak et al ., 2021 ).

There was disagreement among the above studies in relation to the effectiveness of these interventions among healthy individuals. For example, in the case of implementing a Mediterranean diet, one report argued that a healthy diet should be prioritized, indicating significant hazard ratios (HR) of attaining a Mediterranean diet for all-cause mortality (HR = 0.83), CV mortality (HR = 0.75) and CV events (HR = 0.79) among low-risk individuals ( Bo et al ., 2016 ). Others, however, claimed that there was no evidence of healthier participants being more susceptible to changes in physical activity and diet ( Robroek et al ., 2010 ).

Alcohol consumption

Interventions aimed at decreasing alcohol consumption were divided between those being most effective in high-risk drinkers ( Doumas and Hannah, 2008 ; Kirkman et al ., 2018 ), and both moderate and low-risk drinkers ( Matano et al ., 2007 ). These interventions were, at large, seen as cost-saving ( Watson et al ., 2015 ) and feasible in primary care ( Neuner-Jehle et al ., 2013 ). Some studies found a sustained decrease in alcohol consumption in those adhering to the interventions, compared to the control groups at 1 ( Pemberton et al ., 2011 ) and 4 months after the intervention ( Kirkman et al ., 2018 ), whereas others failed to find a significant difference between groups.

Intervention setting

The interventions took place in primary care settings, though these were either in community-based or occupational settings. The findings suggested that there were some discrepancies between these different settings.

When it comes to a community-based setting, the difference is made between interventions conducted on a sample of those visiting primary health care or a sample representative for a population of one community—town, or region. Primary care community-based studies tended to either include participants who were primary care visitors with a long follow-up period, or interventions conducted in primary care clinic centres with a shorter follow-up period, most often using experimental design, sampling individuals living in the community that did not necessarily had an intention to seek care ( Richardson et al ., 2008 ; Hardcastle et al ., 2012 ; Nguyen et al ., 2012 ; Grunfeld et al ., 2013 ; Baumann et al ., 2015 ; Bo et al ., 2016 ; Lidin et al ., 2018 ; Zabaleta-Del-Olmo et al ., 2021 ).

Overall, the community-based studies were conducted on a sample representative for a population of a smaller community ( Kosendiak et al ., 2021 ; Yang and Kim, 2022 ), region ( Lingfors et al ., 2009 ; Wendel-Vos et al ., 2009 ; Gibson et al ., 2014 ; Persson et al ., 2015 ; Bonaccio et al ., 2019 ; Jeong et al ., 2019 ; Lingfors and Persson, 2019 ; Journath et al ., 2020 ) or a country ( Buckland et al ., 2009 ; Blomstedt et al ., 2011 ; Neuner-Jehle et al ., 2013 ), often followed by a longer follow-up period. Finally, some studies were evaluations of previous interventions ( Richardson et al ., 2008 ; Anokye et al ., 2014 ).

Worksite interventions comprised of different occupational roles, often including several of those in the same sample ( Eng et al ., 2016 ), or segmenting based on how physically active the occupation was, e.g. office workers ( Dalager et al ., 2016 ), construction workers ( Gram et al ., 2012 ; Oude Hengel et al ., 2014 ; Viester et al ., 2015 ), sailors ( Hjarnoe and Leppin, 2013 ), farmers ( van Doorn et al ., 2019 ) or simply more active individuals ( Biffi et al ., 2018 ). This had the implication that approaches to intervention differed widely across the studies.

Several interventions were conducted online using a web-based interface, while others were in a professional setting ( Matano et al ., 2007 ; Doumas and Hannah, 2008 ; Robroek et al ., 2010 ; Pemberton et al ., 2011 ; Khadjesari et al ., 2014 ) or in some cases community-based ( Recio-Rodriguez et al ., 2016 ; Kirkman et al ., 2018 ).

Categorization of risk among participants

Many studies applied specific risk criteria based on the participants’ morbidity risks: including groups of low, middle, high risk ( Persson et al ., 2015 ; Bo et al ., 2016 ; Lingfors and Persson, 2019 ), low and high risk ( Baumann et al ., 2015 ), middle and high risk ( Gibson et al ., 2014 ). While some did not distinguish between risk groups ( Wendel-Vos et al ., 2009 ; Blomstedt et al ., 2011 ; Byrne et al ., 2016 ; Journath et al ., 2020 ). In some studies, however, the protocol included mixed populations of those who were healthy and those who had a chronic disease ( Anokye et al ., 2014 ; Bonaccio et al ., 2019 ). Finally, different studies came up with their own meaning of ‘healthy individual’ or ‘healthy population’ based on the health problem they addressed, i.e. having a sedentary lifestyle or high alcohol consumption. Other criteria for being a part of a healthy population were having a high risk for a disease, one or several risks but not the disease itself, or being above a reference value without having a diagnosis.

Ethical implications of healthy controls

Some interventions were screening-result-based, meaning that there was a difference in the treatment of those with good health and those with some complications. In other words, although not excluding healthy individuals, the study protocol included healthy individuals partially receiving full treatment, in the intervention. Studies that excluded those who were healthy from the sample after screening or used them as a control group were excluded from this review. However, some included studies had a healthy control group. Overall, the studies included in this review did not discuss the ethical implications of including healthy populations as controls, or when that was the case, the ethical impact of excluding healthy participants from an intervention.

This scoping review speaks not only of the role and extent of health promotion for healthy individuals in primary care but also of the importance and effects it has on population health. The results showing the association of lifestyle interventions with CVD risk show great implications for future use in primary care, different contexts and feasibility. Physical activity interventions were additionally found to be related to some improvements in mental health.

Interventions aimed at alcohol consumption were found successful in decreasing the amount of drinking sustainably, while the main discussion was based on whether they should be aimed at high-risk only, or at middle- and low-risk drinkers as well, due to mixed results in said groups. The majority of interventions were based in a worksite setting, meaning that this context might be useful for tackling the issue. This approach showed that outcomes might be beneficial even when not reaching the primary goal. Examples of this are findings showing that although not reducing CVD risk, changes in health behaviours were sustained in follow-up ( Baumann et al ., 2015 ), less drastic changes decreasing CVD risk in the healthy population ( Buckland et al ., 2009 ) and beneficial effects of physical activity intervention on worker’s health without an overall increase in physical activity ( McEachan et al ., 2011 ). Finally, in most cases, as mentioned, changes in health behaviours were associated with changes in CVD risk.

Some interventions showed that health promotion benefits could be even bigger ( Bo et al ., 2016 ) or that adherence is higher in healthy participants ( Dalager et al ., 2016 ; Biffi et al ., 2018 ; Jeong et al ., 2019 ), while other authors disagree ( Robroek et al ., 2010 ). This could be traced to the topic of prioritising primary care for healthy, versus only those at high risk/ already with a disease—secondary care approach according to this review definitions. Designing interventions only for high-risk can make them less successful in healthy participants, as displayed in a study by Blomstedt et al . (2011) where self-rated health decreased in 21% of the good baseline health participants at the 10-year follow-up. Furthermore, from the Rose’s (1981) term of prevention paradox—a great benefit for the population can be almost non-existent for an individual, while if we only focus on high-risk cases, many individuals at low-risk can mean worse health outcomes compared to a small number at high-risk ( Rose, 2001 ). In other words, by focusing only on high-risk population, the downsides are care that can be less efficient, less feasible, more expensive and lead to worse health outcomes. This choice should not be exclusive, as excluding either populations can cause ethical concerns. However, this article gives priority to early prevention, by health promotion for healthy individuals in primary care. Additionally, if it is shown that ‘ Systems based on primary care have better population health, health equity, and health care quality, and lower health care expenditure… ’ ( Stange et al. , 2023 ), different treatment of those who are currently healthy presents an obstacle worth mentioning for achieving health equity in primary care. Furthermore, the role of promoting health to healthy populations and their inclusion in interventions is crucial for improving population health in the future.

Articles focusing on smoking cessation, alcoholism, substance misuse interventions were excluded from this scoping review as they represent addictions and are therefore different from lifestyle interventions. Originally, oral health and dental care interventions were to be included, but there were not enough studies matching the scoping review inclusion requirements.

As expected, the process of finding articles appropriate for inclusion was challenging. Even when the inclusion criteria, at first glance, were satisfied, most studies we came across had excluded healthy participants from the sample after screening for being asymptomatic or not having enough risk factors. They were, however, often a part of a control group, and usually received standard care or no care at all. This approach puts healthy individuals in a vulnerable position, by not addressing their needs to change lifestyles that eventually could contribute to an early death or becoming unwell. Our findings suggest that interventions that include healthy individuals could improve quality of life and health status both at the population and individual levels.

Due to studies using different risk criteria, as well as including many study designs and topics, it was hard to make general conclusions. Nevertheless, as a scoping review, we mapped the area of research by identifying the gaps in the evidence base, and summarizing and disseminating research findings ( Arksey and O’Malley, 2005 ), instead of appraising the quality of evidence in different studies.

Concerning the above, a big research gap was detected in studies focusing on, or even including healthy populations. Furthermore, there is a lack of a coherent or comprehensive methodology in assessing the effects of what is considered health promotion, which calls for a more specific approach and a clear definition of the term. Additionally, the question of intervention staff skills should be raised. Is it necessary that health promotion interventions should be conducted by clinically trained professionals or, innovatively, by staff trained in the topic at hand when possible? Another aspect that is important to problematize is whether it is ethical to exclude healthy individuals in health promotion intervention studies even if they would benefit from participating if included? Furthermore, if healthy individuals are systematically discriminated ( Braveman, 2006 ), receive worse treatment and have the risk of worse health outcomes in the future, it is critical to include them in interventions for achieving better health of populations. This has great practical implications for primary care. Similarly, from a cost-benefit perspective, research should address if excluding healthy individuals might affect the cost-effectiveness of health promotion interventions.

An apparent limitation within this review is the culturally uniform sample of studies. Most studies that we were able to identify were a result of research in the global north, with a strong emphasis on either North America or the EU. Only two studies were from less affluent settings in Southeast Asia ( Nguyen et al ., 2012 ; Bo et al ., 2016 ). Given that the findings suggest that these interventions are cost-effective and do not require substantial investments, these programs could have great potential in low-resource settings if more systematically researched.

This scoping review of 42 studies applying salutogenesis in primary care interventions shows that health promotion targeting healthy individuals is relevant and effective. Most interventions were successful in reducing disease-related risks including CVD, CVD mortality, all-cause mortality, but even more importantly success in behavioural change, sustained at follow-up. Additionally, this review shows that health promotion lifestyle interventions can improve mental health, even when having different aims.

Supplementary material is available at Health Promotion International online.

A.B. performed the literature search, performed most of the data analysis and was the major contributor in writing the Methods and Results sections of the manuscript. M.S. formulated the research questions and scope of the study. He gave considerable input to the data analysis, gave input on all sections of the study—including writing and editing—and was the main author of the Introduction and Discussion. Both authors read and approved the final manuscript.

We would like to express our gratitude to Maria Björklund, librarian, at the Faculty of Medicine Library, Lund University at CRC in Malmö, who assisted us in the literature search.

A.B.’s contribution was in part funded by a scholarship she received from the Faculty of Medicine and in part by internal funds at the Division of Social Medicine and Global Health, Lund University, the latter also funded M.S.’s contribution.

The data underlying this article are available in the article and in its online supplementary material.

Álvarez-Bueno , C. , Cavero-Redondo , I. , Martínez-Andrés , M. , Arias-Palencia , N. , Ramos-Blanes , R. and Salcedo-Aguilar , F. ( 2015 ) Effectiveness of multifactorial interventions in primary health care settings for primary prevention of cardiovascular disease: a systematic review of systematic reviews . Preventive Medicine , 76 , S68 – S75 .

Google Scholar

Anokye , N. K. , Lord , J. and Fox-Rushby , J. ( 2014 ) Is brief advice in primary care a cost-effective way to promote physical activity ? British Journal of Sports Medicine , 48 , 202 – 206 .

Arksey , H. and O’Malley , L. ( 2005 ) Scoping studies: towards a methodological framework . International Journal of Social Research Methodology , 8 , 19 – 32 .

Baumann , S. , Toft , U. , Aadahl , M. , Jørgensen , T. and Pisinger , C. ( 2015 ) The long-term effect of screening and lifestyle counseling on changes in physical activity and diet: the Inter99 Study—a randomized controlled trial . The International Journal of Behavioral Nutrition and Physical Activity , 12 , 33 .

Beyer , F. R. , Campbell , F. , Bertholet , N. , Daeppen , J. B. , Saunders , J. B. , Pienaar , E. D. et al. . ( 2019 ) The Cochrane 2018 review on brief interventions in primary care for hazardous and harmful alcohol consumption: a distillation for clinicians and policy makers . Alcohol and Alcoholism (Oxford, Oxfordshire) , 54 , 417 – 427 .

Biffi , A. , Fernando , F. , Adami , P. E. , Messina , M. , Sirico , F. , Di Paolo , F. et al. . ( 2018 ) Ferrari corporate wellness program: results of a pilot analysis and the ‘Drag’ impact in the workplace . High Blood Pressure & Cardiovascular Prevention , 25 , 261 – 266 .

Blomstedt , Y. , Emmelin , M. and Weinehall , L. ( 2011 ) What about healthy participants? The improvement and deterioration of self-reported health at a 10-year follow-up of the Västerbotten Intervention Programme . Global Health Action , 4 , 5435 .

Blomstedt , Y. , Norberg , M. , Stenlund , H. , Nyström , L. , Lönnberg , G. , Boman , K. et al. . ( 2015 ) Impact of a combined community and primary care prevention strategy on all-cause and cardiovascular mortality: a cohort analysis based on 1 million person-years of follow-up in Västerbotten County, Sweden, during 1990-2006 . BMJ Open , 5 , e009651 .

Bo , S. , Ponzo , V. , Goitre , I. , Fadda , M. , Pezzana , A. , Beccuti , G. et al. . ( 2016 ) Predictive role of the Mediterranean diet on mortality in individuals at low cardiovascular risk: a 12-year follow-up population-based cohort study . Journal of Translational Medicine , 14 , 91 .

Bonaccio , M. , Di Castelnuovo , A. , Costanzo , S. , De Curtis , A. , Persichillo , M. , Cerletti , C. et al. ; Moli-sani Study Investigators . ( 2019 ) Impact of combined healthy lifestyle factors on survival in an adult general population and in high-risk groups: prospective results from the Moli-sani Study . Journal of Internal Medicine , 286 , 207 – 220 .

Bortolotti , B. , Menchetti , M. , Bellini , F. , Montaguti , M. B. and Berardi , D. ( 2008 ) Psychological interventions for major depression in primary care: a meta-analytic review of randomized controlled trials . General Hospital Psychiatry , 30 , 293 – 302 .

Braveman , P. ( 2006 ) Health disparities and health equity: concepts and measurement . Annual Review of Public Health , 27 , 167 – 194 .

Buckland , G. , González , C. A. , Agudo , A. , Vilardell , M. , Berenguer , A. , Amiano , P. et al. . ( 2009 ) Adherence to the Mediterranean diet and risk of coronary heart disease in the Spanish EPIC cohort study . American Journal of Epidemiology , 170 , 1518 – 1529 .

Byrne , D. W. , Rolando , L. A. , Aliyu , M. H. , McGown , P. W. , Connor , L. R. , Awalt , B. M. et al. . ( 2016 ) Modifiable healthy lifestyle behaviors: 10-year health outcomes from a health promotion program . American Journal of Preventive Medicine , 51 , 1027 – 1037 .

Cantera , C. M. , Puigdomènech , E. , Ballvé , J. L. , Arias , O. L. , Clemente , L. , Casas , R. et al. . ( 2015 ) Effectiveness of multicomponent interventions in primary healthcare settings to promote continuous smoking cessation in adults: a systematic review . BMJ Open , 5 , e008807 .

Cross , R. , Foster , S. , O’Neil , I. , Rowlands , S. , Warwick-Booth , L. and Woodall , J . ( 2020 ) Health Promotion: Global Principles and Practice . CABI , Wallingford .

Google Preview

Dalager , T. , Justesen , J. B. , Murray , M. , Boyle , E. and Sjøgaard , G. ( 2016 ) Implementing intelligent physical exercise training at the workplace: health effects among office workers—a randomized controlled trial . European Journal of Applied Physiology , 116 , 1433 – 1442 .

Doumas , D. M. and Hannah , E. ( 2008 ) Preventing high-risk drinking in youth in the workplace: a web-based normative feedback program . Journal of Substance Abuse Treatment , 34 , 263 – 271 .

Eng , J. Y. , Moy , F. M. and Bulgiba , A. ( 2016 ) Impact of a workplace health promotion program on employees’ blood pressure in a public university . PLoS One , 11 , e0148307 .

Galaviz , K. I. , Weber , M. B. , Straus , A. , Haw , J. S. , Narayan , K. V. and Ali , M. K. ( 2018 ) Global diabetes prevention interventions: a systematic review and network meta-analysis of the real-world impact on incidence, weight, and glucose . Diabetes Care , 41 , 1526 – 1534 .

Gibson , I. , Flaherty , G. , Cormican , S. , Jones , J. , Kerins , C. , Walsh , A. M. et al. . ( 2014 ) Translating guidelines to practice: findings from a multidisciplinary preventive cardiology programme in the west of Ireland . European Journal of Preventive Cardiology , 21 , 366 – 376 .

Gram , B. , Holtermann , A. , Søgaard , K. and Sjøgaard , G. ( 2012 ) Effect of individualized worksite exercise training on aerobic capacity and muscle strength among construction workers—a randomized controlled intervention study . Scandinavian Journal of Work Environment & Health , 38 , 467 – 475 .

Green , J. , Cross , R. , Woodall , J. and Tones , K. ( 2019 ) Health Promotion: Planning & Strategies , 4th edition. Sage , London .

Grunfeld , E. , Manca , D. , Moineddin , R. , Thorpe , K. E. , Hoch , J. S. , Campbell-Scherer , D. et al. ; BETTER Trial Investigators . ( 2013 ) Improving chronic disease prevention and screening in primary care: results of the BETTER pragmatic cluster randomized controlled trial . BMC Family Practice , 14 , 175 .

Hardcastle , S. , Blake , N. and Hagger , M. S. ( 2012 ) The effectiveness of a motivational interviewing primary-care based intervention on physical activity and predictors of change in a disadvantaged community . Journal of Behavioral Medicine , 35 , 318 – 333 .

Hjarnoe , L. and Leppin , A. ( 2013 ) Health promotion in the Danish maritime setting: challenges and possibilities for changing lifestyle behavior and health among seafarers . BMC Public Health , 13 , 1 – 12 .

Jeong , S. W. , Kim , S. H. , Kang , S. H. , Kim , H. J. , Yoon , C. H. , Youn , T. J. et al. . ( 2019 ) Mortality reduction with physical activity in patients with and without cardiovascular disease . European Heart Journal , 40 , 3547 – 3555 .

Journath , G. , Hammar , N. , Vikström , M. , Linnersjö , A. , Walldius , G. , Krakau , I. et al. . ( 2020 ) A Swedish primary healthcare prevention programme focusing on promotion of physical activity and a healthy lifestyle reduced cardiovascular events and mortality: 22-year follow-up of 5761 study participants and a reference group . British Journal of Sports Medicine , 54 , 1294 – 1299 .

Khadjesari , Z. , Freemantle , N. , Linke , S. , Hunter , R. and Murray , E. ( 2014 ) Health on the web: randomised controlled trial of online screening and brief alcohol intervention delivered in a workplace setting . PLoS One , 9 , e112553 .

Kirkman , J. J. L. , Leo , B. and Moore , J. C. ( 2018 ) Alcohol consumption reduction among a web-based supportive community using the hello Sunday morning blog platform: observational study . Journal of Medical Internet Research , 20 , e196 .

Kosendiak , A. , Felińczak , A. and Szymańska-Chabowska , A. ( 2021 ) The role of physical training in the prevention of cardiovascular disease in a population of healthy people . The Journal of Sports Medicine and Physical Fitness , 61 , 844 – 850 .

Lidin , M. , Hellénius , M. L. , Rydell-Karlsson , M. and Ekblom-Bak , E. ( 2018 ) Long-term effects on cardiovascular risk of a structured multidisciplinary lifestyle program in clinical practice . BMC Cardiovascular Disorders , 18 , 59 .

Lingfors , H. and Persson , L. -G. ( 2019 ) All-cause mortality among young men 24–26 years after a lifestyle health dialogue in a Swedish primary care setting: a longitudinal follow-up register study . BMJ Open , 9 , e022474 .

Lingfors , H. , Persson , L. G. , Lindström , K. , Bengtsson , C. and Lissner , L. ( 2009 ) Effects of a global health and risk assessment tool for prevention of ischemic heart disease in an individual health dialogue compared with a community health strategy only results from the Live for Life health promotion programme . Preventive Medicine , 48 , 20 – 24 .

Matano , R. A. , Koopman , C. , Wanat , S. F. , Winzelberg , A. J. , Whitsell , S. D. , Westrup , D. et al. . ( 2007 ) A pilot study of an interactive web site in the workplace for reducing alcohol consumption . Journal of Substance Abuse Treatment , 32 , 71 – 80 .

Matzer , F. , Nagele , E. , Lerch , N. , Vajda , C. and Fazekas , C. ( 2018 ) Combining walking and relaxation for stress reduction—a randomized cross-over trial in healthy adults . Stress and Health , 34 , 266 – 277 .

McEachan , R. R. C. , Lawton , R. J. , Jackson , C. , Conner , M. , Meads , D. M. and West , R. M. ( 2011 ) Testing a workplace physical activity intervention: a cluster randomized controlled trial . International Journal of Behavioral Nutrition and Physical Activity , 8 , 29 .

Neuner-Jehle , S. , Schmid , M. and Grüninger , U. ( 2013 ) The ‘Health Coaching’ programme: a new patient-centred and visually supported approach for health behaviour change in primary care . BMC Family Practice , 14 , 100 .

Nguyen , Q. N. , Pham , S. T. , Nguyen , V. L. , Weinehall , L. , Wall , S. , Bonita , R. et al. . ( 2012 ) Effectiveness of community-based comprehensive healthy lifestyle promotion on cardiovascular disease risk factors in a rural Vietnamese population: a quasi-experimental study . BMC Cardiovascular Disorders , 12 , 56 .

Nutbeam , D. and Muscat , D. M. ( 2021 ) Health promotion glossary 2021 . Health Promotion International , 36 , 1578 – 1598 .

Oude Hengel , K. M. , Bosmans , J. E. , Van Dongen , J. M. , Bongers , P. M. , Van der Beek , A. J. and Blatter , B. M. ( 2014 ) Prevention program at construction worksites aimed at improving health and work ability is cost-saving to the employer: results from an RCT . American Journal of Industrial Medicine , 57 , 56 – 68 .

Pemberton , M. R. , Williams , J. , Herman-Stahl , M. , Calvin , S. L. , Bradshaw , M. R. , Bray , R. M. et al. . ( 2011 ) Evaluation of two web-based alcohol interventions in the U.S. military . Journal of Studies on Alcohol and Drugs , 72 , 480 – 489 .

Persson , L. G. , Lingfors , H. , Nilsson , M. and Mölstad , S. ( 2015 ) The possibility of lifestyle and biological risk markers to predict morbidity and mortality in a cohort of young men after 26 years follow-up . BMJ Open , 5 , e006798 .

Recio-Rodriguez , J. I. , Agudo-Conde , C. , Martin-Cantera , C. , González-Viejo , M. N. , Fernandez-Alonso , M. D. C. , Arietaleanizbeaskoa , M. S. et al. ; EVIDENT Investigators . ( 2016 ) Short-term effectiveness of a mobile phone app for increasing physical activity and adherence to the Mediterranean diet in primary care: a randomized controlled trial (EVIDENT II study) . Journal of Medical Internet Research , 18 , e331 .

Richardson , G. , van Woerden , H. C. , Morgan , L. , Edwards , R. , Harries , M. , Hancock , E. et al. . ( 2008 ) Healthy hearts—a community-based primary prevention programme to reduce coronary heart disease . BMC Cardiovascular Disorders , 8 , 18 .

Robroek , S. J. , Brouwer , W. , Lindeboom , D. , Oenema , A. and Burdorf , A. ( 2010 ) Demographic, behavioral, and psychosocial correlates of using the website component of a worksite physical activity and healthy nutrition promotion program: a longitudinal study . Journal of Medical Internet Research , 12 , e44 .

Rose , G. ( 1981 ) Strategy of prevention: lessons from cardiovascular disease . British Medical Journal , 282 , 1847 – 1851 .

Rose , G. ( 2001 ) Sick individuals and sick populations . International Journal of Epidemiology , 30 , 427 – 432; discussion 433 .

Stange , K. C. , Miller , W. L. and Etz , R. S. ( 2023 ) The role of primary care in improving population health . The Milbank Quarterly , 101 , 795 – 840 .

Tricco , A. C. , Lillie , E. , Zarin , W. , O’Brien , K. K. , Colquhoun , H. , Levac , D. et al. . ( 2018 ) PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation . Annals of Internal Medicine , 169 , 467 – 473 .

van Doorn , D. , Richardson , N. , Osborne , A. and Blake , C. ( 2019 ) The impact of a workplace cardiovascular health screening programme ‘Farmers have hearts’ on health behaviour change among Irish farmers . Work , 63 , 113 – 123 .

Viester , L. , Verhagen , E. A. , Bongers , P. M. and van der Beek , A. J. ( 2015 ) The effect of a health promotion intervention for construction workers on work-related outcomes: results from a randomized controlled trial . International Archives of Occupational and Environmental Health , 88 , 789 – 798 .

Watson , H. , Godfrey , C. , McFadyen , A. , McArthur , K. , Stevenson , M. and Holloway , A. ( 2015 ) Screening and brief intervention delivery in the workplace to reduce alcohol-related harm: a pilot randomized controlled trial . International Journal of Nursing Studies , 52 , 39 – 48 .

Wendel-Vos , G. C. , Dutman , A. E. , Verschuren , W. M. , Ronckers , E. T. , Ament , A. , van Assema , P. et al. . ( 2009 ) Lifestyle factors of a five-year community-intervention program: the Hartslag Limburg intervention . American Journal of Preventive Medicine , 37 , 50 – 56 .

Yang , S. and Kim , H. ( 2022 ) Effects of a walking exercise-focused health promotion program for middle-aged women in the Korean community . International Journal of Environmental Research and Public Health , 19 , 14947 .

Zabaleta-Del-Olmo , E. , Casajuana-Closas , M. , López-Jiménez , T. , Pombo , H. , Pons-Vigués , M. , Pujol-Ribera , E. et al. . ( 2021 ) Multiple health behaviour change primary care intervention for smoking cessation, physical activity and healthy diet in adults 45 to 75 years old (EIRA study): a hybrid effectiveness-implementation cluster randomised trial . BMC Public Health , 21 , 2208 .

Supplementary data

Email alerts, citing articles via.

• Recommend to Your Librarian
• Journals Career Network

Affiliations

• Online ISSN 1460-2245
• Print ISSN 0957-4824
• Copyright © 2024 Oxford University Press
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

• Systematic Review
• Open access
• Published: 23 May 2024

Systematic literature review of real-world evidence for treatments in HR+/HER2- second-line LABC/mBC after first-line treatment with CDK4/6i

• Veronique Lambert   ORCID: orcid.org/0000-0002-6984-0038 1 ,
• Sarah Kane   ORCID: orcid.org/0009-0006-9341-4836 2   na1 ,
• Belal Howidi   ORCID: orcid.org/0000-0002-1166-7631 2   na1 ,
• Bao-Ngoc Nguyen   ORCID: orcid.org/0000-0001-6026-2270 2   na1 ,
• David Chandiwana   ORCID: orcid.org/0009-0002-3499-2565 3 ,
• Yan Wu   ORCID: orcid.org/0009-0008-3348-9232 1 ,
• Michelle Edwards   ORCID: orcid.org/0009-0001-4292-3140 3 &
• Imtiaz A. Samjoo   ORCID: orcid.org/0000-0003-1415-8055 2   na1  

BMC Cancer volume  24 , Article number:  631 ( 2024 ) Cite this article

1 Altmetric

Metrics details

Cyclin-dependent kinase 4 and 6 inhibitors (CDK4/6i) combined with endocrine therapy (ET) are currently recommended by the National Comprehensive Cancer Network (NCCN) guidelines and the European Society for Medical Oncology (ESMO) guidelines as the first-line (1 L) treatment for patients with hormone receptor-positive, human epidermal growth factor receptor 2-negative, locally advanced/metastatic breast cancer (HR+/HER2- LABC/mBC). Although there are many treatment options, there is no clear standard of care for patients following 1 L CDK4/6i. Understanding the real-world effectiveness of subsequent therapies may help to identify an unmet need in this patient population. This systematic literature review qualitatively synthesized effectiveness and safety outcomes for treatments received in the real-world setting after 1 L CDK4/6i therapy in patients with HR+/ HER2- LABC/mBC.

MEDLINE®, Embase, and Cochrane were searched using the Ovid® platform for real-world evidence studies published between 2015 and 2022. Grey literature was searched to identify relevant conference abstracts published from 2019 to 2022. The review was conducted in accordance with PRISMA guidelines (PROSPERO registration: CRD42023383914). Data were qualitatively synthesized and weighted average median real-world progression-free survival (rwPFS) was calculated for NCCN/ESMO-recommended post-1 L CDK4/6i treatment regimens.

Twenty records (9 full-text articles and 11 conference abstracts) encompassing 18 unique studies met the eligibility criteria and reported outcomes for second-line (2 L) treatments after 1 L CDK4/6i; no studies reported disaggregated outcomes in the third-line setting or beyond. Sixteen studies included NCCN/ESMO guideline-recommended treatments with the majority evaluating endocrine-based therapy; five studies on single-agent ET, six studies on mammalian target of rapamycin inhibitors (mTORi) ± ET, and three studies with a mix of ET and/or mTORi. Chemotherapy outcomes were reported in 11 studies. The most assessed outcome was median rwPFS; the weighted average median rwPFS was calculated as 3.9 months (3.3-6.0 months) for single-agent ET, 3.6 months (2.5–4.9 months) for mTORi ± ET, 3.7 months for a mix of ET and/or mTORi (3.0–4.0 months), and 6.1 months (3.7–9.7 months) for chemotherapy. Very few studies reported other effectiveness outcomes and only two studies reported safety outcomes. Most studies had heterogeneity in patient- and disease-related characteristics.

Conclusions

The real-world effectiveness of current 2 L treatments post-1 L CDK4/6i are suboptimal, highlighting an unmet need for this patient population.

Peer Review reports

Introduction

Breast cancer (BC) is the most diagnosed form of cancer in women with an estimated 2.3 million new cases diagnosed worldwide each year [ 1 ]. BC is the second leading cause of cancer death, accounting for 685,000 deaths worldwide per year [ 2 ]. By 2040, the global burden associated with BC is expected to surpass three million new cases and one million deaths annually (due to population growth and aging) [ 3 ]. Numerous factors contribute to global disparities in BC-related mortality rates, including delayed diagnosis, resulting in a high number of BC cases that have progressed to locally advanced BC (LABC) or metastatic BC (mBC) [ 4 , 5 , 6 ]. In the United States (US), the five-year survival rate for patients who progress to mBC is three times lower (31%) than the overall five-year survival rate for all stages (91%) [ 6 , 7 ].

Hormone receptor (HR) positive (i.e., estrogen receptor and/or progesterone receptor positive) coupled with negative human epidermal growth factor 2 (HER2) expression is the most common subtype of BC, accounting for ∼ 60–70% of all BC cases [ 8 , 9 ]. Historically, endocrine therapy (ET) through estrogen receptor modulation and/or estrogen deprivation has been the standard of care for first-line (1 L) treatment of HR-positive/HER2-negative (HR+/HER2-) mBC [ 10 ]. However, with the approval of the cyclin-dependent kinase 4/6 inhibitor (CDK4/6i) palbociclib in combination with the aromatase inhibitor (AI) letrozole in 2015 by the US Food and Drug Administration (FDA), 1 L treatment practice patterns have evolved such that CDK4/6i (either in combination with AIs or with fulvestrant) are currently considered the standard of care [ 11 , 12 , 13 , 14 , 15 , 16 , 17 ]. Other CDK4/6i (ribociclib and abemaciclib) in combination with ET are approved for the treatment of HR+/HER2- LABC/mBC; 1 L use of ribociclib in combination with an AI was granted FDA approval in March 2017 for postmenopausal women (with expanded approval in July 2018 for pre/perimenopausal women and for use in 1 L with fulvestrant for patients with disease progression on ET as well as for postmenopausal women), and abemaciclib in combination with fulvestrant was granted FDA approval in September 2017 for patients with disease progression following ET and as monotherapy in cases where disease progression occurs following ET and prior chemotherapy in mBC (with expanded approval in February 2018 for use in 1 L in combination with an AI for postmenopausal women) [ 18 , 19 , 20 , 21 ].

Clinical trials investigating the addition of CDK4/6i to ET have demonstrated significant improvement in progression-free survival (PFS) and significant (ribociclib) or numerical (palbociclib and abemaciclib) improvement in overall survival (OS) compared to ET alone in patients with HR+/HER2- advanced or mBC, making this combination treatment the recommended option in the 1 L setting [ 22 , 23 , 24 , 25 , 26 , 27 ]. However, disease progression occurs in a significant portion of patients after 1 L CDK4/6i treatment [ 28 ] and the optimal treatment sequence after progression on CDK4/6i remains unclear [ 29 ]. At the time of this review (literature search conducted December 14, 2022), guidelines by the National Comprehensive Cancer Network (NCCN) and the European Society for Medical Oncology (ESMO) recommend various options for the treatment of HR+/HER2- advanced BC in the second-line (2 L) setting, including fulvestrant monotherapy, mammalian target of rapamycin inhibitors (mTORi; e.g., everolimus) ± ET, alpelisib + fulvestrant (if phosphatidylinositol-4,5-bisphosphate 3-kinase catalytic subunit alpha mutation positive [PIK3CA-m+]), poly-ADP ribose polymerase inhibitors (PARPi) including olaparib or talazoparib (if breast cancer gene/partner and localizer of BRCA2 positive [BRCA/PALB2m+]), and chemotherapy (in cases when a visceral crisis is present) [ 15 , 16 ]. CDK4/6i can also be used in 2 L [ 16 , 30 ]; however, limited data are available to support CDK4/6i rechallenge after its use in the 1 L setting [ 15 ]. Depending on treatments used in the 1 L and 2 L settings, treatment in the third-line setting is individualized based on the patient’s response to prior treatments, tumor load, duration of response, and patient preference [ 9 , 15 ]. Understanding subsequent treatments after 1 L CDK4/6i, and their associated effectiveness, is an important focus in BC research.

Treatment options for HR+/HER2- LABC/mBC continue to evolve, with ongoing research in both clinical trials and in the real-world setting. Real-world evidence (RWE) offers important insights into novel therapeutic regimens and the effectiveness of treatments for HR+/HER2- LABC/mBC. The effectiveness of the current treatment options following 1 L CDK4/6i therapy in the real-world setting highlights the unmet need in this patient population and may help to drive further research and drug development. In this study, we conducted a systematic literature review (SLR) to qualitatively summarize the effectiveness and safety of treatment regimens in the real-world setting after 1 L treatment with CDK4/6i in patients with HR+/HER2- LABC/mBC.

Literature search

An SLR was performed in accordance with the Cochrane Handbook for Systematic Reviews of Interventions [ 31 ] and reported in alignment with the Preferred Reporting Items for Systematic Literature Reviews and Meta-Analyses (PRISMA) statement [ 32 ] to identify all RWE studies assessing the effectiveness and safety of treatments used for patients with HR+/HER2- LABC/mBC following 1 L CDK4/6i therapy and received subsequent treatment in 2 L and beyond (2 L+). The Ovid® platform was used to search MEDLINE® (including Epub Ahead of Print and In-Process, In-Data-Review & Other Non-Indexed Citations), Ovid MEDLINE® Daily, Embase, Cochrane Central Register of Controlled Trials, and Cochrane Database of Systematic Reviews by an experienced medical information specialist. The MEDLINE® search strategy was peer-reviewed independently by a senior medical information specialist before execution using the Peer Review of Electronic Search Strategies (PRESS) checklist [ 33 ]. Searches were conducted on December 14, 2022. The review protocol was developed a priori and registered with the International Prospective Register of Systematic Review (PROSPERO; CRD42023383914) which outlined the population, intervention, comparator, outcome, and study design (PICOS) criteria and methodology used to conduct the review (Table  1 ).

Search strategies utilized a combination of controlled vocabulary (e.g., “HER2 Breast Cancer” or “HR Breast Cancer”) and keywords (e.g., “Retrospective studies”). Vocabulary and syntax were adjusted across databases. Published and validated filters were used to select for study design and were supplemented using additional medical subject headings (MeSH) terms and keywords to select for RWE and nonrandomized studies [ 34 ]. No language restrictions were included in the search strategy. Animal-only and opinion pieces were removed from the results. The search was limited to studies published between January 2015 and December 2022 to reflect the time at which FDA approval was granted for the first CDK4/6i agent (palbociclib) in combination with AI for the treatment of LABC/mBC [ 35 ]. Further search details are presented in Supplementary Material 1 .

Grey literature sources were also searched to identify relevant abstracts and posters published from January 2019 to December 2022 for prespecified relevant conferences including ESMO, San Antonio Breast Cancer Symposium (SABCS), American Society of Clinical Oncology (ASCO), the International Society for Pharmacoeconomics and Outcomes Research (ISPOR US), and the American Association for Cancer Research (AACR). A search of ClinicalTrials.gov was conducted to validate the findings from the database and grey literature searches.

Study selection, data extraction & weighted average calculation

Studies were screened for inclusion using DistillerSR Version 2.35 and 2.41 (DistillerSR Inc. 2021, Ottawa, Canada) by two independent reviewers based on the prespecified PICOS criteria (Table  1 ). A third reviewer was consulted to resolve any discrepancies during the screening process. Studies were included if they reported RWE on patients aged ≥ 18 years with HR+/HER2- LABC/mBC who received 1 L CDK4/6i treatment and received subsequent treatment in 2 L+. Studies were excluded if they reported the results of clinical trials (i.e., non-RWE), were published in any language other than English, and/or were published prior to 2015 (or prior to 2019 for conference abstracts and posters). For studies that met the eligibility criteria, data relating to study design and methodology, details of interventions, patient eligibility criteria and baseline characteristics, and outcome measures such as efficacy, safety, tolerability, and patient-reported outcomes (PROs), were extracted (as available) using a Microsoft Excel®-based data extraction form (Microsoft Corporation, WA, USA). Data extraction was performed by a single reviewer and was confirmed by a second reviewer. Multiple publications identified for the same RWE study, patient population, and setting that reported data for the same intervention were linked and extracted as a single publication. Weighted average median real-world progression-free survival (rwPFS) values were calculated by considering the contribution to the median rwPFS of each study proportional to its respective sample size. These weighted values were then used to compute the overall median rwPFS estimate.

Quality assessment

The Newcastle-Ottawa scale (NOS) for nonrandomized (cohort) studies was used to assess the risk of bias for published, full-text studies [ 36 ]. The NOS allocates a maximum of nine points for the least risk of bias across three domains: (1) Formation of study groups (four points), (2) Comparability between study groups (two points), (3) Outcome ascertainment (three points). NOS scores can be categorized in three groups: very high risk of bias (0 to 3 points), high risk of bias (4 to 6), and low risk of bias (7 to 9) [ 37 ]. Risk of bias assessment was performed by one reviewer and validated by a second independent reviewer to verify accuracy. Due to limited methodological data by which to assess study quality, risk of bias assessment was not performed on conference abstracts or posters. An amendment to the PROSPERO record (CRD42023383914) for this study was submitted in relation to the quality assessment method (specifying usage of the NOS).

The database search identified 3,377 records; after removal of duplicates, 2,759 were screened at the title and abstract stage of which 2,553 were excluded. Out of the 206 reports retrieved and assessed for eligibility, an additional 187 records were excluded after full-text review; most of these studies were excluded for having patients with mixed lines of CDK4/6i treatment (i.e., did not receive CDK4/6i exclusively in 1 L) (Fig.  1 and Table S1 ). The grey literature search identified 753 records which were assessed for eligibility; of which 752 were excluded mainly due to the population not meeting the eligibility criteria (Fig.  1 ). In total, the literature searches identified 20 records (9 published full-text articles and 11 conference abstracts/posters) representing 18 unique RWE studies that met the inclusion criteria. The NOS quality scores for the included full-text articles are provided in Table S2 . The scores ranged from four to six points (out of a total score of nine) and the median score was five, indicating that all the studies suffered from a high risk of bias [ 37 ].

Most studies were retrospective analyses of chart reviews or medical registries, and all studies were published between 2017 and 2022 (Table S3 ). Nearly half of the RWE studies (8 out of 18 studies) were conducted in the US [ 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 ], while the remaining studies included sites in Canada, China, Germany, Italy, Japan, and the United Kingdom [ 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 ]. Sample sizes ranged from as few as 4 to as many as 839 patients across included studies, with patient age ranging from 26 to 86 years old.

Although treatment characteristics in the 1 L setting were not the focus of the present review, these details are captured in Table S3 . Briefly, several RWE studies reported 1 L CDK4/6i use in combination with ET (8 out of 18 studies) or as monotherapy (2 out of 18 studies) (Table S3 ). Treatments used in combination with 1 L CDK4/6i included letrozole, fulvestrant, exemestane, and anastrozole. Where reported (4 out of 18 studies), palbociclib was the most common 1 L CDK4/6i treatment. Many studies (8 out of 18 studies) did not report which specific CDK4/6i treatment(s) were used in 1 L or if its administration was in combination or monotherapy.

Characteristics of treatments after 1 L CDK4/6i therapy

Across all studies included in this review, effectiveness and safety data were only available for treatments administered in the 2 L setting after 1 L CDK4/6i treatment. No studies were identified that reported outcomes for patients treated in the third-line setting or beyond after 1 L CDK4/6i treatment. All 18 studies reported effectiveness outcomes in 2 L, with only two of these studies also describing 2 L safety outcomes. The distribution of outcomes reported in these studies is provided in Table S4 . Studies varied in their reporting of outcomes for 2 L treatments; some studies reported outcomes for a group of 2 L treatments while others described independent outcomes for specific 2 L treatments (i.e., everolimus, fulvestrant, or chemotherapy agents such as eribulin mesylate) [ 42 , 45 , 50 , 54 , 55 ]. Due to the heterogeneity in treatment classes reported in these studies, this data was categorized (as described below) to align with the guidelines provided by NCCN and ESMO [ 15 , 16 ]. The treatment class categorizations for the purpose of this review are: single-agent ET (patients who exclusively received a single-agent ET after 1 L CDK4/6i treatment), mTORi ± ET (patients who exclusively received an mTORi with or without ET after 1 L CDK4/6i treatment), mix of ET and/or mTORi (patients who may have received only ET, only mTORi, and/or both treatments but the studies in this group lacked sufficient information to categorize these patients in the “single-agent ET” or “mTOR ± ET” categories), and chemotherapy (patients who exclusively received chemotherapy after 1 L CDK4/6i treatment). Despite ESMO and NCCN guidelines indicating that limited evidence exists to support rechallenge with CDK4/6i after 1 L CDK4/6i treatment [ 15 , 16 ], two studies reported outcomes for this treatment approach. Data for such patients were categorized as “ CDK4/6i ± ET ” as it was unclear how many patients receiving CDK4/6i rechallenge received concurrent ET. All other patient groups that lacked sufficient information or did not report outcome/safety data independently (i.e., grouped patients with mixed treatments) to categorize as one of the treatment classes described above were grouped as “ other ”.

The majority of studies reported effectiveness outcomes for endocrine-based therapy after 1 L CDK4/6i treatment; five studies for single-agent ET, six studies for mTORi ± ET, and three studies for a mix of ET and/or mTORi (Fig.  2 ). Eleven studies reported effectiveness outcomes for chemotherapy after 1 L CDK4/6i treatment, and only two studies reported effectiveness outcomes for CDK4/6i rechallenge ± ET. Eight studies that described effectiveness outcomes were grouped into the “other” category. Safety data was only reported in two studies: one study evaluating the chemotherapy agent eribulin mesylate and one evaluating the mTORi everolimus.

Effectiveness outcomes

Real-world progression-free survival

Median rwPFS was described in 13 studies (Tables  2 and Table S5 ). Across the 13 studies, the median rwPFS ranged from 2.5 months [ 49 ] to 17.3 months [ 39 ]. Out of the 13 studies reporting median rwPFS, 10 studies reported median rwPFS for a 2 L treatment recommended by ESMO and NCCN guidelines, which ranged from 2.5 months [ 49 ] to 9.7 months [ 45 ].

Weighted average median rwPFS was calculated for 2 L treatments recommended by both ESMO and NCCN guidelines (Fig.  3 ). The weighted average median rwPFS for single-agent ET was 3.9 months ( n  = 92 total patients) and was derived using data from two studies reporting median rwPFS values of 3.3 months ( n  = 70) [ 38 ] and 6.0 months ( n  = 22) [ 40 ]. For one study ( n  = 7) that reported outcomes for single agent ET, median rwPFS was not reached during the follow-up period; as such, this study was excluded from the weighted average median rwPFS calculation [ 49 ].

The weighted average median rwPFS for mTORi ± ET was 3.6 months ( n  = 128 total patients) and was derived based on data from 3 studies with median rwPFS ranging from 2.5 months ( n  = 4) [ 49 ] to 4.9 months ( n  = 25) [ 54 ] (Fig.  3 ). For patients who received a mix of ET and/or mTORi but could not be classified into the single-agent ET or mTORi ± ET treatment classes, the weighted average median rwPFS was calculated to be 3.7 months ( n  = 17 total patients). This was calculated based on data from two studies reporting median rwPFS values of 3.0 months ( n  = 5) [ 46 ] and 4.0 months ( n  = 12) [ 49 ]. Notably, one study of patients receiving ET and/or everolimus reported a median rwPFS duration of 3.0 months; however, this study was excluded from the weighted average median rwPFS calculation for the ET and/or mTORi class as the sample size was not reported [ 53 ].

The weighted average median rwPFS for chemotherapy was 6.1 months ( n  = 499 total patients), calculated using data from 7 studies reporting median rwPFS values ranging from 3.7 months ( n  = 249) [ 38 ] to 9.7 months ( n  = 121) [ 45 ] (Fig.  3 ). One study with a median rwPFS duration of 5.6 months was not included in the weighted average median rwPFS calculation as the study did not report the sample size [ 53 ]. A second study was excluded from the calculation since the reported median rwPFS was not reached during the study period ( n  = 7) [ 41 ].

Although 2 L CDK4/6i ± ET rechallenge lacks sufficient information to support recommendation by ESMO and NCCN guidelines, the limited data currently available for this treatment have shown promising results. Briefly, two studies reported median rwPFS for CDK4/6i ± ET with values of 8.3 months ( n  = 302) [ 38 ] and 17.3 months ( n  = 165) (Table  2 ) [ 39 ]. The remaining median rwPFS studies reported data for patients classified as “Other” (Table S5 ). The “Other” category included median rwPFS outcomes from seven studies, and included a myriad of treatments (e.g., ET, mTOR + ET, chemotherapy, CDK4/6i + ET, alpelisib + fulvestrant, chidamide + ET) for which disaggregated median rwPFS values were not reported.

Overall survival

Median OS for 2 L treatment was reported in only three studies (Table  2 ) [ 38 , 42 , 43 ]. Across the three studies, the 2 L median OS ranged from 5.2 months ( n  = 3) [ 43 ] to 35.7 months ( n  = 302) [ 38 ]. Due to the lack of OS data in most of the studies, weighted averages could not be calculated. No median OS data was reported for the single-agent ET treatment class whereas two studies reported median OS for the mTORi ± ET treatment class, ranging from 5.2 months ( n  = 3) [ 43 ] to 21.8 months ( n  = 54) [ 42 ]. One study reported 2 L median OS of 24.8 months for a single patient treated with chemotherapy [ 43 ]. The median OS data in the CDK4/6i ± ET rechallenge group was 35.7 months ( n  = 302) [ 38 ].

Patient mortality was reported in three studies [ 43 , 44 , 45 ]. No studies reported mortality for the single-agent ET treatment class and only one study reported this outcome for the mTORi ± ET treatment class, where 100% of patients died ( n  = 3) as a result of rapid disease progression [ 43 ]. For the chemotherapy class, one study reported mortality for one patient receiving 2 L capecitabine [ 43 ]. An additional study reported eight deaths (21.7%) following 1 L CDK4/6i treatment; however, this study did not disclose the 2 L treatments administered to these patients [ 44 ].

Other clinical endpoints

The studies included limited information on additional clinical endpoints; two studies reported on time-to-discontinuation (TTD), two reported on duration of response (DOR), and one each on time-to-next-treatment (TTNT), time-to-progression (TTP), objective response rate (ORR), clinical benefit rate (CBR), and stable disease (Tables  2 and Table S5 ).

Safety, tolerability, and patient-reported outcomes

Safety and tolerability data were reported in two studies [ 40 , 45 ]. One study investigating 2 L administration of the chemotherapy agent eribulin mesylate reported 27 patients (22.3%) with neutropenia, 3 patients (2.5%) with febrile neutropenia, 10 patients (8.3%) with peripheral neuropathy, and 14 patients (11.6%) with diarrhea [ 45 ]. Of these, neutropenia of grade 3–4 severity occurred in 9 patients (33.3%) [ 45 ]. A total of 55 patients (45.5%) discontinued eribulin mesylate treatment; 1 patient (0.83%) discontinued treatment due to adverse events [ 45 ]. Another study reported that 5 out of the 22 patients receiving the mTORi everolimus combined with ET in 2 L (22.7%) discontinued treatment due to toxicity [ 40 ]. PROs were not reported in any of the studies included in the SLR.

The objective of this study was to summarize the existing RWE on the effectiveness and safety of therapies for patients with HR+/HER2- LABC/mBC after 1 L CDK4/6i treatment. We identified 18 unique studies reporting specifically on 2 L treatment regimens after 1 L CDK4/6i treatment. The weighted average median rwPFS for NCCN- and ESMO- guideline recommended 2 L treatments ranged from 3.6 to 3.9 months for ET-based treatments and was 6.1 months when including chemotherapy-based regimens. Treatment selection following 1 L CDK4/6i therapy remains challenging primarily due to the suboptimal effectiveness or significant toxicities (e.g., chemotherapy) associated with currently available options [ 56 ]. These results highlight that currently available 2 L treatments for patients with HR+/HER2- LABC/mBC who have received 1 L CDK4/6i are suboptimal, as evidenced by the brief median rwPFS duration associated with ET-based treatments, or notable side effects and toxicity linked to chemotherapy. This conclusion is aligned with a recent review highlighting the limited effectiveness of treatment options for HR+/HER2- LABC/mBC patients post-CDK4/6i treatment [ 56 , 57 ]. Registrational trials which have also shed light on the short median PFS of 2–3 months achieved by ET (i.e., fulvestrant) after 1 L CDK4/6i therapy emphasize the need to develop improved treatment strategies aimed at prolonging the duration of effective ET-based treatment [ 56 ].

The results of this review reveal a paucity of additional real-world effectiveness and safety evidence after 1 L CDK4/6i treatment in HR+/HER2- LABC/mBC. OS and DOR were only reported in two studies while other clinical endpoints (i.e., TTD, TTNT, TTP, ORR, CBR, and stable disease) were only reported in one study each. Similarly, safety and tolerability data were only reported in two studies each, and PROs were not reported in any study. This hindered our ability to provide a comprehensive assessment of real-world treatment effectiveness and safety following 1 L CDK4/6i treatment. The limited evidence may be due to the relatively short period of time that has elapsed since CDK4/6i first received US FDA approval for 1 L treatment of HR+/HER2- LABC/mBC (2015) [ 35 ]. As such, almost half of our evidence was informed by conference abstracts. Similarly, no real-world studies were identified in our review that reported outcomes for treatments in the third- or later-lines of therapy after 1 L CDK4/6i treatment. The lack of data in this patient population highlights a significant gap which limits our understanding of the effectiveness and safety for patients receiving later lines of therapy. As more patients receive CDK4/6i therapy in the 1 L setting, the number of patients requiring subsequent lines of therapy will continue to grow. Addressing this data gap over time will be critical to improve outcomes for patients with HR+/HER2- LABC/mBC following 1 L CDK4/6i therapy.

There are several strengths of this study, including adherence to the guidelines outlined in the Cochrane Handbook to ensure a standardized and reliable approach to the SLR [ 58 ] and reporting of the SLR following PRISMA guidelines to ensure transparency and reproducibility [ 59 ]. Furthermore, the inclusion of only RWE studies allowed us to assess the effectiveness of current standard of care treatments outside of a controlled environment and enabled us to identify an unmet need in this patient population.

This study had some notable limitations, including the lack of safety and additional effectiveness outcomes reported. In addition, the dearth of studies reporting PROs is a limitation, as PROs provide valuable insight into the patient experience and are an important aspect of assessing the impact of 2 L treatments on patients’ quality of life. The studies included in this review also lacked consistent reporting of clinical characteristics (e.g., menopausal status, sites of metastasis, prior surgery) making it challenging to draw comprehensive conclusions or comparisons based on these factors across the studies. Taken together, there exists an important gap in our understanding of the long-term management of patients with HR+/HER2- LABC/mBC. Additionally, the effectiveness results reported in our evidence base were informed by small sample sizes; many of the included studies reported median rwPFS based on less than 30 patients [ 39 , 40 , 41 , 46 , 49 , 51 , 60 ], with two studies not reporting the sample size at all [ 47 , 53 ]. This may impact the generalizability and robustness of the results. Relatedly, the SLR database search was conducted in December 2022; as such, novel agents (e.g., elacestrant and capivasertib + fulvestrant) that have since received FDA approval for the treatment of HR+/HER2- LABC/mBC may impact current 2 L rwPFS outcomes [ 61 , 62 ]. Finally, relative to the number of peer-reviewed full-text articles, this SLR identified eight abstracts and one poster presentation, comprising half (50%) of the included unique studies. As conference abstracts are inherently limited by how much content that can be described due to word limit constraints, this likely had implications on the present synthesis whereby we identified a dearth of real-world effectiveness outcomes in patients with HR+/HER2- LABC/mBC treated with 1 L CDK4/6i therapy.

Future research in this area should aim to address the limitations of the current literature and provide a more comprehensive understanding of optimal sequencing of effective and safe treatment for patients following 1 L CDK4/6i therapy. Specifically, future studies should strive to report robust data related to effectiveness, safety, and PROs for patients receiving 2 L treatment after 1 L CDK4/6i therapy. Future studies should also aim to understand the mechanism underlying CDK4/6i resistance. Addressing these gaps in knowledge may improve the long-term real-world management of patients with HR+/HER2- LABC/mBC. A future update of this synthesis may serve to capture a wider breadth of full-text, peer-reviewed articles to gain a more robust understanding of the safety, effectiveness, and real-world treatment patterns for patients with HR+/HER2- LABC/mBC. This SLR underscores the necessity for ongoing investigation and the development of innovative therapeutic approaches to address these gaps and improve patient outcomes.

This SLR qualitatively summarized the existing real-world effectiveness data for patients with HR+/HER2- LABC/mBC after 1 L CDK4/6i treatment. Results of this study highlight the limited available data and the suboptimal effectiveness of treatments employed in the 2 L setting and underscore the unmet need in this patient population. Additional studies reporting effectiveness and safety outcomes, in addition to PROs, for this patient population are necessary and should be the focus of future research.

PRISMA flow diagram. *Two included conference abstracts reported the same information as already included full-text reports, hence both conference abstracts were not identified as unique. Abbreviations: 1 L = first-line; AACR = American Association of Cancer Research; ASCO = American Society of Clinical Oncology; CDK4/6i = cyclin-dependent kinase 4/6 inhibitor; ESMO = European Society for Medical Oncology; ISPOR = Professional Society for Health Economics and Outcomes Research; n = number of studies; NMA = network meta-analysis; pts = participants; SABCS = San Antonio Breast Cancer Symposium; SLR = systematic literature review.

Number of studies reporting effectiveness outcomes exclusively for each treatment class. *Studies that lack sufficient information on effectiveness outcomes to classify based on the treatment classes outlined in the legend above. Abbreviations: CDK4/6i = cyclin-dependent kinase 4/6 inhibitor; ET = endocrine therapy; mTORi = mammalian target of rapamycin inhibitor.

Weighted average median rwPFS for 2 L treatments (recommended in ESMO/NCCN guidelines) after 1 L CDK4/6i treatment. Circular dot represents weighted average median across studies. Horizontal bars represent the range of values reported in these studies. Abbreviations: CDK4/6i = cyclin-dependent kinase 4/6 inhibitor; ESMO = European Society for Medical Oncology; ET = endocrine therapy, mTORi = mammalian target of rapamycin inhibitor; n = number of patients; NCCN = National Comprehensive Cancer Network; rwPFS = real-world progression-free survival.

Data availability

All data generated or analyzed during this study are included in this published article [and its supplementary information files]. This study is registered with PROSPERO (CRD42023383914).

Abbreviations

Second-line

Second-line treatment setting and beyond

American Association of Cancer Research

Aromatase inhibitor

American Society of Clinical Oncology

• Breast cancer

breast cancer gene/partner and localizer of BRCA2 positive

Clinical benefit rate

Cyclin-dependent kinase 4/6 inhibitor

Complete response

Duration of response

European Society for Medical Oncology

Food and Drug Administration

Human epidermal growth factor receptor 2

Human epidermal growth factor receptor 2 negative

Hormone receptor

Hormone receptor positive

Professional Society for Health Economics and Outcomes Research

Locally advanced breast cancer

Metastatic breast cancer

Medical Literature Analysis and Retrieval System Online

Medical subject headings

Mammalian target of rapamycin inhibitor

National Comprehensive Cancer Network

Newcastle Ottawa Scale

Objective response rate

Poly-ADP ribose polymerase inhibitor

Progression-free survival

Population, Intervention, Comparator, Outcome, Study Design

Partial response

Preferred Reporting Items for Systematic Literature Reviews and Meta-Analyses

Patient-reported outcomes

• Real-world evidence

San Antonio Breast Cancer Symposium

• Systematic literature review

Time-to-discontinuation

Time-to-next-treatment

Time-to-progression

United States

Łukasiewicz S, Czeczelewski M, Forma A, Baj J, Sitarz R, Stanisławek A, Breast, Cancer—Epidemiology. Risk factors, classification, prognostic markers, and current treatment Strategies—An. Updated Rev Cancers. 2021;13(17):4287.

Google Scholar  

World Health Organization (WHO). Breast Cancer Facts Sheet [updated July 12 2023. https://www.who.int/news-room/fact-sheets/detail/breast-cancer .

Arnold M, Morgan E, Rumgay H, Mafra A, Singh D, Laversanne M, et al. Current and future burden of breast cancer: global statistics for 2020 and 2040. Breast. 2022;66:15–23.

Article   PubMed   PubMed Central   Google Scholar  

Wilkinson L, Gathani T. Understanding breast cancer as a global health concern. Br J Radiol. 2022;95(1130):20211033.

Article   PubMed   Google Scholar  

Giaquinto AN, Sung H, Miller KD, Kramer JL, Newman LA, Minihan A et al. Breast Cancer Statistics, 2022. CA: A Cancer Journal for Clinicians. 2022;72(6):524– 41.

National Cancer Institute (NIH). Cancer Stat Facts: Female Breast Cancer [updated 2020. https://seer.cancer.gov/statfacts/html/breast.html .

American Cancer Society. Key Statistics for Breast Cancer [ https://www.cancer.org/cancer/types/breast-cancer/about/how-common-is-breast-cancer.html .

Zagami P, Carey LA. Triple negative breast cancer: pitfalls and progress. npj Breast Cancer. 2022;8(1):95.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Matutino A, Joy AA, Brezden-Masley C, Chia S, Verma S. Hormone receptor-positive, HER2-negative metastatic breast cancer: redrawing the lines. Curr Oncol. 2018;25(Suppl 1):S131–41.

Lloyd MR, Wander SA, Hamilton E, Razavi P, Bardia A. Next-generation selective estrogen receptor degraders and other novel endocrine therapies for management of metastatic hormone receptor-positive breast cancer: current and emerging role. Ther Adv Med Oncol. 2022;14:17588359221113694.

Cardoso F, Senkus E, Costa A, Papadopoulos E, Aapro M, André F, et al. 4th ESO-ESMO International Consensus guidelines for advanced breast Cancer (ABC 4)†. Ann Oncol. 2018;29(8):1634–57.

Article   CAS   PubMed   Google Scholar  

US Food Drug Administration. Palbociclib (Ibrance) 2017 [updated March 31, 2017. https://www.fda.gov/drugs/resources-information-approved-drugs/palbociclib-ibrance .

US Food Drug Administration. FDA expands ribociclib indication in HR-positive, HER2-negative advanced or metastatic breast cancer 2018 [updated July 18. 2018. https://www.fda.gov/drugs/resources-information-approved-drugs/fda-expands-ribociclib-indication-hr-positive-her2-negative-advanced-or-metastatic-breast-cancer .

US Food Drug Administration. FDA approves abemaciclib for HR positive, HER2-negative breast cancer 2017 [updated Sept 28. 2017. https://www.fda.gov/drugs/resources-information-approved-drugs/fda-approves-abemaciclib-hr-positive-her2-negative-breast-cancer .

NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines®). Breast Cancer 2022 [ https://www.nccn.org/professionals/physician_gls/pdf/breast.pdf .

Gennari A, André F, Barrios CH, Cortés J, de Azambuja E, DeMichele A, et al. ESMO Clinical Practice Guideline for the diagnosis, staging and treatment of patients with metastatic breast cancer. Ann Oncol. 2021;32(12):1475–95.

Beaver JA, Amiri-Kordestani L, Charlab R, Chen W, Palmby T, Tilley A, et al. FDA approval: Palbociclib for the Treatment of Postmenopausal Patients with estrogen Receptor-Positive, HER2-Negative metastatic breast Cancer. Clin Cancer Res. 2015;21(21):4760–6.

US Food Drug Administration. Ribociclib (Kisqali) [ https://www.fda.gov/drugs/resources-information-approved-drugs/ribociclib-kisqali#:~:text=On%20March%2013%2C%202017%2C%20the,hormone%20receptor%20(HR)%2Dpositive%2C .

US Food Drug Administration. FDA approves new treatment for certain advanced or metastatic breast cancers [ https://www.fda.gov/news-events/press-announcements/fda-approves-new-treatment-certain-advanced-or-metastatic-breast-cancers .

US Food Drug Administration. FDA expands ribociclib indication in HR-positive, HER2-negative advanced or metastatic breast cancer. 2018 [ https://www.fda.gov/drugs/resources-information-approved-drugs/fda-expands-ribociclib-indication-hr-positive-her2-negative-advanced-or-metastatic-breast-cancer .

US Food Drug Administration. FDA approves abemaciclib as initial therapy for HR-positive, HER2-negative metastatic breast cancer [ https://www.fda.gov/drugs/resources-information-approved-drugs/fda-approves-abemaciclib-initial-therapy-hr-positive-her2-negative-metastatic-breast-cancer .

Turner NC, Slamon DJ, Ro J, Bondarenko I, Im S-A, Masuda N, et al. Overall survival with Palbociclib and fulvestrant in advanced breast Cancer. N Engl J Med. 2018;379(20):1926–36.

Slamon DJ, Neven P, Chia S, Fasching PA, De Laurentiis M, Im SA, et al. Phase III randomized study of Ribociclib and Fulvestrant in hormone Receptor-Positive, human epidermal growth factor receptor 2-Negative advanced breast Cancer: MONALEESA-3. J Clin Oncol. 2018;36(24):2465–72.

Goetz MP, Toi M, Campone M, Sohn J, Paluch-Shimon S, Huober J, et al. MONARCH 3: Abemaciclib as initial therapy for advanced breast Cancer. J Clin Oncol. 2017;35(32):3638–46.

Gopalan PK, Villegas AG, Cao C, Pinder-Schenck M, Chiappori A, Hou W, et al. CDK4/6 inhibition stabilizes disease in patients with p16-null non-small cell lung cancer and is synergistic with mTOR inhibition. Oncotarget. 2018;9(100):37352–66.

Watt AC, Goel S. Cellular mechanisms underlying response and resistance to CDK4/6 inhibitors in the treatment of hormone receptor-positive breast cancer. Breast Cancer Res. 2022;24(1):17.

Goetz M. MONARCH 3: final overall survival results of abemaciclib plus a nonsteroidal aromatase inhibitor as first-line therapy for HR+, HER2- advanced breast cancer. SABCS; 2023.

Munzone E, Pagan E, Bagnardi V, Montagna E, Cancello G, Dellapasqua S, et al. Systematic review and meta-analysis of post-progression outcomes in ER+/HER2– metastatic breast cancer after CDK4/6 inhibitors within randomized clinical trials. ESMO Open. 2021;6(6):100332.

Gennari A, André F, Barrios CH, Cortés J, de Azambuja E, DeMichele A, et al. ESMO Clinical Practice Guideline for the diagnosis, staging and treatment of patients with metastatic breast cancer. Annals of Oncology. 2021;32(12):1475-95.

European Society for Medical Oncology (ESMO). ESMO ​Metastatic Breast Cancer Living Guideline: ER-positive HER2-negative​ Breast Cancer​ [updated May 2023. https://www.esmo.org/living-guidelines/esmo-metastatic-breast-cancer-living-guideline/er-positive-her2-negative-breast-cancer .

Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Welch PM VA, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.2 (updated February 2021). www.training.cochrane.org/handbook : Cochrane; 2021.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. PLoS Med. 2021;18(3):e1003583.

McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS peer review of electronic search strategies: 2015 Guideline Statement. J Clin Epidemiol. 2016;75:40–6.

Fraser C, Murray A, Burr J. Identifying observational studies of surgical interventions in MEDLINE and EMBASE. BMC Med Res Methodol. 2006;6(1):41.

US Food Drug Administration. Palbociclib (Ibrance). Silver Spring, MD: US Food and Drug Administration; 2017.

Book   Google Scholar  

GA Wells BS, D O’Connell J, Peterson V, Welch M, Losos PT. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses [ https://www.ohri.ca/programs/clinical_epidemiology/oxford.asp .

Lo CK-L, Mertz D, Loeb M. Newcastle-Ottawa Scale: comparing reviewers’ to authors’ assessments. BMC Med Res Methodol. 2014;14(1):45.

Martin JM, Handorf EA, Montero AJ, Goldstein LJ. Systemic therapies following progression on first-line CDK4/6-inhibitor treatment: analysis of real-world data. Oncologist. 2022;27(6):441–6.

Kalinsky KM, Kruse M, Smyth EN, Guimaraes CM, Gautam S, Nisbett AR et al. Abstract P1-18-37: Treatment patterns and outcomes associated with sequential and non-sequential use of CDK4 and 6i for HR+, HER2- MBC in the real world. Cancer Research. 2022;82(4_Supplement):P1-18-37-P1-18-37.

Choong GM, Liddell S, Ferre RAL, O’Sullivan CC, Ruddy KJ, Haddad TC, et al. Clinical management of metastatic hormone receptor-positive, HER2-negative breast cancer (MBC) after CDK 4/6 inhibitors: a retrospective single-institution study. Breast Cancer Res Treat. 2022;196(1):229–37.

Xi J, Oza A, Thomas S, Ademuyiwa F, Weilbaecher K, Suresh R, et al. Retrospective Analysis of Treatment Patterns and effectiveness of Palbociclib and subsequent regimens in metastatic breast Cancer. J Natl Compr Canc Netw. 2019;17(2):141–7.

Rozenblit M, Mun S, Soulos P, Adelson K, Pusztai L, Mougalian S. Patterns of treatment with everolimus exemestane in hormone receptor-positive HER2-negative metastatic breast cancer in the era of targeted therapy. Breast Cancer Res. 2021;23(1):14.

Bashour SI, Doostan I, Keyomarsi K, Valero V, Ueno NT, Brown PH, et al. Rapid breast Cancer Disease Progression following cyclin dependent kinase 4 and 6 inhibitor discontinuation. J Cancer. 2017;8(11):2004–9.

Giridhar KV, Choong GM, Leon-Ferre R, O’Sullivan CC, Ruddy K, Haddad T, et al. Abstract P6-18-09: clinical management of metastatic breast cancer (MBC) after CDK 4/6 inhibitors: a retrospective single-institution study. Cancer Res. 2019;79:P6–18.

Article   Google Scholar  

Mougalian SS, Feinberg BA, Wang E, Alexis K, Chatterjee D, Knoth RL, et al. Observational study of clinical outcomes of eribulin mesylate in metastatic breast cancer after cyclin-dependent kinase 4/6 inhibitor therapy. Future Oncol. 2019;15(34):3935–44.

Moscetti LML, Riggi L, Sperduti I, Piacentini FOC, Toss A, Barbieri E, Cortesi L, Canino FMA, Zoppoli G, Frassoldati A, Schirone A, Dominici MECF. SEQUENCE OF TREATMENTS AFTER CDK4/6 THERAPY IN ADVANCED BREAST CANCER (ABC), A GOIRC MULTICENTER RETRO/ PROSPECTIVE STUDY. PRELIMINARY RESULTS IN THE RETROSPECTIVE SERIES OF 116 PATIENTS. Tumori. 2022;108(4S):80.

Menichetti AZE, Giorgi CA, Bottosso M, Leporati R, Giarratano T, Barbieri C, Ligorio F, Mioranza E, Miglietta F, Lobefaro R, Faggioni G, Falci C, Vernaci G, Di Liso E, Girardi F, Griguolo G, Vernieri C, Guarneri V, Dieci MV. CDK 4/6 INHIBITORS FOR METASTATIC BREAST CANCER: A MULTICENTER REALWORLD STUDY. Tumori. 2022;108(4S):70.

Marschner NW, Harbeck N, Thill M, Stickeler E, Zaiss M, Nusch A, et al. 232P Second-line therapies of patients with early progression under CDK4/6-inhibitor in first-line– data from the registry platform OPAL. Annals of Oncology. 2022;33:S643-S4

Gousis C, Lowe KMH, Kapiris M. V. Angelis. Beyond First Line CDK4/6 Inhibitors (CDK4/6i) and Aromatase Inhibitors (AI) in Patients with Oestrogen Receptor Positive Metastatic Breast Cancer (ERD MBC): The Guy’s Cancer Centre Experience. Clinical Oncology2022. p. e178.

Endo Y, Yoshimura A, Sawaki M, Hattori M, Kotani H, Kataoka A, et al. Time to chemotherapy for patients with estrogen receptor-positive breast Cancer and cyclin-dependent kinase 4 and 6 inhibitor use. J Breast Cancer. 2022;25(4):296–306.

Li Y, Li W, Gong C, Zheng Y, Ouyang Q, Xie N, et al. A multicenter analysis of treatment patterns and clinical outcomes of subsequent therapies after progression on palbociclib in HR+/HER2- metastatic breast cancer. Ther Adv Med Oncol. 2021;13:17588359211022890.

Amaro CP, Batra A, Lupichuk S. First-line treatment with a cyclin-dependent kinase 4/6 inhibitor plus an aromatase inhibitor for metastatic breast Cancer in Alberta. Curr Oncol. 2021;28(3):2270–80.

Crocetti SPM, Tassone L, Marcantognini G, Bastianelli L, Della Mora A, Merloni F, Cantini L, Scortichini L, Agostinelli V, Ballatore Z, Savini A, Maccaroni E. Berardi R. What is the best therapeutic sequence for ER-Positive/HER2- Negative metastatic breast cancer in the era of CDK4/6 inhibitors? A single center experience. Tumori. 2020;106(2S).

Nichetti F, Marra A, Giorgi CA, Randon G, Scagnoli S, De Angelis C, et al. 337P Efficacy of everolimus plus exemestane in CDK 4/6 inhibitors-pretreated or naïve HR-positive/HER2-negative breast cancer patients: A secondary analysis of the EVERMET study. Annals of Oncology. 2020;31:S382

Luhn P, O’Hear C, Ton T, Sanglier T, Hsieh A, Oliveri D, et al. Abstract P4-13-08: time to treatment discontinuation of second-line fulvestrant monotherapy for HR+/HER2– metastatic breast cancer in the real-world setting. Cancer Res. 2019;79(4Supplement):P4–13.

Mittal A, Molto Valiente C, Tamimi F, Schlam I, Sammons S, Tolaney SM et al. Filling the gap after CDK4/6 inhibitors: Novel Endocrine and Biologic Treatment options for metastatic hormone receptor positive breast Cancer. Cancers (Basel). 2023;15(7).

Ashai N, Swain SM. Post-CDK 4/6 inhibitor therapy: current agents and novel targets. Cancers (Basel). 2023;15(6).

Higgins JPTTJ, Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated February 2022). www.training.cochrane.org/handbook : Cochrane; 2022.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71.

Serdar CC, Cihan M, Yücel D, Serdar MA. Sample size, power and effect size revisited: simplified and practical approaches in pre-clinical, clinical and laboratory studies. Biochem Med (Zagreb). 2021;31(1):010502.

US Food Drug Administration. FDA approves elacestrant for ER-positive, HER2-negative, ESR1-mutated advanced or metastatic breast cancer [updated January 27 2023. https://www.fda.gov/drugs/resources-information-approved-drugs/fda-approves-elacestrant-er-positive-her2-negative-esr1-mutated-advanced-or-metastatic-breast-cancer .

US Food Drug Administration. FDA approves capivasertib with fulvestrant for breast cancer [updated November 16 2023. https://www.fda.gov/drugs/resources-information-approved-drugs/fda-approves-capivasertib-fulvestrant-breast-cancer .

Download references

Acknowledgements

The authors would like to acknowledge Joanna Bielecki who developed, conducted, and documented the database searches.

This study was funded by Pfizer Inc. (New York, NY, USA) and Arvinas (New Haven, CT, USA).

Author information

Sarah Kane, Belal Howidi, Bao-Ngoc Nguyen and Imtiaz A. Samjoo contributed equally to this work.

Authors and Affiliations

Pfizer, 10017, New York, NY, USA

Veronique Lambert & Yan Wu

EVERSANA, Burlington, ON, Canada

Sarah Kane, Belal Howidi, Bao-Ngoc Nguyen & Imtiaz A. Samjoo

Arvinas, 06511, New Haven, CT, USA

David Chandiwana & Michelle Edwards

You can also search for this author in PubMed   Google Scholar

Contributions

VL, IAS, SK, BH, BN, DC, YW, and ME participated in the conception and design of the study. IAS, SK, BH and BN contributed to the literature review, data collection, analysis, and interpretation of the data. VL, IAS, SK, BH, BN, DC, YW, and ME contributed to the interpretation of the data and critically reviewed for the importance of intellectual content for the work. VL, IAS, SK, BH, BN, DC, YW, and ME were responsible for drafting or reviewing the manuscript and for providing final approval. VL, IAS, SK, BH, BN, DC, YW, and ME meet the International Committee of Medical Journal Editors (ICMJE) criteria for authorship for this article, take responsibility for the integrity of the work, and have given their approval for this version to be published.

Corresponding author

Correspondence to Imtiaz A. Samjoo .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors of this manuscript declare that the research presented was funded by Pfizer Inc. and Arvinas. While the support from Pfizer Inc. and Arvinas was instrumental in facilitating this research, the authors affirm that their interpretation of the data and the content of this manuscript were conducted independently and without bias to maintain the transparency and integrity of the research. IAS, SK, BH, and BN are employees of EVERSANA, Canada, which was a paid consultant to Pfizer in connection with the development of this manuscript.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Lambert, V., Kane, S., Howidi, B. et al. Systematic literature review of real-world evidence for treatments in HR+/HER2- second-line LABC/mBC after first-line treatment with CDK4/6i. BMC Cancer 24 , 631 (2024). https://doi.org/10.1186/s12885-024-12269-8

Download citation

Received : 26 January 2024

Accepted : 16 April 2024

Published : 23 May 2024

DOI : https://doi.org/10.1186/s12885-024-12269-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• First-line CDK4/6i

ISSN: 1471-2407

• Submission enquiries: [email protected]
• General enquiries: [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Grad Med Educ
• v.8(3); 2016 Jul

The Literature Review: A Foundation for High-Quality Medical Education Research

a  These are subscription resources. Researchers should check with their librarian to determine their access rights.

Despite a surge in published scholarship in medical education 1 and rapid growth in journals that publish educational research, manuscript acceptance rates continue to fall. 2 Failure to conduct a thorough, accurate, and up-to-date literature review identifying an important problem and placing the study in context is consistently identified as one of the top reasons for rejection. 3 , 4 The purpose of this editorial is to provide a road map and practical recommendations for planning a literature review. By understanding the goals of a literature review and following a few basic processes, authors can enhance both the quality of their educational research and the likelihood of publication in the Journal of Graduate Medical Education ( JGME ) and in other journals.

The Literature Review Defined

In medical education, no organization has articulated a formal definition of a literature review for a research paper; thus, a literature review can take a number of forms. Depending on the type of article, target journal, and specific topic, these forms will vary in methodology, rigor, and depth. Several organizations have published guidelines for conducting an intensive literature search intended for formal systematic reviews, both broadly (eg, PRISMA) 5 and within medical education, 6 and there are excellent commentaries to guide authors of systematic reviews. 7 , 8

• A literature review forms the basis for high-quality medical education research and helps maximize relevance, originality, generalizability, and impact.
• A literature review provides context, informs methodology, maximizes innovation, avoids duplicative research, and ensures that professional standards are met.
• Literature reviews take time, are iterative, and should continue throughout the research process.
• Researchers should maximize the use of human resources (librarians, colleagues), search tools (databases/search engines), and existing literature (related articles).
• Keeping organized is critical.

Such work is outside the scope of this article, which focuses on literature reviews to inform reports of original medical education research. We define such a literature review as a synthetic review and summary of what is known and unknown regarding the topic of a scholarly body of work, including the current work's place within the existing knowledge . While this type of literature review may not require the intensive search processes mandated by systematic reviews, it merits a thoughtful and rigorous approach.

Purpose and Importance of the Literature Review

An understanding of the current literature is critical for all phases of a research study. Lingard 9 recently invoked the “journal-as-conversation” metaphor as a way of understanding how one's research fits into the larger medical education conversation. As she described it: “Imagine yourself joining a conversation at a social event. After you hang about eavesdropping to get the drift of what's being said (the conversational equivalent of the literature review), you join the conversation with a contribution that signals your shared interest in the topic, your knowledge of what's already been said, and your intention.” 9

The literature review helps any researcher “join the conversation” by providing context, informing methodology, identifying innovation, minimizing duplicative research, and ensuring that professional standards are met. Understanding the current literature also promotes scholarship, as proposed by Boyer, 10 by contributing to 5 of the 6 standards by which scholarly work should be evaluated. 11 Specifically, the review helps the researcher (1) articulate clear goals, (2) show evidence of adequate preparation, (3) select appropriate methods, (4) communicate relevant results, and (5) engage in reflective critique.

Failure to conduct a high-quality literature review is associated with several problems identified in the medical education literature, including studies that are repetitive, not grounded in theory, methodologically weak, and fail to expand knowledge beyond a single setting. 12 Indeed, medical education scholars complain that many studies repeat work already published and contribute little new knowledge—a likely cause of which is failure to conduct a proper literature review. 3 , 4

Likewise, studies that lack theoretical grounding or a conceptual framework make study design and interpretation difficult. 13 When theory is used in medical education studies, it is often invoked at a superficial level. As Norman 14 noted, when theory is used appropriately, it helps articulate variables that might be linked together and why, and it allows the researcher to make hypotheses and define a study's context and scope. Ultimately, a proper literature review is a first critical step toward identifying relevant conceptual frameworks.

Another problem is that many medical education studies are methodologically weak. 12 Good research requires trained investigators who can articulate relevant research questions, operationally define variables of interest, and choose the best method for specific research questions. Conducting a proper literature review helps both novice and experienced researchers select rigorous research methodologies.

Finally, many studies in medical education are “one-offs,” that is, single studies undertaken because the opportunity presented itself locally. Such studies frequently are not oriented toward progressive knowledge building and generalization to other settings. A firm grasp of the literature can encourage a programmatic approach to research.

Approaching the Literature Review

Considering these issues, journals have a responsibility to demand from authors a thoughtful synthesis of their study's position within the field, and it is the authors' responsibility to provide such a synthesis, based on a literature review. The aforementioned purposes of the literature review mandate that the review occurs throughout all phases of a study, from conception and design, to implementation and analysis, to manuscript preparation and submission.

Planning the literature review requires understanding of journal requirements, which vary greatly by journal ( table 1 ). Authors are advised to take note of common problems with reporting results of the literature review. Table 2 lists the most common problems that we have encountered as authors, reviewers, and editors.

Sample of Journals' Author Instructions for Literature Reviews Conducted as Part of Original Research Article a

Common Problem Areas for Reporting Literature Reviews in the Context of Scholarly Articles

Locating and Organizing the Literature

Three resources may facilitate identifying relevant literature: human resources, search tools, and related literature. As the process requires time, it is important to begin searching for literature early in the process (ie, the study design phase). Identifying and understanding relevant studies will increase the likelihood of designing a relevant, adaptable, generalizable, and novel study that is based on educational or learning theory and can maximize impact.

Human Resources

A medical librarian can help translate research interests into an effective search strategy, familiarize researchers with available information resources, provide information on organizing information, and introduce strategies for keeping current with emerging research. Often, librarians are also aware of research across their institutions and may be able to connect researchers with similar interests. Reaching out to colleagues for suggestions may help researchers quickly locate resources that would not otherwise be on their radar.

During this process, researchers will likely identify other researchers writing on aspects of their topic. Researchers should consider searching for the publications of these relevant researchers (see table 3 for search strategies). Additionally, institutional websites may include curriculum vitae of such relevant faculty with access to their entire publication record, including difficult to locate publications, such as book chapters, dissertations, and technical reports.

Strategies for Finding Related Researcher Publications in Databases and Search Engines

Search Tools and Related Literature

Researchers will locate the majority of needed information using databases and search engines. Excellent resources are available to guide researchers in the mechanics of literature searches. 15 , 16

Because medical education research draws on a variety of disciplines, researchers should include search tools with coverage beyond medicine (eg, psychology, nursing, education, and anthropology) and that cover several publication types, such as reports, standards, conference abstracts, and book chapters (see the box for several information resources). Many search tools include options for viewing citations of selected articles. Examining cited references provides additional articles for review and a sense of the influence of the selected article on its field.

Box Information Resources

• Web of Science a
• Education Resource Information Center (ERIC)
• Cumulative Index of Nursing & Allied Health (CINAHL) a
• Google Scholar

Once relevant articles are located, it is useful to mine those articles for additional citations. One strategy is to examine references of key articles, especially review articles, for relevant citations.

Getting Organized

As the aforementioned resources will likely provide a tremendous amount of information, organization is crucial. Researchers should determine which details are most important to their study (eg, participants, setting, methods, and outcomes) and generate a strategy for keeping those details organized and accessible. Increasingly, researchers utilize digital tools, such as Evernote, to capture such information, which enables accessibility across digital workspaces and search capabilities. Use of citation managers can also be helpful as they store citations and, in some cases, can generate bibliographies ( table 4 ).

Citation Managers

Knowing When to Say When

Researchers often ask how to know when they have located enough citations. Unfortunately, there is no magic or ideal number of citations to collect. One strategy for checking coverage of the literature is to inspect references of relevant articles. As researchers review references they will start noticing a repetition of the same articles with few new articles appearing. This can indicate that the researcher has covered the literature base on a particular topic.

Putting It All Together

In preparing to write a research paper, it is important to consider which citations to include and how they will inform the introduction and discussion sections. The “Instructions to Authors” for the targeted journal will often provide guidance on structuring the literature review (or introduction) and the number of total citations permitted for each article category. Reviewing articles of similar type published in the targeted journal can also provide guidance regarding structure and average lengths of the introduction and discussion sections.

When selecting references for the introduction consider those that illustrate core background theoretical and methodological concepts, as well as recent relevant studies. The introduction should be brief and present references not as a laundry list or narrative of available literature, but rather as a synthesized summary to provide context for the current study and to identify the gap in the literature that the study intends to fill. For the discussion, citations should be thoughtfully selected to compare and contrast the present study's findings with the current literature and to indicate how the present study moves the field forward.

To facilitate writing a literature review, journals are increasingly providing helpful features to guide authors. For example, the resources available through JGME include several articles on writing. 17 The journal Perspectives on Medical Education recently launched “The Writer's Craft,” which is intended to help medical educators improve their writing. Additionally, many institutions have writing centers that provide web-based materials on writing a literature review, and some even have writing coaches.

The literature review is a vital part of medical education research and should occur throughout the research process to help researchers design a strong study and effectively communicate study results and importance. To achieve these goals, researchers are advised to plan and execute the literature review carefully. The guidance in this editorial provides considerations and recommendations that may improve the quality of literature reviews.

Comprehensive analysis of digital twins in smart cities: a 4200-paper bibliometric study

• Open access
• Published: 27 May 2024
• Volume 57 , article number  154 , ( 2024 )

Cite this article

You have full access to this open access article

• Rasha F. El-Agamy   ORCID: orcid.org/0009-0005-0519-3870 1 , 2 ,
• Hanaa A. Sayed   ORCID: orcid.org/0000-0003-0728-6323 1 , 3 ,
• Arwa M. AL Akhatatneh   ORCID: orcid.org/0009-0009-2133-1822 1 ,
• Mansourah Aljohani   ORCID: orcid.org/0000-0001-5233-7738 1 &
• Mostafa Elhosseini   ORCID: orcid.org/0000-0002-1259-6193 1 , 4  

This survey paper comprehensively reviews Digital Twin (DT) technology, a virtual representation of a physical object or system, pivotal in Smart Cities for enhanced urban management. It explores DT's integration with Machine Learning for predictive analysis, IoT for real-time data, and its significant role in Smart City development. Addressing the gap in existing literature, this survey analyzes over 4,220 articles from the Web of Science, focusing on unique aspects like datasets, platforms, and performance metrics. Unlike other studies in the field, this research paper distinguishes itself through its comprehensive and bibliometric approach, analyzing over 4,220 articles and focusing on unique aspects like datasets, platforms, and performance metrics. This approach offers an unparalleled depth of analysis, enhancing the understanding of Digital Twin technology in Smart City development and setting a new benchmark in scholarly research in this domain. The study systematically identifies emerging trends and thematic topics, utilizing tools like VOSviewer for data visualization. Key findings include publication trends, prolific authors, and thematic clusters in research. The paper highlights the importance of DT in various urban applications, discusses challenges and limitations, and presents case studies showcasing successful implementations. Distinguishing from prior studies, it offers detailed insights into emerging trends, future research directions, and the evolving role of policy and governance in DT development, thereby making a substantial contribution to the field.

Avoid common mistakes on your manuscript.

1 Introduction

Digital Twin (DT) technology, a cornerstone of the Industry 4.0 era, represents a significant paradigm shift in how we interact with and understand physical systems and assets. Originating from Grieve's 2002 lecture at the University of Michigan (Grieves 2005 ) and later refined by NASA in 2010 (NASA 2010 ), the concept of DT has evolved into a sophisticated, multi-faceted approach to simulation and analysis. A Digital Twin is broadly defined as a digitally created virtual model of a physical object that leverages data to emulate the real-world behavior of the physical entity. It facilitates interaction and interoperability between the physical and virtual entities through interactive feedback, data integration, analysis, and iterative decision-making for optimized control, safety monitoring, and data analysis (Stark et al. 2017 ; Rosen et al. 2015 ).

Kritzinger et al. (Kritzinger et al. 2018 ) further categorized DT into three subtypes: digital model, digital shadow, and digital twin, each representing varying degrees of interaction and correlation between the physical and digital states.

The structure of a DT encompasses hardware and software components connected via middleware. The hardware typically includes IoT sensors and actuators, with the middleware playing a critical role in data management and communication between hardware and software. The software component, often an analytics engine, utilizes machine learning algorithms to transform raw data into actionable insights (Kritzinger et al. 2018 ). As depicted in Fig.  1 , this system encompasses the various components constituting the digital twin architecture.

Digital twin system structure. This diagram illustrates the essential components of a Digital Twin system, showcasing hardware with IoT sensors, middleware for data management, and a software analytics engine

Before going into the various applications of digital twins in various industries, it is important to comprehend the nature of digital twins and two closely connected ideas: digital model and digital shadow. These concepts are crucial for understanding this technology's depth (Kritzinger et al. 2018 ). The first concept is the Digital Model, a static digital representation of a physical object without automatic data exchange between the physical and digital entities. It can take many forms, including simulations, CAD files, 3D models, and mathematical algorithms. Digital models help design, optimize, and test by enabling the visualization, analysis, and manipulation of objects or systems in a digital context. A model is typically an estimation or prediction of how a system, process, or physical thing could function in the future or a certain setting. The second concept is Digital Shadow, which represents unidirectional information flow from the physical object to its digital counterpart, reflecting changes in the physical object. Through sensors, Internet of Things (IoT) devices, or other means, digital shadow gathers data from the asset (a database, a railroad system, or a banking platform). It delivers information that is fed into the model. This indicates that a digital shadow is current with the real object. It is helpful to understand it because it accurately depicts the asset in enough detail. The last concept is the Digital Twin, a dynamic, interactive digital representation capable of simulating, predicting, and interacting with data, showing a reciprocal impact between physical and digital states. Digital twins help with analysis, optimization, and predictive maintenance by simulating, monitoring, and controlling real-world systems or objects. They provide insights for enhancing effectiveness, dependability, and performance, as well as live feedback loops.

Figure  2 demonstrates the evolution from a basic digital model, lacking interactive data exchange, to a digital twin that dynamically mirrors and interacts with its physical counterpart, allowing for a two-way flow of information and continuous adaptation.

The progression from a static digital model to a dynamic digital twin, emphasizing reciprocal interaction between physical and digital assets

Digital twins' ability to reproduce physical items, processes, and systems in a virtual environment makes them useful in various applications. This technology has applications across various sectors and domains, providing several benefits and chances for innovation. For example, in industry, they are used for predictive maintenance, optimizing energy usage in smart buildings, and simulating traffic patterns in smart cities.

In the IoT sector, DTs are pivotal, acting as a critical bridge between the physical and digital realms. They allow for the seamless integration of digital and physical entities, enhancing maintenance capabilities and improving equipment performance monitoring (Fang et al. 2022 ; Mihai et al. 2022 ; Hinchy et al. 2019 ; Guo 2020; Wang and Luo 2021 ; Rajesh et al. 2019 ; Revetria et al. 2019 ). IoT can be seen as the vehicle that drives data to Digital Twins, enabling these virtual entities to replicate and interact with their physical counterparts in real-time. Digital Twins depend heavily on IoT technologies for data acquisition. IoT devices like sensors, RFID tags, and smart wearables collect data from the physical environment, which the digital twin then utilizes for various analyses. This data integration facilitated by IoT is crucial for applications ranging from predictive maintenance in industrial settings to real-time monitoring and augmented reality applications (Rajesh et al. 2019 ; Revetria et al. 2019 ). As described in sources monitoring (Fang et al. 2022 ; Mihai et al. 2022 ; Hinchy et al. 2019 ; Guo et al. 2020 ; Wang and Luo 2021 ; Rajesh et al. 2019 ; Revetria et al. 2019 ), IoT's role is not just about data collection but also about ensuring seamless integration of physical and virtual worlds, thus forming the backbone of any DT system.

While the Internet of Things (IoT) plays a major role in shaping and augmenting the capabilities of digital twins, machine learning augments these capabilities by allowing digital twins to analyze data, forecast, identify anomalies, optimize performance, customize experiences, and learn and improve continuously. Integrating machine learning with DT technology enables real-time, autonomous analysis of extensive data streams, enhancing decision-making and optimizing asset and system performance (Rathore et al. 2021 ; Dong et al. 2019 ; Zohdi 2020 ; Jaensch et al. 2018 ; He et al. 2019 ). Machine Learning, a pivotal branch of Artificial Intelligence, involves algorithms that enable systems to learn and adapt from data without being explicitly programmed. Its relationship with Digital Twin technology is synergistic. Digital Twins, virtual replicas of physical entities, systems, or environments, require advanced analytical capabilities to process and interpret the vast amount of data they receive. This is where Machine Learning comes into play. Machine Learning algorithms in DT systems facilitate the autonomous, real-time analysis of extensive data streams. These algorithms are adept at detecting patterns, making predictions, and optimizing processes based on the data ingested from the physical assets that the digital twins mirror. For instance, Rathore et al. 2021 (Rathore 2021) highlighted how applying advanced AI techniques to data within a DT system enables the creation of an 'intelligent' digital twin. This intelligence is manifested in capabilities like predictive maintenance, operational optimization, and dynamic decision-making based on a continuous stream of sensor and virtual data. The application of various machine learning models, such as Deep Neural Networks (DNNs) or Genetic Algorithms (GAs), is contingent upon the specific requirements and use cases of the intended digital twins (Dong et al. 2019 ; Zohdi 2020 ; Jaensch 2018; He et al. 2019 ). Therefore, Machine Learning is not just a complementary technology for Digital Twins but a fundamental enabler of their advanced functionalities.

In the sector of smart cities, DTs are used for urban planning, traffic management, environmental monitoring, energy management, waste management, public safety, infrastructure maintenance, water management, healthcare, public services, tourism, citizen engagement, economic development, and climate resilience. They provide real-time data crucial for emergency response, optimizing public transportation, and ensuring efficient city operations (Allam and Jones 2021 ; Bouzguenda et al. 2019 ; Svítek et al. 2019 ; Yu et al. 2021 ; Ghosh et al. 2016 ). Smart Cities represent urban areas that integrate various electronic data collection sensors to manage assets, resources, and services efficiently. Digital Twins, within the context of Smart Cities, act as sophisticated tools for urban planning, management, and enhancement of living conditions. They utilize data gathered via IoT devices and analyze it using machine learning algorithms to optimize city operations and decision-making processes. Besides, they contribute to traffic management, environmental monitoring, energy distribution, public safety, and more (Allam and Jones 2021 ; Bouzguenda et al. 2019 ; Svítek et al. 2019 ; Yu et al. 2021 ; Ghosh et al. 2016 ). For example, digital twins utilize data from sensors and cameras to optimize traffic flow and public transportation systems in traffic and transportation management. They use real-time data to monitor air and water quality in environmental monitoring. In energy management, digital twins aid in the operation of smart grids and in identifying potential energy conservation areas. These applications underline the comprehensive impact that Digital Twins, empowered by IoT and ML, can have in transforming urban environments into more efficient, sustainable, and responsive entities.

The primary aim of this paper is to engage in a comprehensive bibliometric analysis, examining the evolving landscape of Digital Twin technology within Smart Cities. The study is dedicated to methodically examining the scholarly dialogue, identifying predominant trends, and revealing key themes and collaborative networks in this area. We aim to provide a detailed, structured understanding of Digital Twin technology's role in urban development, filling a notable void in existing literature reviews. The survey's distinctiveness stems from its thorough data-gathering approach for bibliometric analysis in the field of Digital Twin technology and Smart Cities, selecting the Web of Science database for its broad interdisciplinary coverage and meticulously filtering over 4,220 pertinent articles, enhancing the depth and scope of analysis in these domains.

Our research found that various publications in various literary works are advancing the DT idea. Because there are so many articles available, academics have also published several survey papers that aim to review the current state-of-the-art in digital transformation (DT) development, inform other innovators about potential research gaps, questions, and directions, and point the industry toward potential DT use cases that could yield substantial business value in their particular domain.

Current literature predominantly concentrates on applying digital twin technology within specific facets of smart cities. For instance, Jafari et al. ( 2023 ) and He et al. ( 2023 ) explore the utilization of digital twin (DT) technology in enhancing various sectors of energy management within smart cities, encompassing transportation systems, power grids, and microgrids. Weil (2023) delves into the infrastructure elements of digital twins in smart cities, focusing on storage, computation, and network components. Nica et al. ( 2023 ) investigates Multi-Sensor Fusion Technology's role in sustainable urban governance networks. Dani et al. ( 2023 ) introduces an architectural framework underpinning the flow for digital twin platform development aimed at urban condition monitoring. Lam et al. ( 2023 ) outlines a use case for the 3D visualization of a smart village in Busan, South Korea, employing a 3D Geospatial platform. Paripooranan et al. ( 2020 ) suggests augmented reality (AR)-assisted DT as a pioneering approach towards the future transformation of human-centric industries. Mora (2023) highlights the importance of incorporating innovation management theories into the exploration of smart city transitions, offering novel insights and practical approaches to enhance the governance of smart cities through an innovation management lens. Ariyachandra and Wedawatta ( 2023 ) provides an overview of digital twin technologies' implications on disaster risk management, addressing the challenges of implementing digital twins in smart cities. Additionally, several reviews, including those by Weil (2023) and Wang (2024), focus on bibliometric analyses concerning digital twins in the realm of smart cities.

This work aims to support the other existing survey initiatives and provide a comprehensive comprehension of the DT. The paper gives an in-depth overview of the DT idea, architecture, enabling technologies, applications in smart cities, challenges, performance metrics, datasets, software, and use cases for deploying DTs in diverse industries, complementing prior research. This paper aims to fill a critical gap in understanding the expansive and evolving field of Digital Twin technology and its integration into Smart City development. This study is driven by the need to systematically synthesize and analyze the burgeoning body of research in this interdisciplinary area, providing clarity and direction for future studies. This survey's uniqueness and unprecedented nature stem from its comprehensive and systematic bibliometric analysis of over 4,220 articles on Digital Twin technology and Smart Cities. A focused examination of specialized areas such as datasets, platforms, and performance metrics marks this distinctiveness. The rigorous methodology involving the Web of Science database ensures in-depth interdisciplinary coverage. The survey's meticulous approach in formulating search strategies and selective filtration of relevant articles contributes to its depth and breadth, making it a unique contribution to the field. The significant contributions of this survey paper are listed below:

An overview of the DT definitions, concepts, and architecture in the literature

A Detailed bibliometric study of over 4,220 publications in Digital Twin technology and Smart Cities, including thematic trends analysis like AI and IoT integration.

Examination of datasets, platforms, and performance metrics specific to Digital Twins in urban settings and a critical evaluation of city models.

Applications in Smart Cities: Exploration of Digital Twin technology applications in urban development, encompassing urban planning, energy management, and public health.

Discussion of the challenges in implementing Digital Twin technology in Smart Cities, focusing on data integration, scalability, and security concerns.

Outlining potential research avenues based on current findings, indicating areas for further exploration.

Presentation of practical case studies demonstrating successful Digital Twin integration in urban development.

Summarizing the main findings and implications and a call to action for further research in this evolving field.

The paper's organization follows a clear and structured approach, beginning with Section  1 , an introduction that sets the stage for Digital Twin technology and Smart Cities. It progresses into Section  2 , which provides a detailed bibliometric study, covering objectives, methodology, data collection, and analysis, leading to key findings and implications. Then, Section  3 explores the applications of Digital Twins in Smart Cities. Section  4 discusses some technological aspects of DT. Section  5 presents some examples of datasets and software for developing DT. Section  6 states digital twin performance metrics according to its structure. Section  7 addresses the challenges associated with digital twins. Section  8 introduces some case studies for DT. Section  9 discusses smart city governance in the era of digital twins. Finally, Section  10 summarizes the paper's conclusions and presents future research directions.

2 Bibliometric study on digital twin and smart cities

The primary objective of this research is to perform a bibliometric analysis (Yu and Merritt 2023 ) to acquire a comprehensive understanding of emerging topics, prominent journals, and evolving research trends associated with the application of digital twin technology in smart cities. Additionally, the study aims to shed light on the potential challenges and future research trajectories concerning digital twin technology in the context of smart city development.

2.1 Research methodology

This investigation employed a systematic literature review (SLR) to meticulously explore, assess, and integrate the extant body of knowledge regarding the designated theme, adhering to a rigorously defined protocol (Kyriazopoulou 2015 ). Adopting the SLR methodology is instrumental in delineating the contemporary scholarly landscape of a given topic, thereby uncovering existing research voids and delineating avenues for forthcoming scholarly inquiries (Kitchenham et al. 2009 ). The SLR framework comprises three pivotal phases: planning, execution, and dissemination. The research inquiries were articulated during the planning stage, and criteria for identifying pertinent literature and determining search strategies were established. The execution stage entailed the meticulous gathering and vetting of scholarly works in alignment with the previously established criteria. This phase was initiated with an initial screening of the collected records through their titles and abstracts to ascertain their pertinence to the posed research questions, followed by an in-depth examination of the full-text articles. A bespoke form was devised for the methodical extraction of data, capturing essential information from the chosen articles, such as facets of digital twin components, smart city innovations, and the research lacunae identified therein. Subsequently, the dissemination phase involved the analytical consolidation and synthesis of the compiled literature. The process was underscored by a commitment to transparency and precision, with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) framework guiding the data acquisition methodology (Liberati et al. 2009 ).

2.1.1 Research questions

To delineate the scope of the SLR, the following research questions guided the study:

Q1: What are the components of digital twins in smart city applications?

Q2: What are the existing technologies used in the development of smart city development based on digital twins?

Q3: What are the research gaps and potential areas for future research?

2.1.2 Data collection

This section outlines the steps taken to collect relevant literature for the study. A PRISMA workflow diagram in Fig.  3 illustrates the study's search process. Initial literature searches were conducted in reputable databases such as Web of Science, Direct Science, and Scopus, which were chosen for their extensive coverage of scientific publications and advanced search capabilities. The research strategy applied an advanced search with keywords executed in the Web of Science and Scopus databases with a search string set to ("Digital twin," "virtual twin" or "virtual replica," and "smart city" or "smart cities"), for publications up to September 2023 and set to articles before 2018 were excluded. The period selected for the search is appropriate because there are few publications on digital twins and smart cities before 2018.

PRISMA workflow diagram

2.1.3 Inclusion and exclusion criteria

In this survey, the inclusion and exclusion criteria were meticulously established to ensure a focused and relevant analysis in the fields of Digital Twin technology and Smart Cities. This careful selection was pivotal in delineating the scope of the study.

Inclusion criteria:

Scope of content : Articles must focus on Digital Twin technology and its application within Smart Cities. This includes scholarly articles, conference proceedings, and review articles offering substantial insights into Digital Twin architectures, methodologies for Smart City implementation, technologies employed, and demonstrative case studies or laboratory setups.

Language : Only articles published in English are considered to ensure the clarity and accessibility of the content for our analysis.

Databases : Articles were sourced from the Web of Science, Scopus, and Direct Science databases to ensure a comprehensive and interdisciplinary coverage of the subject matter.

Publication period : Articles published from 2018 to 2023 were included to capture Digital Twin technology's evolution and current state in Smart Cities.

Detail requirements : Articles must present a detailed systematic architecture for a digital twin application and a clear methodology for Smart City implementation. They must also discuss the technologies used and provide demonstrative case studies or laboratory setups.

Exclusion criteria:

Language limitation : Articles published in languages other than English were excluded to maintain consistency and comprehensibility in the analysis.

Irrelevance : Publications unrelated to the direct intersection of Digital Twin technology and Smart Cities, lacking in detailed architecture, clear methodologies, technology discussion, or case studies, were excluded.

Duplication : Duplicated records identified across the databases were removed to ensure the uniqueness and accuracy of the analysis.

Date filter : Articles published before 2018 were excluded to focus the study on more recent developments and applications, reflecting the latest trends and innovations in the field.

The search produced 4,220 records from Web of Science, 382 from Scopus, and 24 from Direct Science. Through a preprocessing step, which involved removing duplicates and applying inclusion and exclusion criteria, 4,507 records were screened. This process yielded 4,073 articles that were deemed eligible for further analysis. The inclusion criteria were specifically targeted at articles that provided detailed systematic architectures for digital twin applications, methodologies for implementing smart cities, descriptions of technologies employed, and demonstrative case studies or laboratory setups.

2.2 Bibliometric study methodology

The methodology of a bibliometric study typically comprises a series of fundamental steps aimed at systematically analyzing scholarly literature within a specific field. These steps include formulating precise research questions to guide the analysis, identifying and selecting appropriate data sources, devising relevant search strategies using carefully chosen keywords, meticulously collecting and preparing the retrieved data, and employing established bibliometric techniques for rigorous data analysis. By adhering to this structured approach, researchers can effectively uncover trends and patterns in scientific publications and citations, thereby gaining valuable insights into the evolving landscape of their area of study (Mora et al. 2019 ). In line with these established practices, this research adopts a systematic approach for collecting, processing, and analyzing academic literature on digital twins within the context of smart cities.

2.3 Data analysis and visualization

This subsection outlines the methodologies and tools implemented to analyze and visualize the bibliometric data. For our study, VOSviewer was selected as the primary tool for managing and interpreting bibliographic data. We utilized network analysis methodologies to generate a range of visual representations. These included co-occurrence analyses, citation and co-citation maps, and keyword co-occurrence maps. Such visualizations were instrumental in uncovering patterns and discerning relationships within the collected dataset.

2.3.1 Publication trends

One of the key indicators in performance analysis is the annual number of publications. This metric serves as an indicator of research productivity. The data collected from 2011 to 2023 reveal a marked increase in publications focused on digital twin technology and smart cities. This surge in research output, demonstrating exponential growth, is depicted in Fig.  4 . This Figure underscores the significance and escalating interest in this interdisciplinary area. Figure  4 illustrates the yearly publication rates concerning digital twins and smart cities. Additionally, Table  1 provides a concise statistical analysis of these findings.

The publications rate of digital twin and smart cities by year from 2011 to 2023

2.3.2 Keyword analysis and research themes

This study's keyword co-occurrence analysis represents a systematic approach to understanding the prevailing keywords associated with digital twin technology and smart cities. The outcomes of this analysis, illustrated in Fig.  5 , reveal a range of predominant research themes and technologies pertinent to the domain of digital twins and smart cities.

Visualization of keyword co-occurrence network

Research themes in digital twin and smart cities:

Theme 1: Integration of AI and big data analytics in digital twins

This theme explores applying advanced deep learning techniques in processing and analyzing digital twin data. Key research areas are identified through terms such as "machine learning," "transfer learning," "simulation," "reinforcement learning," "cloud computing," "AI," "data analysis," "big data," and "forecasting."

Theme 2: Integration of digital twins with IoT

The focus here is IoT technologies, which are central to transmitting and collecting digital twin data. Relevant keywords include "wireless sensor network," "digital devices," "sensors," "5G", "communication," "wireless communications," and "monitoring."

Theme 3: Energy management in digital twins

This theme emphasizes the importance of energy efficiency and sustainability, highlighting keywords such as "energy efficiency," "energy utilization," "sustainability," and "renewable energy."

Theme 4: Security concerns in digital twins

Research in this area deals with the security aspects of digital twins, with keywords like "security," "privacy," "blockchain," and "fault diagnosis".

Theme 5: Cloud computing and digital twins

The final theme investigates the intersection of digital twins with cloud computing technologies, focusing on keywords such as "cloud computing," "edge computing," "fog computing," "blockchain," and "big data analytics."

Predominant technologies in the Digital Twin (DT) domain

Our study analyzed the authors' keywords to ascertain the most prominent technologies within the digital twin sphere. This investigation uses keyword frequency as a metric to identify the key technologies extensively employed in the digital twin (DT) domain. The term 'Internet of Things' (IoT) emerges as the most frequently cited keyword, demonstrated in Fig.  7 . This finding underscores the pivotal role of IoT in the digital twin field, highlighting its extensive research coverage and the ongoing need for in-depth exploration of IoT applications to enhance digital twins' efficacy. Additionally, "AI" and "machine learning" are prominently used to analyze and process large volumes of digital twin data. Other notable technologies such as "cloud computing," "virtual reality," and "digital twin security" have also gained traction. Collectively, these technologies contribute to the efficient storage, visualization, modeling, and security of digital twin data. The data presented in the accompanying table and Fig.  6 substantiate the findings discussed in this subsection.

High-frequency keywords in digital twin research

2.3.3 Analysis of geographical distribution

Examining the geographical distribution in the research and development of digital twin and smart cities technologies offers critical insights into the regional contributions, patterns of collaboration, and prospective areas for advancement. As depicted in Fig.  7 (A), our analysis reveals a broad geographical spread in the field's research activities. We identified key regions contributing significantly to the field by utilizing a citation metric analysis on our dataset, which set a minimum of ten documents and fifty citations per country. China emerges as the leading contributor in terms of citations, followed by the USA, the UK, Italy, and Germany. Furthermore, Fig.  7 (B) corroborates the leadership status of China, the USA, the UK, and Italy in this domain.

A Citations by country in digital twin and smart cities research. B Top 10 publishing countries in digital twin and smart cities research

2.3.4 Analysis of source co-citation

The source co-citation analysis conducted in our study highlights the prominent sources within the domain of digital twins and smart cities. Of 49,275 sources, 433 met the established criterion of a minimum of 50 citations per source. The findings of this analysis are presented in Fig.  8 . The most frequently co-cited journals include IEEE Access, the Journal of Manufacturing Systems, IEEE Transactions on Industrial Informatics, and IEEE Internet of Things. The analysis identified six distinct clusters, each represented by a unique color, as depicted in Fig.  7 (A).

A Co-citation network of sources in digital twin and smart cities research. B Bibliographic coupling network among countries in digital twin and smart cities research

2.3.5 Examination of international collaboration

The observed international collaboration in the digital twin and smart cities sector underscores research's global impact and relevance. Utilizing bibliographic coupling analysis on our dataset, with a set threshold of a minimum of 10 documents and 20 citations per country, 65 out of 103 countries met these criteria. A network visualization visually represents the bibliographic coupling among these countries in Fig.  8 (B).

This analysis collated data on each country's publications, citations, and total link strength. Each node in the figure symbolizes a country whose size reflects its publication count. The visualization reveals that China leads in a collaborative network, boasting approximately 1151 documents and a total link strength of 871,379. Following China are the USA, the UK, and England. Notably, the USA's most extensive collaborations were with China, England, and India, while China's primary collaborations were with the USA, England, and Germany.

The colors in Fig.  8 (B) delineate nine distinct clusters, indicating nations that frequently cite each other's research, suggesting closer collaboration within these groups. This mapping confirms that countries like China, the USA, the UK, Italy, and Germany are at the forefront in advancing research in digital twin and smart cities.

3 Applications of digital twin technology in smart city development

Digital twin technology offers a wide range of applications in smart city development, from optimizing traffic flow and energy usage to improving public safety, Environmental Monitoring and Management, Citizen-Centric Aspects, and Supply Chain Management and Enhancement. By creating virtual replicas of city infrastructure and systems, urban planners and policymakers can visualize potential changes and their impact before implementing them in the physical environment. Furthermore, digital twins can be instrumental in public safety by simulating emergency response scenarios and planning for effective evacuation routes in the event of natural disasters or other crises, as shown in Fig.  9 . As the adoption of digital twins continues to grow, their role in shaping the future of smart cities will become increasingly prominent.

Applications of digital twin technology in smart city development

3.1 Urban planning and management

Urban planning and management encompass the technical and political processes of utilizing land, infrastructure, and buildings within urban areas. This multifaceted domain includes urban design, land use, transportation, zoning, regulation, and environmental planning.

Urban planners and managers increasingly employ digital twin technology to enhance city functions like transportation and sustainability. Digital twins enable more informed decision-making and optimize planning, operations, finance, and strategy. In turn, such systems help reduce carbon emissions and expedite significant projects. Additionally, they enable the simulation of plans before implementation, allowing for the anticipation of potential challenges. The World Economic Forum 2022 recognized the role of digital twins in modeling future sustainable development by integrating digital technology with urban operational systems. This integration facilitates safer, more efficient urban activities. It creates low-carbon, sustainable environments through precise mapping, virtual-real integration, and intelligent feedback of physical and digital urban spaces (Yu and Merritt 2023 ).

Within urban planning and management, digital twins can represent entire cities or specific urban systems, assisting in various ways:

Real-time Monitoring: Integrating sensors and IoT devices with digital twins provides real-time data on urban processes like traffic flow, energy consumption, and air quality.

Simulation and Scenario Testing: Planners can use digital twins to simulate and test different scenarios, assessing the impacts of natural disasters or new transportation systems.

Optimization: Analyzing data from digital twins can identify and address inefficiencies in urban systems.

Public Engagement: Digital twins serve as interactive platforms for public involvement, allowing community members to view proposed changes and provide feedback.

Maintenance and Asset Management: They enable tracking urban infrastructure conditions and predicting maintenance needs.

System Integration: Digital twins facilitate understanding of interdependencies between various urban systems.

Support for Decision-Making: Providing a comprehensive view of the city and its systems, digital twins enhance the decision-making process, ensuring decisions are informed by accurate, up-to-date information.

3.2 Energy management

Energy systems form the backbone of smart cities, ensuring the quality and functionality of these urban environments. This section delves into the application of DTs in energy systems, encompassing transportation systems, power grids, and microgrids.

Digital Twin technology finds varied applications in transportation systems. It supports transportation system infrastructures in several ways, such as monitoring transport systems, traffic forecasting, energy system management, predicting the energy consumption of electric vehicles, IoT-based parking management, analyzing driver behavior, forecasting subway regenerative braking energy, studying pedestrian behavior, controlling health systems, and detecting cyber-physical attacks. DTs can significantly contribute to these areas. For instance, using DTs for transportation system monitoring can reduce maintenance costs. Beyond modeling and planning, DTs facilitate optimal traffic management and provide accurate and extensive traffic and electric vehicle (EV) data, contributing to sustainable development and efficient urban traffic control.

In traffic management, DT technology has been utilized to predict patterns of energy consumption and production (Ketzler et al. 2020 ). Additionally, DTs play a role in IoT-based parking management, improving user services by saving time and reducing parking costs.

Various studies have employed DT technology in diverse contexts. In (Yan et al. 2022 ), authors analyzed real drivers' and pedestrians' behavior using DTs. In (Liu et al. 2020 ; Damjanovic-Behrendt 2018 ), DTs of drivers and vehicles were used in real-time to relay critical information to drivers and vehicles in the physical world. Moreover, in (Crespi et al. 2023 ), the Electric Vehicles (EVs) model employed DTs to monitor the behavior and optimally manage charging programs, using energy consumption parameters and charging capacity and frequency for modeling the virtual twin.

A microgrid is an autonomous energy system characterized by distributed energy resources and interconnected loads. It functions as a manageable entity within the larger grid, enabling it to operate in either island mode or in conjunction with the grid (Ton and Smith 2012 ). The objective of microgrids is to enhance the functionality of energy systems in terms of sustainability, economic viability, efficiency, security, and overall energy management. Key aspects of microgrid performance include reliability, self-sufficiency, security, flexibility, and optimality. Studies on microgrids utilizing the Digital Twin (DT) framework have encompassed areas such as forecasting (Din and Marnerides 2017 ; He et al. 2017 ), management and monitoring (Xu et al. 2019 ; Park et al. 2020 ), fault prediction (Nowocin 2017 ; Goia et al. 2022 ), and security (Huang et al. 2021 ).

The development and implementation of DT-based power grids are instrumental in improving network behavior under various conditions. Network studies employing DT include diverse analyses such as restoration (Biagini et al. 2020 ), reliability (Podvalny and Vasiljev 2021 ), prediction (Park et al. 2020 ), addressing uncertainty (Raqeeb et al. 2022 ), energy hub management (Kuber et al. 2022 ), and ensuring both physical and cyber security. Each of these analyses offers unique insights into network behavior. In reference (Endsley 2016 ), Situation Awareness (SA) is the ability to perceive the elements in a specific environment, understand their properties, and anticipate their future statuses. SA is crucial in augmenting decision-making, especially in complex systems like the Energy Internet of Things (EIoT) (He et al. 2023 ). It provides essential information critical for such systems' operation, enhancing efficiency and effectiveness.

In the application of Digital Twin technology in power systems, several significant challenges emerge:

IT infrastructure limitations: Existing infrastructure often falls short in supporting the data analysis demands of DT environments.

High-performance computing needs: Utilizing high-performance GPUs and cloud services from major providers is essential for adequate support.

Connectivity issues: Software errors and power outages present obstacles in real-time monitoring.

Cybersecurity risks: The extensive data exchange in DT systems heightens vulnerability to cyber-attacks, necessitating secure platforms.

Standardization requirement: The absence of standardized protocols impedes DT development, highlighting the need for unified approaches for model definition, storage, and execution.

The exploration of digital twin applications in energy management reveals several key areas for future development:

Advancements in big models: Addressing challenges in AI, such as limited model generalization and the need for high-quality data, by developing larger, more adaptable models.

Virtual twin structures in power systems: Detailed modeling of power system entities using virtual twins, enabling dynamic visualization and strategy development for urban transformation.

Application of theoretical models: Utilizing chaos and complex system theories (Mir et al. 2022 ) to understand and optimize the nonlinearities in power systems, offering a novel approach to managing system complexities.

3.3 Traffic and mobility management

Traffic and Mobility Management in Smart Cities (Xu et al. 2023 ), enhanced by Digital Twin (DT) technology, represents a significant advancement in urban planning and logistics. DTs enable:

Real-time traffic simulation : Mimicking urban traffic flow to identify and alleviate congestion points.

Public transportation optimization : Analyzing patterns to improve transit routes and schedules.

Pedestrian flow management : Ensuring safer and more efficient pedestrian movement.

Pollution reduction : Aiding in strategies to lower emissions through traffic regulation.

Emergency response enhancement : Assisting in quicker and more efficient routing for emergency services.

Data-driven decision making : Utilizing sensor data for informed traffic management decisions.

Sustainable urban planning : Contributing to long-term urban sustainability goals through efficient mobility solutions.

These applications of DT in traffic and mobility management significantly contribute to creating more livable, efficient, and sustainable urban environments.

3.4 Environmental monitoring and management

Digital twin technology is increasingly integral in urban development, offering real-time insights and solutions for environmental management:

Optimization and prediction: Digital twins, as virtual representations of physical entities, enable process optimization, change monitoring, and future scenario prediction (Wang et al. 2023 ).

Environmental monitoring applications: Usage in water quality monitoring, detecting pollutants, and adapting to changing environmental conditions.

Data integration in smart cities: Interconnection of multiple digital twins, using diverse data sources like temperature and humidity, to forecast environmental conditions (Ivanov et al. 2020 ).

Sensor utilization: Various sensors capture essential environmental data for digital twin construction, including Kinect v2 depth cameras and electronic gloves for manufacturing systems (Nikolakis et al. 2018 ).

Food industry monitoring: Application in monitoring and predicting food quality, employing wireless sensors for environmental factors like humidity and temperature (Defraeye et al. 2019 ).

Agricultural management: Use in agriculture for crop growth monitoring and simulating interventions, aiding in remote farm management (Verdouw et al. 2021 ).

Healthcare applications: Implementation in healthcare for environmental monitoring and mental health management using smartwatch sensors (Bagaria et al. 2019 ).

These diverse applications showcase the role of digital twins in enhancing urban planning, agriculture, healthcare, and more.

3.5 Public health and safety

In the development of Smart Cities, Digital Twin Technology plays a crucial role in enhancing public health and safety (Erol et al. 2020 ). It offers a dynamic and integrated approach to managing complex urban health challenges through simulation and analysis. This technology's applications include:

Disease outbreak prediction and management : Leveraging real-time data to simulate disease spread and plan responses.

Emergency preparedness : Using simulations for natural disasters or public safety incidents to enhance response strategies.

Resource optimization in healthcare : Improving the allocation of healthcare resources like hospital beds and emergency services.

Environmental health monitoring : Tracking and analyzing environmental factors that impact public health, such as pollution levels.

Public safety and incident response : Simulating various scenarios to optimize law enforcement and emergency services.

These applications demonstrate the transformative impact of Digital Twin Technology in public health and safety within Smart Cities.

3.6 Citizen-centric aspects

Technological advancements have focused on urban development and infrastructure management, employing physical sensors such as the Internet of Things (IoT) and satellites (Borrmann et al. 2018 ). However, not all city digital twin implementations have citizen engagement in mind. A citizen-centric digital twin (CCDT) approach views citizens as integral components of a data-driven city, with human sensors playing a key role in addressing city-scale challenges (Saeed et al. 2022 ). This approach distinguishes itself from traditional digital twin frameworks by prioritizing citizens as the central element and integrating technologies like processing, data acquisition, and visualization to enhance citizen involvement in infrastructure governance.

Developing a CCDT requires the execution of numerous processes and technologies. One such technology involves sensors like Volunteered Geographic Information (VGI) (White et al. 2021 ), which transfer data from the actual city to its digital twin, followed by analysis using various analytical tools. Managing data from diverse sources at a city scale presents challenges in scalability, reliability, and the performance of real-time analytics and modeling (Langenheim et al. 2022 ).

The study by Abdeen et al. (Abdeen et al. 2023 ) indicates a scarcity of publications in this field over the past five years. However, a rising trend in CDT interest post-2017 was noted, with publications doubling by the end of 2022 compared to 2019. Research works (Ford and Wolf 2020 ; Fan et al. 2021 ) discuss the application of digital technologies in catastrophic situations and emergency responses. The capabilities of intelligent digital twins in various application fields have been examined (Shahat 2021; Deren et al. 2021 ), with the latter focusing on hazards like epidemic services, traffic control, and flood monitoring. (Shahat et al. 2021 ) concentrated on data simulations, fusion, administration, and collaboration. (Charitonidou 2022 ) addressed citizen participation in decision-making, highlighting that limited variables and processes and overlooking social aspects of urban contexts can render citizen input integration ineffective.

In the literature, various data acquisition mechanisms are employed to support CCDTs. One prominent method is the use of open-source data platforms (OSDP), providing spatiotemporal performance data relevant to CCDT applications like disaster management (Ghaith et al. 2022 ) and public services monitoring (Diakite et al. 2022 ). However, the effectiveness of CCDTs can be compromised if data is unreliable. Another mechanism is crowdsourcing, which generates large quantities of data and is particularly useful when remote or IoT sensors are unavailable (Trusov and Limonova 2020 ). Nevertheless, citizens' data errors or bogus inputs can affect CCDT effectiveness (Trusov and Limonova 2020 ). Visionary concepts for disaster city digital twins with extensive data (images, text, geo maps) have been proposed to enrich CCDT content (Fan et al. 2021 ).

Remote sensors are effective in modeling 3D city aspects of CCDTs and for large-scale urban monitoring (Fan and Mostafavi 2019 ; Fan et al. 2020 ). Geospatial platforms storing and managing data from individual vehicles or pedestrians have been proposed (Lee et al. 2022 ), though data accessibility to all stakeholders remains challenging for CCDT integration. Furthermore, IoT sensors (Nochta et al. 2020 ), deployed in large numbers and integrated effectively, facilitate urban data monitoring but require advanced communication infrastructure.

Advanced AI algorithms also play a crucial role in CCDTs, enhancing citizen engagement. Genetic algorithms (Fan et al. 2021 ) have been used to study the range of disruptions during hazard events, while Convolutional Neural Networks (CNN) (Pang et al. 2021 ) and Burst detection algorithms (Fan et al. 2021 ) help analyze crowdsourced data and social media frequencies providing insights into citizen perspectives and infrastructure governance through CCDTs.

3.7 Supply chain management and enhancement

The supply chain, encompassing the entire spectrum from raw material sourcing to the distribution of finished products, has seen a transformative integration of Digital Twin technology in recent years. Digital Twins, as virtual models of physical assets, offer real-time monitoring, analysis, and optimization across all facets of the supply chain, ranging from procurement to distribution (Tao et al. 2017 ). According to van der Valk et al. (van der Valk et al. 2022 ), these digital replicas enable two-way data exchange between the digital and physical worlds, providing professionals with exceptional visibility and traceability. This level of insight facilitates the identification of complex behavioral patterns and proactive problem detection, which is crucial for maintaining operational continuity.

As Gerlach et al. (Gerlach et al. 2021 ) highlight, Digital Twins are instrumental in offering real-time inventory insights, enabling the simulation of various scenarios, and assisting in planning and forecasting. These capabilities can result in significant cost reductions and process efficiency improvements. The study by Srai et al. (Srai and Settanni 2019 ) explores the optimization opportunities that Digital Twins offer in areas such as transportation resource management, demand–supply analysis, customer service improvement, and revenue enhancement. They also emphasize the role of technology in identifying and addressing inefficiencies.

The influence of Digital Twins in improving stock availability, a key aspect of manufacturing operations, is underscored by Abouzid et al. (Abouzid and Saidi 2023 ). Furthermore, (Lugaresi et al. 2023 ) introduces the concept of "technological labelers" like IoT devices, cloud computing, and advanced analytics, which are crucial in developing a comprehensive digital twin of a company's value chain. IoT integration, in particular, is noted for significantly enhancing supply chain efficiency by providing real-time data and contextual insights. In addition, distorted demand signals can result in various supply chain challenges, which can be effectively addressed using Digital Twin technology (Abouzid and Saidi 2023 ).

In conclusion, the discourse delves into the exploration and development of digital twins within automated manufacturing systems, showcasing the expansive potential of this technology in modernizing and streamlining supply chain management processes (Abideen et al. 2021 ).

4 Technological aspects of digital twin

This section explores the fundamental components and emerging advancements in digital twin technology, covering various technological aspects essential for understanding Digital Twins across disciplines. Each component offers unique insights into critical subjects, including distinctions between Digital Twins and Building Information Modeling (BIM) and the development of Cognitive Twins. By studying frameworks like the Five-Layer Architecture and advancements such as Cloud and Edge Computing integration, this section aims to reveal the technological foundations driving the evolution of Digital Twins. Readers will gain a deeper understanding of the technological breakthroughs shaping the future of Digital Twins and their applications through this comprehensive examination.

4.1 Distinction between digital twins and Building Information Modeling (BIM)

In construction technology, the emerging potential of digital twins and the rapid advancement of smart technologies has garnered significant interest. Although 'digital twin' is a relatively new term in the construction research literature, it is often conflated with Building Information Modeling (BIM), leading to some conceptual ambiguity. It is imperative to clarify the differences between these two concepts.

Building Information Modeling (BIM) is a digital representation of a building or structure's physical and functional characteristics. It is a tool architects, engineers, and construction professionals utilize to create detailed digital models of buildings, encompassing all systems and components, including architectural, plumbing, electrical, and HVAC systems. BIM enables the creation of accurate construction plans, virtual walkthroughs, and performance testing under various scenarios. (Wang and Meng 2021 ) defined BIM as a method that integrates geometric and non-geometric data. The 3D model, often called the BIM model and realized through object-oriented software, is a critical component of BIM (Cerovsek 2011 ). However, BIM primarily manages static data and requires external technologies to update models with real-time data (White 2021). In construction projects and asset management, a vast amount of non-geometric data is essential for informed decision-making but is often underutilized (Khudhair et al. 2021 ). BIM models have limited capacity to handle large volumes of dynamic and multifaceted data, necessitating advanced storage and processing technologies. These limitations can lead to data underutilization, inefficient decision-making, and financial implications. The advent of digital twin technology offers a solution to overcome these constraints inherent in BIM.

Digital twins and BIM represent two distinct technological applications in the construction sector, differentiated by their functions. BIM is most effective in the design and construction phases, while digital twins excel in building maintenance and operations. A digital twin system involves data linkage that transfers information between the physical asset and its virtual counterpart. This indicates that a BIM (Building Information Modeling) model is the initial step toward developing a digital twin in the construction industry. Digital twin technology integrates the BIM model with the physical world, enabling bidirectional data exchange. This connection allows for the real-time updating of the BIM model, enhancing asset implementation and management decision-making. The synergy between BIM and Digital Twin technology has the potential to revolutionize the construction industry. By combining the detailed architectural and structural information provided by BIM with the real-time operational data and analysis capabilities of Digital Twin technology, construction professionals can create comprehensive, accurate digital models of buildings. These models can then continuously monitor and optimize building performance in real-time.

4.2 Framework of the five-layer architecture in digital twins

A digital twin, essentially a digital representation of a physical entity, process, or person contextualized in a virtual environment, is a pivotal tool for organizations to simulate real-world scenarios and outcomes, thus enhancing decision-making capabilities (Moosavi et al. 2021 ). As shown in Fig.  10 , the architecture of digital twins is typically structured into five principal layers, as outlined in (Jones et al. 2020 ):

Five-layer architecture in digital twins

Physical layer : This foundational layer comprises the actual physical objects or entities. It utilizes sensor technology for data acquisition and can receive commands from the virtual layer. This layer provides real-time data feedback to the digital twin model.

Data sensing layer : Responsible for collecting diverse information types, this layer employs various sensors to monitor the system's status and operational process in detail. The data heterogeneity and variety stem from diverse data generation sources, such as IoT sensors, information systems, and wearable devices.

Data transmission layer : As a crucial link, this layer ensures data transmission between the physical and virtual layers. It leverages communication integration protocols and interactive security technologies to facilitate this transfer (Lohtander et al. 2018 ).

Virtual layer : In this layer, the components of the real world are digitally reconstructed. It builds a collection of digital twins using data transmitted from the physical layer, enhanced with historical or integrated network data. This layer dynamically tunes itself based on the real-time data from the physical layer and can be influenced by modifications made in the application layer.

Application layer : This layer visualizes the data and simulations derived from the virtual layer, presenting a graphical model that staff can easily interpret. Modifications in the physical or virtual layer parameters can lead to simulation changes. These can then be revised and optimized based on the observed or extrapolated results.

Each layer in this five-tier architecture plays a distinct yet interconnected role, collectively enabling the digital twin to function as a comprehensive, dynamic system for analyzing, simulating, and enhancing real-world processes and entities.

4.3 Integration of cloud and edge computing in digital twin environments

Cloud computing represents a large-scale computational approach that leverages the Internet to facilitate sharing computing, storage, and other resources, accessible anytime and anywhere on demand. In contrast, edge computing is a novel computational model that processes a portion of data using distributed computing, storage, and network resources between data sources and cloud computing centers.

Edge computing is increasingly recognized for its potential to enhance privacy, reduce latency, conserve energy and costs, and boost reliability. It is particularly well-suited for Digital Twin (DT) scenarios that demand low latency, high bandwidth, high reliability, and stringent privacy measures. In DT-assisted edge computing setups, the framework includes user devices, edge servers, resource devices, and the DT itself. User equipment initiates task requests to the edge server, which then allocates computing resources to the task, with the DT deployed within the edge server. Reviewed literature demonstrates the application of Cloud and Edge technologies in various contexts. Cloud storage is universally employed in these studies. Earlier research also utilized cloud computing for user interfaces, with cloud-rendered 3D models, as indicated in (Xu et al. 2021 ), or through GUIs accessible via web applications as in (Urbina Coronado 2018). While initial studies relegated intensive data processing and analytics to cloud computing due to its superior resource access, recent advancements in edge device capabilities have led to the implementation of edge computing, employing techniques for heavy data analysis or machine learning (Cathey et al. 2021 ; Lu et al. 2021a ; Zhang et al. 2022 ).

Recent digital twin research, such as (Alam and El Saddik 2017 ), employs edge computing, describing a framework where each device is represented as a cloud-based digital twin. This hierarchical architecture involves higher-level digital twins composed of simpler units in a master/slave relationship, enhancing the communicability of traditional cyber-physical systems with cloud servers' advanced computational and storage capacities. Focus on edge-based architectures is evident in (Dong 2019; Lu et al. 2021a ), with research by Dong (Dong 2019) on enhancing energy efficiency in 5G services through deep neural networks and Lu (Lu et al. 2020 ) exploring the use of digital twins in-network replication and machine learning via federated learning.

In industry, studies such as (Lu et al. 2021b ; Zhang et al. 2022 ) concentrate on Smart Vehicles, driven by the rise in edge computing power. Other research, including (Liu et al. 2019 ; Martinez-Velazquez et al. 2019 ), investigates the application of digital twins in healthcare, aiming to provide high-quality, real-time care to senior citizens. Most other studies, such as (Xu et al. 2021 ; Urbina Coronado et al. 2018 ; Hu et al. 2018 ; Bellavista et al. 2021 ), are categorized under Smart Manufacturing, focusing on industrial productivity improvements. Cloud-based digital twins play a crucial role in optimizing IoT device energy consumption and operational efficiency (Li et al. 2020 ), detecting and preventing potential system failures (Cathey et al. 2021 ), and ensuring data privacy and integrity (Wen et al. 2020 ). Thus, cloud computing and IoT emerge as complementary technologies, synergistically advancing the development of smart, interconnected systems.

Research in the Oil and Gas industry reflects a systematic adoption of digital twins and cloud/edge computing. For instance, (Pivano et al. 2019 ) discusses offloading simulations and data analysis to public cloud servers to access greater computational resources and avoid complex local IT infrastructures. Tygesen et al. (Tygesen et al. 2018 ) highlight the role of high-performance cloud computing in wave load modeling, which is essential for maintaining offshore platform integrity. (ASME 2018 ) describes the use of cloud data lakes for data verification and physical model feeding. At the same time, a microservices-based approach has been presented for designing and implementing digital twins using open-source tools (Zborowski 2018 ).

4.4 Implementation of augmented reality in digital twin technology

Augmented Reality (AR) is a technology that merges the real with the virtual, facilitating real-time interaction and 3D registration (Damjanovic-Behrendt and Behrendt 2019 ). It enhances user experience by superimposing graphics, video streams, or holograms onto the physical world (Yin et al. 2023 ). It is supported by various devices such as AR head-mounted displays (HMD), tablets, head-up displays (HUD), projectors, VR HMD with cameras, and 2D screen augmentations.

AR's enhancements are primarily derived from its visualization, interaction, 3D registration, and information collection capabilities as a unified device (Billinghurst et al. 2015 ). AR contributes to Digital Twin (DT) technology in several dimensions. In the virtual twin dimension, AR provides visualization of non-registered geometry, data, workflows, and basic status monitoring and alerting for operators. It also allows users to update DT information, for example, by scanning barcodes or adding annotations. However, the full potential of AR in augmenting DTs remains underexploited. In the hybrid twin dimension, AR enables multi-modal interactions and on-site registered visualization, with a need for further exploration and utilization in cyber-physical interaction functions. In the cognitive twin dimension, AR-assisted DT, bolstered by edge-cloud computing systems, is poised to play a more significant role in areas like visual programming, human–robot collaboration (HRC), product design, and human ergonomics, marking promising future directions for AR-assisted DT.

Applications of AR-assisted DT span a wide range of physical scenarios, encompassing the entire product lifecycle, including the management of production facilities and services. The production process and service phases include design, production, distribution, maintenance, and end-of-life stages, as illustrated in Fig.  11 .

Applications of AR-Assisted digital twins in engineering lifecycle, including design, production, distribution, maintenance, and end-of-life stages

The systematic design process involves prototyping, pilot runs, and testing operations. Real-time data from product usage, collected by sensors, informs smart product service design or redesign, integrating DT in mapping virtual and physical objects (Praschl and Krauss 2022 ). Research in AR device utilization for design falls into three categories: product design (Zheng et al. 2018 ), service design, and system design. Service design, a creative and user-centric process for enhancing or creating new services (Chang et al. 2020 ), is supported by several studies focusing on operation training (Blomkvist et al. 2023 ), driving and flight guidance (Moya et al 2020b ), smart environments (Vidal-Balea et al. 2021 ), smart cities (Lacoche and Villain 2022 ; Ssin et al. 2021a ), and smart wetlands (Ssin et al. 2021b ), aiming to deliver user-centered services that cater to the needs of users and stakeholders.

System design entails developing architecture, components, and core algorithms for AR-assisted DT scenarios. Moya et al. (Aheleroff et al. 2020 ; Moya et al. 2020a ) introduced two self-learning DT systems with screen augmentation for fluid behavior prediction and beam load analysis.

The production process includes goods fabrication or service provision, subdivided into process planning and scheduling (Wiegand et al. 2018 ), monitoring and control (Lemos et al. 2022 ), assembly (Kritzler et al. 2017 ), and robotics-related works (Židek et al. 2021 ). Real-time machine status monitoring and interactive control are prevalent in research, as demonstrated by Paripooranan et al. (Paripooranan et al. 2020 ), who developed an AR-enabled 3D printer DT for alerting abnormal statuses.

In distribution, warehouse management utilizes AR and DT, as shown by Petković et al. (Petković et al. 2019 ) in their use of a warehouse system DT (comprising the warehouse, automated guided vehicles (AGV), and operators with AR HMD) to test a human intention estimation algorithm.

Maintenance work encompasses various strategies and can be categorized into reactive, preventive, and predictive maintenance (Petković et al. 2019 ), adopting different approaches within the AR-assisted DT framework.

4.5 Hybrid twins in mixed reality applications

Mixed Reality (MR) applications offer an interactive experience that blends real and virtual environments, akin to Augmented Reality (AR) (Damjanovic-Behrendt and Behrendt 2019 ). Additionally, the term Extended Reality (XR) encompasses Virtual Reality (VR), AR, and MR and has been included in the research scope. The enhancements brought about by AR are examined across three distinct dimensions of the digital twin: the virtual twin, hybrid twin, and cognitive twin, as depicted in Fig.  12 .

Layered framework for digital twin classification based on augmented reality devices' perceptual capabilities

The virtual twin dimension encompasses data transmission from physical to virtual realms, non-registered visualization, and essential status monitoring and alerting functions based on sensor data. When enhanced by Augmented Reality (AR) devices' perceptual capabilities, this dimension can improve the data transmission process from the physical to the virtual space and suitably update Digital Twin (DT) information. Beyond IoT sensor data, on-site information such as barcodes and workspace details can also be gathered through AR applications, exemplified in warehouse management (Xia et al. 2022 ).

Reference (John Samuel et al. 2022 ) discusses the concept of hybridization in DTs, focusing on refining DT accuracy through self-adaptation and data-driven estimation techniques. This approach integrates physics-based model predictions with process measurements, creating a hybrid digital twin (HT) that facilitates the soft-sensing of otherwise hard-to-predict data.

The hybrid twin dimension emphasizes analysis and feedback from the virtual to the physical world, such as context information-related analysis, visual registration, multi-modal interaction and control, and the functionalities based on these aspects. Traditional DTs manage real-time data analysis, including simulation, prediction, diagnosis, and optimization, feeding back the analysis outcomes from the virtual to the physical world. AR-assisted DTs enhance this analysis with on-site data, adding capabilities like object localization, scene understanding, and cyber-physical interaction computation. For instance, in human–robot collaboration (HRC) assembly (Johansen et al. 2023 ), the hybrid twin dimension offers immersive visual registration beyond traditional 2D interfaces, displaying geometry and key data overlaid on the physical entity in the correct position. In contexts such as assembly (Liu et al. 2022 ; Zhao and Sun 2020 ), maintenance (Meier et al. 2021 ; Li et al. 2021 ; Rabah et al. 2018 ), and manual or semi-automated tasks (Koteleva et al. 2021 ; Rebmann et al. 2020 ; Mandl et al. 2017 ), operators can reference on-site instructions and guidance to work more efficiently. Additionally, geometry overlay for inspection (Catalano et al. 2022 ; Xie et al. 2020 ) or motion preview (Kim and Olsen 2021 ) aids operators in verifying the shape or movement of physical entities against planned outcomes. Users can also add geometry-linked or position-related annotations through AR.

Akroyd et al. (Akroyd et al. 2022 ) introduced the concept of the Universal Digital Twin, a digital twin that leverages a dynamic knowledge graph to enable cross-domain interoperability for DTs.

4.6 Development of cognitive twins in digital twin technology

Cognitive twins represent an advanced form of digital twins endowed with high-level cognitive capabilities encompassing machine and human intelligence. These cognitive twins are designed to address complex and unpredictable situations using enhanced computational power dynamically. Augmented Reality (AR) significantly contributes to the development of cognitive twins as it can function as a wearable computational unit within the edge-cloud architecture (Li et al 2022 ). HoloLens 2, a widely-used AR device, notably possesses substantial computing power (1 T FLOP) compared to wearable devices like sensors. This capability allows training models on high-power devices and their subsequent deployment on HoloLens 2, highlighting one of AR's key benefits to digital twins.

Cognitive Digital Twins (CDTs), originating from the domains of Industry 4.0 and Smart Cities, are recognized for their ability to support autonomous activities (Um et al. 2018 ; Liu et al. 2023 ; Zheng et al. 2021 ). Semantic technologies, including ontology and Knowledge Graph (KG), are vital in interlinking digital twins in virtual spaces. These technologies eliminate ambiguity across heterogeneous systems, thus enhancing digital interoperability and enabling cooperative decision-making (Rožanec et al. 2022 ). As defined by (Pan et al. 2021 ), ontology involves a set of formal and explicit vocabularies characterized by shareability and reusability, describing domain-specific knowledge, entities' attributes, and their interrelationships. While early research primarily focused on utilizing ontology for data modeling and sharing (Rožanec et al. 2022 ), recent studies emphasize that integrating semantics with digital twin technologies can advance the capability and interoperability of CDTs in autonomous and cooperative decision-making (Zheng et al. 2021 ).

The knowledge graph has become increasingly important in developing and managing CDTs because it can delineate relationships between real-world entities or link data (Liu et al. 2023 ). For instance, recent research has explored using knowledge graphs and digital twins in managing assets and tasks in smart manufacturing systems (Guarino et al. 2009 ) and underwater ship inspections (Zheng et al. 2023 ). Some studies have concentrated on methodologies that leverage knowledge graphs to create semantic data models for shaping digital twins (Waszak et al. 2022 ).

Furthermore, the evolving flexibility and customization in futuristic smart manufacturing are closely linked with human intelligence. For example, in human–robot collaboration (HRC) tasks aimed at improving human ergonomics (Steinmetz et al. 2022 ), operators can adjust robot poses through gesture-based interactions with the robot's digital twin. After receiving instructions from human operators, the robot digital twin learns to perform better and meet human needs. Additionally, in the timber prefabrication process (Dimitropoulos et al. 2021 ), AR provides effective interaction methods to enhance mutual understanding between operators and collaborative robots, ultimately facilitating harmonious task sharing.

4.7 Classification of digital twins by scale

Digital twins can be categorized into various types based on their scale and comprehensiveness, including component, asset, system, and process twins (Amtsberg et al. 2021 ).

Component twins : This approach suits large, complex digital twins. The adaptation and uncertainty quantification of the model in such applications can be framed as a Bayesian state estimation problem. Here, data from the physical world is used to infer which models from a model library best represent the digital twins. This approach strategically selects specific components for replication in the digital twins to avoid data redundancy and reduce costs. Microsoft has developed the Azure Digital Twins (ADT) platform (Cinar et al. 2020 ), facilitating model creation and offering a graph API for querying and interacting with these digital twins. The ADT platform enables users to visualize and examine the relationships among components, such as creating 3-D digital twins of a factory with a user-friendly interface. This interface allows operators to monitor the state of each machine. A notable challenge in this scenario involves loading each 3-D object instance into the scene. Repeated loading of the same object in different locations can lead to inefficiencies.

To address this, future developments in component twins could involve a system where a single instance of a 3-D object is streamed, loaded into memory, and rendered multiple times as needed. This approach would optimize the handling of 3-D objects in digital twin environments, enhancing efficiency and reducing the computational load.

Asset twins : This methodology focuses on creating data-driven digital twins using a library of physics-based reduced-order models. When a single model library is shared among numerous assets, this approach can effectively scale to applications requiring a substantial number of digital twins (Krzyczkowski 2019 ). Asset twins involve an estimation process wherein online sensor data from a physical asset determines which models from the library should be integrated into the digital twin. Future advancements in asset twins should enhance the robustness of model selection, particularly in the context of corrupted data. Implementing mechanisms to improve robustness and incorporating various damage models to detect and classify actual asset damage is also essential. GE Healthcare (Kapteyn et al. 2020 ) has noted the application of asset twins in healthcare, addressing challenges such as staffing model design and surgical block schedule optimization.

System twin : Operating at a higher level, system twins amalgamate different assets to form a complete functional system, such as a vehicle's brake system (Aghdam et al. 2021 ). These twins offer insights into asset interactions, thereby augmenting overall performance.

Process twin : Process twins utilize high-performance computing to optimize equipment and manufacturing processes. This is achieved by integrating multidimensional process knowledge models (Aghdam et al. 2021 ). Manufacturers can attain unparalleled efficiency and deeper insights by combining production processes with economic considerations.

Application : A digital twin system integrating Virtual Reality (VR) and Artificial Intelligence (AI) technologies has been developed to monitor and analyze welder behavior. This system exemplifies the practical application of digital twin technology in understanding and improving specific work processes.

5 Datasets, data models, and software for developing digital twins

The transformation of physical assets into digital twins involves an in-depth asset data collection process, which is then utilized to form an exact digital counterpart. This procedure is essential for asset management and predictive maintenance. There are variant data models and datasets used to underpin the digital twin initiative and significantly enhance the effectiveness and capabilities of digital twin implementations while reducing development efforts and optimizing the total cost of ownership. Many software applications have recently been used to create and manage digital twins. This section presents samples of Data models, Datasets, and software applications.

5.1 Smart city data models and datasets

To illustrate the potential of digital twins in smart cities, let us consider examples of digital twin data models and datasets that provide valuable insights for urban planning and management. Digital twin data can be applied in both tangible and virtual realms. These data are pivotal for asset monitoring, operational optimization, and safety enhancement in physical settings. On the other hand, virtual landscapes enable realistic simulations, training endeavors, and strategic planning. This dual use of digital twins highlights their adaptability, effectively bridging the real and digital domains.

One of the cornerstones of DT design and development is modeling data. Data originate from heterogeneous sources, use various protocols, and include their own data attributes, attribute types, and relationships. In order to ensure interoperability, it is necessary not only to standardize the communication between DT components but also to standardize the data format that flows through these components.

3D city modeling transcends mere data acquisition and processing, extending into data management, storage, and exchange. Consequently, open and standardized data models and exchange formats are essential for 3D city modeling. CityGML and its streamlined counterpart, CityJSON (Ledoux et al. 2019 ), are the most established data formats for 3D city models. These formats facilitate representations ranging from basic to richly detailed, depending on the required level of detail (LoD). The building model is depicted in five levels of detail, from LOD0 to LOD4, with higher LoDs offering more detail and accuracy. The aim is to manage the complexity of 3D models effectively.

In their study, the authors (Lei et al. 2022 ) assess 40 authoritative 3D city models that have emerged since 2013. This evaluation yields both quantitative and qualitative insights. The framework developed offers a thorough and structured comprehension of the landscape of semantic 3D geospatial data while also serving as an evaluated compilation of open 3D city models.

In (Ledoux et al. 2019 ), digital twin (DT) initiatives in cities are classified based on the nature of their digital replicas (static or dynamic, i.e., incorporating sensor or IoT data) and the extent of data integration (the data connection between the physical and digital worlds). Various static datasets utilize digital model integration, including Helsinki 3D + , Espoo, Vienna, Zurich 3-4D, and Amsterdam3D. Meanwhile, dynamic datasets such as Digital Twin Munich, Rennes 3D, Virtual Gothenburg, and Sofia-Bulgaria employ digital shadow integration. Furthermore, dynamic datasets like DUET, Fishermans, and Virtual Singapore implement digital twin integration. It can be inferred that most initiatives are digital shadows, given that data connections from the real world to the digital copy are automated. At the same time, the reverse typically involves manual processes (human interventions adapting the physical world). This bidirectional connection warrants further exploration.

Many research projects and similar initiatives mainly focus on collecting and providing IoT data generated from smart cities. For example, the ODAA platform ( 2016 ) Footnote 1 provides open access to data collected from the City of Aarhus using IoT infrastructure deployed within the city. The datasets within the ODAA are categorized across various applications, including energy, population and society, transport, education, and more. Moreover, San Francisco Open Data ( 2024 ) Footnote 2 and the City of Chicago Data Portal ( 2024 ) Footnote 3 provide a centralized collection of relevant smart city datasets that are publicly accessible.

For example, the NYC Open Data Initiative has already leveraged digital twin technology to improve urban planning and citizen engagement. By providing access to a wide range of open data, including information on infrastructure, public services, and environmental factors, the initiative has empowered citizens to actively participate in shaping the city's future.

5.2 Software for digital twin creation and management

Numerous digital twin software applications are available for creating and managing digital twins in buildings, cities, and urban systems. Some notable examples include:

Autodesk revit (Autodesk 2019 ): This software is extensively used for Building Information Modeling (BIM) and is acclaimed for its comprehensive design, documentation, and collaboration tools. It enables architects, engineers, and construction professionals to create detailed 3D models and provides extensive data for informed decision-making throughout a building's lifecycle.

Esri cityengine (Badwi et al. 2022 ): CityEngine is a robust software tool for crafting 3D city models. It is utilized by urban planners and designers to generate detailed and lifelike representations of cities, offering capabilities for cityscape generation, urban environment modeling, and simulation of various urban scenarios. It also integrates with GIS data to enhance city models with geographic information and analysis.

Bentley systems openbuildings designer (Mainisa et al. 2023 ): This BIM software provides advanced building design and construction modeling tools. Architects, engineers, and construction professionals use it for detailed 3D modeling, structural analysis, and effective collaboration throughout the building lifecycle.

Unity reflect (Nämerforslund 2022 ): Unity Reflect is a platform that creates interactive and immersive experiences with digital twins. It supports real-time, high-fidelity 3D modeling for virtual and augmented reality environments, enhancing visualization, interaction, and decision-making processes.

Siemens city performance tool (Al-Obaidy et al. 2022 ): Specifically tailored for urban planning and management, this tool offers a comprehensive platform for analyzing and optimizing the performance of urban systems.

iLens from knowledge lens : This leading Industrial IoT solution addresses Industry 4.0 needs with capabilities in Interface Connectivity, Edge Computing, Monitoring and Control, and Predictive Analytics. iLens is powering diverse industries globally, including Automation, Manufacturing, Energy, and Utilities.

Iotics : Iotics' innovative digital twin technology enables seamless communication across an entire digital ecosystem. It bridges gaps between various entities, from sensors to power stations and individual trains to entire airplane networks, transcending organizational boundaries and differing data languages while maintaining security.

Kavida.ai : This supply chain digital twin platform assists enterprises in making intelligent resiliency decisions. It builds supply chain digital twins using artificial intelligence to help enterprises prevent and mitigate disruptions in real time or before they occur.

MODS reality : This cloud-based application hosts a digital twin of a facility in a point cloud environment, enhancing engineering and streamlining scheduling and work execution management for maintenance and minor modifications, thereby maximizing performance and profitability.

Twinzo : As a mobile-first live digital twin platform focused on operational excellence, twinzo visualizes and reconstructs live data in 3D, offering novel ways to analyze and consume information. It helps customers save significant operational costs and increase production output.

VEERUM's digital twin : This application is a leading visualization and analytics tool that combines CAD, geospatial, document management, IoT, and operational systems. It delivers considerable cost and time savings in operations, maintenance, reliability, and complex capital construction projects.

WillowTwinTM : Revolutionizing the built world, WillowTwinTM is a pioneering software platform for real estate and infrastructure assets. It provides a central hub for all asset data, turning siloed datasets into a virtual replica of the built form. The platform enables proactive, data-driven decision-making in real-time to reduce costs, increase profits, and manage risks.

6 Digital twin performance metrics

Extensive research has been conducted on digital twins (DTs) and their applications, yet a standard method for assessing DT performance remains elusive. Establishing a method for evaluating the performance of DTs is essential for enhancing or monitoring processes and systems within a business context. Such a method could guide researchers and practitioners in developing more effective digital twins (Psarommatis and May 2022 ).

There have been limited studies focusing on specific methodologies for assessing DT performance. Chen et al. (Chen et al. 2021 ) proposed a DT maturity model for managing industrial assets based on Gemini principles, facilitating quantitative evaluation of DT flexibility and implementation levels. Chakraborty et al. (Chakraborty and Adhikari 2021 ) assessed DT performance in a multi-time scale dynamical system using an efficient framework that leverages expectation maximization and a sequential Monte Carlo sampler for developing machine learning-based DTs. Shangguan et al. (Shangguan et al. 2020 ) evaluated DT performance for fault diagnosis using a predefined threshold technique, focusing on accuracy (ACC), specificity (SPE), and sensitivity. Psarommatis et al. (Psarommatis and May 2022 ) introduced a systematic approach for measuring DT performance and flexibility, quantifying it based on four key performance indicators (KPIs). Additionally, they introduced DTflex as a new KPI to evaluate the flexibility of digital twins.

6.1 Performance metrics categories

Although there are no well-established methods or Key Performance Indicators (KPIs) in the field for thoroughly assessing the performance of Digital Twins (DT), this study suggests classifying performance metrics according to three essential elements: software, hardware, and data management middleware. This paradigm makes it possible to evaluate the system's efficacy in detail. A thorough analysis of the body of prior research and industry norms guided the choice of these indicators. We aimed to find measures that captured the essential elements of DT performance by combining knowledge from several sources.

The proposed metrics ensure an adequate evaluation by focusing on DT performance characteristics within each component. For example, metrics about hardware components evaluate attributes like scalability, communication dependability, and sensor precision. These metrics were selected to represent the fundamental hardware performance features essential to DT's operation. Similarly, metrics related to middleware for data management emphasize security, scalability, and efficiency, highlighting middleware's vital role in integrating and controlling data streams. Finally, software component metrics highlight the significance of strong software functions for DT performance by addressing factors such as model integrity, simulation accuracy, and user interface responsiveness. Each metric recommended in this section is supported by its relevance to real-world DT implementations and alignment with broader business or operational objectives. These measures help stakeholders make well-informed decisions by offering practical insights about DT performance. Including these measures also attempts to create a standard framework for assessing DT performance in various applications and domains.

6.1.1 Hardware components

Sensor accuracy: Precision and reliability of physical sensors.

Communication reliability: Efficiency of data transmission between sensors and the digital counterpart.

Hardware scalability: Ability to expand hardware components with increasing data volumes.

Latency in data acquisition: Time taken to acquire and transmit sensor data.

Hardware failure rate: Frequency and severity of failures in sensors or actuators.

6.1.2 Data management middleware

Data integration efficiency: Ease of integrating data from various sources into the DT.

Middleware latency: Time taken for middleware processes to complete tasks.

Data accuracy and consistency: Precision and consistency in data storage and management by middleware.

Scalability of middleware: Ability to handle increasing data volumes without performance degradation.

Data security protocols: Effectiveness of security protocols in protecting data during storage and transit.

6.1.3 Software components

Model fidelity: Accuracy and completeness of the digital model representing the entity.

Simulation accuracy: Precision of simulations compared to real-world scenarios.

Quality of visualization: Clarity and detail of visual representations in the user interface.

User interface responsiveness: Speed and responsiveness of the software interface to user actions.

IoT device integration: Compatibility and integration with various IoT devices.

Scalability of software: Capacity to handle increasing computational loads and data processing demands.

Software security: Protections against cyber threats and unauthorized access.

Interactivity and control: Responsiveness of software to user inputs and control commands.

Updating and maintenance efficiency: Ease of updating and maintaining software components.

Effectiveness of decision support: Capability of the software to provide meaningful insights.

6.2 Best practices for evaluating digital twin performance

As noted by Peter Drucker, Mgt. consultant and author, “You cannot manage what you cannot measure.” This principle is equally applicable to digital twins. The confusion matrix employed in data science can measure digital twins' performance ARC Advisory Group ( 2024 ). Footnote 4 Assessing the performance of digital twins necessitates a thorough approach that considers multiple aspects, including hardware, data management middleware, and software components. Below are some essential practices for effectively assessing the performance of digital twins. The formulation of these best practices necessitated a thorough examination of the current literature on DT performance evaluation. Consultations with some stack holders were also conducted. By combining information from various sources, we hoped to convey the multidimensional nature of DT performance and provide meaningful advice to practitioners and researchers alike. Furthermore, the methods were iteratively refined to ensure their usefulness and applicability across various situations and industries.

Each practice recommended in this section is based on known management principles and its ability to address important difficulties in DT performance evaluation. For example, the emphasis on objective definition and particular Key Performance Indicators (KPIs) demonstrates the significance of goal alignment and measurement precision in achieving effective DT efforts. Similarly, data quality, security assessment, and scalability analysis methods emphasize these variables' importance in assuring the dependability and efficacy of distributed computing systems.

Objective definition: Clearly articulate the goals and objectives of the digital twin implementation to align performance indicators with broader business or operational objectives.

Establish specific KPIs: Identify and set specific Key Performance Indicators (KPIs) that align with the objectives, ensuring they are measurable, relevant, and linked to desired outcomes.

Multidimensional evaluation: Assess performance across multiple dimensions, including accuracy, responsiveness, scalability, security, and usability.

Regular review and update of metrics: Given the evolving nature of digital twin environments, performance metrics should be regularly reviewed and updated to maintain relevance and accuracy.

Focus on data quality and integrity: Emphasize metrics related to data accuracy, consistency, and integrity, as the quality of the digital twin largely depends on the reliability of its data.

Incorporate end-user experience metrics: Include metrics that gauge user satisfaction and adoption, such as visualization quality, interaction, and ease of use.

Measure latency and responsiveness: Evaluate latency in data collection, middleware processing, and software responsiveness to ensure real-time or near-real-time capabilities.

Security performance assessment: Implement metrics to evaluate the efficacy of security measures, including data encryption protocols.

Scalability analysis: Examine the digital twin's scalability, focusing on how well it accommodates increasing data volumes, user numbers, and processing requirements.

Simulation accuracy verification: Regularly validate the accuracy of simulations and virtual representations against actual world scenarios to ensure the digital twin's reliability.

Benchmarking: Compare performance against industry standards or best practices to understand how the digital twin stacks up against similar implementations.

Utilize monitoring technologies: Deploy monitoring technologies that offer real-time insights into the digital twin's operation, enabling proactive issue identification and resolution.

Develop a continuous improvement process: Establish a process for continuous improvement that integrates user feedback and ongoing evaluations, fostering a culture of perpetual enhancement.

By adhering to these practices, organizations can establish a robust framework for assessing and improving the performance of their digital twins, ensuring that these technologies deliver maximum value and effectively contribute to strategic objectives. To sum up, this section's performance indicators are the outcome of a systematic approach guided by academic research and industry observations. We hope to give readers a thorough grasp of how these measures support efficient DT evaluation procedures by outlining the reasoning behind their selection and their applicability to DT performance assessment.

7 Challenges associated with digital twins

Understanding the obstacles encountered while deploying digital twin technology is critical for its successful adoption and improvement. This section elucidates the difficulties various components of digital twin systems face, shedding light on their origins and implications. The challenges outlined are meticulously identified through an extensive review of literature and insights from field and industry experts, signifying their significance in the successful deployment and operation of digital twin systems. This analysis integrates multiple sources to pinpoint these hurdles as key challenges. The study organizes the identified challenges into three main aspects of digital twin technology: hardware, data management middleware, and software. This categorization facilitates a thorough understanding of the complex problems impacting different aspects of digital twin systems. A thorough examination of these challenges across the hardware, data management middleware, and software components aids in bridging the current research gap. Whereas prior studies often discussed these challenges in broad strokes, (Tuhaise et al. 2023 ) divided them into three categories: data transmission, interoperability, and data integration. This research details specific problems within each distinct component of the digital twin framework, thereby offering an in-depth analysis of the inherent obstacles in digital twins. It identifies hardware-related challenges, such as the complexity of sensor integration and issues with hardware reliability, suggesting solutions like adopting standardized sensor interfaces and employing predictive maintenance strategies. Furthermore, the study uncovers problems in data management middleware, including data integration bottlenecks and interoperability issues, recommending developing scalable middleware systems and adopting universal standards to enhance interoperability. The research outlines security vulnerabilities and algorithmic complexity regarding software components, proposing using advanced analytical tools and robust cybersecurity measures as solutions.

By delineating these issues across hardware, middleware, and software components, the study enhances the understanding of digital twin technology and offers actionable recommendations for enhancing the technology’s effectiveness and resilience. As digital twin technology continues to evolve, the findings underscore the necessity of concentrating on these components to surmount challenges and fully exploit the technology's potential across various applications and industries. The examination of digital twin elements and their associated challenges is visually summarized in Fig.  13 , which consists of three parts: (a) delineates the components of a digital twin, (b) identifies the challenges specific to each component, and (c) proposes solutions to these challenges.

Overview of digital twin components, associated challenges, and proposed solutions. Part ( a ) delineates the core components of a Digital Twin (DT). In part ( b ), a detailed breakdown highlights the challenges inherent in each component. Part ( c ) provides insightful solutions strategically proposed to address these challenges and enhance the effectiveness of Digital Twin implementation

7.1 Hardware components

Hardware components are the foundation of digital twin systems, comprising sensors, actuators, and other physical devices. Challenges within this component include:

Sensor integration complexity: Integrating diverse sensors for real-time data poses compatibility and synchronization issues.

Hardware reliability: Ensuring long-term sensor and actuator reliability is essential.

Proposed solutions involve adopting standardized sensor interfaces and implementing predictive maintenance strategies to mitigate these challenges.

7.2 Data management middleware

Middleware plays a crucial role in managing and processing the vast amount of data generated by digital twin systems. Challenges within this component include:

Data integration bottlenecks: Handling diverse data streams can lead to processing delays.

Interoperability issues: Different standards may hinder middleware system compatibility.

Proposed solutions include developing scalable middleware architectures and embracing industry-wide standards for improved interoperability.

7.3 Software components

Software components encompass the algorithms and analytical tools for real-time data analysis and decision-making. Challenges within this component include:

Algorithmic complexity: Complex algorithms for real-time analytics and decision-making need streamlining.

Security vulnerabilities: Software components are susceptible to cybersecurity threats.

Proposed solutions involve utilizing advanced analytical tools and robust cybersecurity protocols to address these challenges.

In conclusion, addressing the challenges linked with digital twins requires a deep understanding of their core components: hardware, data management middleware, and software. This analysis has unveiled various obstacles, from hardware constraints to data integration complexities and software interoperability challenges. A comprehensive perspective is provided by examining these issues across the distinct hardware, middleware, and software components. It is essential to identify and tackle the limitations associated with hardware, the challenges within middleware, and the issues related to software interoperability to enhance the efficiency and robustness of digital twin systems. As digital twin technology evolves, prioritizing these areas will be critical for navigating difficulties and leveraging the technology’s capacity in diverse applications and industries.

8 Case studies

Case studies in the realm of smart cities and digital twins serve as vital illustrations of these technologies in practical scenarios:

Dubai's "Happiness Agenda": A smart city initiative using big data to enhance urban living and measure "happiness" across various criteria. The objective was to involve every citizen in shaping future cities, particularly focusing on citizen engagement. Dubai’s "Happiness Agenda" implementation represents a notable example of a smart city involving its residents in urban development. Dubai has positioned itself as one of the "happiest" places to live by defining citizen "happiness" across multiple criteria. It uses big data analysis to allocate urban resources strategically, enhancing the city's overall "Happiness Index" (Zakzak 2019 ).

West Cambridge site and IFM building: These case studies explore adaptable digital twins at the building level, integrating various data sources and AI-driven decision-making. The West Cambridge site of the University of Cambridge in the UK was chosen as a case study due to its diverse facilities, which include university buildings, sports centers, residence areas, main roads, parking places, and restaurants. This variety allows for testing and evaluating the proposed dynamic digital twin system across different types of infrastructure. Additionally, the site's size and complexity offer an ideal environment to assess the effectiveness of the technology. Access to extensive data sources, collaboration opportunities with experts, and relevance to the academic community further contribute to its suitability as a testbed for the study (Qiuchen Lu et al. 2019 ).

Herrenberg, Germany: A case study demonstrating the use of digital twin technology in urban planning and city management. The case study of Herrenberg might illustrate the implementation and benefits of digital twins in improving urban planning, infrastructure management, and citizen engagement within the city. Herrenberg was selected as a case study for the digital twin due to its relevance to urban challenges, accessibility of diverse data sources, the potential for collaboration with local stakeholders, engagement of the community, and suitability in terms of size and complexity for testing the digital twin technology (Dembski et al. 2020 ).

Cambridge Sub-region: A digital twin pilot is developed, integrating diverse data streams for urban planning and decision-making. The authors stress the significance of including diverse data like IoT sensors, satellite images, social media, and government records to ensure an all-encompassing and precise city representation. The case study presented in the paper is about developing a digital twin pilot for the Cambridge Sub-region. It highlights how integrating various data streams and simulation models can assist urban planning, resource allocation, and decision-making processes. The case study provides insights into the potential benefits of using a city-level digital twin for improving efficiency, sustainability, and resilience in urban environments (Wan et al. 2019 ).

Málaga City: Implementing cognitive analytics in smart city management to enhance transportation, energy, and public services. The focus is on enhancing various aspects of urban life, such as transportation, energy management, waste management, and public services. The case study of Málaga City demonstrates the practical implementation of cognitive analytics to improve decision-making processes, optimize resource allocation, and ultimately enhance the quality of life for its residents (Pérez and Toledo 2017 ).

Ålesund, Norway: The study explores the role of a data-driven digital twin in enhancing urban systems and services within a smart city framework. It suggests using high-quality 3D graphical digital twins (GDTs) of cities to generate 4D visualizations of geolocalized time-series data to enhance citizen engagement. Through a case study conducted in Ålesund, Norway, the methodology utilizes readily available hardware and a game engine to develop immersive environments for presenting complex data sourced from GIS, BIM, demographics, and IoT. The approach emphasizes scalability, transferability, versatility in data integration, adherence to privacy regulations, and dependable data delivery. The paper introduces a pioneering smart city GDT framework, which capitalizes on interactive features and advancements in metrology (Major et al. 2021 ).

Case study in Greece: Details the development and application of digital twins tailored for smart cities, focusing on urban infrastructure improvements. The case study probably illustrates how digital twins optimize city systems, improve efficiency, and facilitate decision-making processes in Greek urban environments. This study might showcase practical examples of implementing digital twin technology to address challenges and enhance the overall functioning of a smart city in Greece (Evangelou et al. 2022 ).

Each case study offers unique insights into the deployment and impact of digital twin technology in various urban settings, highlighting its potential to improve city management and living standards. Table 2 offers an overview of each paper's focus areas, case studies, and key highlights, showcasing their distinct contributions and applications in the field of digital twins in smart cities.

9 Smart city governance in the era of digital twins: addressing challenges and leveraging opportunities

In the evolving discourse on smart cities and digital twin technologies, a critical examination of multi-level governance, organizational practices, and governance dimensions emerges as pivotal. The collective contributions from the referenced studies provide a comprehensive overview of the challenges and strategies in implementing smart city initiatives across different governance frameworks and geographical contexts.

As examined in one study, the integration of Chinese new authoritarian principles into smart government transitions highlights the inherent tensions between state-level directives and local-level implementation, underscoring the complexity of multi-level governance in authoritarian regimes (Zhang and Mora 2023 ). This perspective is enriched by a nuanced exploration of organizational practices within smart city development, revealing how bureaucratic, technocratic, and participatory logics intersect to shape decision-making and citizen engagement in smart city projects (Mora et al. 2023a ). Furthermore, the identification of three key governance dimensions—institutional context for urban innovation, urban innovation ecosystem, and urban digital innovation—provides a framework for understanding the governance mechanisms essential for fostering smart city transitions (Mora et al. 2023b ).

Critical analysis across the studies reveals common challenges in smart city governance, such as interoperability and compatibility issues within the digital ecosystem and integrating a technological dimension in urban development. These challenges underscore the importance of addressing interoperability and compatibility to enhance city planning and management effectively (Quek et al. 2023 ). The discourse extends to the critical analysis of smart urbanism in non-Western contexts, notably in India and Africa, where issues of urban informality, equity, and the inclusivity of smart city initiatives are brought to the forefront (Prasad et al. 2023 ; Tonnarelli and Mora 2023 ). These analyses highlight the necessity of adopting equitable and inclusive smart city development approaches that consider the needs and priorities of all urban dwellers, particularly marginalized communities.

Moreover, the call for empirical studies and the integration of innovation management theory into smart city governance research emphasizes the need for practical guidance and theoretical advancements in managing urban digital innovation (Mora et al. 2023b ). The exploration of human-cyber-physical interactions further illuminates the evolving relationship between technology, governance, and societal dynamics, advocating for a holistic approach that balances technological advancements with ethical and sociocultural considerations (Quek et al. 2023 ).

In conclusion, the amalgamated insights from these studies advocate for a pragmatic, contextually informed, and inclusive approach to smart city governance. By addressing the multifaceted challenges of interoperability, governance, and citizen engagement, and by incorporating a critical perspective on urban informality and inclusivity, this body of work contributes significantly to the scholarly discourse on smart cities and digital twins. The emphasis on empirical research, innovation management, and the integration of technology in urban development underscores the dynamic interplay between technology, governance, and urban development strategies in the quest for sustainable and equitable urban futures.

9.1 Role of DT in smart city governance

Smart city governance constitutes a complex framework fundamental to the effective realization and long-term viability of smart city endeavors. It encompasses the strategic alignment of policies, technological systems, and multifaceted collaborations amongst stakeholders by overarching urban development goals. Digital twin technology plays a pivotal role in enhancing smart city governance by offering innovative solutions across various components:

Policy and strategy formulation : Crafting policies and strategies that guide smart city initiatives in service of the city's broader objectives (Beckers 2022). Digital twins assist in crafting policies and strategies by providing valuable insights derived from real-time data and simulations. City authorities can utilize digital twins to assess the impact of different policies and strategies on urban systems, enabling informed decision-making aligned with broader city objectives.

Collaborative ecosystem : Fostering partnerships spanning government entities, the private sector, academic institutions, and the citizenry, thus leveraging collective knowledge and resources (Beckers 2022). Digital twins foster collaboration among government agencies, private sector entities, academic institutions, and citizens by providing a platform for data sharing and analysis. This collaborative ecosystem enhances collective knowledge and resource utilization, facilitating more effective governance practices and co-creating solutions to urban challenges.

Technological infrastructure : Establishing and administering the technological basis, encompassing data management and digital platforms, that underpins smart city operations (Zhang and Mora 2023 ). As a foundational element of smart city operations, digital twins contribute to establishing and managing the technological infrastructure required for governance. They enable comprehensive data management and visualization, empowering city administrators to monitor urban systems, identify emerging trends, and respond proactively to issues in real-time.

Ethical considerations : Prioritizing ethical concerns by safeguarding data privacy and security and ensuring the equitable deployment of technology (Mora et al. 2023a ). Digital twins support ethical governance by prioritizing data privacy, security, and equitable technology deployment. Through robust data encryption protocols and access controls, digital twins safeguard sensitive information, ensuring that governance processes remain transparent, accountable, and inclusive for all stakeholders.

Public participation : Stimulating citizen involvement in the governance process promotes transparency and inclusiveness (Mora et al. 2023b ). Digital twins facilitate public participation by providing accessible platforms for citizen feedback, collaboration, and co-design of urban solutions. By incorporating citizen inputs into decision-making processes, digital twins help ensure that governance strategies align with community needs and preferences.

Sustainability : Championing sustainable development practices integrated within smart city projects to prioritize environmental stewardship and long-term resilience (Quek et al. 2023 ). By simulating various scenarios and assessing the environmental impact of proposed policies and projects, digital twins enable city authorities to prioritize sustainability and resilience in urban planning and decision-making processes.

9.2 Smart city governance challenges

The pursuit of smart city objectives is frequently hindered by governance crises, underscoring the complexities of managing urban digital transformations. Digital twins offer innovative solutions to navigate the complexities of urban governance and enhance decision-making processes. Here, we explore how digital twins can be utilized to tackle key challenges in smart city governance:

Data privacy and security concerns: Contending with data privacy and security risks associated with the vast collection and storage of urban data (Mora et al. 2023a ). Digital twins incorporate robust data encryption protocols and access controls, ensuring the protection of sensitive information within smart city systems. Digital twins help mitigate privacy and security risks associated with urban data collection and storage by enabling secure data management and transmission.

Digital divide and inequity: Mitigating the digital divide can potentially intensify social disparities within urban communities (Prasad et al. 2023 ). Digital twins promote inclusivity and bridge the digital divide by providing accessible platforms for citizen engagement and participation in governance processes. Through user-friendly interfaces and interactive visualization tools, digital twins empower all citizens to contribute to decision-making, regardless of their technological literacy or socioeconomic status.

Regulatory and legal challenges: Navigating the disparity between the swift pace of technological progress and prevailing regulatory frameworks (Tonnarelli and Mora 2023 ). Digital twins assist city authorities in navigating regulatory and legal frameworks by providing comprehensive data analytics and scenario modeling capabilities. Digital twins facilitate informed policy-making and ensure alignment with legal standards and industry regulations by simulating the impact of proposed regulations and assessing compliance requirements.

Fragmented governance structures: Surmounting the intricacies of multi-stakeholder governance structures, which can obstruct coordinated action (Zhang and Mora 2023 ). Digital twins serve as centralized platforms for data integration and collaboration, overcoming the challenges of fragmented governance structures. By consolidating diverse datasets from multiple stakeholders and domains, digital twins enable seamless information sharing and coordination, fostering synergy among various governmental entities and stakeholders.

Resource constraints: Confronting limitations in financial, technical, and operational capacities is vital to the success of smart city ventures (Quek et al. 2023 ). Digital twins optimize resource utilization and operational efficiencies within smart city governance through predictive analytics and optimization algorithms. By identifying inefficiencies and optimizing resource allocation, digital twins help cities overcome resource constraints and maximize the impact of limited financial, technical, and operational resources.

By integrating digital twin technologies, smart city administration can address these challenges, paving the way for innovative solutions and sustainable urban development. Digital twins offer a comprehensive and data-driven approach to governance, enabling cities to enhance decision-making processes, accountability, and transparency, ultimately enhancing the quality of life for urban residents. While digital twin technologies have the potential to significantly improve urban management through advanced data analytics, simulation, and optimization, their seamless integration into smart city governance requires careful consideration of governance issues. This includes addressing concerns related to data privacy, fostering collaboration among stakeholders, and upholding ethical principles.

10 Conclusions and future research directions

This survey paper employs a meticulous bibliometric methodology, selecting the Web of Science database for its comprehensive coverage and developing precise search criteria to gather over 4,220 relevant articles. The analysis uses advanced tools like VOSviewer for network analyses and visualizations, including co-authorship and keyword co-occurrence maps, enabling a detailed examination of trends and relationships in Digital Twin technology and Smart Cities research. This methodological rigor ensures the study's reliability and contributes to its uniqueness in the field. This survey comprehensively reviews over 4200 publications in the domain of Digital Twins and Smart Cities. It outlines the evolution, applications, and integration of Digital Twins with IoT and AI in urban development. The survey distinguishes itself through extensive bibliometric analysis, focusing on datasets, platforms, software, and performance metrics, and it offers unique insights into the challenges and opportunities within the field. The findings include emerging trends, key thematic areas, and a detailed exploration of various Smart City applications. The paper concludes with implications for urban developers, policymakers, and researchers and recommendations for future research directions. The field of Digital Twin (DT) and Smart Cities is ripe for future research, aiming to overcome current challenges and explore new frontiers. Detailed investigation and development in this area are essential for realizing the full potential of DT technologies in urban environments. The discussions pave the way for sustainable and equitable urban futures, recognizing the dynamic interplay between technology, governance, and urban development strategies.

Future research should focus on:

Enhanced data integration : Developing more efficient methods for integrating diverse data sources within DT systems.

Scalability solutions : Creating scalable DT models suitable for larger and more complex urban environments.

Advanced security protocols : Strengthening cybersecurity measures for DT systems to ensure data privacy and security.

Sophisticated analytical tools : Incorporating cutting-edge AI and machine learning techniques for predictive analytics and decision-making.

Expanding IoT capabilities : Extending the use of IoT in DTs for comprehensive real-time data collection and monitoring.

Sustainable urban development : Leveraging DTs for resource management, focusing on sustainability and environmental conservation.

Citizen engagement models : Developing DTs prioritizing citizen involvement in urban planning and management.

Policy and governance studies : Examining the influence of policy in guiding DT implementation and addressing ethical concerns.

Economic impact assessment : Evaluating the economic implications of DTs, including cost analysis and return on investment.

Real-world case studies : Documenting extensive case studies to assess DTs' practical impact and challenges in urban settings.

Investigating future technological advancements: new applications, and the role of policy and governance in Digital Twins development.

The findings of this paper are poised to influence future research, policy-making, and practical applications in Smart Cities and Digital Twins in significant ways:

Informing future research directions: The comprehensive review of over 4,200 publications provides valuable insights into the current state of Digital Twins and Smart Cities research. Researchers can utilize this information to identify gaps in existing literature and prioritize areas for further investigation. For example, identifying challenges such as data integration bottlenecks and security vulnerabilities can guide future research efforts toward developing solutions to these pressing issues.

Guiding policy development: Policymakers can leverage the findings of this paper to inform the development of policies and regulations related to Digital Twin technology and its application in Smart Cities. By understanding the challenges and opportunities associated with Digital Twins, policymakers can create frameworks that promote innovation while addressing data privacy, cybersecurity, and ethical considerations.

Improving urban planning and management: The insights provided by this paper can assist urban planners and city managers in making informed decisions about adopting and implementing Digital Twins in Smart Cities. By understanding Digital Twin technology's potential benefits and challenges, city officials can develop strategies to optimize urban infrastructure, improve resource management, and enhance citizen services.

Driving technological innovation: The paper identifies emerging trends and technological advancements in Digital Twin technology, such as the integration of AI and IoT, as well as the development of scalable models and advanced security protocols. These insights can inspire innovation in academia and industry, leading to the development of new tools, platforms, and solutions that push the boundaries of Digital Twin technology and its applications in Smart Cities.

Finally, the findings of this article have the potential to spark advances in research, policymaking, and practical applications connected to Digital Twins and Smart Cities, resulting in more efficient, sustainable, and resilient urban development.

Data availability

Data is provided within the manuscript.

http://www.odaa.dk

https://data.sfgov.org

https://data.cityofchicago.org

https://www.arcweb.com/industry-best-practices/measuring-digital-twin-performance-maturity-confusion-matrix

Abdeen FN, Shirowzhan S, Sepasgozar SME (2023) Citizen-centric digital twin development with machine learning and interfaces for maintaining urban infrastructure. Telematics Inform 84:102032. https://doi.org/10.1016/j.tele.2023.102032

Article   Google Scholar  

Abideen AZ, Sundram VPK, Pyeman J, Othman AK, Sorooshian S (2021) Digital twin integrated reinforced learning in supply chain and logistics. Logistics 5(4):84. https://doi.org/10.3390/logistics5040084

Abouzid I, Saidi R (2023) Digital twin implementation approach in supply chain processes. Sci Afr 21:e01821. https://doi.org/10.1016/j.sciaf.2023.e01821

Aghdam ZN, Rahmani AM, Hosseinzadeh M (2021) The role of the internet of things in healthcare: future trends and challenges. Comput Methods Programs Biomed 199:105903. https://doi.org/10.1016/j.cmpb.2020.105903

Aheleroff S, Zhong RY, Xu X, Feng Z, Goyal P (2020) Digital twin enabled mass personalization: a case study of a smart wetland maintenance system. Volume 2: manufacturing processes; manufacturing systems; nano/micro/meso manufacturing; quality and reliability. https://doi.org/10.1115/msec2020-8363

Akroyd J, Harper Z, Soutar D, Farazi F, Bhave A, Mosbach S, Kraft M (2022) Universal digital twin: land use. Data-Centric Eng 3. https://doi.org/10.1017/dce.2021.21

Alam KM, El Saddik A (2017) C2PS: a digital twin architecture reference model for the cloud-based cyber-physical systems. IEEE Access 5:2050–2062. https://doi.org/10.1109/access.2017.2657006

Allam Z, Jones DS (2021) Future (post-COVID) digital, smart and sustainable cities in the wake of 6G: digital twins, immersive realities and new urban economies. Land Use Policy 101:105201. https://doi.org/10.1016/j.landusepol.2020.105201

Al-Obaidy AHMJ, Khalaf SM, Hassan FM (2022) Application of CCME index to assess the water quality of tigris river within Baghdad City, Iraq. IOP Conf Ser: Earth Environ Sci 1088(1):012004. https://doi.org/10.1088/1755-1315/1088/1/012004

Amtsberg F, Yang X, Skoury L, Wagner H-J, Menges A (2021) iHRC: an AR-based interface for intuitive, interactive and coordinated task sharing between humans and robots in building construction. Proceedings of the International Symposium on Automation and Robotics in Construction (IAARC). https://doi.org/10.22260/isarc2021/0006

ARC Advisory Group (2024) Measuring digital twin performance maturity with the confusion matrix. Retrieved from https://www.arcweb.com/industry-best-practices/measuring-digital-twin-performance-maturity-confusion-matrix . Accessed 10 May 2024

Ariyachandra MRMF, Wedawatta G (2023) Digital twin smart cities for disaster risk management: a review of evolving concepts. Sustainability 15(15):11910. https://doi.org/10.3390/su151511910

ASME (2018) Retracted: “Kinematic analysis of the motion of a six degrees of freedom wave energy converter based on the concept of the stewart-gough platform” [ASME 2018 37th international conference on ocean, offshore and arctic engineering, Volume 10: ocean renewable energy, Madrid, Spain, June 17–22, 2018, Conference sponsors: ocean, offshore and arctic engineering division, ISBN: 978–0–7918–5131–9. Paper No. OMAE2018–78601, pp V010T09A047; 10 pages]. Volume 10: ocean renewable energy. https://doi.org/10.1115/OMAE2018-78601

Autodesk Revit Architecture Certification (2019) Mastering Autodesk® Revit® 2020, 1033–1035. Portico. https://doi.org/10.1002/9781119570189.app3

Badwi IM, Ellaithy HM, Youssef HE (2022) 3D-GIS parametric modelling for virtual urban simulation using CityEngine. Ann GIS 28(3):325–341. https://doi.org/10.1080/19475683.2022.2037019

Bagaria N, Laamarti F, Badawi HF, Albraikan A, Martinez Velazquez RA, El Saddik A (2019) Health 4.0: digital twins for health and well-being. Connected health in smart cities. pp 143–152. https://doi.org/10.1007/978-3-030-27844-1_7

Bellavista P, Giannelli C, Mamei M, Mendula M, Picone M (2021) Application-driven network-aware digital twin management in industrial edge environments. IEEE Trans Industr Inf 17(11):7791–7801. https://doi.org/10.1109/tii.2021.3067447

Biagini V, Subasic M, Oudalov A, Kreusel J (2020) The autonomous grid: automation, intelligence and the future of power systems. Energy Res Soc Sci 65:101460. https://doi.org/10.1016/j.erss.2020.101460

Billinghurst M, Clark A, Lee G (2015) A survey of augmented reality. Foundations and Trends® in Human–Computer Interaction, 8(2–3):73–272. https://doi.org/10.1561/1100000049

Blomkvist J, Clatworthy S, Holmlid S (2023) Interlude 1: materiality in design from a practitioner perspective: interview with Markus Edgar Hormeß, Adam Lawrence and Marc Stickdorn (26 April 2022). The materials of service design. pp 83–88. https://doi.org/10.4337/9781802203301.00019

Borrmann A, König M, Koch C, Beetz J (2018). Building information modeling: why? What? How? Building information modeling. pp 1–24. https://doi.org/10.1007/978-3-319-92862-3_1

Bouzguenda I, Alalouch C, Fava N (2019) Towards smart sustainable cities: a review of the role digital citizen participation could play in advancing social sustainability. Sustain Cities Soc 50:101627. https://doi.org/10.1016/j.scs.2019.101627

Catalano M, Chiurco A, Fusto C, Gazzaneo L, Longo F, Mirabelli G, Nicoletti L, Solina V, Talarico S (2022) A digital twin-driven and conceptual framework for enabling extended reality applications: a case study of a brake discs manufacturer. Procedia Comput Sci 200:1885–1893. https://doi.org/10.1016/j.procs.2022.01.389

Cathey G, Benson J, Gupta M, Sandhu R (2021) Edge centric secure data sharing with digital twins in smart ecosystems. 2021 Third IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA). https://doi.org/10.1109/tpsisa52974.2021.00008

Cerovsek T (2011) A review and outlook for a ‘Building Information Model’ (BIM): a multi-standpoint framework for technological development. Adv Eng Inform 25(2):224–244. https://doi.org/10.1016/j.aei.2010.06.003

Chakraborty S, Adhikari S (2021) Machine learning based digital twin for dynamical systems with multiple time-scales. Comput Struct 243:106410. https://doi.org/10.1016/j.compstruc.2020.106410

Chang T-W, Hsiao C-F, Chen C-Y, Huang W-X, Datta S, Mao W-L (2020) Fabricating behavior sensor computing approach for coexisting design environment. Sensors Mater 32(7):2409. https://doi.org/10.18494/sam.2020.2809

Charitonidou M (2022) Urban scale digital twins in data-driven society: challenging digital universalism in urban planning decision-making. Int J Archit Comput 20(2):238–253. https://doi.org/10.1177/14780771211070005

Chen L, Xie X, Lu Q, Parlikad AK, Pitt M, Yang J (2021) Gemini principles-based digital twin maturity model for asset management. Sustainability 13(15):8224. https://doi.org/10.3390/su13158224

Cinar ZM, Nuhu AA, Zeeshan Q, Korhan O (2020) Digital twins for industry 4.0: a review. Industrial engineering in the digital disruption era, 193–203. https://doi.org/10.1007/978-3-030-42416-9_18

City of Chicago Data Portal (2024) Chicago Data Portal. https://data.cityofchicago.org . Accessed 10 May 2024

Crespi N, Drobot AT, Minerva R (2023) The digital twin: what and why? The digital twin.pp 3–20. https://doi.org/10.1007/978-3-031-21343-4_1

Damjanovic-Behrendt V, Behrendt W (2019) An open source approach to the design and implementation of Digital Twins for Smart Manufacturing. Int J Comput Integr Manuf 32(4–5):366–384. https://doi.org/10.1080/0951192x.2019.1599436

Damjanovic-Behrendt V (2018) A digital twin-based privacy enhancement mechanism for the automotive industry. 2018 International Conference on Intelligent Systems (IS). https://doi.org/10.1109/is.2018.8710526

Dani AAH, Supangkat SH, Lubis FF, Nugraha IGBB, Kinanda R, Rizkia I (2023) Development of a smart city platform based on digital twin technology for monitoring and supporting decision-making. Sustainability 15(18):14002. https://doi.org/10.3390/su151814002

Defraeye T, Tagliavini G, Wu W, Prawiranto K, Schudel S, Assefa Kerisima M, Verboven P, Bühlmann A (2019) Digital twins probe into food cooling and biochemical quality changes for reducing losses in refrigerated supply chains. Resour Conserv Recycl 149:778–794. https://doi.org/10.1016/j.resconrec.2019.06.002

Dembski F, Wössner U, Letzgus M, Ruddat M, Yamu C (2020) Urban digital twins for smart cities and citizens: the case study of Herrenberg, Germany. Sustainability 12(6):2307. https://doi.org/10.3390/su12062307

Deren L, Wenbo Y, Zhenfeng S (2021) Smart city based on digital twins. Comput Urban Sci 1(1). https://doi.org/10.1007/s43762-021-00005-y

Diakite AA, Ng L, Barton J, Rigby M, Williams K, Barr S, Zlatanova S (2022) Liveable city digital twin: a pilot project for the city of Liverpool (NSW, Australia). ISPRS Ann Photogramm Remote Sens Spatial Inf Sci X-4/W2-2022:45–52. https://doi.org/10.5194/isprs-annals-x-4-w2-2022-45-2022

Dimitropoulos N, Togias T, Zacharaki N, Michalos G, Makris S (2021) Seamless human-robot collaborative assembly using artificial intelligence and wearable devices. Appl Sci 11(12):5699. https://doi.org/10.3390/app11125699

Din GMU, Marnerides AK (2017) Short term power load forecasting using Deep Neural Networks. 2017 International Conference on Computing, Networking and Communications (ICNC). https://doi.org/10.1109/iccnc.2017.7876196

Dong R, She C, Hardjawana W, Li Y, Vucetic B (2019) Deep learning for hybrid 5G services in mobile edge computing systems: learn from a digital twin. IEEE Trans Wireless Commun 18(10):4692–4707. https://doi.org/10.1109/twc.2019.2927312

Endsley MR (2016) Designing for situation awareness. CRC Press. https://doi.org/10.1201/b11371

Erol T, Mendi AF, Dogan D (2020) The digital twin revolution in healthcare. 2020 4th International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT). https://doi.org/10.1109/ismsit50672.2020.9255249

Evangelou T, Gkeli M, Potsiou C (2022) Building digital twins for smart cities: a case study in Greece. ISPRS Ann Photogramm Remote Sens Spatial Inf Sci X-4/W2-2022:61–68. https://doi.org/10.5194/isprs-annals-x-4-w2-2022-61-2022

Fan C, Mostafavi A (2019) A graph-based method for social sensing of infrastructure disruptions in disasters. Comput-Aided Civ Infrastruct Eng 34(12):1055–1070. https://doi.org/10.1111/mice.12457 . (Portico)

Fan C, Zhang C, Yahja A, Mostafavi A (2021) Disaster City Digital Twin: a vision for integrating artificial and human intelligence for disaster management. Int J Inf Manage 56:102049. https://doi.org/10.1016/j.ijinfomgt.2019.102049

Fan C, Jiang Y, Mostafavi A (2020) Social sensing in disaster city digital twin: integrated textual–visual–geo framework for situational awareness during built environment disruptions. J Manag Eng 36(3). https://doi.org/10.1061/(asce)me.1943-5479.0000745

Fang X, Wang H, Liu G, Tian X, Ding G, Zhang H (2022) Industry application of digital twin: from concept to implementation. Int J Adv Manuf Technol 121(7–8):4289–4312. https://doi.org/10.1007/s00170-022-09632-z

Ford DN, Wolf CM (2020) Smart cities with digital twin systems for disaster management. J Manag Eng 36(4). https://doi.org/10.1061/(asce)me.1943-5479.0000779

Gerlach B, Zarnitz S, Nitsche B, Straube F (2021) Digital supply chain twins—conceptual clarification, use cases and benefits. Logistics 5(4):86. https://doi.org/10.3390/logistics5040086

Ghaith M, Yosri A, El-Dakhakhni W (2022) Digital twin: a city-scale flood imitation framework. Proceedings of the Canadian society of civil engineering annual conference 2021. pp 577–588. https://doi.org/10.1007/978-981-19-1065-4_48

Ghosh D, Chun SA, Shafiq B, Adam NR (2016) Big data-based smart city platform. Proceedings of the 17th international digital government research conference on digital government research. https://doi.org/10.1145/2912160.2912205

Goia B, Cioara T, Anghel I (2022) Virtual power plant optimization in smart grids: a narrative review. Future Internet 14(5):128. https://doi.org/10.3390/fi14050128

Grieves MW (2005) Product lifecycle management: the new paradigm for enterprises. Int J Prod Dev 2(1/2):71. https://doi.org/10.1504/ijpd.2005.006669

Guarino N, Oberle D, Staab S (2009) What is an ontology? Handbook on ontologies. pp 1–17. https://doi.org/10.1007/978-3-540-92673-3_0

Guo D, Zhong RY, Lin P, Lyu Z, Rong Y, Huang GQ (2020) Digital twin-enabled Graduation Intelligent Manufacturing System for fixed-position assembly islands. Robot Comput-Integr Manuf 63:101917. https://doi.org/10.1016/j.rcim.2019.101917

He X, Ai Q, Wang J, Tao F, Pan B, Qiu R, Yang B (2023) Situation awareness of energy internet of things in smart city based on digital twin: from digitization to informatization. IEEE Internet Things J 10(9):7439–7458. https://doi.org/10.1109/jiot.2022.3203823

He Y, Deng J, Li H (2017) Short-term power load forecasting with deep belief network and copula models. 2017 9th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC). https://doi.org/10.1109/ihmsc.2017.50

He X, Ai Q, Qiu RC, Zhang D (2019) Preliminary exploration on digital twin for power systems: challenges, framework, and applications. arXiv preprint arXiv:1909.06977. https://www.semanticscholar.org/paper/Preliminary-Exploration-on-Digital-Twin-for-Power-He-Ai/360cd3a26ad4ab5f3685d676d36001348dd6e1d6 . Accessed 10 May 2024

Hinchy EP, O’Dowd NP, McCarthy CT (2019) Using open-source microcontrollers to enable digital twin communication for smart manufacturing. Procedia Manuf 38:1213–1219. https://doi.org/10.1016/j.promfg.2020.01.212

Hu L, Nguyen N-T, Tao W, Leu MC, Liu XF, Shahriar MR, Al Sunny SMN (2018) Modeling of cloud-based digital twins for smart manufacturing with MT connect. Procedia Manuf 26:1193–1203. https://doi.org/10.1016/j.promfg.2018.07.155

Huang J, Zhao L, Wei F, Cao B (2021) The application of digital twin on power industry. IOP Conf Ser: Earth Environ Sci 647(1):012015. https://doi.org/10.1088/1755-1315/647/1/012015

Ivanov S, Nikolskaya K, Radchenko G, Sokolinsky L, Zymbler M (2020) Digital twin of city: concept overview. 2020 Global Smart Industry Conference (GloSIC). https://doi.org/10.1109/glosic50886.2020.9267879

Jaensch F, Csiszar A, Scheifele C, Verl A (2018) Digital twins of manufacturing systems as a base for machine learning. 2018 25th International Conference on Mechatronics and Machine Vision in Practice (M2VIP). https://doi.org/10.1109/m2vip.2018.8600844

Jafari M, Kavousi-Fard A, Chen T, Karimi M (2023) A review on digital twin technology in smart grid, transportation system and smart city: challenges and future. IEEE Access 11:17471–17484. https://doi.org/10.1109/access.2023.3241588

Johansen ST, Unal P, Albayrak Ö, Ikonen E, Linnestad KJ, Jawahery S, Srivastava AK, Løvfall BT (2023) Hybrid and cognitive digital twins for the process industry. Open Eng 13(1). https://doi.org/10.1515/eng-2022-0418

John Samuel I, Salem O, He S (2022) Defect-oriented supportive bridge inspection system featuring building information modeling and augmented reality. Innov Infrastruct Solut 7(4). https://doi.org/10.1007/s41062-022-00847-3

Jones D, Snider C, Nassehi A, Yon J, Hicks B (2020) Characterising the Digital Twin: a systematic literature review. CIRP J Manuf Sci Technol 29:36–52. https://doi.org/10.1016/j.cirpj.2020.02.002

Kapteyn MG, Knezevic DJ, Huynh DBP, Tran M, Willcox KE (2020) Data-driven physics-based digital twins via a library of component-based reduced-order models. Int J Numer Methods Eng 123(13):2986–3003. https://doi.org/10.1002/nme.6423 . (Portico)

Ketzler B, Naserentin V, Latino F, Zangelidis C, Thuvander L, Logg A (2020) Digital twins for cities: a state of the art review. Built Environ 46(4):547–573. https://doi.org/10.2148/benv.46.4.547

Khudhair A, Li H, Ren G, Liu S (2021) Towards future BIM technology innovations: a bibliometric analysis of the literature. Appl Sci 11(3):1232. https://doi.org/10.3390/app11031232

Kim J, Olsen D (2021) From BIM to inspection: a comparative analysis of assistive embedment inspection. Proceedings of the International Symposium on Automation and Robotics in Construction (IAARC). https://doi.org/10.22260/isarc2021/0123

Kitchenham B, Pearl Brereton O, Budgen D, Turner M, Bailey J, Linkman S (2009) Systematic literature reviews in software engineering – a systematic literature review. Inf Softw Technol 51(1):7–15. https://doi.org/10.1016/j.infsof.2008.09.009

Koteleva N, Valnev V, Frenkel I (2021) Investigation of the effectiveness of an augmented reality and a dynamic simulation system collaboration in oil pump maintenance. Appl Sci 12(1):350. https://doi.org/10.3390/app12010350

Kritzinger W, Karner M, Traar G, Henjes J, Sihn W (2018) Digital Twin in manufacturing: a categorical literature review and classification. IFAC-PapersOnLine 51(11):1016–1022. https://doi.org/10.1016/j.ifacol.2018.08.474

Kritzler M, Funk M, Michahelles F, Rohde W (2017) The virtual twin. Proceedings of the seventh international conference on the internet of things. https://doi.org/10.1145/3131542.3140274

Krzyczkowski D (2019) Introducing azure digital twins. Apress. https://doi.org/10.1007/978-1-4842-5375-5

Kuber S, Sharma M, Bonetti A, Harispuru C, Soroush A (2022) Virtual testing of protection systems using digital twin technology. 2022 75th Annual Conference for Protective Relay Engineers (CPRE). https://doi.org/10.1109/cpre55809.2022.9776572

Kyriazopoulou C (2015) Smart city technologies and architectures - a literature review. Proceedings of the 4th International Conference on Smart Cities and Green ICT Systems. https://doi.org/10.5220/0005407000050016

Lacoche J, Villain É (2022) Prototyping context-aware augmented reality applications for smart environments inside virtual reality. Proceedings of the 17th international joint conference on computer vision, imaging and computer graphics theory and applications. https://doi.org/10.5220/0010768800003124

Lam P-D, Han J, Kwon K-R, Ok S-Y, Lee S-H (2023) Semantic 3D city model data visualization for smar t city digital twin. J Korea Multimed Soc 26(2):116–130. https://doi.org/10.9717/kmms.2023.26.2.116

Langenheim N, Sabri S, Chen Y, Kesmanis A, Felson A, Mueller A, Rajabifard A, Zhang Y (2022) Adapting a digital twin to enable real-time water sensitive urban design decision-making. Int Arch Photogramm Remote Sens Spat Inf Sci XLVIII-4/W4-2022:95–100. https://doi.org/10.5194/isprs-archives-xlviii-4-w4-2022-95-2022

Ledoux H, Arroyo Ohori K, Kumar K, Dukai B, Labetski A, Vitalis S (2019) CityJSON: a compact and easy-to-use encoding of the CityGML data model. Open Geospatial Data Softw Standards 4(1). https://doi.org/10.1186/s40965-019-0064-0

Lee A, Lee K-W, Kim K-H, Shin S-W (2022) A geospatial platform to manage large-scale individual mobility for an urban digital twin platform. Remote Sens 14(3):723. https://doi.org/10.3390/rs14030723

Lei B, Stouffs R, Biljecki F (2022) Assessing and benchmarking 3D city models. Int J Geogr Inf Sci 37(4):788–809. https://doi.org/10.1080/13658816.2022.2140808

Lemos MR, Cardoso VF, Otani M, Da Costa Nunes R, Da Silva VJ, De Lucena Junior VF (2022) Navigation robot training with Deep Q-Learning monitored by Digital Twin. 2022 IEEE International Conference on Consumer Electronics (ICCE). https://doi.org/10.1109/icce53296.2022.9730282

Li W, Rentemeister M, Badeda J, Jöst D, Schulte D, Sauer DU (2020) Digital twin for battery systems: cloud battery management system with online state-of-charge and state-of-health estimation. J Energy Storage 30:101557. https://doi.org/10.1016/j.est.2020.101557

Li C, Zheng P, Li S, Pang Y, Lee CKM (2022) AR-assisted digital twin-enabled robot collaborative manufacturing system with human-in-the-loop. Robot Comput-Integr Manuf 76:102321. https://doi.org/10.1016/j.rcim.2022.102321

Li Y, Zhang Z, Li X, Guan S (2021) Research on equipment maintenance guidance technology based on MR and digital twin. Proceedings of the 2021 5th international conference on electronic information technology and computer engineering. https://doi.org/10.1145/3501409.3501454

Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, Clarke M, Devereaux PJ, Kleijnen J, Moher D (2009) The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol 62(10):e1–e34. https://doi.org/10.1016/j.jclinepi.2009.06.006

Liu Y, Zhang L, Yang Y, Zhou L, Ren L, Wang F, Liu R, Pang Z, Deen MJ (2019) A novel cloud-based framework for the elderly healthcare services using digital twin. IEEE Access 7:49088–49101. https://doi.org/10.1109/access.2019.2909828

Liu S, Wang XV, Wang L (2022) Digital twin-enabled advance execution for human-robot collaborative assembly. CIRP Ann 71(1):25–28. https://doi.org/10.1016/j.cirp.2022.03.024

Liu Y, Pan S, Folz P, Ramparany F, Bolle S, Ballot E, Coupaye T (2023) Cognitive digital twins for freight parking management in last mile delivery under smart cities paradigm. Comput Ind 153:104022. https://doi.org/10.1016/j.compind.2023.104022

Liu Y, Wang Z, Han K, Shou Z, Tiwari P, L. Hansen JH (2020) Sensor fusion of camera and cloud digital twin information for intelligent vehicles. 2020 IEEE intelligent vehicles symposium (IV). https://doi.org/10.1109/iv47402.2020.9304643

Lohtander M, Ahonen N, Lanz M, Ratava J, Kaakkunen J (2018) Micro manufacturing unit and the corresponding 3D-model for the digital twin. Procedia Manuf 25:55–61. https://doi.org/10.1016/j.promfg.2018.06.057

Lu Y, Huang X, Zhang K, Maharjan S, Zhang Y (2021a) Low-latency federated learning and blockchain for edge association in digital twin empowered 6G networks. IEEE Trans Industr Inf 17(7):5098–5107. https://doi.org/10.1109/tii.2020.3017668

Lu Y, Maharjan S, Zhang Y (2021b) Adaptive edge association for wireless digital twin networks in 6G. IEEE Internet Things J 8(22):16219–16230. https://doi.org/10.1109/jiot.2021.3098508

Lu Q, Parlikad AK, Woodall P, Don Ranasinghe G, Xie X, Liang Z, Konstantinou E, Heaton J, Schooling J (2020) Developing a digital twin at building and city levels: case study of West Cambridge Campus. J Manag Eng 36(3). https://doi.org/10.1061/(asce)me.1943-5479.0000763

Lugaresi G, Jemai Z, Sahin E (2023) Digital twins for supply chains: current outlook and future challenges. ECMS 2023 Proceedings Edited by Enrico Vicario, Romeo Bandinelli, Virginia Fani, Michele Mastroianni. https://doi.org/10.7148/2023-0451

Mainisa M, Eka Priana S, Zulhedi Z (2023) Implementasi bim dalam permodelan 3d pembangunan gedung kantor cabang bri batusangkar menggunakan software openbuildings designer. Ensiklopedia Res Community Serv Rev 2(3):147–155. https://doi.org/10.33559/err.v2i3.1771

Major P, Li G, Hildre HP, Zhang H (2021) The use of a data-driven digital twin of a smart city: a case study of Ålesund, Norway. IEEE Instrum Meas Mag 24(7):39–49. https://doi.org/10.1109/mim.2021.9549127

Mandl B, Stehling M, Schmiedinger T, Adam M (2017) Enhancing workplace learning by augmented reality. Proceedings of the seventh international conference on the internet of things. https://doi.org/10.1145/3131542.3140265

Martinez-Velazquez R, Gamez R, El Saddik A (2019) Cardio twin: a digital twin of the human heart running on the edge. 2019 IEEE International Symposium on Medical Measurements and Applications (MeMeA). https://doi.org/10.1109/memea.2019.8802162

Meier N, Müller-Polyzou R, Brach L, Georgiadis A (2021) Digital twin support for laser-based assembly assistance. Procedia CIRP 99:460–465. https://doi.org/10.1016/j.procir.2021.03.066

Mihai S, Yaqoob M, Hung DV, Davis W, Towakel P, Raza M, Karamanoglu M, Barn B, Shetve D, Prasad RV, Venkataraman H, Trestian R, Nguyen HX (2022) Digital twins: a survey on enabling technologies, challenges, trends and future prospects. IEEE Commun Surv Tutorials 24(4):2255–2291. https://doi.org/10.1109/comst.2022.3208773

Mir M, Yaghoobi M, Khairabadi M (2022) A new approach to energy-aware routing in the Internet of Things using improved Grasshopper Metaheuristic Algorithm with Chaos theory and Fuzzy Logic. Multimed Tools Appl 82(4):5133–5159. https://doi.org/10.1007/s11042-021-11841-9

Moosavi J, Naeni LM, Fathollahi-Fard AM, Fiore U (2021) Blockchain in supply chain management: a review, bibliometric, and network analysis. Environ Sci Pollut Res. https://doi.org/10.1007/s11356-021-13094-3

Mora L, Deakin M, Reid A (2019) Combining co-citation clustering and text-based analysis to reveal the main development paths of smart cities. Technol Forecast Soc Chang 142:56–69. https://doi.org/10.1016/j.techfore.2018.07.019

Mora L, Appio FP, Foss NJ, Arellano-Gault D, Zhang X (2023a) Organizing for smart city development: research at the crossroads. Introduction to the special issue. Organ Stud 44(10):1559–1575. https://doi.org/10.1177/01708406231197815

Mora L, Gerli P, Ardito L, Messeni Petruzzelli A (2023b) Smart city governance from an innovation management perspective: theoretical framing, review of current practices, and future research agenda. Technovation 123:102717. https://doi.org/10.1016/j.technovation.2023.102717

Moya B, Badías A, Alfaro I, Chinesta F, Cueto E (2020a) Digital twins that learn and correct themselves. Int J Numer Methods Eng 123(13):3034–3044. https://doi.org/10.1002/nme.6535 . (Portico)

Moya B, Alfaro I, Gonzalez D, Chinesta F, Cueto E (2020b) Physically sound, self-learning digital twins for sloshing fluids. PLoS ONE 15(6):e0234569. https://doi.org/10.1371/journal.pone.0234569

Nämerforslund T (2022) Digital twin performance: unity as a platform for visualizing interactive digital twins. Thesis - Institution of Information Systems and Technology

National Aeronautics and Space Administration (NASA) (2010) Encyclopedia of geography. https://doi.org/10.4135/9781412939591.n797

Nica E, Popescu GH, Poliak M, Kliestik T, Sabie O-M (2023) Digital twin simulation tools, spatial cognition algorithms, and multi-sensor fusion technology in sustainable urban governance networks. Mathematics 11(9):1981. https://doi.org/10.3390/math11091981

Nikolakis N, Alexopoulos K, Xanthakis E, Chryssolouris G (2018) The digital twin implementation for linking the virtual representation of human-based production tasks to their physical counterpart in the factory-floor. Int J Comput Integr Manuf 32(1):1–12. https://doi.org/10.1080/0951192x.2018.1529430

Nochta T, Wan L, Schooling JM, Parlikad AK (2020) A socio-technical perspective on urban analytics: the case of city-scale digital twins. J Urban Technol 28(1–2):263–287. https://doi.org/10.1080/10630732.2020.1798177

Nowocin JK (2017) Microgrid risk reduction for design and validation testing using controller hardware in the loop (Doctoral dissertation, Massachusetts Institute of Technology)

ODAA platform (2016) http://www.odaa.dk . Accessed 10 May 2024

Pan S, Trentesaux D, McFarlane D, Montreuil B, Ballot E, Huang GQ (2021) Digital interoperability in logistics and supply chain management: state-of-the-art and research avenues towards Physical Internet. Comput Ind 128:103435. https://doi.org/10.1016/j.compind.2021.103435

Pang J, Huang Y, Xie Z, Li J, Cai Z (2021) Collaborative city digital twin for the COVID-19 pandemic: a federated learning solution. Tsinghua Sci Technol 26(5):759–771. https://doi.org/10.26599/tst.2021.9010026

Paripooranan CS, Abishek R, Vivek DC, Karthik S (2020) An implementation of AR enabled Digital Twins for 3-D printing. 2020 IEEE International Symposium on Smart Electronic Systems (ISES) (Formerly INiS). https://doi.org/10.1109/ises50453.2020.00043

Park H-A, Byeon G, Son W, Jo H-C, Kim J, Kim S (2020) Digital Twin for operation of microgrid: optimal scheduling in virtual space of Digital Twin. Energies 13(20):5504. https://doi.org/10.3390/en13205504

Pérez JG, Toledo DG (2017) Cognitive analytics of smart cities. Proceedings of the 18th annual international conference on digital government research. https://doi.org/10.1145/3085228.3085265

Petković T, Puljiz D, Marković I, Hein B (2019) Human intention estimation based on hidden Markov model motion validation for safe flexible robotized warehouses. Robot Comput-Integr Manuf 57:182–196. https://doi.org/10.1016/j.rcim.2018.11.004

Pivano L, Nguyen DT, Bruun Ludvigsen K (2019) Digital Twin for drilling operations – towards cloud-based operational planning. Day 3 Wed, May 08, 2019. https://doi.org/10.4043/29316-ms

Podvalny SL, Vasiljev EM (2021) Digital twin for smart electricity distribution networks. IOP Conf Ser: Mater Sci Eng 1035(1):012047. https://doi.org/10.1088/1757-899x/1035/1/012047

Prasad D, Alizadeh T, Dowling R (2023) Smart city planning and the challenges of informality in India. Dialogues in human geography, 204382062311566. https://doi.org/10.1177/20438206231156655

Praschl C, Krauss O (2022) Geo-referenced occlusion models for mixed reality applications using the microsoft HoloLens. Proceedings of the 17th international joint conference on computer vision, imaging and computer graphics theory and applications. https://doi.org/10.5220/0010775200003124

Psarommatis F, May G (2022) A standardized approach for measuring the performance and flexibility of digital twins. Int J Prod Res 61(20):6923–6938. https://doi.org/10.1080/00207543.2022.2139005

Qiuchen Lu V, Parlikad AK, Woodall P, Ranasinghe GD, Heaton J (2019) Developing a dynamic digital twin at a building level: using Cambridge campus as case study. International Conference on Smart Infrastructure and Construction 2019 (ICSIC). https://doi.org/10.1680/icsic.64669.067

Quek HY, Sielker F, Akroyd J, Bhave AN, von Richthofen A, Herthogs P, van der Laag Yamu C, Wan L, Nochta T, Burgess G, Lim MQ, Mosbach S, Kraft M (2023) The conundrum in smart city governance: interoperability and compatibility in an ever-growing ecosystem of digital twins. Data Policy 5. https://doi.org/10.1017/dap.2023.1

Rabah S, Assila A, Khouri E, Maier F, Ababsa F, Bourny V, Maier P, Mérienne F (2018) Towards improving the future of manufacturing through digital twin and augmented reality technologies. Procedia Manuf 17:460–467. https://doi.org/10.1016/j.promfg.2018.10.070

Rajesh PK, Manikandan N, Ramshankar CS, Vishwanathan T, Sathishkumar C (2019) Digital Twin of an automotive brake pad for predictive maintenance. Procedia Comput Sci 165:18–24. https://doi.org/10.1016/j.procs.2020.01.061

Raqeeb A, Bonetti A, Carlsson A, Harispuru C, Pustejovsky M, Wetterstrand N (2022) Functional digital twins of relay protection and relay test equipment enabling benefits in training and remote support. 16th International Conference on Developments in Power System Protection (DPSP 2022). https://doi.org/10.1049/icp.2022.0925

Rathore MM, Shah SA, Shukla D, Bentafat E, Bakiras S (2021) The role of AI, machine learning, and big data in digital twinning: a systematic literature review, challenges, and opportunities. IEEE Access 9:32030–32052. https://doi.org/10.1109/access.2021.3060863

Rebmann A, Knoch S, Emrich A, Fettke P, Loos P (2020) A multi-sensor approach for Digital Twins of manual assembly and commissioning. Procedia Manuf 51:549–556. https://doi.org/10.1016/j.promfg.2020.10.077

Revetria R, Tonelli F, Damiani L, Demartini M, Bisio F, Peruzzo N (2019) A real-time mechanical structures monitoring system based on Digital Twin, Iot and augmented reality. 2019 Spring Simulation Conference (SpringSim). https://doi.org/10.23919/springsim.2019.8732917

Rosen R, von Wichert G, Lo G, Bettenhausen KD (2015) About the importance of autonomy and Digital Twins for the future of manufacturing. IFAC-PapersOnLine 48(3):567–572. https://doi.org/10.1016/j.ifacol.2015.06.141

Rožanec JM, Lu J, Rupnik J, Škrjanc M, Mladenić D, Fortuna B, Zheng X, Kiritsis D (2022) Actionable cognitive twins for decision making in manufacturing. Int J Prod Res 60(2):452–478. https://doi.org/10.1080/00207543.2021.2002967

Saeed Z, Mancini F, Glusac T, Izadpanahi P (2022) Future city, digital twinning and the urban realm: a systematic literature review. Buildings 12(5):685. https://doi.org/10.3390/buildings12050685

San Francisco Open Data (2024) San Francisco Open Data. https://data.sfgov.org . Accessed 10 May 2024

Shahat E, Hyun CT, Yeom C (2021) City Digital Twin potentials: a review and research agenda. Sustainability 13(6):3386. https://doi.org/10.3390/su13063386

Shangguan D, Chen L, Ding J (2020) A Digital Twin-based approach for the fault diagnosis and health monitoring of a complex satellite system. Symmetry 12(8):1307. https://doi.org/10.3390/sym12081307

Srai J, Settanni E (2019) Supply chain digital twins: opportunities and challenges beyond the hype

Ssin S, Cho H, Woo W (2021a) GeoACT: augmented control tower using virtual and real geospatial data. Interact Des Archit 48:122–142. https://doi.org/10.55612/s-5002-048-006

Ssin S, Cho H, Woo W (2021b) GeoVCM: virtual urban digital twin system augmenting virtual and real geo-spacial data. 2021 IEEE International Conference on Consumer Electronics (ICCE). https://doi.org/10.1109/icce50685.2021.9427709

Stark R, Kind S, Neumeyer S (2017) Innovations in digital modelling for next generation manufacturing system design. CIRP Ann 66(1):169–172. https://doi.org/10.1016/j.cirp.2017.04.045

Steinmetz C, Schroeder GN, Sulak A, Tuna K, Binotto A, Rettberg A, Pereira CE (2022) A methodology for creating semantic digital twin models supported by knowledge graphs. 2022 IEEE 27th International Conference on Emerging Technologies and Factory Automation (ETFA). https://doi.org/10.1109/etfa52439.2022.9921499

Svítek M, Skobelev P, Kozhevnikov S (2019) Smart city 5.0 as an urban ecosystem of smart services. Stud Comp Intell:426–438. https://doi.org/10.1007/978-3-030-27477-1_33

Tao F, Cheng J, Qi Q, Zhang M, Zhang H, Sui F (2017) Digital twin-driven product design, manufacturing and service with big data. Int J Adv Manuf Technol 94(9–12):3563–3576. https://doi.org/10.1007/s00170-017-0233-1

Ton DT, Smith MA (2012) The U.S. Department of Energy’s Microgrid Initiative. Electr J 25(8):84–94. https://doi.org/10.1016/j.tej.2012.09.013

Tonnarelli F, Mora L (2023) Smart urbanism in Africa: when theories do not fit with contextual practices. Reg Stud:1–11. https://doi.org/10.1080/00343404.2023.2235407

Trusov A, Limonova EE (2020) The analysis of projective transformation algorithms for image recognition on mobile devices. Twelfth International Conference on Machine Vision (ICMV 2019). https://doi.org/10.1117/12.2559732

Tuhaise VV, Tah JHM, Abanda FH (2023) Technologies for digital twin applications in construction. Autom Constr 152:104931. https://doi.org/10.1016/j.autcon.2023.104931

Tygesen UT, Jepsen MS, Vestermark J, Dollerup N, Pedersen A (2018) The true digital twin concept for fatigue re-assessment of marine structures. Volume 1: offshore technology. https://doi.org/10.1115/omae2018-77915

Um J, Popper J, Ruskowski M (2018) Modular augmented reality platform for smart operator in production environment. 2018 IEEE Industrial Cyber-Physical Systems (ICPS). https://doi.org/10.1109/icphys.2018.8390796

Urbina Coronado PD, Lynn R, Louhichi W, Parto M, Wescoat E, Kurfess T (2018) Part data integration in the shop floor digital twin: mobile and cloud technologies to enable a manufacturing execution system. J Manuf Syst 48:25–33. https://doi.org/10.1016/j.jmsy.2018.02.002

van der Valk H, Strobel G, Winkelmann S, Hunker J, Tomczyk M (2022) Supply chains in the era of Digital Twins – a review. Procedia Comput Sci 204:156–163. https://doi.org/10.1016/j.procs.2022.08.019

Verdouw C, Tekinerdogan B, Beulens A, Wolfert S (2021) Digital twins in smart farming. Agric Syst 189:103046. https://doi.org/10.1016/j.agsy.2020.103046

Vidal-Balea A, Blanco-Novoa O, Fraga-Lamas P, Vilar-Montesinos M, Fernández-Caramés TM (2021) Collaborative augmented digital twin: a novel open-source augmented reality solution for training and maintenance processes in the shipyard of the future. The 4th XoveTIC conference. https://doi.org/10.3390/engproc2021007010

Wan L, Nochta T, Schooling JM (2019) Developing a city-level digital twin –propositions and a case study. International Conference on Smart Infrastructure and Construction 2019 (ICSIC). https://doi.org/10.1680/icsic.64669.187

Wang P, Luo M (2021) A digital twin-based big data virtual and real fusion learning reference framework supported by industrial internet towards smart manufacturing. J Manuf Syst 58:16–32. https://doi.org/10.1016/j.jmsy.2020.11.012

Wang W, He F, Li Y, Tang S, Li X, Xia J, Lv Z (2023) Data information processing of traffic digital twins in smart cities using edge intelligent federation learning. Inf Process Manage 60(2):103171. https://doi.org/10.1016/j.ipm.2022.103171

Wang H, Meng X (2021) BIM-supported knowledge management: potentials and expectations. J Manag Eng 37(4). https://doi.org/10.1061/(asce)me.1943-5479.0000934

Waszak M, Lam AN, Hoffmann V, Elvesater B, Mogos MF, Roman D (2022) Let the asset decide: digital twins with knowledge graphs. 2022 IEEE 19th International Conference on Software Architecture Companion (ICSA-C). https://doi.org/10.1109/icsa-c54293.2022.00014

Wen T, Dobson E, Hvaara R (2020) Mesh learning: a cloud and edge–based computing network providing data–driven solutions to the oil and gas industry. Day 2 Tue, November 03, 2020. https://doi.org/10.4043/30365-ms

White G, Zink A, Codecá L, Clarke S (2021) A digital twin smart city for citizen feedback. Cities 110:103064. https://doi.org/10.1016/j.cities.2020.103064

Wiegand G, Mai C, Liu Y, Hußmann H (2018) Early take-over preparation in stereoscopic 3D. Adjunct proceedings of the 10th international conference on automotive user interfaces and interactive vehicular applications. https://doi.org/10.1145/3239092.3265957

Xia L, Zheng P, Li X, Gao RX, Wang L (2022) Toward cognitive predictive maintenance: a survey of graph-based approaches. J Manuf Syst 64:107–120. https://doi.org/10.1016/j.jmsy.2022.06.002

Xie X, Lu Q, Rodenas-Herraiz D, Parlikad AK, Schooling JM (2020) Visualised inspection system for monitoring environmental anomalies during daily operation and maintenance. Eng Constr Archit Manag 27(8):1835–1852. https://doi.org/10.1108/ecam-11-2019-0640

Xu Y, Sun Y, Liu X, Zheng Y (2019) A Digital-Twin-assisted fault diagnosis using deep transfer learning. IEEE Access 7:19990–19999. https://doi.org/10.1109/access.2018.2890566

Xu W, Cui J, Li L, Yao B, Tian S, Zhou Z (2021) Digital twin-based industrial cloud robotics: framework, control approach and implementation. J Manuf Syst 58:196–209. https://doi.org/10.1016/j.jmsy.2020.07.013

Xu H, Berres A, Yoginath SB, Sorensen H, Nugent PJ, Severino J, Tennille SA, Moore A, Jones W, Sanyal J (2023) Smart mobility in the cloud: enabling real-time situational awareness and cyber-physical control through a Digital Twin for traffic. IEEE Trans Intell Transp Syst 24(3):3145–3156. https://doi.org/10.1109/tits.2022.3226746

Yan M, Gan W, Zhou Y, Wen J, Yao W (2022) Projection method for blockchain-enabled non-iterative decentralized management in integrated natural gas-electric systems and its application in digital twin modelling. Appl Energy 311:118645. https://doi.org/10.1016/j.apenergy.2022.118645

Yin Y, Zheng P, Li C, Wang L (2023) A state-of-the-art survey on augmented reality-assisted Digital Twin for futuristic human-centric industry transformation. Robot Comput-Integr Manuf 81:102515. https://doi.org/10.1016/j.rcim.2022.102515

Yu G, Wang Y, Hu M, Shi L, Mao Z, Sugumaran V (2021) RIOMS: an intelligent system for operation and maintenance of urban roads using spatio-temporal data in smart cities. Futur Gener Comput Syst 115:583–609. https://doi.org/10.1016/j.future.2020.09.010

Yu X, Merritt J (2023) Comparison of city digital twin case studies. Digital twins for smart cities, 123–137. https://doi.org/10.1680/dtsc.66007.123

Zakzak L (2019) Citizen-centric smart city development. Proceedings of the 20th annual international conference on digital government research. https://doi.org/10.1145/3325112.3325236

Zborowski M (2018) Finding meaning, application for the much-discussed “Digital Twin.” J Petrol Technol 70(06):26–32. https://doi.org/10.2118/0618-0026-jpt

Zhang J, Mora L (2023) Nothing but symbolic: Chinese new authoritarianism, smart government, and the challenge of multi-level governance. Gov Inf Q 40(4):101880. https://doi.org/10.1016/j.giq.2023.101880

Zhang K, Cao J, Zhang Y (2022) Adaptive digital twin and multiagent deep reinforcement learning for vehicular edge computing and networks. IEEE Trans Industr Inf 18(2):1405–1413. https://doi.org/10.1109/tii.2021.3088407

Zhao X, Sun Y (2020) Augmented reality assembly guidance method based on situation awareness. In: Proceedings of the 2020 the 10th international workshop on computer science and engineering. WCSE, pp 165–173

Zheng P, Lin T-J, Chen C-H, Xu X (2018) A systematic design approach for service innovation of smart product-service systems. J Clean Prod 201:657–667. https://doi.org/10.1016/j.jclepro.2018.08.101

Zheng X, Lu J, Kiritsis D (2021) The emergence of cognitive digital twin: vision, challenges and opportunities. Int J Prod Res 60(24):7610–7632. https://doi.org/10.1080/00207543.2021.2014591

Zheng P, Li S, Fan J, Li C, Wang L (2023) A collaborative intelligence-based approach for handling human-robot collaboration uncertainties. CIRP Ann 72(1):1–4. https://doi.org/10.1016/j.cirp.2023.04.057

Židek K, Hladký V, Pitel’ J, Demčák J, Hošovský A, Lazorík P (2021) SMART production system with full digitalization for assembly and inspection in concept of industry 4.0. Future access enablers for ubiquitous and intelligent infrastructures. pp 181–192. https://doi.org/10.1007/978-3-030-78459-1_13

Zohdi TI (2020) A machine-learning framework for rapid adaptive digital-twin based fire-propagation simulation in complex environments. Comput Methods Appl Mech Eng 363:112907. https://doi.org/10.1016/j.cma.2020.112907

Article   MathSciNet   Google Scholar  

Download references

Acknowledgements

The authors extend their appreciation to the Deputyship for Research and Innovation, Ministry of Education in Saudi Arabia, for funding this research work through project number 445-5-961.

This research was funded by the Deputyship for Research and Innovation, Ministry of Education in Saudi Arabia, project number 445-5-961.

Author information

Authors and affiliations.

College of Computer Science and Engineering, Taibah University, 46421, Yanbu, Saudi Arabia

Rasha F. El-Agamy, Hanaa A. Sayed, Arwa M. AL Akhatatneh, Mansourah Aljohani & Mostafa Elhosseini

Computer Science Department, Faculty of Science, Tanta University, Tanta, 31527, Egypt

Rasha F. El-Agamy

Department of Computer Science, Faculty of Computers and Information, Assiut University, Assiut, 71516, Egypt

Hanaa A. Sayed

Department of Computers and Control Systems Engineering, Faculty of Engineering, Mansoura University, Mansoura, 35516, Egypt

Mostafa Elhosseini

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization, M.E.; methodology, R.E., H.A., A.A., M.A.; software, A.A. and M.A.; validation, H.A., R.E. and M.E.; formal analysis, R.E.. and A.A.; investigation, A.A. and H.A; resources, M.A.; data curation, R.E., and H.A; writing—original draft preparation, R.E., H.A., A.A., and M.A.; writing—review and editing, R.E., H.A., and M.E.; visualization, R.E., and H.A; supervision, M.E., H.A., and R.E.; project administration, M.E.; funding acquisition, M.E. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Mostafa Elhosseini .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

El-Agamy, R.F., Sayed, H.A., AL Akhatatneh, A.M. et al. Comprehensive analysis of digital twins in smart cities: a 4200-paper bibliometric study. Artif Intell Rev 57 , 154 (2024). https://doi.org/10.1007/s10462-024-10781-8

Download citation

Accepted : 29 April 2024

Published : 27 May 2024

DOI : https://doi.org/10.1007/s10462-024-10781-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Digital twin
• Smart city development
• Bibliometric study
• IoT integration
• Find a journal
• Publish with us
• Track your research

Journal of Materials Chemistry C

High-performance organic thin-film transistors: principles and strategies.

Organic Thin Film Transistors (OTFTs) mark a breakthrough in flexible electronics, offering advantages over traditional inorganic semiconductors through their adaptability, cost efficiency, and low-temperature production. This study highlights OTFTs' evolution, showcasing achievements like mobility rates up to 18 cm²/Vs and effective low-voltage operation. It delves into OTFT architecture, covering semiconductors, dielectrics, and metal contacts, and the impact of device configuration, material choice, and interface quality on performance. Emphasizing advancements in material quality, device stability, and processing techniques, the paper also explores innovative OTFT designs and fabrication methods. Through a focused narrative on recent improvements in electrical properties, this work underlines OTFTs' potential in pioneering applications in electronics, including flexible displays, integrated circuits, and sensors, pointing towards a future of versatile and wearable technology.

• This article is part of the themed collection: Journal of Materials Chemistry C Recent Review Articles

Article information

Download citation, permissions.

Z. Hao, Z. Wu, S. Liu, X. Liu, J. Chen and X. Tang, J. Mater. Chem. C , 2024, Accepted Manuscript , DOI: 10.1039/D4TC01240B

To request permission to reproduce material from this article, please go to the Copyright Clearance Center request page .

If you are an author contributing to an RSC publication, you do not need to request permission provided correct acknowledgement is given.

If you are the author of this article, you do not need to request permission to reproduce figures and diagrams provided correct acknowledgement is given. If you want to reproduce the whole article in a third-party publication (excluding your thesis/dissertation for which permission is not required) please go to the Copyright Clearance Center request page .

Read more about how to correctly acknowledge RSC content .

Social activity

Search articles by author.

This article has not yet been cited.

Advertisements