limitations in literature review

About the LSE Impact Blog
Comments Policy
Popular Posts
Recent Posts
Subscribe to the Impact Blog
Write for us
LSE comment

Neal Haddaway

October 19th, 2020, 8 common problems with literature reviews and how to fix them.

3 comments | 315 shares

Estimated reading time: 5 minutes

Literature reviews are an integral part of the process and communication of scientific research. Whilst systematic reviews have become regarded as the highest standard of evidence synthesis, many literature reviews fall short of these standards and may end up presenting biased or incorrect conclusions. In this post, Neal Haddaway highlights 8 common problems with literature review methods, provides examples for each and provides practical solutions for ways to mitigate them.

Enjoying this blogpost? 📨 Sign up to our mailing list and receive all the latest LSE Impact Blog news direct to your inbox.

Researchers regularly review the literature – it’s an integral part of day-to-day research: finding relevant research, reading and digesting the main findings, summarising across papers, and making conclusions about the evidence base as a whole. However, there is a fundamental difference between brief, narrative approaches to summarising a selection of studies and attempting to reliably and comprehensively summarise an evidence base to support decision-making in policy and practice.

So-called ‘evidence-informed decision-making’ (EIDM) relies on rigorous systematic approaches to synthesising the evidence. Systematic review has become the highest standard of evidence synthesis and is well established in the pipeline from research to practice in the field of health . Systematic reviews must include a suite of specifically designed methods for the conduct and reporting of all synthesis activities (planning, searching, screening, appraising, extracting data, qualitative/quantitative/mixed methods synthesis, writing; e.g. see the Cochrane Handbook ). The method has been widely adapted into other fields, including environment (the Collaboration for Environmental Evidence ) and social policy (the Campbell Collaboration ).

Despite the growing interest in systematic reviews, traditional approaches to reviewing the literature continue to persist in contemporary publications across disciplines. These reviews, some of which are incorrectly referred to as ‘systematic’ reviews, may be susceptible to bias and as a result, may end up providing incorrect conclusions. This is of particular concern when reviews address key policy- and practice- relevant questions, such as the ongoing COVID-19 pandemic or climate change.

These limitations with traditional literature review approaches could be improved relatively easily with a few key procedures; some of them not prohibitively costly in terms of skill, time or resources.

In our recent paper in Nature Ecology and Evolution , we highlight 8 common problems with traditional literature review methods, provide examples for each from the field of environmental management and ecology, and provide practical solutions for ways to mitigate them.

There is a lack of awareness and appreciation of the methods needed to ensure systematic reviews are as free from bias and as reliable as possible: demonstrated by recent, flawed, high-profile reviews. We call on review authors to conduct more rigorous reviews, on editors and peer-reviewers to gate-keep more strictly, and the community of methodologists to better support the broader research community. Only by working together can we build and maintain a strong system of rigorous, evidence-informed decision-making in conservation and environmental management.

Note: This article gives the views of the authors, and not the position of the LSE Impact Blog, nor of the London School of Economics. Please review our comments policy if you have any concerns on posting a comment below

Image credit: Jaeyoung Geoffrey Kang via unsplash

About the author

Neal Haddaway is a Senior Research Fellow at the Stockholm Environment Institute, a Humboldt Research Fellow at the Mercator Research Institute on Global Commons and Climate Change, and a Research Associate at the Africa Centre for Evidence. He researches evidence synthesis methodology and conducts systematic reviews and maps in the field of sustainability and environmental science. His main research interests focus on improving the transparency, efficiency and reliability of evidence synthesis as a methodology and supporting evidence synthesis in resource constrained contexts. He co-founded and coordinates the Evidence Synthesis Hackathon (www.eshackathon.org) and is the leader of the Collaboration for Environmental Evidence centre at SEI. @nealhaddaway

Why is mission creep a problem and not a legitimate response to an unexpected finding in the literature? Surely the crucial points are that the review’s scope is stated clearly and implemented rigorously, not when the scope was finalised.

Pingback: Quick, but not dirty – Can rapid evidence reviews reliably inform policy? | Impact of Social Sciences

#9. Most of them are terribly boring. Which is why I teach students how to make them engaging…and useful.

“But I’m not ready!” Common barriers to writing and how to overcome them

November 16th, 2020.

“Remember a condition of academic writing is that we expose ourselves to critique” – 15 steps to revising journal articles

January 18th, 2017.

A simple guide to ethical co-authorship

March 29th, 2021.

How common is academic plagiarism?

February 8th, 2024.

Visit our sister blog LSE Review of Books

Eight problems with literature reviews and how to fix them

Affiliations.

1 Mercator Research Institute on Climate Change and Global Commons, Berlin, Germany. [email protected].
2 Stockholm Environment Institute, Stockholm, Sweden. [email protected].
3 Africa Centre for Evidence, University of Johannesburg, Johannesburg, South Africa. [email protected].
4 College of Medicine and Health, Exeter University, Exeter, UK.
5 Department of Zoology, University of Cambridge, Cambridge, UK.
6 School of Biological Sciences, University of East Anglia, Norwich, UK.
7 Department of Biological Sciences, Royal Holloway University of London, Egham, UK.
8 Stockholm Environment Institute, Stockholm, Sweden.
9 Department of Zoology, University of Oxford, Oxford, UK.
10 Collaboration for Environmental Evidence, UK Centre, School of Natural Sciences, Bangor University, Bangor, UK.
11 Liljus ltd, London, UK.
12 Department of Forest Sciences, University of Helsinki, Helsinki, Finland.
13 Evidence Synthesis Lab, School of Natural and Environmental Sciences, University of Newcastle, Newcastle-upon-Tyne, UK.
PMID: 33046871
DOI: 10.1038/s41559-020-01295-x

Traditional approaches to reviewing literature may be susceptible to bias and result in incorrect decisions. This is of particular concern when reviews address policy- and practice-relevant questions. Systematic reviews have been introduced as a more rigorous approach to synthesizing evidence across studies; they rely on a suite of evidence-based methods aimed at maximizing rigour and minimizing susceptibility to bias. Despite the increasing popularity of systematic reviews in the environmental field, evidence synthesis methods continue to be poorly applied in practice, resulting in the publication of syntheses that are highly susceptible to bias. Recognizing the constraints that researchers can sometimes feel when attempting to plan, conduct and publish rigorous and comprehensive evidence syntheses, we aim here to identify major pitfalls in the conduct and reporting of systematic reviews, making use of recent examples from across the field. Adopting a 'critical friend' role in supporting would-be systematic reviews and avoiding individual responses to police use of the 'systematic review' label, we go on to identify methodological solutions to mitigate these pitfalls. We then highlight existing support available to avoid these issues and call on the entire community, including systematic review specialists, to work towards better evidence syntheses for better evidence and better decisions.

Environment
Research Design
Systematic Reviews as Topic*

University of Texas Libraries

Literature Reviews

What is a literature review?
Steps in the Literature Review Process
Define your research question
Determine inclusion and exclusion criteria
Choose databases and search
Review Results
Synthesize Results
Analyze Results
Librarian Support

What is a Literature Review?

A literature or narrative review is a comprehensive review and analysis of the published literature on a specific topic or research question. The literature that is reviewed contains: books, articles, academic articles, conference proceedings, association papers, and dissertations. It contains the most pertinent studies and points to important past and current research and practices. It provides background and context, and shows how your research will contribute to the field.

A literature review should:

Provide a comprehensive and updated review of the literature;
Explain why this review has taken place;
Articulate a position or hypothesis;
Acknowledge and account for conflicting and corroborating points of view

From S age Research Methods

Purpose of a Literature Review

A literature review can be written as an introduction to a study to:

Demonstrate how a study fills a gap in research
Compare a study with other research that's been done

Or it can be a separate work (a research article on its own) which:

Organizes or describes a topic
Describes variables within a particular issue/problem

Limitations of a Literature Review

Some of the limitations of a literature review are:

It's a snapshot in time. Unlike other reviews, this one has beginning, a middle and an end. There may be future developments that could make your work less relevant.
It may be too focused. Some niche studies may miss the bigger picture.
It can be difficult to be comprehensive. There is no way to make sure all the literature on a topic was considered.
It is easy to be biased if you stick to top tier journals. There may be other places where people are publishing exemplary research. Look to open access publications and conferences to reflect a more inclusive collection. Also, make sure to include opposing views (and not just supporting evidence).

Source: Grant, Maria J., and Andrew Booth. “A Typology of Reviews: An Analysis of 14 Review Types and Associated Methodologies.” Health Information & Libraries Journal, vol. 26, no. 2, June 2009, pp. 91–108. Wiley Online Library, doi:10.1111/j.1471-1842.2009.00848.x.

Meryl Brodsky : Communication and Information Studies

Hannah Chapman Tripp : Biology, Neuroscience

Carolyn Cunningham : Human Development & Family Sciences, Psychology, Sociology

Larayne Dallas : Engineering

Janelle Hedstrom : Special Education, Curriculum & Instruction, Ed Leadership & Policy

Susan Macicak : Linguistics

Imelda Vetter : Dell Medical School

For help in other subject areas, please see the guide to library specialists by subject .

Periodically, UT Libraries runs a workshop covering the basics and library support for literature reviews. While we try to offer these once per academic year, we find providing the recording to be helpful to community members who have missed the session. Following is the most recent recording of the workshop, Conducting a Literature Review. To view the recording, a UT login is required.

October 26, 2022 recording
Last Updated: Oct 26, 2022 2:49 PM
URL: https://guides.lib.utexas.edu/literaturereviews

USC Libraries
Research Guides

Organizing Your Social Sciences Research Paper

Limitations of the Study
Purpose of Guide
Design Flaws to Avoid
Independent and Dependent Variables
Glossary of Research Terms
Reading Research Effectively
Narrowing a Topic Idea
Broadening a Topic Idea
Extending the Timeliness of a Topic Idea
Academic Writing Style
Applying Critical Thinking
Choosing a Title
Making an Outline
Paragraph Development
Research Process Video Series
Executive Summary
The C.A.R.S. Model
Background Information
The Research Problem/Question
Theoretical Framework
Citation Tracking
Content Alert Services
Evaluating Sources
Primary Sources
Secondary Sources
Tiertiary Sources
Scholarly vs. Popular Publications
Qualitative Methods
Quantitative Methods
Insiderness
Using Non-Textual Elements
Common Grammar Mistakes
Writing Concisely
Avoiding Plagiarism
Footnotes or Endnotes?
Further Readings
Generative AI and Writing
USC Libraries Tutorials and Other Guides
Bibliography

The limitations of the study are those characteristics of design or methodology that impacted or influenced the interpretation of the findings from your research. Study limitations are the constraints placed on the ability to generalize from the results, to further describe applications to practice, and/or related to the utility of findings that are the result of the ways in which you initially chose to design the study or the method used to establish internal and external validity or the result of unanticipated challenges that emerged during the study.

Price, James H. and Judy Murnan. “Research Limitations and the Necessity of Reporting Them.” American Journal of Health Education 35 (2004): 66-67; Theofanidis, Dimitrios and Antigoni Fountouki. "Limitations and Delimitations in the Research Process." Perioperative Nursing 7 (September-December 2018): 155-163. .

Importance of...

Always acknowledge a study's limitations. It is far better that you identify and acknowledge your study’s limitations than to have them pointed out by your professor and have your grade lowered because you appeared to have ignored them or didn't realize they existed.

Keep in mind that acknowledgment of a study's limitations is an opportunity to make suggestions for further research. If you do connect your study's limitations to suggestions for further research, be sure to explain the ways in which these unanswered questions may become more focused because of your study.

Acknowledgment of a study's limitations also provides you with opportunities to demonstrate that you have thought critically about the research problem, understood the relevant literature published about it, and correctly assessed the methods chosen for studying the problem. A key objective of the research process is not only discovering new knowledge but also to confront assumptions and explore what we don't know.

Claiming limitations is a subjective process because you must evaluate the impact of those limitations . Don't just list key weaknesses and the magnitude of a study's limitations. To do so diminishes the validity of your research because it leaves the reader wondering whether, or in what ways, limitation(s) in your study may have impacted the results and conclusions. Limitations require a critical, overall appraisal and interpretation of their impact. You should answer the question: do these problems with errors, methods, validity, etc. eventually matter and, if so, to what extent?

Price, James H. and Judy Murnan. “Research Limitations and the Necessity of Reporting Them.” American Journal of Health Education 35 (2004): 66-67; Structure: How to Structure the Research Limitations Section of Your Dissertation. Dissertations and Theses: An Online Textbook. Laerd.com.

Descriptions of Possible Limitations

All studies have limitations . However, it is important that you restrict your discussion to limitations related to the research problem under investigation. For example, if a meta-analysis of existing literature is not a stated purpose of your research, it should not be discussed as a limitation. Do not apologize for not addressing issues that you did not promise to investigate in the introduction of your paper.

Here are examples of limitations related to methodology and the research process you may need to describe and discuss how they possibly impacted your results. Note that descriptions of limitations should be stated in the past tense because they were discovered after you completed your research.

Possible Methodological Limitations

Sample size -- the number of the units of analysis you use in your study is dictated by the type of research problem you are investigating. Note that, if your sample size is too small, it will be difficult to find significant relationships from the data, as statistical tests normally require a larger sample size to ensure a representative distribution of the population and to be considered representative of groups of people to whom results will be generalized or transferred. Note that sample size is generally less relevant in qualitative research if explained in the context of the research problem.
Lack of available and/or reliable data -- a lack of data or of reliable data will likely require you to limit the scope of your analysis, the size of your sample, or it can be a significant obstacle in finding a trend and a meaningful relationship. You need to not only describe these limitations but provide cogent reasons why you believe data is missing or is unreliable. However, don’t just throw up your hands in frustration; use this as an opportunity to describe a need for future research based on designing a different method for gathering data.
Lack of prior research studies on the topic -- citing prior research studies forms the basis of your literature review and helps lay a foundation for understanding the research problem you are investigating. Depending on the currency or scope of your research topic, there may be little, if any, prior research on your topic. Before assuming this to be true, though, consult with a librarian! In cases when a librarian has confirmed that there is little or no prior research, you may be required to develop an entirely new research typology [for example, using an exploratory rather than an explanatory research design ]. Note again that discovering a limitation can serve as an important opportunity to identify new gaps in the literature and to describe the need for further research.
Measure used to collect the data -- sometimes it is the case that, after completing your interpretation of the findings, you discover that the way in which you gathered data inhibited your ability to conduct a thorough analysis of the results. For example, you regret not including a specific question in a survey that, in retrospect, could have helped address a particular issue that emerged later in the study. Acknowledge the deficiency by stating a need for future researchers to revise the specific method for gathering data.
Self-reported data -- whether you are relying on pre-existing data or you are conducting a qualitative research study and gathering the data yourself, self-reported data is limited by the fact that it rarely can be independently verified. In other words, you have to the accuracy of what people say, whether in interviews, focus groups, or on questionnaires, at face value. However, self-reported data can contain several potential sources of bias that you should be alert to and note as limitations. These biases become apparent if they are incongruent with data from other sources. These are: (1) selective memory [remembering or not remembering experiences or events that occurred at some point in the past]; (2) telescoping [recalling events that occurred at one time as if they occurred at another time]; (3) attribution [the act of attributing positive events and outcomes to one's own agency, but attributing negative events and outcomes to external forces]; and, (4) exaggeration [the act of representing outcomes or embellishing events as more significant than is actually suggested from other data].

Possible Limitations of the Researcher

Access -- if your study depends on having access to people, organizations, data, or documents and, for whatever reason, access is denied or limited in some way, the reasons for this needs to be described. Also, include an explanation why being denied or limited access did not prevent you from following through on your study.
Longitudinal effects -- unlike your professor, who can literally devote years [even a lifetime] to studying a single topic, the time available to investigate a research problem and to measure change or stability over time is constrained by the due date of your assignment. Be sure to choose a research problem that does not require an excessive amount of time to complete the literature review, apply the methodology, and gather and interpret the results. If you're unsure whether you can complete your research within the confines of the assignment's due date, talk to your professor.
Cultural and other type of bias -- we all have biases, whether we are conscience of them or not. Bias is when a person, place, event, or thing is viewed or shown in a consistently inaccurate way. Bias is usually negative, though one can have a positive bias as well, especially if that bias reflects your reliance on research that only support your hypothesis. When proof-reading your paper, be especially critical in reviewing how you have stated a problem, selected the data to be studied, what may have been omitted, the manner in which you have ordered events, people, or places, how you have chosen to represent a person, place, or thing, to name a phenomenon, or to use possible words with a positive or negative connotation. NOTE : If you detect bias in prior research, it must be acknowledged and you should explain what measures were taken to avoid perpetuating that bias. For example, if a previous study only used boys to examine how music education supports effective math skills, describe how your research expands the study to include girls.
Fluency in a language -- if your research focuses , for example, on measuring the perceived value of after-school tutoring among Mexican-American ESL [English as a Second Language] students and you are not fluent in Spanish, you are limited in being able to read and interpret Spanish language research studies on the topic or to speak with these students in their primary language. This deficiency should be acknowledged.

Structure and Writing Style

Information about the limitations of your study are generally placed either at the beginning of the discussion section of your paper so the reader knows and understands the limitations before reading the rest of your analysis of the findings, or, the limitations are outlined at the conclusion of the discussion section as an acknowledgement of the need for further study. Statements about a study's limitations should not be buried in the body [middle] of the discussion section unless a limitation is specific to something covered in that part of the paper. If this is the case, though, the limitation should be reiterated at the conclusion of the section.

If you determine that your study is seriously flawed due to important limitations , such as, an inability to acquire critical data, consider reframing it as an exploratory study intended to lay the groundwork for a more complete research study in the future. Be sure, though, to specifically explain the ways that these flaws can be successfully overcome in a new study.

But, do not use this as an excuse for not developing a thorough research paper! Review the tab in this guide for developing a research topic . If serious limitations exist, it generally indicates a likelihood that your research problem is too narrowly defined or that the issue or event under study is too recent and, thus, very little research has been written about it. If serious limitations do emerge, consult with your professor about possible ways to overcome them or how to revise your study.

When discussing the limitations of your research, be sure to:

Describe each limitation in detailed but concise terms;
Explain why each limitation exists;
Provide the reasons why each limitation could not be overcome using the method(s) chosen to acquire or gather the data [cite to other studies that had similar problems when possible];
Assess the impact of each limitation in relation to the overall findings and conclusions of your study; and,
If appropriate, describe how these limitations could point to the need for further research.

Remember that the method you chose may be the source of a significant limitation that has emerged during your interpretation of the results [for example, you didn't interview a group of people that you later wish you had]. If this is the case, don't panic. Acknowledge it, and explain how applying a different or more robust methodology might address the research problem more effectively in a future study. A underlying goal of scholarly research is not only to show what works, but to demonstrate what doesn't work or what needs further clarification.

Aguinis, Hermam and Jeffrey R. Edwards. “Methodological Wishes for the Next Decade and How to Make Wishes Come True.” Journal of Management Studies 51 (January 2014): 143-174; Brutus, Stéphane et al. "Self-Reported Limitations and Future Directions in Scholarly Reports: Analysis and Recommendations." Journal of Management 39 (January 2013): 48-75; Ioannidis, John P.A. "Limitations are not Properly Acknowledged in the Scientific Literature." Journal of Clinical Epidemiology 60 (2007): 324-329; Pasek, Josh. Writing the Empirical Social Science Research Paper: A Guide for the Perplexed. January 24, 2012. Academia.edu; Structure: How to Structure the Research Limitations Section of Your Dissertation. Dissertations and Theses: An Online Textbook. Laerd.com; What Is an Academic Paper? Institute for Writing Rhetoric. Dartmouth College; Writing the Experimental Report: Methods, Results, and Discussion. The Writing Lab and The OWL. Purdue University.

Writing Tip

Don't Inflate the Importance of Your Findings!

After all the hard work and long hours devoted to writing your research paper, it is easy to get carried away with attributing unwarranted importance to what you’ve done. We all want our academic work to be viewed as excellent and worthy of a good grade, but it is important that you understand and openly acknowledge the limitations of your study. Inflating the importance of your study's findings could be perceived by your readers as an attempt hide its flaws or encourage a biased interpretation of the results. A small measure of humility goes a long way!

Another Writing Tip

Negative Results are Not a Limitation!

Negative evidence refers to findings that unexpectedly challenge rather than support your hypothesis. If you didn't get the results you anticipated, it may mean your hypothesis was incorrect and needs to be reformulated. Or, perhaps you have stumbled onto something unexpected that warrants further study. Moreover, the absence of an effect may be very telling in many situations, particularly in experimental research designs. In any case, your results may very well be of importance to others even though they did not support your hypothesis. Do not fall into the trap of thinking that results contrary to what you expected is a limitation to your study. If you carried out the research well, they are simply your results and only require additional interpretation.

Lewis, George H. and Jonathan F. Lewis. “The Dog in the Night-Time: Negative Evidence in Social Research.” The British Journal of Sociology 31 (December 1980): 544-558.

Yet Another Writing Tip

Sample Size Limitations in Qualitative Research

Sample sizes are typically smaller in qualitative research because, as the study goes on, acquiring more data does not necessarily lead to more information. This is because one occurrence of a piece of data, or a code, is all that is necessary to ensure that it becomes part of the analysis framework. However, it remains true that sample sizes that are too small cannot adequately support claims of having achieved valid conclusions and sample sizes that are too large do not permit the deep, naturalistic, and inductive analysis that defines qualitative inquiry. Determining adequate sample size in qualitative research is ultimately a matter of judgment and experience in evaluating the quality of the information collected against the uses to which it will be applied and the particular research method and purposeful sampling strategy employed. If the sample size is found to be a limitation, it may reflect your judgment about the methodological technique chosen [e.g., single life history study versus focus group interviews] rather than the number of respondents used.

Boddy, Clive Roland. "Sample Size for Qualitative Research." Qualitative Market Research: An International Journal 19 (2016): 426-432; Huberman, A. Michael and Matthew B. Miles. "Data Management and Analysis Methods." In Handbook of Qualitative Research . Norman K. Denzin and Yvonna S. Lincoln, eds. (Thousand Oaks, CA: Sage, 1994), pp. 428-444; Blaikie, Norman. "Confounding Issues Related to Determining Sample Size in Qualitative Research." International Journal of Social Research Methodology 21 (2018): 635-641; Oppong, Steward Harrison. "The Problem of Sampling in qualitative Research." Asian Journal of Management Sciences and Education 2 (2013): 202-210.

<< Previous: 8. The Discussion
Next: 9. The Conclusion >>
Last Updated: May 25, 2024 4:09 PM
URL: https://libguides.usc.edu/writingguide

Research guides

Writing a Literature Review

Phase 1: scope of review, it's a literature review of what, precisely.

Need to Have a Precise Topic It is essential that one defines a research topic very carefully. For example, it should not be too far-reaching. The following is much too broad:

"Life and Times of Sigmund Freud"

However, this is more focused and specific and, accordingly, a more appropriate topic:

"An Analysis of the Relationship of Freud and Jung in the International Psychoanalytic Association, 1910-1914"

Limitations of Study In specifying precisely one's research topic, one is also specifying appropriate limitations on the research. Limiting, for example, by time, personnel, gender, age, location, nationality, etc. results in a more focused and meaningful topic.

Scope of the Literature Review It is also important to determine the precise scope of the literature review. For example,

What exactly will you cover in your review?
How comprehensive will it be?
How long? About how many citations will you use?
How detailed? Will it be a review of ALL relevant material or will the scope be limited to more recent material, e.g., the last five years.
Are you focusing on methodological approaches; on theoretical issues; on qualitative or quantitative research?
Will you broaden your search to seek literature in related disciplines?
Will you confine your reviewed material to English language only or will you include research in other languages too?

In evaluating studies, timeliness is more significant for some subjects than others. Scientists generally need more recent material. However, currency is often less of a factor for scholars in arts/humanities. Research published in 1920 about Plato's philosophy might be more relevant than recent studies.

<< Previous: Home
Next: Phase 2: Finding Information >>
Last Updated: Dec 5, 2023 2:26 PM
Subjects: Education , General
Tags: literature_review , literature_review_in_education

Advantages and disadvantages of literature review

This comprehensive article explores some of the advantages and disadvantages of literature review in research. Reviewing relevant literature is a key area in research, and indeed, it is a research activity in itself. It helps researchers investigate a particular topic in detail. However, it has some limitations as well.

What is literature review?

In order to understand the advantages and disadvantages of literature review, it is important to understand what a literature review is and how it differs from other methods of research. According to Jones and Gratton (2009) a literature review essentially consists of critically reading, evaluating, and organising existing literature on a topic to assess the state of knowledge in the area. It is sometimes called critical review.

A literature review is a select analysis of existing research which is relevant to a researcher’s selected topic, showing how it relates to their investigation. It explains and justifies how their investigation may help answer some of the questions or gaps in the chosen area of study (University of Reading, 2022).

A literature review is a term used in the field of research to describe a systematic and methodical investigation of the relevant literature on a particular topic. In other words, it is an analysis of existing research on a topic in order to identify any relevant studies and draw conclusions about the topic.

A literature review is not the same as a bibliography or a database search. Rather than simply listing references to sources of information, a literature review involves critically evaluating and summarizing existing research on a topic. As such, it is a much more detailed and complex process than simply searching databases and websites, and it requires a lot of effort and skills.

Advantages of literature review

Information synthesis

A literature review is a very thorough and methodical exercise. It can be used to synthesize information and draw conclusions about a particular topic. Through a careful evaluation and critical summarization, researchers can draw a clear and comprehensive picture of the chosen topic.

Familiarity with the current knowledge

According to the University of Illinois (2022), literature reviews allow researchers to gain familiarity with the existing knowledge in their selected field, as well as the boundaries and limitations of that field.

Creation of new body of knowledge

One of the key advantages of literature review is that it creates new body of knowledge. Through careful evaluation and critical summarisation, researchers can create a new body of knowledge and enrich the field of study.

Answers to a range of questions

Literature reviews help researchers analyse the existing body of knowledge to determine the answers to a range of questions concerning a particular subject.

Disadvantages of literature review

Time consuming

As a literature review involves collecting and evaluating research and summarizing the findings, it requires a significant amount of time. To conduct a comprehensive review, researchers need to read many different articles and analyse a lot of data. This means that their review will take a long time to complete.

Lack of quality sources

Researchers are expected to use a wide variety of sources of information to present a comprehensive review. However, it may sometimes be challenging for them to identify the quality sources because of the availability of huge numbers in their chosen field. It may also happen because of the lack of past empirical work, particularly if the selected topic is an unpopular one.

Descriptive writing

One of the major disadvantages of literature review is that instead of critical appreciation, some researchers end up developing reviews that are mostly descriptive. Their reviews are often more like summaries of the work of other writers and lack in criticality. It is worth noting that they must go beyond describing the literature.

Key features of literature review

Clear organisation

A literature review is typically a very critical and thorough process. Universities usually recommend students a particular structure to develop their reviews. Like all other academic writings, a review starts with an introduction and ends with a conclusion. Between the beginning and the end, researchers present the main body of the review containing the critical discussion of sources.

No obvious bias

A key feature of a literature review is that it should be very unbiased and objective. However, it should be mentioned that researchers may sometimes be influenced by their own opinions of the world.

Proper citation

One of the key features of literature review is that it must be properly cited. Researchers should include all the sources that they have used for information. They must do citations and provide a reference list by the end in line with a recognized referencing system such as Harvard.

To conclude this article, it can be said that a literature review is a type of research that seeks to examine and summarise existing research on a particular topic. It is an essential part of a dissertation/thesis. However, it is not an easy thing to handle by an inexperienced person. It also requires a lot of time and patience.

Hope you like this ‘Advantages and disadvantages of literature review’. Please share this with others to support our research work.

Author: M Rahman

M Rahman writes extensively online and offline with an emphasis on business management, marketing, and tourism. He is a lecturer in Management and Marketing. He holds an MSc in Tourism & Hospitality from the University of Sunderland. Also, graduated from Leeds Metropolitan University with a BA in Business & Management Studies and completed a DTLLS (Diploma in Teaching in the Life-Long Learning Sector) from London South Bank University.

How to be a good team player, competitive advantage for tourist destinations, advantages and disadvantages of snowball sampling.

Get science-backed answers as you write with Paperpal's Research feature

What is a Literature Review? How to Write It (with Examples)

A literature review is a critical analysis and synthesis of existing research on a particular topic. It provides an overview of the current state of knowledge, identifies gaps, and highlights key findings in the literature. 1 The purpose of a literature review is to situate your own research within the context of existing scholarship, demonstrating your understanding of the topic and showing how your work contributes to the ongoing conversation in the field. Learning how to write a literature review is a critical tool for successful research. Your ability to summarize and synthesize prior research pertaining to a certain topic demonstrates your grasp on the topic of study, and assists in the learning process.

What is the purpose of literature review?
a. Habitat Loss and Species Extinction:
b. Range Shifts and Phenological Changes:
c. Ocean Acidification and Coral Reefs:
d. Adaptive Strategies and Conservation Efforts:

How to write a good literature review

Choose a Topic and Define the Research Question:
Decide on the Scope of Your Review:
Select Databases for Searches:
Conduct Searches and Keep Track:
Review the Literature:
Organize and Write Your Literature Review:
How to write a literature review faster with Paperpal?
Frequently asked questions

What is a literature review?

A well-conducted literature review demonstrates the researcher’s familiarity with the existing literature, establishes the context for their own research, and contributes to scholarly conversations on the topic. One of the purposes of a literature review is also to help researchers avoid duplicating previous work and ensure that their research is informed by and builds upon the existing body of knowledge.

What is the purpose of literature review?

A literature review serves several important purposes within academic and research contexts. Here are some key objectives and functions of a literature review: 2

1. Contextualizing the Research Problem: The literature review provides a background and context for the research problem under investigation. It helps to situate the study within the existing body of knowledge.

2. Identifying Gaps in Knowledge: By identifying gaps, contradictions, or areas requiring further research, the researcher can shape the research question and justify the significance of the study. This is crucial for ensuring that the new research contributes something novel to the field.

Find academic papers related to your research topic faster. Try Research on Paperpal

3. Understanding Theoretical and Conceptual Frameworks: Literature reviews help researchers gain an understanding of the theoretical and conceptual frameworks used in previous studies. This aids in the development of a theoretical framework for the current research.

4. Providing Methodological Insights: Another purpose of literature reviews is that it allows researchers to learn about the methodologies employed in previous studies. This can help in choosing appropriate research methods for the current study and avoiding pitfalls that others may have encountered.

5. Establishing Credibility: A well-conducted literature review demonstrates the researcher’s familiarity with existing scholarship, establishing their credibility and expertise in the field. It also helps in building a solid foundation for the new research.

6. Informing Hypotheses or Research Questions: The literature review guides the formulation of hypotheses or research questions by highlighting relevant findings and areas of uncertainty in existing literature.

Literature review example

Let’s delve deeper with a literature review example: Let’s say your literature review is about the impact of climate change on biodiversity. You might format your literature review into sections such as the effects of climate change on habitat loss and species extinction, phenological changes, and marine biodiversity. Each section would then summarize and analyze relevant studies in those areas, highlighting key findings and identifying gaps in the research. The review would conclude by emphasizing the need for further research on specific aspects of the relationship between climate change and biodiversity. The following literature review template provides a glimpse into the recommended literature review structure and content, demonstrating how research findings are organized around specific themes within a broader topic.

Literature Review on Climate Change Impacts on Biodiversity:

Climate change is a global phenomenon with far-reaching consequences, including significant impacts on biodiversity. This literature review synthesizes key findings from various studies:

a. Habitat Loss and Species Extinction:

Climate change-induced alterations in temperature and precipitation patterns contribute to habitat loss, affecting numerous species (Thomas et al., 2004). The review discusses how these changes increase the risk of extinction, particularly for species with specific habitat requirements.

b. Range Shifts and Phenological Changes:

Observations of range shifts and changes in the timing of biological events (phenology) are documented in response to changing climatic conditions (Parmesan & Yohe, 2003). These shifts affect ecosystems and may lead to mismatches between species and their resources.

c. Ocean Acidification and Coral Reefs:

The review explores the impact of climate change on marine biodiversity, emphasizing ocean acidification’s threat to coral reefs (Hoegh-Guldberg et al., 2007). Changes in pH levels negatively affect coral calcification, disrupting the delicate balance of marine ecosystems.

d. Adaptive Strategies and Conservation Efforts:

Recognizing the urgency of the situation, the literature review discusses various adaptive strategies adopted by species and conservation efforts aimed at mitigating the impacts of climate change on biodiversity (Hannah et al., 2007). It emphasizes the importance of interdisciplinary approaches for effective conservation planning.

Strengthen your literature review with factual insights. Try Research on Paperpal for free!

Writing a literature review involves summarizing and synthesizing existing research on a particular topic. A good literature review format should include the following elements.

Introduction: The introduction sets the stage for your literature review, providing context and introducing the main focus of your review.

Opening Statement: Begin with a general statement about the broader topic and its significance in the field.
Scope and Purpose: Clearly define the scope of your literature review. Explain the specific research question or objective you aim to address.
Organizational Framework: Briefly outline the structure of your literature review, indicating how you will categorize and discuss the existing research.
Significance of the Study: Highlight why your literature review is important and how it contributes to the understanding of the chosen topic.
Thesis Statement: Conclude the introduction with a concise thesis statement that outlines the main argument or perspective you will develop in the body of the literature review.

Body: The body of the literature review is where you provide a comprehensive analysis of existing literature, grouping studies based on themes, methodologies, or other relevant criteria.

Organize by Theme or Concept: Group studies that share common themes, concepts, or methodologies. Discuss each theme or concept in detail, summarizing key findings and identifying gaps or areas of disagreement.
Critical Analysis: Evaluate the strengths and weaknesses of each study. Discuss the methodologies used, the quality of evidence, and the overall contribution of each work to the understanding of the topic.
Synthesis of Findings: Synthesize the information from different studies to highlight trends, patterns, or areas of consensus in the literature.
Identification of Gaps: Discuss any gaps or limitations in the existing research and explain how your review contributes to filling these gaps.
Transition between Sections: Provide smooth transitions between different themes or concepts to maintain the flow of your literature review.

Write and Cite as you go with Paperpal Research. Start now for free.

Conclusion: The conclusion of your literature review should summarize the main findings, highlight the contributions of the review, and suggest avenues for future research.

Summary of Key Findings: Recap the main findings from the literature and restate how they contribute to your research question or objective.
Contributions to the Field: Discuss the overall contribution of your literature review to the existing knowledge in the field.
Implications and Applications: Explore the practical implications of the findings and suggest how they might impact future research or practice.
Recommendations for Future Research: Identify areas that require further investigation and propose potential directions for future research in the field.
Final Thoughts: Conclude with a final reflection on the importance of your literature review and its relevance to the broader academic community.

Conducting a literature review

Conducting a literature review is an essential step in research that involves reviewing and analyzing existing literature on a specific topic. It’s important to know how to do a literature review effectively, so here are the steps to follow: 1

Choose a Topic and Define the Research Question:

Select a topic that is relevant to your field of study.
Clearly define your research question or objective. Determine what specific aspect of the topic do you want to explore?

Decide on the Scope of Your Review:

Determine the timeframe for your literature review. Are you focusing on recent developments, or do you want a historical overview?
Consider the geographical scope. Is your review global, or are you focusing on a specific region?
Define the inclusion and exclusion criteria. What types of sources will you include? Are there specific types of studies or publications you will exclude?

Select Databases for Searches:

Identify relevant databases for your field. Examples include PubMed, IEEE Xplore, Scopus, Web of Science, and Google Scholar.
Consider searching in library catalogs, institutional repositories, and specialized databases related to your topic.

Conduct Searches and Keep Track:

Develop a systematic search strategy using keywords, Boolean operators (AND, OR, NOT), and other search techniques.
Record and document your search strategy for transparency and replicability.
Keep track of the articles, including publication details, abstracts, and links. Use citation management tools like EndNote, Zotero, or Mendeley to organize your references.

Review the Literature:

Evaluate the relevance and quality of each source. Consider the methodology, sample size, and results of studies.
Organize the literature by themes or key concepts. Identify patterns, trends, and gaps in the existing research.
Summarize key findings and arguments from each source. Compare and contrast different perspectives.
Identify areas where there is a consensus in the literature and where there are conflicting opinions.
Provide critical analysis and synthesis of the literature. What are the strengths and weaknesses of existing research?

Organize and Write Your Literature Review:

Literature review outline should be based on themes, chronological order, or methodological approaches.
Write a clear and coherent narrative that synthesizes the information gathered.
Use proper citations for each source and ensure consistency in your citation style (APA, MLA, Chicago, etc.).
Conclude your literature review by summarizing key findings, identifying gaps, and suggesting areas for future research.

Whether you’re exploring a new research field or finding new angles to develop an existing topic, sifting through hundreds of papers can take more time than you have to spare. But what if you could find science-backed insights with verified citations in seconds? That’s the power of Paperpal’s new Research feature!

How to write a literature review faster with Paperpal?

Paperpal, an AI writing assistant, integrates powerful academic search capabilities within its writing platform. With the Research feature, you get 100% factual insights, with citations backed by 250M+ verified research articles, directly within your writing interface with the option to save relevant references in your Citation Library. By eliminating the need to switch tabs to find answers to all your research questions, Paperpal saves time and helps you stay focused on your writing.

Here’s how to use the Research feature:

Ask a question: Get started with a new document on paperpal.com. Click on the “Research” feature and type your question in plain English. Paperpal will scour over 250 million research articles, including conference papers and preprints, to provide you with accurate insights and citations.
Review and Save: Paperpal summarizes the information, while citing sources and listing relevant reads. You can quickly scan the results to identify relevant references and save these directly to your built-in citations library for later access.
Cite with Confidence: Paperpal makes it easy to incorporate relevant citations and references into your writing, ensuring your arguments are well-supported by credible sources. This translates to a polished, well-researched literature review.

The literature review sample and detailed advice on writing and conducting a review will help you produce a well-structured report. But remember that a good literature review is an ongoing process, and it may be necessary to revisit and update it as your research progresses. By combining effortless research with an easy citation process, Paperpal Research streamlines the literature review process and empowers you to write faster and with more confidence. Try Paperpal Research now and see for yourself.

Frequently asked questions

A literature review is a critical and comprehensive analysis of existing literature (published and unpublished works) on a specific topic or research question and provides a synthesis of the current state of knowledge in a particular field. A well-conducted literature review is crucial for researchers to build upon existing knowledge, avoid duplication of efforts, and contribute to the advancement of their field. It also helps researchers situate their work within a broader context and facilitates the development of a sound theoretical and conceptual framework for their studies.

Literature review is a crucial component of research writing, providing a solid background for a research paper’s investigation. The aim is to keep professionals up to date by providing an understanding of ongoing developments within a specific field, including research methods, and experimental techniques used in that field, and present that knowledge in the form of a written report. Also, the depth and breadth of the literature review emphasizes the credibility of the scholar in his or her field.

Before writing a literature review, it’s essential to undertake several preparatory steps to ensure that your review is well-researched, organized, and focused. This includes choosing a topic of general interest to you and doing exploratory research on that topic, writing an annotated bibliography, and noting major points, especially those that relate to the position you have taken on the topic.

Literature reviews and academic research papers are essential components of scholarly work but serve different purposes within the academic realm. 3 A literature review aims to provide a foundation for understanding the current state of research on a particular topic, identify gaps or controversies, and lay the groundwork for future research. Therefore, it draws heavily from existing academic sources, including books, journal articles, and other scholarly publications. In contrast, an academic research paper aims to present new knowledge, contribute to the academic discourse, and advance the understanding of a specific research question. Therefore, it involves a mix of existing literature (in the introduction and literature review sections) and original data or findings obtained through research methods.

Literature reviews are essential components of academic and research papers, and various strategies can be employed to conduct them effectively. If you want to know how to write a literature review for a research paper, here are four common approaches that are often used by researchers. Chronological Review: This strategy involves organizing the literature based on the chronological order of publication. It helps to trace the development of a topic over time, showing how ideas, theories, and research have evolved. Thematic Review: Thematic reviews focus on identifying and analyzing themes or topics that cut across different studies. Instead of organizing the literature chronologically, it is grouped by key themes or concepts, allowing for a comprehensive exploration of various aspects of the topic. Methodological Review: This strategy involves organizing the literature based on the research methods employed in different studies. It helps to highlight the strengths and weaknesses of various methodologies and allows the reader to evaluate the reliability and validity of the research findings. Theoretical Review: A theoretical review examines the literature based on the theoretical frameworks used in different studies. This approach helps to identify the key theories that have been applied to the topic and assess their contributions to the understanding of the subject. It’s important to note that these strategies are not mutually exclusive, and a literature review may combine elements of more than one approach. The choice of strategy depends on the research question, the nature of the literature available, and the goals of the review. Additionally, other strategies, such as integrative reviews or systematic reviews, may be employed depending on the specific requirements of the research.

The literature review format can vary depending on the specific publication guidelines. However, there are some common elements and structures that are often followed. Here is a general guideline for the format of a literature review: Introduction: Provide an overview of the topic. Define the scope and purpose of the literature review. State the research question or objective. Body: Organize the literature by themes, concepts, or chronology. Critically analyze and evaluate each source. Discuss the strengths and weaknesses of the studies. Highlight any methodological limitations or biases. Identify patterns, connections, or contradictions in the existing research. Conclusion: Summarize the key points discussed in the literature review. Highlight the research gap. Address the research question or objective stated in the introduction. Highlight the contributions of the review and suggest directions for future research.

Both annotated bibliographies and literature reviews involve the examination of scholarly sources. While annotated bibliographies focus on individual sources with brief annotations, literature reviews provide a more in-depth, integrated, and comprehensive analysis of existing literature on a specific topic. The key differences are as follows:

References

Denney, A. S., & Tewksbury, R. (2013). How to write a literature review.  Journal of criminal justice education ,  24 (2), 218-234.
Pan, M. L. (2016).  Preparing literature reviews: Qualitative and quantitative approaches . Taylor & Francis.
Cantero, C. (2019). How to write a literature review.  San José State University Writing Center .

Paperpal is an AI writing assistant that help academics write better, faster with real-time suggestions for in-depth language and grammar correction. Trained on millions of research manuscripts enhanced by professional academic editors, Paperpal delivers human precision at machine speed.

Try it for free or upgrade to  Paperpal Prime , which unlocks unlimited access to premium features like academic translation, paraphrasing, contextual synonyms, consistency checks and more. It’s like always having a professional academic editor by your side! Go beyond limitations and experience the future of academic writing.  Get Paperpal Prime now at just US$19 a month!

6 Tips for Post-Doc Researchers to Take Their Career to the Next Level

Self-plagiarism in research: what it is and how to avoid it, you may also like, how to ace grant writing for research funding..., powerful academic phrases to improve your essay writing , how to write a high-quality conference paper, how paperpal’s research feature helps you develop and..., how paperpal is enhancing academic productivity and accelerating..., how to write a successful book chapter for..., academic editing: how to self-edit academic text with..., 4 ways paperpal encourages responsible writing with ai, what are scholarly sources and where can you..., how to write a hypothesis types and examples .

Limitations of the literature – A guide to a sticky problem

One of the more difficult tasks set for a Masters and especially PhD student is to identify limitations in the literature under review. Postgraduate students are required not only to “tell” (repeat what an author/s say) but to critically engage the literature being reviewed. This work typically appears in a separate chapter in the thesis or dissertation under the title Literature Review. To help you, here are 10 typical examples often used to identify limitations in a particular body of literature—with a caveat—using education examples.

1. The sample size is too small and therefore not representative of the phenomenon being studied This is fine but be aware of the fact that a single case study (e.g. the leadership practices of a non-traditional principal) is completely acceptable in a qualitative study. Here depth makes up for spread (representivity) for the purpose of the study is different e.g. an ethnographic case study of one principal.

2. The context is limiting and therefore not representative of all This too is acceptable e.g. a study of subject competency levels of 10 science teachers in rural Limpopo province. However, always specific WHAT it is about the context that is limiting e.g. nothing about science teachers in urban areas. A common mistake is to say ‘that study was done in America’ but that is not helpful in itself. What is it about the American context that might not be applicable in the Southern African context?

3. The study is dated as in out-of-date e.g. 1985 This is important as a criticism except in the case of historical studies or in reference to a classical piece of work that set the standard for understanding a particular problem. For example, Michael Apple’s 1979 Ideology and Curriculum might be dated but it is the landmark criticism in the politics of curriculum and often deserves referencing as a launching pad for more recent studies.

4. The period of observation was too short This is valuable especially in the qualitative study of classrooms where the researcher visits a history class of 10 teachers once and then jumps to make major findings and conclusions from a 40-minute lesson per teacher. This is a common problem with qualitative research in education in South Africa—the lack of extended engagement in the field.

5. The theoretical framework or theory is limiting A researcher might be using a particular theory to explain events e.g. a behaviorist account of learner discipline where you believe that a constructivist account offers more insight into why learners misbehave in classrooms. The onus is still on you to explain why the rival theory is ‘better’ than the one used.

6. The methodological approach is inappropriate for the question posed A study uses a self-reporting questionnaire completed by teachers to determine their competency levels in mathematics teaching. You could argue that direct observation of actual teaching is a much more direct measure of teaching competency in mathematics since teachers might overestimate their own levels of competency in the subject (by the way, there is research to back you up with such a claim)

7. The research question or instrument(s) is biased A study that asks ‘why teachers are incompetent’ or ‘the students are lazy’ already assumes the fact ahead of the inquiry itself. The research question can and should be posed in an open-ended manner to allow for more than one outcome. In the controversial SU study on Coloured women’s cognitive abilities and health styles it was found that the measuring instrument used was found in other studies to be flawed. And the study of code-switching in a Grade 6 language classroom would clearly show bias if the researcher was competent in English alone—unless, of course, this deficiency is remedied in the study design.

8. The study does not differentiate between subjects or contexts A report might give the results of a study for academic performance in Grade 12 economics in the National Senior Certificate, which is fine, but does not differentiate between the results of the former white schools and those of black schools thereby concealing variable performance and, for the sake of reform, where exactly added support might be needed.

9. The literature reviewed does not cover the subject at all or from a particular perspective The conclusion that ‘there is no research on topic X’ is a common one among students and often wrong. The fact that you have not yet found literature on topic X does not mean that such references do not exist. However, there are instances in which the case can and should be made that topic X appears to be understudied given the literature reviewed. Hint: always check with your supervisor and other experts whether the ‘not yet researched’ claim is justified. More plausible are studies which critique a dominant perspective on a subject and offer a new lens on the same issue (see point #5) above.

10. The study is based largely on opinion and uses primarily secondary sources This is a very valid criticism of many articles posing as research studies where the evidence is often anecdotal (not ‘thick descriptions’ as anthropologists like to point out) and drawing on other opinion pieces rather than solid research on the topic.

REMEMBER: THESE NOTES ARE NOT ABOUT HOW TO IDENTIFY LIMITATIONS IN LITERATURE REVIEWED FOR YOUR STUDY. IT IS NOT ABOUT LISTING THE LIMITATIONS OF YOUR STUDY IN THE COURSE OF WRITING UP A RESEARCH PROPOSAL OR THE THESIS ITSELF, EVEN THOUGH THIS COULD HELP YOU DEFEND SELF-IDENTIFIED LIMITATIONS OF YOUR RESEARCH.

The upsides to a pandemic: from jobs to teaching, it’s not all bad news

Let’s face facts, the 2020 school year is lost. so what to do, you may also like, corrupted: a study of chronic dysfunction in south..., learning under lockdown – 100 student voices, ten elements of a good literature review, ten rules for turning a dissertation into a....

This was very helpful to me.

Truly helpful. So much valuable information on this site that’s worth spreading across all social media platforms.

Health profession education hackathons: a scoping review of current trends and best practices

Azadeh Rooholamini ORCID: orcid.org/0000-0002-9638-7953 1 &
Mahla Salajegheh ORCID: orcid.org/0000-0003-0651-3467 1

BMC Medical Education volume 24 , Article number: 554 ( 2024 ) Cite this article

Metrics details

While the concept of hacking in education has gained traction in recent years, there is still much uncertainty surrounding this approach. As such, this scoping review seeks to provide a detailed overview of the existing literature on hacking in health profession education and to explore what we know (and do not know) about this emerging trend.

This was a scoping review study using specific keywords conducted on 8 databases (PubMed, Embase, Scopus, Web of Science, ERIC, PsycINFO, Education Source, CINAHL) with no time limitation. To find additional relevant studies, we conducted a forward and backward searching strategy by checking the reference lists and citations of the included articles. Studies reporting the concept and application of hacking in education and those articles published in English were included. Titles, abstracts, and full texts were screened and the data were extracted by 2 authors.

Twenty-two articles were included. The findings are organized into two main categories, including (a) a Description of the interventions and expected outcomes and (b) Aspects of hacking in health profession education.

Hacking in health profession education refers to a positive application that has not been explored before as discovering creative and innovative solutions to enhance teaching and learning. This includes implementing new instructional methods, fostering collaboration, and critical thinking to utilize unconventional approaches.

Peer Review reports

Introduction

Health professions education is a vital component of healthcare systems to provide students with the knowledge, skills, and attitudes necessary to provide high-quality care to patients [ 1 ]. However, with the advent of innovative technologies and changing global dynamics, there is a growing need to incorporate new educational methods to prepare medical science students for the future [ 2 ].

Although traditional methods can be effective for certain learning objectives and in specific contexts and may create a stable and predictable learning environment, beneficial for introducing foundational concepts, memorization, and repetition, however, they may not fully address the diverse needs and preferences of today’s learners [ 3 ]. Some of their limitations may be limited engagement, passive learning, lack of personalization, and limited creativity and critical thinking [ 4 ].

As Du et al. (2022) revealed the traditional teaching model fails to capture the complex needs of today’s students who require practical and collaborative learning experiences. Students nowadays crave interactive learning methods that enable them to apply theoretical knowledge in real-world situations [ 5 ].

To achieve innovation in health professions education, engaging students and helping them learn, educators should use diverse and new educational methods [ 6 ]. Leary et al. (2022) described how schools of nursing can integrate innovation into their mission and expressed that education officials must think strategically about the knowledge and skills the next generation of students will need to learn, to build an infrastructure that supports innovation in education, research, and practice, and provide meaningful collaboration with other disciplines to solve challenging problems. Such efforts should be structured and built on a deliberate plan and include curricular innovations, and experiential learning in the classroom, as well as in practice and research [ 7 ].

The incorporation of technology in education is another aspect that cannot be ignored. Technology has revolutionized the way we communicate and learn, providing opportunities for students to access information and resources beyond the traditional education setting. According to the advancement of technology in education, hacking in education is an important concept in this field [ 8 ].

Hack has become an increasingly popular term in recent years, with its roots in the world of computer programming and technology [ 9 ]. However, the term “hack” is not limited solely to the realm of computers and technology. It can also refer to a creative approach to problem-solving, a willingness to challenge established norms, and a desire to find new and innovative ways to accomplish tasks [ 10 ]. At its core, hacking involves exploring and manipulating technology systems to gain a deeper understanding of how they work. This process of experimentation and discovery can be applied to many different fields, including education [ 11 ].

In education, the concept of “hack” has become popular as educators seek innovative ways to engage students and improve learning outcomes. As Wizel (2019) described “hack in education” involves applying hacker mentality and techniques, such as using technology creatively and challenging traditional structures, to promote innovation within the educational system [ 12 ]. These hacking techniques encompass various strategies like gamification, hackathons, creating new tools and resources for education, use of multimedia presentations, online forums, and educational apps for project-based learning [ 9 ]. Butt et al. (2020) demonstrated the effectiveness of hack in education in promoting cross-disciplinary learning in medical education [ 13 ]. However, concerns exist about the negative connotations and ethical implications of hacking in education, with some educators hesitant to embrace these techniques in their classrooms [ 7 , 14 ].

However, while the concept of hack in education has gained traction in recent years, there is still a great deal of uncertainty surrounding its implementation and efficacy. As such, this scoping review seeks to provide a comprehensive overview of the existing literature on hacking in health profession education (HPE), to explore what we know (and do not know) about this emerging trend. To answer this research question, this study provided a comprehensive review of the literature related to hacking in HPE. Specifically, it explored the various ways in which educators are using hack techniques to improve learning outcomes, increase student engagement, and promote creativity in the classroom.

Methods and materials

This scoping review was performed based on the Arksey and O’Malley Framework [ 15 ] and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement to answer some questions about the hacking approach in health professions education [ 16 ].

Search strategies

The research question was “What are the aspects of hacking in education?“. We used the PCC framework which is commonly used in scoping reviews to develop the research question [ 17 ]. In such a way the Population assumed as learners, the Concept supposed as aspects of hacking in education, and the Context is considered to be the health profession education.

A systematic literature search was conducted on June 2023, using the following terms and their combinations: hack OR hacking OR hackathon AND education, professional OR “medical education” OR “medical training” OR “nursing education” OR “dental education” OR “pharmacy education” OR “health professions education” OR “health professional education” OR “higher education” OR “healthcare education” OR “health care education” OR “students, health occupations” OR “medical student” OR “nursing student” OR “dental student” OR “pharmacy student” OR “schools, health occupations” OR “medical school” OR “nursing school” OR “dental school” OR “pharmacy school”) in 8 databases (PubMed, Embase, Scopus, Web of Science, ERIC, PsycINFO, Education Source, CINAHL) with no time limitation. (A copy of the search strategy is included in Appendix 1 ). To find additional relevant studies, we conducted a forward and backward searching strategy by checking the reference lists and citations of the included articles.

Inclusion and exclusion criteria

Original research reporting the different aspects of hacking in health professions education and published in English was included. We excluded commentaries, editorials, opinion pieces, perspectives, reviews, calls for change, needs assessments, and other studies in which no real interventions had been employed.

Study identification

After removing the duplicates, each study potentially meeting the inclusion criteria was independently screened by 2 authors (A.R. and M.S.). Then, the full texts of relevant papers were assessed independently by the 2 authors for relevance and inclusion. Disagreements at either step were resolved when needed until a consensus was reached.

Quality assessment of the studies

We used the BEME checklist [ 18 ], consisting of 11 indicators, to assess the quality of studies. Each indicator was rated as “met,” “unmet,” or “unclear.” To be deemed of high quality, articles should meet at least 7 indicators. The quality of the full text of potentially relevant studies was assessed by 2 authors (A.R. and M.S.). Disagreements were resolved through discussion. No study was removed based on the results of the quality assessment.

Data extraction and synthesis

To extract the data from the studies, a data extraction form was designed based on the results of the entered studies. A narrative synthesis was applied as a method for comparing, contrasting, synthesizing, and interpreting the results of the selected papers. All outcomes relevant to the review question were reported. The two authors reviewed and coded each included study using the data extraction form independently.

A total of 645 titles were found, with a further four titles identified through the hand-searching of reference lists of all reviewed articles. After removing the duplicate references, 422 references remained. After title screening, 250 studies were considered for abstract screening, and 172 studies were excluded. After the abstract screening, 73 studies were considered for full-text screening, and 177 studies were excluded due to reasons such as:1. being irrelevant, 2. loss of data, and 3. language limitation. 22 studies were included in the final analysis. The 2020 PRISMA diagram for the included studies is shown in Fig. 1 . The quality was evaluated as “high” in 12 studies, “moderate” in 7 studies, and “low” in 3 studies.

PRISMA flow diagram for included studies

The review findings are organized into two main categories: (a) Description of the interventions and expected outcomes and (b) Aspects of hacking in health profession education.

Description of the interventions and expected outcomes

The description of the studies included the geographical context of the interventions, type, and number of participants, focus of the intervention, evaluation methodology, and outcomes. Table 1 displays a summary of these features.

Geographical context

Of the 22 papers reviewed, 11 studies (45.4%) took place in the United States of America [ 7 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 ], two studies in Pakistan [ 13 , 29 ], one study performed in international locations [ 30 ], and the remainder being in the United Kingdom [ 31 ], Germany [ 32 ], Finland [ 33 ], Australia [ 34 ], Austria [ 35 ], Thailand [ 36 ], Africa [ 37 ], and Canada [ 38 ].

Type and number of participants

Hacking in HPE interventions covered a wide range and multiple audiences. The majority of interventions targeted students (17 studies, 77.2%) [ 7 , 13 , 20 , 21 , 23 , 24 , 25 , 26 , 27 , 29 , 30 , 31 , 32 , 33 , 36 , 37 , 38 ]. Their field of education was reported differently including medicine, nursing, engineering, design, business, kinesiology, and computer sciences. Also, they were undergraduates, postgraduates, residents, and post-docs. Ten interventions (45.4%) were designed for physicians [ 13 , 19 , 21 , 24 , 25 , 26 , 28 , 29 , 33 , 35 ]. Their field of practice was reported diverse including psychology, radiology, surgery, and in some cases not specified. Eight (36.3%) studies focused on staff which included healthcare staff, employees of the university, nurses, care experts, and public health specialists [ 13 , 22 , 26 , 28 , 29 , 30 , 32 , 35 ]. Interestingly, nine of the hacking in HPE interventions (40.9%) welcomed specialists from other fields outside of health sciences and medicine [ 13 , 19 , 22 , 25 , 26 , 28 , 29 , 33 , 35 ]. Their field of practice was very diverse including engineers, theologians, artists, entrepreneurs, designers, informaticists, IT professionals, business professionals, industry members, data scientists, and user interface designers. The next group of participants was faculty with 5 studies (22.7%) [ 7 , 23 , 32 , 34 , 36 ]. An intervention (4.5%) targeted the researchers [ 27 ]. The number of participants in the interventions ranged from 12 to 396. Three studies did not specify the number of their participants.

The focus of the intervention

The half of interventions aimed to improve HPE (12 studies, 54.5%) [ 7 , 13 , 21 , 23 , 24 , 26 , 28 , 30 , 31 , 32 , 34 , 38 ], with a secondary emphasis on enhancing clinical or health care [ 19 , 22 , 25 , 29 , 33 , 35 , 36 , 37 ]. Two studies highlighted the improvement in entrepreneurship skills of health professions [ 19 , 20 ]. One study aimed to improve the research skills of health professionals [ 27 ].

Evaluation methodology

Methods to evaluate hacking in HPE interventions included end-of-program questionnaires, pre-and post-test measures to assess attitudinal or cognitive change, self-assessment of post-training performance, project-based assessment through expert judgment and feedback, interviews with participants, and direct observations of behavior.

Hacking in HPE interventions has resulted in positive outcomes for participants. Five studies found high levels of satisfaction for participants with the intervention [ 21 , 31 , 32 , 33 , 37 ]. Some studies evaluated learning, which included changes in attitudes, knowledge, and skills. In most studies, participants demonstrated a gain in knowledge regarding awareness of education’s strengths and problems, in the desire to improve education by enhancement of awareness for technological possibilities [ 7 , 13 , 19 , 21 , 23 , 30 , 32 , 33 , 34 , 35 , 38 ]. Some studies found improving participant familiarity with healthcare innovation [ 19 , 22 , 24 , 25 , 26 , 33 , 36 , 37 ]. Some participants reported a positive change in attitudes towards HPE as a result of their involvement in hacking interventions. They cited a greater awareness of personal strengths and limitations, increased motivation, more confidence, and a notable appreciation of the benefits of professional development [ 20 , 21 , 29 , 34 ]. Some studies also demonstrated behavioral change. In one study, changes were noted in developing a successful proof-of-concept of a radiology training module with elements of gamification, enhancement engagement, and learning outcomes in radiology training [ 28 ]. In a study, participants reported building relationships when working with other members which may be students, faculty, or healthcare professionals [ 7 ]. Five studies found a high impact on participant perceptions and attitudes toward interdisciplinary collaboration [ 22 , 26 , 27 , 36 , 38 ].

Aspects of hacking in health profession education

The special insights of hacking in HPE included the adaptations considered in the interventions, the challenges of interventions, the suggestions for future interventions, and Lessons learned.

Adaptations

The adaptations are considered to improve the efficacy of hacking in HPE interventions. We found that 21 interventions were described as hackathons. Out of this number, some were only hackathons, and some others had benefited from hackathons besides other implications of hacking in education. Therefore, most of the details in this part of the findings are presented with a focus on hackathons. The hackathon concept has been limited to the industry and has not been existing much in education [ 39 , 40 ]. In the context of healthcare, hackathons are events exposing healthcare professionals to innovative methodologies while working with interdisciplinary teams to co-create solutions to the problems they see in their practice [ 19 , 22 , 24 , 25 , 30 , 41 , 42 ].

Some hackathons used various technologies for internal and external interactions during the hackathon including Zoom, Gmail, WhatsApp, Google Meet, etc [ 37 ]. . . Almost all hackathons were planned and performed in the following steps including team formation, team working around the challenges, finding innovative solutions collaboratively, presenting the solutions and being evaluating based on some criteria including whether they work, are good ideas with a suitable problem/solution fit, how a well-designed experience and execution, etc. For example, in the hackathon conducted by Pathanasethpong et al. (2017), the judging criteria included innovativeness, feasibility, and value of the projects [ 36 ]. Also, they managed the cultural differences between the participants through strong support of leadership, commitment, flexibility, respect for culture, and willingness to understand each other’s needs [ 36 ].

Despite valuable adaptations, several challenges were reported. The hackathons faced some challenges such as limited internet connectivity, time limitations, limited study sample, power supply, associated costs, lack of diversity among participants, start-up culture, and lack of organizational support [ 13 , 19 , 25 , 28 , 30 , 34 , 37 ]. Some interventions reported the duration of the hackathon was deemed too short to develop comprehensive solutions [ 37 ]. One study identified that encouraging experienced physicians and other healthcare experts to participate in healthcare hackathons is an important challenge [ 26 ].

Suggestions for the future

Future hackathons should provide internet support for participants and judges, invite investors and philanthropists to provide seed funding for winning teams, and enable equal engagement of all participants to foster interdisciplinary collaboration [ 37 ]. Subsequent hackathons have to evaluate the effect of implementation or durability of the new knowledge in practice [ 19 , 28 ]. Wang et al. (2018) performed a hackathon to bring together interdisciplinary teams of students and professionals to collaborate, brainstorm, and build solutions to unmet clinical needs. They suggested that future healthcare hackathon organizers a balanced distribution of participants and mentors, publicize the event to diverse clinical specialties, provide monetary prizes and investor networking opportunities for post-hackathon development, and establish a formal vetting process for submitted needs that incorporates faculty review and well-defined evaluation criteria [ 22 ]. Most interventions had an overreliance on self-assessments to assess their effectiveness. To move forward, we should consider the use of novel assessment methods [ 30 ].

Lessons learned

Based on the findings of hackathons, they have developed efficient solutions to different problems related to public health and medical education. Some of these solutions included developing novel computer algorithms, designing and building model imaging devices, designing more approachable online patient user websites, developing initial prototypes, developing or optimizing data analysis tools, and creating a mobile app to optimize hospital logistics [ 25 , 26 , 27 , 36 ]. Staziaki et al. (2022) performed an intervention to develop a radiology curriculum. Their strategies were creating new tools and resources, gamification, and conducting a hackathon with colleagues from five different countries. They revealed a radiology training module that utilized gamification elements, including experience points and a leaderboard, for annotation of chest radiographs of patients with tuberculosis [ 28 ].

Most hackathons provide an opportunity for medical health professionals to inter-professional and inter-university collaboration and use technology to produce innovative solutions to public health and medical education [ 7 , 23 , 26 , 30 , 37 , 38 ]. For example, one study discussed that hackathons allowed industry experts and mentors to connect with students [ 37 ]. In the study by Mosene et al. (2023), results offer an insight into the possibilities of hackathons as a teaching/learning event for educational development and thus can be used for large-scale-assessments and qualitative interviews for motivational aspects to participate in hackathons, development of social skills and impact on job orientation [ 32 ].

The participants’ willingness to continue working on the projects after the hackathons was also reported in some papers [ 13 , 29 , 33 ]. One study highlights the potential of hackathons to address unmet workforce needs and the preference of female surgeons for small-group discussions and workshops [ 24 ]. Craddock et al. (2016) discussed that their intervention provided a unique opportunity for junior researchers and those from developing economies who have limited opportunities to interact with peers and senior scientists outside their home institution [ 27 ].

Dameff et al. (2019) developed and evaluated a novel high-fidelity simulation-based cybersecurity training program for healthcare providers. They found significant improvements in the knowledge and confidence of participants related to clinical cybersecurity after completing the simulation exercise. They also reported high levels of satisfaction with the training program [ 21 ].

This scoping review provided a detailed overview of the existing literature on hacking in health profession education and explored what we know (and do not know) about this emerging trend. Our results emphasized the increasing pattern of utilizing hacking in HPE for enhancing teaching and learning, problem-solving, and product generation. Our findings revealed that elements of hacking in HPE can include; innovation, creativity, critical thinking, and collaboration. Innovation is a critical element of hacking in education that holds different meanings for different disciplines. Those involved in HPE consider innovation to create new tools and resources [ 7 , 28 ], hackathons [ 13 , 19 , 20 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 ], gamification [ 28 ], and simulation-based training [ 21 ].

This study by introducing a different perspective or a new application of hacking that has not been explored before allows for a broader understanding of hacking and its potential positive applications in HPE. Although it does mention “hacking,” it does not refer to the malicious or illegal activities often associated with the term [ 43 , 44 ]. The results of this study indicate incorporating hacking into HPE aimed at improving education and enhancing clinical or healthcare had positive outcomes in learning, attitudes, knowledge, and skills. Embracing hacking in HPE revolutionizes traditional teaching methods, promotes interdisciplinary collaboration, leverages cutting-edge technologies, and cultivates a culture of lifelong learning, ultimately enhancing clinical outcomes and the healthcare system as a whole [ 13 , 20 , 21 , 22 , 26 , 27 , 28 , 30 , 31 , 32 , 33 , 34 , 36 , 37 , 38 ].

This study reveals that hackathons are more prominent in the United States of America (USA) education system compared to other countries due to the culture of innovation and entrepreneurship [ 7 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 ]. It is important to note that while hackathons are more prominent in the USA, they are also gaining popularity in other countries [ 13 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 ]. This mindset directly contributes to designing effective interventions and driving innovation across different countries and regions around the world. In comparison to other educational interventions, in hacking within education studies, the geographical context, the focus of the intervention, and outcomes can play a significant role in shaping the educational intervention. The relationship between them can be explained through Socio-cultural theory which emphasizes the influence of social interactions and cultural factors in learning and development [ 45 ]. According to this theory, factors such as cultural values, societal norms, availability of technological resources, access to educational opportunities, and collaboration with local communities all play a role in shaping the outcomes of hacking in education. In light of the findings, creating a positive impact on education through “hacking” as innovation requires adaptations and overcoming challenges. Adaptations could involve modifying traditional teaching methods, incorporating new technologies into the learning process, or adopting new pedagogical approaches, such as project-based learning or blended learning [ 40 ]. Adapting education through hacking means finding innovative solutions to improve teaching methods, student engagement, and overall learning outcomes [ 46 ]. Challenges refer to the obstacles or barriers that educators, leaders, or organizations may face when trying to implement innovative changes in education could be related to resistance to change, lack of resources or funding, bureaucratic hurdles, or simply the complexities of navigating a rapidly changing educational landscape [ 47 ]. Therefore, driving positive change requires leading with creativity, perseverance, and collaboration [ 48 ]. In this way, different leadership and management approaches and models can help to create change. For example, studies show that Kotter’s 8-Step Change theory can be considered a guide for educators to lead innovation in education through hacking [ 49 ].

With a clear definition of innovation, the next is to consider how to systematize and embed a culture of innovation within the educational organization. An important component of this strategy is tying innovation to professional, school, and university priorities. Innovation is a human-centered endeavor and requires key stakeholders’ engagement to identify challenges and opportunities. Our findings emphasized that while meeting with multiple stakeholders is critical, developing other champions of an innovation focus is essential. Consider resources available in developing internal and external advisory members, local entrepreneurs, or leaders in innovation roles. Other strategies can be used to guide the design and development of innovation programs including co-design sessions, focus groups, and the use of external consultants.

Faculty members are the main actors of change and the most effective source of creativity in education. They have a significant role to play in driving change in education by preparing the ground for creativity, adapting to new changes, and stimulating change within the classroom. They can create a positive and innovative learning environment that benefits both students and the entire organization [ 50 , 51 ].

For many faculty members, innovation will be a new area of inquiry. Hence, based on our findings we recommend to the planners and organizers of faculty development programs to design and implement some programs about innovation in the teaching and learning process considering these three key elements: building knowledge, acquiring skills in applying rigorous innovation methodologies to identifying and solving problems, and generating opportunities to participate in innovation activities can way to develop an interest in innovation and elevate it as a school goal and priority [ 51 , 52 ].

Overall, these findings demonstrate that the hackathon effectively met its objectives in the case of HPE by promoting interdisciplinary collaboration, building relationships, facilitating learning, developing innovation, knowledge acquisition, practical problem-solving skills, cross-disciplinary tools for teaching and learning, and inquiry-based learning. In addition, findings reveal the positive outcomes of hackathons in HPE including increasing confidence levels as innovators, enhancing awareness of technological possibilities for future healthcare givers, improved familiarity with healthcare innovation and teaching entrepreneurship, improving engagement, and learning outcomes in training, high participant satisfaction, and increased motivation with the program. Also, Hackathon in HPE emphasizes the role of multidisciplinary teams and technology in solving medical education problems and encourages disciplinary collaborations to improve data collection and analysis [ 7 , 13 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 ]. A potential gap of knowledge in this study is the lack of research on the long-term impact and sustainability of hacking in HPE. While the study highlights the positive outcomes of incorporating hacking into education, it does not delve into the long-term effects or address the potential challenges in maintaining and sustaining these innovative practices. Additionally, there is limited mention of the assessment methods used to measure the effectiveness of hacking in education, which could be an area for further investigation.

Some limitations of this study are including, this comprehensive study includes a straightforward research question, a predefined search strategy, and inclusion and exclusion criteria for studies that summarize all relevant studies, allowing for a detailed understanding of the available evidence. This had some limitations when it came to collecting eligible articles. Since this review extracted only published research, there are educational interventions that are reported at conferences but have not yet been published in the literature. The moderate quality of full-text studies is indeed a limitation of this study. Future research should consider including higher-quality full-text studies to enhance the robustness of the findings.

Although we searched for articles using general keywords, these were limited to hackathon keywords. Further research is needed to conduct hackathons in HPE to drive sustained innovation and crowd-source solutions. First, research should investigate how to enhance faculty and student engagement and retention to foster hackathons in HPE. Second, a multidisciplinary study is crucial to strike a balance between embracing innovation and evaluating its impact to ensure its successful integration into the education system. Third, future research could focus on exploring the long-term impact, sustainability, and assessment methods of incorporating hackathons in HPE.

Hacking in the health profession educational context refers to the positive applications in teaching and learning that have not been explored before. Embracing hacking requires adaptations, overcoming challenges, and driving change through creativity, perseverance, and collaboration. The goal of hacking in health profession education is to create a more dynamic, adaptable, and effective educational system that meets the needs of all learners and prepares them for success in the rapidly evolving 21st-century economy.

Data availability

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Van Schalkwyk SC, Hafler J, Brewer TF, Maley MA, Margolis C, McNamee L, et al. Transformative learning as pedagogy for the health professions: a scoping review. Med Educ. 2019;53(6):547–58.

Article Google Scholar

Green M, Wayne DB, Neilson EG. Medical education 2020—charting a path forward. JAMA. 2019;322(10):934–5.

Koolivand H, Shooreshi MM, Safari-Faramani R, Borji M, Mansoory MS, Moradpoor H, et al. Comparison of the effectiveness of virtual reality-based education and conventional teaching methods in dental education: a systematic review. BMC Med Educ. 2024;24(1):8.

Saini S, Kamath G, Mathew M, DSouza D. Case based interprofessional learning versus traditional teaching methods for medical, nursing, and physiotherapy students. Internet J Allied Health Sci Pract. 2024;22(1):27.

Google Scholar

Du L, Zhao L, Xu T, Wang Y, Zu W, Huang X et al. Blended learning vs traditional teaching: the potential of a novel teaching strategy in nursing education-a systematic review and meta-analysis. Nurse Educ Pract. 2022:103354.

Gyimah N. Assessing technological innovation on education in the world of coronavirus (COVID-19). Annal Immunol Immunotherapy. 2022;4(1):000158.

Leary M, Villarruel AM, Richmond TS. Creating an innovation infrastructure in academic nursing. J Prof Nurs. 2022;38:83–8.

Záhorec J, Nagyová A, Hašková A. Teachers’ Attitudes to Incorporation Digital Means in Teaching Process in Relation to the Subjects they Teach. Int J Eng Pedagogy. 2019;9(4).

Barpi F, Dalmazzo D, De Blasio A, Vinci F. Hacking higher education: rethinking the EduHack course. Educ Sci. 2021;11(2):40.

Kim HJ, Jang HY. Sustainable technology integration in underserved area schools: the impact of perceived student change on teacher continuance intention. Sustainability. 2020;12(12):4802.

Iglesias-Sánchez PP, Jambrino-Maldonado C, de las Heras-Pedrosa C. Training entrepreneurial competences with open innovation paradigm in higher education. Sustainability. 2019;11(17):4689.

Wizel M, editor. Teachers as Hackers: Implications for 21st Century Teacher Education. 7th Teaching & Education Conference; 2019; London: Lesley University.

Butt WA, Shahood Q, Farooqi WH, Ghias K, Sabzwari S, Mian A. Healthcare hackathons: fostering medical education through innovation in a developing country: a case study from Pakistan. BMJ Innovations. 2020;7(1):1–6.

Maimon D, Louderback ER. Cyber-dependent crimes: an interdisciplinary review. Annual Rev Criminol. 2019;2:191–216.

Arksey H, O’malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Int J Surg. 2021;88:105906.

Pollock D, Peters MD, Khalil H, McInerney P, Alexander L, Tricco AC, et al. Recommendations for the extraction, analysis, and presentation of results in scoping reviews. JBI Evid Synthesis. 2023;21(3):520–32.

Buckley S, Coleman J, Davison I, Khan KS, Zamora J, Malick S, et al. The educational effects of portfolios on undergraduate student learning: a best evidence Medical Education (BEME) systematic review. BEME Guide 11 Med Teacher. 2009;31(4):282–98.

Preiksaitis C, Dayton JR, Kabeer R, Bunney G, Boukhman M. Teaching principles of Medical Innovation and Entrepreneurship through hackathons: Case Study and qualitative analysis. JMIR Med Educ. 2023;9(1):e43916.

Kagan O, Sciasci NG, Koszalinski RS, Kagan DH, Leary M, Nadel H. Nurses’ confidence in starting a new venture, startup or project in the context of nurse-led hackathons: results of prehackathon survey. Nurs Outlook. 2023;71(3):101961.

Dameff CJ, Selzer JA, Fisher J, Killeen JP, Tully JL. Clinical cybersecurity training through novel high-fidelity simulations. J Emerg Med. 2019;56(2):233–8.

Wang JK, Roy SK, Barry M, Chang RT, Bhatt AS. Institutionalizing healthcare hackathons to promote diversity in collaboration in medicine. BMC Med Educ. 2018;18(1):1–9.

Saffari SS, Frederick Lambert R, Dang L, Pagni S, Dragan IF. Integrating student feedback during Dental Curriculum Hack-A-thon. BMC Med Educ. 2018;18:1–6.

Ruzgar NM, Ahuja C, Kristin EY, Sallam A, Rosenthal R, Killelea B. How we do it: creation of a workforce development-focused track at a surgical hackathon. J Surg Educ. 2020;77(5):1028–32.

Cooper K, Siefert A, Weinreb J. Skills beyond the Reading room: training in innovation and collaboration at a radiology hackathon. J Am Coll Radiol. 2018;15(3):466–8.

Silver JK, Binder DS, Zubcevik N, Zafonte RD. Healthcare hackathons provide educational and innovation opportunities: a case study and best practice recommendations. J Med Syst. 2016;40:1–7.

Cameron Craddock R, Margulies S, Bellec D, Nolan Nichols P, Alcauter B, Barrios SA et al. F,. Brainhack: a collaborative workshop for the open neuroscience community. GigaScience. 2016;5(1):s13742-016-0121-x.

Staziaki PV, Santinha JA, Coelho MO, Angulo D, Hussain M, Folio LR. Gamification in radiology training module developed during the society for imaging informatics in medicine annual meeting hackathon. J Digit Imaging. 2022;35(3):714–22.

Butt WA, Shariff A, Khan S, Mian AI. Global surgery hackathons: a Case Study from Pakistan. Surg Innov. 2021;28(4):496–501.

Wang JK, Pamnani RD, Capasso R, Chang RT. An extended hackathon model for collaborative education in medical innovation. J Med Syst. 2018;42:1–8.

Kienzler H, Fontanesi C. Learning through inquiry: a global health hackathon. Teach High Educ. 2017;22(2):129–42.

Mosene K, Kleinesper C, Prokop G, Caroli F, Teufel D, Berberat PO et al. OPEN Hackathon at the TUM School of Medicine, Germany. GMS J Med Educ. 2023;40(2).

Kolog EA, Sutinen E, Nygren E. Hackathon for learning digital theology in computer science. Int J Mod Educ Comput Sci. 2016;8(6):1.

Brand G, Collins J, Bedi G, Bonnamy J, Barbour L, Ilangakoon C, et al. I teach it because it is the biggest threat to health: integrating sustainable healthcare into health professions education. Med Teach. 2021;43(3):325–33.

Ströckl DE, Perchtaler M, Oberzaucher J, editors. Interdisciplinary Hackathons-A method to embed digitization in Healthcare Education. dHealth; 2022.

Pathanasethpong A, Soomlek C, Morley K, Morley M, Polpinit P, Dagan A, et al. Tackling regional public health issues using mobile health technology: event report of an mHealth hackathon in Thailand. JMIR mHealth uHealth. 2017;5(10):e8259.

Babatunde AO, Brimmo FO, Arama UO, Onyinyechi MG, Josephat KA, Osiene AO. A Public Health Hackathon for Medical Students in Africa: Process, Outcome and Recommendations. medRxiv. 2023:2023.01. 28.23284802.

Muñoz-Leija MA, Paul BR, Shi G, Dixit I, Quiroga-Garza A, Elizondo-Omaña RE, et al. THE HIVE: a multidisciplinary approach to medical education. Eur J Anat. 2021;25(1):101–6.

Oyetade KE, Zuva T, Harmse A. Factors influencing Hackathon Adoption for Learning Information Technology (IT) programming modules. TEM J. 2022;11(3).

Horton PA, Jordan SS, Weiner S, Lande M, editors. Project-based learning among engineering students during short-form hackathon events. 2018 ASEE Annual Conference & Exposition; 2018.

Walker A, Ko N. Bringing medicine to the digital age via hackathons and beyond. J Med Syst. 2016;40:1–3.

Ahmed R, Mian AI. A case for global surgery in Pakistan: implementation through Multi-disciplinary Engagement. JPMA J Pakistan Med Association. 2019;69(1):S98–100.

Clarke R, Youngstein T. Cyberattack on Britain’s National Health Service—a wake-up call for modern medicine. N Engl J Med. 2017;377(5):409–11.

Grimes S, Wirth A. Holding the line: events that shaped healthcare cybersecurity. Biomedical Instrum Technol. 2017;51(s6):30–2.

Mahn H, John-Steiner V. Vygotsky and sociocultural approaches to teaching and learning. Handbook of Psychology, Second Edition. 2012;7.

Wizel M. Preparing educational hackers. Contemporary pedagogies in teacher education and development. IntechOpen; 2018.

Ávila LV, Leal Filho W, Brandli L, Macgregor CJ, Molthan-Hill P, Özuyar PG, et al. Barriers to innovation and sustainability at universities around the world. J Clean Prod. 2017;164:1268–78.

Matthew CT, Sternberg RJ. Leading innovation through collaboration. Innovation through collaboration. Volume 12. Emerald Group Publishing Limited; 2006. pp. 27–52.

Kotter J. The 8-step process for leading change. Kotter Int. 2012.

Potter EM. Perceptions of Creativity among Faculty in Higher Education. 2013.

Steinert Y, Mann K, Centeno A, Dolmans D, Spencer J, Gelula M, et al. A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME Guide 8. Med Teach. 2006;28(6):497–526.

Steinert Y. Perspectives on faculty development: aiming for 6/6 by 2020. Perspect Med Educ. 2012;1:31–42.

Download references

Acknowledgements

Not applicable.

This study was conducted with the financial support of the Vice-Chancellor for Research and Technology of Kerman University of Medical Sciences (project number: 402000210).The role of the funding body was to provide support for data collection and analysis.

Author information

Authors and affiliations.

Department of Medical Education, Medical Education Development Center, Kerman University of Medical Sciences, Kerman, Iran

Azadeh Rooholamini & Mahla Salajegheh

You can also search for this author in PubMed Google Scholar

Contributions

AR and MS formulated the research idea, extracted data, and performed the analysis of the data, wrote the manuscript, and edited the draft of the paper. All authors approved the final manuscript.

Corresponding author

Correspondence to Mahla Salajegheh .

Ethics declarations

Ethical approval and consent to participate.

The Ethics Committee of Kerman University of Medical Sciences approved the study (No: IR.KMU.REC.1402.251).

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Rooholamini, A., Salajegheh, M. Health profession education hackathons: a scoping review of current trends and best practices. BMC Med Educ 24 , 554 (2024). https://doi.org/10.1186/s12909-024-05519-7

Download citation

Received : 05 December 2023

Accepted : 06 May 2024

Published : 21 May 2024

DOI : https://doi.org/10.1186/s12909-024-05519-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Medical education
Health profession education

BMC Medical Education

ISSN: 1472-6920

Submission enquiries: [email protected]
General enquiries: [email protected]

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

View all journals
Explore content
About the journal
Publish with us
Sign up for alerts
Review Article
Published: 22 May 2024

Review of evidence for treatments of acute non arteritic anterior ischemic optic neuropathy

Pooja Parthasarathi ORCID: orcid.org/0009-0004-0363-2940 1 &
Heather E. Moss ORCID: orcid.org/0000-0003-4373-447X 1 , 2

Eye ( 2024 ) Cite this article

1 Altmetric

Metrics details

Optic nerve diseases
Vision disorders

To review treatment modalities that have been studied in acute non arteritic anterior ischemic optic neuropathy (NAION).

We performed a comprehensive literature search of English language publications in the last 5 years, with human species and NAION. Articles were reviewed to identify those that described original research on treatment of acute NAION. Study type, setting, duration, interventions, and results were extracted and articles were reviewed for biases and limitations.

We identified 22 kinds of treatment varying by compound and modality. These include topical, intravitreal, and systemic drugs as well as surgical approaches. Evidence for efficacy ranges from expert opinion to randomized control trials.

Conclusions

Although several treatments are utilized in practice, none of these have high quality evidence of efficacy to improve visual outcomes. Continued collaborative research is necessary to complete high quality studies in order identify effective therapies for this rare and blinding disease.

目的: 回顾急性非动脉炎性前部缺血性视神经病变 (NAION) 的治疗方法。

方法: 我们对过去五年的英语文献进行全面检索, 检索词包括人类物种和NAION。通过回顾文献, 以确定描述急性NAION治疗的原始研究的文献。研究提取了研究类型、研究设计、持续时间、干预措施和结果, 并审查了文章的偏倚和局限性。

结果: 我们确定了22种不同药物和治疗方法。其中包括局部、玻璃体内和全身药物以及手术方法。疗效证据包括专家意见到随机对照试验等。

结论: 虽然几种治疗方法已用于实践, 但这些方法都没有高质量的疗效证据可改善视力效果。持续的合作研究对于完成高质量的研究是必要的, 以确定这种罕见和致盲疾病的有效疗法。

This is a preview of subscription content, access via your institution

Access options

Subscribe to this journal

Receive 18 print issues and online access

251,40 € per year

only 13,97 € per issue

Buy this article

Purchase on Springer Link
Instant access to full article PDF

Prices may be subject to local taxes which are calculated during checkout

How do I recognise and manage visual snow syndrome?

Topical insulin in neurotrophic keratopathy after diabetic vitrectomy

MOG antibody-associated optic neuritis

Hayreh SS Ischemic optic neuropathy. Prog Retinal Eye Res. 2009;28:34–62.

Article Google Scholar

Wu KY, Evoy F. Naion: Diagnosis and management. American Academy of Ophthalmology. 2023. https://www.aao.org/eyenet/article/naion-diagnosis-and-management . Accessed 7 Feb 2024.

Lee J-Y, Park K-A, Oh SY Prevalence and incidence of non-arteritic anterior ischaemic optic neuropathy in South Korea: a nationwide population-based study. Br J Ophthalmol. 2018;102:936–41.

Article PubMed Google Scholar

Johnson LN, Arnold AC. Incidence of nonarteritic and arteritic anterior ischemic optic neuropathy. J Neuro Ophthalmol. 1994;14. https://doi.org/10.1097/00041327-199403000-00011 .

Cestari DM, Gaier ED, Bouzika P, Blachley TS, De Lott LB, Rizzo JF, et al. Demographic, systemic, and ocular factors associated with nonarteritic anterior ischemic optic neuropathy. Ophthalmology. 2016;123:2446–55.

Ischemic Optic Neuropathy Decompression Trial Study Group. Characteristics of patients with nonarteritic anterior ischemic optic neuropathy eligible for the ischemic optic neuropathy decompression trial. Arch Ophthalmol. 1996;114:1366–74.

Rizzo JF, Lessell S Optic neuritis and ischemic optic neuropathy. Arch Ophthalmol. 1991;109:1668–72.

Tisavipat N, Stiebel-Kalish H, Palevski D, Bialer OY, Moss HE, Chaitanuwong P, et al. Acute optic neuropathy in older adults: differentiating between MOGAD optic neuritis and non-arteritic anterior ischemic optic neuropathy. Neurology 2024;11:e200214.

Google Scholar

Arnold A, Wang M. Ischemic optic neuropathies. In: Ophthalmology. Elsevier; 2014. p. 884–9.

Part I: Anatomy. 2020–2021 BCSC Basic and Clinical Science CourseTM. 2021. Accessed 7 Feb 2024.

Hayreh S, Podhajsky P, Zimmerman MB Role of nocturnal arterial hypotension in optic nerve head ischemic disorders. Ophthalmologica. 1999;213:76–96.

Article CAS PubMed Google Scholar

Purvin V, King R, Kawasaki A, Yee R Anterior ischemic optic neuropathy in eyes with optic disc drusen. Arch Ophthalmol. 2004;122:48–53.

Modarres M, Sanjari MS, Falavarjani KG Vitrectomy and release of presumed epipapillary vitreous traction for treatment of nonarteritic anterior ischemic optic neuropathy associated with partial posterior vitreous detachment. Ophthalmology. 2007;114:340–4.

Kupersmith MJ, Sibony PA, Dave S Nonarteritic anterior ischemic optic neuropathy induced retinal folds and deformations. Investig Ophthalmol Vis Sci. 2017;58:4286–91.

Thompson AC, Bhatti MT, Gospe SM Spectral-domain optical coherence tomography of the vitreopapillary interface in acute nonarteritic anterior ischemic optic neuropathy. Am J Ophthalmol. 2018;195:199–208.

Wheeler LA, Lai R, Woldemussie E. From the lab to the clinic: activation of an alpha-2 agonist pathway is neuroprotective in models of retinal and optic nerve injury. Eur J Ophthalmol. 1999;9. https://doi.org/10.1177/112067219900901s09 .

Wilhelm B, Lüdtke H, Wilhelm H Efficacy and tolerability of 0.2% brimonidine tartrate for the treatment of acute non-arteritic anterior ischemic optic neuropathy (NAION): A 3-month, double-masked, randomised, placebo-controlled trial. Graefe’s Arch Clin Exp Ophthalmol. 2005;244:551–8.

Fazzone HE Does topical brimonidine tartrate help naion? Br J Ophthalmol. 2003;87:1193–4.

Article CAS PubMed PubMed Central Google Scholar

Wood JPM, Schmidt KG, Melena J, Chidlow G, Allmeier H, Osborne NN The β-adrenoceptor antagonists metipranolol and timolol are retinal neuroprotectants: comparison with betaxolol. Exp Eye Res. 2003;76:505–16.

Treatment Study for Ischemic Optic Neuropathy With Ophthalmic Timolol Maleate 0.5%. ClinicalTrials.gov [Internet]. [cited 2024 Feb 5].

Leonhardt H, Grigoleit HG Effects of pentoxifylline on red blood cell deformability and blood viscosity under hyperosmolar conditions. Naunyn-Schmiedebergs Arch für Pharmakologie. 1977;299:197–200.

Article CAS Google Scholar

Dawson DL, Cutler BS, Hiatt WR, Hobson RW, Martin JD, Bortey EB, et al. A comparison of cilostazol and pentoxifylline for treating intermittent claudication. Am J Med. 2000;109:523–30.

Jull AB, Arroll B, Parag V, Waters J Pentoxifylline for treating venous leg ulcers. Cochrane Database Syst Rev. 2012;2012:CD001733.

PubMed Central Google Scholar

Prokosch V, Thanos S Visual outcome of patients following NAION after treatment with adjunctive fluocortolone. Restor Neurol Neurosci. 2014;32:381–9.

CAS PubMed Google Scholar

Gale J, Chu ER, Frousiakis SE, Srinivas S, Karanjia R, Tan O, et al. Effects of pentoxifylline on blood flow in patients with non-arteritic ischemic optic neuropathy. Investig Ophthalmol Vis Sci. 2014;55:6232.

Beck RW, Hayreh SS, Podhajsky PA, Tan ES, Moke PS Aspirin therapy in nonarteritic anterior ischemic optic neuropathy. Am J Ophthalmol. 1997;123:212–7.

Newman NJ, Scherer R, Langenberg P, Kelman S, Feldon S, Kaufman D, et al. The fellow eye in NAION: report from the ischemic optic neuropathy decompression trial follow-up study. Am J Ophthalmol. 2002;134:317–28.

Botelho PJ, Johnson LN, Arnold AC The effect of aspirin on the visual outcome of nonarteritic anterior ischemic optic neuropathy. Am J Ophthalmol. 1996;121:450–1.

Chauhan BC, LeVatte TL, Jollimore CA, Yu PK, Reitsamer HA, Kelly MEM, et al. Model of endothelin-1–induced chronic optic neuropathy in rat. Investig Ophthalmol Vis Sci. 2004;45:144–52.

Resch H, Karl K, Weigert G, Wolzt M, Hommer A, Schmetterer L, et al. Effect of dual endothelin receptor blockade on ocular blood flow in patients with glaucoma and healthy subjects. Investig Ophthalmol Vis Sci. 2009;50:358–63.

Chiquet C, Vignal C, Gohier P, Heron E, Thuret G, Rougier MB, et al. Treatment of nonarteritic anterior ischemic optic neuropathy with an endothelin antagonist: ENDOTHELION (ENDOTHELin antagonist receptor in Ischemic Optic Neuropathy)—a multicentre randomised controlled trial protocol. Trials. 2022;23:916.

University Hospital, Grenoble. Effect of Bosentan in Patients With Non Arteritic Ischemic Optic Neuropathy [Internet]. clinicaltrials.gov; 2022 Jul [cited 2023 Dec 31]. Report No.: NCT02377271.

Bodis-Wollner I Visual deficits related to dopamine deficiency in experimental animals and Parkinson’s disease patients. Trends Neurosci. 1990;13:296–302.

Johnson LN, Gould TJ, Krohel GB Effect of levodopa and carbidopa on recovery of visual function in patients with nonarteritic anterior ischemic optic neuropathy of longer than six months’ duration. Am J Ophthalmol. 1996;121:77–83.

Johnson LN, Guy ME, Krohel GB, Madsen RW Levodopa may improve vision loss in recent-onset, nonarteritic anterior ischemic optic neuropathy. Ophthalmology. 2000;107:521–6.

Beck RW, Ferris FL Does levodopa improve visual function in NAION? Ophthalmology. 2000;107:1431–4.

2020–2021 BCSC Basic and Clinical Science CourseTM [Internet]. [cited 2023 Dec 27].

Hayreh SS Anterior ischaemic optic neuropathy. III. Treatment, prophylaxis, and differential diagnosis. Br J Ophthalmol. 1974;58:981–9.

Rebolleda G, Pérez-López M, Casas-LLera P, Contreras I, Muñoz-Negrete FJ Visual and anatomical outcomes of non-arteritic anterior ischemic optic neuropathy with high-dose systemic corticosteroids. Graefe’s Arch Clin Exp Ophthalmol. 2013;251:255–60.

Hayreh SS, Zimmerman MB Non-arteritic anterior ischemic optic neuropathy: role of systemic corticosteroid therapy. Graefe’s Arch Clin Exp Ophthalmol. 2008;246:1029–46.

Saxena R, Singh D, Sharma M, James M, Sharma P, Menon V Steroids versus no steroids in nonarteritic anterior ischemic optic neuropathy: a randomized controlled trial. Ophthalmology. 2018;125:1623–7.

Pakravan M, Sanjari N, Esfandiari H, Pakravan P, Yaseri M The effect of high-dose steroids, and normobaric oxygen therapy, on recent onset non-arteritic anterior ischemic optic neuropathy: a randomized clinical trial. Graefe’s Arch Clin Exp Ophthalmol. 2016;254:2043–8.

Kaderli B, Avci R, Yucel A, Guler K, Gelisken O Intravitreal triamcinolone improves recovery of visual acuity in nonarteritic anterior ischemic optic neuropathy. J Neuro-Ophthalmol. 2007;27:164–8.

Sohn BJ, Chun BY, Kwon JY The effect of an intravitreal triamcinolone acetonide injection for acute nonarteritic anterior ischemic optic neuropathy. Korean J Ophthalmol. 2009;23:59–61.

Article PubMed PubMed Central Google Scholar

Radoi C, Garcia T, Brugniart C, Ducasse A, Arndt C Intravitreal triamcinolone injections in non-arteritic anterior ischemic optic neuropathy. Graefe’s Arch Clin Exp Ophthalmol. 2014;252:339–45.

Zhong L, Bradley J, Schubert W, Ahmed E, Adamis AP, Shima DT, et al. Erythropoietin promotes survival of retinal ganglion cells in DBA/2J Glaucoma Mice. Investig Ophthalmol Vis Sci. 2007;48:1212–8.

Pakravan M, Esfandiari H, Hassanpour K, Razavi S, Pakravan P The effect of combined systemic erythropoietin and steroid on non-arteritic anterior ischemic optic neuropathy: a prospective study. Curr Eye Res. 2017;42:1079–84.

Nikkhah H, Golalipour M, Doozandeh A, Pakravan M, Yaseri M, Esfandiari H The effect of systemic erythropoietin and oral prednisolone on recent-onset non-arteritic anterior ischemic optic neuropathy: a randomized clinical trial. Graefe’s Arch Clin Exp Ophthalmol. 2020;258:2291–7.

Modarres M, Falavarjani KG, Nazari H, Sanjari MS, Aghamohammadi F, Homaii M, et al. Intravitreal erythropoietin injection for the treatment of non-arteritic anterior ischaemic optic neuropathy. Br J Ophthalmol. 2011;95:992–5.

Pece A, Querques G, Quinto A, Isola V Intravitreal ranibizumab injection for nonarteritic ischemic optic neuropathy. J Ocul Pharmacol Ther. 2010;26:523–7.

Rootman DB, Gill HS, Margolin EA Intravitreal bevacizumab for the treatment of nonarteritic anterior ischemic optic neuropathy: a prospective trial. Eye. 2013;27:538–44.

Saatci AO, Taskin O, Selver OB, Yaman A, Bajin MS Efficacy of intravitreal ranibizumab injection in acute nonarteritic ischemic optic neuropathy: a long-term follow up. Open Ophthalmol J. 2013;7:58–62.

Bennett JL, Thomas S, Olson JL, Mandava N Treatment of nonarteritic anterior ischemic optic neuropathy with intravitreal bevacizumab. J Neuro Ophthalmol. 2007;27:238–40.

Mansour AM, Schwartz SG, Gregori NZ, Yehoshua Z, Freund KB, Dellatorre K, et al. Insight into 8 patients with nonarteritic anterior ischemic optic neuropathy following anti-VEGF injections. J Neuro-Ophthalmol. 2012;32:193.

Liu PK, Wen YT, Lin W, Kapupara K, Tai M, Tsai RK Neuroprotective effects of low-dose G-CSF plus meloxicam in a rat model of anterior ischemic optic neuropathy. Sci Rep. 2020 ;10:10351.

Abri Aghdam K, Aghajani A, Ashraf Khorasani M, Soltan Sanjari M, Chaibakhsh S, Habibi A, et al. Intravitreal injection of the granulocyte-colony stimulating factor for the treatment of non-arteritic anterior ischemic optic neuropathy: a pilot study. Semin Ophthalmol. 2021;36:649–57.

Ahmed Z, Kalinski H, Berry M, Almasieh M, Ashush H, Slager N, et al. Ocular neuroprotection by siRNA targeting caspase-2. Cell Death Dis. 2011;2:e173.

Antoszyk A, Katz B, Singh R, Gurses-Ozden R, Erlich S, Rothenstein D, et al. A phase I open label, dose escalation trial Of QPI-1007 delivered by a single intravitreal (IVT) injection to subjects with low visual acuity and acute non-arteritic anterior ischemic optic neuropathy (NAION). Investig Ophthalmol Vis Sci. 2013;54:4575.

Quark Pharmaceuticals. A Phase 2/3, Randomized, Double-Masked, Sham-Controlled Trial of QPI-1007 Delivered By Single or Multi-Dose Intravitreal Injection(s) to Subjects With Acute Nonarteritic Anterior Ischemic Optic Neuropathy (NAION) [Internet]. clinicaltrials.gov; 2020 Jul [cited 2023 Dec 31]. Report No.: NCT02341560.

Weiss JN, Levy S, Benes SC Stem Cell Ophthalmology Treatment Study: bone marrow derived stem cells in the treatment of non-arteritic ischemic optic neuropathy (NAION). Stem Cell Investig. 2017;4:94.

Pastor JC, Pastor-Idoate S, López-Paniagua M, Para M, Blazquez F, Murgui E, et al. Intravitreal allogeneic mesenchymal stem cells: a non-randomized phase II clinical trial for acute non-arteritic optic neuropathy. Stem Cell Research & Therapy [Internet]. 2023 [cited 2024 Jan 23];14. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10512539/

Rath EZ, Hazan Z, Adamsky K, Solomon A, Segal ZI, Levin LA Randomized controlled phase 2a study of RPh201 in previous nonarteritic anterior ischemic optic neuropathy. J Neuro Ophthalmol. 2019;39:291–8.

Study Details | Efficacy & Safety of RPh201 Treatment in Patients With Previous Nonarteritic Anterior Ischemic Optic Neuropathy (NAION) | ClinicalTrials.gov [Internet]. [cited 2024 Feb 5].

Fibrinogen, cholesterol, and smoking as risk factors for non-arteritic anterior ischaemic optic neuropathy—PubMed [Internet]. [cited 2024 Jan 3].

Haas A, Walzl M, Jesenik F, Walzl B, Berghold A, Berglöff J, et al. Application of HELP in nonarteritic anterior ischemic optic neuropathy: a prospective, randomized, controlled study. Graefe’s Arch Clin Exp Ophthalmol. 1997;235:14–9.

Tolias CM, Reinert M, Seiler R, Gilman C, Scharf A, Bullock MR Normobaric hyperoxia–induced improvement in cerebral metabolism and reduction in intracranial pressure in patients with severe head injury: a prospective historical cohort-matched study. J Neurosurg. 2004;101:435–44.

Sharifipour F, Baradaran-Rafii A, Idani E, Zamani M, Jabbarpoor Bonyadi MH Oxygen therapy for acute ocular chemical or thermal burns: a pilot study. Am J Ophthalmol. 2011;151:823–8.

Arnold AC, Hepler RS, Lieber M, Alexander JM Hyperbaric oxygen therapy for nonarteritic anterior ischemic optic neuropathy. Am J Ophthalmol. 1996;122:535–41.

Ischemic Optic Neuropathy Decompression Trial Research Group. Ischemic optic neuropathy decompression trial twenty-four–month update. Arch Ophthalmol. 2000;118:793–7

Dickersin K Optic nerve decompression surgery for nonarteritic anterior ischemic optic neuropathy (NAION) is not effective and may be harmful. JAMA J Am Med Assoc. 1995;273:625.

Levin LA Lessons from the ischemic optic neuropathy decompression trial: a decade later. Arch Ophthalmol. 2007;125:1570–1.

Chen TW, Wu PY, Wen YT, Desai TD, Huang CT, Liu PK, et al. Vitamin B3 provides neuroprotection via antioxidative stress in a rat model of anterior ischemic optic neuropathy. Antioxidants. 2022;11:2422.

De Moraes CG, John SWM, Williams PA, Blumberg DM, Cioffi GA, Liebmann JM Nicotinamide and pyruvate for neuroenhancement in open-angle glaucoma: a phase 2 randomized clinical trial. JAMA Ophthalmol. 2022;140:11–8.

Boccaccini A, Cavaterra D, Carnevale C, Tanga L, Marini S, Bocedi A, et al. Novel frontiers in neuroprotective therapies in glaucoma: Molecular and clinical aspects. Mol Asp Med. 2023;94:101225.

Kitaoka Y, Sase K, Tsukahara C, Fujita N, Arizono I, Kogo J, et al. Axonal protection by netarsudil, a ROCK inhibitor, is linked to an AMPK-autophagy pathway in TNF-induced optic nerve degeneration. Investig Ophthalmol Vis Sci. 2022;63:4.

Lawrence ECN, Guo M, Schwartz TD, Wu J, Lu J, Nikonov S, et al. Topical and systemic GLP-1R agonist administration both rescue retinal ganglion cells in hypertensive glaucoma. Front Cell Neurosci. 2023;17:1156829.

Download references

NIH P30 EY026877, Unrestricted Grant from Research to Prevent Blindness.

Author information

Authors and affiliations.

Department of Ophthalmology, Stanford University, Palo Alto, CA, 94303, USA

Pooja Parthasarathi & Heather E. Moss

Department of Neurology & Neurological Sciences, Stanford University, Palo Alto, CA, 94303, USA

Heather E. Moss

You can also search for this author in PubMed Google Scholar

Contributions

PP and HEM designed the review protocol following which PP was responsible for the literature search, extraction of data, formulation of the literature matrix, and writing of the manuscript with appropriate and timely guidance from HEM, who reviewed the manuscript and provided feedback.

Corresponding author

Correspondence to Pooja Parthasarathi .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary table, rights and permissions.

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Cite this article.

Parthasarathi, P., Moss, H.E. Review of evidence for treatments of acute non arteritic anterior ischemic optic neuropathy. Eye (2024). https://doi.org/10.1038/s41433-024-03136-8

Download citation

Received : 07 February 2024

Revised : 08 April 2024

Accepted : 07 May 2024

Published : 22 May 2024

DOI : https://doi.org/10.1038/s41433-024-03136-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

Explore articles by subject
Guide to authors
Editorial policies

Comprehensive analysis of digital twins in smart cities: a 4200-paper bibliometric study

Open access
Published: 27 May 2024
Volume 57 , article number 154 , ( 2024 )

Cite this article

You have full access to this open access article

Rasha F. El-Agamy ORCID: orcid.org/0009-0005-0519-3870 1 , 2 ,
Hanaa A. Sayed ORCID: orcid.org/0000-0003-0728-6323 1 , 3 ,
Arwa M. AL Akhatatneh ORCID: orcid.org/0009-0009-2133-1822 1 ,
Mansourah Aljohani ORCID: orcid.org/0000-0001-5233-7738 1 &
Mostafa Elhosseini ORCID: orcid.org/0000-0002-1259-6193 1 , 4

This survey paper comprehensively reviews Digital Twin (DT) technology, a virtual representation of a physical object or system, pivotal in Smart Cities for enhanced urban management. It explores DT's integration with Machine Learning for predictive analysis, IoT for real-time data, and its significant role in Smart City development. Addressing the gap in existing literature, this survey analyzes over 4,220 articles from the Web of Science, focusing on unique aspects like datasets, platforms, and performance metrics. Unlike other studies in the field, this research paper distinguishes itself through its comprehensive and bibliometric approach, analyzing over 4,220 articles and focusing on unique aspects like datasets, platforms, and performance metrics. This approach offers an unparalleled depth of analysis, enhancing the understanding of Digital Twin technology in Smart City development and setting a new benchmark in scholarly research in this domain. The study systematically identifies emerging trends and thematic topics, utilizing tools like VOSviewer for data visualization. Key findings include publication trends, prolific authors, and thematic clusters in research. The paper highlights the importance of DT in various urban applications, discusses challenges and limitations, and presents case studies showcasing successful implementations. Distinguishing from prior studies, it offers detailed insights into emerging trends, future research directions, and the evolving role of policy and governance in DT development, thereby making a substantial contribution to the field.

Avoid common mistakes on your manuscript.

1 Introduction

Digital Twin (DT) technology, a cornerstone of the Industry 4.0 era, represents a significant paradigm shift in how we interact with and understand physical systems and assets. Originating from Grieve's 2002 lecture at the University of Michigan (Grieves 2005 ) and later refined by NASA in 2010 (NASA 2010 ), the concept of DT has evolved into a sophisticated, multi-faceted approach to simulation and analysis. A Digital Twin is broadly defined as a digitally created virtual model of a physical object that leverages data to emulate the real-world behavior of the physical entity. It facilitates interaction and interoperability between the physical and virtual entities through interactive feedback, data integration, analysis, and iterative decision-making for optimized control, safety monitoring, and data analysis (Stark et al. 2017 ; Rosen et al. 2015 ).

Kritzinger et al. (Kritzinger et al. 2018 ) further categorized DT into three subtypes: digital model, digital shadow, and digital twin, each representing varying degrees of interaction and correlation between the physical and digital states.

The structure of a DT encompasses hardware and software components connected via middleware. The hardware typically includes IoT sensors and actuators, with the middleware playing a critical role in data management and communication between hardware and software. The software component, often an analytics engine, utilizes machine learning algorithms to transform raw data into actionable insights (Kritzinger et al. 2018 ). As depicted in Fig. 1 , this system encompasses the various components constituting the digital twin architecture.

Digital twin system structure. This diagram illustrates the essential components of a Digital Twin system, showcasing hardware with IoT sensors, middleware for data management, and a software analytics engine

Before going into the various applications of digital twins in various industries, it is important to comprehend the nature of digital twins and two closely connected ideas: digital model and digital shadow. These concepts are crucial for understanding this technology's depth (Kritzinger et al. 2018 ). The first concept is the Digital Model, a static digital representation of a physical object without automatic data exchange between the physical and digital entities. It can take many forms, including simulations, CAD files, 3D models, and mathematical algorithms. Digital models help design, optimize, and test by enabling the visualization, analysis, and manipulation of objects or systems in a digital context. A model is typically an estimation or prediction of how a system, process, or physical thing could function in the future or a certain setting. The second concept is Digital Shadow, which represents unidirectional information flow from the physical object to its digital counterpart, reflecting changes in the physical object. Through sensors, Internet of Things (IoT) devices, or other means, digital shadow gathers data from the asset (a database, a railroad system, or a banking platform). It delivers information that is fed into the model. This indicates that a digital shadow is current with the real object. It is helpful to understand it because it accurately depicts the asset in enough detail. The last concept is the Digital Twin, a dynamic, interactive digital representation capable of simulating, predicting, and interacting with data, showing a reciprocal impact between physical and digital states. Digital twins help with analysis, optimization, and predictive maintenance by simulating, monitoring, and controlling real-world systems or objects. They provide insights for enhancing effectiveness, dependability, and performance, as well as live feedback loops.

Figure 2 demonstrates the evolution from a basic digital model, lacking interactive data exchange, to a digital twin that dynamically mirrors and interacts with its physical counterpart, allowing for a two-way flow of information and continuous adaptation.

The progression from a static digital model to a dynamic digital twin, emphasizing reciprocal interaction between physical and digital assets

Digital twins' ability to reproduce physical items, processes, and systems in a virtual environment makes them useful in various applications. This technology has applications across various sectors and domains, providing several benefits and chances for innovation. For example, in industry, they are used for predictive maintenance, optimizing energy usage in smart buildings, and simulating traffic patterns in smart cities.

In the IoT sector, DTs are pivotal, acting as a critical bridge between the physical and digital realms. They allow for the seamless integration of digital and physical entities, enhancing maintenance capabilities and improving equipment performance monitoring (Fang et al. 2022 ; Mihai et al. 2022 ; Hinchy et al. 2019 ; Guo 2020; Wang and Luo 2021 ; Rajesh et al. 2019 ; Revetria et al. 2019 ). IoT can be seen as the vehicle that drives data to Digital Twins, enabling these virtual entities to replicate and interact with their physical counterparts in real-time. Digital Twins depend heavily on IoT technologies for data acquisition. IoT devices like sensors, RFID tags, and smart wearables collect data from the physical environment, which the digital twin then utilizes for various analyses. This data integration facilitated by IoT is crucial for applications ranging from predictive maintenance in industrial settings to real-time monitoring and augmented reality applications (Rajesh et al. 2019 ; Revetria et al. 2019 ). As described in sources monitoring (Fang et al. 2022 ; Mihai et al. 2022 ; Hinchy et al. 2019 ; Guo et al. 2020 ; Wang and Luo 2021 ; Rajesh et al. 2019 ; Revetria et al. 2019 ), IoT's role is not just about data collection but also about ensuring seamless integration of physical and virtual worlds, thus forming the backbone of any DT system.

While the Internet of Things (IoT) plays a major role in shaping and augmenting the capabilities of digital twins, machine learning augments these capabilities by allowing digital twins to analyze data, forecast, identify anomalies, optimize performance, customize experiences, and learn and improve continuously. Integrating machine learning with DT technology enables real-time, autonomous analysis of extensive data streams, enhancing decision-making and optimizing asset and system performance (Rathore et al. 2021 ; Dong et al. 2019 ; Zohdi 2020 ; Jaensch et al. 2018 ; He et al. 2019 ). Machine Learning, a pivotal branch of Artificial Intelligence, involves algorithms that enable systems to learn and adapt from data without being explicitly programmed. Its relationship with Digital Twin technology is synergistic. Digital Twins, virtual replicas of physical entities, systems, or environments, require advanced analytical capabilities to process and interpret the vast amount of data they receive. This is where Machine Learning comes into play. Machine Learning algorithms in DT systems facilitate the autonomous, real-time analysis of extensive data streams. These algorithms are adept at detecting patterns, making predictions, and optimizing processes based on the data ingested from the physical assets that the digital twins mirror. For instance, Rathore et al. 2021 (Rathore 2021) highlighted how applying advanced AI techniques to data within a DT system enables the creation of an 'intelligent' digital twin. This intelligence is manifested in capabilities like predictive maintenance, operational optimization, and dynamic decision-making based on a continuous stream of sensor and virtual data. The application of various machine learning models, such as Deep Neural Networks (DNNs) or Genetic Algorithms (GAs), is contingent upon the specific requirements and use cases of the intended digital twins (Dong et al. 2019 ; Zohdi 2020 ; Jaensch 2018; He et al. 2019 ). Therefore, Machine Learning is not just a complementary technology for Digital Twins but a fundamental enabler of their advanced functionalities.

In the sector of smart cities, DTs are used for urban planning, traffic management, environmental monitoring, energy management, waste management, public safety, infrastructure maintenance, water management, healthcare, public services, tourism, citizen engagement, economic development, and climate resilience. They provide real-time data crucial for emergency response, optimizing public transportation, and ensuring efficient city operations (Allam and Jones 2021 ; Bouzguenda et al. 2019 ; Svítek et al. 2019 ; Yu et al. 2021 ; Ghosh et al. 2016 ). Smart Cities represent urban areas that integrate various electronic data collection sensors to manage assets, resources, and services efficiently. Digital Twins, within the context of Smart Cities, act as sophisticated tools for urban planning, management, and enhancement of living conditions. They utilize data gathered via IoT devices and analyze it using machine learning algorithms to optimize city operations and decision-making processes. Besides, they contribute to traffic management, environmental monitoring, energy distribution, public safety, and more (Allam and Jones 2021 ; Bouzguenda et al. 2019 ; Svítek et al. 2019 ; Yu et al. 2021 ; Ghosh et al. 2016 ). For example, digital twins utilize data from sensors and cameras to optimize traffic flow and public transportation systems in traffic and transportation management. They use real-time data to monitor air and water quality in environmental monitoring. In energy management, digital twins aid in the operation of smart grids and in identifying potential energy conservation areas. These applications underline the comprehensive impact that Digital Twins, empowered by IoT and ML, can have in transforming urban environments into more efficient, sustainable, and responsive entities.

The primary aim of this paper is to engage in a comprehensive bibliometric analysis, examining the evolving landscape of Digital Twin technology within Smart Cities. The study is dedicated to methodically examining the scholarly dialogue, identifying predominant trends, and revealing key themes and collaborative networks in this area. We aim to provide a detailed, structured understanding of Digital Twin technology's role in urban development, filling a notable void in existing literature reviews. The survey's distinctiveness stems from its thorough data-gathering approach for bibliometric analysis in the field of Digital Twin technology and Smart Cities, selecting the Web of Science database for its broad interdisciplinary coverage and meticulously filtering over 4,220 pertinent articles, enhancing the depth and scope of analysis in these domains.

Our research found that various publications in various literary works are advancing the DT idea. Because there are so many articles available, academics have also published several survey papers that aim to review the current state-of-the-art in digital transformation (DT) development, inform other innovators about potential research gaps, questions, and directions, and point the industry toward potential DT use cases that could yield substantial business value in their particular domain.

Current literature predominantly concentrates on applying digital twin technology within specific facets of smart cities. For instance, Jafari et al. ( 2023 ) and He et al. ( 2023 ) explore the utilization of digital twin (DT) technology in enhancing various sectors of energy management within smart cities, encompassing transportation systems, power grids, and microgrids. Weil (2023) delves into the infrastructure elements of digital twins in smart cities, focusing on storage, computation, and network components. Nica et al. ( 2023 ) investigates Multi-Sensor Fusion Technology's role in sustainable urban governance networks. Dani et al. ( 2023 ) introduces an architectural framework underpinning the flow for digital twin platform development aimed at urban condition monitoring. Lam et al. ( 2023 ) outlines a use case for the 3D visualization of a smart village in Busan, South Korea, employing a 3D Geospatial platform. Paripooranan et al. ( 2020 ) suggests augmented reality (AR)-assisted DT as a pioneering approach towards the future transformation of human-centric industries. Mora (2023) highlights the importance of incorporating innovation management theories into the exploration of smart city transitions, offering novel insights and practical approaches to enhance the governance of smart cities through an innovation management lens. Ariyachandra and Wedawatta ( 2023 ) provides an overview of digital twin technologies' implications on disaster risk management, addressing the challenges of implementing digital twins in smart cities. Additionally, several reviews, including those by Weil (2023) and Wang (2024), focus on bibliometric analyses concerning digital twins in the realm of smart cities.

This work aims to support the other existing survey initiatives and provide a comprehensive comprehension of the DT. The paper gives an in-depth overview of the DT idea, architecture, enabling technologies, applications in smart cities, challenges, performance metrics, datasets, software, and use cases for deploying DTs in diverse industries, complementing prior research. This paper aims to fill a critical gap in understanding the expansive and evolving field of Digital Twin technology and its integration into Smart City development. This study is driven by the need to systematically synthesize and analyze the burgeoning body of research in this interdisciplinary area, providing clarity and direction for future studies. This survey's uniqueness and unprecedented nature stem from its comprehensive and systematic bibliometric analysis of over 4,220 articles on Digital Twin technology and Smart Cities. A focused examination of specialized areas such as datasets, platforms, and performance metrics marks this distinctiveness. The rigorous methodology involving the Web of Science database ensures in-depth interdisciplinary coverage. The survey's meticulous approach in formulating search strategies and selective filtration of relevant articles contributes to its depth and breadth, making it a unique contribution to the field. The significant contributions of this survey paper are listed below:

An overview of the DT definitions, concepts, and architecture in the literature

A Detailed bibliometric study of over 4,220 publications in Digital Twin technology and Smart Cities, including thematic trends analysis like AI and IoT integration.

Examination of datasets, platforms, and performance metrics specific to Digital Twins in urban settings and a critical evaluation of city models.

Applications in Smart Cities: Exploration of Digital Twin technology applications in urban development, encompassing urban planning, energy management, and public health.

Discussion of the challenges in implementing Digital Twin technology in Smart Cities, focusing on data integration, scalability, and security concerns.

Outlining potential research avenues based on current findings, indicating areas for further exploration.

Presentation of practical case studies demonstrating successful Digital Twin integration in urban development.

Summarizing the main findings and implications and a call to action for further research in this evolving field.

The paper's organization follows a clear and structured approach, beginning with Section 1 , an introduction that sets the stage for Digital Twin technology and Smart Cities. It progresses into Section 2 , which provides a detailed bibliometric study, covering objectives, methodology, data collection, and analysis, leading to key findings and implications. Then, Section 3 explores the applications of Digital Twins in Smart Cities. Section 4 discusses some technological aspects of DT. Section 5 presents some examples of datasets and software for developing DT. Section 6 states digital twin performance metrics according to its structure. Section 7 addresses the challenges associated with digital twins. Section 8 introduces some case studies for DT. Section 9 discusses smart city governance in the era of digital twins. Finally, Section 10 summarizes the paper's conclusions and presents future research directions.

2 Bibliometric study on digital twin and smart cities

The primary objective of this research is to perform a bibliometric analysis (Yu and Merritt 2023 ) to acquire a comprehensive understanding of emerging topics, prominent journals, and evolving research trends associated with the application of digital twin technology in smart cities. Additionally, the study aims to shed light on the potential challenges and future research trajectories concerning digital twin technology in the context of smart city development.

2.1 Research methodology

This investigation employed a systematic literature review (SLR) to meticulously explore, assess, and integrate the extant body of knowledge regarding the designated theme, adhering to a rigorously defined protocol (Kyriazopoulou 2015 ). Adopting the SLR methodology is instrumental in delineating the contemporary scholarly landscape of a given topic, thereby uncovering existing research voids and delineating avenues for forthcoming scholarly inquiries (Kitchenham et al. 2009 ). The SLR framework comprises three pivotal phases: planning, execution, and dissemination. The research inquiries were articulated during the planning stage, and criteria for identifying pertinent literature and determining search strategies were established. The execution stage entailed the meticulous gathering and vetting of scholarly works in alignment with the previously established criteria. This phase was initiated with an initial screening of the collected records through their titles and abstracts to ascertain their pertinence to the posed research questions, followed by an in-depth examination of the full-text articles. A bespoke form was devised for the methodical extraction of data, capturing essential information from the chosen articles, such as facets of digital twin components, smart city innovations, and the research lacunae identified therein. Subsequently, the dissemination phase involved the analytical consolidation and synthesis of the compiled literature. The process was underscored by a commitment to transparency and precision, with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) framework guiding the data acquisition methodology (Liberati et al. 2009 ).

2.1.1 Research questions

To delineate the scope of the SLR, the following research questions guided the study:

Q1: What are the components of digital twins in smart city applications?

Q2: What are the existing technologies used in the development of smart city development based on digital twins?

Q3: What are the research gaps and potential areas for future research?

2.1.2 Data collection

This section outlines the steps taken to collect relevant literature for the study. A PRISMA workflow diagram in Fig. 3 illustrates the study's search process. Initial literature searches were conducted in reputable databases such as Web of Science, Direct Science, and Scopus, which were chosen for their extensive coverage of scientific publications and advanced search capabilities. The research strategy applied an advanced search with keywords executed in the Web of Science and Scopus databases with a search string set to ("Digital twin," "virtual twin" or "virtual replica," and "smart city" or "smart cities"), for publications up to September 2023 and set to articles before 2018 were excluded. The period selected for the search is appropriate because there are few publications on digital twins and smart cities before 2018.

PRISMA workflow diagram

2.1.3 Inclusion and exclusion criteria

In this survey, the inclusion and exclusion criteria were meticulously established to ensure a focused and relevant analysis in the fields of Digital Twin technology and Smart Cities. This careful selection was pivotal in delineating the scope of the study.

Inclusion criteria:

Scope of content : Articles must focus on Digital Twin technology and its application within Smart Cities. This includes scholarly articles, conference proceedings, and review articles offering substantial insights into Digital Twin architectures, methodologies for Smart City implementation, technologies employed, and demonstrative case studies or laboratory setups.

Language : Only articles published in English are considered to ensure the clarity and accessibility of the content for our analysis.

Databases : Articles were sourced from the Web of Science, Scopus, and Direct Science databases to ensure a comprehensive and interdisciplinary coverage of the subject matter.

Publication period : Articles published from 2018 to 2023 were included to capture Digital Twin technology's evolution and current state in Smart Cities.

Detail requirements : Articles must present a detailed systematic architecture for a digital twin application and a clear methodology for Smart City implementation. They must also discuss the technologies used and provide demonstrative case studies or laboratory setups.

Exclusion criteria:

Language limitation : Articles published in languages other than English were excluded to maintain consistency and comprehensibility in the analysis.

Irrelevance : Publications unrelated to the direct intersection of Digital Twin technology and Smart Cities, lacking in detailed architecture, clear methodologies, technology discussion, or case studies, were excluded.

Duplication : Duplicated records identified across the databases were removed to ensure the uniqueness and accuracy of the analysis.

Date filter : Articles published before 2018 were excluded to focus the study on more recent developments and applications, reflecting the latest trends and innovations in the field.

The search produced 4,220 records from Web of Science, 382 from Scopus, and 24 from Direct Science. Through a preprocessing step, which involved removing duplicates and applying inclusion and exclusion criteria, 4,507 records were screened. This process yielded 4,073 articles that were deemed eligible for further analysis. The inclusion criteria were specifically targeted at articles that provided detailed systematic architectures for digital twin applications, methodologies for implementing smart cities, descriptions of technologies employed, and demonstrative case studies or laboratory setups.

2.2 Bibliometric study methodology

The methodology of a bibliometric study typically comprises a series of fundamental steps aimed at systematically analyzing scholarly literature within a specific field. These steps include formulating precise research questions to guide the analysis, identifying and selecting appropriate data sources, devising relevant search strategies using carefully chosen keywords, meticulously collecting and preparing the retrieved data, and employing established bibliometric techniques for rigorous data analysis. By adhering to this structured approach, researchers can effectively uncover trends and patterns in scientific publications and citations, thereby gaining valuable insights into the evolving landscape of their area of study (Mora et al. 2019 ). In line with these established practices, this research adopts a systematic approach for collecting, processing, and analyzing academic literature on digital twins within the context of smart cities.

2.3 Data analysis and visualization

This subsection outlines the methodologies and tools implemented to analyze and visualize the bibliometric data. For our study, VOSviewer was selected as the primary tool for managing and interpreting bibliographic data. We utilized network analysis methodologies to generate a range of visual representations. These included co-occurrence analyses, citation and co-citation maps, and keyword co-occurrence maps. Such visualizations were instrumental in uncovering patterns and discerning relationships within the collected dataset.

2.3.1 Publication trends

One of the key indicators in performance analysis is the annual number of publications. This metric serves as an indicator of research productivity. The data collected from 2011 to 2023 reveal a marked increase in publications focused on digital twin technology and smart cities. This surge in research output, demonstrating exponential growth, is depicted in Fig. 4 . This Figure underscores the significance and escalating interest in this interdisciplinary area. Figure 4 illustrates the yearly publication rates concerning digital twins and smart cities. Additionally, Table 1 provides a concise statistical analysis of these findings.

The publications rate of digital twin and smart cities by year from 2011 to 2023

2.3.2 Keyword analysis and research themes

This study's keyword co-occurrence analysis represents a systematic approach to understanding the prevailing keywords associated with digital twin technology and smart cities. The outcomes of this analysis, illustrated in Fig. 5 , reveal a range of predominant research themes and technologies pertinent to the domain of digital twins and smart cities.

Visualization of keyword co-occurrence network

Research themes in digital twin and smart cities:

Theme 1: Integration of AI and big data analytics in digital twins

This theme explores applying advanced deep learning techniques in processing and analyzing digital twin data. Key research areas are identified through terms such as "machine learning," "transfer learning," "simulation," "reinforcement learning," "cloud computing," "AI," "data analysis," "big data," and "forecasting."

Theme 2: Integration of digital twins with IoT

The focus here is IoT technologies, which are central to transmitting and collecting digital twin data. Relevant keywords include "wireless sensor network," "digital devices," "sensors," "5G", "communication," "wireless communications," and "monitoring."

Theme 3: Energy management in digital twins

This theme emphasizes the importance of energy efficiency and sustainability, highlighting keywords such as "energy efficiency," "energy utilization," "sustainability," and "renewable energy."

Theme 4: Security concerns in digital twins

Research in this area deals with the security aspects of digital twins, with keywords like "security," "privacy," "blockchain," and "fault diagnosis".

Theme 5: Cloud computing and digital twins

The final theme investigates the intersection of digital twins with cloud computing technologies, focusing on keywords such as "cloud computing," "edge computing," "fog computing," "blockchain," and "big data analytics."

Predominant technologies in the Digital Twin (DT) domain

Our study analyzed the authors' keywords to ascertain the most prominent technologies within the digital twin sphere. This investigation uses keyword frequency as a metric to identify the key technologies extensively employed in the digital twin (DT) domain. The term 'Internet of Things' (IoT) emerges as the most frequently cited keyword, demonstrated in Fig. 7 . This finding underscores the pivotal role of IoT in the digital twin field, highlighting its extensive research coverage and the ongoing need for in-depth exploration of IoT applications to enhance digital twins' efficacy. Additionally, "AI" and "machine learning" are prominently used to analyze and process large volumes of digital twin data. Other notable technologies such as "cloud computing," "virtual reality," and "digital twin security" have also gained traction. Collectively, these technologies contribute to the efficient storage, visualization, modeling, and security of digital twin data. The data presented in the accompanying table and Fig. 6 substantiate the findings discussed in this subsection.

High-frequency keywords in digital twin research

2.3.3 Analysis of geographical distribution

Examining the geographical distribution in the research and development of digital twin and smart cities technologies offers critical insights into the regional contributions, patterns of collaboration, and prospective areas for advancement. As depicted in Fig. 7 (A), our analysis reveals a broad geographical spread in the field's research activities. We identified key regions contributing significantly to the field by utilizing a citation metric analysis on our dataset, which set a minimum of ten documents and fifty citations per country. China emerges as the leading contributor in terms of citations, followed by the USA, the UK, Italy, and Germany. Furthermore, Fig. 7 (B) corroborates the leadership status of China, the USA, the UK, and Italy in this domain.

A Citations by country in digital twin and smart cities research. B Top 10 publishing countries in digital twin and smart cities research

2.3.4 Analysis of source co-citation

The source co-citation analysis conducted in our study highlights the prominent sources within the domain of digital twins and smart cities. Of 49,275 sources, 433 met the established criterion of a minimum of 50 citations per source. The findings of this analysis are presented in Fig. 8 . The most frequently co-cited journals include IEEE Access, the Journal of Manufacturing Systems, IEEE Transactions on Industrial Informatics, and IEEE Internet of Things. The analysis identified six distinct clusters, each represented by a unique color, as depicted in Fig. 7 (A).

A Co-citation network of sources in digital twin and smart cities research. B Bibliographic coupling network among countries in digital twin and smart cities research

2.3.5 Examination of international collaboration

The observed international collaboration in the digital twin and smart cities sector underscores research's global impact and relevance. Utilizing bibliographic coupling analysis on our dataset, with a set threshold of a minimum of 10 documents and 20 citations per country, 65 out of 103 countries met these criteria. A network visualization visually represents the bibliographic coupling among these countries in Fig. 8 (B).

This analysis collated data on each country's publications, citations, and total link strength. Each node in the figure symbolizes a country whose size reflects its publication count. The visualization reveals that China leads in a collaborative network, boasting approximately 1151 documents and a total link strength of 871,379. Following China are the USA, the UK, and England. Notably, the USA's most extensive collaborations were with China, England, and India, while China's primary collaborations were with the USA, England, and Germany.

The colors in Fig. 8 (B) delineate nine distinct clusters, indicating nations that frequently cite each other's research, suggesting closer collaboration within these groups. This mapping confirms that countries like China, the USA, the UK, Italy, and Germany are at the forefront in advancing research in digital twin and smart cities.

3 Applications of digital twin technology in smart city development

Digital twin technology offers a wide range of applications in smart city development, from optimizing traffic flow and energy usage to improving public safety, Environmental Monitoring and Management, Citizen-Centric Aspects, and Supply Chain Management and Enhancement. By creating virtual replicas of city infrastructure and systems, urban planners and policymakers can visualize potential changes and their impact before implementing them in the physical environment. Furthermore, digital twins can be instrumental in public safety by simulating emergency response scenarios and planning for effective evacuation routes in the event of natural disasters or other crises, as shown in Fig. 9 . As the adoption of digital twins continues to grow, their role in shaping the future of smart cities will become increasingly prominent.

Applications of digital twin technology in smart city development

3.1 Urban planning and management

Urban planning and management encompass the technical and political processes of utilizing land, infrastructure, and buildings within urban areas. This multifaceted domain includes urban design, land use, transportation, zoning, regulation, and environmental planning.

Urban planners and managers increasingly employ digital twin technology to enhance city functions like transportation and sustainability. Digital twins enable more informed decision-making and optimize planning, operations, finance, and strategy. In turn, such systems help reduce carbon emissions and expedite significant projects. Additionally, they enable the simulation of plans before implementation, allowing for the anticipation of potential challenges. The World Economic Forum 2022 recognized the role of digital twins in modeling future sustainable development by integrating digital technology with urban operational systems. This integration facilitates safer, more efficient urban activities. It creates low-carbon, sustainable environments through precise mapping, virtual-real integration, and intelligent feedback of physical and digital urban spaces (Yu and Merritt 2023 ).

Within urban planning and management, digital twins can represent entire cities or specific urban systems, assisting in various ways:

Real-time Monitoring: Integrating sensors and IoT devices with digital twins provides real-time data on urban processes like traffic flow, energy consumption, and air quality.

Simulation and Scenario Testing: Planners can use digital twins to simulate and test different scenarios, assessing the impacts of natural disasters or new transportation systems.

Optimization: Analyzing data from digital twins can identify and address inefficiencies in urban systems.

Public Engagement: Digital twins serve as interactive platforms for public involvement, allowing community members to view proposed changes and provide feedback.

Maintenance and Asset Management: They enable tracking urban infrastructure conditions and predicting maintenance needs.

System Integration: Digital twins facilitate understanding of interdependencies between various urban systems.

Support for Decision-Making: Providing a comprehensive view of the city and its systems, digital twins enhance the decision-making process, ensuring decisions are informed by accurate, up-to-date information.

3.2 Energy management

Energy systems form the backbone of smart cities, ensuring the quality and functionality of these urban environments. This section delves into the application of DTs in energy systems, encompassing transportation systems, power grids, and microgrids.

Digital Twin technology finds varied applications in transportation systems. It supports transportation system infrastructures in several ways, such as monitoring transport systems, traffic forecasting, energy system management, predicting the energy consumption of electric vehicles, IoT-based parking management, analyzing driver behavior, forecasting subway regenerative braking energy, studying pedestrian behavior, controlling health systems, and detecting cyber-physical attacks. DTs can significantly contribute to these areas. For instance, using DTs for transportation system monitoring can reduce maintenance costs. Beyond modeling and planning, DTs facilitate optimal traffic management and provide accurate and extensive traffic and electric vehicle (EV) data, contributing to sustainable development and efficient urban traffic control.

In traffic management, DT technology has been utilized to predict patterns of energy consumption and production (Ketzler et al. 2020 ). Additionally, DTs play a role in IoT-based parking management, improving user services by saving time and reducing parking costs.

Various studies have employed DT technology in diverse contexts. In (Yan et al. 2022 ), authors analyzed real drivers' and pedestrians' behavior using DTs. In (Liu et al. 2020 ; Damjanovic-Behrendt 2018 ), DTs of drivers and vehicles were used in real-time to relay critical information to drivers and vehicles in the physical world. Moreover, in (Crespi et al. 2023 ), the Electric Vehicles (EVs) model employed DTs to monitor the behavior and optimally manage charging programs, using energy consumption parameters and charging capacity and frequency for modeling the virtual twin.

A microgrid is an autonomous energy system characterized by distributed energy resources and interconnected loads. It functions as a manageable entity within the larger grid, enabling it to operate in either island mode or in conjunction with the grid (Ton and Smith 2012 ). The objective of microgrids is to enhance the functionality of energy systems in terms of sustainability, economic viability, efficiency, security, and overall energy management. Key aspects of microgrid performance include reliability, self-sufficiency, security, flexibility, and optimality. Studies on microgrids utilizing the Digital Twin (DT) framework have encompassed areas such as forecasting (Din and Marnerides 2017 ; He et al. 2017 ), management and monitoring (Xu et al. 2019 ; Park et al. 2020 ), fault prediction (Nowocin 2017 ; Goia et al. 2022 ), and security (Huang et al. 2021 ).

The development and implementation of DT-based power grids are instrumental in improving network behavior under various conditions. Network studies employing DT include diverse analyses such as restoration (Biagini et al. 2020 ), reliability (Podvalny and Vasiljev 2021 ), prediction (Park et al. 2020 ), addressing uncertainty (Raqeeb et al. 2022 ), energy hub management (Kuber et al. 2022 ), and ensuring both physical and cyber security. Each of these analyses offers unique insights into network behavior. In reference (Endsley 2016 ), Situation Awareness (SA) is the ability to perceive the elements in a specific environment, understand their properties, and anticipate their future statuses. SA is crucial in augmenting decision-making, especially in complex systems like the Energy Internet of Things (EIoT) (He et al. 2023 ). It provides essential information critical for such systems' operation, enhancing efficiency and effectiveness.

In the application of Digital Twin technology in power systems, several significant challenges emerge:

IT infrastructure limitations: Existing infrastructure often falls short in supporting the data analysis demands of DT environments.

High-performance computing needs: Utilizing high-performance GPUs and cloud services from major providers is essential for adequate support.

Connectivity issues: Software errors and power outages present obstacles in real-time monitoring.

Cybersecurity risks: The extensive data exchange in DT systems heightens vulnerability to cyber-attacks, necessitating secure platforms.

Standardization requirement: The absence of standardized protocols impedes DT development, highlighting the need for unified approaches for model definition, storage, and execution.

The exploration of digital twin applications in energy management reveals several key areas for future development:

Advancements in big models: Addressing challenges in AI, such as limited model generalization and the need for high-quality data, by developing larger, more adaptable models.

Virtual twin structures in power systems: Detailed modeling of power system entities using virtual twins, enabling dynamic visualization and strategy development for urban transformation.

Application of theoretical models: Utilizing chaos and complex system theories (Mir et al. 2022 ) to understand and optimize the nonlinearities in power systems, offering a novel approach to managing system complexities.

3.3 Traffic and mobility management

Traffic and Mobility Management in Smart Cities (Xu et al. 2023 ), enhanced by Digital Twin (DT) technology, represents a significant advancement in urban planning and logistics. DTs enable:

Real-time traffic simulation : Mimicking urban traffic flow to identify and alleviate congestion points.

Public transportation optimization : Analyzing patterns to improve transit routes and schedules.

Pedestrian flow management : Ensuring safer and more efficient pedestrian movement.

Pollution reduction : Aiding in strategies to lower emissions through traffic regulation.

Emergency response enhancement : Assisting in quicker and more efficient routing for emergency services.

Data-driven decision making : Utilizing sensor data for informed traffic management decisions.

Sustainable urban planning : Contributing to long-term urban sustainability goals through efficient mobility solutions.

These applications of DT in traffic and mobility management significantly contribute to creating more livable, efficient, and sustainable urban environments.

3.4 Environmental monitoring and management

Digital twin technology is increasingly integral in urban development, offering real-time insights and solutions for environmental management:

Optimization and prediction: Digital twins, as virtual representations of physical entities, enable process optimization, change monitoring, and future scenario prediction (Wang et al. 2023 ).

Environmental monitoring applications: Usage in water quality monitoring, detecting pollutants, and adapting to changing environmental conditions.

Data integration in smart cities: Interconnection of multiple digital twins, using diverse data sources like temperature and humidity, to forecast environmental conditions (Ivanov et al. 2020 ).

Sensor utilization: Various sensors capture essential environmental data for digital twin construction, including Kinect v2 depth cameras and electronic gloves for manufacturing systems (Nikolakis et al. 2018 ).

Food industry monitoring: Application in monitoring and predicting food quality, employing wireless sensors for environmental factors like humidity and temperature (Defraeye et al. 2019 ).

Agricultural management: Use in agriculture for crop growth monitoring and simulating interventions, aiding in remote farm management (Verdouw et al. 2021 ).

Healthcare applications: Implementation in healthcare for environmental monitoring and mental health management using smartwatch sensors (Bagaria et al. 2019 ).

These diverse applications showcase the role of digital twins in enhancing urban planning, agriculture, healthcare, and more.

3.5 Public health and safety

In the development of Smart Cities, Digital Twin Technology plays a crucial role in enhancing public health and safety (Erol et al. 2020 ). It offers a dynamic and integrated approach to managing complex urban health challenges through simulation and analysis. This technology's applications include:

Disease outbreak prediction and management : Leveraging real-time data to simulate disease spread and plan responses.

Emergency preparedness : Using simulations for natural disasters or public safety incidents to enhance response strategies.

Resource optimization in healthcare : Improving the allocation of healthcare resources like hospital beds and emergency services.

Environmental health monitoring : Tracking and analyzing environmental factors that impact public health, such as pollution levels.

Public safety and incident response : Simulating various scenarios to optimize law enforcement and emergency services.

These applications demonstrate the transformative impact of Digital Twin Technology in public health and safety within Smart Cities.

3.6 Citizen-centric aspects

Technological advancements have focused on urban development and infrastructure management, employing physical sensors such as the Internet of Things (IoT) and satellites (Borrmann et al. 2018 ). However, not all city digital twin implementations have citizen engagement in mind. A citizen-centric digital twin (CCDT) approach views citizens as integral components of a data-driven city, with human sensors playing a key role in addressing city-scale challenges (Saeed et al. 2022 ). This approach distinguishes itself from traditional digital twin frameworks by prioritizing citizens as the central element and integrating technologies like processing, data acquisition, and visualization to enhance citizen involvement in infrastructure governance.

Developing a CCDT requires the execution of numerous processes and technologies. One such technology involves sensors like Volunteered Geographic Information (VGI) (White et al. 2021 ), which transfer data from the actual city to its digital twin, followed by analysis using various analytical tools. Managing data from diverse sources at a city scale presents challenges in scalability, reliability, and the performance of real-time analytics and modeling (Langenheim et al. 2022 ).

The study by Abdeen et al. (Abdeen et al. 2023 ) indicates a scarcity of publications in this field over the past five years. However, a rising trend in CDT interest post-2017 was noted, with publications doubling by the end of 2022 compared to 2019. Research works (Ford and Wolf 2020 ; Fan et al. 2021 ) discuss the application of digital technologies in catastrophic situations and emergency responses. The capabilities of intelligent digital twins in various application fields have been examined (Shahat 2021; Deren et al. 2021 ), with the latter focusing on hazards like epidemic services, traffic control, and flood monitoring. (Shahat et al. 2021 ) concentrated on data simulations, fusion, administration, and collaboration. (Charitonidou 2022 ) addressed citizen participation in decision-making, highlighting that limited variables and processes and overlooking social aspects of urban contexts can render citizen input integration ineffective.

In the literature, various data acquisition mechanisms are employed to support CCDTs. One prominent method is the use of open-source data platforms (OSDP), providing spatiotemporal performance data relevant to CCDT applications like disaster management (Ghaith et al. 2022 ) and public services monitoring (Diakite et al. 2022 ). However, the effectiveness of CCDTs can be compromised if data is unreliable. Another mechanism is crowdsourcing, which generates large quantities of data and is particularly useful when remote or IoT sensors are unavailable (Trusov and Limonova 2020 ). Nevertheless, citizens' data errors or bogus inputs can affect CCDT effectiveness (Trusov and Limonova 2020 ). Visionary concepts for disaster city digital twins with extensive data (images, text, geo maps) have been proposed to enrich CCDT content (Fan et al. 2021 ).

Remote sensors are effective in modeling 3D city aspects of CCDTs and for large-scale urban monitoring (Fan and Mostafavi 2019 ; Fan et al. 2020 ). Geospatial platforms storing and managing data from individual vehicles or pedestrians have been proposed (Lee et al. 2022 ), though data accessibility to all stakeholders remains challenging for CCDT integration. Furthermore, IoT sensors (Nochta et al. 2020 ), deployed in large numbers and integrated effectively, facilitate urban data monitoring but require advanced communication infrastructure.

Advanced AI algorithms also play a crucial role in CCDTs, enhancing citizen engagement. Genetic algorithms (Fan et al. 2021 ) have been used to study the range of disruptions during hazard events, while Convolutional Neural Networks (CNN) (Pang et al. 2021 ) and Burst detection algorithms (Fan et al. 2021 ) help analyze crowdsourced data and social media frequencies providing insights into citizen perspectives and infrastructure governance through CCDTs.

3.7 Supply chain management and enhancement

The supply chain, encompassing the entire spectrum from raw material sourcing to the distribution of finished products, has seen a transformative integration of Digital Twin technology in recent years. Digital Twins, as virtual models of physical assets, offer real-time monitoring, analysis, and optimization across all facets of the supply chain, ranging from procurement to distribution (Tao et al. 2017 ). According to van der Valk et al. (van der Valk et al. 2022 ), these digital replicas enable two-way data exchange between the digital and physical worlds, providing professionals with exceptional visibility and traceability. This level of insight facilitates the identification of complex behavioral patterns and proactive problem detection, which is crucial for maintaining operational continuity.

As Gerlach et al. (Gerlach et al. 2021 ) highlight, Digital Twins are instrumental in offering real-time inventory insights, enabling the simulation of various scenarios, and assisting in planning and forecasting. These capabilities can result in significant cost reductions and process efficiency improvements. The study by Srai et al. (Srai and Settanni 2019 ) explores the optimization opportunities that Digital Twins offer in areas such as transportation resource management, demand–supply analysis, customer service improvement, and revenue enhancement. They also emphasize the role of technology in identifying and addressing inefficiencies.

The influence of Digital Twins in improving stock availability, a key aspect of manufacturing operations, is underscored by Abouzid et al. (Abouzid and Saidi 2023 ). Furthermore, (Lugaresi et al. 2023 ) introduces the concept of "technological labelers" like IoT devices, cloud computing, and advanced analytics, which are crucial in developing a comprehensive digital twin of a company's value chain. IoT integration, in particular, is noted for significantly enhancing supply chain efficiency by providing real-time data and contextual insights. In addition, distorted demand signals can result in various supply chain challenges, which can be effectively addressed using Digital Twin technology (Abouzid and Saidi 2023 ).

In conclusion, the discourse delves into the exploration and development of digital twins within automated manufacturing systems, showcasing the expansive potential of this technology in modernizing and streamlining supply chain management processes (Abideen et al. 2021 ).

4 Technological aspects of digital twin

This section explores the fundamental components and emerging advancements in digital twin technology, covering various technological aspects essential for understanding Digital Twins across disciplines. Each component offers unique insights into critical subjects, including distinctions between Digital Twins and Building Information Modeling (BIM) and the development of Cognitive Twins. By studying frameworks like the Five-Layer Architecture and advancements such as Cloud and Edge Computing integration, this section aims to reveal the technological foundations driving the evolution of Digital Twins. Readers will gain a deeper understanding of the technological breakthroughs shaping the future of Digital Twins and their applications through this comprehensive examination.

4.1 Distinction between digital twins and Building Information Modeling (BIM)

In construction technology, the emerging potential of digital twins and the rapid advancement of smart technologies has garnered significant interest. Although 'digital twin' is a relatively new term in the construction research literature, it is often conflated with Building Information Modeling (BIM), leading to some conceptual ambiguity. It is imperative to clarify the differences between these two concepts.

Building Information Modeling (BIM) is a digital representation of a building or structure's physical and functional characteristics. It is a tool architects, engineers, and construction professionals utilize to create detailed digital models of buildings, encompassing all systems and components, including architectural, plumbing, electrical, and HVAC systems. BIM enables the creation of accurate construction plans, virtual walkthroughs, and performance testing under various scenarios. (Wang and Meng 2021 ) defined BIM as a method that integrates geometric and non-geometric data. The 3D model, often called the BIM model and realized through object-oriented software, is a critical component of BIM (Cerovsek 2011 ). However, BIM primarily manages static data and requires external technologies to update models with real-time data (White 2021). In construction projects and asset management, a vast amount of non-geometric data is essential for informed decision-making but is often underutilized (Khudhair et al. 2021 ). BIM models have limited capacity to handle large volumes of dynamic and multifaceted data, necessitating advanced storage and processing technologies. These limitations can lead to data underutilization, inefficient decision-making, and financial implications. The advent of digital twin technology offers a solution to overcome these constraints inherent in BIM.

Digital twins and BIM represent two distinct technological applications in the construction sector, differentiated by their functions. BIM is most effective in the design and construction phases, while digital twins excel in building maintenance and operations. A digital twin system involves data linkage that transfers information between the physical asset and its virtual counterpart. This indicates that a BIM (Building Information Modeling) model is the initial step toward developing a digital twin in the construction industry. Digital twin technology integrates the BIM model with the physical world, enabling bidirectional data exchange. This connection allows for the real-time updating of the BIM model, enhancing asset implementation and management decision-making. The synergy between BIM and Digital Twin technology has the potential to revolutionize the construction industry. By combining the detailed architectural and structural information provided by BIM with the real-time operational data and analysis capabilities of Digital Twin technology, construction professionals can create comprehensive, accurate digital models of buildings. These models can then continuously monitor and optimize building performance in real-time.

4.2 Framework of the five-layer architecture in digital twins

A digital twin, essentially a digital representation of a physical entity, process, or person contextualized in a virtual environment, is a pivotal tool for organizations to simulate real-world scenarios and outcomes, thus enhancing decision-making capabilities (Moosavi et al. 2021 ). As shown in Fig. 10 , the architecture of digital twins is typically structured into five principal layers, as outlined in (Jones et al. 2020 ):

Five-layer architecture in digital twins

Physical layer : This foundational layer comprises the actual physical objects or entities. It utilizes sensor technology for data acquisition and can receive commands from the virtual layer. This layer provides real-time data feedback to the digital twin model.

Data sensing layer : Responsible for collecting diverse information types, this layer employs various sensors to monitor the system's status and operational process in detail. The data heterogeneity and variety stem from diverse data generation sources, such as IoT sensors, information systems, and wearable devices.

Data transmission layer : As a crucial link, this layer ensures data transmission between the physical and virtual layers. It leverages communication integration protocols and interactive security technologies to facilitate this transfer (Lohtander et al. 2018 ).

Virtual layer : In this layer, the components of the real world are digitally reconstructed. It builds a collection of digital twins using data transmitted from the physical layer, enhanced with historical or integrated network data. This layer dynamically tunes itself based on the real-time data from the physical layer and can be influenced by modifications made in the application layer.

Application layer : This layer visualizes the data and simulations derived from the virtual layer, presenting a graphical model that staff can easily interpret. Modifications in the physical or virtual layer parameters can lead to simulation changes. These can then be revised and optimized based on the observed or extrapolated results.

Each layer in this five-tier architecture plays a distinct yet interconnected role, collectively enabling the digital twin to function as a comprehensive, dynamic system for analyzing, simulating, and enhancing real-world processes and entities.

4.3 Integration of cloud and edge computing in digital twin environments

Cloud computing represents a large-scale computational approach that leverages the Internet to facilitate sharing computing, storage, and other resources, accessible anytime and anywhere on demand. In contrast, edge computing is a novel computational model that processes a portion of data using distributed computing, storage, and network resources between data sources and cloud computing centers.

Edge computing is increasingly recognized for its potential to enhance privacy, reduce latency, conserve energy and costs, and boost reliability. It is particularly well-suited for Digital Twin (DT) scenarios that demand low latency, high bandwidth, high reliability, and stringent privacy measures. In DT-assisted edge computing setups, the framework includes user devices, edge servers, resource devices, and the DT itself. User equipment initiates task requests to the edge server, which then allocates computing resources to the task, with the DT deployed within the edge server. Reviewed literature demonstrates the application of Cloud and Edge technologies in various contexts. Cloud storage is universally employed in these studies. Earlier research also utilized cloud computing for user interfaces, with cloud-rendered 3D models, as indicated in (Xu et al. 2021 ), or through GUIs accessible via web applications as in (Urbina Coronado 2018). While initial studies relegated intensive data processing and analytics to cloud computing due to its superior resource access, recent advancements in edge device capabilities have led to the implementation of edge computing, employing techniques for heavy data analysis or machine learning (Cathey et al. 2021 ; Lu et al. 2021a ; Zhang et al. 2022 ).

Recent digital twin research, such as (Alam and El Saddik 2017 ), employs edge computing, describing a framework where each device is represented as a cloud-based digital twin. This hierarchical architecture involves higher-level digital twins composed of simpler units in a master/slave relationship, enhancing the communicability of traditional cyber-physical systems with cloud servers' advanced computational and storage capacities. Focus on edge-based architectures is evident in (Dong 2019; Lu et al. 2021a ), with research by Dong (Dong 2019) on enhancing energy efficiency in 5G services through deep neural networks and Lu (Lu et al. 2020 ) exploring the use of digital twins in-network replication and machine learning via federated learning.

In industry, studies such as (Lu et al. 2021b ; Zhang et al. 2022 ) concentrate on Smart Vehicles, driven by the rise in edge computing power. Other research, including (Liu et al. 2019 ; Martinez-Velazquez et al. 2019 ), investigates the application of digital twins in healthcare, aiming to provide high-quality, real-time care to senior citizens. Most other studies, such as (Xu et al. 2021 ; Urbina Coronado et al. 2018 ; Hu et al. 2018 ; Bellavista et al. 2021 ), are categorized under Smart Manufacturing, focusing on industrial productivity improvements. Cloud-based digital twins play a crucial role in optimizing IoT device energy consumption and operational efficiency (Li et al. 2020 ), detecting and preventing potential system failures (Cathey et al. 2021 ), and ensuring data privacy and integrity (Wen et al. 2020 ). Thus, cloud computing and IoT emerge as complementary technologies, synergistically advancing the development of smart, interconnected systems.

Research in the Oil and Gas industry reflects a systematic adoption of digital twins and cloud/edge computing. For instance, (Pivano et al. 2019 ) discusses offloading simulations and data analysis to public cloud servers to access greater computational resources and avoid complex local IT infrastructures. Tygesen et al. (Tygesen et al. 2018 ) highlight the role of high-performance cloud computing in wave load modeling, which is essential for maintaining offshore platform integrity. (ASME 2018 ) describes the use of cloud data lakes for data verification and physical model feeding. At the same time, a microservices-based approach has been presented for designing and implementing digital twins using open-source tools (Zborowski 2018 ).

4.4 Implementation of augmented reality in digital twin technology

Augmented Reality (AR) is a technology that merges the real with the virtual, facilitating real-time interaction and 3D registration (Damjanovic-Behrendt and Behrendt 2019 ). It enhances user experience by superimposing graphics, video streams, or holograms onto the physical world (Yin et al. 2023 ). It is supported by various devices such as AR head-mounted displays (HMD), tablets, head-up displays (HUD), projectors, VR HMD with cameras, and 2D screen augmentations.

AR's enhancements are primarily derived from its visualization, interaction, 3D registration, and information collection capabilities as a unified device (Billinghurst et al. 2015 ). AR contributes to Digital Twin (DT) technology in several dimensions. In the virtual twin dimension, AR provides visualization of non-registered geometry, data, workflows, and basic status monitoring and alerting for operators. It also allows users to update DT information, for example, by scanning barcodes or adding annotations. However, the full potential of AR in augmenting DTs remains underexploited. In the hybrid twin dimension, AR enables multi-modal interactions and on-site registered visualization, with a need for further exploration and utilization in cyber-physical interaction functions. In the cognitive twin dimension, AR-assisted DT, bolstered by edge-cloud computing systems, is poised to play a more significant role in areas like visual programming, human–robot collaboration (HRC), product design, and human ergonomics, marking promising future directions for AR-assisted DT.

Applications of AR-assisted DT span a wide range of physical scenarios, encompassing the entire product lifecycle, including the management of production facilities and services. The production process and service phases include design, production, distribution, maintenance, and end-of-life stages, as illustrated in Fig. 11 .

Applications of AR-Assisted digital twins in engineering lifecycle, including design, production, distribution, maintenance, and end-of-life stages

The systematic design process involves prototyping, pilot runs, and testing operations. Real-time data from product usage, collected by sensors, informs smart product service design or redesign, integrating DT in mapping virtual and physical objects (Praschl and Krauss 2022 ). Research in AR device utilization for design falls into three categories: product design (Zheng et al. 2018 ), service design, and system design. Service design, a creative and user-centric process for enhancing or creating new services (Chang et al. 2020 ), is supported by several studies focusing on operation training (Blomkvist et al. 2023 ), driving and flight guidance (Moya et al 2020b ), smart environments (Vidal-Balea et al. 2021 ), smart cities (Lacoche and Villain 2022 ; Ssin et al. 2021a ), and smart wetlands (Ssin et al. 2021b ), aiming to deliver user-centered services that cater to the needs of users and stakeholders.

System design entails developing architecture, components, and core algorithms for AR-assisted DT scenarios. Moya et al. (Aheleroff et al. 2020 ; Moya et al. 2020a ) introduced two self-learning DT systems with screen augmentation for fluid behavior prediction and beam load analysis.

The production process includes goods fabrication or service provision, subdivided into process planning and scheduling (Wiegand et al. 2018 ), monitoring and control (Lemos et al. 2022 ), assembly (Kritzler et al. 2017 ), and robotics-related works (Židek et al. 2021 ). Real-time machine status monitoring and interactive control are prevalent in research, as demonstrated by Paripooranan et al. (Paripooranan et al. 2020 ), who developed an AR-enabled 3D printer DT for alerting abnormal statuses.

In distribution, warehouse management utilizes AR and DT, as shown by Petković et al. (Petković et al. 2019 ) in their use of a warehouse system DT (comprising the warehouse, automated guided vehicles (AGV), and operators with AR HMD) to test a human intention estimation algorithm.

Maintenance work encompasses various strategies and can be categorized into reactive, preventive, and predictive maintenance (Petković et al. 2019 ), adopting different approaches within the AR-assisted DT framework.

4.5 Hybrid twins in mixed reality applications

Mixed Reality (MR) applications offer an interactive experience that blends real and virtual environments, akin to Augmented Reality (AR) (Damjanovic-Behrendt and Behrendt 2019 ). Additionally, the term Extended Reality (XR) encompasses Virtual Reality (VR), AR, and MR and has been included in the research scope. The enhancements brought about by AR are examined across three distinct dimensions of the digital twin: the virtual twin, hybrid twin, and cognitive twin, as depicted in Fig. 12 .

Layered framework for digital twin classification based on augmented reality devices' perceptual capabilities

The virtual twin dimension encompasses data transmission from physical to virtual realms, non-registered visualization, and essential status monitoring and alerting functions based on sensor data. When enhanced by Augmented Reality (AR) devices' perceptual capabilities, this dimension can improve the data transmission process from the physical to the virtual space and suitably update Digital Twin (DT) information. Beyond IoT sensor data, on-site information such as barcodes and workspace details can also be gathered through AR applications, exemplified in warehouse management (Xia et al. 2022 ).

Reference (John Samuel et al. 2022 ) discusses the concept of hybridization in DTs, focusing on refining DT accuracy through self-adaptation and data-driven estimation techniques. This approach integrates physics-based model predictions with process measurements, creating a hybrid digital twin (HT) that facilitates the soft-sensing of otherwise hard-to-predict data.

The hybrid twin dimension emphasizes analysis and feedback from the virtual to the physical world, such as context information-related analysis, visual registration, multi-modal interaction and control, and the functionalities based on these aspects. Traditional DTs manage real-time data analysis, including simulation, prediction, diagnosis, and optimization, feeding back the analysis outcomes from the virtual to the physical world. AR-assisted DTs enhance this analysis with on-site data, adding capabilities like object localization, scene understanding, and cyber-physical interaction computation. For instance, in human–robot collaboration (HRC) assembly (Johansen et al. 2023 ), the hybrid twin dimension offers immersive visual registration beyond traditional 2D interfaces, displaying geometry and key data overlaid on the physical entity in the correct position. In contexts such as assembly (Liu et al. 2022 ; Zhao and Sun 2020 ), maintenance (Meier et al. 2021 ; Li et al. 2021 ; Rabah et al. 2018 ), and manual or semi-automated tasks (Koteleva et al. 2021 ; Rebmann et al. 2020 ; Mandl et al. 2017 ), operators can reference on-site instructions and guidance to work more efficiently. Additionally, geometry overlay for inspection (Catalano et al. 2022 ; Xie et al. 2020 ) or motion preview (Kim and Olsen 2021 ) aids operators in verifying the shape or movement of physical entities against planned outcomes. Users can also add geometry-linked or position-related annotations through AR.

Akroyd et al. (Akroyd et al. 2022 ) introduced the concept of the Universal Digital Twin, a digital twin that leverages a dynamic knowledge graph to enable cross-domain interoperability for DTs.

4.6 Development of cognitive twins in digital twin technology

Cognitive twins represent an advanced form of digital twins endowed with high-level cognitive capabilities encompassing machine and human intelligence. These cognitive twins are designed to address complex and unpredictable situations using enhanced computational power dynamically. Augmented Reality (AR) significantly contributes to the development of cognitive twins as it can function as a wearable computational unit within the edge-cloud architecture (Li et al 2022 ). HoloLens 2, a widely-used AR device, notably possesses substantial computing power (1 T FLOP) compared to wearable devices like sensors. This capability allows training models on high-power devices and their subsequent deployment on HoloLens 2, highlighting one of AR's key benefits to digital twins.

Cognitive Digital Twins (CDTs), originating from the domains of Industry 4.0 and Smart Cities, are recognized for their ability to support autonomous activities (Um et al. 2018 ; Liu et al. 2023 ; Zheng et al. 2021 ). Semantic technologies, including ontology and Knowledge Graph (KG), are vital in interlinking digital twins in virtual spaces. These technologies eliminate ambiguity across heterogeneous systems, thus enhancing digital interoperability and enabling cooperative decision-making (Rožanec et al. 2022 ). As defined by (Pan et al. 2021 ), ontology involves a set of formal and explicit vocabularies characterized by shareability and reusability, describing domain-specific knowledge, entities' attributes, and their interrelationships. While early research primarily focused on utilizing ontology for data modeling and sharing (Rožanec et al. 2022 ), recent studies emphasize that integrating semantics with digital twin technologies can advance the capability and interoperability of CDTs in autonomous and cooperative decision-making (Zheng et al. 2021 ).

The knowledge graph has become increasingly important in developing and managing CDTs because it can delineate relationships between real-world entities or link data (Liu et al. 2023 ). For instance, recent research has explored using knowledge graphs and digital twins in managing assets and tasks in smart manufacturing systems (Guarino et al. 2009 ) and underwater ship inspections (Zheng et al. 2023 ). Some studies have concentrated on methodologies that leverage knowledge graphs to create semantic data models for shaping digital twins (Waszak et al. 2022 ).

Furthermore, the evolving flexibility and customization in futuristic smart manufacturing are closely linked with human intelligence. For example, in human–robot collaboration (HRC) tasks aimed at improving human ergonomics (Steinmetz et al. 2022 ), operators can adjust robot poses through gesture-based interactions with the robot's digital twin. After receiving instructions from human operators, the robot digital twin learns to perform better and meet human needs. Additionally, in the timber prefabrication process (Dimitropoulos et al. 2021 ), AR provides effective interaction methods to enhance mutual understanding between operators and collaborative robots, ultimately facilitating harmonious task sharing.

4.7 Classification of digital twins by scale

Digital twins can be categorized into various types based on their scale and comprehensiveness, including component, asset, system, and process twins (Amtsberg et al. 2021 ).

Component twins : This approach suits large, complex digital twins. The adaptation and uncertainty quantification of the model in such applications can be framed as a Bayesian state estimation problem. Here, data from the physical world is used to infer which models from a model library best represent the digital twins. This approach strategically selects specific components for replication in the digital twins to avoid data redundancy and reduce costs. Microsoft has developed the Azure Digital Twins (ADT) platform (Cinar et al. 2020 ), facilitating model creation and offering a graph API for querying and interacting with these digital twins. The ADT platform enables users to visualize and examine the relationships among components, such as creating 3-D digital twins of a factory with a user-friendly interface. This interface allows operators to monitor the state of each machine. A notable challenge in this scenario involves loading each 3-D object instance into the scene. Repeated loading of the same object in different locations can lead to inefficiencies.

To address this, future developments in component twins could involve a system where a single instance of a 3-D object is streamed, loaded into memory, and rendered multiple times as needed. This approach would optimize the handling of 3-D objects in digital twin environments, enhancing efficiency and reducing the computational load.

Asset twins : This methodology focuses on creating data-driven digital twins using a library of physics-based reduced-order models. When a single model library is shared among numerous assets, this approach can effectively scale to applications requiring a substantial number of digital twins (Krzyczkowski 2019 ). Asset twins involve an estimation process wherein online sensor data from a physical asset determines which models from the library should be integrated into the digital twin. Future advancements in asset twins should enhance the robustness of model selection, particularly in the context of corrupted data. Implementing mechanisms to improve robustness and incorporating various damage models to detect and classify actual asset damage is also essential. GE Healthcare (Kapteyn et al. 2020 ) has noted the application of asset twins in healthcare, addressing challenges such as staffing model design and surgical block schedule optimization.

System twin : Operating at a higher level, system twins amalgamate different assets to form a complete functional system, such as a vehicle's brake system (Aghdam et al. 2021 ). These twins offer insights into asset interactions, thereby augmenting overall performance.

Process twin : Process twins utilize high-performance computing to optimize equipment and manufacturing processes. This is achieved by integrating multidimensional process knowledge models (Aghdam et al. 2021 ). Manufacturers can attain unparalleled efficiency and deeper insights by combining production processes with economic considerations.

Application : A digital twin system integrating Virtual Reality (VR) and Artificial Intelligence (AI) technologies has been developed to monitor and analyze welder behavior. This system exemplifies the practical application of digital twin technology in understanding and improving specific work processes.

5 Datasets, data models, and software for developing digital twins

The transformation of physical assets into digital twins involves an in-depth asset data collection process, which is then utilized to form an exact digital counterpart. This procedure is essential for asset management and predictive maintenance. There are variant data models and datasets used to underpin the digital twin initiative and significantly enhance the effectiveness and capabilities of digital twin implementations while reducing development efforts and optimizing the total cost of ownership. Many software applications have recently been used to create and manage digital twins. This section presents samples of Data models, Datasets, and software applications.

5.1 Smart city data models and datasets

To illustrate the potential of digital twins in smart cities, let us consider examples of digital twin data models and datasets that provide valuable insights for urban planning and management. Digital twin data can be applied in both tangible and virtual realms. These data are pivotal for asset monitoring, operational optimization, and safety enhancement in physical settings. On the other hand, virtual landscapes enable realistic simulations, training endeavors, and strategic planning. This dual use of digital twins highlights their adaptability, effectively bridging the real and digital domains.

One of the cornerstones of DT design and development is modeling data. Data originate from heterogeneous sources, use various protocols, and include their own data attributes, attribute types, and relationships. In order to ensure interoperability, it is necessary not only to standardize the communication between DT components but also to standardize the data format that flows through these components.

3D city modeling transcends mere data acquisition and processing, extending into data management, storage, and exchange. Consequently, open and standardized data models and exchange formats are essential for 3D city modeling. CityGML and its streamlined counterpart, CityJSON (Ledoux et al. 2019 ), are the most established data formats for 3D city models. These formats facilitate representations ranging from basic to richly detailed, depending on the required level of detail (LoD). The building model is depicted in five levels of detail, from LOD0 to LOD4, with higher LoDs offering more detail and accuracy. The aim is to manage the complexity of 3D models effectively.

In their study, the authors (Lei et al. 2022 ) assess 40 authoritative 3D city models that have emerged since 2013. This evaluation yields both quantitative and qualitative insights. The framework developed offers a thorough and structured comprehension of the landscape of semantic 3D geospatial data while also serving as an evaluated compilation of open 3D city models.

In (Ledoux et al. 2019 ), digital twin (DT) initiatives in cities are classified based on the nature of their digital replicas (static or dynamic, i.e., incorporating sensor or IoT data) and the extent of data integration (the data connection between the physical and digital worlds). Various static datasets utilize digital model integration, including Helsinki 3D + , Espoo, Vienna, Zurich 3-4D, and Amsterdam3D. Meanwhile, dynamic datasets such as Digital Twin Munich, Rennes 3D, Virtual Gothenburg, and Sofia-Bulgaria employ digital shadow integration. Furthermore, dynamic datasets like DUET, Fishermans, and Virtual Singapore implement digital twin integration. It can be inferred that most initiatives are digital shadows, given that data connections from the real world to the digital copy are automated. At the same time, the reverse typically involves manual processes (human interventions adapting the physical world). This bidirectional connection warrants further exploration.

Many research projects and similar initiatives mainly focus on collecting and providing IoT data generated from smart cities. For example, the ODAA platform ( 2016 ) Footnote 1 provides open access to data collected from the City of Aarhus using IoT infrastructure deployed within the city. The datasets within the ODAA are categorized across various applications, including energy, population and society, transport, education, and more. Moreover, San Francisco Open Data ( 2024 ) Footnote 2 and the City of Chicago Data Portal ( 2024 ) Footnote 3 provide a centralized collection of relevant smart city datasets that are publicly accessible.

For example, the NYC Open Data Initiative has already leveraged digital twin technology to improve urban planning and citizen engagement. By providing access to a wide range of open data, including information on infrastructure, public services, and environmental factors, the initiative has empowered citizens to actively participate in shaping the city's future.

5.2 Software for digital twin creation and management

Numerous digital twin software applications are available for creating and managing digital twins in buildings, cities, and urban systems. Some notable examples include:

Autodesk revit (Autodesk 2019 ): This software is extensively used for Building Information Modeling (BIM) and is acclaimed for its comprehensive design, documentation, and collaboration tools. It enables architects, engineers, and construction professionals to create detailed 3D models and provides extensive data for informed decision-making throughout a building's lifecycle.

Esri cityengine (Badwi et al. 2022 ): CityEngine is a robust software tool for crafting 3D city models. It is utilized by urban planners and designers to generate detailed and lifelike representations of cities, offering capabilities for cityscape generation, urban environment modeling, and simulation of various urban scenarios. It also integrates with GIS data to enhance city models with geographic information and analysis.

Bentley systems openbuildings designer (Mainisa et al. 2023 ): This BIM software provides advanced building design and construction modeling tools. Architects, engineers, and construction professionals use it for detailed 3D modeling, structural analysis, and effective collaboration throughout the building lifecycle.

Unity reflect (Nämerforslund 2022 ): Unity Reflect is a platform that creates interactive and immersive experiences with digital twins. It supports real-time, high-fidelity 3D modeling for virtual and augmented reality environments, enhancing visualization, interaction, and decision-making processes.

Siemens city performance tool (Al-Obaidy et al. 2022 ): Specifically tailored for urban planning and management, this tool offers a comprehensive platform for analyzing and optimizing the performance of urban systems.

iLens from knowledge lens : This leading Industrial IoT solution addresses Industry 4.0 needs with capabilities in Interface Connectivity, Edge Computing, Monitoring and Control, and Predictive Analytics. iLens is powering diverse industries globally, including Automation, Manufacturing, Energy, and Utilities.

Iotics : Iotics' innovative digital twin technology enables seamless communication across an entire digital ecosystem. It bridges gaps between various entities, from sensors to power stations and individual trains to entire airplane networks, transcending organizational boundaries and differing data languages while maintaining security.

Kavida.ai : This supply chain digital twin platform assists enterprises in making intelligent resiliency decisions. It builds supply chain digital twins using artificial intelligence to help enterprises prevent and mitigate disruptions in real time or before they occur.

MODS reality : This cloud-based application hosts a digital twin of a facility in a point cloud environment, enhancing engineering and streamlining scheduling and work execution management for maintenance and minor modifications, thereby maximizing performance and profitability.

Twinzo : As a mobile-first live digital twin platform focused on operational excellence, twinzo visualizes and reconstructs live data in 3D, offering novel ways to analyze and consume information. It helps customers save significant operational costs and increase production output.

VEERUM's digital twin : This application is a leading visualization and analytics tool that combines CAD, geospatial, document management, IoT, and operational systems. It delivers considerable cost and time savings in operations, maintenance, reliability, and complex capital construction projects.

WillowTwinTM : Revolutionizing the built world, WillowTwinTM is a pioneering software platform for real estate and infrastructure assets. It provides a central hub for all asset data, turning siloed datasets into a virtual replica of the built form. The platform enables proactive, data-driven decision-making in real-time to reduce costs, increase profits, and manage risks.

6 Digital twin performance metrics

Extensive research has been conducted on digital twins (DTs) and their applications, yet a standard method for assessing DT performance remains elusive. Establishing a method for evaluating the performance of DTs is essential for enhancing or monitoring processes and systems within a business context. Such a method could guide researchers and practitioners in developing more effective digital twins (Psarommatis and May 2022 ).

There have been limited studies focusing on specific methodologies for assessing DT performance. Chen et al. (Chen et al. 2021 ) proposed a DT maturity model for managing industrial assets based on Gemini principles, facilitating quantitative evaluation of DT flexibility and implementation levels. Chakraborty et al. (Chakraborty and Adhikari 2021 ) assessed DT performance in a multi-time scale dynamical system using an efficient framework that leverages expectation maximization and a sequential Monte Carlo sampler for developing machine learning-based DTs. Shangguan et al. (Shangguan et al. 2020 ) evaluated DT performance for fault diagnosis using a predefined threshold technique, focusing on accuracy (ACC), specificity (SPE), and sensitivity. Psarommatis et al. (Psarommatis and May 2022 ) introduced a systematic approach for measuring DT performance and flexibility, quantifying it based on four key performance indicators (KPIs). Additionally, they introduced DTflex as a new KPI to evaluate the flexibility of digital twins.

6.1 Performance metrics categories

Although there are no well-established methods or Key Performance Indicators (KPIs) in the field for thoroughly assessing the performance of Digital Twins (DT), this study suggests classifying performance metrics according to three essential elements: software, hardware, and data management middleware. This paradigm makes it possible to evaluate the system's efficacy in detail. A thorough analysis of the body of prior research and industry norms guided the choice of these indicators. We aimed to find measures that captured the essential elements of DT performance by combining knowledge from several sources.

The proposed metrics ensure an adequate evaluation by focusing on DT performance characteristics within each component. For example, metrics about hardware components evaluate attributes like scalability, communication dependability, and sensor precision. These metrics were selected to represent the fundamental hardware performance features essential to DT's operation. Similarly, metrics related to middleware for data management emphasize security, scalability, and efficiency, highlighting middleware's vital role in integrating and controlling data streams. Finally, software component metrics highlight the significance of strong software functions for DT performance by addressing factors such as model integrity, simulation accuracy, and user interface responsiveness. Each metric recommended in this section is supported by its relevance to real-world DT implementations and alignment with broader business or operational objectives. These measures help stakeholders make well-informed decisions by offering practical insights about DT performance. Including these measures also attempts to create a standard framework for assessing DT performance in various applications and domains.

6.1.1 Hardware components

Sensor accuracy: Precision and reliability of physical sensors.

Communication reliability: Efficiency of data transmission between sensors and the digital counterpart.

Hardware scalability: Ability to expand hardware components with increasing data volumes.

Latency in data acquisition: Time taken to acquire and transmit sensor data.

Hardware failure rate: Frequency and severity of failures in sensors or actuators.

6.1.2 Data management middleware

Data integration efficiency: Ease of integrating data from various sources into the DT.

Middleware latency: Time taken for middleware processes to complete tasks.

Data accuracy and consistency: Precision and consistency in data storage and management by middleware.

Scalability of middleware: Ability to handle increasing data volumes without performance degradation.

Data security protocols: Effectiveness of security protocols in protecting data during storage and transit.

6.1.3 Software components

Model fidelity: Accuracy and completeness of the digital model representing the entity.

Simulation accuracy: Precision of simulations compared to real-world scenarios.

Quality of visualization: Clarity and detail of visual representations in the user interface.

User interface responsiveness: Speed and responsiveness of the software interface to user actions.

IoT device integration: Compatibility and integration with various IoT devices.

Scalability of software: Capacity to handle increasing computational loads and data processing demands.

Software security: Protections against cyber threats and unauthorized access.

Interactivity and control: Responsiveness of software to user inputs and control commands.

Updating and maintenance efficiency: Ease of updating and maintaining software components.

Effectiveness of decision support: Capability of the software to provide meaningful insights.

6.2 Best practices for evaluating digital twin performance

As noted by Peter Drucker, Mgt. consultant and author, “You cannot manage what you cannot measure.” This principle is equally applicable to digital twins. The confusion matrix employed in data science can measure digital twins' performance ARC Advisory Group ( 2024 ). Footnote 4 Assessing the performance of digital twins necessitates a thorough approach that considers multiple aspects, including hardware, data management middleware, and software components. Below are some essential practices for effectively assessing the performance of digital twins. The formulation of these best practices necessitated a thorough examination of the current literature on DT performance evaluation. Consultations with some stack holders were also conducted. By combining information from various sources, we hoped to convey the multidimensional nature of DT performance and provide meaningful advice to practitioners and researchers alike. Furthermore, the methods were iteratively refined to ensure their usefulness and applicability across various situations and industries.

Each practice recommended in this section is based on known management principles and its ability to address important difficulties in DT performance evaluation. For example, the emphasis on objective definition and particular Key Performance Indicators (KPIs) demonstrates the significance of goal alignment and measurement precision in achieving effective DT efforts. Similarly, data quality, security assessment, and scalability analysis methods emphasize these variables' importance in assuring the dependability and efficacy of distributed computing systems.

Objective definition: Clearly articulate the goals and objectives of the digital twin implementation to align performance indicators with broader business or operational objectives.

Establish specific KPIs: Identify and set specific Key Performance Indicators (KPIs) that align with the objectives, ensuring they are measurable, relevant, and linked to desired outcomes.

Multidimensional evaluation: Assess performance across multiple dimensions, including accuracy, responsiveness, scalability, security, and usability.

Regular review and update of metrics: Given the evolving nature of digital twin environments, performance metrics should be regularly reviewed and updated to maintain relevance and accuracy.

Focus on data quality and integrity: Emphasize metrics related to data accuracy, consistency, and integrity, as the quality of the digital twin largely depends on the reliability of its data.

Incorporate end-user experience metrics: Include metrics that gauge user satisfaction and adoption, such as visualization quality, interaction, and ease of use.

Measure latency and responsiveness: Evaluate latency in data collection, middleware processing, and software responsiveness to ensure real-time or near-real-time capabilities.

Security performance assessment: Implement metrics to evaluate the efficacy of security measures, including data encryption protocols.

Scalability analysis: Examine the digital twin's scalability, focusing on how well it accommodates increasing data volumes, user numbers, and processing requirements.

Simulation accuracy verification: Regularly validate the accuracy of simulations and virtual representations against actual world scenarios to ensure the digital twin's reliability.

Benchmarking: Compare performance against industry standards or best practices to understand how the digital twin stacks up against similar implementations.

Utilize monitoring technologies: Deploy monitoring technologies that offer real-time insights into the digital twin's operation, enabling proactive issue identification and resolution.

Develop a continuous improvement process: Establish a process for continuous improvement that integrates user feedback and ongoing evaluations, fostering a culture of perpetual enhancement.

By adhering to these practices, organizations can establish a robust framework for assessing and improving the performance of their digital twins, ensuring that these technologies deliver maximum value and effectively contribute to strategic objectives. To sum up, this section's performance indicators are the outcome of a systematic approach guided by academic research and industry observations. We hope to give readers a thorough grasp of how these measures support efficient DT evaluation procedures by outlining the reasoning behind their selection and their applicability to DT performance assessment.

7 Challenges associated with digital twins

Understanding the obstacles encountered while deploying digital twin technology is critical for its successful adoption and improvement. This section elucidates the difficulties various components of digital twin systems face, shedding light on their origins and implications. The challenges outlined are meticulously identified through an extensive review of literature and insights from field and industry experts, signifying their significance in the successful deployment and operation of digital twin systems. This analysis integrates multiple sources to pinpoint these hurdles as key challenges. The study organizes the identified challenges into three main aspects of digital twin technology: hardware, data management middleware, and software. This categorization facilitates a thorough understanding of the complex problems impacting different aspects of digital twin systems. A thorough examination of these challenges across the hardware, data management middleware, and software components aids in bridging the current research gap. Whereas prior studies often discussed these challenges in broad strokes, (Tuhaise et al. 2023 ) divided them into three categories: data transmission, interoperability, and data integration. This research details specific problems within each distinct component of the digital twin framework, thereby offering an in-depth analysis of the inherent obstacles in digital twins. It identifies hardware-related challenges, such as the complexity of sensor integration and issues with hardware reliability, suggesting solutions like adopting standardized sensor interfaces and employing predictive maintenance strategies. Furthermore, the study uncovers problems in data management middleware, including data integration bottlenecks and interoperability issues, recommending developing scalable middleware systems and adopting universal standards to enhance interoperability. The research outlines security vulnerabilities and algorithmic complexity regarding software components, proposing using advanced analytical tools and robust cybersecurity measures as solutions.

By delineating these issues across hardware, middleware, and software components, the study enhances the understanding of digital twin technology and offers actionable recommendations for enhancing the technology’s effectiveness and resilience. As digital twin technology continues to evolve, the findings underscore the necessity of concentrating on these components to surmount challenges and fully exploit the technology's potential across various applications and industries. The examination of digital twin elements and their associated challenges is visually summarized in Fig. 13 , which consists of three parts: (a) delineates the components of a digital twin, (b) identifies the challenges specific to each component, and (c) proposes solutions to these challenges.

Overview of digital twin components, associated challenges, and proposed solutions. Part ( a ) delineates the core components of a Digital Twin (DT). In part ( b ), a detailed breakdown highlights the challenges inherent in each component. Part ( c ) provides insightful solutions strategically proposed to address these challenges and enhance the effectiveness of Digital Twin implementation

7.1 Hardware components

Hardware components are the foundation of digital twin systems, comprising sensors, actuators, and other physical devices. Challenges within this component include:

Sensor integration complexity: Integrating diverse sensors for real-time data poses compatibility and synchronization issues.

Hardware reliability: Ensuring long-term sensor and actuator reliability is essential.

Proposed solutions involve adopting standardized sensor interfaces and implementing predictive maintenance strategies to mitigate these challenges.

7.2 Data management middleware

Middleware plays a crucial role in managing and processing the vast amount of data generated by digital twin systems. Challenges within this component include:

Data integration bottlenecks: Handling diverse data streams can lead to processing delays.

Interoperability issues: Different standards may hinder middleware system compatibility.

Proposed solutions include developing scalable middleware architectures and embracing industry-wide standards for improved interoperability.

7.3 Software components

Software components encompass the algorithms and analytical tools for real-time data analysis and decision-making. Challenges within this component include:

Algorithmic complexity: Complex algorithms for real-time analytics and decision-making need streamlining.

Security vulnerabilities: Software components are susceptible to cybersecurity threats.

Proposed solutions involve utilizing advanced analytical tools and robust cybersecurity protocols to address these challenges.

In conclusion, addressing the challenges linked with digital twins requires a deep understanding of their core components: hardware, data management middleware, and software. This analysis has unveiled various obstacles, from hardware constraints to data integration complexities and software interoperability challenges. A comprehensive perspective is provided by examining these issues across the distinct hardware, middleware, and software components. It is essential to identify and tackle the limitations associated with hardware, the challenges within middleware, and the issues related to software interoperability to enhance the efficiency and robustness of digital twin systems. As digital twin technology evolves, prioritizing these areas will be critical for navigating difficulties and leveraging the technology’s capacity in diverse applications and industries.

8 Case studies

Case studies in the realm of smart cities and digital twins serve as vital illustrations of these technologies in practical scenarios:

Dubai's "Happiness Agenda": A smart city initiative using big data to enhance urban living and measure "happiness" across various criteria. The objective was to involve every citizen in shaping future cities, particularly focusing on citizen engagement. Dubai’s "Happiness Agenda" implementation represents a notable example of a smart city involving its residents in urban development. Dubai has positioned itself as one of the "happiest" places to live by defining citizen "happiness" across multiple criteria. It uses big data analysis to allocate urban resources strategically, enhancing the city's overall "Happiness Index" (Zakzak 2019 ).

West Cambridge site and IFM building: These case studies explore adaptable digital twins at the building level, integrating various data sources and AI-driven decision-making. The West Cambridge site of the University of Cambridge in the UK was chosen as a case study due to its diverse facilities, which include university buildings, sports centers, residence areas, main roads, parking places, and restaurants. This variety allows for testing and evaluating the proposed dynamic digital twin system across different types of infrastructure. Additionally, the site's size and complexity offer an ideal environment to assess the effectiveness of the technology. Access to extensive data sources, collaboration opportunities with experts, and relevance to the academic community further contribute to its suitability as a testbed for the study (Qiuchen Lu et al. 2019 ).

Herrenberg, Germany: A case study demonstrating the use of digital twin technology in urban planning and city management. The case study of Herrenberg might illustrate the implementation and benefits of digital twins in improving urban planning, infrastructure management, and citizen engagement within the city. Herrenberg was selected as a case study for the digital twin due to its relevance to urban challenges, accessibility of diverse data sources, the potential for collaboration with local stakeholders, engagement of the community, and suitability in terms of size and complexity for testing the digital twin technology (Dembski et al. 2020 ).

Cambridge Sub-region: A digital twin pilot is developed, integrating diverse data streams for urban planning and decision-making. The authors stress the significance of including diverse data like IoT sensors, satellite images, social media, and government records to ensure an all-encompassing and precise city representation. The case study presented in the paper is about developing a digital twin pilot for the Cambridge Sub-region. It highlights how integrating various data streams and simulation models can assist urban planning, resource allocation, and decision-making processes. The case study provides insights into the potential benefits of using a city-level digital twin for improving efficiency, sustainability, and resilience in urban environments (Wan et al. 2019 ).

Málaga City: Implementing cognitive analytics in smart city management to enhance transportation, energy, and public services. The focus is on enhancing various aspects of urban life, such as transportation, energy management, waste management, and public services. The case study of Málaga City demonstrates the practical implementation of cognitive analytics to improve decision-making processes, optimize resource allocation, and ultimately enhance the quality of life for its residents (Pérez and Toledo 2017 ).

Ålesund, Norway: The study explores the role of a data-driven digital twin in enhancing urban systems and services within a smart city framework. It suggests using high-quality 3D graphical digital twins (GDTs) of cities to generate 4D visualizations of geolocalized time-series data to enhance citizen engagement. Through a case study conducted in Ålesund, Norway, the methodology utilizes readily available hardware and a game engine to develop immersive environments for presenting complex data sourced from GIS, BIM, demographics, and IoT. The approach emphasizes scalability, transferability, versatility in data integration, adherence to privacy regulations, and dependable data delivery. The paper introduces a pioneering smart city GDT framework, which capitalizes on interactive features and advancements in metrology (Major et al. 2021 ).

Case study in Greece: Details the development and application of digital twins tailored for smart cities, focusing on urban infrastructure improvements. The case study probably illustrates how digital twins optimize city systems, improve efficiency, and facilitate decision-making processes in Greek urban environments. This study might showcase practical examples of implementing digital twin technology to address challenges and enhance the overall functioning of a smart city in Greece (Evangelou et al. 2022 ).

Each case study offers unique insights into the deployment and impact of digital twin technology in various urban settings, highlighting its potential to improve city management and living standards. Table 2 offers an overview of each paper's focus areas, case studies, and key highlights, showcasing their distinct contributions and applications in the field of digital twins in smart cities.

9 Smart city governance in the era of digital twins: addressing challenges and leveraging opportunities

In the evolving discourse on smart cities and digital twin technologies, a critical examination of multi-level governance, organizational practices, and governance dimensions emerges as pivotal. The collective contributions from the referenced studies provide a comprehensive overview of the challenges and strategies in implementing smart city initiatives across different governance frameworks and geographical contexts.

As examined in one study, the integration of Chinese new authoritarian principles into smart government transitions highlights the inherent tensions between state-level directives and local-level implementation, underscoring the complexity of multi-level governance in authoritarian regimes (Zhang and Mora 2023 ). This perspective is enriched by a nuanced exploration of organizational practices within smart city development, revealing how bureaucratic, technocratic, and participatory logics intersect to shape decision-making and citizen engagement in smart city projects (Mora et al. 2023a ). Furthermore, the identification of three key governance dimensions—institutional context for urban innovation, urban innovation ecosystem, and urban digital innovation—provides a framework for understanding the governance mechanisms essential for fostering smart city transitions (Mora et al. 2023b ).

Critical analysis across the studies reveals common challenges in smart city governance, such as interoperability and compatibility issues within the digital ecosystem and integrating a technological dimension in urban development. These challenges underscore the importance of addressing interoperability and compatibility to enhance city planning and management effectively (Quek et al. 2023 ). The discourse extends to the critical analysis of smart urbanism in non-Western contexts, notably in India and Africa, where issues of urban informality, equity, and the inclusivity of smart city initiatives are brought to the forefront (Prasad et al. 2023 ; Tonnarelli and Mora 2023 ). These analyses highlight the necessity of adopting equitable and inclusive smart city development approaches that consider the needs and priorities of all urban dwellers, particularly marginalized communities.

Moreover, the call for empirical studies and the integration of innovation management theory into smart city governance research emphasizes the need for practical guidance and theoretical advancements in managing urban digital innovation (Mora et al. 2023b ). The exploration of human-cyber-physical interactions further illuminates the evolving relationship between technology, governance, and societal dynamics, advocating for a holistic approach that balances technological advancements with ethical and sociocultural considerations (Quek et al. 2023 ).

In conclusion, the amalgamated insights from these studies advocate for a pragmatic, contextually informed, and inclusive approach to smart city governance. By addressing the multifaceted challenges of interoperability, governance, and citizen engagement, and by incorporating a critical perspective on urban informality and inclusivity, this body of work contributes significantly to the scholarly discourse on smart cities and digital twins. The emphasis on empirical research, innovation management, and the integration of technology in urban development underscores the dynamic interplay between technology, governance, and urban development strategies in the quest for sustainable and equitable urban futures.

9.1 Role of DT in smart city governance

Smart city governance constitutes a complex framework fundamental to the effective realization and long-term viability of smart city endeavors. It encompasses the strategic alignment of policies, technological systems, and multifaceted collaborations amongst stakeholders by overarching urban development goals. Digital twin technology plays a pivotal role in enhancing smart city governance by offering innovative solutions across various components:

Policy and strategy formulation : Crafting policies and strategies that guide smart city initiatives in service of the city's broader objectives (Beckers 2022). Digital twins assist in crafting policies and strategies by providing valuable insights derived from real-time data and simulations. City authorities can utilize digital twins to assess the impact of different policies and strategies on urban systems, enabling informed decision-making aligned with broader city objectives.

Collaborative ecosystem : Fostering partnerships spanning government entities, the private sector, academic institutions, and the citizenry, thus leveraging collective knowledge and resources (Beckers 2022). Digital twins foster collaboration among government agencies, private sector entities, academic institutions, and citizens by providing a platform for data sharing and analysis. This collaborative ecosystem enhances collective knowledge and resource utilization, facilitating more effective governance practices and co-creating solutions to urban challenges.

Technological infrastructure : Establishing and administering the technological basis, encompassing data management and digital platforms, that underpins smart city operations (Zhang and Mora 2023 ). As a foundational element of smart city operations, digital twins contribute to establishing and managing the technological infrastructure required for governance. They enable comprehensive data management and visualization, empowering city administrators to monitor urban systems, identify emerging trends, and respond proactively to issues in real-time.

Ethical considerations : Prioritizing ethical concerns by safeguarding data privacy and security and ensuring the equitable deployment of technology (Mora et al. 2023a ). Digital twins support ethical governance by prioritizing data privacy, security, and equitable technology deployment. Through robust data encryption protocols and access controls, digital twins safeguard sensitive information, ensuring that governance processes remain transparent, accountable, and inclusive for all stakeholders.

Public participation : Stimulating citizen involvement in the governance process promotes transparency and inclusiveness (Mora et al. 2023b ). Digital twins facilitate public participation by providing accessible platforms for citizen feedback, collaboration, and co-design of urban solutions. By incorporating citizen inputs into decision-making processes, digital twins help ensure that governance strategies align with community needs and preferences.

Sustainability : Championing sustainable development practices integrated within smart city projects to prioritize environmental stewardship and long-term resilience (Quek et al. 2023 ). By simulating various scenarios and assessing the environmental impact of proposed policies and projects, digital twins enable city authorities to prioritize sustainability and resilience in urban planning and decision-making processes.

9.2 Smart city governance challenges

The pursuit of smart city objectives is frequently hindered by governance crises, underscoring the complexities of managing urban digital transformations. Digital twins offer innovative solutions to navigate the complexities of urban governance and enhance decision-making processes. Here, we explore how digital twins can be utilized to tackle key challenges in smart city governance:

Data privacy and security concerns: Contending with data privacy and security risks associated with the vast collection and storage of urban data (Mora et al. 2023a ). Digital twins incorporate robust data encryption protocols and access controls, ensuring the protection of sensitive information within smart city systems. Digital twins help mitigate privacy and security risks associated with urban data collection and storage by enabling secure data management and transmission.

Digital divide and inequity: Mitigating the digital divide can potentially intensify social disparities within urban communities (Prasad et al. 2023 ). Digital twins promote inclusivity and bridge the digital divide by providing accessible platforms for citizen engagement and participation in governance processes. Through user-friendly interfaces and interactive visualization tools, digital twins empower all citizens to contribute to decision-making, regardless of their technological literacy or socioeconomic status.

Regulatory and legal challenges: Navigating the disparity between the swift pace of technological progress and prevailing regulatory frameworks (Tonnarelli and Mora 2023 ). Digital twins assist city authorities in navigating regulatory and legal frameworks by providing comprehensive data analytics and scenario modeling capabilities. Digital twins facilitate informed policy-making and ensure alignment with legal standards and industry regulations by simulating the impact of proposed regulations and assessing compliance requirements.

Fragmented governance structures: Surmounting the intricacies of multi-stakeholder governance structures, which can obstruct coordinated action (Zhang and Mora 2023 ). Digital twins serve as centralized platforms for data integration and collaboration, overcoming the challenges of fragmented governance structures. By consolidating diverse datasets from multiple stakeholders and domains, digital twins enable seamless information sharing and coordination, fostering synergy among various governmental entities and stakeholders.

Resource constraints: Confronting limitations in financial, technical, and operational capacities is vital to the success of smart city ventures (Quek et al. 2023 ). Digital twins optimize resource utilization and operational efficiencies within smart city governance through predictive analytics and optimization algorithms. By identifying inefficiencies and optimizing resource allocation, digital twins help cities overcome resource constraints and maximize the impact of limited financial, technical, and operational resources.

By integrating digital twin technologies, smart city administration can address these challenges, paving the way for innovative solutions and sustainable urban development. Digital twins offer a comprehensive and data-driven approach to governance, enabling cities to enhance decision-making processes, accountability, and transparency, ultimately enhancing the quality of life for urban residents. While digital twin technologies have the potential to significantly improve urban management through advanced data analytics, simulation, and optimization, their seamless integration into smart city governance requires careful consideration of governance issues. This includes addressing concerns related to data privacy, fostering collaboration among stakeholders, and upholding ethical principles.

10 Conclusions and future research directions

This survey paper employs a meticulous bibliometric methodology, selecting the Web of Science database for its comprehensive coverage and developing precise search criteria to gather over 4,220 relevant articles. The analysis uses advanced tools like VOSviewer for network analyses and visualizations, including co-authorship and keyword co-occurrence maps, enabling a detailed examination of trends and relationships in Digital Twin technology and Smart Cities research. This methodological rigor ensures the study's reliability and contributes to its uniqueness in the field. This survey comprehensively reviews over 4200 publications in the domain of Digital Twins and Smart Cities. It outlines the evolution, applications, and integration of Digital Twins with IoT and AI in urban development. The survey distinguishes itself through extensive bibliometric analysis, focusing on datasets, platforms, software, and performance metrics, and it offers unique insights into the challenges and opportunities within the field. The findings include emerging trends, key thematic areas, and a detailed exploration of various Smart City applications. The paper concludes with implications for urban developers, policymakers, and researchers and recommendations for future research directions. The field of Digital Twin (DT) and Smart Cities is ripe for future research, aiming to overcome current challenges and explore new frontiers. Detailed investigation and development in this area are essential for realizing the full potential of DT technologies in urban environments. The discussions pave the way for sustainable and equitable urban futures, recognizing the dynamic interplay between technology, governance, and urban development strategies.

Future research should focus on:

Enhanced data integration : Developing more efficient methods for integrating diverse data sources within DT systems.

Scalability solutions : Creating scalable DT models suitable for larger and more complex urban environments.

Advanced security protocols : Strengthening cybersecurity measures for DT systems to ensure data privacy and security.

Sophisticated analytical tools : Incorporating cutting-edge AI and machine learning techniques for predictive analytics and decision-making.

Expanding IoT capabilities : Extending the use of IoT in DTs for comprehensive real-time data collection and monitoring.

Sustainable urban development : Leveraging DTs for resource management, focusing on sustainability and environmental conservation.

Citizen engagement models : Developing DTs prioritizing citizen involvement in urban planning and management.

Policy and governance studies : Examining the influence of policy in guiding DT implementation and addressing ethical concerns.

Economic impact assessment : Evaluating the economic implications of DTs, including cost analysis and return on investment.

Real-world case studies : Documenting extensive case studies to assess DTs' practical impact and challenges in urban settings.

Investigating future technological advancements: new applications, and the role of policy and governance in Digital Twins development.

The findings of this paper are poised to influence future research, policy-making, and practical applications in Smart Cities and Digital Twins in significant ways:

Informing future research directions: The comprehensive review of over 4,200 publications provides valuable insights into the current state of Digital Twins and Smart Cities research. Researchers can utilize this information to identify gaps in existing literature and prioritize areas for further investigation. For example, identifying challenges such as data integration bottlenecks and security vulnerabilities can guide future research efforts toward developing solutions to these pressing issues.

Guiding policy development: Policymakers can leverage the findings of this paper to inform the development of policies and regulations related to Digital Twin technology and its application in Smart Cities. By understanding the challenges and opportunities associated with Digital Twins, policymakers can create frameworks that promote innovation while addressing data privacy, cybersecurity, and ethical considerations.

Improving urban planning and management: The insights provided by this paper can assist urban planners and city managers in making informed decisions about adopting and implementing Digital Twins in Smart Cities. By understanding Digital Twin technology's potential benefits and challenges, city officials can develop strategies to optimize urban infrastructure, improve resource management, and enhance citizen services.

Driving technological innovation: The paper identifies emerging trends and technological advancements in Digital Twin technology, such as the integration of AI and IoT, as well as the development of scalable models and advanced security protocols. These insights can inspire innovation in academia and industry, leading to the development of new tools, platforms, and solutions that push the boundaries of Digital Twin technology and its applications in Smart Cities.

Finally, the findings of this article have the potential to spark advances in research, policymaking, and practical applications connected to Digital Twins and Smart Cities, resulting in more efficient, sustainable, and resilient urban development.

Data availability

Data is provided within the manuscript.

http://www.odaa.dk

https://data.sfgov.org

https://data.cityofchicago.org

https://www.arcweb.com/industry-best-practices/measuring-digital-twin-performance-maturity-confusion-matrix

Abdeen FN, Shirowzhan S, Sepasgozar SME (2023) Citizen-centric digital twin development with machine learning and interfaces for maintaining urban infrastructure. Telematics Inform 84:102032. https://doi.org/10.1016/j.tele.2023.102032

Article Google Scholar

Abideen AZ, Sundram VPK, Pyeman J, Othman AK, Sorooshian S (2021) Digital twin integrated reinforced learning in supply chain and logistics. Logistics 5(4):84. https://doi.org/10.3390/logistics5040084

Abouzid I, Saidi R (2023) Digital twin implementation approach in supply chain processes. Sci Afr 21:e01821. https://doi.org/10.1016/j.sciaf.2023.e01821

Aghdam ZN, Rahmani AM, Hosseinzadeh M (2021) The role of the internet of things in healthcare: future trends and challenges. Comput Methods Programs Biomed 199:105903. https://doi.org/10.1016/j.cmpb.2020.105903

Aheleroff S, Zhong RY, Xu X, Feng Z, Goyal P (2020) Digital twin enabled mass personalization: a case study of a smart wetland maintenance system. Volume 2: manufacturing processes; manufacturing systems; nano/micro/meso manufacturing; quality and reliability. https://doi.org/10.1115/msec2020-8363

Akroyd J, Harper Z, Soutar D, Farazi F, Bhave A, Mosbach S, Kraft M (2022) Universal digital twin: land use. Data-Centric Eng 3. https://doi.org/10.1017/dce.2021.21

Alam KM, El Saddik A (2017) C2PS: a digital twin architecture reference model for the cloud-based cyber-physical systems. IEEE Access 5:2050–2062. https://doi.org/10.1109/access.2017.2657006

Allam Z, Jones DS (2021) Future (post-COVID) digital, smart and sustainable cities in the wake of 6G: digital twins, immersive realities and new urban economies. Land Use Policy 101:105201. https://doi.org/10.1016/j.landusepol.2020.105201

Al-Obaidy AHMJ, Khalaf SM, Hassan FM (2022) Application of CCME index to assess the water quality of tigris river within Baghdad City, Iraq. IOP Conf Ser: Earth Environ Sci 1088(1):012004. https://doi.org/10.1088/1755-1315/1088/1/012004

Amtsberg F, Yang X, Skoury L, Wagner H-J, Menges A (2021) iHRC: an AR-based interface for intuitive, interactive and coordinated task sharing between humans and robots in building construction. Proceedings of the International Symposium on Automation and Robotics in Construction (IAARC). https://doi.org/10.22260/isarc2021/0006

ARC Advisory Group (2024) Measuring digital twin performance maturity with the confusion matrix. Retrieved from https://www.arcweb.com/industry-best-practices/measuring-digital-twin-performance-maturity-confusion-matrix . Accessed 10 May 2024

Ariyachandra MRMF, Wedawatta G (2023) Digital twin smart cities for disaster risk management: a review of evolving concepts. Sustainability 15(15):11910. https://doi.org/10.3390/su151511910

ASME (2018) Retracted: “Kinematic analysis of the motion of a six degrees of freedom wave energy converter based on the concept of the stewart-gough platform” [ASME 2018 37th international conference on ocean, offshore and arctic engineering, Volume 10: ocean renewable energy, Madrid, Spain, June 17–22, 2018, Conference sponsors: ocean, offshore and arctic engineering division, ISBN: 978–0–7918–5131–9. Paper No. OMAE2018–78601, pp V010T09A047; 10 pages]. Volume 10: ocean renewable energy. https://doi.org/10.1115/OMAE2018-78601

Autodesk Revit Architecture Certification (2019) Mastering Autodesk® Revit® 2020, 1033–1035. Portico. https://doi.org/10.1002/9781119570189.app3

Badwi IM, Ellaithy HM, Youssef HE (2022) 3D-GIS parametric modelling for virtual urban simulation using CityEngine. Ann GIS 28(3):325–341. https://doi.org/10.1080/19475683.2022.2037019

Bagaria N, Laamarti F, Badawi HF, Albraikan A, Martinez Velazquez RA, El Saddik A (2019) Health 4.0: digital twins for health and well-being. Connected health in smart cities. pp 143–152. https://doi.org/10.1007/978-3-030-27844-1_7

Bellavista P, Giannelli C, Mamei M, Mendula M, Picone M (2021) Application-driven network-aware digital twin management in industrial edge environments. IEEE Trans Industr Inf 17(11):7791–7801. https://doi.org/10.1109/tii.2021.3067447

Biagini V, Subasic M, Oudalov A, Kreusel J (2020) The autonomous grid: automation, intelligence and the future of power systems. Energy Res Soc Sci 65:101460. https://doi.org/10.1016/j.erss.2020.101460

Billinghurst M, Clark A, Lee G (2015) A survey of augmented reality. Foundations and Trends® in Human–Computer Interaction, 8(2–3):73–272. https://doi.org/10.1561/1100000049

Blomkvist J, Clatworthy S, Holmlid S (2023) Interlude 1: materiality in design from a practitioner perspective: interview with Markus Edgar Hormeß, Adam Lawrence and Marc Stickdorn (26 April 2022). The materials of service design. pp 83–88. https://doi.org/10.4337/9781802203301.00019

Borrmann A, König M, Koch C, Beetz J (2018). Building information modeling: why? What? How? Building information modeling. pp 1–24. https://doi.org/10.1007/978-3-319-92862-3_1

Bouzguenda I, Alalouch C, Fava N (2019) Towards smart sustainable cities: a review of the role digital citizen participation could play in advancing social sustainability. Sustain Cities Soc 50:101627. https://doi.org/10.1016/j.scs.2019.101627

Catalano M, Chiurco A, Fusto C, Gazzaneo L, Longo F, Mirabelli G, Nicoletti L, Solina V, Talarico S (2022) A digital twin-driven and conceptual framework for enabling extended reality applications: a case study of a brake discs manufacturer. Procedia Comput Sci 200:1885–1893. https://doi.org/10.1016/j.procs.2022.01.389

Cathey G, Benson J, Gupta M, Sandhu R (2021) Edge centric secure data sharing with digital twins in smart ecosystems. 2021 Third IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA). https://doi.org/10.1109/tpsisa52974.2021.00008

Cerovsek T (2011) A review and outlook for a ‘Building Information Model’ (BIM): a multi-standpoint framework for technological development. Adv Eng Inform 25(2):224–244. https://doi.org/10.1016/j.aei.2010.06.003

Chakraborty S, Adhikari S (2021) Machine learning based digital twin for dynamical systems with multiple time-scales. Comput Struct 243:106410. https://doi.org/10.1016/j.compstruc.2020.106410

Chang T-W, Hsiao C-F, Chen C-Y, Huang W-X, Datta S, Mao W-L (2020) Fabricating behavior sensor computing approach for coexisting design environment. Sensors Mater 32(7):2409. https://doi.org/10.18494/sam.2020.2809

Charitonidou M (2022) Urban scale digital twins in data-driven society: challenging digital universalism in urban planning decision-making. Int J Archit Comput 20(2):238–253. https://doi.org/10.1177/14780771211070005

Chen L, Xie X, Lu Q, Parlikad AK, Pitt M, Yang J (2021) Gemini principles-based digital twin maturity model for asset management. Sustainability 13(15):8224. https://doi.org/10.3390/su13158224

Cinar ZM, Nuhu AA, Zeeshan Q, Korhan O (2020) Digital twins for industry 4.0: a review. Industrial engineering in the digital disruption era, 193–203. https://doi.org/10.1007/978-3-030-42416-9_18

City of Chicago Data Portal (2024) Chicago Data Portal. https://data.cityofchicago.org . Accessed 10 May 2024

Crespi N, Drobot AT, Minerva R (2023) The digital twin: what and why? The digital twin.pp 3–20. https://doi.org/10.1007/978-3-031-21343-4_1

Damjanovic-Behrendt V, Behrendt W (2019) An open source approach to the design and implementation of Digital Twins for Smart Manufacturing. Int J Comput Integr Manuf 32(4–5):366–384. https://doi.org/10.1080/0951192x.2019.1599436

Damjanovic-Behrendt V (2018) A digital twin-based privacy enhancement mechanism for the automotive industry. 2018 International Conference on Intelligent Systems (IS). https://doi.org/10.1109/is.2018.8710526

Dani AAH, Supangkat SH, Lubis FF, Nugraha IGBB, Kinanda R, Rizkia I (2023) Development of a smart city platform based on digital twin technology for monitoring and supporting decision-making. Sustainability 15(18):14002. https://doi.org/10.3390/su151814002

Defraeye T, Tagliavini G, Wu W, Prawiranto K, Schudel S, Assefa Kerisima M, Verboven P, Bühlmann A (2019) Digital twins probe into food cooling and biochemical quality changes for reducing losses in refrigerated supply chains. Resour Conserv Recycl 149:778–794. https://doi.org/10.1016/j.resconrec.2019.06.002

Dembski F, Wössner U, Letzgus M, Ruddat M, Yamu C (2020) Urban digital twins for smart cities and citizens: the case study of Herrenberg, Germany. Sustainability 12(6):2307. https://doi.org/10.3390/su12062307

Deren L, Wenbo Y, Zhenfeng S (2021) Smart city based on digital twins. Comput Urban Sci 1(1). https://doi.org/10.1007/s43762-021-00005-y

Diakite AA, Ng L, Barton J, Rigby M, Williams K, Barr S, Zlatanova S (2022) Liveable city digital twin: a pilot project for the city of Liverpool (NSW, Australia). ISPRS Ann Photogramm Remote Sens Spatial Inf Sci X-4/W2-2022:45–52. https://doi.org/10.5194/isprs-annals-x-4-w2-2022-45-2022

Dimitropoulos N, Togias T, Zacharaki N, Michalos G, Makris S (2021) Seamless human-robot collaborative assembly using artificial intelligence and wearable devices. Appl Sci 11(12):5699. https://doi.org/10.3390/app11125699

Din GMU, Marnerides AK (2017) Short term power load forecasting using Deep Neural Networks. 2017 International Conference on Computing, Networking and Communications (ICNC). https://doi.org/10.1109/iccnc.2017.7876196

Dong R, She C, Hardjawana W, Li Y, Vucetic B (2019) Deep learning for hybrid 5G services in mobile edge computing systems: learn from a digital twin. IEEE Trans Wireless Commun 18(10):4692–4707. https://doi.org/10.1109/twc.2019.2927312

Endsley MR (2016) Designing for situation awareness. CRC Press. https://doi.org/10.1201/b11371

Erol T, Mendi AF, Dogan D (2020) The digital twin revolution in healthcare. 2020 4th International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT). https://doi.org/10.1109/ismsit50672.2020.9255249

Evangelou T, Gkeli M, Potsiou C (2022) Building digital twins for smart cities: a case study in Greece. ISPRS Ann Photogramm Remote Sens Spatial Inf Sci X-4/W2-2022:61–68. https://doi.org/10.5194/isprs-annals-x-4-w2-2022-61-2022

Fan C, Mostafavi A (2019) A graph-based method for social sensing of infrastructure disruptions in disasters. Comput-Aided Civ Infrastruct Eng 34(12):1055–1070. https://doi.org/10.1111/mice.12457 . (Portico)

Fan C, Zhang C, Yahja A, Mostafavi A (2021) Disaster City Digital Twin: a vision for integrating artificial and human intelligence for disaster management. Int J Inf Manage 56:102049. https://doi.org/10.1016/j.ijinfomgt.2019.102049

Fan C, Jiang Y, Mostafavi A (2020) Social sensing in disaster city digital twin: integrated textual–visual–geo framework for situational awareness during built environment disruptions. J Manag Eng 36(3). https://doi.org/10.1061/(asce)me.1943-5479.0000745

Fang X, Wang H, Liu G, Tian X, Ding G, Zhang H (2022) Industry application of digital twin: from concept to implementation. Int J Adv Manuf Technol 121(7–8):4289–4312. https://doi.org/10.1007/s00170-022-09632-z

Ford DN, Wolf CM (2020) Smart cities with digital twin systems for disaster management. J Manag Eng 36(4). https://doi.org/10.1061/(asce)me.1943-5479.0000779

Gerlach B, Zarnitz S, Nitsche B, Straube F (2021) Digital supply chain twins—conceptual clarification, use cases and benefits. Logistics 5(4):86. https://doi.org/10.3390/logistics5040086

Ghaith M, Yosri A, El-Dakhakhni W (2022) Digital twin: a city-scale flood imitation framework. Proceedings of the Canadian society of civil engineering annual conference 2021. pp 577–588. https://doi.org/10.1007/978-981-19-1065-4_48

Ghosh D, Chun SA, Shafiq B, Adam NR (2016) Big data-based smart city platform. Proceedings of the 17th international digital government research conference on digital government research. https://doi.org/10.1145/2912160.2912205

Goia B, Cioara T, Anghel I (2022) Virtual power plant optimization in smart grids: a narrative review. Future Internet 14(5):128. https://doi.org/10.3390/fi14050128

Grieves MW (2005) Product lifecycle management: the new paradigm for enterprises. Int J Prod Dev 2(1/2):71. https://doi.org/10.1504/ijpd.2005.006669

Guarino N, Oberle D, Staab S (2009) What is an ontology? Handbook on ontologies. pp 1–17. https://doi.org/10.1007/978-3-540-92673-3_0

Guo D, Zhong RY, Lin P, Lyu Z, Rong Y, Huang GQ (2020) Digital twin-enabled Graduation Intelligent Manufacturing System for fixed-position assembly islands. Robot Comput-Integr Manuf 63:101917. https://doi.org/10.1016/j.rcim.2019.101917

He X, Ai Q, Wang J, Tao F, Pan B, Qiu R, Yang B (2023) Situation awareness of energy internet of things in smart city based on digital twin: from digitization to informatization. IEEE Internet Things J 10(9):7439–7458. https://doi.org/10.1109/jiot.2022.3203823

He Y, Deng J, Li H (2017) Short-term power load forecasting with deep belief network and copula models. 2017 9th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC). https://doi.org/10.1109/ihmsc.2017.50

He X, Ai Q, Qiu RC, Zhang D (2019) Preliminary exploration on digital twin for power systems: challenges, framework, and applications. arXiv preprint arXiv:1909.06977. https://www.semanticscholar.org/paper/Preliminary-Exploration-on-Digital-Twin-for-Power-He-Ai/360cd3a26ad4ab5f3685d676d36001348dd6e1d6 . Accessed 10 May 2024

Hinchy EP, O’Dowd NP, McCarthy CT (2019) Using open-source microcontrollers to enable digital twin communication for smart manufacturing. Procedia Manuf 38:1213–1219. https://doi.org/10.1016/j.promfg.2020.01.212

Hu L, Nguyen N-T, Tao W, Leu MC, Liu XF, Shahriar MR, Al Sunny SMN (2018) Modeling of cloud-based digital twins for smart manufacturing with MT connect. Procedia Manuf 26:1193–1203. https://doi.org/10.1016/j.promfg.2018.07.155

Huang J, Zhao L, Wei F, Cao B (2021) The application of digital twin on power industry. IOP Conf Ser: Earth Environ Sci 647(1):012015. https://doi.org/10.1088/1755-1315/647/1/012015

Ivanov S, Nikolskaya K, Radchenko G, Sokolinsky L, Zymbler M (2020) Digital twin of city: concept overview. 2020 Global Smart Industry Conference (GloSIC). https://doi.org/10.1109/glosic50886.2020.9267879

Jaensch F, Csiszar A, Scheifele C, Verl A (2018) Digital twins of manufacturing systems as a base for machine learning. 2018 25th International Conference on Mechatronics and Machine Vision in Practice (M2VIP). https://doi.org/10.1109/m2vip.2018.8600844

Jafari M, Kavousi-Fard A, Chen T, Karimi M (2023) A review on digital twin technology in smart grid, transportation system and smart city: challenges and future. IEEE Access 11:17471–17484. https://doi.org/10.1109/access.2023.3241588

Johansen ST, Unal P, Albayrak Ö, Ikonen E, Linnestad KJ, Jawahery S, Srivastava AK, Løvfall BT (2023) Hybrid and cognitive digital twins for the process industry. Open Eng 13(1). https://doi.org/10.1515/eng-2022-0418

John Samuel I, Salem O, He S (2022) Defect-oriented supportive bridge inspection system featuring building information modeling and augmented reality. Innov Infrastruct Solut 7(4). https://doi.org/10.1007/s41062-022-00847-3

Jones D, Snider C, Nassehi A, Yon J, Hicks B (2020) Characterising the Digital Twin: a systematic literature review. CIRP J Manuf Sci Technol 29:36–52. https://doi.org/10.1016/j.cirpj.2020.02.002

Kapteyn MG, Knezevic DJ, Huynh DBP, Tran M, Willcox KE (2020) Data-driven physics-based digital twins via a library of component-based reduced-order models. Int J Numer Methods Eng 123(13):2986–3003. https://doi.org/10.1002/nme.6423 . (Portico)

Ketzler B, Naserentin V, Latino F, Zangelidis C, Thuvander L, Logg A (2020) Digital twins for cities: a state of the art review. Built Environ 46(4):547–573. https://doi.org/10.2148/benv.46.4.547

Khudhair A, Li H, Ren G, Liu S (2021) Towards future BIM technology innovations: a bibliometric analysis of the literature. Appl Sci 11(3):1232. https://doi.org/10.3390/app11031232

Kim J, Olsen D (2021) From BIM to inspection: a comparative analysis of assistive embedment inspection. Proceedings of the International Symposium on Automation and Robotics in Construction (IAARC). https://doi.org/10.22260/isarc2021/0123

Kitchenham B, Pearl Brereton O, Budgen D, Turner M, Bailey J, Linkman S (2009) Systematic literature reviews in software engineering – a systematic literature review. Inf Softw Technol 51(1):7–15. https://doi.org/10.1016/j.infsof.2008.09.009

Koteleva N, Valnev V, Frenkel I (2021) Investigation of the effectiveness of an augmented reality and a dynamic simulation system collaboration in oil pump maintenance. Appl Sci 12(1):350. https://doi.org/10.3390/app12010350

Kritzinger W, Karner M, Traar G, Henjes J, Sihn W (2018) Digital Twin in manufacturing: a categorical literature review and classification. IFAC-PapersOnLine 51(11):1016–1022. https://doi.org/10.1016/j.ifacol.2018.08.474

Kritzler M, Funk M, Michahelles F, Rohde W (2017) The virtual twin. Proceedings of the seventh international conference on the internet of things. https://doi.org/10.1145/3131542.3140274

Krzyczkowski D (2019) Introducing azure digital twins. Apress. https://doi.org/10.1007/978-1-4842-5375-5

Kuber S, Sharma M, Bonetti A, Harispuru C, Soroush A (2022) Virtual testing of protection systems using digital twin technology. 2022 75th Annual Conference for Protective Relay Engineers (CPRE). https://doi.org/10.1109/cpre55809.2022.9776572

Kyriazopoulou C (2015) Smart city technologies and architectures - a literature review. Proceedings of the 4th International Conference on Smart Cities and Green ICT Systems. https://doi.org/10.5220/0005407000050016

Lacoche J, Villain É (2022) Prototyping context-aware augmented reality applications for smart environments inside virtual reality. Proceedings of the 17th international joint conference on computer vision, imaging and computer graphics theory and applications. https://doi.org/10.5220/0010768800003124

Lam P-D, Han J, Kwon K-R, Ok S-Y, Lee S-H (2023) Semantic 3D city model data visualization for smar t city digital twin. J Korea Multimed Soc 26(2):116–130. https://doi.org/10.9717/kmms.2023.26.2.116

Langenheim N, Sabri S, Chen Y, Kesmanis A, Felson A, Mueller A, Rajabifard A, Zhang Y (2022) Adapting a digital twin to enable real-time water sensitive urban design decision-making. Int Arch Photogramm Remote Sens Spat Inf Sci XLVIII-4/W4-2022:95–100. https://doi.org/10.5194/isprs-archives-xlviii-4-w4-2022-95-2022

Ledoux H, Arroyo Ohori K, Kumar K, Dukai B, Labetski A, Vitalis S (2019) CityJSON: a compact and easy-to-use encoding of the CityGML data model. Open Geospatial Data Softw Standards 4(1). https://doi.org/10.1186/s40965-019-0064-0

Lee A, Lee K-W, Kim K-H, Shin S-W (2022) A geospatial platform to manage large-scale individual mobility for an urban digital twin platform. Remote Sens 14(3):723. https://doi.org/10.3390/rs14030723

Lei B, Stouffs R, Biljecki F (2022) Assessing and benchmarking 3D city models. Int J Geogr Inf Sci 37(4):788–809. https://doi.org/10.1080/13658816.2022.2140808

Lemos MR, Cardoso VF, Otani M, Da Costa Nunes R, Da Silva VJ, De Lucena Junior VF (2022) Navigation robot training with Deep Q-Learning monitored by Digital Twin. 2022 IEEE International Conference on Consumer Electronics (ICCE). https://doi.org/10.1109/icce53296.2022.9730282

Li W, Rentemeister M, Badeda J, Jöst D, Schulte D, Sauer DU (2020) Digital twin for battery systems: cloud battery management system with online state-of-charge and state-of-health estimation. J Energy Storage 30:101557. https://doi.org/10.1016/j.est.2020.101557

Li C, Zheng P, Li S, Pang Y, Lee CKM (2022) AR-assisted digital twin-enabled robot collaborative manufacturing system with human-in-the-loop. Robot Comput-Integr Manuf 76:102321. https://doi.org/10.1016/j.rcim.2022.102321

Li Y, Zhang Z, Li X, Guan S (2021) Research on equipment maintenance guidance technology based on MR and digital twin. Proceedings of the 2021 5th international conference on electronic information technology and computer engineering. https://doi.org/10.1145/3501409.3501454

Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, Clarke M, Devereaux PJ, Kleijnen J, Moher D (2009) The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol 62(10):e1–e34. https://doi.org/10.1016/j.jclinepi.2009.06.006

Liu Y, Zhang L, Yang Y, Zhou L, Ren L, Wang F, Liu R, Pang Z, Deen MJ (2019) A novel cloud-based framework for the elderly healthcare services using digital twin. IEEE Access 7:49088–49101. https://doi.org/10.1109/access.2019.2909828

Liu S, Wang XV, Wang L (2022) Digital twin-enabled advance execution for human-robot collaborative assembly. CIRP Ann 71(1):25–28. https://doi.org/10.1016/j.cirp.2022.03.024

Liu Y, Pan S, Folz P, Ramparany F, Bolle S, Ballot E, Coupaye T (2023) Cognitive digital twins for freight parking management in last mile delivery under smart cities paradigm. Comput Ind 153:104022. https://doi.org/10.1016/j.compind.2023.104022

Liu Y, Wang Z, Han K, Shou Z, Tiwari P, L. Hansen JH (2020) Sensor fusion of camera and cloud digital twin information for intelligent vehicles. 2020 IEEE intelligent vehicles symposium (IV). https://doi.org/10.1109/iv47402.2020.9304643

Lohtander M, Ahonen N, Lanz M, Ratava J, Kaakkunen J (2018) Micro manufacturing unit and the corresponding 3D-model for the digital twin. Procedia Manuf 25:55–61. https://doi.org/10.1016/j.promfg.2018.06.057

Lu Y, Huang X, Zhang K, Maharjan S, Zhang Y (2021a) Low-latency federated learning and blockchain for edge association in digital twin empowered 6G networks. IEEE Trans Industr Inf 17(7):5098–5107. https://doi.org/10.1109/tii.2020.3017668

Lu Y, Maharjan S, Zhang Y (2021b) Adaptive edge association for wireless digital twin networks in 6G. IEEE Internet Things J 8(22):16219–16230. https://doi.org/10.1109/jiot.2021.3098508

Lu Q, Parlikad AK, Woodall P, Don Ranasinghe G, Xie X, Liang Z, Konstantinou E, Heaton J, Schooling J (2020) Developing a digital twin at building and city levels: case study of West Cambridge Campus. J Manag Eng 36(3). https://doi.org/10.1061/(asce)me.1943-5479.0000763

Lugaresi G, Jemai Z, Sahin E (2023) Digital twins for supply chains: current outlook and future challenges. ECMS 2023 Proceedings Edited by Enrico Vicario, Romeo Bandinelli, Virginia Fani, Michele Mastroianni. https://doi.org/10.7148/2023-0451

Mainisa M, Eka Priana S, Zulhedi Z (2023) Implementasi bim dalam permodelan 3d pembangunan gedung kantor cabang bri batusangkar menggunakan software openbuildings designer. Ensiklopedia Res Community Serv Rev 2(3):147–155. https://doi.org/10.33559/err.v2i3.1771

Major P, Li G, Hildre HP, Zhang H (2021) The use of a data-driven digital twin of a smart city: a case study of Ålesund, Norway. IEEE Instrum Meas Mag 24(7):39–49. https://doi.org/10.1109/mim.2021.9549127

Mandl B, Stehling M, Schmiedinger T, Adam M (2017) Enhancing workplace learning by augmented reality. Proceedings of the seventh international conference on the internet of things. https://doi.org/10.1145/3131542.3140265

Martinez-Velazquez R, Gamez R, El Saddik A (2019) Cardio twin: a digital twin of the human heart running on the edge. 2019 IEEE International Symposium on Medical Measurements and Applications (MeMeA). https://doi.org/10.1109/memea.2019.8802162

Meier N, Müller-Polyzou R, Brach L, Georgiadis A (2021) Digital twin support for laser-based assembly assistance. Procedia CIRP 99:460–465. https://doi.org/10.1016/j.procir.2021.03.066

Mihai S, Yaqoob M, Hung DV, Davis W, Towakel P, Raza M, Karamanoglu M, Barn B, Shetve D, Prasad RV, Venkataraman H, Trestian R, Nguyen HX (2022) Digital twins: a survey on enabling technologies, challenges, trends and future prospects. IEEE Commun Surv Tutorials 24(4):2255–2291. https://doi.org/10.1109/comst.2022.3208773

Mir M, Yaghoobi M, Khairabadi M (2022) A new approach to energy-aware routing in the Internet of Things using improved Grasshopper Metaheuristic Algorithm with Chaos theory and Fuzzy Logic. Multimed Tools Appl 82(4):5133–5159. https://doi.org/10.1007/s11042-021-11841-9

Moosavi J, Naeni LM, Fathollahi-Fard AM, Fiore U (2021) Blockchain in supply chain management: a review, bibliometric, and network analysis. Environ Sci Pollut Res. https://doi.org/10.1007/s11356-021-13094-3

Mora L, Deakin M, Reid A (2019) Combining co-citation clustering and text-based analysis to reveal the main development paths of smart cities. Technol Forecast Soc Chang 142:56–69. https://doi.org/10.1016/j.techfore.2018.07.019

Mora L, Appio FP, Foss NJ, Arellano-Gault D, Zhang X (2023a) Organizing for smart city development: research at the crossroads. Introduction to the special issue. Organ Stud 44(10):1559–1575. https://doi.org/10.1177/01708406231197815

Mora L, Gerli P, Ardito L, Messeni Petruzzelli A (2023b) Smart city governance from an innovation management perspective: theoretical framing, review of current practices, and future research agenda. Technovation 123:102717. https://doi.org/10.1016/j.technovation.2023.102717

Moya B, Badías A, Alfaro I, Chinesta F, Cueto E (2020a) Digital twins that learn and correct themselves. Int J Numer Methods Eng 123(13):3034–3044. https://doi.org/10.1002/nme.6535 . (Portico)

Moya B, Alfaro I, Gonzalez D, Chinesta F, Cueto E (2020b) Physically sound, self-learning digital twins for sloshing fluids. PLoS ONE 15(6):e0234569. https://doi.org/10.1371/journal.pone.0234569

Nämerforslund T (2022) Digital twin performance: unity as a platform for visualizing interactive digital twins. Thesis - Institution of Information Systems and Technology

National Aeronautics and Space Administration (NASA) (2010) Encyclopedia of geography. https://doi.org/10.4135/9781412939591.n797

Nica E, Popescu GH, Poliak M, Kliestik T, Sabie O-M (2023) Digital twin simulation tools, spatial cognition algorithms, and multi-sensor fusion technology in sustainable urban governance networks. Mathematics 11(9):1981. https://doi.org/10.3390/math11091981

Nikolakis N, Alexopoulos K, Xanthakis E, Chryssolouris G (2018) The digital twin implementation for linking the virtual representation of human-based production tasks to their physical counterpart in the factory-floor. Int J Comput Integr Manuf 32(1):1–12. https://doi.org/10.1080/0951192x.2018.1529430

Nochta T, Wan L, Schooling JM, Parlikad AK (2020) A socio-technical perspective on urban analytics: the case of city-scale digital twins. J Urban Technol 28(1–2):263–287. https://doi.org/10.1080/10630732.2020.1798177

Nowocin JK (2017) Microgrid risk reduction for design and validation testing using controller hardware in the loop (Doctoral dissertation, Massachusetts Institute of Technology)

ODAA platform (2016) http://www.odaa.dk . Accessed 10 May 2024

Pan S, Trentesaux D, McFarlane D, Montreuil B, Ballot E, Huang GQ (2021) Digital interoperability in logistics and supply chain management: state-of-the-art and research avenues towards Physical Internet. Comput Ind 128:103435. https://doi.org/10.1016/j.compind.2021.103435

Pang J, Huang Y, Xie Z, Li J, Cai Z (2021) Collaborative city digital twin for the COVID-19 pandemic: a federated learning solution. Tsinghua Sci Technol 26(5):759–771. https://doi.org/10.26599/tst.2021.9010026

Paripooranan CS, Abishek R, Vivek DC, Karthik S (2020) An implementation of AR enabled Digital Twins for 3-D printing. 2020 IEEE International Symposium on Smart Electronic Systems (ISES) (Formerly INiS). https://doi.org/10.1109/ises50453.2020.00043

Park H-A, Byeon G, Son W, Jo H-C, Kim J, Kim S (2020) Digital Twin for operation of microgrid: optimal scheduling in virtual space of Digital Twin. Energies 13(20):5504. https://doi.org/10.3390/en13205504

Pérez JG, Toledo DG (2017) Cognitive analytics of smart cities. Proceedings of the 18th annual international conference on digital government research. https://doi.org/10.1145/3085228.3085265

Petković T, Puljiz D, Marković I, Hein B (2019) Human intention estimation based on hidden Markov model motion validation for safe flexible robotized warehouses. Robot Comput-Integr Manuf 57:182–196. https://doi.org/10.1016/j.rcim.2018.11.004

Pivano L, Nguyen DT, Bruun Ludvigsen K (2019) Digital Twin for drilling operations – towards cloud-based operational planning. Day 3 Wed, May 08, 2019. https://doi.org/10.4043/29316-ms

Podvalny SL, Vasiljev EM (2021) Digital twin for smart electricity distribution networks. IOP Conf Ser: Mater Sci Eng 1035(1):012047. https://doi.org/10.1088/1757-899x/1035/1/012047

Prasad D, Alizadeh T, Dowling R (2023) Smart city planning and the challenges of informality in India. Dialogues in human geography, 204382062311566. https://doi.org/10.1177/20438206231156655

Praschl C, Krauss O (2022) Geo-referenced occlusion models for mixed reality applications using the microsoft HoloLens. Proceedings of the 17th international joint conference on computer vision, imaging and computer graphics theory and applications. https://doi.org/10.5220/0010775200003124

Psarommatis F, May G (2022) A standardized approach for measuring the performance and flexibility of digital twins. Int J Prod Res 61(20):6923–6938. https://doi.org/10.1080/00207543.2022.2139005

Qiuchen Lu V, Parlikad AK, Woodall P, Ranasinghe GD, Heaton J (2019) Developing a dynamic digital twin at a building level: using Cambridge campus as case study. International Conference on Smart Infrastructure and Construction 2019 (ICSIC). https://doi.org/10.1680/icsic.64669.067

Quek HY, Sielker F, Akroyd J, Bhave AN, von Richthofen A, Herthogs P, van der Laag Yamu C, Wan L, Nochta T, Burgess G, Lim MQ, Mosbach S, Kraft M (2023) The conundrum in smart city governance: interoperability and compatibility in an ever-growing ecosystem of digital twins. Data Policy 5. https://doi.org/10.1017/dap.2023.1

Rabah S, Assila A, Khouri E, Maier F, Ababsa F, Bourny V, Maier P, Mérienne F (2018) Towards improving the future of manufacturing through digital twin and augmented reality technologies. Procedia Manuf 17:460–467. https://doi.org/10.1016/j.promfg.2018.10.070

Rajesh PK, Manikandan N, Ramshankar CS, Vishwanathan T, Sathishkumar C (2019) Digital Twin of an automotive brake pad for predictive maintenance. Procedia Comput Sci 165:18–24. https://doi.org/10.1016/j.procs.2020.01.061

Raqeeb A, Bonetti A, Carlsson A, Harispuru C, Pustejovsky M, Wetterstrand N (2022) Functional digital twins of relay protection and relay test equipment enabling benefits in training and remote support. 16th International Conference on Developments in Power System Protection (DPSP 2022). https://doi.org/10.1049/icp.2022.0925

Rathore MM, Shah SA, Shukla D, Bentafat E, Bakiras S (2021) The role of AI, machine learning, and big data in digital twinning: a systematic literature review, challenges, and opportunities. IEEE Access 9:32030–32052. https://doi.org/10.1109/access.2021.3060863

Rebmann A, Knoch S, Emrich A, Fettke P, Loos P (2020) A multi-sensor approach for Digital Twins of manual assembly and commissioning. Procedia Manuf 51:549–556. https://doi.org/10.1016/j.promfg.2020.10.077

Revetria R, Tonelli F, Damiani L, Demartini M, Bisio F, Peruzzo N (2019) A real-time mechanical structures monitoring system based on Digital Twin, Iot and augmented reality. 2019 Spring Simulation Conference (SpringSim). https://doi.org/10.23919/springsim.2019.8732917

Rosen R, von Wichert G, Lo G, Bettenhausen KD (2015) About the importance of autonomy and Digital Twins for the future of manufacturing. IFAC-PapersOnLine 48(3):567–572. https://doi.org/10.1016/j.ifacol.2015.06.141

Rožanec JM, Lu J, Rupnik J, Škrjanc M, Mladenić D, Fortuna B, Zheng X, Kiritsis D (2022) Actionable cognitive twins for decision making in manufacturing. Int J Prod Res 60(2):452–478. https://doi.org/10.1080/00207543.2021.2002967

Saeed Z, Mancini F, Glusac T, Izadpanahi P (2022) Future city, digital twinning and the urban realm: a systematic literature review. Buildings 12(5):685. https://doi.org/10.3390/buildings12050685

San Francisco Open Data (2024) San Francisco Open Data. https://data.sfgov.org . Accessed 10 May 2024

Shahat E, Hyun CT, Yeom C (2021) City Digital Twin potentials: a review and research agenda. Sustainability 13(6):3386. https://doi.org/10.3390/su13063386

Shangguan D, Chen L, Ding J (2020) A Digital Twin-based approach for the fault diagnosis and health monitoring of a complex satellite system. Symmetry 12(8):1307. https://doi.org/10.3390/sym12081307

Srai J, Settanni E (2019) Supply chain digital twins: opportunities and challenges beyond the hype

Ssin S, Cho H, Woo W (2021a) GeoACT: augmented control tower using virtual and real geospatial data. Interact Des Archit 48:122–142. https://doi.org/10.55612/s-5002-048-006

Ssin S, Cho H, Woo W (2021b) GeoVCM: virtual urban digital twin system augmenting virtual and real geo-spacial data. 2021 IEEE International Conference on Consumer Electronics (ICCE). https://doi.org/10.1109/icce50685.2021.9427709

Stark R, Kind S, Neumeyer S (2017) Innovations in digital modelling for next generation manufacturing system design. CIRP Ann 66(1):169–172. https://doi.org/10.1016/j.cirp.2017.04.045

Steinmetz C, Schroeder GN, Sulak A, Tuna K, Binotto A, Rettberg A, Pereira CE (2022) A methodology for creating semantic digital twin models supported by knowledge graphs. 2022 IEEE 27th International Conference on Emerging Technologies and Factory Automation (ETFA). https://doi.org/10.1109/etfa52439.2022.9921499

Svítek M, Skobelev P, Kozhevnikov S (2019) Smart city 5.0 as an urban ecosystem of smart services. Stud Comp Intell:426–438. https://doi.org/10.1007/978-3-030-27477-1_33

Tao F, Cheng J, Qi Q, Zhang M, Zhang H, Sui F (2017) Digital twin-driven product design, manufacturing and service with big data. Int J Adv Manuf Technol 94(9–12):3563–3576. https://doi.org/10.1007/s00170-017-0233-1

Ton DT, Smith MA (2012) The U.S. Department of Energy’s Microgrid Initiative. Electr J 25(8):84–94. https://doi.org/10.1016/j.tej.2012.09.013

Tonnarelli F, Mora L (2023) Smart urbanism in Africa: when theories do not fit with contextual practices. Reg Stud:1–11. https://doi.org/10.1080/00343404.2023.2235407

Trusov A, Limonova EE (2020) The analysis of projective transformation algorithms for image recognition on mobile devices. Twelfth International Conference on Machine Vision (ICMV 2019). https://doi.org/10.1117/12.2559732

Tuhaise VV, Tah JHM, Abanda FH (2023) Technologies for digital twin applications in construction. Autom Constr 152:104931. https://doi.org/10.1016/j.autcon.2023.104931

Tygesen UT, Jepsen MS, Vestermark J, Dollerup N, Pedersen A (2018) The true digital twin concept for fatigue re-assessment of marine structures. Volume 1: offshore technology. https://doi.org/10.1115/omae2018-77915

Um J, Popper J, Ruskowski M (2018) Modular augmented reality platform for smart operator in production environment. 2018 IEEE Industrial Cyber-Physical Systems (ICPS). https://doi.org/10.1109/icphys.2018.8390796

Urbina Coronado PD, Lynn R, Louhichi W, Parto M, Wescoat E, Kurfess T (2018) Part data integration in the shop floor digital twin: mobile and cloud technologies to enable a manufacturing execution system. J Manuf Syst 48:25–33. https://doi.org/10.1016/j.jmsy.2018.02.002

van der Valk H, Strobel G, Winkelmann S, Hunker J, Tomczyk M (2022) Supply chains in the era of Digital Twins – a review. Procedia Comput Sci 204:156–163. https://doi.org/10.1016/j.procs.2022.08.019

Verdouw C, Tekinerdogan B, Beulens A, Wolfert S (2021) Digital twins in smart farming. Agric Syst 189:103046. https://doi.org/10.1016/j.agsy.2020.103046

Vidal-Balea A, Blanco-Novoa O, Fraga-Lamas P, Vilar-Montesinos M, Fernández-Caramés TM (2021) Collaborative augmented digital twin: a novel open-source augmented reality solution for training and maintenance processes in the shipyard of the future. The 4th XoveTIC conference. https://doi.org/10.3390/engproc2021007010

Wan L, Nochta T, Schooling JM (2019) Developing a city-level digital twin –propositions and a case study. International Conference on Smart Infrastructure and Construction 2019 (ICSIC). https://doi.org/10.1680/icsic.64669.187

Wang P, Luo M (2021) A digital twin-based big data virtual and real fusion learning reference framework supported by industrial internet towards smart manufacturing. J Manuf Syst 58:16–32. https://doi.org/10.1016/j.jmsy.2020.11.012

Wang W, He F, Li Y, Tang S, Li X, Xia J, Lv Z (2023) Data information processing of traffic digital twins in smart cities using edge intelligent federation learning. Inf Process Manage 60(2):103171. https://doi.org/10.1016/j.ipm.2022.103171

Wang H, Meng X (2021) BIM-supported knowledge management: potentials and expectations. J Manag Eng 37(4). https://doi.org/10.1061/(asce)me.1943-5479.0000934

Waszak M, Lam AN, Hoffmann V, Elvesater B, Mogos MF, Roman D (2022) Let the asset decide: digital twins with knowledge graphs. 2022 IEEE 19th International Conference on Software Architecture Companion (ICSA-C). https://doi.org/10.1109/icsa-c54293.2022.00014

Wen T, Dobson E, Hvaara R (2020) Mesh learning: a cloud and edge–based computing network providing data–driven solutions to the oil and gas industry. Day 2 Tue, November 03, 2020. https://doi.org/10.4043/30365-ms

White G, Zink A, Codecá L, Clarke S (2021) A digital twin smart city for citizen feedback. Cities 110:103064. https://doi.org/10.1016/j.cities.2020.103064

Wiegand G, Mai C, Liu Y, Hußmann H (2018) Early take-over preparation in stereoscopic 3D. Adjunct proceedings of the 10th international conference on automotive user interfaces and interactive vehicular applications. https://doi.org/10.1145/3239092.3265957

Xia L, Zheng P, Li X, Gao RX, Wang L (2022) Toward cognitive predictive maintenance: a survey of graph-based approaches. J Manuf Syst 64:107–120. https://doi.org/10.1016/j.jmsy.2022.06.002

Xie X, Lu Q, Rodenas-Herraiz D, Parlikad AK, Schooling JM (2020) Visualised inspection system for monitoring environmental anomalies during daily operation and maintenance. Eng Constr Archit Manag 27(8):1835–1852. https://doi.org/10.1108/ecam-11-2019-0640

Xu Y, Sun Y, Liu X, Zheng Y (2019) A Digital-Twin-assisted fault diagnosis using deep transfer learning. IEEE Access 7:19990–19999. https://doi.org/10.1109/access.2018.2890566

Xu W, Cui J, Li L, Yao B, Tian S, Zhou Z (2021) Digital twin-based industrial cloud robotics: framework, control approach and implementation. J Manuf Syst 58:196–209. https://doi.org/10.1016/j.jmsy.2020.07.013

Xu H, Berres A, Yoginath SB, Sorensen H, Nugent PJ, Severino J, Tennille SA, Moore A, Jones W, Sanyal J (2023) Smart mobility in the cloud: enabling real-time situational awareness and cyber-physical control through a Digital Twin for traffic. IEEE Trans Intell Transp Syst 24(3):3145–3156. https://doi.org/10.1109/tits.2022.3226746

Yan M, Gan W, Zhou Y, Wen J, Yao W (2022) Projection method for blockchain-enabled non-iterative decentralized management in integrated natural gas-electric systems and its application in digital twin modelling. Appl Energy 311:118645. https://doi.org/10.1016/j.apenergy.2022.118645

Yin Y, Zheng P, Li C, Wang L (2023) A state-of-the-art survey on augmented reality-assisted Digital Twin for futuristic human-centric industry transformation. Robot Comput-Integr Manuf 81:102515. https://doi.org/10.1016/j.rcim.2022.102515

Yu G, Wang Y, Hu M, Shi L, Mao Z, Sugumaran V (2021) RIOMS: an intelligent system for operation and maintenance of urban roads using spatio-temporal data in smart cities. Futur Gener Comput Syst 115:583–609. https://doi.org/10.1016/j.future.2020.09.010

Yu X, Merritt J (2023) Comparison of city digital twin case studies. Digital twins for smart cities, 123–137. https://doi.org/10.1680/dtsc.66007.123

Zakzak L (2019) Citizen-centric smart city development. Proceedings of the 20th annual international conference on digital government research. https://doi.org/10.1145/3325112.3325236

Zborowski M (2018) Finding meaning, application for the much-discussed “Digital Twin.” J Petrol Technol 70(06):26–32. https://doi.org/10.2118/0618-0026-jpt

Zhang J, Mora L (2023) Nothing but symbolic: Chinese new authoritarianism, smart government, and the challenge of multi-level governance. Gov Inf Q 40(4):101880. https://doi.org/10.1016/j.giq.2023.101880

Zhang K, Cao J, Zhang Y (2022) Adaptive digital twin and multiagent deep reinforcement learning for vehicular edge computing and networks. IEEE Trans Industr Inf 18(2):1405–1413. https://doi.org/10.1109/tii.2021.3088407

Zhao X, Sun Y (2020) Augmented reality assembly guidance method based on situation awareness. In: Proceedings of the 2020 the 10th international workshop on computer science and engineering. WCSE, pp 165–173

Zheng P, Lin T-J, Chen C-H, Xu X (2018) A systematic design approach for service innovation of smart product-service systems. J Clean Prod 201:657–667. https://doi.org/10.1016/j.jclepro.2018.08.101

Zheng X, Lu J, Kiritsis D (2021) The emergence of cognitive digital twin: vision, challenges and opportunities. Int J Prod Res 60(24):7610–7632. https://doi.org/10.1080/00207543.2021.2014591

Zheng P, Li S, Fan J, Li C, Wang L (2023) A collaborative intelligence-based approach for handling human-robot collaboration uncertainties. CIRP Ann 72(1):1–4. https://doi.org/10.1016/j.cirp.2023.04.057

Židek K, Hladký V, Pitel’ J, Demčák J, Hošovský A, Lazorík P (2021) SMART production system with full digitalization for assembly and inspection in concept of industry 4.0. Future access enablers for ubiquitous and intelligent infrastructures. pp 181–192. https://doi.org/10.1007/978-3-030-78459-1_13

Zohdi TI (2020) A machine-learning framework for rapid adaptive digital-twin based fire-propagation simulation in complex environments. Comput Methods Appl Mech Eng 363:112907. https://doi.org/10.1016/j.cma.2020.112907

Article MathSciNet Google Scholar

Download references

Acknowledgements

The authors extend their appreciation to the Deputyship for Research and Innovation, Ministry of Education in Saudi Arabia, for funding this research work through project number 445-5-961.

This research was funded by the Deputyship for Research and Innovation, Ministry of Education in Saudi Arabia, project number 445-5-961.

Author information

Authors and affiliations.

College of Computer Science and Engineering, Taibah University, 46421, Yanbu, Saudi Arabia

Rasha F. El-Agamy, Hanaa A. Sayed, Arwa M. AL Akhatatneh, Mansourah Aljohani & Mostafa Elhosseini

Computer Science Department, Faculty of Science, Tanta University, Tanta, 31527, Egypt

Rasha F. El-Agamy

Department of Computer Science, Faculty of Computers and Information, Assiut University, Assiut, 71516, Egypt

Hanaa A. Sayed

Department of Computers and Control Systems Engineering, Faculty of Engineering, Mansoura University, Mansoura, 35516, Egypt

Mostafa Elhosseini

You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization, M.E.; methodology, R.E., H.A., A.A., M.A.; software, A.A. and M.A.; validation, H.A., R.E. and M.E.; formal analysis, R.E.. and A.A.; investigation, A.A. and H.A; resources, M.A.; data curation, R.E., and H.A; writing—original draft preparation, R.E., H.A., A.A., and M.A.; writing—review and editing, R.E., H.A., and M.E.; visualization, R.E., and H.A; supervision, M.E., H.A., and R.E.; project administration, M.E.; funding acquisition, M.E. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Mostafa Elhosseini .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

El-Agamy, R.F., Sayed, H.A., AL Akhatatneh, A.M. et al. Comprehensive analysis of digital twins in smart cities: a 4200-paper bibliometric study. Artif Intell Rev 57 , 154 (2024). https://doi.org/10.1007/s10462-024-10781-8

Download citation

Accepted : 29 April 2024

Published : 27 May 2024

DOI : https://doi.org/10.1007/s10462-024-10781-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Digital twin
Smart city development
Bibliometric study
IoT integration
Find a journal
Publish with us
Track your research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

Advanced Search
Journal List
PMC10486481

Cancer Vaccine Therapeutics: Limitations and Effectiveness—A Literature Review

Mariusz kaczmarek.

1 Department of Medical Biotechnology, Poznan University of Medical Sciences, 61-866 Poznań, Poland

2 Department of Cancer Diagnostics and Immunology, Greater Poland Cancer Center, 61-866 Poznań, Poland

Justyna Poznańska

3 Scientific Society of Cancer Immunology, Poznań University of Medical Sciences, 61-866 Poznań, Poland; moc.oohay@iksnanzopanytsuj (J.P.);

Filip Fechner

Natasza michalska, sara paszkowska, adrianna napierała, andrzej mackiewicz.

In recent years, there has been a surge of interest in tumor microenvironment-associated cancer vaccine therapies. These innovative treatments aim to activate and enhance the body’s natural immune response against cancer cells by utilizing specific antigens present in the tumor microenvironment. The goal is to achieve a complete clinical response, where all measurable cancer cells are either eliminated or greatly reduced in size. With their potential to revolutionize cancer treatment, these therapies represent a promising avenue for researchers and clinicians alike. Despite over 100 years of research, the success of therapeutic cancer vaccines has been variable, particularly in advanced cancer patients, with various limitations, including the heterogeneity of the tumor microenvironment, the presence of immunosuppressive cells, and the potential for tumor escape mechanisms. Additionally, the effectiveness of these therapies may be limited by the variability of the patient’s immune system response and the difficulty in identifying appropriate antigens for each patient. Despite these challenges, tumor microenvironment-targeted vaccine cancer therapies have shown promising results in preclinical and clinical studies and have the potential to become a valuable addition to current cancer treatment and “curative” options. While chemotherapeutic and monoclonal antibody treatments remain popular, ongoing research is needed to optimize the design and delivery of these therapies and to identify biomarkers that can predict response and guide patient selection. This comprehensive review explores the mechanisms of cancer vaccines, various delivery methods, and the role of adjuvants in improving treatment outcomes. It also discusses the historical background of cancer vaccine research and examines the current state of major cancer vaccination immunotherapies. Furthermore, the limitations and effectiveness of each vaccine type are analyzed, providing insights into the future of cancer vaccine development.

1. Introduction

Tumor microenvironments (TMEs) involve a mixed composition of transforming immune cells, blood vessels, stromal cells, and extracellular matrix, creating tumors that are exclusive to their location and diverse in composition among individual patients [ 1 ]. The metastatic progression, along with the distinct and peculiar composition of the TME, plays the most significant role in the patient’s response to treatment [ 2 ] All components of the transforming TME determine the changes and behavior of surrounding macromolecules and tissue development, creating complicated variations in immune response and tumor behavior, like suppression or stimulation of tumor growth [ 1 , 3 ].

The first standardized cancer immunotherapy utilized for the treatment of malignant tumors was developed by William B. Coley in 1891. For 40 years following this discovery, the ‘Coley Toxin’, developed from Streptococcal bacteria, stimulated immune system responses and was used to treat patients with bone and soft tissue sarcomas, shrinking the tumor significantly. Since the end of Coley’s research in 1933, the study of TMEs in correlation to cancer vaccine development skyrocketed, leading to cures and positive treatment outcomes of liver cancer related to hepatitis B, cervical cancer associated with human papillomavirus, colon cancer, melanomas, and bladder cancer, to name a few [ 4 ].

Cancer vaccination, also referred to as cancer immunization or cancer immunotherapy, is a therapeutic approach aimed at activating the immune system to recognize and combat cancer cells. Its primary objective is to prevent tumor growth, recurrence, or metastasis while enhancing the immune system’s capacity to identify and eliminate cancer cells. The mechanisms of cancer vaccines involve eliciting an immune response targeting specific tumor-associated antigens (TAAs), which are proteins expressed by cancer cells. This immune response involves the activation of T cells, B cells, and other immune cells, leading to the destruction of cancer cells. Cancer vaccines can serve as preventive measures in high-risk populations, known as prophylactic vaccines, and as treatment options for individuals already diagnosed with cancer, referred to as therapeutic vaccines. By harnessing the power of the immune system, cancer vaccination holds promise in providing effective strategies for cancer prevention and treatment.

Additionally, adjuvants are essential components of cancer vaccines, as they enhance immune responses by activating innate immune pathways. Adjuvants, such as Toll-like receptor (TLR) agonists, cytokines, and immune checkpoint inhibitors, have been utilized to improve vaccine efficacy. TLR agonists, such as CpG oligodeoxynucleotides (CpG-ODNs), stimulate antigen-presenting cells (APCs) and promote antigen presentation, while immune checkpoint inhibitors block inhibitory signaling pathways, allowing sustained immune activation. These adjuvants have shown promising results in preclinical and clinical studies, contributing to the development of novel cancer vaccines. Despite the challenges of cancer vaccine development regarding efficacy, several therapeutic vaccination strategies are under development and are being evaluated in preclinical and clinical trials [ 5 ].

Despite the progress made in the field of cancer vaccines, it is important to address the limitations and potential side effects of cancer vaccines. Tumor heterogeneity, immunosuppressive TMEs, and immune tolerance mechanisms pose significant challenges for vaccine efficacy. The identification of suitable TAAs and the selection of optimal adjuvants remain critical for successful vaccine development, and the overall effectiveness of cancer vaccines may vary among different cancer types and individual patients, necessitating personalized approaches. Common side effects include injection site pain, headache, influenza-like illness, fever, nausea, diarrhea, rashes, erythema, pruritus, myalgia, and dyspnea. Serious adverse events are less common but may involve immune system disorders, psychiatric disorders, and pulmonary embolism. While rare, varying levels of toxicity have been observed in some cases. Vaccines and their adjuvants can also lead to additional complications, such as hyponatremia, liver enzyme elevation, anemia, colitis, and increased creatinine levels. Furthermore, vaccine-induced immune responses, particularly T cell responses, have the potential to cause tumor pseudo-progression, as highlighted by M. Platten et al. (2021) [ 6 ]. It is essential to carefully monitor and manage these immune-related reactions to ensure the safety and efficacy of cancer vaccines.

In this comprehensive review article, we provide a concise overview mechanism underlying cancer vaccines, the various delivery methods employed, and the role adjuvants have in enhancing treatment outcomes. We also delve into the historical background of cancer vaccine research and explore the current state of four major cancer vaccination immunotherapies, including traditional cell-based vaccines, second-generation microbial vector vaccines, peptide vaccines, and third-generation genetic vaccines. Furthermore, we discuss the limitations and effectiveness of each vaccine type to provide a well-rounded perspective on the future of cancer vaccine development.

2. Cellular Composition of Tumor Microenvironment

Cancer vaccine therapeutics hold great promise in stimulating the immune system to target cancer cells. However, their effectiveness is influenced by the complex cellular composition of the TME. The TME encompasses immune cells such as T cells, B cells, and natural killer cells (NK), each playing distinct roles in the anti-tumor immune response. T cells exhibit phenotypic plasticity, allowing them to differentiate into effector T cells or immunosuppressive regulatory T cells (Tregs) [ 3 , 7 ]. Immune checkpoint receptors expressed by T cells, such as PD-1 and CTLA-4, modulate T cell activation and function. Blocking these immune checkpoint pathways has shown success in cancer immunotherapy.

B cells, on the other hand, possess multifaceted functions within the TME. They can produce antibodies, regulate antigen processing and presentation, and exhibit pro- and anti-tumorigenic properties [ 8 ]. Regulatory B cells (Bregs) secrete immunosuppressive proteins, like IL-10 and IL-35, dampening the anti-tumor immune response [ 1 , 9 ]. The dynamic interactions between these immune cells and the tumor cells within the TME shape the efficacy of cancer vaccines. Furthermore, the TME comprises other components, such as dendritic cells (DCs), neutrophils, tumor-associated macrophages (TAMs), and cancer-associated fibroblasts (CAFs), each with distinct contributions to tumor progression and immune modulation.

While cancer vaccines hold the potential to stimulate the immune system to target cancer cells, there are limitations to their effectiveness. The TME’s immunosuppressive nature, genetic instability, and heterogeneity pose challenges to cancer vaccine therapy. Targeting specific components of the TME, such as immune checkpoints or fibroblast activation, may help overcome these limitations and enhance the effectiveness of cancer vaccines.

3. Mechanisms of Cancer Vaccines

Cancer vaccines employ various mechanisms to stimulate the immune system and generate an effective anti-tumor response. One common approach involves the use of DCs, which are potent APCs. In one method, DCs are collected from the patient’s blood or generated in the laboratory. They are then matured and activated using immune-stimulating molecules or tumor antigens. After loading the DCs with tumor-specific antigens (TSAs) derived from tumor cells or genetic material, the loaded DCs are administered back to the patient. These DCs migrate to lymphoid organs, where they interact with immune cells, such as T cells, B cells, and NK) cells. The DCs present the tumor antigens to CD4+ helper T cells and CD8+ cytotoxic T lymphocytes (CTLs), leading to their activation. The activated T cells provide help signals to other immune cells, enhancing the immune response against tumor cells. CTLs specifically recognize and target cancer cells expressing the tumor antigens, resulting in their elimination. Additionally, this vaccine aims to induce a memory response, allowing for a more effective immune response upon subsequent encounters with tumor cells.

Another strategy involves the use of whole-cell preparations derived from cancer cells. Cancer cells are collected from the patient’s tumor or established cancer cell lines. These cells are inactivated or genetically modified to reduce their ability to grow and cause disease. When administered back to the patient, the whole cells are recognized by various immune cells, including DCs, macrophages, and NK cells, triggering an immediate non-specific inflammatory response. The activated immune cells, in turn, process the tumor antigens, present them to T cells, and initiate an immune response. CD4+ helper T cells provide help signals to other immune cells, while CD8+ CTLs recognize and eliminate tumor cells expressing the presented antigens. Whole-cell cancer vaccines also aim to induce a memory response for enhanced immune protection against tumor recurrence.

Induced pluripotent stem cell (iPSC)-based cancer vaccines represent a promising approach. iPSCs are generated from somatic cells and differentiated into TME-specific cells, such as TAFs, endothelial cells, or immune cells. These iPSC-derived cells express antigens characteristic of the TME, including TSAs or molecules involved in immunosuppression. Upon administration to the patient, these cells are recognized by immune cells, leading to the activation of a robust immune response. APCs, primarily DCs, take up iPSC-derived antigens, process them, and present them to T cells. The activated T cells, particularly CD8+ CTLs, recognize and target TME components expressing TSAs. The expansion of effector cells contributes to the elimination of tumor cells, and the induction of a memory response enables a more rapid and effective immune response upon subsequent encounters with tumor cells.

In situ cancer vaccines are administered directly into the tumor site or a nearby lymph node. This approach involves the activation of APCs, such as DCs, macrophages, neutrophils, and NK cells, within the TME. The vaccine induces an inflammatory response, cytokine production, and immune cell recruitment. APC take up tumor antigens released during vaccine administration, process them, and present them to T cells. This triggers the activation of CD8+ CTLs and CD4+ helper T cells, which work synergistically to eliminate tumor cells. The activated immune cells produce effector molecules and mediate the destruction of tumor cells within the TME. In situ cancer vaccines also aim to generate a memory response for heightened protection against tumor recurrence.

Viral-based cancer vaccines utilize modified viruses to directly activate the immune response. The modified virus interacts with immune cells, including DCs, macrophages, and NK cells, triggering an inflammatory response and the release of pro-inflammatory cytokines and chemokines. The virus particles are phagocytosed by immune cells, and the TAAs expressed by the virus or delivered to infected cells are processed and presented to T cells. CD8+ CTLs recognize the presented TAAs, leading to their activation and expansion. CD4+ helper T cells provide help signals to other immune cells, and antibodies produced against the TAAs can directly bind to tumor cells, facilitating their destruction. Viral-based vaccines aim to induce a memory response, enabling a more rapid and robust immune response upon subsequent encounters with tumor cells expressing the same TAAs.

Similarly, bacteria-based cancer vaccines utilize modified bacteria to activate the immune response. The modified bacteria interact with immune cells, triggering an inflammatory response and the production of pro-inflammatory cytokines, chemokines, and other signaling molecules. APCs, particularly DCs, phagocytose the bacteria and process them, leading to the presentation of TAAs. CD8+ CTLs recognize the presented TAAs and become activated, while CD4+ helper T cells provide help signals to other immune cells. Antibodies produced against the TAAs can directly bind to tumor cells, facilitating their destruction. Bacteria-based vaccines aim to induce a memory response, leading to a more rapid and robust immune response upon subsequent encounters with tumor cells expressing the same TAAs.

Peptide cancer vaccines involve the administration of specific peptides derived from TAAs. These peptides are taken up by APCs primarily DCs, which process them and present them on their surface using major histocompatibility complex (MHC) molecules. CD8+ CTLs recognize the presented peptides on MHC class I molecules, leading to their activation and expansion. CD4+ helper T cells recognize the peptides presented on MHC class II molecules and provide help signals to other immune cells. B cells can be activated by the peptides presented by DCs, leading to the production of antibodies specific to the TAAs. The antibodies can directly bind to tumor cells, facilitating their destruction, and a memory response is induced for enhanced immune protection.

DNA and RNA cancer vaccines involve the administration of DNA or RNA molecules encoding TAAs. The administered DNA or RNA is taken up by cells, such as muscle cells or DCs, and the TAAs are produced within these cells. APCs cells, particularly DCs, take up the TAAs and present them on their surface using MHC molecules. CD8+ CTLs recognize the presented TAAs on MHC class I molecules, leading to their activation and expansion. CD4+ helper T cells recognize the TAAs presented on MHC class II molecules and provide help signals to other immune cells. B cells can be activated by the TAAs presented by DCs, leading to the production of antibodies specific to the TAAs. The antibodies can directly bind to tumor cells, facilitating their destruction, and a memory response is induced for enhanced immune protection.

Exosome-based cancer vaccines utilize exosomes loaded with TAAs or nucleic acids encoding TAAs. These exosomes are taken up by APCs cells, primarily DCs, which become activated and enhance their antigen-presenting capabilities. The TAAs delivered by exosomes are processed and presented on the surface of DCs using molecules. CD8+ CTLs recognize the presented TAAs on MHC class I molecules, leading to their activation and expansion. CD4+ helper T cells recognize the TAAs presented on MHC class II molecules and provide help signals to other immune cells. B cells can be activated by the TAAs presented by DCs, leading to the production of antibodies specific to the TAAs. The antibodies can directly bind to tumor cells, facilitating their destruction, and a memory response is induced for heightened immune protection.

4. Cancer Vaccine Progress and Development

Tumor cells are known to exhibit genetic instability, resulting in numerous somatic mutations, such as deletions, insertions, point mutations, and translocations. This genetic complexity can lead to the production of abnormal proteins, making them attractive targets for immunotherapy. By focusing on patient-specific proteins, immunotherapy offers a potential solution to address challenges associated with self-tolerance and treatment efficacy [ 10 ].

The average age of cancer diagnosis is around 66 years, and as patients age, the TME becomes increasingly complex. Consequently, it is imperative to explore alternative targets for cancer vaccines that specifically address age-related variations in the TME, thereby optimizing treatment outcomes [ 11 ]. In a study by Grizzle et al. (2007), it was observed that older mice had an increased presence of myeloid-derived suppressor cells in the TME compared to younger mice, resulting in impaired T cell responses [ 12 ]. This highlights the decline in efficacy and dysregulation of the immune system, necessitating the use of adjuvants and strategies to elicit robust cellular responses in cancer vaccines [ 13 ].

Compared to conventional treatment methods, such as chemotherapy and radiotherapy, vaccine immunotherapy has emerged as a powerful tool and an area of intense research for inducing or reactivating anti-tumor immune responses [ 14 ]. Recent advancements have revealed the potential of vaccine immunotherapy to precisely target specific tumors within specific regions of the body, enabling tailored interventions at the cellular and tissue level. These breakthroughs represent considerable progress in cancer treatment strategies, offering promising outcomes on a global scale.

The selection of a cancer vaccine is a critical decision that can impact the speed, intensity, and duration of the immune response. The clinical effectiveness of a given treatment is determined by its ability to prolong patient survival, improve clinical response, achieve partial response, or enhance disease stability. Several factors, including cancer stage, patient age, the immunosuppressive nature of the TME, and the choice of antigens and adjuvants, influence treatment outcomes [ 13 , 15 ]. The variability of treatment responses among different population cohorts can be substantial, with exceptional success observed in some cases but limited efficacy in specific subgroups [ 16 ]. Despite the promising potential of vaccine immunotherapy, achieving consistently positive clinical outcomes remains a significant challenge, which has raised concerns about its widespread acceptance [ 16 ].

Cancer vaccines are meticulously developed and tailored to target specific antigens, with the aim of modulating the immunosuppressive TME and eliciting effective immune responses. A key focus of research lies in combining cancer vaccines with various therapies, such as radiotherapy, oncolytic viruses, cytokine-based approaches, and physical therapy, to enhance their efficacy and impact [ 17 ]. Vaccines can be designed for TAAs, TSAs, cancer germline antigens (CGAs), and virus-associated antigens, each serving unique roles in cancer immunotherapy [ 18 ].

TAAs are expressed in minimal amounts in healthy cells but are overexpressed in tumor cells, often because of posttranslational modifications. Despite extensive research on TAAs, such as prostatic acid phosphatase for prostate cancer and carcinoembryonic antigen for gastrointestinal cancer, their limited tumor-specificity has hindered their success as therapy targets. Notably, chimeric TAA receptor T cell therapy targeting CD19 has shown promise in patients with acute lymphoblastic leukemia [ 19 ]. However, it is important to consider that targeting TAAs can lead to adverse effects, including colitis, hepatitis, and in some cases, even death [ 20 ].

In contrast, TSAs are absent in healthy cells and arise from nonsynonymous mutations. Since TSAs are typically absent from normal tissue, targeting them in therapy offers the potential to minimize the risk of autoimmune responses, making them a promising target for cancer treatment [ 18 , 21 ]. Unlike TAAs, TSAs are not subjected to central tolerance mechanisms that eliminate T and B cells reactive to self-antigens.

CGAs, on the other hand, are predominantly expressed in reproductive tissues, such as trophoblast and fetal ovaries. They are selectively expressed in specific tumor types due to epigenetic dysregulation. Virus-associated antigens originate from oncogenic viral proteins that integrate into the host genome, promoting tumorigenesis [ 18 ]. All these mentioned antigens can be presented by the major histocompatibility complex and recognized by T cells, playing crucial roles in immunotherapeutic approaches against cancer.

The main cancer vaccine types discussed in this review are based on composition and include (1) tumor or immune cell-based vaccines, (2) peptide-based vaccines, (3) microbial vector-based vaccines, (4) nucleic acid-based vaccines, (5) exosome-based vaccines, (6) induced pluripotent stem cell-based vaccines, (7) in situ vaccines, and finally (8) a discussion on combination vaccine methods. Table 1 provides a comprehensive overview of the mechanisms underlying each vaccine, detailing the specific stimuli employed, the elicited T cell responses, and the accessory cells involved in the process ( Table 1 ).

Stimulus, T cell Response and Accessory Cells utilized in vaccine types discussed in this article.

4.1. Cell-Based Vaccines

Cell-based vaccines use immune cells, such as DCs, to stimulate an immune response against cancer cells. The mechanism of action of cell-based vaccines involves the collection of immune cells from a patient’s blood or tumor and the activation and expansion of these cells in a laboratory setting. The activated immune cells are then reintroduced into the patient’s body, where they target and attack cancer cells.

Cell-based vaccines offer two approaches: allogeneic and autologous tumor cells. Allogeneic vaccines, while lacking personalization, present a time-saving advantage. In contrast, autologous vaccines utilize the patient’s own tumor cells, ensuring antigen compatibility but at the expense of increased costs and time requirements. The selection between these approaches depends on several factors, including the patient’s specific needs, cancer stage, and resource availability.

One significant drawback of cell-based vaccines is the potential for human leukocyte antigen (HLA) mismatch. This mismatch can shift the focus onto HLA molecules rather than the immune system itself, affecting the effectiveness of the vaccine [ 31 ]. Cell-based vaccines can be further categorized into two types: DC vaccines and whole-cell vaccines.

Refining the balance between personalization and practicality is a crucial consideration in the development of cell-based cancer vaccines, as it ensures the optimal utilization of resources while maximizing the potential benefits for patients.

4.1.1. Dendritic Cell Vaccines

DC vaccination, a pioneering approach explored in clinical trials during the 1990s, seeks to trigger an effective anti-tumor immune response against tumor antigens through multiple mechanisms. The fundamental principle behind DC vaccines involves introducing specific tumor antigens to the immune system, thereby activating immune cells, especially T cells. This activation empowers T cells to recognize and eliminate cancerous cells, fostering a targeted attack against the disease. Notably, DC vaccines possess the ability to induce the production of cytokines, which play a crucial role in modifying the TME. These cytokines can bolster the immune response against cancer cells and promote the growth of blood vessels, facilitating the improved delivery of immune cells and anticancer drugs to the TME. This multifaceted approach holds the potential to enhance the overall efficacy of cancer immunotherapy [ 32 ]. Despite ongoing clinical trials investigating various protocols and treatment regimens, DC vaccines have yet to realize their full potential in clinical practice [ 33 ]. Continued research and refinement of DC vaccination strategies aim to overcome existing limitations and unleash the full therapeutic power of this promising approach in the fight against cancer.

Using a cytokine maturation cocktail, DC vaccines are reduced by isolating autologous monocytes from a patient’s peripheral blood and are then exposed to cytokines initiating the differentiation of immature DCs [ 34 ]. This cytokine mixture prepares the DCs for specific functions like lymph node homing. They are then loaded with whole tumor lysate, DC-tumor cell fusion, virus, specific shared TAAs, like peptides and nucleic acid, or unique neoantigens from the tumor cells [ 22 ]. It is introduced to the patient’s immune system, causing the production and presentation of tumor antigens to CD8+ and CD4+ T cells, engaging both the innate and adaptive immunities and leading to the development of immunological memory in the case of tumor relapse [ 23 ]. Some studies reported that antigen-loaded DC vaccines induced stronger immune responses than vaccines composed of antigens and adjuvants alone [ 35 ].

Patients with late-stage disease have dominant immunosuppressive mechanisms; they can have weakened responses to DC vaccines and T cell activation leading to decreased efficacy in patients with metastasis [ 22 ]. While this concept is not entirely understood, given that most vaccines can induce an immune response against a specific antigen, the reason could pertain to the immunosuppressive TME where tumor-specific T cells, which would otherwise be inactivated to continue to expand and carry out effective functions, would then transfer the TME from suppressive to inflammatory. Duraiswamy et al. (2013) confirmed in preclinical studies that combination therapy of the tumor cell vaccine along with a blockade of programmed cell death protein 1 (PD-1) and/or cytotoxic T-lymphocyte-associated antigen 4 (CTLA-4) could improve tumor control [ 36 ].

The limited efficacy observed in DC vaccines can be attributed to suboptimal protocols that fail to generate an optimal T cell response. One factor contributing to this limitation is the use of granulocyte-macrophage colony stimulating factor (GM-CSF) for maturing peripheral blood mononuclear cells, which results in the production of monocyte-derived DCs that have a limited ability to migrate to lymph nodes [ 37 , 38 ]. The migration of DCs to lymph nodes is crucial for T cell-antigen interaction, and the use of monocyte-derived DC vaccines hampers this essential step [ 39 , 40 , 41 ].

Furthermore, it has been highlighted by Roy et al. (2020) that the production of DC vaccines can be costly and complex, further posing challenges to their widespread adoption [ 22 ]. These factors have contributed to the need for refinement in the manufacturing and administration of DC vaccines to enhance their effectiveness.

4.1.2. Whole-Cell Vaccines

Whole-cell cancer vaccines have a rich history of development dating back to the 1950s. Among these, Bacillus of Calmette-Guérin (BCG) stands as a significant milestone. In 1990, BCG became the first approved whole-cell vaccine for cancer therapy, specifically for the treatment of bladder cancer. TICE ® BCG, an attenuated, live culture preparation of the BCG strain of Mycobacterium Bovis, gained United States Food and Drug Administration approval in 1998. It was intended for intravesical use to combat recurrent tumors in patients with carcinoma in situ of the urinary bladder and prevent the recurrence of Stage TaT1 papillary tumors with an elevated risk of relapse. This pioneering approval marked a significant advancement in whole-cell cancer vaccine development [ 42 ].

Whole-cell cancer vaccines are designed to trigger an immune response against multiple antigens expressed by cancer cells. Unlike targeted therapies that focus on specific antigens, these vaccines expose the immune system to the entirety of the cancer cell. By targeting the entire tumor cell, these vaccines have the potential to activate a diverse array of immune cells, including T cells, B cells, and NK cells. This comprehensive immune activation holds great promise for bolstering the immune system’s capacity to effectively target and eliminate the complex TME.

Autologous whole-cell vaccines are produced from tumor cells isolated directly from the patient, which are collected with the abundance of TAAs in the TME, making this the ideal strategy for creating a natural mode of immune response by the adaptive immune system. However, multiple inhibitory signals on T cells and the APCs responsible for activating the T cell response prevent effective immune activation and recognition of growing tumors in patients [ 31 ]. GVAX is a cancer vaccine genetically modified to secrete the immune stimulatory cytokine and GM-CSF preventing further cell division, effective immune activation, and recognition of growing tumors in patients [ 31 , 43 ]. One of the main concerns discussed is the prohibitive cost and complexity of their production [ 44 ].

In a notable study by Xia et al. (2016) conducted at the Center for Cancer Research, potential biomarkers were identified to distinguish productive and unproductive immune responses in the context of whole-cell cancer vaccines [ 45 ]. Additionally, the research explored the utilization of fetal bovine growth supplements in cell cultures for investigations related to whole-cell vaccine research and manufacturing. The findings of the study were encouraging, indicating that whole-cell cancer vaccines have the capacity to induce long-lasting remissions and hold promise for advancement into late-stage clinical trials. Moreover, the study highlighted a potential strategy for enhancing vaccine efficacy by removing or reducing the presence of fetal bovine serum. These insights shed light on the potential of whole-cell cancer vaccines and provide valuable considerations for future research and development in the field [ 45 ].

The utilization of whole tumor cells as a vaccine offers a unique advantage compared to targeting a single protein or peptide tumor antigen. This approach allows for the presentation of all potential antigens expressed by cancer cells, removing the need to identify the optimal antigen for a specific cancer type. By exposing the immune system to the entirety of the tumor cell, whole-cell vaccines can elicit a broader immune response that involves the activation of T cells, B cells, and NK cells. Consequently, this approach holds great promise for the development of more efficient and adaptable cancer vaccines [ 31 ].

Despite the increasing incidence and mortality rates associated with melanoma, conventional treatments, such as chemotherapy and radiotherapy, have shown limited success in improving the overall survival of high-risk melanoma patients. However, the development of the allogeneic whole-cell melanoma vaccine, known as AGI-101H, has offered promising outcomes. Originally composed of autologous melanoma cells combined with two modified allogeneic cell lines expressing interleukin 6 (IL-6) and its soluble gp80 receptor (sIL-6R), early clinical studies in human melanoma patients were initiated in 1995, marking one of the pioneering gene therapy clinical trials globally. Notably, complete and partial clinical responses, as well as long-term survival, were observed. Subsequently, due to challenges in obtaining sufficient autologous melanoma cells, the vaccine composition was modified to include allogeneic melanoma cell lines. The current iteration of AGI-101H incorporates two melanoma cell lines retrovirally transduced with a designer cytokine gene known as Hyper-IL6 or H6, which serves as a molecular adjuvant.

Moreover, chronic exposure of vaccine cells to H6 in an autocrine manner activates the Janus Kinase 1-Signal Transducer and Activator of Transcription 3-Octamer-Binding Transcription Factor 4 (JAK1-STAT3-OCT4) pathway, resulting in a shift towards a melanoma stem-cell-like phenotype or induced stem cell phenotype. Immunization of advanced melanoma patients with AGI-101H has demonstrated a significant increase in overall survival compared to traditional chemo- and radiotherapeutic approaches [ 46 ]. Similar promising results have been observed in renal cell carcinoma and prostate cancer models, where modified TME cells with the H6 adjuvant led to improved clinical outcomes and enhanced anti-tumor immune responses [ 47 , 48 ].

A study conducted by Nawrocki et al. (2001) revealed significant humoral responses, with approximately 50% of patients showing immunoglobulin G (IgG) responses to allogenic melanoma cells and 40% of patients responding to autologous cells. Intriguingly, since 1997 Nawrocki has achieved success in treating stage IV melanoma patients using genetically modified cellular vaccines (GMTV). Out of 16 patients, a clinical response was observed following GMTV immunization, and in 4 patients, complete regression of metastases was observed [ 49 ]. These findings demonstrate the potential of modified melanoma vaccines in eliciting immune responses and achieving positive clinical outcomes in melanoma patients. Phase II trials were conducted by Mackiewicz et al. (2012) to determine the efficacy and toxicity of adjuvant treatment using the Hyper-IL-6 gene-modified whole-cell allogeneic melanoma vaccine in patients with stage 3 and 4 resected diseases [ 50 ]. While there was minor toxicity observed related to local vaccination reaction, redness, edema, and itching at the injection site, there were no grade 3 or grade 4 toxicities observed. In fact, there was a significant increase in disease-free survival and overall survival of patients with continuous vaccination methods [ 50 ].

Przybyla et al. (2021) conducted a study highlighting the spontaneous development of CD8+ T cell responses to melanoma-associated antigens in healthy individuals [ 51 ]. These antigens, including Tyrosinase, MAGE-A3, Melan/Mart-1, gp100, and NY-ESO-1, are regularly expressed by normal melanocytes. The research findings suggest that healthy individuals possess natural autoimmunity directed against melanocytes, which may provide protection against the progression to malignant melanoma. This phenomenon was also observed in studies involving breast cancer patients, where healthy women exhibited increased levels of spontaneous T cell autoreactivity to HER-2, while women with breast cancer were found to lack this cellular response [ 52 , 53 ]. These findings shed light on the complex interplay between the immune system and cancer development, suggesting the potential role of pre-existing immune responses in preventing malignant transformations.

Even more recently, Kwiatkowska-Borowczyk et al., (2018) revealed that immunization of patients resulted in the generation of cytotoxic CD8+ T cells specific for Aldehyde Dehydrogenase 1 Family Member A1 (ALDH1A1) along with the production of specific anti-ALDH1 antibodies [ 54 ]. Kazimierczak et al., (2020) discovered a new biological marker for monitoring melanoma immunotherapy, where there is a positive correlation between the production of antibodies to BNIP3L/NIX and the clinical outcome of melanoma patients treated with the AGI-101H vaccine [ 55 ]. Increased expression of BNIP3L is also positively correlated with patient overall survival (OS) in melanoma. BNIP3L is induced by tumor suppressor p53 and, in response to hypoxic TME conditions, plays a crucial role in the clearance of damaged mitochondria with mitochondrial autophagy [ 55 ].

Further research conducted in 2015 by Mackiewicz et al. discussed the efficacy of AGI-101H as a candidate for the combination treatment of non-resected advanced melanoma. Immune checkpoint inhibitors or tumor hypoxia normalization agents are the perfect combination approaches to be added to AGI-101H vaccine therapy. Furthermore, patients that were treated in an adjuvant setting and received reinduction with AGI-101H showed a 70% reduction in risk of death compared with patients not reinduced [ 48 ]. Throughout the entirety of AGI-101H trials conducted in both adjuvant and non-resected melanoma contexts, involving a participant pool exceeding 400 patients who collectively received 40,000 vaccine doses, solely anticipated grade 1 and 2 toxicities were observed. In contrast to alternative treatment strategies like IL-2, INF-alpha, ipilimumab, nivolumab, or anti-PD-1L, which have been associated with severe grades 3 and 4 toxicities carrying the potential for life-threatening consequences, AGI-101H offers a markedly favorable safety profile [ 48 , 50 , 56 ].

Recent experimental studies in mice models demonstrated the increased effectiveness of the melanoma vaccine modified with H6 and admixed with murine melanoma stem-like cells (MSC) or induced mice pluripotent stem cells (miPSCs). The above vaccine significantly inhibited tumor growth and extended disease free-survival and OS compared to the vaccine alone in animal models. The MSC and miPSC additions increased the stemness of the basic vaccine, thus decreasing the local TME immunosuppression.

Despite noteworthy progress in cancer immunotherapy, our comprehension of what constitutes a favorable immune response beyond the AGI-101H melanoma cancer vaccine remains incomplete. It is imperative to investigate the factors contributing to a positive response and the underlying reasons for the substantial variations in the response among different patient populations. Moreover, continued research is necessary to explore the interaction between whole-cell vaccination and the TME. Such research has the potential to provide a more comprehensive understanding of immune responses and further enhance the effectiveness of cancer immunotherapy.

4.2. Induced Pluripotent Stem Cell-Based Vaccines

In 2006, Shinya Yamanaka made a groundbreaking discovery by developing iPSCs, building upon earlier work by Sir John Gurdon in 1962. Gurdon successfully reprogrammed the somatic cells of tadpoles into pluripotent embryonic cells, paving the way for subsequent cloning experiments [ 24 ].

iPSCs hold immense potential as a valuable tool in personalized medicine due to their ability to be derived from a patient’s own cells, ensuring compatibility and reducing the risk of immune rejection. In the field of cancer vaccine development, iPSCs play a crucial role. They can be generated by reprogramming adult somatic cells and exhibit gene expression profiles similar to embryonic stem cells. These iPSCs possess unique characteristics, combining the immunogenic properties of TME-extracted cells with the ability to proliferate in culture. Additionally, iPSCs possess the remarkable capability to differentiate into various germ cell lines, including endoderm, mesoderm, and ectoderm, offering a diverse range of cell types that can be utilized for innovative immunotherapeutic strategies [ 24 ].

iPSC vaccines offer a promising approach to precisely target the TME by engineering induced pluripotent stem cells. Through manipulation, iPSCs can be directed to generate TME-specific antigens found on various cell types within the TME, such as TAFs (transcription factors associated with the TATA-binding protein), endothelial cells, and immune cells. By presenting these TME-specific antigens, iPSC vaccines elicit an immune response that specifically targets the components of the TME. Moreover, iPSCs can be engineered to express immunostimulatory molecules or cytokines, enhancing the activation of immune cells and fostering an improved anti-tumor immune response within the TME. By expressing TSAs, iPSC vaccines activate T cells, including cytotoxic CD8+ T cells, which recognize and eliminate tumor cells within the TME. Consequently, iPSC vaccines provide a targeted and potent strategy for combating cancer within the complex TME.

Studies have demonstrated the effectiveness of iPSC vaccines in preventing tumor growth in genetically comparable cancer models. Notably, iPSC vaccines have shown indications of humoral and cellular immune responses when used prophylactically. They can reduce metastatic tumor burden, modify immune responses by influencing the balance of Th17 cells, and increase the infiltration of Gr1+CD11b+ myeloid cells into tumors. Remarkably, studies by Kooreman et al. (2018) have shown that unvaccinated recipients receiving T cells from vaccinated tumor-bearing mice exhibit an antigen-specific anti-tumor response [ 57 ].

More recently, Kishi et al. (2021) demonstrated the potential of iPSCs in combination with a histone deacetylase inhibitor (HDACi) called Valproic Acid (VPA) [ 58 ]. This combination improved the survival rate, reduced tumor volume, and transformed an immunosuppressive TME into an immune-activated TME in mouse models of aggressive triple-negative breast cancer cell lines. This research study’s main objective was to use iPSCs to target cancer stem cells otherwise resistant to conventional therapies. However, it is worth noting that iPSC production can be time-consuming, taking several months, and there is a risk of teratoma formation once injected, as iPSCs are immature progenitor cells [ 22 , 57 ].

4.3. In Situ Cancer Vaccines

In the early 2000s, Dr. Cornelis Melief and his colleagues at Leiden University Medical Center in the Netherlands pioneered a groundbreaking achievement: the development of the first in situ cancer vaccine. Named TriMix, this vaccine was specifically designed to activate and mobilize DCs within the TME. By doing so, it effectively stimulated an immune response against cancer cells. TriMix works by presenting TSAs to the immune system, triggering a targeted immune response against the cancer cells dwelling in the TME. This pioneering breakthrough laid the foundation for subsequent advancements in the field of in situ cancer vaccines. Since then, a wide range of in situ cancer vaccines have been developed and rigorously tested in both preclinical and clinical studies, further expanding our understanding of their potential in the fight against cancer.

Subsequent studies have revealed the remarkable potential of in situ cancer vaccines to address the immunosuppressive TME. Research conducted by Locy et al., in 2018 demonstrated that these vaccines possess the unique ability to convert various aspects of the TME, enabling effector T cells to infiltrate the tumor site and effectively eliminate cancer cells [ 19 ]. A key attribute of these vaccines lies in their capacity to induce distinct patterns of cytokine secretion. As discussed by Lurje et al., in 2021, this capability leads to diverse profiles of tumor-infiltrating lymphocytes (TIL), including variations in their type, quantity, and activation status [ 59 ]. Such heterogeneity culminates in immunogenic cell death (ICD), a process crucial for mounting a potent immune response against cancer. These significant findings highlight the transformative potential of in situ cancer vaccines in reshaping the immune landscape within tumors, thereby bolstering the body’s ability to combat cancer cells.

In situ cancer vaccines offer a distinct advantage by eliciting a potent and specific immune response against tumors while mitigating many of the systemic side effects commonly associated with traditional chemotherapy and radiation treatments. One remarkable attribute of these vaccines is their ability to generate the immune response directly within the TME, thereby holding the potential to effectively target and eliminate metastatic cancer in distant sites.

Recent research conducted by Wang et al., in 2020 sheds light on another significant benefit of in situ cancer vaccines. Their study reveals that an in-situ gel vaccine induces a process known as TAM repolarization, shifting these cells toward the M1 phenotype. This repolarization is associated with enhanced efficacy of the anti-tumor vaccine and prolonged survival in both preclinical and clinical settings [ 60 ].

4.4. Microbial Vector Vaccines

The concept of microbial vector cancer vaccines traces its origins back to the late 19th century and the mid-20th century. In 1891, the groundwork for bacterial-based vaccines was laid, followed by advancements in viral-based vaccines in the 1950s. These pioneering developments paved the way for the utilization of microbial vectors in cancer vaccine research. Microbial vector vaccines have emerged as a compelling approach due to their ability to stimulate antigen presentation through both MHC class I and class II pathways. This dual mechanism enhances the immunogenicity of these vaccines [ 61 , 62 ]. Live vector-based vaccines, a subset of microbial vector vaccines, consist of recombinant viral and bacterial vectors. These vectors are designed to carry the required antigens of the vaccine, replicate within host cells, and elicit robust immune responses against the targeted disease.

4.4.1. Viral-Based Vaccines

In 1972, a significant milestone was achieved by Jackson et al., who successfully generated recombinant DNA from the SV40 virus. This breakthrough paved the way for further exploration and utilization of viral-based vaccines, particularly through the development of the vaccinia virus vector. Subsequently, a diverse array of novel vectors emerged and underwent evaluation in clinical trials, expanding the repertoire of viral-based cancer vaccines [ 63 ].

Viral-based vaccines employ genetically modified viruses as delivery vehicles, specifically targeting cancer cells while sparing healthy cells. Upon infiltrating cancer cells, these viruses replicate and express TAAs, which are then presented to immune cells. By presenting TAAs to the immune system, viral-based vaccines can elicit a potent immune response against cancer cells expressing these antigens. This approach harnesses the power of the immune system to selectively recognize and eliminate cancer cells, offering a highly targeted strategy for cancer immunotherapy. As emphasized by D’Alise et al. in 2022, viral-based vaccines hold significant potential to provide an effective and precise approach to combating cancer [ 64 ]. Among the most extensively studied vectors today are adenovirus and vaccinia virus. These vectors have demonstrated remarkable immune-stimulating properties, particularly in inducing the activation of CTLs, without necessitating the use of adjuvants [ 25 ].

Oncolytic virotherapy represents a compelling approach to enhance the effectiveness of cancer vaccines by orchestrating the modulation of the TME and selectively targeting and eliminating malignant tissue while preserving normal cells and surrounding tissues [ 65 ]. The oncolytic virus strategy in immunotherapy capitalizes on the ability of viruses to infect and replicate within tumor cells, resulting in cell death and the subsequent release of antigens and viral remnants. Crucially, the behavior and impact of the vaccine are determined by the vector and viral components’ capacity to stimulate the innate immune response, triggering interferon production and cytokine release [ 25 ]. The lysis of cancer cells leads to the liberation of various immunogenic components, including pathogen-associated molecular patterns (PAMPs), damage-associated molecular patterns (DAMPs), viral proteins, nucleic acids, TAAs, and immunogenic neoepitopes. These released factors initiate a cascade of immune responses, contributing to the activation of NK cells, which target cancer cells exhibiting reduced expression of MHC I. This process also induces ICD, further bolstering the immune response. Moreover, these cascades foster the initiation of de novo T cell responses against TAAs and neoantigens [ 22 ]. This multifaceted immune activation provides a robust defense mechanism against cancer, leveraging the innate and adaptive immune systems to combat malignant cells.

There has been a recent proposal for a new concept of oncolytic viral therapy where the virus is engineered in hybrid vectors to circumvent the different side effects of individual viral strains. In a particular study by Martínez-Vélez et al., (2019), the potential of the oncolytic adenovirus Delta-24-RGD elicits an anti-tumor effect on a variety of pediatric glioma [ 66 ]. Most preclinical studies within the past 10 years have focused on the HSV1716 oncolytic herpes virus treatment for neuroblastoma. In all studies, the oncolytic virus displayed no toxicity. While there are minimal studies on oncolytic adenovirus for the treatment of high-grade pediatric brain tumors, Delta-24-RGD has started clinical trials, demonstrating safe measures and efficacy against gliomas [ 66 ].

Some limitations of viral-based cancer vaccines can be dependent on pre-existing immunity to the virus used in the vaccine, which can reduce the effectiveness of the treatment. There is also a risk of adverse effects from the viral vector used in the vaccine, such as inflammation or allergic reactions. Despite these limitations, viral-based cancer vaccines have shown some effectiveness in preclinical and clinical studies; for example, the FDA has approved the use of a viral-based cancer vaccine called Sipuleucel-T for the treatment of advanced prostate cancer. Clinical trials are ongoing for other types of cancer, including lung cancer, melanoma, and glioblastoma. To improve the effectiveness of viral-based cancer vaccines, researchers are exploring various strategies, such as combining the vaccine with other immunotherapies targeting multiple antigens and optimizing the dosing and timing of the treatment.

Viral vaccines have emerged as powerful tools in targeting tumor angiogenesis, leading to regression or impeding the progression of distant metastases from the site of administration. Notably, talimogene laherparepvec, an FDA-approved oncolytic virus based on herpes simplex virus type 1 (HSV-1), marketed as T-VEC (ImlygicTM), has shown efficacy in treating surgically unresectable metastatic melanoma. Another promising viral vaccine in this category is PSA-TRICOM (Prostvac-VF), currently undergoing clinical trials ( {"type":"clinical-trial","attrs":{"text":"NCT02326805","term_id":"NCT02326805"}} NCT02326805 ) for advanced prostate cancer [ 22 , 26 ]. The PSA-TRICOM platform employs a strategic approach wherein it enables the expression of prostate-specific antigen, a marker for prostate cancer while incorporating three T cell receptor-stimulating co-regulators (TRICOM). This incorporation facilitates the activation of previously dormant DCs and T cells within the TME, as explained by Thomas and Prendergast in 2016 [ 26 ].

It is important to note that, due to in situ virus replication, the viral dose needs to be incrementally increased over time, and adherence to a scheduled vaccine protocol is imperative. These considerations ensure optimal virus-mediated therapeutic outcomes and sustained immune activation.

4.4.2. Bacteria-Based Vaccines

Long before the discovery of viral-based vaccines was the use of bacteria-based vaccines for cancer immunotherapy. As mentioned previously, in 1891, William Coley successfully used mixtures of live and inactivated Streptococcus pyogenes and Serratia marcescens in a novel treatment of sarcoma, leading to tumor regression [ 62 ]. Coley proved that the diverse use of bacteria-based vaccine vectors can be used to vaccinate against variable TAAs, deliver cytokines, and target immunosuppressive molecules. Elimination or conversion using chemotherapeutics or attenuated Listeria , along with tumor-killing agents to the TME using bacterial vectors, have been known to increase the effectiveness of cancer vaccines in older age patients [ 67 ].

In specific subsets of patients, the utilization of bacteria-based vaccines has demonstrated the capacity to generate long-term immunity and effectively target metastases. This remarkable ability, coupled with the potential to impede tumor proliferation, prevent metastasis, and hinder disease recurrence, positions bacteria-based vaccines as a formidable alternative to conventional therapies [ 60 ]. One promising strategy lies in directing the bacteria to the TME, a mechanism that holds significant potential in reducing systemic toxicity and enhancing therapeutic efficacy. By precisely delivering the vaccines to the TME, bacteria-based vaccines can more effectively engage with the tumor cells and trigger an immune response with heightened specificity.

It is important to acknowledge that the mechanisms and pathways employed by these vaccines can vary significantly, necessitating a deeper understanding of the future development of next-generation vaccines. As we delve further into their intricate workings, researchers can unlock new insights and leverage this knowledge to refine and advance the field of cancer vaccines.

Bacterial vectors, such as Listeria monocytogenes (Lm), Lactobacillus casei ( L. casei ), Lactobacillus lactis , and Salmonella , have undergone clinical trials and demonstrated their ability to elicit both innate and adaptive immune responses. Importantly, these bacterial vectors have shown the potential to reduce systemic toxicity, thereby enhancing therapeutic efficacy. By modifying the bacterial vectors to express cytokines and tumor antigens (TAs), such as IL-2 or mesothelin, they can induce targeted immune responses from T and NK cells that specifically recognize tumor cells [ 22 ]. One notable advantage of using bacterial vectors is the potential to simplify and economize the manufacturing and vaccination process by circumventing the challenges associated with target antigen purification. This approach offers a straightforward and cost-effective means for the large-scale production and administration of vaccines [ 60 ].

It is worth noting that certain bacteria can pose a risk factor for specific types of cancer. Among the well-known examples is Helicobacter pylori , which is strongly associated with gastric cancer [ 68 ]. Additionally, bacterial infections caused by Chlamydia trachomatis , Neisseria gonorrhoeae , and Treponema pallidum (syphilis) have been linked to an increased risk of developing cervical cancer.

4.5. Peptide Vaccines

While the focus of immunization has been largely based on infectious diseases and the treatment of allergies, current immunization efforts focus on noninfectious diseases, primarily directed against cancers. In fact, most cells in the TME are differentiated based on the regulation of endogenous proteins or mutations in those proteins, being a suitable target for vaccines [ 27 ].

The main goal of peptide immunogen is to induce protective T cell and B cell immunity. The earliest peptide vaccination study came from virus-derived CD8+ T cell epitopes, which was reported in the late 1980s, and discovered that mice vaccinated with small synthetic peptides can be recognized by CD8+ CTL [ 27 ]. Polypeptides and protein-based vaccines can significantly activate T cells resulting in a heightened immune response and enhanced T cell activation. They also sporadically fail to induce memory CD8+ T cell responses [ 22 , 26 ]. Peptide-based vaccines can be composed of TSAs, TAAs, CGAs, and virus-associated antigens; however, the most distinct categories tackled in this review include TAA and TSA [ 22 ].

Synthetic peptides used for the production of these vaccines usually consist of 20–30 amino acids that target tumor antigen-associated epitopes. In comparison to genetic vaccines, peptide vaccines are not able to encode full-length tumor antigens [ 69 ]. Most peptide vaccines can be delivered in the form of synthetic T cell epitopes or in combination with T cell and B cell epitopes into one therapeutic preparation. Synthetic T cell epitopes bind to HLA classes I and II of the APCs stimulating CD8+ T cells or CD4+ T helper cells to target TAAs or TSAs. These HLA-peptide complexes activate immune responses from CTLs and T helper cells. Another approach is using whole proteins as antigen carriers, namely liposomes, microemulsions, immune-stimulating complexes, and other microparticle systems.

Peptide cancer vaccines work by targeting specific TAAs expressed by cancer cells within the TME. These vaccines consist of short peptide sequences that mimic the TAAs and are administered to stimulate an immune response against the cancer cells. When peptide cancer vaccines are introduced into the TME, they are processed by APCs, such as DCs. The APCs internalize the peptides and present them on their surface in complex with MHC molecules. This presentation allows the peptides to be recognized by T cells, specifically CD8+ CTLs, which are capable of directly killing cancer cells expressing the targeted TAAs. The interaction between the presented peptide-MHC complex and the T cell receptor (TCR) triggers the activation of the CTLs. This activation leads to the expansion of a population of tumor-specific CTLs that can specifically recognize and target cancer cells expressing the TAAs. These CTLs can infiltrate the tumor and exert their cytotoxic effects, thereby combating the cancer cells within the TME [ 70 ].

Recent research has highlighted the potential of an IDO-specific (indoleamine 2,3-dioxygenase-specific) peptide vaccine as a targeted therapeutic strategy within the TME. This innovative vaccine specifically aims to address the immunosuppressive effects of IDO+ cells, a key component in myeloid-originated tumors. By inducing an IDO-specific immune response, the vaccine effectively targets and reduces the number of these suppressive cells while simultaneously increasing the presence of CD8+ T cells. These encouraging results suggest that combining this vaccine with other cancer vaccines may amplify their efficacy [ 71 ].

Peptide vaccines offer several advantages, including ease of synthesis, low production costs, low carcinogenic potential, and high chemical stability. However, they may exhibit low immunogenicity, and thus, combination strategies should be considered [ 18 , 69 ]. To enhance their effectiveness, computer-based algorithms can be employed to screen amino acid sequences and identify those with the greatest compatibility. Experimental testing of the selected sequences for antigen specificity is also essential [ 71 ]. It should be noted that while peptide vaccines have shown the ability to delay immune responses and inhibit tumor growth, significant tumor shrinkage has yet to be consistently demonstrated [ 69 ].

When discussing peptide vaccination, delivery of the peptide antigen becomes a crucial consideration due to its potential toxicity. Li et al. (2014) have extensively explored various administration routes, including transdermal patches, subcutaneous injection, and intravenous injection, assessing their safety and efficacy [ 27 ].

The transdermal route of administration has been the safest and commendable by being needle-free and eliminating the need for healthcare professionals to administer the vaccine. Particularly, this can help with systemic immunity and increase shelf life and stability while reducing the cost of application [ 72 ]. Regarding the subcutaneous injection and intravenous injection of a peptide vaccine, it was determined that these methods were also safe by detailing the immediate adverse effects within 30 min of administration.

Regardless of the several advantages, peptides are typically more immunogenic when used with next-generation adjuvants than without. Future development and improvement of the safety concerns related to the use of adjuvants and particulate peptide vaccine delivery systems should be considered. The specific limitations of older adjuvants include a lack of cellular immune stimulation, degradation on freeze drying, and the possibility of adverse local reactions; thus, newer developments in the delivery system focus on overcoming these imperfections [ 73 , 74 ]. Substances that create adverse reactions, like toxins, lipids, and lipopolysaccharides, are typically avoided in the development of this vaccine.

In recent trials, incorporating proteins or peptides in cancer immunotherapy has shown promising results, particularly in targeting novel cellular antigens, such as preferentially expressed antigens in melanoma (PRAME). PRAME is a cancer-testis antigen expressed in solid tumors and hematologic malignancies, with high expression correlating with a good prognosis in acute myeloid leukemia [ 75 ].

Pujol et al. (2016) conducted a phase I dose escalation study targeting PRAME antigens in clinical trials ( {"type":"clinical-trial","attrs":{"text":"NCT01159964","term_id":"NCT01159964"}} NCT01159964 ). The study aimed to assess the safety and immunogenicity of an immunotherapeutic consisting of recombinant PRAME and the immunostimulant AS15 [ 76 ]. Sixty patients with PRAME-positive resected non-small cell lung carcinoma (NSCLC) were divided into three groups, receiving different doses of PRAME antigen immunotherapy. The study found detectable anti-PRAME antibodies in all patients after four doses, with the highest percentage of patients showing PRAME-specific CD4+ T cell responses in the group receiving the highest dose (500 μg). However, no predefined CD8+ T cell responses were detected, and further development of the immunotherapeutic was halted when comparable results were not achieved in NSCLC patients [ 76 ].

A specific subtype of diffuse glioma is characterized by a mutation in isocitrate dehydrogenase 1 (IDH1), with the most common mutation affecting codon 132, resulting in the IDH1(R132H) protein. In a study by Schumacher et al. (2014), an IDH1(R132H)-specific vaccine demonstrated efficacy in inducing a targeted T helper cell response against IDH1(R132H) + tumors in MHC-humanized mouse models [ 77 ].

To further investigate the safety and immunogenicity of the IDH1-specific vaccine in humans, a multicenter phase I trial was conducted on 32 patients with newly diagnosed grade 3 and 4 IDH1(R132H) + astrocytoma. The trial administered an IDH1 targeted peptide-based vaccine (IDH1-vac) containing 300 ± 30 μg of immunogenic peptide. The patients, both male and female, ranged in age from 18 to 65 years. The trial group was then divided into three subgroups based on their previous treatment: TG1 received radiotherapy alone, TG2 received three cycles of TMZ (temozolomide) chemotherapy alone, and TG3 received both radiotherapy and chemotherapy. Among the participants, 17 underwent complete tumor resection, 12 had a subtotal resection, and 3 underwent a biopsy. Methylation analysis categorized 14 patients as low grade and 10 as high grade. The IDH1-vac was administered periodically at specific weeks, and blood samples were collected for immunogenicity testing. MRI (magnetic resonance imaging) scans were performed at designated time points to monitor the patients’ response to the vaccine.

In a clinical trial ( {"type":"clinical-trial","attrs":{"text":"NCT02454634","term_id":"NCT02454634"}} NCT02454634 ) evaluating the IDH1-specific vaccine in patients with IDH1(R132H) + astrocytoma, vaccine-related side effects were limited to grade 1, and no significant toxicity that would impact the treatment regimen was observed. Encouragingly, the vaccine-elicited immune responses in 93.3% of patients spanning multiple MHC alleles. T cell responses were detected in 26 out of 30 patients, while B cell responses were observed in 28 out of 30 patients. Analysis of the IDH1-vac-induced T cells using flow cytometry revealed the production of tumor necrosis factor (TNF), interferon-γ (INF-γ), and interleukin-17 (IL-17) by T helper cells.

Follow-up analysis showed favorable three-year progression-free and death-free rates, with 63% (95% CI 44–77) and 84% (95% CI 67–93) of patients experiencing no disease progression or death, respectively. Notably, patients who did not develop an immune response had a higher risk of disease progression within two years compared to those with an immune response, whose two-year progression-free rate was 82% (95% CI 62.3–92.1). These findings from Platten et al. (2021) underscore the safety, immunogenicity, and potential clinical benefit of the IDH1-specific vaccine in patients with IDH1(R132H) + astrocytoma. The vaccine-induced immune responses, particularly the T cell responses producing TNF, INF-γ, and IL-17, appear to be associated with improved clinical outcomes [ 6 ].

4.6. Nucleic Acid-Based Vaccines

Gene-based vaccines utilize DNA or RNA to deliver the coding region of an antigen, stimulating a host immune response and production of selected antigens [ 78 ]. These vaccines have several advantages. (1) They can encode full-length tumor antigens, allowing for the presentation of multiple epitopes and a broader T cell response. (2) Vaccination can activate DCs and increase pro-inflammatory cytokine levels. (3) Fusion genes can be produced to enhance the generation of T-helper memory response.

Choosing the right plasmid, typically of bacterial origin, is crucial in gene-based vaccine production. However, the main challenge lies in the delivery method [ 18 , 78 ]. While electroporation and viral vectors show high efficiency, they are difficult to use in clinical practice and may lead to variable immune responses and unwanted side effects.

Nucleic-acid-based vaccines offer the advantage of delivering multiple antigens and targeting various TAAs or somatic tumor mutations. This activation of both humoral and cell-mediated immune responses increases the chances of overcoming vaccine resistance. The two main categories of gene-based vaccines are DNA- and RNA-based vaccines.

4.6.1. DNA Vaccines

DNA vaccines are plasmid-based vaccines that deliver genes encoding tumor antigens, triggering an innate immune response. The presence of unmethylated CpG motifs in the delivered DNA stimulates immune response pathways. DNA vaccines are known to elicit a potent CD8+ T cell response against neoantigens compared to RNA and peptide vaccines [ 22 , 28 ]. They are stable, cost-effective, and easy to produce and can be stored without strict cold chain requirements.

To enhance targeted delivery, engineered cell-derived exosomes and synthetic nanoparticle complexes have shown promise [ 23 , 79 ]. Delivery techniques, such as electroporation, sonoporation, gene guns, or DNA tattooing, are commonly used. However, as a monotherapy, DNA vaccines have limitations and are insufficient to escape immune system recognition and attack [ 22 ].

To maximize efficacy, DNA vaccines are often combined with other strategies and adjuvants, such as cytokines, immune checkpoint blockade, chemotherapy, radiotherapy, and endocrine therapy [ 80 ]. However, concerns exist regarding the possibility of plasmid DNA integrating into the chromosome, the use of genes encoding cytokines or co-stimulatory molecules, and the potential undesirable effects of the expressed antigen itself [ 81 ].

4.6.2. RNA Vaccines

RNA vaccines for cancer are a relatively new field, but the use of RNA as a therapeutic agent has been explored for some time. Early attempts to develop RNA vaccines encountered challenges with stability, immunogenicity, and delivery. However, considerable progress was made in the early 2000s, with researchers demonstrating the potential of RNA vaccines for cancer immunotherapy in animal models. They showed that injecting RNA encoding TSAs could induce a robust immune response and delay tumor growth [ 82 ]. Following this breakthrough, various companies and academic institutions began developing RNA vaccine candidates for different diseases. The culmination of these efforts came in 2020 when the first RNA-based vaccines for COVID-19 were authorized for emergency use. Developed by Pfizer-BioNTech and Moderna, these vaccines marked a major milestone in the history of RNA vaccines.

RNA-based cancer vaccines have gained significant interest in recent years, with numerous studies investigating their potential in treating diverse types of cancers. In a notable development, the U.S. FDA approved the first RNA vaccine for cancer, called “Imlygic” or talimogene laherparepvec, in 2020. This vaccine utilizes a modified herpes simplex virus to selectively replicate within cancer cells, causing them to rupture. It also releases GM-CSF, a protein that stimulates the immune system to target and attack cancer cells.

There are three main types of RNAs being studied as cancer vaccines: non-replicating unmodified mRNA, modified mRNA, and self-amplifying mRNA derived from viruses [ 83 ]. Naked RNA is unstable, so researchers have explored different delivery methods, such as the “gene gun”, protamine condensation, and encapsulation, to enhance stability and performance [ 22 ]. Unlike DNA vaccines, mRNA vaccines can be translated into both dividing and non-dividing cells, and they do not integrate into the genome sequence [ 82 ]. mRNA vaccines offer advantages such as high potency, safe administration, rapid development potential, and cost-effective manufacturing.

Cafri et al. (2020) conducted studies focused on the detection and selection of neoantigens expressed by autologous cancer and recognized by TILs [ 29 ]. Metastatic gastrointestinal tumors were harvested, and TILs were collected for analysis. Using high-throughput immunologic screening, tumor-specific mutations were sequenced, and TSAs were identified using long peptides and tandem minigenes. In this study, four patients with gastrointestinal cancer, who had previously been treated with TILs or anti-PD1 agents, were intramuscularly injected with a personalized mRNA-4650 vaccine. The vaccine contained up to 20 selected antigens, including mutations in TP53 (tumor protein 1 gene), KRAS (Kirsten rat sarcoma virus), or PIK3CA, as well as up to 15 in silico-predicted HLA class 1 potential neoantigens. Two patients received eight injections of 0.13 mg, while the other two received four injections of 0.39 mg. The patients experienced mild toxicities, mostly grade 1 and 2, which resolved quickly without severe adverse effects. The study observed that approximately 15.7% of the predicted antigens induced a T cell-specific immune response, with 59% of the epitopes being CD4+ and 41% CD8+ epitopes. The number of vaccination-induced mutations varied, ranging from 2 to 6 per patient. It is worth noting that one patient did not show evidence of an immune response following vaccination [ 29 ].

4.7. Exosome-Based Vaccines

Recent studies show that tumor-derived exosomes are highly enriched in tumor antigens, MHC molecules, heat-shock proteins, and inducible co-stimulatory molecules found in the TME. Exosomes can trigger the differentiation of fibroblasts into CAF, leading to elevated smooth muscle actin expression and angiogenesis with the promotion of VEGF, fibroblast growth factor (FGF), platelet-derived growth factor (PDGF), basic FGF, transforming growth factor β (TGF-β), TNF-α, and interleukin-8 (IL-8) [ 30 ]. This is a promising research area for vaccine development as they behave as signaling molecules in cancer promotion and TME remodeling [ 84 ]. While exosomes are also capable of delivering functional RNAs to target cells, in combination with immunostimulatory agents, they can trigger potent CD8+ T cell anti-tumor responses while serving as useful biomarkers in cancer screening and early diagnosis [ 23 ].

A hepatocellular carcinoma (HCC) study described in the Lou et al. (2015) article on exosome delivery of anticancer drugs across the blood-brain barrier demonstrated the delivery of chemotherapeutic agents, like 5-FU and sorafenib, via adipose tissue-derived mesenchymal stem cell (AMSC) exosomes. miR-122, a key regulator of liver physiology and disease biology, was modified for additional use in this exosome treatment to increase the chemosensitivity of HCC cells [ 85 ]. In zebrafish models described in the same article, brain endothelial-cell-derived exosomes could pass through the blood-brain barrier and deliver medications, like paclitaxel and doxorubicin, for the treatment of brain cancer [ 86 ].

Perhaps the greatest concern of exosome vaccination regarding the TME is tumor resistance to therapy. Epithelial-mesenchymal transition (EMT) is promoted by carrying factors, like TGF-β, HIF1α (hypoxia-inducible factor 1-alpha), β-catenin, IL-6, β-catenin or vimentin, casein kinase, and several miRNAs. Induction of EMT may lead to tumor resistance to therapy through anti-apoptotic pathway promotion via drug efflux or drug sequestration, signal transduction alteration, and immune cell modulation [ 87 ].

5. Delivery Methods

The efficacy of these vaccines relies not only on the selection of appropriate antigens but also on the development of efficient delivery methods. Several strategies have been employed to deliver cancer vaccines tailored to optimize immune responses and enhance therapeutic outcomes. Here, we discuss common delivery methods for cancer vaccines and provide examples of their application in clinical settings.

Injection-based delivery is the most widely utilized approach for cancer vaccines, allowing for direct delivery of the vaccine into the body. Intramuscular or subcutaneous injections are commonly used, such as in the administration of HPV vaccines like Gardasil and Cervarix, which help prevent cervical cancer and other HPV-associated malignancies [ 88 , 89 ]. Intratumoral injections have also been employed, particularly for localized tumors, to induce a potent immune response within the TME [ 90 ].

Intravenous (IV) infusion provides a systemic delivery route, enabling the distribution of cancer vaccines throughout the body. PROVENGE, an FDA-approved vaccine for advanced prostate cancer, is administered via intravenous infusion [ 91 ]. This vaccine utilizes autologous DCs loaded with a fusion protein to activate the patient’s immune system against prostate cancer cells.

Oral administration offers a convenient and non-invasive delivery method for certain cancer vaccines. An example is the oral polio vaccine, which has been repurposed for the treatment of glioblastoma. Known as PVSRIPO, this vaccine utilizes a modified poliovirus to target and kill tumor cells [ 92 ].

Topical application involves directly applying the cancer vaccine to the skin or mucous membranes. Talimogene laherparepvec (T-VEC), an oncolytic virus-based vaccine, is administered through direct injection into cutaneous melanoma lesions [ 93 ]. By selectively replicating within tumor cells, T-VEC leads to their destruction and the induction of a systemic immune response against the cancer.

Gene delivery represents another approach in cancer vaccine development, involving the delivery of genetic material encoding TSAs. For example, the ADVAXIS-HPV vaccine utilizes a live attenuated bacterium to deliver a tumor antigen (HPV E7) gene into the body. The genetic material is expressed by cells, promoting immune recognition and tumor-specific immune responses [ 94 ].

Electroporation enhances vaccine delivery by applying electric pulses to the skin or tumor site. This method increases cell membrane permeability, enabling efficient uptake of the vaccine. Clinical trials have utilized the TriGrid™ electroporation system to enhance the delivery of melanoma-specific vaccines, such as the gp100 peptide vaccine, resulting in improved immune responses [ 95 ].

Nanoparticle-based delivery offers a versatile platform for cancer vaccine administration, providing enhanced stability, targeted delivery, and controlled release of vaccine components. Various nanoparticle-based vaccines, including lipid-based nanoparticles, polymer nanoparticles, and virus-like particles, are being investigated. These systems can encapsulate tumor antigens, adjuvants, or genetic material, facilitating efficient antigen presentation and immune stimulation [ 96 ].

The development of effective cancer vaccines necessitates the utilization of appropriate delivery methods. The diverse range of delivery approaches, including injection, intravenous infusion, oral administration, topical application, gene delivery, electroporation, and nanoparticle-based delivery, allows for tailored immunization strategies. Ongoing research aims to optimize vaccine delivery, combining multiple strategies to elicit robust and durable anti-tumor immune responses, thereby improving therapeutic outcomes for cancer patients.

6. Improving Vaccine Outcomes

Since the first approved adjuvant, called alum, was developed in 1926 by Alexander Glenny, hundreds of materials have been studied as adjuvants. It consists of aluminum salts, such as aluminum hydroxide or aluminum phosphate. Alum adjuvants enhance the immune response by promoting the activation of APCs and the release of pro-inflammatory signals, leading to the recruitment and activation of immune cells. It also enhances the uptake and presentation of antigens to immune cells, thereby stimulating both antibody and cellular immune responses. In 1936, Freund’s complete adjuvant was developed but was found to induce severe local necrotic ulcers and is considered too toxic for human use; thus, alum quickly became the gold standard for efficacy. Leading from alum to cytokines, TLR agonists, saponins, mineral salts, emulsions, and bacterial components, like lipopolysaccharides and lipophile phospholipids, led to the development of less toxic and a more stable FDA-approved vaccination adjuvant [ 13 ] MF59, virosome, AS03, AF03, and monophosphorylate lipid A (MPLA) in conjunction with alum are just some of the newly approved adjuvants used.

TLRs are a type of pattern recognition receptor that recognizes specific molecular patterns on pathogens. TLR agonists mimic these patterns and activate the innate immune system, leading to increased antigen presentation, enhanced production of pro-inflammatory cytokines, and improved activation of adaptive immune responses. Various TLR agonists, such as MPLA, can serve as adjuvants to improve vaccine efficacy. An example of the use of a liposome as an adjuvant is the peptide-CpG-liposome composite vaccine developed by Park et al. (2018). It was observed to induce humoral responses and inhibit cancer growth in pancreatic cancer, metastatic hepatocellular carcinoma, and colon cancer murine tumor models [ 97 ].

MPLA is a chemically modified derivative of lipopolysaccharides that displays reduced toxicity while maintaining the immunostimulatory activity seen in lipopolysaccharides [ 5 , 98 ]. Because they are lipid-based vesicles, they can encapsulate antigens and deliver them to immune cells. They can function as adjuvants by improving antigen presentation, enhancing antibody production, and promoting cellular immune responses. Liposomes can be designed to carry various immune-stimulating molecules, such as CpG DNA, which activates TLR9 and triggers immune activation.

We see MPLA used specifically in the HBV (hepatitis B virus) vaccine Fendrix for patients with renal insufficiency and in broad HPV vaccines like Cervarix. With a history of safe and effective use, MPLA technology has shown wide success in clinical trials and is being considered for peptide vaccine delivery. It is important to note that the effectiveness of MPLA as an adjuvant in targeting the TME can depend on numerous factors, including the specific cancer type, the stage of the disease, and the overall immune status of the individual. Combination approaches using MPLA with other immunotherapeutic agents or treatments may also be explored to further enhance the anti-tumor immune response and target the TME more effectively.

Saponins, extracted from the Quillaja saponaria plant (QS-21), function as immunostimulant adjuvants having strong anti-inflammatory properties. QS-21 is one of the most widely used adjuvants in vaccines. Saponins can induce the activation of APCsing cells, stimulate the production of pro-inflammatory cytokines, and promote the generation of both humoral and cellular immune responses.

GM-CSF is commonly used as an immunostimulatory adjuvant, where cytokines direct the differentiation, proliferation, and activation of macrophages and DCs with a focus on cDC1 and Th1 responses [ 99 ]. GM-CSF induces an influx of immune cells, including DCs, macrophages, eosinophils, and T cells, at the vaccination site, which is crucial for initiating and orchestrating immune responses against pathogens or tumors.

GM-CSF cytokines have been found to be more effective than other researched cytokines, such as IL-2, IL-4, IL-5, and y-IFN, as it activates a tumor-specific T cell response and is used in the production of GM-CSF-secreting whole-cell cancer vaccines [ 31 , 100 , 101 ]. The GM-CSF vaccine adjuvant produced conflicting results in clinical trials, some showing only a weak effect in enhancing the immune response of the vaccine [ 13 ]. In others, there were no positive effects at all [ 102 , 103 ]. However, high clinical efficacy has been found in combination with other immunotherapy [ 103 ].

These different adjuvants work through various mechanisms to enhance the immune response, including promoting antigen uptake and presentation, activating innate immune cells, inducing the production of pro-inflammatory cytokines, and directing immune responses towards a desired profile (e.g., Th1 responses). By improving the immune response, adjuvants help to generate stronger and more long-lasting protective immunity against pathogens or antigens of interest.

It is important to note that the effectiveness of adjuvants can vary depending on the specific vaccine formulation, target pathogen or antigen, and the characteristics of the immune system in individuals. Combination approaches using multiple adjuvants or adjuvants in conjunction with other immunotherapeutic strategies are also being explored to further enhance the immune response and target specific immunological pathways effectively.

Radiotherapy, Chemotherapy, and Naturopathy

In preclinical studies conducted by Adler et al. (1998), vaccines given five weeks after radiotherapy were most effective [ 104 ]. Recent studies have observed an increase in cancer cell resistance to chemotherapy and radiotherapy and the promotion of tumor angiogenesis, leading to a greater risk for tumor invasion and metastasis [ 105 ].

Some clinical responses can be rapid, being active and efficient within just a few weeks with fast, noticeable regression. However, in comparison to chemotherapy and radiotherapy, where the response can take 2–3 months, therapeutic vaccine methods alone may be considered as having a slower immune response and could take several months to be effective. With several studies and assessments, it was determined that treatment methods like radiotherapy and chemotherapy added to the adjuvant show a higher success rate in treatment. In pancreatic ductal adenoma (PDA), discussed in a study by Mandili et al. (2020), chemotherapy enhances the immune response to TAA [ 106 ]. Vaccinations combined with Ipilimumab, for example, could increase the density of proliferating intratumoral CD8+ T cells or ISCOMATRIX (a saponin-based adjuvant). Ideally, this would develop a strong CD4+ and CD8+ T cell response [ 107 , 108 ].

With clinical trials, Cuzzubbo et al. (2021) also described the importance of using a more naturopathic method to help increase immune system strength, which could lead to increased effectiveness in cancer vaccines [ 13 ]. They explained that since radical treatments like chemotherapy and radiotherapy could strongly weaken the immune system, changing diet, increasing physical effort, and reducing stress could increase the effectiveness of anticancer vaccines [ 13 ]. Webber et al. (2018) stated that a ketogenic diet may produce clinical application and described its effectiveness as being similar to adjuvant therapy in cancer patients. They also mentioned how the effectiveness and safety of the ketogenic diet as supplementary therapy depended on the tumor location and genotype [ 109 ]. This could be something to consider during treatment.

7. Limitations of Cancer Vaccines

Despite the immense potential for cancer vaccination, various limitations hinder the widespread application and optimal efficacy of these vaccine types. The following presents an overview of the limitations associated with each vaccine category, shedding light on the challenges they pose in clinical translation.

Cell-based cancer vaccines encounter several limitations that hinder their widespread adoption and effectiveness. The process of isolating and culturing patient-specific immune cells demands significant resources and expertise, leading to high production costs that strain healthcare systems and limit patient access to this personalized therapy. This necessity for individualized vaccines tailored to specific tumor antigens poses challenges in mass production and distribution, impacting the scalability and logistical feasibility of these vaccines. Moreover, tumors can establish suppressive microenvironments that impede the activated immune cells’ efficacy, hindering their ability to recognize and combat tumor cells effectively. Tumor cells may thus develop immune evasion mechanisms, further compromising the overall efficacy of the vaccine and facilitating tumor progression. Additionally, the transportation and storage of cell-based vaccines under stringent conditions pose logistical obstacles, potentially jeopardizing the viability of these vaccines during distribution [ 110 ].

Ethical considerations constitute one of the primary concerns regarding iPSC-based vaccines. The process of generating iPSCs may involve the use of embryonic stem cells and genetic manipulation, raising ethical questions that demand careful evaluation before proceeding with clinical applications. The safety profile of iPSCs is a critical aspect that requires extensive preclinical safety assessments to address the potential risks of tumor formation or unwanted cellular responses. Additionally, like other vaccine types, iPSC-based vaccines may encounter immune evasion challenges, impacting their ability to provoke robust anti-tumor immune responses [ 111 ].

The identification of specific tumor antigens for in situ cancer vaccines can be a formidable task, particularly for cancers characterized by high genetic heterogeneity. The intricate TME may pose immune suppression, diminishing the immune response elicited by in situ vaccines and consequently reducing their overall efficacy. Furthermore, they might be limited to localized tumor sites, potentially offering suboptimal effectiveness against metastatic tumors [ 112 , 113 ].

Pre-existing immunity to the viral vectors used in microbial vector vaccines can compromise their effectiveness, particularly in individuals who have been previously exposed to the vector. Safety concerns associated with certain microbial vectors need to be thoroughly addressed to prevent adverse immune responses in vaccinated patients. Additionally, the limited cargo capacity of microbial vectors poses a challenge in delivering multiple tumor antigens simultaneously, potentially limiting the scope of immune responses generated by the vaccines [ 114 ].

The delivery of nucleic acids (DNA or RNA) to target cells demands specialized delivery systems or technologies to ensure efficient and effective transfection. The transient expression of antigens encoded by nucleic-acid-based vaccines may require multiple doses to sustain an adequate and prolonged immune response. Furthermore, the development of immune tolerance to the encoded antigens can limit the vaccine’s long-term efficacy, necessitating strategies to mitigate immune tolerance and improve vaccine durability [ 115 ].

Peptide-based cancer vaccines encounter several other limitations that impact their efficacy and applicability. Their focus on specific tumor antigens may lead to the oversight of other relevant antigens that could be targeted by the immune system, thereby limiting their overall effectiveness in eliciting a comprehensive anti-tumor response. Additionally, the requirement for peptides to bind to specific HLA molecules for immune recognition restricts their use in patients with compatible HLA types, potentially excluding a significant proportion of the patient population. The genetic diversity of tumors further complicates the efficacy of peptide vaccines, as some tumor cells may lack the targeted peptides, leading to suboptimal immune responses. Compared to whole-cell-based vaccines, peptide vaccines may induce weaker immune responses, necessitating the implementation of supplementary strategies to enhance their immunogenicity. Lastly, the incorporation of adjuvants or immune-stimulating agents is often necessary to potentiate the immune response elicited by peptide vaccines, resulting in more complex vaccine formulations [ 116 ].

The scalability of exosome production for exosome-based vaccines is a challenge that requires careful optimization to ensure sufficient vaccine availability for large-scale therapeutic use. Efficient loading of exosomes with an adequate amount of tumor antigens is essential to maximize their immunogenicity and enhance their potency as vaccines. Moreover, exosomes’ potential to modulate the immune response may have unintended consequences on vaccine efficacy and safety, necessitating a comprehensive understanding of their interactions with the immune system.

8. Conclusions

Cancer vaccines represent a promising avenue in the fight against cancer, harnessing the power of the immune system to prevent tumor growth, recurrence, or metastasis while enhancing its ability to recognize and eliminate cancer cells. The intricate composition of TMEs and the diverse responses they elicit play a crucial role in determining treatment outcomes. The activation of T cells is vital for effective immune responses against cancer, while B cells contribute to both tumor suppression and promotion. NK cells hold the potential to eliminate tumor cells but face challenges within the immunosuppressive TME. DCs play a crucial role in antigen presentation and T cell activation but may be impaired by tumor-derived factors. Neutrophils and TAMs exhibit dynamic roles, capable of switching between pro-tumor and anti-tumor states, influencing tumor initiation and progression. Understanding the complex interactions within TMEs is essential for designing effective cancer vaccines.

Delivery methods and the inclusion of adjuvants have proven pivotal in optimizing cancer vaccine efficacy. Various delivery approaches, such as peptide-based, nucleic acid-based, protein-based, viral vector-based, and DC-based vaccines, offer distinct advantages and challenges that need to be considered for successful implementation. The integration of adjuvants, such as TLR agonists, cytokines, and immune checkpoint inhibitors, enhances immune responses and promotes sustained immune activation, further augmenting vaccine efficacy.

Overcoming the challenges associated with cancer vaccines requires a multifaceted approach driven by innovative research and clinical advancements. To enhance the accessibility and affordability of cell-based vaccines, streamlining the production and distribution processes while leveraging personalized medicine advancements, such as biomarkers and genomics, can tailor vaccines to individual patients’ needs. Addressing immune evasion and the suppressive TME demands a deeper understanding of tumor immune escape mechanisms, potentially leading to novel strategies for circumventing immune suppression and improving cell-based vaccine efficacy. Additionally, optimizing peptide vaccines entails broadening the range of tumor antigens considered and exploring heteroclitic peptides to expand their applicability across various HLA types. Combining vaccines with immune checkpoint inhibitors or other immunomodulatory agents could synergistically enhance vaccine responses. For iPSC-based vaccines, rigorous preclinical safety assessments and refined protocols for generating iPSCs are paramount for safe and reliable clinical translation. Furthermore, exosome-based vaccines could benefit from improved production methods and enhanced loading techniques to ensure potent and consistent vaccines with sufficient tumor antigens. Understanding the interactions between exosomes and the immune system will provide insights into their immunomodulatory effects and safety profiles. Conducting well-designed clinical trials with long-term follow-ups is essential to assess the efficacy, safety, and durability of cancer vaccines across diverse patient populations. By collectively pursuing these avenues of research, we can overcome current limitations, ushering in an era of more effective and personalized cancer immunotherapies that hold the promise of controlling and defeating cancer.

To overcome these challenges, ongoing research is focused on TME profiling, molecular pathway mapping, and an improved understanding of TME-mediated patterns. The aim is to optimize the development of personalized cancer vaccines and mitigate the potential risks. By gaining deeper insights into the intricate dynamics of the TME, we can refine the design of these vaccines and strive for curative outcomes, particularly for patients with advanced metastatic diseases. While TME heterogeneity, the presence of immunosuppressive cells, and tumor escape mechanisms pose significant hurdles, combination therapies that incorporate diverse vaccine types, adjuvants, and chemotherapeutic/radiotherapeutic strategies are showing promise in enhancing immune activation and improving treatment outcomes. Although the success rates of current cancer vaccines vary, continued TME profiling and molecular pathway mapping offer valuable opportunities to advance our understanding of TME-mediated patterns.

Funding Statement

This research received no external funding.

Conflicts of Interest

The authors declare no conflict of interest.

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

IMAGES

21 Research Limitations Examples (2024)
systematic review limitations section
how to write limitations in literature review
Limitations of literature review papers.
how to write limitations in literature review
Literature Review Guidelines

VIDEO

Literature Review: Find Research gap and limitation in an article!
How to Do a Good Literature Review for Research Paper and Thesis
𝐁𝐫𝐞𝐚𝐤𝐢𝐧𝐠 𝐁𝐚𝐫𝐫𝐢𝐞𝐫𝐬: 𝐑𝐮𝐜𝐡𝐢𝐭𝐚 𝐒𝐚𝐡𝐮𝐤𝐚𝐫𝐢'𝐬 𝐉𝐨𝐮𝐫𝐧𝐞𝐲 𝐰𝐢𝐭𝐡 𝐎𝐬𝐭𝐞𝐨𝐠𝐞𝐧𝐞𝐬𝐢𝐬 𝐈𝐦𝐩𝐞𝐫𝐟𝐞𝐜𝐭𝐚
A Comprehensive Guide to Systematic Literature Review (SLR)
Chapter 1. Part 2. How to write SOP, Significance of the Study, Hypotheses, Scope and Limitations
English Literature (MDC) Previous UG 1st & 2nd Sem Papers || Kashmir University

COMMENTS

8 common problems with literature reviews and how to fix them
In our recent paper in Nature Ecology and Evolution, we highlight 8 common problems with traditional literature review methods, provide examples for each from the field of environmental management and ecology, and provide practical solutions for ways to mitigate them. Problem. Solution. Lack of relevance - limited stakeholder engagement can ...
Eight problems with literature reviews and how to fix them
Environment. Policy*. Research Design. Systematic Reviews as Topic*. Traditional approaches to reviewing literature may be susceptible to bias and result in incorrect decisions. This is of particular concern when reviews address policy- and practice-relevant questions. Systematic reviews have been introduced as a more rigorous approach to ...
Literature review as a research methodology: An overview and guidelines
A literature review can broadly be described as a more or less systematic way of collecting and synthesizing previous research (Baumeister & Leary, 1997; Tranfield, Denyer, & Smart, 2003). ... the limitations of the search method, and the quality of the search process, and they often lack details on how the analysis was conducted. Second, in ...
What is a literature review?
A literature or narrative review is a comprehensive review and analysis of the published literature on a specific topic or research question. The literature that is reviewed contains: books, articles, academic articles, conference proceedings, association papers, and dissertations. It contains the most pertinent studies and points to important ...
Eight problems with literature reviews and how to fix them
Gillian Petrok ofsky, Andrew S. Pullin9, Sini Savilaakso 10,11 and Gavin B. Stewart 12. T raditional approaches to reviewing literature may be susceptible to bias and result in incorrect decisions ...
Guidance on Conducting a Systematic Literature Review
Introduction. Literature review is an essential feature of academic research. Fundamentally, knowledge advancement must be built on prior existing work. To push the knowledge frontier, we must know where the frontier is. By reviewing relevant literature, we understand the breadth and depth of the existing body of work and identify gaps to explore.
How to Write a Literature Review
Examples of literature reviews. Step 1 - Search for relevant literature. Step 2 - Evaluate and select sources. Step 3 - Identify themes, debates, and gaps. Step 4 - Outline your literature review's structure. Step 5 - Write your literature review.
Eight problems with literature reviews and how to fix them
The authors also excluded/ignored grey literature (see point 5, below). In a review of tropical forest management impacts 32 and in a review of forest conservation policies 33, searches for ...
Writing a literature review
A formal literature review is an evidence-based, in-depth analysis of a subject. There are many reasons for writing one and these will influence the length and style of your review, but in essence a literature review is a critical appraisal of the current collective knowledge on a subject. Rather than just being an exhaustive list of all that ...
Writing the Literature Review: Common Mistakes and Best Practices
The literature review is an essential component of academic research writing, providing a comprehensive overview of existing research and informing the development of new studies. ... But this time, a solution to the original study's limitation is offered in boldfaced font as well as support for that solution using the findings of yet another ...
Ten Simple Rules for Writing a Literature Review
Literature reviews are in great demand in most scientific fields. Their need stems from the ever-increasing output of scientific publications .For example, compared to 1991, in 2008 three, eight, and forty times more papers were indexed in Web of Science on malaria, obesity, and biodiversity, respectively .Given such mountains of papers, scientists cannot be expected to examine in detail every ...
Systematic reviews: Brief overview of methods, limitations, and
CONCLUSION. Siddaway 16 noted that, "The best reviews synthesize studies to draw broad theoretical conclusions about what the literature means, linking theory to evidence and evidence to theory" (p. 747). To that end, high quality systematic reviews are explicit, rigorous, and reproducible. It is these three criteria that should guide authors seeking to write a systematic review or editors ...
Systematic and other reviews: criteria and complexities
A systematic review follows explicit methodology to answer a well-defined research question by searching the literature comprehensively, evaluating the quantity and quality of research evidence rigorously, and analyzing the evidence to synthesize an answer to the research question. The evidence gathered in systematic reviews can be qualitative ...
Approaching literature review for academic purposes: The Literature
A sophisticated literature review (LR) can result in a robust dissertation/thesis by scrutinizing the main problem examined by the academic study; anticipating research hypotheses, methods and results; and maintaining the interest of the audience in how the dissertation/thesis will provide solutions for the current gaps in a particular field.
Limitations of the Study
The limitations of the study are those characteristics of design or methodology that impacted or influenced the interpretation of the findings from your research. ... Lack of prior research studies on the topic-- citing prior research studies forms the basis of your literature review and helps lay a foundation for understanding the research ...
Libraries: Writing a Literature Review: Phase 1: Scope of Review
In specifying precisely one's research topic, one is also specifying appropriate limitations on the research. Limiting, for example, by time, personnel, gender, age, location, nationality, etc. results in a more focused and meaningful topic. Scope of the Literature Review. It is also important to determine the precise scope of the literature ...
Advantages and disadvantages of literature review
According to the University of Illinois (2022), literature reviews allow researchers to gain familiarity with the existing knowledge in their selected field, as well as the boundaries and limitations of that field. Creation of new body of knowledge. One of the key advantages of literature review is that it creates new body of knowledge.
What is a Literature Review? How to Write It (with Examples)
A literature review is a critical analysis and synthesis of existing research on a particular topic. It provides an overview of the current state of knowledge, identifies gaps, and highlights key findings in the literature. 1 The purpose of a literature review is to situate your own research within the context of existing scholarship ...
The benefits and challenges of using systematic reviews in
The challenges and limitations of doing this are discussed in Duvendack et al. ... Beyond these general reflections, we offer five specific conclusions: first, applying systematic review principles to a literature review is highly valuable as it increases breadth, improves transparency and emphasises the importance of empirical evidence over ...
Systematic reviews: Brief overview of methods, limitations, and resources
REVIEW State of Reviews Systematic reviews: Brief overview of methods, limitations, and resources Jacqueline K. Owens Correspondence Jacqueline K. Owens. Email: [email protected] INTRODUCTION Systematic reviews are a valuable resource for nurses in academia and practice.1‐3 Well done systematic reviews, which include but
Limitations of the literature
This work typically appears in a separate chapter in the thesis or dissertation under the title Literature Review. To help you, here are 10 typical examples often used to identify limitations in a particular body of literature—with a caveat—using education examples. 1. The sample size is too small and therefore not representative of the ...
The Literature Review: A Foundation for High-Quality Medical Education
Purpose and Importance of the Literature Review. An understanding of the current literature is critical for all phases of a research study. Lingard 9 recently invoked the "journal-as-conversation" metaphor as a way of understanding how one's research fits into the larger medical education conversation. As she described it: "Imagine yourself joining a conversation at a social event.
Full article: Children's Perceptions and Experiences of Their
When conducting a literature review, there is always a risk of missing relevant articles. ... A possible limitation could be including studies older than 15 years (n = 5) since the importance of involving children as next of kin in the care have been noticed in legislation, regulations and routines during the last decades. On the other hand in ...
Health profession education hackathons: a scoping review of current
This had some limitations when it came to collecting eligible articles. Since this review extracted only published research, there are educational interventions that are reported at conferences but have not yet been published in the literature. The moderate quality of full-text studies is indeed a limitation of this study.
Sustainability
Over the last two decades, numerous studies have highlighted the significance of integrating sustainability into higher education. Consequently, there has been a growing interest in the literature on engineering education for sustainable development, emphasizing the inclusion of this concept within engineering curricula and recognizing the pivotal role that engineers play in achieving the ...
Review of evidence for treatments of acute non arteritic anterior
To review treatment modalities that have been studied in acute non arteritic anterior ischemic optic neuropathy (NAION). We performed a comprehensive literature search of English language ...
Chapter 9 Methods for Literature Reviews
9.3. Types of Review Articles and Brief Illustrations. EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic.
Comprehensive analysis of digital twins in smart cities: a ...
Language limitation: Articles published in languages other than English were excluded to maintain consistency and comprehensibility in the analysis. ... The challenges outlined are meticulously identified through an extensive review of literature and insights from field and industry experts, signifying their significance in the successful ...
Deep learning techniques for solar tracking systems: A systematic
Thus, this systematic literature review aims to provide an overview of the state-of-the-art of DL techniques for solar tracking systems. It examines dataset usage, preprocessing methods, feature engineering methods, DL algorithms, and performance metrics used in the identified studies. ... traditional ML models have limitations in processing ...
Cancer Vaccine Therapeutics: Limitations and Effectiveness—A Literature
7. Limitations of Cancer Vaccines. Despite the immense potential for cancer vaccination, various limitations hinder the widespread application and optimal efficacy of these vaccine types. The following presents an overview of the limitations associated with each vaccine category, shedding light on the challenges they pose in clinical translation.

Neal Haddaway

About the author

Leave a Comment Cancel reply

Related Posts

“But I’m not ready!” Common barriers to writing and how to overcome them

“Remember a condition of academic writing is that we expose ourselves to critique” – 15 steps to revising journal articles

A simple guide to ethical co-authorship

How common is academic plagiarism?

Eight problems with literature reviews and how to fix them

Literature Reviews

What is a Literature Review?

Purpose of a Literature Review

Limitations of a Literature Review

Organizing Your Social Sciences Research Paper

Importance of...

Descriptions of Possible Limitations

Structure and Writing Style

Writing Tip

Another Writing Tip

Yet Another Writing Tip

Writing a Literature Review

Author: M Rahman

Related Posts

What is a Literature Review? How to Write It (with Examples)

Table of Contents

How to write a good literature review

What is a literature review?

What is the purpose of literature review?

Literature review example

Literature Review on Climate Change Impacts on Biodiversity:

a. Habitat Loss and Species Extinction:

b. Range Shifts and Phenological Changes:

c. Ocean Acidification and Coral Reefs:

d. Adaptive Strategies and Conservation Efforts:

Conducting a literature review

Choose a Topic and Define the Research Question:

Decide on the Scope of Your Review:

Select Databases for Searches:

Conduct Searches and Keep Track:

Review the Literature:

Organize and Write Your Literature Review:

How to write a literature review faster with Paperpal?

Frequently asked questions

Related Reads:

6 Tips for Post-Doc Researchers to Take Their Career to the Next Level

Limitations of the literature – A guide to a sticky problem

The upsides to a pandemic: from jobs to teaching, it’s not all bad news

Leave a Comment Cancel Reply

Health profession education hackathons: a scoping review of current trends and best practices

Introduction

Methods and materials

Search strategies

Inclusion and exclusion criteria

Study identification

Quality assessment of the studies

Data extraction and synthesis

Description of the interventions and expected outcomes

Geographical context

Type and number of participants

The focus of the intervention

Evaluation methodology

Aspects of hacking in health profession education

Adaptations

Suggestions for the future

Lessons learned

Data availability

Acknowledgements

Author information

Contributions

Corresponding author

Ethics declarations

Consent for publication

Additional information

Electronic supplementary material

Supplementary Material 1

About this article

Share this article

BMC Medical Education

Review of evidence for treatments of acute non arteritic anterior ischemic optic neuropathy

Conclusions