method literature research

Research Methods

• Getting Started
• Literature Review Research
• Research Design
• Research Design By Discipline
• SAGE Research Methods
• Teaching with SAGE Research Methods

Literature Review

• What is a Literature Review?
• What is NOT a Literature Review?
• Purposes of a Literature Review
• Types of Literature Reviews
• Literature Reviews vs. Systematic Reviews
• Systematic vs. Meta-Analysis

Literature Review  is a comprehensive survey of the works published in a particular field of study or line of research, usually over a specific period of time, in the form of an in-depth, critical bibliographic essay or annotated list in which attention is drawn to the most significant works.

Also, we can define a literature review as the collected body of scholarly works related to a topic:

• Summarizes and analyzes previous research relevant to a topic
• Includes scholarly books and articles published in academic journals
• Can be an specific scholarly paper or a section in a research paper

The objective of a Literature Review is to find previous published scholarly works relevant to an specific topic

• Help gather ideas or information
• Keep up to date in current trends and findings
• Help develop new questions

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Helps focus your own research questions or problems
• Discovers relationships between research studies/ideas.
• Suggests unexplored ideas or populations
• Identifies major themes, concepts, and researchers on a topic.
• Tests assumptions; may help counter preconceived ideas and remove unconscious bias.
• Identifies critical gaps, points of disagreement, or potentially flawed methodology or theoretical approaches.
• Indicates potential directions for future research.

All content in this section is from Literature Review Research from Old Dominion University 

Keep in mind the following, a literature review is NOT:

Not an essay 

Not an annotated bibliography  in which you summarize each article that you have reviewed.  A literature review goes beyond basic summarizing to focus on the critical analysis of the reviewed works and their relationship to your research question.

Not a research paper   where you select resources to support one side of an issue versus another.  A lit review should explain and consider all sides of an argument in order to avoid bias, and areas of agreement and disagreement should be highlighted.

A literature review serves several purposes. For example, it

• provides thorough knowledge of previous studies; introduces seminal works.
• helps focus one’s own research topic.
• identifies a conceptual framework for one’s own research questions or problems; indicates potential directions for future research.
• suggests previously unused or underused methodologies, designs, quantitative and qualitative strategies.
• identifies gaps in previous studies; identifies flawed methodologies and/or theoretical approaches; avoids replication of mistakes.
• helps the researcher avoid repetition of earlier research.
• suggests unexplored populations.
• determines whether past studies agree or disagree; identifies controversy in the literature.
• tests assumptions; may help counter preconceived ideas and remove unconscious bias.

As Kennedy (2007) notes*, it is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the original studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally that become part of the lore of field. In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews.

Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are several approaches to how they can be done, depending upon the type of analysis underpinning your study. Listed below are definitions of types of literature reviews:

Argumentative Review      This form examines literature selectively in order to support or refute an argument, deeply imbedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to to make summary claims of the sort found in systematic reviews.

Integrative Review      Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication.

Historical Review      Few things rest in isolation from historical precedent. Historical reviews are focused on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review      A review does not always focus on what someone said [content], but how they said it [method of analysis]. This approach provides a framework of understanding at different levels (i.e. those of theory, substantive fields, research approaches and data collection and analysis techniques), enables researchers to draw on a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection and data analysis, and helps highlight many ethical issues which we should be aware of and consider as we go through our study.

Systematic Review      This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyse data from the studies that are included in the review. Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?"

Theoretical Review      The purpose of this form is to concretely examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review help establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

* Kennedy, Mary M. "Defining a Literature."  Educational Researcher  36 (April 2007): 139-147.

All content in this section is from The Literature Review created by Dr. Robert Larabee USC

Robinson, P. and Lowe, J. (2015),  Literature reviews vs systematic reviews.  Australian and New Zealand Journal of Public Health, 39: 103-103. doi: 10.1111/1753-6405.12393

What's in the name? The difference between a Systematic Review and a Literature Review, and why it matters . By Lynn Kysh from University of Southern California

Systematic review or meta-analysis?

A  systematic review  answers a defined research question by collecting and summarizing all empirical evidence that fits pre-specified eligibility criteria.

A  meta-analysis  is the use of statistical methods to summarize the results of these studies.

Systematic reviews, just like other research articles, can be of varying quality. They are a significant piece of work (the Centre for Reviews and Dissemination at York estimates that a team will take 9-24 months), and to be useful to other researchers and practitioners they should have:

• clearly stated objectives with pre-defined eligibility criteria for studies
• explicit, reproducible methodology
• a systematic search that attempts to identify all studies
• assessment of the validity of the findings of the included studies (e.g. risk of bias)
• systematic presentation, and synthesis, of the characteristics and findings of the included studies

Not all systematic reviews contain meta-analysis. 

Meta-analysis is the use of statistical methods to summarize the results of independent studies. By combining information from all relevant studies, meta-analysis can provide more precise estimates of the effects of health care than those derived from the individual studies included within a review.  More information on meta-analyses can be found in  Cochrane Handbook, Chapter 9 .

A meta-analysis goes beyond critique and integration and conducts secondary statistical analysis on the outcomes of similar studies.  It is a systematic review that uses quantitative methods to synthesize and summarize the results.

An advantage of a meta-analysis is the ability to be completely objective in evaluating research findings.  Not all topics, however, have sufficient research evidence to allow a meta-analysis to be conducted.  In that case, an integrative review is an appropriate strategy. 

Some of the content in this section is from Systematic reviews and meta-analyses: step by step guide created by Kate McAllister.

• << Previous: Getting Started
• Next: Research Design >>
• Last Updated: Aug 21, 2023 4:07 PM
• URL: https://guides.lib.udel.edu/researchmethods

Harvey Cushing/John Hay Whitney Medical Library

• Collections
• Research Help

YSN Doctoral Programs: Steps in Conducting a Literature Review

• Biomedical Databases
• Global (Public Health) Databases
• Soc. Sci., History, and Law Databases
• Grey Literature
• Trials Registers
• Data and Statistics
• Public Policy
• Google Tips
• Recommended Books
• Steps in Conducting a Literature Review

What is a literature review?

A literature review is an integrated analysis -- not just a summary-- of scholarly writings and other relevant evidence related directly to your research question.  That is, it represents a synthesis of the evidence that provides background information on your topic and shows a association between the evidence and your research question.

A literature review may be a stand alone work or the introduction to a larger research paper, depending on the assignment.  Rely heavily on the guidelines your instructor has given you.

Why is it important?

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Discovers relationships between research studies/ideas.
• Identifies major themes, concepts, and researchers on a topic.
• Identifies critical gaps and points of disagreement.
• Discusses further research questions that logically come out of the previous studies.

APA7 Style resources

APA Style Blog - for those harder to find answers

1. Choose a topic. Define your research question.

Your literature review should be guided by your central research question.  The literature represents background and research developments related to a specific research question, interpreted and analyzed by you in a synthesized way.

• Make sure your research question is not too broad or too narrow.  Is it manageable?
• Begin writing down terms that are related to your question. These will be useful for searches later.
• If you have the opportunity, discuss your topic with your professor and your class mates.

2. Decide on the scope of your review

How many studies do you need to look at? How comprehensive should it be? How many years should it cover? 

• This may depend on your assignment.  How many sources does the assignment require?

3. Select the databases you will use to conduct your searches.

Make a list of the databases you will search. 

Where to find databases:

• use the tabs on this guide
• Find other databases in the Nursing Information Resources web page
• More on the Medical Library web page
• ... and more on the Yale University Library web page

4. Conduct your searches to find the evidence. Keep track of your searches.

• Use the key words in your question, as well as synonyms for those words, as terms in your search. Use the database tutorials for help.
• Save the searches in the databases. This saves time when you want to redo, or modify, the searches. It is also helpful to use as a guide is the searches are not finding any useful results.
• Review the abstracts of research studies carefully. This will save you time.
• Use the bibliographies and references of research studies you find to locate others.
• Check with your professor, or a subject expert in the field, if you are missing any key works in the field.
• Ask your librarian for help at any time.
• Use a citation manager, such as EndNote as the repository for your citations. See the EndNote tutorials for help.

Review the literature

Some questions to help you analyze the research:

• What was the research question of the study you are reviewing? What were the authors trying to discover?
• Was the research funded by a source that could influence the findings?
• What were the research methodologies? Analyze its literature review, the samples and variables used, the results, and the conclusions.
• Does the research seem to be complete? Could it have been conducted more soundly? What further questions does it raise?
• If there are conflicting studies, why do you think that is?
• How are the authors viewed in the field? Has this study been cited? If so, how has it been analyzed?

Tips: 

• Review the abstracts carefully.  
• Keep careful notes so that you may track your thought processes during the research process.
• Create a matrix of the studies for easy analysis, and synthesis, across all of the studies.
• << Previous: Recommended Books
• Last Updated: Jan 4, 2024 10:52 AM
• URL: https://guides.library.yale.edu/YSNDoctoral

Research Methods and Design

• Action Research
• Case Study Design

Literature Review

• Quantitative Research Methods
• Qualitative Research Methods
• Mixed Methods Study
• Indigenous Research and Ethics This link opens in a new window
• Identifying Empirical Research Articles This link opens in a new window
• Research Ethics and Quality
• Data Literacy
• Get Help with Writing Assignments

A literature review is a discussion of the literature (aka. the "research" or "scholarship") surrounding a certain topic. A good literature review doesn't simply summarize the existing material, but provides thoughtful synthesis and analysis. The purpose of a literature review is to orient your own work within an existing body of knowledge. A literature review may be written as a standalone piece or be included in a larger body of work.

You can read more about literature reviews, what they entail, and how to write one, using the resources below. 

Am I the only one struggling to write a literature review?

Dr. Zina O'Leary explains the misconceptions and struggles students often have with writing a literature review. She also provides step-by-step guidance on writing a persuasive literature review.

An Introduction to Literature Reviews

Dr. Eric Jensen, Professor of Sociology at the University of Warwick, and Dr. Charles Laurie, Director of Research at Verisk Maplecroft, explain how to write a literature review, and why researchers need to do so. Literature reviews can be stand-alone research or part of a larger project. They communicate the state of academic knowledge on a given topic, specifically detailing what is still unknown.

This is the first video in a whole series about literature reviews. You can find the rest of the series in our SAGE database, Research Methods:

Videos covering research methods and statistics

Identify Themes and Gaps in Literature (with real examples) | Scribbr

Finding connections between sources is key to organizing the arguments and structure of a good literature review. In this video, you'll learn how to identify themes, debates, and gaps between sources, using examples from real papers.

4 Tips for Writing a Literature Review's Intro, Body, and Conclusion | Scribbr

While each review will be unique in its structure--based on both the existing body of both literature and the overall goals of your own paper, dissertation, or research--this video from Scribbr does a good job simplifying the goals of writing a literature review for those who are new to the process. In this video, you’ll learn what to include in each section, as well as 4 tips for the main body illustrated with an example.

• Literature Review This chapter in SAGE's Encyclopedia of Research Design describes the types of literature reviews and scientific standards for conducting literature reviews.
• UNC Writing Center: Literature Reviews This handout from the Writing Center at UNC will explain what literature reviews are and offer insights into the form and construction of literature reviews in the humanities, social sciences, and sciences.
• Purdue OWL: Writing a Literature Review The overview of literature reviews comes from Purdue's Online Writing Lab. It explains the basic why, what, and how of writing a literature review.

Organizational Tools for Literature Reviews

One of the most daunting aspects of writing a literature review is organizing your research. There are a variety of strategies that you can use to help you in this task. We've highlighted just a few ways writers keep track of all that information! You can use a combination of these tools or come up with your own organizational process. The key is choosing something that works with your own learning style.

Citation Managers

Citation managers are great tools, in general, for organizing research, but can be especially helpful when writing a literature review. You can keep all of your research in one place, take notes, and organize your materials into different folders or categories. Read more about citations managers here:

• Manage Citations & Sources

Concept Mapping

Some writers use concept mapping (sometimes called flow or bubble charts or "mind maps") to help them visualize the ways in which the research they found connects.

There is no right or wrong way to make a concept map. There are a variety of online tools that can help you create a concept map or you can simply put pen to paper. To read more about concept mapping, take a look at the following help guides:

• Using Concept Maps From Williams College's guide, Literature Review: A Self-guided Tutorial

Synthesis Matrix

A synthesis matrix is is a chart you can use to help you organize your research into thematic categories. By organizing your research into a matrix, like the examples below, can help you visualize the ways in which your sources connect. 

• Walden University Writing Center: Literature Review Matrix Find a variety of literature review matrix examples and templates from Walden University.
• Writing A Literature Review and Using a Synthesis Matrix An example synthesis matrix created by NC State University Writing and Speaking Tutorial Service Tutors. If you would like a copy of this synthesis matrix in a different format, like a Word document, please ask a librarian. CC-BY-SA 3.0
• << Previous: Case Study Design
• Next: Quantitative Research Methods >>
• Last Updated: Feb 6, 2024 9:20 AM

CityU Home - CityU Catalog

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

• An introductory paragraph that explains what your working topic and thesis is
• A forecast of key topics or texts that will appear in the review
• Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
• Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically Evaluate: Mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

• Summarize the key findings you have taken from the literature and emphasize their significance
• Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

• Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
• Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
• Qualitative versus quantitative research
• Empirical versus theoretical scholarship
• Divide the research by sociological, historical, or cultural sources
• Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

• It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
• Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
• Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
• Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 5. The Literature Review
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

• Give a new interpretation of old material or combine new with old interpretations,
• Trace the intellectual progression of the field, including major debates,
• Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
• Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

• Place each work in the context of its contribution to understanding the research problem being studied.
• Describe the relationship of each work to the others under consideration.
• Identify new ways to interpret prior research.
• Reveal any gaps that exist in the literature.
• Resolve conflicts amongst seemingly contradictory previous studies.
• Identify areas of prior scholarship to prevent duplication of effort.
• Point the way in fulfilling a need for additional research.
• Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE : Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

• An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
• Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
• An explanation of how each work is similar to and how it varies from the others,
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

• Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
• Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
• Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
• Persuasiveness -- which of the author's theses are most convincing or least convincing?
• Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

• Current Situation : Information necessary to understand the current topic or focus of the literature review.
• Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
• History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
• Standards : Description of the way in which you present your information.
• Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

• Sources in your literature review do not clearly relate to the research problem;
• You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
• Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
• Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
• Does not describe the search procedures that were used in identifying the literature to review;
• Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
• Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

• How are they organizing their ideas?
• What methods have they used to study the problem?
• What theories have been used to explain, predict, or understand their research problem?
• What sources have they cited to support their conclusions?
• How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

• Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
• Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
• Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

• << Previous: Theoretical Framework
• Next: Citation Tracking >>
• Last Updated: Apr 22, 2024 9:12 AM
• URL: https://libguides.usc.edu/writingguide

Archer Library

Qualitative research: literature review .

• Archer Library This link opens in a new window
• Schedule a Reference Appointment This link opens in a new window
• Qualitative Research Handout This link opens in a new window
• Locating Books
• ebook Collections This link opens in a new window
• A to Z Database List This link opens in a new window
• Research & Stats
• Literature Review Resources
• Citation & Reference

Exploring the literature review 

Literature review model: 6 steps.

Adapted from The Literature Review , Machi & McEvoy (2009, p. 13).

Your Literature Review

Step 2: search, boolean search strategies, search limiters, ★ ebsco & google drive.

1. Select a Topic

"All research begins with curiosity" (Machi & McEvoy, 2009, p. 14)

Selection of a topic, and fully defined research interest and question, is supervised (and approved) by your professor. Tips for crafting your topic include:

• Be specific. Take time to define your interest.
• Topic Focus. Fully describe and sufficiently narrow the focus for research.
• Academic Discipline. Learn more about your area of research & refine the scope.
• Avoid Bias. Be aware of bias that you (as a researcher) may have.
• Document your research. Use Google Docs to track your research process.
• Research apps. Consider using Evernote or Zotero to track your research.

Consider Purpose

What will your topic and research address?

In The Literature Review: A Step-by-Step Guide for Students , Ridley presents that literature reviews serve several purposes (2008, p. 16-17).  Included are the following points:

• Historical background for the research;
• Overview of current field provided by "contemporary debates, issues, and questions;"
• Theories and concepts related to your research;
• Introduce "relevant terminology" - or academic language - being used it the field;
• Connect to existing research - does your work "extend or challenge [this] or address a gap;" 
• Provide "supporting evidence for a practical problem or issue" that your research addresses.

★ Schedule a research appointment

At this point in your literature review, take time to meet with a librarian. Why? Understanding the subject terminology used in databases can be challenging. Archer Librarians can help you structure a search, preparing you for step two. How? Contact a librarian directly or use the online form to schedule an appointment. Details are provided in the adjacent Schedule an Appointment box.

2. Search the Literature

Collect & Select Data: Preview, select, and organize

AU Library is your go-to resource for this step in your literature review process. The literature search will include books and ebooks, scholarly and practitioner journals, theses and dissertations, and indexes. You may also choose to include web sites, blogs, open access resources, and newspapers. This library guide provides access to resources needed to complete a literature review.

Books & eBooks: Archer Library & OhioLINK

Databases: scholarly & practitioner journals.

Review the Library Databases tab on this library guide, it provides links to recommended databases for Education & Psychology, Business, and General & Social Sciences.

Expand your journal search; a complete listing of available AU Library and OhioLINK databases is available on the Databases  A to Z list . Search the database by subject, type, name, or do use the search box for a general title search. The A to Z list also includes open access resources and select internet sites.

Databases: Theses & Dissertations

Review the Library Databases tab on this guide, it includes Theses & Dissertation resources. AU library also has AU student authored theses and dissertations available in print, search the library catalog for these titles.

Did you know? If you are looking for particular chapters within a dissertation that is not fully available online, it is possible to submit an ILL article request . Do this instead of requesting the entire dissertation.

Newspapers:  Databases & Internet

Consider current literature in your academic field. AU Library's database collection includes The Chronicle of Higher Education and The Wall Street Journal .  The Internet Resources tab in this guide provides links to newspapers and online journals such as Inside Higher Ed , COABE Journal , and Education Week .

Search Strategies & Boolean Operators

There are three basic boolean operators:  AND, OR, and NOT.

Used with your search terms, boolean operators will either expand or limit results. What purpose do they serve? They help to define the relationship between your search terms. For example, using the operator AND will combine the terms expanding the search. When searching some databases, and Google, the operator AND may be implied.

Overview of boolean terms

About the example: Boolean searches were conducted on November 4, 2019; result numbers may vary at a later date. No additional database limiters were set to further narrow search returns.

Database Search Limiters

Database strategies for targeted search results.

Most databases include limiters, or additional parameters, you may use to strategically focus search results.  EBSCO databases, such as Education Research Complete & Academic Search Complete provide options to:

• Limit results to full text;
• Limit results to scholarly journals, and reference available;
• Select results source type to journals, magazines, conference papers, reviews, and newspapers
• Publication date

Keep in mind that these tools are defined as limiters for a reason; adding them to a search will limit the number of results returned.  This can be a double-edged sword.  How? 

• If limiting results to full-text only, you may miss an important piece of research that could change the direction of your research. Interlibrary loan is available to students, free of charge. Request articles that are not available in full-text; they will be sent to you via email.
• If narrowing publication date, you may eliminate significant historical - or recent - research conducted on your topic.
• Limiting resource type to a specific type of material may cause bias in the research results.

Use limiters with care. When starting a search, consider opting out of limiters until the initial literature screening is complete. The second or third time through your research may be the ideal time to focus on specific time periods or material (scholarly vs newspaper).

★ Truncating Search Terms

Expanding your search term at the root.

Truncating is often referred to as 'wildcard' searching. Databases may have their own specific wildcard elements however, the most commonly used are the asterisk (*) or question mark (?).  When used within your search. they will expand returned results.

Asterisk (*) Wildcard

Using the asterisk wildcard will return varied spellings of the truncated word. In the following example, the search term education was truncated after the letter "t."

Explore these database help pages for additional information on crafting search terms.

• EBSCO Connect: Searching with Wildcards and Truncation Symbols
• EBSCO Connect: Searching with Boolean Operators
• EBSCO Connect: EBSCOhost Search Tips
• EBSCO Connect: Basic Searching with EBSCO
• ProQuest Help: Search Tips
• ERIC: How does ERIC search work?

★ EBSCO Databases & Google Drive

Tips for saving research directly to Google drive.

Researching in an EBSCO database?

It is possible to save articles (PDF and HTML) and abstracts in EBSCOhost databases directly to Google drive. Select the Google Drive icon, authenticate using a Google account, and an EBSCO folder will be created in your account. This is a great option for managing your research. If documenting your research in a Google Doc, consider linking the information to actual articles saved in drive.

EBSCO Databases & Google Drive

EBSCOHost Databases & Google Drive: Managing your Research

This video features an overview of how to use Google Drive with EBSCO databases to help manage your research. It presents information for connecting an active Google account to EBSCO and steps needed to provide permission for EBSCO to manage a folder in Drive.

About the Video:  Closed captioning is available, select CC from the video menu.  If you need to review a specific area on the video, view on YouTube and expand the video description for access to topic time stamps.  A video transcript is provided below.

• EBSCOhost Databases & Google Scholar

Defining Literature Review

What is a literature review.

A definition from the Online Dictionary for Library and Information Sciences .

A literature review is "a comprehensive survey of the works published in a particular field of study or line of research, usually over a specific period of time, in the form of an in-depth, critical bibliographic essay or annotated list in which attention is drawn to the most significant works" (Reitz, 2014). 

A systemic review is "a literature review focused on a specific research question, which uses explicit methods to minimize bias in the identification, appraisal, selection, and synthesis of all the high-quality evidence pertinent to the question" (Reitz, 2014).

Recommended Reading

About this page

EBSCO Connect [Discovery and Search]. (2022). Searching with boolean operators. Retrieved May, 3, 2022 from https://connect.ebsco.com/s/?language=en_US

EBSCO Connect [Discover and Search]. (2022). Searching with wildcards and truncation symbols. Retrieved May 3, 2022; https://connect.ebsco.com/s/?language=en_US

Machi, L.A. & McEvoy, B.T. (2009). The literature review . Thousand Oaks, CA: Corwin Press: 

Reitz, J.M. (2014). Online dictionary for library and information science. ABC-CLIO, Libraries Unlimited . Retrieved from https://www.abc-clio.com/ODLIS/odlis_A.aspx

Ridley, D. (2008). The literature review: A step-by-step guide for students . Thousand Oaks, CA: Sage Publications, Inc.

Archer Librarians

Schedule an appointment.

Contact a librarian directly (email), or submit a request form. If you have worked with someone before, you can request them on the form.

• ★ Archer Library Help • Online Reqest Form
• Carrie Halquist • Reference & Instruction
• Jessica Byers • Reference & Curation
• Don Reams • Corrections Education & Reference
• Diane Schrecker • Education & Head of the IRC
• Tanaya Silcox • Technical Services & Business
• Sarah Thomas • Acquisitions & ATS Librarian
• << Previous: Research & Stats
• Next: Literature Review Resources >>
• Last Updated: Nov 17, 2023 7:46 AM
• URL: https://libguides.ashland.edu/qualitative

Archer Library • Ashland University © Copyright 2023. An Equal Opportunity/Equal Access Institution.

Chapter Four: Theory, Methodologies, Methods, and Evidence

Research Methods

You are viewing the first edition of this textbook. a second edition is available – please visit the latest edition for updated information..

This page discusses the following topics:

Research Goals

Research method types.

Before discussing research   methods , we need to distinguish them from  methodologies  and  research skills . Methodologies, linked to literary theories, are tools and lines of investigation: sets of practices and propositions about texts and the world. Researchers using Marxist literary criticism will adopt methodologies that look to material forces like labor, ownership, and technology to understand literature and its relationship to the world. They will also seek to understand authors not as inspired geniuses but as people whose lives and work are shaped by social forces.

Example: Critical Race Theory Methodologies

Critical Race Theory may use a variety of methodologies, including

• Interest convergence: investigating whether marginalized groups only achieve progress when dominant groups benefit as well
• Intersectional theory: investigating how multiple factors of advantage and disadvantage around race, gender, ethnicity, religion, etc. operate together in complex ways
• Radical critique of the law: investigating how the law has historically been used to marginalize particular groups, such as black people, while recognizing that legal efforts are important to achieve emancipation and civil rights
• Social constructivism: investigating how race is socially constructed (rather than biologically grounded)
• Standpoint epistemology: investigating how knowledge relates to social position
• Structural determinism: investigating how structures of thought and of organizations determine social outcomes

To identify appropriate methodologies, you will need to research your chosen theory and gather what methodologies are associated with it. For the most part, we can’t assume that there are “one size fits all” methodologies.

Research skills are about how you handle materials such as library search engines, citation management programs, special collections materials, and so on.

Research methods  are about where and how you get answers to your research questions. Are you conducting interviews? Visiting archives? Doing close readings? Reviewing scholarship? You will need to choose which methods are most appropriate to use in your research and you need to gain some knowledge about how to use these methods. In other words, you need to do some research into research methods!

Your choice of research method depends on the kind of questions you are asking. For example, if you want to understand how an author progressed through several drafts to arrive at a final manuscript, you may need to do archival research. If you want to understand why a particular literary work became a bestseller, you may need to do audience research. If you want to know why a contemporary author wrote a particular work, you may need to do interviews. Usually literary research involves a combination of methods such as  archival research ,  discourse analysis , and  qualitative research  methods.

Literary research methods tend to differ from research methods in the hard sciences (such as physics and chemistry). Science research must present results that are reproducible, while literary research rarely does (though it must still present evidence for its claims). Literary research often deals with questions of meaning, social conventions, representations of lived experience, and aesthetic effects; these are questions that reward dialogue and different perspectives rather than one great experiment that settles the issue. In literary research, we might get many valuable answers even though they are quite different from one another. Also in literary research, we usually have some room to speculate about answers, but our claims have to be plausible (believable) and our argument comprehensive (meaning we don’t overlook evidence that would alter our argument significantly if it were known).

A literary researcher might select the following:

Theory: Critical Race Theory

Methodology: Social Constructivism

Method: Scholarly

Skills: Search engines, citation management

Wendy Belcher, in  Writing Your Journal Article in 12 Weeks , identifies two main approaches to understanding literary works: looking at a text by itself (associated with New Criticism ) and looking at texts as they connect to society (associated with Cultural Studies ). The goal of New Criticism is to bring the reader further into the text. The goal of Cultural Studies is to bring the reader into the network of discourses that surround and pass through the text. Other approaches, such as Ecocriticism, relate literary texts to the Sciences (as well as to the Humanities).

The New Critics, starting in the 1940s,  focused on meaning within the text itself, using a method they called “ close reading .” The text itself becomes e vidence for a particular reading. Using this approach, you should summarize the literary work briefly and q uote particularly meaningful passages, being sure to introduce quotes and then interpret them (never let them stand alone). Make connections within the work; a sk  “why” and “how” the various parts of the text relate to each other.

Cultural Studies critics see all texts  as connected to society; the critic  therefore has to connect a text to at least one political or social issue. How and why does  the text reproduce particular knowledge systems (known as discourses) and how do these knowledge systems relate to issues of power within the society? Who speaks and when? Answering these questions helps your reader understand the text in context. Cultural contexts can include the treatment of gender (Feminist, Queer), class (Marxist), nationality, race, religion, or any other area of human society.

Other approaches, such as psychoanalytic literary criticism , look at literary texts to better understand human psychology. A psychoanalytic reading can focus on a character, the author, the reader, or on society in general. Ecocriticism  look at human understandings of nature in literary texts.

We select our research methods based on the kinds of things we want to know. For example, we may be studying the relationship between literature and society, between author and text, or the status of a work in the literary canon. We may want to know about a work’s form, genre, or thematics. We may want to know about the audience’s reading and reception, or about methods for teaching literature in schools.

Below are a few research methods and their descriptions. You may need to consult with your instructor about which ones are most appropriate for your project. The first list covers methods most students use in their work. The second list covers methods more commonly used by advanced researchers. Even if you will not be using methods from this second list in your research project, you may read about these research methods in the scholarship you find.

Most commonly used undergraduate research methods:

• Scholarship Methods:  Studies the body of scholarship written about a particular author, literary work, historical period, literary movement, genre, theme, theory, or method.
• Textual Analysis Methods:  Used for close readings of literary texts, these methods also rely on literary theory and background information to support the reading.
• Biographical Methods:  Used to study the life of the author to better understand their work and times, these methods involve reading biographies and autobiographies about the author, and may also include research into private papers, correspondence, and interviews.
• Discourse Analysis Methods:  Studies language patterns to reveal ideology and social relations of power. This research involves the study of institutions, social groups, and social movements to understand how people in various settings use language to represent the world to themselves and others. Literary works may present complex mixtures of discourses which the characters (and readers) have to navigate.
• Creative Writing Methods:  A literary re-working of another literary text, creative writing research is used to better understand a literary work by investigating its language, formal structures, composition methods, themes, and so on. For instance, a creative research project may retell a story from a minor character’s perspective to reveal an alternative reading of events. To qualify as research, a creative research project is usually combined with a piece of theoretical writing that explains and justifies the work.

Methods used more often by advanced researchers:

• Archival Methods: Usually involves trips to special collections where original papers are kept. In these archives are many unpublished materials such as diaries, letters, photographs, ledgers, and so on. These materials can offer us invaluable insight into the life of an author, the development of a literary work, or the society in which the author lived. There are at least three major archives of James Baldwin’s papers: The Smithsonian , Yale , and The New York Public Library . Descriptions of such materials are often available online, but the materials themselves are typically stored in boxes at the archive.
• Computational Methods:  Used for statistical analysis of texts such as studies of the popularity and meaning of particular words in literature over time.
• Ethnographic Methods:  Studies groups of people and their interactions with literary works, for instance in educational institutions, in reading groups (such as book clubs), and in fan networks. This approach may involve interviews and visits to places (including online communities) where people interact with literary works. Note: before you begin such work, you must have  Institutional Review Board (IRB)  approval “to protect the rights and welfare of human participants involved in research.”
• Visual Methods:  Studies the visual qualities of literary works. Some literary works, such as illuminated manuscripts, children’s literature, and graphic novels, present a complex interplay of text and image. Even works without illustrations can be studied for their use of typography, layout, and other visual features.

Regardless of the method(s) you choose, you will need to learn how to apply them to your work and how to carry them out successfully. For example, you should know that many archives do not allow you to bring pens (you can use pencils) and you may not be allowed to bring bags into the archives. You will need to keep a record of which documents you consult and their location (box number, etc.) in the archives. If you are unsure how to use a particular method, please consult a book about it. [1] Also, ask for the advice of trained researchers such as your instructor or a research librarian.

• What research method(s) will you be using for your paper? Why did you make this method selection over other methods? If you haven’t made a selection yet, which methods are you considering?
• What specific methodological approaches are you most interested in exploring in relation to the chosen literary work?
• What is your plan for researching your method(s) and its major approaches?
• What was the most important lesson you learned from this page? What point was confusing or difficult to understand?

Write your answers in a webcourse discussion page.

• Introduction to Research Methods: A Practical Guide for Anyone Undertaking a Research Project  by Catherine, Dr. Dawson
• Practical Research Methods: A User-Friendly Guide to Mastering Research Techniques and Projects  by Catherine Dawson
• Qualitative Inquiry and Research Design: Choosing Among Five Approaches  by John W. Creswell  Cheryl N. Poth
• Qualitative Research Evaluation Methods: Integrating Theory and Practice  by Michael Quinn Patton
• Research Design: Qualitative, Quantitative, and Mixed Methods Approaches  by John W. Creswell  J. David Creswell
• Research Methodology: A Step-by-Step Guide for Beginners  by Ranjit Kumar
• Research Methodology: Methods and Techniques  by C.R. Kothari

Strategies for Conducting Literary Research Copyright © 2021 by Barry Mauer & John Venecek is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Dudley RA, Frolich A, Robinowitz DL, et al. Strategies To Support Quality-based Purchasing: A Review of the Evidence. Rockville (MD): Agency for Healthcare Research and Quality (US); 2004 Jul. (Technical Reviews, No. 10.)

Strategies To Support Quality-based Purchasing: A Review of the Evidence.

2 methods for literature search.

• Technical Expert Advisory Panel

For advice on the scope of the project, refinement of the key questions, and preparation of this technical review, we consulted technical experts in the following fields: employer purchasing strategies, provider performance assessment, consumer use of report cards and consumer preferences for health care information, risk adjustment, and economics. (See Appendix A , available at www.ahrq.gov/clinic/epcindex.htm .)

• Target Audiences and Population

The decisionmakers addressed in this technical review are purchasers (both private purchasers such as employers and public purchasers such as the Centers for Medicare & Medicaid Services and State Medicaid programs), executives in health plans that must negotiate incentive arrangements with provider organizations or individual providers, executives in provider organizations that must negotiate incentive arrangements with providers, public health officials and other organizations interested in creating health care performance reports for public release, and policymakers. For the purpose of this report, provider organizations include all clinical health providers such as physicians, nurses, and hospitals. Public health officials and policymakers include those at the local, State, Federal, and international levels.

The ultimate target population of this report is the U.S. population at risk for morbidity or mortality resulting from quality problems in the provision of health care. We are interested in QBP strategies that affect the entire U.S. population—all members of which are at risk for receiving poor quality care—including those of all racial and ethnic backgrounds, all ages, and both genders.

• Key Questions

We developed the key questions in collaboration with AHRQ, the Alliance (the nominating partner), and our Technical Expert Panel. The goal of these discussions was to identify the issues purchasers interested in QBP faced so that, if the available research offered conclusions about these aspects of QBP, the various stakeholders would be in a better position to select optimal approaches to QBP.

The key questions for which literature, ongoing research, or results from analyses were sought in preparation of this report were:

• What is the evidence on the extent to which health plans and employers use incentives to improve quality and efficiency?
• Does the use of financial incentives for quality and efficiency actually increase the probability that patients receive high quality, efficient care?
• The basis of the incentive (structure, process, outcome)?
• The nature of the incentive (bonus, penalties or holdback, tiering or patient steerage/referral)?
• To whom the incentive is targeted (plan vs. provider group vs. individual provider)?
• The payer of the incentive (purchaser vs. plan vs. medical group)?
• The magnitude of the incentive?
• Does the use of nonfinancial incentives for quality and efficiency actually increase the probability that patients receive high quality, efficient care?
• The nature of the incentive (public release of performance report vs. confidential performance report)?
• Does greater spending result in higher quality?
• What are the cost savings for the health care provider and purchaser as a result of the quality improvement?
• What are the cost savings associated with different approaches to preventing medical errors or otherwise improving quality?
• What specific processes and structures result in quantifiable cost savings? Who realizes the savings? How should they be shared?
• What contextual variables (e.g., provider supply, employer number and market share, health plan competition, organizational system/infrastructure, employee demographics) positively or negatively influence the effectiveness of financial and nonfinancial incentives for providers?
• Literature Review Methods

Based on input from our expert advisors, our conceptual model, and practical considerations, we developed literature review methods that included: inclusion and exclusion criteria to identify potentially relevant articles, search strategies to retrieve articles, abstract review protocols, and a system of scoring published studies for completeness.

Inclusion and Exclusion Criteria

To be considered an article that provided evidence regarding one of the key questions above, the article had to address one of the predictor variables and either quality (as measured by processes or outcomes) or cost. In addition, the intervention in the trial had to be a strategy that could plausibly be introduced by a purchaser. Our focus was on articles that provided definitive primary data from randomized, controlled trials, but we also included systematic reviews to determine whether these contained any additional information not covered by the primary randomized, controlled trial reports.

We excluded articles that did not meet specific criteria in terms of the quality of the research and reporting. These were:

• Intervention randomized
• Inclusion/exclusion criteria clear and appropriate
• Greater than 75% follow-up
• Note: two criteria usually used to judge the quality of a randomized, controlled trial—provision of placebo to the control group and blinding of the subjects—are not applicable in this situation
• Information source appropriate
• Information source adequately searched
• Data abstraction performed by at least 2 independent reviewers
• Principal measures of effect and the methods of combining results appropriate

Search Strategy

The objective of our search strategy was to identify all published QBP randomized trials and all ongoing research into QBP strategies. For the literature review, we used standard search strategies involving the querying of two online databases (MEDLINE ® and Cochrane) using key words, followed by evaluation of the bibliographies of relevant articles, Web sites of relevant organizations (especially of funding agencies providing project summaries and of employer organizations pursuing QBP), and reference lists provided by our Technical Expert Panel ( Table 1 ).

Table 1. Information sources for literature review and catalog of ongoing research.

Database Searches

To identify potentially relevant articles in the medical literature, we searched MEDLINE ® and Cochrane databases and references provided by our Expert Advisors.

MEDLINE ® search strategies. We searched MEDLINE ® (January 1980 to December 15, 2003) for English language articles using the search terms described in Table 2 . Some citations were reviewed and articles were retrieved in more than one of the searches listed below.

Table 2. MEDLINE ® searches to identify potentially relevant primary data.

Cochrane search strategies. We searched the Cochrane databases from January 1, 1990 through December 15, 2003 (OVID, Evidence Based Medicine Reviews Multifile) using the search terms described in Table 3 .

Table 3. Search terms and citations for Cochrane databases.

Abstract Review

To identify potentially relevant articles for focused searching, at least two investigators (to ensure consistent application of the inclusion and exclusion criteria) reviewed each citation and, whenever an abstract was available, the abstract. Discrepancies in inclusion were resolved by discussion and re-review.

Evaluating Published Articles for Completeness of Reporting

We assessed each of the published articles for their completeness in reporting the factors we identified in our conceptual model that could influence a provider's response to incentives. Specifically, we scored them for the inclusion (or not) of descriptions of the elements in Table 4 . We also recorded the type of care (preventive care, acute care, or chronic care) to which the quality measured pertained.

Table 4. Evaluating randomized controlled trials for completeness of reporting.

• Identifying Ongoing Research

Based on input from our expert advisors, our conceptual model, and practical considerations, we developed methods to catalog ongoing research into QBP that involved specifying: inclusion and exclusion criteria to identify potentially relevant research projects, search strategies to retrieve project abstracts, abstract review protocols, and a system of describing the study design of ongoing research projects.

Since the search for ongoing research focused on projects not yet reported in the literature, the criteria for identifying relevant projects focused on the planned intervention. Two types of research potentially met our inclusion criteria: projects designed as randomized controlled trials, or projects with interventions using QBP methods as described above (i.e., payment or performance reporting strategies) and applied at the community level (or in a broader geographic region, such as a State) that included historical or contemporaneous non-randomized control groups.

We searched online health services research databases (HSRProj and AHRQ's Grants-On-Line Database or GOLD). We also searched the Web sites of other funders or coordinators of projects (e.g., the Leapfrog Group at www.leapfroggroup.org/RewardingResults/ ). Finally, we inquired of staff at AHRQ, the Robert Wood Johnson Foundation, the California HealthCare Foundation, and the Commonwealth Fund whether there was ongoing research that met our inclusion criteria being funded by those organizations. Table 5 lists our information sources for this aspect of the report.

Table 5. Information sources for the catalog of ongoing research.

We searched the two available databases for ongoing health services research, using a similar search strategy for each ( Tables 6 and 7 ). We accessed HSRProj through the National Library of Medicine's Gateway database at gateway.nlm.nih.gov/gw/Cmd and GOLD at www.gold.ahrq.gov .

Table 6. Search terms and citations for GOLD.

Table 7. Search terms and citations for HSRProj database.

GOLD search strategies . We searched GOLD through February 15, 2004 for grants funded by AHRQ using the categories described in Table 6 . Through our combination of searches, we eventually evaluated all projects in GOLD.

HSRProj search strategies . We searched the HSRProj database through February 15, 2004 using the categories described in Table 7 .

Grant Abstract Review

Two investigators reviewed the abstracts of projects identified from the database searches to assess relevance to the technical review. Discrepancies in inclusion were resolved by discussion and re-review and by discussion with project officers at funding agencies or with the principal investigator of the project under consideration.

Describing the Study Design of Ongoing Research

For each research project, we interviewed either project staff (usually the principal investigator) or the project officer to determine the study design. We obtained information about the intervention—performance measures and incentives used—and the control group. The information sought is described in Table 8 .

Table 8. Design information sought about ongoing research.

• Cite this Page Dudley RA, Frolich A, Robinowitz DL, et al. Strategies To Support Quality-based Purchasing: A Review of the Evidence. Rockville (MD): Agency for Healthcare Research and Quality (US); 2004 Jul. (Technical Reviews, No. 10.) 2, Methods for Literature Search.
• PDF version of this title (1.7M)

In this Page

Other titles in these collections.

• AHRQ Technical Reviews
• Health Services/Technology Assessment Text (HSTAT)

Recent Activity

• Methods for Literature Search - Strategies To Support Quality-based Purchasing Methods for Literature Search - Strategies To Support Quality-based Purchasing

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

SJSU | School of Information

Fostering the cultivation of practices in multimodal and culturally responsive literature review research methods.

Published : April 17th, 2024 by Dr. Kristen Radsliff Rebmann

When I talk to students about their program of study, scholarship in LIS, and their identity as researchers, they often tell me that they have no interest in doing research and that they just want to be librarians. Furthermore, I’ve been asked why the iSchool has developed a required course in research methods: INFO 285.  In response to these queries, I try to emphasize that research absolutely is in our wheelhouse as information professionals (a librarian superpower) and that students should take the opportunity in INFO 285 to deepen their skill set and competencies relating to research and writing for scholarly communication.

That said, student resistance made me consider how I (as a faculty member) can do a better job of designing a course that meets students “where they are”, while supporting their development as information professionals who excel in research and other forms of scholarly production. With these experiences in mind I sent out, back in 2021, to develop a new section of info 285 that might support students in a domain of research methods that would deepen their knowledge of research practices but also connect them with competencies that they could immediately use in the workplace.

As a faculty member that has been teaching INFO 275 for nearly a decade (a course relating to the design of programs and services for diverse populations), I contemplated designing a new section of INFO 285 that is both equity-forward and is primed to provide a framework for research in diversity, equity, and inclusion.  I was inspired by SJSU’s language around acknowledging and celebrating this type of scholarship of engagement.  Yet, I realized that there are so many inductive and critical theories out there that to focus on one related methodology, connected to equity-forward research, would be very limiting or (at least) so niche that the course may not fill.  So, I embarked on a new journey to work towards identifying research methods that are new or articulated in new, exciting ways.  I read many new (to me) textbooks and articles on novel methodologies and new takes on established approaches.

I asked myself: What would be exciting AND useful for our graduate students?

In my travels, I came across Anthony J. Onwuegbuzie and Rebecca Frels’ culturally relevant approach to writing literature reviews for programs of study and publication: Seven Steps to a Comprehensive Literature Review – A Multimodal and Cultural Approach .  I connected strongly with the textbook’s emphasis on the use of multimodal texts in charting the landscape of a topic and the authors’ core argument that researchers must take a reflexive stance in their work - reckoning with diverse voices as they make intellectual claims and form arguments in support of moving the field forward.  I found this the perfect text to support a new course in literature review research methods – a methodology that would support both scholarship and professional writing.

So, what IS a comprehensive literature review anyway?

The authors’ own definition of the comprehensive literature review appears on page 4 of the text.

”The comprehensive Literature Review is a methodology, conducted either to stand alone or to inform primary research at multiple stages of their research process which optimally involves the use of mixed research techniques inclusive of culture, ethics, and multimodal texts and settings in a systematic, holistic, synergistic and cyclical process of exploring, interpreting, synthesizing, and communicating published and or unpublished information.”

If you’ve worked as a researcher for many years like I have, you’ll find and read many, many literature reviews but notice along the way that many authors deploy a methodological framework that “makes visible” their epistemological and ontological standpoints.  Literature review authors also traditionally “stay in their lane” when writing about their chosen topics.  This textbook is a reaction against these dispositions. 

What makes Onwuegbuzie and Frels’ comprehensive literature review methodology multimodal, and cultural in its approach?

On page 39 of the textbook, Onwuegbuzie and Frels argue that the multimodal characteristics of our face2face and online experiences of the world require (not only) culturally progressive and ethical research approaches to the world but an approach that includes information harvested from resources harvested in multiple modalities.  The acronym they introduce, MODES, refers to information harvested in the forms of media, observations, documents, experts, and secondary data.

Further, their CLR framework operationalizes the process of locating literature review methods within the author(s) own belief system and stances but also to acknowledge the assets and wealth of knowledge across the many research paradigms and cultural communities that exist in the many field producing knowledge.These efforts represent an important movement within the field: creating a literature review methodology that makes visible the belief systems that shape knowledge production and the value of incorporating diverse intellectual traditions and communities of practice into the information that is collected and synthesized.

I was very proud when students in my course were able to publish their literature review that charts the landscape of refugee services in library and information science.  You can find their open access article here: Refugees’ Digital Equity, Inclusion, and Access in Public Libraries: A Narrative Review .

Onwuegbuzie, A. J. and Frels, R. (2016). Seven steps to a comprehensive literature review: A multimodal and cultural approach. (1st ed.) Sage. https://uk.sagepub.com/en-gb/eur/seven-steps-to-a-comprehensive-literature-review/book238001

Stoner, J., Sagran, N., Cervantes, D., Baseley, S., & Borgolini, S. (2022). Refugees’ Digital Equity, Inclusion, and Access in Public Libraries: A Narrative Review. Library Philosophy & Practice. (p. 7219). Available: https://digitalcommons.unl.edu/libphilprac/7219

Post new comment

• Request new password
• Support portal

Hybrid intelligence failure analysis for industry 4.0: a literature review and future prospective

• Open access
• Published: 22 April 2024

Cite this article

You have full access to this open access article

• Mahdi Mokhtarzadeh   ORCID: orcid.org/0000-0002-0348-6718 1 , 2 ,
• Jorge Rodríguez-Echeverría 1 , 2 , 3 ,
• Ivana Semanjski 1 , 2 &
• Sidharta Gautama 1 , 2  

Industry 4.0 and advanced technology, such as sensors and human–machine cooperation, provide new possibilities for infusing intelligence into failure analysis. Failure analysis is the process of identifying (potential) failures and determining their causes and effects to enhance reliability and manufacturing quality. Proactive methodologies, such as failure mode and effects analysis (FMEA), and reactive methodologies, such as root cause analysis (RCA) and fault tree analysis (FTA), are used to analyze failures before and after their occurrence. This paper focused on failure analysis methodologies intelligentization literature applied to FMEA, RCA, and FTA to provide insights into expert-driven, data-driven, and hybrid intelligence failure analysis advancements. Types of data to establish an intelligence failure analysis, tools to find a failure’s causes and effects, e.g., Bayesian networks, and managerial insights are discussed. This literature review, along with the analyses within it, assists failure and quality analysts in developing effective hybrid intelligence failure analysis methodologies that leverage the strengths of both proactive and reactive methods.

Avoid common mistakes on your manuscript.

Introduction

Failure analysis entails activities to identify, categorize, and prioritize (potential) failures and determine causes and effects of each failure and failure propagation and interdependencies (Rausand & Øien, 1996 ). Failure analysis significance in manufacturing has grown since Industry 3.0 to mitigate defects and/or failures in production processes, thereby maximizing reliability and quality and minimizing production interruptions, associated risks, and costs (Wu et al., 2021 ; Ebeling, 2019 ).

Failure analysis methodologies have been supported by mathematical, statistical, and graph theories and tools, including MCDM theory, fuzzy theory, six-sigma, SPC, DOE, simulation, Pareto charts, and analysis of mean and variance (Oliveira et al., 2021 ; Huang et al., 2020 ; Tari & Sabater, 2004 ). Industry 4.0 is driven by (real-time) data from sensors, the Internet of Things (IoT), such as Internet-enabled machines and tools, and artificial intelligence (AI). Advances in artificial intelligence theory and technology have brought new tools to strengthen failure analysis methodologies (Oztemel & Gursev, 2020 ). Examples of tools include Bayesian networks (BNs), case-based reasoning (CBR), neural networks, classifications, clusterings algorithms, principal component analysis (PCA), deep learning, decision trees, and ontology-driven methods (Zheng et al., 2021 ). These Industry 4.0 advancments enable more efficient data collection and analysis, enhancing predictive capabilities, increasing efficiency and automation, and improving collaboration and knowledge sharing.

Failure analysis methodologies can be categorized into expert-driven, data-driven, and hybrid ones. Expert-driven failure analysis methods rely on experts’ knowledge and experience (Yucesan et al., 2021 ; Huang et al., 2020 ). This approach is useful when the data is limited or when there is a high degree of uncertainty. Expert-driven methods are also useful when the failure structure is complex and difficult to understand. However, this approach is limited by the availability and expertise of the experts, and is prone to bias and subjective interpretations (Liu et al., 2013 ).

Data-driven failure analysis methods, on the other hand, rely on statistical analysis and machine learning algorithms to identify patterns in the data and predict the causes of the failure (Zhang et al., 2023 ; Mazzoleni et al., 2017 ). This approach is useful when there is a large amount of data available and when the failure structure is well-defined. However, data-driven methods is limited by the quality and completeness of the data (Oliveira et al., 2021 ).

Until recently, most tools have focused on replacing humans with artificial intelligence (Yang et al., 2020 ; Filz et al., 2021b ), which causes them to remove human intellect and capabilities from intelligence systems. Hybrid intelligence creates hybrid human–machine intelligence systems, in which humans and machines collaborate synergistically, proactively, and purposefully to augment human intellect and capabilities rather than replace them with machine intellect and capabilities to achieve shared goals (Akata et al., 2020 ).

Collaboration between humans and machines can enhance the failure analysis process, allowing for analyses that were previously unattainable by either humans or machines alone. Thus, hybrid failure analysis provides a more comprehensive analysis of the failure by incorporating strengths of both expert-driven and data-driven approaches to identify the most likely causes and effects of failures (Dellermann et al., 2019 ; van der Aalst, 2021 ).

Benefits from a smart failure analysis may include reduced costs and production stoppages, improved use of human resources, improved use of knowledge, improved failure, root causes, and effects identification, and real-time failure analysis. Yet, only a few studies specifically addressed hybrid failure analysis (Chhetri et al., 2023 ). A case example of hybrid expert data-driven failure analysis involves using data from similar product assemblies to construct a Bayesian network for proccess failure mode and effects analysis (pFMEA), while also incorporating expert knowledge as constraints based on the specific product being analyzed (Chhetri et al., 2023 ).

Over the past few years, several literature reviews, as reported in Section Literature review , have been accomplished under different outlooks in relation to different failure analysis methodologies including failure mode and effects analysis (FMEA), root cause analysis (RCA), and fault tree analysis (FTA). Currently, most existing literature does not systematically summarize the research status of these failure analysis methodologies from the perspective of Industry 4.0 and (hybrid) intelligence failure analysis with the benefits from new technologies. Therefore, this study aims to review, categorize, and analyze the literature of these three general failure analysis methodologies in production systems. The objective is to provide researchers with a comprehensive overview of these methodologies, with a specific focus on hybrid intelligence, and its benefits for quality issues in production. We address two questions "How can failure analysis methodologies benefit from hybrid intelligence?" and "Which tools are suitable for a good fusion of human and machine intelligence?" Consequently, the main contributions of this study to the failure analysis literature are as follows:

Analysis of 86 papers out of 7113 papers from FMEA, RCA, and FTA with respect to methods and data types that might be useful for a hybrid intelligence failure analysis.

Identification of data and methods to construct and detect multiple failures within different research related to FMEA, RCA, and FTA methodologies.

Identification of the most effective methods for analyzing failures, identifying their sources and effects, and assessing related risks.

Proposal of a categorization of research based on the levels of automation/intelligence, along with the identification of limitations in current research in this regard.

Provision of hybrid intelligent failure analysis future research, along with other future directions such as future research on failure propagation and correlation.

The plan of this paper is as follows. Section Literature review briefly introduces related literature reviews on FMEA, RCA, and FTA. A brief description of other failure analysis methodologies is also provided. Section Research methodology presents our review methodology, including the review scope and protocols, defining both our primary and secondary questions, and the criteria for selecting journals and papers to be reviewed. A bibliography summary of the selected papers is provided. Literature has been categorized in Section Literature categorization based on the four general steps of a failure analysis methodology, involving failure structure detection, failure event probability detection, failure risk analysis, and outputs. Managerial insights, limitations, and future research are discussed in Section Managerial insights, limitations, and future research . This assists researchers with applications and complexity, levels of intelligence, how knowledge is introduced into the failure analysis. A more in-depth discussion of hybrid intelligence, failure propagation and correlation, hybrid methodologies, and other areas of future research is also included. Conclusions are presented in Section Conclusion .

Literature review

General and industry/field-specific failure analysis methodologies have been developed over the last few decades. In this section, we provide useful review papers regarding FMEA, RCA, and FTA, which are the focus of our paper. Additionally, some other general and industry/field-specific failure analysis methodologies are briefly discussed.

FMEA is a most commonly used bottom-up proactive qualitative methodologies for potential quality failure analysis (Huang et al., 2020 ; Stamatis, 2003 ). Among its extensions, process FMEA (pFMEA) proactively identifies potential quality failures in production processes such as assembly lines (Johnson & Khan, 2003 ). Typically, (p)FMEA uses expert knowledge to determine potential failures, effects, and causes, and to prioritize the failures based on the risk priority number (RPN). RPN is a product of severity, occurrence, and detection rates for each failure (Wu et al., 2021 ). Some of the FMEA shortcomings include time-consuming, subjectivity, inability to determine multiple failures, and failure propagation and interdependency (Liu et al., 2013 ).

RCA is a bottom-up reactive quantitative methodology that determines the causal mechanism behind a failure to prevent the recurrence of the failure in manufacturing processes (Oliveira et al., 2023 ). To locate, identify, and/or explain the reasons behind the occurrence of root causes, RCA utilizes statistical analysis tools, such as regression, statistical process control (SPC), design of experiments (DOE), PCA, and cause-effect diagram (Williams, 2001 ). Limited ability to predict future failures and difficulty in identifying complex or systemic issues are among RCA limitations (Yuniarto, 2012 ).

FTA is a top-down reactive graphical method to model failure propagation through a system, i.e., how component failures lead to system failures (Kumar & Kaushik, 2020 ). FTA uses qualitative data to model the structure of a system and quantitative data, including probabilities and graph methods such as minimal cut/path sets, binary decision diagrams, simulation, and BNs, to model failures propagation. Requiring extensive data, limited ability to identify contributing factors, and time-consuming are among the FTA limitations (Ruijters & Stoelinga, 2015 ).

In recent years, several literature reviews have been conducted on failure analysis methodologies, exploring various perspectives and approaches. Liu et al. ( 2013 ) reviewed FMEA risk evaluation tools including rule-based systems, mathematical programming, and multi-criteria decision-making (MCDM). They concluded that artificial intelligence and MCDM tools, particularly fuzzy rule base systems, grey theory, and cost-based models, are the most cited tools to prioritize risks in FMEA. Liu et al. ( 2019a ) and Dabous et al. ( 2021 ) reviewed MCDM tools application for FMEA. Papers with different areas, automotive, electronics, machinery and equipment, and steel manufacturing were considered. The most used MCDM tools, namely technique for order of preference by similarity to ideal solution (TOPSIS), analytic hierarchy process (AHP), decision-making trial and evaluation laboratory (DEMATEL), and grey theory, were identified.

Spreafico et al. ( 2017 ) provided a FMEA/Failure mode, effects, and criticality analysis (FMECA) critical review by classifying FMEA/FMECA limitations and issues and reviewing suggested improvements and solutions for the limitations. FMEA issues were classified into four groups of applicabilities, cause and effect analysis, risk analysis, and problem-solving. Main problems (and solutions) are being time-consuming (integration with design tools, using more structured templates, and automation), lack of secondary effects modeling (integration with other tools such as FTA, BN, and Petri net), being too subjective (using statistical evaluation and cost-based approaches), and lack in evaluating the implementation of a solution (using the improved presentation of the results and integration with other tools such as maintenance management tools), respectively. Huang et al. ( 2020 ) provided a bibliographic analysis of FMEA and its applications in manufacturing, marine, healthcare, aerospace, and electronics. Wu et al. ( 2021 ) sorted out potential failure mode identification approaches such as analyzing entry point for system failure mode identification, failure mode recognition tools, and failure mode specification description. Then a review of FMEA risk assessment tools had been provided.

Oliveira et al. ( 2023 ) reviewed automatic RCA literature in manufacturing. Different data types, location-time, physical, and log-action, that are usually used were identified. Industries with the most use of RCA are ranked, semiconductor, chemical, automotive, and others. Then different tools used to automate RCA, including decision trees, regression models, classification methods, clustering methods, neural networks, BNs, PCA, statistical tests, and control charts, were discussed. Ruijters and Stoelinga ( 2015 ) provided FTA qualitative and quantitative analysis methods. Also, different types of FTA, standard FTA, dynamic FTA, and other extensions, were discussed. Zhu and Zhang ( 2022 ) also reviewed dynamic FTA. Cai et al. ( 2017 ) reviewed the application of BN in fault diagnosis. First, an overview of BN types (static, dynamic, and object-oriented), structure modeling, parameters modeling, and interference has been provided. Then applicability of BN for fault identification in process, energy, structural, manufacturing, and network systems has been discussed. BN verification and validation methods are provided. Future prospects including integration of big data with BN, real-time fault diagnosis BN inference algorithms, and hybrid fault diagnosis methods are finally resulted. More relevant BN reviews include BN application in reliability (Insua et al., 2020 ) and safety and risk assessments (Kabir & Papadopoulos, 2019 ).

The integration of FMEA, RCA, and FTA holds immense potential for quality and production managers to minimize failures and enhance system efficiency. By capitalizing on the unique strengths of each approach, the integration of these failure analysis methodologies enables a more comprehensive and effective examination of failures. However, existing studies and literature reviews have predominantly focused on individual methodologies, leading to a lack of integration and limited familiarity with three approaches among engineers and industry experts. To address this gap and promote the integration of them, this study aims to review the progress of intelligence failure analysis within FMEA, RCA, and FTA.

Other general failure analysis methodologies include, but are not limited to, the following methodologies. Event Tree Analysis, similar to FTA, is a graphical representation that models the progression of events following an initiating event, helping to analyze the potential consequences (Ruijters & Stoelinga, 2015 ). Bow-Tie Analysis, usually used in risk management, visualizes the relationship between different potential causes of a hazard and their possible consequences (Khakzad et al., 2012 ). Human Reliability Analysis focuses on assessing the probability of human error and its potential impact on systems and processes (French et al., 2011 ). The Fishbone Diagram visually represents potential causes of a problem to identify root causes by categorizing them into specific factors like people, process, equipment, materials, etc.

There are also industry-specific methodologies, including but not limited to the following ones. Electrostatic Discharge (ESD) Failure Analysis focuses on identifying failures caused by electrostatic discharge, a common concern in the electronics industry. Hazard and Operability Study is widely used in the chemical industry to examine deviations from the design intent and identify potential hazards and operability issues. Incident Response and Post-Incident Analysis, in the IT industry, is used for analyzing and responding to security incidents, with a focus on preventing future occurrences. Hazard Analysis and Critical Control Points is a systematic preventive approach to food safety that identifies, evaluates, and controls hazards throughout the production process. Maximum credible accident analysis assesses and mitigates the most severe accidents that could occur in high-risk industries. For more information on industry-specific methodologies, an interested reader may consult the paper on that industry, as they are wide and out of the scope of this paper for deep discussion.

Our review focuses on the historical progress of (hybrid) intelligence failure analysis to identify and classify methodologies and tools used within them. In Industry 4.0, (hybrid) intelligence failure analysis can contribute to improve quality management and automate quality through an improved human cyber-physical experience. Different from the abovementioned reviews, the purpose of our study is to provide a rich comprehensive understanding of the recent developments in these methodologies from industry 4.0 and hybrid intelligence, the benefits of making them intelligent, i.e., (augmented) automatic and/or data-driven, and their limitations.

Research methodology

A systematic literature review analyses a particular knowledge domain’s body of literature to provide insights into research and practice and identify research gaps (Thomé et al., 2016 ). This section discusses our review scope and protocols, defining both our primary and secondary questions, and the criteria for selecting journals and papers to be reviewed. A bibliography analysis of the selected papers is also presented, including distributions by year, affiliation, and journals.

Review scope and protocol

We follow Thomé et al. ( 2016 ) 8-step literature review methodology to assure a rigorous literature review of intelligence, automated/data-driven, failure analysis methodology for Industry 4.0.

In Step 1, our (hybrid) intelligence failure analysis problem is planned and formulated by identifying the needs, scope, and questions for this research. Our initial need for this literature review comes from a relevant industrial project entitled "assembly quality management using system intelligence" which aims to reduce the quality failures in assembly lines. The trend towards automated and data-driven methodologies in recent years signifies the need for this systematic literature review. Thus, three general failure analysis methodologies, FMEA, RCA, and FTA, are reviewed with respect to tools to make them intelligent and to derive benefits from hybrid intelligence.

Our primary questions are as follows. (i) What are the failure analysis general methodologies and what tools have been used to make them intelligent? (ii) How these methodologies may benefit from hybrid intelligence? (iii) What are the strengths and weaknesses of these methodologies and tools? Our secondary questions are as follows. (i) How intelligent are these tools? (ii) What types of data do they use? Which tools allow a good fusion of human and machine intelligence? (iii) How well do they identify the root causes of failures? (iv) What are the possible future prospectives?

Distribution of papers by year and affliation

Step 2 concerns searching the literature by selecting relevant journals, databases, keywords, and criteria to include or exclude papers. We select the SCOPUS database to scan the relevant paper from 1990 to the first half of 2022. SCOPUS contains all high-quality English publications and covers other databases such as ScienceDirect and IEEE Xplore. A two-level keyword structure is used. The first level retrieves all papers that have either failure mode and effect analysis, FMEA, failure mode and effects and criticality analysis, FMECA, fault tree analysis, FTA, event tree analysis, ETA, root cause analysis, RCA, failure identification, failure analysis, or fault diagnosis in the title, abstract, and/or keywords. The second level limits the retrieved paper by the first level keywords to papers that have either Bayesian network, BN, automated, automatic, automation, smart, intelligence or data-driven in the title, abstract, and/or keywords.

To ensure the scientific rigor of our literature review process, we have removed papers that met at least one of the following criteria: Publications with concise and/or ambiguous information that would make it impossible to re-implement the tools and methodologies described in the paper later on. Publications in low-level journals, i.e., journals in the third quarter (Q3) or lower in the Scimago Journal & Country Rank. Papers with subject areas that are irrelevant to our research topic, such as physics and astronomy.

Steps 3 and 4 involve gathering data and evaluating data quality. We download papers and check their sources according to exclusion criteria. Step 5 concerns data analysis. Step 6 focuses on interpreting the data. The final selected papers are analyzed and interpreted in Section Managerial insights, limitations, and future research . Step 7 involves preparing the results and report. Step 8 requires the review to be updated continuously.

Discussion and statistical analysis

Here is a bibliometric analysis of our literature review. About 15,977 papers were found in our first search. By excluding criteria, we shortened the search to 7113. Then, we checked the titles of 7113 papers including 4359 conference and 2754 journal papers. We downloaded 1,203 papers to read their abstracts and skim their bodies. Then, 1114 low-quality/irrelevant papers were excluded. The remaining 86 high-quality papers were examined for this study.

Distributions of papers by year and affiliation are shown in Fig. 1 . 28 countries have contributed in total. Most affiliations are in advanced countries including China, Germany, and the UK. Surprisingly, we found no publications from Japan and only five from the USA. Only one papers had been published between 1990 and 1999 because of limited data and technology, e.g., sensors and industrial cameras. A slow growth observed between 2000 and 2014 coincides with the technology advancement and Industry 4.0 emergence. The advanced technology and researchers focus on Industry 4.0 have led to significant growth every year since 2015. Worth to note that 2022 information is incomplete because this research has been conducted in the middle of 2022. We expect more publications, at least equal to 2021, for 2022.

Papers distribution by journal is in Fig. 2 . 58 journals and conferences have contributed. Journals with a focus on production and quality, e.g., International Journal of Production Research , have published most papers. Technology-focused journals, e.g., IEEE Access , also have contributed.

Distribution of papers by journal

Literature categorization

Selected papers are now categorized based on the four general steps of a failure analysis methodology, involving failure structure detection, failure event probabilities detection, failure risk analysis, and outputs. Then, a statistical analysis of these categorizations is provided.

These four steps of a failure analysis methodology are illustrated in Fig. 3 . The first two steps deal with input data. In step 1, the failure structure is identified, encompassing all (possible) failures, the failure propagation structure, failure interdependency, and causes and effects. Step 2 involves detecting event probabilities in a failure structure. For example, classical FMEA scores each failure with severity, occurrence, and detection rates.

Four general steps of a failure analysis methodology

To analyze failures in a (production) system, data should be collected to identify the failure structure and detect failures. Reactive methodologies, such as RCA, are data-driven and typically gather available data in a system, while proactive methodologies, such as FMEA, are expert-driven and gather data through expert knowledge. However, a (hybrid) intelligence failure analysis methodology should take advantage of both advanced technologies, such as sensors and Internet-enabled machines and tools, and experts to automatically gather required data, combining proactive and reactive approaches, and providing highly reliable analyses and solutions.

In step 3, all input data are processed to determine the associated risk value with each failure, and the most probable causes (usually based on an observed or potential effect). Typically, a main tool, such as Bayesian networks, neural rule-based systems, statistical analysis, or expert analysis, is used to determine root causes, classify failures, and/or rank failures.

Step 4 outputs results that may include failures and sources, reasons behind the sources, and mitigation actions. The output of this tool is post-processed to provide possible solutions and information that is explainable and easy to use for both humans and machines.

Steps 1: failure structure

Failure structure identification is the first step in a failure analysis methodology. (Potential) failures, causes, effects, and/or failure interdependency are identified. We categorize the literature to develop a (hybrid) intelligence failure methodology to identify failure structure, causes, effects, interdependencies, and relationships between failures, failures and causes, and failures and effects.

Traditionally, experts have defined failure structures by analyzing causes, effects, and the interdependency of failures. However, recent studies have explored alternative approaches to identifying failure structures, leveraging available data sources such as problem-solving databases, design forms, and process descriptions. Problem-solving databases include quality issue records, maintenance records, failure analysis records, and CBR databases. These records could be stored in structured databases and sheets, or unstructured texts. Design forms may include design FMEA forms, reliability characteristics, and product quality characteristics. Process descriptions may include operations, stations, and key operational characteristics. Moreover, simulation can be used to generate failures, causes, and effects (Snooke & Price, 2012 ). Design forms and process descriptions are generated by experts, usually for other purposes, and are re-used for failure analysis. Problem-solving databases could be generated by experts, such as previous FMEAs, or by an automated failure analysis methodology, such as automated RCA. Table 1 classifies studies based on the data sources used to identify the failure structure.

Data processing methods

To define failure structure from operational expert-driven data, no specific tool has been used. In the industry, failure structures are typically defined by an expert (or group of experts). When expert-driven or data-driven historical data and/or design forms and process descriptions are available, ontology-driven algorithms, including heuristics (Sayed & Lohse, 2014 ; Zhou et al., 2015 ; Steenwinckel et al., 2018 ; Xu & Dang, 2023 ) and SysML modeling language (Hecht & Baum, 2019 ), process/system decomposition (the operation, the station, and the key characteristics levels) (Zuo et al., 2016 ; Khorshidi et al., 2015 ; Zhou et al., 2015 ), rule-based algorithms that use CBR (Yang et al., 2018 ; Liu & Ke, 2007 ; Xu & Dang, 2023 ; Oliveira et al., 2022 , 2021 ), and FTA/BN modeling from FMEA/expert data (Yang et al., 2022 ; Steenwinckel et al., 2018 ; Palluat et al., 2006 ) and from Perti net (Yang & Liu, 1998 ) have been suggested. Rivera Torres et al. ( 2018 ) divided a system into components and related failures to each of the components to make a tree of components and failures.

Component-failure matrix is generated using unstructured and quality problem texts mining from historical documents such as bills of material and failure analysis. Apriori algorithms were used to find synonyms in the set of failure modes (Xu et al., 2020 ). The 8D method is used to describe a failure. Ontology was used to store and retrieve data in a knowledge base CBR system.

Yang et al. ( 2022 ), Leu and Chang ( 2013 ) and Waghen and Ouali ( 2021 ) have suggested building a BN structure from the FTA model. Wang et al. ( 2018 ) has proposed to use the fault feature diagram, the fault-labeled transition system based on the Kripke structure to describe the system behavior. The MASON (manufacturing semantic ontology) has been used to construct the structure of the failure class by Psarommatis and Kiritsis ( 2022 ). Teoh and Case ( 2005 ) has developed a functional diagram to construct a failure structure between components of a system and to identify causes and effect propagation. Yang et al. ( 2018 ) used an FMEA style CBR to collect failures to search for similarity. They then used CBR to build a BN using a heuristic algorithm.

Step 2: failure detection

Failure detection data are gathered to determine the strength of relationships among failures, causes, and effects.

Failure detection can be based on operational or historical expert-driven data, as well as data-driven historical and/or real-time data obtained from sensors. Such data can come from a variety of sources, including design and control parameters (such as machine age or workpiece geometry), state variables (such as power demand), performance criteria (such as process time or acoustic emission), and internal/external influencing factors (such as environmental conditions) (Filz et al., 2021b ; Dey & Stori, 2005 ). These data are usually used to determine occurrence probability of failures. To determine the severity and detection probabilities of failures, conditional severity utility data/tables may be used (Lee, 2001 ). Simulation can also be used to determine occurrence, severity, and detection (Price & Taylor, 2002 ). Table 2 summarizes types of data that are usually used to detect failures in the literature.

Processing data refers to the transformation of raw data into meaningful information. A data processing tool is needed that provides accurate and complete information about the system and relationships between data and potential failures.

First, data from different sources should be pre-processed. In a data pre-processing step, data is cleaned, edited, reduced, or wrangled to ensure or enhance performance, such as replacing a missing value with the mean value of the entire column (Filz et al., 2021b ; Schuh et al., 2021 ; Zhang et al., 2023 ; Musumeci et al., 2020 ; Jiao et al., 2020 ; Yang et al., 2015 ; Chien et al., 2017 ).

Data then may need to be processed according to the tools used in Step 3. Common data processing methods between all tools include data normalization using the min-max method (Filz et al., 2021b ; Musumeci et al., 2020 ) and other methods (Yang et al., 2018 ; Schuh et al., 2021 ; Jiao et al., 2020 ; Sariyer et al., 2021 ; Chien et al., 2017 ).

Feature selection/extraction algorithms have been used to select the most important features of data (Filz et al., 2021b ; Xu & Dang, 2020 ; Mazzoleni et al., 2017 ; Duan et al., 2020 ; Schuh et al., 2021 ; Zhang et al., 2023 ; Musumeci et al., 2020 ; Yang et al., 2015 ; Sariyer et al., 2021 ).

For BN-based failure analysis, maximum entropy theory is proposed to calculate failure probabilities from expert-based data (Rastayesh et al., 2019 ). Fuzzy methods have also been used to convert linguistic terms to occurrence probabilities (Yucesan et al., 2021 ; Wan et al., 2019 ; Nie et al., 2019 ; Nepal & Yadav, 2015 ; Ma & Wu, 2020 ; Li et al., 2013 ; Duan et al., 2020 ). Euclidean distance-based similarity measure (Chang et al., 2015 ) and fuzzy rule base RPN model (Tay et al., 2015 ), heuristic algorithms (Brahim et al., 2019 ; Dey & Stori, 2005 ; Yang et al., 2022 ), and a fuzzy probability function (Khorshidi et al., 2015 ) have been suggested to build failure probabilities.

Failure analysis data may be incomplete, inaccurate, imprecise, and limited. Therefore, several studies have used tools to deal with uncertainty in data. The most commonly used methods are fuzzy FMEA (Yang et al., 2022 ; Nepal & Yadav, 2015 ; Ma & Wu, 2020 ), fuzzy BN (Yucesan et al., 2021 ; Wan et al., 2019 ; Nie et al., 2019 ), fuzzy MCDM (Yucesan et al., 2021 ; Nie et al., 2019 ; Nepal & Yadav, 2015 ), fuzzy neural network (Tay et al., 2015 ; Palluat et al., 2006 ), and fuzzy evidential reasoning and Petri nets (Shi et al., 2020 ).

Step 3: analysis

A failure analysis tool is essential for conducting any failure analysis. Table 3 categorizes various data-driven tools, such as BNs, Clustering/Classification, Rule-based Reasoning, and other tools used in the literature and the aspects they support.

BNs model probabilistic relationships among failure causes, modes, and effects using directed acyclic graphs and conditional probabilities. Pieces of evidence, i.e., known variables, are propagated through the graph to evaluate unobserved variables (Cai et al., 2017 ). For example, Rastayesh et al. ( 2019 ) applied BNs for FMEA and perform risk analysis of a Proton Exchange Membrane Fuel Cell. Various elements and levels of the system were identified along with possible routes of failure, including failure causes, modes, and effects. A BN was constructed to perform the failure analysis. Some other examples of the BNs application include an assembly system (Sayed & Lohse, 2014 ), kitchen equipment manufacturing (Yucesan et al., 2021 ), and Auxiliary Power Unit (APU) fault isolation (Yang et al., 2015 ).

Classification assigns predefined labels to input data based on learned patterns, Clustering organizes data into groups based on similarities. Neural networks are commonly used for failure classification and have been employed in most studies. Hence, we separated these studies from those that used other clustering/classification tools. Neural networks consist of layers of interconnected nodes, with an input layer receiving data, one or more hidden layers for processing, and an output layer providing the final classification (Jiang et al., 2024 ). For example, Ma and Wu ( 2020 ) applied neural networks to assess the quality of 311 apartments in Shanghai, China, for FMEA. The input includes various APIs collected for the apartments, and the output was the risk rate of each apartment. In another study, Ma et al. ( 2021 ) applied neural networks for RCA to predict the root causes of multiple quality problems in an automobile factory. Some other examples of the neural networks application include industrial valve manufacturing (Pang et al., 2021 ), complex cyber–physical systems (Liu et al., 2021 ), and an electronic module designed for use in a medical device (Psarommatis & Kiritsis, 2022 ).

Other clustering/classification tools include evolving tree (Chang et al., 2015 ), reinforced concrete columns (Mangalathu et al., 2020 ), K-means, random forest algorithms (Xu & Dang, 2020 ; Chien et al., 2017 ; Oliveira et al., 2022 , 2021 ), contrasting clusters (Zhang et al., 2023 ), K-nearest neighbors (Ma et al., 2021 ), self-organizing maps (Gómez-Andrades et al., 2015 ), and Naive Bayes (Schuh et al., 2021 ; Yang et al., 2015 ).

Rule-based reasoning represents knowledge in the form of "if-then" rules. Rule-based reasoning involves a knowledge base containing the rules and a reasoning engine that applies these rules to incoming data or situations. For instance, Jacobo et al. ( 2007 ) utilized rule-based reasoning for analyzing failures in mechanical components. This approach serves as a knowledgeable assistant, offering guidance to less experienced users with foundational knowledge in materials science and related engineering fields throughout the failure analysis process. Also, the application of the rule-based reasoning for wind turbines FMEA is studied by (Zhou et al., 2015 ).

Other tools include gradient-boosted trees, logistic regression (Filz et al., 2021b ), CBR (Tönnes, 2018 ; Camarillo et al., 2018 ; Jacobo et al., 2007 ), analyzing sensitivities of the machining operation by the stream of variations and errors probability distribution determination (Zuo et al., 2016 ), causal reasoning (Teoh & Case, 2005 ), probabilistic Boolean networks with interventions (Rivera Torres et al., 2018 ), principal component analysis (PCA) (Duan et al., 2020 ; Zhang et al., 2023 ; Jiao et al., 2020 ; Sun et al., 2021 ), factor ranking algorithms (Oliveira et al., 2022 , 2021 ), heuristics and/or new frameworks (Camarillo et al., 2018 ; Yang et al., 2009 , 2020 ; Snooke & Price, 2012 ; Xu & Dang, 2023 ; Rokach & Hutter, 2012 ; Wang et al., 2018 ; Hecht & Baum, 2019 ; Yang & Liu, 1998 ; Liu & Ke, 2007 ), and mathematical optimization methods (Khorshidi et al., 2015 ).

These tools may be integrated by other tools including sequential state switching and artificial anomaly association in a neural network (Liu et al., 2021 ), MCDM/optimization (Yucesan et al., 2021 ; Jomthanachai et al., 2021 ; Ma et al., 2021 ; Sun et al., 2021 ), game theory (Mangalathu et al., 2020 ), fuzzy evidential reasoning and Petri nets (Shi et al., 2020 ), and maximum spanning tree, conditional Granger causality, and multivariate time series (Chen et al., 2018 ).

Step 4: output

A data analysis process can benefit not only humans but also machines and tools in a hybrid intelligence failure analysis methodology. Therefore, the output information should be carefully designed. Table 4 ranks the output data, and the list of studies for each output is available in Online Appendix EC.1. Most studies have focused on automatically identifying the root causes of failures, which is the primary objective of a failure analysis methodology. In addition, researchers have also focused on failure occurrence rating, ranking, and classification. While automatically finding the root causes of failures is important, a hybrid intelligence failure analysis process needs to interpret the related data and information and automatically provide mitigation actions for both operators and machines. However, only a few studies have proposed tools to automatically find possible mitigation actions, usually based on CBR databases and only readable for humans. Therefore, future studies may focus on finding possible automated mitigation actions for failures and developing a quality inspection strategy.

Data post-processing

A data post-processing step transforms data from the main tool into readable, actionable, and useful information for both humans and machines. Adapting solutions from similar failures in a database (i.e., CBR) to propose a solution for a detected failure has been proposed by Tönnes ( 2018 ), Camarillo et al. ( 2018 ), Hecht and Baum ( 2019 ), Jacobo et al. ( 2007 ), Liu and Ke ( 2007 ) and Ma et al. ( 2021 ). Simulation to analyze different scenarios (Psarommatis & Kiritsis, 2022 ; Jomthanachai et al., 2021 ; Chien et al., 2017 ; Oliveira et al., 2022 ), mathematical optimization model (Khorshidi et al., 2015 ; Ma et al., 2021 ) and self-organizing map (SOM) neural network (Chang et al., 2017 ) to automatically select the best corrective action have also been proposed. Also, fuzzy rule-based systems to obtain RPN (Nepal & Yadav, 2015 ) and visualisation (Xu & Dang, 2020 ; Yang et al., 2009 ) are discussed.

The statistical analysis of the paper reveals that most FMEA-based studies rely solely on expert-based information to construct failure structures, while RCA-based papers tend to use a hybrid of problem-solving and system-related data. This is depicted in Fig. 4 , which shows the distribution of papers by data used over time. FMEA is used to identify potential failures when there is not enough data available to construct a failure structure based on system-based data. The trend shows some effort to use data, instead of expert knowledge, to construct failure structures, using data from similar products/processes. RCA and FTA are a reactive methodology that analyzes more information than FMEA. Advances in data mining techniques, along with increased data availability, have led to a growing trend of using data to construct failure structures. For a comprehensive and reliable intelligence failure analysis, a combination of all kinds of data is necessary. It is worth noting that Waghen and Ouali ( 2021 ) proposed a heuristic method to augment failure structure identification that uses expert and historical data. They suggested engaging expert knowledge when historical data are insufficient to identify a failure structure and/or the reliability of a failure structure is low. Other studies have solely focused on failure identification through expert knowledge or historical data, without considering the potential benefits of combining different types of data.

Input data statistical analysis

While most FMEA-based papers use only expert-based data to determine failure probability, there is a significant growth in the utilization of problem-solving data and a hybrid of problem-solving and system-related data, i.e., production line data, over time. RCA and FTA usually tend to use more problem-solving and system-related data. Moreover, this figure and Fig. 5 show that the literature on RCA has been growing in recent years, while the trend for FMEA has remained the same over time. We found that Filz et al. ( 2021b ), Mazzoleni et al. ( 2017 ), Ma and Wu ( 2020 ) and Yang et al. ( 2015 ) improved FMEA to use a combination of expert-based, problem-solving, and system-related data to determine potential failures and their causes. They analyzed these data using deep learning, classification, and neural networks, respectively. Duan et al. ( 2020 ), Ma et al. ( 2021 ) tried to use the benefits of both expert-based data and problem-solving and system-related data in the RCA context. They analyzed the root cause of failures using neural networks.

The distribution of papers by the tools used is shown in Fig. 5 . BNs have been mainly used within the context of FMEA methodologies with a growing trend during the recent years, while RCA researchers have used them less frequently. BNs have the potential to model failure propagation, multi-failure scenarios, and solution analysis to propose potential solutions. However, all of the studies reviewed in this paper only used BNs to identify the root causes of failures. BNs offer a clear graphical representation of failures, their causes, and their effects, which facilitates the interpretation of results by humans. They also provide an easy way for humans to intervene and analyze the sensitivity of results and correct processed data if it appears unrealistic. BNs are well-developed tool and have the ability to work with expert-based, historical, and system-based data, even when data is fuzzy or limited. Developing methodologies that leverage the advantages of BNs seems promising for FMEA, RCA, and FTA.

Tools distribution statistical analysis

RCA and FTA are reliant on various tools over time with no trend of using a specific tool, such as PCA and regression, due to their need for a large amount of data. However, these methods have limitations in incorporating both human and machine intelligence and mostly rely on machine intelligence. Although neural networks and classification algorithms have gained attention in both FMEA and RCA during the last few years, they are black boxes and difficult for humans to modify. Also, classification algorithms typically do not address failure propagation or multi-failure modes. BNs offer a promising alternative, as they can model failure propagation, multiple-failures, and provide a clear graphical representation of failures, causes, and effects. Furthermore, BNs can incorporate both expert-based and historical data, making them well-suited for FMEA, RCA, and FTA. Therefore, developing methodologies that fully leverage the benefits of BNs in these domains would be valuable.

Managerial insights, limitations, and future research

In this section, we discuss managerial insights, limitations, and future research related to different aspects of a Hybrid Intelligence failure analysis methodology. The aim is to assist researchers in focusing on relevant recommendations. Section Section Applications and complexity delves into the applications and complexity of each study, and provides examples for each tool. Section Levels of automation/intelligence presents the levels of intelligence for a failure analysis methodology. Section Introducing knowledge into tools discusses how knowledge is introduced into the failure analysis tools for an effective failure analysis. A more in-depth discussion of hybrid intelligence is in Section Hybrid intelligence . The last three sections provide insights into failure propagation and correlation, hybrid methodologies, and other areas of future research.

Applications and complexity

Intelligent FMEA, RCA, and FTA have been applied to various applications, including production quality management, computer systems, reliability and safety, chemical systems, and others. Table 5 presents the distribution of reviewed papers by application. The list of studies per application is available in Online Appendix EC.2. Production quality management has been the most common application of intelligent failure analysis methodologies due to the significant costs associated with quality assurance. Smart failure analysis methodologies have also been impacted by the increased use of sensors and IoT to collect precise data from machines, tools, operators, and stations, as well as powerful computers to analyze the data. Computer systems failure analysis and system reliability and safety rank second, while chemical systems rank third, as these systems often require specific methodologies, such as hazard and operability analysis.

We checked every paper dataset to find information about the complexity of their case-study and reasons behind their good results to help readers select a validated study on a large set of data. An enriched dataset of problem-solving data are used by Xu et al. ( 2020 ), Du et al. ( 2012 ), Oliveira et al. ( 2021 ), Gómez-Andrades et al. ( 2015 ), Leu and Chang ( 2013 ), Price and Taylor ( 2002 ), Sariyer et al. ( 2021 ), Gomez-Andrades et al. ( 2016 ) and Xu and Dang ( 2023 ). An enriched dataset of historical problem-solving and sensors data is used by

Filz et al. ( 2021b ), Sun et al. ( 2021 ), Mazzoleni et al. ( 2017 ), Hireche et al. ( 2018 ), Yanget al. ( 2015 ) Demirbaga et al. ( 2021 ), Waghen and Ouali ( 2021 ), Zhang et al. ( 2023 ), Oliveira et al. ( 2022 ), Sun et al. ( 2021 ). Data from the system and processes are used by Teoh and Case ( 2005 ), Ma et al. ( 2021 ), Schuh et al. ( 2021 ), Waghen and Ouali ( 2021 ). Other studies demonstrated their methodology on a small problem.

Levels of automation/intelligence

Failure analysis intelligence can be divided into five levels based on the data used. Level 1 involves analyzing failures using expert-based data with the use of intelligence tools. This level can be further improved by incorporating fuzzy-based tools, such as fuzzy BNs, fuzzy neural networks, and fuzzy rule-based systems. If the amount of historical data can be increased over time, we suggest using BNs in a heuristic-based algorithm, as they have the capability to work with all possible data, resulting in fewer modifications in the failure analysis methodology over time. Good examples for Level 1 include Yucesan et al. ( 2021 ) and Brahim et al. ( 2019 ).

Level 2 involves analyzing failures using experts to identify failure structures and problem-solving and system-related data to determine failure probabilities. This level can be used by a professional team who can correctly and completely identify failure structure. It can also be used by those who work with variable structures where updating the structure requires a lot of data modification. Identifying failure structures and analyzing failures are both automated at level 3. This level is the most applicable when a good amount of data is available. BNs, classification algorithms, and neural networks are among the best tools to analyze failure within RCA, FMEA, and FTA methodologies. Studies such as Filz et al. ( 2021b ) Zuo et al. ( 2016 ), Dey and Stori ( 2005 ), Mangalathu et al. ( 2020 ), Yang et al. ( 2015 ) and Ma et al. ( 2021 ) are good examples for Levels 2 and 3.

In level 4, mitigation actions are also determined automatically. This level represents a whole automation of failure analysis. BNs are among the few tools that can encompass all steps of failure analysis. As such, we suggest using them. CBR databases can be used by BNs plus system-based data to provide possible corrective actions. Tönnes ( 2018 ), Zuo et al. ( 2016 ) and Hecht and Baum ( 2019 ) are among good studies for Level 4. Chang et al. ( 2017 ) has focused to automate and visualize corrective actions using a self-organizing map (SOM) neural network in an FMEA methodology. Future research should concentrate on the development of an automated FMEA that dynamically updates the current RPN (Risk Priority Number). This can aid in predicting failures in parts or components of a system using a "Live RPN." The predictive capability of such a tool can be utilized to optimize the overall system. It enables the transformation of a manufacturing system into a self-controlling system, allowing adjustments based on current parameters (Filz et al., 2021b ).

Level 5 is a hybrid intelligence approach to failure analysis that encompasses all other levels and can be implemented within FMEA, RCA, and FTA methodologies when a limited amount of historical and system-based data is available until a comprehensive CBR database is built. BNs provide a good graphical representation and can work with all possible data types. The advantages of BNs are significant enough to be suggested for hybrid intelligence failure analysis. However, we did not find any comprehensive study for this level. A combination of studies that proposed methods to use integrated expert-based, problem-solving, and system-based data, such as Waghen and Ouali ( 2021 ); Filz et al. ( 2021b ), is suggested. Nonetheless, this level remains open and needs to be the focus of future research by scholars. To facilitate the implementation of hybrid intelligence failure analysis, a user-friendly interface is crucial for operators to interact with. Several studies have proposed user-interface applications for this purpose, including (Chan & McNaught, 2008 ; Camarillo et al., 2018 ; Li et al., 2013 ; Jacobo et al., 2007 ; Yang et al., 2009 , 2020 ; Demirbaga et al., 2021 ; Snooke & Price, 2012 ; Palluat et al., 2006 ).

Introducing knowledge into tools

In this section, we analyze which types of knowledge, expert-driven, data-driven, or a hybrid of both, are usually used with which tools and what the implications are for providing insights on suitable tools for hybrid intelligence failure analyses.

Figure 6 shows the distribution of literature based on the input data, tools, and outputs (four general steps of a failure analysis methodology in Fig. 3 ). The first column of nodes shows various combinations of types of knowledge, expert-driven, data-driven, or a hybrid of both, that are usually used in the literature to identify the structure of failure and to detect the probability of failures. The second column of nodes shows various tools that are used to analyze the failure. The third column of nodes shows outputs of a failure analysis. The number of studies with each particular focus is shown by the thickness of an arrow. Details are in Appendix EC.1.

Literature distribution based on inputs, tools, and outputs

The following studies have tried to introduce knowledge and data from expert and data based sources to a failure analysis methodology. Filz et al. ( 2021b ) utilized expert knowledge to identify the structure of failure, the components involved, and the necessary sensors to be used. They then employed sensors to capture data and leveraged problem-solving data from the recorded expert archive to identify failures in a deep learning model. Similarly, Musumeci et al. ( 2020 ) used supervised algorithms to classify failures. Mazzoleni et al. ( 2017 ) they used data from sensors to select the most effective features related to a failure, and subsequently employed sensor data and failure expert data-sets within a gradient boosting tree algorithm to identify the possibility of the failure. Duan et al. ( 2020 ) used data from different sources in a similar way for a neural network to identify the root cause of a failure. Ma and Wu ( 2020 ) utilized expert knowledge to identify failures in construction projects. Subsequently, expert datasets were employed in conjunction with project performance indices to predict the possibility of a failure and determine the root cause of the failure using a neural network tool.

Hireche et al. ( 2018 ), Yang et al. ( 2015 ) gathered data from sensors to determine the conditions of each failure/component node. Then, a BN was used to identify the risks and causes. A multi-level tree is developed by Waghen and Ouali ( 2021 ). Each level contains a solution, pattern, and condition level. Solutions are retrieved from a historical failure database as a combination of certain patterns. The pattern in each problem has been identified and related to the solution using a supervised machine-learning tool. Each level is linked to the next level until the root cause of a failure is correctly identified.

Other usefull tips for introducing knowledge from different sources to a failure analysis methodology can be found in the following studies. Zuo et al. ( 2016 ) divided a multi-operation machining process operation, station, and key characteristics levels. Stream of variations (SoV) was used to evaluate the sensitivities of the machining operations level by level. Results were used to find the sources affecting the quality. Distribution techniques for each quality precision using multi-objective optimization were chosen. Dey and Stori ( 2005 ) used a message-passing method (Pearl, 1988 ) to update a BN using data from sensors to estimate the condition of the system and update the CPTs, when each sensor output is considered as a node in the BN. Chan and McNaught ( 2008 ) also used sensor data to change the probabilities in a BN. A user interface is also developed to make inferences and present the results to operators.

Rokach and Hutter ( 2012 ) used the sequence of machines and a commonality graph of steps and failure causes data to cluster failures to find commonalities between them. A GO methodology is used by Liu et al. ( 2019b ) to model the system and a heuristic is used to construct BN structure and probabilities from the GO methodology model. Teoh and Case ( 2005 ) developed an objective-oriented framework that considers conceptual design information. A hierarchy of components, an assembly tree, and a functional diagram are built to capture data from processes and feed it to FMEA. Bhardwaj et al. ( 2022 ) used historical data from a similar system to estimate failure detection probabilities. Hecht and Baum ( 2019 ) used SysML to describe components and failures.

Zhou et al. ( 2015 ) used a tree of a system. Two classes of knowledge, shallow knowledge and deep knowledge, were gathered to generate rules for failure analysis. The former indicates the experiential knowledge of domain experts, and the latter is the knowledge about the structure and basic principle of the diagnosis system. Liu and Ke ( 2007 ) used CBR to find similar problems and solutions, text mining to find key concepts of the failure in the historical failure record texts, and rule mining to find hidden patterns among system features and failures. Filz et al. ( 2021a ) gathered process parameters after each station using a quality check station. Then a self-organizing Map was used to find failure propagation and cause and effect. Ma et al. ( 2021 ) used data from the system to determine features of problems, products, and operators. Data from problem-solving databases was used to find new failures and classified them using the features and historical data.

Psarommatis and Kiritsis ( 2022 ) developed a methodology that uses data-driven and knowledge-based approaches, an ontology base on the MASON ontology to describe the production domain and enrich the available data. Wang et al. ( 2018 ) developed a data acquisition system including a monitor, sensor, and filter modules. A fault diagram models failure propagation. They extended the Kripke structure by proposing the feature-labeled transition system, which is used to distinguish the behavior of the transition relationship by adding a signature to the transition relationship.

This section highlights that in the realm of failure analysis, a majority of research papers have utilized a hybrid approach, combining expert and data knowledge for tasks such as failure detection, classification, and feature selection. However, to achieve real-time failure analysis, a more effective integration of these two sources is crucial. This integration should enable operators and engineers to provide timely input to the system and observe immediate results. Furthermore, only a limited number of studies have specifically focused on the identification of failure structures using either data or a hybrid of expert and data knowledge.

The use of BNs has emerged as a highly promising approach for achieving real-time input and structure identification in the field of failure analysis. By leveraging both expert knowledge and data sources, BNs have the capability to effectively incorporate expert knowledge as constraints within structure identification algorithms. Unlike traditional classification algorithms that are primarily designed for continuous data, BNs are versatile in handling both discrete and continuous data types. Moreover, BNs possess several strengths that make them particularly suitable for failure analysis. They excel at performing real-time inferences, engaging in counterfactual reasoning, and effectively managing confounding factors. Given these advantages, it is essential to allocate more attention to the application of BNs in hybrid intelligence failure analysis. This involves further exploration of their capabilities and conducting comparative analyses with other tools to assess their effectiveness in various scenarios. By focusing on BNs and conducting comprehensive evaluations, researchers can enhance the understanding and adoption of these powerful tools for improved failure analysis in real-time settings.

Hybrid intelligence

A collaborative failure analysis methodology is needed, in which artificial intelligence tools, machines, and humans can communicate. While hybrid intelligence has gained attention in various fields, literature on the subject for failure analysis is still limited. For example, Piller et al. ( 2022 ) discussed methods to enhance productivity in manufacturing using hybrid intelligence. They explored considerations such as task allocation between humans and machines and the degree of machine intelligence integrated into manufacturing processes. Petrescu and Krishen ( 2023 ) and references within have delved into the benefits and future directions of hybrid intelligence for marketing analytics. Mirbabaie et al. ( 2021 ) has reviewed challenges associated with hybrid intelligence, focusing particularly on conversational agents in hospital settings. Ye et al. ( 2022 ) developed a parallel cognition model. This model draws on both a psychological model and user behavioral data to adaptively learn an individual’s cognitive knowledge. Lee et al. ( 2020 ) combined a data-driven prediction model with a rule-based system to benefit from the combination of human and machine intelligence for personalized rehabilitation assessment.

An artificial intelligence tool should not only provide its final results but also provide its reasoning. A human can analyze the artificial intelligence tool reasoning through a user-interface application and correct possible mistakes instantly and effortlessly. To enable this capability, the use of a white-box artificial tool, such as Bayesian networks, is essential. Explainable AI aids in comprehending and trusting the decision-making process of the hybrid intelligence system by providing the reasoning behind it (Confalonieri et al., 2021 ). Moreover, a machine should be able to interpret and implement an artificial intelligence tool and/or human solutions. Artificial intelligence tools, machines, and humans can learn from mistakes (Correia et al., 2023 ).

To fully exploit the complementarity in human–machine collaborations and effectively utilize the strengths of both, it is important to recognize and understand their roles, limitations, and capabilities in the context of failure analysis. Future research should focus on developing a clear plan for their teamwork and joint actions, including determining the optimal sensor types and locations, quality inspection stations, and human/machine analysis processes. In other words, How to design a decision support system that integrates both human knowledge and machine intelligence with respect to quality management? should be answered. Additionally, tools should be developed to propose possible mitigation actions based on the unique characteristics of the system, environment, humans, and machines. To achieve this, system-related data along with CBR data can be analyzed to find potential mitigation actions.

A general framework for human–machine fusion could involve the following steps: identifying applicable human knowledge and machine data for the problem, determining machine intelligence tools that facilitate the integration of human–machine elements like BNs, identifying the suitable points in the decision-making process to combine human knowledge and machine intelligence effectively, designing the user interface, and incorporating online learning using input from human knowledge (Jarrahi et al., 2022 ). However, human–machine fusion is not an easy task due to the complexity of human–machine interaction, the need for effective and online methods to work with both human and machine data, and the challenge of online learning from human knowledge. For instance, while ChatGPT interacts well with humans, it currently does not update its knowledge using human knowledge input for future cases (Dellermann et al., 2019 ; Correia et al., 2023 ).

Failure propagation and correlation

Most FMEA papers concentrated on analyzing failures in individual products, processes, or machines. It is essential to acknowledge that production processes and machines are interconnected, leading to the correlation and propagation of failures among them. Consequently, it becomes crucial to address the challenge of analyzing failures in multiple machines. To effectively tackle this issue, a holistic approach is necessary. Rather than focusing solely on individual machines, take a broader perspective by considering the entire production system to identify the interdependencies and interactions among different machines, multiple processes, and within the system.

For an intelligence failure analysis, it is necessary to exploit detailed system-related data to carefully and comprehensively identify the relations between different parts of a system, product, and/or process. Some papers have suggested methods to identify failure propagation and correlation (Wang et al., 2021 ; Zhu et al., 2021 ; Chen et al., 2017 ). They usually proposed methods to analyze correlations only between failures or risk criteria using MCDM or statistical methods. However, an intelligence failure analysis should go beyond this and identify failure propagation and correlation among parts of a system.

In the literature, Chen and Jiao ( 2017 ) proposed finite state machine (FSM) theory to model the interactive behaviors between the components, constructing the transition process of fault propagation through the extraction of the state, input, output, and state function of the component. Zuo et al. ( 2016 ) used SoV to model propagation of variations from station to station and operation to operation. A propagation from one station (operation) to the next station (operation) was modeled using a regression like formula. Ament and Goch ( 2001 ) used quality check data after each station to train a neural network for failure progagation and estimate the relationships betweenfailure in stations using a regression model to find patterns in quality check data. Ma et al. ( 2021 ) used patterns in data to classify failures and identify causes.

To conduct an intelligence failure analysis, it is important to identify every part involved, their roles, characteristics, and states. The analysis should include the identification of failure propagation and effects on functions, parts, and other failures. One approach to analyzing failures is through simulation, which can help assess the changes in the characteristics of every part of a system, including humans, machines, and the environment. To analyze the complexity of failure propagation and mutual interactions among different parts of a system, data-driven tools and heuristic algorithms need to be developed. These tools should be capable of managing a large bill of materials and analyzing the failure structure beyond the traditional statistical and MCDM methods. Rule mining can be a useful tool for detecting failure correlation and propagation, especially in situations where there is limited data available, and human interpretation is crucial.

Hybrid methodologies

FMEA, RCA, and FTA methodologies are all complementary and can improve each other’s performance. Furthermore, the availability of data, advanced tools to process data, and the ability to gather online data may lead to a unified FMEA, RCA, and FTA methodology. The reason for this is that while FMEA tries to find potential failures, RCA and FTA try to find root causes of failures, they use similar data and tools to analyze data.

In the literature, FTA has been used as an internal part of FMEA by Steenwinckel et al. ( 2018 ), Palluat et al. ( 2006 )and RCA by Chen et al. ( 2018 ). Using automated mappings from FMEA data to a domain-specific ontology and rules derived from a constructed FTA, Steenwinckel et al. ( 2018 ) annotated and reasoned on sensor observations. Palluat et al. ( 2006 ) used FTA to illustrate the failure structure of a system within an FMEA methodology and developed a neuro-fuzzy network to analyze failures. Chen et al. ( 2018 ) used FTA and graph theory tools, such as the maximum spanning tree, to find the root cause of failures in an RCA methodology. However, studies on the integration of these methodologies regarding the availability of data, tools, and applications should be done to use their advantages within a unified methodology that detects potential failures, finds root causes and effects, and improves the system.

Other future research

Several promising future research directions can be pursued. Cost-based and economic quantification approaches can be integrated into intelligent methodologies to enable more informed decision-making related to failures, their effects, and corrective actions. Additionally, incorporating customer satisfaction criteria, such as using the Kano model, can be useful in situations where there are several costly failures in a system, and budget constraints make it necessary to select the most effective corrective action. This approach has been successfully applied in previous studies (Madzík & Kormanec, 2020 ), and can help optimize decision-making in complex failure scenarios.

Data management is a critical aspect of intelligence methodologies, given the large volume and diverse types of data that need to be processed. Therefore, it is important to design reliable databases that can store and retrieve all necessary data. Ontology can be a valuable tool to help integrate and connect different types of data (Rajpathak & De, 2016 ; Ebrahimipour et al., 2010 ). However, it is also essential to consider issues such as data obsolescence and updates, especially when corrective actions are taken and root causes are removed. Failure to address these issues can lead to incorrect analysis and decision-making.

Traditionally, only single failures were considered in analysis because analyzing a combination of multiple failures was impossible. However, in a system, two or more failures may occur simultaneously or sequentially. It is also possible that a failure occurs as a consequence of another failure. These circumstances are complicated because each failure can have several root causes, and another failure is only one of its causes. Therefore, a clear and powerful tool, such as Bayesian Networks (BNs), should be used to analyze failures and accurately identify possible causes.

The traditional failure analysis methodologies had limitations such as repeatability, subjectivity, and time consumption, which have been addressed by intelligence failure analysis. However, there is a need for more focus on explainability, objective evaluation criteria, and results reliability as some intelligent tools, such as neural networks, act as black boxes. Therefore, suitable tools, such as BNs, should be well-developed and adapted for (hybrid) intelligence failure analysis. Details such as the time and location of the detected failure, possible factors of the causes, such as location, time, conditions, and description of the cause, and reasons behind the causes, such as human fatigue, should be considered within a methodology. These can help to go beyond the CBR and propose intelligence solutions based on the reasons behind a cause. While RCA has implemented these data to a limited extent, FMEA lacks such implementation.

This paper has collected information on both proactive and reactive failure analysis methodologies from 86 papers that focus on FMEA, RCA, or FTA. The goal is to identify areas for improvement, trends, and open problems regarding intelligent failure analysis. This information can help researchers learn the benefits of both methodologies, use their tools, and integrate them to strengthen failure analysis. Each paper has been read and analyzed to extract data and tools used within the paper and their benefits. It was observed that the literature on the three methodologies, FMEA, RCA, and FTA, is diverse. In Industry 4.0, the availability of data, and advances in technology are helping these methodologies benefit from the same tools, such as BNs and neural networks, and make them more integrated.

The literature was classified based on the data needed for a (hybrid) intelligence failure analysis methodology and the tools used for failure analysis to be data-driven and automated. In addition, trends to make these methodologies smart and possible future research in this regard were discussed.

Two main classes of failure structure and failure detection data are usually needed for a failure analysis methodology, each of which can be classified as expert-driven and data-driven. However, a combination of all types of data can lead to more reliable failure analysis. Most papers focused on operational and historical expert-driven and/or data-driven problem-solving data. Among the tools used within FMEA, RCA, and FTA methodologies, BNs have the capability to make a methodology smart and interact with both humans and machines to benefit from hybrid intelligence. BNs not only can analyze failures to identify root causes but also can analyze possible solutions to provide necessary action to prevent failures. A BN’s are also capable of real-time inference, counterfactual reasoning, and managing confounding factors. BNs handle both discrete and continuous data types, unlike traditional classification algorithms. Besides BNs, classification by neural networks, other classification tools, rule-based algorithms, and other tools have been proposed in the literature.

Finally, managerial insights and future research are provided. Most studies have focused on the determination of root causes. It is necessary to automatically find possible mitigation and corrective actions. This step of a failure analysis methodology needs more interaction with humans. Thus, the benefits of hybrid intelligence can be more evident here. It is imperative for humans and machines to work together to properly identify and resolve failures. System-related data should be analyzed to find possible corrective actions. This data is usually available for both proactive and reactive methodologies. Our study showed an effectively tool to integrate knowledge from experts and sensors in needed, enabling operators and engineers to provide timely input and observe immediate results. There is a need to identify failure structures using a hybrid approach that combines expert and data knowledge. Real-time input and structure identification with Bayesian networks can be achieved through the use of Bayesian networks. Further exploration of BNs and comparative analyses with other tools is necessary to enhance understanding and adoption of the best tools for a hybrid intelligence failure analysis in real-time scenarios to prevent failures.

Data availability

There is no data related to this paper.

Agrawal, V., Panigrahi, B. K., & Subbarao, P. (2016). Intelligent decision support system for detection and root cause analysis of faults in coal mills. IEEE Transactions on Fuzzy Systems, 25 (4), 934–944.

Article   Google Scholar  

Akata, Z., Balliet, D., De Rijke, M., Dignum, F., Dignum, V., Eiben, G., Fokkens, A., Grossi, D., Hindriks, K., Hoos, H., et al. (2020). A research agenda for hybrid intelligence: Augmenting human intellect with collaborative, adaptive, responsible, and explainable artificial intelligence. Computer, 53 (08), 18–28.

Al-Mamory, S. O., & Zhang, H. (2009). Intrusion detection alarms reduction using root cause analysis and clustering. Computer Communications, 32 (2), 419–430.

Ament, C., & Goch, G. (2001). A process oriented approach to automated quality control. CIRP Annals, 50 (1), 251–254.

Bhardwaj, U., Teixeira, A., & Soares, C. G. (2022). Bayesian framework for reliability prediction of subsea processing systems accounting for influencing factors uncertainty. Reliability Engineering & System Safety, 218 , 108143.

Brahim, I. B., Addouche, S. A., El Mhamedi, A., & Boujelbene, Y. (2019). Build a Bayesian network from FMECA in the production of automotive parts: Diagnosis and prediction. IFAC-PapersOnLine, 52 (13), 2572–2577.

Cai, B., Huang, L., & Xie, M. (2017). Bayesian networks in fault diagnosis. IEEE Transactions on Industrial Informatics, 13 (5), 2227–2240.

Camarillo, A., Ríos, J., & Althoff, K. D. (2018). Knowledge-based multi-agent system for manufacturing problem solving process in production plants. Journal of Manufacturing Systems, 47 , 115–127.

Chan, A., & McNaught, K. R. (2008). Using Bayesian networks to improve fault diagnosis during manufacturing tests of mobile telephone infrastructure. Journal of the Operational Research Society, 59 (4), 423–430.

Chang, W. L., Pang, L. M., & Tay, K. M. (2017). Application of self-organizing map to failure modes and effects analysis methodology. Neurocomputing, 249 , 314–320.

Chang, W. L., Tay, K. M., & Lim, C. P. (2015). Clustering and visualization of failure modes using an evolving tree. Expert Systems with Applications, 42 (20), 7235–7244.

Chen, H. S., Yan, Z., Zhang, X., Liu, Y., & Yao, Y. (2018). Root cause diagnosis of process faults using conditional Granger causality analysis and maximum spanning tree. IFAC-PapersOnLine, 51 (18), 381–386.

Chen, L., Jiao, J., Wei, Q., & Zhao, T. (2017). An improved formal failure analysis approach for safety-critical system based on mbsa. Engineering Failure Analysis, 82 , 713–725.

Chen, X., & Jiao, J. (2017). A fault propagation modeling method based on a finite state machine. Annual Reliability and Maintainability Symposium (RAMS), 2017 , 1–7.

Google Scholar  

Chhetri, T. R., Aghaei, S., Fensel, A., Göhner, U., Gül-Ficici, S., & Martinez-Gil, J. (2023). Optimising manufacturing process with Bayesian structure learning and knowledge graphs. Computer Aided Systems Theory - EUROCAST, 2022 , 594–602.

Chien, C. F., Liu, C. W., & Chuang, S. C. (2017). Analysing semiconductor manufacturing big data for root cause detection of excursion for yield enhancement. International Journal of Production Research, 55 (17), 5095–5107.

Clancy, R., O’Sullivan, D., & Bruton, K. (2023). Data-driven quality improvement approach to reducing waste in manufacturing. The TQM Journal, 35 (1), 51–72.

Confalonieri, R., Coba, L., Wagner, B., & Besold, T. R. (2021). A historical perspective of explainable artificial intelligence. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 11 (1), e1391.

Correia, A., Grover, A., Schneider, D., Pimentel, A. P., Chaves, R., De Almeida, M. A., & Fonseca, B. (2023). Designing for hybrid intelligence: A taxonomy and survey of crowd-machine interaction. Applied Sciences, 13 (4), 2198.

Dabous, S. A., Ibrahim, F., Feroz, S., & Alsyouf, I. (2021). Integration of failure mode, effects, and criticality analysis with multi-criteria decision-making in engineering applications: Part I- manufacturing industry. Engineering Failure Analysis, 122 , 105264.

Dellermann, D., Ebel, P., Söllner, M., & Leimeister, J. M. (2019). Hybrid intelligence. Business & Information Systems Engineering, 61 , 637–643.

Demirbaga, U., Wen, Z., Noor, A., Mitra, K., Alwasel, K., Garg, S., Zomaya, A. Y., & Ranjan, R. (2021). Autodiagn: An automated real-time diagnosis framework for big data systems. IEEE Transactions on Computers, 71 (5), 1035–1048.

Dey, S., & Stori, J. (2005). A Bayesian network approach to root cause diagnosis of process variations. International Journal of Machine Tools and Manufacture, 45 (1), 75–91.

Du, S., Lv, J., & Xi, L. (2012). A robust approach for root causes identification in machining processes using hybrid learning algorithm and engineering knowledge. Journal of Intelligent Manufacturing, 23 (5), 1833–1847.

Duan, P., He, Z., He, Y., Liu, F., Zhang, A., & Zhou, D. (2020). Root cause analysis approach based on reverse cascading decomposition in QFD and fuzzy weight ARM for quality accidents. Computers & Industrial Engineering, 147 , 106643.

Ebeling, C. E. (2019). An introduction to reliability and maintainability engineering . Waveland Press.

Ebrahimipour, V., Rezaie, K., & Shokravi, S. (2010). An ontology approach to support FMEA studies. Expert Systems with Applications, 37 (1), 671–677.

Filz, M. A., Gellrich, S., Lang, F., Zietsch, J., Abraham, T., & Herrmann, C. (2021). Data-driven analysis of product property propagation to support process-integrated quality management in manufacturing systems. Procedia CIRP, 104 , 900–905.

Filz, M. A., Langner, J. E. B., Herrmann, C., & Thiede, S. (2021). Data-driven failure mode and effect analysis (FMEA) to enhance maintenance planning. Computers in Industry, 129 , 103451.

French, S., Bedford, T., Pollard, S. J., & Soane, E. (2011). Human reliability analysis: A critique and review for managers. Safety Science, 49 (6), 753–763.

Gomez-Andrades, A., Barco, R., Serrano, I., Delgado, P., Caro-Oliver, P., & Munoz, P. (2016). Automatic root cause analysis based on traces for LTE self-organizing networks. IEEE Wireless Communications, 23 (3), 20–28.

Gómez-Andrades, A., Munoz, P., Serrano, I., & Barco, R. (2015). Automatic root cause analysis for lte networks based on unsupervised techniques. IEEE Transactions on Vehicular Technology, 65 (4), 2369–2386.

Hecht, M., & Baum, D. (2019). Failure propagation modeling in FMEAs for reliability, safety, and cybersecurity using SysML. Procedia Computer Science, 153 , 370–377.

Hireche, C., Dezan, C., Mocanu, S., Heller, D., & Diguet, J. P. (2018). Context/resource-aware mission planning based on BNs and concurrent MDPs for autonomous UAVs. Sensors, 18 (12), 4266.

Huang, J., You, J. X., Liu, H. C., & Song, M. S. (2020). Failure mode and effect analysis improvement: A systematic literature review and future research agenda. Reliability Engineering & System Safety, 199 , 106885.

Insua, D. R., Ruggeri, F., Soyer, R., & Wilson, S. (2020). Advances in Bayesian decision making in reliability. European Journal of Operational Research, 282 (1), 1–18.

Jacobo, V., Ortiz, A., Cerrud, Y., & Schouwenaars, R. (2007). Hybrid expert system for the failure analysis of mechanical elements. Engineering Failure Analysis, 14 (8), 1435–1443.

Jarrahi, M. H., Lutz, C., & Newlands, G. (2022). Artificial intelligence, human intelligence and hybrid intelligence based on mutual augmentation. Big Data & Society, 9 (2), 20539517221142824.

Jiang, S., Qin, S., Pulsipher, J. L., & Zavala, V. M. (2024). Convolutional neural networks: Basic concepts and applications in manufacturing. Artificial Intelligence in Manufacturing, 8 , 63–102.

Jiao, J., Zhen, W., Zhu, W., & Wang, G. (2020). Quality-related root cause diagnosis based on orthogonal kernel principal component regression and transfer entropy. IEEE Transactions on Industrial Informatics, 17 (9), 6347–6356.

Johnson, K., & Khan, M. K. (2003). A study into the use of the process failure mode and effects analysis (PFMEA) in the automotive industry in the UK. Journal of Materials Processing Technology, 139 (1–3), 348–356.

Jomthanachai, S., Wong, W. P., & Lim, C. P. (2021). An application of data envelopment analysis and machine learning approach to risk management. IEEE Access, 9 , 85978–85994.

Kabir, S., & Papadopoulos, Y. (2019). Applications of Bayesian networks and Petri nets in safety, reliability, and risk assessments: A review. Safety Science, 115 , 154–175.

Khakzad, N., Khan, F., & Amyotte, P. (2012). Dynamic risk analysis using bow-tie approach. Reliability Engineering & System Safety, 104 , 36–44.

Khorshidi, H. A., Gunawan, I., & Ibrahim, M. Y. (2015). Data-driven system reliability and failure behavior modeling using FMECA. IEEE Transactions on Industrial Informatics, 12 (3), 1253–1260.

Kumar, M., & Kaushik, M. (2020). System failure probability evaluation using fault tree analysis and expert opinions in intuitionistic fuzzy environment. Journal of Loss Prevention in the Process Industries, 67 , 104236.

Lee BH (2001) Using Bayes belief networks in industrial FMEA modeling and analysis. Annual Reliability and Maintainability Symposium. 2001 Proceedings. International Symposium on Product Quality and Integrity (Cat. No.01CH37179) , pp. 7–15.

Lee MH, Siewiorek DP, Smailagic A, Bernardino A, Bermúdez i Badia S (2020) Interactive hybrid approach to combine machine and human intelligence for personalized rehabilitation assessment. Proceedings of the ACM Conference on Health, Inference, and Learning , pp. 160–169.

Leu, S. S., & Chang, C. M. (2013). Bayesian-network-based safety risk assessment for steel construction projects. Accident Analysis & Prevention, 54 , 122–133.

Li, B., Han, T., & Kang, F. (2013). Fault diagnosis expert system of semiconductor manufacturing equipment using a Bayesian network. International Journal of Computer Integrated Manufacturing, 26 (12), 1161–1171.

Liu, C., Lore, K. G., Jiang, Z., & Sarkar, S. (2021). Root-cause analysis for time-series anomalies via spatiotemporal graphical modeling in distributed complex systems. Knowledge-Based Systems, 211 , 106527.

Liu, D. R., & Ke, C. K. (2007). Knowledge support for problem-solving in a production process: A hybrid of knowledge discovery and case-based reasoning. Expert Systems with Applications, 33 (1), 147–161.

Liu, H. C., Chen, X. Q., Duan, C. Y., & Wang, Y. M. (2019). Failure mode and effect analysis using multi-criteria decision making methods: A systematic literature review. Computers & Industrial Engineering, 135 , 881–897.

Liu, H. C., Liu, L., & Liu, N. (2013). Risk evaluation approaches in failure mode and effects analysis: A literature review. Expert Systems with Applications, 40 (2), 828–838.

Liu, L., Fan, D., Wang, Z., Yang, D., Cui, J., Ma, X., & Ren, Y. (2019). Enhanced GO methodology to support failure mode, effects and criticality analysis. Journal of Intelligent Manufacturing, 30 (3), 1451–1468.

Ma, G., & Wu, M. (2020). A big data and FMEA-based construction quality risk evaluation model considering project schedule for shanghai apartment projects. International Journal of Quality & Reliability Management, 37 (1), 18–33.

Ma, Q., Li, H., & Thorstenson, A. (2021). A big data-driven root cause analysis system: Application of machine learning in quality problem solving. Computers & Industrial Engineering, 160 , 107580.

Madzík, P., & Kormanec, P. (2020). Developing the integrated approach of Kano model and failure mode and effect analysis. Total Quality Management & Business Excellence, 31 (15–16), 1788–1810.

Mangalathu, S., Hwang, S. H., & Jeon, J. S. (2020). Failure mode and effects analysis of RC members based on machine-learning-based shapley additive explanations (shap) approach. Engineering Structures, 219 , 110927.

Mazzoleni, M., Maccarana, Y., & Previdi, F. (2017). A comparison of data-driven fault detection methods with application to aerospace electro-mechanical actuators. IFAC-PapersOnLine, 50 (1), 12797–12802.

Mirbabaie, M., Stieglitz, S., & Frick, N. R. (2021). Hybrid intelligence in hospitals: Towards a research agenda for collaboration. Electronic Markets, 31 , 365–387.

Musumeci, F., Magni, L., Ayoub, O., Rubino, R., Capacchione, M., Rigamonti, G., Milano, M., Passera, C., & Tornatore, M. (2020). Supervised and semi-supervised learning for failure identification in microwave networks. IEEE Transactions on Network and Service Management, 18 (2), 1934–1945.

Nepal, B., & Yadav, O. P. (2015). Bayesian belief network-based framework for sourcing risk analysis during supplier selection. International Journal of Production Research, 53 (20), 6114–6135.

Nie, W., Liu, W., Wu, Z., Chen, B., & Wu, L. (2019). Failure mode and effects analysis by integrating Bayesian fuzzy assessment number and extended gray relational analysis-technique for order preference by similarity to ideal solution method. Quality and Reliability Engineering International, 35 (6), 1676–1697.

Oliveira, E. E., Miguéis, V. L., & Borges, J. L. (2021). Understanding overlap in automatic root cause analysis in manufacturing using causal inference. IEEE Access, 10 , 191–201.

Oliveira, E. E., Miguéis, V. L., & Borges, J. L. (2022). On the influence of overlap in automatic root cause analysis in manufacturing. International Journal of Production Research, 60 (21), 6491–6507.

Oliveira, E. E., Miguéis, V. L., & Borges, J. L. (2023). Automatic root cause analysis in manufacturing: An overview & conceptualization. Journal of Intelligent Manufacturing, 34 , 2061–2078.

Oztemel, E., & Gursev, S. (2020). Literature review of industry 4.0 and related technologies. Journal of Intelligent Manufacturing, 31 (1), 127–182.

Palluat, N., Racoceanu, D., & Zerhouni, N. (2006). A neuro-fuzzy monitoring system: Application to flexible production systems. Computers in Industry, 57 (6), 528–538.

Pang, J., Zhang, N., Xiao, Q., Qi, F., & Xue, X. (2021). A new intelligent and data-driven product quality control system of industrial valve manufacturing process in CPS. Computer Communications, 175 , 25–34.

Pearl, J. (1988). Probabilistic reasoning in intelligent systems: Networks of plausible inference . Morgan kaufmann.

Petrescu, M., & Krishen, A. S. (2023). Hybrid intelligence: Human-ai collaboration in marketing analytics. Journal of Marketing Analytics, 11 (3), 263–274.

Piller, F. T., Nitsch, V., & van der Aalst, W. (2022). Hybrid intelligence in next generation manufacturing: An outlook on new forms of collaboration between human and algorithmic decision-makers in the factory of the future (pp. 139–158). Forecasting Next Generation Manufacturing: Digital Shadows, Human-Machine Collaboration, and Data-driven Business Models.

Price, C. J., & Taylor, N. S. (2002). Automated multiple failure FMEA. Reliability Engineering & System Safety, 76 (1), 1–10.

Psarommatis, F., & Kiritsis, D. (2022). A hybrid decision support system for automating decision making in the event of defects in the era of zero defect manufacturing. Journal of Industrial Information Integration, 26 , 100263.

Rajpathak, D., & De, S. (2016). A data-and ontology-driven text mining-based construction of reliability model to analyze and predict component failures. Knowledge and Information Systems, 46 (1), 87–113.

Rastayesh, S., Bahrebar, S., Blaabjerg, F., Zhou, D., Wang, H., & Dalsgaard Sørensen, J. (2019). A system engineering approach using FMEA and Bayesian network for risk analysis-a case study. Sustainability, 12 (1), 77.

Rausand, M., & Øien, K. (1996). The basic concepts of failure analysis. Reliability Engineering & System Safety, 53 (1), 73–83.

Rivera Torres, P. J., Serrano Mercado, E. I., Llanes Santiago, O., & Anido Rifón, L. (2018). Modeling preventive maintenance of manufacturing processes with probabilistic Boolean networks with interventions. Journal of Intelligent Manufacturing, 29 (8), 1941–1952.

Rokach, L., & Hutter, D. (2012). Automatic discovery of the root causes for quality drift in high dimensionality manufacturing processes. Journal of Intelligent Manufacturing, 23 (5), 1915–1930.

Ruijters, E., & Stoelinga, M. (2015). Fault tree analysis: A survey of the state-of-the-art in modeling, analysis and tools. Computer Science Review, 15 , 29–62.

Sariyer, G., Mangla, S. K., Kazancoglu, Y., Ocal Tasar, C., & Luthra, S. (2021). Data analytics for quality management in Industry 4.0 from a MSME perspective. Annals of Operations Research, 23 , 1–19.

Sayed, M. S., & Lohse, N. (2014). Ontology-driven generation of Bayesian diagnostic models for assembly systems. The International Journal of Advanced Manufacturing Technology, 74 (5), 1033–1052.

Schuh, G., Gützlaff, A., Thomas, K., & Welsing, M. (2021). Machine learning based defect detection in a low automated assembly environment. Procedia CIRP, 104 , 265–270.

Shi, H., Wang, L., Li, X. Y., & Liu, H. C. (2020). A novel method for failure mode and effects analysis using fuzzy evidential reasoning and fuzzy Petri nets. Journal of Ambient Intelligence and Humanized Computing, 11 (6), 2381–2395.

Snooke, N., & Price, C. (2012). Automated FMEA based diagnostic symptom generation. Advanced Engineering Informatics, 26 (4), 870–888.

Spreafico, C., Russo, D., & Rizzi, C. (2017). A state-of-the-art review of FMEA/FMECA including patents. Computer Science Review, 25 , 19–28.

Stamatis, D. H. (2003). Failure mode and effect analysis: FMEA from theory to execution . ASQ Quality Press.

Steenwinckel B, Heyvaert P, De Paepe D, Janssens O, Vanden Hautte S, Dimou A, De Turck F, Van Hoecke S, Ongenae F (2018) Towards adaptive anomaly detection and root cause analysis by automated extraction of knowledge from risk analyses. 9th International Semantic Sensor Networks Workshop, Co-Located with 17th International Semantic Web Conference (ISWC 2018) , Vol. 2213, pp. 17–31.

Sun, Y., Qin, W., Zhuang, Z., & Xu, H. (2021). An adaptive fault detection and root-cause analysis scheme for complex industrial processes using moving window KPCA and information geometric causal inference. Journal of Intelligent Manufacturing, 32 (7), 2007–2021.

Tari, J. J., & Sabater, V. (2004). Quality tools and techniques: Are they necessary for quality management? International Journal of Production Economics, 92 (3), 267–280.

Tay, K. M., Jong, C. H., & Lim, C. P. (2015). A clustering-based failure mode and effect analysis model and its application to the edible bird nest industry. Neural Computing and Applications, 26 (3), 551–560.

Teoh, P. C., & Case, K. (2004). Failure modes and effects analysis through knowledge modelling. Journal of Materials Processing Technology, 153 , 253–260.

Teoh, P. C., & Case, K. (2005). An evaluation of failure modes and effects analysis generation method for conceptual design. International Journal of Computer Integrated Manufacturing, 18 (4), 279–293.

Thomé, A. M. T., Scavarda, L. F., & Scavarda, A. J. (2016). Conducting systematic literature review in operations management. Production Planning & Control, 27 (5), 408–420.

Tönnes, W. (2018). Applying data of historical defects to increase efficiency of rework in assembly. Procedia CIRP, 72 , 255–260.

van der Aalst, W. M. (2021). Hybrid intelligence: To automate or not to automate, that is the question. International Journal of Information Systems and Project Management, 9 (2), 5–20.

Waghen, K., & Ouali, M. S. (2021). Multi-level interpretable logic tree analysis: A data-driven approach for hierarchical causality analysis. Expert Systems with Applications, 178 , 115035.

Wan, C., Yan, X., Zhang, D., Qu, Z., & Yang, Z. (2019). An advanced fuzzy Bayesian-based FMEA approach for assessing maritime supply chain risks. Transportation Research Part E, 125 , 222–240.

Wang, L., Li, S., Wei, O., Huang, M., & Hu, J. (2018). An automated fault tree generation approach with fault configuration based on model checking. IEEE Access, 6 , 46900–46914.

Wang, Q., Jia, G., Jia, Y., & Song, W. (2021). A new approach for risk assessment of failure modes considering risk interaction and propagation effects. Reliability Engineering & System Safety, 216 , 108044.

Williams, P. M. (2001). Techniques for root cause analysis. Baylor University Medical Center Proceedings, 14 (2), 154–157.

Wu, Z., Liu, W., & Nie, W. (2021). Literature review and prospect of the development and application of FMEA in manufacturing industry. The International Journal of Advanced Manufacturing Technology, 112 (5), 1409–1436.

Xu, Z., & Dang, Y. (2020). Automated digital cause-and-effect diagrams to assist causal analysis in problem-solving: A data-driven approach. International Journal of Production Research, 58 (17), 5359–5379.

Xu, Z., & Dang, Y. (2023). Data-driven causal knowledge graph construction for root cause analysis in quality problem solving. International Journal of Production Research, 61 (10), 3227–3245.

Xu, Z., Dang, Y., Munro, P., & Wang, Y. (2020). A data-driven approach for constructing the component-failure mode matrix for FMEA. Journal of Intelligent Manufacturing, 31 (1), 249–265.

Yang, C., Zou, Y., Lai, P., & Jiang, N. (2015). Data mining-based methods for fault isolation with validated fmea model ranking. Applied Intelligence, 43 (4), 913–923.

Yang, S., Bian, C., Li, X., Tan, L., & Tang, D. (2018). Optimized fault diagnosis based on FMEA-style CBR and BN for embedded software system. The International Journal of Advanced Manufacturing Technology, 94 (9), 3441–3453.

Yang, S., Liu, H., Zhang, Y., Arndt, T., Hofmann, C., Häfner, B., & Lanza, G. (2020). A data-driven approach for quality analytics of screwing processes in a global learning factory. Procedia Manufacturing, 45 , 454–459.

Yang, S., & Liu, T. (1998). A Petri net approach to early failure detection and isolation for preventive maintenance. Quality and Reliability Engineering International, 14 (5), 319-330.

Yang, Y. J., Xiong, Y. L., Zhang, X. Y., Wang, G. H., & Zou, B. (2022). Reliability analysis of continuous emission monitoring system with common cause failure based on fuzzy FMECA and Bayesian networks. Annals of Operations Research, 311 , 451–467.

Yang, Z. X., Zheng, Y. Y., & Xue, J. X. (2009). Development of automatic fault tree synthesis system using decision matrix. International Journal of Production Economics, 121 (1), 49–56.

Ye, P., Wang, X., Zheng, W., Wei, Q., & Wang, F. Y. (2022). Parallel cognition: Hybrid intelligence for human-machine interaction and management. Frontiers of Information Technology & Electronic Engineering, 23 (12), 1765–1779.

Yucesan, M., Gul, M., & Celik, E. (2021). A holistic FMEA approach by fuzzy-based Bayesian network and best-worst method. Complex & Intelligent Systems, 7 (3), 1547–1564.

Yuniarto, H. (2012). The shortcomings of existing root cause analysis tools. Proceedings of the World Congress on Engineering, 3 , 186–191.

Zhang, S., Xie, X., & Qu, H. (2023). A data-driven workflow for evaporation performance degradation analysis: A full-scale case study in the herbal medicine manufacturing industry. Journal of Intelligent Manufacturing, 34 , 651–668.

Zheng, T., Ardolino, M., Bacchetti, A., & Perona, M. (2021). The applications of industry 4.0 technologies in manufacturing context: a systematic literature review. International Journal of Production Research, 59 (6), 1922–1954.

Zhou, A., Yu, D., & Zhang, W. (2015). A research on intelligent fault diagnosis of wind turbines based on ontology and FMECA. Advanced Engineering Informatics, 29 (1), 115–125.

Zhu, C., & Zhang, T. (2022). A review on the realization methods of dynamic fault tree. Quality and Reliability Engineering International, 38 (6), 3233–3251.

Zhu, J. H., Chen, Z. S., Shuai, B., Pedrycz, W., Chin, K. S., & Martínez, L. (2021). Failure mode and effect analysis: A three-way decision approach. Engineering Applications of Artificial Intelligence, 106 , 104505.

Zuo, X., Li, B., & Yang, J. (2016). Error sensitivity analysis and precision distribution for multi-operation machining processes based on error propagation model. The International Journal of Advanced Manufacturing Technology, 86 (1), 269–280.

Download references

This research is funded by Flanders Make under the project AQUME_SBO, project number 2022-0151. Flanders Make is the Flemish strategic research center for the manufacturing industry in Belgium.

Author information

Authors and affiliations.

Department of Industrial Systems Engineering and Product Design, Ghent University, 9052, Ghent, Belgium

Mahdi Mokhtarzadeh, Jorge Rodríguez-Echeverría, Ivana Semanjski & Sidharta Gautama

FlandersMake@UGent–corelab ISyE, Lommel, Belgium

Escuela Superior Politécnica del Litoral, ESPOL, Facultad de Ingeniería en Electricidad y Computación, ESPOL Polytechnic University, Campus Gustavo Galindo, Km 30.5 Vía Perimetral, P.O. Box 09-01-5863, 090112, Guayaquil, Ecuador

Jorge Rodríguez-Echeverría

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Mahdi Mokhtarzadeh .

Ethics declarations

Competing interest.

The authors report there are no competing interests to declare.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (pdf 137 KB)

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Mokhtarzadeh, M., Rodríguez-Echeverría, J., Semanjski, I. et al. Hybrid intelligence failure analysis for industry 4.0: a literature review and future prospective. J Intell Manuf (2024). https://doi.org/10.1007/s10845-024-02376-5

Download citation

Received : 27 June 2023

Accepted : 14 March 2024

Published : 22 April 2024

DOI : https://doi.org/10.1007/s10845-024-02376-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Automated failure analysis
• Data-driven failure analysis
• Human–machine cooperation
• Find a journal
• Publish with us
• Track your research