types of literature in research

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.

Handbook of eHealth Evaluation: An Evidence-based Approach [Internet].

Chapter 9 methods for literature reviews.

Guy Paré and Spyros Kitsiou .

9.1. Introduction

Literature reviews play a critical role in scholarship because science remains, first and foremost, a cumulative endeavour ( vom Brocke et al., 2009 ). As in any academic discipline, rigorous knowledge syntheses are becoming indispensable in keeping up with an exponentially growing eHealth literature, assisting practitioners, academics, and graduate students in finding, evaluating, and synthesizing the contents of many empirical and conceptual papers. Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generating new frameworks and theories; and (e) identifying topics or questions requiring more investigation ( Paré, Trudel, Jaana, & Kitsiou, 2015 ).

Literature reviews can take two major forms. The most prevalent one is the “literature review” or “background” section within a journal paper or a chapter in a graduate thesis. This section synthesizes the extant literature and usually identifies the gaps in knowledge that the empirical study addresses ( Sylvester, Tate, & Johnstone, 2013 ). It may also provide a theoretical foundation for the proposed study, substantiate the presence of the research problem, justify the research as one that contributes something new to the cumulated knowledge, or validate the methods and approaches for the proposed study ( Hart, 1998 ; Levy & Ellis, 2006 ).

The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ( Paré et al., 2015 ). Rather than providing a base for a researcher’s own work, it creates a solid starting point for all members of the community interested in a particular area or topic ( Mulrow, 1987 ). The so-called “review article” is a journal-length paper which has an overarching purpose to synthesize the literature in a field, without collecting or analyzing any primary data ( Green, Johnson, & Adams, 2006 ).

When appropriately conducted, review articles represent powerful information sources for practitioners looking for state-of-the art evidence to guide their decision-making and work practices ( Paré et al., 2015 ). Further, high-quality reviews become frequently cited pieces of work which researchers seek out as a first clear outline of the literature when undertaking empirical studies ( Cooper, 1988 ; Rowe, 2014 ). Scholars who track and gauge the impact of articles have found that review papers are cited and downloaded more often than any other type of published article ( Cronin, Ryan, & Coughlan, 2008 ; Montori, Wilczynski, Morgan, Haynes, & Hedges, 2003 ; Patsopoulos, Analatos, & Ioannidis, 2005 ). The reason for their popularity may be the fact that reading the review enables one to have an overview, if not a detailed knowledge of the area in question, as well as references to the most useful primary sources ( Cronin et al., 2008 ). Although they are not easy to conduct, the commitment to complete a review article provides a tremendous service to one’s academic community ( Paré et al., 2015 ; Petticrew & Roberts, 2006 ). Most, if not all, peer-reviewed journals in the fields of medical informatics publish review articles of some type.

The main objectives of this chapter are fourfold: (a) to provide an overview of the major steps and activities involved in conducting a stand-alone literature review; (b) to describe and contrast the different types of review articles that can contribute to the eHealth knowledge base; (c) to illustrate each review type with one or two examples from the eHealth literature; and (d) to provide a series of recommendations for prospective authors of review articles in this domain.

9.2. Overview of the Literature Review Process and Steps

As explained in Templier and Paré (2015) , there are six generic steps involved in conducting a review article:

• formulating the research question(s) and objective(s),
• searching the extant literature,
• screening for inclusion,
• assessing the quality of primary studies,
• extracting data, and
• analyzing data.

Although these steps are presented here in sequential order, one must keep in mind that the review process can be iterative and that many activities can be initiated during the planning stage and later refined during subsequent phases ( Finfgeld-Connett & Johnson, 2013 ; Kitchenham & Charters, 2007 ).

Formulating the research question(s) and objective(s): As a first step, members of the review team must appropriately justify the need for the review itself ( Petticrew & Roberts, 2006 ), identify the review’s main objective(s) ( Okoli & Schabram, 2010 ), and define the concepts or variables at the heart of their synthesis ( Cooper & Hedges, 2009 ; Webster & Watson, 2002 ). Importantly, they also need to articulate the research question(s) they propose to investigate ( Kitchenham & Charters, 2007 ). In this regard, we concur with Jesson, Matheson, and Lacey (2011) that clearly articulated research questions are key ingredients that guide the entire review methodology; they underscore the type of information that is needed, inform the search for and selection of relevant literature, and guide or orient the subsequent analysis. Searching the extant literature: The next step consists of searching the literature and making decisions about the suitability of material to be considered in the review ( Cooper, 1988 ). There exist three main coverage strategies. First, exhaustive coverage means an effort is made to be as comprehensive as possible in order to ensure that all relevant studies, published and unpublished, are included in the review and, thus, conclusions are based on this all-inclusive knowledge base. The second type of coverage consists of presenting materials that are representative of most other works in a given field or area. Often authors who adopt this strategy will search for relevant articles in a small number of top-tier journals in a field ( Paré et al., 2015 ). In the third strategy, the review team concentrates on prior works that have been central or pivotal to a particular topic. This may include empirical studies or conceptual papers that initiated a line of investigation, changed how problems or questions were framed, introduced new methods or concepts, or engendered important debate ( Cooper, 1988 ). Screening for inclusion: The following step consists of evaluating the applicability of the material identified in the preceding step ( Levy & Ellis, 2006 ; vom Brocke et al., 2009 ). Once a group of potential studies has been identified, members of the review team must screen them to determine their relevance ( Petticrew & Roberts, 2006 ). A set of predetermined rules provides a basis for including or excluding certain studies. This exercise requires a significant investment on the part of researchers, who must ensure enhanced objectivity and avoid biases or mistakes. As discussed later in this chapter, for certain types of reviews there must be at least two independent reviewers involved in the screening process and a procedure to resolve disagreements must also be in place ( Liberati et al., 2009 ; Shea et al., 2009 ). Assessing the quality of primary studies: In addition to screening material for inclusion, members of the review team may need to assess the scientific quality of the selected studies, that is, appraise the rigour of the research design and methods. Such formal assessment, which is usually conducted independently by at least two coders, helps members of the review team refine which studies to include in the final sample, determine whether or not the differences in quality may affect their conclusions, or guide how they analyze the data and interpret the findings ( Petticrew & Roberts, 2006 ). Ascribing quality scores to each primary study or considering through domain-based evaluations which study components have or have not been designed and executed appropriately makes it possible to reflect on the extent to which the selected study addresses possible biases and maximizes validity ( Shea et al., 2009 ). Extracting data: The following step involves gathering or extracting applicable information from each primary study included in the sample and deciding what is relevant to the problem of interest ( Cooper & Hedges, 2009 ). Indeed, the type of data that should be recorded mainly depends on the initial research questions ( Okoli & Schabram, 2010 ). However, important information may also be gathered about how, when, where and by whom the primary study was conducted, the research design and methods, or qualitative/quantitative results ( Cooper & Hedges, 2009 ). Analyzing and synthesizing data : As a final step, members of the review team must collate, summarize, aggregate, organize, and compare the evidence extracted from the included studies. The extracted data must be presented in a meaningful way that suggests a new contribution to the extant literature ( Jesson et al., 2011 ). Webster and Watson (2002) warn researchers that literature reviews should be much more than lists of papers and should provide a coherent lens to make sense of extant knowledge on a given topic. There exist several methods and techniques for synthesizing quantitative (e.g., frequency analysis, meta-analysis) and qualitative (e.g., grounded theory, narrative analysis, meta-ethnography) evidence ( Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005 ; Thomas & Harden, 2008 ).

9.3. Types of Review Articles and Brief Illustrations

EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic. Our classification scheme is largely inspired from Paré and colleagues’ (2015) typology. Below we present and illustrate those review types that we feel are central to the growth and development of the eHealth domain.

9.3.1. Narrative Reviews

The narrative review is the “traditional” way of reviewing the extant literature and is skewed towards a qualitative interpretation of prior knowledge ( Sylvester et al., 2013 ). Put simply, a narrative review attempts to summarize or synthesize what has been written on a particular topic but does not seek generalization or cumulative knowledge from what is reviewed ( Davies, 2000 ; Green et al., 2006 ). Instead, the review team often undertakes the task of accumulating and synthesizing the literature to demonstrate the value of a particular point of view ( Baumeister & Leary, 1997 ). As such, reviewers may selectively ignore or limit the attention paid to certain studies in order to make a point. In this rather unsystematic approach, the selection of information from primary articles is subjective, lacks explicit criteria for inclusion and can lead to biased interpretations or inferences ( Green et al., 2006 ). There are several narrative reviews in the particular eHealth domain, as in all fields, which follow such an unstructured approach ( Silva et al., 2015 ; Paul et al., 2015 ).

Despite these criticisms, this type of review can be very useful in gathering together a volume of literature in a specific subject area and synthesizing it. As mentioned above, its primary purpose is to provide the reader with a comprehensive background for understanding current knowledge and highlighting the significance of new research ( Cronin et al., 2008 ). Faculty like to use narrative reviews in the classroom because they are often more up to date than textbooks, provide a single source for students to reference, and expose students to peer-reviewed literature ( Green et al., 2006 ). For researchers, narrative reviews can inspire research ideas by identifying gaps or inconsistencies in a body of knowledge, thus helping researchers to determine research questions or formulate hypotheses. Importantly, narrative reviews can also be used as educational articles to bring practitioners up to date with certain topics of issues ( Green et al., 2006 ).

Recently, there have been several efforts to introduce more rigour in narrative reviews that will elucidate common pitfalls and bring changes into their publication standards. Information systems researchers, among others, have contributed to advancing knowledge on how to structure a “traditional” review. For instance, Levy and Ellis (2006) proposed a generic framework for conducting such reviews. Their model follows the systematic data processing approach comprised of three steps, namely: (a) literature search and screening; (b) data extraction and analysis; and (c) writing the literature review. They provide detailed and very helpful instructions on how to conduct each step of the review process. As another methodological contribution, vom Brocke et al. (2009) offered a series of guidelines for conducting literature reviews, with a particular focus on how to search and extract the relevant body of knowledge. Last, Bandara, Miskon, and Fielt (2011) proposed a structured, predefined and tool-supported method to identify primary studies within a feasible scope, extract relevant content from identified articles, synthesize and analyze the findings, and effectively write and present the results of the literature review. We highly recommend that prospective authors of narrative reviews consult these useful sources before embarking on their work.

Darlow and Wen (2015) provide a good example of a highly structured narrative review in the eHealth field. These authors synthesized published articles that describe the development process of mobile health ( m-health ) interventions for patients’ cancer care self-management. As in most narrative reviews, the scope of the research questions being investigated is broad: (a) how development of these systems are carried out; (b) which methods are used to investigate these systems; and (c) what conclusions can be drawn as a result of the development of these systems. To provide clear answers to these questions, a literature search was conducted on six electronic databases and Google Scholar . The search was performed using several terms and free text words, combining them in an appropriate manner. Four inclusion and three exclusion criteria were utilized during the screening process. Both authors independently reviewed each of the identified articles to determine eligibility and extract study information. A flow diagram shows the number of studies identified, screened, and included or excluded at each stage of study selection. In terms of contributions, this review provides a series of practical recommendations for m-health intervention development.

9.3.2. Descriptive or Mapping Reviews

The primary goal of a descriptive review is to determine the extent to which a body of knowledge in a particular research topic reveals any interpretable pattern or trend with respect to pre-existing propositions, theories, methodologies or findings ( King & He, 2005 ; Paré et al., 2015 ). In contrast with narrative reviews, descriptive reviews follow a systematic and transparent procedure, including searching, screening and classifying studies ( Petersen, Vakkalanka, & Kuzniarz, 2015 ). Indeed, structured search methods are used to form a representative sample of a larger group of published works ( Paré et al., 2015 ). Further, authors of descriptive reviews extract from each study certain characteristics of interest, such as publication year, research methods, data collection techniques, and direction or strength of research outcomes (e.g., positive, negative, or non-significant) in the form of frequency analysis to produce quantitative results ( Sylvester et al., 2013 ). In essence, each study included in a descriptive review is treated as the unit of analysis and the published literature as a whole provides a database from which the authors attempt to identify any interpretable trends or draw overall conclusions about the merits of existing conceptualizations, propositions, methods or findings ( Paré et al., 2015 ). In doing so, a descriptive review may claim that its findings represent the state of the art in a particular domain ( King & He, 2005 ).

In the fields of health sciences and medical informatics, reviews that focus on examining the range, nature and evolution of a topic area are described by Anderson, Allen, Peckham, and Goodwin (2008) as mapping reviews . Like descriptive reviews, the research questions are generic and usually relate to publication patterns and trends. There is no preconceived plan to systematically review all of the literature although this can be done. Instead, researchers often present studies that are representative of most works published in a particular area and they consider a specific time frame to be mapped.

An example of this approach in the eHealth domain is offered by DeShazo, Lavallie, and Wolf (2009). The purpose of this descriptive or mapping review was to characterize publication trends in the medical informatics literature over a 20-year period (1987 to 2006). To achieve this ambitious objective, the authors performed a bibliometric analysis of medical informatics citations indexed in medline using publication trends, journal frequencies, impact factors, Medical Subject Headings (MeSH) term frequencies, and characteristics of citations. Findings revealed that there were over 77,000 medical informatics articles published during the covered period in numerous journals and that the average annual growth rate was 12%. The MeSH term analysis also suggested a strong interdisciplinary trend. Finally, average impact scores increased over time with two notable growth periods. Overall, patterns in research outputs that seem to characterize the historic trends and current components of the field of medical informatics suggest it may be a maturing discipline (DeShazo et al., 2009).

9.3.3. Scoping Reviews

Scoping reviews attempt to provide an initial indication of the potential size and nature of the extant literature on an emergent topic (Arksey & O’Malley, 2005; Daudt, van Mossel, & Scott, 2013 ; Levac, Colquhoun, & O’Brien, 2010). A scoping review may be conducted to examine the extent, range and nature of research activities in a particular area, determine the value of undertaking a full systematic review (discussed next), or identify research gaps in the extant literature ( Paré et al., 2015 ). In line with their main objective, scoping reviews usually conclude with the presentation of a detailed research agenda for future works along with potential implications for both practice and research.

Unlike narrative and descriptive reviews, the whole point of scoping the field is to be as comprehensive as possible, including grey literature (Arksey & O’Malley, 2005). Inclusion and exclusion criteria must be established to help researchers eliminate studies that are not aligned with the research questions. It is also recommended that at least two independent coders review abstracts yielded from the search strategy and then the full articles for study selection ( Daudt et al., 2013 ). The synthesized evidence from content or thematic analysis is relatively easy to present in tabular form (Arksey & O’Malley, 2005; Thomas & Harden, 2008 ).

One of the most highly cited scoping reviews in the eHealth domain was published by Archer, Fevrier-Thomas, Lokker, McKibbon, and Straus (2011) . These authors reviewed the existing literature on personal health record ( phr ) systems including design, functionality, implementation, applications, outcomes, and benefits. Seven databases were searched from 1985 to March 2010. Several search terms relating to phr s were used during this process. Two authors independently screened titles and abstracts to determine inclusion status. A second screen of full-text articles, again by two independent members of the research team, ensured that the studies described phr s. All in all, 130 articles met the criteria and their data were extracted manually into a database. The authors concluded that although there is a large amount of survey, observational, cohort/panel, and anecdotal evidence of phr benefits and satisfaction for patients, more research is needed to evaluate the results of phr implementations. Their in-depth analysis of the literature signalled that there is little solid evidence from randomized controlled trials or other studies through the use of phr s. Hence, they suggested that more research is needed that addresses the current lack of understanding of optimal functionality and usability of these systems, and how they can play a beneficial role in supporting patient self-management ( Archer et al., 2011 ).

9.3.4. Forms of Aggregative Reviews

Healthcare providers, practitioners, and policy-makers are nowadays overwhelmed with large volumes of information, including research-based evidence from numerous clinical trials and evaluation studies, assessing the effectiveness of health information technologies and interventions ( Ammenwerth & de Keizer, 2004 ; Deshazo et al., 2009 ). It is unrealistic to expect that all these disparate actors will have the time, skills, and necessary resources to identify the available evidence in the area of their expertise and consider it when making decisions. Systematic reviews that involve the rigorous application of scientific strategies aimed at limiting subjectivity and bias (i.e., systematic and random errors) can respond to this challenge.

Systematic reviews attempt to aggregate, appraise, and synthesize in a single source all empirical evidence that meet a set of previously specified eligibility criteria in order to answer a clearly formulated and often narrow research question on a particular topic of interest to support evidence-based practice ( Liberati et al., 2009 ). They adhere closely to explicit scientific principles ( Liberati et al., 2009 ) and rigorous methodological guidelines (Higgins & Green, 2008) aimed at reducing random and systematic errors that can lead to deviations from the truth in results or inferences. The use of explicit methods allows systematic reviews to aggregate a large body of research evidence, assess whether effects or relationships are in the same direction and of the same general magnitude, explain possible inconsistencies between study results, and determine the strength of the overall evidence for every outcome of interest based on the quality of included studies and the general consistency among them ( Cook, Mulrow, & Haynes, 1997 ). The main procedures of a systematic review involve:

• Formulating a review question and developing a search strategy based on explicit inclusion criteria for the identification of eligible studies (usually described in the context of a detailed review protocol).
• Searching for eligible studies using multiple databases and information sources, including grey literature sources, without any language restrictions.
• Selecting studies, extracting data, and assessing risk of bias in a duplicate manner using two independent reviewers to avoid random or systematic errors in the process.
• Analyzing data using quantitative or qualitative methods.
• Presenting results in summary of findings tables.
• Interpreting results and drawing conclusions.

Many systematic reviews, but not all, use statistical methods to combine the results of independent studies into a single quantitative estimate or summary effect size. Known as meta-analyses , these reviews use specific data extraction and statistical techniques (e.g., network, frequentist, or Bayesian meta-analyses) to calculate from each study by outcome of interest an effect size along with a confidence interval that reflects the degree of uncertainty behind the point estimate of effect ( Borenstein, Hedges, Higgins, & Rothstein, 2009 ; Deeks, Higgins, & Altman, 2008 ). Subsequently, they use fixed or random-effects analysis models to combine the results of the included studies, assess statistical heterogeneity, and calculate a weighted average of the effect estimates from the different studies, taking into account their sample sizes. The summary effect size is a value that reflects the average magnitude of the intervention effect for a particular outcome of interest or, more generally, the strength of a relationship between two variables across all studies included in the systematic review. By statistically combining data from multiple studies, meta-analyses can create more precise and reliable estimates of intervention effects than those derived from individual studies alone, when these are examined independently as discrete sources of information.

The review by Gurol-Urganci, de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) on the effects of mobile phone messaging reminders for attendance at healthcare appointments is an illustrative example of a high-quality systematic review with meta-analysis. Missed appointments are a major cause of inefficiency in healthcare delivery with substantial monetary costs to health systems. These authors sought to assess whether mobile phone-based appointment reminders delivered through Short Message Service ( sms ) or Multimedia Messaging Service ( mms ) are effective in improving rates of patient attendance and reducing overall costs. To this end, they conducted a comprehensive search on multiple databases using highly sensitive search strategies without language or publication-type restrictions to identify all rct s that are eligible for inclusion. In order to minimize the risk of omitting eligible studies not captured by the original search, they supplemented all electronic searches with manual screening of trial registers and references contained in the included studies. Study selection, data extraction, and risk of bias assessments were performed inde­­pen­dently by two coders using standardized methods to ensure consistency and to eliminate potential errors. Findings from eight rct s involving 6,615 participants were pooled into meta-analyses to calculate the magnitude of effects that mobile text message reminders have on the rate of attendance at healthcare appointments compared to no reminders and phone call reminders.

Meta-analyses are regarded as powerful tools for deriving meaningful conclusions. However, there are situations in which it is neither reasonable nor appropriate to pool studies together using meta-analytic methods simply because there is extensive clinical heterogeneity between the included studies or variation in measurement tools, comparisons, or outcomes of interest. In these cases, systematic reviews can use qualitative synthesis methods such as vote counting, content analysis, classification schemes and tabulations, as an alternative approach to narratively synthesize the results of the independent studies included in the review. This form of review is known as qualitative systematic review.

A rigorous example of one such review in the eHealth domain is presented by Mickan, Atherton, Roberts, Heneghan, and Tilson (2014) on the use of handheld computers by healthcare professionals and their impact on access to information and clinical decision-making. In line with the methodological guide­lines for systematic reviews, these authors: (a) developed and registered with prospero ( www.crd.york.ac.uk/ prospero / ) an a priori review protocol; (b) conducted comprehensive searches for eligible studies using multiple databases and other supplementary strategies (e.g., forward searches); and (c) subsequently carried out study selection, data extraction, and risk of bias assessments in a duplicate manner to eliminate potential errors in the review process. Heterogeneity between the included studies in terms of reported outcomes and measures precluded the use of meta-analytic methods. To this end, the authors resorted to using narrative analysis and synthesis to describe the effectiveness of handheld computers on accessing information for clinical knowledge, adherence to safety and clinical quality guidelines, and diagnostic decision-making.

In recent years, the number of systematic reviews in the field of health informatics has increased considerably. Systematic reviews with discordant findings can cause great confusion and make it difficult for decision-makers to interpret the review-level evidence ( Moher, 2013 ). Therefore, there is a growing need for appraisal and synthesis of prior systematic reviews to ensure that decision-making is constantly informed by the best available accumulated evidence. Umbrella reviews , also known as overviews of systematic reviews, are tertiary types of evidence synthesis that aim to accomplish this; that is, they aim to compare and contrast findings from multiple systematic reviews and meta-analyses ( Becker & Oxman, 2008 ). Umbrella reviews generally adhere to the same principles and rigorous methodological guidelines used in systematic reviews. However, the unit of analysis in umbrella reviews is the systematic review rather than the primary study ( Becker & Oxman, 2008 ). Unlike systematic reviews that have a narrow focus of inquiry, umbrella reviews focus on broader research topics for which there are several potential interventions ( Smith, Devane, Begley, & Clarke, 2011 ). A recent umbrella review on the effects of home telemonitoring interventions for patients with heart failure critically appraised, compared, and synthesized evidence from 15 systematic reviews to investigate which types of home telemonitoring technologies and forms of interventions are more effective in reducing mortality and hospital admissions ( Kitsiou, Paré, & Jaana, 2015 ).

9.3.5. Realist Reviews

Realist reviews are theory-driven interpretative reviews developed to inform, enhance, or supplement conventional systematic reviews by making sense of heterogeneous evidence about complex interventions applied in diverse contexts in a way that informs policy decision-making ( Greenhalgh, Wong, Westhorp, & Pawson, 2011 ). They originated from criticisms of positivist systematic reviews which centre on their “simplistic” underlying assumptions ( Oates, 2011 ). As explained above, systematic reviews seek to identify causation. Such logic is appropriate for fields like medicine and education where findings of randomized controlled trials can be aggregated to see whether a new treatment or intervention does improve outcomes. However, many argue that it is not possible to establish such direct causal links between interventions and outcomes in fields such as social policy, management, and information systems where for any intervention there is unlikely to be a regular or consistent outcome ( Oates, 2011 ; Pawson, 2006 ; Rousseau, Manning, & Denyer, 2008 ).

To circumvent these limitations, Pawson, Greenhalgh, Harvey, and Walshe (2005) have proposed a new approach for synthesizing knowledge that seeks to unpack the mechanism of how “complex interventions” work in particular contexts. The basic research question — what works? — which is usually associated with systematic reviews changes to: what is it about this intervention that works, for whom, in what circumstances, in what respects and why? Realist reviews have no particular preference for either quantitative or qualitative evidence. As a theory-building approach, a realist review usually starts by articulating likely underlying mechanisms and then scrutinizes available evidence to find out whether and where these mechanisms are applicable ( Shepperd et al., 2009 ). Primary studies found in the extant literature are viewed as case studies which can test and modify the initial theories ( Rousseau et al., 2008 ).

The main objective pursued in the realist review conducted by Otte-Trojel, de Bont, Rundall, and van de Klundert (2014) was to examine how patient portals contribute to health service delivery and patient outcomes. The specific goals were to investigate how outcomes are produced and, most importantly, how variations in outcomes can be explained. The research team started with an exploratory review of background documents and research studies to identify ways in which patient portals may contribute to health service delivery and patient outcomes. The authors identified six main ways which represent “educated guesses” to be tested against the data in the evaluation studies. These studies were identified through a formal and systematic search in four databases between 2003 and 2013. Two members of the research team selected the articles using a pre-established list of inclusion and exclusion criteria and following a two-step procedure. The authors then extracted data from the selected articles and created several tables, one for each outcome category. They organized information to bring forward those mechanisms where patient portals contribute to outcomes and the variation in outcomes across different contexts.

9.3.6. Critical Reviews

Lastly, critical reviews aim to provide a critical evaluation and interpretive analysis of existing literature on a particular topic of interest to reveal strengths, weaknesses, contradictions, controversies, inconsistencies, and/or other important issues with respect to theories, hypotheses, research methods or results ( Baumeister & Leary, 1997 ; Kirkevold, 1997 ). Unlike other review types, critical reviews attempt to take a reflective account of the research that has been done in a particular area of interest, and assess its credibility by using appraisal instruments or critical interpretive methods. In this way, critical reviews attempt to constructively inform other scholars about the weaknesses of prior research and strengthen knowledge development by giving focus and direction to studies for further improvement ( Kirkevold, 1997 ).

Kitsiou, Paré, and Jaana (2013) provide an example of a critical review that assessed the methodological quality of prior systematic reviews of home telemonitoring studies for chronic patients. The authors conducted a comprehensive search on multiple databases to identify eligible reviews and subsequently used a validated instrument to conduct an in-depth quality appraisal. Results indicate that the majority of systematic reviews in this particular area suffer from important methodological flaws and biases that impair their internal validity and limit their usefulness for clinical and decision-making purposes. To this end, they provide a number of recommendations to strengthen knowledge development towards improving the design and execution of future reviews on home telemonitoring.

9.4. Summary

Table 9.1 outlines the main types of literature reviews that were described in the previous sub-sections and summarizes the main characteristics that distinguish one review type from another. It also includes key references to methodological guidelines and useful sources that can be used by eHealth scholars and researchers for planning and developing reviews.

Typology of Literature Reviews (adapted from Paré et al., 2015).

As shown in Table 9.1 , each review type addresses different kinds of research questions or objectives, which subsequently define and dictate the methods and approaches that need to be used to achieve the overarching goal(s) of the review. For example, in the case of narrative reviews, there is greater flexibility in searching and synthesizing articles ( Green et al., 2006 ). Researchers are often relatively free to use a diversity of approaches to search, identify, and select relevant scientific articles, describe their operational characteristics, present how the individual studies fit together, and formulate conclusions. On the other hand, systematic reviews are characterized by their high level of systematicity, rigour, and use of explicit methods, based on an “a priori” review plan that aims to minimize bias in the analysis and synthesis process (Higgins & Green, 2008). Some reviews are exploratory in nature (e.g., scoping/mapping reviews), whereas others may be conducted to discover patterns (e.g., descriptive reviews) or involve a synthesis approach that may include the critical analysis of prior research ( Paré et al., 2015 ). Hence, in order to select the most appropriate type of review, it is critical to know before embarking on a review project, why the research synthesis is conducted and what type of methods are best aligned with the pursued goals.

9.5. Concluding Remarks

In light of the increased use of evidence-based practice and research generating stronger evidence ( Grady et al., 2011 ; Lyden et al., 2013 ), review articles have become essential tools for summarizing, synthesizing, integrating or critically appraising prior knowledge in the eHealth field. As mentioned earlier, when rigorously conducted review articles represent powerful information sources for eHealth scholars and practitioners looking for state-of-the-art evidence. The typology of literature reviews we used herein will allow eHealth researchers, graduate students and practitioners to gain a better understanding of the similarities and differences between review types.

We must stress that this classification scheme does not privilege any specific type of review as being of higher quality than another ( Paré et al., 2015 ). As explained above, each type of review has its own strengths and limitations. Having said that, we realize that the methodological rigour of any review — be it qualitative, quantitative or mixed — is a critical aspect that should be considered seriously by prospective authors. In the present context, the notion of rigour refers to the reliability and validity of the review process described in section 9.2. For one thing, reliability is related to the reproducibility of the review process and steps, which is facilitated by a comprehensive documentation of the literature search process, extraction, coding and analysis performed in the review. Whether the search is comprehensive or not, whether it involves a methodical approach for data extraction and synthesis or not, it is important that the review documents in an explicit and transparent manner the steps and approach that were used in the process of its development. Next, validity characterizes the degree to which the review process was conducted appropriately. It goes beyond documentation and reflects decisions related to the selection of the sources, the search terms used, the period of time covered, the articles selected in the search, and the application of backward and forward searches ( vom Brocke et al., 2009 ). In short, the rigour of any review article is reflected by the explicitness of its methods (i.e., transparency) and the soundness of the approach used. We refer those interested in the concepts of rigour and quality to the work of Templier and Paré (2015) which offers a detailed set of methodological guidelines for conducting and evaluating various types of review articles.

To conclude, our main objective in this chapter was to demystify the various types of literature reviews that are central to the continuous development of the eHealth field. It is our hope that our descriptive account will serve as a valuable source for those conducting, evaluating or using reviews in this important and growing domain.

• Ammenwerth E., de Keizer N. An inventory of evaluation studies of information technology in health care. Trends in evaluation research, 1982-2002. International Journal of Medical Informatics. 2004; 44 (1):44–56. [ PubMed : 15778794 ]
• Anderson S., Allen P., Peckham S., Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. 2008; 6 (7):1–12. [ PMC free article : PMC2500008 ] [ PubMed : 18613961 ] [ CrossRef ]
• Archer N., Fevrier-Thomas U., Lokker C., McKibbon K. A., Straus S.E. Personal health records: a scoping review. Journal of American Medical Informatics Association. 2011; 18 (4):515–522. [ PMC free article : PMC3128401 ] [ PubMed : 21672914 ]
• Arksey H., O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005; 8 (1):19–32.
• A systematic, tool-supported method for conducting literature reviews in information systems. Paper presented at the Proceedings of the 19th European Conference on Information Systems ( ecis 2011); June 9 to 11; Helsinki, Finland. 2011.
• Baumeister R. F., Leary M.R. Writing narrative literature reviews. Review of General Psychology. 1997; 1 (3):311–320.
• Becker L. A., Oxman A.D. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Overviews of reviews; pp. 607–631.
• Borenstein M., Hedges L., Higgins J., Rothstein H. Introduction to meta-analysis. Hoboken, nj : John Wiley & Sons Inc; 2009.
• Cook D. J., Mulrow C. D., Haynes B. Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine. 1997; 126 (5):376–380. [ PubMed : 9054282 ]
• Cooper H., Hedges L.V. In: The handbook of research synthesis and meta-analysis. 2nd ed. Cooper H., Hedges L. V., Valentine J. C., editors. New York: Russell Sage Foundation; 2009. Research synthesis as a scientific process; pp. 3–17.
• Cooper H. M. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society. 1988; 1 (1):104–126.
• Cronin P., Ryan F., Coughlan M. Undertaking a literature review: a step-by-step approach. British Journal of Nursing. 2008; 17 (1):38–43. [ PubMed : 18399395 ]
• Darlow S., Wen K.Y. Development testing of mobile health interventions for cancer patient self-management: A review. Health Informatics Journal. 2015 (online before print). [ PubMed : 25916831 ] [ CrossRef ]
• Daudt H. M., van Mossel C., Scott S.J. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. bmc Medical Research Methodology. 2013; 13 :48. [ PMC free article : PMC3614526 ] [ PubMed : 23522333 ] [ CrossRef ]
• Davies P. The relevance of systematic reviews to educational policy and practice. Oxford Review of Education. 2000; 26 (3-4):365–378.
• Deeks J. J., Higgins J. P. T., Altman D.G. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Analysing data and undertaking meta-analyses; pp. 243–296.
• Deshazo J. P., Lavallie D. L., Wolf F.M. Publication trends in the medical informatics literature: 20 years of “Medical Informatics” in mesh . bmc Medical Informatics and Decision Making. 2009; 9 :7. [ PMC free article : PMC2652453 ] [ PubMed : 19159472 ] [ CrossRef ]
• Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy. 2005; 10 (1):45–53. [ PubMed : 15667704 ]
• Finfgeld-Connett D., Johnson E.D. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. Journal of Advanced Nursing. 2013; 69 (1):194–204. [ PMC free article : PMC3424349 ] [ PubMed : 22591030 ]
• Grady B., Myers K. M., Nelson E. L., Belz N., Bennett L., Carnahan L. … Guidelines Working Group. Evidence-based practice for telemental health. Telemedicine Journal and E Health. 2011; 17 (2):131–148. [ PubMed : 21385026 ]
• Green B. N., Johnson C. D., Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. Journal of Chiropractic Medicine. 2006; 5 (3):101–117. [ PMC free article : PMC2647067 ] [ PubMed : 19674681 ]
• Greenhalgh T., Wong G., Westhorp G., Pawson R. Protocol–realist and meta-narrative evidence synthesis: evolving standards ( rameses ). bmc Medical Research Methodology. 2011; 11 :115. [ PMC free article : PMC3173389 ] [ PubMed : 21843376 ]
• Gurol-Urganci I., de Jongh T., Vodopivec-Jamsek V., Atun R., Car J. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database System Review. 2013; 12 cd 007458. [ PMC free article : PMC6485985 ] [ PubMed : 24310741 ] [ CrossRef ]
• Hart C. Doing a literature review: Releasing the social science research imagination. London: SAGE Publications; 1998.
• Higgins J. P. T., Green S., editors. Cochrane handbook for systematic reviews of interventions: Cochrane book series. Hoboken, nj : Wiley-Blackwell; 2008.
• Jesson J., Matheson L., Lacey F.M. Doing your literature review: traditional and systematic techniques. Los Angeles & London: SAGE Publications; 2011.
• King W. R., He J. Understanding the role and methods of meta-analysis in IS research. Communications of the Association for Information Systems. 2005; 16 :1.
• Kirkevold M. Integrative nursing research — an important strategy to further the development of nursing science and nursing practice. Journal of Advanced Nursing. 1997; 25 (5):977–984. [ PubMed : 9147203 ]
• Kitchenham B., Charters S. ebse Technical Report Version 2.3. Keele & Durham. uk : Keele University & University of Durham; 2007. Guidelines for performing systematic literature reviews in software engineering.
• Kitsiou S., Paré G., Jaana M. Systematic reviews and meta-analyses of home telemonitoring interventions for patients with chronic diseases: a critical assessment of their methodological quality. Journal of Medical Internet Research. 2013; 15 (7):e150. [ PMC free article : PMC3785977 ] [ PubMed : 23880072 ]
• Kitsiou S., Paré G., Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. Journal of Medical Internet Research. 2015; 17 (3):e63. [ PMC free article : PMC4376138 ] [ PubMed : 25768664 ]
• Levac D., Colquhoun H., O’Brien K. K. Scoping studies: advancing the methodology. Implementation Science. 2010; 5 (1):69. [ PMC free article : PMC2954944 ] [ PubMed : 20854677 ]
• Levy Y., Ellis T.J. A systems approach to conduct an effective literature review in support of information systems research. Informing Science. 2006; 9 :181–211.
• Liberati A., Altman D. G., Tetzlaff J., Mulrow C., Gøtzsche P. C., Ioannidis J. P. A. et al. Moher D. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine. 2009; 151 (4):W-65. [ PubMed : 19622512 ]
• Lyden J. R., Zickmund S. L., Bhargava T. D., Bryce C. L., Conroy M. B., Fischer G. S. et al. McTigue K. M. Implementing health information technology in a patient-centered manner: Patient experiences with an online evidence-based lifestyle intervention. Journal for Healthcare Quality. 2013; 35 (5):47–57. [ PubMed : 24004039 ]
• Mickan S., Atherton H., Roberts N. W., Heneghan C., Tilson J.K. Use of handheld computers in clinical practice: a systematic review. bmc Medical Informatics and Decision Making. 2014; 14 :56. [ PMC free article : PMC4099138 ] [ PubMed : 24998515 ]
• Moher D. The problem of duplicate systematic reviews. British Medical Journal. 2013; 347 (5040) [ PubMed : 23945367 ] [ CrossRef ]
• Montori V. M., Wilczynski N. L., Morgan D., Haynes R. B., Hedges T. Systematic reviews: a cross-sectional study of location and citation counts. bmc Medicine. 2003; 1 :2. [ PMC free article : PMC281591 ] [ PubMed : 14633274 ]
• Mulrow C. D. The medical review article: state of the science. Annals of Internal Medicine. 1987; 106 (3):485–488. [ PubMed : 3813259 ] [ CrossRef ]
• Evidence-based information systems: A decade later. Proceedings of the European Conference on Information Systems ; 2011. Retrieved from http://aisel ​.aisnet.org/cgi/viewcontent ​.cgi?article ​=1221&context ​=ecis2011 .
• Okoli C., Schabram K. A guide to conducting a systematic literature review of information systems research. ssrn Electronic Journal. 2010
• Otte-Trojel T., de Bont A., Rundall T. G., van de Klundert J. How outcomes are achieved through patient portals: a realist review. Journal of American Medical Informatics Association. 2014; 21 (4):751–757. [ PMC free article : PMC4078283 ] [ PubMed : 24503882 ]
• Paré G., Trudel M.-C., Jaana M., Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management. 2015; 52 (2):183–199.
• Patsopoulos N. A., Analatos A. A., Ioannidis J.P. A. Relative citation impact of various study designs in the health sciences. Journal of the American Medical Association. 2005; 293 (19):2362–2366. [ PubMed : 15900006 ]
• Paul M. M., Greene C. M., Newton-Dame R., Thorpe L. E., Perlman S. E., McVeigh K. H., Gourevitch M.N. The state of population health surveillance using electronic health records: A narrative review. Population Health Management. 2015; 18 (3):209–216. [ PubMed : 25608033 ]
• Pawson R. Evidence-based policy: a realist perspective. London: SAGE Publications; 2006.
• Pawson R., Greenhalgh T., Harvey G., Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy. 2005; 10 (Suppl 1):21–34. [ PubMed : 16053581 ]
• Petersen K., Vakkalanka S., Kuzniarz L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology. 2015; 64 :1–18.
• Petticrew M., Roberts H. Systematic reviews in the social sciences: A practical guide. Malden, ma : Blackwell Publishing Co; 2006.
• Rousseau D. M., Manning J., Denyer D. Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. The Academy of Management Annals. 2008; 2 (1):475–515.
• Rowe F. What literature review is not: diversity, boundaries and recommendations. European Journal of Information Systems. 2014; 23 (3):241–255.
• Shea B. J., Hamel C., Wells G. A., Bouter L. M., Kristjansson E., Grimshaw J. et al. Boers M. amstar is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology. 2009; 62 (10):1013–1020. [ PubMed : 19230606 ]
• Shepperd S., Lewin S., Straus S., Clarke M., Eccles M. P., Fitzpatrick R. et al. Sheikh A. Can we systematically review studies that evaluate complex interventions? PLoS Medicine. 2009; 6 (8):e1000086. [ PMC free article : PMC2717209 ] [ PubMed : 19668360 ]
• Silva B. M., Rodrigues J. J., de la Torre Díez I., López-Coronado M., Saleem K. Mobile-health: A review of current state in 2015. Journal of Biomedical Informatics. 2015; 56 :265–272. [ PubMed : 26071682 ]
• Smith V., Devane D., Begley C., Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. bmc Medical Research Methodology. 2011; 11 (1):15. [ PMC free article : PMC3039637 ] [ PubMed : 21291558 ]
• Sylvester A., Tate M., Johnstone D. Beyond synthesis: re-presenting heterogeneous research literature. Behaviour & Information Technology. 2013; 32 (12):1199–1215.
• Templier M., Paré G. A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems. 2015; 37 (6):112–137.
• Thomas J., Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. bmc Medical Research Methodology. 2008; 8 (1):45. [ PMC free article : PMC2478656 ] [ PubMed : 18616818 ]
• Reconstructing the giant: on the importance of rigour in documenting the literature search process. Paper presented at the Proceedings of the 17th European Conference on Information Systems ( ecis 2009); Verona, Italy. 2009.
• Webster J., Watson R.T. Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly. 2002; 26 (2):11.
• Whitlock E. P., Lin J. S., Chou R., Shekelle P., Robinson K.A. Using existing systematic reviews in complex systematic reviews. Annals of Internal Medicine. 2008; 148 (10):776–782. [ PubMed : 18490690 ]

This publication is licensed under a Creative Commons License, Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0): see https://creativecommons.org/licenses/by-nc/4.0/

• Cite this Page Paré G, Kitsiou S. Chapter 9 Methods for Literature Reviews. In: Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
• PDF version of this title (4.5M)
• Disable Glossary Links

In this Page

• Introduction
• Overview of the Literature Review Process and Steps
• Types of Review Articles and Brief Illustrations
• Concluding Remarks

Related information

• PMC PubMed Central citations
• PubMed Links to PubMed

Recent Activity

• Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Ev... Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Evidence-based Approach

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

Methodological Approaches to Literature Review

• Living reference work entry
• First Online: 09 May 2023
• Cite this living reference work entry

• Dennis Thomas 2 ,
• Elida Zairina 3 &
• Johnson George 4  

523 Accesses

1 Citations

The literature review can serve various functions in the contexts of education and research. It aids in identifying knowledge gaps, informing research methodology, and developing a theoretical framework during the planning stages of a research study or project, as well as reporting of review findings in the context of the existing literature. This chapter discusses the methodological approaches to conducting a literature review and offers an overview of different types of reviews. There are various types of reviews, including narrative reviews, scoping reviews, and systematic reviews with reporting strategies such as meta-analysis and meta-synthesis. Review authors should consider the scope of the literature review when selecting a type and method. Being focused is essential for a successful review; however, this must be balanced against the relevance of the review to a broad audience.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Institutional subscriptions

Akobeng AK. Principles of evidence based medicine. Arch Dis Child. 2005;90(8):837–40.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Alharbi A, Stevenson M. Refining Boolean queries to identify relevant studies for systematic review updates. J Am Med Inform Assoc. 2020;27(11):1658–66.

Article   PubMed   PubMed Central   Google Scholar  

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.

Article   Google Scholar  

Aromataris E MZE. JBI manual for evidence synthesis. 2020.

Google Scholar  

Aromataris E, Pearson A. The systematic review: an overview. Am J Nurs. 2014;114(3):53–8.

Article   PubMed   Google Scholar  

Aromataris E, Riitano D. Constructing a search strategy and searching for evidence. A guide to the literature search for a systematic review. Am J Nurs. 2014;114(5):49–56.

Babineau J. Product review: covidence (systematic review software). J Canad Health Libr Assoc Canada. 2014;35(2):68–71.

Baker JD. The purpose, process, and methods of writing a literature review. AORN J. 2016;103(3):265–9.

Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7(9):e1000326.

Bramer WM, Rethlefsen ML, Kleijnen J, Franco OH. Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study. Syst Rev. 2017;6(1):1–12.

Brown D. A review of the PubMed PICO tool: using evidence-based practice in health education. Health Promot Pract. 2020;21(4):496–8.

Cargo M, Harris J, Pantoja T, et al. Cochrane qualitative and implementation methods group guidance series – paper 4: methods for assessing evidence on intervention implementation. J Clin Epidemiol. 2018;97:59–69.

Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med. 1997;126(5):376–80.

Article   CAS   PubMed   Google Scholar  

Counsell C. Formulating questions and locating primary studies for inclusion in systematic reviews. Ann Intern Med. 1997;127(5):380–7.

Cummings SR, Browner WS, Hulley SB. Conceiving the research question and developing the study plan. In: Cummings SR, Browner WS, Hulley SB, editors. Designing Clinical Research: An Epidemiological Approach. 4th ed. Philadelphia (PA): P Lippincott Williams & Wilkins; 2007. p. 14–22.

Eriksen MB, Frandsen TF. The impact of patient, intervention, comparison, outcome (PICO) as a search strategy tool on literature search quality: a systematic review. JMLA. 2018;106(4):420.

Ferrari R. Writing narrative style literature reviews. Medical Writing. 2015;24(4):230–5.

Flemming K, Booth A, Hannes K, Cargo M, Noyes J. Cochrane qualitative and implementation methods group guidance series – paper 6: reporting guidelines for qualitative, implementation, and process evaluation evidence syntheses. J Clin Epidemiol. 2018;97:79–85.

Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Inf Libr J. 2009;26(2):91–108.

Green BN, Johnson CD, Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. J Chiropr Med. 2006;5(3):101–17.

Gregory AT, Denniss AR. An introduction to writing narrative and systematic reviews; tasks, tips and traps for aspiring authors. Heart Lung Circ. 2018;27(7):893–8.

Harden A, Thomas J, Cargo M, et al. Cochrane qualitative and implementation methods group guidance series – paper 5: methods for integrating qualitative and implementation evidence within intervention effectiveness reviews. J Clin Epidemiol. 2018;97:70–8.

Harris JL, Booth A, Cargo M, et al. Cochrane qualitative and implementation methods group guidance series – paper 2: methods for question formulation, searching, and protocol development for qualitative evidence synthesis. J Clin Epidemiol. 2018;97:39–48.

Higgins J, Thomas J. In: Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.3, updated February 2022). Available from www.training.cochrane.org/handbook.: Cochrane; 2022.

International prospective register of systematic reviews (PROSPERO). Available from https://www.crd.york.ac.uk/prospero/ .

Khan KS, Kunz R, Kleijnen J, Antes G. Five steps to conducting a systematic review. J R Soc Med. 2003;96(3):118–21.

Landhuis E. Scientific literature: information overload. Nature. 2016;535(7612):457–8.

Lockwood C, Porritt K, Munn Z, Rittenmeyer L, Salmond S, Bjerrum M, Loveday H, Carrier J, Stannard D. Chapter 2: Systematic reviews of qualitative evidence. In: Aromataris E, Munn Z, editors. JBI Manual for Evidence Synthesis. JBI; 2020. Available from https://synthesismanual.jbi.global . https://doi.org/10.46658/JBIMES-20-03 .

Chapter   Google Scholar  

Lorenzetti DL, Topfer L-A, Dennett L, Clement F. Value of databases other than medline for rapid health technology assessments. Int J Technol Assess Health Care. 2014;30(2):173–8.

Moher D, Liberati A, Tetzlaff J, Altman DG, the PRISMA Group. Preferred reporting items for (SR) and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;6:264–9.

Mulrow CD. Systematic reviews: rationale for systematic reviews. BMJ. 1994;309(6954):597–9.

Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018;18(1):143.

Munthe-Kaas HM, Glenton C, Booth A, Noyes J, Lewin S. Systematic mapping of existing tools to appraise methodological strengths and limitations of qualitative research: first stage in the development of the CAMELOT tool. BMC Med Res Methodol. 2019;19(1):1–13.

Murphy CM. Writing an effective review article. J Med Toxicol. 2012;8(2):89–90.

NHMRC. Guidelines for guidelines: assessing risk of bias. Available at https://nhmrc.gov.au/guidelinesforguidelines/develop/assessing-risk-bias . Last published 29 August 2019. Accessed 29 Aug 2022.

Noyes J, Booth A, Cargo M, et al. Cochrane qualitative and implementation methods group guidance series – paper 1: introduction. J Clin Epidemiol. 2018b;97:35–8.

Noyes J, Booth A, Flemming K, et al. Cochrane qualitative and implementation methods group guidance series – paper 3: methods for assessing methodological limitations, data extraction and synthesis, and confidence in synthesized qualitative findings. J Clin Epidemiol. 2018a;97:49–58.

Noyes J, Booth A, Moore G, Flemming K, Tunçalp Ö, Shakibazadeh E. Synthesising quantitative and qualitative evidence to inform guidelines on complex interventions: clarifying the purposes, designs and outlining some methods. BMJ Glob Health. 2019;4(Suppl 1):e000893.

Peters MD, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. Int J Evid Healthcare. 2015;13(3):141–6.

Polanin JR, Pigott TD, Espelage DL, Grotpeter JK. Best practice guidelines for abstract screening large-evidence systematic reviews and meta-analyses. Res Synth Methods. 2019;10(3):330–42.

Article   PubMed Central   Google Scholar  

Shea BJ, Grimshaw JM, Wells GA, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7(1):1–7.

Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. Brit Med J. 2017;358

Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. Br Med J. 2016;355

Stroup DF, Berlin JA, Morton SC, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. JAMA. 2000;283(15):2008–12.

Tawfik GM, Dila KAS, Mohamed MYF, et al. A step by step guide for conducting a systematic review and meta-analysis with simulation data. Trop Med Health. 2019;47(1):1–9.

The Critical Appraisal Program. Critical appraisal skills program. Available at https://casp-uk.net/ . 2022. Accessed 29 Aug 2022.

The University of Melbourne. Writing a literature review in Research Techniques 2022. Available at https://students.unimelb.edu.au/academic-skills/explore-our-resources/research-techniques/reviewing-the-literature . Accessed 29 Aug 2022.

The Writing Center University of Winconsin-Madison. Learn how to write a literature review in The Writer’s Handbook – Academic Professional Writing. 2022. Available at https://writing.wisc.edu/handbook/assignments/reviewofliterature/ . Accessed 29 Aug 2022.

Thompson SG, Sharp SJ. Explaining heterogeneity in meta-analysis: a comparison of methods. Stat Med. 1999;18(20):2693–708.

Tricco AC, Lillie E, Zarin W, et al. A scoping review on the conduct and reporting of scoping reviews. BMC Med Res Methodol. 2016;16(1):15.

Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

Yoneoka D, Henmi M. Clinical heterogeneity in random-effect meta-analysis: between-study boundary estimate problem. Stat Med. 2019;38(21):4131–45.

Yuan Y, Hunt RH. Systematic reviews: the good, the bad, and the ugly. Am J Gastroenterol. 2009;104(5):1086–92.

Download references

Author information

Authors and affiliations.

Centre of Excellence in Treatable Traits, College of Health, Medicine and Wellbeing, University of Newcastle, Hunter Medical Research Institute Asthma and Breathing Programme, Newcastle, NSW, Australia

Dennis Thomas

Department of Pharmacy Practice, Faculty of Pharmacy, Universitas Airlangga, Surabaya, Indonesia

Elida Zairina

Centre for Medicine Use and Safety, Monash Institute of Pharmaceutical Sciences, Faculty of Pharmacy and Pharmaceutical Sciences, Monash University, Parkville, VIC, Australia

Johnson George

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Johnson George .

Section Editor information

College of Pharmacy, Qatar University, Doha, Qatar

Derek Charles Stewart

Department of Pharmacy, University of Huddersfield, Huddersfield, United Kingdom

Zaheer-Ud-Din Babar

Rights and permissions

Reprints and permissions

Copyright information

© 2023 Springer Nature Switzerland AG

About this entry

Cite this entry.

Thomas, D., Zairina, E., George, J. (2023). Methodological Approaches to Literature Review. In: Encyclopedia of Evidence in Pharmaceutical Public Health and Health Services Research in Pharmacy. Springer, Cham. https://doi.org/10.1007/978-3-030-50247-8_57-1

Download citation

DOI : https://doi.org/10.1007/978-3-030-50247-8_57-1

Received : 22 February 2023

Accepted : 22 February 2023

Published : 09 May 2023

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-50247-8

Online ISBN : 978-3-030-50247-8

eBook Packages : Springer Reference Biomedicine and Life Sciences Reference Module Biomedical and Life Sciences

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Tutorial: Evaluating Information: Scholarly Literature Types

• Evaluating Information
• Scholarly Literature Types
• Primary vs. Secondary Articles
• Peer Review
• Systematic Reviews & Meta-Analysis
• Gray Literature
• Evaluating Like a Boss
• Evaluating AV

Types of scholarly literature

You will encounter many types of articles and it is important to distinguish between these different categories of scholarly literature. Keep in mind the following definitions.

Peer-reviewed (or refereed):  Refers to articles that have undergone a rigorous review process, often including revisions to the original manuscript, by peers in their discipline, before publication in a scholarly journal. This can include empirical studies, review articles, meta-analyses among others.

Empirical study (or primary article): An empirical study is one that aims to gain new knowledge on a topic through direct or indirect observation and research. These include quantitative or qualitative data and analysis. In science, an empirical article will often include the following sections: Introduction, Methods, Results, and Discussion.

Review article:  In the scientific literature, this is a type of article that provides a synthesis of existing research on a particular topic. These are useful when you want to get an idea of a body of research that you are not yet familiar with. It differs from a systematic review in that it does not aim to capture ALL of the research on a particular topic.

Systematic review:  This is a methodical and thorough literature review focused on a particular research question. It's aim is to identify and synthesize all of the scholarly research on a particular topic in an unbiased, reproducible way to provide evidence for practice and policy-making. It may involve a meta-analysis (see below). 

Meta-analysis:  This is a type of research study that combines or contrasts data from different independent studies in a new analysis in order to strengthen the understanding of a particular topic. There are many methods, some complex, applied to performing this type of analysis.

Types of non-formally published scholarly literature

What is gray literature.

Gray (or grey) literature is literature produced by individuals or organizations outside of commercial and/or academic publishers. This type of non-formally published substantive information (often not formally peer-reviewed; especially important in all kinds of sciences) can include information such:

• theses and dissertations
• technical reports  
• working papers 
• government reports
• evaluation and think tank reports and resources
• conference proceedings, papers and posters
• publications from NGOs, INGOs, think tanks and policy institutes
• unpublished clinical trials
• and much more

The sources you select will be informed by your research question and field of study, but should likely include, at a minimum, theses and dissertations.

Why Search the Gray Literature?

Most of gray literature is considered less prestigious, reliable, and "official" than publication in a peer-reviewed journal. But they are still fully legitimate avenues of publication. Often they are used to publicize early findings, before a study is entirely complete. Or, in the case of theses, they are published as a condition of receiving an advanced degree. Government technical reports are issued either by agencies that do scientific research themselves or else by a lab that has received government funding. Increasingly, such labs may be required to publish technical reports as a condition of receiving such funding. Gray literature may be cited like any other paper although with the caveat mentioned before that it is considered less "official" and reliable than peer-reviewed scientific papers.

When doing evidence synthesiis, it's important because the intent is to synthesize  all available evidence  that is applicable to your research question. There is a strong bias in scientific publishing toward publishing studies that show some sort of significant effect. Meanwhile, many studies and trials that show no effect end up going unpublished. But knowing that an intervention had no effect is just as important as knowing that it did have an effect when it comes to making decisions for practice and policy-making. While not peer-reviewed, gray literature represents a valuable body of information that is critical to consider when synthesizing and evaluating all available evidence.

• << Previous: Evaluating Information
• Next: Primary vs. Secondary Articles >>
• Last Updated: Oct 20, 2021 11:11 AM
• URL: https://guides.library.cornell.edu/evaluate

• UConn Library
• Literature Review: The What, Why and How-to Guide
• Introduction

Literature Review: The What, Why and How-to Guide — Introduction

• Getting Started
• How to Pick a Topic
• Strategies to Find Sources
• Evaluating Sources & Lit. Reviews
• Tips for Writing Literature Reviews
• Writing Literature Review: Useful Sites
• Citation Resources
• Other Academic Writings

What are Literature Reviews?

So, what is a literature review? "A literature review is an account of what has been published on a topic by accredited scholars and researchers. In writing the literature review, your purpose is to convey to your reader what knowledge and ideas have been established on a topic, and what their strengths and weaknesses are. As a piece of writing, the literature review must be defined by a guiding concept (e.g., your research objective, the problem or issue you are discussing, or your argumentative thesis). It is not just a descriptive list of the material available, or a set of summaries." Taylor, D.  The literature review: A few tips on conducting it . University of Toronto Health Sciences Writing Centre.

Goals of Literature Reviews

What are the goals of creating a Literature Review?  A literature could be written to accomplish different aims:

• To develop a theory or evaluate an existing theory
• To summarize the historical or existing state of a research topic
• Identify a problem in a field of research 

Baumeister, R. F., & Leary, M. R. (1997). Writing narrative literature reviews .  Review of General Psychology , 1 (3), 311-320.

What kinds of sources require a Literature Review?

• A research paper assigned in a course
• A thesis or dissertation
• A grant proposal
• An article intended for publication in a journal

All these instances require you to collect what has been written about your research topic so that you can demonstrate how your own research sheds new light on the topic.

Types of Literature Reviews

What kinds of literature reviews are written?

Narrative review: The purpose of this type of review is to describe the current state of the research on a specific topic/research and to offer a critical analysis of the literature reviewed. Studies are grouped by research/theoretical categories, and themes and trends, strengths and weakness, and gaps are identified. The review ends with a conclusion section which summarizes the findings regarding the state of the research of the specific study, the gaps identify and if applicable, explains how the author's research will address gaps identify in the review and expand the knowledge on the topic reviewed.

• Example : Predictors and Outcomes of U.S. Quality Maternity Leave: A Review and Conceptual Framework:  10.1177/08948453211037398  

Systematic review : "The authors of a systematic review use a specific procedure to search the research literature, select the studies to include in their review, and critically evaluate the studies they find." (p. 139). Nelson, L. K. (2013). Research in Communication Sciences and Disorders . Plural Publishing.

• Example : The effect of leave policies on increasing fertility: a systematic review:  10.1057/s41599-022-01270-w

Meta-analysis : "Meta-analysis is a method of reviewing research findings in a quantitative fashion by transforming the data from individual studies into what is called an effect size and then pooling and analyzing this information. The basic goal in meta-analysis is to explain why different outcomes have occurred in different studies." (p. 197). Roberts, M. C., & Ilardi, S. S. (2003). Handbook of Research Methods in Clinical Psychology . Blackwell Publishing.

• Example : Employment Instability and Fertility in Europe: A Meta-Analysis:  10.1215/00703370-9164737

Meta-synthesis : "Qualitative meta-synthesis is a type of qualitative study that uses as data the findings from other qualitative studies linked by the same or related topic." (p.312). Zimmer, L. (2006). Qualitative meta-synthesis: A question of dialoguing with texts .  Journal of Advanced Nursing , 53 (3), 311-318.

• Example : Women’s perspectives on career successes and barriers: A qualitative meta-synthesis:  10.1177/05390184221113735

Literature Reviews in the Health Sciences

• UConn Health subject guide on systematic reviews Explanation of the different review types used in health sciences literature as well as tools to help you find the right review type
• << Previous: Getting Started
• Next: How to Pick a Topic >>
• Last Updated: Sep 21, 2022 2:16 PM
• URL: https://guides.lib.uconn.edu/literaturereview

ACAP LEARNING RESOURCES

ACAP Pathfinder: Literature Review

• What is a Literature Review?
• Examples of Literature Reviews
• The Research Question

Types of Literature

• How to Search
• Recording the Search
• Reference Management
• Evaluating the Literature

Your research question or thesis statement will inform the types of literature that will best suit the review. There are many different types literature and they come from a variety of sources. The resources described below provide you with a general outline of the types of literature available via the library. Go to the Choosing Resources page in the Information Skills guide to learn more about where and how to find these resources in the library and on the internet.

• THEORY-BASED
• PHILOSOPHICAL
• STATISTICAL

Research Literature

Some edited books, journal articles, theses and government publications will employ research paradigms and methodologies to support a hypothesis. Methodologies can be broadly categorised as longitudinal, qualitative or quantitative. Many of these publications are peer-reviewed, meaning they have been checked by a panel of experts before publication. Most library databases offer a peer-reviewed checkbox which will filter search results in this way. They will also allow you to filter results according to research methodology, focus group, geographical location and much more. Research literature is an essential component of your literature review.

Theory-based Literature

Literature that is informed and tested by research, these books, articles and reference sources will attempt to explain, describe, define and provide a background or theoretical framework for a field of inquiry. These sources may include the original works of primary theorists as well as works which build upon, critique and discuss these primary sources while connecting it to the latest research. This type of literature is also an important component of any literature review.

Philosophical Literature

Information sources such as books and articles, which deal with the underlying beliefs, attitudes and concepts that form the basic assumptions or building blocks within a profession or field of study.  This kind of literature formulates critical inquiry from either an  ethical, epistemological, metaphysical or logical standpoint.  The extent to which you use philosophical literature will depend on the focus and subject matter of your review but it may be useful when constructing a background or theoretical base in your writing.

Empirical or Practice-based Literature

Statistical Reports

Grey Literature

Types of Resources

• FILM & VIDEO
• NEWS & MAGAZINES
• REFERENCE SOURCES
• CONFERENCES & THESES
• GOVERNMENT & POLICIES

Description

Provide an overview of a subject area or of a number of related topics.  They may also include detailed information about a specific topic or topics. Search for print books using  MultiSearch .  These items may be collected from library shelves or requested from other campuses. eBooks are searchable from  MultiSearch  or  A-Z Databases  and may be read online or downloaded to any PC or device.

Use books to gather comprehensive information on a topic. In the library, you will find mostly academic, non-fiction items which may be used in your assessment tasks to sketch out an overview on a subject or to illustrate an in-depth understanding.

• Essentials of Psychology Concepts and Applications
• Addiction: Psychology and Treatment

Relevant Links

• http://libguides.navitas.com/borrow
• http://libguides.navitas.com/eresources/books

Journal publications, sometimes called periodicals or serials, contain articles which offer research, reports, reviews, letters and other papers on specific topics. They are usually published weekly, monthly, quarterly or yearly. Academic publications are often peer-reviewed and as such provide up-to-date information from authorised sources. Search for journal articles via  MultiSearch  and  A-Z  Databases . Search for journal publications using  A-Z Journals . You can also search for articles on the internet via  Google Scholar  or  Researchgate . 

Journal articles can provide you with more up-to-date information on specific aspects of a topic, and in smaller more digestible packages than books. Use journal publications to access scholarly research on a topic, often from a unique or new perspective.

• International Journal of Clinical & Health Psychology
• Khan, F., Chong, J., Theisen, J., Fraley, R., Young, J., & Hankin, B. (2020). Development and Change in Attachment: A Multiwave Assessment of Attachment and Its Correlates Across Childhood and Adolescence.  Journal of Personality and Social Psychology ,  118 (6), 1188–1206. https://doi.org/10.1037/pspi0000211
• http://libguides.navitas.com/eresources/articles

Audiovisual material in the form of DVDs, podcasts, online streaming and so on, is searchable from  MultiSearch  or  A-Z Databases . Videos in the library include movies, conferences and seminars, tutorials, documentaries and much more.

AV items may be used as instructional material or gain an understanding of a topic by way of visual or concrete examples. 

• Narrative Family Therapy
• Waltz with Bashir
• http://libguides.navitas.com/eresources/videos

News items are published at regular intervals and provide new information about various topics of interest to the general public. Magazines are also produced regularly and may focus on a particular subject area or cover a range of topics, again for consumption by the general public. The information within news publications and magazines are not in themselves scholarly works but may refer to academic sources. These sources can be found by searching  MultiSearch  or  A-Z Databases

To access recent or new information about current affairs, social, economic or political issues which provide an overview or introduction in digestible and readable packages.  While not scholarly information, these resources may be required for use in particular assessment tasks or point towards recent research in a particular field. 

• Australia targeted in cyber crime increase
• New Scientist
• Search for news on Google

Reference items such as dictionaries, encyclopaedias, handbooks and industry standards provide an overview of a subject area, or include specific definitions, technical or practical information. They are searchable via  MultiSearch  and  A-Z Databases . Reference works at ACAP cover a range of topics including but not limited to sociology, social work and philosophy, psychology, counselling and mental illness, legal materials and policies, diagnoses, drug overviews, care planning and best practices for healthcare workers.

Useful for a broad understanding of a topic, theory or theorist or for assessment tasks which require technical or practical information about a particular industry or field of inquiry. 

• Evidence-Based Policy
• Diagnostic and statistical manual of mental disorders : DSM-5

A website is a set of related and interlinked pages which are hosted on the World Wide Web (WWW). A webpage refers to the individual pages contained within these sites. To search the WWW, download an internet browser (we recommend Google Chrome) and use a search engine such as Google to find content.

The websites of particular institutions and government departments may be useful for your assessment tasks. You might also want to look at scholarly sites, such as Google Scholar, Researchgate, State Libraries, academic publishers and journal indexing services, to find the most scholarly information on the Web. Links recommended by the library can be found on the  Useful Websites  page. 

• Google Scholar
• Dulwich Centre Resources

Conference papers are presented at conferences which are usually themed around a specific subject area or set of related topics and presented as a collection of proceedings. Some papers may be peer-reviewed and are searchable from  MultiSearch , within  journal databases ,  Dissertation Express  or via  Libraries Australia Trove database . A thesis or dissertation involves personal research, written by a candidate for an advanced university degree. Theses are also searchable from  MultiSearch ,  Trove , or within individual academic institutional repositories and indexing databases such as  PQDT Open ,  NDLTD ,  CORE  and  DART .

Conference proceedings and theses can provide you with an in-depth look at some of the latest research on specific aspects of a topic.

• Counselor education: A personal growth & personal development experience
• Awareness of memory deficits in Parkinson’s disease
• http://libguides.navitas.com/ill

Legal resources include documents such as, but not limited to Bills, Acts, regulations, statutory laws, by-laws, proceedings of Parliament, legal cases and tribunal decisions. Some of these resources may be found by searching  MultiSearch , in databases such as  AustLII  or  JADE  or within legislative sites for individual states and territories. You will also find cases and tribunal decisions on the websites of regulatory authorities such as  AHPRA .

Use these information sources when you need to refer to current laws, records, cases and decisions in your assessment tasks.   

• AustLII Cases & Legislation
• Library Resources for Criminology & Justice: Law & Legislation

Government policies, reports, gazettes, media releases and parliamentary publications such as Hansards, are available from various websites here in Australia.  You can also search Google to find individual publications.  MultiSearch  and  journal  databases  will include some government papers and reports in search results.  However, you should also directly consult  federal  and  state  departments and agencies, Libraries Australia  GovPubs,  the  Analysis and Policy Observatory  and State and  Federal  Parliamentary libraries. 

Use these sources in your assessment tasks to access up-to-date and authoritative information on subject areas which may be affected by Federal or State government.

• Domestic and family violence and parenting: mixed method insights into impact and support needs - final report
• National Statement on Ethical Conduct in Human Research

Statistical reports and data from sites such as the  Australian Bureau of Statistics  gather information for a particular field of research or to report on the views and habits of the population. Data collection may be performed via interviews, questionnaires, surveys, censuses and so on. You'll also be able to search for and access statistical information using  MultiSearch  and  journal databases . Other important sites for statistics include the  Australian Institute of Family Studies ,  Australian Institute of Health & Welfare  and  HILDA .

An important part of the research in any field of study, statistical reporting and datasets are useful for description, analysis and comparison in your assessment tasks.

• 4329.0.00.003 - Patterns of Use of Mental Health Services and Prescription Medications, 2011
• Parent-child contact after separation
• << Previous: The Research Question
• Next: How to Search >>
• Last Updated: Sep 13, 2023 10:53 AM
• URL: https://libguides.navitas.com/literature-review

Scholarly Literature Types

Types of scholarly literature, non-formally published substantive literature.

• Acknowledgements
• Peer Review This link opens in a new window
• Examples of Academic Sources
• Qualitative vs Quantitative Research
• Anatomy of a Scholarly Article

You will encounter many types of articles and it is important to distinguish between these different categories of scholarly literature. Keep in mind the following definitions.

Peer-reviewed (or refereed):  Refers to articles that have undergone a rigorous review process, often including revisions to the original manuscript, by peers in their discipline, before publication in a scholarly journal. This can include empirical studies, review articles, meta-analyses among others.

Empirical study (or primary article): An empirical study is one that aims to gain new knowledge on a topic through direct or indirect observation and research. These include quantitative or qualitative data and analysis. In science, an empirical article will often include the following sections: Introduction, Methods, Results, and Discussion.

Review article:  In the scientific literature, this is a type of article that provides a synthesis of existing research on a particular topic. These are useful when you want to get an idea of a body of research that you are not yet familiar with. It differs from a systematic review in that it does not aim to capture ALL of the research on a particular topic.

Systematic review:  This is a methodical and thorough literature review focused on a particular research question. It's aim is to identify and synthesize all of the scholarly research on a particular topic in an unbiased, reproducible way to provide evidence for practice and policy-making. It may involve a meta-analysis (see below). 

Meta-analysis:  This is a type of research study that combines or contrasts data from different independent studies in a new analysis in order to strengthen the understanding of a particular topic. There are many methods, some complex, applied to performing this type of analysis.

What is Grey Literature?

Grey literature is literature produced by individuals or organizations outside of commercial and/or academic publishers. This type of non-formally published substantive information (often not formally peer-reviewed; especially important in all kinds of sciences) can include information such:

• theses and dissertations
• technical reports 
• working papers 
• government reports
• evaluation and think tank reports and resources
• conference proceedings, papers and posters
• publications from NGOs, INGOs, think tanks and policy institutes
• unpublished clinical trials
• and much more

The sources you select will be informed by your research question and field of study, but should likely include, at a minimum, theses and dissertations.

Why Search the Gray Literature?

Most of gray literature is considered less prestigious, reliable, and "official" than publication in a peer-reviewed journal. But they are still fully legitimate avenues of publication. Often they are used to publicize early findings, before a study is entirely complete. Or, in the case of theses, they are published as a condition of receiving an advanced degree. Government technical reports are issued either by agencies that do scientific research themselves or else by a lab that has received government funding. Increasingly, such labs may be required to publish technical reports as a condition of receiving such funding. Gray literature may be cited like any other paper although with the caveat mentioned before that it is considered less "official" and reliable than peer-reviewed scientific papers.

When doing evidence synthesis, it's important because the intent is to synthesize  all available evidence  that is applicable to your research question. There is a strong bias in scientific publishing toward publishing studies that show some sort of significant effect. Meanwhile, many studies and trials that show no effect end up going unpublished. But knowing that an intervention had no effect is just as important as knowing that it did have an effect when it comes to making decisions for practice and policy-making. While not peer-reviewed, gray literature represents a valuable body of information that is critical to consider when synthesizing and evaluating all available evidence.

The guide is based on the Cornell University Library Tutorial: Scholarly Literature Types.

• Next: Peer Review >>
• Last Updated: Sep 13, 2023 5:23 PM
• URL: https://mcphs.libguides.com/ScholarlySources

• Locations and Hours
• UCLA Library
• Research Guides
• Biomedical Library Guides

Systematic Reviews

• Types of Literature Reviews

What Makes a Systematic Review Different from Other Types of Reviews?

• Planning Your Systematic Review
• Database Searching
• Creating the Search
• Search Filters and Hedges
• Grey Literature
• Managing and Appraising Results
• Further Resources

Reproduced from Grant, M. J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26: 91–108. doi:10.1111/j.1471-1842.2009.00848.x

• << Previous: Home
• Next: Planning Your Systematic Review >>
• Last Updated: Apr 17, 2024 2:02 PM
• URL: https://guides.library.ucla.edu/systematicreviews

Conducting a Literature Review: Types of Literature

• Introduction
• 1. Choose Your Topic

Types of Literature

• 3. Search the Literature
• 4. Read & Analyze the Literature
• 5. Write the Review
• Keeping Track of Information
• Style Guides

Different types of publications have different characteristics.

Primary Literature Primary sources means original studies, based on direct observation, use of statistical records, interviews, or experimental methods, of actual practices or the actual impact of practices or policies. They are authored by researchers, contains original research data, and are usually published in a peer-reviewed journal. Primary literature may also include conference papers, pre-prints, or preliminary reports. Also called empirical research .

Secondary Literature Secondary literature consists of interpretations and evaluations that are derived from or refer to the primary source literature. Examples include review articles (such as meta-analysis and systematic reviews) and reference works. Professionals within each discipline take the primary literature and synthesize, generalize, and integrate new research.

Tertiary Literature Tertiary literature consists of a distillation and collection of primary and secondary sources such as textbooks, encyclopedia articles, and guidebooks or handbooks. The purpose of tertiary literature is to provide an overview of key research findings and an introduction to principles and practices within the discipline.

Adapted from the Information Services Department of the Library of the Health Sciences-Chicago , University of Illinois at Chicago.

Types of Scientific Publications

These examples and descriptions of publication types will give you an idea of how to use various works and why you would want to write a particular kind of paper.

• Scholarly article aka empirical article
• Review article
• Conference paper

Scholarly (aka empirical) article -- example

Empirical studies use data derived from observation or experiment. Original research papers (also called primary research articles) that describe empirical studies and their results are published in academic journals.  Articles that report empirical research contain different sections which relate to the steps of the scientific method.

      Abstract - The abstract provides a very brief summary of the research.

     Introduction - The introduction sets the research in a context, which provides a review of related research and develops the hypotheses for the research.

     Method - The method section describes how the research was conducted.

     Results - The results section describes the outcomes of the study.

     Discussion - The discussion section contains the interpretations and implications of the study.

     References - A references section lists the articles, books, and other material cited in the report.

Review article -- example

A review article summarizes a particular field of study and places the recent research in context. It provides an overview and is an excellent introduction to a subject area. The references used in a review article are helpful as they lead to more in-depth research.

Many databases have limits or filters to search for review articles. You can also search by keywords like review article, survey, overview, summary, etc.

Conference proceedings, abstracts and reports -- example

Conference proceedings, abstracts and reports are not usually peer-reviewed.  A conference article is similar to a scholarly article insofar as it is academic. Conference articles are published much more quickly than scholarly articles. You can find conference papers in many of the same places as scholarly articles.

How Do You Identify Empirical Articles?

To identify an article based on empirical research, look for the following characteristics:

     The article is published in a peer-reviewed journal .

     The article includes charts, graphs, or statistical analysis .

     The article is substantial in size , likely to be more than 5 pages long.

     The article contains the following parts (the exact terms may vary): abstract, introduction, method, results, discussion, references .

• << Previous: 2. Identify Databases & Resources to Search
• Next: 3. Search the Literature >>
• Last Updated: Sep 25, 2023 3:04 PM
• URL: https://cob-bs.libguides.com/literaturereview

Systematic Reviews: Types of literature review, methods, & resources

• Types of literature review, methods, & resources
• Protocol and registration
• Search strategy
• Medical Literature Databases to search
• Study selection and appraisal
• Data Extraction/Coding/Study characteristics/Results
• Reporting the quality/risk of bias
• Manage citations using RefWorks This link opens in a new window
• GW Box file storage for PDF's This link opens in a new window

Analytical reviews

GUIDELINES FOR HOW TO CARRY OUT AN ANALYTICAL REVIEW OF QUANTITATIVE RESEARCH

Enhancing the QUAlity and Transparency Of health Research (EQUATOR) network. (Tracking and listing over 550 reporting guidelines for various different study types including Randomised trials, Systematic reviews, Study protocols, Diagnostic/prognostic studies, Case reports, Clinical practice guidelines, Animal pre-clinical studies, etc). http://www.equator-network.org/resource-centre/library-of-health-research-reporting/

When comparing therapies :

PRISMA (Guideline on how to perform and write-up a systematic review and/or meta-analysis of the outcomes reported in multiple clinical trials of therapeutic interventions. PRISMA  replaces the previous QUORUM statement guidelines ):  Liberati, A,, Altman, D,, Moher, D, et al. (2009). The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration.  Plos Medicine, 6 (7):e1000100. doi:10.1371/journal.pmed.1000100 

When comparing diagnostic methods :

Checklist for Artificial Intelligence in Medical Imaging (CLAIM). CLAIM is modeled after the STARD guideline and has been extended to address applications of AI in medical imaging that include classification, image reconstruction, text analysis, and workflow optimization. The elements described here should be viewed as a “best practice” to guide authors in presenting their research. Reported in Mongan, J., Moy, L., & Kahn, C. E., Jr (2020). Checklist for Artificial Intelligence in Medical Imaging (CLAIM): A Guide for Authors and Reviewers.  Radiology. Artificial intelligence ,  2 (2), e200029. https://doi.org/10.1148/ryai.2020200029

STAndards for the Reporting of Diagnostic accuracy studies (STARD) Statement. (Reporting guidelines for writing up a study comparing the accuracy of competing diagnostic methods)  http://www.stard-statement.org/

When evaluating clinical practice guidelines :

AGREE Research Trust (ART) (2013).  Appraisal of Guidelines for Research & Evaluation (AGREE-II) . (A 23-item instrument for as sessing th e quality of Clinical Practice Guidelines. Used internationally for evaluating or deciding which guidelines could be recommended for use in practice or to inform health policy decisions.)

National Guideline Clearinghouse Extent of Adherence to Trustworthy Standards (NEATS) Instrument (2019). (A 15-item instrument using scales of 1-5 to evaluate a guideline's adherence to the Institute of Medicine's standard for trustworthy guidelines. It has good external validity among guideline developers and good interrater reliability across trained reviewers.)

When reviewing genetics studies

Human genetics review reporting guidelines.  Little J, Higgins JPT (eds.). The HuGENet™ HuGE Review Handbook, version 1.0 . 

When you need to re-analyze individual participant data

If you wish to collect, check, and re-analyze individual participant data (IPD) from clinical trials addressing a particular research question, you should follow the  PRISMA-IPD  guidelines as reported in  Stewart, L.A., Clarke, M., Rovers, M., et al. (2015). Preferred Reporting Items for a Systematic Review and Meta-analysis of Individual Participant Data: The PRISMA-IPD Statement. JAMA, 313(16):1657-1665. doi:10.1001/jama.2015.3656 .

When comparing Randomized studies involving animals, livestock, or food:

O’Connor AM, et al. (2010).  The REFLECT statement: methods and processes of creating reporting guidelines for randomized controlled trials for livestock and food safety by modifying the CONSORT statement.  Zoonoses Public Health. 57(2):95-104. Epub 2010/01/15. doi: 10.1111/j.1863-2378.2009.01311.x. PubMed PMID: 20070653.

Sargeant JM, et al. (2010).  The REFLECT Statement: Reporting Guidelines for Randomized Controlled Trials in Livestock and Food Safety: Explanation and Elaboration.  Zoonoses Public Health. 57(2):105-36. Epub 2010/01/15. doi: JVB1312 [pii] 10.1111/j.1863-2378.2009.01312.x. PubMed PMID: 20070652.

GUIDELINES FOR HOW TO WRITE UP FOR PUBLICATION THE RESULTS OF ONE QUANTITATIVE CLINICAL TRIAL

When reporting the results of a Randomized Controlled Trial :

Consolidated Standards of Reporting Trials (CONSORT) Statement. (2010 reporting guideline for writing up a Randomized Controlled Clinical Trial).  http://www.consort-statement.org . Since updated in 2022, see Butcher, M. A., et al. (2022). Guidelines for Reporting Outcomes in Trial Reports: The CONSORT-Outcomes 2022 Extension . JAMA : the Journal of the American Medical Association, 328(22), 2252–2264. https://doi.org/10.1001/jama.2022.21022

Kilkenny, C., Browne, W. J., Cuthill, I. C., Emerson, M., & Altman, D. G. (2010). Improving bioscience research reporting: The ARRIVE guidelines for reporting animal research. PLoS Biology, 8(6), e1000412–e1000412. https://doi.org/10.1371/journal.pbio.1000412 (A 20-item checklist, following the CONSORT approach, listing the information that published articles reporting research using animals should include, such as the number and specific characteristics of animals used; details of housing and husbandry; and the experimental, statistical, and analytical methods used to reduce bias.)

Narrative reviews

GUIDELINES  FOR HOW TO CARRY OUT  A  NARRATIVE REVIEW / QUALITATIVE RESEARCH /  OBSERVATIONAL STUDIES

Campbell, M. (2020). Synthesis without meta-analysis (SWiM) in systematic reviews: reporting guideline. BMJ, 368. doi: https://doi.org/10.1136/bmj.l6890  (guideline on how to analyse evidence for a narrative review, to provide a recommendation based on heterogenous study types).

Community Preventive Services Task Force (2021).  The Methods Manual for Community Guide Systematic Reviews . (Public Health Prevention systematic review guidelines)

Enhancing the QUAlity and Transparency Of health Research (EQUATOR) network. (Tracking and listing over 550 reporting guidelines for various different study types including Observational studies, Qualitative research, Quality improvement studies, and Economic evaluations). http://www.equator-network.org/resource-centre/library-of-health-research-reporting/

Cochrane Qualitative & Implementation Methods Group. (2019). Training resources. Retrieved from  https://methods.cochrane.org/qi/training-resources . (Training materials for how to do a meta-synthesis, or qualitative evidence synthesis). 

Cornell University Library (2019). Planning worksheet for structured literature reviews. Retrieved 4/8/22 from  https://osf.io/tnfm7/  (offers a framework for a narrative literature review).

Green, B. N., Johnson, C. D., & Adams, A. (2006).  Writing narrative literature reviews for peer-reviewed journals: secrets of the trade . Journal of Chiropractic Medicine, 5(3): 101-117. DOI: 10.1016/ S0899-3467 (07)60142-6.  This is a very good article about what to take into consideration when writing any type of narrative review.

When reviewing observational studies/qualitative research :

STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) statement. (Reporting guidelines for various types of health sciences observational studies).  http://www.strobe-statement.org 

Meta-analysis of Observational Studies in Epidemiology (MOOSE)  http://jama.jamanetwork.com/article.aspx?articleid=192614

RATS Qualitative research systematic review guidelines.  https://www.equator-network.org/reporting-guidelines/qualitative-research-review-guidelines-rats/

Methods/Guidance

Right Review , this decision support website provides an algorithm to help reviewers choose a review methodology from among 41 knowledge synthesis methods.

The Systematic Review Toolbox , an online catalogue of tools that support various tasks within the systematic review and wider evidence synthesis process. Maintained by the UK University of York Health Economics Consortium, Newcastle University NIHR Innovation Observatory, and University of Sheffield School of Health and Related Research.

Institute of Medicine. (2011).  Finding What Works in Health Care: Standards for Systematic Reviews . Washington, DC: National Academies  (Systematic review guidelines from the Health and Medicine Division (HMD) of the U.S. National Academies of Sciences, Engineering, and Medicine (formerly called the Institute of Medicine)).

International Committee of Medical Journal Editors (2022).  Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly work in Medical Journals . Guidance on how to prepare a manuscript for submission to a Medical journal.

Cochrane Handbook of Systematic Reviews of Interventions (International Cochrane Collaboration systematic review guidelines). The various Cochrane review groups comporise around 30,000 physicians around the world working in the disciplines on reviews of interventions with very detailed methods for verifying the validity of the research methods and analysis performed in screened-in Randmized Controlled Clinical Trials. Typically published Cochrane Reviews are the most exhaustive review of the evidence of effectiveness of a particular drug or intervention, and include a statistical meta-analysis. Similar to practice guidelines, Cochrane reviews are periodically revised and updated.

Joanna Briggs Institute (JBI) Manual of Evidence Synthesis . (International systematic review guidelines). Based at the University of Adelaide, South Australia, and collaborating with around 80 academic and medical entities around the world. Unlike Cochrane Reviews that strictly focus on efficacy of interventions, JBI offers a broader, inclusive approach to evidence, to accommodate a range of diverse questions and study designs. The JBI manual provides guidance on how to analyse and include both quantitative and qualitative research.

Cochrane Methods Support Unit, webinar recordings on methodological support questions 

Cochrane Qualitative & Implementation Methods Group. (2019). Training resources. Retrieved from https://methods.cochrane.org/qi/training-resources . (How to do a meta-synthesis, or qualitative evidence synthesis). 

Center for Reviews and Dissemination (University of York, England) (2009).  Systematic Reviews: CRD's guidance for undertaking systematic reviews in health care . (British systematic review guidelines). 

Agency for Health Research & Quality (AHRQ) (2013). Methods guide for effectiveness and comparative effectiveness reviews . (U.S. comparative effectiveness review guidelines)

Hunter, K. E., et al. (2022). Searching clinical trials registers: guide for systematic reviewers.  BMJ (Clinical research ed.) ,  377 , e068791. https://doi.org/10.1136/bmj-2021-068791

Patient-Centered Outcomes Research Institute (PCORI).  The PCORI Methodology Report . (A 47-item methodology checklist for U.S. patient-centered outcomes research. Established under the Patient Protection and Affordable Care Act, PCORI funds the development of guidance on the comparative effectivess of clinical healthcare, similar to the UK National Institute for Clinical Evidence but without reporting cost-effectiveness QALY metrics). 

Canadian Agency for Drugs and Technologies in Health (CADTH) (2019). Grey Matters: a practical tool for searching health-related grey literature. Retrieved from https://www.cadth.ca/resources/finding-evidence/grey-matters . A checklist of N American & international online databases and websites you can use to search for unpublished reports, posters, and policy briefs, on topics including general medicine and nursing, public and mental health, health technology assessment, drug and device regulatory, approvals, warnings, and advisories.

Hempel, S., Xenakis, L., & Danz, M. (2016). Systematic Reviews for Occupational Safety and Health Questions: Resources for Evidence Synthesis. Retrieved 8/15/16 from http://www.rand.org/pubs/research_reports/RR1463.html . NIOSH guidelines for how to carry out a systematic review in the occupational safety and health domain.

A good source for reporting guidelines is the  NLM's  Research Reporting Guidelines and Initiatives .

Grading of Recommendations Assessment, Development and Evaluation (GRADE). (An international group of academics/clinicians working to promote a common approach to grading the quality of evidence and strength of recommendations.) 

Phillips, B., Ball, C., Sackett, D., et al. (2009). Oxford Centre for Evidence Based Medicine: Levels of Evidence. Retrieved 3/20/17 from https://www.cebm.net/wp-content/uploads/2014/06/CEBM-Levels-of-Evidence-2.1.pdf . (Another commonly used criteria for grading the quality of evidence and strength of recommendations, developed in part by EBM guru David Sackett.) 

Systematic Reviews for Animals & Food  (guidelines including the REFLECT statement for carrying out a systematic review on animal health, animal welfare, food safety, livestock, and agriculture)

Grant, M. J., & Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies . Health Information & Libraries Journal, 26(2), 91-108. doi:10.1111/j.1471-1842.2009.00848.x. (Describes 14 different types of literature and systematic review, useful for thinking at the outset about what sort of literature review you want to do.)

Sutton, A., Clowes, M., Preston, L., & Booth, A. (2019). Meeting the review family: exploring review types and associated information retrieval requirements . Health information and libraries journal, 36(3), 202–222. doi:10.1111/hir.12276  (An updated look at different types of literature review, expands on the Grant & Booth 2009 article listed above).

Garrard, J. (2007).  Health Sciences Literature Review Made Easy: The Matrix Method  (2nd Ed.).   Sudbury, MA:  Jones & Bartlett Publishers. (Textbook of health sciences literature search methods).

Zilberberg, M. (2012).  Between the lines: Finding the truth in medical literature . Goshen, MA: Evimed Research Press. (Concise book on foundational concepts of evidence-based medicine).

Lang, T. (2009). The Value of Systematic Reviews as Research Activities in Medical Education . In: Lang, T. How to write, publish, & present in the health sciences : a guide for clinicians & laboratory researchers. Philadelphia : American College of Physicians.  (This book chapter has a helpful bibliography on systematic review and meta-analysis methods)

Brown, S., Martin, E., Garcia, T., Winter, M., García, A., Brown, A., Cuevas H.,  & Sumlin, L. (2013). Managing complex research datasets using electronic tools: a meta-analysis exemplar . Computers, Informatics, Nursing: CIN, 31(6), 257-265. doi:10.1097/NXN.0b013e318295e69c. (This article advocates for the programming of electronic fillable forms in Adobe Acrobat Pro to feed data into Excel or SPSS for analysis, and to use cloud based file sharing systems such as Blackboard, RefWorks, or EverNote to facilitate sharing knowledge about the decision-making process and keep data secure. Of particular note are the flowchart describing this process, and their example screening form used for the initial screening of abstracts).

Brown, S., Upchurch, S., & Acton, G. (2003). A framework for developing a coding scheme for meta-analysis . Western Journal Of Nursing Research, 25(2), 205-222. (This article describes the process of how to design a coded data extraction form and codebook, Table 1 is an example of a coded data extraction form that can then be used to program a fillable form in Adobe Acrobat or Microsoft Access).

Elamin, M. B., Flynn, D. N., Bassler, D., Briel, M., Alonso-Coello, P., Karanicolas, P., & ... Montori, V. M. (2009). Choice of data extraction tools for systematic reviews depends on resources and review complexity .  Journal Of Clinical Epidemiology ,  62 (5), 506-510. doi:10.1016/j.jclinepi.2008.10.016  (This article offers advice on how to decide what tools to use to extract data for analytical systematic reviews).

Riegelman R.   Studying a Study and Testing a Test: Reading Evidence-based Health Research , 6th Edition.  Lippincott Williams & Wilkins, 2012. (Textbook of quantitative statistical methods used in health sciences research).

Rathbone, J., Hoffmann, T., & Glasziou, P. (2015). Faster title and abstract screening? Evaluating Abstrackr, a semi-automated online screening program for systematic reviewers. Systematic Reviews, 480. doi:10.1186/s13643-015-0067-6

Guyatt, G., Rennie, D., Meade, M., & Cook, D. (2015). Users' guides to the medical literature (3rd ed.). New York: McGraw-Hill Education Medical.  (This is a foundational textbook on evidence-based medicine and of particular use to the reviewer who wants to learn about the different types of published research article e.g. "what is a case report?" and to understand what types of study design best answer what types of clinical question).

Glanville, J., Duffy, S., Mccool, R., & Varley, D. (2014). Searching ClinicalTrials.gov and the International Clinical Trials Registry Platform to inform systematic reviews: what are the optimal search approaches? Journal of the Medical Library Association : JMLA, 102(3), 177–183. https://doi.org/10.3163/1536-5050.102.3.007

Ouzzani, M., Hammady, H., Fedorowicz, Z., & Elmagarmid, A. (2016). Rayyan a web and mobile app for systematic reviews.  Systematic Reviews, 5 : 210, DOI: 10.1186/s13643-016-0384-4. http://rdcu.be/nzDM

Kwon Y, Lemieux M, McTavish J, Wathen N. (2015). Identifying and removing duplicate records from systematic review searches. J Med Libr Assoc. 103 (4): 184-8. doi: 10.3163/1536-5050.103.4.004. https://www.ncbi.nlm.nih.gov/pubmed/26512216

Bramer WM, Giustini D, de Jonge GB, Holland L, Bekhuis T. (2016). De-duplication of database search results for systematic reviews in EndNote. J Med Libr Assoc. 104 (3):240-3. doi: 10.3163/1536-5050.104.3.014. Erratum in: J Med Libr Assoc. 2017 Jan;105(1):111. https://www.ncbi.nlm.nih.gov/pubmed/27366130

McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. J Clin Epidemiol. 2016;75:40–46. doi: 10.1016/j.jclinepi.2016.01.021 . PRESS is a guideline with a checklist for librarians to critically appraise the search strategy for a systematic review literature search.

Clark, JM, Sanders, S, Carter, M, Honeyman, D, Cleo, G, Auld, Y, Booth, D, Condron, P, Dalais, C, Bateup, S, Linthwaite, B, May, N, Munn, J, Ramsay, L, Rickett, K, Rutter, C, Smith, A, Sondergeld, P, Wallin, M, Jones, M & Beller, E 2020, 'Improving the translation of search strategies using the Polyglot Search Translator: a randomized controlled trial',  Journal of the Medical Library Association , vol. 108, no. 2, pp. 195-207.

Journal articles describing systematic review methods can be searched for in PubMed using this search string in the PubMed search box: sysrev_methods [sb] . 

Software tools for systematic reviews

• Covidence GW in 2019 has bought a subscription to this Cloud based tool for facilitating screening decisions, used by the Cochrane Collaboration. Register for an account.
• NVIVO for analysis of qualitative research NVIVO is used for coding interview data to identify common themes emerging from interviews with several participants. GW faculty, staff, and students may download NVIVO software.
• RedCAP RedCAP is software that can be used to create survey forms for research or data collection or data extraction. It has very detailed functionality to enable data exchange with Electronic Health Record Systems, and to integrate with study workflow such as scheduling follow up reminders for study participants.
• SRDR tool from AHRQ Free, web-based and has a training environment, tutorials, and example templates of systematic review data extraction forms
• RevMan 5 RevMan 5 is the desktop version of the software used by Cochrane systematic review teams. RevMan 5 is free for academic use and can be downloaded and configured to run as stand alone software that does not connect with the Cochrane server if you follow the instructions at https://training.cochrane.org/online-learning/core-software-cochrane-reviews/revman/revman-5-download/non-cochrane-reviews
• Rayyan Free, web-based tool for collecting and screening citations. It has options to screen with multiple people, masking each other.
• GradePro Free, web application to create, manage and share summaries of research evidence (called Evidence Profiles and Summary of Findings Tables) for reviews or guidelines, uses the GRADE criteria to evaluate each paper under review.
• DistillerSR Needs subscription. Create coded data extraction forms from templates.
• EPPI Reviewer Needs subscription. Like DistillerSR, tool for text mining, data clustering, classification and term extraction
• SUMARI Needs subscription. Qualitative data analysis.
• Dedoose Needs subscription. Qualitative data analysis, similar to NVIVO in that it can be used to code interview transcripts, identify word co-occurence, cloud based.
• Meta-analysis software for statistical analysis of data for quantitative reviews SPSS, SAS, and STATA are popular analytical statistical software that include macros for carrying out meta-analysis. Himmelfarb has SPSS on some 3rd floor computers, and GW affiliates may download SAS to your own laptop from the Division of IT website. To perform mathematical analysis of big data sets there are statistical analysis software libraries in the R programming language available through GitHub and RStudio, but this requires advanced knowledge of the R and Python computer languages and data wrangling/cleaning.
• PRISMA 2020 flow diagram generator The PRISMA Statement website has a page listing example flow diagram templates and a link to software for creating PRISMA 2020 flow diagrams using R software.

GW researchers may want to consider using Refworks to manage citations, and GW Box to store the full text PDF's of review articles. You can also use online survey forms such as Qualtrics, RedCAP, or Survey Monkey, to design and create your own coded fillable forms, and export the data to Excel or one of the qualitative analytical software tools listed above.

Forest Plot Generators

• RevMan 5 the desktop version of the software used by Cochrane systematic review teams. RevMan 5 is free for academic use and can be downloaded and configured to run as stand alone software that does not connect with the Cochrane server if you follow the instructions at https://training.cochrane.org/online-learning/core-software-cochrane-reviews/revman/revman-5-download/non-cochrane-reviews.
• Meta-Essentials a free set of workbooks designed for Microsoft Excel that, based on your input, automatically produce meta-analyses including Forest Plots. Produced for Erasmus University Rotterdam joint research institute.
• Neyeloff, Fuchs & Moreira Another set of Excel worksheets and instructions to generate a Forest Plot. Published as Neyeloff, J.L., Fuchs, S.C. & Moreira, L.B. Meta-analyses and Forest plots using a microsoft excel spreadsheet: step-by-step guide focusing on descriptive data analysis. BMC Res Notes 5, 52 (2012). https://doi-org.proxygw.wrlc.org/10.1186/1756-0500-5-52
• For R programmers instructions are at https://cran.r-project.org/web/packages/forestplot/vignettes/forestplot.html and you can download the R code package from https://github.com/gforge/forestplot
• << Previous: Home
• Next: Protocol and registration >>

• Last Updated: May 8, 2024 11:07 AM
• URL: https://guides.himmelfarb.gwu.edu/systematic_review

• Himmelfarb Intranet
• Privacy Notice
• Terms of Use
• GW is committed to digital accessibility. If you experience a barrier that affects your ability to access content on this page, let us know via the Accessibility Feedback Form .
• Himmelfarb Health Sciences Library
• 2300 Eye St., NW, Washington, DC 20037
• Phone: (202) 994-2850
• [email protected]
• https://himmelfarb.gwu.edu

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 5. The Literature Review
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

• Give a new interpretation of old material or combine new with old interpretations,
• Trace the intellectual progression of the field, including major debates,
• Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
• Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

• Place each work in the context of its contribution to understanding the research problem being studied.
• Describe the relationship of each work to the others under consideration.
• Identify new ways to interpret prior research.
• Reveal any gaps that exist in the literature.
• Resolve conflicts amongst seemingly contradictory previous studies.
• Identify areas of prior scholarship to prevent duplication of effort.
• Point the way in fulfilling a need for additional research.
• Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE : Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

• An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
• Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
• An explanation of how each work is similar to and how it varies from the others,
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

• Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
• Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
• Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
• Persuasiveness -- which of the author's theses are most convincing or least convincing?
• Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

• Current Situation : Information necessary to understand the current topic or focus of the literature review.
• Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
• History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
• Standards : Description of the way in which you present your information.
• Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

• Sources in your literature review do not clearly relate to the research problem;
• You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
• Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
• Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
• Does not describe the search procedures that were used in identifying the literature to review;
• Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
• Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

• How are they organizing their ideas?
• What methods have they used to study the problem?
• What theories have been used to explain, predict, or understand their research problem?
• What sources have they cited to support their conclusions?
• How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

• Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
• Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
• Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

• << Previous: Theoretical Framework
• Next: Citation Tracking >>
• Last Updated: May 15, 2024 9:53 AM
• URL: https://libguides.usc.edu/writingguide

Capstone and PICO Project Toolkit

• Starting a Project: Overview
• Developing a Research Question
• Selecting Databases
• Expanding a Search
• Refining/Narrowing a Search
• Saving Searches
• Critical Appraisal & Levels of Evidence
• Citing & Managing References
• Database Tutorials
• Types of Literature Reviews
• Finding Full Text
• Term Glossary

Choosing a Review Type

For guidance related to choosing a review type, see:

• "What Type of Review is Right for You?" - Decision Tree (PDF) This decision tree, from Cornell University Library, highlights key difference between narrative, systematic, umbrella, scoping and rapid reviews.
• Reviewing the literature: choosing a review design Noble, H., & Smith, J. (2018). Reviewing the literature: Choosing a review design. Evidence Based Nursing, 21(2), 39–41. https://doi.org/10.1136/eb-2018-102895
• What synthesis methodology should I use? A review and analysis of approaches to research synthesis Schick-Makaroff, K., MacDonald, M., Plummer, M., Burgess, J., & Neander, W. (2016). What synthesis methodology should I use? A review and analysis of approaches to research synthesis. AIMS Public Health, 3 (1), 172-215. doi:10.3934/publichealth.2016.1.172 More information less... ABSTRACT: Our purpose is to present a comprehensive overview and assessment of the main approaches to research synthesis. We use "research synthesis" as a broad overarching term to describe various approaches to combining, integrating, and synthesizing research findings.
• Right Review - Decision Support Tool Not sure of the most suitable review method? Answer a few questions and be guided to suitable knowledge synthesis methods. Updated in 2022 and featured in the Journal of Clinical Epidemiology 10.1016/j.jclinepi.2022.03.004

Types of Evidence Synthesis / Literature Reviews

Literature reviews are comprehensive summaries and syntheses of the previous research on a given topic.  While narrative reviews are common across all academic disciplines, reviews that focus on appraising and synthesizing research evidence are increasingly important in the health and social sciences.  

Most evidence synthesis methods use formal and explicit methods to identify, select and combine results from multiple studies, making evidence synthesis a form of meta-research.  

The review purpose, methods used and the results produced vary among different kinds of literature reviews; some of the common types of literature review are detailed below.

Common Types of Literature Reviews 1

Narrative (literature) review.

• A broad term referring to reviews with a wide scope and non-standardized methodology
• Search strategies, comprehensiveness of literature search, time range covered and method of synthesis will vary and do not follow an established protocol

Integrative Review

• A type of literature review based on a systematic, structured literature search
• Often has a broadly defined purpose or review question
• Seeks to generate or refine and theory or hypothesis and/or develop a holistic understanding of a topic of interest
• Relies on diverse sources of data (e.g. empirical, theoretical or methodological literature; qualitative or quantitative studies)

Systematic Review

• Systematically and transparently collects and categorize existing evidence on a question of scientific, policy or management importance
• Follows a research protocol that is established a priori
• Some sub-types of systematic reviews include: SRs of intervention effectiveness, diagnosis, prognosis, etiology, qualitative evidence, economic evidence, and more.
• Time-intensive and often takes months to a year or more to complete 
• The most commonly referred to type of evidence synthesis; sometimes confused as a blanket term for other types of reviews

Meta-Analysis

• Statistical technique for combining the findings from disparate quantitative studies
• Uses statistical methods to objectively evaluate, synthesize, and summarize results
• Often conducted as part of a systematic review

Scoping Review

• Systematically and transparently collects and categorizes existing evidence on a broad question of scientific, policy or management importance
• Seeks to identify research gaps, identify key concepts and characteristics of the literature and/or examine how research is conducted on a topic of interest
• Useful when the complexity or heterogeneity of the body of literature does not lend itself to a precise systematic review
• Useful if authors do not have a single, precise review question
• May critically evaluate existing evidence, but does not attempt to synthesize the results in the way a systematic review would 
• May take longer than a systematic review

Rapid Review

• Applies a systematic review methodology within a time-constrained setting
• Employs methodological "shortcuts" (e.g., limiting search terms and the scope of the literature search), at the risk of introducing bias
• Useful for addressing issues requiring quick decisions, such as developing policy recommendations

Umbrella Review

• Reviews other systematic reviews on a topic
• Often defines a broader question than is typical of a traditional systematic review
• Most useful when there are competing interventions to consider

1. Adapted from:

Eldermire, E. (2021, November 15). A guide to evidence synthesis: Types of evidence synthesis. Cornell University LibGuides. https://guides.library.cornell.edu/evidence-synthesis/types

Nolfi, D. (2021, October 6). Integrative Review: Systematic vs. Scoping vs. Integrative. Duquesne University LibGuides. https://guides.library.duq.edu/c.php?g=1055475&p=7725920

Delaney, L. (2021, November 24). Systematic reviews: Other review types. UniSA LibGuides. https://guides.library.unisa.edu.au/SystematicReviews/OtherReviewTypes

Further Reading: Exploring Different Types of Literature Reviews

• A typology of reviews: An analysis of 14 review types and associated methodologies Grant, M. J., & Booth, A. (2009). A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information and Libraries Journal, 26 (2), 91-108. doi:10.1111/j.1471-1842.2009.00848.x More information less... ABSTRACT: The expansion of evidence-based practice across sectors has lead to an increasing variety of review types. However, the diversity of terminology used means that the full potential of these review types may be lost amongst a confusion of indistinct and misapplied terms. The objective of this study is to provide descriptive insight into the most common types of reviews, with illustrative examples from health and health information domains.
• Clarifying differences between review designs and methods Gough, D., Thomas, J., & Oliver, S. (2012). Clarifying differences between review designs and methods. Systematic Reviews, 1 , 28. doi:10.1186/2046-4053-1-28 More information less... ABSTRACT: This paper argues that the current proliferation of types of systematic reviews creates challenges for the terminology for describing such reviews....It is therefore proposed that the most useful strategy for the field is to develop terminology for the main dimensions of variation.
• Are we talking the same paradigm? Considering methodological choices in health education systematic review Gordon, M. (2016). Are we talking the same paradigm? Considering methodological choices in health education systematic review. Medical Teacher, 38 (7), 746-750. doi:10.3109/0142159X.2016.1147536 More information less... ABSTRACT: Key items discussed are the positivist synthesis methods meta-analysis and content analysis to address questions in the form of "whether and what" education is effective. These can be juxtaposed with the constructivist aligned thematic analysis and meta-ethnography to address questions in the form of "why." The concept of the realist review is also considered. It is proposed that authors of such work should describe their research alignment and the link between question, alignment and evidence synthesis method selected.
• Meeting the review family: Exploring review types and associated information retrieval requirements Sutton, A., Clowes, M., Preston, L., & Booth, A. (2019). Meeting the review family: Exploring review types and associated information retrieval requirements. Health Information & Libraries Journal, 36(3), 202–222. doi: 10.1111/hir.12276

Integrative Reviews

"The integrative review method is an approach that allows for the inclusion of diverse methodologies (i.e. experimental and non-experimental research)." (Whittemore & Knafl, 2005, p. 547).

• The integrative review: Updated methodology Whittemore, R., & Knafl, K. (2005). The integrative review: Updated methodology. Journal of Advanced Nursing, 52 (5), 546–553. doi:10.1111/j.1365-2648.2005.03621.x More information less... ABSTRACT: The aim of this paper is to distinguish the integrative review method from other review methods and to propose methodological strategies specific to the integrative review method to enhance the rigour of the process....An integrative review is a specific review method that summarizes past empirical or theoretical literature to provide a more comprehensive understanding of a particular phenomenon or healthcare problem....Well-done integrative reviews present the state of the science, contribute to theory development, and have direct applicability to practice and policy.

• Conducting integrative reviews: A guide for novice nursing researchers Dhollande, S., Taylor, A., Meyer, S., & Scott, M. (2021). Conducting integrative reviews: A guide for novice nursing researchers. Journal of Research in Nursing, 26(5), 427–438. https://doi.org/10.1177/1744987121997907
• Rigour in integrative reviews Whittemore, R. (2007). Rigour in integrative reviews. In C. Webb & B. Roe (Eds.), Reviewing Research Evidence for Nursing Practice (pp. 149–156). John Wiley & Sons, Ltd. https://doi.org/10.1002/9780470692127.ch11

Scoping Reviews

Scoping reviews are evidence syntheses that are conducted systematically, but begin with a broader scope of question than traditional systematic reviews, allowing the research to 'map' the relevant literature on a given topic.

• Scoping studies: Towards a methodological framework Arksey, H., & O'Malley, L. (2005). Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology, 8 (1), 19-32. doi:10.1080/1364557032000119616 More information less... ABSTRACT: We distinguish between different types of scoping studies and indicate where these stand in relation to full systematic reviews. We outline a framework for conducting a scoping study based on our recent experiences of reviewing the literature on services for carers for people with mental health problems.
• Scoping studies: Advancing the methodology Levac, D., Colquhoun, H., & O'Brien, K. K. (2010). Scoping studies: Advancing the methodology. Implementation Science, 5 (1), 69. doi:10.1186/1748-5908-5-69 More information less... ABSTRACT: We build upon our experiences conducting three scoping studies using the Arksey and O'Malley methodology to propose recommendations that clarify and enhance each stage of the framework.
• Methodology for JBI scoping reviews Peters, M. D. J., Godfrey, C. M., McInerney, P., Baldini Soares, C., Khalil, H., & Parker, D. (2015). The Joanna Briggs Institute reviewers’ manual: Methodology for JBI scoping reviews [PDF]. Retrieved from The Joanna Briggs Institute website: http://joannabriggs.org/assets/docs/sumari/Reviewers-Manual_Methodology-for-JBI-Scoping-Reviews_2015_v2.pdf More information less... ABSTRACT: Unlike other reviews that address relatively precise questions, such as a systematic review of the effectiveness of a particular intervention based on a precise set of outcomes, scoping reviews can be used to map the key concepts underpinning a research area as well as to clarify working definitions, and/or the conceptual boundaries of a topic. A scoping review may focus on one of these aims or all of them as a set.

Systematic vs. Scoping Reviews: What's the Difference? 

YouTube Video 4 minutes, 45 seconds

Rapid Reviews

Rapid reviews are systematic reviews that are undertaken under a tighter timeframe than traditional systematic reviews. 

• Evidence summaries: The evolution of a rapid review approach Khangura, S., Konnyu, K., Cushman, R., Grimshaw, J., & Moher, D. (2012). Evidence summaries: The evolution of a rapid review approach. Systematic Reviews, 1 (1), 10. doi:10.1186/2046-4053-1-10 More information less... ABSTRACT: Rapid reviews have emerged as a streamlined approach to synthesizing evidence - typically for informing emergent decisions faced by decision makers in health care settings. Although there is growing use of rapid review "methods," and proliferation of rapid review products, there is a dearth of published literature on rapid review methodology. This paper outlines our experience with rapidly producing, publishing and disseminating evidence summaries in the context of our Knowledge to Action (KTA) research program.
• What is a rapid review? A methodological exploration of rapid reviews in Health Technology Assessments Harker, J., & Kleijnen, J. (2012). What is a rapid review? A methodological exploration of rapid reviews in Health Technology Assessments. International Journal of Evidence‐Based Healthcare, 10 (4), 397-410. doi:10.1111/j.1744-1609.2012.00290.x More information less... ABSTRACT: In recent years, there has been an emergence of "rapid reviews" within Health Technology Assessments; however, there is no known published guidance or agreed methodology within recognised systematic review or Health Technology Assessment guidelines. In order to answer the research question "What is a rapid review and is methodology consistent in rapid reviews of Health Technology Assessments?", a study was undertaken in a sample of rapid review Health Technology Assessments from the Health Technology Assessment database within the Cochrane Library and other specialised Health Technology Assessment databases to investigate similarities and/or differences in rapid review methodology utilised.
• Rapid Review Guidebook Dobbins, M. (2017). Rapid review guidebook. Hamilton, ON: National Collaborating Centre for Methods and Tools.
• NCCMT Summary and Tool for Dobbins' Rapid Review Guidebook National Collaborating Centre for Methods and Tools. (2017). Rapid review guidebook. Hamilton, ON: McMaster University. Retrieved from http://www.nccmt.ca/knowledge-repositories/search/308
• << Previous: Database Tutorials
• Next: Finding Full Text >>
• Last Updated: May 15, 2024 12:45 PM
• URL: https://guides.nyu.edu/pico

Chapter Four: Theory, Methodologies, Methods, and Evidence

Research Methods

You are viewing the first edition of this textbook. a second edition is available – please visit the latest edition for updated information..

This page discusses the following topics:

Research Goals

Research method types.

Before discussing research   methods , we need to distinguish them from  methodologies  and  research skills . Methodologies, linked to literary theories, are tools and lines of investigation: sets of practices and propositions about texts and the world. Researchers using Marxist literary criticism will adopt methodologies that look to material forces like labor, ownership, and technology to understand literature and its relationship to the world. They will also seek to understand authors not as inspired geniuses but as people whose lives and work are shaped by social forces.

Example: Critical Race Theory Methodologies

Critical Race Theory may use a variety of methodologies, including

• Interest convergence: investigating whether marginalized groups only achieve progress when dominant groups benefit as well
• Intersectional theory: investigating how multiple factors of advantage and disadvantage around race, gender, ethnicity, religion, etc. operate together in complex ways
• Radical critique of the law: investigating how the law has historically been used to marginalize particular groups, such as black people, while recognizing that legal efforts are important to achieve emancipation and civil rights
• Social constructivism: investigating how race is socially constructed (rather than biologically grounded)
• Standpoint epistemology: investigating how knowledge relates to social position
• Structural determinism: investigating how structures of thought and of organizations determine social outcomes

To identify appropriate methodologies, you will need to research your chosen theory and gather what methodologies are associated with it. For the most part, we can’t assume that there are “one size fits all” methodologies.

Research skills are about how you handle materials such as library search engines, citation management programs, special collections materials, and so on.

Research methods  are about where and how you get answers to your research questions. Are you conducting interviews? Visiting archives? Doing close readings? Reviewing scholarship? You will need to choose which methods are most appropriate to use in your research and you need to gain some knowledge about how to use these methods. In other words, you need to do some research into research methods!

Your choice of research method depends on the kind of questions you are asking. For example, if you want to understand how an author progressed through several drafts to arrive at a final manuscript, you may need to do archival research. If you want to understand why a particular literary work became a bestseller, you may need to do audience research. If you want to know why a contemporary author wrote a particular work, you may need to do interviews. Usually literary research involves a combination of methods such as  archival research ,  discourse analysis , and  qualitative research  methods.

Literary research methods tend to differ from research methods in the hard sciences (such as physics and chemistry). Science research must present results that are reproducible, while literary research rarely does (though it must still present evidence for its claims). Literary research often deals with questions of meaning, social conventions, representations of lived experience, and aesthetic effects; these are questions that reward dialogue and different perspectives rather than one great experiment that settles the issue. In literary research, we might get many valuable answers even though they are quite different from one another. Also in literary research, we usually have some room to speculate about answers, but our claims have to be plausible (believable) and our argument comprehensive (meaning we don’t overlook evidence that would alter our argument significantly if it were known).

A literary researcher might select the following:

Theory: Critical Race Theory

Methodology: Social Constructivism

Method: Scholarly

Skills: Search engines, citation management

Wendy Belcher, in  Writing Your Journal Article in 12 Weeks , identifies two main approaches to understanding literary works: looking at a text by itself (associated with New Criticism ) and looking at texts as they connect to society (associated with Cultural Studies ). The goal of New Criticism is to bring the reader further into the text. The goal of Cultural Studies is to bring the reader into the network of discourses that surround and pass through the text. Other approaches, such as Ecocriticism, relate literary texts to the Sciences (as well as to the Humanities).

The New Critics, starting in the 1940s,  focused on meaning within the text itself, using a method they called “ close reading .” The text itself becomes e vidence for a particular reading. Using this approach, you should summarize the literary work briefly and q uote particularly meaningful passages, being sure to introduce quotes and then interpret them (never let them stand alone). Make connections within the work; a sk  “why” and “how” the various parts of the text relate to each other.

Cultural Studies critics see all texts  as connected to society; the critic  therefore has to connect a text to at least one political or social issue. How and why does  the text reproduce particular knowledge systems (known as discourses) and how do these knowledge systems relate to issues of power within the society? Who speaks and when? Answering these questions helps your reader understand the text in context. Cultural contexts can include the treatment of gender (Feminist, Queer), class (Marxist), nationality, race, religion, or any other area of human society.

Other approaches, such as psychoanalytic literary criticism , look at literary texts to better understand human psychology. A psychoanalytic reading can focus on a character, the author, the reader, or on society in general. Ecocriticism  look at human understandings of nature in literary texts.

We select our research methods based on the kinds of things we want to know. For example, we may be studying the relationship between literature and society, between author and text, or the status of a work in the literary canon. We may want to know about a work’s form, genre, or thematics. We may want to know about the audience’s reading and reception, or about methods for teaching literature in schools.

Below are a few research methods and their descriptions. You may need to consult with your instructor about which ones are most appropriate for your project. The first list covers methods most students use in their work. The second list covers methods more commonly used by advanced researchers. Even if you will not be using methods from this second list in your research project, you may read about these research methods in the scholarship you find.

Most commonly used undergraduate research methods:

• Scholarship Methods:  Studies the body of scholarship written about a particular author, literary work, historical period, literary movement, genre, theme, theory, or method.
• Textual Analysis Methods:  Used for close readings of literary texts, these methods also rely on literary theory and background information to support the reading.
• Biographical Methods:  Used to study the life of the author to better understand their work and times, these methods involve reading biographies and autobiographies about the author, and may also include research into private papers, correspondence, and interviews.
• Discourse Analysis Methods:  Studies language patterns to reveal ideology and social relations of power. This research involves the study of institutions, social groups, and social movements to understand how people in various settings use language to represent the world to themselves and others. Literary works may present complex mixtures of discourses which the characters (and readers) have to navigate.
• Creative Writing Methods:  A literary re-working of another literary text, creative writing research is used to better understand a literary work by investigating its language, formal structures, composition methods, themes, and so on. For instance, a creative research project may retell a story from a minor character’s perspective to reveal an alternative reading of events. To qualify as research, a creative research project is usually combined with a piece of theoretical writing that explains and justifies the work.

Methods used more often by advanced researchers:

• Archival Methods: Usually involves trips to special collections where original papers are kept. In these archives are many unpublished materials such as diaries, letters, photographs, ledgers, and so on. These materials can offer us invaluable insight into the life of an author, the development of a literary work, or the society in which the author lived. There are at least three major archives of James Baldwin’s papers: The Smithsonian , Yale , and The New York Public Library . Descriptions of such materials are often available online, but the materials themselves are typically stored in boxes at the archive.
• Computational Methods:  Used for statistical analysis of texts such as studies of the popularity and meaning of particular words in literature over time.
• Ethnographic Methods:  Studies groups of people and their interactions with literary works, for instance in educational institutions, in reading groups (such as book clubs), and in fan networks. This approach may involve interviews and visits to places (including online communities) where people interact with literary works. Note: before you begin such work, you must have  Institutional Review Board (IRB)  approval “to protect the rights and welfare of human participants involved in research.”
• Visual Methods:  Studies the visual qualities of literary works. Some literary works, such as illuminated manuscripts, children’s literature, and graphic novels, present a complex interplay of text and image. Even works without illustrations can be studied for their use of typography, layout, and other visual features.

Regardless of the method(s) you choose, you will need to learn how to apply them to your work and how to carry them out successfully. For example, you should know that many archives do not allow you to bring pens (you can use pencils) and you may not be allowed to bring bags into the archives. You will need to keep a record of which documents you consult and their location (box number, etc.) in the archives. If you are unsure how to use a particular method, please consult a book about it. [1] Also, ask for the advice of trained researchers such as your instructor or a research librarian.

• What research method(s) will you be using for your paper? Why did you make this method selection over other methods? If you haven’t made a selection yet, which methods are you considering?
• What specific methodological approaches are you most interested in exploring in relation to the chosen literary work?
• What is your plan for researching your method(s) and its major approaches?
• What was the most important lesson you learned from this page? What point was confusing or difficult to understand?

Write your answers in a webcourse discussion page.

• Introduction to Research Methods: A Practical Guide for Anyone Undertaking a Research Project  by Catherine, Dr. Dawson
• Practical Research Methods: A User-Friendly Guide to Mastering Research Techniques and Projects  by Catherine Dawson
• Qualitative Inquiry and Research Design: Choosing Among Five Approaches  by John W. Creswell  Cheryl N. Poth
• Qualitative Research Evaluation Methods: Integrating Theory and Practice  by Michael Quinn Patton
• Research Design: Qualitative, Quantitative, and Mixed Methods Approaches  by John W. Creswell  J. David Creswell
• Research Methodology: A Step-by-Step Guide for Beginners  by Ranjit Kumar
• Research Methodology: Methods and Techniques  by C.R. Kothari

Strategies for Conducting Literary Research Copyright © 2021 by Barry Mauer & John Venecek is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

Types of Literature Review

There are many types of literature review. The choice of a specific type depends on your research approach and design. The following types of literature review are the most popular in business studies:

Narrative literature review , also referred to as traditional literature review, critiques literature and summarizes the body of a literature. Narrative review also draws conclusions about the topic and identifies gaps or inconsistencies in a body of knowledge. You need to have a sufficiently focused research question to conduct a narrative literature review

Systematic literature review requires more rigorous and well-defined approach compared to most other types of literature review. Systematic literature review is comprehensive and details the timeframe within which the literature was selected. Systematic literature review can be divided into two categories: meta-analysis and meta-synthesis.

When you conduct meta-analysis you take findings from several studies on the same subject and analyze these using standardized statistical procedures. In meta-analysis patterns and relationships are detected and conclusions are drawn. Meta-analysis is associated with deductive research approach.

Meta-synthesis, on the other hand, is based on non-statistical techniques. This technique integrates, evaluates and interprets findings of multiple qualitative research studies. Meta-synthesis literature review is conducted usually when following inductive research approach.

Scoping literature review , as implied by its name is used to identify the scope or coverage of a body of literature on a given topic. It has been noted that “scoping reviews are useful for examining emerging evidence when it is still unclear what other, more specific questions can be posed and valuably addressed by a more precise systematic review.” [1] The main difference between systematic and scoping types of literature review is that, systematic literature review is conducted to find answer to more specific research questions, whereas scoping literature review is conducted to explore more general research question.

Argumentative literature review , as the name implies, examines literature selectively in order to support or refute an argument, deeply imbedded assumption, or philosophical problem already established in the literature. It should be noted that a potential for bias is a major shortcoming associated with argumentative literature review.

Integrative literature review reviews , critiques, and synthesizes secondary data about research topic in an integrated way such that new frameworks and perspectives on the topic are generated. If your research does not involve primary data collection and data analysis, then using integrative literature review will be your only option.

Theoretical literature review focuses on a pool of theory that has accumulated in regard to an issue, concept, theory, phenomena. Theoretical literature reviews play an instrumental role in establishing what theories already exist, the relationships between them, to what degree existing theories have been investigated, and to develop new hypotheses to be tested.

At the earlier parts of the literature review chapter, you need to specify the type of your literature review your chose and justify your choice. Your choice of a specific type of literature review should be based upon your research area, research problem and research methods.  Also, you can briefly discuss other most popular types of literature review mentioned above, to illustrate your awareness of them.

[1] Munn, A. et. al. (2018) “Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach” BMC Medical Research Methodology

  John Dudovskiy

Literature Review: Types of literature reviews

• Traditional or narrative literature reviews
• Scoping Reviews
• Systematic literature reviews
• Annotated bibliography
• Keeping up to date with literature
• Finding a thesis
• Evaluating sources and critical appraisal of literature
• Managing and analysing your literature
• Further reading and resources

Types of literature reviews

The type of literature review you write will depend on your discipline and whether you are a researcher writing your PhD, publishing a study in a journal or completing an assessment task in your undergraduate study.

A literature review for a subject in an undergraduate degree will not be as comprehensive as the literature review required for a PhD thesis.

An undergraduate literature review may be in the form of an annotated bibliography or a narrative review of a small selection of literature, for example ten relevant articles. If you are asked to write a literature review, and you are an undergraduate student, be guided by your subject coordinator or lecturer.

The common types of literature reviews will be explained in the pages of this section.

• Narrative or traditional literature reviews
• Critically Appraised Topic (CAT)
• Scoping reviews
• Annotated bibliographies

These are not the only types of reviews of literature that can be conducted. Often the term "review" and "literature" can be confusing and used in the wrong context. Grant and Booth (2009) attempt to clear up this confusion by discussing 14 review types and the associated methodology, and advantages and disadvantages associated with each review.

Grant, M. J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies . Health Information & Libraries Journal, 26 , 91–108. doi:10.1111/j.1471-1842.2009.00848.x

What's the difference between reviews?

Researchers, academics, and librarians all use various terms to describe different types of literature reviews, and there is often inconsistency in the ways the types are discussed. Here are a couple of simple explanations.

• The image below describes common review types in terms of speed, detail, risk of bias, and comprehensiveness:

"Schematic of the main differences between the types of literature review" by Brennan, M. L., Arlt, S. P., Belshaw, Z., Buckley, L., Corah, L., Doit, H., Fajt, V. R., Grindlay, D., Moberly, H. K., Morrow, L. D., Stavisky, J., & White, C. (2020). Critically Appraised Topics (CATs) in veterinary medicine: Applying evidence in clinical practice. Frontiers in Veterinary Science, 7 , 314. https://doi.org/10.3389/fvets.2020.00314 is licensed under CC BY 3.0

• The table below lists four of the most common types of review , as adapted from a widely used typology of fourteen types of reviews (Grant & Booth, 2009).  

Grant, M.J. & Booth, A. (2009).  A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26 (2), 91-108. https://doi.org/10.1111/j.1471-1842.2009.00848.x

See also the Library's  Literature Review guide.

Critical Appraised Topic (CAT)

For information on conducting a Critically Appraised Topic or CAT

Callander, J., Anstey, A. V., Ingram, J. R., Limpens, J., Flohr, C., & Spuls, P. I. (2017).  How to write a Critically Appraised Topic: evidence to underpin routine clinical practice.  British Journal of Dermatology (1951), 177(4), 1007-1013. https://doi.org/10.1111/bjd.15873 

Books on Literature Reviews

• << Previous: Home
• Next: Traditional or narrative literature reviews >>
• Last Updated: May 12, 2024 12:18 PM
• URL: https://libguides.csu.edu.au/review

Charles Sturt University is an Australian University, TEQSA Provider Identification: PRV12018. CRICOS Provider: 00005F.

Literature Reviews

• Getting Started

Selecting a Review Type

Defining the scope of your review, four common types of reviews.

• Developing a Research Question
• Searching the Literature
• Searching Tips
• ChatGPT [beta]
• Documenting your Search
• Using Citation Managers
• Concept Mapping
• Writing the Review
• Further Resources

More Review Types

This article by Sutton & Booth (2019) explores 48 distinct types of Literature Reviews:

Which Review is Right for You?

The  Right Review tool  has questions about your lit review process and plans. It offers a qualitative and quantitative option. At completion, you are given a lit review type recommendation.

You'll want to think about the kind of review you are doing. Is it a selective or comprehensive review? Is the review part of a larger work or a stand-alone work ?

For example, if you're writing the Literature Review section of a journal article, that's a selective review which is part of a larger work. Alternatively, if you're writing a review article, that's a comprehensive review which is a stand-alone work. Thinking about this will help you develop the scope of the review.

This exercise will help define the scope of your Literature Review, setting the boundaries for which literature to include and which to exclude.

A FEW GENERAL CONSIDERATIONS WHEN DEFINING SCOPE

• Which populations to investigate — this can include gender, age, socio-economic status, race, geographic location, etc., if the research area includes humans.
• What years to include — if researching the legalization of medicinal cannabis, you might only look at the previous 20 years; but if researching dolphin mating practices, you might extend many more decades.
• Which subject areas — if researching artificial intelligence, subject areas could be computer science, robotics, or health sciences
• How many sources  — a selective review for a class assignment might only need ten, while a comprehensive review for a dissertation might include hundreds. There is no one right answer.
• There will be many other considerations that are more specific to your topic. 

Most databases will allow you to limit years and subject areas, so look for those tools while searching. See the Searching Tips tab for information on how use these tools.

LITERATURE REVIEW

• Often used as a generic term to describe any type of review
• More precise definition:  Published materials that provide an examination of published literature . Can cover wide range of subjects at various levels of comprehensiveness.
• Identifies gaps in research, explains importance of topic, hypothesizes future work, etc.
• Usually written as part of a larger work like a journal article or dissertation

SCOPING REVIEW

• Conducted to address broad research questions with the goal of understanding the extent of research that has been conducted.
• Provides a preliminary assessment of the potential size and scope of available research literature. It aims to identify the nature and extent of research evidence (usually including ongoing research) 
• Doesn't assess the quality of the literature gathered (i.e. presence of literature on a topic shouldn’t be conflated w/ the quality of that literature)
• " Preparing scoping reviews for publication using methodological guides and reporting standards " is a great article to read on Scoping Reviews

SYSTEMATIC REVIEW

• Common in the health sciences ( Taubman Health Sciences Library guide to Systematic Reviews )
• Goal: collect all literature that meets specific criteria (methodology, population, treatment, etc.) and then appraise its quality and synthesize it
• Follows strict protocol for literature collection, appraisal and synthesis
• Typically performed by research teams 
• Takes 12-18 months to complete
• Often written as a stand alone work

META-ANALYSIS

• Goes one step further than a systematic review by statistically combining the results of quantitative studies to provide a more precise effect of the results. 
• << Previous: Getting Started
• Next: Developing a Research Question >>
• Last Updated: May 9, 2024 11:44 AM
• URL: https://guides.lib.umich.edu/litreview

• Research Guides
• University Libraries

Architecture

Common paper types.

• Federal Government Information
• Industry Associations
• Author Profile
• Literature Review
• Scoping Review
• Systematic Review

Understanding Literature Reviews 

I.  Getting Started with a Workshop Video  (Highly recommended!)

• Searching for Literature Reviews: Before You Write, You Have to Find   https://www.youtube.com/watch?v=9la5ytz9MmM

          A lecture by the Writing Center, TAMU.

II.  What is a Literature Review?

• Generally, the purpose of a review is to analyze critically a segment of a published body of knowledge through summary, classification, and comparison of prior research studies, reviews of literature, and theoretical articles. <http://writing.wisc.edu/Handbook/ReviewofLiterature.html>  
•  A literature review can be just a simple summary of the sources, but it usually has an organizational pattern and combines both summary and synthesis. A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information. It might give a new interpretation of old material or combine new with old interpretations. Or it might trace the intellectual progression of the field, including major debates. And depending on the situation, the literature review may evaluate the sources and advise the reader on the most pertinent or relevant.  < http://writingcenter.unc.edu/resources/handouts-demos/specific-writing-assignments/literature-reviews >  
•  A literature review is a body of text that aims to review the critical points of current knowledge including substantive findings as well as theoretical and methodological contributions to a particular topic...Its ultimate goal is to bring the reader up to date with current literature on a topic and forms the basis for another goal, such as future research that may be needed in the area. <http://en.wikipedia.org/wiki/Literature_review>

III.  What Major Steps Literature Reviews Require?

• 1. Develop a review protocol. Protocols define the scope of studies that will be reviewed, the process through which studies will be identified, and the outcomes that will be examined. Protocols also specify the time period during which relevant studies will have been conducted, the outcomes to be examined in the review, and keyword strategies for the literature search. 2. Identify relevant studies, often through a systematic search of the literature. 3. Screen studies for relevance and the adequacy of study design, implementation, and reporting. 4. Retrieve and summarize information on the intervention studied, the study characteristics, and the study findings. 5. Combine findings within studies and across studies when relevant. < http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_procedures_v2_1_standards_handbook.pdf>  
• The basic stages in a typical research project are: i) identify your topic of interest, ii) perform a literature review, iii) generate related questions, iv) state your unsolved problem or hypothesis, v) find or develop a solution, and vi) document your results.  
• The four stages required: Problem formulation—which topic or field is being examined and what are its component issues? Literature search—finding materials relevant to the subject being explored Data evaluation—determining which literature makes a significant contribution to the understanding of the topic Analysis and interpretation—discussing the findings and conclusions of pertinent literature < http://library.ucsc.edu/help/howto/write-a-literature-review#components >

IV.    What Basic Elements Comprise a Literature Review?   

• An overview of the subject, issue or theory under consideration, along with the objectives of the literature review
• Division of works under review into categories (e.g. those in support of a particular position, those against, and those offering alternative theses entirely)
• Explanation of how each work is similar to and how it varies from the others
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research    

          < http://library.ucsc.edu/help/howto/write-a-literature-review#components > V.    Which Citation Tool Are You Going to Use to Manage the Search Results?

•   Choose your citation tool before conducing your literature reviews. If you decide to use  RefWorks , the information can be found at  http://tamu.libguides.com/refworks .          

VII. Other Useful Guides

• Literature Reviews (University of North Carolina at Chapel Hill)
• The Literature Review: A Few Tips On Conducting It
• How to Write a Literature Review  (UCSC)
• Learn how to write a review of literature  (WISC)
• Reviewing the Literature

Scoping Reviews

Scoping reviews search for concepts by mapping the language and data which surrounds those concepts to synthesize evidence and assess the scope of an area of inquiry. The goal can be to assess how much data or evidence is available regarding a certain area of interest. The review should be systematically conducted and reported with a transparent and repeatable method.

• not a systematic review
• map the extent, range and nature of literature
• determine possible gaps in the literature on a topic
• scoping reviews are not limited to peer review literature
• normally conducted by a team, not one person

Step 1: Identifying the Research Question

• too broad a question may lead to a large number of papers to review
• too narrow will compromise the breadth and depth of the review
• conduct preliminary search to determine if a scoping review already exits
• Is there enough literature to conduct a scoping review

Step 2: Indentifying Relevant Studies

• Define the inclusion and exclusion criteria
• Need to have diverse perspectives
• Inclusion criteria are aligned with research question
• Refine search strategy based on research found

Step 3: Selecting Studies to Be Included in the Review

• Organize papers
• Import collected citations from Endnote, RefWorks, etc to Covidence or Rayyan
• Select papers for inclusion
• Having additional reviewers will accelerate the pace of the review

What is a Systematic Review?

A systematic review is a r esearch method  that is designed to answer a research question(s) by identifying, coding, appraising, and synthesizing a group of studies investigating the same question(s). This method is utilized in disciplines such as education, behavioral and social sciences, medicine, public health, and others to answer a variety of questions: effectiveness of an intervention or policy, prevalence, tests/diagnosis, and more.

What assistance can the library provide?

Evidence Review Services

Steps of a Systematic Review (PIECES)

P: Plan  - Decide on the methods of the systematic review before conducting it.

• Use Campbell Collaboration (MECCIR)  Conduct Standards  and  Searching for Studies: A Guide to Information Retrieval for Campbell Systematic Reviews  (Feb 2017) as guides.
• Identify search terms, including keywords, synonyms, and subject headings to search selected databases.
• Perform  scoping searches  in selected databases (to find background literature that may help you  refine your research question  and set inclusion/exclusion criteria).
• Set i nclusion/exclusion criteria  (such as specific aspects of a population; types of interventions, outcomes, and study designs; and limiters such as publication date range and language.

I: Identify  - Systematically search for studies using predetermined criteria.

• Finalize selection of databases to search.
• Design a search strategy using subject headings, keywords, Boolean operators, and/or proximity operators for each database.
• Export results from each database into a citation management program such as  RefWorks  or  EndNote .
• Remove duplicate citations using the citation management program, save remaining citations as an RIS file, then import the RIS file into a Screening program such as  Rayyan  or  Covidence .
• Document the process using the  PRISMA flow diagram .  

E: Evaluate  - Sort all retrieved articles into included or excluded categories; then assess the risk of bias for each included study.

• First screening ( use exclusion criteria to  exclude  studies): Sort (screen) studies in  Rayyan  or  Covidence  - based on article title and abstract - and mark as exclude, include, or undecided. (McGill University Library has a helpful guide:  Rayyan for Systematic Reviews . For additional assistance, please contact TAMU librarians  - not the McGill librarians).
• Second screening  (use inclusion criteria to i nclude  studies): Load PDFs into  Rayyan  for studies marked as include and undecided; then mark as include or exclude based on screening of the full text. 
• Assess quality of included studies. Example quality criteria are available in  Standard Quality Assessment Criteria for Evaluating Primary Research Papers from a Variety of Fields .

C: Collect/Combine  - Create a coding form to capture study characteristics; then synthesize data qualitatively or quantitatively.

• Use Google Forms or other software to create a coding form.

E: Explain  - Contextualize synthesis results, noting strengths and weaknesses of the studies.

S: Summarize  - Report or describe methods and results in a clear and transparent manner.

• Use Campbell Collaboration (MECCIR)  Reporting Standards  as a guide.
• Use the  PRISMA Checklist  to report methods. For more detailed guidance use the  PRISMA Statement  and  PRISMA Elaboration & Explanation   documents.

Standards & Guidelines for Systematic Reviews

Includes documents for conducting and reporting systematic reviews in education and social sciences. Last updated May 2017.

Version 1.3. Last updated November 2017

Based on chapter 6 of the Cochrane Handbook, provides guidance on the information retrieval process including literature sources, planning and executing searches, and documentation. Last updated Feb 2017.

Produced by the Institute of Education Sciences (IES) -- the statistics, research, and evaluation arm of the U.S. Department of Education.

Develops methods for SRs and research synthesis in education and social sciences; based within the department of Social Science, University College London.

Includes links to the PRISMA checklist and diagram. An evidence-based minimum set of items for reporting systematic reviews and meta-analyses. Focuses on randomized trials, but can also be used interventions.

The PRESS (Peer Review of Electronic Search Strategies) Checklist is a guide for librarians and information specialists in evaluating electronic search strategies.

Software Tools

Link to sign up using Texas A&M's Covidence account. Sign up using an @tamu.edu email address.

Free web application developed at Qatar Computing Research Institute (Data Analytics) to help authors manage, screen, and collaborate on SRs.

From Campbell Collaboration

From the EppiCentre. "MetaLight is a software application designed to support the teaching and learning of meta-analysis." Freely available and uses the Silverlight browser plugin.

One resource for creating the coding form.

A catalog of tools that can be used to support the SR process.

• << Previous: Books
• Next: Federal Government Information >>
• Last Updated: May 16, 2024 10:34 AM
• URL: https://tamu.libguides.com/c.php?g=607940

Evidence-based Medicine: Types of Literature

• Core EBM Databases
• Ask an Answerable Question (PICO)
• Acquire the Evidence
• Appraise the Evidence
• Apply the evidence in your clinical practice.
• Assess your performance
• Study Designs

Types of Literature

• Clinical Guidelines
• How to Find Books
• PubMed Guide This link opens in a new window
• EBM Resources
• Systematic Review Guide This link opens in a new window
• Literature Review Guide This link opens in a new window

Different types of publications have different characteristics.

Primary Literature Primary sources means original studies, based on direct observation, use of statistical records, interviews, or experimental methods, of actual practices or the actual impact of practices or policies. They are authored by researchers, contains original research data, and are usually published in a peer-reviewed journal. Primary literature may also include conference papers, pre-prints, or preliminary reports. Also called empirical research .

Secondary Literature Secondary literature consists of interpretations and evaluations that are derived from or refer to the primary source literature. Examples include review articles (such as meta-analysis and systematic reviews) and reference works. Professionals within each discipline take the primary literature and synthesize, generalize, and integrate new research.

Tertiary Literature Tertiary literature consists of a distillation and collection of primary and secondary sources such as textbooks, encyclopedia articles, and guidebooks or handbooks. The purpose of tertiary literature is to provide an overview of key research findings and an introduction to principles and practices within the discipline.

Adapted from the Information Services Department of the Library of the Health Sciences-Chicago , University of Illinois at Chicago.

Types of Scientific Publications

These examples and descriptions of publication types will give you an idea of how to use various works and why you would want to write a particular kind of paper.

• Scholarly article aka empirical article
• Review article
• Conference paper

Scholarly (aka empirical) article -- example

Empirical studies use data derived from observation or experiment. Original research papers (also called primary research articles) that describe empirical studies and their results are published in academic journals.  Articles that report empirical research contain different sections which relate to the steps of the scientific method.

      Abstract - The abstract provides a very brief summary of the research.

     Introduction - The introduction sets the research in a context, which provides a review of related research and develops the hypotheses for the research.

     Method - The method section describes how the research was conducted.

     Results - The results section describes the outcomes of the study.

     Discussion - The discussion section contains the interpretations and implications of the study.

     References - A references section lists the articles, books, and other material cited in the report.

Review article -- example

A review article summarizes a particular field of study and places the recent research in context. It provides an overview and is an excellent introduction to a subject area. The references used in a review article are helpful as they lead to more in-depth research.

Many databases have limits or filters to search for review articles. You can also search by keywords like review article, survey, overview, summary, etc.

Conference proceedings, abstracts and reports -- example

Conference proceedings, abstracts and reports are not usually peer-reviewed.  A conference article is similar to a scholarly article insofar as it is academic. Conference articles are published much more quickly than scholarly articles. You can find conference papers in many of the same places as scholarly articles.

How Do You Identify Empirical Articles?

To identify an article based on empirical research, look for the following characteristics:

     The article is published in a peer-reviewed journal .

     The article includes charts, graphs, or statistical analysis .

     The article is substantial in size , likely to be more than 5 pages long.

     The article contains the following parts (the exact terms may vary): abstract, introduction, method, results, discussion, references .

• << Previous: Study Designs
• Next: Clinical Guidelines >>
• Last Updated: Mar 13, 2024 2:23 PM
• URL: https://research.library.gsu.edu/EBM

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 16 May 2024

Testing the reliability of an AI-based large language model to extract ecological information from the scientific literature

• Andrew V. Gougherty 1 &
• Hannah L. Clipp 1  

npj Biodiversity volume  3 , Article number:  13 ( 2024 ) Cite this article

Metrics details

• Data mining
• Invasive species
• Macroecology

Artificial intelligence-based large language models (LLMs) have the potential to substantially improve the efficiency and scale of ecological research, but their propensity for delivering incorrect information raises significant concern about their usefulness in their current state. Here, we formally test how quickly and accurately an LLM performs in comparison to a human reviewer when tasked with extracting various types of ecological data from the scientific literature. We found the LLM was able to extract relevant data over 50 times faster than the reviewer and had very high accuracy (>90%) in extracting discrete and categorical data, but it performed poorly when extracting certain quantitative data. Our case study shows that LLMs offer great potential for generating large ecological databases at unprecedented speed and scale, but additional quality assurance steps are required to ensure data integrity.

Introduction

The recent public release of multiple artificial intelligence (AI)-based language generating chatbots has garnered significant attention from both the public and scientific communities 1 , 2 . The ability of large language models (LLMs) to quickly process and synthesize large amounts of text and return a reasonable response to user queries has led to the suggestion that scientists could potentially begin to shift mundane, laborious, or time-consuming tasks to AI systems 3 . However, while it seems promising that LLMs can generate correct technical answers and seemingly reasonable responses, the tendency for LLMs to sometimes “hallucinate” or return objectively wrong information raises significant concern about whether LLMs, in their current (publicly available) state, can be relied upon to produce accurate results. Furthermore, biases in the training data can perpetuate errors that can be difficult to understand, given the “black box” nature of LLMs and the frequent lack of transparency in the data used for training 4 . As such, it is uncertain whether LLMs in their current form offer a useful tool for scientists that could improve productivity, efficiency, learning, and teaching or whether LLMs should be avoided as a research tool due to imprecision and unreliability. With AI being increasingly used in ecological studies, and the possibility of AI-based systems generating novel, testable hypotheses and predictions 5 , 6 , there is a pressing need to characterize the ability of AI-based systems to interact with ecological data.

Here, we sought to formally test the ability of an LLM to extract ecological information from scientific reports and, in the process, generate a database that could undergo further analysis. We focused on scientific reports of plant pathogens occurring on new hosts or in new geographic regions (i.e., emerging infectious diseases [EIDs]). These reports provide a valuable real-world case study, as thousands of new disease reports are published annually in the scientific literature—indeed, there are entire journals dedicated to the topic (e.g., New Disease Reports , Online ISSN:2044-0588)—produced at a rate that would challenge any researcher to keep up-to-date on this rapidly expanding literature 7 . These reports also provide important information for understanding the spread of invasive species, which may harm ecosystems 8 , native communities 9 , and crop production 10 , and for informing future management and surveillance of invasive species. Based on the results of our case study, we identified the strengths and weaknesses of the LLM in extracting different types of data, and we conclude by commenting on the potential usefulness of LLMs in general as a research tool.

Results and discussion

In total, data extraction via the LLM took approximately 5 min, while the reviewer took approximately 268 min to review the 100 reports, representing an over 50-fold difference. The LLM had a strong ability to accurately identify the pathogens, hosts, years, and countries described in the reports. Of the 103 pathogens described in the reports, 98.1% matched those identified by the reviewer (Kappa = 0.98, CI = 0.95–1.0). The only instances that were not exact matches were associated with the alder yellows pathogen. In this case, the reviewer correctly identified the pathogen as “Alder yellows phytoplasma,” while the LLM identified the pathogen as “Candidatus Phytoplasma alni.” While this pathogen name seems correct, it was not specifically mentioned in the report, nor were we able to find any mention of this specific pathogen name in the literature. Interestingly, this species name seems to follow the convention of other phytoplasmas (e.g., Candidatus Phytoplasma ulmi , Ca. P . fraxini ). Host identities were also matched with high accuracy. Of the 132 hosts identified by the reviewer or LLM, 91.7% were exact matches (Kappa = 0.92, CI = 0.87–0.96). The greatest errors were due to omission—that is, instances where the reviewer identified a host that the LLM did not. These were exclusively cases where multiple hosts were listed in the report, but the LLM failed to identify all of the hosts. A similar trend was found for the year during which EIDs were observed. Generally, the reviewer and LLM returned identical years, with an overall accuracy of 72.1%, which consisted of 106 “true positive” matches, 11 “false positive” cases, 14 mismatching values, 15 errors of omission [due primarily to the LLM missing hosts or locations identified by the reviewer], and 1 error of commission out of 147 total cases. Mismatched values often occurred when the EIDs were observed across a range of multiple years. Countries were identified with the highest accuracy rate—effectively, all countries identified by the LLM were exact matches (100%) to those identified by the reviewer (Kappa = 1.0).

When latitude/longitude coordinates were supplied in the report ( N  = 34 out of the 100 total reports, comprising 44 unique locations where diseases were first recorded; Fig. 1 ), they tended to be similar between the LLM and reviewer, but the LLM frequently struggled with converting given coordinates to decimal degrees. There were 46 total unique locations identified by both the reviewer and LLM from the reports, and 34.0% of the latitude/longitude coordinate values were an exact match (aside from negligible rounding issues that occurred when converting to decimal degrees and resulted in mean absolute differences in latitude and longitude of 0.0002 and 0.0004, respectively). For 16 locations, minor mismatches arose from seemingly random discrepancies in converting to decimal degrees, which resulted in mean absolute differences in latitude and longitude of 0.1369 and 0.0022, respectively. For 8 locations, the LLM completely failed to convert correctly to decimal degrees, returning latitude and longitude values that had mean absolute differences of 0.1733 and 0.1097, respectively. The remaining mismatches were due to errors of omission ( N  = 4), errors of commission ( N  = 2), and a case where 2 different locations were conflated and treated as a single location. Excluding those latter 7 cases, absolute differences ranged from 0–1.8383 (mean = 0.1052) for latitude and from 0–0.2800 (mean = 0.0270) for longitude.

Gray points ( N  = 44) are unique locations provided explicitly in the disease reports and identified by the human reviewer, and all other points ( N  = 110) are those identified by the large language model (LLM). In many cases, geographic coordinates were not provided in the disease reports, but the LLM automatically geocoded 70 unique locations, with high accuracy (98.6%) for placement in the correct country (black points) but uncertain precision (e.g., 3 sets of coordinates were located in bodies of water [see blue points]). Approximately 34.0% of the coordinates provided in the reports were precise matches (aside from negligible rounding issues) between values identified by both the reviewer and LLM (i.e., gray points with black border). Mismatches were due to small discrepancies in converting to decimal degrees ( N  = 16; yellow points), complete failure to correctly convert to decimal degrees ( N  = 8; orange points), errors of omission ( N  = 4; gray points with no border), errors of commission ( N  = 2; red points; note that these coordinates also failed to correctly convert to decimal degrees and resulted in the 2 points east of Japan, which should have been located in Australia), and conflation of 2 different sets of coordinates as a single set of coordinates ( N  = 1; purple point).

It is worth noting that there were 2 cases where the reviewer was able to interpret location-related information that the LLM could not. The first instances were 2 errors of commission by the LLM in returning latitude/longitude coordinates that were not identified by the reviewer; both were from a single report that contained 2 sets of coordinates for surveys that had been conducted to monitor for tree diseases but were not the actual coordinates of the first records of the disease. The reviewer was able to distinguish the context and indicated NA for the latitude/longitude coordinates, as they were not provided, but the LLM returned the 2 sets of tree survey coordinates. Thus, in this case, the LLM appeared to simply return any coordinates that were explicitly included in a report, even though they did not correspond to the requested location of a first disease record. The second instance was where the authors of the report apparently recorded the wrong longitude coordinate, resulting in a location that was far outside of the country where the disease was observed. The reviewer was able to correctly identify this inconsistency, but the LLM did not. While this is not entirely surprising, it verifies the LLM does not confirm the data it returns are internally consistent and, therefore, may be unlikely to identify errors in the inputted data without additional instruction.

Surprisingly, even when the report did not explicitly supply latitude/longitude coordinates, the LLM geocoded the locations (Fig. 1 ), producing coordinates for 70 unique locations with high accuracy (98.6%) for placement within the correct country. We were not expecting this behavior, but automatically geocoding the locations adds significant value to the extracted data, as climatic, land use and other environmental data could then be extracted for those locations. However, the LLM’s method of determining the geocoded location when it is not provided in the report is unknown, and we were unable to determine the reason for the single set of coordinates that were located within the incorrect country (~5 km from the border of the correct country). Furthermore, the precision of the geocoded locations is uncertain, particularly as 3 sets of coordinates were situated within bodies of water (Fig. 1 ), ranging 40 m to 5.2 km from the nearest shoreline.

The least accurate data extracted by the LLM were for pathogen incidence, which had an overall accuracy of 23.8% (consisting of 25 “true positive” matches, 10 “true negative” matches, 95 “false positive” cases, 1 mismatching value, 15 errors of omission [due primarily to the LLM missing hosts or locations identified by the reviewer], and 1 error of commission out of a total of 147 cases). Although the prompt specifically stated that NAs should be used when the data were not available in the report, the LLM assigned 53 of the 100 total reports with no incidence data as 100% incidence, whereas the reviewer returned an NA, and it was not clear how the LLM reached this answer. Of the cases where the reviewer identified an incidence value, the LLM frequently extracted the same value (96.2% matches, N  = 25/26), which seems to indicate that the LLM can extract numeric data, but the frequency of “false positive” cases was concerning.

Implications, limitations, and future directions

The workflow we utilized to automate data extraction from scientific reports highlights the potential for LLMs to rapidly generate large databases with relatively high accuracy, opening the potential for researchers to address new questions at a scale that was previously not possible. That said, there are numerous caveats to the approach we present, and to LLMs generally, that should be noted. First, the reports from which we extracted data were known a priori to have relevant pathogen, host, and geographic information. This use of generative AI is likely “safer” than some others, as we only asked the LLM to extract data we suspected were in the reports rather than find new data or generate text that was not already in the report. Further, the reports were short in length and relatively data-dense, so while the LLM was usually able to identify relevant data, it is yet unclear how well it would interact with longer text sources (e.g., journal articles with thousands of words). Further, the data we were extracting were relatively simplistic (e.g., pest/host scientific names, country names), which likely facilitated high accuracy rates. Our results are largely consistent with ref. 11 , who suggested LLMs respond best to simple, straightforward tasks that do not require multiple sequential steps. Despite its simplicity, however, we note this type of information is valuable for tracking pathogens, pests, and invasive species in new areas and could be useful for both automating surveillance and identifying high-risk pathogens and pests before they arrive in a new region.

The LLM’s ability to distinguish between pathogen and host names was particularly useful as it indicates the LLM is not simply searching for formatting clues of scientific names (e.g., italics). Rather, it implies the pathogen and host names are evident from the context of the text and/or that possibly some of these scientific names occur in the data used to train the LLM. Interestingly, several of the pathogens in the reports were parasitic plants, which the LLM correctly identified as the pathogen—again suggesting an ability to distinguish between the antagonistic species and host species. However, the LLM did not consistently extract the quantitative data correctly. The tendency to assign 100% incidence when no incidence data were provided was somewhat disconcerting, and it was unclear why this occurred. If the general workflow from our case study were to be used to generate quantitative datasets, numerous quality assurance steps would need to be taken to ensure reasonable accuracy before further analysis. The fast pace of development of publicly available LLMs could overcome these issues in the near future, and formal comparison of multiple LLMs could clarify differences in their ability to interact with ecological data.

Despite the relatively high accuracy of the data extracted, the workflow could yet be improved. For instance, although the “off-the-shelf” LLM from our case study produced acceptable results for much of the data, fine-tuning an LLM could help improve the accuracy of the incidence or other quantitative data 12 . An LLM trained in ecological text could be particularly advantageous for more complex types of data, such as when effect sizes are needed for a meta-analysis. However, even the ability to reliably extract discrete/categorical data can prove tremendously valuable as these types of data could be used to identify novel ecological associations, biological introductions/invasions, and interactions between species and their environment. The ability of LLMs to interpret an expanding range of languages can also help overcome biases that may emerge when focusing solely on the English-language scientific literature 13 , 14 . However, the growing number and capabilities of LLMs, along with their associated data processing requirements, should warrant a certain amount of reflection on their use as cumulative environmental costs have yet to be fully realized 15 .

Source text

We used reports from a recent study on the global accumulation of emerging infectious tree diseases 7 . EIDs are generally defined as diseases occurring in a new geographic region, on a new host, or recently increasing in impact 16 . For plant species, EIDs are frequently documented in the literature as “First reports”, in which authors describe the conditions where the pathogen was detected and the methodological approaches used to identify the pathogen. These reports are frequently short in length, similar to the word count of a typical abstract (e.g., current guidelines for Plant Disease , the source of the disease reports used in our case study, state that reports should be ≤2985 characters). Because these reports are known to contain new and important ecological information, they offer a unique opportunity to test an LLM’s ability to extract relevant ecological information from the scientific literature. We used the first 100 reports from ref. 7 , which represent unique hosts and pathogens reported in new regions.

LLM data extraction

For our case study testing the ability of an LLM to extract various types of ecological data from the scientific literature, we elected to use the publicly available text-bison-001 generative text model from Google. As a generative text model, text-bison-001 is designed to return only the relevant text requested, which can then be parsed to a table, without the superfluous conversational text that would accompany responses by a chat model (e.g., OpenAI’s ChatGPT, Google’s Bard). In addition, the text-bison-001 model can be accessed freely from an API, which allows for an entirely scripted workflow that bypasses the need to manually copy and paste data to and from a web browser.

We prompted the LLM to extract multiple pieces of information from the disease reports, including the scientific names of the pathogen and hosts described in the report, the incidence of the pathogen (i.e., percent of hosts affected by the pathogen), and when (i.e., year) and where (i.e., country) the pathogen was detected. The development of the prompt required some iterative experimentation to ensure that the requested data were returned accurately and consistently. In initial testing, for instance, we found that the LLM sometimes returned geographic coordinates as degrees–minutes–seconds or returned common names when scientific names were available. We found that explicitly stating the desired format for these variables increased the consistency of data extraction. Furthermore, because the data table was returned as a single text string, we realized that we needed to request that columns be delimited by a vertical bar (as opposed to a comma), because locations occasionally included commas, and this specification was necessary to properly delimit the table. As part of our workflow, the title and text of each report were appended to a text prompt, which described the data we wished to extract and the desired format of the response. The prompt read:

“The following is an abstract describing a plant pathogen on a new host or in a new geographic area.

I’d like to know (i) what is the scientific name of the pathogen?

(ii) what is the scientific name of the host?

(iii) what percentage of hosts were infected by the pathogen?

(iv) what year was the pathogen sampled?

(v) where was the pathogen observed?

(vi) what country was the pathogen observed in? and

(vii) what are the latitude/longitude, in decimal degrees, of the location where the pathogen was observed.

For the coordinates, don’t include letters to indicate the cardinal directions, but use negative numbers to indicate west and south.

It is very important that the latitude and longitude be returned in decimal degrees.

If there are multiple pathogens, hosts, or locations, include each as a separate row.

If any of the information is not included in the abstract, use NA.

Use a vertical bar | to delimit the table. Use the column names: ‘Pathogen’ [scientific name of pathogen], ‘Host’ [scientific name of host], ‘Percentage’ [percent of hosts infected], ‘Year’ [year pathogen was sampled], ‘Location’ [location where the pathogen was observed], ‘Country’ [country where pathogen was observed], ‘Latitude’ [latitude in decimal degrees], ‘Longitude’ [longitude in decimal degrees].

Always use scientific names when possible. Do not summarize the abstract. Return only a table. Here is the title, followed by the abstract:”

We interacted with the LLM through Google’s developer API, which was accessed with the httr package 17 in the R statistical program 18 . The data from each report were returned as a single text string, with rows delimited with a newline designator (\n) and columns delimited with a vertical bar (|). The workflow was entirely scripted, and the table with the relevant responses was saved as an Excel file. The entire script to interact with the LLM is available at https://doi.org/10.6084/m9.figshare.24646302 .

In initial testing, we found the LLM occasionally flagged the prompt and reports as being “derogatory” or “toxic” and would not return a response. It was not immediately clear why these particular reports might be considered derogatory or toxic. We were able to adjust the thresholds for the allowable derogatory/toxic content level in the response, which fixed the issue for the problematic reports. Of the 100 reports tested, 87 returned a result without adjusting the allowable derogatory/toxicity level, and 13 returned a result only after adjusting the allowable derogatory/toxicity level. Furthermore, to improve the repeatability of the responses, we set the “temperature” of the response to zero. Temperature controls the degree of creativity and stochasticity in the response. Setting the temperature to zero made the responses more deterministic, as the model always selected the highest probability response (see the API guide: https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/text ).

We tested how well the data extracted by the LLM compared to those extracted by an independent human reviewer (H. L. Clipp; hereafter, reviewer), who had not worked with these data previously. For the discrete variables (i.e., the identity of the pathogen, host, country, and year of sampling), we calculated 2 validation statistics: (i) an overall accuracy metric that was calculated as the number of exact matches between the LLM and reviewer divided by the total number of unique returns by the LLM and reviewer, and (ii) Cohen’s Kappa using the psych package in the R statistical program 19 . For the quantitative variables (i.e., latitude/longitude, incidence), we calculated LLM accuracy as the percentage of unique values that matched between those identified by both the LLM and reviewer, and we tallied the reasons for discrepancies (e.g., errors of commission/omission). We additionally calculated the absolute differences between unique latitude and longitude values returned by the LLM and reviewer. Any discrepancies between the reviewer and the LLM were assessed by the first author (A. V. Gougherty) to confirm the reviewer had extracted the correct information. We allowed some flexibility when the reviewer and LLM had minor disagreements. For instance, when hosts were identified to the subspecific level, we considered it a match whether or not the reviewer or LLM included “subsp.”, “ssp”, or no specific designation for the sub-specific epithet. Similarly, when the host was identified only to the genus level, we allowed the inclusion, or not, of “sp.” or “spp.” as a species identifier.

Data availability

The text of the disease reports and script to interact with the LLM is available at https://doi.org/10.6084/m9.figshare.24646302 .

Ayala, P. Are large language models right for scientific research. https://www.cas.org/resources/cas-insights/emerging-science/are-large-language-models-right-scientific-research (2023).

Birhane, A., Kasirzadeh, A., Leslie, D. & Wachter, S. Science in the age of large language models. Nat. Rev. Phys. 5 , 277–280 (2023).

Article   Google Scholar  

Borger, J. G. et al. Artificial intelligence takes center stage: exploring the capabilities and implications of ChatGPT and other AI-assisted technologies in scientific research and education. Immunol. Cell Biol. 101 , 923–935 (2023).

Article   PubMed   Google Scholar  

O’Neill, M. & Connor, M. Amplifying limitations, harms and risks of large language models. Preprint at http://arxiv.org/abs/2307.04821 (2023).

Agathokleous, E., Saitanis, C. J., Fang, C. & Yu, Z. Use of ChatGPT: what does it mean for biology and environmental science? Sci. Tot. Env. 888 , 164154 (2023).

Article   CAS   Google Scholar  

Doi, H., Osawa, T. & Tsutsumida, N. The Role of Large Language Models in Ecology and Biodiversity Conservation: Opportunities and Challenges . https://www.authorea.com/users/313786/articles/648830-the-role-of-large-language-models-in-ecology-and-biodiversity-conservation-opportunities-and-challenges?commit=3cda2d0c695b11ef0925fb8164511add4d006b63 ; https://doi.org/10.22541/au.168657324.49460085/v1 (2023).

Gougherty, A. V. Emerging tree diseases are accumulating rapidly in the native and non-native ranges of Holarctic trees. NeoBiota 87 , 143–160 (2023).

Lovett, G. M., Canham, C. D., Arthur, M. A., Weathers, K. C. & Fitzhugh, R. D. Forest ecosystem responses to exotic pests and pathogens in eastern North America. BioScience 56 , 395–405 (2006).

Fei, S., Morin, R. S., Oswalt, C. M. & Liebhold, A. M. Biomass losses resulting from insect and disease invasions in US forests. Proc. Natl Acad. Sci. USA 116 , 17371–17376 (2019).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Rossman, A. Y. The impact of invasive fungi on agricultural ecosystems in the United States. in Ecological Impacts of Non-Native Invertebrates and Fungi on Terrestrial Ecosystems (eds Langor, D. W. & Sweeney, J.) 97–107 (Springer Netherlands, Dordrecht, 2009). https://doi.org/10.1007/978-1-4020-9680-8_7 .

Merow, C., Serra-Diaz, J. M., Enquist, B. J. & Wilson, A. M. AI chatbots can boost scientific coding. Nat. Ecol. Evolut. 7 , 960–962 (2023).

Le Guillarme, N. & Thuiller, W. TaxoNERD: deep neural models for the recognition of taxonomic entities in the ecological and evolutionary literature. Methods Ecol. Evolut. 13 , 625–641 (2022).

Nuñez, M. A. & Amano, T. Monolingual searches can limit and bias results in global literature reviews. Nat. Ecol. Evolut. 5 , 264–264 (2021).

Zenni, R. D. et al. Multi-lingual literature searches are needed to unveil global knowledge. J. Appl. Ecol. 60 , 380–383 (2023).

Bender, E. M., Gebru, T., McMillan-Major, A. & Shmitchell, S. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? in Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency 610–623 (Association for Computing Machinery, New York, NY, USA, 2021). https://doi.org/10.1145/3442188.3445922 .

Anderson, P. K. et al. Emerging infectious diseases of plants: pathogen pollution, climate change and agrotechnology drivers. Trends Ecol. Evolut. 19 , 535–544 (2004).

Wickham, H. Httr: Tools for Working with URLs and HTTP. R Package Version 1.4.5 . (2023).

R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing (2023).

Revelle, W. Psych: Procedures for Psychological, Psychometric, and Personality Research (Evanston, IL, 2024).

Download references

Author information

Authors and affiliations.

USDA Forest Service Northern Research Station, Delaware, OH, USA

Andrew V. Gougherty & Hannah L. Clipp

You can also search for this author in PubMed   Google Scholar

Contributions

A.V.G. conceptualized the work, extracted data from the LLM, and wrote the first draft of the paper. H.L.C. extracted the data and edited the paper.

Corresponding author

Correspondence to Andrew V. Gougherty .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Gougherty, A.V., Clipp, H.L. Testing the reliability of an AI-based large language model to extract ecological information from the scientific literature. npj biodivers 3 , 13 (2024). https://doi.org/10.1038/s44185-024-00043-9

Download citation

Received : 08 December 2023

Accepted : 08 March 2024

Published : 16 May 2024

DOI : https://doi.org/10.1038/s44185-024-00043-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Researcher Views on Multi-omics Return of Results to Research Participants: Insights from the Molecular Transducers of Physical Activity Consortium (MoTrPAC) Study

• Find this author on Google Scholar
• Find this author on PubMed
• Search for this author on this site
• ORCID record for Kelly E Ormond
• For correspondence: [email protected]
• ORCID record for Caroline Stanclift
• ORCID record for Chloe M Reuter
• ORCID record for Jennefer N Carter
• ORCID record for Kathleen E. Murphy
• ORCID record for Malene E Lindholm
• ORCID record for Matthew T Wheeler
• Info/History
• Preview PDF

Background: There is growing consensus in favor of returning individual specific research results that are clinically actionable, valid, and reliable. However, deciding what and how research results should be returned remains a considerable challenge. Researchers are key stakeholders in return of results decision-making and implementation. Multi-omics data contains medically relevant findings that could be considered for return. We sought to understand researchers' views regarding the potential for multi-omics data derived return of results from a large, national consortium generating multi-omics data. Methods: Researchers from the Molecular Transducers of Physical Activity Consortium (MoTrPAC) were recruited for in-depth semi-structured interviews. To assess understanding of potential clinical utility for types of data collected and attitudes towards return of results in multi-omic clinical studies, we devised an interview guide focusing on types of results generated in the study which could hypothetically be returned based on review of the literature and professional expertise of team members. The semi-structured interviews were recorded, transcribed verbatim and co-coded. Thematic trends were identified for reporting. Results: We interviewed a total of 16 individuals representative of 11 sites and 6 research roles across MoTrPAC. Many respondents expressed positive attitudes regarding hypothetical multi-omics results return, citing participant rights to their data and perception of minimal harm. Ethical and logistical concerns around the return of multi-omics results were raised, including: uncertain clinical validity, a lack of expertise to communicate results, and an unclear obligation regarding whether to return multi-omics results. Further, researchers called for more guidance from funding agencies and increased researcher education regarding return of results. Conclusion: Overall, researchers expressed positive attitudes toward multi-omic return of results in principle, particularly if medically actionable. However, competing ethical considerations, logistical constraints, and need for more external guidance were raised as key implementation concerns. Future studies should consider views and experiences of other relevant stakeholders, specifically clinical genomics professionals and study participants, regarding the clinical utility of multi-omics information and multi-omics results return.

Competing Interest Statement

Kathleen Murphy declares holding Illumina stock. The remaining authors declare no competing interests. Funding This paper was supported in part by the NIH Common Fund (award number U24OD026629).

Funding Statement

This paper was supported in part by the NIH Common Fund (award number U24OD026629).

Author Declarations

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

The IRB of Stanford University School of Medicine gave ethical approval for this work (Protocol 57147).

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Data Availability

De-identified transcripts, interview guides and other data are available upon reasonable request to the corresponding author.

View the discussion thread.

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Citation Manager Formats

• EndNote (tagged)
• EndNote 8 (xml)
• RefWorks Tagged
• Ref Manager
• Tweet Widget
• Facebook Like
• Google Plus One
• Addiction Medicine (324)
• Allergy and Immunology (627)
• Anesthesia (163)
• Cardiovascular Medicine (2371)
• Dentistry and Oral Medicine (289)
• Dermatology (206)
• Emergency Medicine (379)
• Endocrinology (including Diabetes Mellitus and Metabolic Disease) (836)
• Epidemiology (11768)
• Forensic Medicine (10)
• Gastroenterology (702)
• Genetic and Genomic Medicine (3736)
• Geriatric Medicine (350)
• Health Economics (633)
• Health Informatics (2395)
• Health Policy (932)
• Health Systems and Quality Improvement (896)
• Hematology (341)
• HIV/AIDS (782)
• Infectious Diseases (except HIV/AIDS) (13308)
• Intensive Care and Critical Care Medicine (767)
• Medical Education (365)
• Medical Ethics (104)
• Nephrology (398)
• Neurology (3501)
• Nursing (198)
• Nutrition (524)
• Obstetrics and Gynecology (674)
• Occupational and Environmental Health (663)
• Oncology (1823)
• Ophthalmology (537)
• Orthopedics (218)
• Otolaryngology (287)
• Pain Medicine (232)
• Palliative Medicine (66)
• Pathology (446)
• Pediatrics (1033)
• Pharmacology and Therapeutics (426)
• Primary Care Research (420)
• Psychiatry and Clinical Psychology (3175)
• Public and Global Health (6138)
• Radiology and Imaging (1280)
• Rehabilitation Medicine and Physical Therapy (747)
• Respiratory Medicine (826)
• Rheumatology (379)
• Sexual and Reproductive Health (372)
• Sports Medicine (323)
• Surgery (402)
• Toxicology (50)
• Transplantation (172)
• Urology (145)

• Open access
• Published: 14 May 2024

The development of probiotics and prebiotics therapy to ulcerative colitis: a therapy that has gained considerable momentum

• Jing Guo 1 ,
• Liping Li 1 ,
• Yue Cai 2 &
• Yongbo Kang 1  

Cell Communication and Signaling volume  22 , Article number:  268 ( 2024 ) Cite this article

22 Accesses

1 Altmetric

Metrics details

Ulcerative colitis (UC) is increasingly common, and it is gradually become a kind of global epidemic. UC is a type of inflammatory bowel disease (IBD), and it is a lifetime recurrent disease. UC as a common disease has become a financial burden for many people and has the potential to develop into cancer if not prevented or treated. There are multiple factors such as genetic factors, host immune system disorders, and environmental factors to cause UC. A growing body of research have suggested that intestinal microbiota as an environmental factor play an important role in the occurrence and development of UC. Meanwhile, evidence to date suggests that manipulating the gut microbiome may represent effective treatment for the prevention or management of UC. In addition, the main clinical drugs to treat UC are amino salicylate and corticosteroid. These clinical drugs always have some side effects and low success rate when treating patients with UC. Therefore, there is an urgent need for safe and efficient methods to treat UC. Based on this, probiotics and prebiotics may be a valuable treatment for UC. In order to promote the wide clinical application of probiotics and prebiotics in the treatment of UC. This review aims to summarize the recent literature as an aid to better understanding how the probiotics and prebiotics contributes to UC while evaluating and prospecting the therapeutic effect of the probiotics and prebiotics in the treatment of UC based on previous publications.

Introduction

Ulcerative colitis (UC) is a chronic non-specific intestinal inflammatory disease [ 1 ]. UC becomes an important health problem, because it’s high morbidity. Especially in newly industrialized countries [ 2 ]. Research shows that the incidence of UC is 10 to 20 patients per 100,000 people every year [ 3 ]. UC often presents with recurrent attacks. And the inflammatory of UC will become a factor of colon cancer in the long run [ 4 ]. The pathogenic factors of UC are sophisticated, it is related to intestinal microbiota, immune function of the body (For example, UC is closely related with Th2 cells) [ 5 ], genetic factor and environment factor (e.g. life-style, dietary habits) and so on [ 6 ]. wherein, intestinal microbiota is one of the most important factor that arise UC [ 7 ]. Therefore, we can use probiotics to regulate the intestinal flora in the treatment of UC [ 8 , 9 ]. A growing body of research has shown that probiotics and prebiotics can bring about remission the symptoms of UC improving intestinal mucosal homeostasis, ameliorating the intestinal microbiota environment, regulating the body’s immune function. Therefore, probiotics and prebiotics may be a very safe and efficient treatment for UC. At the same time, it can greatly reduce the financial burden of patients. Furthermore, New techniques have made it possible to attempt systematic studies of probiotics prebiotics, which can provide more specific information about their functions and pathological variations. This review summarizes cutting-edge research on probiotics and prebiotics treatment for UC, existing issues in probiotics treatment and prebiotics therapy, the future of probiotics and prebiotics, and microbial therapeutics.

Pathogenesis of UC

Ulcerative colitis is a chronic inflammatory disorder of the gastrointestinal tract. It is characterized by a progressive decline in health. UC is marked by inflammation of the mucosal lining, usually confined to the colon and rectum [ 10 ]. The pathogenesis of UC is closely related to a variety of factors, such as genetics and environment [ 11 ]. Statistically, genetics can only explain 7.5% of the variation in disease and has little predictive power for phenotype. Therefore, it has limited clinical application. Examples of loci associated with increased susceptibility to UC including genes associated with barrier function and human leukocyte antigen, such as HNF4A and CDH1 [ 12 , 13 ]. Environment plays an important role in the development of UC. Such as, living condition, hygiene, diet, etc. While UC is mainly due to immune dysfunction and intestinal barrier dysfunction. Colonic epithelial cells (colonocytes), as the first line of defense of the gut immune system, are closely related to the pathogenesis of the UC. Research findings, the expression of peroxisome proliferator-activated receptor γ (PPAR γ) is reduced in the colonocytes in patients with UC. And the reduced expression of PPAR γ, which is a nuclear receptor that downregulates inflammation, will stimulate an inflammatory cascade responses through a series of immune responses, leading to the production of large quantities of inflammatory factors [ 14 ]. Also, when certain genes in the intestinal epithelium are functionally deficient, it may lead to disruption of the intestinal barrier function [ 10 ]. The deficiency or malfunction of various immune cells and the abnormal expression of cytokines, which play an important signaling function, can also lead to inflammation, which, if prolonged, can lead to the development of UC. The intestinal immune system also involves the intrinsic and adaptive immunity [ 15 ], involving a variety of immune cells and molecules and others. If dendritic cells abundantly express Toll-like receptors (TLR) which can recognize pathogen pattern receptors, this will leads to the activation of several inflammatory signaling pathway, such as NF-κB [ 16 ] and MAPK pathway, triggering an inflammatory response. The production of large amounts of pro-inflammatory factors affects the differentiation of immune cells such as T cell differentiation towards subpopulation. For example, massive activation of Th2 cells leads to high expression of IL-13, which induces apoptosis of epithelial cells and disrupts the integrity of mucosal barrier [ 17 , 18 ]. Other T helper cells also play an important role in UC. And some research suggest that Breg deficiency may also associated with UC [ 19 ]. The damage of the intestinal mucosal barrier is also an important causative factor in UC. Intestinal secretory dysfunction such as decreased secretion of antimicrobial peptides and mucus layer, or structural defects of intestinal barrier including occludin, ZO-1, ZO-2 and so on. It has been found that the disruption of human gut microbiota, the largest collection of microbes within the body [ 20 ], is critical in the progression of UC, but the specific mechanism is not yet clear.

The role of gut microflora in UC

Gut microflora lives on intestinal mucosal and forms bacterial layer. Thus, there is a strong and complex relationship between gut microbiota and gut. Intestinal dysbacteriosis can leads to a decrease in intestinal defense function and immune regulatory function. Furthermore, the decrease of the body immune function and an increase in associated pathogenic factors leading to the intestinal mucosal invasion or exacerbates the gastrointestinal diseases [ 6 ]. Recently, a large number of studies have shown that alterations of intestinal microbiota can play an important role in the occurrence and development of UC. Meanwhile, some studies have shed light on UC subjects exhibiting alterations in the relative abundance of “beneficial” and potentially “harmful” bacteria compared to healthy subjects. The existence of a link between UC and the gut microbiota was indicated based on studies in animals and patients with UC. Changes of gut microbiota together with their-derived products and metabolites account for the important factors to promote UC occurrence. Here, the possible mechanisms of microbiome-gut action in promoting UC occurrence are discussed as well as outlined in Fig.  1 .

The mechanism of UC caused by dysbiosis of gut microbiota. Research findings, the decline of certain beneficial bacteria inhibits the conversion of food protein into organic acid which can nourish epithelial cells and inhibit pathogenic bacteria. Firmicutes as a major producer of butyrate (a kind of SCFAs), its decline leads to lower intestinal SCFAs. Leading the decreased secretion of epithelial repair cytokine interleukin-18, reduced the integrity of epithelial cells, and inhibited goblet cells secrete mucin and modification of tight junctions. And the decline of some gut microbiota also can lead to a decrease of indoles and their derivatives (e.g., IAA, IPA and IAID) which is produced by tryptophan. Thereby reducing the activation of AhR, a member of the activation of PER-ARNT-SIM (PAS) superfamily of transcription factors. The activation of AhR can inhibited the expression of NF-κB in a manner dependent on suppressor of cytokine signaling 2 (SOCS2). And AhR can also maintains the integrity of intestinal barrier activation by increasing the expressions of intestinal tight junction protein (TJPs) or activating the AhR-Nrf2 pathway. All of these effects were reversed due to the decrease of IAA, IPA or IAID. Thus lead to the increase of inflammatory factors (e.g., TNF-α and IL-17) and oxidative damage. Other researchers found that certain pathogenic bacteria such as Bacteroides (B.) fragilis and capsular lipopolysaccharide A can activate NF-κB signaling pathway and promote the secretion of inflammatory factors. The gut microbiota dysbiosis can also lead to the decreased synthesis of secondary bile acid. And secondary acid act as high-affinity ligands for TGR5 and FXR, its decline can promote NF-κB activation to synthesize inflammatory and the expression of proinflammatory cytokines secreted by monocyte and downregulate the expression of FGF19 and promote the synthesis of bile acids thus increasing its toxicity effect on tissues. As an intestinal pathogen, the increase of sulphate-reducing bacteria leads to cell disintegration and inflammatory via toxic sulfide. All of these can lead to the occurrence and development of UC.

A large number of studies have shown that patients with UC have a decrease in the bacterial diversity of gut microbiota [ 21 ]. Animal study results indicate a close association between gut microbiota and UC. Li et al. found that Firmicutes and Proteobacteria increased, whereas Bacteroidetes decreased in UC rats. And Lactobacillus , Lachnospiraceae_NK4A136_group , Prevotella_9 and Bacteroides were dominant genera in the model group [ 22 ]. Consistent with animal studies, the existence of a link between UC and the gut microbiota was indicated based on studies in patients with UC. Guo et al. also found that the abundance of Bacteroides and Clostridium sub-cluster XIVab as well as the concentration of organic acids significantly decrease by comparing with healthy individuals [ 23 ]. Similarly, Mizoguchi et al. shown that UC patients harbored relatively more abundant Actinobacteria, Proteobacteria and Tenericutes [ 24 ]. A comparison between UC and healthy individuals differed in the composition and diversity of the microbiota, with an upward trend in the Clostridium cluster IX and a decreased Clostridium cluster XIVa in patients with UC [ 25 ]. Consistent with the above results, there is a reduced amounts of bacterial groups from the Clostridium cluster XIVa , and the levels of Bacteroidetes was increased [ 26 ].

In addition, Kotlowski et al. found that the numbers of Escherichia coli were high in the rectal tissue of patients with UC [ 27 ]. By comparing with healthy controls, Xu et al. showed that the inflamed mucosa had more Proteobacteria (e.g. Escherichia–Shigella ) and fewer Firmicutes (e.g., Enterococcus ) [ 28 ]. As demonstrated by Schwiertz et al., Patients with active UC have lower cell counts of Bifidobacterium than healthy controls [ 29 ]. Another study found that the sulfate-reducing bacteria which is the dominant microflora in UC, it may proliferate with the release of toxic sulfide [ 30 ].

Recently, Verma et al. shown that during the active and remission stages of UC cases, the proportions of Bacteroides , Eubacterium , and Lactobacillus spp. are decrease [ 31 ]. Similarly, in another analysis of mucosa-associated flora in UC patients, it was learned that UC patients contained proportionally less Firmicutes , and correspondingly more Bacteroidetes [ 32 ]. Tahara et al. demonstrated that Fusobacterium nucleatum is common which is isolate from human intestinal biopsy from UC, compared to healthy controls [ 33 ].

In keeping with these results, Machiels et al. found that there is a decrease of the Roseburia hominis and Faecalibacterium prausnitzii in patients with UC [ 34 ]. Lepage et al. demonstrated that patients with UC are characterized by more Actinobacteria and Proteobacteria and less bacteria from the Lachnospiraceae and Ruminococcaceae families [ 35 ]. Likewise, a significant reduction was found on the UC mucosa compared with the non-IBD controls, that is levels of Clostridium clostridioforme , the Eubacterium rectale group, Faecalibacterium prausnitzii , Bifidobacteria , Lactobacilli , and Clostridium butyricum [ 36 ]. Consistent with the above results of this study, patients with UC in remission compared to that of controls, there is a loss of Bacteroides , Escherichia , Eubacterium , Lactobacillus , and Ruminococcus spp [ 37 ].

Recently, Hu et al. [ 38 ] found that the decreased of the dominant bacteria that digest food carbohydrates to short chain fatty acid (SCFA) lead to the reduce of intestinal barrier integrity (for example, the decrease of TJPs in colon). Guo et al. [ 23 ] also found that SCFAs can affect the secretion of the epithelial repair cytokine interleukin-18. And they found that the decreased of Bacteroides and Clostridium sub-cluster XIVab leading to the decrease of organic acid, which reduces the trophic effect of organic acid to epithelial cells and the inhibitory effect on pathogenic bacteria [ 39 ]. Agus et al. [ 40 ] found that the reduced of certain intestinal flora inhibited the conversion of tryptophan to indole and its derivatives, and AhR as a receptor of indole and its derivatives, its activation will reduced, thereby inhibiting the intestinal TJP and AhR-Nrf2 pathway, leading to the reduced of intestinal barrier integrity and increased oxidative stress [ 41 ]. Rothhammer et al. [ 42 ] demonstrated that the reduce of AhR can promote the activation of NF-κB pathway in a manner dependent on suppressor of cytokine signaling 2 (SOCS2), then increase the expression of a number of inflammatory factors, including TNF-α and IL-12 et al. It is reported that some bacteria regulate the secretion of TNF-α and IL-12 by activating the NF-κB pathway through TLR2 receptor [ 43 ]. Iracheta et al. [ 44 ] found that primary bile acid are converted to secondary bile acid by gut microorganisms after being secreted into gut through a series of reactions, and that a decline of these gut microorganisms leads to a decrease of secondary bile acid. The decrease of secondary bile acid, which act as high-affinity ligands for TGR5 and FXR, leads to a decreased activation of TGF5 and FXR. The inhibitory effect of TGR5 on NF-κB is reduced, thereby promoting the activation of NF-κB. Reduced activation of FXR down-regulates the expression of FGF19, then its inhibitory effect to hepatic bile acid is declined, leading to a further increase of bile acid and exacerbating the development of inflammation [ 45 ]. And the decrease of secondary bile acids promote the secretion of pro-inflammatory factors by monocytes [ 46 ]. Figliuolo et al. [ 47 ] found that the increase of sulphate-reducing bacteria lead to an increase of toxic sulfide, which cause the disruption of gut epithelial cell and increase intestinal inflammatory.

Taken together, these results provide further insights into a role for gut microbiota in the pathogenesis of UC and might potentially serve as guidance for the interventions of UC by manipulating gut microbiota.

Research advances existing challenges IBD treatment

At present, there are many various treatment methods for IBD. Conventional treatment is the use of pharmacotherapy, including aminosalicylates, corticosteroids (CSs), immunomodulators (e.g., thiopurines (TPs), methotrexate (MTX), and calcineurin inhibitors), and biologics (e.g., pro-inflammatory cytokine inhibitors and integrin antagonists). Surgical resection and other methods including apheresis therapy, antibiotics, probiotics and prebiotics can also be used for treatment [ 48 ]. However, the side effects and high reccurence rate of these substances and methods limit there application. For example, research found, although aminosalicylates have been used in the treatment of IBD for the past 80 years, its efficacy remains controversial. And its mild side effects include diarrhea, nausea, abdominal pain, flatulence and others [ 49 ]. Severe cases can lead to infertility and anemia. CSs inhibits the transcription of certain inflammatory factors [ 50 ] and regulate the expression of certain anti-inflammatory genes [ 51 ] through certain signaling pathways. And it has many side effects, including diabetes mellitus, hypertension, venous thromboembolism (VTE), etc [ 52 ].. Some patients may also have dependence on this medication [ 53 ]. TPs inhibits intestinal inflammatory response by regulating T cell proliferation and activation. But TPs can cause side effects such as liver damage [ 54 ] and gastrointestinal intolerance [ 55 ]. MTS excerts its effects also by downregulating inflammatory factors. But it can cause adverse reactions such as fatigue, diarrhea, pneumonia and rash [ 51 ]. Calcineurin inhibitors also supresses inflammatory responses by interfering with signaling pathways. The incidence of side effects of calcineurin inhibitors is high, including renal function damage, hyperkalemia and infectious diseases and so on [ 56 ]. Anti-TNF therapy will inhibit the secretion of pro-inflammatory factor TNF-α. Anti-IL-12/23 therapy works by inhibiting the production of pro-inflammatory factor IL-12 and IL-23 by antigen-presenting cells. Anti-integrin therapy inhibits the accumulation of white blood cells in intestinal and alleviates intestinal inflammatory. But these biological agents are expensive and many patients may experience unresponsive and intolerant states. Therefore, it is urgent to study effective and safe methods to treat UC.

In the recent years, regulating gut microbiota has become a hot topic in the treatment of UC. Therefore, as a promising method for treating IBD, probiotics act as live microorganisms have therapeutic effects on IBD which is caused by intestinal ecological disorders and other reasons. The treatment of IBD can be achieved through its antioxidant effects [ 57 ], the regulatory effect on gut microbiota [ 58 ], anti-inflammatory effect [ 59 ], the promotion effect to intestinal barrier integrity [ 60 ] and so on. As an indigestible food ingredient, prebiotics can also be used to treat or alleviate UC by regulating the redox system, immune system, etc. It can also selectively regulate colon microbiota, for example, enhancement of beneficial intestinal bacteria and inhibition of the growth of pathogenic microorganisms. All of these suggests that probiotics and prebiotics have a lot of room to develop as new form of treatment.

Effect and mechanism of probiotics and prebiotics in treating UC

Probiotics are nonpathogenic living microorganisms which, when administered in adequate amounts, have been shown to confer health benefits to the host and regulate intestinal microecological balance. Probiotics are widely used in medical application to prevent or treat many diseases, such as obesity [ 61 ], hepatocellular Carcinoma [ 62 ], autoimmune hepatitis [ 63 ], diabetic retinopathy [ 64 ], and alcoholic liver disease [ 65 ] and so on. The therapeutic effects of probiotics on UC have also been confirmed in animals and humans (Tables  1 and 2 ). Thus, therapeutic interventions with probiotics may offer new treatment for UC. Here, the possible effects and mechanisms of probiotics in the treatment of UC are summarized in Fig.  2 .

The potential mechanism of probiotics in alleviating Ulcerative Colitis (UC). Probiotics that enter the gut can bind with corresponding receptors (e.g. PTK) which are on the intestinal epithelial cells, then inhibit its stimulation to MAPKKK (e.g. TNK1, ASK1, MEKK1, MLK3), further suppress the activation of MAPKK (e.g. MKK3/6, MKK4/7) which are activated by MAPKKK, thereby inhibiting the activation of MAPK (e.g. p38, JNK1,2,3). Blocking the transcription factor transcribe of relevant genes (e.g. Cyclin D1, Raf). Finally, inhibition the inflammatory, apoptosis, and differentiation activated by this pathway. Meanwhile, probiotics protect the intestinal barrier by increasing the levels of tight junction proteins of ZO-1 and Occludin between intestinal epithelial cells, preventing the invasion of pathogenic microorganisms. In addition, probiotics can bind with its receptors (e.g. TLR) on the intestinal epithelial cells, inhibiting the activation of adaptor protein (e.g. RIP1) and suppressing the recruitment of TAB/TAK complex, thereby inhibiting the ubiquitination degradation of IκB by ubiquitinatingNEMO. Prevents the release of NF-κB proteins (RelA/p50) to nucleus. Ultimately inhibits the transcription of proinflammatory factors (e.g. TNF-β) and reduces the promotion effect of TNFα releasing by macrophages to this pathway. Meanwhile, probiotics act on intestinal epithelial cells-associated receptors (e.g. TLR), then phosphorylate AKT, and inhibit the degradation of Nrf2. Nrf2 enters the nucleus and promotes the expression of a range of cytoprotective genes (e.g. SOD, CAT, GSH).

Probiotics therapy

Experimental studies.

Convincing evidence from animal studies indicate that probiotics treatment can relieve UC (Table  1 ). Wu et al. [ 66 ] found that the use of Bifidobacterium longum CCFM1206 to treat Dextran-Sulfate-Sodium (DSS) induced Colitis mice will promotes the conversion of Glucoraphanin (GRP) to sulforaphane (SFN). SFN help to upregulate the Nrf2 signaling pathway and inhibit the NF-κB activity, which can ameliorate DSS-induced colitis. The result also indicated that the intervention of B.longum CCFM1206 could relieve the dysbiosis of intestinal microbiota. That is, promoted the proportion of Alistipes , Bifidobacterium , Blautia and Lachnospiraceae NK4A136 group and inhibited the proportion of Acinetobacter , and Lachnospiraceae A2 in the gut. Similar study, Han et al. [ 67 ] demonstrated that Bifidobacterium infantis enhances genetic stability by maintaining the balance of gut flora to increase anaphase-promoting complex subunit 7 (APC7) expression in colonic tissues, changing gut flora such as an increase in B.infantis . Then reducing DSS-induced colonic inflammation. Consistent with the above results, Fu et al. [ 68 ] found that Bacteroides xylanisolvens AY11-1 regulate the intestinal microbiota through the efficient degradation of alginate, improving the dysbiosis of intestinal ecology and promoting the growth of beneficial bacteria, for example, the increase of Blautia spp and Prevotellaceae UCG-001. Then ameliorated the symptoms of DSS-induced UC in mice. Wang et al. [ 69 ] revealed that the administration of probiotic Companilactobacillus crustorum MN047 in DSS-induced UC mice resulted in the expression of tight junctions, and down-regulation of pro-inflammatory and chemokine expression. It was also found that an increase of goblet cells, MUCs, TFF3, and TJs in the probiotic group, which demonstrated that the treat with CCMN could enhance the gut barrier function. And confirmed by fecal microbiota transplantation (FMT), the mechanisms of CCMN alleviating UC were partly due to its modulation to gut microbiota. The result showed that an increase in Bacteroidaceae and Burkholderiaceae and a decrease in Akkermansiaceae and Eggerthellaceae . Hu et al. [ 70 ] also found that Selenium-enriched Bifidobacterium longum DD98 administration alleviated the symptoms caused by DSS, inhibited the expression of the pro-inflammatory cytokines, decreased the level of oxidative stress, promoted the expression of tight junction proteins, inhibited the activation of toll-like receptor 4 (TLR4), and regulated the gut flora. They found that after the treatment of Se-B. longum DD98, the phylum of Bacteroidetes decreased and the phylum of Firmicutes increased. All of the above can be effective attenuated DSS-induced colitis in mice. In another study, the results of Han et al.’s [ 71 ] study of Lacticaseibacillus rhamnosus Hao9 in DSS-induced UC mice showed that the use of Hao9 attenuated weight loss which is caused by DSS, lowered DAI scores, attenuated colonic damage and inflammatory infiltrates and promoted the growth of Faecalibaculum and Romboutsia in the gut. The researcher attributed the observed effects of Hao9 on UC to its ability to inhibit lipopolysaccharide-induced intestinal IκB activation of mice. Consistent with the above results, Huang et al. [ 72 ] also showed that Lactobacillus paracasei R3 supplementation improved the general symptoms of murine colitis, attenuated inflammatory cell infiltration and more. And it was showed that the imbalance of Treg/Th17 cell in the intestinal inflammation caused by DSS was restored after treatment with L.p R3. Similarly, Xu et al. [ 73 ] investigated the effect of Saccharomyces boulardii and its postbiotics on DSS-induced UC in mice, showing that both S. boulardii elements and its postbiotics could significantly alleviate weight loss, reduce colonic tissue damage, regulate the balance of pro/anti-inflammatory cytokines in serum and colon, promote the expression of colonic tight junction proteins, and regulate the stability of intestinal microecology in mice. Changing in the bacterial flora were characterized by a significant increase in Turcibacter at the genus level, which collectively attenuate DSS-induced colitis. Komaki et al. [ 74 ] administered Lactococcus lactis subsp.lactis JCM5850 to mices with colitis induced by DSS and found that moderate amounts of L. lactis had a mitigating effect on colitis. In keeping with these results, Hizay et al. [ 75 ] also found that Lactobacillus acidophilus reduces abnormally high levels of serotonin in colon tissue in acetic acid-induced UC and relieves inflammation in intestinal tissue. As with the results above, Gao et al. [ 76 ] made Saccharomyces boulardii into suspension, observing its effect on DSS induced colitis in mice. The results suggested that S. boulardii can alleviate the clinical symotoms of colitis in mice exposed to DSS and the histological lesions. And it was found that the mechanism of S. boulardii to treat UC is inhibite nuclear transcription factor kappa B (NF-κB) and activate nuclear factor erythroid 2-related factor 2 (Nrf2) signaling pathway. As demonstrated by He [ 77 ] et al., Enterococcus faecium administration prevented DSS-induced intestinal inflammation and intestinal flora dysbiosis and particially repaired the damage to intestinal mucosal barrier and tight junctions. The modulatory effect on intestinal flora was characterized by an increase in Butyricicoccus sp., Lactobacillus sp., and Bifidobacterium sp. and a decrease in Ochrobactrum sp. and Acinetobacter sp. .

By studying the effects of tetrapeptide from maize (TPM) and probiotic (5 Lactobacillus strains: L.animalis- BA12, L.bulgaricus- LB42, L.paracasei- LC86, L.casei- LC89 and L.plantarum- LP90) in mice with DSS-induced UC, Li et al. [ 78 ] found that it could reduce the level of oxidative stress, attenuate the loss of kidney and colon, and regulate the intestinal flora to alleviate the inflammatory effects of UC. Wherein, the modulation effect to gut microbiota in manifested as an increase in Muribaculaceae, Alistipes, Ligilactobacillus and Lactobacillus . Recently, Shang et al. [ 79 ] reported that Bifidobacterium bifidum H3-R2 can effectively alleviate of pathogenesis by inhibiting inflammatory signaling, maintaining intestinal ecological homeostasis, and protecting colonic integrity. B.bifidum H3-R2 administration similarly affected the composition of gut microbiota, showing that B.bifidum H3-R2 caused a significant increase in the abundance of Bifidobacterium and Lactobacillus and a decrease in Enterobacter , Enterococcus and Streptococcus . Chen et al. [ 80 ] also discovered Lactobacillus fermentum ZS40 could inhibit DSS-induced mice colon shortening, colon damage, and intestinal wall thickening. It does so by inhibiting the activation of NF-κB and MAPK signaling pathways, and ultimately relieved inflammation.

To sum up, these results provide important clues for the design and use of more effective probiotic agents to treat UC and may provide new insights into the mechanisms by which host-microbe interactions confer the protective effect. And probiotics as additionally supplemented active micro-organisms, may have better value in clinical applications as drugs in the future [ 81 ].

Clinical studies

There are many contributing factors to UC, but much evidence suggests a strong link between host gut microbes and the treatment of UC pathogenesis, and suggests that mediation of gut microbes is the key to treating UC. Probiotics have been shown to alleviate UC by altering the composition of the gut microbiota and many other ways. A growing number of clinical trials have also demonstrated the therapeutic effects of probiotics in UC (Table  2 ). As early as 2010, Hegazy et al.’s [ 82 ] study showed that administration of probiotics ( Lactobacillus delbruekii and L. fermentum ) not only decreased the NF-κB DNA binding activity, but also reduced the accumulation of leukocytes, and down-regulated levels of pro-inflammatory factors, and thereby ameliorated the severity of the colitis. Similarly, in order to study the long-term effect of probiotics on UC, Palumbo et al. [ 83 ] conducted a clinical study and the results of the study showed that patients in the probiotics ( L. salivarius , L.acidophilus and B.bifidus strain BGN4) treatment group had better outcomes which is reflected through MMDAI. Thus, the use of probiotics may enhance the anti-inflammatory effect. Similar results, Bjarnason et al. [ 84 ] tried to prove the impact of the probiotic Symprove (including Lactobacillus rhamnosus NCIMB 30,174, Lactobacillus plantarum NCIMB 30,173, Lactobacillus acidophilus NCIMB 30,175 and Enterococcus faecium NCIMB 30,176) which contains four naturally occurring bacterial strain for this experiment. Research showed that Symprove are associated with reduced intestinal inflammation in UC patients. In line with these results, Tsuda et al. [ 85 ] gave patients with moderate to severe UC treated with BIO-THREE (containing Streptococcus faecalisa T-110, Clostridium butyricum TO-A and Bacillus mesentericus TO-A). Researchers found that the treated with BIO-THREE were able to improve clinical and endoscopic examinations in about half of UC patients who were intolerant to conventional therapy. And its intake improved intestinal microflora, the main change may be an increase in bifidobacteria. After a six-week study, Agraib et al. [ 86 ] found that patients in the probiotic (containing nine Lactobacillus and five Bifidobacterium species) group had higher levels of anti-inflammatoty and better clinical symptoms compared with the placebo group. Groeger et al. [ 87 ] demonstrated that Bifidobacterium infantis 35,624 achieved palliate effect to UC primarily by reducing intestinal inflammatory biomarkers (e.g. CRP, TNF-α, IL-6). In 2021, the study conducted by Gu et al. [ 88 ] revealed that Akkermansia muciniphila activate aryl hydrocarbon receptor (AhR) signaling, inhibite Kyn pathway (KP) activation, and restore the down-regulation of anti-inflammatory factors through increasing the levels of indoleacetic acid (IAA) and indole acrylic acid (IA) in the tryptophan (Trp) metabolic pathway. Similarly, the mitigation effect of probiotics (containing L.casei Zhang, L.plantarum P-8 and B.animalis subsp. lactis V9) was demonstrated in a trail by Chen et al. [ 89 ] in the treatment of UC. And the researchers found that the probiotic group had more beneficial bacteria, such as Eubacterium ramulus , Pediococcus pentosaceus , Bacteroides fragilis and Weissella cibaria .

All in all, these clinical studies have shown that the effectiveness of treating UC patients with probiotics is increasingly being proven. Above all, probiotics intervention might be a potentially effective approach in the treatment of UC by restoration of gut microbiota. Meanwhile, therapies that may most efficiently bring the disease under control are still being sought.

Prebiotics therapy

Prebiotics are selectively fermentable, non-digestible oligosaccharides, or ingredients. They function to accelerate beneficial bacterial growth and suppress harmful bacterial growth, thus adjusting the balance of gut microbiota. In addition, they can lead to the production of SCFAs, regulate immune response, control gene expression in bacterial cells, and improve absorption of micronutrients. And prebiotics are used to treat a wide variety of disease, such as obesity [ 90 ], chronic enteritis [ 91 ], skin disease [ 92 ] and autism spectrum disorder [ 93 ]. The therapeutic effects of prebiotics on UC have also been confirmed in animal and humans (Tables  3 and 4 ). Thus, Prebiotics can be used as a novel dietary management approach for UC. Here, the possible effects and mechanisms of prebiotics in the treatment of UC are summarized in Fig.  3 .

The mechanism of prebiotics in alleviating Ulcerative Colitis in Mice. It was found that the mechanism of prebiotics alleviate UC is probably through inhibiting of the TLR4/NF-κB signaling pathway, the JAK2/STAT3 signaling pathway, and regulating the ratio of T cell subsets. Firstly, prebiotics inhibit the activation effect of lipopolysaccharides from Gram-positive bacteria on TLR4 receptors, thereby inhibiting NF-κB from being released into nucleus and thus reducing the transcription of pro-inflammatory factors. Secondly, prebiotics can inhibit the activation of cytokine receptors by IL-6, thus suppress the entry of STAT3 into the nucleus and likewise inhibit its production of pro-inflammatory factors. Thirdly, prebiotics can inhibit of the conversion of naive T cells into Th17 cells and promote of their conversion into Treg cells, causing an increase of the expression of anti-inflammatory. (This mechanism diagram was drawn by Figdraw ( https://www.figdraw.com ))

Convincing evidence from animal studies indicate that prebiotics treatment can relieve UC. Koleva et al. [ 94 ] showed that fructo-oligosaccharides (FOS) promoted Bifidobacterium spp. and inulin and FOS can all decrease Clostridium cluster XI in rats, while Bifidobacterium spp. and Clostridium cluster XI correlated negatively and positively, respectively, to chronic intestinal inflammation. That is, both this two fructans inhibited intestinal inflammation. Hoentjen et al. [ 95 ] also orally administered a prebiotic combination of chicory-derived long-chain inulin-type fructans and short-chain inulin fraction oligofructose to HLA-B27 transgenic rats and found that this prebiotic can significantly reduce colitis and demonstrated that this effect was not only related to the gut microbiota, but also to immunomodulatory effects. They found that the prebiotic can promote the increase of bifidobacteria and endogenous lactobacilli. In immunomodulation, for example, it is possible to increase TGF-β in cecum. Wang et al. [ 96 ] allowed C57BL/6 mice with UC to receive oral administration of stachyose which is a prebiotic that traditionally extracted from plants for a period of time, and demonstrated the effect of stachyose on the recovery of body weight and found that it can reduced colonic tissue damage, lowered the level of pro-inflammatory cytokines, and restored the dysbiosis of the intestinal microbiota imblance (reduce the abundance of Escherichia_Shigella , Parabacteroides , Romboutsia and Turicibacter and raise the abundance of Alistipes and Roseburia ). In the study of Lunken et al. [ 97 ], they used an adoptive T-cell transfer mice model of colitis to examine the effects of enriching exclusive enteral nutrition (EEN) with inulin-type fructans (IN) (ENN IN) on colitis and found that a less deterioration of the mucus layer, increased butyrate production, and the expansion of anti-inflammatory T-cell subsets, including IL-10 producing Foxp3 ＋ Tregs. And they also found an increased relative abundance of beneficial microbes ( Bifidobacterium spp. and Anaerostipes caccae ) and an reduced relative abundance of potentially pathogenic microbes ( Escherichia Shigella spp.). All of these results continue to prove the benefits of prebiotics in UC. Li et al. [ 98 ] established the DSS-induced mice model of colitis by evaluating the therapeutic effects of prebiotics high-substituted hydroxypropyl cellulose (HHPC) and low-substituted hydroxypropyl cellulose (LHPC) on UC, and the results confirmed that these two prebiotics dose-dependently ameliorated the inflammation in colitis mice, inhibited pro-inflammatory cytokine and regulated the balance of intestinal flora, including increased the relative abundance of Bacteroides and Alloprevotella genus and reduced the relative abundance of Firmicutes. Kanauchi et al. [ 99 ] investigated the effect of Germinated barley foodstuff (GBF), a prebiotic product, on the gut environment and found that it can inhibited the expression of STAT3 and NF-κB, thereby reducing the inflammatory response of the epithelium.      

In summary, these animal experiments have showed the good effect of prebiotic therapy alone or in combination to UC. This provides a new direction in the clinical treatment of UC.

Many clinical studies have demonstrated the benefits of prebiotics for people with UC. Oligofructose and Inulin as the oligosaccharide fraction of Raftilose and the oligosaccharide fraction of Raftiline, which was obtained by the extraction of chicory roots, Gibson et al. [ 100 ] have demonstrated the stimulatory effect of these two substances on intestinal bifibacteria, which is a bacterium thought to be beneficial to health through clinical experiment and reduced some pathogenic bacteria that can produce toxins or hydrolyzed proteins, including bacteroides, clostridia, and fusobacteria. Vulevic et al. [ 101 ] found that Galactooligosaccharides (GOSs) promoted the population of beneficial bacteria, especially bifidobacteria and lactobacilli, and reduced numbers of less beneficial bacteria (bacteroides, the C. histolyticum group, E. coli , and Desulfovibrio spp.), and also enhanced the immune response and reduced the production of pro-inflammatory factors. Similarly. Casellas et al. [ 102 ] demonstrated that oligofructose-enriched inulin reduced intestinal inflammation by measuring fecal calprotectin levels in patients. Faghfoori et al. [ 103 ] administrated germinated barley foodstuff (GBF) to patients with UC and showed that GBF were able to reduce serum levels of pro-inflammatory including IL-6, IL-8,TNF-α. As demonstrated by Mitsuyama et al. [ 104 ], by determining the changes of microorganisms in the feces of patients with UC after four weeks of oral administration of GBF, the results proved that prebiotics can increase the concentration of fecal Bifidobacterium and Eubacterium limosum and increase the concentration of colonic butyrate, which is a source of energy for epithelium. And decreased the presence of Bacteroides .

Ryan et al. [ 105 ] conducted in vitro and in vivo experiments and demonstrated the promoting effect of \( {2}^{{\prime }}\) -fucosyllactose ( \( {2}^{{\prime }}\) -FL) which is a prebiotic human milk oligosaccharide on butyric acid producers, including Bifidobacterium , Clostridium cluster XIVa and Roseburia spp. Butyric acid, on the other hand, as a kind of SCFA, can inhibit the inflammatory response. In this study, they also found a significant increase in fecal Faecalibacterium prausnitzii , Anaerotruncus colihominis , and Pseudoflavonifractor species. Consistent with the above results, Suzuki et al. [ 106 ] tested the effectiveness of Bifidogenic growth stimulator (BGS) which is a prebiotic preparation produced by Propionibacterium freudenreichii isolated from Swiss cheese in patients with UC and found that it can selectively stimulated the activation of Bifidobacteria , which not only produced butyrate to nourish colonocytes and inhibited cytokine production and activation of NF-κB pathway, but also improved the balance of the intestinal microflora to maintain intestinal mucosal integrity and prevented intestinal damage. In the clinical study by Li et al. [ 107 ], they demonstrated the potential of Xylo-oligosaccharide (XOS) to alleviate microecological dysbiosis in patients with UC by measuring the effect of XOS on the intestinal flora. They found that XOS promotes the proliferation of Bifidobacteria , which produces a variety of organic acids and inhibits the growth of harmful bacteria by altering their metabolites.

In conclusion, these clinical studies demonstrated the palliative effects of prebiotics on UC, showing that prebiotics hold promise as primary or adjunctive maintenance therapy for UC.

Concluding remarks

UC as a common disease has become a financial burden for many people and has the potential to develop into cancer if not prevented or treated. Therefore, it is important to identify and intervene in a timely manner. The pathogenesis of UC is complex, that’s why it’s important to find a reliable treatment. There is a strong and complex relationship between gut microbiota and gut. Crucially, growing evidence strongly suggests that the gut microbiota plays a pivotal role in intestinal defense function, immune regulatory function, inflammatory responses, as a result, the development and progression of UC. Meanwhile, mechanistic studies have demonstrated these particular species of intestinal commensal bacteria capable of playing either a protective or pathogenic role in UC development. Traditional treatment methods come with a lot of side effects. And probiotics and prebiotics emerge as a new therapeutic modality to modulate the gut microbiota. Based on these, numerous animal and clinical studies have shown that regulating gut microbiota may be an effective strategy to treat UC.

Probiotics being able to confer notable health benefits by modulating the composition of gut microbiota and restoring the physiological bacterial flora. However, while an increasing number of studies have pointed to the therapeutic effects of probiotics on UC, the available data in this field remain limited and the relevant scientific work is still in its early stages. Thus, further research is still necessary. Firstly, due to the complex relationship between gut microbiota and UC, in order to better use probiotics to treat UC, it is necessary to further study the mechanism of intestinal flora affecting the occurrence and development of UC through more animal and clinical experiments. Secondly, we need to know how these probiotics regulate gut microbiota or how they function in the intestinal and what factors contribute to their long-run stability in both health and disease. Changes in certain pathway molecules can be probed to determine the specific mechanism of probiotic treats UC. Meanwhile, in the study of probiotics in the treatment of UC, we should pay more attention to the etiology and pathogenesis. Based on this, the composition and metabolites of probiotics should be of great concern. In particularly, it should be thoroughly studied for their antioxidant effects, anti-inflammatory properties, maintenance of the intestinal homeostasis, regulation of mucosal immune homeostasis, and so on. Some key probiotic components and metabolites may be highly effective postbiotic in the treatment of UC. Thirdly, most medications for the treatment of UC have many adverse effects. Meanwhile, probiotics have great potential as drugs to treat UC. Therefore, it may be more cost-efficient to invest more effort in probiotics than in developing new anti-inflammatory drugs. Fourth, in order to provide more effective probiotics to clinical, we can study the beneficial gut microbiota of healthy humans to dig out more and better probiotics. At the same time, it is necessary to search for the most effective probiotic compositions for the treatment of UC. Fifth, more clinical rationalized trials should be carried out to determine whether probiotics is safe and effective in the treatment of UC. Furthermore, because the composition of the gut microbiota is related to region, ethnicity, and diet, it is necessary to study large samples of people in different regions. Sixth, we must figure out the route of administration of the probiotics as well as the dosage, to ensure the probiotics will maximize the benefits in patient’s body under safe administration. Seventh, in order to make it easier and more convenient for patients to use probiotics, such as how to keep probiotics maintain highly active in some way and make it easier for patients to take, we should further explore the production and preservation of probiotics. Last but not least, to accepted by patients as a reliable treatment, it should be clarified for which patients a particular probiotic is effective, or which is preferable for a single probiotic or a blend of strains. So, there are still many problems it faces. In the future, probiotic therapy may be a potentially useful approach for UC, but research in this area has just started.

Prebiotics offer an exciting new approach to dietary management of gastrointestinal disorders including UC. It has been accepted as a dietary food ingredient that helps to nourish gut microbes, which can improve health and prevent UC. But while many studies to date have demonstrated the beneficial effects of prebiotics in UC, it still faces numerous challenges. Now many studies have the limitation of too small a sample size or lack of a control group, so the evidence for a significant effect of prebiotics is still lacking. The dosage of prebiotics is also a question to be confirmed, if too high a dosage will lead to tolerance, or if a higher dosage of prebiotics will produce better results when well tolerated. With so many types of prebiotics available, it is also deserving of further study as to which prebiotics have better results for which type of UC patients. Although a large number of in vitro and in vivo experiments have confirmed the positive effects of prebiotics, there is still a need for more clinical trials or animal experiments to further evaluate their specific effects.The specific mechanism by which we found that prebiotics alleviate UC remains unclear. It’s worth exploring further. In-depth experiments are needed to further elucidate the role of prebiotics in patients with UC, whether it is their own structure or their metabolites that play a role. And to meet the needs of consumers, new strategies for cost-effective and efficient prebiotics can be developed. Prebiotics, as a food-sourced ingredient for the treatment of UC, offer a new clinical direction, and it is important to study its good effects and side effects as clearly as possible. Therefore, in any case, the prospect of the application of prebiotics in UC is worthy of attention and expectation.

Certainly, in order to gain wider acceptance and recognition for probiotics and prebiotics to treat UC, further research is urgently required.

Data availability

The data that support the findings of this study are available from the corresponding author, upon reasonable request.

Hijova E, Soltesova A. Effects of probiotics and prebiotics in ulcerative colitis. Bratislava Med J. 2013;114(09):540–3.

Article   CAS   Google Scholar  

Ng SC, Shi HY, Hamidi N, Underwood FE, Tang W, Benchimol EI, et al. The Worldwide Incidence and Prevalence of Inflammatory Bowel Disease in the 21 St Century: a systematic review of Population-Based studies. Gastroenterology. 2017;152(5):S970–1.

Article   Google Scholar  

Iheozor-Ejiofor Z, Kaur L, Gordon M, Baines PA, Sinopoulou V, Akobeng AK. Probiotics for maintenance of remission in ulcerative colitis. Cochrane Database of Systematic Reviews. 2020;3(3):CD007443.

Mladenova D, Kohonen-Corish MRJ. Mouse models of inflammatory bowel disease - insights into the mechanisms of inflammation-associated Colorectal Cancer. Vivo. 2012;26(4):627–46.

CAS   Google Scholar  

Li J, Ueno A, Fort Gasia M, Luider J, Wang T, Hirota C, et al. Profiles of Lamina Propria T Helper Cell subsets discriminate between Ulcerative Colitis and Crohnʼs Disease. Inflamm Bowel Dis. 2016;22(8):1779–92.

Article   PubMed   Google Scholar  

Shen Z-H, Zhu C-X, Quan Y-S, Yang Z-Y, Wu S, Luo W-W, et al. Relationship between intestinal microbiota and ulcerative colitis: mechanisms and clinical application of probiotics and fecal microbiota transplantation. World J Gastroenterol. 2018;24(1):5–14.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Yao SY, Zhao ZX, Wang WJ, Liu XL. Bifidobacterium Longum: Protection against Inflammatory Bowel Disease. Journal of Immunology Research. 2021:8030297.

Zou JF, Shen YM, Chen MJ, Zhang ZM, Xiao SW, Liu C, et al. Lizhong decoction ameliorates ulcerative colitis in mice via modulating gut microbiota and its metabolites. Appl Microbiol Biotechnol. 2020;104(13):5999–6012.

Article   CAS   PubMed   Google Scholar  

Rather IA, Majumder R, Alshammari FH, Park JG, Bajpai VK. Ulcerative colitis and probiotics: an overview. Pak J Pharm Sci. 2016;29(5):1877–80.

CAS   PubMed   Google Scholar  

Peyrin-Biroulet L, Sandborn W, Sands BE, Reinisch W, Bemelman W, Bryant RV, et al. Selecting therapeutic targets in inflammatory bowel disease (STRIDE): determining therapeutic goals for treat-to-target. Am J Gastroenterol. 2015;110(9):1324–38.

Kushkevych I, Martinkova K, Vitezova M, Rittmann SKR. Intestinal microbiota and perspectives of the Use of Meta-Analysis for comparison of Ulcerative Colitis studies. J Clin Med. 2021;10(3):462.

Jostins L, Ripke S, Weersma RK, Duerr RH, McGovern DP, Hui KY, et al. Host-microbe interactions have shaped the genetic architecture of inflammatory bowel disease. Nature. 2012;491(7422):119–24.

Barrett JC, Lee JC, Lees CW, Prescott NJ, Anderson CA, Phillips A, et al. Genome-wide association study of ulcerative colitis identifies three new susceptibility loci, including the < i > HNF4A region. Nat Genet. 2009;41(12):1330–U99.

Ordas I, Eckmann L, Talamini M, Baumgart DC, Sandborn WJ. Ulcerative colitis. Lancet. 2012;380(9853):1606–19.

Sands BE, Kaplan GG. therapeutic review the role of tnfα α in ulcerative colitis.2007;47(8):930-41.

Zhang FX, Kirschning CJ, Mancinelli R, Xu XP, Jin Y, Faure E, et al. Bacterial lipopolysaccharide activates nuclear factor-kappab through interleukin-1 signaling mediators in cultured human dermal endothelial cells and mononuclear phagocytes. J Biol Chem. 1999;274(12):7611–4.

Selin KA, Hedin CRH, Villablanca EJ. Immunological networks defining the heterogeneity of Inflammatory Bowel diseases. J Crohns Colitis. 2021;15(11):1959–73.

Article   PubMed   PubMed Central   Google Scholar  

Heller F, Florian P, Bojarski C, Richter J, Christ M, Hillenbrand B, et al. Interleukin-13 is the Key Effector Th2 Cytokine in Ulcerative Colitis that affects epithelial tight junctions, apoptosis, and cell restitution. Gastroenterology. 2005;129(2):550–64.

Zhu QG, Rui K, Wang SJ, Tian J. Advances of Regulatory B cells in Autoimmune diseases. Front Immunol. 2021;12:12:592914.

Dave M, Higgins PD, Middha S, Rioux KP. The human gut microbioma current knowledge, challenges, and future directions. Translational Res. 2012;160(4):246–57.

He XL, Gao J, Peng L, Hu TT, Wan Y, Zhou MJ, et al. Bacterial O-GlcNAcase genes abundance decreases in ulcerative colitis patients and its administration ameliorates colitis in mice. Gut. 2021;70(10):1872–83.

Li BL, Du PL, Du Y, Zhao DY, Cai YR, Yang Q et al. Luteolin alleviates inflammation and modulates gut microbiota in ulcerative colitis rats. Life Sci. 2021;269:269:119008.

Guo XY, Liu XJ, Hao JY. Gut microbiota in ulcerative colitis: insights on pathogenesis and treatment. J Dig Dis. 2020;21(3):147–59.

Mizoguchi E, Tong M, Li X, Wegener Parfrey L, Roth B, Ippoliti A et al. A Modular Organization of the Human Intestinal Mucosal Microbiota and its Association with Inflammatory Bowel Disease. PLoS ONE. 2013;8(11):e80702.

Andoh A, Ida S, Tsujikawa T, Benno Y, Fujiyama Y. Terminal restriction fragment polymorphism analyses of fecal microbiota in five siblings including two with ulcerative colitis. Clin J Gastroenterol. 2009;2(5):343–5.

Fuentes S, Rossen NG, van der Spek MJ, Hartman JHA, Huuskonen L, Korpela K, et al. Microbial shifts and signatures of long-term remission in ulcerative colitis after faecal microbiota transplantation. ISME J. 2017;11(8):1877–89.

Kotlowski R, Bernstein CN, Sepehri S, Krause DO. High prevalence of Escherichia coli belonging to the B2 + D phylogenetic group in inflammatory bowel disease. Gut. 2007;56(5):669–75.

Xu J, Chen N, Wu Z, Song Y, Zhang Y, Wu N et al. 5-Aminosalicylic acid alters the gut bacterial microbiota in patients with Ulcerative Colitis. Front Microbiol. 2018;9:1274.

Schwiertz A, Jacobi M, Frick J-S, Richter M, Rusch K, Köhler H. Microbiota in Pediatric Inflammatory Bowel Disease. J Pediatr. 2010;157(2):240–e41.

Khalil NA, Walton GE, Gibson GR, Tuohy KM, Andrews SC. In vitrobatch cultures of gut microbiota from healthy and ulcerative colitis (UC) subjects suggest that sulphate-reducing bacteria levels are raised in UC and by a protein-rich diet. Int J Food Sci Nutr. 2013;65(1):79–88.

Verma R, Verma AK, Ahuja V, Paul J. Real-Time Analysis of Mucosal Flora in patients with inflammatory bowel disease in India. J Clin Microbiol. 2010;48(11):4279–82.

Walker AW, Sanderson JD, Churcher C, Parkes GC, Hudspith BN, Rayment N et al. High-throughput clone library analysis of the mucosa-associated microbiota reveals dysbiosis and differences between inflamed and non-inflamed regions of the intestine in inflammatory bowel disease. BMC Microbiol. 2011;11:7.

Tahara T, Shibata T, Kawamura T, Okubo M, Ichikawa Y, Sumi K, et al. Fusobacterium detected in Colonic Biopsy and Clinicopathological features of Ulcerative Colitis in Japan. Dig Dis Sci. 2014;60(1):205–10.

Machiels K, Joossens M, Sabino J, De Preter V, Arijs I, Eeckhaut V, et al. A decrease of the butyrate-producing speciesRoseburia hominisandFaecalibacterium prausnitziidefines dysbiosis in patients with ulcerative colitis. Gut. 2014;63(8):1275–83.

Lepage P, Häsler R, Spehlmann ME, Rehman A, Zvirbliene A, Begun A, et al. Twin study indicates loss of Interaction between Microbiota and Mucosa of patients with Ulcerative Colitis. Gastroenterology. 2011;141(1):227–36.

Fite A, Macfarlane S, Furrie E, Bahrami B, Cummings JH, Steinke DT, et al. Longitudinal analyses of gut mucosal microbiotas in Ulcerative Colitis in Relation to Patient Age and Disease Severity and Duration. J Clin Microbiol. 2013;51(3):849–56.

Ott SJ, Plamondon S, Hart A, Begun A, Rehman A, Kamm MA, et al. Dynamics of the Mucosa-Associated Flora in Ulcerative Colitis patients during remission and clinical relapse. J Clin Microbiol. 2008;46(10):3510–3.

Hu Y, Chen Z, Xu C, Kan S, Chen D. Disturbances of the gut microbiota and microbiota-derived metabolites in Inflammatory Bowel Disease. Nutrients. 2022;14(23):5140.

Andreopoulou M, Tsiouris V, Georgopoulou I. Effects of organic acids on the gut ecosystem and on the performance of broiler chickens. J Hellenic Veterinary Med Soc. 2017;65(4):289.

Agus A, Planchais J, Sokol H. Gut microbiota regulation of Tryptophan Metabolism in Health and Disease. Cell Host Microbe. 2018;23(6):716–24.

Singh R, Chandrashekharappa S, Bodduluri SR, Baby BV, Hegde B, Kotla NG et al. Enhancement of the gut barrier integrity by a microbial metabolite through the Nrf2 pathway. Nat Commun. 2019;10(1):89.

Rothhammer V, Mascanfroni ID, Bunse L, Takenaka MC, Kenison JE, Mayo L, et al. Type I interferons and microbial metabolites of tryptophan modulate astrocyte activity and central nervous system inflammation via the aryl hydrocarbon receptor. Nat Med. 2016;22(6):586–97.

Q W, RM M, BA C, M C-D, KJ Z. D G, - A bacterial carbohydrate links innate and adaptive responses through Toll-like. D– 2985109r. (– 0022-1007 (Print)):- 2853-63.

A I-V CDC, K JPAA. K, L A, - FXR and TGR5 Agonists Ameliorate Liver Injury, Steatosis, and Inflammation After. D– 101695860. (– 2471-254X (Electronic)):- 1379-91.

Agus A, Clément K, Sokol H. Gut microbiota-derived metabolites as central regulators in metabolic disorders. Gut: J Br Soc Gastroenterol. 2021;70(6):1174–82.

Wahlstrom A, Kovatcheva-Datchary P, Stahlman M, Backhed F, Marschall H-U. Crosstalk between bile acids and gut microbiota and its impact on farnesoid receptor signalling. Dig Dis. 2017;35(3):246–50.

Figliuolo VR, Coutinho-Silva R, Lara Melo Coutinho CM. Contribution of sulfate-reducing bacteria to homeostasis disruption during intestinal inflammation. Life Sci. 2018;215:145–51.

Cai Z, Wang S, Li J. Treatment of inflammatory bowel disease: a Comprehensive Review. Front Med (Lausanne). 2021;8:765474.

Tremblay L, Pineton de Chambrun G, De Vroey B, Lavogiez C, Delaporte E, Colombel JF, et al. Stevens-Johnson syndrome with sulfasalazine treatment: report of two cases. J Crohns Colitis. 2011;5(5):457–60.

Hayashi R, Wada H, Ito K, Adcock IM. Effects of glucocorticoids on gene transcription. Eur J Pharmacol. 2004;500(1–3):51–62.

Lichtenstein GR, Abreu MT, Cohen R, Tremaine W, American Gastroenterological A. American Gastroenterological Association Institute technical review on corticosteroids, immunomodulators, and infliximab in inflammatory bowel disease. Gastroenterology. 2006;130(3):940–87.

Curkovic I, Egbring M, Kullak-Ublick GA. Risks of inflammatory bowel disease treatment with glucocorticosteroids and aminosalicylates. Dig Dis. 2013;31(3–4):368–73.

Chhaya V, Saxena S, Cecil E, Subramanian V, Curcin V, Majeed A, et al. Steroid dependency and trends in prescribing for inflammatory bowel disease - a 20-year national population-based study. Aliment Pharmacol Ther. 2016;44(5):482–94.

D’Haens G. Systematic review: second-generation vs. conventional corticosteroids for induction of remission in ulcerative colitis. Aliment Pharmacol Ther. 2016;44(10):1018–29.

Jharap B, Seinen ML, de Boer NK, van Ginkel JR, Linskens RK, Kneppelhout JC, et al. Thiopurine therapy in inflammatory bowel disease patients: analyses of two 8-year intercept cohorts. Inflamm Bowel Dis. 2010;16(9):1541–9.

Herrlinger KR, Barthel DN, Schmidt KJ, Buning J, Barthel CS, Wehkamp J, et al. Infliximab as rescue medication for patients with severe ulcerative/indeterminate colitis refractory to tacrolimus. Aliment Pharmacol Ther. 2010;31(9):1036–41.

Moura FA, de Andrade KQ, Dos Santos JCF, Araujo ORP, Goulart MOF. Antioxidant therapy for treatment of inflammatory bowel disease: does it work? Redox Biol. 2015;6:617–39.

Hui L, Wei-Hsien L, Hanying Z, Haotian F, Wen Z, Wei-Lian H et al. Bifidobacterium lactis BL-99 protects mice with osteoporosis caused by colitis via gut inflammation and gut microbiota regulation. Food & Function. 2022;13(3):1482-1494.

Aghamohammad S, Sepehr A, Miri ST, Najafi S, Pourshafie MR, Rohani M. The role of combining Probiotics in Preventing and Controlling inflammation: a focus on the anti-inflammatory and Immunomodulatory effects of Probiotics in an in vitro model of IBD. Can J Gastroenterol Hepatol. 2022:2045572.

White R, Atherly T, Guard B, Rossi G, Wang C, Mosher C, et al. Randomized, controlled trial evaluating the effect of multi-strain probiotic on the mucosal microbiota in canine idiopathic inflammatory bowel disease. Gut Microbes. 2017;8(5):451–66.

Kang Y, Cai Y. The development of probiotics therapy to obesity: a therapy that has gained considerable momentum. Hormones. 2018;17(2):141–51.

Kang Y, Cai Y, Yang Y. The gut Microbiome and Hepatocellular Carcinoma: implications for early diagnostic biomarkers and Novel therapies. Liver Cancer. 2022;11(2):113–25.

Li LP, Kang YB. The gut Microbiome and Autoimmune Hepatitis: implications for early diagnostic biomarkers and Novel therapies. Molecular Nutrition & Food Research; 2023;67(24):e2300043.

Cai Y, Kang YB. Gut microbiota and metabolites in diabetic retinopathy: insights into pathogenesis for novel therapeutic strategies. Biomed Pharmacother. 2023;164:114994.

Liu HX, Kang X, Yang XD, Yang H, Kuang XY, Ren P, et al. Compound Probiotic ameliorates Acute Alcoholic Liver Disease in mice by modulating gut microbiota and maintaining intestinal barrier. Probiotics Antimicrob Proteins. 2023;15(1):185–201.

Wu J, Cui S, Tang X, Zhang Q, Jin Y, Zhao J, et al. Bifidobacterium longum CCFM1206 promotes the Biotransformation of Glucoraphanin to Sulforaphane that contributes to amelioration of Dextran-Sulfate-Sodium-Induced Colitis in mice. J Agric Food Chem. 2023;71(2):1100–12.

Han T, Hu X, Li K, Zhang D, Zhang Y, Li J. Bifidobacterium infantis maintains Genome Stability in Ulcerative Colitis via regulating anaphase-promoting Complex Subunit 7. Front Microbiol. 2021;12:761113.

Fu T, Wang Y, Ma M, Dai W, Pan L, Shang Q et al. Isolation of Alginate-degrading Bacteria from the human gut microbiota and Discovery of Bacteroides xylanisolvens AY11-1 as a Novel Anti-colitis Probiotic Bacterium. Nutrients. 2023;15(6):1352.

Wang T, Shi C, Wang S, Zhang Y, Wang S, Ismael M, et al. Protective effects of Companilactobacillus Crustorum MN047 against Dextran Sulfate Sodium-Induced Ulcerative Colitis: a fecal microbiota transplantation study. J Agric Food Chem. 2022;70(5):1547–61.

Hu Y, Jin X, Gao F, Lin T, Zhu H, Hou X et al. Selenium-enriched Bifidobacterium longum DD98 effectively ameliorates dextran sulfate sodium-induced ulcerative colitis in mice. Front Microbiol. 2022;13:955112.

Han M, Liao W, Si X, Bai C, Gai Z. Protective effects of lacticaseibacillus rhamnosus Hao9 on dextran sulphate sodium-induced ulcerative colitis in mice. J Appl Microbiol. 2022;133(3):2039–49.

Huang J, Yang Z, Li Y, Chai X, Liang Y, Lin B et al. Lactobacillus paracasei R3 protects against dextran sulfate sodium (DSS)-induced colitis in mice via regulating Th17/Treg cell balance. J Translational Med. 2021;19(1):356.

Xu X, Wu J, Jin Y, Huang K, Zhang Y, Liang Z. Both Saccharomyces boulardii and its postbiotics alleviate Dextran Sulfate Sodium-Induced Colitis in mice, Association with modulating inflammation and intestinal microbiota. Nutrients. 2023;15(6):1484.

Komaki S, Haque A, Miyazaki H, Matsumoto T, Nakamura S. Unexpected effect of probiotics by Lactococcus lactis subsp. lactis against colitis induced by dextran sulfate sodium in mice. J Infect Chemother. 2020;26(6):549–53.

Hizay A, Dag K, Oz N, Comak-Gocer EM, Ozbey-Unlu O, Ucak M et al. Lactobacillus acidophilus regulates abnormal serotonin availability in experimental ulcerative colitis. Anaerobe. 2023;80:102710.

Gao H, Li Y, Sun J, Xu H, Wang M, Zuo X, et al. Saccharomyces Boulardii ameliorates Dextran Sulfate Sodium-Induced Ulcerative Colitis in mice by regulating NF-κB and Nrf2 signaling pathways. Oxidative Med Cell Longev. 2021;2021:1–14.

Google Scholar  

He W, Ni W, Zhao J, M’Koma A. Enterococcus faecium alleviates Gut Barrier Injury in C57BL/6 mice with Dextran Sulfate Sodium-Induced Ulcerative Colitis. Gastroenterol Res Pract. 2021;2021:1–9.

Li ZG, Zhang S, Xu L, Fang XX, Wan YZ, Yu DH, et al. A tetrapeptide from maize combined with probiotics exerted strong anti-inflammatory effects and modulated gut microbiota in DSS-induced colitis mice. Food Funct. 2022;13(24):12602–18.

Shang JC, Yang S, Tang ZX, Chen YH, Duan BF, Meng XC. Bifidobacterium bifidum H3-R2 and its Molecular Communication within the context of Ulcerative Colitis. J Agric Food Chem. 2022;70(37):11678–88.

Chen ZX, Yi L, Pan YN, Long XY, Mu JF, Yi RK et al. Lactobacillus fermentum ZS40 ameliorates inflammation in mice with Ulcerative Colitis Induced by Dextran Sulfate Sodium. Front Pharmacol. 2021;12:700217.

Ren Z, Hong Y, Huo Y, Peng L, Lv H, Chen J et al. Prospects of probiotic adjuvant drugs in clinical treatment. Nutrients. 2022;14(22):4723.

Hegazy SK. Effect of probiotics on pro-inflammatory cytokines and NF-κB activation in ulcerative colitis. World J Gastroenterol. 2010;16(33):4145-51.

Palumbo VD, Romeo M, Gammazza AM, Carini F, Damiani P, Damiano G, et al. The long-term effects of probiotics in the therapy of ulcerative colitis: a clinical study. Biomedical Papers-Olomouc. 2016;160(3):372–7.

Bjarnason I, Sission G, Hayee B. A randomised, double-blind, placebo-controlled trial of a multi-strain probiotic in patients with asymptomatic ulcerative colitis and Crohn’s disease. Inflammopharmacology. 2019;27(3):465–73.

Tsuda Y, Yoshimatsu Y, Aoki H, Nakamura K, Irie M, Fukuda K, et al. Clinical effectiveness of probiotics therapy (BIO-THREE) in patients with ulcerative colitis refractory to conventional therapy. Scand J Gastroenterol. 2009;42(11):1306–11.

Agraib LM, Yamani MI, Tayyem R, Abu-Sneineh AT, Rayyan YM. Probiotic supplementation induces remission and changes in the immunoglobulins and inflammatory response in active ulcerative colitis patients: a pilot, randomized, double-blind, placebo-controlled study. Clin Nutr ESPEN. 2022;51:83–91.

Groeger D, O’Mahony L, Murphy EF, Bourke JF, Dinan TG, Kiely B, et al. Bifidobacterium infantis35624 modulates host inflammatory processes beyond the gut. Gut Microbes. 2014;4(4):325–39.

Gu ZY, Pei WL, Shen YH, Wang LJ, Zhu J, Zhang Y, et al. Akkermansia muciniphila and its outer protein Amuc_1100 regulates tryptophan metabolism in colitis. Food Funct. 2021;12(20):10184–95.

Chen P, Xu HY, Tang H, Zhao FY, Yang CC, Kwok LY, et al. Modulation of gut mucosal microbiota as a mechanism of probiotics-based adjunctive therapy for ulcerative colitis. Microb Biotechnol. 2020;13(6):2032–43.

Carnahan S, Balzer A, Panchal SK, Brown L. Prebiotics in obesity. Panminerva Med. 2014;56(2):165–75.

Looijer-van Langen MA, Dieleman LA. Prebiotics in chronic intestinal inflammation. Inflamm Bowel Dis. 2009;15(3):454–62.

Baquerizo Nole KL, Yim E, Keri JE. Probiotics and prebiotics in dermatology. J Am Acad Dermatol. 2014;71(4):814–21.

Sanlier N, Kocabas S. The effect of probiotic, prebiotic and gut microbiota on ASD: a review and future perspectives. Crit Rev Food Sci Nutr. 2023;63(15):2319–30.

Koleva PT, Valcheva RS, Sun X, Ganzle MG, Dieleman LA. Inulin and fructo-oligosaccharides have divergent effects on colitis and commensal microbiota in HLA-B27 transgenic rats. Br J Nutr. 2012;108(9):1633–43.

Hoentjen F, Welling GW, Harmsen HJM, Zhang X, Snart J, Tannock GW, et al. Reduction of colitis by prebiotics in HLA-B27 transgenic rats is associated with microflora changes and immunomodulation. Inflamm Bowel Dis. 2005;11(11):977–85.

Wang C, Bai J, Wang B, Yu L, Tian F, Zhao J, et al. Stachyose modulates gut microbiota and alleviates DSS-induced ulcerative colitis in mice. Food Sci Hum Wellness. 2023;12(6):2211–20.

Lunken GR, Tsai K, Schick A, Lisko DJ, Cook L, Vallance BA, et al. Prebiotic enriched exclusive Enteral Nutrition suppresses colitis via gut microbiome modulation and expansion of anti-inflammatory T cells in a mouse model of colitis. Cell Mol Gastroenterol Hepatol. 2021;12(4):1251–66.

Li X, Lv H, Shi F, Song J, Zhang Z. The potential therapeutic effects of hydroxypropyl cellulose on acute murine colitis induced by DSS. Carbohydr Polym. 2022;289:119430.

Kanauchi O, Serizawa I, Araki Y, Suzuki A, Andoh A, Fujiyama Y, et al. Germinated barley foodstuff, a prebiotic product, ameliorates inflammation of colitis through modulation of the enteric environment. J Gastroenterol. 2003;38(2):134.

Gibson GR, Beatty ER, Wang X, Cummings JH, Gibson GR, Beatty ER, Wang X, Cummings JH. Selective stimulation of Bifidobacteria in the human colon by oligofructose and inulin. Gastroenterology. 1995;108(4):975– 82.

Vulevic J, Drakoularakou A, Yaqoob P, Tzortzis G, Gibson GR. Modulation of the fecal microflora profile and immune function by a novel trans-galactooligosaccharide mixture (B-GOS) in healthy elderly volunteers. Am J Clin Nutr. 2008;88(5):1438–46.

Casellas F, Borruel N, Torrejon A, Varela E, Antolin M, Guarner F, et al. Oral oligofructose-enriched inulin supplementation in acute ulcerative colitis is well tolerated and associated with lowered faecal calprotectin. Aliment Pharmacol Ther. 2007;25(9):1061–7.

Faghfoori Z, Navai L, Shakerhosseini R, Somi MH, Nikniaz Z, Norouzi MF. Effects of an oral supplementation of germinated barley foodstuff on serum tumour necrosis factor-alpha, interleukin-6 and– 8 in patients with ulcerative colitis. Ann Clin Biochem. 2011;48(Pt 3):233–7.

Mitsuyama K, Saiki T, Kanauchi O, Iwanaga T, Sata M. Treatment of ulcerative colitis with germinated barley foodstuff feeding: a pilot study. Aliment Pharmacol Ther. 1998;12(12):1225–30.

Ryan JJ, Monteagudo-Mera A, Contractor N, Gibson GR. Impact of 2’-Fucosyllactose on Gut Microbiota Composition in Adults with Chronic Gastrointestinal Conditions: Batch Culture Fermentation Model and Pilot Clinical Trial Findings. Nutrients. 2021;13(3):938.

Suzuki A, Mitsuyama K, Koga H, Tomiyasu N, Masuda J, Takaki K, et al. Bifidogenic growth stimulator for the treatment of active ulcerative colitis: a pilot study. Nutrition. 2006;22(1):76–81.

Li Z, Li Z, Zhu L, Dai N, Sun G, Peng L, et al. Effects of Xylo-Oligosaccharide on the gut microbiota of patients with Ulcerative Colitis in Clinical Remission. Front Nutr. 2021;8:778542.

Download references

Acknowledgements

Not applicable.

This work was supported by Shanxi Province Natural Science Foundation (Grant No. 202203021221182), Science Research Start-up Fund for Doctor of Shanxi Medical University (Grant No. XD1807), Science Research Start-up Fund for Doctor of Shanxi Province (Grant No.SD1807), Scientific and Technological Innovation Programs of Higher Education Institutions in Shanxi (Grant No. 2019L0425), and Shanxi Province Science Foundation for Youths (Grant No. 201901D211314).

Author information

Authors and affiliations.

Department of microbiology and immunology, School of Basic Medical Sciences, Shanxi Medical University, Taiyuan, Shanxi, China

Jing Guo, Liping Li & Yongbo Kang

Faculty of Life science and Technology, Kunming University of Science and Technology, Kunming, Yunnan, China

You can also search for this author in PubMed   Google Scholar

Contributions

Jing Guo wrote the main manuscript text with the help of Liping Li, Xinwei Huang and Yongbo Kang. All authors reviewed the manuscript.

Corresponding author

Correspondence to Yongbo Kang .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Guo, J., Li, L., Cai, Y. et al. The development of probiotics and prebiotics therapy to ulcerative colitis: a therapy that has gained considerable momentum. Cell Commun Signal 22 , 268 (2024). https://doi.org/10.1186/s12964-024-01611-z

Download citation

Received : 10 October 2023

Accepted : 10 April 2024

Published : 14 May 2024

DOI : https://doi.org/10.1186/s12964-024-01611-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Ulcerative colitis
• Gut microflora
• Inflammation

Cell Communication and Signaling

ISSN: 1478-811X

• General enquiries: [email protected]