example of synthesis of related literature and studies

• University of Oregon Libraries
• Research Guides

How to Write a Literature Review

• 6. Synthesize
• Literature Reviews: A Recap
• Reading Journal Articles
• Does it Describe a Literature Review?
• 1. Identify the Question
• 2. Review Discipline Styles
• Searching Article Databases
• Finding Full-Text of an Article
• Citation Chaining
• When to Stop Searching
• 4. Manage Your References
• 5. Critically Analyze and Evaluate

Synthesis Visualization

Synthesis matrix example.

• 7. Write a Literature Review

• Synthesis Worksheet

About Synthesis

Approaches to synthesis.

You can sort the literature in various ways, for example:

How to Begin?

Read your sources carefully and find the main idea(s) of each source

Look for similarities in your sources – which sources are talking about the same main ideas? (for example, sources that discuss the historical background on your topic)

Use the worksheet (above) or synthesis matrix (below) to get organized

This work can be messy. Don't worry if you have to go through a few iterations of the worksheet or matrix as you work on your lit review!

Four Examples of Student Writing

In the four examples below, only ONE shows a good example of synthesis: the fourth column, or  Student D . For a web accessible version, click the link below the image.

Long description of "Four Examples of Student Writing" for web accessibility

• Download a copy of the "Four Examples of Student Writing" chart

Click on the example to view the pdf.

From Jennifer Lim

• << Previous: 5. Critically Analyze and Evaluate
• Next: 7. Write a Literature Review >>
• Last Updated: Jan 10, 2024 4:46 PM
• URL: https://researchguides.uoregon.edu/litreview

Contact Us Library Accessibility UO Libraries Privacy Notices and Procedures

1501 Kincaid Street Eugene, OR 97403 P: 541-346-3053 F: 541-346-3485

• Visit us on Facebook
• Visit us on Twitter
• Visit us on Youtube
• Visit us on Instagram
• Report a Concern
• Nondiscrimination and Title IX
• Accessibility
• Privacy Policy
• Find People

Literature Syntheis 101

How To Synthesise The Existing Research (With Examples)

By: Derek Jansen (MBA) | Expert Reviewer: Eunice Rautenbach (DTech) | August 2023

One of the most common mistakes that students make when writing a literature review is that they err on the side of describing the existing literature rather than providing a critical synthesis of it. In this post, we’ll unpack what exactly synthesis means and show you how to craft a strong literature synthesis using practical examples.

This post is based on our popular online course, Literature Review Bootcamp . In the course, we walk you through the full process of developing a literature review, step by step. If it’s your first time writing a literature review, you definitely want to use this link to get 50% off the course (limited-time offer).

Overview: Literature Synthesis

• What exactly does “synthesis” mean?
• Aspect 1: Agreement
• Aspect 2: Disagreement
• Aspect 3: Key theories
• Aspect 4: Contexts
• Aspect 5: Methodologies
• Bringing it all together

What does “synthesis” actually mean?

As a starting point, let’s quickly define what exactly we mean when we use the term “synthesis” within the context of a literature review.

Simply put, literature synthesis means going beyond just describing what everyone has said and found. Instead, synthesis is about bringing together all the information from various sources to present a cohesive assessment of the current state of knowledge in relation to your study’s research aims and questions .

Put another way, a good synthesis tells the reader exactly where the current research is “at” in terms of the topic you’re interested in – specifically, what’s known , what’s not , and where there’s a need for more research .

So, how do you go about doing this?

Well, there’s no “one right way” when it comes to literature synthesis, but we’ve found that it’s particularly useful to ask yourself five key questions when you’re working on your literature review. Having done so,  you can then address them more articulately within your actual write up. So, let’s take a look at each of these questions.

1. Points Of Agreement

The first question that you need to ask yourself is: “Overall, what things seem to be agreed upon by the vast majority of the literature?”

For example, if your research aim is to identify which factors contribute toward job satisfaction, you’ll need to identify which factors are broadly agreed upon and “settled” within the literature. Naturally, there may at times be some lone contrarian that has a radical viewpoint , but, provided that the vast majority of researchers are in agreement, you can put these random outliers to the side. That is, of course, unless your research aims to explore a contrarian viewpoint and there’s a clear justification for doing so. 

Identifying what’s broadly agreed upon is an essential starting point for synthesising the literature, because you generally don’t want (or need) to reinvent the wheel or run down a road investigating something that is already well established . So, addressing this question first lays a foundation of “settled” knowledge.

Need a helping hand?

2. Points Of Disagreement

Related to the previous point, but on the other end of the spectrum, is the equally important question: “Where do the disagreements lie?” .

In other words, which things are not well agreed upon by current researchers? It’s important to clarify here that by disagreement, we don’t mean that researchers are (necessarily) fighting over it – just that there are relatively mixed findings within the empirical research , with no firm consensus amongst researchers.

This is a really important question to address as these “disagreements” will often set the stage for the research gap(s). In other words, they provide clues regarding potential opportunities for further research, which your study can then (hopefully) contribute toward filling. If you’re not familiar with the concept of a research gap, be sure to check out our explainer video covering exactly that .

3. Key Theories

The next question you need to ask yourself is: “Which key theories seem to be coming up repeatedly?” .

Within most research spaces, you’ll find that you keep running into a handful of key theories that are referred to over and over again. Apart from identifying these theories, you’ll also need to think about how they’re connected to each other. Specifically, you need to ask yourself:

• Are they all covering the same ground or do they have different focal points  or underlying assumptions ?
• Do some of them feed into each other and if so, is there an opportunity to integrate them into a more cohesive theory?
• Do some of them pull in different directions ? If so, why might this be?
• Do all of the theories define the key concepts and variables in the same way, or is there some disconnect? If so, what’s the impact of this ?

Simply put, you’ll need to pay careful attention to the key theories in your research area, as they will need to feature within your theoretical framework , which will form a critical component within your final literature review. This will set the foundation for your entire study, so it’s essential that you be critical in this area of your literature synthesis.

If this sounds a bit fluffy, don’t worry. We deep dive into the theoretical framework (as well as the conceptual framework) and look at practical examples in Literature Review Bootcamp . If you’d like to learn more, take advantage of our limited-time offer to get 60% off the standard price.

4. Contexts

The next question that you need to address in your literature synthesis is an important one, and that is: “Which contexts have (and have not) been covered by the existing research?” .

For example, sticking with our earlier hypothetical topic (factors that impact job satisfaction), you may find that most of the research has focused on white-collar , management-level staff within a primarily Western context, but little has been done on blue-collar workers in an Eastern context. Given the significant socio-cultural differences between these two groups, this is an important observation, as it could present a contextual research gap .

In practical terms, this means that you’ll need to carefully assess the context of each piece of literature that you’re engaging with, especially the empirical research (i.e., studies that have collected and analysed real-world data). Ideally, you should keep notes regarding the context of each study in some sort of catalogue or sheet, so that you can easily make sense of this before you start the writing phase. If you’d like, our free literature catalogue worksheet is a great tool for this task.

5. Methodological Approaches

Last but certainly not least, you need to ask yourself the question: “What types of research methodologies have (and haven’t) been used?”

For example, you might find that most studies have approached the topic using qualitative methods such as interviews and thematic analysis. Alternatively, you might find that most studies have used quantitative methods such as online surveys and statistical analysis.

But why does this matter?

Well, it can run in one of two potential directions . If you find that the vast majority of studies use a specific methodological approach, this could provide you with a firm foundation on which to base your own study’s methodology . In other words, you can use the methodologies of similar studies to inform (and justify) your own study’s research design .

On the other hand, you might argue that the lack of diverse methodological approaches presents a research gap , and therefore your study could contribute toward filling that gap by taking a different approach. For example, taking a qualitative approach to a research area that is typically approached quantitatively. Of course, if you’re going to go against the methodological grain, you’ll need to provide a strong justification for why your proposed approach makes sense. Nevertheless, it is something worth at least considering.

Regardless of which route you opt for, you need to pay careful attention to the methodologies used in the relevant studies and provide at least some discussion about this in your write-up. Again, it’s useful to keep track of this on some sort of spreadsheet or catalogue as you digest each article, so consider grabbing a copy of our free literature catalogue if you don’t have anything in place.

Bringing It All Together

Alright, so we’ve looked at five important questions that you need to ask (and answer) to help you develop a strong synthesis within your literature review.  To recap, these are:

• Which things are broadly agreed upon within the current research?
• Which things are the subject of disagreement (or at least, present mixed findings)?
• Which theories seem to be central to your research topic and how do they relate or compare to each other?
• Which contexts have (and haven’t) been covered?
• Which methodological approaches are most common?

Importantly, you’re not just asking yourself these questions for the sake of asking them – they’re not just a reflection exercise. You need to weave your answers to them into your actual literature review when you write it up. How exactly you do this will vary from project to project depending on the structure you opt for, but you’ll still need to address them within your literature review, whichever route you go.

The best approach is to spend some time actually writing out your answers to these questions, as opposed to just thinking about them in your head. Putting your thoughts onto paper really helps you flesh out your thinking . As you do this, don’t just write down the answers – instead, think about what they mean in terms of the research gap you’ll present , as well as the methodological approach you’ll take . Your literature synthesis needs to lay the groundwork for these two things, so it’s essential that you link all of it together in your mind, and of course, on paper.

Psst… there’s more!

This post is an extract from our bestselling short course, Literature Review Bootcamp . If you want to work smart, you don't want to miss this .

You Might Also Like:

excellent , thank you

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

The Sheridan Libraries

• Write a Literature Review
• Sheridan Libraries
• Find This link opens in a new window
• Evaluate This link opens in a new window

Get Organized

• Lit Review Prep Use this template to help you evaluate your sources, create article summaries for an annotated bibliography, and a synthesis matrix for your lit review outline.

Synthesize your Information

Synthesize: combine separate elements to form a whole.

Synthesis Matrix

A synthesis matrix helps you record the main points of each source and document how sources relate to each other.

After summarizing and evaluating your sources, arrange them in a matrix or use a citation manager to help you see how they relate to each other and apply to each of your themes or variables.  

By arranging your sources by theme or variable, you can see how your sources relate to each other, and can start thinking about how you weave them together to create a narrative.

• Step-by-Step Approach
• Example Matrix from NSCU
• Matrix Template
• << Previous: Summarize
• Next: Integrate >>
• Last Updated: Sep 26, 2023 10:25 AM
• URL: https://guides.library.jhu.edu/lit-review

How to Synthesize Written Information from Multiple Sources

Shona McCombes

Content Manager

B.A., English Literature, University of Glasgow

Shona McCombes is the content manager at Scribbr, Netherlands.

Learn about our Editorial Process

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

On This Page:

When you write a literature review or essay, you have to go beyond just summarizing the articles you’ve read – you need to synthesize the literature to show how it all fits together (and how your own research fits in).

Synthesizing simply means combining. Instead of summarizing the main points of each source in turn, you put together the ideas and findings of multiple sources in order to make an overall point.

At the most basic level, this involves looking for similarities and differences between your sources. Your synthesis should show the reader where the sources overlap and where they diverge.

Unsynthesized Example

Franz (2008) studied undergraduate online students. He looked at 17 females and 18 males and found that none of them liked APA. According to Franz, the evidence suggested that all students are reluctant to learn citations style. Perez (2010) also studies undergraduate students. She looked at 42 females and 50 males and found that males were significantly more inclined to use citation software ( p < .05). Findings suggest that females might graduate sooner. Goldstein (2012) looked at British undergraduates. Among a sample of 50, all females, all confident in their abilities to cite and were eager to write their dissertations.

Synthesized Example

Studies of undergraduate students reveal conflicting conclusions regarding relationships between advanced scholarly study and citation efficacy. Although Franz (2008) found that no participants enjoyed learning citation style, Goldstein (2012) determined in a larger study that all participants watched felt comfortable citing sources, suggesting that variables among participant and control group populations must be examined more closely. Although Perez (2010) expanded on Franz’s original study with a larger, more diverse sample…

Step 1: Organize your sources

After collecting the relevant literature, you’ve got a lot of information to work through, and no clear idea of how it all fits together.

Before you can start writing, you need to organize your notes in a way that allows you to see the relationships between sources.

One way to begin synthesizing the literature is to put your notes into a table. Depending on your topic and the type of literature you’re dealing with, there are a couple of different ways you can organize this.

Summary table

A summary table collates the key points of each source under consistent headings. This is a good approach if your sources tend to have a similar structure – for instance, if they’re all empirical papers.

Each row in the table lists one source, and each column identifies a specific part of the source. You can decide which headings to include based on what’s most relevant to the literature you’re dealing with.

For example, you might include columns for things like aims, methods, variables, population, sample size, and conclusion.

For each study, you briefly summarize each of these aspects. You can also include columns for your own evaluation and analysis.

The summary table gives you a quick overview of the key points of each source. This allows you to group sources by relevant similarities, as well as noticing important differences or contradictions in their findings.

Synthesis matrix

A synthesis matrix is useful when your sources are more varied in their purpose and structure – for example, when you’re dealing with books and essays making various different arguments about a topic.

Each column in the table lists one source. Each row is labeled with a specific concept, topic or theme that recurs across all or most of the sources.

Then, for each source, you summarize the main points or arguments related to the theme.

The purposes of the table is to identify the common points that connect the sources, as well as identifying points where they diverge or disagree.

Step 2: Outline your structure

Now you should have a clear overview of the main connections and differences between the sources you’ve read. Next, you need to decide how you’ll group them together and the order in which you’ll discuss them.

For shorter papers, your outline can just identify the focus of each paragraph; for longer papers, you might want to divide it into sections with headings.

There are a few different approaches you can take to help you structure your synthesis.

If your sources cover a broad time period, and you found patterns in how researchers approached the topic over time, you can organize your discussion chronologically .

That doesn’t mean you just summarize each paper in chronological order; instead, you should group articles into time periods and identify what they have in common, as well as signalling important turning points or developments in the literature.

If the literature covers various different topics, you can organize it thematically .

That means that each paragraph or section focuses on a specific theme and explains how that theme is approached in the literature.

Source Used with Permission: The Chicago School

If you’re drawing on literature from various different fields or they use a wide variety of research methods, you can organize your sources methodologically .

That means grouping together studies based on the type of research they did and discussing the findings that emerged from each method.

If your topic involves a debate between different schools of thought, you can organize it theoretically .

That means comparing the different theories that have been developed and grouping together papers based on the position or perspective they take on the topic, as well as evaluating which arguments are most convincing.

Step 3: Write paragraphs with topic sentences

What sets a synthesis apart from a summary is that it combines various sources. The easiest way to think about this is that each paragraph should discuss a few different sources, and you should be able to condense the overall point of the paragraph into one sentence.

This is called a topic sentence , and it usually appears at the start of the paragraph. The topic sentence signals what the whole paragraph is about; every sentence in the paragraph should be clearly related to it.

A topic sentence can be a simple summary of the paragraph’s content:

“Early research on [x] focused heavily on [y].”

For an effective synthesis, you can use topic sentences to link back to the previous paragraph, highlighting a point of debate or critique:

“Several scholars have pointed out the flaws in this approach.” “While recent research has attempted to address the problem, many of these studies have methodological flaws that limit their validity.”

By using topic sentences, you can ensure that your paragraphs are coherent and clearly show the connections between the articles you are discussing.

As you write your paragraphs, avoid quoting directly from sources: use your own words to explain the commonalities and differences that you found in the literature.

Don’t try to cover every single point from every single source – the key to synthesizing is to extract the most important and relevant information and combine it to give your reader an overall picture of the state of knowledge on your topic.

Step 4: Revise, edit and proofread

Like any other piece of academic writing, synthesizing literature doesn’t happen all in one go – it involves redrafting, revising, editing and proofreading your work.

Checklist for Synthesis

•   Do I introduce the paragraph with a clear, focused topic sentence?
•   Do I discuss more than one source in the paragraph?
•   Do I mention only the most relevant findings, rather than describing every part of the studies?
•   Do I discuss the similarities or differences between the sources, rather than summarizing each source in turn?
•   Do I put the findings or arguments of the sources in my own words?
•   Is the paragraph organized around a single idea?
•   Is the paragraph directly relevant to my research question or topic?
•   Is there a logical transition from this paragraph to the next one?

Further Information

How to Synthesise: a Step-by-Step Approach

Help…I”ve Been Asked to Synthesize!

Learn how to Synthesise (combine information from sources)

How to write a Psychology Essay

Literature Reviews

• 5. Synthesize your findings
• Getting started
• Types of reviews
• 1. Define your research question
• 2. Plan your search
• 3. Search the literature
• 4. Organize your results

How to synthesize

Approaches to synthesis.

• 6. Write the review
• Artificial intelligence (AI) tools
• Thompson Writing Studio This link opens in a new window
• Need to write a systematic review? This link opens in a new window

Contact a Librarian

Ask a Librarian

In the synthesis step of a literature review, researchers analyze and integrate information from selected sources to identify patterns and themes. This involves critically evaluating findings, recognizing commonalities, and constructing a cohesive narrative that contributes to the understanding of the research topic.

Here are some examples of how to approach synthesizing the literature:

💡 By themes or concepts

🕘 Historically or chronologically

📊 By methodology

These organizational approaches can also be used when writing your review. It can be beneficial to begin organizing your references by these approaches in your citation manager by using folders, groups, or collections.

Create a synthesis matrix

A synthesis matrix allows you to visually organize your literature.

Topic: ______________________________________________

Topic: Chemical exposure to workers in nail salons

• << Previous: 4. Organize your results
• Next: 6. Write the review >>
• Last Updated: Apr 3, 2024 12:40 PM
• URL: https://guides.library.duke.edu/lit-reviews

Services for...

• Faculty & Instructors
• Graduate Students
• Undergraduate Students
• International Students
• Patrons with Disabilities

• Harmful Language Statement
• Re-use & Attribution / Privacy
• Support the Libraries

Library Guides

Literature reviews: synthesis.

• Criticality

Synthesise Information

So, how can you create paragraphs within your literature review that demonstrates your knowledge of the scholarship that has been done in your field of study?  

You will need to present a synthesis of the texts you read.  

Doug Specht, Senior Lecturer at the Westminster School of Media and Communication, explains synthesis for us in the following video:  

Synthesising Texts  

What is synthesis? 

Synthesis is an important element of academic writing, demonstrating comprehension, analysis, evaluation and original creation.  

With synthesis you extract content from different sources to create an original text. While paraphrase and summary maintain the structure of the given source(s), with synthesis you create a new structure.  

The sources will provide different perspectives and evidence on a topic. They will be put together when agreeing, contrasted when disagreeing. The sources must be referenced.  

Perfect your synthesis by showing the flow of your reasoning, expressing critical evaluation of the sources and drawing conclusions.  

When you synthesise think of "using strategic thinking to resolve a problem requiring the integration of diverse pieces of information around a structuring theme" (Mateos and Sole 2009, p448). 

Synthesis is a complex activity, which requires a high degree of comprehension and active engagement with the subject. As you progress in higher education, so increase the expectations on your abilities to synthesise. 

How to synthesise in a literature review: 

Identify themes/issues you'd like to discuss in the literature review. Think of an outline.  

Read the literature and identify these themes/issues.  

Critically analyse the texts asking: how does the text I'm reading relate to the other texts I've read on the same topic? Is it in agreement? Does it differ in its perspective? Is it stronger or weaker? How does it differ (could be scope, methods, year of publication etc.). Draw your conclusions on the state of the literature on the topic.  

Start writing your literature review, structuring it according to the outline you planned.  

Put together sources stating the same point; contrast sources presenting counter-arguments or different points.  

Present your critical analysis.  

Always provide the references. 

The best synthesis requires a "recursive process" whereby you read the source texts, identify relevant parts, take notes, produce drafts, re-read the source texts, revise your text, re-write... (Mateos and Sole, 2009). 

What is good synthesis?  

The quality of your synthesis can be assessed considering the following (Mateos and Sole, 2009, p439):  

Integration and connection of the information from the source texts around a structuring theme. 

Selection of ideas necessary for producing the synthesis. 

Appropriateness of the interpretation.  

Elaboration of the content.  

Example of Synthesis

Original texts (fictitious): 

Synthesis: 

Animal experimentation is a subject of heated debate. Some argue that painful experiments should be banned. Indeed it has been demonstrated that such experiments make animals suffer physically and psychologically (Chowdhury 2012; Panatta and Hudson 2016). On the other hand, it has been argued that animal experimentation can save human lives and reduce harm on humans (Smith 2008). This argument is only valid for toxicological testing, not for tests that, for example, merely improve the efficacy of a cosmetic (Turner 2015). It can be suggested that animal experimentation should be regulated to only allow toxicological risk assessment, and the suffering to the animals should be minimised.   

Bibliography

Mateos, M. and Sole, I. (2009). Synthesising Information from various texts: A Study of Procedures and Products at Different Educational Levels. European Journal of Psychology of Education,  24 (4), 435-451. Available from https://doi.org/10.1007/BF03178760 [Accessed 29 June 2021].

• << Previous: Structure
• Next: Criticality >>
• Last Updated: Nov 18, 2023 10:56 PM
• URL: https://libguides.westminster.ac.uk/literature-reviews

CONNECT WITH US

Literature Review How To

• Things To Consider
• Synthesizing Sources
• Video Tutorials
• Books On Literature Reviews

What is Synthesis

What is Synthesis? Synthesis writing is a form of analysis related to comparison and contrast, classification and division. On a basic level, synthesis requires the writer to pull together two or more summaries, looking for themes in each text. In synthesis, you search for the links between various materials in order to make your point. Most advanced academic writing, including literature reviews, relies heavily on synthesis. (Temple University Writing Center)  

How To Synthesize Sources in a Literature Review

Literature reviews synthesize large amounts of information and present it in a coherent, organized fashion. In a literature review you will be combining material from several texts to create a new text – your literature review.

You will use common points among the sources you have gathered to help you synthesize the material. This will help ensure that your literature review is organized by subtopic, not by source. This means various authors' names can appear and reappear throughout the literature review, and each paragraph will mention several different authors. 

When you shift from writing summaries of the content of a source to synthesizing content from sources, there is a number things you must keep in mind: 

• Look for specific connections and or links between your sources and how those relate to your thesis or question.
• When writing and organizing your literature review be aware that your readers need to understand how and why the information from the different sources overlap.
• Organize your literature review by the themes you find within your sources or themes you have identified. 
• << Previous: Things To Consider
• Next: Video Tutorials >>
• Last Updated: Nov 30, 2018 4:51 PM
• URL: https://libguides.csun.edu/literature-review

Report ADA Problems with Library Services and Resources

A Guide to Evidence Synthesis: What is Evidence Synthesis?

• Meet Our Team
• Our Published Reviews and Protocols
• What is Evidence Synthesis?
• Types of Evidence Synthesis
• Evidence Synthesis Across Disciplines
• Finding and Appraising Existing Systematic Reviews
• 0. Develop a Protocol
• 1. Draft your Research Question
• 2. Select Databases
• 3. Select Grey Literature Sources
• 4. Write a Search Strategy
• 5. Register a Protocol
• 6. Translate Search Strategies
• 7. Citation Management
• 8. Article Screening
• 9. Risk of Bias Assessment
• 10. Data Extraction
• 11. Synthesize, Map, or Describe the Results
• Evidence Synthesis Institute for Librarians
• Open Access Evidence Synthesis Resources

What are Evidence Syntheses?

What are evidence syntheses.

According to the Royal Society, 'evidence synthesis' refers to the process of bringing together information from a range of sources and disciplines to inform debates and decisions on specific issues. They generally include a methodical and comprehensive literature synthesis focused on a well-formulated research question.  Their aim is to identify and synthesize all  of the scholarly research on a particular topic, including both published and unpublished studies. Evidence syntheses are conducted in an unbiased, reproducible way to provide evidence for practice and policy-making, as well as to identify gaps in the research. Evidence syntheses may also include a meta-analysis, a more quantitative process of synthesizing and visualizing data retrieved from various studies. 

Evidence syntheses are much more time-intensive than traditional literature reviews and require a multi-person research team. See this PredicTER tool to get a sense of a systematic review timeline (one type of evidence synthesis). Before embarking on an evidence synthesis, it's important to clearly identify your reasons for conducting one. For a list of types of evidence synthesis projects, see the next tab.

How Does a Traditional Literature Review Differ From an Evidence Synthesis?

How does a systematic review differ from a traditional literature review.

One commonly used form of evidence synthesis is a systematic review.  This table compares a traditional literature review with a systematic review.

Video: Reproducibility and transparent methods (Video 3:25)

Reporting Standards

There are some reporting standards for evidence syntheses. These can serve as guidelines for protocol and manuscript preparation and journals may require that these standards are followed for the review type that is being employed (e.g. systematic review, scoping review, etc). ​

• PRISMA checklist Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses.
• PRISMA-P Standards An updated version of the original PRISMA standards for protocol development.
• PRISMA - ScR Reporting guidelines for scoping reviews and evidence maps
• PRISMA-IPD Standards Extension of the original PRISMA standards for systematic reviews and meta-analyses of individual participant data.
• EQUATOR Network The EQUATOR (Enhancing the QUAlity and Transparency Of health Research) Network is an international initiative that seeks to improve the reliability and value of published health research literature by promoting transparent and accurate reporting and wider use of robust reporting guidelines. They provide a list of various standards for reporting in systematic reviews.

Video: Guidelines and reporting standards

PRISMA Flow Diagram

The  PRISMA  flow diagram depicts the flow of information through the different phases of an evidence synthesis. It maps the search (number of records identified), screening (number of records included and excluded), and selection (reasons for exclusion).  Many evidence syntheses include a PRISMA flow diagram in the published manuscript.

See below for resources to help you generate your own PRISMA flow diagram.

• PRISMA Flow Diagram Tool
• PRISMA Flow Diagram Word Template
• << Previous: Our Published Reviews and Protocols
• Next: Types of Evidence Synthesis >>
• Last Updated: Apr 22, 2024 2:24 PM
• URL: https://guides.library.cornell.edu/evidence-synthesis

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Synthesizing Sources

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

When you look for areas where your sources agree or disagree and try to draw broader conclusions about your topic based on what your sources say, you are engaging in synthesis. Writing a research paper usually requires synthesizing the available sources in order to provide new insight or a different perspective into your particular topic (as opposed to simply restating what each individual source says about your research topic).

Note that synthesizing is not the same as summarizing.  

• A summary restates the information in one or more sources without providing new insight or reaching new conclusions.
• A synthesis draws on multiple sources to reach a broader conclusion.

There are two types of syntheses: explanatory syntheses and argumentative syntheses . Explanatory syntheses seek to bring sources together to explain a perspective and the reasoning behind it. Argumentative syntheses seek to bring sources together to make an argument. Both types of synthesis involve looking for relationships between sources and drawing conclusions.

In order to successfully synthesize your sources, you might begin by grouping your sources by topic and looking for connections. For example, if you were researching the pros and cons of encouraging healthy eating in children, you would want to separate your sources to find which ones agree with each other and which ones disagree.

After you have a good idea of what your sources are saying, you want to construct your body paragraphs in a way that acknowledges different sources and highlights where you can draw new conclusions.

As you continue synthesizing, here are a few points to remember:

• Don’t force a relationship between sources if there isn’t one. Not all of your sources have to complement one another.
• Do your best to highlight the relationships between sources in very clear ways.
• Don’t ignore any outliers in your research. It’s important to take note of every perspective (even those that disagree with your broader conclusions).

Example Syntheses

Below are two examples of synthesis: one where synthesis is NOT utilized well, and one where it is.

Parents are always trying to find ways to encourage healthy eating in their children. Elena Pearl Ben-Joseph, a doctor and writer for KidsHealth , encourages parents to be role models for their children by not dieting or vocalizing concerns about their body image. The first popular diet began in 1863. William Banting named it the “Banting” diet after himself, and it consisted of eating fruits, vegetables, meat, and dry wine. Despite the fact that dieting has been around for over a hundred and fifty years, parents should not diet because it hinders children’s understanding of healthy eating.

In this sample paragraph, the paragraph begins with one idea then drastically shifts to another. Rather than comparing the sources, the author simply describes their content. This leads the paragraph to veer in an different direction at the end, and it prevents the paragraph from expressing any strong arguments or conclusions.

An example of a stronger synthesis can be found below.

Parents are always trying to find ways to encourage healthy eating in their children. Different scientists and educators have different strategies for promoting a well-rounded diet while still encouraging body positivity in children. David R. Just and Joseph Price suggest in their article “Using Incentives to Encourage Healthy Eating in Children” that children are more likely to eat fruits and vegetables if they are given a reward (855-856). Similarly, Elena Pearl Ben-Joseph, a doctor and writer for Kids Health , encourages parents to be role models for their children. She states that “parents who are always dieting or complaining about their bodies may foster these same negative feelings in their kids. Try to keep a positive approach about food” (Ben-Joseph). Martha J. Nepper and Weiwen Chai support Ben-Joseph’s suggestions in their article “Parents’ Barriers and Strategies to Promote Healthy Eating among School-age Children.” Nepper and Chai note, “Parents felt that patience, consistency, educating themselves on proper nutrition, and having more healthy foods available in the home were important strategies when developing healthy eating habits for their children.” By following some of these ideas, parents can help their children develop healthy eating habits while still maintaining body positivity.

In this example, the author puts different sources in conversation with one another. Rather than simply describing the content of the sources in order, the author uses transitions (like "similarly") and makes the relationship between the sources evident.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• AIMS Public Health
• v.3(1); 2016

What Synthesis Methodology Should I Use? A Review and Analysis of Approaches to Research Synthesis

Kara schick-makaroff.

1 Faculty of Nursing, University of Alberta, Edmonton, AB, Canada

Marjorie MacDonald

2 School of Nursing, University of Victoria, Victoria, BC, Canada

Marilyn Plummer

3 College of Nursing, Camosun College, Victoria, BC, Canada

Judy Burgess

4 Student Services, University Health Services, Victoria, BC, Canada

Wendy Neander

Associated data, additional file 1.

When we began this process, we were doctoral students and a faculty member in a research methods course. As students, we were facing a review of the literature for our dissertations. We encountered several different ways of conducting a review but were unable to locate any resources that synthesized all of the various synthesis methodologies. Our purpose is to present a comprehensive overview and assessment of the main approaches to research synthesis. We use ‘research synthesis’ as a broad overarching term to describe various approaches to combining, integrating, and synthesizing research findings.

We conducted an integrative review of the literature to explore the historical, contextual, and evolving nature of research synthesis. We searched five databases, reviewed websites of key organizations, hand-searched several journals, and examined relevant texts from the reference lists of the documents we had already obtained.

We identified four broad categories of research synthesis methodology including conventional, quantitative, qualitative, and emerging syntheses. Each of the broad categories was compared to the others on the following: key characteristics, purpose, method, product, context, underlying assumptions, unit of analysis, strengths and limitations, and when to use each approach.

Conclusions

The current state of research synthesis reflects significant advancements in emerging synthesis studies that integrate diverse data types and sources. New approaches to research synthesis provide a much broader range of review alternatives available to health and social science students and researchers.

1. Introduction

Since the turn of the century, public health emergencies have been identified worldwide, particularly related to infectious diseases. For example, the Severe Acute Respiratory Syndrome (SARS) epidemic in Canada in 2002-2003, the recent Ebola epidemic in Africa, and the ongoing HIV/AIDs pandemic are global health concerns. There have also been dramatic increases in the prevalence of chronic diseases around the world [1] – [3] . These epidemiological challenges have raised concerns about the ability of health systems worldwide to address these crises. As a result, public health systems reform has been initiated in a number of countries. In Canada, as in other countries, the role of evidence to support public health reform and improve population health has been given high priority. Yet, there continues to be a significant gap between the production of evidence through research and its application in practice [4] – [5] . One strategy to address this gap has been the development of new research synthesis methodologies to deal with the time-sensitive and wide ranging evidence needs of policy makers and practitioners in all areas of health care, including public health.

As doctoral nursing students facing a review of the literature for our dissertations, and as a faculty member teaching a research methods course, we encountered several ways of conducting a research synthesis but found no comprehensive resources that discussed, compared, and contrasted various synthesis methodologies on their purposes, processes, strengths and limitations. To complicate matters, writers use terms interchangeably or use different terms to mean the same thing, and the literature is often contradictory about various approaches. Some texts [6] , [7] – [9] did provide a preliminary understanding about how research synthesis had been taken up in nursing, but these did not meet our requirements. Thus, in this article we address the need for a comprehensive overview of research synthesis methodologies to guide public health, health care, and social science researchers and practitioners.

Research synthesis is relatively new in public health but has a long history in other fields dating back to the late 1800s. Research synthesis, a research process in its own right [10] , has become more prominent in the wake of the evidence-based movement of the 1990s. Research syntheses have found their advocates and detractors in all disciplines, with challenges to the processes of systematic review and meta-analysis, in particular, being raised by critics of evidence-based healthcare [11] – [13] .

Our purpose was to conduct an integrative review of the literature to explore the historical, contextual, and evolving nature of research synthesis [14] – [15] . We synthesize and critique the main approaches to research synthesis that are relevant for public health, health care, and social scientists. Research synthesis is the overarching term we use to describe approaches to combining, aggregating, integrating, and synthesizing primary research findings. Each synthesis methodology draws on different types of findings depending on the purpose and product of the chosen synthesis (see Additional File 1 ).

3. Method of Review

Based on our current knowledge of the literature, we identified these approaches to include in our review: systematic review, meta-analysis, qualitative meta-synthesis, meta-narrative synthesis, scoping review, rapid review, realist synthesis, concept analysis, literature review, and integrative review. Our first step was to divide the synthesis types among the research team. Each member did a preliminary search to identify key texts. The team then met to develop search terms and a framework to guide the review.

Over the period of 2008 to 2012 we extensively searched the literature, updating our search at several time points, not restricting our search by date. The dates of texts reviewed range from 1967 to 2015. We used the terms above combined with the term “method* (e.g., “realist synthesis” and “method*) in the database Health Source: Academic Edition (includes Medline and CINAHL). This search yielded very few texts on some methodologies and many on others. We realized that many documents on research synthesis had not been picked up in the search. Therefore, we also searched Google Scholar, PubMed, ERIC, and Social Science Index, as well as the websites of key organizations such as the Joanna Briggs Institute, the University of York Centre for Evidence-Based Nursing, and the Cochrane Collaboration database. We hand searched several nursing, social science, public health and health policy journals. Finally, we traced relevant documents from the references in obtained texts.

We included works that met the following inclusion criteria: (1) published in English; (2) discussed the history of research synthesis; (3) explicitly described the approach and specific methods; or (4) identified issues, challenges, strengths and limitations of the particular methodology. We excluded research reports that resulted from the use of particular synthesis methodologies unless they also included criteria 2, 3, or 4 above.

Based on our search, we identified additional types of research synthesis (e.g., meta-interpretation, best evidence synthesis, critical interpretive synthesis, meta-summary, grounded formal theory). Still, we missed some important developments in meta-analysis, for example, identified by the journal's reviewers that have now been discussed briefly in the paper. The final set of 197 texts included in our review comprised theoretical, empirical, and conceptual papers, books, editorials and commentaries, and policy documents.

In our preliminary review of key texts, the team inductively developed a framework of the important elements of each method for comparison. In the next phase, each text was read carefully, and data for these elements were extracted into a table for comparison on the points of: key characteristics, purpose, methods, and product; see Additional File 1 ). Once the data were grouped and extracted, we synthesized across categories based on the following additional points of comparison: complexity of the process, degree of systematization, consideration of context, underlying assumptions, unit of analysis, and when to use each approach. In our results, we discuss our comparison of the various synthesis approaches on the elements above. Drawing only on documents for the review, ethics approval was not required.

We identified four broad categories of research synthesis methodology: Conventional, quantitative, qualitative, and emerging syntheses. From our dataset of 197 texts, we had 14 texts on conventional synthesis, 64 on quantitative synthesis, 78 on qualitative synthesis, and 41 on emerging syntheses. Table 1 provides an overview of the four types of research synthesis, definitions, types of data used, products, and examples of the methodology.

Although we group these types of synthesis into four broad categories on the basis of similarities, each type within a category has unique characteristics, which may differ from the overall group similarities. Each could be explored in greater depth to tease out their unique characteristics, but detailed comparison is beyond the scope of this article.

Additional File 1 presents one or more selected types of synthesis that represent the broad category but is not an exhaustive presentation of all types within each category. It provides more depth for specific examples from each category of synthesis on the characteristics, purpose, methods, and products than is found in Table 1 .

4.1. Key Characteristics

4.1.1. what is it.

Here we draw on two types of categorization. First, we utilize Dixon Woods et al.'s [49] classification of research syntheses as being either integrative or interpretive . (Please note that integrative syntheses are not the same as an integrative review as defined in Additional File 1 .) Second, we use Popay's [80] enhancement and epistemological models .

The defining characteristics of integrative syntheses are that they involve summarizing the data achieved by pooling data [49] . Integrative syntheses include systematic reviews, meta-analyses, as well as scoping and rapid reviews because each of these focus on summarizing data. They also define concepts from the outset (although this may not always be true in scoping or rapid reviews) and deal with a well-specified phenomenon of interest.

Interpretive syntheses are primarily concerned with the development of concepts and theories that integrate concepts [49] . The analysis in interpretive synthesis is conceptual both in process and outcome, and “the product is not aggregations of data, but theory” [49] , [p.12]. Interpretive syntheses involve induction and interpretation, and are primarily conceptual in process and outcome. Examples include integrative reviews, some systematic reviews, all of the qualitative syntheses, meta-narrative, realist and critical interpretive syntheses. Of note, both quantitative and qualitative studies can be either integrative or interpretive

The second categorization, enhancement versus epistemological , applies to those approaches that use multiple data types and sources [80] . Popay's [80] classification reflects the ways that qualitative data are valued in relation to quantitative data.

In the enhancement model , qualitative data adds something to quantitative analysis. The enhancement model is reflected in systematic reviews and meta-analyses that use some qualitative data to enhance interpretation and explanation. It may also be reflected in some rapid reviews that draw on quantitative data but use some qualitative data.

The epistemological model assumes that quantitative and qualitative data are equal and each has something unique to contribute. All of the other review approaches, except pure quantitative or qualitative syntheses, reflect the epistemological model because they value all data types equally but see them as contributing different understandings.

4.1.2. Data type

By and large, the quantitative approaches (quantitative systematic review and meta-analysis) have typically used purely quantitative data (i.e., expressed in numeric form). More recently, both Cochrane [81] and Campbell [82] collaborations are grappling with the need to, and the process of, integrating qualitative research into a systematic review. The qualitative approaches use qualitative data (i.e., expressed in words). All of the emerging synthesis types, as well as the conventional integrative review, incorporate qualitative and quantitative study designs and data.

4.1.3. Research question

Four types of research questions direct inquiry across the different types of syntheses. The first is a well-developed research question that gives direction to the synthesis (e.g., meta-analysis, systematic review, meta-study, concept analysis, rapid review, realist synthesis). The second begins as a broad general question that evolves and becomes more refined over the course of the synthesis (e.g., meta-ethnography, scoping review, meta-narrative, critical interpretive synthesis). In the third type, the synthesis begins with a phenomenon of interest and the question emerges in the analytic process (e.g., grounded formal theory). Lastly, there is no clear question, but rather a general review purpose (e.g., integrative review). Thus, the requirement for a well-defined question cuts across at least three of the synthesis types (e.g., quantitative, qualitative, and emerging).

4.1.4. Quality appraisal

This is a contested issue within and between the four synthesis categories. There are strong proponents of quality appraisal in the quantitative traditions of systematic review and meta-analysis based on the need for strong studies that will not jeopardize validity of the overall findings. Nonetheless, there is no consensus on pre-defined criteria; many scales exist that vary dramatically in composition. This has methodological implications for the credibility of findings [83] .

Specific methodologies from the conventional, qualitative, and emerging categories support quality appraisal but do so with caveats. In conventional integrative reviews appraisal is recommended, but depends on the sampling frame used in the study [18] . In meta-study, appraisal criteria are explicit but quality criteria are used in different ways depending on the specific requirements of the inquiry [54] . Among the emerging syntheses, meta-narrative review developers support appraisal of a study based on criteria from the research tradition of the primary study [67] , [84] – [85] . Realist synthesis similarly supports the use of high quality evidence, but appraisal checklists are viewed with scepticism and evidence is judged based on relevance to the research question and whether a credible inference may be drawn [69] . Like realist, critical interpretive syntheses do not judge quality using standardized appraisal instruments. They will exclude fatally flawed studies, but there is no consensus on what ‘fatally flawed’ means [49] , [71] . Appraisal is based on relevance to the inquiry, not rigor of the study.

There is no agreement on quality appraisal among qualitative meta-ethnographers with some supporting and others refuting the need for appraisal. [60] , [62] . Opponents of quality appraisal are found among authors of qualitative (grounded formal theory and concept analysis) and emerging syntheses (scoping and rapid reviews) because quality is not deemed relevant to the intention of the synthesis; the studies being reviewed are not effectiveness studies where quality is extremely important. These qualitative synthesis are often reviews of theoretical developments where the concept itself is what is important, or reviews that provide quotations from the raw data so readers can make their own judgements about the relevance and utility of the data. For example, in formal grounded theory, the purpose of theory generation and authenticity of data used to generate the theory is not as important as the conceptual category. Inaccuracies may be corrected in other ways, such as using the constant comparative method, which facilitates development of theoretical concepts that are repeatedly found in the data [86] – [87] . For pragmatic reasons, evidence is not assessed in rapid and scoping reviews, in part to produce a timely product. The issue of quality appraisal is unresolved across the terrain of research synthesis and we consider this further in our discussion.

4.2. Purpose

All research syntheses share a common purpose -- to summarize, synthesize, or integrate research findings from diverse studies. This helps readers stay abreast of the burgeoning literature in a field. Our discussion here is at the level of the four categories of synthesis. Beginning with conventional literature syntheses, the overall purpose is to attend to mature topics for the purpose of re-conceptualization or to new topics requiring preliminary conceptualization [14] . Such syntheses may be helpful to consider contradictory evidence, map shifting trends in the study of a phenomenon, and describe the emergence of research in diverse fields [14] . The purpose here is to set the stage for a study by identifying what has been done, gaps in the literature, important research questions, or to develop a conceptual framework to guide data collection and analysis.

The purpose of quantitative systematic reviews is to combine, aggregate, or integrate empirical research to be able to generalize from a group of studies and determine the limits of generalization [27] . The focus of quantitative systematic reviews has been primarily on aggregating the results of studies evaluating the effectiveness of interventions using experimental, quasi-experimental, and more recently, observational designs. Systematic reviews can be done with or without quantitative meta-analysis but a meta-analysis always takes place within the context of a systematic review. Researchers must consider the review's purpose and the nature of their data in undertaking a quantitative synthesis; this will assist in determining the approach.

The purpose of qualitative syntheses is broadly to synthesize complex health experiences, practices, or concepts arising in healthcare environments. There may be various purposes depending on the qualitative methodology. For example, in hermeneutic studies the aim may be holistic explanation or understanding of a phenomenon [42] , which is deepened by integrating the findings from multiple studies. In grounded formal theory, the aim is to produce a conceptual framework or theory expected to be applicable beyond the original study. Although not able to generalize from qualitative research in the statistical sense [88] , qualitative researchers usually do want to say something about the applicability of their synthesis to other settings or phenomena. This notion of ‘theoretical generalization’ has been referred to as ‘transferability’ [89] – [90] and is an important criterion of rigour in qualitative research. It applies equally to the products of a qualitative synthesis in which the synthesis of multiple studies on the same phenomenon strengthens the ability to draw transferable conclusions.

The overarching purpose of emerging syntheses is challenging the more traditional types of syntheses, in part by using data from both quantitative and qualitative studies with diverse designs for analysis. Beyond this, however, each emerging synthesis methodology has a unique purpose. In meta-narrative review, the purpose is to identify different research traditions in the area, synthesize a complex and diverse body of research. Critical interpretive synthesis shares this characteristic. Although a distinctive approach, critical interpretive synthesis utilizes a modification of the analytic strategies of meta-ethnography [61] (e.g., reciprocal translational analysis, refutational synthesis, and lines of argument synthesis) but goes beyond the use of these to bring a critical perspective to bear in challenging the normative or epistemological assumptions in the primary literature [72] – [73] . The unique purpose of a realist synthesis is to amalgamate complex empirical evidence and theoretical understandings within a diverse body of literature to uncover the operative mechanisms and contexts that affect the outcomes of social interventions. In a scoping review, the intention is to find key concepts, examine the range of research in an area, and identify gaps in the literature. The purpose of a rapid review is comparable to that of a scoping review, but done quickly to meet the time-sensitive information needs of policy makers.

4.3. Method

4.3.1. degree of systematization.

There are varying degrees of systematization across the categories of research synthesis. The most systematized are quantitative systematic reviews and meta-analyses. There are clear processes in each with judgments to be made at each step, although there are no agreed upon guidelines for this. The process is inherently subjective despite attempts to develop objective and systematic processes [91] – [92] . Mullen and Ramirez [27] suggest that there is often a false sense of rigour implied by the terms ‘systematic review’ and ‘meta-analysis’ because of their clearly defined procedures.

In comparison with some types of qualitative synthesis, concept analysis is quite procedural. Qualitative meta-synthesis also has defined procedures and is systematic, yet perhaps less so than concept analysis. Qualitative meta-synthesis starts in an unsystematic way but becomes more systematic as it unfolds. Procedures and frameworks exist for some of the emerging types of synthesis [e.g., [50] , [63] , [71] , [93] ] but are not linear, have considerable flexibility, and are often messy with emergent processes [85] . Conventional literature reviews tend not to be as systematic as the other three types. In fact, the lack of systematization in conventional literature synthesis was the reason for the development of more systematic quantitative [17] , [20] and qualitative [45] – [46] , [61] approaches. Some authors in the field [18] have clarified processes for integrative reviews making them more systematic and rigorous, but most conventional syntheses remain relatively unsystematic in comparison with other types.

4.3.2. Complexity of the process

Some synthesis processes are considerably more complex than others. Methodologies with clearly defined steps are arguably less complex than the more flexible and emergent ones. We know that any study encounters challenges and it is rare that a pre-determined research protocol can be followed exactly as intended. Not even the rigorous methods associated with Cochrane [81] systematic reviews and meta-analyses are always implemented exactly as intended. Even when dealing with numbers rather than words, interpretation is always part of the process. Our collective experience suggests that new methodologies (e.g., meta-narrative synthesis and realist synthesis) that integrate different data types and methods are more complex than conventional reviews or the rapid and scoping reviews.

4.4. Product

The products of research syntheses usually take three distinct formats (see Table 1 and Additional File 1 for further details). The first representation is in tables, charts, graphical displays, diagrams and maps as seen in integrative, scoping and rapid reviews, meta-analyses, and critical interpretive syntheses. The second type of synthesis product is the use of mathematical scores. Summary statements of effectiveness are mathematically displayed in meta-analyses (as an effect size), systematic reviews, and rapid reviews (statistical significance).

The third synthesis product may be a theory or theoretical framework. A mid-range theory can be produced from formal grounded theory, meta-study, meta-ethnography, and realist synthesis. Theoretical/conceptual frameworks or conceptual maps may be created in meta-narrative and critical interpretive syntheses, and integrative reviews. Concepts for use within theories are produced in concept analysis. While these three product types span the categories of research synthesis, narrative description and summary is used to present the products resulting from all methodologies.

4.5. Consideration of context

There are diverse ways that context is considered in the four broad categories of synthesis. Context may be considered to the extent that it features within primary studies for the purpose of the review. Context may also be understood as an integral aspect of both the phenomenon under study and the synthesis methodology (e.g., realist synthesis). Quantitative systematic reviews and meta-analyses have typically been conducted on studies using experimental and quasi-experimental designs and more recently observational studies, which control for contextual features to allow for understanding of the ‘true’ effect of the intervention [94] .

More recently, systematic reviews have included covariates or mediating variables (i.e., contextual factors) to help explain variability in the results across studies [27] . Context, however, is usually handled in the narrative discussion of findings rather than in the synthesis itself. This lack of attention to context has been one criticism leveled against systematic reviews and meta-analyses, which restrict the types of research designs that are considered [e.g., [95] ].

When conventional literature reviews incorporate studies that deal with context, there is a place for considering contextual influences on the intervention or phenomenon. Reviews of quantitative experimental studies tend to be devoid of contextual considerations since the original studies are similarly devoid, but context might figure prominently in a literature review that incorporates both quantitative and qualitative studies.

Qualitative syntheses have been conducted on the contextual features of a particular phenomenon [33] . Paterson et al. [54] advise researchers to attend to how context may have influenced the findings of particular primary studies. In qualitative analysis, contextual features may form categories by which the data can be compared and contrasted to facilitate interpretation. Because qualitative research is often conducted to understand a phenomenon as a whole, context may be a focus, although this varies with the qualitative methodology. At the same time, the findings in a qualitative synthesis are abstracted from the original reports and taken to a higher level of conceptualization, thus removing them from the original context.

Meta-narrative synthesis [67] , [84] , because it draws on diverse research traditions and methodologies, may incorporate context into the analysis and findings. There is not, however, an explicit step in the process that directs the analyst to consider context. Generally, the research question guiding the synthesis is an important factor in whether context will be a focus.

More recent iterations of concept analysis [47] , [96] – [97] explicitly consider context reflecting the assumption that a concept's meaning is determined by its context. Morse [47] points out, however, that Wilson's [98] approach to concept analysis, and those based on Wilson [e.g., [45] ], identify attributes that are devoid of context, while Rodgers' [96] , [99] evolutionary method considers context (e.g., antecedents, consequences, and relationships to other concepts) in concept development.

Realist synthesis [69] considers context as integral to the study. It draws on a critical realist logic of inquiry grounded in the work of Bhaskar [100] , who argues that empirical co-occurrence of events is insufficient for inferring causation. One must identify generative mechanisms whose properties are causal and, depending on the situation, may nor may not be activated [94] . Context interacts with program/intervention elements and thus cannot be differentiated from the phenomenon [69] . This approach synthesizes evidence on generative mechanisms and analyzes contextual features that activate them; the result feeds back into the context. The focus is on what works, for whom, under what conditions, why and how [68] .

4.6. Underlying Philosophical and Theoretical Assumptions

When we began our review, we ‘assumed’ that the assumptions underlying synthesis methodologies would be a distinguishing characteristic of synthesis types, and that we could compare the various types on their assumptions, explicit or implicit. We found, however, that many authors did not explicate the underlying assumptions of their methodologies, and it was difficult to infer them. Kirkevold [101] has argued that integrative reviews need to be carried out from an explicit philosophical or theoretical perspective. We argue this should be true for all types of synthesis.

Authors of some emerging synthesis approaches have been very explicit about their assumptions and philosophical underpinnings. An implicit assumption of most emerging synthesis methodologies is that quantitative systematic reviews and meta-analyses have limited utility in some fields [e.g., in public health – [13] , [102] ] and for some kinds of review questions like those about feasibility and appropriateness versus effectiveness [103] – [104] . They also assume that ontologically and epistemologically, both kinds of data can be combined. This is a significant debate in the literature because it is about the commensurability of overarching paradigms [105] but this is beyond the scope of this review.

Realist synthesis is philosophically grounded in critical realism or, as noted above, a realist logic of inquiry [93] , [99] , [106] – [107] . Key assumptions regarding the nature of interventions that inform critical realism have been described above in the section on context. See Pawson et al. [106] for more information on critical realism, the philosophical basis of realist synthesis.

Meta-narrative synthesis is explicitly rooted in a constructivist philosophy of science [108] in which knowledge is socially constructed rather than discovered, and what we take to be ‘truth’ is a matter of perspective. Reality has a pluralistic and plastic character, and there is no pre-existing ‘real world’ independent of human construction and language [109] . See Greenhalgh et al. [67] , [85] and Greenhalgh & Wong [97] for more discussion of the constructivist basis of meta-narrative synthesis.

In the case of purely quantitative or qualitative syntheses, it may be an easier matter to uncover unstated assumptions because they are likely to be shared with those of the primary studies in the genre. For example, grounded formal theory shares the philosophical and theoretical underpinnings of grounded theory, rooted in the theoretical perspective of symbolic interactionism [110] – [111] and the philosophy of pragmatism [87] , [112] – [114] .

As with meta-narrative synthesis, meta-study developers identify constructivism as their interpretive philosophical foundation [54] , [88] . Epistemologically, constructivism focuses on how people construct and re-construct knowledge about a specific phenomenon, and has three main assumptions: (1) reality is seen as multiple, at times even incompatible with the phenomenon under consideration; (2) just as primary researchers construct interpretations from participants' data, meta-study researchers also construct understandings about the primary researchers' original findings. Thus, meta-synthesis is a construction of a construction, or a meta-construction; and (3) all constructions are shaped by the historical, social and ideological context in which they originated [54] . The key message here is that reports of any synthesis would benefit from an explicit identification of the underlying philosophical perspectives to facilitate a better understanding of the results, how they were derived, and how they are being interpreted.

4.7. Unit of Analysis

The unit of analysis for each category of review is generally distinct. For the emerging synthesis approaches, the unit of analysis is specific to the intention. In meta-narrative synthesis it is the storyline in diverse research traditions; in rapid review or scoping review, it depends on the focus but could be a concept; and in realist synthesis, it is the theories rather than programs that are the units of analysis. The elements of theory that are important in the analysis are mechanisms of action, the context, and the outcome [107] .

For qualitative synthesis, the units of analysis are generally themes, concepts or theories, although in meta-study, the units of analysis can be research findings (“meta-data-analysis”), research methods (“meta-method”) or philosophical/theoretical perspectives (“meta-theory”) [54] . In quantitative synthesis, the units of analysis range from specific statistics for systematic reviews to effect size of the intervention for meta-analysis. More recently, some systematic reviews focus on theories [115] – [116] , therefore it depends on the research question. Similarly, within conventional literature synthesis the units of analysis also depend on the research purpose, focus and question as well as on the type of research methods incorporated into the review. What is important in all research syntheses, however, is that the unit of analysis needs to be made explicit. Unfortunately, this is not always the case.

4.8. Strengths and Limitations

In this section, we discuss the overarching strengths and limitations of synthesis methodologies as a whole and then highlight strengths and weaknesses across each of our four categories of synthesis.

4.8.1. Strengths of Research Syntheses in General

With the vast proliferation of research reports and the increased ease of retrieval, research synthesis has become more accessible providing a way of looking broadly at the current state of research. The availability of syntheses helps researchers, practitioners, and policy makers keep up with the burgeoning literature in their fields without which evidence-informed policy or practice would be difficult. Syntheses explain variation and difference in the data helping us identify the relevance for our own situations; they identify gaps in the literature leading to new research questions and study designs. They help us to know when to replicate a study and when to avoid excessively duplicating research. Syntheses can inform policy and practice in a way that well-designed single studies cannot; they provide building blocks for theory that helps us to understand and explain our phenomena of interest.

4.8.2. Limitations of Research Syntheses in General

The process of selecting, combining, integrating, and synthesizing across diverse study designs and data types can be complex and potentially rife with bias, even with those methodologies that have clearly defined steps. Just because a rigorous and standardized approach has been used does not mean that implicit judgements will not influence the interpretations and choices made at different stages.

In all types of synthesis, the quantity of data can be considerable, requiring difficult decisions about scope, which may affect relevance. The quantity of available data also has implications for the size of the research team. Few reviews these days can be done independently, in particular because decisions about inclusion and exclusion may require the involvement of more than one person to ensure reliability.

For all types of synthesis, it is likely that in areas with large, amorphous, and diverse bodies of literature, even the most sophisticated search strategies will not turn up all the relevant and important texts. This may be more important in some synthesis methodologies than in others, but the omission of key documents can influence the results of all syntheses. This issue can be addressed, at least in part, by including a library scientist on the research team as required by some funding agencies. Even then, it is possible to miss key texts. In this review, for example, because none of us are trained in or conduct meta-analyses, we were not even aware that we had missed some new developments in this field such as meta-regression [117] – [118] , network meta-analysis [119] – [121] , and the use of individual patient data in meta-analyses [122] – [123] .

One limitation of systematic reviews and meta-analyses is that they rapidly go out of date. We thought this might be true for all types of synthesis, although we wondered if those that produce theory might not be somewhat more enduring. We have not answered this question but it is open for debate. For all types of synthesis, the analytic skills and the time required are considerable so it is clear that training is important before embarking on a review, and some types of review may not be appropriate for students or busy practitioners.

Finally, the quality of reporting in primary studies of all genres is variable so it is sometimes difficult to identify aspects of the study essential for the synthesis, or to determine whether the study meets quality criteria. There may be flaws in the original study, or journal page limitations may necessitate omitting important details. Reporting standards have been developed for some types of reviews (e.g., systematic review, meta-analysis, meta-narrative synthesis, realist synthesis); but there are no agreed upon standards for qualitative reviews. This is an important area for development in advancing the science of research synthesis.

4.8.3. Strengths and Limitations of the Four Synthesis Types

The conventional literature review and now the increasingly common integrative review remain important and accessible approaches for students, practitioners, and experienced researchers who want to summarize literature in an area but do not have the expertise to use one of the more complex methodologies. Carefully executed, such reviews are very useful for synthesizing literature in preparation for research grants and practice projects. They can determine the state of knowledge in an area and identify important gaps in the literature to provide a clear rationale or theoretical framework for a study [14] , [18] . There is a demand, however, for more rigour, with more attention to developing comprehensive search strategies and more systematic approaches to combining, integrating, and synthesizing the findings.

Generally, conventional reviews include diverse study designs and data types that facilitate comprehensiveness, which may be a strength on the one hand, but can also present challenges on the other. The complexity inherent in combining results from studies with diverse methodologies can result in bias and inaccuracies. The absence of clear guidelines about how to synthesize across diverse study types and data [18] has been a challenge for novice reviewers.

Quantitative systematic reviews and meta-analyses have been important in launching the field of evidence-based healthcare. They provide a systematic, orderly and auditable process for conducting a review and drawing conclusions [25] . They are arguably the most powerful approaches to understanding the effectiveness of healthcare interventions, especially when intervention studies on the same topic show very different results. When areas of research are dogged by controversy [25] or when study results go against strongly held beliefs, such approaches can reduce the uncertainty and bring strong evidence to bear on the controversy.

Despite their strengths, they also have limitations. Systematic reviews and meta-analyses do not provide a way of including complex literature comprising various types of evidence including qualitative studies, theoretical work, and epidemiological studies. Only certain types of design are considered and qualitative data are used in a limited way. This exclusion limits what can be learned in a topic area.

Meta-analyses are often not possible because of wide variability in study design, population, and interventions so they may have a narrow range of utility. New developments in meta-analysis, however, can be used to address some of these limitations. Network meta-analysis is used to explore relative efficacy of multiple interventions, even those that have never been compared in more conventional pairwise meta-analyses [121] , allowing for improved clinical decision making [120] . The limitation is that network meta-analysis has only been used in medical/clinical applications [119] and not in public health. It has not yet been widely accepted and many methodological challenges remain [120] – [121] . Meta-regression is another development that combines meta-analytic and linear regression principles to address the fact that heterogeneity of results may compromise a meta-analysis [117] – [118] . The disadvantage is that many clinicians are unfamiliar with it and may incorrectly interpret results [117] .

Some have accused meta-analysis of combining apples and oranges [124] raising questions in the field about their meaningfulness [25] , [28] . More recently, the use of individual rather than aggregate data has been useful in facilitating greater comparability among studies [122] . In fact, Tomas et al. [123] argue that meta-analysis using individual data is now the gold standard although access to the raw data from other studies may be a challenge to obtain.

The usefulness of systematic reviews in synthesizing complex health and social interventions has also been challenged [102] . It is often difficult to synthesize their findings because such studies are “epistemologically diverse and methodologically complex” [ [69] , p.21]. Rigid inclusion/exclusion criteria may allow only experimental or quasi-experimental designs into consideration resulting in lost information that may well be useful to policy makers for tailoring an intervention to the context or understanding its acceptance by recipients.

Qualitative syntheses may be the type of review most fraught with controversy and challenge, while also bringing distinct strengths to the enterprise. Although these methodologies provide a comprehensive and systematic review approach, they do not generally provide definitive statements about intervention effectiveness. They do, however, address important questions about the development of theoretical concepts, patient experiences, acceptability of interventions, and an understanding about why interventions might work.

Most qualitative syntheses aim to produce a theoretically generalizable mid-range theory that explains variation across studies. This makes them more useful than single primary studies, which may not be applicable beyond the immediate setting or population. All provide a contextual richness that enhances relevance and understanding. Another benefit of some types of qualitative synthesis (e.g., grounded formal theory) is that the concept of saturation provides a sound rationale for limiting the number of texts to be included thus making reviews potentially more manageable. This contrasts with the requirements of systematic reviews and meta-analyses that require an exhaustive search.

Qualitative researchers debate about whether the findings of ontologically and epistemological diverse qualitative studies can actually be combined or synthesized [125] because methodological diversity raises many challenges for synthesizing findings. The products of different types of qualitative syntheses range from theory and conceptual frameworks, to themes and rich descriptive narratives. Can one combine the findings from a phenomenological study with the theory produced in a grounded theory study? Many argue yes, but many also argue no.

Emerging synthesis methodologies were developed to address some limitations inherent in other types of synthesis but also have their own issues. Because each type is so unique, it is difficult to identify overarching strengths of the entire category. An important strength, however, is that these newer forms of synthesis provide a systematic and rigorous approach to synthesizing a diverse literature base in a topic area that includes a range of data types such as: both quantitative and qualitative studies, theoretical work, case studies, evaluations, epidemiological studies, trials, and policy documents. More than conventional literature reviews and systematic reviews, these approaches provide explicit guidance on analytic methods for integrating different types of data. The assumption is that all forms of data have something to contribute to knowledge and theory in a topic area. All have a defined but flexible process in recognition that the methods may need to shift as knowledge develops through the process.

Many emerging synthesis types are helpful to policy makers and practitioners because they are usually involved as team members in the process to define the research questions, and interpret and disseminate the findings. In fact, engagement of stakeholders is built into the procedures of the methods. This is true for rapid reviews, meta-narrative syntheses, and realist syntheses. It is less likely to be the case for critical interpretive syntheses.

Another strength of some approaches (realist and meta-narrative syntheses) is that quality and publication standards have been developed to guide researchers, reviewers, and funders in judging the quality of the products [108] , [126] – [127] . Training materials and online communities of practice have also been developed to guide users of realist and meta-narrative review methods [107] , [128] . A unique strength of critical interpretive synthesis is that it takes a critical perspective on the process that may help reconceptualize the data in a way not considered by the primary researchers [72] .

There are also challenges of these new approaches. The methods are new and there may be few published applications by researchers other than the developers of the methods, so new users often struggle with the application. The newness of the approaches means that there may not be mentors available to guide those unfamiliar with the methods. This is changing, however, and the number of applications in the literature is growing with publications by new users helping to develop the science of synthesis [e.g., [129] ]. However, the evolving nature of the approaches and their developmental stage present challenges for novice researchers.

4.9. When to Use Each Approach

Choosing an appropriate approach to synthesis will depend on the question you are asking, the purpose of the review, and the outcome or product you want to achieve. In Additional File 1 , we discuss each of these to provide guidance to readers on making a choice about review type. If researchers want to know whether a particular type of intervention is effective in achieving its intended outcomes, then they might choose a quantitative systemic review with or without meta-analysis, possibly buttressed with qualitative studies to provide depth and explanation of the results. Alternately, if the concern is about whether an intervention is effective with different populations under diverse conditions in varying contexts, then a realist synthesis might be the most appropriate.

If researchers' concern is to develop theory, they might consider qualitative syntheses or some of the emerging syntheses that produce theory (e.g., critical interpretive synthesis, realist review, grounded formal theory, qualitative meta-synthesis). If the aim is to track the development and evolution of concepts, theories or ideas, or to determine how an issue or question is addressed across diverse research traditions, then meta-narrative synthesis would be most appropriate.

When the purpose is to review the literature in advance of undertaking a new project, particularly by graduate students, then perhaps an integrative review would be appropriate. Such efforts contribute towards the expansion of theory, identify gaps in the research, establish the rationale for studying particular phenomena, and provide a framework for interpreting results in ways that might be useful for influencing policy and practice.

For researchers keen to bring new insights, interpretations, and critical re-conceptualizations to a body of research, then qualitative or critical interpretive syntheses will provide an inductive product that may offer new understandings or challenges to the status quo. These can inform future theory development, or provide guidance for policy and practice.

5. Discussion

What is the current state of science regarding research synthesis? Public health, health care, and social science researchers or clinicians have previously used all four categories of research synthesis, and all offer a suitable array of approaches for inquiries. New developments in systematic reviews and meta-analysis are providing ways of addressing methodological challenges [117] – [123] . There has also been significant advancement in emerging synthesis methodologies and they are quickly gaining popularity. Qualitative meta-synthesis is still evolving, particularly given how new it is within the terrain of research synthesis. In the midst of this evolution, outstanding issues persist such as grappling with: the quantity of data, quality appraisal, and integration with knowledge translation. These topics have not been thoroughly addressed and need further debate.

5.1. Quantity of Data

We raise the question of whether it is possible or desirable to find all available studies for a synthesis that has this requirement (e.g., meta-analysis, systematic review, scoping, meta-narrative synthesis [25] , [27] , [63] , [67] , [84] – [85] ). Is the synthesis of all available studies a realistic goal in light of the burgeoning literature? And how can this be sustained in the future, particularly as the emerging methodologies continue to develop and as the internet facilitates endless access? There has been surprisingly little discussion on this topic and the answers will have far-reaching implications for searching, sampling, and team formation.

Researchers and graduate students can no longer rely on their own independent literature search. They will likely need to ask librarians for assistance as they navigate multiple sources of literature and learn new search strategies. Although teams now collaborate with library scientists, syntheses are limited in that researchers must make decisions on the boundaries of the review, in turn influencing the study's significance. The size of a team may also be pragmatically determined to manage the search, extraction, and synthesis of the burgeoning data. There is no single answer to our question about the possibility or necessity of finding all available articles for a review. Multiple strategies that are situation specific are likely to be needed.

5.2. Quality Appraisal

While the issue of quality appraisal has received much attention in the synthesis literature, scholars are far from resolution. There may be no agreement about appraisal criteria in a given tradition. For example, the debate rages over the appropriateness of quality appraisal in qualitative synthesis where there are over 100 different sets of criteria and many do not overlap [49] . These differences may reflect disciplinary and methodological orientations, but diverse quality appraisal criteria may privilege particular types of research [49] . The decision to appraise is often grounded in ontological and epistemological assumptions. Nonetheless, diversity within and between categories of synthesis is likely to continue unless debate on the topic of quality appraisal continues and evolves toward consensus.

5.3. Integration with Knowledge Translation

If research syntheses are to make a difference to practice and ultimately to improve health outcomes, then we need to do a better job of knowledge translation. In the Canadian Institutes of Health Research (CIHR) definition of knowledge translation (KT), research or knowledge synthesis is an integral component [130] . Yet, with few exceptions [131] – [132] , very little of the research synthesis literature even mentions the relationship of synthesis to KT nor does it discuss strategies to facilitate the integration of synthesis findings into policy and practice. The exception is in the emerging synthesis methodologies, some of which (e.g., realist and meta-narrative syntheses, scoping reviews) explicitly involve stakeholders or knowledge users. The argument is that engaging them in this way increases the likelihood that the knowledge generated will be translated into policy and practice. We suggest that a more explicit engagement with knowledge users in all types of synthesis would benefit the uptake of the research findings.

Research synthesis neither makes research more applicable to practice nor ensures implementation. Focus must now turn seriously towards translation of synthesis findings into knowledge products that are useful for health care practitioners in multiple areas of practice and develop appropriate strategies to facilitate their use. The burgeoning field of knowledge translation has, to some extent, taken up this challenge; however, the research-practice gap continues to plague us [133] – [134] . It is a particular problem for qualitative syntheses [131] . Although such syntheses have an important place in evidence-informed practice, little effort has gone into the challenge of translating the findings into useful products to guide practice [131] .

5.4. Limitations

Our study took longer than would normally be expected for an integrative review. Each of us were primarily involved in our own dissertations or teaching/research positions, and so this study was conducted ‘off the sides of our desks.’ A limitation was that we searched the literature over the course of 4 years (from 2008–2012), necessitating multiple search updates. Further, we did not do a comprehensive search of the literature after 2012, thus the more recent synthesis literature was not systematically explored. We did, however, perform limited database searches from 2012–2015 to keep abreast of the latest methodological developments. Although we missed some new approaches to meta-analysis in our search, we did not find any new features of the synthesis methodologies covered in our review that would change the analysis or findings of this article. Lastly, we struggled with the labels used for the broad categories of research synthesis methodology because of our hesitancy to reinforce the divide between quantitative and qualitative approaches. However, it was very difficult to find alternative language that represented the types of data used in these methodologies. Despite our hesitancy in creating such an obvious divide, we were left with the challenge of trying to find a way of characterizing these broad types of syntheses.

6. Conclusion

Our findings offer methodological clarity for those wishing to learn about the broad terrain of research synthesis. We believe that our review makes transparent the issues and considerations in choosing from among the four broad categories of research synthesis. In summary, research synthesis has taken its place as a form of research in its own right. The methodological terrain has deep historical roots reaching back over the past 200 years, yet research synthesis remains relatively new to public health, health care, and social sciences in general. This is rapidly changing. New developments in systematic reviews and meta-analysis, and the emergence of new synthesis methodologies provide a vast array of options to review the literature for diverse purposes. New approaches to research synthesis and new analytic methods within existing approaches provide a much broader range of review alternatives for public health, health care, and social science students and researchers.

Acknowledgments

KSM is an assistant professor in the Faculty of Nursing at the University of Alberta. Her work on this article was largely conducted as a Postdoctoral Fellow, funded by KRESCENT (Kidney Research Scientist Core Education and National Training Program, reference #KRES110011R1) and the Faculty of Nursing at the University of Alberta.

MM's work on this study over the period of 2008-2014 was supported by a Canadian Institutes of Health Research Applied Public Health Research Chair Award (grant #92365).

We thank Rachel Spanier who provided support with reference formatting.

List of Abbreviations (in Additional File 1 )

Conflict of interest: The authors declare that they have no conflicts of interest in this article.

Authors' contributions: KSM co-designed the study, collected data, analyzed the data, drafted/revised the manuscript, and managed the project.

MP contributed to searching the literature, developing the analytic framework, and extracting data for the Additional File.

JB contributed to searching the literature, developing the analytic framework, and extracting data for the Additional File.

WN contributed to searching the literature, developing the analytic framework, and extracting data for the Additional File.

All authors read and approved the final manuscript.

Additional Files: Additional File 1 – Selected Types of Research Synthesis

This Additional File is our dataset created to organize, analyze and critique the literature that we synthesized in our integrative review. Our results were created based on analysis of this Additional File.

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 5. The Literature Review
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

• Give a new interpretation of old material or combine new with old interpretations,
• Trace the intellectual progression of the field, including major debates,
• Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
• Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

• Place each work in the context of its contribution to understanding the research problem being studied.
• Describe the relationship of each work to the others under consideration.
• Identify new ways to interpret prior research.
• Reveal any gaps that exist in the literature.
• Resolve conflicts amongst seemingly contradictory previous studies.
• Identify areas of prior scholarship to prevent duplication of effort.
• Point the way in fulfilling a need for additional research.
• Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE : Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

• An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
• Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
• An explanation of how each work is similar to and how it varies from the others,
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

• Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
• Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
• Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
• Persuasiveness -- which of the author's theses are most convincing or least convincing?
• Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

• Current Situation : Information necessary to understand the current topic or focus of the literature review.
• Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
• History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
• Standards : Description of the way in which you present your information.
• Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

• Sources in your literature review do not clearly relate to the research problem;
• You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
• Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
• Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
• Does not describe the search procedures that were used in identifying the literature to review;
• Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
• Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

• How are they organizing their ideas?
• What methods have they used to study the problem?
• What theories have been used to explain, predict, or understand their research problem?
• What sources have they cited to support their conclusions?
• How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

• Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
• Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
• Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

• << Previous: Theoretical Framework
• Next: Citation Tracking >>
• Last Updated: Apr 22, 2024 9:12 AM
• URL: https://libguides.usc.edu/writingguide

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Chapter 7: Synthesizing Sources

Learning objectives.

At the conclusion of this chapter, you will be able to:

• synthesize key sources connecting them with the research question and topic area.

7.1 Overview of synthesizing

7.1.1 putting the pieces together.

Combining separate elements into a whole is the dictionary definition of synthesis.  It is a way to make connections among and between numerous and varied source materials.  A literature review is not an annotated bibliography, organized by title, author, or date of publication.  Rather, it is grouped by topic to create a whole view of the literature relevant to your research question.

Your synthesis must demonstrate a critical analysis of the papers you collected as well as your ability to integrate the results of your analysis into your own literature review.  Each paper collected should be critically evaluated and weighed for “adequacy, appropriateness, and thoroughness” ( Garrard, 2017 ) before inclusion in your own review.  Papers that do not meet this criteria likely should not be included in your literature review.

Begin the synthesis process by creating a grid, table, or an outline where you will summarize, using common themes you have identified and the sources you have found. The summary grid or outline will help you compare and contrast the themes so you can see the relationships among them as well as areas where you may need to do more searching. Whichever method you choose, this type of organization will help you to both understand the information you find and structure the writing of your review.  Remember, although “the means of summarizing can vary, the key at this point is to make sure you understand what you’ve found and how it relates to your topic and research question” ( Bennard et al., 2014 ).

As you read through the material you gather, look for common themes as they may provide the structure for your literature review.  And, remember, research is an iterative process: it is not unusual to go back and search information sources for more material.

At one extreme, if you are claiming, ‘There are no prior publications on this topic,’ it is more likely that you have not found them yet and may need to broaden your search.  At another extreme, writing a complete literature review can be difficult with a well-trod topic.  Do not cite it all; instead cite what is most relevant.  If that still leaves too much to include, be sure to reference influential sources…as well as high-quality work that clearly connects to the points you make. ( Klingner, Scanlon, & Pressley, 2005 ).

7.2 Creating a summary table

Literature reviews can be organized sequentially or by topic, theme, method, results, theory, or argument.  It’s important to develop categories that are meaningful and relevant to your research question.  Take detailed notes on each article and use a consistent format for capturing all the information each article provides.  These notes and the summary table can be done manually, using note cards.  However, given the amount of information you will be recording, an electronic file created in a word processing or spreadsheet is more manageable. Examples of fields you may want to capture in your notes include:

• Authors’ names
• Article title
• Publication year
• Main purpose of the article
• Methodology or research design
• Participants
• Measurement
• Conclusions

  Other fields that will be useful when you begin to synthesize the sum total of your research:

• Specific details of the article or research that are especially relevant to your study
• Key terms and definitions
• Strengths or weaknesses in research design
• Relationships to other studies
• Possible gaps in the research or literature (for example, many research articles conclude with the statement “more research is needed in this area”)
• Finally, note how closely each article relates to your topic.  You may want to rank these as high, medium, or low relevance.  For papers that you decide not to include, you may want to note your reasoning for exclusion, such as ‘small sample size’, ‘local case study,’ or ‘lacks evidence to support assertion.’

This short video demonstrates how a nursing researcher might create a summary table.

7.2.1 Creating a Summary Table

  Summary tables can be organized by author or by theme, for example:

For a summary table template, see http://blogs.monm.edu/writingatmc/files/2013/04/Synthesis-Matrix-Template.pdf

7.3 Creating a summary outline

An alternate way to organize your articles for synthesis it to create an outline. After you have collected the articles you intend to use (and have put aside the ones you won’t be using), it’s time to identify the conclusions that can be drawn from the articles as a group.

  Based on your review of the collected articles, group them by categories.  You may wish to further organize them by topic and then chronologically or alphabetically by author.  For each topic or subtopic you identified during your critical analysis of the paper, determine what those papers have in common.  Likewise, determine which ones in the group differ.  If there are contradictory findings, you may be able to identify methodological or theoretical differences that could account for the contradiction (for example, differences in population demographics).  Determine what general conclusions you can report about the topic or subtopic as the entire group of studies relate to it.  For example, you may have several studies that agree on outcome, such as ‘hands on learning is best for science in elementary school’ or that ‘continuing education is the best method for updating nursing certification.’ In that case, you may want to organize by methodology used in the studies rather than by outcome.

Organize your outline in a logical order and prepare to write the first draft of your literature review.  That order might be from broad to more specific, or it may be sequential or chronological, going from foundational literature to more current.  Remember, “an effective literature review need not denote the entire historical record, but rather establish the raison d’etre for the current study and in doing so cite that literature distinctly pertinent for theoretical, methodological, or empirical reasons.” ( Milardo, 2015, p. 22 ).

As you organize the summarized documents into a logical structure, you are also appraising and synthesizing complex information from multiple sources.  Your literature review is the result of your research that synthesizes new and old information and creates new knowledge.

7.4 Additional resources:

Literature Reviews: Using a Matrix to Organize Research / Saint Mary’s University of Minnesota

Literature Review: Synthesizing Multiple Sources / Indiana University

Writing a Literature Review and Using a Synthesis Matrix / Florida International University

 Sample Literature Reviews Grid / Complied by Lindsay Roberts

Select three or four articles on a single topic of interest to you. Then enter them into an outline or table in the categories you feel are important to a research question. Try both the grid and the outline if you can to see which suits you better. The attached grid contains the fields suggested in the video .

Literature Review Table  

Test yourself.

• Select two articles from your own summary table or outline and write a paragraph explaining how and why the sources relate to each other and your review of the literature.
• In your literature review, under what topic or subtopic will you place the paragraph you just wrote?

Image attribution

Literature Reviews for Education and Nursing Graduate Students Copyright © by Linda Frederiksen is licensed under a Creative Commons Attribution 4.0 International License , except where otherwise noted.

Share This Book

• Langson Library
• Science Library
• Grunigen Medical Library
• Law Library
• Connect From Off-Campus
• Accessibility
• Gateway Study Center

Email this link

Systematic reviews & evidence synthesis methods.

• Schedule a Consultation / Meet our Team
• What is Evidence Synthesis?
• Types of Evidence Synthesis
• Evidence Synthesis Across Disciplines
• Finding and Appraising Existing Systematic Reviews
• 1. Develop a Protocol
• 2. Draft your Research Question
• 3. Select Databases
• 4. Select Grey Literature Sources
• 5. Write a Search Strategy
• 6. Register a Protocol
• 7. Translate Search Strategies
• 8. Citation Management
• 9. Article Screening
• 10. Risk of Bias Assessment
• 11. Data Extraction
• 12. Synthesize, Map, or Describe the Results
• Open Access Evidence Synthesis Resources

What are Evidence Syntheses?

According to the Royal Society, 'evidence synthesis' refers to the process of bringing together information from a range of sources and disciplines to inform debates and decisions on specific issues. They generally include a methodical and comprehensive literature synthesis focused on a well-formulated research question. Their aim is to identify and synthesize all of the scholarly research on a particular topic, including both published and unpublished studies. Evidence syntheses are conducted in an unbiased, reproducible way to provide evidence for practice and policy-making, as well as to identify gaps in the research. Evidence syntheses may also include a meta-analysis, a more quantitative process of synthesizing and visualizing data retrieved from various studies.

Evidence syntheses are much more time-intensive than traditional literature reviews and require a multi-person research team. See this PredicTER tool to get a sense of a systematic review timeline (one type of evidence synthesis). Before embarking on an evidence synthesis, it's important to clearly identify your reasons for conducting one. For a list of types of evidence synthesis projects, see the Types of Evidence Synthesis tab.

How Does a Traditional Literature Review Differ From an Evidence Synthesis?

One commonly used form of evidence synthesis is a systematic review. This table compares a traditional literature review with a systematic review.

Video: Reproducibility and transparent methods (Video 3:25)

Reporting Standards

There are some reporting standards for evidence syntheses. These can serve as guidelines for protocol and manuscript preparation and journals may require that these standards are followed for the review type that is being employed (e.g. systematic review, scoping review, etc).​

• PRISMA checklist Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses.
• PRISMA-P Standards An updated version of the original PRISMA standards for protocol development.
• PRISMA - ScR Reporting guidelines for scoping reviews and evidence maps
• PRISMA-IPD Standards Extension of the original PRISMA standards for systematic reviews and meta-analyses of individual participant data.
• EQUATOR Network The EQUATOR (Enhancing the QUAlity and Transparency Of health Research) Network is an international initiative that seeks to improve the reliability and value of published health research literature by promoting transparent and accurate reporting and wider use of robust reporting guidelines. They provide a list of various standards for reporting in systematic reviews.

Video: Guidelines and reporting standards

PRISMA Flow Diagram

The PRISMA flow diagram depicts the flow of information through the different phases of an evidence synthesis. It maps the search (number of records identified), screening (number of records included and excluded), and selection (reasons for exclusion). Many evidence syntheses include a PRISMA flow diagram in the published manuscript.

See below for resources to help you generate your own PRISMA flow diagram.

• PRISMA Flow Diagram Tool
• PRISMA Flow Diagram Word Template
• << Previous: Schedule a Consultation / Meet our Team
• Next: Types of Evidence Synthesis >>
• Last Updated: Apr 22, 2024 7:04 PM
• URL: https://guides.lib.uci.edu/evidence-synthesis

Off-campus? Please use the Software VPN and choose the group UCIFull to access licensed content. For more information, please Click here

Software VPN is not available for guests, so they may not have access to some content when connecting from off-campus.

Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design (2020)

Chapter: chapter 2 - literature review and synthesis.

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

4 Literature Review and Synthesis Literature Review Purpose of Literature Review Performance-based seismic design (PBSD) for infrastructure in the United States is a developing field, with new research, design, and repair technologies; definitions; and method- ologies being advanced every year. A synthesis report, NCHRP Synthesis 440: Performance- Based Seismic Bridge Design (Marsh and Stringer 2013), was created to capture PBSD understanding up to that point. This synthesis report described the background, objec- tives, and research up until 2011 to 2012 and synthesized the information, including areas where knowledge gaps existed. The literature review in this research report focuses on new infor mation developed after the efforts of NCHRP Synthesis 440. The intention is that this research report will fuel the next challenge: developing a methodology to implement PBSD for bridge design. Literature Review Process Marsh and Stringer (2013) performed an in-depth bridge practice review by sending a questionnaire to all 50 states, with particular attention to regions with higher seismic hazards. The survey received responses from a majority of those agencies. This process was continued in the current project with a request for new information or research that the state depart- ment of transportation (DOT) offices have participated in or are aware of through other organizations. The research team reached out to the list of states and researchers in Table 1. An X within a box is placed in front of their names if they responded. The team also examined the websites of the state DOTs that participated to investigate whether something was studied locally, especially work being developed in California. The research team made an additional effort to perform a practice review of bridge designs, research, and other design industries, specifically in the building industry. The building industry has been developing PBSD for more than 20 years, and some of their developments are appli- cable to bridge design. These combined efforts have allowed the research team to assemble an overview of the state of PBSD engineering details and deployment since Marsh and Stringer’s (2013) report was published. NCHRP Synthesis 440 primarily dealt with the effects of strong ground motion shaking. Secondary effects such as tsunami/seiche, ground failure (surface rupture, liquefaction, or slope failure), fire, and flood were outside the scope of this study. Regardless, their impact on bridges may be substantial, and investigation into their effects is undoubtedly important. C H A P T E R 2

Literature Review and Synthesis 5 The following e-mail was sent to the owners and researchers. Dear (individual): We are assisting Modjeski & Masters with the development of proposed guidelines for Performance- Based Seismic Bridge Design, as part of NCHRP [Project] 12-106. Lee Marsh and our Team at BergerABAM are continuing our efforts from NCHRP Synthesis 440, which included a literature review up to December of 2011. From this timeframe forward, we are looking for published research, contractual language, or owner documents that deal with the following categories: 1. Seismic Hazards (seismic hazard levels, hazard curves, return periods, geo-mean vs. maximum direc- tion, probabilistic vs. deterministic ground motions, conditional mean spectrum, etc.) 2. Structure Response (engineering design parameters, materials and novel columns, isolation bearings, modeling techniques, etc.) 3. Damage Limit States (performance descriptions, displacement ductility, drift ratios, strain limits, rotation curvature, etc.) 4. Potential for Loss (damage descriptions, repairs, risk of collapse, economical loss, serviceability loss, etc.) 5. Performance Design Techniques (relating hazard to design to performance to risk, and how to assess [these] levels together) If you are aware of this type of resource, please provide a contact that we can work with to get this information or provide a published reference we can gather. Your assistance is appreciated. We want to minimize your time, and ask that you respond by Wednesday, 8 February 2017. Thank you again, Research Team Synthesis of PBSD (2012–2016) Objectives of NCHRP Synthesis 440 The synthesis gathered data from a number of different but related areas. Marsh and Stringer (2013), herein referred to as NCHRP Synthesis 440, set the basis for this effort. The research report outline follows what has been added to the NCHRP Synthesis 440 effort since 2012. The information gathered that supplements NCHRP Synthesis 440 includes, but is not limited to, the following topics. • Public and engineering expectations of seismic design and the associated regulatory framework Participation State Alaska DOT Arkansas DOT California DOT (Caltrans) Illinois DOT Indiana DOT Missouri DOT Montana DOT Nevada DOT Oregon DOT South Carolina DOT Utah DOT Washington State DOT Table 1. List of state DOT offices and their participation.

6 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design • Seismic hazard analysis • Structural analysis and design • Damage analysis • Loss analysis • Organization-specific criteria for bridges • Project-specific criteria Where new or updated information is available for these areas, a summary is included. Marsh and Stringer (2013) also identified gaps in the knowledge base of PBSD, current as of 2012, that need to be closed. Knowledge gaps certainly exist in all facets of PBSD; however, key knowledge gaps that should be closed in order to implement PBSD are covered. • Gaps related to seismic hazard prediction • Gaps related to structural analysis • Gaps related to damage prediction • Gaps related to performance • Gaps related to loss prediction • Gaps related to regulatory oversight and training • Gaps related to decision making These knowledge gaps have been filled in somewhat in this research report but, for the most part, these areas are still the key concepts that require additional development to further the development of a PBSD guide specification. Public and Engineering Expectations of Seismic Design and the Associated Regulatory Framework The public expectation of a structure, including a bridge, is that it will withstand an earthquake, but there is a limited understanding of what that actually means. Decision makers struggle to understand how a bridge meeting the current requirements of the AASHTO Guide Specifications for LRFD Seismic Bridge Design (2011), herein referred to as AASHTO guide specifications, will perform after either the expected (design) or a higher level earthquake. Decision makers understand the basis of life safety, wherein the expectation is that no one will perish from a structure collapsing, but often mistakenly believe that the structure will also be usable after the event. In higher level earthquakes, even in some lower level events, this is not true without repair, retrofit, or replacement. In the past decade, there has been an increased awareness by owners and decision makers as to the basis of seismic design. As a result, a need has developed for performance criteria so that economic and social impacts can be interwoven with seismic design into the decision processes (see Figure 1). Several states, including California, Oregon, and the State of Washington, are working toward resiliency plans, although these are developed under different titles or programs within the states. Resiliency has been defined in several ways: (1) amount of damage from an event measured in fatalities, structural replacement cost, and recovery time and (2) the time to resto- ration of lifelines, reoccupation of homes and structures, and, in the short term, resumption of normal living routines. The California DOT Caltrans has generated risk models and is in the process of developing a new seismic design specification to address PBSD in bridge design. The risk models and specifications are not published yet, but the use in PBSD is discussed in greater detail later in this chapter.

Literature Review and Synthesis 7 The State of Washington The State of Washington’s resiliency plan, outlined in Washington State Emergency Management Council–Seismic Safety Committee (2012), works to identify actions and policies before, during, and after an earthquake event that can leverage existing policies, plans, and initiatives to realize disaster resilience within a 50-year life cycle. The hazard level used for trans- portation planning is the 1000 year event. The goals for transportation systems vary depending on the type of service a route provides, as shown in following components of the plan. For major corridors such as Interstates 5, 90, and 405 and floating bridges SR 520, I-90, and Hood Canal, the target timeframe for response and recovery is between 1 to 3 days and 1 to 3 months, depending on location. The current anticipated timeframe based on current capacity and without modifications is between 3 months to 1 year and 1 to 3 years, depending on location. The actual response and recovery time will depend on a number of factors. For example: 1. The number of Washington State DOT personnel who are able to report to work may be limited by a variety of circumstances, including where personnel were at the time of the earthquake and whether they sustained injuries. 2. Bridges and roadways in earthquake-affected areas must be inspected. How long this takes will depend on the number and accessibility of the structures and the availability of qualified inspectors. 3. Some bridges and segments of road may be rendered unusable or only partially usable as a result of the earthquake or secondary effects. The response and recovery timeframe will depend on the number, the location, and the extent of the damage. 4. Certain earthquake scenarios could result in damage to the Ballard Locks and cause the water level in Lake Washington to drop below the level required to operate the floating bridges. 5. Depending on the scenario and local conditions, liquefaction and slope failure could damage both interstates and planned detours. During the first 3 days after the event, the Washington State Department of Transportation (Washington State DOT) will inspect bridges and begin repairs as needed. Washington State DOT’s first priority will be to open key routes for emergency response vehicles. Subsequent phases of recovery will include setting up detours where necessary and regulating the type and Figure 1. PBSD decision-making process (Guidelines Figure 2.0-1). References to guidelines figures and tables within parentheses indicate the proposed AASHTO guidelines.

8 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design volume of traffic, to give the public as much access as possible while damaged roads and bridges are repaired. For major and minor arterials, which encompass arterial roadways (including bridges) other than the interstates (so therefore includes state highways and many city and county roads), the target timeframe for response and recovery is between 0 to 24 hours and 3 months to 1 year, depending on location; the percentage of roadways that are open for use will increase over this period. Anticipated timeframe based on current capacity is between 1 week to 1 month and 1 to 3 years, depending on location; the percentage of roadways that are open for use will increase over this period. The goal of Washington State Emergency Management Council’s resiliency plan is to establish a means to coordinate agencies, public–private partnerships, and standards toward these resiliency goals. The plan outlines goals for recovery times for transportation systems in terms of hours, days, weeks, months, and years, with targets to achieve different levels of recovery (see Table 2) as follows. Similar recovery timeframe processes were established for service sectors (e.g., hospitals, law enforcement, and education); utilities; ferries, airports, ports, and navigable waterways; mass transit; and housing. The overall resiliency plan also discusses the degree to which the recovery of one component or sector would depend on the restoration of another. The key interdependencies that the participants identified include information and communication technologies, transportation, electricity, fuel, domestic water supplies, wastewater systems, finance and banking, and planning and community development. It appears that the implementation of the Washington State Emergency Management Council’s initiative, originally assumed to take 2.5 to 3 years in 2012, has not seen significant development since then. However, the State’s initiative to develop a more resilient community has been extended down to the county level, with King County’s efforts referenced in Rahman et al. (2014) and, at the city level, with the City of Seattle referenced in CEMP (2015). This reflects the commitment needed not only by the legislature and the state departments but also by other agencies (e.g., county, city, or utilities) and the public to take an interest in, and provide funding for, the development of a resiliency plan. The recovery continuum is presented graphically in Figure 2. Developing this relationship with other agency plans is an iterative process that will take time, as shown in Figure 3. Identifying the critical sectors of the agency is necessary to develop a resiliency model and determine how to approach a disaster recovery framework. King County worked from Washington State’s initiative to develop Figure 4. The Oregon DOT Oregon DOT has developed a variation of the approach identified by the State of Wash- ington; further discussion is found later in this chapter. Other Resilience Documents The building industry has recently seen the development of two additional documents that address PBSD in terms of expectations and process. The REDi Rating System from REDi (2013) sets an example for incorporating resilience- based design into the PBSD process. This document outlines structural resilience objectives for organizational resilience, building resilience, loss assessment, and ambient resilience to evaluate and rate the decision making and design methodology using PBSD for a specific project.

Literature Review and Synthesis 9 The document is one of the only references that addresses a system to develop probabilistic methods to estimate downtime. The overall intent is to provide a roadmap to resilience. This roadmap is intended to allow owners to resume business operation and to provide livable conditions quickly after an earthquake. The Los Angeles Tall Buildings Structural Design Council (LATBSDC 2014) created an alter- native procedure specific to their location. Design specification criteria are identified and modi- fications are described as appropriate for the PBSD approach to tall buildings in this localized Minimal (A minimum level of service is restored, primarily for the use of emergency responders, repair crews, and vehicles transporting food and other critical supplies.) Functional (Although service is not yet restored to full capacity, it is sufficient to get the economy moving again—for example, some truck/freight traffic can be accommodated. There may be fewer lanes in use, some weight restrictions, and lower speed limits.) Operational (Restoration is up to 80 to 90 percent of capacity: A full level of service has been restored and is sufficient to allow people to commute to school and to work.) Time needed for recovery to 80 to 90 percent operational given current conditions. Source: Washington State Emergency Management Council–Seismic Safety Committee (2012). Table 2. Washington State’s targets of recovery.

10 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design Source: Adapted from FHWA by CEMP (2015). Figure 2. Recovery continuum process. Source: CEMP (2015). Figure 3. Relationship of disaster recovery framework to other city plans. region. This procedure is a good example of how PBSD criteria and methodology need to be established locally, with a knowledge of risk, resources, and performance needs in order to set the criteria for true PBSD. Seismic Hazard Prediction As outlined in NCHRP Synthesis 440, the seismic hazard includes the regional tectonics and the local site characteristics from either a deterministic or probabilistic viewpoint. The deterministic form allows the assessment of shaking at a site as a function of the controlling earthquake that can occur on all the identified faults or sources. The probabilistic approach

Literature Review and Synthesis 11 defines an acceleration used in design that would be exceeded during a given window of time (e.g., a 7% chance of exceedance in 75 years). The following subsections provide a summary of procedures currently used within AASHTO, as well as new issues that should be eventually addressed in light of approaches used by the building industry. AASHTO Probabilistic Approach As summarized in the AASHTO guide specifications, the current approach used by AASHTO involves the use of a probabilistic hazard model with a nominal return period of 1000 years. Baker (2013) noted that the probabilistic seismic hazard analysis involves the following five steps: 1. Identify all earthquake sources capable of producing damaging ground motions. 2. Characterize the distribution of earthquake magnitudes (the rates at which earthquakes of various magnitudes are expected to occur). 3. Characterize the distribution of source-to-site distances associated with potential earthquakes. 4. Predict the resulting distribution of ground motion intensity as a function of earthquake magnitude, distance, and so forth. 5. Combine uncertainties in earthquake size, location, and ground motion intensity, using a calculation known as the total probability theorem. While implementation of the five steps in the probabilistic approach is beyond what most practicing bridge engineers can easily perform, AASHTO, working through the U.S. Geological Survey, developed a website hazard tool that allows implementation of the probabilistic proce- dure based on the latitude and longitude of a bridge site. The product of the website includes peak ground acceleration (PGA), spectral acceleration at 0.2 s (Ss), and spectral acceleration at 1 s (S1). These values are for a reference-site condition comprising soft rock/stiff soil, having a time-averaged shear wave velocity (Vs) over the upper 100 feet of soil profile equal to 2500 feet per second (fps). The Geological Survey website can also correct for local site conditions following procedures in the AASHTO Guide Specifications for LRFD Seismic Bridge Design. One of the limitations of the current U.S. Geological Survey hazard website is that it is based on a seismic hazard model developed in 2002. The Geological Survey updated its seismic model in 2008 and then in 2014; however, these updates are currently not implemented within the AASHTO hazard model on the Geological Survey’s website. Oregon and the State of Washington have updated the seismic hazard map used by the Oregon DOT and the Washington State Source: Rahman et al. (2014). Figure 4. Resilient King County critical sectors and corresponding subsectors.

12 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design DOT to include the 2014 U.S. Geological Survey hazard model; however, most state DOTs are still using the out-of-date hazard model. Use of the outdated hazard model introduces some inconsistencies in ground motion prediction, relative to the current Geological Survey hazard website tool at some locations. Discussions are ongoing between NCHRP and the U.S. Geological Survey to update the 2002 website tool. Another issue associated with the current AASHTO probabilistic method is that it is based on the geomean of the ground motion. In other words, the ground motion prediction equations in the hazard model are based on the geomean of recorded earthquake motions. These motions are not necessarily the largest motion. The building industry recognized that the maximum direction could result in larger ground motions and introduced maximum direction corrections. These corrections increase spectral acceleration by a factor of 1.1 and S1 by a factor of 1.3. The relevance of this correction to bridges is discussed in the next subsection of this review. The building industry also introduced a risk-of-collapse correction to the hazard model results. This correction is made to Ss and S1. The size of the correction varies from approximately 0.8 to 1.2 within the continental United States. It theoretically adjusts the hazard curves to provide a 1% risk of collapse in 50 years. The risk-of-collapse corrections were developed by the U.S. Geological Survey for a range of building structures located throughout the United States. Although no similar corrections have been developed for bridges, the rationale for the adjust- ment needs to be further evaluated to determine if the rationale should be applied to bridge structures. As a final point within this discussion of probabilistic methods within the AASHTO guide specifications, there are several other areas of seismic response that need to be considered. These include near-fault and basin effects on ground motions, as well as a long-period transition factor. The near-fault and basin adjustments correct the Ss and S1 spectral accelerations for locations near active faults and at the edge of basins, respectively. These adjustments typically increase spectral accelerations at longer periods (> 1 s) by 10% to 20%, depending on specifics of the site. The long-period transition identifies the point at which response spectral ordinates are no longer proportional to the 1/T decay with increasing period. These near-fault, basin, and long-period adjustments have been quantified within the building industry guidance documents but remain, for the most part, undefined within the AASHTO guide specifications. As bridge discussions and research move closer to true probabilistic format for PBSD, these issues need to be addressed as part of a future implementation process. Correction for Maximum Direction of Motion Over the last decade, a debate has been under way within the building industry regarding the appropriate definition of design response spectra (Stewart et al. 2011). The essence of the argument relates to the representation of bidirectional motion via response spectra. In both the AASHTO LRFD Bridge Design Specifications (2014), as well as the AASHTO Guide Specifications for LRFD Seismic Bridge Design (SGS), response spectra are established by defining spectral ordinates at two or three different periods from design maps developed by the U.S. Geological Survey for a return period of 1000 years. The resulting spectra are then adjusted for local site conditions, resulting in the final design spectra. In establishing the design maps for parameters such as Ss and S1, the U.S. Geological Survey has traditionally relied upon probabilistic seismic hazard analysis, which utilizes ground motion prediction equations (GMPEs) defined by the geometric mean of the two principal directions of recorded motion. In 2006, Boore introduced a new rotation independent geometric mean definition termed GMRotI50 (Boore et al. 2006). Then, in 2010, Boore developed a new defini- tion that does not rely upon the geometric mean termed RotD50 spectra, which can be generi- cally expressed as RotDNN spectra, where NN represents the percentile of response (i.e., 50 is

Literature Review and Synthesis 13 consistent with the median, 0 is the minimum, and 100 is the maximum). The NGA–West2 project GMPEs utilized RotD50 spectra for the ground motion models; however, the 2009 National Earthquake Hazards Reduction Program (NEHRP) provisions adopted a factor to modify the median response, RotD50, to the maximum possible response, RotD100 as the spectra for the design maps (Stewart et al. 2011). Introducing RotD100 resulted in a 10% to 30% increase in spectral ordinates results relative to the geometric mean, which has traditionally been used as a basis of seismic design. In order to appreciate the impact of these choices, a brief discussion of RotDNN spectra is warranted. As described in Boore (2010), for a given recording station, the two orthogonal- component time series are combined into a single time series corresponding to different rotation angles, as shown in Equation 1: aROT(t ; θ) = a1(t)cos(θ) + a2(t)sin(θ) (1) where a1(t ) and a2(t ) are the orthogonal horizontal component acceleration time series and θ is the rotation angle. For example, consider the two orthogonal horizontal component time series, H1 and H2, shown in Figure 5. The single time series corresponding to the rotation angle θ is created by combining the Direction 1 and Direction 2 time series. Then, the response spectrum for that single time series can be obtained, as shown in the figure. The process is repeated for a range of azimuths from 0° to one rotation-angle increment less than 180°. If the rotation-angle increment is θ, then there will be 180/θ single time series, as well as 180/θ corresponding response spectra. For example, if θ = 30°, then there will be six single time series (the original two, as well as four generated time series), as well as six response spectra, as shown in Figure 6. Once the response spectra for all rotation angles are obtained, then the nth percentile of the spectral amplitude over all rotation angles for each period is computed (e.g., RotD50 is the median value and RotD100 is the largest value for all rotation angles). For example, at a given period of 1 s, the response spectra values for all rotation angles are sorted, and the RotD100 value would be the largest value from all rotation angles while RotD50 would be the median. This is repeated for all periods, with potentially different rotation angles, producing the largest Source: Palma (2019). Figure 5. Combination of time series to generate rotation dependent spectra.

14 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design response at any given period (period-dependent rotation angle.) Figure 7 shows an example of the two orthogonal horizontal components, as well as the RotD50 and RotD100 spectra for the as-recorded ground motion from the 2011 Christchurch, New Zealand, earthquake at Kaiapoi North School station. As can be seen in the sample spectra (see Figure 7), the RotD100 spectrum represents a sub- stantial increase in demand when compared with the RotD50 spectrum. The main question facing the bridge community from this point onward is the appropriate selection of response spectra definition. This question can only be answered by developing sample designs to both the RotD50 and RotD100 spectra, which would then be evaluated via no-linear time history analysis. Such a study will require multiple bridge configurations and multiple ground motions. As an example of the potential impact, Figure 8 shows the results of a single-degree-of- freedom bridge column designed according to both RotD50 and RotD100 spectra, along with the resulting nonlinear time history analysis. The column was designed using direct displacement- based design to achieve a target displacement of 45 cm. It is clear from the results in Figure 8d that the nonlinear response of the column designed to the RotD100 spectrum matches the target Source: Palma (2019). Figure 6. Example of time series rotations with an angle increment (p) of 30ç. Source: Palma (2019). Figure 7. Sample spectra for a recorded ground motion pair.

Literature Review and Synthesis 15 reasonably well, while designing to the RotD50 spectrum results in displacements that are much greater than expected. This is, of course, only one result of an axisymmetric system. In the future (and outside the scope of this project), a systematic study could be conducted for both single degree of freedom and multiple degrees of freedom systems. The literature on this topic can be divided into two categories: (1) response spectra definitions and (2) impact on seismic response. The majority of the literature addresses the former. For example, Boore et al. (2006) and Boore (2010) introduced orientation-independent measures of seismic intensity from two horizontal ground motions. Boore et al. (2006) proposed two measures of the geometric mean of the seismic intensity, which are independent of the in-situ orientations of the sensors. One measure uses period-dependent rotation angles to quantify the spectral intensity, denoted GMRotDnn. The other measure is the GMRotInn, where I stands for period-independent. The ground motion prediction equations of Abrahamson and Silva (1997), Figure 8. Single bridge column designed according to both RotD50 and RotD100 spectra (Tabas EQ = Tabas earthquake and displ. = displacement).

16 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design Boore et al. (1997), Campbell and Bozorgnia (2003), and Sadigh et al. (1997) have been updated using GMRotI50 as the dependent variable. Since more users within the building industry expressed the desire to use the maximum spec- tral response over all the rotation angles without geometric means, Boore (2010) introduced the measures of ground-shaking intensity irrespective of the sensor orientation. The measures are RotDnn and RotInn, whose computation is similar to GMRotDnn and GMRotInn without computing the geometric means. With regard to impact on seismic response, the opinion paper by Stewart et al. (2011) and the work by Mackie et al. (2011) on the impact of incidence angle on bridge response are relevant. Specifically, Stewart et al. (2011) noted the importance of computational analysis of structures (which had not been done as of 2011) in proposing appropriate spectra definitions. Other Methodologies for Addressing Seismic Ground Motion Hazards There are several other reports that address the question of the methodology that may be utilized in developing the seismic hazard. These recent studies endeavored to create a method- ology that is easier for engineers, as users, to understand how to tie the seismic hazard to the performance expectation. The variability of these approaches also demonstrates the broad range of options and therefore a limited understanding by practitioners in the bridge design industry. Following are some examples that apply to PBSD. Wang et al. (2016) performed a probabilistic seismic risk analysis (SRA) based on a single ground motion parameter (GMP). For structures whose responses can be better predicted using multiple GMPs, a vector-valued SRA (VSRA) gives accurate estimates of risk. A simplified approach to VSRA, which can substantially improve computational efficiency without losing accuracy, and a new seismic hazard de-aggregation procedure are proposed. This approach and the new seismic hazard de-aggregation procedure would allow an engineer to determine a set of controlling earthquakes in terms of magnitude, source–site distance, and occurrence rate for the site of interest. Wang et al. presented two numerical examples to validate the effectiveness and accuracy of the simplified approach. Factors affecting the approximations in the simplified approach were discussed. Kwong and Chopra (2015) investigated the issue of selecting and scaling ground motions as input excitations for response history analyses of buildings in performance-based earthquake engineering. Many ground motion selection and modification procedures have been developed to select ground motions for a variety of objectives. This report focuses on the selection and scaling of single, horizontal components of ground motion for estimating seismic demand hazard curves of multistory frames at a given site. Worden et al. (2012) used a database of approximately 200,000 modified Mercalli intensity (MMI) observations of California earthquakes collected from U.S. Geological Survey reports, along with a comparable number of peak ground motion amplitudes from California seismic networks, to develop probabilistic relationships between MMI and peak ground velocity (PGV), PGA, and 0.3-s, 1-s, and 3-s 5% damped pseudo-spectral acceleration. After associating each ground motion observation with an MMI computed from all the seismic responses within 2 kilometers of the observation, a joint probability distribution between MMI and ground motion was derived. A reversible relationship was then derived between MMI and each ground motion parameter by using a total least squares regression to fit a bilinear function to the median of the stacked probability distributions. Among the relationships, the fit-to-peak ground velocity has the smallest errors, although linear combinations of PGA and PGV give nominally better results. The magnitude and distance terms also reduce the overall residuals and are justifiable on an information theoretical basis.

Literature Review and Synthesis 17 Another approach to developing the appropriate seismic hazard comes out of Europe. Delavaud et al. (2012) presented a strategy to build a logic tree for ground motion prediction in European countries. Ground motion prediction equations and weights have been determined so that the logic tree captures epistemic uncertainty in ground motion prediction for six different tectonic regions in Europe. This includes selecting candidate GMPEs and simultaneously running them through a panel of six experts to generate independent logic trees and rank the GMPEs on available test data. The collaboration of this information is used to set a weight to the GMPEs and create a consensus logic tree. This output then is run through a sensitivity analysis of the proposed weights on the seismic hazard before setting a final logic tree for the GMPEs. Tehrani and Mitchell (2014) used updated seismic hazard maps for Montreal, Canada to develop a uniform hazard spectra for Site Class C and a seismic hazard curve to analyze bridges in the localized area. Kramer and Greenfield (2016) evaluated three case studies following the 2011 Tohoku earthquake to better understand and design for liquefaction. Existing case history databases are incomplete with respect to many conditions for which geotechnical engineers are often required to evaluate liquefaction potential. These include liquefaction at depth, liquefaction of relatively dense soils, and liquefaction of gravelly soils. Kramer and Greenfield’s investigation of the three case histories will add to the sparse existing data for those conditions, and their interpretations will aid in the validation and development of predictive procedures for liquefaction potential evaluation. Structural Analysis and Design Predicting the structural response to the earthquake ground motions is critical for the PBSD process. NCHRP Synthesis 440 outlined several analysis methods that can be used to accomplish this task. The multimodal linear dynamic procedures are outlined in AASHTO LRFD Bridge Design Specifications (AASHTO 2014) and AASHTO Guide Specifications for LRFD Seismic Bridge Design (AASHTO 2011), although the Guide Specifications also include the parameters for performing a model pushover analysis in addition to prescriptive detail practices to ensure energy-dissipating systems behave as intended and other elements are capacity-protected. Other methods of analysis may be better suited for PBSD, but the initial PBSD approach will likely follow the procedures of the AASHTO guide specifications, with multi-level hazards and performance expectations. Limited research and code development have been accomplished since NCHRP Synthesis 440, but one new analysis method, outlined in Babazadeh et al. (2015), includes a three-dimensional finite element model simulation that is used to efficiently predict intermediate damage limit states in a consistent manner, with the experimental observations extracted from the actual tested columns. Other recent articles of structural analysis identified areas of improvement in the current design methodology that may be beneficial to PBSD. Huff and Pezeshk (2016) compared the substitute structure method methodology for isolated bearings with the displacement-based design methodology for ordinary bridges and showed that these two methodologies vary in estimating inelastic displacements. Huff (2016a) identified issues that are generally simplified or ignored in current practice of predicting inelastic behavior of bridges during earthquakes, both on the capacity (in the section of the element type and geometric nonlinearities) and demand (issues related to viscous dampening levels) sides of the process. The current SGS methodology for nonlinear static procedures were compared in Hajihashemi et al. (2017) with recent methodologies for multimodal pushover procedures that take into account all significant modes of the structure and with modified equivalent linearization procedures developed for

18 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design FEMA-440 (FEMA 2005). All of these analysis articles identify areas of current discussion on how to improve the analytical procedures proposed in the SGS. NCHRP Synthesis 440 focused primarily on new analysis methods, but a recent increased focus, in both academia and industry, has to do with new materials and systems and their impacts on PBSD. The evolution of enhanced seismic performance has been wrapped into several research topics, such as accelerated bridge construction (ABC), novel columns, and PBSD. The following are several aspects, though not all-encompassing, which have been improved upon in the last 6 years or so. Improving Structural Analysis Through Better Material Data The analysis and performance of a bridge are controlled with material property parameters incorporated into the seismic analysis models, specifically for the push-over analysis method. AASHTO Guide Specifications for LRFD Seismic Bridge Design (AASHTO 2011) specifies the strain limits to use for ASTM A706 (Grade 60) and ASTM A615 Grade 60 reinforcement. These strain limits come from Caltrans study of 1,100 mill certificates for ASTM A706 Grade 60 in the mid-1990s for projects in Caltrans bridge construction. The results were reported as elongation—not strain—at peak stress, so select bar pull tests were performed to correlate elongation to strain at peak stress. This was assumed to be a conservative approach, though it has recently been validated with a new ASTM A706 Grade 80 study at North Carolina State University by Overby et al. (2015a), which showed Caltrans numbers, by comparison, for Grade 60 are reasonable and conservative. Overby et al. (2015b) developed stress strain parameters for ASTM A706 Grade 80 reinforcing steel. Approximately 800 tests were conducted on bars ranging from #4 to #18 from multiple heats from three producing mills. Statistical results were presented for elastic modulus, yield strain and stress, strain-hardening strain, strain at maximum stress, and ultimate stress. Research is currently under way at North Carolina State University that aims to identify strain limit states, plastic hinge lengths, and equivalent viscous damping models for bridge columns constructed from A706 Grade 80 reinforcing steel. Work is also under way at the University of California, San Diego, on applications of Grade 80 rebar for capacity-protected members such as bridge cap beams. Design Using New Materials and Systems Structural analysis and design are fundamentally about structural response to the earthquake ground motion and the analysis methods used to develop this relationship. The complexity of the analysis depends on the geometry of the structure and elements and the extent of inelastic behavior. This is coupled with the damage, or performance criteria but has been broken out for the purposes of this report and NCHRP Synthesis 440. Next generation bridge columns, often referred to as novel columns, are improving as a tool for engineers to control both the structural analysis, as the make-up of the material changes the inelastic behavior, and the element performance of bridges in higher seismic hazards. The energy-dissipating benefits of low damage materials—such as ultrahigh-performance concrete (UHPC), engineered cementi- tious composites (ECC), and shape memory alloy, fiber-reinforced polymer (FRP) wraps and tubes, elastomeric bearings, and post-tensioned strands or bars—can be utilized by engineers to improve seismic performance and life-cycle costs after a significant seismic event. Recent (Saiidi et al. 2017) studies tested various combinations of these materials to determine if there are columns that can be built with these materials that are equivalent to, or better than, conventional reinforced concrete columns (in terms of cost, complexity, and construction duration) but that improve seismic performance, provide greater ductility, reduce damage, and accommodate a quicker recovery time and reduce loss in both the bridge and the economic environment.

Literature Review and Synthesis 19 Accelerated bridge construction is also a fast-developing field in bridge engineering, with draft guide specifications for design and construction currently being developed for adop- tion by AASHTO for AASHTO LRFD Bridge Design Specifications (AASHTO 2014). ABC has economic impacts that go beyond seismic engineering, but research is focusing on details and connections for accelerated construction in higher seismic regions, moving two research paths forward at the same time. Tazarv and Saiidi (2014) incorporated ABC research with novel column research to evaluate combined novel column materials that can be constructed quickly. The research focused on the performance of materials and how to incorporate them into practice. Key mechanical properties of reinforcing SMA were defined as follows: • Observed yield strength (fyo) is the stress at the initiation of nonlinearity on the first cycle of loading to the upper plateau. • Austenite modulus (k1) is the average slope between 15% to 70% of fyo. • Post yield stiffness (k2) is the average slope of curve between 2.5% and 3.5% of strain on the upper plateau of the first cycle of loading to 6% strain. • Austenite yield strength (fy) is the stress at the intersection of line passing through origin with slope of k1 and line passing through stress at 3% strain with slope of k2. • Lower plateau inflection strength (fi) is the stress at the inflection point of lower plateau during unloading from the first cycle to 6% strain. • Lower plateau stress factor, β = 1 – (fi/fy). • Residual strain (eres) is the tensile strain after one cycle to 6% and unloading to 1 ksi (7 MPa). • Recoverable super-elastic strain (er) is maximum strain with at least 90% strain recovery capacity. Using the ASTM standard for tensile testing, er ≤ 6%. • Martensite modulus (k3) is the slope of the curve between 8% to 9% strain, subsequent to one cycle of loading to 6% strain, unloading to 1 ksi (7 MPa) and reloading to the ultimate stress. • Secondary post-yield stiffness ratio, α = k3/k1. • Ultimate strain (eu) is strain at failure. A graphical representation is shown in Figure 9, and minimum and expected mechanical properties are listed in Table 3. Other researchers, such as at the University of Washington, are currently testing grouted bars using conventional grouts and finding that these development lengths can be reduced greatly. However, it is the force transfer of the grouted duct to the reinforcing outside the duct that may Figure 9. NiTi SE SMA nonlinear model.

20 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design require additional length to adequately develop the energy-dissipating or capacity-protecting system that was intended by the designer for performance of the bridge in a high seismic event. Tazarv and Saiidi (2014) identified other material properties such as UHPC and ECC, shown in Tables 4 and 5, respectively. Tazarv and Saiidi (2014) also addressed grouted splice sleeve couplers, self-consolidating concrete (SCC), and other connection types that could be used in ABC and novel column configurations, testing these materials in the laboratory to see if various combinations produced a logical system to be carried forward in research, design, and implementation. Trono et al. (2015) studied a rocking post-tensioned hybrid fiber-reinforced concrete (HyFRC) bridge column that was designed to limit damage and residual drifts and that was tested dynamically under earthquake excitation. The column utilized post-tensioned strands, HyFRC, and a combination of unbonded and headed longitudinal reinforcement. There have been two projects related to the field of novel columns and ABC through the National Cooperative Highway Research Program. One project was NCHRP Project 12-101, which resulted in NCHRP Report 864, 2 volumes (Saiidi et al. 2017), and the other project was NCHRP Project 12-105, which resulted in NCHRP Research Report 935 (Saiidi et al. 2020). NCHRP Project 12-101 identified three novel column systems—specifically, SMA and ECC, ECC and FRP, and hybrid rocking column using post-tensioned strands and fiber-reinforced Parameter Tensile Compressive,ExpectedbExpectedbMinimuma Table 3. Minimum expected reinforcing NiTi SE SMA mechanical properties. Properties Range Poisson’s Ratio 0.2 Creep Coefficient* 0.2 to 0.8 Total Shrinkage** *Depends on curing conditions and age of loading. up to 900x10-6 Equation Compressive Strength (f'UHPC) f'UHPC 20 to 30 ksi, (140 to 200 MPa) Coefficient of Thermal Expansion (5.5 to 8.5)x10 -6/°F, (10 to 15)x10-6/°C Specific Creep* (0.04 to 0.3)x10 -6/psi, (6 to 45)x10-6/MPa A time-dependent equation for UHPC strength is available. Tensile Cracking Strength (ft,UHPC) ft,UHPC = 6.7 (psi) f'UHPCEUHPC = 49000 (psi) 0.9 to 1.5 ksi, (6 to 10 MPa) Modulus of Elasticity (EUHPC) 6000 to 10000 ksi, (40 to 70 GPa) **Combination of drying shrinkage and autogenous shrinkage and depends on curing method. Table 4. UHPC mechanical properties.

Literature Review and Synthesis 21 polymer confinement—and compared them to a conventional reinforced column. The research and properties of the material are provided; incorporating laboratory tests and calibration, design examples are created to help engineers understand how to use these advanced materials in a linear elastic seismic demand model and to determine performance using a pushover analysis. It is worth noting that ductility requirements do not accurately capture the perfor- mance capabilities of these novel columns, and drift ratio limits are being used instead, similar to the building industry. NCHRP Project 12-101 also provided evaluation criteria that can be evaluated and incorporated by AASHTO into a guide specification or into AASHTO Guide Specifications for LRFD Seismic Bridge Design (AASHTO 2011) directly. NCHRP Project 12-105 synthesized research, design codes, specifications, and contract language throughout all 50 states and combined the knowledge base and lessons learned for ABC into proposed guide specifications for both design and construction. This work focused on connections, and most of that information is related to seismic performance of ABC elements and systems. Earthquake resisting elements (ERE) and earthquake resisting systems (ERS) are specifically identified, defined, and prescribed for performance in AASHTO guide specifica- tions (AASHTO 2011) but only implicitly applied in AASHTO LRFD Bridge Design Specifications (AASHTO 2014). Since NCHRP Project 12-105 is applicable to both of these design resources, ERE and ERS are discussed in terms of how to apply performance to the force-based seismic design practice of AASHTO LRFD Bridge Design Specifications (AASHTO 2014). The proposed guide specification language also identifies when performance of materials have to be incor- porated into the design, say in higher seismic hazards, and when it is acceptable to apply ABC connections and detailing practices with prescriptive design methodologies. As the industry’s understanding of performance increases, the engineering industry is accepting the benefits that come from a more user-defined engineering practice that is implemented by identifying material properties; evaluating hazards and soil and structural responses; and verifying performance through strain limits, damage limits states, moment curvature, displacements, and ductility. These tools and advancements in ABC and novel column designs, including other material property performance and analytical methodologies, are allowing PBSD to advance in other areas, such as hazard prediction, loss prediction, and the owner decision-making process. Feng et al. (2014a) studied the application of fiber-based analysis to predict the nonlinear response of reinforced concrete bridge columns. Specifically considered were predictions of overall force-deformation hysteretic response and strain gradients in plastic hinge regions. The authors also discussed the relative merits of force-based and displacement-based fiber elements and proposed a technique for prediction of nonlinear strain distribution based on the modified compression field theory. Fulmer et al. (2013) developed a new steel bridge system that is based upon ABC techniques that employ an external socket to connect a circular steel pier to a cap beam through the use of grout and shear studs. The resulting system develops a plastic hinge in the pipe away from the column-to-cap interface. An advantage of the design is ease of construction, as no field welding Properties Range Flexural Strength 1.5 to 4.5 ksi (10 to 30 MPa) Modulus of Elasticity 2600 to 5000 ksi (18 to 34 GPa) Ultimate Tensile Strain 1 to 8% Ultimate Tensile Strength 0.6 to 1.7 ksi (4 to 12 MPa) First Crack Strength 0.4 to 1.0 ksi (3 to 7 MPa) Compressive Strength 3 to 14 ksi (20 to 95 MPa) Table 5. ECC mechanical properties.

22 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design is required: the two assemblies are placed together and the annular space between the column and cap filled with grout. Figure 10 shows the details of this connection, and Figure 11 shows a test of the system. Another system being investigated is isolation bearings or dampening devices. Xie and Zhiang (2016) investigated the effectiveness and optimal design of protective devices for the seismic protection of highway bridges. Fragility functions are first derived by probabilistic seismic demand analysis, repair cost ratios are then derived using a performance-based methodol- ogy, and the associated component failure probability. Subsequently, the researchers tried to identify the optimal design parameters of protective devices for six design cases with various combinations of isolation bearings and fluid dampers and discussed the outcomes. Damage mitigation through isolation and energy dissipation devices is continually improving based on research, development, and implementation in the field. Recent events within the State of Washington, Alaska, and other state agencies have shown that the benefits of these tools can be compromised if the intended performance cannot be sustained for the 75-year design life of the structure. Mackie and Stojadinovic (2015) outlined performance criteria for fabrica- tion and construction that need to be administered properly, and engineers should consider the effects of moisture, salts, or other corrosive environmental conditions that can affect the performance of the isolation or energy-dissipating system. Another constraint with these systems can be the proprietary nature that occurs as a specific isolation or energy-dissipating system is utilized to develop a specific performance expectation that can only be accomplished with the prescribed system. This proprietary nature of these systems can create issues for certain funding sources that require equal bidding opportunities and the project expense that can accompany a proprietary system. To address this type of design constraint, Illinois DOT has been developing an earthquake-resisting system (ERS) to leverage the displacement capacity available at typical bearings in order to provide seismic protection to substructures of typical bridges. LaFave et al. (2013a) identified the effects and design parameters, Source: Fulmer et al. (2013). 5" 4 at 5" O.C. A A A-A Connection Details 45° UT 100% 3 8" 12 Studs Spaced Around Cross Section 30°Typ. 15° Offset Studs Inside Pipe from Cap Beam CL HSS16x0.500 Pipe 24x0.500 2'-0"2 14 " 4 at 5" O.C. 212"-34 "Ø Shear Studs 1'-11" Pipe Stud Detail Grout Provided By and Placed by NCSU Figure 10. Grouted shear stud bridge system.

Literature Review and Synthesis 23 such as fuse capacity, shear response, and sliding response, which can be used to account for more standard bearing configurations in seismic analysis, especially lower seismic hazard regions. A variation on the use of bearings in order to improve seismic performance of a pier wall configuration was outlined in Bignell et al. (2006). Historically, pinned, rocking, and sliding bearings have been used with interior pier walls and steel girder superstructures. These bearing configurations were compared with replacement elastomeric bearing configurations and details for structural analysis techniques, damage limit states, and structural fragility, and performance through probability distributions were utilized as a PBSD process for determining solutions to seismic isolation and enhanced seismic performance. The foundation conditions, pier wall effects, bearing type, and even embankment effects to structural performance were included in this evaluation. Another approach to enhanced performance is modifications to foundation elements or increased understanding and modeling of soil–structure interaction, specifically where lateral spread or liquefaction design conditions make conventional bridge design and elements imprac- tical. One example of this is the seismic design and performance of bridges constructed with rocking foundations, as evaluated in Antonellis and Panagiotou (2013). This type of rocking goes beyond the loss of contact area currently allowed in the guide specifications. The applica- tion of columns supported on rocking foundations accommodates large deformations, while there is far less damage, and can re-center after large earthquakes. Another approach is to tie a tolerable displacement of an individual deep foundation element to a movement that would cause adverse performance, excessive maintenance issues, or functionality problems with the bridge structure. Roberts et al. (2011) established a performance-based soil–structure–interaction design approach for drilled shafts. Chiou and Tsai (2014) evaluated displacement ductility of an in-ground hinging of a fixed head pile. Assessment formulas were developed for the displacement ductility capacity of a fixed-head pile in cohesion-less soils. The parameters in the formulas included the sectional over-strength ratio and curvature ductility capacity, as well as a modification factor for consider- ing soil nonlinearity. The modification factor is a function of the displacement ratio of the pile’s ultimate displacement to the effective soil yield displacement, which is constructed through a number of numerical pushover analyses. Source: Fulmer et al. (2013). Figure 11. Photograph of completed system before seismic testing showing hinge locations.

24 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design Damage Analysis As stated in NCHRP Synthesis 440, it is a fundamental need for the PBSD methodology to determine the type of damage and the likelihood that such damage will occur in the particular components of the structural system. This determination is of vital importance, as the damage sustained by a structure (and its nonstructural components) is directly relatable to the use or loss of a system after an earthquake. Therefore, there is a need to be able to reliably link structural and nonstructural response (internal forces, deformations, accelerations, and displacements) to damage. This is the realm of damage analyses, where damage is defined as discrete observable damage states (e.g., yield, spalling, longitudinal bar buckling, and bar fracture). Although the primary focus of the discussions is on structural components, similar considerations must be made for nonstructural components as well. NCHRP Synthesis 440 outlined an initial discussion on types of structural damage observed during historic earthquakes and laboratory experiments, prefaced the methods that have been developed to predict damage, identified structural details and concepts that could be used to reduce damage even in strong ground shaking, and reviewed post-event inspection tools. The new materials discussed in previous sections also apply to this discussion but are not repeated herein. Accurate damage prediction relies upon accurate definitions of performance limit states at the material level (i.e., strain limits) and the corresponding relationship between strain and displacement. Examples of recent research follow. Research by Feng et al. (2014b, 2014c) used finite element analysis validated by experimental test results to develop a model for predicting the tension strain corresponding to bar buckling. The model considers the impact of loading history on the boundary conditions of longitudinal bar restraint provided by the transverse steel. Goodnight et al. (2016a) identified strain limits to initiate bar buckling based on experimental results from 30 column tests (Equation 2). Following additional bidirectional tests on 12 columns, Equation 2 was revised to Equation 3. In addition, strain limit state equations were proposed for the compression strain in concrete to cause spiral yielding (Goodnight et al. 2017a). Goodnight et al. (2016b) also developed a new plastic hinge length model based on the data collected during those tests, which accounts for the actual curvature distribution in RC bridge columns. The revised model separates the strain penetration component from the flexural component while also recognizing that the hinge length for compression is smaller than that for tension. Brown et al. (2015) developed strain limit state (Equation 4) (tube wall local buckling) and equivalent viscous damping equations for reinforced concrete filled steel tubes (RCFSTs). The recommendations of the authors were based upon reversed cyclic tests of 12 RCFSTs of variable D/t (diameter to thickness) ratios. 0.03 700 0.1 (2)bucklingbar f E P f A s s yhe s ce g ε = + ρ − ′ 0.032 790 0.14 (3)bucklingbar f E P f A s s yhe s ce g ε = + ρ − ′ 0.021 9100 (4)tension buckling D t yε = − ≥ ε

Literature Review and Synthesis 25 where rs = reinforcement ratio, fyhe = expected yield strength of the steel tube (ksi), Es = elastic modulus of steel (ksi), P = axial load (kip), f ′ce = expected concrete strength (ksi), Ag = gross area of concrete (in.2), D = diameter of tube (in.), t = thickness of tube (in.), and ey = yield strain for steel (in./in.). Loss Analysis The PBSD combines the seismic hazard, structural, and damage analysis into a performance matrix that can be estimated into a loss metric. There are many loss metrics that can be used by, and that are important to, stakeholders and decision makers (discussed in detail in NCHRP Synthesis 440), but all these metrics can be boiled down to three main categories: deaths, dollars, and downtime. Bertero (2014) discussed earthquake lessons, in terms of loss, to be considered in both design and construction of buildings. At the beginning of 2010, two large earthquakes struck the Americas. The January 12, 2010, Haiti earthquake with a magnitude 7.0 produced about 300,000 deaths (second by the number of fatalities in world history after the 1556 Shaanxi, China earthquake). A month later, the February 27, 2010, Maule Chilean earthquake with a magnitude 8.8 (an energy release 500 times bigger than that from the Haiti earthquake) produced 500 deaths, most due to the resulting tsunami. However, the Chilean earthquake caused more than $30 billion of direct damage, left dozens of hospitals and thousands of schools nonoperational, and caused a general blackout for several hours, as well as the loss of service of essential communications facilities, crucial to take control of the chaotic after-earthquake situ- ation. Bertero (2014) compared the severity of both earthquakes and comments on their effects to life and the economy of the affected countries, as well as the features of the seismic codes or the absence of codes. An example of risk analysis with PBSD is utilized in Bensi et al. (2011), with the development of a Bayesian network (BN) methodology for performing infrastructure seismic risk assessment and providing decision support with an emphasis on immediate post-earthquake applications. A BN is a probabilistic graphical model that represents a set of random variables and their probabilistic dependencies. The proposed methodology consists of four major components: (1) a seismic demand model of ground motion intensity as a spatially distributed random field, accounting for multiple sources and including finite fault rupture and directivity effects; (2) a model for seismic performance of point-site and distributed components; (3) models of system performance as a function of component states; and (4) models of post-earthquake decision making for inspection and operation or shutdown of components. The use of the term Bayesian to describe this approach comes from the well-known Bayes rule, attributed to the 18th-century mathematician and philosopher Thomas Bayes: A B AB B B A B A( ) ( )( ) ( ) ( ) ( )= =Pr Pr Pr Pr Pr Pr (5) Pr(AB) is the probability of joint occurrence of Events A and B; Pr(A) is the marginal probability of Event A; Pr(A|B) is the conditional probability of Event A, given that Event B

26 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design has occurred; and Pr(B) is the marginal probability of Event B. The quantity Pr(B | A) is known as the likelihood of the observed Event B. Note that the probability of Event A appears on both sides of Equation 5. The Bayes rule describes how the probability of Event A changes given information gained about the occurrence of Event B. For discrete nodes, a conditional probability table is attached to each node that provides the conditional probability mass function (PMF) of the random variable represented by the node, given each of the mutually exclusive combinations of the states of its parents. For nodes without parents (e.g., X1 and X2 in Figure 12), known as root nodes, a marginal probability table is assigned. The joint PMF of all random variables X in the BN is constructed as the product of the conditional PMFs: (6) 1 p x p x Pa xi ii n∏( ) ( )( )= = Bensi et al. (2011) goes on to introduce BN models further and discusses how to incorporate BN-based seismic demand models into bridge design. The BN methodology is applied to modeling of random fields, construction of an approximate transformation matrix, and numer- ical investigation of approximation methods, including a discussion on the effect of correlation approximations on system reliability. Modeling component performance with BNs to capture seismic fragility of point-site components and distributed components, as well as modeling system performance of BNs with both qualitative and conventional methods, is explained. This reference goes on to identify efficient minimal link set (MLS), minimal cut set (MCS) formulations, optimal ordering of efficient MLS and MCS formulations, and heuristic augmen- tation that can be utilized with the BN methodology. Bensi et al. (2011) continues the PBSD process by addressing the owner decision-making process (see more discussion later in the report) and how to incorporate this model into that process. Two example problems are provided utilizing this methodology, including a California high-speed rail system that incorporates the bridge modeling into the example. Similarly, in Tehrani and Mitchell (2014), the seismic performance of 15 continuous four- span bridges with different arrangements of column heights and diameters was studied using incremental dynamic analysis (IDA). These bridges were designed using the Canadian Highway Bridge Design Code provisions (CSA 2006). The IDA procedure has been adopted by some guidelines to determine the seismic performance, collapse capacity, and fragility of buildings. Similar concepts can be used for the seismic assessment of bridges. Fragility curves can be devel- oped using the IDA results to predict the conditional probability that a certain damage state is exceeded at a given intensity measure value. Assuming that the IDA data are lognormally distributed, it is possible to develop the fragility curves at collapse (or any other damage state) by computing only the median collapse capacity and the logarithmic standard deviation of the IDA results for any given damage state. The fragility curves can then be analytically computed using Equation 7 as follows: ln ln (7)50% TOT P failure S x x S a a C( )( ) ( )= = Φ − β     where function F = cumulative normal distribution function, SCa 50% = median capacity determined from the IDA, and βTOT = total uncertainty caused by record-to-record variability, design requirements, test data, and structural modeling. Figure 12. A simple BN.

Literature Review and Synthesis 27 The seismic risk associated with exceeding different damage states in the columns, includ- ing yielding, cover spalling, bar buckling, and structural collapse (i.e., dynamic instability) was predicted. Some simplified equations were derived for Montreal, Quebec, Canada, to estimate the mean annual probability of exceeding different damage states in the columns using the IDA results. Repair and retrofit procedures are linked to loss predictions, as outlined in the FHWA’s retro- fitting manual (Buckle et al. 2006). Several chapters/articles address analysis, methodologies, effects, analytical tools, and costs for retrofit and repairs to mitigate damage or return a structure to a serviceable condition. Zimmerman et al. (2013) is one example, in which numerical techniques and seismic retrofit solutions for shear-critical reinforced concrete columns was investigated, utilizing test data of a reinforced concrete column with widely spaced transverse reinforcement. The study focused on the analysis method of nonlinear trusses and the retrofit option known as supplemental gravity columns, which is an example of how loss prediction and the analysis process are linked and should be iterated through PBSD. Organization-Specific Criteria for Bridges and Project-Specific Criteria NCHRP Synthesis 440 has two sections of criteria: organization-specific criteria for bridges and project-specific criteria. New information for both of these sections since NCHRP Synthesis 440 published is combined. The California DOT (Caltrans) Caltrans is currently updating their Seismic Design Criteria (SDC) to specify requirements to meet the performance goals for newly designed Ordinary Standard and Recovery Standard con- crete bridges. Nonstandard bridges require Project-Specific Seismic Design Criteria, in addition to the SDC, to address their nonstandard features. For both standard and nonstandard bridges, Caltrans is also categorizing their inventory in terms of Ordinary Bridges, Recovery Bridges, and Important Bridges. Some states have had issues with terms like Important or Essential, as a bridge is considered important to those that utilize each bridge. Caltrans is using these terms to correlate with loss analysis of an owner’s infrastructure and the time to reopen the bridge to support lifeline and recovery corridors. The bridge performance is also evaluated using a dual-seismic hazard; for Caltrans SDC they are listed as a Safety Evaluation Earthquake (SEE) for Ordinary Bridges. Both SEE and Functional Evaluation Earthquake (FEE) for Recovery Bridges are summarized in Table 6. Caltrans SDC revisions will also provide updates to the design parameters in Chapter 3 of the SDC and updates to both the analysis methods and displacement ductility demand values in Chapter 4 of the SDC. The adjustments to the displacement ductility demand values are revised to limit the bridge displacements beyond the initial yielding point of the ERE, specifically if a recovery standard bridge is being designed. The revisions to their SDC is an example of how PBSD is being gradually introduced as a better method of dealing with the hazards, soil–structure interaction, analysis tools, methodologies, material properties, damage states, performance, and loss. Similar revisions are being made to Seismic Design Specifications of Highway Bridges, as detailed in Japan Road Association (JRA) revisions in 2012. A synopsis of the revisions is provided in Kuwabara et al. (2013). The JRA specifications apply to Japanese road bridges and consist of five parts: Part I, Common; Part II, Steel Bridges; Part III, Concrete Bridges; Part IV, Substruc- tures; and Part V, Seismic Design. The revisions are based on improvements in terms of safety,

28 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design serviceability, and durability of bridges. Based on those lessons, design earthquake ground motions corresponding to the subduction-type earthquake were revised, and the requirements for easy and secure maintenance (inspection and repair works) for the bridges were clearly specified. JRA has clarified their performance of ERE conventionally reinforced columns for a dual-level (SPL 2 and SPL 3) seismic performance evaluation, as summarized in Table 7. The JRA 2012 revisions also address connection failures between reinforced concrete steel piles and the pile-supported spread footing to improve structural detailing and performance at the head of the piles. This is similar to research performed by the University of Washington, see Stephens et al. (2015) and Stephens et al. (2016) for both Caltrans and Washington State DOT, respectively, to evaluate capacity protecting this region and even considering the development of plastic hinges at these locations for combined hazard events or large lateral spreading and liquefaction occurrences. Caltrans also funded a study by Saini and Saiidi (2014) to address probabilistic seismic design of bridge columns using a probabilistic damage control approach and reliability analysis. Source: Caltrans. BRIDGE CATEGORY SEISMIC HAZARD EVALUATION LEVEL POST EARTHQUAKE DAMAGE STATE EXPECTED POST EARTHQUAKE SERVICE LEVEL Table 6. Caltrans draft proposed seismic design bridge performance criteria. SPL2 SPL3 Note: SPL1: Fully operational is required. Limit state of bridge is serviceability limit state. Negligible structural damage and nonstructural damage are allowed. Table 7. Seismic performance of bridge and limit states of conventionally reinforced concrete bridge column.

Literature Review and Synthesis 29 The probabilistic damage control approach uses the extent of lateral displacement nonlinearity defined by Damage Index (DI) to measure the performance of bridge columns. DI is a measure of damage from the lower measure of zero damage to the ultimate measure of a collapse mecha- nism for an element that has been subjected to base excitations. The performance objective was defined based on predefined apparent Damage States (DS), and the DS were correlated to DIs based on a previous study at the University of Nevada, Reno (Figure 13) (Vosooghi and Saiidi 2010). A statistical analysis of the demand damage index (DIL) was performed to develop fragility curves (load model) and to determine the reliability index for each DS. The results of the reliability analyses were analyzed, and a direct probabilistic damage control approach was developed to calibrate design DI to obtain a desired reliability index against failure. The calculated reliability indices and fragility curves showed that the proposed method could be effectively used in seismic design of new bridges, as well as in seismic assessment of existing bridges. The DS and DI are summarized with performance levels defined by Caltrans in Table 8, which shows the correlation between DS and DI. Figure 14 shows a fragility curve using lognormal distribution. Figure 15 shows both the fragility curves (upper two graphs) and reliability indices (lower two graphs) for four column bents (FCBs), with 4-foot diameter columns that are 30 feet in length in Site D for both the 1000 year and 2500 year seismic events. Note: O-ST = ordinary standard bridge, O-NST = ordinary nonstandard bridge, Rec. = recovery bridge, Imp. = important bridge, and NA = not applicable. Damage State (DS) Service to Public Service to Emergency Emergency Repair Design Damage Index (DI) Earthquake Levels (Years) Table 8. Design performance levels. DI P (D I { D S) Figure 13. Correlation between DS and DI.

30 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design Figure 14. Fragility curve. 100% 80% 60% 40% 20% 0% 0.00 0.20 0.40 0.60 0.80 1.00 P (D I L ) DIL 4.0 3.0 2.0 1.0 0.0 R el ia bi lit y In de x | D S DS3 DS4 DS5 DS6 Damage State (DS) 6.0 5.0 4.0 3.0 2.0 1.0 0.0 R el ia bi lit y In de x | D S DS3 DS4 DS5 DS6 Damage State (DS) (a) (b) (d)(c) 0.00 0.20 0.40 0.60 0.80 1.00 DIL 100% 80% 60% 40% 20% 0% P (D I L ) Figure 15. Fragility curves and reliability indices for FCBs with 4-foot columns in Site D. The Oregon DOT The Oregon DOT is developing a global plan for addressing resiliency in order to improve recovery for the next Cascadia Earthquake and Tsunami, using PBSD in terms of applying applicable hazards, identifying critical services, developing a comprehensive assessment of structures and systems, and updating public policies. The resilience goals are similar to those discussed at the beginning of this chapter, with the following statement: Oregon citizens will not only be protected from life-threatening physical harm, but because of risk reduction measures and pre-disaster planning, communities will recover more quickly and with less continuing vulnerability following a Cascadia subduction zone earthquake and tsunami.

Literature Review and Synthesis 31 Research has shown that the next great (magnitude 9.0) Cascadia subduction zone earth- quake is pending, as shown in Figure 16. This comparison of historical subduction zone earthquakes in northern California, Oregon, and Washington covers 10000 years of seismic history. The evidence of a pending event has made decision makers and the public take notice and put forth resources to develop strategies revolving around PBSD. Oregon’s performance-based features are modified from NCHRP Synthesis 440 to account for a third hazard condition: Cascadia Subduction Zone Earthquake (CSZE) in Oregon DOT’s Bridge Design and Drafting Manual—Section 1, Design (Oregon DOT 2016a; see also Oregon DOT 2016b). Design of new bridges on and west of US 97 references two levels of perfor- mance criteria: life safety and operational. Design of new bridges east of US 97 requires life safety criteria only. Seismic design criteria for life safety and operational criteria are described as follows. • “Life Safety” Criteria: Design all bridges for a 1,000-year return period earthquake (7 percent prob- ability of exceedance in 75 years) to meet the “Life Safety” criteria using the 2014 USGS Hazard Maps. The probabilistic hazard maps for an average return period of 1,000 years and 500 years are available at ODOT Bridge Section website, but not available on USGS website. To satisfy the “Life Safety” criteria, use Response Modification Factors from LRFD Table 3.10.7.1-1 using an importance category of “other.” • “Operational” Criteria: Design all bridges on and west of US 97 to remain “Operational” after a full rupture of Cascadia Subduction Zone Earthquake (CSZE). The full-rupture CSZE hazard maps are available at the ODOT Bridge Section website. To satisfy the “Operational” criteria, use Response Modification Factors from LRFD Table 3.10.7.1-1 using an importance category of “essential.” When requested in writing by a local agency, the “Operational” criteria for local bridges may be waived. The CSZE is a deterministic event, and a deterministic design response spectrum must be generated. To allow for consistency and efficiency in design for the CSZE, an application for generating the design response spectra has been developed by Portland State University (Nako et al. 2009). AASHTO guide specifications values for Table 3.4.2.3-1 are modified into two tables for (1) values of Site Factor, Fpga, at zero-period on the acceleration spectrum and (2) values of Site Factor, Fa, for short-period range of acceleration spectrum. Table 3.4.2.3-2 is replaced with values of Site Factor, Fv, for long-period range of acceleration spectrum. For seismic retrofit projects, the lower level ground motion is modified to be the CSZE with full rupture, as seen in Table 9. Performance levels, including performance level zero (PL0), are specified based on bridge importance and the anticipated service life (ASL) category required. Source: OSSPAC (2013). Figure 16. Cascadia earthquake timeline.

32 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design The South Carolina DOT South Carolina Department of Transportation (South Carolina DOT) has updated its geo- technical design manual (South Carolina DOT 2019). Chapters 12, 13, and 14 for geo technical seismic analysis, hazard, and design, respectively, have been updated to current practices and research, including incorporation of PBSD hazard prediction. South Carolina DOT is also updating their site coefficients to be more appropriate for South Carolina’s geologic and seismic conditions; see Andrus et al. (2014). Note that with the revisions, South Carolina DOT issued a design memorandum in November 2015 that revised the substructure unit quantitative damage criteria (maximum ductility demand) table (Table 7.1 of the SCDOT Seismic Design Specifications for Highway Bridges). See Table 10. The Utah DOT The Utah DOT and Brigham Young University (see Franke et al. 2014a, 2014b, 2015a, 2015b, 2015c, 2016) are researching the ability for engineers to apply the benefits of the full performance- based probabilistic earthquake analysis without requiring specialized software, training, or education. There is an emphasis on differences between deterministic and performance-based procedures for assessing liquefaction hazards and how the output can vary significantly with these two methodologies, especially in areas of low seismicity. Guidance is provided regarding when to use each of the two methodologies and how to bind the analysis effort. Additionally, a simplified performance-based procedure for assessment of liquefaction triggering using liquefaction loading maps was developed with this research. The components of this tool, as well as step-by-step procedures for the liquefaction initiation and lateral spread displacement models, are provided. The tool incorporates the simplified performance-based procedures determined with this research. National Highway Institute Marsh et al. (2014) referenced a manual for the National Highway Institute’s training course for engineers to understand displacement-based LRFD seismic analysis and design of bridges, which is offered through state agencies and open to industry engineers and geotechnical engi- neers. This course helps designers understand the principles behind both force-based AASHTO (AASHTO 2014) and displacement-based AASHTO (AASHTO 2011) methodologies, including a deeper understanding of what performance means in a seismic event. Other similar courses are also being offered to industry and are improving the understanding of practicing engineers. Federal Emergency Management Agency The Federal Emergency Management Agency (FEMA) has developed a series of design guidelines for seismic performance assessment of buildings and three of the five documents EARTHQUAKE GROUND MOTION BRIDGE IMPORTANCE and SERVICE LIFE CATEGORY Table 9. Modifications to minimum performance levels for retrofitted bridges.

Literature Review and Synthesis 33 are referenced in FEMA (2012a, 2012b, 2012c). A step-by-step methodology and explanation of implementation are provided for an intensity-based assessment and for a time-based assess- ment. The process of identifying and developing appropriate fragility curves is demonstrated. A software program called Performance Assessment Calculation Tool has also been developed with a user manual that is included in the FEMA documents to help engineers apply PBSD to the building industry. Japan Road Association The Japan Road Association (JRA) Design Specifications have been revised based on the performance-based design code concept in response to the international harmonization of design codes and the flexible employment of new structures and new construction methods. Figure 17 shows the code structure for seismic design using the JRA Design Specifications. The performance matrix is based on a two-level ground motion (Earthquakes 1 and 2), with the first one based on an interpolate-type earthquake and magnitude of around 8, and the second one with a magnitude of around 7 with a short distance to the structure. Kuwabara et al. (2013) outlined the incremental revisions from the JRA Design Specif i- cations between 2002 and 2012. These revisions include, but are not limited to, the ductility design method of reinforced concrete bridges, plastic hinge length equation, evaluation of hollow columns, and the introduction of high-strength steel reinforcement. Following the 2016 earthquake in Kumamoto, Japan, a new version of the JRA Design Specifications is in the works. Note: Analysis for FEE is not required for OC III bridges. Source: South Carolina DOT (2015). Design Earthquake Operational Classification (OC)Bridge Systems Table 10. South Carolina DOT substructure unit quantitative damage criteria (maximum ductility demand ld).

34 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design Identification of Knowledge Gaps The resources to develop guide specifications for PBSD are improving with examples such as the upcoming Seismic Design Criteria, Version 2 from Caltrans, which will address aspects of PBSD and the building industry’s efforts to develop practices in PBSD and tools for engineers and owners to collaborate on solutions based on performance criteria and expectations. There is still a perception that the bridge industry could better predict likely performance in large, damaging earthquakes than is being done at the present, and there are still gaps in that knowledge base that need to be closed. Most of the knowledge gaps listed in Marsh and Stringer (2013) are still applicable today; see Table 11. The technology readiness levels represent what has been developed and used; what research is done, ongoing, and being discussed; and what only exists in concept. Knowledge gaps certainly exist in all facets of PBSD; however, other key knowledge gaps beyond those listed in NCHRP Synthesis 440 (Marsh and Stringer 2013) that should be closed in order to improve the implementation of PBSD are covered. Objectives of Codes Mandated Specifications Overall Goals Functional Requirements (Basic Requirements) Performance Requirement Level Verification Methods and Acceptable Solutions Can be Modified or May be Selected with Necessary Verifications Importance, Loads, Design Ground Motion, Limit States Principles of Performance Verification Verifications of Seismic Performances (Static and Dynamic Verifications) Evaluation of Limit States of Members (RC and Steel Columns, Bearings, Foundations and Superstructure) Unseating Prevention Systems Principles of Seismic Design Figure 17. Code structure for seismic design using JRA design specifications. TRL Description 0-25 25-50 50-75 75-100 1 PBSD concept exists 2 Seismic hazard deployable 3 Structural analysis deployable 4 Damage analysis deployable 5 Loss analysis deployable 6 Owners willing and skilled in PBSD 7 Design guidelines 8 Demonstration projects 9 Proven effectiveness in earthquake Technology Readiness Level (TRL) % of Development Complete Table 11. Technology readiness levels for PBSD.

Literature Review and Synthesis 35 Gaps related to structural analysis can include minimum and expected properties for reinforcing greater than Grade 80, stainless steel, and other materials that can improve serviceability and in some conditions performance. Oregon DOT has been using stainless steel in their bridges located along the coastline and other highly corrosive environments to extend the service life of the bridge; however, many of these locations are also prone to large CSZE and the use of these materials in earthquake resisting elements is still being developed. In the State of Washington’s resiliency plan, outlined in Washington State Emergency Management Council–Seismic Safety Committee (2012), what is missing is a link between damage levels and return to service. This is a knowledge gap given what we know structurally and what this report is suggesting as a desired goal for post-earthquake recovery. Gaps related to decision makers can include bridge collapse. It is not intended that the PBSD guide specifications will address tsunami events, but the JRA specifications do address tsunami as well as landslide effects. Figures 18 and 19 are examples of these other types of failure systems and show the collapse of bridges caused by effects other than ground motion (Kuwabara et al. 2013). The decision to combine these types of effects with a seismic hazard, even combining liquefaction, down drag, and lateral spreading effects, needs additional clarification and is currently left up to the owner to assess implications of probability, safety, and cost ramifications. Liang and Lee (2013) summarized that in order to update the extreme event design limit states in the AASHTO 2014, combinations of all nonextreme and extreme loads need to be formulated on the same probability-based platform. Accounting for more than one-time variable load creates a complex situation, in which all of the possible load combinations, even many that are not needed for the purpose of bridge design, have to be determined. A formulation of a criterion to determine if a specific term is necessary to be included or rejected is described, and a comparison of the value of a given failure probability to the total pre-set permissible design failure probability can be chosen as this criterion. Figure 18. Collapse of bridge due to landslide. (Note: Reprinted courtesy of the National Institute of Standards and Technology, U.S. Department of Commerce. Not copyrightable in the United States). Source: Kuwabara et al. (2013).

36 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design While the seismic hazard definition was once thought to be relatively well understood, there is a growing knowledge gap related to the effect of rotation angle on intensity of ground motions and how the use of a geometric mean of the motions, or other methods of including the effect of rotation angle (RotDxx), should be incorporated into seismic design. This issue is not specific to PBSD; like all seismic design methods, PBSD is reliant on a full understanding of the hazard definition for proper implementation. The knowledge gaps identified in NCHRP Synthesis 440 are still applicable. Many of these knowledge gaps will become evident to both engineers and decision makers as the PBSD guidelines are developed. Overall, the baseline information to develop PBSD guide specifica- tions are in place. Industry’s end goal of understanding the relationship between risk-based decision making and design decisions and methodologies to meet performance goals is going to be an iterative process. Figure 19. Collapse of bridge due to tsunami. (Note: Reprinted courtesy of the National Institute of Standards and Technology, U.S. Department of Commerce. Not copyrightable in the United States). Source: Kuwabara et al. (2013).

Performance-based seismic design (PBSD) for infrastructure in the United States is a developing field, with new research, design, and repair technologies; definitions; and methodologies being advanced every year.

The TRB National Cooperative Highway Research Program's NCHRP Research Report 949: Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design presents a methodology to analyze and determine the seismic capacity requirements of bridge elements expressed in terms of service and damage levels of bridges under a seismic hazard. The methodology is presented as proposed AASHTO guidelines for performance-based seismic bridge design with ground motion maps and detailed design examples illustrating the application of the proposed guidelines and maps.

Supplemental materials to the report include an Appendix A - SDOF Column Investigation Sample Calculations and Results and Appendix B - Hazard Comparison.

READ FREE ONLINE

Welcome to OpenBook!

You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

Do you want to take a quick tour of the OpenBook's features?

Show this book's table of contents , where you can jump to any chapter by name.

...or use these buttons to go back to the previous chapter or skip to the next one.

Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

To search the entire text of this book, type in your search term here and press Enter .

Share a link to this book page on your preferred social network or via email.

View our suggested citation for this chapter.

Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

Get Email Updates

Do you enjoy reading reports from the Academies online for free ? Sign up for email notifications and we'll let you know about new publications in your areas of interest when they're released.

Should I do a synthesis (i.e. literature review)?

• Questions & Quandaries
• Published: 18 April 2024

Cite this article

• H. Carrie Chen 1 ,
• Ayelet Kuper 2 , 3 , 4 ,
• Jennifer Cleland 5 &
• Patricia O’Sullivan 6  

107 Accesses

1 Altmetric

Explore all metrics

This column is intended to address the kinds of knotty problems and dilemmas with which many scholars grapple in studying health professions education. In this article, the authors address the question of whether one should conduct a literature review or knowledge synthesis, considering the why, when, and how, as well as its potential pitfalls. The goal is to guide supervisors and students who are considering whether to embark on a literature review in education research.

Avoid common mistakes on your manuscript.

Two junior colleagues come to you to ask your advice about carrying out a literature review on a particular topic. “Should they?” immediately pops into your mind, followed closely by, if yes, then what kind of literature review is appropriate? Our experience is that colleagues often come to suggest a literature review to “kick start” their research (in fact, some academic programs require them as part of degree requirements), without a full understanding of the work involved, the different types of literature review, and what type of literature review might be most suitable for their research question. In this Questions and Quandaries, we address the question of literature reviews in education research, considering the why, when, and how, as well as potential pitfalls.

First, what is meant by literature review? The term literature review has been used to refer to both a review of the literature and a knowledge synthesis (Maggio et al., 2018 ; Siddaway et al., 2019 ). For our purposes, we employ the term as commonly used to refer to a knowledge synthesis , which is a formal comprehensive review of the existing body of literature on a topic. It is a research approach that critically integrates and synthesizes available evidence from multiple studies to provide insight and allow the drawing of conclusions. It is an example of Boyer’s scholarship of integration (Boyer, 1990 ). In contrast, a review of the literature is a relatively casual and expedient method for attaining a general overview of the state of knowledge on a given topic to make the argument that a new study is needed. In this interpretation, a literature review serves as a key starting point for anyone conducting research by identifying gaps in the literature, informing the study question, and situating one’s study in the field.

Whether a formal knowledge synthesis should be done depends on if a review is needed and what the rationale is for the review. The first question to consider is whether a literature review already exists. If no, is there enough literature published on the topic to warrant a review? If yes, does the previous review need updating? How long has it been since the last review and has the literature expanded so much or are there important new studies that need integrating to justify an updated review? Or were there flaws in the previous review that one intends to address with a new review? Or does one intend to address a different question than the focus of the previous review?

If the knowledge synthesis is to be done, it should be driven by a research question. What is the research question? Can it be answered by a review? What is the purpose of the synthesis? There are two main purposes for knowledge synthesis– knowledge support and decision support. Knowledge support summarizes the evidence while decision support takes additional analytical steps to allow for decision-making in particular contexts (Mays et al., 2005 ).

If the purpose is to provide knowledge support, then the question is how or what will the knowledge synthesis add to the literature? Will it establish the state of knowledge in an area, identify gaps in the literature/knowledge base, and/or map opportunities for future research? Cornett et al., performed a scoping review of the literature on professional identity, focusing on how professional identity is described, why the studies where done, and what constructs of identity were used. Their findings advanced understanding of the state of knowledge by indicating that professional identity studies were driven primarily by the desire to examine the impact of political, social and healthcare reforms and advances, and that the various constructs of professional identity across the literature could be categorized into five themes (Cornett et al., 2023 ).

If, on the other hand, the purpose of the knowledge synthesis is to provide decision support, for whom will the synthesis be relevant and how will it improve practice? Will the synthesis result in tools such as guidelines or recommendations for practitioners and policymakers? An example of a knowledge synthesis for decision support is a systematic review conducted by Spencer and colleagues to examine the validity evidence for use of the Ottawa Surgical Competency Operating Room Evaluation (OSCORE) assessment tool. The authors summarized their findings with recommendations for educational practice– namely supporting the use of the OSCORE for in-the-moment entrustment decisions by frontline supervisors in surgical fields but cautioning about the limited evidence for support of its use in summative promotions decisions or non-surgical contexts (Spencer et al., 2022 ).

If a knowledge synthesis is indeed appropriate, its methodology should be informed by its research question and purpose. We do not have the space to discuss the various types of knowledge synthesis except to say that several types have been described in the literature. The five most common types in health professions education are narrative reviews, systematic reviews, umbrella reviews (meta-syntheses), scoping reviews, and realist reviews (Maggio et al., 2018 ). These represent different epistemologies, serve different review purposes, use different methods, and result in different review outcomes (Gordon, 2016 ).

Each type of review lends itself best to answering a certain type of research question. For instance, narrative reviews generally describe what is known about a topic without necessarily answering a specific empirical question (Maggio et al., 2018 ). A recent example of a narrative review focused on schoolwide wellbeing programs, describing what is known about the key characteristics and mediating factors that influence student support and identifying critical tensions around confidentiality that could make or break programs (Tan et al., 2023 ). Umbrella reviews, on the other hand, synthesize evidence from multiple reviews or meta-analyses and can illuminate agreement, inconsistencies, or evolution of evidence on a topic. For example, an umbrella review on problem-based learning highlighted the shift in research focus over time from does it work, to how does it work, to how does it work in different contexts, and pointed to directions for new research (Hung et al., 2019 ).

Practical questions for those considering a literature review include whether one has the time required and an appropriate team to conduct a high-quality knowledge synthesis. Regardless of the type of knowledge synthesis and use of quantitative or qualitative methods, all require rigorous and clear methods that allow for reproducibility. This can take time, up to 12–18 months. A high-quality knowledge synthesis also requires a team whose members have expertise not only in the content matter, but also in knowledge synthesis methodology and in literature searches (i.e. a librarian). A team with multiple reviewers with a variety of perspectives can also help manage the volume of large reviews, minimize potential biases, and strengthen the critical analysis.

Finally, a pitfall one should be careful to avoid is merely summarizing everything in the literature without critical evaluation and integration of the information. A knowledge synthesis that merely bean counts or presents a collection of unconnected information that has not been reflected upon or critically analyzed does not truly advance knowledge or decision-making. Rather, it leads us back to our original question of whether it should have been done in the first place.

Boyer, E. L. (1990). Scholarship reconsidered: Priorities of the professoriate (pp. 18–21). Princeton University Press.

Cornett, M., Palermo, C., & Ash, S. (2023). Professional identity research in the health professions—a scoping review. Advances in Health Sciences Education , 28 (2), 589–642.

Article   Google Scholar  

Gordon, M. (2016). Are we talking the same paradigm? Considering methodological choices in health education systematic review. Medical Teacher , 38 (7), 746–750.

Hung, W., Dolmans, D. H. J. M., & van Merrienboer, J. J. G. (2019). A review to identify key perspectives in PBL meta-analyses and reviews: Trends, gaps and future research directions. Advances in Health Sciences Education , 24 , 943–957.

Maggio, L. A., Thomas, A., & Durning, S. J. (2018). Knowledge synthesis. In T. Swanwick, K. Forrest, & B. C. O’Brien (Eds.), Understanding Medical Education: Evidence, theory, and practice (pp. 457–469). Wiley.

Mays, N., Pope, C., & Popay, J. (2005). Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. Journal of Health Services Research & Policy , 10 (1_suppl), 6–20.

Siddaway, A. P., Wood, A. M., & Hedges, L. V. (2019). How to do a systematic review: A best practice guide for conducting and reporting narrative reviews, meta-analyses, and meta-syntheses. Annual Review of Psychology , 70 , 747–770.

Spencer, M., Sherbino, J., & Hatala, R. (2022). Examining the validity argument for the Ottawa Surgical Competency operating room evaluation (OSCORE): A systematic review and narrative synthesis. Advances in Health Sciences Education , 27 , 659–689.

Tan, E., Frambach, J., Driessen, E., & Cleland, J. (2023). Opening the black box of school-wide student wellbeing programmes: A critical narrative review informed by activity theory. Advances in Health Sciences Education . https://doi.org/10.1007/s10459-023-10261-8 . Epub ahead of print 02 July 2023.

Download references

Author information

Authors and affiliations.

Georgetown University School of Medicine, Washington, DC, USA

H. Carrie Chen

Wilson Centre for Research in Education, University Health Network, University of Toronto, Toronto, Canada

Ayelet Kuper

Division of General Internal Medicine, Sunnybrook Health Sciences Center, Toronto, Canada

Department of Medicine, University of Toronto, Toronto, Canada

Lee Kong Chian School of Medicine, Nanyang Technological University Singapore, Singapore, Singapore

Jennifer Cleland

University of California San Francisco School of Medicine, San Francisco, CA, USA

Patricia O’Sullivan

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to H. Carrie Chen .

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Chen, H.C., Kuper, A., Cleland, J. et al. Should I do a synthesis (i.e. literature review)?. Adv in Health Sci Educ (2024). https://doi.org/10.1007/s10459-024-10335-1

Download citation

Published : 18 April 2024

DOI : https://doi.org/10.1007/s10459-024-10335-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Find a journal
• Publish with us
• Track your research

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

CHAPTER 2 REVIEW OF RELATED LITERATURE AND STUDIES

Related Papers

Ana Liza Sigue

Motoky Hayakawa

Rene E Ofreneo

... Rene E. Ofreneo University of the Philippines School of Labor and Industrial Relations ... Erickson, Christopher L.; Kuruvilla, Sarosh ; Ofreneo, Rene E.; and Ortiz, Maria Asuncion , &amp;quot;Recent Developments in Employment Relations in the Philippines&amp;quot; (2001). ...

Lucita Lazo

This book begins by looking at the status of women in Filipino society and their place in the general socio-economic situation. It continues with sections on education and training in the Philippines and work and training. The next section reviews the constraints to women’s participation in training. In the summary the author gives a general overview of the situation of women and opportunities for work and training in the Philippines and offers some practical suggestions for the enhancement of women’s training and development.

EducationInvestor Global

Tony Mitchener

Rosalyn Eder

In this article, I examine the role of CHED and the Technical Panels (TPs) in the “production” of the globally competitive Filipina/o worker. For this paper, I draw on relevant literature on the topic and take nurse education, which is rooted in the colonial system established during the US-American occupation, as an example of how CHED and the TPs could be more linked to labor migration. I use the colonial difference - a space that offers critical insights and interpretation - to illustrate how coloniality remains hidden under the cloak of modernity. Link to the article: http://dx.doi.org/10.1080/00131946.2016.1214913

Asia Pacific Journal of Management and Sustainable Development ISSN 2782-8557(Print)

Ryan O Tayco , Pio Supat

This study aims to determine the employability of the Negros Oriental State University graduates from 2016 to 2020. Employability is measured using different dimensions-from the graduates' side including the perspectives of the employers. A total of 1, 056 NORSU graduates and 68 employers locally and abroad answered the questionnaire through online and offline survey methods. Basic statistics were used and simple linear regression was also used to estimate the relationship between manifestations of respondents in NORSU VMGOs and the job performance as perceived by the employers. Most of the respondents in the study are presently employed and work locally. Many of them stay and accept the job because of the salaries and benefits they received, a career challenge, and related to the course they have taken in college. The study shows that the curriculum used and competencies learned by the NORSU graduates are relevant to their job. Competencies such as communication skills, human relations skills, critical thinking skills, and problem-solving skills are found to be useful by the respondents. It is found that the manifestation of the respondents is very high and homogenous. The same can be said with job performance as perceived by employers in terms of attitudes and values, skills and competencies, and knowledge. Furthermore, job performance and the manifestation of NORSU VMGOs have a significant relationship. That is, those respondents who have higher job performance in terms of attitude and values, skills and competencies, and knowledge have higher manifestations of NORSU VMGOs.

Annals of Tropical Research

Pedro Armenia

Ezekiel Succor

RELATED PAPERS

Châu Bùi Thị Ngọc

Physical Review C

Biointerphases

Bálint Szabó

Arch. Inst. Cardiol. …

Gustavo Miranda

TAP CHI SINH HOC

Dalton Trans.

peter goodrich

Environmental Health Perspectives

Ilan Kirsch

Nineteenth-Century Music Review

Michelle Meinhart

Latin American Applied Research - An international journal

Muhammad Mustafa

Akdeniz Spor Bilimleri Dergisi

emre şimşek

International journal of pharmaceutics

Martina Marchetti-Deschmann

Helen Thein

ACM Transactions on Reconfigurable Technology and Systems

Pete Sedcole

Ali Aljaafreh

fatimatus solihah

Edward Melhuish

La pintura romana en Hispania. Del estudio de campo a su puesta en valor

Gonzalo Castillo Alcántara

Andy Ramirez

Historia Actual Online

Marcio Orozco

SEMEION MED

Souad Eddarouich

Revista ION

ALEJANDRO ACOSTA CARDENAS

Verbum Vitae

Janusz Lemanski

Risk Analysis

William Grenzebach

Acta Veterinaria-beograd

Tamara Popović

Global Ecology and Conservation

Agnieszka Piernik

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

HYPOTHESIS AND THEORY article

This article is part of the research topic.

Using Case Study and Narrative Pedagogy to Guide Students Through the Process of Science

Molecular Storytelling: A Conceptual Framework for Teaching and Learning with Molecular Case Studies Provisionally Accepted

• 1 School of Interdisciplinary Arts and Sciences, University of Washington Bothell, United States
• 2 Institute for Quantitative Biomedicine, Rutgers, The State University of New Jersey, United States
• 3 Research Collaboratory for Structural Bioinformatics Protein Data Bank, Rutgers, The State University of New Jersey,, United States

The final, formatted version of the article will be published soon.

Molecular case studies (MCSs) provide educational opportunities to explore biomolecular structure and function using data from public bioinformatics resources. The conceptual basis for the design of MCSs has yet to be fully discussed in the literature, so we present molecular storytelling as a conceptual framework for teaching with case studies. Whether the case study aims to understand the biology of a specific disease and design its treatments or track the evolution of a biosynthetic pathway, vast amounts of structural and functional data, freely available in public bioinformatics resources, can facilitate rich explorations in atomic detail. To help biology and chemistry educators use these resources for instruction, a community of scholars collaborated to create the Molecular CaseNet. This community uses storytelling to explore biomolecular structure and function while teaching biology and chemistry. In this article, we define the structure of an MCS and present an example. Then, we articulate the evolution of a conceptual framework for developing and using MCSs. Finally, we related our framework to the development of technological, pedagogical, and content knowledge (TPCK) for educators in the Molecular CaseNet. The report conceptualizes an interdisciplinary framework for teaching about the molecular world and informs lesson design and education research.

Keywords: Molecular education, Case studies, Technological pedagogical and content knowledge (TPCK), Molecular structure and function, molecular visualization, Bioinformatics education, conceptual modeling

Received: 31 Jan 2024; Accepted: 23 Apr 2024.

Copyright: © 2024 Trujillo and Dutta. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

* Correspondence: Prof. Caleb M. Trujillo, University of Washington Bothell, School of Interdisciplinary Arts and Sciences, Bothell, United States

People also looked at