critical synthesis literature review

Literature Syntheis 101

How To Synthesise The Existing Research (With Examples)

By: Derek Jansen (MBA) | Expert Reviewer: Eunice Rautenbach (DTech) | August 2023

One of the most common mistakes that students make when writing a literature review is that they err on the side of describing the existing literature rather than providing a critical synthesis of it. In this post, we’ll unpack what exactly synthesis means and show you how to craft a strong literature synthesis using practical examples.

This post is based on our popular online course, Literature Review Bootcamp . In the course, we walk you through the full process of developing a literature review, step by step. If it’s your first time writing a literature review, you definitely want to use this link to get 50% off the course (limited-time offer).

Overview: Literature Synthesis

• What exactly does “synthesis” mean?
• Aspect 1: Agreement
• Aspect 2: Disagreement
• Aspect 3: Key theories
• Aspect 4: Contexts
• Aspect 5: Methodologies
• Bringing it all together

What does “synthesis” actually mean?

As a starting point, let’s quickly define what exactly we mean when we use the term “synthesis” within the context of a literature review.

Simply put, literature synthesis means going beyond just describing what everyone has said and found. Instead, synthesis is about bringing together all the information from various sources to present a cohesive assessment of the current state of knowledge in relation to your study’s research aims and questions .

Put another way, a good synthesis tells the reader exactly where the current research is “at” in terms of the topic you’re interested in – specifically, what’s known , what’s not , and where there’s a need for more research .

So, how do you go about doing this?

Well, there’s no “one right way” when it comes to literature synthesis, but we’ve found that it’s particularly useful to ask yourself five key questions when you’re working on your literature review. Having done so,  you can then address them more articulately within your actual write up. So, let’s take a look at each of these questions.

1. Points Of Agreement

The first question that you need to ask yourself is: “Overall, what things seem to be agreed upon by the vast majority of the literature?”

For example, if your research aim is to identify which factors contribute toward job satisfaction, you’ll need to identify which factors are broadly agreed upon and “settled” within the literature. Naturally, there may at times be some lone contrarian that has a radical viewpoint , but, provided that the vast majority of researchers are in agreement, you can put these random outliers to the side. That is, of course, unless your research aims to explore a contrarian viewpoint and there’s a clear justification for doing so. 

Identifying what’s broadly agreed upon is an essential starting point for synthesising the literature, because you generally don’t want (or need) to reinvent the wheel or run down a road investigating something that is already well established . So, addressing this question first lays a foundation of “settled” knowledge.

Need a helping hand?

2. Points Of Disagreement

Related to the previous point, but on the other end of the spectrum, is the equally important question: “Where do the disagreements lie?” .

In other words, which things are not well agreed upon by current researchers? It’s important to clarify here that by disagreement, we don’t mean that researchers are (necessarily) fighting over it – just that there are relatively mixed findings within the empirical research , with no firm consensus amongst researchers.

This is a really important question to address as these “disagreements” will often set the stage for the research gap(s). In other words, they provide clues regarding potential opportunities for further research, which your study can then (hopefully) contribute toward filling. If you’re not familiar with the concept of a research gap, be sure to check out our explainer video covering exactly that .

3. Key Theories

The next question you need to ask yourself is: “Which key theories seem to be coming up repeatedly?” .

Within most research spaces, you’ll find that you keep running into a handful of key theories that are referred to over and over again. Apart from identifying these theories, you’ll also need to think about how they’re connected to each other. Specifically, you need to ask yourself:

• Are they all covering the same ground or do they have different focal points  or underlying assumptions ?
• Do some of them feed into each other and if so, is there an opportunity to integrate them into a more cohesive theory?
• Do some of them pull in different directions ? If so, why might this be?
• Do all of the theories define the key concepts and variables in the same way, or is there some disconnect? If so, what’s the impact of this ?

Simply put, you’ll need to pay careful attention to the key theories in your research area, as they will need to feature within your theoretical framework , which will form a critical component within your final literature review. This will set the foundation for your entire study, so it’s essential that you be critical in this area of your literature synthesis.

If this sounds a bit fluffy, don’t worry. We deep dive into the theoretical framework (as well as the conceptual framework) and look at practical examples in Literature Review Bootcamp . If you’d like to learn more, take advantage of our limited-time offer to get 60% off the standard price.

4. Contexts

The next question that you need to address in your literature synthesis is an important one, and that is: “Which contexts have (and have not) been covered by the existing research?” .

For example, sticking with our earlier hypothetical topic (factors that impact job satisfaction), you may find that most of the research has focused on white-collar , management-level staff within a primarily Western context, but little has been done on blue-collar workers in an Eastern context. Given the significant socio-cultural differences between these two groups, this is an important observation, as it could present a contextual research gap .

In practical terms, this means that you’ll need to carefully assess the context of each piece of literature that you’re engaging with, especially the empirical research (i.e., studies that have collected and analysed real-world data). Ideally, you should keep notes regarding the context of each study in some sort of catalogue or sheet, so that you can easily make sense of this before you start the writing phase. If you’d like, our free literature catalogue worksheet is a great tool for this task.

5. Methodological Approaches

Last but certainly not least, you need to ask yourself the question: “What types of research methodologies have (and haven’t) been used?”

For example, you might find that most studies have approached the topic using qualitative methods such as interviews and thematic analysis. Alternatively, you might find that most studies have used quantitative methods such as online surveys and statistical analysis.

But why does this matter?

Well, it can run in one of two potential directions . If you find that the vast majority of studies use a specific methodological approach, this could provide you with a firm foundation on which to base your own study’s methodology . In other words, you can use the methodologies of similar studies to inform (and justify) your own study’s research design .

On the other hand, you might argue that the lack of diverse methodological approaches presents a research gap , and therefore your study could contribute toward filling that gap by taking a different approach. For example, taking a qualitative approach to a research area that is typically approached quantitatively. Of course, if you’re going to go against the methodological grain, you’ll need to provide a strong justification for why your proposed approach makes sense. Nevertheless, it is something worth at least considering.

Regardless of which route you opt for, you need to pay careful attention to the methodologies used in the relevant studies and provide at least some discussion about this in your write-up. Again, it’s useful to keep track of this on some sort of spreadsheet or catalogue as you digest each article, so consider grabbing a copy of our free literature catalogue if you don’t have anything in place.

Bringing It All Together

Alright, so we’ve looked at five important questions that you need to ask (and answer) to help you develop a strong synthesis within your literature review.  To recap, these are:

• Which things are broadly agreed upon within the current research?
• Which things are the subject of disagreement (or at least, present mixed findings)?
• Which theories seem to be central to your research topic and how do they relate or compare to each other?
• Which contexts have (and haven’t) been covered?
• Which methodological approaches are most common?

Importantly, you’re not just asking yourself these questions for the sake of asking them – they’re not just a reflection exercise. You need to weave your answers to them into your actual literature review when you write it up. How exactly you do this will vary from project to project depending on the structure you opt for, but you’ll still need to address them within your literature review, whichever route you go.

The best approach is to spend some time actually writing out your answers to these questions, as opposed to just thinking about them in your head. Putting your thoughts onto paper really helps you flesh out your thinking . As you do this, don’t just write down the answers – instead, think about what they mean in terms of the research gap you’ll present , as well as the methodological approach you’ll take . Your literature synthesis needs to lay the groundwork for these two things, so it’s essential that you link all of it together in your mind, and of course, on paper.

Psst… there’s more!

This post is an extract from our bestselling Udemy Course, Literature Review Bootcamp . If you want to work smart, you don't want to miss this .

You Might Also Like:

excellent , thank you

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

• University of Oregon Libraries
• Research Guides

How to Write a Literature Review

• 6. Synthesize
• Literature Reviews: A Recap
• Reading Journal Articles
• Does it Describe a Literature Review?
• 1. Identify the Question
• 2. Review Discipline Styles
• Searching Article Databases
• Finding Full-Text of an Article
• Citation Chaining
• When to Stop Searching
• 4. Manage Your References
• 5. Critically Analyze and Evaluate

Synthesis Visualization

Synthesis matrix example.

• 7. Write a Literature Review

• Synthesis Worksheet

About Synthesis

Approaches to synthesis.

You can sort the literature in various ways, for example:

How to Begin?

Read your sources carefully and find the main idea(s) of each source

Look for similarities in your sources – which sources are talking about the same main ideas? (for example, sources that discuss the historical background on your topic)

Use the worksheet (above) or synthesis matrix (below) to get organized

This work can be messy. Don't worry if you have to go through a few iterations of the worksheet or matrix as you work on your lit review!

Four Examples of Student Writing

In the four examples below, only ONE shows a good example of synthesis: the fourth column, or  Student D . For a web accessible version, click the link below the image.

Long description of "Four Examples of Student Writing" for web accessibility

• Download a copy of the "Four Examples of Student Writing" chart

Click on the example to view the pdf.

From Jennifer Lim

• << Previous: 5. Critically Analyze and Evaluate
• Next: 7. Write a Literature Review >>
• Last Updated: Jan 10, 2024 4:46 PM
• URL: https://researchguides.uoregon.edu/litreview

Contact Us Library Accessibility UO Libraries Privacy Notices and Procedures

1501 Kincaid Street Eugene, OR 97403 P: 541-346-3053 F: 541-346-3485

• Visit us on Facebook
• Visit us on Twitter
• Visit us on Youtube
• Visit us on Instagram
• Report a Concern
• Nondiscrimination and Title IX
• Accessibility
• Privacy Policy
• Find People

The Sheridan Libraries

• Write a Literature Review
• Sheridan Libraries
• Find This link opens in a new window
• Evaluate This link opens in a new window

Get Organized

• Lit Review Prep Use this template to help you evaluate your sources, create article summaries for an annotated bibliography, and a synthesis matrix for your lit review outline.

Synthesize your Information

Synthesize: combine separate elements to form a whole.

Synthesis Matrix

A synthesis matrix helps you record the main points of each source and document how sources relate to each other.

After summarizing and evaluating your sources, arrange them in a matrix or use a citation manager to help you see how they relate to each other and apply to each of your themes or variables.  

By arranging your sources by theme or variable, you can see how your sources relate to each other, and can start thinking about how you weave them together to create a narrative.

• Step-by-Step Approach
• Example Matrix from NSCU
• Matrix Template
• << Previous: Summarize
• Next: Integrate >>
• Last Updated: Sep 26, 2023 10:25 AM
• URL: https://guides.library.jhu.edu/lit-review

Encyclopedia of Evidence in Pharmaceutical Public Health and Health Services Research in Pharmacy pp 1–15 Cite as

Methodological Approaches to Literature Review

• Dennis Thomas 2 ,
• Elida Zairina 3 &
• Johnson George 4  
• Living reference work entry
• First Online: 09 May 2023

436 Accesses

The literature review can serve various functions in the contexts of education and research. It aids in identifying knowledge gaps, informing research methodology, and developing a theoretical framework during the planning stages of a research study or project, as well as reporting of review findings in the context of the existing literature. This chapter discusses the methodological approaches to conducting a literature review and offers an overview of different types of reviews. There are various types of reviews, including narrative reviews, scoping reviews, and systematic reviews with reporting strategies such as meta-analysis and meta-synthesis. Review authors should consider the scope of the literature review when selecting a type and method. Being focused is essential for a successful review; however, this must be balanced against the relevance of the review to a broad audience.

• Literature review
• Systematic review
• Meta-analysis
• Scoping review
• Research methodology

This is a preview of subscription content, log in via an institution .

Akobeng AK. Principles of evidence based medicine. Arch Dis Child. 2005;90(8):837–40.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Alharbi A, Stevenson M. Refining Boolean queries to identify relevant studies for systematic review updates. J Am Med Inform Assoc. 2020;27(11):1658–66.

Article   PubMed   PubMed Central   Google Scholar  

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.

Article   Google Scholar  

Aromataris E MZE. JBI manual for evidence synthesis. 2020.

Google Scholar  

Aromataris E, Pearson A. The systematic review: an overview. Am J Nurs. 2014;114(3):53–8.

Article   PubMed   Google Scholar  

Aromataris E, Riitano D. Constructing a search strategy and searching for evidence. A guide to the literature search for a systematic review. Am J Nurs. 2014;114(5):49–56.

Babineau J. Product review: covidence (systematic review software). J Canad Health Libr Assoc Canada. 2014;35(2):68–71.

Baker JD. The purpose, process, and methods of writing a literature review. AORN J. 2016;103(3):265–9.

Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7(9):e1000326.

Bramer WM, Rethlefsen ML, Kleijnen J, Franco OH. Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study. Syst Rev. 2017;6(1):1–12.

Brown D. A review of the PubMed PICO tool: using evidence-based practice in health education. Health Promot Pract. 2020;21(4):496–8.

Cargo M, Harris J, Pantoja T, et al. Cochrane qualitative and implementation methods group guidance series – paper 4: methods for assessing evidence on intervention implementation. J Clin Epidemiol. 2018;97:59–69.

Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med. 1997;126(5):376–80.

Article   CAS   PubMed   Google Scholar  

Counsell C. Formulating questions and locating primary studies for inclusion in systematic reviews. Ann Intern Med. 1997;127(5):380–7.

Cummings SR, Browner WS, Hulley SB. Conceiving the research question and developing the study plan. In: Cummings SR, Browner WS, Hulley SB, editors. Designing Clinical Research: An Epidemiological Approach. 4th ed. Philadelphia (PA): P Lippincott Williams & Wilkins; 2007. p. 14–22.

Eriksen MB, Frandsen TF. The impact of patient, intervention, comparison, outcome (PICO) as a search strategy tool on literature search quality: a systematic review. JMLA. 2018;106(4):420.

Ferrari R. Writing narrative style literature reviews. Medical Writing. 2015;24(4):230–5.

Flemming K, Booth A, Hannes K, Cargo M, Noyes J. Cochrane qualitative and implementation methods group guidance series – paper 6: reporting guidelines for qualitative, implementation, and process evaluation evidence syntheses. J Clin Epidemiol. 2018;97:79–85.

Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Inf Libr J. 2009;26(2):91–108.

Green BN, Johnson CD, Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. J Chiropr Med. 2006;5(3):101–17.

Gregory AT, Denniss AR. An introduction to writing narrative and systematic reviews; tasks, tips and traps for aspiring authors. Heart Lung Circ. 2018;27(7):893–8.

Harden A, Thomas J, Cargo M, et al. Cochrane qualitative and implementation methods group guidance series – paper 5: methods for integrating qualitative and implementation evidence within intervention effectiveness reviews. J Clin Epidemiol. 2018;97:70–8.

Harris JL, Booth A, Cargo M, et al. Cochrane qualitative and implementation methods group guidance series – paper 2: methods for question formulation, searching, and protocol development for qualitative evidence synthesis. J Clin Epidemiol. 2018;97:39–48.

Higgins J, Thomas J. In: Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.3, updated February 2022). Available from www.training.cochrane.org/handbook.: Cochrane; 2022.

International prospective register of systematic reviews (PROSPERO). Available from https://www.crd.york.ac.uk/prospero/ .

Khan KS, Kunz R, Kleijnen J, Antes G. Five steps to conducting a systematic review. J R Soc Med. 2003;96(3):118–21.

Landhuis E. Scientific literature: information overload. Nature. 2016;535(7612):457–8.

Lockwood C, Porritt K, Munn Z, Rittenmeyer L, Salmond S, Bjerrum M, Loveday H, Carrier J, Stannard D. Chapter 2: Systematic reviews of qualitative evidence. In: Aromataris E, Munn Z, editors. JBI Manual for Evidence Synthesis. JBI; 2020. Available from https://synthesismanual.jbi.global . https://doi.org/10.46658/JBIMES-20-03 .

Chapter   Google Scholar  

Lorenzetti DL, Topfer L-A, Dennett L, Clement F. Value of databases other than medline for rapid health technology assessments. Int J Technol Assess Health Care. 2014;30(2):173–8.

Moher D, Liberati A, Tetzlaff J, Altman DG, the PRISMA Group. Preferred reporting items for (SR) and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;6:264–9.

Mulrow CD. Systematic reviews: rationale for systematic reviews. BMJ. 1994;309(6954):597–9.

Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018;18(1):143.

Munthe-Kaas HM, Glenton C, Booth A, Noyes J, Lewin S. Systematic mapping of existing tools to appraise methodological strengths and limitations of qualitative research: first stage in the development of the CAMELOT tool. BMC Med Res Methodol. 2019;19(1):1–13.

Murphy CM. Writing an effective review article. J Med Toxicol. 2012;8(2):89–90.

NHMRC. Guidelines for guidelines: assessing risk of bias. Available at https://nhmrc.gov.au/guidelinesforguidelines/develop/assessing-risk-bias . Last published 29 August 2019. Accessed 29 Aug 2022.

Noyes J, Booth A, Cargo M, et al. Cochrane qualitative and implementation methods group guidance series – paper 1: introduction. J Clin Epidemiol. 2018b;97:35–8.

Noyes J, Booth A, Flemming K, et al. Cochrane qualitative and implementation methods group guidance series – paper 3: methods for assessing methodological limitations, data extraction and synthesis, and confidence in synthesized qualitative findings. J Clin Epidemiol. 2018a;97:49–58.

Noyes J, Booth A, Moore G, Flemming K, Tunçalp Ö, Shakibazadeh E. Synthesising quantitative and qualitative evidence to inform guidelines on complex interventions: clarifying the purposes, designs and outlining some methods. BMJ Glob Health. 2019;4(Suppl 1):e000893.

Peters MD, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. Int J Evid Healthcare. 2015;13(3):141–6.

Polanin JR, Pigott TD, Espelage DL, Grotpeter JK. Best practice guidelines for abstract screening large-evidence systematic reviews and meta-analyses. Res Synth Methods. 2019;10(3):330–42.

Article   PubMed Central   Google Scholar  

Shea BJ, Grimshaw JM, Wells GA, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7(1):1–7.

Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. Brit Med J. 2017;358

Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. Br Med J. 2016;355

Stroup DF, Berlin JA, Morton SC, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. JAMA. 2000;283(15):2008–12.

Tawfik GM, Dila KAS, Mohamed MYF, et al. A step by step guide for conducting a systematic review and meta-analysis with simulation data. Trop Med Health. 2019;47(1):1–9.

The Critical Appraisal Program. Critical appraisal skills program. Available at https://casp-uk.net/ . 2022. Accessed 29 Aug 2022.

The University of Melbourne. Writing a literature review in Research Techniques 2022. Available at https://students.unimelb.edu.au/academic-skills/explore-our-resources/research-techniques/reviewing-the-literature . Accessed 29 Aug 2022.

The Writing Center University of Winconsin-Madison. Learn how to write a literature review in The Writer’s Handbook – Academic Professional Writing. 2022. Available at https://writing.wisc.edu/handbook/assignments/reviewofliterature/ . Accessed 29 Aug 2022.

Thompson SG, Sharp SJ. Explaining heterogeneity in meta-analysis: a comparison of methods. Stat Med. 1999;18(20):2693–708.

Tricco AC, Lillie E, Zarin W, et al. A scoping review on the conduct and reporting of scoping reviews. BMC Med Res Methodol. 2016;16(1):15.

Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

Yoneoka D, Henmi M. Clinical heterogeneity in random-effect meta-analysis: between-study boundary estimate problem. Stat Med. 2019;38(21):4131–45.

Yuan Y, Hunt RH. Systematic reviews: the good, the bad, and the ugly. Am J Gastroenterol. 2009;104(5):1086–92.

Download references

Author information

Authors and affiliations.

Centre of Excellence in Treatable Traits, College of Health, Medicine and Wellbeing, University of Newcastle, Hunter Medical Research Institute Asthma and Breathing Programme, Newcastle, NSW, Australia

Dennis Thomas

Department of Pharmacy Practice, Faculty of Pharmacy, Universitas Airlangga, Surabaya, Indonesia

Elida Zairina

Centre for Medicine Use and Safety, Monash Institute of Pharmaceutical Sciences, Faculty of Pharmacy and Pharmaceutical Sciences, Monash University, Parkville, VIC, Australia

Johnson George

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Johnson George .

Section Editor information

College of Pharmacy, Qatar University, Doha, Qatar

Derek Charles Stewart

Department of Pharmacy, University of Huddersfield, Huddersfield, United Kingdom

Zaheer-Ud-Din Babar

Rights and permissions

Reprints and permissions

Copyright information

© 2023 Springer Nature Switzerland AG

About this entry

Cite this entry.

Thomas, D., Zairina, E., George, J. (2023). Methodological Approaches to Literature Review. In: Encyclopedia of Evidence in Pharmaceutical Public Health and Health Services Research in Pharmacy. Springer, Cham. https://doi.org/10.1007/978-3-030-50247-8_57-1

Download citation

DOI : https://doi.org/10.1007/978-3-030-50247-8_57-1

Received : 22 February 2023

Accepted : 22 February 2023

Published : 09 May 2023

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-50247-8

Online ISBN : 978-3-030-50247-8

eBook Packages : Springer Reference Biomedicine and Life Sciences Reference Module Biomedical and Life Sciences

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Holiday opening hours

Opening hours for our libraries and Library Chat have been adjusted over the Easter holidays. Please check our library opening hours page before you visit.

Which review is that? A guide to review types.

• Which review is that?
• Review Comparison Chart
• Decision Tool
• Critical Review
• Integrative Review
• Narrative Review
• State of the Art Review
• Narrative Summary
• Systematic Review
• Meta-analysis
• Comparative Effectiveness Review
• Diagnostic Systematic Review
• Network Meta-analysis
• Prognostic Review
• Psychometric Review
• Review of Economic Evaluations
• Systematic Review of Epidemiology Studies
• Living Systematic Reviews
• Umbrella Review
• Review of Reviews
• Rapid Review
• Rapid Evidence Assessment
• Rapid Realist Review
• Qualitative Evidence Synthesis
• Qualitative Interpretive Meta-synthesis
• Qualitative Meta-synthesis
• Qualitative Research Synthesis
• Framework Synthesis - Best-fit Framework Synthesis
• Meta-aggregation
• Meta-ethnography
• Meta-interpretation
• Meta-narrative Review
• Meta-summary
• Thematic Synthesis
• Mixed Methods Synthesis
• Narrative Synthesis
• Bayesian Meta-analysis
• EPPI-Centre Review

Critical Interpretive Synthesis

• Realist Synthesis - Realist Review
• Scoping Review
• Mapping Review
• Systematised Review
• Concept Synthesis
• Expert Opinion - Policy Review
• Technology Assessment Review
• Methodological Review
• Systematic Search and Review

"Critical interpretive synthesis involves an iterative approach to refining the research question and searching and selecting from the literature (using theoretical sampling) and defining and applying codes and categories. It also has a particular approach to appraising quality, using relevance – i.e. likely contribution to theory development – rather than methodological characteristics as a means of determining the 'quality' of individual papers" (Barnett-Page et al, 2009).

Further Reading/Resources

Chalmiers, M. A., Karaki, F., Muriki, M., Mody, S., Chen, A., & de Bocanegra, H. T. (2021). Refugee women's experiences with contraceptive care after resettlement in high-income countries: A Critical Interpretive Synthesis. Contraception . Full Text

References Barnett-Page, E., & Thomas, J. (2009). Methods for the synthesis of qualitative research: a critical review. BMC medical research methodology , 9 (1), 1-11. Full Text

• << Previous: EPPI-Centre Review
• Next: Realist Synthesis - Realist Review >>
• Last Updated: Mar 5, 2024 1:14 PM
• URL: https://unimelb.libguides.com/whichreview

Library Guides

Literature reviews: synthesis.

• Criticality

Synthesise Information

So, how can you create paragraphs within your literature review that demonstrates your knowledge of the scholarship that has been done in your field of study?  

You will need to present a synthesis of the texts you read.  

Doug Specht, Senior Lecturer at the Westminster School of Media and Communication, explains synthesis for us in the following video:  

Synthesising Texts  

What is synthesis? 

Synthesis is an important element of academic writing, demonstrating comprehension, analysis, evaluation and original creation.  

With synthesis you extract content from different sources to create an original text. While paraphrase and summary maintain the structure of the given source(s), with synthesis you create a new structure.  

The sources will provide different perspectives and evidence on a topic. They will be put together when agreeing, contrasted when disagreeing. The sources must be referenced.  

Perfect your synthesis by showing the flow of your reasoning, expressing critical evaluation of the sources and drawing conclusions.  

When you synthesise think of "using strategic thinking to resolve a problem requiring the integration of diverse pieces of information around a structuring theme" (Mateos and Sole 2009, p448). 

Synthesis is a complex activity, which requires a high degree of comprehension and active engagement with the subject. As you progress in higher education, so increase the expectations on your abilities to synthesise. 

How to synthesise in a literature review: 

Identify themes/issues you'd like to discuss in the literature review. Think of an outline.  

Read the literature and identify these themes/issues.  

Critically analyse the texts asking: how does the text I'm reading relate to the other texts I've read on the same topic? Is it in agreement? Does it differ in its perspective? Is it stronger or weaker? How does it differ (could be scope, methods, year of publication etc.). Draw your conclusions on the state of the literature on the topic.  

Start writing your literature review, structuring it according to the outline you planned.  

Put together sources stating the same point; contrast sources presenting counter-arguments or different points.  

Present your critical analysis.  

Always provide the references. 

The best synthesis requires a "recursive process" whereby you read the source texts, identify relevant parts, take notes, produce drafts, re-read the source texts, revise your text, re-write... (Mateos and Sole, 2009). 

What is good synthesis?  

The quality of your synthesis can be assessed considering the following (Mateos and Sole, 2009, p439):  

Integration and connection of the information from the source texts around a structuring theme. 

Selection of ideas necessary for producing the synthesis. 

Appropriateness of the interpretation.  

Elaboration of the content.  

Example of Synthesis

Original texts (fictitious): 

Synthesis: 

Animal experimentation is a subject of heated debate. Some argue that painful experiments should be banned. Indeed it has been demonstrated that such experiments make animals suffer physically and psychologically (Chowdhury 2012; Panatta and Hudson 2016). On the other hand, it has been argued that animal experimentation can save human lives and reduce harm on humans (Smith 2008). This argument is only valid for toxicological testing, not for tests that, for example, merely improve the efficacy of a cosmetic (Turner 2015). It can be suggested that animal experimentation should be regulated to only allow toxicological risk assessment, and the suffering to the animals should be minimised.   

Bibliography

Mateos, M. and Sole, I. (2009). Synthesising Information from various texts: A Study of Procedures and Products at Different Educational Levels. European Journal of Psychology of Education,  24 (4), 435-451. Available from https://doi.org/10.1007/BF03178760 [Accessed 29 June 2021].

• << Previous: Structure
• Next: Criticality >>
• Last Updated: Nov 18, 2023 10:56 PM
• URL: https://libguides.westminster.ac.uk/literature-reviews

CONNECT WITH US

• U.S. Locations
• UMGC Europe
• Learn Online
• Find Answers
• 855-655-8682
• Current Students

Online Guide to Writing and Research

Thinking strategies and writing patterns, explore more of umgc.

• Online Guide to Writing

Critical Strategies and Writing

One of the basic academic writing activities is researching your topic and what others have said about it. Your goal should be to draw thoughts, observations, and claims about your topic from your research. We call this process of drawing from multiple sources “ synthesis .” Click on the accordion items below for more information.

Definition, Use, and Sample Synthesis

Synthesis defined.

Synthesis Emerges from Analysis

Synthesis emerges from analytical activities we discussed on the previous page: comparative analysis and analysis for cause and effect . For example, to communicate where scholars agree and where they disagree, one must analyze their work for similarities and differences. Also crucial for understanding scholarly discourse is understanding how a particular work of scholarship shapes the scholarship of others, causing them to head in new directions.

When Should Synthesis Be Used?

When to Use Synthesis

Many college assignments require synthesis. A literature review, for example, requires that you make explanatory claims regarding a body of research. These should go beyond summary (mere description) to provide helpful characterizations that aid in understanding. Literature reviews can stand on their own, but often they are a part of a research paper, and research papers are where you will probably use synthesis most often.

The purpose of a research paper is to derive meaning from a body of information collected through research. It is your job, as the writer, to communicate that meaning to your readers. Doing this requires that you develop an informed and educated opinion of what your research suggests about your subject. Communicating this opinion requires synthesis.

Sample Synthesis

In 1655, an embassy of Dutch Jews led by Rabbi Menassah ben Israel traveled to London to meet with the Commonwealth’s new Lord Protector, Oliver Cromwell. The informal “Readmission” of Jews—who had been expelled from England by royal edict in 1290—resulting from the Whitehall conference was once hailed as a high point in the history of toleration. Yet in recent years, scholars have increasingly challenged the progressive nature of this event, both in its substance and its motivation (Kaplan 2007; Katznelson 2010; Walsham 2006) . “Toleration” in this case, as in many others, did not entail religious freedom or civic equality; Jews in England were granted legal residency and permitted to worship privately, but citizenship, public worship, and the printing of anything that “opposeth the Christian religion” remained off the cards. As for its motivation, Edward Whalley’s twofold argument was representative: the Jews “will bring in much wealth into this Commonwealth: and where wee both pray for theyre conversion and beleeve it shal be, I knowe not why wee shold deny the means” (Marshall 2006, 381–82) 

(Bejan, 2015, p. 1103).

The author of the above passage, Teresa Bejan, has synthesized the work of a number of other scholars (Kaplan, Katznelson, Walsham, and Marshall) to situate her argument.  Note how not all of these scholars are directly quoted, but they are cited because their work forms the basis of Bejan's work.

Key Takeaways

• Synthesizing allows you to carry an argument or stance you adopt within a paper in your own words, based on conclusions you have come to about the topic.
• Synthesizing contributes to confidence about your stance and topic.

Mailing Address: 3501 University Blvd. East, Adelphi, MD 20783 This work is licensed under a  Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License . © 2022 UMGC. All links to external sites were verified at the time of publication. UMGC is not responsible for the validity or integrity of information located at external sites.

Table of Contents: Online Guide to Writing

Chapter 1: College Writing

How Does College Writing Differ from Workplace Writing?

What Is College Writing?

Why So Much Emphasis on Writing?

Chapter 2: The Writing Process

Doing Exploratory Research

Getting from Notes to Your Draft

Introduction

Prewriting - Techniques to Get Started - Mining Your Intuition

Prewriting: Targeting Your Audience

Prewriting: Techniques to Get Started

Prewriting: Understanding Your Assignment

Rewriting: Being Your Own Critic

Rewriting: Creating a Revision Strategy

Rewriting: Getting Feedback

Rewriting: The Final Draft

Techniques to Get Started - Outlining

Techniques to Get Started - Using Systematic Techniques

Thesis Statement and Controlling Idea

Writing: Getting from Notes to Your Draft - Freewriting

Writing: Getting from Notes to Your Draft - Summarizing Your Ideas

Writing: Outlining What You Will Write

Chapter 3: Thinking Strategies

A Word About Style, Voice, and Tone

A Word About Style, Voice, and Tone: Style Through Vocabulary and Diction

Critical Strategies and Writing: Analysis

Critical Strategies and Writing: Evaluation

Critical Strategies and Writing: Persuasion

Critical Strategies and Writing: Synthesis

Developing a Paper Using Strategies

Kinds of Assignments You Will Write

Patterns for Presenting Information

Patterns for Presenting Information: Critiques

Patterns for Presenting Information: Discussing Raw Data

Patterns for Presenting Information: General-to-Specific Pattern

Patterns for Presenting Information: Problem-Cause-Solution Pattern

Patterns for Presenting Information: Specific-to-General Pattern

Patterns for Presenting Information: Summaries and Abstracts

Supporting with Research and Examples

Writing Essay Examinations

Writing Essay Examinations: Make Your Answer Relevant and Complete

Writing Essay Examinations: Organize Thinking Before Writing

Writing Essay Examinations: Read and Understand the Question

Chapter 4: The Research Process

Planning and Writing a Research Paper

Planning and Writing a Research Paper: Ask a Research Question

Planning and Writing a Research Paper: Cite Sources

Planning and Writing a Research Paper: Collect Evidence

Planning and Writing a Research Paper: Decide Your Point of View, or Role, for Your Research

Planning and Writing a Research Paper: Draw Conclusions

Planning and Writing a Research Paper: Find a Topic and Get an Overview

Planning and Writing a Research Paper: Manage Your Resources

Planning and Writing a Research Paper: Outline

Planning and Writing a Research Paper: Survey the Literature

Planning and Writing a Research Paper: Work Your Sources into Your Research Writing

Research Resources: Where Are Research Resources Found? - Human Resources

Research Resources: What Are Research Resources?

Research Resources: Where Are Research Resources Found?

Research Resources: Where Are Research Resources Found? - Electronic Resources

Research Resources: Where Are Research Resources Found? - Print Resources

Structuring the Research Paper: Formal Research Structure

Structuring the Research Paper: Informal Research Structure

The Nature of Research

The Research Assignment: How Should Research Sources Be Evaluated?

The Research Assignment: When Is Research Needed?

The Research Assignment: Why Perform Research?

Chapter 5: Academic Integrity

Academic Integrity

Giving Credit to Sources

Giving Credit to Sources: Copyright Laws

Giving Credit to Sources: Documentation

Giving Credit to Sources: Style Guides

Integrating Sources

Practicing Academic Integrity

Practicing Academic Integrity: Keeping Accurate Records

Practicing Academic Integrity: Managing Source Material

Practicing Academic Integrity: Managing Source Material - Paraphrasing Your Source

Practicing Academic Integrity: Managing Source Material - Quoting Your Source

Practicing Academic Integrity: Managing Source Material - Summarizing Your Sources

Types of Documentation

Types of Documentation: Bibliographies and Source Lists

Types of Documentation: Citing World Wide Web Sources

Types of Documentation: In-Text or Parenthetical Citations

Types of Documentation: In-Text or Parenthetical Citations - APA Style

Types of Documentation: In-Text or Parenthetical Citations - CSE/CBE Style

Types of Documentation: In-Text or Parenthetical Citations - Chicago Style

Types of Documentation: In-Text or Parenthetical Citations - MLA Style

Types of Documentation: Note Citations

Chapter 6: Using Library Resources

Finding Library Resources

Chapter 7: Assessing Your Writing

How Is Writing Graded?

How Is Writing Graded?: A General Assessment Tool

The Draft Stage

The Draft Stage: The First Draft

The Draft Stage: The Revision Process and the Final Draft

The Draft Stage: Using Feedback

The Research Stage

Using Assessment to Improve Your Writing

Chapter 8: Other Frequently Assigned Papers

Reviews and Reaction Papers: Article and Book Reviews

Reviews and Reaction Papers: Reaction Papers

Writing Arguments

Writing Arguments: Adapting the Argument Structure

Writing Arguments: Purposes of Argument

Writing Arguments: References to Consult for Writing Arguments

Writing Arguments: Steps to Writing an Argument - Anticipate Active Opposition

Writing Arguments: Steps to Writing an Argument - Determine Your Organization

Writing Arguments: Steps to Writing an Argument - Develop Your Argument

Writing Arguments: Steps to Writing an Argument - Introduce Your Argument

Writing Arguments: Steps to Writing an Argument - State Your Thesis or Proposition

Writing Arguments: Steps to Writing an Argument - Write Your Conclusion

Writing Arguments: Types of Argument

Appendix A: Books to Help Improve Your Writing

Dictionaries

General Style Manuals

Researching on the Internet

Special Style Manuals

Writing Handbooks

Appendix B: Collaborative Writing and Peer Reviewing

Collaborative Writing: Assignments to Accompany the Group Project

Collaborative Writing: Informal Progress Report

Collaborative Writing: Issues to Resolve

Collaborative Writing: Methodology

Collaborative Writing: Peer Evaluation

Collaborative Writing: Tasks of Collaborative Writing Group Members

Collaborative Writing: Writing Plan

General Introduction

Peer Reviewing

Appendix C: Developing an Improvement Plan

Working with Your Instructor’s Comments and Grades

Appendix D: Writing Plan and Project Schedule

Devising a Writing Project Plan and Schedule

Reviewing Your Plan with Others

By using our website you agree to our use of cookies. Learn more about how we use cookies by reading our  Privacy Policy .

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base
• Working with sources
• Synthesizing Sources | Examples & Synthesis Matrix

Synthesizing Sources | Examples & Synthesis Matrix

Published on July 4, 2022 by Eoghan Ryan . Revised on May 31, 2023.

Synthesizing sources involves combining the work of other scholars to provide new insights. It’s a way of integrating sources that helps situate your work in relation to existing research.

Synthesizing sources involves more than just summarizing . You must emphasize how each source contributes to current debates, highlighting points of (dis)agreement and putting the sources in conversation with each other.

You might synthesize sources in your literature review to give an overview of the field or throughout your research paper when you want to position your work in relation to existing research.

Table of contents

Example of synthesizing sources, how to synthesize sources, synthesis matrix, other interesting articles, frequently asked questions about synthesizing sources.

Let’s take a look at an example where sources are not properly synthesized, and then see what can be done to improve it.

This paragraph provides no context for the information and does not explain the relationships between the sources described. It also doesn’t analyze the sources or consider gaps in existing research.

Research on the barriers to second language acquisition has primarily focused on age-related difficulties. Building on Lenneberg’s (1967) theory of a critical period of language acquisition, Johnson and Newport (1988) tested Lenneberg’s idea in the context of second language acquisition. Their research seemed to confirm that young learners acquire a second language more easily than older learners. Recent research has considered other potential barriers to language acquisition. Schepens, van Hout, and van der Slik (2022) have revealed that the difficulties of learning a second language at an older age are compounded by dissimilarity between a learner’s first language and the language they aim to acquire. Further research needs to be carried out to determine whether the difficulty faced by adult monoglot speakers is also faced by adults who acquired a second language during the “critical period.”

Scribbr Citation Checker New

The AI-powered Citation Checker helps you avoid common mistakes such as:

• Missing commas and periods
• Incorrect usage of “et al.”
• Ampersands (&) in narrative citations
• Missing reference entries

To synthesize sources, group them around a specific theme or point of contention.

As you read sources, ask:

• What questions or ideas recur? Do the sources focus on the same points, or do they look at the issue from different angles?
• How does each source relate to others? Does it confirm or challenge the findings of past research?
• Where do the sources agree or disagree?

Once you have a clear idea of how each source positions itself, put them in conversation with each other. Analyze and interpret their points of agreement and disagreement. This displays the relationships among sources and creates a sense of coherence.

Consider both implicit and explicit (dis)agreements. Whether one source specifically refutes another or just happens to come to different conclusions without specifically engaging with it, you can mention it in your synthesis either way.

Synthesize your sources using:

• Topic sentences to introduce the relationship between the sources
• Signal phrases to attribute ideas to their authors
• Transition words and phrases to link together different ideas

To more easily determine the similarities and dissimilarities among your sources, you can create a visual representation of their main ideas with a synthesis matrix . This is a tool that you can use when researching and writing your paper, not a part of the final text.

In a synthesis matrix, each column represents one source, and each row represents a common theme or idea among the sources. In the relevant rows, fill in a short summary of how the source treats each theme or topic.

This helps you to clearly see the commonalities or points of divergence among your sources. You can then synthesize these sources in your work by explaining their relationship.

If you want to know more about ChatGPT, AI tools , citation , and plagiarism , make sure to check out some of our other articles with explanations and examples.

• ChatGPT vs human editor
• ChatGPT citations
• Is ChatGPT trustworthy?
• Using ChatGPT for your studies
• What is ChatGPT?
• Chicago style
• Paraphrasing

 Plagiarism

• Types of plagiarism
• Self-plagiarism
• Avoiding plagiarism
• Academic integrity
• Consequences of plagiarism
• Common knowledge

The only proofreading tool specialized in correcting academic writing - try for free!

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Try for free

Synthesizing sources means comparing and contrasting the work of other scholars to provide new insights.

It involves analyzing and interpreting the points of agreement and disagreement among sources.

You might synthesize sources in your literature review to give an overview of the field of research or throughout your paper when you want to contribute something new to existing research.

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

Topic sentences help keep your writing focused and guide the reader through your argument.

In an essay or paper , each paragraph should focus on a single idea. By stating the main idea in the topic sentence, you clarify what the paragraph is about for both yourself and your reader.

At college level, you must properly cite your sources in all essays , research papers , and other academic texts (except exams and in-class exercises).

Add a citation whenever you quote , paraphrase , or summarize information or ideas from a source. You should also give full source details in a bibliography or reference list at the end of your text.

The exact format of your citations depends on which citation style you are instructed to use. The most common styles are APA , MLA , and Chicago .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Ryan, E. (2023, May 31). Synthesizing Sources | Examples & Synthesis Matrix. Scribbr. Retrieved April 1, 2024, from https://www.scribbr.com/working-with-sources/synthesizing-sources/

Is this article helpful?

Eoghan Ryan

Other students also liked, signal phrases | definition, explanation & examples, how to write a literature review | guide, examples, & templates, how to find sources | scholarly articles, books, etc., "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

How to Synthesize Written Information from Multiple Sources

Shona McCombes

Content Manager

B.A., English Literature, University of Glasgow

Shona McCombes is the content manager at Scribbr, Netherlands.

Learn about our Editorial Process

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

On This Page:

When you write a literature review or essay, you have to go beyond just summarizing the articles you’ve read – you need to synthesize the literature to show how it all fits together (and how your own research fits in).

Synthesizing simply means combining. Instead of summarizing the main points of each source in turn, you put together the ideas and findings of multiple sources in order to make an overall point.

At the most basic level, this involves looking for similarities and differences between your sources. Your synthesis should show the reader where the sources overlap and where they diverge.

Unsynthesized Example

Franz (2008) studied undergraduate online students. He looked at 17 females and 18 males and found that none of them liked APA. According to Franz, the evidence suggested that all students are reluctant to learn citations style. Perez (2010) also studies undergraduate students. She looked at 42 females and 50 males and found that males were significantly more inclined to use citation software ( p < .05). Findings suggest that females might graduate sooner. Goldstein (2012) looked at British undergraduates. Among a sample of 50, all females, all confident in their abilities to cite and were eager to write their dissertations.

Synthesized Example

Studies of undergraduate students reveal conflicting conclusions regarding relationships between advanced scholarly study and citation efficacy. Although Franz (2008) found that no participants enjoyed learning citation style, Goldstein (2012) determined in a larger study that all participants watched felt comfortable citing sources, suggesting that variables among participant and control group populations must be examined more closely. Although Perez (2010) expanded on Franz’s original study with a larger, more diverse sample…

Step 1: Organize your sources

After collecting the relevant literature, you’ve got a lot of information to work through, and no clear idea of how it all fits together.

Before you can start writing, you need to organize your notes in a way that allows you to see the relationships between sources.

One way to begin synthesizing the literature is to put your notes into a table. Depending on your topic and the type of literature you’re dealing with, there are a couple of different ways you can organize this.

Summary table

A summary table collates the key points of each source under consistent headings. This is a good approach if your sources tend to have a similar structure – for instance, if they’re all empirical papers.

Each row in the table lists one source, and each column identifies a specific part of the source. You can decide which headings to include based on what’s most relevant to the literature you’re dealing with.

For example, you might include columns for things like aims, methods, variables, population, sample size, and conclusion.

For each study, you briefly summarize each of these aspects. You can also include columns for your own evaluation and analysis.

The summary table gives you a quick overview of the key points of each source. This allows you to group sources by relevant similarities, as well as noticing important differences or contradictions in their findings.

Synthesis matrix

A synthesis matrix is useful when your sources are more varied in their purpose and structure – for example, when you’re dealing with books and essays making various different arguments about a topic.

Each column in the table lists one source. Each row is labeled with a specific concept, topic or theme that recurs across all or most of the sources.

Then, for each source, you summarize the main points or arguments related to the theme.

The purposes of the table is to identify the common points that connect the sources, as well as identifying points where they diverge or disagree.

Step 2: Outline your structure

Now you should have a clear overview of the main connections and differences between the sources you’ve read. Next, you need to decide how you’ll group them together and the order in which you’ll discuss them.

For shorter papers, your outline can just identify the focus of each paragraph; for longer papers, you might want to divide it into sections with headings.

There are a few different approaches you can take to help you structure your synthesis.

If your sources cover a broad time period, and you found patterns in how researchers approached the topic over time, you can organize your discussion chronologically .

That doesn’t mean you just summarize each paper in chronological order; instead, you should group articles into time periods and identify what they have in common, as well as signalling important turning points or developments in the literature.

If the literature covers various different topics, you can organize it thematically .

That means that each paragraph or section focuses on a specific theme and explains how that theme is approached in the literature.

Source Used with Permission: The Chicago School

If you’re drawing on literature from various different fields or they use a wide variety of research methods, you can organize your sources methodologically .

That means grouping together studies based on the type of research they did and discussing the findings that emerged from each method.

If your topic involves a debate between different schools of thought, you can organize it theoretically .

That means comparing the different theories that have been developed and grouping together papers based on the position or perspective they take on the topic, as well as evaluating which arguments are most convincing.

Step 3: Write paragraphs with topic sentences

What sets a synthesis apart from a summary is that it combines various sources. The easiest way to think about this is that each paragraph should discuss a few different sources, and you should be able to condense the overall point of the paragraph into one sentence.

This is called a topic sentence , and it usually appears at the start of the paragraph. The topic sentence signals what the whole paragraph is about; every sentence in the paragraph should be clearly related to it.

A topic sentence can be a simple summary of the paragraph’s content:

“Early research on [x] focused heavily on [y].”

For an effective synthesis, you can use topic sentences to link back to the previous paragraph, highlighting a point of debate or critique:

“Several scholars have pointed out the flaws in this approach.” “While recent research has attempted to address the problem, many of these studies have methodological flaws that limit their validity.”

By using topic sentences, you can ensure that your paragraphs are coherent and clearly show the connections between the articles you are discussing.

As you write your paragraphs, avoid quoting directly from sources: use your own words to explain the commonalities and differences that you found in the literature.

Don’t try to cover every single point from every single source – the key to synthesizing is to extract the most important and relevant information and combine it to give your reader an overall picture of the state of knowledge on your topic.

Step 4: Revise, edit and proofread

Like any other piece of academic writing, synthesizing literature doesn’t happen all in one go – it involves redrafting, revising, editing and proofreading your work.

Checklist for Synthesis

•   Do I introduce the paragraph with a clear, focused topic sentence?
•   Do I discuss more than one source in the paragraph?
•   Do I mention only the most relevant findings, rather than describing every part of the studies?
•   Do I discuss the similarities or differences between the sources, rather than summarizing each source in turn?
•   Do I put the findings or arguments of the sources in my own words?
•   Is the paragraph organized around a single idea?
•   Is the paragraph directly relevant to my research question or topic?
•   Is there a logical transition from this paragraph to the next one?

Further Information

How to Synthesise: a Step-by-Step Approach

Help…I”ve Been Asked to Synthesize!

Learn how to Synthesise (combine information from sources)

How to write a Psychology Essay

• Technical advance
• Open access
• Published: 26 July 2006

Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups

• Mary Dixon-Woods 1 ,
• Debbie Cavers 2 ,
• Shona Agarwal 3 ,
• Ellen Annandale 4 ,
• Antony Arthur 5 ,
• Janet Harvey 6 ,
• Ron Hsu 1 ,
• Savita Katbamna 7 ,
• Richard Olsen 7 ,
• Lucy Smith 1 ,
• Richard Riley 1 &
• Alex J Sutton 1  

BMC Medical Research Methodology volume  6 , Article number:  35 ( 2006 ) Cite this article

105k Accesses

1058 Citations

49 Altmetric

Metrics details

Conventional systematic review techniques have limitations when the aim of a review is to construct a critical analysis of a complex body of literature. This article offers a reflexive account of an attempt to conduct an interpretive review of the literature on access to healthcare by vulnerable groups in the UK

This project involved the development and use of the method of Critical Interpretive Synthesis (CIS). This approach is sensitised to the processes of conventional systematic review methodology and draws on recent advances in methods for interpretive synthesis.

Many analyses of equity of access have rested on measures of utilisation of health services, but these are problematic both methodologically and conceptually. A more useful means of understanding access is offered by the synthetic construct of candidacy. Candidacy describes how people's eligibility for healthcare is determined between themselves and health services. It is a continually negotiated property of individuals, subject to multiple influences arising both from people and their social contexts and from macro-level influences on allocation of resources and configuration of services. Health services are continually constituting and seeking to define the appropriate objects of medical attention and intervention, while at the same time people are engaged in constituting and defining what they understand to be the appropriate objects of medical attention and intervention. Access represents a dynamic interplay between these simultaneous, iterative and mutually reinforcing processes. By attending to how vulnerabilities arise in relation to candidacy, the phenomenon of access can be better understood, and more appropriate recommendations made for policy, practice and future research.

By innovating with existing methods for interpretive synthesis, it was possible to produce not only new methods for conducting what we have termed critical interpretive synthesis, but also a new theoretical conceptualisation of access to healthcare. This theoretical account of access is distinct from models already extant in the literature, and is the result of combining diverse constructs and evidence into a coherent whole. Both the method and the model should be evaluated in other contexts.

Peer Review reports

Like many areas of healthcare practice and policy, the literature on access to healthcare is large, diverse, and complex. It includes empirical work using both qualitative and quantitative methods; editorial comment and theoretical work; case studies; evaluative, epidemiological, trial, descriptive, sociological, psychological, management, and economics papers, as well as policy documents and political statements. "Access" itself has not been consistently defined or operationalised across the field. There are substantial adjunct literatures, including those on quality in healthcare, priority-setting, and patient satisfaction. A review of the area would be of most benefit if it were to produce a "mid-range" theoretical account of the evidence and existing theory that is neither so abstract that it lacks empirical applicability nor so specific that its explanatory scope is limited.

In this paper, we suggest that conventional systematic review methodology is ill-suited to the challenges that conducting such a review would pose, and describe the development of a new form of review which we term "Critical Interpretive Synthesis" (CIS). This approach draws is sensitised to the range of issues involved in conducting reviews that conventional systematic review methodology has identified, but draws on a distinctive tradition of qualitative inquiry, including recent interpretive approaches to review [ 1 ]. We suggest that that using CIS to synthesise a diverse body of evidence enables the generation of theory with strong explanatory power. We illustrate this briefly using an example based on synthesis of the literature on access to healthcare in the UK by socio-economically disadvantaged people.

Aggregative and interpretive reviews

Conventional systematic review developed as a specific methodology for searching for, appraising, and synthesising findings of primary studies [ 2 ]. It offers a way of systematising, rationalising, and making more explicit the processes of review, and has demonstrated considerable benefits in synthesising certain forms of evidence where the aim is to test theories, perhaps especially about "what works". It is more limited when the aim, as here, is to include many different forms of evidence with the aim of generating theory [ 3 ]. Conventional systematic review methods are thus better suited to the production of aggregative rather than interpretive syntheses.

This distinction between aggregative and interpretive syntheses, noted by Noblit and Hare in their ground-breaking book on meta-ethnography[ 1 ] allows a useful (though necessarily crude) categorisation of two principal approaches to conducting reviews [ 4 ]. Aggregative reviews are concerned with assembling and pooling data, may use techniques such as meta-analysis, and require a basic comparability between phenomena so that the data can be aggregated for analysis. Their defining characteristics are a focus on summarising data , and an assumption that the concepts (or variables) under which those data are to be summarised are largely secure and well specified. Key concepts are defined at an early stage in the review and form the categories under which the data from empirical studies are to be summarised.

Interpretive reviews, by contrast, see the essential tasks of synthesis as involving both induction and interpretation. Their primary concern is with the development of concepts and theories that integrate those concepts. An interpretive review will therefore avoid specifying concepts in advance of the synthesis. The interpretive analysis that yields the synthesis is conceptual in process and output. The product of the synthesis is not aggregations of data, but theory grounded in the studies included in the review. Although there is a tendency at present to conduct interpretive synthesis only of qualitative studies, it should in principle be possible and indeed desirable to conduct interpretive syntheses of all forms of evidence, since theory-building need not be based only on one form of evidence. Indeed, Glaser and Strauss [ 5 ] in their seminal text, included an (often forgotten) chapter on the use of quantitative data for theory-building.

Recent years have seen the emergence of a range of methods that draw on a more interpretive tradition, but these also have limitations when attempting a synthesis of a large and complex body of evidence. In general, the use to date of interpretive approaches to synthesis has been confined to the synthesis of qualitative research only [ 6 – 8 ]. Meta-ethnography, an approach in which there has been recent significant activity and innovation, has similarly been used solely to synthesise qualitative studies, and has typically been used only with small samples [ 9 – 11 ]. Few approaches have attempted to apply an interpretive approach to the whole corpus of evidence (regardless of study type) included in a review, and few have treated the literature they examine as itself an object of scrutiny, for example by questioning the ways in which the literature constructs its problematics, the nature of the assumptions on the literature draw, or what has influenced proposed solutions.

In this paper we offer a reflexive account of our attempt to conduct an interpretive synthesis of all types of evidence relevant to access to National Health Service (NHS) healthcare in the UK by potentially vulnerable groups. These groups had been defined at the outset by the funders of the project (the UK Department of Health Service Delivery and Organisation R&D Programme) as children, older people, members of minority ethnicities, men/women, and socio-economically disadvantaged people. We explain in particular our development of Critical Interpretive Synthesis as a method for conducting this review.

Formulating the review question

Conventional systematic review methodology [ 12 , 13 ] emphasises the need for review questions to be precisely formulated. A tightly focused research question allows the parameters of the review to be identified and the study selection criteria to be defined in advance, and in turn limits the amount of evidence required to address the review question.

This strategy is successful where the phenomenon of interest, the populations, interventions, and outcomes are all well specified – i.e. if the aim of the review is aggregative . For our project, it was neither possible nor desirable to specify in advance the precise review question, a priori definitions, or categories under which the data could be summarised, since one of its aims was to allow the definition of the phenomenon of access to emerge from our analysis of the literature [ 14 ]. This is not to say that we did not have a review question, only that it was not a specific hypothesis. Instead it was, as Greenhalgh and colleagues [ 15 ] describe, tentative, fuzzy and contested at the outset of the project. It did include a focus on equity and on how access, particularly for potentially vulnerable groups, can best be understood in the NHS, a health care system that is, unlike most in the world, free at the point of use.

The approach we used to further specify the review question was highly iterative, modifying the question in response to search results and findings from retrieved items. It treated, as Eakin and Mykhalovskiy [ 16 ] suggest, the question as a compass rather than an anchor, and as something that would not finally be settled until the end of the review. In the process of refining the question, we benefited from the multidisciplinary nature of our review team: this allowed a range of perspectives to be incorporated into the process, something that was also helpful and important in other elements of the review.

Searching the literature

A defining characteristic of conventional systematic review methodology is its use of explicit searching strategies, and its requirement that reviewers be able to give a clear account of how they searched for relevant evidence, such that the search methods can be reproduced. [ 2 ] Searching normally involves a range of strategies, but relies heavily on electronic bibliographic databases.

We piloted the use of a highly structured search strategy using protocol-driven searches across a range of electronic databases but, like Greenhalgh and Peacock [ 17 ] found that was this unsatisfactory. In particular, it risked missing relevant materials by failing to pick up papers that, while not ostensibly about "access", were nonetheless important to the aim of the review. We then developed a more organic process that fitted better with the emergent and exploratory nature of the review questions. This combined a number of strategies, including searching of electronic databases; searching websites; reference chaining; and contacts with experts. Crucially, we also used expertise within the team to identify relevant literature from adjacent fields not immediately or obviously relevant to the question of "access".

However, searching generated thousands of potentially relevant items – at one stage over 100,000 records. A literature of this size would clearly be unmanageable, and well exceed the capacity of the review team. We therefore redefined the aim of the searching phase. Rather than aiming for comprehensive identification and inclusion of all relevant literature, as would be required under conventional systematic review methodology, we saw the purpose of the searching phase as identifying potentially relevant papers to provide a sampling frame. Our sampling frame eventually totalled approximately 1,200 records.

Conventional systematic review methodology limits the number of papers to be included in a review by having tightly specified inclusion criteria for papers. Effectively, this strategy constructs the field to be known as having specific boundaries, defined as research that has specifically addressed the review question, used particular study designs and fulfilled the procedural requirements for the proper execution of these. Interpretive reviews might construct the field to be known rather differently, seeing the boundaries as more diffuse and ill-defined, as potentially overlapping with other fields, and as shifting as the review progresses. Nonetheless, there is a need to limit the number of papers to be included in an interpretive synthesis not least for practical reasons, including the time available. Sampling is also warranted theoretically, in that the focus in interpretive synthesis is on the development of concepts and theory rather than on exhaustive summary of all data. A number of authors [ 18 – 20 ] suggest drawing on the sampling techniques of primary qualitative research, including principles of theoretical sampling and theoretical saturation, when conducting a synthesis of qualitative literature.

For purposes of our synthesis, we used purposive sampling initially to select papers that were clearly concerned with aspects of access to healthcare, partly informed by an earlier scoping study [ 21 ] and later used theoretical sampling to add, test and elaborate the emerging analysis. Sampling therefore involved a constant dialectic process conducted concurrently with theory generation.

Determination of quality

Conventional systematic review methodology uses assessment of study quality in a number of ways. First, as indicated above, studies included in a review may be limited to particular study designs, often using a "hierarchy of evidence" approach that sees some designs (e.g. randomized controlled trials) as being more robust than others (e.g. case-control studies). Second, it is usual to devise broad inclusion criteria – for example adequate randomisation for RCTs – and to exclude studies that fail to meet these. Third, an appraisal of included studies, perhaps using a structured quality checklist, may be undertaken to allow sensitivity analyses aimed at assessing the effects of weaker papers.

Using this approach when confronted with a complex literature, including qualitative research, poses several challenges. No hierarchy of study designs exists for qualitative research. How or whether to appraise papers for inclusion in an interpretive reviews has received a great deal of attention, but there is little sign of an emergent consensus [ 22 ]. Some argue that formal appraisals of quality may not be necessary, and some argue that there is a risk of discounting important studies for the sake of "surface mistakes" [ 23 ]. Others propose that weak papers should be excluded from the review altogether, and several published syntheses of qualitative research have indeed used quality criteria to make decisions about excluding papers. [ 10 , 24 ]

We aimed to prioritise papers that appeared to be relevant, rather than particular study types or papers that met particular methodological standards. We might therefore be said to be prioritising "signal" (likely relevance) over "noise" (the inverse of methodological quality) [ 25 ]. We felt it important, for purposes of an interpretive review, that a low threshold be applied to maximise the inclusion and contribution of a wide variety of papers at the level of concepts . We therefore took a two-pronged approach to quality. First, we decided that only papers that were deemed to be fatally flawed would be excluded. Second, once in the review, the synthesis itself crucially involved judgements and interpretations of credibility and contribution, as we discuss later.

To identify fatally flawed papers, we used the criteria in Table 1 , adapted from those proposed (at the time of our review) by the National Health Service (NHS) National Electronic Library for Health for the evaluation of qualitative research, to inform judgements on the quality of the papers. These criteria were used for assessing all empirical papers (but not those classified as 'reviews') regardless of study type. The final judgement about inclusion of the review rested both on an assessment of relevance as well as on the assessment of the quality of the individual papers. Decisions about relevance and quality were recorded, and a small sample of decisions about relevance and quality was reviewed. In the event, very few papers – approximately 20 – were excluded on grounds of being "fatally flawed", because even weak papers were often judged to have potentially high relevance. The value of deferring judgements of credibility and contribution until the synthesis became increasingly evident.

Most fundamentally, as the review progressed, we became increasingly convinced that the assumption that all studies deemed to have satisfactorily fulfilled criteria of execution and reporting can contribute equally to a synthesis is flawed. As we discuss further below, one of the distinctive characteristics of a critical interpretive synthesis is its emphasis not only on summary of data reported in the literature but also on a more fundamental critique, which may involve questioning taken-for-granted assumptions.

Data extraction

A data-extraction pro-forma was initially devised to assist in systematically identifying characteristics of research participants, methods of data collection, methods of data analysis and major findings of each paper. For both qualitative and quantitative papers, this involved extracting the titles of the categories and sub-categories using the terms used in the paper itself and a summary of the relevant material. Practically, however, it proved impossible to conduct this form of data extraction on all documents included in the review, including very large documents. We therefore summarised some documents more informally, for example using highlighter pen. More generally, the value of formal data extraction for purposes of this type of study will require further evaluation.

Conducting an interpretive synthesis

We had intended, at the outset of this project, to use meta-ethnography, a method for interpretive synthesis where there is currently an active programme of methodological research, [ 9 – 11 ] as our approach to synthesis. However, this had previously only been used to synthesise qualitative studies. Our experiences of working with a large sample of papers using multiple methods led us to refine and respecify some of the concepts and techniques of meta-ethnography in order to enable synthesis of a very large and methodologically diverse literature. Eventually we had made so many amendments and additions to the original methodology that we felt it was more appropriate, helpful and informative to deem it a new methodology with its own title and processes. It is this approach which we term critical interpretive synthesis (CIS). It is important to emphasise, however, that CIS is an approach to review and is not solely a method for synthesis.

Meta-ethnography, as originally proposed [ 1 ], involves three major strategies.:

1. Reciprocal translational analysis (RTA). The key metaphors, themes, or concepts in each study report are identified. An attempt is then made to translate the concepts into each other. Judgements about the ability of the concept of one study to capture concepts from others are based on attributes of the themes themselves, and the concept that is "most adequate" is chosen.

2. Refutational synthesis. Contradictions between the study reports are characterised, and an attempt made to explain them.

3. Lines-of-argument synthesis (LOA) involves building a general interpretation grounded in the findings of the separate studies. The themes or categories that are most powerful in representing the entire dataset are identified by constant comparisons between individual accounts.

Reciprocal translational analysis

Reciprocal translational analysis involves translating findings of one paper into another by systematically comparing findings from each study, using techniques such as maps. [ 9 ] We encountered considerable methodological and practical problems in trying to apply RTA across a large set of papers, in part because of the kinds of iterations we were conducting in refining the sample. These meant that there were difficulties in identifying a stable "set" of papers on which an RTA could be conducted. RTA appears to be most suitable for a well-defined, relatively small (fewer than 50) and complete set of papers, because substitution or deletion of papers causes problems with both identifying index concepts and showing which concepts from other papers translate into these. A further problem is that, when confronted with a very large and diverse literature such as ours, RTA tends to provide only a summary in terms that have already been used in the literature. Although this may be a useful strategy as a stage on the way to a more interpretive synthesis, its value may be more limited than is the case for smaller samples of qualitative study reports where its benefits have been more evident.

Before our review, RTA had previously only been used for synthesising interpretive research, not a large and diverse body of literature, so this may be one reason why it was unsuccessful for our purposes. It is important to distinguish between the doubtful value of RTA in our synthesis (particularly because of the size and diversity of the literature), and the doubtful use of RTA in general. The diversity of the literature would also have prevented us from undertaking an aggregative synthesis using meta-analysis, but this clearly could not be read as a criticism of meta-analysis itself, but of its limitations when applying it to a diverse literature.

Lines of argument synthesis

Recent work [ 9 – 11 ] has innovated in the methodology of lines-of-argument (LOA) synthesis originally proposed by Noblit and Hare by building on Schutz's [ 26 ] notions of "orders" of constructs Schutz used the idea of "first order construct" to refer to the everyday understandings of ordinary people and "second order construct" to refer to the constructs of the social sciences. The explanations and theories used by authors in primary study reports could therefore be seen as second order interpretations. This recent work uses LOA synthesis to develop what are referred to as "third order" interpretations, which build on the explanations and interpretations of the constituent studies, and are simultaneously consistent with the original results while extending beyond them. Our experiences have led us to respecify some of this approach.

We suggest that the appropriate way of conceptualising the output of an LOA synthesis is as a synthesising argument . This argument integrates evidence from across the studies in the review into a coherent theoretical framework comprising a network of constructs and the relationships between them. Its function is to provide more insightful, formalised, and generalisable ways of understanding a phenomenon. A synthesising argument can be generated through detailed analysis of the evidence included in a review, analogous to the analysis undertaken in primary qualitative research. It may require the generation of what we call synthetic constructs , which are the result of a transformation of the underlying evidence into a new conceptual form. Synthetic constructs are grounded in the evidence, but result from an interpretation of the whole of that evidence, and allow the possibility of several disparate aspects of a phenomenon being unified in a more useful and explanatory way.

What we have called a "synthetic construct" might also be seen as a "third order construct". We suggest that the term "synthetic construct" is a more useful term because it is more explicit, and also because we emphasise that a synthesising argument need not consist solely of synthetic constructs. Instead, synthesising arguments may explicitly link not only synthetic constructs, but also second order constructs already reported in the literature. In effect, therefore, our approach does not make this precise distinction between second and third order constructs.

Refutational syntheses

We further suggest that what Noblit and Hare [ 1 ] call "refutational syntheses" are best conducted as part of the analysis that produces the synthesising argument Few published meta-ethnographies have in fact reported a separate refutational synthesis. It is, we suggest, more productive instead to adopt a critical and reflexive approach to the literature, including consideration of contradictions and flaws in evidence and theory.

An important element of producing a synthesising argument is the need, when conducting the analysis, to consider and reflect on the credibility of the evidence, to make critical judgements about how it contributes to the development of the synthesising argument, and to root the synthesising argument appropriately in critique of existing evidence. Clearly, credibility depends on the quality of the research, its currency, and the robustness of its theoretical base. But more generally, a critical interpretive synthesis is critical in the broader sense of critique rather than this more limited sense of critical appraisal, in which each study is judged against the standards of its type. Critique may involve identification of the research traditions or meta-narratives that have guided particular fields of research [ 27 ] as well as critical analysis of particular forms of discourses. Its aim is therefore to treat the literature as warranting critical scrutiny in its own right.

Conducting the analysis

Our analysis of the evidence, in order to produce a synthesising argument, was similar to that undertaken in primary qualitative research. We began with detailed inspection of the papers, gradually identifying recurring themes and developing a critique. We then generated themes that helped to explain the phenomena being described in the literature, constantly comparing the theoretical structures we were developing against the data in the papers, and attempting to specify the categories of our analysis and the relationships between them. To facilitate the process of identifying patterns, themes, and categories across the large volumes of text-based data in our study, we used QSR N5 software. However, it is important to note that, as with any qualitative analysis, full transparency is not possible because of the creative, interpretive processes involved. Nonetheless, the large multidisciplinary team involved in the review, and the continual dialogue made necessary by this, helped to introduce "checks and balances" that guarded against framing of the analysis according to a single perspective.

A key feature of this process that distinguishes it from some other current approaches to interpretive synthesis (and indeed of much primary qualitative research) was its aim of being critical : its questioning of the ways in which the literature had constructed the problematics of access, the nature of the assumptions on which it drew, and what has influenced its choice of proposed solutions. Our critique of the literature was thus dynamic, recursive and reflexive, and, rather than being a stage in which individual papers are excluded or weighted, it formed a key part of the synthesis, informing the sampling and selection of material and playing a key role in theory generation.

Findings: access to healthcare by socio-economically disadvantaged people

Our critical interpretive synthesis of the literature on access to healthcare by socio-economically disadvantaged people in the UK included 119 papers. Early analytic categories were tentative and contingent, but gradually became firmed up and more highly specified as our analysis continued. Our synthesis involved a critique of the tendency to use measures of utilisation as a means of assessing the extent to which access to healthcare is equitable. It further involved the generation of a synthesising argument that has the synthetic construct of candidacy at its core. For space reasons, we can report here only a brief illustrative summary.

Critique of utilisation as a measure of access

Much of the evidence on whether access to healthcare in the UK is equitable has relied on measuring utilisation of health services. This approach measures the units of healthcare (consultations, procedures, etc) that people have actually consumed. The literature suggests that different groups have identifiable patterns of use of services, but the significance of these is often difficult to interpret. General practice (GP) consultation rates among socio-economically disadvantaged people have generally been found to be higher [ 28 , 29 ] though some recent work has suggested that social class variables are generally insignificant in explaining health service use [ 30 ] Studies that have attempted to adjust for need, usually on the basis of estimates of morbidity, have generally suggested that the apparent excess of GP consultation can be explained by higher need [ 31 ].

Our critique of the literature suggests that utilisation is a generally unhelpful measure of equity of access. Not only do the logistical and practical problems of conducting utilisation studies pose substantial threats to validity and reliability, these studies are problematic for other reasons. They rely on a largely untested set of normative (i.e. ideas about how the world ought to be) and somewhat questionable assumptions about the "correct" level of utilisation, and on a difficult-to-measure (or conceputalise) estimates of "need". They often invoke normative assumptions about need relative to some apparently privileged though often ill-defined reference group (such as "affluent" people), and therefore risk failing to identify problems in access for that reference group. Misleadingly reassuring results may be produced that indicate that "need" and use or receipt are proportionate. We argue that utilisation, or, more appropriately, receipt of healthcare is the outcome of many different complex processes, which all need to be recognised if access is to be properly understood.

Our analysis suggested that a focus instead on candidacy , a synthetic construct that we generated during the course of our analysis, would demonstrate the vulnerabilities associated with socio-economic disadvantage, emphasise the highly dynamic, multi-dimensional and contingent character of access, and allow a more insightful interpretation of the evidence on receipt of healthcare.

Our synthesising argument around access to healthcare by socio-economically disadvantaged people is organised around a set of central concepts and, in particular, the core synthetic category of "candidacy". Candidacy functions as a synthetic construct because it is the product of the transformation of the evidence into a new conceptual form. It is distinct from earlier uses of the term "candidacy", including its use in the lay epidemiology of heart disease [ 32 ].

We have defined candidacy as follows: candidacy describes the ways in which people's eligibility for medical attention and intervention is jointly negotiated between individuals and health services. Our synthesising argument runs as follows: candidacy is a dynamic and contingent process, constantly being defined and redefined through interactions between individuals and professionals, including how "cases" are constructed. Accomplishing access to healthcare requires considerable work on the part of users, and the amount, difficulty, and complexity of that work may operate as barriers to receipt of care. The social patterning of perceptions of health and health services, and a lack of alignment between the priorities and competencies of disadvantaged people and the organisation of health services, conspire to create vulnerabilities. Candidacy is managed in the context of operating conditions that are influenced by individuals, the setting and environment in which care takes place, situated activity, the dynamics of face-to-face activity, and aspects of self (such as gender), the typifications staff use in categorising people and diseases, availability of economic and other resources such as time, local pressures, and policy imperatives.

Identification of candidacy

How people recognise their symptoms as needing medical attention or intervention is clearly key to understanding how they assert a claim to candidacy. Our analysis suggests that people in more deprived circumstances are likely to manage health and to recognise candidacy as a series of crises. There is significant evidence of lower use of preventive services among more deprived groups, [ 33 , 34 ] as well as evidence of higher use of accident and emergency facilities, emergency admissions and out-of-hours use [ 35 , 35 , 37 , 38 ]. Among more deprived groups, there is a tendency to seek help in response to specific events that are seen as warranting candidacy. "Warning signs" may be downgraded in importance by socio-economically disadvantaged populations because of a lack of a positive conceptualisation of health, [ 39 , 40 ] the normalisation of symptoms within deprived communities [ 41 – 43 ], and fear of being "blamed" by health professionals [ 44 ].

Using services requires considerable work on the part of people. First, people must be aware of the services on offer, and there has been persistent concern that more deprived people may lack awareness of some services [ 45 , 46 ]. Second, using health services requires the mobilisation of a range of practical resources that may be variably available in the population. A key practical resource that impacts on the ability to seek care for the socio-economically disadvantaged, for example, is transport [ 44 , 47 , 48 ]. Other practical resources that may impact on the ability of disadvantaged groups to negotiate health services include more rigid patterns of working life [ 47 ]. Goddard and Smith [ 49 ] summarise evidence suggesting that those from more deprived social groups face financial costs of attending health services which, though not sufficient to dissuade them from using services when they are ill (i.e. in response to a specific "event"), act as a barrier to attending "optional" services related to health promotion and health prevention

The permeability of services

Patterns of use of health services reflect issues in the organisation of services as much as they reflect a tendency to manage health as a series of crises on the part of disadvantaged people. We generated the synthetic construct of "permeability" to refer to the ease with which people can use services. Porous services require few qualifications of candidacy to use them, and may require the mobilisation of relatively fewer resources. Such services might include Accident and Emergency departments. Services that are less permeable demand qualifications (such as a referral), and also demand a higher degree of cultural alignment between themselves and their users, particularly in respect of the extent to which people feel comfortable with the organisational values of the service. Such services might include out-patients clinics in hospitals.

Services that are less permeable tend to have high levels of default by socio-economically disadvantaged people [ 50 – 53 ]. Appointments systems, for example, are a threat to permeability by socio-economically disadvantaged people because they require resources and competencies (including stable addresses, being able to read, and being able to present in particular places at particular times [ 33 , 50 , 54 ] In addition, the extent to which people feel alienated from the cultural values of health services and their satisfaction with services have important implications for which services they choose to use [ 41 , 55 ].

Appearances at health services

Appearing at health services involves people in asserting a claim to candidacy for medical attention or intervention. Whatever the nature of the claim, making it clearly involves work that requires a set of competencies, including the ability to formulate and articulate the issue for which help is being sought, and the ability to present credibly. More deprived people are at risk in these situations: they may be less used to or less able to provide coherent abstracted explanations of need, and may feel intimidated by their social distance from health professionals. Sword [ 56 ] points out that people with low incomes may feel alienated by the power relations that often characterise encounters with professionals. Dixon et al [ 57 ] and, in the US, Cooper and Roter [ 58 ] suggest that middle class people may be more adept at using their "voice" to demand better and extensive services: they may be more articulate, more confident, and more persistent, while people from lower class backgrounds are less verbally active. Somerset et al [ 59 ] report that in making referral decisions, patients' social status and their ability to articulate verbally act as background (and unexpressed) influences that affect the likelihood of referral.

Adjudications

Once a patient has asserted their candidacy by presenting to health services, the professional judgements made about that candidacy strongly influence subsequent access to attention and interventions. We generated the synthetic construct of "adjudication" to refer to the judgements and decisions made by professionals which allow or inhibit continued progression of candidacy. May et al's [ 60 ] analysis suggests doctors' practices are often exercised through a repertoire of routine judgements about the possibilities presented by individual patients and the routinely available means of solving these. These typifications are, we suggest, strongly influenced by local conditions, including the operating conditions in which practitioners work and sensitivity to resource constraints. Candidacy of socially disadvantaged people appears to be at risk of being judged to be less eligible, at least for some types of interventions, although the evidence that this happens is not particularly strong.

Our analysis suggests that it is likely that professionals' perceptions of patients who are likely to "do well" as a result of interventions may disadvantage people in more deprived circumstances. As Hughes and Griffiths [ 61 ] identify, clinical decisions may rest on often implicit social criteria about which patients "ought" to receive care People in disadvantaged groups are more likely to smoke, to be overweight and to have co-morbidities, and professional perceptions of the cultural and health capital required to convert a unit of health provision into a given unit of health gain may function as barriers to healthcare [ 34 ]. In addition, perceptions of social "deservingness" may play a role [ 61 , 62 ]. Goddard and Smith [ 49 ] summarise evidence suggesting that independent of the severity of the disease, some GPs are more likely to refer the economically active and those with dependants. Clearly, there is potential for socially disadvantaged people to be disfavoured in such decisions.

Offers and resistance

Much of the work on utilisation of healthcare explicitly or implicitly assumes that non-utilisation is a direct reflection of non-offer. However, this type of normative analysis fails to acknowledge that people may choose to refuse offers. There is some evidence of patterns of resistance to offers. Referral implies that a GP has identified particular features of candidacy and is seeking to match those to a service that deals with that form of candidacy, but patients can resist being referred [ 42 , 63 ] and can resist offers of medication [ 64 , 65 ].

Operating conditions and the local production of candidacy

A small body of recent research has identified what might be called local influences on the production of candidacy, and in our analysis these are hugely important. These are the contingent and locally specific influences on interactions between practitioners and patients, which may be emergent over time through repeated encounters. Crucial to the local production of candidacy is the perceived or actual availability and suitability of resources to address that candidacy [ 60 , 63 ].

Demands from health policy-makers and managers for syntheses of evidence that are useful, rigorous and relevant are fuelling interest in the development of methods that can allow the integration of diverse types of evidence [ 66 ]. With the diversity of techniques for evidence synthesis now beginning to appear, those using existing, 'new' or evolving techniques need to produce critical reflexive accounts of their experiences of using the methods [ 3 ]. Our experience of conducting a review of access to healthcare, where there is a large, amorphous and complex body of literature, and a need to assemble the findings into a form that is useful in informing policy and that is empirically and theoretically grounded [ 67 ], has led us to propose a new method – Critical Interpretive Synthesis – which is sensitised to the kinds of processes involved in conventional systematic review while drawing on a distinctively qualitative tradition of inquiry.

Conventional systematic review methodology is well-suited to aggregative syntheses, where what is required is a summary of the findings of the literature under a set of categories which are largely pre-specified, secure, and well-defined. It has been important in drawing attention to the weaknesses of informal reviews, including perceived failures in their procedural specification and the possibility that the (thus) undisciplined reviewer might be chaotic or negligent in identifying the relevant evidence, or might construct idiosyncratic theories and marshall the evidence in support of these. It has thus revealed some of the pitfalls of informal literature review. Conventional systematic review methodology has demonstrated considerable benefits in synthesising certain forms of evidence where the aim is to test theories (in the form of hypotheses), perhaps especially about "what works". However, this approach is limited when the aim, confronted with a complex body of evidence, is to generate theory [ 15 , 27 ].

Current methods for conducting an interpretive synthesis of the literature, (such as meta-ethnography) are also limited, in part because application of many interpretive methods for synthesis has remained confined to studies reporting qualitative research. Realist synthesis [ 68 ], which does include diverse forms of evidence, is oriented towards theory evaluation, in particular by focusing on theories of change. Methods for including qualitative and quantitative evidence in systematic reviews developed by the EPPI Centre at the Institute of Education, London, have involved refinements and extensions of conventional systematic review methodology [ 6 – 8 ], and have limited their application of interpretive techniques to synthesis of qualitative evidence.

More generally, many current approaches fail to be sufficiently critical , in the sense of offering a critique. There is rarely an attempt to reconceptualise the phenomenon of interest, to provide a more sweeping critique of the ways in which the literature in the area have chosen to represent it, or to question the epistemological and normative assumptions of the literature. With notable exceptions such as the recent approach of meta-narrative analysis [ 15 ], critique of papers in current approaches to review tends to be limited to appraisal of the methodological specificities of the individual papers.

Conducting an interpretive review of the literature on access to healthcare by vulnerable groups in the UK therefore required methodological innovation that would be alert to the issues raised by systematic review methodology but also move beyond both its limitations and those of other current interpretive methods. The methods for review that we developed in this project (Table 2 ) built on conventional systematic review methodology in their sensitivity to the need for attentiveness to a range of methodological processes. Crucially, in doing so, we drew explicitly on traditions of qualitative research inquiry, and in particular on the principles of grounded theory [ 5 ].

In addition to its explicit orientation towards theory generation, perhaps what most distinguishes CIS from conventional systematic review methods is its rejection of a "stage" approach to review. Processes of question formulation, searching, selection, data extraction, critique and synthesis are characterised as iterative, interactive, dynamic and recursive rather than as fixed procedures to be accomplished in a pre-defined sequence. CIS recognises the need for flexibility in the conduct of review, and future work would need to assess how far formal methods of critical appraisal and data extraction will be essential elements of the method. Our experience suggests that while attention to scientific quality is required, more generally the emphasis should be on critique rather than critical appraisal, and an ongoing critical orientation to the material examined and to emerging theoretical ideas. Formal data extraction may also be an unnecessarily constraining and burdensome process.

CIS emphasises the need for theoretical categories to be generated from the available evidence and for those categories to be submitted to rigorous scrutiny as the review progresses. Further, it emphasises a need for constant reflexivity to inform the emerging theoretical notions, and guides the sampling of articles. Although CIS demands attention to flaws in study design, execution and reporting in our judgements of the quality of individual papers, its critical approach goes beyond standard approaches. Thus, in our review, some methodologically weak papers were important in terms of their theoretical contribution, or in terms of demonstrating the breadth of evidence considered in the construction of particular categories, or in terms of providing a more comprehensive summary of the evidence, while a single strong paper might be pivotal in the development of the synthesis. Hughes and Griffiths' paper on micro-rationing of healthcare [ 61 ], for example, was a key paper in helping to generate the construct of candidacy that later came to unify the themes of our analysis. The critical interpretation in our analysis focused on how a synthesising argument could be fashioned from the available evidence, given the quality of the evidence and the kinds of critiques that could be offered of the theory and assumptions that lay behind particular approaches. In treating the literature as an object of scrutiny in its own right, CIS problematises the literature in ways that are quite distinctive from most current approaches to literature reviewing.

Access to healthcare

The CIS approaches we adopted deferred final definition of the phenomenon of access and the appropriate ways of conceptualising it until our analysis was complete. Our critique of the current literature focused on the inadequacies of studies of utilisation as a guide to explaining inequities in health care. The conceptual model of access that we developed emphasises candidacy as the core organising construct, and recasts access as highly dynamic and contingent, and subject to constant negotiation.

In this conceptual model of access to healthcare, health services are continually constituting and seeking to define the appropriate objects of medical attention and intervention, while at the same time people are engaged in constituting and defining what they understand to be the appropriate objects of medical attention and intervention. Candidacy describes how people's eligibility for healthcare is determined between themselves and health services. Candidacy is a continually negotiated property of individuals, subject to multiple influences arising both from people and their social contexts and from macro-level influences on allocation of resources and configuration of services. "Access" represents a dynamic interplay between these simultaneous, iterative and mutually reinforcing processes. By attending to how vulnerabilities arise in relation to candidacy, the phenomenon of access can be much better understood, and more appropriate recommendations made for policy, practice and future research. Although our review focused on the UK, we suggest that the construct of candidacy is transferable, and has useful explanatory value in other contexts.

In addition to the core construct of candidacy, our analysis required the production of a number of other linked synthetic constructs – constructs generated through an attempt to summarise and integrate diverse concepts and data – including "adjudications" and "offers". It was also possible to link existing "second order" constructs, for example relating to help-seeking as the identification of candidacy by patients, into the synthesising argument, and making these work as synthesising constructs. We feel that this approach allows maximum benefit to be gained from previous analyses as well as the new synthesis.

Reflections on the method

Clearly, questions can be raised about the validity and credibility of the CIS analysis we have presented here. Conventional systematic review methodology sets great store by the reproducibility of its protocols and findings. It would certainly have been possible to produce an account of the evidence that was more reproducible. For example, we could have used the evidence to produce a thematic summary that stuck largely to the terms and concepts used in the evidence itself. However, we felt it important that we produced an interpretation of the evidence that could produce new insights and fresh ways of understanding the phenomenon of access, and that the "critical voice" of our interpretation was maintained throughout the analysis. Simply to have produced a thematic summary of what the literature was saying would have run the risk of accepting that the accounts offered in the evidence-base were the only valid way of understanding the phenomenon of access to healthcare by vulnerable groups. We therefore make no claim to reproducibility, but wish to address some possible concerns. First, it could be argued that a different team using the same set of papers would have produced a different theoretical model. However, the same would be true for qualitative researchers working with primary qualitative data, who accept that other possible interpretations might be given to, say, the same set of transcripts. Clearly, the production of a synthesizing argument, as an interpretive process, produces one privileged reading of the evidence, and, as the product of an authorial voice, it cannot be defended as an inherently reproducible process or product. We would suggest, however, that our analysis can be defended on the grounds that it is demonstrably grounded in the evidence; that it is plausible; that it offers insights that are consistent with the available evidence; and that it can generate testable hypotheses and empirically valuable questions for future research.

Second, subjecting a question to continual review and refinement, as we did, may make it more difficult for those conducting critical interpretive reviews to demonstrate, as required by conventional systematic review methodology, the "transparency", comprehensiveness, and reproducibility of search strategies. This dilemma between the "answerable" question and the "meaningful" question has received little attention, but it underpins key tensions between the two ends of the academic/pragmatic systematic review spectrum. On balance, faced with a large and amorphous body of evidence in an area such as access to healthcare, and given the aims of an interpretive synthesis, we feel that our decision not to limit the focus of the review at the outset, and our subsequent sampling strategies, were well justified. Our decision not to commit to a particular view of what access might be and how it should be assessed at the outset of the project was critical to our subsequent development of a more satisfactory understanding of access.

Third, it could be argued that we have synthesized too small a sample of the available papers, or that the processes used to select the papers are not transparent. We recognize that we have analyzed and synthesized only a fraction of all relevant papers in the area of access to healthcare by vulnerable groups. However, a common strategy in conventional systematic review is to limit the study types to be included; this strategy also might result in only a proportion of the potentially relevant literature being synthesised. While we have described our methods for sampling as purposive, it is possible that another team using the same approach could have come up with a different sample, because, particularly in the later stages of our review, our sampling was highly intuitive and guided by the emerging theory.

The final version of the conceptual model of access to healthcare that we eventually developed did not emerge until quite late in the review process, and much of the later sampling was directed at testing and purposively challenging the theory as we began to develop it. Again, such forms of searching and sampling do not lend themselves easily to reproducibility or indeed auditability. Testing whether the interpretations change in response to different findings will be an important focus for future research, which will also need to evaluate whether apparently disconfirming evidence is the result of methodological flaws or poses a genuine challenge to theory.

Conducting interpretive reviews in challenging areas where there is a large body of diverse evidence demands an approach that can draw on the strengths of conventional systematic review methodology and on the recent advances in methods for interpretive synthesis. We have termed the approach we developed to this review "critical interpretive synthesis". We believe that this methodology offers the potential for insight, vividness, illumination, and reconceptualisation of research questions, particularly in challenging areas such as access to healthcare, and look forward to further evaluations of its application.

Noblit G, Hare R: Meta-ethnography: synthesising qualitative studies. 1988, Newbury Park, CA: Sage

Chapter   Google Scholar  

Egger M, Davey Smith G, Phillips A: Meta-analysis: principles and procedures. BMJ. 1997, 315: 1533-1537.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Dixon-Woods M, Bonas S, Booth A, Jones DR, Miller T, Sutton AJ, Shaw RL, Smith JA, Young B: How can systematic reviews incorporate qualitative research? A critical perspective. Qualitative Research. 2006, 6: 27-44. 10.1177/1468794106058867.

Article   Google Scholar  

Dixon-Woods M, Agarwal S, Jones D, Young B, Sutton A: Synthesising qualitative and quantitative evidence: a review of possible methods. J Health Serv Res Policy. 2005, 10: 45-53. 10.1258/1355819052801804.

Article   PubMed   Google Scholar  

Glaser B, Strauss A: The discovery of grounded theory: strategies for qualitative research. 1967, New York: Aldine de Gruyter

Google Scholar  

Shepherd J, Harden A, Rees R, Brunton G, Garcia J, Oliver S, Oakley A: Young people and healthy eating: a systematic review of research on barriers and facilitators. Health Educ Res. 2006, 21: 239-257. 10.1093/her/cyh060.

Article   CAS   PubMed   Google Scholar  

Harden A, Garcia J, Oliver S, Rees R, Shepherd J, Brunton G, Oakley A: Applying systematic review methods to studies of people's views: an example from public health research. J Epidemiol Community Health. 2004, 58: 794-800. 10.1136/jech.2003.014829.

Article   PubMed   PubMed Central   Google Scholar  

Thomas J, Harden A, Oakley A, Oliver S, Sutcliffe K, Rees R, Brunton G, Kavanagh J: Integrating qualitative research with trials in systematic reviews. BMJ. 2004, 328: 1010-1012. 10.1136/bmj.328.7446.1010.

Pound P, Britten N, Morgan M, Yardley L, Pope C, Daker-White G, Campbell R: Resisting medicines: a synthesis of qualitative studies of medicine taking. Soc Sci Med. 2005, 61: 133-155. 10.1016/j.socscimed.2004.11.063.

Campbell R, Pound P, Pope C, et al: Evaluating meta-ethnography: a synthesis of qualitative research on lay experiences of diabetes and diabetes care. Soc Sci Med. 2003, 56: 671-684. 10.1016/S0277-9536(02)00064-3.

Britten N, Campbell R, Pope C, Donovan J, Morgan M, Pill R: Using meta-ethnography to synthesise qualitative research: a worked example. J Health Serv Res Policy. 2002, 7: 209-215. 10.1258/135581902320432732.

NHS Centre for Reviews and Dissemination: Undertaking systematic reviews of research on effectiveness: CRD's guidance for those carrying out or commissioning reviews. 2001, York: NHS CRD

Alderson P, Green S, Higgins JP: Cochrane Reviewers' Handbook 4.2. 2004, The Cochrane Library, 2

Jensen L, Allen M: Meta-synthesis of qualitative findings. Qual Health Res. 1996, 6: 553-560.

Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O, Peacock R: Storylines of research in diffusion of innovation: a meta-narrative approach to systematic review. Soc Sci Med. 2005, 61: 417-430. 10.1016/j.socscimed.2004.12.001.

Eakin JM, Mykhalovskiy E: Reframing the evaluation of qualitative health research: reflections on a review of appraisal guidelines in the health sciences. J Eval Clin Pract. 2003, 9: 187-194. 10.1046/j.1365-2753.2003.00392.x.

Greenhalgh T, Peacock R: Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. BMJ. 2005, 331: 1064-1065. 10.1136/bmj.38636.593461.68.

Schreiber RCD, Stern P: Qualitative meta-analysis. In: Completing a qualitative project: details and dialogue. Edited by: Morse JM. 1997, Thousand Oaks, CA: Sage

Booth A: Cochrane or cock-eyed? How shoul we conduct systematic reviews of qualitative research?. Proceedings of the Qualitative Evidence-based Practice Conference, Taking a Critical Stance. 2001, Coventry University, May 14 2001

Paterson B, Thorne S, C C, Jillings C: Meta-study of qualitative health research. 2001, Thousand Oaks: Sage

Gulliford M, Hughes D, Figeroa-Munoz J, Hudson M, Connell P, Morgan M, Beech R, Gibson B, Arumugam C, Mohiddin A, Sedgwick J: Access to health care: Report of a scoping exercise for NHS Service Delivery and Organisation R&D (NCCSDO). National Co-ordinating Centre for NHS Service Delivery and Organisation Research. 2001

Dixon-Woods M, Shaw RL, Agarwal S, Smith JA: The problem of appraising qualitative research. Qual Saf Health Care. 2004, 13: 223-225. 10.1136/qshc.2003.008714.

Sandelowski M, Docherty S, Emden C: Focus on qualitative methods. Qualitative meta-synthesis: issues and techniques. Res Nurs Health. 1997, 20: 365-371. 10.1002/(SICI)1098-240X(199708)20:4<365::AID-NUR9>3.0.CO;2-E.

Paterson B, Thorne S, Dewis M: Adapting to and managing diabetes. Image. 1998, 30: 57-62.

CAS   Google Scholar  

Edwards A, Elwyn G, Hood K, Rollnick S: Judging the 'weighting of evidence' in systematic reviews: introducing rigour into the qualitative overview stage by assessing Signal and Noise. J Eval Clin Pract. 2000, 6: 177-184. 10.1046/j.1365-2753.2000.00212.x.

Schutz A: Collected Papers. 1962, The Hague: Martinus Nijhoff, 1:

Greenhalgh T, Robert G, Bate P, Kyriakidou O, Macfarlane F, Peacock R: A systematic review of the literature on diffusion, dissemination and sustainability of innovations in health service delivery. 2004, London: NHS SDO Programme

Carr-Hill R, Rice N, Roland M: Socioeconomic determinants of rates of consultation in general practice based on fourth national morbidity survey of general practices. BMJ. 1996, 312: 1008-1012.

Carlisle R, Johnstone S: The relationship between census-derived socio-economic variables and general practice consultation rates in three town centre practices. Br J Gen Pract. 1998, 48: 1675-1678.

CAS   PubMed   PubMed Central   Google Scholar  

Sutton M, Gravelle H, Leyland A, Morris S, Muirhead M, Noble M, Parkin D, Windmeijer F: Small area study of supply and demand determinants of healthcare resource use and estimation of relative needs for primary care trusts in England. English Department of Health/Advisory Committee on Resource Allocation. 2002

Wyke S, Hunt K, Walker J, Wilson P: Frequent attendance, socio-economic status and burden of ill health: an investigation in the West of Scotland. Eur J Gen Pract. 2003, 9: 48-

Davison C, Davey Smith G, Frankel S: Lay epidemiology and the prevention paradox: the implications of coronary candidacy for health education. Sociol Health Illn. 1991, 13: 1-19. 10.1111/1467-9566.ep11340301.

Martin R, Sterne J: Social and economic variation in general practice consultation rates amongst men aged 16–39. Health Statistics Quarterly. 2001, 09: 29-36.

Watt G: The inverse care law today. Lancet. 2002, 360: 252-254. 10.1016/S0140-6736(02)09466-7.

Majeed F, Cook D, Hilton S, Poloniecki J, Hagen A: Annual night visiting rates in 129 general practices in one family health services authority: Association with patient and general practice characteristics. Br J Gen Pract. 1995, 45: 531-535.

Reid F, Cook D, Majeed A: Explaining variation in hospital admission rates between general practices: cross sectional study. BMJ. 1999, 319: 98-103.

Blatchford O, Capewell S, Murray S, Blatchford M: Emergency medical admissions in Glasgow: general practices vary despite adjustment for age, sex, and deprivation. Br J Gen Pract. 1999, 49: 551-554.

Carlisle R, Groom L, Avery A, Boot D, Earwicker S: Relation of out-of-hours activity by general practice and accident and emergency services with deprivation in Nottingham: longitudinal survey. BMJ. 1998, 316: 520-523.

Cornwell J: Hard Earned Lives. 1984, London: Tavistock

Calnan M: Health and Illness: the lay perspective. Tavistock. 1987

Richards H, Reid M, Watt C: Socioeconomic variations in responses to chest pain: qualitative study. BMJ. 2002, 324: 1308-1310. 10.1136/bmj.324.7349.1308.

Gardner K, Chapple A: Barriers to referral in patients with angina: qualitative study. BMJ. 1999, 319: 418-421.

Hunt K, Emslie C, Watt G: Lay constructions of a family history of heart disease: potential for misunderstandings in the clinical encounter?. Lancet. 2001, 357: 1168-1171. 10.1016/S0140-6736(00)04334-8.

Tod A, Read C, Lacey A, Abbott J: Barriers to uptake of services for coronary heart disease: qualitative study. BMJ. 2001, 323: 214-217. 10.1136/bmj.323.7306.214.

Anderson E, Petersen S, Wailoo M: Health concerns and needs of traveller families. Health Visitor. 1997, 70: 148-150.

House of Commons Health Committee: Inequalities in access to maternity services. Eighth report of session 2002–2003. 2003, London: Stationery Office

Field K, Briggs D: Socio-economic and locational determinants of accessibility and utilization of primary health-care. Health Soc Care Community. 2001, 9: 294-308. 10.1046/j.0966-0410.2001.00303.x.

Shipman C, Payne F, Dale J, Jessopp L: Patient-perceived benefits of and barriers to using out-of-hours primary care centres. Fam Pract. 2001, 18: 149-155. 10.1093/fampra/18.2.149.

Goddard M, Smith P: Equity of access to health care. 1998, York: University of York

George A, Rubin G: Non-attendance in general practice: A systematic review and its implications for access to primary health care. Fam Pract. 2003, 20: 178-184. 10.1093/fampra/20.2.178.

McClure R, Newell S, Edwards S: Patient characteristics affecting attendance at general outpatient clinics. Arch Dis Child. 1996, 74: 121-125.

Macleod U, Ross S, Twelves C, George W, Gillis C, Watt G: Primary and secondary care management of women with early breast cancer from affluent and deprived areas: retrospective review of hospital and general practice records. BMJ. 2000, 320: 1442-1445. 10.1136/bmj.320.7247.1442.

Sharp D, Hamilton W: Non-attendance at general practices and outpatient clinics. BMJ. 2002, 323: 1081-1082. 10.1136/bmj.323.7321.1081.

van Cleemput P: Health care needs of travellers. Arch Dis Child. 2000, 82: 32-37. 10.1136/adc.82.1.32.

Rogers A, Hassell K, Nicolas G: Demanding patients? Analysing the use of primary care. 1999, Buckingham: Open University Press

Sword W: A socio-ecological approach to understanding barriers to prenatal care for women of low income. J Adv Nurs. 1999, 29: 1170-1177. 10.1046/j.1365-2648.1999.00986.x.

Dixon A, Le Grand J, Henderson J, Murray R, Poteliakhoff E: Is the NHS equitable? A review of the evidence. London. LSE Health and Social Care. 2003

Cooper L, Roter D: Patient-provider communication: the effect of race and ethnicity on process and outcomes of healthcare. Unequal treatment: confronting racial and ethnic disparities in health care. Edited by: Smedley BD, Stith AY, Nelson AR. 2003, Washington DC: National Academies Press

Somerset M, Faulkner A, Shaw A, Dunn L, Sharp D: Obstacles on the path to a primary-care led National Health Service: complexities of outpatient care. Soc Sci Med. 1999, 48: 213-225. 10.1016/S0277-9536(98)00338-4.

May C, Allison G, Chapple A, Chew-Graham C, Dixon C, Gask L, Graham R, Rogers A, Roland M: Framing the doctor-patient relationship in chronic illness: a comparative study of general practitioners' accounts. Sociol Health Illn. 2004, 26: 135-158. 10.1111/j.1467-9566.2004.00384.x.

Hughes D, Griffiths L: Ruling in and Ruling out: two approaches to the micro-rationing of health care. Soc Sci Med. 1997, 44: 589-599. 10.1016/S0277-9536(96)00207-9.

Dodier N, Camus A: Openness and specialisation: dealing with patients in a hospital emergency service. Sociol Health Illn. 1998, 20: 413-444. 10.1111/1467-9566.00109.

Fuat A, Hungin A, Murphy J: Barriers to accurate diagnosis and effective management of heart failure in primary care: qualitative study. BMJ. 2003, 326: 196-200. 10.1136/bmj.326.7382.196.

Britten N, Stevenson F, Gafaranga J, Barry C, Bradley C: The expression of aversion to medicines in general practice consultations. Soc Sci Med. 2004, 59: 1495-1503. 10.1016/j.socscimed.2004.01.019.

Scahfheutle E, Hassell K, Noyce P, Weiss M: Access to medicines: cost as an influence on the views and behaviour of patients. Health Soc Care Community. 2002, 10: 187-195. 10.1046/j.1365-2524.2002.00356.x.

Sheldon TA: Making evidence synthesis more useful for management and policy-making. J Health Serv Res Policy. 2005, 10 (Suppl 1): 1-5. 10.1258/1355819054308521.

Dixon-Woods M, Fitzpatrick R, Roberts K: Including qualitative research in systematic reviews: problems and opportunities. J Eval Clin Pract. 2001, 7: 125-133. 10.1046/j.1365-2753.2001.00257.x.

Pawson R, Greenhalgh T, Harvey G, Walshe K: Realist review-a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005, 10 (Suppl 1): 21-34. 10.1258/1355819054308530.

Pre-publication history

The pre-publication history for this paper can be accessed here: http://www.biomedcentral.com/1471-2288/6/35/prepub

Download references

Acknowledgements

This project was funded by the NHS Service Delivery and Organisation (SDO) Research Programme.

Author information

Authors and affiliations.

Department of Health Sciences, University of Leicester, 22–28 Princess Road West, Leicester, LE1 6TP, UK

Mary Dixon-Woods, Ron Hsu, Lucy Smith, Richard Riley & Alex J Sutton

Division of Oncology/General Practice, University of Edinburgh, Edinburgh Centre for Neuro-Oncology, Western General Hospital, Crewe Road South, Edinburgh, EH4 2XU, UK

Debbie Cavers

Department of Health Sciences, University of Leicester, Leicester General Hospital, Leicester, LE5 4PW, UK

Shona Agarwal

Department of Sociology, University of Leicester, Leicester, LE1 7RH, UK

Ellen Annandale

School of Nursing, University of Nottingham, Queens Medical Centre, Nottingham, NG7 2HA, UK

Antony Arthur

Centre for Research in Social Policy, Loughborough University, Leicestershire, LE11 3TU, UK

Janet Harvey

Nuffield Research Unit, Department of Health Sciences, University of Leicester, 22–28 Princess Road West, Leicester, LE1 6TP, UK

Savita Katbamna & Richard Olsen

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Mary Dixon-Woods .

Additional information

Competing interests.

The author(s) declare that they have no competing interests.

Authors' contributions

MDW designed the project, led and supervised its execution, and drafted the manuscript. EA, AA, JH, RH, SK, RO, LS, RR and AJS participated in the design of the study. All authors engaged in searching, screening, sampling, data extraction, and critical appraisal/critique activities, and contributed to the thematic analysis. DC and SA managed the searching, maintained the databases and coded material using N6 software. All authors contributed to the draft of the manuscript.

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( http://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article.

Dixon-Woods, M., Cavers, D., Agarwal, S. et al. Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Med Res Methodol 6 , 35 (2006). https://doi.org/10.1186/1471-2288-6-35

Download citation

Received : 19 April 2006

Accepted : 26 July 2006

Published : 26 July 2006

DOI : https://doi.org/10.1186/1471-2288-6-35

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• National Health Service
• Review Question
• Disadvantaged People
• Order Construct
• Diverse Literature

BMC Medical Research Methodology

ISSN: 1471-2288

• General enquiries: [email protected]

Research to Action

The Global Guide to Research Impact

Social Media

Framing challenges

Synthetic literature reviews: An introduction

By Steve Wallis and Bernadette Wright 26/05/2020

Whether you are writing a funding proposal or an academic paper, you will most likely be required to start with a literature review of some kind. Despite (or because of) the work involved, a literature review is a great opportunity to showcase your knowledge on a topic. In this post, we’re going to take it one step further. We’re going to tell you a very practical approach to conducting literature reviews that allows you to show that you are advancing scientific knowledge before your project even begins. Also – and this is no small bonus – this approach lets you show how your literature review will lead to a more successful project.

Literature review – start with the basics

A literature review helps you shape effective solutions to the problems you (and your organisation) are facing. A literature review also helps you demonstrate the value of your activities. You can show how much you add to the process before you spend any money collecting new data. Finally, your literature review helps you avoid reinventing the wheel by showing you what relevant research already exists, so that you can target your new research more efficiently and more effectively.

We all want to conduct good research and have a meaningful impact on people’s lives. To do this, a literature review is a critical step. For funders, a literature review is especially important because it shows how much useful knowledge the writer already has.

Past methods of literature reviews tend to be focused on ‘muscle power’, that is spending more time and more effort to review more papers and adhering more closely to accepted standards. Examples of standards for conducting literature reviews include the PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions and the guidelines for assessing the quality and applicability of systematic reviews developed by the Task Force on Systematic Review and Guidelines . Given the untold millions of papers in many disciplines, even a large literature review that adheres to the best guidelines does little to move us toward integrated knowledge in and across disciplines.

In short, we need we need to work smarter, not harder!

Synthetic literature reviews

One approach that can provide more benefit is the synthetic literature review. Synthetic meaning synthesised or integrated, not artificial. Rather than explaining and reflecting on the results of previous studies (as is typically done in literature reviews), a synthetic literature review strives to create a new and more useful theoretical perspective by rigorously integrating the results of previous studies.

Many people find the process of synthesis difficult, elusive, or mysterious. When presenting their views and making recommendations for research, they tend to fall back on intuition (which is neither harder nor smarter).

After defining your research topic (‘poverty’ for example), the next step is to search the literature for existing theories or models of poverty that have been developed from research. You can use Google Scholar or your institutional database, or the assistance of a research librarian. A broad topic such as ‘poverty’, however, will lead you to millions of articles. You’ll narrow that field by focusing more closely on your topic and adding search terms. For example, you might be more interested in poverty among Latino communities in central California. You might also focus your search according to the date of the study (often, but not always, more recent results are preferred), or by geographic location. Continue refining and focusing your search until you have a workable number of papers (depending on your available time and resources). You might also take this time to throw out the papers that seem to be less relevant.

Skim those papers to be sure that they are really relevant to your topic. Once you have chosen a workable number of relevant papers, it is time to start integrating them.

Next, sort them according to the quality of their data.

Next, read the theory presented in each paper and create a diagram of the theory. The theory may be found in a section called ‘theory’ or sometimes in the ‘introduction’. For research papers, that presented theory may have changed during the research process, so you should look for the theory in the ‘findings’, ‘results’, or ‘discussion’ sections.

That diagram should include all relevant concepts from the theory and show the causal connections between the concepts that have been supported by research (some papers will present two theories, one before and one after the research – use the second one – only the hypotheses that have been supported by the research).

For a couple of brief and partial example from a recent interdisciplinary research paper, one theory of poverty might say ‘Having more education will help people to stay out of poverty’, while another might say ‘The more that the economy develops, the less poverty there will be’.

We then use those statements to create a diagram as we have in Figure 1.

Figure 1. Two (simple, partial) theories of poverty. (We like to use dashed lines to indicate ’causes less’, and solid lines to indicate ’causes more’)

When you have completed a diagram for each theory, the next step is to synthesise (integrate) them where the concepts are the same (or substantively similar) between two or more theories. With causal diagrams such as these, the process of synthesis becomes pretty direct. We simply combine the two (or more) theories to create a synthesised theory, such as in Figure 2.

Figure 2. Two theories synthesised where they overlap (in this case theories of poverty)

Much like a road map, a causal diagram of a theory with more concepts and more connecting arrows is more useful for navigation. You can show that your literature review is better than previous reviews by showing that you have taken a number of fragmented theories (as in Figure 1) and synthesised them to create a more coherent theory (as in Figure 2).

To go a step further, you may use Integrative Propositional Analysis (IPA) to quantify the extent to which your research has improved the structure and potential usefulness of your knowledge through the synthesis. Another source is our new book from Practical Mapping for Applied Research and Program Evaluation (see especially Chapter 5). (For the basics, you can look at Chapter One for free on the publisher’s site by clicking on the ‘Preview’ tab here. )

Once you become comfortable with the process, you will certainly be working ‘smarter’ and showcasing your knowledge to funders!

Contribute Write a blog post, post a job or event, recommend a resource

Partner with Us Are you an institution looking to increase your impact?

Most Recent Posts

• Call for Applications: 2024 L’Oréal-UNESCO For Women in Science Young Talents Programmes
• Keeping up with the AI revolution
• Four lessons for improving Theory of Change use in research uptake M&E
• Empowering Climate Action: The Essential Role of Knowledge Brokering
• Memorandum of Understanding-a Communication Tool

This Week's Most Read

• What do we mean by ‘impact’?
• How to write actionable policy recommendations
• 12ft Ladder: Making research accessible
• Gap analysis for literature reviews and advancing useful knowledge
• How to develop input, activity, output, outcome and impact indicators 
• Policymaker, policy maker, or policy-maker?
• Outcome Mapping: A Basic Introduction
• AEN Evidence 23 – Online Access Registration now open!
• Key questions to ask when putting together a Theory of Change for Research Uptake (Part 1 of 2)
• How to plan, write and communicate an effective Policy Brief: Three steps to success

Research To Action (R2A) is a learning platform for anyone interested in maximising the impact of research and capturing evidence of impact.

The site publishes practical resources on a range of topics including research uptake, communications, policy influence and monitoring and evaluation. It captures the experiences of practitioners and researchers working on these topics and facilitates conversations between this global community through a range of social media platforms.

R2A is produced by a small editorial team, led by CommsConsult . We welcome suggestions for and contributions to the site.

Subscribe to our newsletter!

Our contributors

Browse all authors

Friends and partners

• Global Development Network (GDN)
• Institute of Development Studies (IDS)
• International Initiative for Impact Evaluation (3ie)
• On Think Tanks
• Politics & Ideas
• Research for Development (R4D)
• Research Impact

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

• An introductory paragraph that explains what your working topic and thesis is
• A forecast of key topics or texts that will appear in the review
• Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
• Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically Evaluate: Mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

• Summarize the key findings you have taken from the literature and emphasize their significance
• Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

• Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
• Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
• Qualitative versus quantitative research
• Empirical versus theoretical scholarship
• Divide the research by sociological, historical, or cultural sources
• Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

• It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
• Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
• Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
• Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.

Handbook of eHealth Evaluation: An Evidence-based Approach [Internet].

Chapter 9 methods for literature reviews.

Guy Paré and Spyros Kitsiou .

9.1. Introduction

Literature reviews play a critical role in scholarship because science remains, first and foremost, a cumulative endeavour ( vom Brocke et al., 2009 ). As in any academic discipline, rigorous knowledge syntheses are becoming indispensable in keeping up with an exponentially growing eHealth literature, assisting practitioners, academics, and graduate students in finding, evaluating, and synthesizing the contents of many empirical and conceptual papers. Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generating new frameworks and theories; and (e) identifying topics or questions requiring more investigation ( Paré, Trudel, Jaana, & Kitsiou, 2015 ).

Literature reviews can take two major forms. The most prevalent one is the “literature review” or “background” section within a journal paper or a chapter in a graduate thesis. This section synthesizes the extant literature and usually identifies the gaps in knowledge that the empirical study addresses ( Sylvester, Tate, & Johnstone, 2013 ). It may also provide a theoretical foundation for the proposed study, substantiate the presence of the research problem, justify the research as one that contributes something new to the cumulated knowledge, or validate the methods and approaches for the proposed study ( Hart, 1998 ; Levy & Ellis, 2006 ).

The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ( Paré et al., 2015 ). Rather than providing a base for a researcher’s own work, it creates a solid starting point for all members of the community interested in a particular area or topic ( Mulrow, 1987 ). The so-called “review article” is a journal-length paper which has an overarching purpose to synthesize the literature in a field, without collecting or analyzing any primary data ( Green, Johnson, & Adams, 2006 ).

When appropriately conducted, review articles represent powerful information sources for practitioners looking for state-of-the art evidence to guide their decision-making and work practices ( Paré et al., 2015 ). Further, high-quality reviews become frequently cited pieces of work which researchers seek out as a first clear outline of the literature when undertaking empirical studies ( Cooper, 1988 ; Rowe, 2014 ). Scholars who track and gauge the impact of articles have found that review papers are cited and downloaded more often than any other type of published article ( Cronin, Ryan, & Coughlan, 2008 ; Montori, Wilczynski, Morgan, Haynes, & Hedges, 2003 ; Patsopoulos, Analatos, & Ioannidis, 2005 ). The reason for their popularity may be the fact that reading the review enables one to have an overview, if not a detailed knowledge of the area in question, as well as references to the most useful primary sources ( Cronin et al., 2008 ). Although they are not easy to conduct, the commitment to complete a review article provides a tremendous service to one’s academic community ( Paré et al., 2015 ; Petticrew & Roberts, 2006 ). Most, if not all, peer-reviewed journals in the fields of medical informatics publish review articles of some type.

The main objectives of this chapter are fourfold: (a) to provide an overview of the major steps and activities involved in conducting a stand-alone literature review; (b) to describe and contrast the different types of review articles that can contribute to the eHealth knowledge base; (c) to illustrate each review type with one or two examples from the eHealth literature; and (d) to provide a series of recommendations for prospective authors of review articles in this domain.

9.2. Overview of the Literature Review Process and Steps

As explained in Templier and Paré (2015) , there are six generic steps involved in conducting a review article:

• formulating the research question(s) and objective(s),
• searching the extant literature,
• screening for inclusion,
• assessing the quality of primary studies,
• extracting data, and
• analyzing data.

Although these steps are presented here in sequential order, one must keep in mind that the review process can be iterative and that many activities can be initiated during the planning stage and later refined during subsequent phases ( Finfgeld-Connett & Johnson, 2013 ; Kitchenham & Charters, 2007 ).

Formulating the research question(s) and objective(s): As a first step, members of the review team must appropriately justify the need for the review itself ( Petticrew & Roberts, 2006 ), identify the review’s main objective(s) ( Okoli & Schabram, 2010 ), and define the concepts or variables at the heart of their synthesis ( Cooper & Hedges, 2009 ; Webster & Watson, 2002 ). Importantly, they also need to articulate the research question(s) they propose to investigate ( Kitchenham & Charters, 2007 ). In this regard, we concur with Jesson, Matheson, and Lacey (2011) that clearly articulated research questions are key ingredients that guide the entire review methodology; they underscore the type of information that is needed, inform the search for and selection of relevant literature, and guide or orient the subsequent analysis. Searching the extant literature: The next step consists of searching the literature and making decisions about the suitability of material to be considered in the review ( Cooper, 1988 ). There exist three main coverage strategies. First, exhaustive coverage means an effort is made to be as comprehensive as possible in order to ensure that all relevant studies, published and unpublished, are included in the review and, thus, conclusions are based on this all-inclusive knowledge base. The second type of coverage consists of presenting materials that are representative of most other works in a given field or area. Often authors who adopt this strategy will search for relevant articles in a small number of top-tier journals in a field ( Paré et al., 2015 ). In the third strategy, the review team concentrates on prior works that have been central or pivotal to a particular topic. This may include empirical studies or conceptual papers that initiated a line of investigation, changed how problems or questions were framed, introduced new methods or concepts, or engendered important debate ( Cooper, 1988 ). Screening for inclusion: The following step consists of evaluating the applicability of the material identified in the preceding step ( Levy & Ellis, 2006 ; vom Brocke et al., 2009 ). Once a group of potential studies has been identified, members of the review team must screen them to determine their relevance ( Petticrew & Roberts, 2006 ). A set of predetermined rules provides a basis for including or excluding certain studies. This exercise requires a significant investment on the part of researchers, who must ensure enhanced objectivity and avoid biases or mistakes. As discussed later in this chapter, for certain types of reviews there must be at least two independent reviewers involved in the screening process and a procedure to resolve disagreements must also be in place ( Liberati et al., 2009 ; Shea et al., 2009 ). Assessing the quality of primary studies: In addition to screening material for inclusion, members of the review team may need to assess the scientific quality of the selected studies, that is, appraise the rigour of the research design and methods. Such formal assessment, which is usually conducted independently by at least two coders, helps members of the review team refine which studies to include in the final sample, determine whether or not the differences in quality may affect their conclusions, or guide how they analyze the data and interpret the findings ( Petticrew & Roberts, 2006 ). Ascribing quality scores to each primary study or considering through domain-based evaluations which study components have or have not been designed and executed appropriately makes it possible to reflect on the extent to which the selected study addresses possible biases and maximizes validity ( Shea et al., 2009 ). Extracting data: The following step involves gathering or extracting applicable information from each primary study included in the sample and deciding what is relevant to the problem of interest ( Cooper & Hedges, 2009 ). Indeed, the type of data that should be recorded mainly depends on the initial research questions ( Okoli & Schabram, 2010 ). However, important information may also be gathered about how, when, where and by whom the primary study was conducted, the research design and methods, or qualitative/quantitative results ( Cooper & Hedges, 2009 ). Analyzing and synthesizing data : As a final step, members of the review team must collate, summarize, aggregate, organize, and compare the evidence extracted from the included studies. The extracted data must be presented in a meaningful way that suggests a new contribution to the extant literature ( Jesson et al., 2011 ). Webster and Watson (2002) warn researchers that literature reviews should be much more than lists of papers and should provide a coherent lens to make sense of extant knowledge on a given topic. There exist several methods and techniques for synthesizing quantitative (e.g., frequency analysis, meta-analysis) and qualitative (e.g., grounded theory, narrative analysis, meta-ethnography) evidence ( Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005 ; Thomas & Harden, 2008 ).

9.3. Types of Review Articles and Brief Illustrations

EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic. Our classification scheme is largely inspired from Paré and colleagues’ (2015) typology. Below we present and illustrate those review types that we feel are central to the growth and development of the eHealth domain.

9.3.1. Narrative Reviews

The narrative review is the “traditional” way of reviewing the extant literature and is skewed towards a qualitative interpretation of prior knowledge ( Sylvester et al., 2013 ). Put simply, a narrative review attempts to summarize or synthesize what has been written on a particular topic but does not seek generalization or cumulative knowledge from what is reviewed ( Davies, 2000 ; Green et al., 2006 ). Instead, the review team often undertakes the task of accumulating and synthesizing the literature to demonstrate the value of a particular point of view ( Baumeister & Leary, 1997 ). As such, reviewers may selectively ignore or limit the attention paid to certain studies in order to make a point. In this rather unsystematic approach, the selection of information from primary articles is subjective, lacks explicit criteria for inclusion and can lead to biased interpretations or inferences ( Green et al., 2006 ). There are several narrative reviews in the particular eHealth domain, as in all fields, which follow such an unstructured approach ( Silva et al., 2015 ; Paul et al., 2015 ).

Despite these criticisms, this type of review can be very useful in gathering together a volume of literature in a specific subject area and synthesizing it. As mentioned above, its primary purpose is to provide the reader with a comprehensive background for understanding current knowledge and highlighting the significance of new research ( Cronin et al., 2008 ). Faculty like to use narrative reviews in the classroom because they are often more up to date than textbooks, provide a single source for students to reference, and expose students to peer-reviewed literature ( Green et al., 2006 ). For researchers, narrative reviews can inspire research ideas by identifying gaps or inconsistencies in a body of knowledge, thus helping researchers to determine research questions or formulate hypotheses. Importantly, narrative reviews can also be used as educational articles to bring practitioners up to date with certain topics of issues ( Green et al., 2006 ).

Recently, there have been several efforts to introduce more rigour in narrative reviews that will elucidate common pitfalls and bring changes into their publication standards. Information systems researchers, among others, have contributed to advancing knowledge on how to structure a “traditional” review. For instance, Levy and Ellis (2006) proposed a generic framework for conducting such reviews. Their model follows the systematic data processing approach comprised of three steps, namely: (a) literature search and screening; (b) data extraction and analysis; and (c) writing the literature review. They provide detailed and very helpful instructions on how to conduct each step of the review process. As another methodological contribution, vom Brocke et al. (2009) offered a series of guidelines for conducting literature reviews, with a particular focus on how to search and extract the relevant body of knowledge. Last, Bandara, Miskon, and Fielt (2011) proposed a structured, predefined and tool-supported method to identify primary studies within a feasible scope, extract relevant content from identified articles, synthesize and analyze the findings, and effectively write and present the results of the literature review. We highly recommend that prospective authors of narrative reviews consult these useful sources before embarking on their work.

Darlow and Wen (2015) provide a good example of a highly structured narrative review in the eHealth field. These authors synthesized published articles that describe the development process of mobile health ( m-health ) interventions for patients’ cancer care self-management. As in most narrative reviews, the scope of the research questions being investigated is broad: (a) how development of these systems are carried out; (b) which methods are used to investigate these systems; and (c) what conclusions can be drawn as a result of the development of these systems. To provide clear answers to these questions, a literature search was conducted on six electronic databases and Google Scholar . The search was performed using several terms and free text words, combining them in an appropriate manner. Four inclusion and three exclusion criteria were utilized during the screening process. Both authors independently reviewed each of the identified articles to determine eligibility and extract study information. A flow diagram shows the number of studies identified, screened, and included or excluded at each stage of study selection. In terms of contributions, this review provides a series of practical recommendations for m-health intervention development.

9.3.2. Descriptive or Mapping Reviews

The primary goal of a descriptive review is to determine the extent to which a body of knowledge in a particular research topic reveals any interpretable pattern or trend with respect to pre-existing propositions, theories, methodologies or findings ( King & He, 2005 ; Paré et al., 2015 ). In contrast with narrative reviews, descriptive reviews follow a systematic and transparent procedure, including searching, screening and classifying studies ( Petersen, Vakkalanka, & Kuzniarz, 2015 ). Indeed, structured search methods are used to form a representative sample of a larger group of published works ( Paré et al., 2015 ). Further, authors of descriptive reviews extract from each study certain characteristics of interest, such as publication year, research methods, data collection techniques, and direction or strength of research outcomes (e.g., positive, negative, or non-significant) in the form of frequency analysis to produce quantitative results ( Sylvester et al., 2013 ). In essence, each study included in a descriptive review is treated as the unit of analysis and the published literature as a whole provides a database from which the authors attempt to identify any interpretable trends or draw overall conclusions about the merits of existing conceptualizations, propositions, methods or findings ( Paré et al., 2015 ). In doing so, a descriptive review may claim that its findings represent the state of the art in a particular domain ( King & He, 2005 ).

In the fields of health sciences and medical informatics, reviews that focus on examining the range, nature and evolution of a topic area are described by Anderson, Allen, Peckham, and Goodwin (2008) as mapping reviews . Like descriptive reviews, the research questions are generic and usually relate to publication patterns and trends. There is no preconceived plan to systematically review all of the literature although this can be done. Instead, researchers often present studies that are representative of most works published in a particular area and they consider a specific time frame to be mapped.

An example of this approach in the eHealth domain is offered by DeShazo, Lavallie, and Wolf (2009). The purpose of this descriptive or mapping review was to characterize publication trends in the medical informatics literature over a 20-year period (1987 to 2006). To achieve this ambitious objective, the authors performed a bibliometric analysis of medical informatics citations indexed in medline using publication trends, journal frequencies, impact factors, Medical Subject Headings (MeSH) term frequencies, and characteristics of citations. Findings revealed that there were over 77,000 medical informatics articles published during the covered period in numerous journals and that the average annual growth rate was 12%. The MeSH term analysis also suggested a strong interdisciplinary trend. Finally, average impact scores increased over time with two notable growth periods. Overall, patterns in research outputs that seem to characterize the historic trends and current components of the field of medical informatics suggest it may be a maturing discipline (DeShazo et al., 2009).

9.3.3. Scoping Reviews

Scoping reviews attempt to provide an initial indication of the potential size and nature of the extant literature on an emergent topic (Arksey & O’Malley, 2005; Daudt, van Mossel, & Scott, 2013 ; Levac, Colquhoun, & O’Brien, 2010). A scoping review may be conducted to examine the extent, range and nature of research activities in a particular area, determine the value of undertaking a full systematic review (discussed next), or identify research gaps in the extant literature ( Paré et al., 2015 ). In line with their main objective, scoping reviews usually conclude with the presentation of a detailed research agenda for future works along with potential implications for both practice and research.

Unlike narrative and descriptive reviews, the whole point of scoping the field is to be as comprehensive as possible, including grey literature (Arksey & O’Malley, 2005). Inclusion and exclusion criteria must be established to help researchers eliminate studies that are not aligned with the research questions. It is also recommended that at least two independent coders review abstracts yielded from the search strategy and then the full articles for study selection ( Daudt et al., 2013 ). The synthesized evidence from content or thematic analysis is relatively easy to present in tabular form (Arksey & O’Malley, 2005; Thomas & Harden, 2008 ).

One of the most highly cited scoping reviews in the eHealth domain was published by Archer, Fevrier-Thomas, Lokker, McKibbon, and Straus (2011) . These authors reviewed the existing literature on personal health record ( phr ) systems including design, functionality, implementation, applications, outcomes, and benefits. Seven databases were searched from 1985 to March 2010. Several search terms relating to phr s were used during this process. Two authors independently screened titles and abstracts to determine inclusion status. A second screen of full-text articles, again by two independent members of the research team, ensured that the studies described phr s. All in all, 130 articles met the criteria and their data were extracted manually into a database. The authors concluded that although there is a large amount of survey, observational, cohort/panel, and anecdotal evidence of phr benefits and satisfaction for patients, more research is needed to evaluate the results of phr implementations. Their in-depth analysis of the literature signalled that there is little solid evidence from randomized controlled trials or other studies through the use of phr s. Hence, they suggested that more research is needed that addresses the current lack of understanding of optimal functionality and usability of these systems, and how they can play a beneficial role in supporting patient self-management ( Archer et al., 2011 ).

9.3.4. Forms of Aggregative Reviews

Healthcare providers, practitioners, and policy-makers are nowadays overwhelmed with large volumes of information, including research-based evidence from numerous clinical trials and evaluation studies, assessing the effectiveness of health information technologies and interventions ( Ammenwerth & de Keizer, 2004 ; Deshazo et al., 2009 ). It is unrealistic to expect that all these disparate actors will have the time, skills, and necessary resources to identify the available evidence in the area of their expertise and consider it when making decisions. Systematic reviews that involve the rigorous application of scientific strategies aimed at limiting subjectivity and bias (i.e., systematic and random errors) can respond to this challenge.

Systematic reviews attempt to aggregate, appraise, and synthesize in a single source all empirical evidence that meet a set of previously specified eligibility criteria in order to answer a clearly formulated and often narrow research question on a particular topic of interest to support evidence-based practice ( Liberati et al., 2009 ). They adhere closely to explicit scientific principles ( Liberati et al., 2009 ) and rigorous methodological guidelines (Higgins & Green, 2008) aimed at reducing random and systematic errors that can lead to deviations from the truth in results or inferences. The use of explicit methods allows systematic reviews to aggregate a large body of research evidence, assess whether effects or relationships are in the same direction and of the same general magnitude, explain possible inconsistencies between study results, and determine the strength of the overall evidence for every outcome of interest based on the quality of included studies and the general consistency among them ( Cook, Mulrow, & Haynes, 1997 ). The main procedures of a systematic review involve:

• Formulating a review question and developing a search strategy based on explicit inclusion criteria for the identification of eligible studies (usually described in the context of a detailed review protocol).
• Searching for eligible studies using multiple databases and information sources, including grey literature sources, without any language restrictions.
• Selecting studies, extracting data, and assessing risk of bias in a duplicate manner using two independent reviewers to avoid random or systematic errors in the process.
• Analyzing data using quantitative or qualitative methods.
• Presenting results in summary of findings tables.
• Interpreting results and drawing conclusions.

Many systematic reviews, but not all, use statistical methods to combine the results of independent studies into a single quantitative estimate or summary effect size. Known as meta-analyses , these reviews use specific data extraction and statistical techniques (e.g., network, frequentist, or Bayesian meta-analyses) to calculate from each study by outcome of interest an effect size along with a confidence interval that reflects the degree of uncertainty behind the point estimate of effect ( Borenstein, Hedges, Higgins, & Rothstein, 2009 ; Deeks, Higgins, & Altman, 2008 ). Subsequently, they use fixed or random-effects analysis models to combine the results of the included studies, assess statistical heterogeneity, and calculate a weighted average of the effect estimates from the different studies, taking into account their sample sizes. The summary effect size is a value that reflects the average magnitude of the intervention effect for a particular outcome of interest or, more generally, the strength of a relationship between two variables across all studies included in the systematic review. By statistically combining data from multiple studies, meta-analyses can create more precise and reliable estimates of intervention effects than those derived from individual studies alone, when these are examined independently as discrete sources of information.

The review by Gurol-Urganci, de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) on the effects of mobile phone messaging reminders for attendance at healthcare appointments is an illustrative example of a high-quality systematic review with meta-analysis. Missed appointments are a major cause of inefficiency in healthcare delivery with substantial monetary costs to health systems. These authors sought to assess whether mobile phone-based appointment reminders delivered through Short Message Service ( sms ) or Multimedia Messaging Service ( mms ) are effective in improving rates of patient attendance and reducing overall costs. To this end, they conducted a comprehensive search on multiple databases using highly sensitive search strategies without language or publication-type restrictions to identify all rct s that are eligible for inclusion. In order to minimize the risk of omitting eligible studies not captured by the original search, they supplemented all electronic searches with manual screening of trial registers and references contained in the included studies. Study selection, data extraction, and risk of bias assessments were performed inde­­pen­dently by two coders using standardized methods to ensure consistency and to eliminate potential errors. Findings from eight rct s involving 6,615 participants were pooled into meta-analyses to calculate the magnitude of effects that mobile text message reminders have on the rate of attendance at healthcare appointments compared to no reminders and phone call reminders.

Meta-analyses are regarded as powerful tools for deriving meaningful conclusions. However, there are situations in which it is neither reasonable nor appropriate to pool studies together using meta-analytic methods simply because there is extensive clinical heterogeneity between the included studies or variation in measurement tools, comparisons, or outcomes of interest. In these cases, systematic reviews can use qualitative synthesis methods such as vote counting, content analysis, classification schemes and tabulations, as an alternative approach to narratively synthesize the results of the independent studies included in the review. This form of review is known as qualitative systematic review.

A rigorous example of one such review in the eHealth domain is presented by Mickan, Atherton, Roberts, Heneghan, and Tilson (2014) on the use of handheld computers by healthcare professionals and their impact on access to information and clinical decision-making. In line with the methodological guide­lines for systematic reviews, these authors: (a) developed and registered with prospero ( www.crd.york.ac.uk/ prospero / ) an a priori review protocol; (b) conducted comprehensive searches for eligible studies using multiple databases and other supplementary strategies (e.g., forward searches); and (c) subsequently carried out study selection, data extraction, and risk of bias assessments in a duplicate manner to eliminate potential errors in the review process. Heterogeneity between the included studies in terms of reported outcomes and measures precluded the use of meta-analytic methods. To this end, the authors resorted to using narrative analysis and synthesis to describe the effectiveness of handheld computers on accessing information for clinical knowledge, adherence to safety and clinical quality guidelines, and diagnostic decision-making.

In recent years, the number of systematic reviews in the field of health informatics has increased considerably. Systematic reviews with discordant findings can cause great confusion and make it difficult for decision-makers to interpret the review-level evidence ( Moher, 2013 ). Therefore, there is a growing need for appraisal and synthesis of prior systematic reviews to ensure that decision-making is constantly informed by the best available accumulated evidence. Umbrella reviews , also known as overviews of systematic reviews, are tertiary types of evidence synthesis that aim to accomplish this; that is, they aim to compare and contrast findings from multiple systematic reviews and meta-analyses ( Becker & Oxman, 2008 ). Umbrella reviews generally adhere to the same principles and rigorous methodological guidelines used in systematic reviews. However, the unit of analysis in umbrella reviews is the systematic review rather than the primary study ( Becker & Oxman, 2008 ). Unlike systematic reviews that have a narrow focus of inquiry, umbrella reviews focus on broader research topics for which there are several potential interventions ( Smith, Devane, Begley, & Clarke, 2011 ). A recent umbrella review on the effects of home telemonitoring interventions for patients with heart failure critically appraised, compared, and synthesized evidence from 15 systematic reviews to investigate which types of home telemonitoring technologies and forms of interventions are more effective in reducing mortality and hospital admissions ( Kitsiou, Paré, & Jaana, 2015 ).

9.3.5. Realist Reviews

Realist reviews are theory-driven interpretative reviews developed to inform, enhance, or supplement conventional systematic reviews by making sense of heterogeneous evidence about complex interventions applied in diverse contexts in a way that informs policy decision-making ( Greenhalgh, Wong, Westhorp, & Pawson, 2011 ). They originated from criticisms of positivist systematic reviews which centre on their “simplistic” underlying assumptions ( Oates, 2011 ). As explained above, systematic reviews seek to identify causation. Such logic is appropriate for fields like medicine and education where findings of randomized controlled trials can be aggregated to see whether a new treatment or intervention does improve outcomes. However, many argue that it is not possible to establish such direct causal links between interventions and outcomes in fields such as social policy, management, and information systems where for any intervention there is unlikely to be a regular or consistent outcome ( Oates, 2011 ; Pawson, 2006 ; Rousseau, Manning, & Denyer, 2008 ).

To circumvent these limitations, Pawson, Greenhalgh, Harvey, and Walshe (2005) have proposed a new approach for synthesizing knowledge that seeks to unpack the mechanism of how “complex interventions” work in particular contexts. The basic research question — what works? — which is usually associated with systematic reviews changes to: what is it about this intervention that works, for whom, in what circumstances, in what respects and why? Realist reviews have no particular preference for either quantitative or qualitative evidence. As a theory-building approach, a realist review usually starts by articulating likely underlying mechanisms and then scrutinizes available evidence to find out whether and where these mechanisms are applicable ( Shepperd et al., 2009 ). Primary studies found in the extant literature are viewed as case studies which can test and modify the initial theories ( Rousseau et al., 2008 ).

The main objective pursued in the realist review conducted by Otte-Trojel, de Bont, Rundall, and van de Klundert (2014) was to examine how patient portals contribute to health service delivery and patient outcomes. The specific goals were to investigate how outcomes are produced and, most importantly, how variations in outcomes can be explained. The research team started with an exploratory review of background documents and research studies to identify ways in which patient portals may contribute to health service delivery and patient outcomes. The authors identified six main ways which represent “educated guesses” to be tested against the data in the evaluation studies. These studies were identified through a formal and systematic search in four databases between 2003 and 2013. Two members of the research team selected the articles using a pre-established list of inclusion and exclusion criteria and following a two-step procedure. The authors then extracted data from the selected articles and created several tables, one for each outcome category. They organized information to bring forward those mechanisms where patient portals contribute to outcomes and the variation in outcomes across different contexts.

9.3.6. Critical Reviews

Lastly, critical reviews aim to provide a critical evaluation and interpretive analysis of existing literature on a particular topic of interest to reveal strengths, weaknesses, contradictions, controversies, inconsistencies, and/or other important issues with respect to theories, hypotheses, research methods or results ( Baumeister & Leary, 1997 ; Kirkevold, 1997 ). Unlike other review types, critical reviews attempt to take a reflective account of the research that has been done in a particular area of interest, and assess its credibility by using appraisal instruments or critical interpretive methods. In this way, critical reviews attempt to constructively inform other scholars about the weaknesses of prior research and strengthen knowledge development by giving focus and direction to studies for further improvement ( Kirkevold, 1997 ).

Kitsiou, Paré, and Jaana (2013) provide an example of a critical review that assessed the methodological quality of prior systematic reviews of home telemonitoring studies for chronic patients. The authors conducted a comprehensive search on multiple databases to identify eligible reviews and subsequently used a validated instrument to conduct an in-depth quality appraisal. Results indicate that the majority of systematic reviews in this particular area suffer from important methodological flaws and biases that impair their internal validity and limit their usefulness for clinical and decision-making purposes. To this end, they provide a number of recommendations to strengthen knowledge development towards improving the design and execution of future reviews on home telemonitoring.

9.4. Summary

Table 9.1 outlines the main types of literature reviews that were described in the previous sub-sections and summarizes the main characteristics that distinguish one review type from another. It also includes key references to methodological guidelines and useful sources that can be used by eHealth scholars and researchers for planning and developing reviews.

Typology of Literature Reviews (adapted from Paré et al., 2015).

As shown in Table 9.1 , each review type addresses different kinds of research questions or objectives, which subsequently define and dictate the methods and approaches that need to be used to achieve the overarching goal(s) of the review. For example, in the case of narrative reviews, there is greater flexibility in searching and synthesizing articles ( Green et al., 2006 ). Researchers are often relatively free to use a diversity of approaches to search, identify, and select relevant scientific articles, describe their operational characteristics, present how the individual studies fit together, and formulate conclusions. On the other hand, systematic reviews are characterized by their high level of systematicity, rigour, and use of explicit methods, based on an “a priori” review plan that aims to minimize bias in the analysis and synthesis process (Higgins & Green, 2008). Some reviews are exploratory in nature (e.g., scoping/mapping reviews), whereas others may be conducted to discover patterns (e.g., descriptive reviews) or involve a synthesis approach that may include the critical analysis of prior research ( Paré et al., 2015 ). Hence, in order to select the most appropriate type of review, it is critical to know before embarking on a review project, why the research synthesis is conducted and what type of methods are best aligned with the pursued goals.

9.5. Concluding Remarks

In light of the increased use of evidence-based practice and research generating stronger evidence ( Grady et al., 2011 ; Lyden et al., 2013 ), review articles have become essential tools for summarizing, synthesizing, integrating or critically appraising prior knowledge in the eHealth field. As mentioned earlier, when rigorously conducted review articles represent powerful information sources for eHealth scholars and practitioners looking for state-of-the-art evidence. The typology of literature reviews we used herein will allow eHealth researchers, graduate students and practitioners to gain a better understanding of the similarities and differences between review types.

We must stress that this classification scheme does not privilege any specific type of review as being of higher quality than another ( Paré et al., 2015 ). As explained above, each type of review has its own strengths and limitations. Having said that, we realize that the methodological rigour of any review — be it qualitative, quantitative or mixed — is a critical aspect that should be considered seriously by prospective authors. In the present context, the notion of rigour refers to the reliability and validity of the review process described in section 9.2. For one thing, reliability is related to the reproducibility of the review process and steps, which is facilitated by a comprehensive documentation of the literature search process, extraction, coding and analysis performed in the review. Whether the search is comprehensive or not, whether it involves a methodical approach for data extraction and synthesis or not, it is important that the review documents in an explicit and transparent manner the steps and approach that were used in the process of its development. Next, validity characterizes the degree to which the review process was conducted appropriately. It goes beyond documentation and reflects decisions related to the selection of the sources, the search terms used, the period of time covered, the articles selected in the search, and the application of backward and forward searches ( vom Brocke et al., 2009 ). In short, the rigour of any review article is reflected by the explicitness of its methods (i.e., transparency) and the soundness of the approach used. We refer those interested in the concepts of rigour and quality to the work of Templier and Paré (2015) which offers a detailed set of methodological guidelines for conducting and evaluating various types of review articles.

To conclude, our main objective in this chapter was to demystify the various types of literature reviews that are central to the continuous development of the eHealth field. It is our hope that our descriptive account will serve as a valuable source for those conducting, evaluating or using reviews in this important and growing domain.

• Ammenwerth E., de Keizer N. An inventory of evaluation studies of information technology in health care. Trends in evaluation research, 1982-2002. International Journal of Medical Informatics. 2004; 44 (1):44–56. [ PubMed : 15778794 ]
• Anderson S., Allen P., Peckham S., Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. 2008; 6 (7):1–12. [ PMC free article : PMC2500008 ] [ PubMed : 18613961 ] [ CrossRef ]
• Archer N., Fevrier-Thomas U., Lokker C., McKibbon K. A., Straus S.E. Personal health records: a scoping review. Journal of American Medical Informatics Association. 2011; 18 (4):515–522. [ PMC free article : PMC3128401 ] [ PubMed : 21672914 ]
• Arksey H., O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005; 8 (1):19–32.
• A systematic, tool-supported method for conducting literature reviews in information systems. Paper presented at the Proceedings of the 19th European Conference on Information Systems ( ecis 2011); June 9 to 11; Helsinki, Finland. 2011.
• Baumeister R. F., Leary M.R. Writing narrative literature reviews. Review of General Psychology. 1997; 1 (3):311–320.
• Becker L. A., Oxman A.D. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Overviews of reviews; pp. 607–631.
• Borenstein M., Hedges L., Higgins J., Rothstein H. Introduction to meta-analysis. Hoboken, nj : John Wiley & Sons Inc; 2009.
• Cook D. J., Mulrow C. D., Haynes B. Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine. 1997; 126 (5):376–380. [ PubMed : 9054282 ]
• Cooper H., Hedges L.V. In: The handbook of research synthesis and meta-analysis. 2nd ed. Cooper H., Hedges L. V., Valentine J. C., editors. New York: Russell Sage Foundation; 2009. Research synthesis as a scientific process; pp. 3–17.
• Cooper H. M. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society. 1988; 1 (1):104–126.
• Cronin P., Ryan F., Coughlan M. Undertaking a literature review: a step-by-step approach. British Journal of Nursing. 2008; 17 (1):38–43. [ PubMed : 18399395 ]
• Darlow S., Wen K.Y. Development testing of mobile health interventions for cancer patient self-management: A review. Health Informatics Journal. 2015 (online before print). [ PubMed : 25916831 ] [ CrossRef ]
• Daudt H. M., van Mossel C., Scott S.J. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. bmc Medical Research Methodology. 2013; 13 :48. [ PMC free article : PMC3614526 ] [ PubMed : 23522333 ] [ CrossRef ]
• Davies P. The relevance of systematic reviews to educational policy and practice. Oxford Review of Education. 2000; 26 (3-4):365–378.
• Deeks J. J., Higgins J. P. T., Altman D.G. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Analysing data and undertaking meta-analyses; pp. 243–296.
• Deshazo J. P., Lavallie D. L., Wolf F.M. Publication trends in the medical informatics literature: 20 years of “Medical Informatics” in mesh . bmc Medical Informatics and Decision Making. 2009; 9 :7. [ PMC free article : PMC2652453 ] [ PubMed : 19159472 ] [ CrossRef ]
• Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy. 2005; 10 (1):45–53. [ PubMed : 15667704 ]
• Finfgeld-Connett D., Johnson E.D. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. Journal of Advanced Nursing. 2013; 69 (1):194–204. [ PMC free article : PMC3424349 ] [ PubMed : 22591030 ]
• Grady B., Myers K. M., Nelson E. L., Belz N., Bennett L., Carnahan L. … Guidelines Working Group. Evidence-based practice for telemental health. Telemedicine Journal and E Health. 2011; 17 (2):131–148. [ PubMed : 21385026 ]
• Green B. N., Johnson C. D., Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. Journal of Chiropractic Medicine. 2006; 5 (3):101–117. [ PMC free article : PMC2647067 ] [ PubMed : 19674681 ]
• Greenhalgh T., Wong G., Westhorp G., Pawson R. Protocol–realist and meta-narrative evidence synthesis: evolving standards ( rameses ). bmc Medical Research Methodology. 2011; 11 :115. [ PMC free article : PMC3173389 ] [ PubMed : 21843376 ]
• Gurol-Urganci I., de Jongh T., Vodopivec-Jamsek V., Atun R., Car J. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database System Review. 2013; 12 cd 007458. [ PMC free article : PMC6485985 ] [ PubMed : 24310741 ] [ CrossRef ]
• Hart C. Doing a literature review: Releasing the social science research imagination. London: SAGE Publications; 1998.
• Higgins J. P. T., Green S., editors. Cochrane handbook for systematic reviews of interventions: Cochrane book series. Hoboken, nj : Wiley-Blackwell; 2008.
• Jesson J., Matheson L., Lacey F.M. Doing your literature review: traditional and systematic techniques. Los Angeles & London: SAGE Publications; 2011.
• King W. R., He J. Understanding the role and methods of meta-analysis in IS research. Communications of the Association for Information Systems. 2005; 16 :1.
• Kirkevold M. Integrative nursing research — an important strategy to further the development of nursing science and nursing practice. Journal of Advanced Nursing. 1997; 25 (5):977–984. [ PubMed : 9147203 ]
• Kitchenham B., Charters S. ebse Technical Report Version 2.3. Keele & Durham. uk : Keele University & University of Durham; 2007. Guidelines for performing systematic literature reviews in software engineering.
• Kitsiou S., Paré G., Jaana M. Systematic reviews and meta-analyses of home telemonitoring interventions for patients with chronic diseases: a critical assessment of their methodological quality. Journal of Medical Internet Research. 2013; 15 (7):e150. [ PMC free article : PMC3785977 ] [ PubMed : 23880072 ]
• Kitsiou S., Paré G., Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. Journal of Medical Internet Research. 2015; 17 (3):e63. [ PMC free article : PMC4376138 ] [ PubMed : 25768664 ]
• Levac D., Colquhoun H., O’Brien K. K. Scoping studies: advancing the methodology. Implementation Science. 2010; 5 (1):69. [ PMC free article : PMC2954944 ] [ PubMed : 20854677 ]
• Levy Y., Ellis T.J. A systems approach to conduct an effective literature review in support of information systems research. Informing Science. 2006; 9 :181–211.
• Liberati A., Altman D. G., Tetzlaff J., Mulrow C., Gøtzsche P. C., Ioannidis J. P. A. et al. Moher D. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine. 2009; 151 (4):W-65. [ PubMed : 19622512 ]
• Lyden J. R., Zickmund S. L., Bhargava T. D., Bryce C. L., Conroy M. B., Fischer G. S. et al. McTigue K. M. Implementing health information technology in a patient-centered manner: Patient experiences with an online evidence-based lifestyle intervention. Journal for Healthcare Quality. 2013; 35 (5):47–57. [ PubMed : 24004039 ]
• Mickan S., Atherton H., Roberts N. W., Heneghan C., Tilson J.K. Use of handheld computers in clinical practice: a systematic review. bmc Medical Informatics and Decision Making. 2014; 14 :56. [ PMC free article : PMC4099138 ] [ PubMed : 24998515 ]
• Moher D. The problem of duplicate systematic reviews. British Medical Journal. 2013; 347 (5040) [ PubMed : 23945367 ] [ CrossRef ]
• Montori V. M., Wilczynski N. L., Morgan D., Haynes R. B., Hedges T. Systematic reviews: a cross-sectional study of location and citation counts. bmc Medicine. 2003; 1 :2. [ PMC free article : PMC281591 ] [ PubMed : 14633274 ]
• Mulrow C. D. The medical review article: state of the science. Annals of Internal Medicine. 1987; 106 (3):485–488. [ PubMed : 3813259 ] [ CrossRef ]
• Evidence-based information systems: A decade later. Proceedings of the European Conference on Information Systems ; 2011. Retrieved from http://aisel ​.aisnet.org/cgi/viewcontent ​.cgi?article ​=1221&context ​=ecis2011 .
• Okoli C., Schabram K. A guide to conducting a systematic literature review of information systems research. ssrn Electronic Journal. 2010
• Otte-Trojel T., de Bont A., Rundall T. G., van de Klundert J. How outcomes are achieved through patient portals: a realist review. Journal of American Medical Informatics Association. 2014; 21 (4):751–757. [ PMC free article : PMC4078283 ] [ PubMed : 24503882 ]
• Paré G., Trudel M.-C., Jaana M., Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management. 2015; 52 (2):183–199.
• Patsopoulos N. A., Analatos A. A., Ioannidis J.P. A. Relative citation impact of various study designs in the health sciences. Journal of the American Medical Association. 2005; 293 (19):2362–2366. [ PubMed : 15900006 ]
• Paul M. M., Greene C. M., Newton-Dame R., Thorpe L. E., Perlman S. E., McVeigh K. H., Gourevitch M.N. The state of population health surveillance using electronic health records: A narrative review. Population Health Management. 2015; 18 (3):209–216. [ PubMed : 25608033 ]
• Pawson R. Evidence-based policy: a realist perspective. London: SAGE Publications; 2006.
• Pawson R., Greenhalgh T., Harvey G., Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy. 2005; 10 (Suppl 1):21–34. [ PubMed : 16053581 ]
• Petersen K., Vakkalanka S., Kuzniarz L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology. 2015; 64 :1–18.
• Petticrew M., Roberts H. Systematic reviews in the social sciences: A practical guide. Malden, ma : Blackwell Publishing Co; 2006.
• Rousseau D. M., Manning J., Denyer D. Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. The Academy of Management Annals. 2008; 2 (1):475–515.
• Rowe F. What literature review is not: diversity, boundaries and recommendations. European Journal of Information Systems. 2014; 23 (3):241–255.
• Shea B. J., Hamel C., Wells G. A., Bouter L. M., Kristjansson E., Grimshaw J. et al. Boers M. amstar is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology. 2009; 62 (10):1013–1020. [ PubMed : 19230606 ]
• Shepperd S., Lewin S., Straus S., Clarke M., Eccles M. P., Fitzpatrick R. et al. Sheikh A. Can we systematically review studies that evaluate complex interventions? PLoS Medicine. 2009; 6 (8):e1000086. [ PMC free article : PMC2717209 ] [ PubMed : 19668360 ]
• Silva B. M., Rodrigues J. J., de la Torre Díez I., López-Coronado M., Saleem K. Mobile-health: A review of current state in 2015. Journal of Biomedical Informatics. 2015; 56 :265–272. [ PubMed : 26071682 ]
• Smith V., Devane D., Begley C., Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. bmc Medical Research Methodology. 2011; 11 (1):15. [ PMC free article : PMC3039637 ] [ PubMed : 21291558 ]
• Sylvester A., Tate M., Johnstone D. Beyond synthesis: re-presenting heterogeneous research literature. Behaviour & Information Technology. 2013; 32 (12):1199–1215.
• Templier M., Paré G. A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems. 2015; 37 (6):112–137.
• Thomas J., Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. bmc Medical Research Methodology. 2008; 8 (1):45. [ PMC free article : PMC2478656 ] [ PubMed : 18616818 ]
• Reconstructing the giant: on the importance of rigour in documenting the literature search process. Paper presented at the Proceedings of the 17th European Conference on Information Systems ( ecis 2009); Verona, Italy. 2009.
• Webster J., Watson R.T. Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly. 2002; 26 (2):11.
• Whitlock E. P., Lin J. S., Chou R., Shekelle P., Robinson K.A. Using existing systematic reviews in complex systematic reviews. Annals of Internal Medicine. 2008; 148 (10):776–782. [ PubMed : 18490690 ]

This publication is licensed under a Creative Commons License, Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0): see https://creativecommons.org/licenses/by-nc/4.0/

• Cite this Page Paré G, Kitsiou S. Chapter 9 Methods for Literature Reviews. In: Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
• PDF version of this title (4.5M)
• Disable Glossary Links

In this Page

• Introduction
• Overview of the Literature Review Process and Steps
• Types of Review Articles and Brief Illustrations
• Concluding Remarks

Related information

• PMC PubMed Central citations
• PubMed Links to PubMed

Recent Activity

• Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Ev... Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Evidence-based Approach

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

Synthesising the literature as part of a literature review

Affiliation.

• 1 University of Manchester, England.
• PMID: 25783281
• DOI: 10.7748/ns.29.29.44.e8957

This article examines how to synthesise and critique research literature. To place the process of synthesising the research literature into context, the article explores the critiquing process by breaking it down into seven sequential steps. The article explains how and why these steps need to be kept in mind if a robust comprehensive literature search and analysis are to be achieved. The article outlines how to engage in the critiquing process and explains how the literature review needs to be assembled to generate a logical and reasoned debate to examine a topic of interest or research in more detail.

Keywords: Critical analysis; critique; evaluation; integrative review; literature review; literature search; research; research question; search strategy; synthesis.

• Research / standards*
• Research Design*
• Review Literature as Topic*