example of integrative literature review

Integrative Reviews

• Introduction
• Guidelines and procedures
• Management tools
• Define the question
• Check the topic
• Determine inclusion/exclusion criteria
• Develop a protocol
• Identify keywords
• Databases and search strategies
• Grey literature
• Manage and organise
• Screen & Select
• Locate full text
• Extract data

Example reviews

• Examples of integrative reviews
• Accessing help This link opens in a new window
• Systematic Style Reviews Guide This link opens in a new window

Please choose the tab below for your discipline to see relevant examples.

For more information about how to conduct and write reviews, please see the Guidelines section of this guide.

• Health & medicine
• Social sciences
• Does health literacy mediate the relationship between socioeconomic status and health disparities? Integrative review. (2019).
• Exploring healthcare professionals’ understanding and experiences of artificial intelligence technology use in the delivery of healthcare: An integrative review. (2020).
• An integrated review of the role of communication in veterinary clinical practice. (2020).
• Living labs for health: An integrative literature review. (2020).
• What’s in a definition? Holistic nursing, integrative health care, and integrative nursing: Report of an integrated literature review. (2019).
• Lawsuits in health: An integrative review. (2019).
• The onset of late paleolithic in North China: An integrative review of the Shuidonggou site complex, China. (2018).
• The risk of family violence after incarceration: An integrative review. (2020).
• Social work practices for young people with complex needs: An integrative review. (2018).
• Ubiquitous yet ambiguous: An integrative review of unpaid work. (2018).
• Experiences of individuals with disabilities sheltering during natural disasters: An integrative review. (2019).
• Foraging and feeding ecology of platanista: An integrative review. (2018).
• Geodiversity: An integrative review as a contribution to the sustainable management of the whole of nature. (2018).
• An integrative review of granular sludge for the biological removal of nutrients and recalcitrant organic matter from wastewater. (2018).
• Evaluating the presence of pesticides in bananas: An integrative review. (2020).
• << Previous: Publish
• Next: Accessing help >>
• Last Updated: May 13, 2024 11:34 AM
• URL: https://libguides.jcu.edu.au/integrative-reviews

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• v.26(5); 2021 Aug

Conducting integrative reviews: a guide for novice nursing researchers

Shannon dhollande.

Lecturer, School of Nursing, Midwifery & Social Sciences, CQ University Brisbane, Australia

Annabel Taylor

Professor, School of Nursing, Midwifery & Social Sciences, CQ University Brisbane, Australia

Silke Meyer

Associate Professor, School of Social Sciences, Monash University, Australia

Emergency Consultant, Emergency Department, Caboolture Hospital, Australia

Integrative reviews within healthcare promote a holistic understanding of the research topic. Structure and a comprehensive approach within reviews are important to ensure the reliability in their findings.

This paper aims to provide a framework for novice nursing researchers undertaking integrative reviews.

Established methods to form a research question, search literature, extract data, critically appraise extracted data and analyse review findings are discussed and exemplified using the authors’ own review as a comprehensive and reliable approach for the novice nursing researcher undertaking an integrative literature review.

Providing a comprehensive audit trail that details how an integrative literature review has been conducted increases and ensures the results are reproducible. The use of established tools to structure the various components of an integrative review increases robustness and readers’ confidence in the review findings.

Implications for practice

Novice nursing researchers may increase the reliability of their results by employing a framework to guide them through the process of conducting an integrative review.

A literature review is a critical analysis of published research literature based on a specified topic ( Pluye et al., 2016 ). Literature reviews identify literature then examine its strengths and weaknesses to determine gaps in knowledge ( Pluye et al. 2016 ). Literature reviews are an integral aspect of research projects; indeed, many reviews constitute a publication in themselves ( Snyder, 2019 ). There are various types of literature reviews based largely on the type of literature sourced ( Cronin et al. 2008 ). These include systematic literature reviews, traditional, narrative and integrative literature reviews ( Snyder, 2019 ). Aveyard and Bradbury-Jones (2019) found more than 35 commonly used terms to describe literature reviews. Within healthcare, systematic literature reviews initially gained traction and widespread support because of their reproducibility and focus on arriving at evidence-based conclusions that could influence practice and policy development ( Boell and Cecez-Kecmanovic, 2015 ). Yet, it became apparent that healthcare-related treatment options needed to review broader spectrums of research for treatment options to be considered comprehensive, holistic and patient orientated ( Boell and Cecez-Kecmanovic, 2015 ). Stern et al. (2014) suggest that despite the focus in healthcare on quantitative research not all pertinent questions surrounding the provision of care can be answered from this approach. To devise solutions to multidimensional problems, all forms of trustworthy evidence need to be considered ( Stern et al. 2014 ).

Integrative reviews assimilate research data from various research designs to reach conclusions that are comprehensive and reliable ( Soares et al. 2014 ). For example, an integrative review considers both qualitative and quantitative research to reach its conclusions. This approach promotes the development of a comprehensive understanding of the topic from a synthesis of all forms of available evidence ( Russell, 2005 ; Torraco, 2005 ). The strengths of an integrative review include its capacity to analyse research literature, evaluate the quality of the evidence, identify knowledge gaps, amalgamate research from various research designs, generate research questions and develop theoretical frameworks ( Russell, 2005 ). Aveyard and Bradbury-Jones (2019) suggested that integrative reviews exhibit similar characteristics to systematic reviews and may therefore be regarded as rigorous.

Integrative reviews value both qualitative and quantitative research which are built upon differing epistemological paradigms. Both types of research are vital in developing the evidence base that guides healthcare provision ( Leppäkoski and Paavilainen, 2012 ). Therefore, integrative reviews may influence policy development as their conclusions have considered a broad range of appropriate literature ( Whittemore and Knafl, 2005 ). An integrative approach to evidence synthesis allows healthcare professionals to make better use of all available evidence and apply it to the clinical practice environment ( Souza et al. 2010 ). For example, Aveyard and Bradbury-Jones (2019) found in excess of 12 different types of reviews employed to guide healthcare practice. The healthcare profession requires both quantitative and qualitative forms of research to establish the robust evidence base that enables the provision of evidence-based patient-orientated healthcare.

Integrative reviews require a specific set of skills to identify and synthesise literature ( Boell and Cecez-Kecmanovic, 2010 ). There remains a paucity of literature that provides explicit guidance to novice nursing researchers on how to conduct an integrative review and importantly how to ensure the results and conclusions are both comprehensive and reliable. Furthermore, novice nursing researchers may receive little formal training to develop the skills required to generate a comprehensive integrative review ( Boote and Beile, 2005 ). Aveyard and Bradbury-Jones (2019) also emphasised the limited literature providing guidance surrounding integrative reviews. Therefore, novice nursing researchers need to rely on published guidance to assist them. In this regard this paper, using an integrative review conducted by the authors as a case study, aims to provide a framework for novice nursing researchers conducting integrative reviews.

Developing the framework

In conducting integrative reviews, the novice nursing researcher may need to employ a framework to ensure the findings are comprehensive and reliable ( Boell and Cecez-Kecmanovic, 2010 ; Snyder, 2019 ). A framework to guide novice nursing researchers in conducting integrative reviews has been adapted by the authors and will now be described and delineated. This framework used various published literature to guide its creation, namely works by Aveyard and Bradbury-Jones (2019) , Nelson (2014), Stern et al. (2014) , Whittemore and Knafl (2005) , Pluye et al. (2009) , Moher et al., (2009) and Attride-Stirling, (2001) . The suggested framework involves seven steps ( Figure 1 ).

Integrative review framework ( Cooke et al. 2012 ; Riva et al. 2012 ).

Step 1: Write the review question

The review question acts as a foundation for an integrative study ( Riva et al. 2012 ). Yet, a review question may be difficult to articulate for the novice nursing researcher as it needs to consider multiple factors specifically, the population or sample, the interventions or area under investigation, the research design and outcomes and any benefit to the treatment ( Riva et al. 2012 ). A well-written review question aids the researcher to develop their research protocol/design and is of vital importance when writing an integrative review.

To articulate a review question there are numerous tools available to the novice nursing researcher to employ. These tools include variations on the PICOTs template (PICOT, PICO, PIO), and the Spider template. The PICOTs template is an established tool for structuring a research question. Yet, the SPIDER template has gained acceptance despite the need for further research to determine its applicability to multiple research contexts ( Cooke et al., 2012 ). Templates are recommended to aid the novice nursing researcher in effectively delineating and deconstructing the various elements within their review question. Delineation aids the researcher to refine the question and produce more targeted results within a literature search. In the case study, the review question was to: identify, evaluate and synthesise current knowledge and healthcare approaches to women presenting due to intimate partner violence (IPV) within emergency departments (ED). This review objective is delineated in the review question templates shown in Table 1 .

Comparison of elements involved with a PICOTS and SPIDER review question.

( Cooke et al. 2012 ; Riva et al. 2012 ).

Step 2: Determine the search strategy

In determining a search strategy, it is important for the novice nursing researcher to consider the databases employed, the search terms, the Boolean operators, the use of truncation and the use of subject headings. Furthermore, Nelson (2014) suggests that a detailed description of the search strategy should be included within integrative reviews to ensure readers are able to reproduce the results.

The databases employed within a search strategy need to consider the research aim and the scope of information contained within the database. Many databases vary in their coverage of specific journals and associated literature, such as conference proceedings ( Boell and Cecez-Kecmanovic, 2010 ). Therefore, the novice nursing researcher should consult several databases when conducting their searches. For example, search strategies within the healthcare field may utilise databases such as Cumulative Index to Nursing and Allied Healthcare Literature (CINAHL), Cochrane Library, Science Direct, ProQuest, Web of Science, Scopus and PsychInfo ( Cronin et al. 2008 ). These databases among others are largely considered appropriate repositories of reliable data that novice researchers may utilise when researching within healthcare. The date in which the searches are undertaken should be within the search strategy as searches undertaken after this date may generate increased results in line with the publication of further studies.

Utilising an established template to generate a research question allows for the delineation of key elements within the question as seen above. These key elements may assist the novice nursing researcher in determining the search terms they employ. Furthermore, keywords on published papers may provide the novice nursing researcher with alternative search terms, synonyms and introduce the researcher to key terminology employed within their field ( Boell and Cecez-Kecmanovic, 2010 ). For example, within the case study undertaken the search terms included among others: ‘domestic violence’, ‘domestic abuse’, ‘intimate partner violence and/or abuse’. To refine the search to the correct healthcare environment the terms ‘emergency department’ and/or ‘emergency room’ were employed. To link search terms, the researcher should consider their use of Boolean operators ‘And’ ‘Or’ and ‘Not’ and their use of truncation ( Cronin et al. 2008 ). Truncation is the shortening of words which in literature searches may increase the number of search results. Medical subject headings (MeSH) or general subject headings should be employed where appropriate and within this case study the headings included ‘nursing’, ‘domestic violence’ and ‘intimate partner violence’.

Inclusion and exclusion criteria allow the novice nursing researcher to reduce and refine the search parameters and locate the specific data they seek. Appropriate use of inclusion and exclusion criteria permits relevant data to be sourced as wider searches can produce a large amount of disparate data, whereas a search that is too narrow may result in the omission of significant findings ( Boell and Cecez-Kecmanovic, 2010 ). The novice nursing researcher needs to be aware that generating a large volume of search results may not necessarily result in relevant data being identified. Within integrative reviews there is potential for a large volume of data to be sourced and therefore time and resources required to complete the review need to be considered ( Heyvaert et al. 2017 ). The analysis and refining of a large volume of data can become a labour-intensive exercise for the novice nursing researcher ( Boell and Cecez-Kecmanovic, 2010 ).

Stern et al. (2014) suggest various elements that should be considered within inclusion/exclusion criteria:

• the type of studies included;
• the topic under exploration;
• the outcomes;
• publication language;
• the time period; and
• the methods employed.

The use of limiters or exclusion criteria are an effective method to manage the amount of time it takes to undertake searches and limit the volume of research generated. Yet, exclusion criteria may introduce biases in the search results and should therefore be used with caution and to produce specific outcomes by the novice nursing researcher ( Hammerstrøm et al. 2017 ).

Whittemore and Knafl (2005) suggest that randomised controlled trials, prospective and retrospective cohort studies, case control studies, cross sectional studies, systematic reviews and meta-analyses should all be included within the search strategy. Therefore, there are no biases based on the type of publication sourced ( Hammerstrøm et al. 2017 ).

There should be no restriction on the sample size within the studies recognising that qualitative studies generally have smaller sample sizes, and to capture the breadth of research available. There was no restriction on the date of publication within the case study as quality literature was limited. Scoping widely is an important strategy within integrative reviews to produce comprehensive results. A manual citation search of the reference list of all sourced papers was also undertaken by a member of the research team.

Literature may be excluded if those papers were published in a language foreign to the researcher with no accepted translation available. Though limiting papers based on translation availability may introduce some bias, this does ensure the review remains free from translational errors and cultural misinterpretations. In the case study, research conducted in developing countries with a markedly different healthcare service and significant resource limitations were excluded due to their lack of generalisability and clinical relevance; though this may have introduced a degree of location bias ( Nelson, 2014 ).

A peer review of the search strategy by an individual who specialises in research data searches such as a research librarian may be a viable method in which the novice healthcare researcher can ensure the search strategy is appropriate and able to generate the required data. One such tool that a novice nurse may employ is the Peer Review of the Search Strategy (PRESS) checklist. A peer review of the caste study was undertaken by a research librarian. All recommendations were incorporated into the search strategy which included removing a full text limiter, and changes to the Boolean and proximity operators.

After the search strategy has been implemented the researcher removes duplicate results and screened the retrieved publications based on their titles and abstracts. A second screening was then undertaken based on the full text of retrieved publications to remove papers that were irrelevant to the research question. Full text copies should then be obtained for critical appraisal employing validated methods.

Step 3: Critical appraisal of search results

The papers identified within the search strategy should undergo a critical appraisal to determine if they are appropriate and of sufficient quality to be included within the review. This should be conducted or reviewed by a second member of the research team, which occurred within this case study. Any discrepancies were discussed until consensus was achieved. A critical appraisal allows the novice healthcare researcher to appraise the relevance and trustworthiness of a study and, therefore, determine its applicability to their research (CASP, 2013). There are several established tools a novice nurse can employ in which to structure their critical appraisal. These include the Scoring System for Mixed-Methods Research and Mixed Studies Reviews developed by Pluye et al. (2009) and the Critical Appraisal Skills Programme (CASP, 2018) Checklists.

The review undertaken by the authors employed the scoring system for mixed-methods research and mixed-studies reviews developed by Pluye et al. (2009) . This scoring system was specifically designed for reviews employing studies from various research designs and therefore was utilised with ease ( Table 2 ).

The scoring system for mixed-methods research and mixed-studies reviews ( Pluye et al. 2009 ).

Using the CASP checklist aids the novice nursing researcher to examine the methodology of identified papers to establish validity. This critical appraisal tool contains 10 items. These items are yes or no questions that assist the researcher to determine (a) if the results of the paper are valid, (b) what the results are and (c) if it is relevant in the context of their study. For example, the checklist asks the researcher to consider the presence of a clear statement surrounding the aims of the research, and to consider why and how the research is important in regard to their topic (CASP, 2013). This checklist supports the nurse researcher to assess the validity, results and significance of research, and therefore appropriately decide on its inclusion within the review ( Krainovich-Miller et al., 2009 ).

Step 4: Summarise the search results

A summary of the results generated by literature searches is important to exemplify how comprehensive the literature is or conversely to identify if there are gaps in research. This summary should include the number of, and type of papers included within the review post limiters, screening and critical appraisal of search results. For example, within the review detailed throughout this paper the search strategy resulted in the inclusion of 25 qualitative and six quantitative papers ( Bakon et al. 2019 ). Many papers provide a summary of their search results visually in a Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram ( Boell and Cecez-Kecmanovic, 2015 ). PRISMA is a method of reporting that enables readers to assess the robustness of the results ( Leclercq et al. 2019 ; Moher et al. 2009 ). PRISMA promotes the transparency of the search process by delineating various items within the search process ( Leclercq et al. 2019 ; Moher et al. 2009 ). Researchers may decide how rigorously they follow this process yet should provide a rationale for any deviations ( Leclercq et al. 2019 ; Moher et al, 2009 ). Figure 2 is an example of the PRISMA flow diagram as it was applied within the case study.

Example PRISMA flow diagram ( Bakon et al. 2019 ; Moher et al. 2009 ).

Step 5: Data extraction and reduction

Data can be extracted from the critically appraised papers identified through the search strategy employing extraction tables. Within the case study data were clearly delineated, as suggested by Boell and Cecez-Kecmanovic (2010) , into extraction or comparison tables ( Table 3 ). These tables specify the authors, the date of publication, year of publication, site where the research was conducted and the key findings. Setting out the data into tables facilitates the comparison of these variables and aids the researcher to determine the appropriateness of the papers’ inclusion or exclusion within the review ( Whittemore and Knafl, 2005 ).

Example of a data extraction table.

Step 6: Analysis

Thematic analysis is widely used in integrative research ( Attride-Stirling, 2001 ). In this section we will discuss the benefits of employing a structured approach to thematic analysis including the formation of a thematic network. A thematic network is a visual diagram or depiction of the themes displaying their interconnectivity. Thematic analysis with the development of a thematic network is a way of identifying themes at various levels and depicting the observed relationships and organisation of these themes ( Attride-Stirling, 2001 ). There are numerous methods and tools available in which to conduct a thematic analysis that may be of use to the novice healthcare researcher conducting an integrative review. The approach used in a thematic analysis is important though a cursory glance at many literature reviews will reveal that many authors do not delineate the methods they employ. This includes the thematic analysis approach suggested by Thomas and Harden (2008) and the approach to thematic networking suggested by Attride-Stirling (2001) .

Thomas and Harden (2008) espouse a three-step approach to thematic analysis which includes: (a) coding, (b) organisation of codes into descriptive themes, and (c) the amalgamation of descriptive themes into analytical themes. The benefit of this approach lies in its simplicity and the ease with which a novice nurse researcher can apply the required steps. In contrast, the benefit of the approach suggested by Attride-Stirling (2001) lies in its ability to move beyond analysis and generate a visual thematic network which facilitates a critical interpretation and synthesis of the data.

Thematic networks typically depict three levels: basic themes, organising themes and global themes ( Attride-Stirling, 2001 ). The thematic network can then be developed. A thematic network is a visual depiction that appears graphically as a web like design ( Attride-Stirling, 2001 ). Thematic networks emphasise the relationships and interconnectivity of the network. It is an illustrative tool that facilitates interpretation of the data ( Attride-Stirling, 2001 ).

The benefits of employing a thematic analysis and networking within integrative reviews is the flexibility inherent within the approach, which allows the novice nursing researcher to provide a comprehensive accounting of the data ( Nowell et al. 2017 ). Thematic analysis is also an easily grasped form of data analysis that is useful for exploring various perspectives on specific topics and highlighting knowledge gaps ( Nowell et al. 2017 ). Thematic analysis and networking is also useful as a method to summarise large or diversified data sets to produce insightful conclusions ( Attride-Stirling, 2001 ; Nowell et al. 2017 ). The ability to assimilate data from various seemingly disparate perspectives may be challenging for the novice nursing researcher conducting an integrative review yet this integration of data by thematic analysis and networking was is integral.

To ensure the trustworthiness of results, novice nursing researchers need to clearly articulate each stage within the chosen method of data analysis ( Attride-Stirling, 2001 ; Nowell et al. 2017 ). The method employed in data analysis needs to be precise and exhaustively delineated ( Attride-Stirling, 2001 ; Nowell et al. 2017 ). Attride-Stirling (2001) suggests six steps within her methods of thematic analysis and networking. These steps include:

• code material;
• identify themes;
• construct thematic network;
• describe and explore the thematic network;
• summarise thematic network findings; and
• interpret patterns to identify implications.

In employing the approach suggested by Attride-Stirling (2001) within the case study the coding of specific findings within the data permitted the development of various themes ( Table 4 ). Inclusion of both quantitative and qualitative findings within the themes facilitated integration of the data which identified patterns and generated insights into the current care provided to IPV victims within ED.

Coding and theme formation.

Step 7: Conclusions and implications

A conclusion is important to remind the reader why the research topic is important. The researcher can then follow advice by Higginbottom (2015) who suggests that in drawing and writing research conclusions the researcher has an opportunity to explain the significance of the findings. The researcher may also need to explain these conclusions in light of the study limitations and parameters. Higginbottom (2015) emphasises that a conclusion is not a summary or reiteration of the results but a section which details the broader implications of the research and translates this knowledge into a format that is of use to the reader. The implications of the review findings for healthcare practice, for healthcare education and research should be considered.

Employing this structured and comprehensive framework within the case study the authors were able to determine that there remains a marked barrier in the provision of healthcare within the ED to women presenting with IPV-related injury. By employing an integrative approach multiple forms of literature were reviewed, and a considerable gap was identified. Therefore, further research may need to focus on the developing a structured healthcare protocol to aid ED clinicians to meet the needs of this vulnerable patient population.

Integrative reviews can be conducted with success when they follow a structured approach. This paper proposes a framework that novice nursing researchers can employ. Applying our stepped framework within an integrative review will strengthen the robustness of the study and facilitate its translation into policy and practice. This framework was employed by the authors to identify, evaluate and synthesise current knowledge and approaches of health professionals surrounding the care provision of women presenting due to IPV within emergency departments. The recommendations from the case study are currently being translated and implemented into the practice environment.

Key points for policy, practice and/or research

• Integrative literature reviews are required within nursing to consider elements of care provision from a holistic perspective.
• There is currently limited literature providing explicit guidance on how to undertake an integrative literature review.
• Clear delineation of the integrative literature review process demonstrates how the knowledge base was understood, organised and analysed.
• Nurse researchers may utilise this guidance to ensure the reliability of their integrative review.

Shannon Dhollande is a Lecturer, registered nurse and researcher. Her research explores the provision of emergency care to vulnerable populations.

Annabel Taylor is a Professorial Research Fellow at CQ University who with her background in social work explores methods of addressing gendered violence such as domestic violence.

Silke Meyer is an Associate Professor in Criminology and the Deputy Director of the Gender and Family Violence Prevention Centre at Monash University.

Mark Scott is an Emergency Medical Consultant with a track record in advancing emergency healthcare through implementation of evidence-based healthcare.

Declaration of conflicting interests: The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethics: Due to the nature of this article this article did not require ethical approval.

Funding: The author(s) received no financial support for the research, authorship, and/or publication of this article.

Shannon Dhollande https://orcid.org/0000-0003-3181-7606

Silke Meyer https://orcid.org/0000-0003-3964-042X

• Attride-Stirling J. (2001) Thematic networks: An analytic tool for qualitative research . Qualitative Research 1 ( 3 ): 385–405. [ Google Scholar ]
• Aveyard H, Bradbury-Jones C. (2019) An analysis of current practices in undertaking literature reviews in nursing: Findings from a focused mapping review and synthesis . BMC Medical Research Methodology 19 ( 1 ): 1–9. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Bakon S, Taylor A, Meyer S, et al. (2019) The provision of emergency healthcare for women who experience intimate partner violence: Part 1 An integrative review . Emergency Nurse 27 : 19–25. [ PubMed ] [ Google Scholar ]
• Boell S, Cecez-Kecmanovic D. (2010) Literature reviews and the Hermeneutic circle . Australian Academic & Research Libraries 41 ( 2 ): 129–144. [ Google Scholar ]
• Boell S, Cecez-Kecmanovic D. (2015) On being systematic in literature reviews . Journal of Information Technology 30 : 161–173. [ Google Scholar ]
• Boote D, Beile P. (2005) Scholars before researchers: On the centrality of the dissertation literature review in research preparation . Educational Researcher 34 ( 6 ): 3–15. [ Google Scholar ]
• Cooke A, Smith D, Booth A. (2012) Beyond PICO: The SPIDER tool for qualitative evidence synthesis . Qualitative Health Research 22 ( 10 ): 1435–1443. [ PubMed ] [ Google Scholar ]
• Critical Appraisal Skills Programme (CASP) (2018) CASP Checklists . casp-uk.net/casp-tools-checklists.
• Cronin P, Ryan F, Coughlan M. (2008) Undertaking a literature review: A step by step approach . British Journal of Nursing 17 ( 1 ): 38–43. [ PubMed ] [ Google Scholar ]
• Hammerstrøm K, Wade A, Jørgensen A, et al. (2017) Searching for relevant studie . In: Heyvaert M, Hannes K, Onghena P. (eds) Using Mixed Methods Research Synthesis for Literature Reviews , Thousand Oaks, CA: SAGE, pp. 69–112. [ Google Scholar ]
• Heyvaert M, Maes B, Onghena P, et al. (2017) Introduction to MMRS literature reviews . In: Heyvaert M, Hannes K, Onghena P. (eds) Using Mixed Methods Research Synthesis for Literature Reviews , Thousand Oaks, CA: SAGE, pp. 1–22. [ Google Scholar ]
• Higginbottom G. (2015) Drawing conclusions from your research . In: Higginbottom G, Liamputtong P. (eds) Participatory Qualitative Research Methodologies in Health , London: SAGE, pp. 80–89. [ Google Scholar ]
• Krainovich-Miller B, Haber J, Yost J, et al. (2009) Evidence-based practice challenge: Teaching critical appraisal of systematic reviews and clinical practice guidelines to graduate students . Journal of Nursing Education 48 ( 4 ): 186–195. [ PubMed ] [ Google Scholar ]
• Leclercq V, Beaudart C, Ajamieh S, et al. (2019) Meta-analyses indexed in PsycINFO had a better completeness of reporting when they mention PRISMA . Journal of Clinical Epidemiology 115 : 46–54. [ PubMed ] [ Google Scholar ]
• Leppäkoski T, Paavilainen E. (2012) Triangulation as a method to create a preliminary model to identify and intervene in intimate partner violence . Applied Nursing Research 25 : 171–180. [ PubMed ] [ Google Scholar ]
• Moher D, Liberati A, Tetzlaff J, et al. (2009) Preferred reporting items for systematic reviews and meta-analyses: The PRISMA Statement . PLoS Med 6 ( 7 ): e1000097. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Nelson H. (2014) Systematic Reviews to Answer Health Care Questions , Philadelphia, PA: Wolters Kluwer Health. [ Google Scholar ]
• Nowell L, Norris J, White D, et al. (2017) Thematic analysis . International Journal of Qualitative Methods 16 ( 1 ): 1–13. [ Google Scholar ]
• Pluye P, Gagnon M, Griffiths F, et al. (2009) A scoring system for appraising mixed methods research, and concomitantly appraising qualitative, quantitative and mixed methods primary studies in mixed studies reviews . International Journal of Nursing Studies 46 : 529–546. [ PubMed ] [ Google Scholar ]
• Pluye P, Hong Q, Bush P, et al. (2016) Opening up the definition of systematic literature review: The plurality of worldviews, methodologies and methods for reviews and syntheses . Journal of Clinical Epidemiology 73 : 2–5. [ PubMed ] [ Google Scholar ]
• Riva JJ, Malik KM, Burnie SJ, et al. (2012) What is your research question? An introduction to the PICOT format for clinicians . The Journal of the Canadian Chiropractic Association 56 ( 3 ): 167–171. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Russell C. (2005) An overview of the integrative research review . Progress in Transplantation 15 ( 1 ): 8–13. [ PubMed ] [ Google Scholar ]
• Snyder H. (2019) Literature review as a research methodology: An overview and guidelines . Journal of Business Research 104 : 333–339. [ Google Scholar ]
• Soares C, Hoga L, Peduzzi M, et al. (2014) Integrative review: Concepts and methods used in nursing . Revista da Escola de Enfermagem da USP 48 ( 2 ): 335–345. [ PubMed ] [ Google Scholar ]
• Souza M, Silva M, Carvalho R. (2010) Integrative review: What is it? How to do it? Einstein 8 ( 1 ): 102–106. [ PubMed ] [ Google Scholar ]
• Stern C, Jordan Z, McArthur A. (2014) Developing the review question and inclusion criteria . American Journal of Nursing 114 ( 4 ): 53–56. [ PubMed ] [ Google Scholar ]
• Thomas J, Harden A. (2008) Methods for the thematic synthesis of qualitative research in systematic reviews . BMC Medical Research Methodology 8 : 45. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Torraco R. (2005) Writing integrative literature reviews: Guidelines and examples . Human Resource Development Review 4 ( 3 ): 356–367. [ Google Scholar ]
• Whittemore R, Knafl K. (2005) The integrative review: Updated methodology . Journal of Advanced Nursing 52 ( 5 ): 546–553. [ PubMed ] [ Google Scholar ]

Overview of the Integrative Review

• First Online: 18 February 2020

Cite this chapter

• Coleen E. Toronto 3  

5838 Accesses

21 Citations

A reviewer selects literature on a topic and synthesizes and disseminates review findings to a targeted discipline. There are many review methods available for a reviewer to consider. The review question(s) will determine which review method is most appropriate to use. If the reviewer seeks to examine a topic broadly and wants to identify the current state of science on a particular phenomenon, an integrative review would be the method chosen. There are limited resources available to a reviewer to access when seeking guidance on how to conduct an integrative review. This chapter provides an overview of the integrative review method and provides a framework for the different steps that are recommended when conducting such a review.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as EPUB and PDF
• Read on any device
• Instant download
• Own it forever
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Aromataris E, Munn Z (eds) (2017) Joanna Briggs Institute reviewer’s manual. The Joanna Briggs Institute. https://reviewersmanual.joannabriggs.org/

Aveyard H, Bradbury-Jones C (2019) An analysis of current practices in undertaking literature reviews in nursing: findings from a focused mapping review and synthesis. BMC Med Res Methodol 19:1. https://doi.org/10.1186/s12874-019-0751-7

Article   Google Scholar  

Conner BT (2014) Demystifying literature reviews. Am Nurse Today 9(1):13–14

Google Scholar  

Cooper HM (1982) Scientific guidelines for conducting integrative research reviews. Rev Educ Res 52:291–302. https://doi.org/10.2307/1170314

Cooper HM (1984) The integrative research review: a systematic approach. SAGE Publications, Beverly Hills, CA, p 11

Coughlan M, Cronin P (2017) Doing a literature review in nursing, health and social care, 2nd edn. SAGE Publications, Thousand Oaks, CA, p 12

de Souza MT, da Silva MD, de Carvalho R (2010) Integrative review: what is it? How to do it? Einstein (São Paulo) 8(1):102–106. https://doi.org/10.1590/s1679-45082010rw1134

Denney AS, Tewksbury R (2013) How to write a literature review. J Crim Justice Educ 24(2):218–234. https://doi.org/10.1080/10511253.2012.730617

Ganong LH (1987) Integrative review of nursing research. Res Nurs Health 10(1):1–11

Article   CAS   Google Scholar  

Gough D, Thomas J, Oliver S (2012) Clarifying differences between review designs and methods. Syst Rev 1:28. https://doi.org/10.1186/2046-4053-1-28

Article   PubMed   PubMed Central   Google Scholar  

Grant MJ, Booth A (2009) A typology of reviews: an analysis of review types and associated methodologies. Health Inf Libr J 26:91–108. https://doi.org/10.1111/j.1471-1842.2009.00848.x

Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (eds) (2019) Cochrane handbook for systematic reviews of interventions version 6.0 (updated July 2019). Cochrane. www.training.cochrane.org/handbook

Hopia H, Latvala E, Liimatainen L (2016) Reviewing the methodology of an integrative review. Scand J Caring Sci 30:662–669. https://doi.org/10.1111/scs.12327

Article   PubMed   Google Scholar  

Institute of Medicine (2011) Finding what works in health care: standards for systematic reviews. The National Academies Press, Washington, DC. https://doi.org/10.17226/13059

Book   Google Scholar  

Jackson GB (1980) Methods for integrative reviews. Rev Educ Res Rev 50:438–460. https://doi.org/10.2307/1170440

Moher D, Liberati A, Tetzlaff J, Altman DG (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol 62(10):1006–1012. https://doi.org/10.1016/j.jclinepi.2009.06.005

Morris ZS, Wooding S, Grant J (2011) The answer is 17 years, what is the question: understanding time lags in translational research. J R Soc Med 104(12):510–520. https://doi.org/10.1258/jrsm.2011.110180

Oermann MH, Hays JC (2016) Writing for publication in nursing, 3rd edn. Springer, New York, NY, p 136

Polit DF, Beck CT (2018) Essentials of nursing research: appraising evidence for nursing practice, 9th edn. Wolters Kluwer, Philadelphia, p 96

Russell CL (2005) An overview of the integrative research review. Prog Transplant 15(1):8–13. https://doi.org/10.7182/prtr.15.1.0n13660r26g725kj

Soares CB, Hoga LAK, Peduzzi M, Sangaleti C, Yonekura T, Silva D (2014) Integrative review: concepts and methods used in nursing. Rev Esc Enferm USP 48(2):329–339. https://doi.org/10.1590/s0080-6234201400002000020

Toronto CE, Quinn B, Remington R (2018) Characteristics of reviews published in nursing literature: a methodological review. ANS Adv Nurs Sci 41(1):30–40. https://doi.org/10.1097/ANS.0000000000000180

Torraco RJ (2005) Writing integrative literature reviews: guidelines and examples. Hum Resour Dev Rev. SAGE Publications 4(3):356–367. https://doi.org/10.1177/1534484305278283

Torraco RJ (2016) Writing integrative literature reviews: using the past and present to explore the future. Hum Resour Dev Rev 15(4):404–428. https://doi.org/10.1177/1534484316671606

Whittemore R (2007) Rigour in integrative reviews. In: Webb C, Roe B (eds) Reviewing research evidence for nursing practice: in systematic reviews. Blackwell Publishing, Malden, MA, p 151

Whittemore R, Chao A, Jang M, Minges KE, Park C (2014) Methods for research synthesis: an overview. Heart Lung 43(5):453–461. https://doi.org/10.1016/j.hrtlng.2014.05.014

Whittemore R, Knafl K (2005) The integrative review: updated methodology. J Adv Nurs 52(5):546–553. https://doi.org/10.1111/j.1365-2648.2005.03621.x

Download references

Author information

Authors and affiliations.

School of Nursing, Curry College, Milton, MA, USA

Coleen E. Toronto

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Coleen E. Toronto .

Editor information

Editors and affiliations.

Department of Nursing, Framingham State University, Framingham, MA, USA

Ruth Remington

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Toronto, C.E. (2020). Overview of the Integrative Review. In: Toronto, C., Remington, R. (eds) A Step-by-Step Guide to Conducting an Integrative Review. Springer, Cham. https://doi.org/10.1007/978-3-030-37504-1_1

Download citation

DOI : https://doi.org/10.1007/978-3-030-37504-1_1

Published : 18 February 2020

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-37503-4

Online ISBN : 978-3-030-37504-1

eBook Packages : Medicine Medicine (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

University of Houston Libraries

• Literature Reviews in the Health Sciences
• Review Comparison Chart
• Decision Tools
• Systematic Review
• Meta-Analysis
• Scoping Review
• Mapping Review
• Integrative Review
• Rapid Review
• Realist Review
• Umbrella Review
• Review of Complex Interventions
• Diagnostic Test Accuracy Review
• Narrative Literature Reviews
• Standards and Guidelines

Navigate the links below to jump to a specific section of the page:

When is an Integrative Review methodology appropriate?

Outline of stages, methods and guidance, examples of integrative reviews, supplementary resources.

"An integrative review is a specific review method that summarizes past empirical or theoretical literature to provide a greater comprehensive understanding of a particular phenomenon or healthcare problem" (Broome, 1993). Thus, integrative reviews have the potential to build upon nursing science, informing research, practice, and policy initiatives.

An integrative review method is an approach that allows for the inclusion of diverse methodologies (i.e. experimental and non-experimental research) and have the potential to play a greater role in evidence-based practice for nursing ( Whittemore & Knafl, 2005 ).

Characteristics:

• An integrative review is best designed for nursing research
• The problem must be clearly defined
• define concepts
• review theories
• review evidence/point out gaps in the literature
• analyze methodological issues

When to Use It: According to Toronto & Remington (2020) , Whittmore & Knafl (2005) , and Broome (2000)  an integrative review approach is best suited for:

• A research scope focused more broadly at a phenomenon of interest rather than a systematic review and allows for diverse research, which may contain theoretical and methodological literature to address the aim of the review.
• Supporting a wide range of inquiry, such as defining concepts, reviewing theories, or analyzing methodological issues.
• Examining the complexity of nursing practice more broadly by using diverse data sources.

The following stages of conducting an integrative review are derived from  Whittemore & Knafl (2005) .

Timeframe:  12+ months

*Varies beyond the type of review. Depends on many factors such as but not limited to: resources available, the quantity and quality of the literature, and the expertise or experience of reviewers" ( Grant & Booth, 2009 ).

Question:  Formulation of a problem, may be related to practice and/or policy especially in nursing.

Is your review question a complex intervention?  Learn more about  Reviews of Complex Interventions .

Sources and searches:  Comprehensive but with a specific focus, integrated methodologies-experimental and non-experimental research. Purposive Sampling may be employed. Database searching is recommended along with grey literature searching. "Other recommended approaches to searching the literature include ancestry searching, journal hand searching, networking, and searching research registries." Search is transparent and reproducible.

Selection:  Selected as related to problem identified or question, Inclusion of empirical and theoretical reports and diverse study methodologies. 

Appraisal:  "How quality is evaluated in an integrative review will vary depending on the sampling frame." Limited/varying methods of critical appraisal and can be complex. "In a review that encompasses theoretical and empirical sources, two quality criteria instruments could be developed for each type of source and scores could be used as criteria for inclusion/exclusion or as a variable in the data analysis stage."

Synthesis:  Narrative synthesis for qualitative and quantitative studies. Data extracted for study characteristics and concept. Synthesis may be in the form of a table, diagram or model to portray results. "Extracted data are compared item by item so that similar data are categorized and grouped together."  

The method consists of:

• data reduction
• data display
• data comparison
• conclusion drawing,
• verification 

The following resources are considered to be the best guidance for conduct in the field of integrative reviews.

Methods & Guidance

• Hopia, H., Latvala, E., & Liimatainen, L. (2016). Reviewing the methodology of an integrative review .  Scandinavian journal of caring sciences ,  30 (4), 662–669. doi: 10.1111/scs.12327
• Russell C. L. (2005). An overview of the integrative research review .  Progress in transplantation ,  15 (1), 8–13. doi: 10.1177/152692480501500102
• Whittemore, R., & Knafl, K. (2005). The integrative review: updated methodology .  Journal of advanced nursing ,  52 (5), 546–553. doi: 10.1111/j.1365-2648.2005.03621.x

Reporting Guideline

There is currently no reporting guideline for integrative reviews.

• Collins, J. W., Zoucha, R., Lockhart, J. S., & Mixer, S. J. (2018). Cultural aspects of end-of-life care planning for African Americans: an integrative review of literature .  Journal of transcultural nursing ,  29 (6), 578–590. doi: 10.1177/1043659617753042
• Cowdell, F., Booth, A., & Appleby, B. (2017). Knowledge mobilization in bridging patient-practitioner-researcher boundaries: a systematic integrative review protocol .  Journal of advanced nursing ,  73 (11), 2757–2764. doi: 10.1111/jan.13378
• Frisch, N. C., & Rabinowitsch, D. (2019). What's in a definition? Holistic nursing, integrative health care, and integrative nursing: report of an integrated literature review .  Journal of holistic nursing ,  37 (3), 260–272. doi: 10.1177/0898010119860685
• Kim, J., Kim, Y. L., Jang, H., Cho, M., Lee, M., Kim, J., & Lee, H. (2020). Living labs for health: an integrative literature review .  European journal of public health ,  30 (1), 55–63. doi: 10.1093/eurpub/ckz105
• Luckett, T., Sellars, M., Tieman, J., Pollock, C. A., Silvester, W., Butow, P. N., Detering, K. M., Brennan, F., & Clayton, J. M. (2014). Advance care planning for adults with CKD: a systematic integrative review .  American journal of kidney diseases ,  63 (5), 761–770. doi: 10.1053/j.ajkd.2013.12.007
• Shinners, L., Aggar, C., Grace, S., & Smith, S. (2020). Exploring healthcare professionals' understanding and experiences of artificial intelligence technology use in the delivery of healthcare: an integrative review .  Health informatics journal ,  26 (2), 1225–1236. doi: 10.1177/1460458219874641
• Silva, D., Tavares, N. V., Alexandre, A. R., Freitas, D. A., Brêda, M. Z., Albuquerque, M. C., & Melo, V. L. (2015). Depressão e risco de suicídio entre profissionais de Enfermagem: revisão integrative [Depression and suicide risk among nursing professionals: an integrative review] .  Revista da Escola de Enfermagem da U S P ,  49 (6), 1027–1036. doi: 10.1590/S0080-623420150000600020
• Stormacq, C., Van den Broucke, S., & Wosinski, J. (2019). Does health literacy mediate the relationship between socioeconomic status and health disparities? integrative review .  Health promotion international ,  34 (5), e1–e17. doi: 10.1093/heapro/day062
• Broome M.E. (1993). Integrative literature reviews for the development of concepts. In Rodgers, B. L., & Knafl, K. A. (Eds.),  Concept development in nursing  (2nd ed., pp. 231-250). W.B. Saunders Company.
• da Silva, R. N., Brandão, M., & Ferreira, M. A. (2020). Integrative Review as a Method to Generate or to Test Nursing Theory .  Nursing science quarterly ,  33 (3), 258–263. doi: 10.1177/0894318420920602
• Garritty, C., Gartlehner, G., Nussbaumer-Streit, B., King, V. J., Hamel, C., Kamel, C., Affengruber, L., & Stevens, A. (2021). Cochrane Rapid Reviews Methods Group offers evidence-informed guidance to conduct rapid reviews .  Journal of clinical epidemiology ,  130 , 13–22. doi: 10.1016/j.jclinepi.2020.10.007
• Grant, M. J., & Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies .  Health information and libraries journal ,  26 (2), 91–108. doi: 10.1111/j.1471-1842.2009.00848.x

Toronto, C. E., & Remington, R. (2020).  A Step-By-Step Guide to Conducting an Integrative Review.  Springer International Publishing AG. doi: 10.1007/978-3-030-37504-1

• Torraco, R. J. (2005). Writing integrative literature reviews: guidelines and examples .  Human Resource Development Review, 4 (3), 356–367. doi: 10.1177/1534484305278283
• Whittemore. (2007). Rigour in Integrative Reviews . In Webb, C., & Roe, B. (Eds.),  Reviewing Research Evidence for Nursing Practice (pp. 149–156). Blackwell Publishing Ltd. https://doi.org/10.1002/9780470692127.ch11
• << Previous: Mapping Review
• Next: Rapid Review >>

Other Names for an Integrative Review

• Integrative Literature Review
• Systematic Integrative Review
• Integrative Research Review

Limitations of an Integrative Review

The following challenges of integrative reviews are derived from Toronto & Remington (2020) , Whitmore & Knafl (2005) , and Broome (2000) .

• The combination and complexity of incorporating diverse methodologies can contribute to lack of rigor, inaccuracy, and bias.
• Methods of analysis, synthesis, and conclusion-drawing remain poorly formulated.
• Combining empirical and theoretical reports can be difficult.
• There is no current guidance on reporting.

Medical Librarian

• Last Updated: Sep 5, 2023 11:14 AM
• URL: https://guides.lib.uh.edu/reviews

• Integrative Review

What is an Integrative Review?

An  integrative review provides a broader summary of the literature and includes findings from a range of research designs. It gathers and synthesizes  both empirical and theoretical evidence  relevant to a clearly defined problem. It may include case studies, observational studies, and meta-analyses, but may also include practice applications, theory, and guidelines. It is the only approach that allows for the combination of diverse methodologies. Its aim is to develop a holistic understanding   of the topic, present the state of the science and contribute to theory development.  The integrative review has been advocated as important for evidence-based practice initiatives in nursing  (Hopia et al., 2016).

Integrative reviews are popular in nursing because they use diverse data sources to investigate the complexity of nursing practice. An integrative review addresses the current state of the evidence, the quality of the available evidence, identifies gaps in the literature and suggests future directions for research and practice The clinical question(s)   of an integrative review   is broader  than that of a systematic review, yet should be clearly stated and well-defined. As with a systematic review, an integrative review requires a transparent and rigorous systematic approach  (Remington & Toronto, 2020).

Integrative reviews synthesize research data from various research designs to reach comprehensive and reliable conclusions. An integrative review helps to develop a comprehensive understanding of the topic by synthesizing  all forms of available evidence (Dhollande et al., 2021). They allow healthcare professionals to use all available evidence from both  qualitative and quantitative research to provide a more holistic understanding of the topic, which can then be applied to clinical practice. Sampling for an integrative review may include experimental and nonexperimental (empirical) and theoretical literature (Remington & Toronto, 2020). 

From:  Kutcher, & LeBaron, V. T. (2022). A simple guide for completing an integrative review using an example article.  Journal of Professional Nursing,  40 , 13-19. https://doi.org/10.1016/j.profnurs.2022.02.004

See Table 2: Steps of the integrative review (IR) process with key points and lessons learned

Steps of the Integrative Review Process

1: Select a Topic:  Formulate a purpose and/or review question(s).   An integrative review can be used to answer research questions related to nursing and other disciplines.   Clearly identify a problem from a gap in the literature. Perform a quick search for other literature reviews related to the topic of interest to avoid duplication. Integrative review questions should be  broad in scope, but narrow enough that the search is manageable.  It should be  well-defined,  and  clearly stated . Provide background on the topic and justification for the integrative review. Do a quick literature search to determine if any recent integrative or other types of reviews on or related to the topic have been performed.

Quality Appraisal Tools for Integrative Reviews

Critical Appraisal Skills Programme (CASP) Checklists  Appraisal checklists designed for use with Systematic Reviews, Randomized Controlled Trials, Cohort Studies,  Case Control  Studies, Economic Evaluations, Diagnostic Studies, Qualitative studies and Clinical Prediction Rule.

Mixed Methods Appraisal Tool (MMAT)  The MMAT is a critical appraisal tool that is designed for the appraisal stage of systematic mixed studies reviews, i.e., reviews that include qualitative, quantitative and mixed methods studies. It permits to appraise the methodological quality of five categories to studies: qualitative research, randomized controlled trials, non-randomized studies, quantitative descriptive studies, and mixed methods studies. (Hong et al., 2018).

Hong, Q. N., Fàbregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P., Gagnon, M.-P., Griffiths, F., Nicolau, B., O’Cathain, A., Rousseau, M.-C., Vedel, I., & Pluye, P. (2018). The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers.  Education for Information, 34 (4), 285–291. https://doi.org/10.3233/EFI-180221

More Information

For more information on integrative reviews:

Dhollande, S., Taylor, A., Meyer, S., & Scott, M. (2021). Conducting integrative reviews: A guide for novice nursing researchers.  Journal of Research in Nursing, 26( 5), 427–438. https://doi.org/10.1177/1744987121997907

Evans, D. (2007). Integrative reviews: Overview of methods. In C. Webb, & B. Roe (Eds.),  Reviewing research evidence for nursing practice: Systematic reviews  (pp. 135 - 148). John Wiley & Sons, Incorporated.

Hopia, Latvala, E., & Liimatainen, L. (2016). Reviewing the methodology of an integrative review.  Scandinavian Journal of Caring Sciences,  30 (4), 662–669. https://doi.org/10.1111/scs.12327

Kutcher, & LeBaron, V. T. (2022). A simple guide for completing an integrative review using an example article.  Journal of Professional Nursing,  40 , 13-19. https://doi.org/10.1016/j.profnurs.2022.02.004

Oermann, M. H., & Knafl, K. A. (2021). Strategies for completing a successful integrative review.  Nurse Author & Editor (Blackwell) ,  31 (3/4), 65–68. https://doi-org.libproxy.adelphi.edu/10.1111/nae2.30

Toronto, C. E., & Remington, R. (Eds.). (2020).  A step-by-step guide to conducting an integrative review . Springer.

Whittemore, R., & Knafl, K. (2005). The integrative review: updated methodology.  Journal of Advanced Nursing ,  52 (5), 546–553. https://doi.org/10.1111/j.1365-2648.2005.03621.x

Whittemore, R. (2007). Rigour in integrative reviews. In C. Webb, & B. Roe (Eds.),  Reviewing research evidence for nursing practice: Systematic reviews  (pp. 149 - 156). John Wiley & Sons, Incorporated.

• << Previous: Evidence Synthesis Review Types - Overview
• Next: Scoping Review >>
• Types of Questions
• Key Features and Limitations
• Is a Systematic Review Right for Your Research?
• Scoping Review
• Rapid Review
• Meta-Analysis/Meta-Synthesis
• Selecting a Review Type
• Reducing Bias
• Guidelines for Student Researchers
• Training Resources
• Register Your Protocol
• Handbooks & Manuals
• Reporting Guidelines
• PRESS 2015 Guidelines
• Search Strategies
• Selected Databases
• Grey Literature
• Handsearching
• Citation Searching
• Study Types & Terminology
• Quantitative vs. Qualitative Research
• Critical Appraisal of Studies
• Broad Functionality Programs & Tools
• Search Strategy Tools
• Deduplication Tools
• CItation Screening
• Critical Appraisal Tools
• Quality Assessment/Risk of Bias Tools
• Data Collection/Extraction
• Meta Analysis Tools
• Books on Systematic Reviews
• Finding Systematic Review Articles in the Databases
• Systematic Review Journals
• More Resources
• Evidence-Based Practice Research in Nursing
• Citation Management Programs
• Last Updated: May 3, 2024 5:52 PM
• URL: https://libguides.adelphi.edu/Systematic_Reviews

Integrative review: what is it? How to do it?

Affiliation.

• 1 Faculdade de Enfermagem do Hospital Israelita Albert Einstein - FEHIAE, São Paulo, SP, Brazil.
• PMID: 26761761
• DOI: 10.1590/S1679-45082010RW1134

Introduction: The integrative review is the methodology that provides synthesis of knowledge and applicability of results of significant studies to practice.

Objective: To present the phases of an integrative review and the relevant aspects to be taken into account when using this methodological resource.

Methods: This study was based on bibliographic search and on the experience of the authors when performing an integrative review.

Results: Presentation of the six stages of the integrative review process: preparing the guiding question, searching or sampling the literature, data collection, critical analysis of the studies included, discussion of results and presentation of the integrative review.

Conclusions: Considering the need to assure care based on scientific evidence, the integrative review has been identified as a unique tool in healthcare for it synthesizes investigations available on the given topic and guides practice based on scientific knowledge.

Integrative Review

• Introduction
• Systematic vs. Scoping vs. Integrative
• Related Guides
• Getting Help

Framework for Integrative Reviews

• This table summarizes the  Whittemore and Knafl (2005) integrative review methodology. 

Integrative Review Search Planning Form

• << Previous: Introduction
• Next: Resources >>
• Last Updated: Feb 20, 2024 3:59 PM
• URL: https://guides.library.duq.edu/integrative

• Matters Arising
• Open access
• Published: 13 May 2024

Extrapolation errors in Liu et al.’s CAM integrative review of health care professionals in New Zealand

• Jillian Marie McDowell   ORCID: orcid.org/0000-0003-4351-9817 1 ,
• Susan Heather Kohut 2 &
• Debra Betts 3  

BMC Complementary Medicine and Therapies volume  24 , Article number:  187 ( 2024 ) Cite this article

Metrics details

Matters Arising to this article was published on 13 May 2024

The Original Article was published on 13 February 2021

This letter is to highlight errors made by Liu et al. in their 2020 paper in BMC Complementary Medicine and Therapies, “Complementary and alternative medicine—practice, attitudes, and knowledge among healthcare professionals in New Zealand: an integrative review”. Substantial errors in their citation of the recent research and methodology by McDowell, Kohut & Betts (2019) pertaining to the practice of acupuncture in New Zealand by physiotherapists are presented. The actual results of McDowell et al.’s work and the true state of acupuncture use by their sample group is reported.

Peer Review reports

This letter is to highlight errors made by Liu et al. in their 2020 paper in BMC Complementary Medicine and Therapies, “Complementary and alternative medicine—practice, attitudes, and knowledge among healthcare professionals in New Zealand: an integrative review” [ 1 ], when citing the paper “Safe acupuncture and dry needling during pregnancy: A survey of New Zealand physiotherapists’ practice”, published in 2019 in Integrative Medicine Research [ 2 ]. Extrapolation errors are evident in their abstract, results and discussion’s statistics. Furthermore, the methodology has been mis-cited, as has interpretation of postgraduate physiotherapy acupuncture training in New Zealand (NZ).

Extrapolation errors – statistical

In Liu et al.’s abstract on complementary and alternative medicine (CAM) they state:

“When treating pregnant women, 48.4% of physiotherapists practise acupuncture , and 37.3% of midwives recommend CAM” page 1 [ 1 ].

These statistics are repeated in their results section for prevalence of practice, use and referral for CAM:

“Findings indicated that around 25% of GPs practise CAM, and 82.3% refer patients to CAM practitioners. When treating pregnant women, 48.4% of physiotherapists practise acupuncture, and 37.3% of midwives recommend CAM” page 4 [ 1 ],

and again, in their discussion:

“Findings indicated that CAM use is common among New Zealand healthcare professionals: around 25% of GPs practise CAM, and 82.3% refer patients to CAM practitioners; when treating pregnant women, 48.4% of physiotherapists practise acupuncture, and 37.3% of midwives recommend CAM” page 7 [ 1 ].

More accurately McDowell et al. [ 2 ] reported that 60 physiotherapists (48.4% of the 124 total respondents) practise acupuncture during pregnancy. This represented 0.8% of the 7337 registered physiotherapists in NZ. Liu et al.’s citation would infer that 48.4% of all registered physiotherapists in NZ (3551/7337) practice acupuncture for pregnant women; a 60-fold overestimation. McDowell et al. very clearly reported the limitations of their study;

“The results of this purposive study may not be indicative of the opinion of all NZ physiotherapy acupuncturists and dry needlers” page 37 [ 2 ] ,

and Liu et al.’s interpretative flaws may lead to future publication errors if it is cited further.

Extrapolation error—Post graduate training

The following statement in Liu et al.’s paper is inaccurate and misleading.

“Findings indicated that approximately one quarter of GPs have received formal training in CAM, while 44.4% of physiotherapist received (postgraduate) training in acupuncture” page 6 [ 1 ].

More correctly 53% of physiotherapists ( n  = 32) of those who answered in the affirmative to McDowell et al.’s survey, (i.e., who would provide acupuncture during pregnancy), had undergone post-graduate training with a resulting university-based qualification. It is important to note that physiotherapists choose to study and practice acupuncture in NZ as a postgraduate competency. Yet again Liu et al. have committed a statistical error of extrapolating a very small survey finding to the entire NZ physiotherapy population.

Methodology citation error

In their paper Liu et al. also state:

“Instruments used in the eight surveys varied, without psychometric properties reported; four studies used instruments that were developed and adapted from previous studies” . page 4 [ 1 ].

McDowell et al.’s paper clearly describes the development of the electronic survey questionnaire, expert peer review, and piloting prior to the main survey. It was not adapted from a previous study as cited by Liu et al. above.

The current state of acupuncture practice in NZ by physiotherapists

The practice of acupuncture is not currently regulated for physiotherapists in NZ, being considered “a modality within the practice of physiotherapy” [ 3 ]. Registered physiotherapists may practice within a defined field of practice as autonomous practitioners but must “complete or be currently undertaking, relevant, sufficient, and appropriate education and training and continuing professional development to maintain such knowledge and competence” [ 4 ]. NZ registered physiotherapists who have gained post graduate qualifications in acupuncture or have trained in dry needing are termed “physiotherapy acupuncturists” to identify as a separate profession to acupuncturists.

At the time of the survey 7337 physiotherapists were registered to practice in NZ [ 5 ]. Sixty nine percent of those were members of their national society, Physiotherapy New Zealand (PNZ) [ 6 ], and only 310 were members of PNZ’s special interest group the Physiotherapy Acupuncture Association of New Zealand (PAANZ). Further demographic information on the total number of NZ physiotherapists who have undertaken relevant education or who are currently practicing needling was not available.

Acupuncture has been taught and practiced by physiotherapists in NZ since 1992 [ 7 , 8 , 9 ]. However, the total number of physiotherapists utilising this complementary and alternative medicine in NZ is unknown. This has been a limiting factor for several research surveys to date [ 10 , 11 ]. The paper “Safe acupuncture and dry needling during pregnancy: New Zealand physiotherapists’ opinion and practice”, was published in 2019 in Integrative Medicine Research [ 2 ]. It is important to emphasise that the paper’s results did not have access to the entire population of potential respondents. The Privacy Act of New Zealand [ 12 ] prohibits the direct use of Physiotherapy Board of New Zealand (PBNZ) membership lists to target the entire survey population. McDowell et al.’s survey recruitment was via invitation only by PNZ e-mail, with additional links available through PAANZ and PNZ webpages, closed acupuncture group Facebook pages and physiotherapy newsletters within NZ.

Whilst McDowell et al.’s primary aim was to gather basic demographics and examine the opinions, practice and level of understanding held by NZ physiotherapy acupuncturists providing acupuncture and dry needling for pregnant women, their results were never intended to be reflective of the entire population of physiotherapists in New Zealand.

Regrettably, this fact seems to have been lost by Liu et al. when extrapolating McDowell et al.’s data as verbatim statistics for the entire NZ physiotherapy population.

Survey response rates have been on the decline for the past decade in the field of health related research [ 13 , 14 ], with response rates consistently lower than those of surveys of the general population [ 15 ]. Field et al. [ 16 ] reported response rates of between 13 and 39% are common in healthcare research. Cunningham et al. [ 13 ] found an overall response rate of 35% amongst physicians in Canada on a sensitive topic (medical billing practices) which was comparable to 39.6% response rate reported in the meta-analysis by Cook [ 17 ]. The McDowell et al.’s survey noted the existence of a low response rate (particularly from physiotherapy acupuncturists with only dry needling training), and a high number of missing responses within individual responses. This sample may not represent any definable population larger than NZ physiotherapy acupuncturists, a concept that appears to have been neglected by Liu et al.’s integrative review [ 1 ].

Further research

Research-based practice is central to improving the quality of health care by obtaining information about the knowledge, attitudes, practice patterns, the needs of practitioners [ 18 ] and to evaluate the impact of clinical research on practice [ 13 ]. Soliciting practitioner input is also essential when existing healthcare policies are being updated or to inform new policies [ 19 ].

Liu et al. have failed to recognize the limitations of McDowell et al.’s research sample, thus committing a statistical error of extrapolating a small survey finding to the entire NZ physiotherapy population. This raises concern that others may read their paper and cite it further (especially with the error prominent in the abstract), compounding errors in perceived practice if they are compared to the inflated NZ data. To date four papers have cited Liu et al.’s paper [ 20 , 21 , 22 , 23 ], fortunately without specifically referring to the erroneous sections. We ask that Lui et al.’s statements be corrected via publication of this correspondence, that is indexed and bidirectionally linked to the original article to prevent further and future errors.

Availability of data and materials

Not applicable.

Liu L, Tang Y, Baxter GD, Yin H, Tumilty S. Complementary and alternative medicine - practice, attitudes, and knowledge among healthcare professionals in New Zealand: an integrative review. BMC Complement Med Ther. 2021;21(1):63.

Article   CAS   PubMed   PubMed Central   Google Scholar  

McDowell JM, Kohut SH, Betts D. Safe acupuncture and dry needling during pregnancy: New Zealand physiotherapists’ opinion and practice. J Integrative Med. 2019;17(1):30–7.

Article   Google Scholar  

Physiotherapy Board of New Zealand. Acupuncture, a modality within the practice of physiotherapy. In: Touch The Newsletter of the Physiotherapy Board of New Zealand. 2004:9.

Physiotherapy Board of New Zealand. Physiotherapists practising in a defined field standard. Wellington: Physiotherapy Board of New Zealand; 2021.

Google Scholar  

Physiotherapy Board of New Zealand. Annual Report 2020/21. Wellington; 2021.

Kirby S. PNZ Leadership day. Wellington. 2021.

Kohut SH, Larmer PJ, Johnson G. Western acupuncture education for New Zealand physiotherapists. PTR. 2013;16(2):106–12.

Scrymgeour J. Moving on. A History of the New Zealand Society of Physiotherapists Inc. 1973–1999. Wellington: New Zealand Society of Physiotherapists; 2000.

Rapson L, Ellis N, Turnball D, Madzokere H, Haker E, Carballo AM. Introduction of acupuncture into some countries. In: Hopwood V, Lovesey M, Mokone S, editors. Acupuncture and Related Techniques in Physical Therapy. New York: Churchill Livingstone; 1997. p. 163–73.

McDowell JM, Johnson GM. Acupuncture needling styles and reports of associated adverse reactions to acupuncture. Med Acupunct. 2014;26(5):271–8.

McDowell JM, Johnson GM, Hale L. Adverse reactions to acupuncture: policy recommendations based on practitioner opinion in New Zealand. NZ J Physiother. 2013;41(3):94–101.

New Zealand Government. Privacy Act, (30 June 2020).

Cunningham CT, Quan H, Hemmelgarn B, Noseworthy T, Beck CA, Dixon E, et al. Exploring physician specialist response rates to web-based surveys. BMC Med Res Methodol. 2015;15(1):32.

Article   PubMed   PubMed Central   Google Scholar  

Burke M, Hodgins M. Is ‘Dear colleague’ enough? Improving response rates in surveys of healthcare professionals. Nurse Res. 2015;23(1):8–15.

Article   PubMed   Google Scholar  

Martins Y, Lederman RI, Lowenstein CL, Joffe S, Neville BA, Hastings BT, et al. Increasing response rates from physicians in oncology research: a structured literature review and data from a recent physician survey. Br J Cancer. 2012;106(6):1021–6.

Field TS, Cadoret CA, Brown ML, Ford M, Greene SM, Hill D, et al. Surveying physicians: do components of the “Total Design Approach” to optimizing survey response rates apply to physicians? Med Care. 2002;40(7):596–605.

Cook C, Heath F, Thompson RL. A meta-analysis of response rates in web- or internet-based surveys. Educ Psychol Measur. 2016;60(6):821–36.

Bonevski B, Magin P, Horton G, Foster M, Girgis A. Response rates in GP surveys - trialling two recruitment strategies. Aust Fam Physician. 2011;40(6):427–30.

PubMed   Google Scholar  

Cushman DM, Scholl LV, Ludlow M, Cunningham S, Teramoto M. Accuracy of rib palpation for dry needling of deep periscapular musculature, measured with ultrasound. J Bodyw Mov Ther. 2021;26:7–11.

Graca S, Betts D, Roberts K, Smith CA, Armour M. The changing clinical landscape in acupuncture for women’s health: a cross-sectional online survey in New Zealand and Australia. BMC Complement Med Ther. 2022;22(1):94.

Yingying P, Zang L, Wang X, Yang X. Effect of continuous care combined with constraint-induced movement therapy based on a continuing care health platform on MBI and FMA scores of acute stroke patients. J Healthc Eng. 2022;2022:5299969.

Yue T, Li J, Yang J, Fan D. Spinal manipulative therapy and acupuncture for chronic low back pain: a systematic review and meta-analysis of RCTs. Research Square. 2021;PREPRINT (Version 1). https://doi.org/10.21203/rs.3.rs-1192409/v1 .

Lesniewska M, Koziol I, Budzynska J, Kopystecka A, Patryn R. Opinions and experiences of physicians regarding complementary and alternative medicine (CAM). A research report from Poland. Research Square. 2022;PREPRINT (Version1). https://doi.org/10.21203/rs.3.rs-1138578/v1

Download references

Acknowledgements

No funding was received for the original survey “Safe acupuncture and dry needling during pregnancy: New Zealand physiotherapists’ opinion and practice”. A prize to encourage participant completion of the survey was donated by AcuFuture, NZ.

Author information

Authors and affiliations.

Prohealth Physiotherapy, 124 Kelvin St, Invercargill, New Zealand

Jillian Marie McDowell

School of Clinical Sciences, Faculty of Health and Environmental Sciences, Auckland University of Technology, Auckland, New Zealand

Susan Heather Kohut

New Zealand School of Acupuncture and Traditional Chinese Medicine, Wellington, New Zealand

Debra Betts

You can also search for this author in PubMed   Google Scholar

Contributions

JM, SK and DB were the authors of the original paper. JM drafted the letter. All authors read and approved the final version.

Corresponding author

Correspondence to Jillian Marie McDowell .

Ethics declarations

Ethics approval and consent to participate.

The survey referred to in this correspondence was undertaken with an electronic questionnaire. Ethics approval for this study was granted by the Auckland University of Technology Ethics Committee, application number 17/100. Participants were provided with an explanation of the study, consent details and researcher contact details.

Consent for publication

Not applicable (No identifying data).

Competing interests

JM and SK are executive members of, and tutors for the Physiotherapy Acupuncture Association of New Zealand (PAANZ). They are responsible for updating the PAANZ Guidelines for Safe Acupuncture practice biannually. SK is on the executive of International Acupuncture Association of Physical Therapists. DB is employed by the New Zealand School of Acupuncture and Traditional Chinese Medicine as a supervisor for a maternity acupuncture clinic at Hutt Valley Hospital, New Zealand.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

McDowell, J.M., Kohut, S.H. & Betts, D. Extrapolation errors in Liu et al.’s CAM integrative review of health care professionals in New Zealand. BMC Complement Med Ther 24 , 187 (2024). https://doi.org/10.1186/s12906-023-04259-3

Download citation

Received : 21 September 2021

Accepted : 14 November 2023

Published : 13 May 2024

DOI : https://doi.org/10.1186/s12906-023-04259-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Physiotherapy
• Acupuncture
• Extrapolation

BMC Complementary Medicine and Therapies

ISSN: 2662-7671

• Submission enquiries: [email protected]
• General enquiries: [email protected]