systematic approach to literature review

• Find My Rep

You are here

Systematic Approaches to a Successful Literature Review

• Andrew Booth - The University of Sheffield, UK
• Anthea Sutton - The University of Sheffield, UK
• Mark Clowes - Sheffield University, UK
• Marrissa Martyn-St James - Sheffield University, UK
• Description

The perfect project support for any social sciences student, this edition also includes a new chapter on analysing mixed methods research.

Supplements

Student Resources (Free to access) A literature review starter template to demonstrate the sections you need to include for a successful written review. A source credibility checklist to help you assess and think critically about the sources you choose. A source tracker template to help you keep track of your sources and know what you need to include in your audit trail. A downloadable exercise workbook and suggested answers.  A collated list of tried-and-tested tools , including freely available technologies to help you search databases efficiently, plan your work, and keep track of references. A project diary template and example. A complete glossary of terms. Instructor Resources (Log-in needed) PowerPoint slide templates including 10-15 slides per chapter, which can be downloaded and customized for use in your own presentations.

The Systematic Approaches to a Successful Literature Review (third edition) by Andrew Booth, Anthea Sutton, Mark Clowes and Marrissa Martyn-St James is a comprehensive overview of the entire evidence synthesis process – from selecting the appropriate method for an evidence synthesis topic all the way to the analysis and dissemination of the review. This book is of relevance to anyone interested in evidence synthesis – from trainees to researchers to decision-makers. Anyone can learn something from this book, whether you are a beginner, intermediate, or advance researcher in evidence synthesis. This book is perfect for university-level courses or for anyone interested in evidence synthesis. The exercises, toolbox, key learning points, and frequently asked questions were particularly helpful in advancing my learning.

For our masters level students doing their literature review dissertation this provides effective guidance in approaching their work in a systematic fashion.

Great resource. Easy to read, with helpful tables and diagrams that catch the students' attention and they find easy to recall. The examples and up-to-date links to external sources are also invaluable springboards for the students.

Post-COVID many more students prefer to use the electronic versions of books and the library is also keen to adopt more books in this format, so this is very helpful to enable the maximum number of students to access the helpful text with easy to follow guidance .

I have put this at the top of my reading list for my module on evidence based practice which is like a mini dissertation for level 5 students. It is a comprehensive read and sets out the stages involved in a successful literature review. My students have a problem with this part of the module and this book is a godsend

This book is very important for students to understand how to do an in-depth literature review as a support and motivation for their research.

I did not receive an inspection copy to use

Good comprehensive text - easy to follow

Very clear and useful

Preview this book

For instructors, select a purchasing option.

• Electronic Order Options VitalSource Amazon Kindle Google Play eBooks.com Kobo

How to Do a Systematic Review: A Best Practice Guide for Conducting and Reporting Narrative Reviews, Meta-Analyses, and Meta-Syntheses

Affiliations.

• 1 Behavioural Science Centre, Stirling Management School, University of Stirling, Stirling FK9 4LA, United Kingdom; email: [email protected].
• 2 Department of Psychological and Behavioural Science, London School of Economics and Political Science, London WC2A 2AE, United Kingdom.
• 3 Department of Statistics, Northwestern University, Evanston, Illinois 60208, USA; email: [email protected].
• PMID: 30089228
• DOI: 10.1146/annurev-psych-010418-102803

Systematic reviews are characterized by a methodical and replicable methodology and presentation. They involve a comprehensive search to locate all relevant published and unpublished work on a subject; a systematic integration of search results; and a critique of the extent, nature, and quality of evidence in relation to a particular research question. The best reviews synthesize studies to draw broad theoretical conclusions about what a literature means, linking theory to evidence and evidence to theory. This guide describes how to plan, conduct, organize, and present a systematic review of quantitative (meta-analysis) or qualitative (narrative review, meta-synthesis) information. We outline core standards and principles and describe commonly encountered problems. Although this guide targets psychological scientists, its high level of abstraction makes it potentially relevant to any subject area or discipline. We argue that systematic reviews are a key methodology for clarifying whether and how research findings replicate and for explaining possible inconsistencies, and we call for researchers to conduct systematic reviews to help elucidate whether there is a replication crisis.

Keywords: evidence; guide; meta-analysis; meta-synthesis; narrative; systematic review; theory.

• Guidelines as Topic
• Meta-Analysis as Topic*
• Publication Bias
• Review Literature as Topic
• Systematic Reviews as Topic*

Systematic literature reviews

Systematic approaches to literature review searching.

This guide is primarily for those undertaking a literature review. It outlines how to approach the searching phase systematically in order to identify relevant literature on a research question.

Why be systematic? This approach can:

• Provide a robust overview of the available literature on your topic
• Ensure relevant literature is identified and key publications are not overlooked
• Reduce irrelevant search results through search planning
• Help you to create a reproducible search strategy.

In addition, applying a systematic approach will allow you to work more efficiently.

A literature review may form an essential part of the research process, for example as a major component of a thesis or dissertation. Alternatively, a review may constitute a research project in itself - as a peer-reviewed publication in a journal, or as a report from a research funded project.

While the searching phase of any literature review should be approached in a systematic manner, you do not need to follow all of the techniques outlined in this guide. The methods you choose are dependent on the time and resources you have available, and the purpose of your literature review.

Systematic reviews vs. systematic approaches

A full systematic review aims to comprehensively identify, evaluate and integrate the findings of all relevant studies on a particular research question. A systematic approach involves a rigorous and structured search strategy, without necessarily attempting to include all available research on a particular topic.

• Open access
• Published: 14 August 2018

Defining the process to literature searching in systematic reviews: a literature review of guidance and supporting studies

• Chris Cooper   ORCID: orcid.org/0000-0003-0864-5607 1 ,
• Andrew Booth 2 ,
• Jo Varley-Campbell 1 ,
• Nicky Britten 3 &
• Ruth Garside 4  

BMC Medical Research Methodology volume  18 , Article number:  85 ( 2018 ) Cite this article

203k Accesses

205 Citations

118 Altmetric

Metrics details

Systematic literature searching is recognised as a critical component of the systematic review process. It involves a systematic search for studies and aims for a transparent report of study identification, leaving readers clear about what was done to identify studies, and how the findings of the review are situated in the relevant evidence.

Information specialists and review teams appear to work from a shared and tacit model of the literature search process. How this tacit model has developed and evolved is unclear, and it has not been explicitly examined before.

The purpose of this review is to determine if a shared model of the literature searching process can be detected across systematic review guidance documents and, if so, how this process is reported in the guidance and supported by published studies.

A literature review.

Two types of literature were reviewed: guidance and published studies. Nine guidance documents were identified, including: The Cochrane and Campbell Handbooks. Published studies were identified through ‘pearl growing’, citation chasing, a search of PubMed using the systematic review methods filter, and the authors’ topic knowledge.

The relevant sections within each guidance document were then read and re-read, with the aim of determining key methodological stages. Methodological stages were identified and defined. This data was reviewed to identify agreements and areas of unique guidance between guidance documents. Consensus across multiple guidance documents was used to inform selection of ‘key stages’ in the process of literature searching.

Eight key stages were determined relating specifically to literature searching in systematic reviews. They were: who should literature search, aims and purpose of literature searching, preparation, the search strategy, searching databases, supplementary searching, managing references and reporting the search process.

Conclusions

Eight key stages to the process of literature searching in systematic reviews were identified. These key stages are consistently reported in the nine guidance documents, suggesting consensus on the key stages of literature searching, and therefore the process of literature searching as a whole, in systematic reviews. Further research to determine the suitability of using the same process of literature searching for all types of systematic review is indicated.

Peer Review reports

Systematic literature searching is recognised as a critical component of the systematic review process. It involves a systematic search for studies and aims for a transparent report of study identification, leaving review stakeholders clear about what was done to identify studies, and how the findings of the review are situated in the relevant evidence.

Information specialists and review teams appear to work from a shared and tacit model of the literature search process. How this tacit model has developed and evolved is unclear, and it has not been explicitly examined before. This is in contrast to the information science literature, which has developed information processing models as an explicit basis for dialogue and empirical testing. Without an explicit model, research in the process of systematic literature searching will remain immature and potentially uneven, and the development of shared information models will be assumed but never articulated.

One way of developing such a conceptual model is by formally examining the implicit “programme theory” as embodied in key methodological texts. The aim of this review is therefore to determine if a shared model of the literature searching process in systematic reviews can be detected across guidance documents and, if so, how this process is reported and supported.

Identifying guidance

Key texts (henceforth referred to as “guidance”) were identified based upon their accessibility to, and prominence within, United Kingdom systematic reviewing practice. The United Kingdom occupies a prominent position in the science of health information retrieval, as quantified by such objective measures as the authorship of papers, the number of Cochrane groups based in the UK, membership and leadership of groups such as the Cochrane Information Retrieval Methods Group, the HTA-I Information Specialists’ Group and historic association with such centres as the UK Cochrane Centre, the NHS Centre for Reviews and Dissemination, the Centre for Evidence Based Medicine and the National Institute for Clinical Excellence (NICE). Coupled with the linguistic dominance of English within medical and health science and the science of systematic reviews more generally, this offers a justification for a purposive sample that favours UK, European and Australian guidance documents.

Nine guidance documents were identified. These documents provide guidance for different types of reviews, namely: reviews of interventions, reviews of health technologies, reviews of qualitative research studies, reviews of social science topics, and reviews to inform guidance.

Whilst these guidance documents occasionally offer additional guidance on other types of systematic reviews, we have focused on the core and stated aims of these documents as they relate to literature searching. Table  1 sets out: the guidance document, the version audited, their core stated focus, and a bibliographical pointer to the main guidance relating to literature searching.

Once a list of key guidance documents was determined, it was checked by six senior information professionals based in the UK for relevance to current literature searching in systematic reviews.

Identifying supporting studies

In addition to identifying guidance, the authors sought to populate an evidence base of supporting studies (henceforth referred to as “studies”) that contribute to existing search practice. Studies were first identified by the authors from their knowledge on this topic area and, subsequently, through systematic citation chasing key studies (‘pearls’ [ 1 ]) located within each key stage of the search process. These studies are identified in Additional file  1 : Appendix Table 1. Citation chasing was conducted by analysing the bibliography of references for each study (backwards citation chasing) and through Google Scholar (forward citation chasing). A search of PubMed using the systematic review methods filter was undertaken in August 2017 (see Additional file 1 ). The search terms used were: (literature search*[Title/Abstract]) AND sysrev_methods[sb] and 586 results were returned. These results were sifted for relevance to the key stages in Fig.  1 by CC.

The key stages of literature search guidance as identified from nine key texts

Extracting the data

To reveal the implicit process of literature searching within each guidance document, the relevant sections (chapters) on literature searching were read and re-read, with the aim of determining key methodological stages. We defined a key methodological stage as a distinct step in the overall process for which specific guidance is reported, and action is taken, that collectively would result in a completed literature search.

The chapter or section sub-heading for each methodological stage was extracted into a table using the exact language as reported in each guidance document. The lead author (CC) then read and re-read these data, and the paragraphs of the document to which the headings referred, summarising section details. This table was then reviewed, using comparison and contrast to identify agreements and areas of unique guidance. Consensus across multiple guidelines was used to inform selection of ‘key stages’ in the process of literature searching.

Having determined the key stages to literature searching, we then read and re-read the sections relating to literature searching again, extracting specific detail relating to the methodological process of literature searching within each key stage. Again, the guidance was then read and re-read, first on a document-by-document-basis and, secondly, across all the documents above, to identify both commonalities and areas of unique guidance.

Results and discussion

Our findings.

We were able to identify consensus across the guidance on literature searching for systematic reviews suggesting a shared implicit model within the information retrieval community. Whilst the structure of the guidance varies between documents, the same key stages are reported, even where the core focus of each document is different. We were able to identify specific areas of unique guidance, where a document reported guidance not summarised in other documents, together with areas of consensus across guidance.

Unique guidance

Only one document provided guidance on the topic of when to stop searching [ 2 ]. This guidance from 2005 anticipates a topic of increasing importance with the current interest in time-limited (i.e. “rapid”) reviews. Quality assurance (or peer review) of literature searches was only covered in two guidance documents [ 3 , 4 ]. This topic has emerged as increasingly important as indicated by the development of the PRESS instrument [ 5 ]. Text mining was discussed in four guidance documents [ 4 , 6 , 7 , 8 ] where the automation of some manual review work may offer efficiencies in literature searching [ 8 ].

Agreement between guidance: Defining the key stages of literature searching

Where there was agreement on the process, we determined that this constituted a key stage in the process of literature searching to inform systematic reviews.

From the guidance, we determined eight key stages that relate specifically to literature searching in systematic reviews. These are summarised at Fig. 1 . The data extraction table to inform Fig. 1 is reported in Table  2 . Table 2 reports the areas of common agreement and it demonstrates that the language used to describe key stages and processes varies significantly between guidance documents.

For each key stage, we set out the specific guidance, followed by discussion on how this guidance is situated within the wider literature.

Key stage one: Deciding who should undertake the literature search

The guidance.

Eight documents provided guidance on who should undertake literature searching in systematic reviews [ 2 , 4 , 6 , 7 , 8 , 9 , 10 , 11 ]. The guidance affirms that people with relevant expertise of literature searching should ‘ideally’ be included within the review team [ 6 ]. Information specialists (or information scientists), librarians or trial search co-ordinators (TSCs) are indicated as appropriate researchers in six guidance documents [ 2 , 7 , 8 , 9 , 10 , 11 ].

How the guidance corresponds to the published studies

The guidance is consistent with studies that call for the involvement of information specialists and librarians in systematic reviews [ 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 ] and which demonstrate how their training as ‘expert searchers’ and ‘analysers and organisers of data’ can be put to good use [ 13 ] in a variety of roles [ 12 , 16 , 20 , 21 , 24 , 25 , 26 ]. These arguments make sense in the context of the aims and purposes of literature searching in systematic reviews, explored below. The need for ‘thorough’ and ‘replicable’ literature searches was fundamental to the guidance and recurs in key stage two. Studies have found poor reporting, and a lack of replicable literature searches, to be a weakness in systematic reviews [ 17 , 18 , 27 , 28 ] and they argue that involvement of information specialists/ librarians would be associated with better reporting and better quality literature searching. Indeed, Meert et al. [ 29 ] demonstrated that involving a librarian as a co-author to a systematic review correlated with a higher score in the literature searching component of a systematic review [ 29 ]. As ‘new styles’ of rapid and scoping reviews emerge, where decisions on how to search are more iterative and creative, a clear role is made here too [ 30 ].

Knowing where to search for studies was noted as important in the guidance, with no agreement as to the appropriate number of databases to be searched [ 2 , 6 ]. Database (and resource selection more broadly) is acknowledged as a relevant key skill of information specialists and librarians [ 12 , 15 , 16 , 31 ].

Whilst arguments for including information specialists and librarians in the process of systematic review might be considered self-evident, Koffel and Rethlefsen [ 31 ] have questioned if the necessary involvement is actually happening [ 31 ].

Key stage two: Determining the aim and purpose of a literature search

The aim: Five of the nine guidance documents use adjectives such as ‘thorough’, ‘comprehensive’, ‘transparent’ and ‘reproducible’ to define the aim of literature searching [ 6 , 7 , 8 , 9 , 10 ]. Analogous phrases were present in a further three guidance documents, namely: ‘to identify the best available evidence’ [ 4 ] or ‘the aim of the literature search is not to retrieve everything. It is to retrieve everything of relevance’ [ 2 ] or ‘A systematic literature search aims to identify all publications relevant to the particular research question’ [ 3 ]. The Joanna Briggs Institute reviewers’ manual was the only guidance document where a clear statement on the aim of literature searching could not be identified. The purpose of literature searching was defined in three guidance documents, namely to minimise bias in the resultant review [ 6 , 8 , 10 ]. Accordingly, eight of nine documents clearly asserted that thorough and comprehensive literature searches are required as a potential mechanism for minimising bias.

The need for thorough and comprehensive literature searches appears as uniform within the eight guidance documents that describe approaches to literature searching in systematic reviews of effectiveness. Reviews of effectiveness (of intervention or cost), accuracy and prognosis, require thorough and comprehensive literature searches to transparently produce a reliable estimate of intervention effect. The belief that all relevant studies have been ‘comprehensively’ identified, and that this process has been ‘transparently’ reported, increases confidence in the estimate of effect and the conclusions that can be drawn [ 32 ]. The supporting literature exploring the need for comprehensive literature searches focuses almost exclusively on reviews of intervention effectiveness and meta-analysis. Different ‘styles’ of review may have different standards however; the alternative, offered by purposive sampling, has been suggested in the specific context of qualitative evidence syntheses [ 33 ].

What is a comprehensive literature search?

Whilst the guidance calls for thorough and comprehensive literature searches, it lacks clarity on what constitutes a thorough and comprehensive literature search, beyond the implication that all of the literature search methods in Table 2 should be used to identify studies. Egger et al. [ 34 ], in an empirical study evaluating the importance of comprehensive literature searches for trials in systematic reviews, defined a comprehensive search for trials as:

a search not restricted to English language;

where Cochrane CENTRAL or at least two other electronic databases had been searched (such as MEDLINE or EMBASE); and

at least one of the following search methods has been used to identify unpublished trials: searches for (I) conference abstracts, (ii) theses, (iii) trials registers; and (iv) contacts with experts in the field [ 34 ].

Tricco et al. (2008) used a similar threshold of bibliographic database searching AND a supplementary search method in a review when examining the risk of bias in systematic reviews. Their criteria were: one database (limited using the Cochrane Highly Sensitive Search Strategy (HSSS)) and handsearching [ 35 ].

Together with the guidance, this would suggest that comprehensive literature searching requires the use of BOTH bibliographic database searching AND supplementary search methods.

Comprehensiveness in literature searching, in the sense of how much searching should be undertaken, remains unclear. Egger et al. recommend that ‘investigators should consider the type of literature search and degree of comprehension that is appropriate for the review in question, taking into account budget and time constraints’ [ 34 ]. This view tallies with the Cochrane Handbook, which stipulates clearly, that study identification should be undertaken ‘within resource limits’ [ 9 ]. This would suggest that the limitations to comprehension are recognised but it raises questions on how this is decided and reported [ 36 ].

What is the point of comprehensive literature searching?

The purpose of thorough and comprehensive literature searches is to avoid missing key studies and to minimize bias [ 6 , 8 , 10 , 34 , 37 , 38 , 39 ] since a systematic review based only on published (or easily accessible) studies may have an exaggerated effect size [ 35 ]. Felson (1992) sets out potential biases that could affect the estimate of effect in a meta-analysis [ 40 ] and Tricco et al. summarize the evidence concerning bias and confounding in systematic reviews [ 35 ]. Egger et al. point to non-publication of studies, publication bias, language bias and MEDLINE bias, as key biases [ 34 , 35 , 40 , 41 , 42 , 43 , 44 , 45 , 46 ]. Comprehensive searches are not the sole factor to mitigate these biases but their contribution is thought to be significant [ 2 , 32 , 34 ]. Fehrmann (2011) suggests that ‘the search process being described in detail’ and that, where standard comprehensive search techniques have been applied, increases confidence in the search results [ 32 ].

Does comprehensive literature searching work?

Egger et al., and other study authors, have demonstrated a change in the estimate of intervention effectiveness where relevant studies were excluded from meta-analysis [ 34 , 47 ]. This would suggest that missing studies in literature searching alters the reliability of effectiveness estimates. This is an argument for comprehensive literature searching. Conversely, Egger et al. found that ‘comprehensive’ searches still missed studies and that comprehensive searches could, in fact, introduce bias into a review rather than preventing it, through the identification of low quality studies then being included in the meta-analysis [ 34 ]. Studies query if identifying and including low quality or grey literature studies changes the estimate of effect [ 43 , 48 ] and question if time is better invested updating systematic reviews rather than searching for unpublished studies [ 49 ], or mapping studies for review as opposed to aiming for high sensitivity in literature searching [ 50 ].

Aim and purpose beyond reviews of effectiveness

The need for comprehensive literature searches is less certain in reviews of qualitative studies, and for reviews where a comprehensive identification of studies is difficult to achieve (for example, in Public health) [ 33 , 51 , 52 , 53 , 54 , 55 ]. Literature searching for qualitative studies, and in public health topics, typically generates a greater number of studies to sift than in reviews of effectiveness [ 39 ] and demonstrating the ‘value’ of studies identified or missed is harder [ 56 ], since the study data do not typically support meta-analysis. Nussbaumer-Streit et al. (2016) have registered a review protocol to assess whether abbreviated literature searches (as opposed to comprehensive literature searches) has an impact on conclusions across multiple bodies of evidence, not only on effect estimates [ 57 ] which may develop this understanding. It may be that decision makers and users of systematic reviews are willing to trade the certainty from a comprehensive literature search and systematic review in exchange for different approaches to evidence synthesis [ 58 ], and that comprehensive literature searches are not necessarily a marker of literature search quality, as previously thought [ 36 ]. Different approaches to literature searching [ 37 , 38 , 59 , 60 , 61 , 62 ] and developing the concept of when to stop searching are important areas for further study [ 36 , 59 ].

The study by Nussbaumer-Streit et al. has been published since the submission of this literature review [ 63 ]. Nussbaumer-Streit et al. (2018) conclude that abbreviated literature searches are viable options for rapid evidence syntheses, if decision-makers are willing to trade the certainty from a comprehensive literature search and systematic review, but that decision-making which demands detailed scrutiny should still be based on comprehensive literature searches [ 63 ].

Key stage three: Preparing for the literature search

Six documents provided guidance on preparing for a literature search [ 2 , 3 , 6 , 7 , 9 , 10 ]. The Cochrane Handbook clearly stated that Cochrane authors (i.e. researchers) should seek advice from a trial search co-ordinator (i.e. a person with specific skills in literature searching) ‘before’ starting a literature search [ 9 ].

Two key tasks were perceptible in preparing for a literature searching [ 2 , 6 , 7 , 10 , 11 ]. First, to determine if there are any existing or on-going reviews, or if a new review is justified [ 6 , 11 ]; and, secondly, to develop an initial literature search strategy to estimate the volume of relevant literature (and quality of a small sample of relevant studies [ 10 ]) and indicate the resources required for literature searching and the review of the studies that follows [ 7 , 10 ].

Three documents summarised guidance on where to search to determine if a new review was justified [ 2 , 6 , 11 ]. These focused on searching databases of systematic reviews (The Cochrane Database of Systematic Reviews (CDSR) and the Database of Abstracts of Reviews of Effects (DARE)), institutional registries (including PROSPERO), and MEDLINE [ 6 , 11 ]. It is worth noting, however, that as of 2015, DARE (and NHS EEDs) are no longer being updated and so the relevance of this (these) resource(s) will diminish over-time [ 64 ]. One guidance document, ‘Systematic reviews in the Social Sciences’, noted, however, that databases are not the only source of information and unpublished reports, conference proceeding and grey literature may also be required, depending on the nature of the review question [ 2 ].

Two documents reported clearly that this preparation (or ‘scoping’) exercise should be undertaken before the actual search strategy is developed [ 7 , 10 ]).

The guidance offers the best available source on preparing the literature search with the published studies not typically reporting how their scoping informed the development of their search strategies nor how their search approaches were developed. Text mining has been proposed as a technique to develop search strategies in the scoping stages of a review although this work is still exploratory [ 65 ]. ‘Clustering documents’ and word frequency analysis have also been tested to identify search terms and studies for review [ 66 , 67 ]. Preparing for literature searches and scoping constitutes an area for future research.

Key stage four: Designing the search strategy

The Population, Intervention, Comparator, Outcome (PICO) structure was the commonly reported structure promoted to design a literature search strategy. Five documents suggested that the eligibility criteria or review question will determine which concepts of PICO will be populated to develop the search strategy [ 1 , 4 , 7 , 8 , 9 ]. The NICE handbook promoted multiple structures, namely PICO, SPICE (Setting, Perspective, Intervention, Comparison, Evaluation) and multi-stranded approaches [ 4 ].

With the exclusion of The Joanna Briggs Institute reviewers’ manual, the guidance offered detail on selecting key search terms, synonyms, Boolean language, selecting database indexing terms and combining search terms. The CEE handbook suggested that ‘search terms may be compiled with the help of the commissioning organisation and stakeholders’ [ 10 ].

The use of limits, such as language or date limits, were discussed in all documents [ 2 , 3 , 4 , 6 , 7 , 8 , 9 , 10 , 11 ].

Search strategy structure

The guidance typically relates to reviews of intervention effectiveness so PICO – with its focus on intervention and comparator - is the dominant model used to structure literature search strategies [ 68 ]. PICOs – where the S denotes study design - is also commonly used in effectiveness reviews [ 6 , 68 ]. As the NICE handbook notes, alternative models to structure literature search strategies have been developed and tested. Booth provides an overview on formulating questions for evidence based practice [ 69 ] and has developed a number of alternatives to the PICO structure, namely: BeHEMoTh (Behaviour of interest; Health context; Exclusions; Models or Theories) for use when systematically identifying theory [ 55 ]; SPICE (Setting, Perspective, Intervention, Comparison, Evaluation) for identification of social science and evaluation studies [ 69 ] and, working with Cooke and colleagues, SPIDER (Sample, Phenomenon of Interest, Design, Evaluation, Research type) [ 70 ]. SPIDER has been compared to PICO and PICOs in a study by Methley et al. [ 68 ].

The NICE handbook also suggests the use of multi-stranded approaches to developing literature search strategies [ 4 ]. Glanville developed this idea in a study by Whitting et al. [ 71 ] and a worked example of this approach is included in the development of a search filter by Cooper et al. [ 72 ].

Writing search strategies: Conceptual and objective approaches

Hausner et al. [ 73 ] provide guidance on writing literature search strategies, delineating between conceptually and objectively derived approaches. The conceptual approach, advocated by and explained in the guidance documents, relies on the expertise of the literature searcher to identify key search terms and then develop key terms to include synonyms and controlled syntax. Hausner and colleagues set out the objective approach [ 73 ] and describe what may be done to validate it [ 74 ].

The use of limits

The guidance documents offer direction on the use of limits within a literature search. Limits can be used to focus literature searching to specific study designs or by other markers (such as by date) which limits the number of studies returned by a literature search. The use of limits should be described and the implications explored [ 34 ] since limiting literature searching can introduce bias (explored above). Craven et al. have suggested the use of a supporting narrative to explain decisions made in the process of developing literature searches and this advice would usefully capture decisions on the use of search limits [ 75 ].

Key stage five: Determining the process of literature searching and deciding where to search (bibliographic database searching)

Table 2 summarises the process of literature searching as reported in each guidance document. Searching bibliographic databases was consistently reported as the ‘first step’ to literature searching in all nine guidance documents.

Three documents reported specific guidance on where to search, in each case specific to the type of review their guidance informed, and as a minimum requirement [ 4 , 9 , 11 ]. Seven of the key guidance documents suggest that the selection of bibliographic databases depends on the topic of review [ 2 , 3 , 4 , 6 , 7 , 8 , 10 ], with two documents noting the absence of an agreed standard on what constitutes an acceptable number of databases searched [ 2 , 6 ].

The guidance documents summarise ‘how to’ search bibliographic databases in detail and this guidance is further contextualised above in terms of developing the search strategy. The documents provide guidance of selecting bibliographic databases, in some cases stating acceptable minima (i.e. The Cochrane Handbook states Cochrane CENTRAL, MEDLINE and EMBASE), and in other cases simply listing bibliographic database available to search. Studies have explored the value in searching specific bibliographic databases, with Wright et al. (2015) noting the contribution of CINAHL in identifying qualitative studies [ 76 ], Beckles et al. (2013) questioning the contribution of CINAHL to identifying clinical studies for guideline development [ 77 ], and Cooper et al. (2015) exploring the role of UK-focused bibliographic databases to identify UK-relevant studies [ 78 ]. The host of the database (e.g. OVID or ProQuest) has been shown to alter the search returns offered. Younger and Boddy [ 79 ] report differing search returns from the same database (AMED) but where the ‘host’ was different [ 79 ].

The average number of bibliographic database searched in systematic reviews has risen in the period 1994–2014 (from 1 to 4) [ 80 ] but there remains (as attested to by the guidance) no consensus on what constitutes an acceptable number of databases searched [ 48 ]. This is perhaps because thinking about the number of databases searched is the wrong question, researchers should be focused on which databases were searched and why, and which databases were not searched and why. The discussion should re-orientate to the differential value of sources but researchers need to think about how to report this in studies to allow findings to be generalised. Bethel (2017) has proposed ‘search summaries’, completed by the literature searcher, to record where included studies were identified, whether from database (and which databases specifically) or supplementary search methods [ 81 ]. Search summaries document both yield and accuracy of searches, which could prospectively inform resource use and decisions to search or not to search specific databases in topic areas. The prospective use of such data presupposes, however, that past searches are a potential predictor of future search performance (i.e. that each topic is to be considered representative and not unique). In offering a body of practice, this data would be of greater practicable use than current studies which are considered as little more than individual case studies [ 82 , 83 , 84 , 85 , 86 , 87 , 88 , 89 , 90 ].

When to database search is another question posed in the literature. Beyer et al. [ 91 ] report that databases can be prioritised for literature searching which, whilst not addressing the question of which databases to search, may at least bring clarity as to which databases to search first [ 91 ]. Paradoxically, this links to studies that suggest PubMed should be searched in addition to MEDLINE (OVID interface) since this improves the currency of systematic reviews [ 92 , 93 ]. Cooper et al. (2017) have tested the idea of database searching not as a primary search method (as suggested in the guidance) but as a supplementary search method in order to manage the volume of studies identified for an environmental effectiveness systematic review. Their case study compared the effectiveness of database searching versus a protocol using supplementary search methods and found that the latter identified more relevant studies for review than searching bibliographic databases [ 94 ].

Key stage six: Determining the process of literature searching and deciding where to search (supplementary search methods)

Table 2 also summaries the process of literature searching which follows bibliographic database searching. As Table 2 sets out, guidance that supplementary literature search methods should be used in systematic reviews recurs across documents, but the order in which these methods are used, and the extent to which they are used, varies. We noted inconsistency in the labelling of supplementary search methods between guidance documents.

Rather than focus on the guidance on how to use the methods (which has been summarised in a recent review [ 95 ]), we focus on the aim or purpose of supplementary search methods.

The Cochrane Handbook reported that ‘efforts’ to identify unpublished studies should be made [ 9 ]. Four guidance documents [ 2 , 3 , 6 , 9 ] acknowledged that searching beyond bibliographic databases was necessary since ‘databases are not the only source of literature’ [ 2 ]. Only one document reported any guidance on determining when to use supplementary methods. The IQWiG handbook reported that the use of handsearching (in their example) could be determined on a ‘case-by-case basis’ which implies that the use of these methods is optional rather than mandatory. This is in contrast to the guidance (above) on bibliographic database searching.

The issue for supplementary search methods is similar in many ways to the issue of searching bibliographic databases: demonstrating value. The purpose and contribution of supplementary search methods in systematic reviews is increasingly acknowledged [ 37 , 61 , 62 , 96 , 97 , 98 , 99 , 100 , 101 ] but understanding the value of the search methods to identify studies and data is unclear. In a recently published review, Cooper et al. (2017) reviewed the literature on supplementary search methods looking to determine the advantages, disadvantages and resource implications of using supplementary search methods [ 95 ]. This review also summarises the key guidance and empirical studies and seeks to address the question on when to use these search methods and when not to [ 95 ]. The guidance is limited in this regard and, as Table 2 demonstrates, offers conflicting advice on the order of searching, and the extent to which these search methods should be used in systematic reviews.

Key stage seven: Managing the references

Five of the documents provided guidance on managing references, for example downloading, de-duplicating and managing the output of literature searches [ 2 , 4 , 6 , 8 , 10 ]. This guidance typically itemised available bibliographic management tools rather than offering guidance on how to use them specifically [ 2 , 4 , 6 , 8 ]. The CEE handbook provided guidance on importing data where no direct export option is available (e.g. web-searching) [ 10 ].

The literature on using bibliographic management tools is not large relative to the number of ‘how to’ videos on platforms such as YouTube (see for example [ 102 ]). These YouTube videos confirm the overall lack of ‘how to’ guidance identified in this study and offer useful instruction on managing references. Bramer et al. set out methods for de-duplicating data and reviewing references in Endnote [ 103 , 104 ] and Gall tests the direct search function within Endnote to access databases such as PubMed, finding a number of limitations [ 105 ]. Coar et al. and Ahmed et al. consider the role of the free-source tool, Zotero [ 106 , 107 ]. Managing references is a key administrative function in the process of review particularly for documenting searches in PRISMA guidance.

Key stage eight: Documenting the search

The Cochrane Handbook was the only guidance document to recommend a specific reporting guideline: Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [ 9 ]. Six documents provided guidance on reporting the process of literature searching with specific criteria to report [ 3 , 4 , 6 , 8 , 9 , 10 ]. There was consensus on reporting: the databases searched (and the host searched by), the search strategies used, and any use of limits (e.g. date, language, search filters (The CRD handbook called for these limits to be justified [ 6 ])). Three guidance documents reported that the number of studies identified should be recorded [ 3 , 6 , 10 ]. The number of duplicates identified [ 10 ], the screening decisions [ 3 ], a comprehensive list of grey literature sources searched (and full detail for other supplementary search methods) [ 8 ], and an annotation of search terms tested but not used [ 4 ] were identified as unique items in four documents.

The Cochrane Handbook was the only guidance document to note that the full search strategies for each database should be included in the Additional file 1 of the review [ 9 ].

All guidance documents should ultimately deliver completed systematic reviews that fulfil the requirements of the PRISMA reporting guidelines [ 108 ]. The guidance broadly requires the reporting of data that corresponds with the requirements of the PRISMA statement although documents typically ask for diverse and additional items [ 108 ]. In 2008, Sampson et al. observed a lack of consensus on reporting search methods in systematic reviews [ 109 ] and this remains the case as of 2017, as evidenced in the guidance documents, and in spite of the publication of the PRISMA guidelines in 2009 [ 110 ]. It is unclear why the collective guidance does not more explicitly endorse adherence to the PRISMA guidance.

Reporting of literature searching is a key area in systematic reviews since it sets out clearly what was done and how the conclusions of the review can be believed [ 52 , 109 ]. Despite strong endorsement in the guidance documents, specifically supported in PRISMA guidance, and other related reporting standards too (such as ENTREQ for qualitative evidence synthesis, STROBE for reviews of observational studies), authors still highlight the prevalence of poor standards of literature search reporting [ 31 , 110 , 111 , 112 , 113 , 114 , 115 , 116 , 117 , 118 , 119 ]. To explore issues experienced by authors in reporting literature searches, and look at uptake of PRISMA, Radar et al. [ 120 ] surveyed over 260 review authors to determine common problems and their work summaries the practical aspects of reporting literature searching [ 120 ]. Atkinson et al. [ 121 ] have also analysed reporting standards for literature searching, summarising recommendations and gaps for reporting search strategies [ 121 ].

One area that is less well covered by the guidance, but nevertheless appears in this literature, is the quality appraisal or peer review of literature search strategies. The PRESS checklist is the most prominent and it aims to develop evidence-based guidelines to peer review of electronic search strategies [ 5 , 122 , 123 ]. A corresponding guideline for documentation of supplementary search methods does not yet exist although this idea is currently being explored.

How the reporting of the literature searching process corresponds to critical appraisal tools is an area for further research. In the survey undertaken by Radar et al. (2014), 86% of survey respondents (153/178) identified a need for further guidance on what aspects of the literature search process to report [ 120 ]. The PRISMA statement offers a brief summary of what to report but little practical guidance on how to report it [ 108 ]. Critical appraisal tools for systematic reviews, such as AMSTAR 2 (Shea et al. [ 124 ]) and ROBIS (Whiting et al. [ 125 ]), can usefully be read alongside PRISMA guidance, since they offer greater detail on how the reporting of the literature search will be appraised and, therefore, they offer a proxy on what to report [ 124 , 125 ]. Further research in the form of a study which undertakes a comparison between PRISMA and quality appraisal checklists for systematic reviews would seem to begin addressing the call, identified by Radar et al., for further guidance on what to report [ 120 ].

Limitations

Other handbooks exist.

A potential limitation of this literature review is the focus on guidance produced in Europe (the UK specifically) and Australia. We justify the decision for our selection of the nine guidance documents reviewed in this literature review in section “ Identifying guidance ”. In brief, these nine guidance documents were selected as the most relevant health care guidance that inform UK systematic reviewing practice, given that the UK occupies a prominent position in the science of health information retrieval. We acknowledge the existence of other guidance documents, such as those from North America (e.g. the Agency for Healthcare Research and Quality (AHRQ) [ 126 ], The Institute of Medicine [ 127 ] and the guidance and resources produced by the Canadian Agency for Drugs and Technologies in Health (CADTH) [ 128 ]). We comment further on this directly below.

The handbooks are potentially linked to one another

What is not clear is the extent to which the guidance documents inter-relate or provide guidance uniquely. The Cochrane Handbook, first published in 1994, is notably a key source of reference in guidance and systematic reviews beyond Cochrane reviews. It is not clear to what extent broadening the sample of guidance handbooks to include North American handbooks, and guidance handbooks from other relevant countries too, would alter the findings of this literature review or develop further support for the process model. Since we cannot be clear, we raise this as a potential limitation of this literature review. On our initial review of a sample of North American, and other, guidance documents (before selecting the guidance documents considered in this review), however, we do not consider that the inclusion of these further handbooks would alter significantly the findings of this literature review.

This is a literature review

A further limitation of this review was that the review of published studies is not a systematic review of the evidence for each key stage. It is possible that other relevant studies could help contribute to the exploration and development of the key stages identified in this review.

This literature review would appear to demonstrate the existence of a shared model of the literature searching process in systematic reviews. We call this model ‘the conventional approach’, since it appears to be common convention in nine different guidance documents.

The findings reported above reveal eight key stages in the process of literature searching for systematic reviews. These key stages are consistently reported in the nine guidance documents which suggests consensus on the key stages of literature searching, and therefore the process of literature searching as a whole, in systematic reviews.

In Table 2 , we demonstrate consensus regarding the application of literature search methods. All guidance documents distinguish between primary and supplementary search methods. Bibliographic database searching is consistently the first method of literature searching referenced in each guidance document. Whilst the guidance uniformly supports the use of supplementary search methods, there is little evidence for a consistent process with diverse guidance across documents. This may reflect differences in the core focus across each document, linked to differences in identifying effectiveness studies or qualitative studies, for instance.

Eight of the nine guidance documents reported on the aims of literature searching. The shared understanding was that literature searching should be thorough and comprehensive in its aim and that this process should be reported transparently so that that it could be reproduced. Whilst only three documents explicitly link this understanding to minimising bias, it is clear that comprehensive literature searching is implicitly linked to ‘not missing relevant studies’ which is approximately the same point.

Defining the key stages in this review helps categorise the scholarship available, and it prioritises areas for development or further study. The supporting studies on preparing for literature searching (key stage three, ‘preparation’) were, for example, comparatively few, and yet this key stage represents a decisive moment in literature searching for systematic reviews. It is where search strategy structure is determined, search terms are chosen or discarded, and the resources to be searched are selected. Information specialists, librarians and researchers, are well placed to develop these and other areas within the key stages we identify.

This review calls for further research to determine the suitability of using the conventional approach. The publication dates of the guidance documents which underpin the conventional approach may raise questions as to whether the process which they each report remains valid for current systematic literature searching. In addition, it may be useful to test whether it is desirable to use the same process model of literature searching for qualitative evidence synthesis as that for reviews of intervention effectiveness, which this literature review demonstrates is presently recommended best practice.

Abbreviations

Behaviour of interest; Health context; Exclusions; Models or Theories

Cochrane Database of Systematic Reviews

The Cochrane Central Register of Controlled Trials

Database of Abstracts of Reviews of Effects

Enhancing transparency in reporting the synthesis of qualitative research

Institute for Quality and Efficiency in Healthcare

National Institute for Clinical Excellence

Population, Intervention, Comparator, Outcome

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

Setting, Perspective, Intervention, Comparison, Evaluation

Sample, Phenomenon of Interest, Design, Evaluation, Research type

STrengthening the Reporting of OBservational studies in Epidemiology

Trial Search Co-ordinators

Booth A. Unpacking your literature search toolbox: on search styles and tactics. Health Information & Libraries Journal. 2008;25(4):313–7.

Article   Google Scholar  

Petticrew M, Roberts H. Systematic reviews in the social sciences: a practical guide. Oxford: Blackwell Publishing Ltd; 2006.

Book   Google Scholar  

Institute for Quality and Efficiency in Health Care (IQWiG). IQWiG Methods Resources. 7 Information retrieval 2014 [Available from: https://www.ncbi.nlm.nih.gov/books/NBK385787/ .

NICE: National Institute for Health and Care Excellence. Developing NICE guidelines: the manual 2014. Available from: https://www.nice.org.uk/media/default/about/what-we-do/our-programmes/developing-nice-guidelines-the-manual.pdf .

Sampson M. MJ, Lefebvre C, Moher D, Grimshaw J. Peer Review of Electronic Search Strategies: PRESS; 2008.

Google Scholar  

Centre for Reviews & Dissemination. Systematic reviews – CRD’s guidance for undertaking reviews in healthcare. York: Centre for Reviews and Dissemination, University of York; 2009.

eunetha: European Network for Health Technology Assesment Process of information retrieval for systematic reviews and health technology assessments on clinical effectiveness 2016. Available from: http://www.eunethta.eu/sites/default/files/Guideline_Information_Retrieval_V1-1.pdf .

Kugley SWA, Thomas J, Mahood Q, Jørgensen AMK, Hammerstrøm K, Sathe N. Searching for studies: a guide to information retrieval for Campbell systematic reviews. Oslo: Campbell Collaboration. 2017; Available from: https://www.campbellcollaboration.org/library/searching-for-studies-information-retrieval-guide-campbell-reviews.html

Lefebvre C, Manheimer E, Glanville J. Chapter 6: searching for studies. In: JPT H, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions; 2011.

Collaboration for Environmental Evidence. Guidelines for Systematic Review and Evidence Synthesis in Environmental Management.: Environmental Evidence:; 2013. Available from: http://www.environmentalevidence.org/wp-content/uploads/2017/01/Review-guidelines-version-4.2-final-update.pdf .

The Joanna Briggs Institute. Joanna Briggs institute reviewers’ manual. 2014th ed: the Joanna Briggs institute; 2014. Available from: https://joannabriggs.org/assets/docs/sumari/ReviewersManual-2014.pdf

Beverley CA, Booth A, Bath PA. The role of the information specialist in the systematic review process: a health information case study. Health Inf Libr J. 2003;20(2):65–74.

Article   CAS   Google Scholar  

Harris MR. The librarian's roles in the systematic review process: a case study. Journal of the Medical Library Association. 2005;93(1):81–7.

PubMed   PubMed Central   Google Scholar  

Egger JB. Use of recommended search strategies in systematic reviews and the impact of librarian involvement: a cross-sectional survey of recent authors. PLoS One. 2015;10(5):e0125931.

Li L, Tian J, Tian H, Moher D, Liang F, Jiang T, et al. Network meta-analyses could be improved by searching more sources and by involving a librarian. J Clin Epidemiol. 2014;67(9):1001–7.

Article   PubMed   Google Scholar  

McGowan J, Sampson M. Systematic reviews need systematic searchers. J Med Libr Assoc. 2005;93(1):74–80.

Rethlefsen ML, Farrell AM, Osterhaus Trzasko LC, Brigham TJ. Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviews. J Clin Epidemiol. 2015;68(6):617–26.

Weller AC. Mounting evidence that librarians are essential for comprehensive literature searches for meta-analyses and Cochrane reports. J Med Libr Assoc. 2004;92(2):163–4.

Swinkels A, Briddon J, Hall J. Two physiotherapists, one librarian and a systematic literature review: collaboration in action. Health Info Libr J. 2006;23(4):248–56.

Foster M. An overview of the role of librarians in systematic reviews: from expert search to project manager. EAHIL. 2015;11(3):3–7.

Lawson L. OPERATING OUTSIDE LIBRARY WALLS 2004.

Vassar M, Yerokhin V, Sinnett PM, Weiher M, Muckelrath H, Carr B, et al. Database selection in systematic reviews: an insight through clinical neurology. Health Inf Libr J. 2017;34(2):156–64.

Townsend WA, Anderson PF, Ginier EC, MacEachern MP, Saylor KM, Shipman BL, et al. A competency framework for librarians involved in systematic reviews. Journal of the Medical Library Association : JMLA. 2017;105(3):268–75.

Cooper ID, Crum JA. New activities and changing roles of health sciences librarians: a systematic review, 1990-2012. Journal of the Medical Library Association : JMLA. 2013;101(4):268–77.

Crum JA, Cooper ID. Emerging roles for biomedical librarians: a survey of current practice, challenges, and changes. Journal of the Medical Library Association : JMLA. 2013;101(4):278–86.

Dudden RF, Protzko SL. The systematic review team: contributions of the health sciences librarian. Med Ref Serv Q. 2011;30(3):301–15.

Golder S, Loke Y, McIntosh HM. Poor reporting and inadequate searches were apparent in systematic reviews of adverse effects. J Clin Epidemiol. 2008;61(5):440–8.

Maggio LA, Tannery NH, Kanter SL. Reproducibility of literature search reporting in medical education reviews. Academic medicine : journal of the Association of American Medical Colleges. 2011;86(8):1049–54.

Meert D, Torabi N, Costella J. Impact of librarians on reporting of the literature searching component of pediatric systematic reviews. Journal of the Medical Library Association : JMLA. 2016;104(4):267–77.

Morris M, Boruff JT, Gore GC. Scoping reviews: establishing the role of the librarian. Journal of the Medical Library Association : JMLA. 2016;104(4):346–54.

Koffel JB, Rethlefsen ML. Reproducibility of search strategies is poor in systematic reviews published in high-impact pediatrics, cardiology and surgery journals: a cross-sectional study. PLoS One. 2016;11(9):e0163309.

Article   PubMed   PubMed Central   CAS   Google Scholar  

Fehrmann P, Thomas J. Comprehensive computer searches and reporting in systematic reviews. Research Synthesis Methods. 2011;2(1):15–32.

Booth A. Searching for qualitative research for inclusion in systematic reviews: a structured methodological review. Systematic Reviews. 2016;5(1):74.

Article   PubMed   PubMed Central   Google Scholar  

Egger M, Juni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health technology assessment (Winchester, England). 2003;7(1):1–76.

Tricco AC, Tetzlaff J, Sampson M, Fergusson D, Cogo E, Horsley T, et al. Few systematic reviews exist documenting the extent of bias: a systematic review. J Clin Epidemiol. 2008;61(5):422–34.

Booth A. How much searching is enough? Comprehensive versus optimal retrieval for technology assessments. Int J Technol Assess Health Care. 2010;26(4):431–5.

Papaioannou D, Sutton A, Carroll C, Booth A, Wong R. Literature searching for social science systematic reviews: consideration of a range of search techniques. Health Inf Libr J. 2010;27(2):114–22.

Petticrew M. Time to rethink the systematic review catechism? Moving from ‘what works’ to ‘what happens’. Systematic Reviews. 2015;4(1):36.

Betrán AP, Say L, Gülmezoglu AM, Allen T, Hampson L. Effectiveness of different databases in identifying studies for systematic reviews: experience from the WHO systematic review of maternal morbidity and mortality. BMC Med Res Methodol. 2005;5

Felson DT. Bias in meta-analytic research. J Clin Epidemiol. 1992;45(8):885–92.

Article   PubMed   CAS   Google Scholar  

Franco A, Malhotra N, Simonovits G. Publication bias in the social sciences: unlocking the file drawer. Science. 2014;345(6203):1502–5.

Hartling L, Featherstone R, Nuspl M, Shave K, Dryden DM, Vandermeer B. Grey literature in systematic reviews: a cross-sectional study of the contribution of non-English reports, unpublished studies and dissertations to the results of meta-analyses in child-relevant reviews. BMC Med Res Methodol. 2017;17(1):64.

Schmucker CM, Blümle A, Schell LK, Schwarzer G, Oeller P, Cabrera L, et al. Systematic review finds that study data not published in full text articles have unclear impact on meta-analyses results in medical research. PLoS One. 2017;12(4):e0176210.

Egger M, Zellweger-Zahner T, Schneider M, Junker C, Lengeler C, Antes G. Language bias in randomised controlled trials published in English and German. Lancet (London, England). 1997;350(9074):326–9.

Moher D, Pham B, Lawson ML, Klassen TP. The inclusion of reports of randomised trials published in languages other than English in systematic reviews. Health technology assessment (Winchester, England). 2003;7(41):1–90.

Pham B, Klassen TP, Lawson ML, Moher D. Language of publication restrictions in systematic reviews gave different results depending on whether the intervention was conventional or complementary. J Clin Epidemiol. 2005;58(8):769–76.

Mills EJ, Kanters S, Thorlund K, Chaimani A, Veroniki A-A, Ioannidis JPA. The effects of excluding treatments from network meta-analyses: survey. BMJ : British Medical Journal. 2013;347

Hartling L, Featherstone R, Nuspl M, Shave K, Dryden DM, Vandermeer B. The contribution of databases to the results of systematic reviews: a cross-sectional study. BMC Med Res Methodol. 2016;16(1):127.

van Driel ML, De Sutter A, De Maeseneer J, Christiaens T. Searching for unpublished trials in Cochrane reviews may not be worth the effort. J Clin Epidemiol. 2009;62(8):838–44.e3.

Buchberger B, Krabbe L, Lux B, Mattivi JT. Evidence mapping for decision making: feasibility versus accuracy - when to abandon high sensitivity in electronic searches. German medical science : GMS e-journal. 2016;14:Doc09.

Lorenc T, Pearson M, Jamal F, Cooper C, Garside R. The role of systematic reviews of qualitative evidence in evaluating interventions: a case study. Research Synthesis Methods. 2012;3(1):1–10.

Gough D. Weight of evidence: a framework for the appraisal of the quality and relevance of evidence. Res Pap Educ. 2007;22(2):213–28.

Barroso J, Gollop CJ, Sandelowski M, Meynell J, Pearce PF, Collins LJ. The challenges of searching for and retrieving qualitative studies. West J Nurs Res. 2003;25(2):153–78.

Britten N, Garside R, Pope C, Frost J, Cooper C. Asking more of qualitative synthesis: a response to Sally Thorne. Qual Health Res. 2017;27(9):1370–6.

Booth A, Carroll C. Systematic searching for theory to inform systematic reviews: is it feasible? Is it desirable? Health Info Libr J. 2015;32(3):220–35.

Kwon Y, Powelson SE, Wong H, Ghali WA, Conly JM. An assessment of the efficacy of searching in biomedical databases beyond MEDLINE in identifying studies for a systematic review on ward closures as an infection control intervention to control outbreaks. Syst Rev. 2014;3:135.

Nussbaumer-Streit B, Klerings I, Wagner G, Titscher V, Gartlehner G. Assessing the validity of abbreviated literature searches for rapid reviews: protocol of a non-inferiority and meta-epidemiologic study. Systematic Reviews. 2016;5:197.

Wagner G, Nussbaumer-Streit B, Greimel J, Ciapponi A, Gartlehner G. Trading certainty for speed - how much uncertainty are decisionmakers and guideline developers willing to accept when using rapid reviews: an international survey. BMC Med Res Methodol. 2017;17(1):121.

Ogilvie D, Hamilton V, Egan M, Petticrew M. Systematic reviews of health effects of social interventions: 1. Finding the evidence: how far should you go? J Epidemiol Community Health. 2005;59(9):804–8.

Royle P, Milne R. Literature searching for randomized controlled trials used in Cochrane reviews: rapid versus exhaustive searches. Int J Technol Assess Health Care. 2003;19(4):591–603.

Pearson M, Moxham T, Ashton K. Effectiveness of search strategies for qualitative research about barriers and facilitators of program delivery. Eval Health Prof. 2011;34(3):297–308.

Levay P, Raynor M, Tuvey D. The Contributions of MEDLINE, Other Bibliographic Databases and Various Search Techniques to NICE Public Health Guidance. 2015. 2015;10(1):19.

Nussbaumer-Streit B, Klerings I, Wagner G, Heise TL, Dobrescu AI, Armijo-Olivo S, et al. Abbreviated literature searches were viable alternatives to comprehensive searches: a meta-epidemiological study. J Clin Epidemiol. 2018;102:1–11.

Briscoe S, Cooper C, Glanville J, Lefebvre C. The loss of the NHS EED and DARE databases and the effect on evidence synthesis and evaluation. Res Synth Methods. 2017;8(3):256–7.

Stansfield C, O'Mara-Eves A, Thomas J. Text mining for search term development in systematic reviewing: A discussion of some methods and challenges. Research Synthesis Methods.n/a-n/a.

Petrova M, Sutcliffe P, Fulford KW, Dale J. Search terms and a validated brief search filter to retrieve publications on health-related values in Medline: a word frequency analysis study. Journal of the American Medical Informatics Association : JAMIA. 2012;19(3):479–88.

Stansfield C, Thomas J, Kavanagh J. 'Clustering' documents automatically to support scoping reviews of research: a case study. Res Synth Methods. 2013;4(3):230–41.

PubMed   Google Scholar  

Methley AM, Campbell S, Chew-Graham C, McNally R, Cheraghi-Sohi S. PICO, PICOS and SPIDER: a comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews. BMC Health Serv Res. 2014;14:579.

Andrew B. Clear and present questions: formulating questions for evidence based practice. Library Hi Tech. 2006;24(3):355–68.

Cooke A, Smith D, Booth A. Beyond PICO: the SPIDER tool for qualitative evidence synthesis. Qual Health Res. 2012;22(10):1435–43.

Whiting P, Westwood M, Bojke L, Palmer S, Richardson G, Cooper J, et al. Clinical effectiveness and cost-effectiveness of tests for the diagnosis and investigation of urinary tract infection in children: a systematic review and economic model. Health technology assessment (Winchester, England). 2006;10(36):iii-iv, xi-xiii, 1–154.

Cooper C, Levay P, Lorenc T, Craig GM. A population search filter for hard-to-reach populations increased search efficiency for a systematic review. J Clin Epidemiol. 2014;67(5):554–9.

Hausner E, Waffenschmidt S, Kaiser T, Simon M. Routine development of objectively derived search strategies. Systematic Reviews. 2012;1(1):19.

Hausner E, Guddat C, Hermanns T, Lampert U, Waffenschmidt S. Prospective comparison of search strategies for systematic reviews: an objective approach yielded higher sensitivity than a conceptual one. J Clin Epidemiol. 2016;77:118–24.

Craven J, Levay P. Recording database searches for systematic reviews - what is the value of adding a narrative to peer-review checklists? A case study of nice interventional procedures guidance. Evid Based Libr Inf Pract. 2011;6(4):72–87.

Wright K, Golder S, Lewis-Light K. What value is the CINAHL database when searching for systematic reviews of qualitative studies? Syst Rev. 2015;4:104.

Beckles Z, Glover S, Ashe J, Stockton S, Boynton J, Lai R, et al. Searching CINAHL did not add value to clinical questions posed in NICE guidelines. J Clin Epidemiol. 2013;66(9):1051–7.

Cooper C, Rogers M, Bethel A, Briscoe S, Lowe J. A mapping review of the literature on UK-focused health and social care databases. Health Inf Libr J. 2015;32(1):5–22.

Younger P, Boddy K. When is a search not a search? A comparison of searching the AMED complementary health database via EBSCOhost, OVID and DIALOG. Health Inf Libr J. 2009;26(2):126–35.

Lam MT, McDiarmid M. Increasing number of databases searched in systematic reviews and meta-analyses between 1994 and 2014. Journal of the Medical Library Association : JMLA. 2016;104(4):284–9.

Bethel A, editor Search summary tables for systematic reviews: results and findings. HLC Conference 2017a.

Aagaard T, Lund H, Juhl C. Optimizing literature search in systematic reviews - are MEDLINE, EMBASE and CENTRAL enough for identifying effect studies within the area of musculoskeletal disorders? BMC Med Res Methodol. 2016;16(1):161.

Adams CE, Frederick K. An investigation of the adequacy of MEDLINE searches for randomized controlled trials (RCTs) of the effects of mental health care. Psychol Med. 1994;24(3):741–8.

Kelly L, St Pierre-Hansen N. So many databases, such little clarity: searching the literature for the topic aboriginal. Canadian family physician Medecin de famille canadien. 2008;54(11):1572–3.

Lawrence DW. What is lost when searching only one literature database for articles relevant to injury prevention and safety promotion? Injury Prevention. 2008;14(6):401–4.

Lemeshow AR, Blum RE, Berlin JA, Stoto MA, Colditz GA. Searching one or two databases was insufficient for meta-analysis of observational studies. J Clin Epidemiol. 2005;58(9):867–73.

Sampson M, Barrowman NJ, Moher D, Klassen TP, Pham B, Platt R, et al. Should meta-analysts search Embase in addition to Medline? J Clin Epidemiol. 2003;56(10):943–55.

Stevinson C, Lawlor DA. Searching multiple databases for systematic reviews: added value or diminishing returns? Complementary Therapies in Medicine. 2004;12(4):228–32.

Suarez-Almazor ME, Belseck E, Homik J, Dorgan M, Ramos-Remus C. Identifying clinical trials in the medical literature with electronic databases: MEDLINE alone is not enough. Control Clin Trials. 2000;21(5):476–87.

Taylor B, Wylie E, Dempster M, Donnelly M. Systematically retrieving research: a case study evaluating seven databases. Res Soc Work Pract. 2007;17(6):697–706.

Beyer FR, Wright K. Can we prioritise which databases to search? A case study using a systematic review of frozen shoulder management. Health Info Libr J. 2013;30(1):49–58.

Duffy S, de Kock S, Misso K, Noake C, Ross J, Stirk L. Supplementary searches of PubMed to improve currency of MEDLINE and MEDLINE in-process searches via Ovid. Journal of the Medical Library Association : JMLA. 2016;104(4):309–12.

Katchamart W, Faulkner A, Feldman B, Tomlinson G, Bombardier C. PubMed had a higher sensitivity than Ovid-MEDLINE in the search for systematic reviews. J Clin Epidemiol. 2011;64(7):805–7.

Cooper C, Lovell R, Husk K, Booth A, Garside R. Supplementary search methods were more effective and offered better value than bibliographic database searching: a case study from public health and environmental enhancement (in Press). Research Synthesis Methods. 2017;

Cooper C, Booth, A., Britten, N., Garside, R. A comparison of results of empirical studies of supplementary search techniques and recommendations in review methodology handbooks: A methodological review. (In Press). BMC Systematic Reviews. 2017.

Greenhalgh T, Peacock R. Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. BMJ (Clinical research ed). 2005;331(7524):1064–5.

Article   PubMed Central   Google Scholar  

Hinde S, Spackman E. Bidirectional citation searching to completion: an exploration of literature searching methods. PharmacoEconomics. 2015;33(1):5–11.

Levay P, Ainsworth N, Kettle R, Morgan A. Identifying evidence for public health guidance: a comparison of citation searching with web of science and Google scholar. Res Synth Methods. 2016;7(1):34–45.

McManus RJ, Wilson S, Delaney BC, Fitzmaurice DA, Hyde CJ, Tobias RS, et al. Review of the usefulness of contacting other experts when conducting a literature search for systematic reviews. BMJ (Clinical research ed). 1998;317(7172):1562–3.

Westphal A, Kriston L, Holzel LP, Harter M, von Wolff A. Efficiency and contribution of strategies for finding randomized controlled trials: a case study from a systematic review on therapeutic interventions of chronic depression. Journal of public health research. 2014;3(2):177.

Matthews EJ, Edwards AG, Barker J, Bloor M, Covey J, Hood K, et al. Efficient literature searching in diffuse topics: lessons from a systematic review of research on communicating risk to patients in primary care. Health Libr Rev. 1999;16(2):112–20.

Bethel A. Endnote Training (YouTube Videos) 2017b [Available from: http://medicine.exeter.ac.uk/esmi/workstreams/informationscience/is_resources,_guidance_&_advice/ .

Bramer WM, Giustini D, de Jonge GB, Holland L, Bekhuis T. De-duplication of database search results for systematic reviews in EndNote. Journal of the Medical Library Association : JMLA. 2016;104(3):240–3.

Bramer WM, Milic J, Mast F. Reviewing retrieved references for inclusion in systematic reviews using EndNote. Journal of the Medical Library Association : JMLA. 2017;105(1):84–7.

Gall C, Brahmi FA. Retrieval comparison of EndNote to search MEDLINE (Ovid and PubMed) versus searching them directly. Medical reference services quarterly. 2004;23(3):25–32.

Ahmed KK, Al Dhubaib BE. Zotero: a bibliographic assistant to researcher. J Pharmacol Pharmacother. 2011;2(4):303–5.

Coar JT, Sewell JP. Zotero: harnessing the power of a personal bibliographic manager. Nurse Educ. 2010;35(5):205–7.

Moher D, Liberati A, Tetzlaff J, Altman DG, The PG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097.

Sampson M, McGowan J, Tetzlaff J, Cogo E, Moher D. No consensus exists on search reporting methods for systematic reviews. J Clin Epidemiol. 2008;61(8):748–54.

Toews LC. Compliance of systematic reviews in veterinary journals with preferred reporting items for systematic reviews and meta-analysis (PRISMA) literature search reporting guidelines. Journal of the Medical Library Association : JMLA. 2017;105(3):233–9.

Booth A. "brimful of STARLITE": toward standards for reporting literature searches. Journal of the Medical Library Association : JMLA. 2006;94(4):421–9. e205

Faggion CM Jr, Wu YC, Tu YK, Wasiak J. Quality of search strategies reported in systematic reviews published in stereotactic radiosurgery. Br J Radiol. 2016;89(1062):20150878.

Mullins MM, DeLuca JB, Crepaz N, Lyles CM. Reporting quality of search methods in systematic reviews of HIV behavioral interventions (2000–2010): are the searches clearly explained, systematic and reproducible? Research Synthesis Methods. 2014;5(2):116–30.

Yoshii A, Plaut DA, McGraw KA, Anderson MJ, Wellik KE. Analysis of the reporting of search strategies in Cochrane systematic reviews. Journal of the Medical Library Association : JMLA. 2009;97(1):21–9.

Bigna JJ, Um LN, Nansseu JR. A comparison of quality of abstracts of systematic reviews including meta-analysis of randomized controlled trials in high-impact general medicine journals before and after the publication of PRISMA extension for abstracts: a systematic review and meta-analysis. Syst Rev. 2016;5(1):174.

Akhigbe T, Zolnourian A, Bulters D. Compliance of systematic reviews articles in brain arteriovenous malformation with PRISMA statement guidelines: review of literature. Journal of clinical neuroscience : official journal of the Neurosurgical Society of Australasia. 2017;39:45–8.

Tao KM, Li XQ, Zhou QH, Moher D, Ling CQ, Yu WF. From QUOROM to PRISMA: a survey of high-impact medical journals' instructions to authors and a review of systematic reviews in anesthesia literature. PLoS One. 2011;6(11):e27611.

Wasiak J, Tyack Z, Ware R. Goodwin N. Jr. Poor methodological quality and reporting standards of systematic reviews in burn care management. International wound journal: Faggion CM; 2016.

Tam WW, Lo KK, Khalechelvam P. Endorsement of PRISMA statement and quality of systematic reviews and meta-analyses published in nursing journals: a cross-sectional study. BMJ Open. 2017;7(2):e013905.

Rader T, Mann M, Stansfield C, Cooper C, Sampson M. Methods for documenting systematic review searches: a discussion of common issues. Res Synth Methods. 2014;5(2):98–115.

Atkinson KM, Koenka AC, Sanchez CE, Moshontz H, Cooper H. Reporting standards for literature searches and report inclusion criteria: making research syntheses more transparent and easy to replicate. Res Synth Methods. 2015;6(1):87–95.

McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol. 2016;75:40–6.

Sampson M, McGowan J, Cogo E, Grimshaw J, Moher D, Lefebvre C. An evidence-based practice guideline for the peer review of electronic search strategies. J Clin Epidemiol. 2009;62(9):944–52.

Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ (Clinical research ed). 2017;358.

Whiting P, Savović J, Higgins JPT, Caldwell DM, Reeves BC, Shea B, et al. ROBIS: a new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol. 2016;69:225–34.

Relevo R, Balshem H. Finding evidence for comparing medical interventions: AHRQ and the effective health care program. J Clin Epidemiol. 2011;64(11):1168–77.

Medicine Io. Standards for Systematic Reviews 2011 [Available from: http://www.nationalacademies.org/hmd/Reports/2011/Finding-What-Works-in-Health-Care-Standards-for-Systematic-Reviews/Standards.aspx .

CADTH: Resources 2018.

Download references

Acknowledgements

CC acknowledges the supervision offered by Professor Chris Hyde.

This publication forms a part of CC’s PhD. CC’s PhD was funded through the National Institute for Health Research (NIHR) Health Technology Assessment (HTA) Programme (Project Number 16/54/11). The open access fee for this publication was paid for by Exeter Medical School.

RG and NB were partially supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care South West Peninsula.

The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.

Author information

Authors and affiliations.

Institute of Health Research, University of Exeter Medical School, Exeter, UK

Chris Cooper & Jo Varley-Campbell

HEDS, School of Health and Related Research (ScHARR), University of Sheffield, Sheffield, UK

Andrew Booth

Nicky Britten

European Centre for Environment and Human Health, University of Exeter Medical School, Truro, UK

Ruth Garside

You can also search for this author in PubMed   Google Scholar

Contributions

CC conceived the idea for this study and wrote the first draft of the manuscript. CC discussed this publication in PhD supervision with AB and separately with JVC. CC revised the publication with input and comments from AB, JVC, RG and NB. All authors revised the manuscript prior to submission. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Chris Cooper .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional file

Additional file 1:.

Appendix tables and PubMed search strategy. Key studies used for pearl growing per key stage, working data extraction tables and the PubMed search strategy. (DOCX 30 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Cooper, C., Booth, A., Varley-Campbell, J. et al. Defining the process to literature searching in systematic reviews: a literature review of guidance and supporting studies. BMC Med Res Methodol 18 , 85 (2018). https://doi.org/10.1186/s12874-018-0545-3

Download citation

Received : 20 September 2017

Accepted : 06 August 2018

Published : 14 August 2018

DOI : https://doi.org/10.1186/s12874-018-0545-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Literature Search Process
• Citation Chasing
• Tacit Models
• Unique Guidance
• Information Specialists

BMC Medical Research Methodology

ISSN: 1471-2288

• General enquiries: [email protected]

Browse Econ Literature

• Working papers
• Software components
• Book chapters
• JEL classification

More features

• Subscribe to new research

RePEc Biblio

Author registration.

• Economics Virtual Seminar Calendar NEW!

How to Undertake an Impactful Literature Review: Understanding Review Approaches and Guidelines for High-impact Systematic Literature Reviews

• Author & abstract
• Related works & more

Corrections

• Amrita Chakraborty
• Arpan Kumar Kar

Suggested Citation

Download full text from publisher.

Follow serials, authors, keywords & more

Public profiles for Economics researchers

Various research rankings in Economics

RePEc Genealogy

Who was a student of whom, using RePEc

Curated articles & papers on economics topics

Upload your paper to be listed on RePEc and IDEAS

New papers by email

Subscribe to new additions to RePEc

EconAcademics

Blog aggregator for economics research

Cases of plagiarism in Economics

About RePEc

Initiative for open bibliographies in Economics

News about RePEc

Questions about IDEAS and RePEc

RePEc volunteers

Participating archives

Publishers indexing in RePEc

Privacy statement

Found an error or omission?

Opportunities to help RePEc

Get papers listed

Have your research listed on RePEc

Open a RePEc archive

Have your institution's/publisher's output listed on RePEc

Get RePEc data

Use data assembled by RePEc

Differences and common ground in the frameworks of health-related quality of life in traditional Chinese medicine and modern medicine: a systematic review

• Open access
• Published: 13 May 2024

Cite this article

You have full access to this open access article

• Yifan Ding   ORCID: orcid.org/0009-0006-8112-628X 1 ,
• Zhuxin Mao 2 ,
• Nan Luo 3 ,
• Zhihao Yang 4 &
• Jan Busschbach 1  

This systematic review aims to explore the conceptualization of health-related quality of life (HRQoL) in China. With HRQoL influenced by both modern medicine (MM) and traditional Chinese medicine (TCM), the study seeks to identify differences and common ground between the frameworks of MM and TCM as defined in the literature.

A systematic literature search was conducted across three Chinese databases and four English databases. The data was extracted including title, author(s), publication year, region, aim, method, category, and result. When sorting data, we broke down the HRQoL frameworks into concepts, domains and facets, with a focus on overlapped facets between the frameworks of MM and TCM.

A total of 31 studies were included. In the perspective of TCM, HRQoL is centered around three key 'concepts': (1) 'xingshentongyi' (unity of body and spirit), (2) 'tianrenheyi' (harmony between man and nature), and (3) 'qiqing' (seven emotional forms). In contrast, the MM framework comprises 'physical,' 'mental,' 'social,' and 'environment' domains. Out of the 59 unique facets identified, 28 are common to both TCM and MM, 9 specific to TCM, and 22 specific to MM. 'Appetite,' 'sleep,' and 'energy' are the most frequently mentioned facets in both frameworks.

The concept of HRQoL in China encompasses frameworks rooted in both TCM and MM. While TCM and MM have distinct healthcare approaches, they share overlapping domains when measuring HRQoL through questionnaires. Furthermore, TCM and MM demonstrate considerable convergence in terms of HRQoL facets, showing the potential for utilizing HRQoL instruments across different cultural settings.

Avoid common mistakes on your manuscript.

Plain English summary

Health-related quality of life (HRQoL) is about how people perceive their own health. The instruments used to measure it, even in China, are mostly devised in the West. However, it is increasingly understood that cultural distinctions might impact how we understand HRQoL, causing doubts about whether Western tools accurately represent this concept. This suggests that instruments created in the West might not fully grasp the health experiences of people in China, considering their unique cultural background. This systematic review study tries to understand from the literature how Chinese people think about HRQoL. It found two different ways of thinking about it: one based on modern medicine and the other on traditional Chinese medicine. Surprisingly, these ways of thinking lead to similar ideas about HRQoL. This study demonstrates the adaptability of HRQoL instruments across cultural settings with minimal adjustments. This challenging the notion that diverse cultures yield entirely distinct perspectives on HRQoL.

Introduction

Health-related quality of life (HRQoL) is a multi-dimensional concept. Despite the challenge of defining HRQoL, there is a broad consensus that HRQoL focuses on revealing people’s subjective evaluation of health [ 1 , 2 , 3 , 4 ]. Obtaining HRQoL information is crucial for comprehending patient health, making informed medical decisions [ 5 , 6 ], evaluating healthcare interventions, guiding public health policies, and ultimately contributing to evidence-based medicine [ 7 , 8 , 9 ]. Some HRQoL outcomes are preference-based, informing decisions on health products, technologies, and policies through QALY calculations [ 7 ]. Note that in this paper, we focus on HRQoL, rather than quality of life (QoL) in general.

The way we conceptualize HRQoL determines the specific health facets that we include in HRQoL measurements. To ensure valid and reliable HRQoL observations, standardized instruments are utilized. However, the recognition of cultural influences on HRQoL conceptualization has led to discussions about the suitability of Western-developed HRQoL measures in China [ 10 , 11 ]. Specifically, when Western instruments, grounded in a distinct cultural context, are applied to measure HRQoL in China, dimensions important to the Chinese population's understanding of health might be overlooked or not accurately captured. This misalignment can lead to an incomplete representation of the true HRQoL experienced by Chinese individuals [ 12 ].

For example, Prior et al. found that participants from Cantonese-speaking communities in England described HRQoL using facets related to traditional Chinese medicine (TCM), such as ‘demons’, ‘food’, and ‘weather’, alongside common Western facets [ 13 ]. Similarly, a Q-methodological study emphasized the importance of 'spirits' and 'body constitution' in describing HRQoL among Chinese living in China, which were often absent in commonly used instruments [ 11 ]. Additionally, Mao et al. compared Western and Chinese-developed HRQoL instruments and identified exclusive HRQoL domains in China, including ‘emotion control’, ‘weather adaption’, ‘social adaption’, ‘spirit’, and ‘complexion’ [ 8 ]. These new aspects of Chinese HRQoL appear to be linked to TCM, indicating that TCM could influence how people perceive and describe HRQoL. Alternatively, it's possible that the Chinese perspective on HRQoL shapes practices within TCM. Thus, it can be argued that Western-developed instruments may inadequately capture the health experiences of Chinese populations within their cultural context, for instance in neglecting the influence of TCM [ 8 , 10 , 14 ]. Consequently, using Western-developed HRQoL instruments to assess HRQoL in Chinese populations may not yield optimal results if there are conceptual differences between Western and Chinese perspectives [ 10 , 12 ].

Despite debates regarding the applicability of Western HRQoL instruments, the influence of culture on HRQoL conceptualization and instrument development remains unclear [ 15 ]. In China, efforts have been made by the scientific community to define HRQoL within a Chinese cultural context and use it to guide instrument development. However, there has been no systematic assessment, comparison, or synthesis of these Chinese-specific definitions or instruments. Consequently, it is still unclear how HRQoL is defined in China, and how these ‘Chinese definitions’ differ from those used in imported HRQoL instruments. This study aims to systematically review published studies that describe the theoretical and operationalised conceptualization of HRQoL. The objective is to explore and synthesize perceptions of HRQoL within a Chinese cultural setting.

This systematic review followed the guideline of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [ 16 ], including the following four parts: search strategy, identification and selection, data extraction and quality assessment.

Search strategy and inclusion criteria

A comprehensive search was conducted across three Chinese databases (CNKI, Weipu, and Wanfang) and four English databases (EMBASE, MEDLINE, Web of Science, and Cochrane), in addition to a restricted search on Google Scholar. We applied specific inclusion criteria: (a) reporting HRQoL measures developed in a Chinese cultural setting; (b) discussing the definition of HRQoL in Chinese cultural setting or constructing a conceptual framework of HRQoL specifically to Chinese culture; (c) qualitative interviews exploring Chinese people's understandings of HRQoL. We excluded articles focusing on Western-based perspectives of HRQoL.

Both of detailed Chinese and English search strategies were presented in the Appendix 1 . The Chinese search strategy was developed through an internal discussion between the four Chinese researchers in team. When formulating the English strategy, we sought professional assistance from the librarians at Erasmus MC, followed by a discussion within the entire research team. To ensure a comprehensive search, we took into account potential spelling errors and synonyms when formulating the search strategies, as can be seen in the search strategies presented in the Appendix 1 .

Identification, selection and exclusion criteria

The research team reviewed all types of publications including original research, reviews, commentaries and dissertations. However, conference reports were excluded due to their potential lack of rigor and incomplete data. There were no constraints based on publication date other than the specified end date (March 3rd, 2022).

Two bilingual reviewers (YD and ZM) reviewed the titles and abstracts, applying the following exclusion criteria: (a) studies focused on a specific disease; (b) studies focused on a particular population (e.g., elderly or specific region); (c) studies utilizing existing instruments such as SF-36, EQ-5D, etc.; (d) studies focused on other constructs (e.g., happiness, well-being, life satisfaction); (e) studies focused on the target population which is not Chinese. Any disagreements between the reviewers were resolved through internal team discussions (ZM, ZY, and NL). The first author (YD) then conducted independent reviews of the resulting full-text articles. In cases where YD had doubts regarding the eligibility of a paper, the doubts were discussed with the review team. The review team consisted of one member with methodological expertise (ZM) and two individuals well-versed in the topic (ZY and NL). If disagreements persisted, they were resolved through consensus, but in case a consensus was not reached, an external opinion (JB) was sought.

Data extraction and analysis

According to the searching results, the key information of all eligible studies was extracted including the title, author, publish year, region, study aim, methods, and results (see Appendix 2 ). After reviewing, all eligible articles were categorized into two groups based on their content: definition and framework. The definition category focused on providing a general definition of HRQoL, while the framework category presented a detailed hierarchical system organizing specific health facets into health domains, and further grouped into health concepts. The category of each paper is documented in Appendix 2 . In the “ Results ” section, we extracted specific definitions from papers focusing on the general definition. For those paper that focused on the frameworks, we outlined the main hierarchical structure of domains and concepts. The specific facets for each instrument were documented in Appendix 3 .

This review aimed to synthesize the conceptualization of HRQoL, both theoretically (narrative definition) and operationally (HRQoL instrument presentation). In our approach to HRQoL frameworks, we employed a four-level hierarchy, which included concepts (the first level), domains (the second level), facets (the third level), and items (the fourth level). 'Concepts' represent the higher-order theoretical components of the theory, while 'domains' are the second-order aspects defined using 'items.' Given that closely related items may have different wording (e.g., walking, mobility, movement), we grouped similar items under the term 'facets'. In this study, we looked for the overlap in facets between MM and TCM. However, during the analysis process, we observed that some studies presented HRQoL from a TCM perspective, while others did not. These two perspectives resulted in two distinct conceptual frameworks (see “ Results ” section, Figs.  3 and 4 ).

For studies focusing on a general definition of HRQoL, frequently-mentioned health concepts (sometimes also health domains) were extracted. For studies providing detailed descriptive system, all specific facets were extracted. The extracted facets (see Appendix 3 ) were grouped by the reviewers, and the frequency of each facet was recorded (see Appendix 4 ). During the grouping process, we referred to the existing categories of descriptive systems as found in papers because most of them were already grouped by the designers of frameworks or instruments. If facets were classified into different groups in different frameworks or instruments, we made the classification after discussions within the review team. The grouping was independently conducted by two reviewers (YD and ZM), and any inconsistencies were resolved through internal team discussions.

Quality assessment

Normally, quality assessment for eligible studies is essential to a systematic review. However, after internal discussion, we opted not to conduct such an assessment in this review for maximizing the inclusion of studies. We were concerned that performing a quality assessment might inadvertently exclude relevant studies, resulting in the loss of valuable information. Nevertheless, the paper clearly written outside the scientific domain was excluded. These were articles that lacked 'scientific' jargon and failed to provide sufficient literature references to support the claims made in the article. For instance, it was found that a paper titled '20 New Concepts of Healthy Living,' which appeared more like a glossy magazine article than a scholarly contribution. As a result, such articles were considered unsuitable for inclusion in our study.

After removing duplicates, a total of 6437 English articles and 6169 Chinese articles were identified through database searches. Following title and abstract screening, 33 English studies and 87 Chinese studies remained. Full-text selection further reduced the number to 6 English articles and 25 Chinese articles. The detailed information of the included paper was provided in Appendix 2 . The selection process was illustrated in Figs.  1 and 2 of the PRISMA flow charts.

PRISMA flow diagram (English database)

PRISMA flow diagram (Chinese database)

During full-text review, the 31 included papers were primarily divided into two categories: focusing on general definitions of HRQoL and frameworks of HRQoL. Among them, 14 papers focus on providing a general definition of HRQoL, 11 papers delve into HRQoL frameworks, while 6 papers contain information pertaining to both the general definition and the framework. Additionally, a distinction was made between studies based on TCM perspectives and those using the perspective of modern medicine (MM). The categorization of each paper was provided in Appendix 2 .

The perspective of TCM on HRQoL

Based on the included papers focusing on general definition, we found that these papers introduced the foundational theory of TCM, which not only supplied potential corroborative evidence for TCM framework but also laid the foundation upon which TCM framework was developed.

The notion of 'Yin Yang' theory emerged most prominently as the foundational principle of TCM [ 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 ]. Additionally, the notion of 'Ping ren' [ 21 , 22 , 27 , 28 , 29 , 30 , 31 , 32 ] and 'He' [ 33 , 34 ] were also highlighted as the basis principle of TCM. The meanings of these principles are as follows. According to TCM, everything in the universe can be divided into two opposite but interconnected forces: Yin and Yang. For instance, the earth represents Yin, while the sky represents Yang, reflecting a universal concept. Specifically for human, the spirit is considered Yin, while the body is considered Yang. Health is believed to arise from the balance between Yin and Yang, both internally and within the surrounding environment. Additionally, individuals' health is influenced by the balance between different materials and the harmony between their 'Yin essence' and 'Yang spirit' [ 20 ].

Under the guidance of the general 'Yin Yang' theory, the most recognized definition of health is 'Ping ren' [ 21 , 22 , 27 , 28 , 29 , 30 , 31 , 32 ], which refers to a person in a healthy state with harmonious blood circulation, calm breathing, and steady pulse. TCM practitioners commonly view these indicators as signs of good health [ 19 , 22 , 24 ]. The concept of 'Yin Yang' aligns with another related phrase, 'He,' which emphasizes ‘harmony’ as the essence of health. 'He' represents the harmony between ‘Qi’ (energy in the broadest sense possible) and blood (important indicators of physical health in TCM) and the harmony of the spirit (associated with mental and emotional well-being). Both 'Yin Yang' and 'He' reflect characteristic philosophical thoughts in Chinese traditional culture [ 26 , 30 ].

In the identified studies, the three most commonly used concepts under the 'Yin Yang' theory for defining and measuring health were as follows: ‘Xing Shen Tong Yi'(形神统一) [ 17 , 18 , 19 , 20 , 22 , 25 , 26 , 27 , 31 , 33 , 35 , 36 , 37 ], 'Tian Ren He Yi'(天人合一) [ 17 , 19 , 20 , 22 , 25 , 26 , 27 , 28 , 35 , 37 , 38 ] and 'Qi Qing'(七情) [ 17 , 25 , 27 , 36 , 37 ].

Xing Shen Tong Yi

'Xing Shen Tong Yi' means the unity of body and spirit, which can be specifically divided into two domains: 'xing' and 'shen'. The 'xing' domain refers to the body structure, such as the viscera, meridians, essence, blood, muscles, limbs, and bones [ 19 ], focusing on physical health. By summarizing the included studies, the following facets were identified: stamina, sleep, appetite and digestion, complexion, stool, mobility, self-care, discomfort, pain, urination, voice, and constitution. These indicators can be used to confirm whether a person is physically healthy or not.

The 'Shen' domain has a broader meaning, which refers to both the gods of nature (i.e. natural phenomena and laws) and the mental state of the human body (i.e. emotions, will, personality, memory, and perception) [ 22 ]. By summarizing the included studies, the following facets were identified: thinking, verbal expression, consciousness, spirit of the eyes, memory, concentration, fatigue, confidence, and satisfaction. 'Xing Shen Tong Yi' also reflects the basic view of holism in TCM that people are an organic integrity, and that the body and mind are closely interdependent [ 20 ].

Tian Ren He Yi

TCM emphasizes the unity of the human body and its relationship with the external environment [ 17 , 27 ]. 'Tian Ren He Yi' emphasizes the harmony between humans and environment [ 17 , 20 , 35 ], which can be divided into two domains: natural environment and social environment.

In the 'natural environment' domain, TCM highlights the interconnectedness between humans and nature. Any changes or movements in nature will directly or indirectly affect the human body, such as seasonal climate changes, regional differences, leading to physiological discomfort or pathological changes [ 17 ]. This domain encompasses two facets: climate adaptation and adjustment, as well as dwelling conditions.

In the 'social environment' domain, TCM recognizes that humans are not only part of nature but also part of society. Health is influenced by various social factors, including political, economic, cultural, marriage, and interpersonal aspects. These factors will affect people's emotional changes, which can further impact the health of human. Common facets in this domain include socialization (e.g., communication, sex life, and loneliness), family (including family relationships, support, and conflicts), work (including relationships with colleagues and work performance), and economic conditions [ 39 ].

The concept of 'Tian Ren He Yi' is the direct embodiment of the relationship between human and environment. People's adaptation to the external environment is also the external embodiment of human health. Maintaining harmony between humans and the natural environment is of great importance [ 19 , 20 , 35 ].

In TCM, the concept of 'Qi Qing' refers to the seven basic emotions: joy, anger, worry, pensiveness, grief, fear, and anxiety [ 17 ]. According to TCM, these emotions are closely linked to the five internal organs to our body: heart, liver, spleen, lung, and kidney [ 36 , 37 ]. Each emotion corresponds to a specific organ, and when emotions become intense or unstable, they can disrupt the balance of ‘Qi’ (energy in the broadest sense possible) and blood, leading to physical problems. For example, anger affects the liver, joy affects the heart, pensiveness affects the spleen, worry affects the lung, and fear affects the kidney [ 17 ]. Therefore, maintaining emotional stability is essential for overall health and proper organ functioning.

Additionally, some of the definitions of HRQoL mentioned in the eligible studies may not fall within the purview of 'Yin Yang'. For instance, Confucianism emphasizes positive attitude and family relationships [ 18 , 22 ], as well as the concept of ‘moderation’ to maintain a healthy state [ 22 ]. Taoism highlights conforming to nature and maintaining peace of mind without overthinking [ 22 , 30 , 31 , 40 ]. Buddhism places importance on virtue and the theory of karma, where good behavior leads to positive outcomes, including physical health and mental satisfaction [ 18 , 22 ].

MM framework

From MM's perspective, two definitions were discussed. Firstly, one study adopted the definition from 'Ci Hai,' an authoritative Chinese dictionary [ 35 ]: ‘The state of well-developed organs and systems, normal functions, strong physique, full of energy, and high labor efficiency, which are usually measured through physical examination and various physiological indicators’ [ 41 ]. Secondly, most studies on HRQoL were based on the World Health Organization's (WHO) definition of ‘health’ [ 10 , 18 , 19 ]: ‘health is a complete state of physical, mental, and social well-being, not simply the absence of disease or infirmity’ [ 42 ]. The WHO definition of ‘health’ was not only used in defining HRQoL but also incorporated into the development of HRQoL instruments. Most studies defined HRQoL using the following three concepts: physical health, mental health, and social health, with some suggesting modifications. In the Chinese context, the concept of 'environment' such as dwelling conditions and changes in climate may be added [ 18 , 21 , 22 , 23 , 38 ]. The detailed framework results are shown in Figs.  3 and 4 .

Traditional Chinese medicine framework

Modern medicine framework

Physical health

The concept of 'physical health' can be divided into two domains: 'physical function' and 'physical senses'. In the domain of ‘physical function’, researchers focus on basic functions of body, such as appetite, sleep, mobility, daily activities, self-care, vision, hearing, body constitution, physical exercise, recreational activities, and heavy physical work. In the domain of ‘physical senses’, attention is given to discomfort, pain, and disease, reflected in facets such as pain, fatigue or tiredness, energy or vitality, body weight, dependence on medication, discomfort, complexion, body strength, and family medical history.

Mental health

'Mental health' consists of three domains: emotional experiences, mind-frame, and cognitive functions. In the domain of ‘emotional experiences’, common facets include depression, stress, anxiety, regularity in daily life, sex life, loneliness, worry, anger, fear, happy mood, no worry, and no pressure. The ‘mind-frame’ domain encompasses facets like self-confidence, morality, positive attitude, peace, breadth of mind, and sense of life satisfaction. Lastly, the ‘cognitive function’ domain includes facets such as memory, concentration ability, sharp mind, clear mind, decision-making ability, and thinking ability.

Social health

'Social health' can be divided into three domains: interpersonal relationships, work or school, and economic condition. ‘Interpersonal relationships’ emphasize the ability to communication with others and the adaptability to the environment. It includes the following facets: social interactions, good relationships and communications with others (e.g. friends, family and colleagues), social support, adaptability to social environment and burdens to others. The ‘work or school’ domain concerns task completion and performance in studies or job. Additionally, ‘economic conditions’ are also a crucial component of overall mental health.

Environment

Apart from the three fundamental WHO health concepts, namely physical, mental, and social health, the concept of 'environment' has also been recognized as an important factor in the included studies [ 18 , 21 , 22 , 23 , 38 ]. This domain encompasses the ability to adapt to changes in weather and living conditions.

Overlap between TCM and MM

Both TCM and MM break down their concepts of HRQoL into 'domains', as depicted in Figs.  3 and 4 . These domains demonstrate significant overlap between TCM and MM perspectives. They encompass intrapersonal (such as physical and emotional aspects) and interpersonal domains (such as relationships and work).

When these domains are operationalized into survey questions, the resulting 'facets' further reveal the similarities between TCM and MM. Figure  5 illustrates that many facets share common wording. Among the 59 facets identified, 28 were addressed in both TCM and MM, 9 were specific to TCM, and 22 were specific to MM. Among the shared facets, we found that 'appetite', 'sleep' and 'energy' were the most frequently mentioned facets in both frameworks.

Intersection diagram of the facets of TCM and the MM frameworks

This paper aims to explore and synthesize perceptions of HRQoL within a Chinese cultural setting. Our systematic review of relevant research reveals that two distinct perspectives existed in defining the conceptual frameworks of HRQoL in China: TCM and MM. To provide a comprehensive overview of these perspectives, we summarized the conceptual frameworks of both perspectives.

Firstly, it is challenging to define HRQoL. Mayo et al. proposed distinguishing between QoL and HRQoL [ 43 ]. However, Chinese studies often use the WHO's definition of 'health' instead of Mayo's HRQoL definition. The Chinese studies used the WHO’s definition of 'health' for defining HRQoL and also used the WHO definition of 'health' for the development of HRQoL instruments [ 10 , 18 , 19 ]. Meanwhile, examining HRQoL from TCM perspective reveals a focus on broader QoL aspects. TCM's emphasis on 'a balance between oneself and the environment' relates more to QoL than the narrow 'impact of disease and treatment' in the HRQoL definition from Mayo.

The two frameworks exhibit certain differences. On the level of concepts, TCM takes a holistic view, emphasizing the interconnectedness and interaction between body and mind, while MM does not. TCM framework uses terms like 'unity' and 'harmony' to underscore this holistic perspective and interaction, considering the inseparability of body and spirit [ 34 ]. By contrast, MM seems to be influenced by the theory of 'mind–body dualism,' separating the mind and body. According to the MM framework, individuals consist of two separate substances: body and mind, which can be described independently. As a result of these fundamental differences, the descriptions of HRQoL from these two frameworks consistently differ. Notably, though the HRQoL conceptual framework of TCM emphasizes the unity of health, instruments developed based on this conceptual framework often do not fully capture this unity. For instance, the Chinese Medicine Quality of Life-11 Dimensions (CQ-11D) emphasizes the unity of body and spirit but it still measures ‘body and ‘mind’ as two independent dimensions [ 33 ]. Therefore, we have observed that the idea of 'holistic' is difficult to embody in TCM instruments. This can be seen in Fig.  3 .

Differences in specific facets describing HRQoL are also evident (Fig.  5 ). The TCM framework incorporates unique diagnostic indicators, many of which are specific to TCM-related 'symptoms' such as 'defecation' and 'spirit of eyes'. In contrast, MM focuses on measurable physical phenomena like 'vision' and 'weight'. Our analysis highlights two main reasons for these differences. Firstly, TCM places emphasis on 'image thinking' [ 44 , 45 ], which involves intuitively grasping the abstract meaning of the world and its universal connections through intuition, metaphor, symbol, association, and analogy [ 45 , 46 , 47 ]. TCM employs various 'images,' such as tongue image, pulse image, and syndrome image [ 47 ]. Based on this idea, TCM generally collects patients' symptoms through ‘making observations, listening to breathing, asking about symptoms and taking the pulse (the four fundamental methods for diagnosis in TCM)’ for analysis and syndrome differentiation. However, MM focuses on analysing pathological mechanisms using objective and measurable examination results [ 48 ]. Secondly, the discrepancy in specific facets is related to a well-accepted concept known as sub-health, which refers to a state between illness and health [ 49 , 50 ]. Sub-health is characterized by experiencing different types of symptoms, both physical (e.g., pain, discomfort, fatigue) and mental (e.g., negative emotions, poor memory, inattention), for more than three months without any clear clinical attribution [ 51 , 52 ]. In MM, there is no specific diagnose for such health status specific diseases when individuals experience prolonged fatigue despite normal clinical indicators. However, TCM theories have significantly contributed to the widespread recognition and acceptance of the sub-health concept.

Despite some differences, these two conceptual frameworks also share similarities. As mentioned before, while each framework has its unique facets, there is a significant overlap between them, as shown in the middle section of Fig.  5 . This overlap may be inherent to TCM and MM, but can also be the result ‘borrowing’ aspect of TCM into MM and vice versa. The integration of these two frameworks can be observed in Fig.  5 . MM also acknowledges the interconnectedness and interaction of body and mind when considering HRQoL. For instance, in the MM framework, there are facets that resemble TCM facets, such as 'complexion', 'energy', and 'constitution'. Similarly, in the TCM framework, facets emphasized by MM seem to be borrowed, like 'self-care' and 'mobility'. Among the shared facets, we found that 'appetite', 'sleep' and 'energy' were the most frequently mentioned facets in both frameworks. These can be considered vital elements in how people in China define HRQoL. The importance of 'appetite' can be attributed to various factors, including China’s historical experiences with varying food supply and the social aspect of eating [ 53 ]. For example, sharing meals serves as a common way for people to establish and express connections with one another in China. Additionally, 'sleep' was also recognized as an essential facet in most HRQoL instruments within the Chinese cultural setting. TCM regards 'sleep' as crucial for well-being and preventing illness [ 54 , 55 , 56 ]. Compared to TCM, MM only recently acknowledges that sleep is vital for cognitive function, emotion, memory, endocrine balance, and immunity [ 57 ]. Furthermore, 'energy' was also frequently mentioned and covered by the two frameworks. In Chinese culture, 'energy' is closely related to the concept of 'Qi' (also spelled chi) in TCM [ 58 , 59 ]. Qi is believed to exist in all things, including air, water, food, and sunlight, and is often translated as 'vital energy'—a fundamental substance that builds and sustains the body [ 60 ]. Furthermore, it is important to recognize that concepts from two types of frameworks are divided into 'domains', and these domains exhibit considerable overlap (Figs.  3 , 4 ). These figures demonstrate that both frameworks describe HRQoL from four aspects: physical, mental, social, and adaptability to the natural environment. Although the theoretical basis and classification of the two frameworks may differ, the content is quite similar.

Although there are many overlapping facets between two frameworks, we still chose to describe HRQoL using two separate frameworks rather than combining them into one in this article. There are three main reasons: firstly, the original intention was to provide a comprehensive understanding of HRQoL in Chinese cultural. After a systematic literature review and content classification, we found that presenting two conceptual frameworks from different perspectives better represents the definition of HRQoL in China. Secondly, both frameworks have unique facets that cannot be merged. TCM includes facets such as 'urinate', 'stool', and 'spirit of eye', whereas the MM-based framework includes 'body weight', 'dependence on medication', and 'burdens to others'. Thirdly, even for overlapping facets, they have different classifications in each framework due to distinct theories. For example, 'emotions' holds different positions in the two frameworks. In TCM, it is considered a 'concept' contributing to overall health, ranked alongside 'the unity of form and spirit' and 'the unity of man and nature'. In MM, 'emotion' is a domain under the concept of 'mental health'. This divergence arises from MM viewing emotions as only a component of mental health, while TCM recognizes that emotions not only affect an individual's mental state but also have specific relationships with different organs in the body. As a result, TCM emphasizes the importance of maintaining emotional stability and avoiding extreme emotions for good health, leading us to list it separately in the TCM framework.

However, it would be beneficial to integrate these frameworks in future research. This integration aligns with the strategy of 'integrating Chinese and modern medicine', which is widely practiced in clinical practice in China. Integrative medicine, with its efficacy and complementary advantages, has emerged as one of the major medical systems alongside MM and TCM [ 61 ]. In Chinese clinical guidelines, it's common to find a combination of conventional disease diagnoses and traditional syndromes. While there are globally recognized standards for diagnosing diseases (such as the International Classification of Diseases, ICD-10), there's also an emphasis on identifying syndromes that reflect traditional Chinese characteristics [ 61 ]. Supported by the government policy, 'integrating Chinese and modern medicine' will continuously be the trend and our results also support this conclusion. It is evident that in the Chinese context, the definition of HRQoL cannot be adequately captured from a single perspective alone. Therefore, future research should aim to integrate the two HRQoL frameworks to provide a more comprehensive understanding.

The conclusion evokes questions whether we need a specific Chinese HRQoL instrument or if existing ones are sufficient. While instruments focused on 'Integrating Chinese and Modern medicine' naturally differ from Western-developed ones, it's important to assess if they truly benefit Chinese researchers internationally. We need to consider if a 'real Chinese HRQoL instrument' would be valuable outside of China, which could potentially devalue research from China and contradict the findings of the present study. Despite China having two well-developed HRQoL concepts, one possibly unique to China, they seem quite similar in terms of their main ideas.

As mentioned in the methods section, a limitation is the absence of a quality evaluation for the included articles. The second limitation is the inherent subjectively in classifying the concepts, domains and facets. In order to address this, we will employ 'concept mapping' in the future to provide a more objective exploration of the overlap in dimensions and facets between MM and TCM. The third limitation is that our research only included the general population and generic questionnaires. Patient-Reported Outcomes (PROs) are often diseases specific, such as cancer [ 62 ], but we were limited in the use of them due to the variance in patient groups, which would in turn impact how HRQoL is defined. However, this does not imply that PROs cannot be investigated. One potential solution could be to categorize specific disease groups. For instance, several authors have examined the applications and characteristics of PRO instruments as primary and secondary outcomes in randomized clinical trials in China [ 63 ], as well as the use of PROs in clinical trials of TCM [ 64 ] to promote and standardize PROs in China.

The present study is the first systematic review summarizing the conception of HRQoL in the Chinese cultural setting. We found two distinct frameworks of HRQoL exist in China, each basing on its own unique theory: MM and TCM, but arriving at some similar concepts, domains and facets.

Data availability

The data supporting the findings of this study are accessible in both the article and its supplementary material.

Shumaker, S. A., & Berzon, R. A. (1995). The international assessment of health-related quality of life: Theory, translation, measurement and analysis.

Bowling, A. (1995). Measuring disease: A review of disease-specific quality of life measurement scales (second edition). Quality of Life Research, 12 , 1147–1148.

Google Scholar  

Fayers, P. M., & Machin, D. (2007). Quality of life: The assessment, analysis and interpretation of patient-reported outcomes.

Costa, D. S. J., Mercieca-Bebber, R., Rutherford, C., Tait, M. A., & King, M. T. (2021). How is quality of life defined and assessed in published research? Quality of Life Research, 30 (8), 2109–2121.

Article   PubMed   Google Scholar  

Fitzpatrick, R., Fletcher, A., Gore, S., Jones, D., Spiegelhalter, D., & Cox, D. (1992). Quality of life measures in health care. I: Applications and issues in assessment. BMJ, 305 (6861), 1074–1077.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Wilson, I. B., & Cleary, P. D. (1995). Linking clinical variables with health-related quality of life. A conceptual model of patient outcomes. JAMA, 273 (1), 59–65.

Article   CAS   PubMed   Google Scholar  

Yao, Q., Liu, C., Zhang, Y., & Xu, L. (2019). Changes in health-related quality of life of Chinese populations measured by the EQ-5D-3 L: A comparison of the 2008 and 2013 National Health Services Surveys. Health and Quality of Life Outcomes, 17 (1), 43.

Article   PubMed   PubMed Central   Google Scholar  

Mao, Z., Ahmed, S., Graham, C., Kind, P., Sun, Y. N., & Yu, C. H. (2021). Similarities and differences in health-related quality-of-life concepts between the East and the West: A qualitative analysis of the content of health-related quality-of-life measures. Value Health Reg Issues, 24 , 96–106.

Karimi, M., & Brazier, J. (2016). Health, health-related quality of life, and quality of life: What is the difference? PharmacoEconomics, 34 (7), 645–649.

Li, M., Bao, Z., Lv, G., Zhou, J., Chen, P., & Luo, N. (2021). Culture-related health disparities in quality of life: Assessment of instrument dimensions among Chinese. Frontiers in Public Health, 9 , 663904.

Mao, Z., Ahmed, S., Graham, C., & Kind, P. (2020). Exploring subjective constructions of health in China: A Q-methodological investigation. Health and Quality of Life Outcomes, 18 (1), 165.

Cheung, Y. B., & Thumboo, J. (2006). Developing health-related quality-of-life instruments for use in Asia: The issues. PharmacoEconomics, 24 (7), 643–650.

Prior, L., Chun, P. L., & Huat, S. B. (2000). Beliefs and accounts of illness. Views from two Cantonese-speaking communities in England. Sociology of Health & Illness, 22 (6), 815–839.

Article   Google Scholar  

Yang, F., Jiang, S., He, X. N., Li, H. C., Wu, H. Y., Zhang, T. T., & Wu, J. (2021). Do rural residents in China understand EQ-5D-5L as intended? Evidence from a qualitative study. Pharmacoecon Open, 5 (1), 101–109.

Xiang, Y. T., Chiu, H. F., & Ungvari, G. S. (2010). Quality of life and mental health in Chinese culture. Current Opinion in Psychiatry, 23 (1), 43–47.

Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., Stewart, L. A., & Group, P.-P. (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic Reviews, 4 (1), 1.

Leung, K. F., Liu, F. B., Zhao, L., Fang, J. Q., Chan, K., & Lin, L. Z. (2005). Development and validation of the Chinese quality of life instrument. Health and Quality of Life Outcomes, 3 , 26.

Zhu, D. (1999). Confucius’ views on humanity and human health. Yun nan jiao yu xue yuan xue bao, 06 , 16–20.

Wang, J., & Tang, Y. (2011). A study on the concept of health in traditional Chinese medicine. Zhong yi za zhi, 52 (12), 995–997.

Ke, J., Zeng, T., Zhou, X., Yan, Q., Xiao, M., Wang, Y., & Sun, Y. (2019). Constructing the conceptual framework of health in traditional Chinese medicine culture. Zhong hua hu li jiao yu, 16 (05), 389–392.

Yu, Y., Du, J., Xue, C., & Yang, Q. (2011). The traditional chinese medicine concept of health in the Huangdi Neijing. Zhong guo zhong yi ji chu yi xue za zhi, 17 (02), 147–148.

Zhang, H., Zhang, F., Zhao, Y., & Li, Z. (2015). The concept of health from the perspective of Chinese culture. Zhong hua hu li za zhi, 50 (10), 1236–1239.

Wu, Y., Xie, G., Li, Y., Zhou, B., Zhang, P., Shi, P., Ren, F., & Ma, L. (2005). Development and evaluation of a general quality of life scale for Chinese people. Zhong hua liu xing bing xue za zhi, 10 , 27–32.

Zhu, F. (2012). Exploring the meaning of mind-body health from the perspective of Yin-Yang theory. Jiang su zhong yi yao, 44 (09), 3–5.

Li, X. (2007). Study and evaluation of PRO (patient-reported outcomes) properties of the Chinese health-related quality of life scale. Guang zhou zhong yi yao da xue .

Ni, H., He, Y., Wu, Y., Xu, M., & Guo, Y. (2014). Exploring the evolution of health concepts in Eastern and Western traditions: Implications and insights. Nan jing zhong yi yao da xue xue bao, 15 (02), 79–83.

Zhao, L., Liu, F., Liang, G., Fang, J., Lin, L., & Chen, J. (2004). Development and exploration of the theoretical structure model of the Chinese health-related quality of life scale. Zhong guo lin chuang kang fu, 16 , 3132–3134.

Liu, F., Zhao, L., Liang, G., & Lin, L. (2007). Theoretical and structural model exploration of traditional Chinese medicine health status scale. Xin zhong yao, 09 , 10–12.

CAS   Google Scholar  

Zhao, L., & Chen, J. (2003). The concept of traditional Chinese medicine’s approach to health. Yi xue yu zhe xue, 12 , 58–59.

Wu, N. (2016). The "Five-character Formula" of the health connotation in Huangdi Neijing, Shi jie zhong yi yao xue hui lian he hui zhong yi zhi wei bing zhuan ye wei yuan hui shou jie guo ji zhong yi zhi wei bing xue shuda hui lun wen ji (pp. 83–86). Nanjing.

Wang, P. (2021). An exploration of the health standards in the Huangdi Neijing. Zhong hua zhong yi yao za zhi, 36 (1), 17–21.

Wang, Q. (2012). A study on the concept and connotation of health status in traditional Chinese medicine. Zhong hua jian kang guan li xue za zhi, 6 (6), 419–420.

Zhu, W., Gao, H., Zhang, M., Shi, L., Tao, L., Chen, Y., Shi, L., Yu, C., Han, S, Li, S., & Wei J. (2022). The development of the traditional Chinese Medicine Quality of Life Assessment Scale (CQ-11D). Zhong guo yao wu jing ji xue, 17 (05), 10–15+39.

Chirico, F. (2016). Spiritual well-being in the 21st century: It’s time to review the current WHO’s health definition. Journal of Health and Social Sciences, 1 (1), 11–16.

Song, T., Shen, H., Ni, H., & He, Y. (2014). A philological study and exegesis of the etymology of health. Zhong hua zhong yi yao xue kan, 32 (06), 1299–1301.

Liu, F., Zhao, L., Lang, J., Lin, L., Liang, G., & Fang J. (2006). Development of the Chinese health-related quality of life scale (CH-QOL). 2006 Guang dong sheng zhong yi, zhong xi yi jie he pi wei xiao hua bing xue shu hui yi ji guo jia ji ji xu jiao yu xiang mu xiao hua bing jin zhan yan tao ban lun wen hui bian (pp. 293–298). Guang Zhou.

Lang, J. (2005). Development, evaluation, and initial application of the Chinese Health-Related Quality of Life Scale (CH-QOL). Zhong shan da xue.

Wu, D., & Lai, S. (2007). Discussion on the concept of health in traditional Chinese medicine and its operationalization in measurement. Zhong guo zhong xi yi jie he za zhi, 02 , 174–177.

Du, Q. (2014). Reflections on the relationship between Yin and Yang in the ’Yellow Emperor’s inner canon: Discussion on the correspondence of Yin and Yang. Ya tai chuan tong yi yao, 10 (09), 2–3.

Fan, P. (2019). Exploring the health thoughts in chinese traditional culture: A textual analysis of the Qingjing Jing. Lao zi xue kan, 01 , 79–87.

Ni, W., Zhang, W., & Ji, X. (1994). Modern Chinese dictionary.

Grad, F. P. (2002). The preamble of the constitution of the World Health Organization. Bulletin of the World Health Organization, 80 (12), 981–984.

PubMed   Google Scholar  

Mayo, N. E., Figueiredo, S., Ahmed, S., & Bartlett, S. J. (2017). Montreal accord on patient-reported outcomes (PROs) use series—Paper 2: Terminology proposed to measure what matters in health. Journal of Clinical Epidemiology, 89 , 119–124.

An, D., Li, L., Mao, H., & Liang, Y. (2020). A comparative study of traditional Chinese medicine’s “Xiang” and Western medicine’s ’phenotype. Zhong yi yao xin xi, 37 (05), 24–26.

Chen, M., & Li, X. (2022). The cultivation of traditional chinese medicine thinking in clinical medical students in Western Medical Colleges and Universities. Zhong guo zhong yi yao xian dai yuan cheng jiao yu, 20 (22), 15–17.

Xu, Y., Jiang, L., & Yuan, H. (2018). Clinical applications and examples of symbolic and numeric thinking. Zhong hua zhong yi yao za zhi, 33 (07), 2926–2929.

Xing, Y. (2014). A brief analysis of the concept of symbolic thinking in traditional Chinese medicine. Zhong yi za zhi, 55 (15), 1347–1348.

Yang, T., Wang, D., Feng, M., Huang, Z., Duan, R., Bu, P., Guo, F., & Wen, C. (2023). A study on the normalization and hierarchical processing framework for multimodal sensor data based on traditional Chinese medicine four diagnosis, western medical tests, and bioinformatics. Shi jie ke xue ji shu-zhong yi yao xian dai hua, 25 (02), 798–805.

Zhang, J., Wang, D., He, Q., Sun, G., Xiao, D., Zeng, X., & Mao, X. (2021). Exploring the content and extension of sub-health science in traditional Chinese medicine. Zhong hua zhong yi yao za zhi, 36 (07), 3777–3781.

Sun, T., He, Q., Zhu, R., Wang, T., Fan, X., Wei, Y., Xu, R., He, L., Zhu, Y., & Ma, W. (2022). Guidelines for the classification of sub-health states in traditional Chinese medicine. Shi jie zhong xi yi jie he za zhi, 17 (09), 1891–1893+1898.

Yan, Y. X., Liu, Y. Q., Li, M., Hu, P. F., Guo, A. M., Yang, X. H., Qiu, J. J., Yang, S. S., Shen, J., Zhang, L. P., & Wang, W. (2009). Development and evaluation of a questionnaire for measuring suboptimal health status in urban Chinese. Journal of Epidemiology, 19 (6), 333–341.

Liu, B., He, L., Xie, Y., & Hu, J. (2004). Development of a questionnaire for investigating the fundamental traditional Chinese medicine syndrome characteristics in sub-health states. Zhong guo zhong yi ji chu yi xue za zhi, 09 , 23–28.

Tian, R. G., Tian, K., Zhao, D. D., & Wang, C. H. (2018). Food culture in China: From social political perspectives. Trames-Journal of the Humanities and Social Sciences, 22 (4), 345–364.

Wu, J. F., Hu, R., Li, M., Liu, S. S., Zhang, X. Z., He, J., Chen, J. X., & Li, X. J. (2021). Diagnosis of sleep disorders in traditional Chinese medicine based on adaptive neuro-fuzzy inference system. Biomedical Signal Processing and Control, 70 .

Wang, Z. (2008). Clinical experience in treating insomnia with a five elements approach focusing on the liver. Shi zhen guo yi yi yao, 19 (5).

Wang, R., & Jin, R. (2006). The relationship between traditional Chinese medicine and the liver in insomnia. Paper presented at the Di si jie quan guo zhong yi shui mian yi xue xue shu yan tao hui, Xian.

Sun, Y., Wang, H., Jin, T., Qiu, F., & Wang, X. (2022). Prevalence of sleep problems among Chinese medical students: A systematic review and meta-analysis. Frontiers in Psychiatry, 13 , 753419.

Liu, T. (2018). The scientific hypothesis of an “energy system” in the human body. Journal of Traditional Chinese Medical Sciences, 5 , 29–34.

Ou, B., Huang, D., Hampsch-Woodill, M., & Flanagan, J. A. (2003). When east meets west: The relationship between yin-yang and antioxidation-oxidation. The FASEB Journal, 17 (2), 127–129.

Xiao, D., Li, S., Zhu, Q., Fang, H., & Pan, H. (2016). Energy has the same characteristics of Qi in traditional Chinese medicine. World Journal of Integrated Traditional and Western Medicine, 2 (02), 14–18.

Xu, H., Shi, D., Liu, B., & Chen, K. (2009). A proposal for building integrative clinical guidelines for traditional Chinese and western medicine based on clinical practice data. Zhong guo zhong xi yi jie he za zhi, 29 (06), 544–547.

Osoba, D. (2011). Health-related quality of life and cancer clinical trials. Therapeutic Advances in Medical Oncology, 3 (2), 57–71.

Zhou, H., Yao, M., Gu, X., Liu, M., Zeng, R., Li, Q., Chen, T., He, W., Chen, X., & Yuan, G. (2022). Application of patient-reported outcome measurements in clinical trials in China. JAMA Network Open, 5 (5), e2211644.

Dong, Y., Liu, L., Zhang, X., Gong, Y., Yan, S., Li, W., Li, S., Rong, H., & Liu, J. (2023). A cross-sectional study on the application of patient-reported outcome measurements in clinical trials of traditional Chinese medicine in mainland China. Frontiers in Pharmacology, 14 , 1159906.

Download references

Open Access funding enabled and organized by Erasmus Medical Center. This work was funded by EuroQol Foundation, the Netherlands. The study sponsor had no role in the design of the study, data collection, data management, data analysis, data interpretation, and issues regarding the publication of results. Views expressed by the authors in the publication do not necessarily reflect the views of the EuroQol Foundation.

Author information

Authors and affiliations.

Department of Psychiatry, Erasmus MC, University Medical Center Rotterdam, Rotterdam, The Netherlands

Yifan Ding & Jan Busschbach

Centre for Health Economics Research and Modelling Infectious Diseases, Vaccine and Infectious Disease Institute, University of Antwerp, Antwerp, Belgium

Saw Swee Hock School of Public Health, National University of Singapore, Singapore, Singapore

Health Services Management Department, Guizhou Medical University, Guiyang, China

Zhihao Yang

You can also search for this author in PubMed   Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation was performed by Zhuxin Mao, Zhihao Yang and Nan Luo. Data collection and analysis were performed by Yifan Ding. The first draft of the manuscript was written by Yifan Ding and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Yifan Ding .

Ethics declarations

Conflict of interest.

The authors have no relevant financial or non-financial interests to disclose.

Ethical approval

Not applicable.

Consent to participate

Consent for publication, additional information, publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 19 KB)

Supplementary file2 (xlsx 23 kb), supplementary file3 (xlsx 26 kb), supplementary file4 (xlsx 15 kb), supplementary file5 (docx 187 kb), supplementary file6 (docx 33 kb), supplementary file7 (docx 19 kb), rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Ding, Y., Mao, Z., Luo, N. et al. Differences and common ground in the frameworks of health-related quality of life in traditional Chinese medicine and modern medicine: a systematic review. Qual Life Res (2024). https://doi.org/10.1007/s11136-024-03669-1

Download citation

Accepted : 21 April 2024

Published : 13 May 2024

DOI : https://doi.org/10.1007/s11136-024-03669-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Health-related quality of life
• Traditional Chinese medicine
• Modern medicine
• Find a journal
• Publish with us
• Track your research

• Open access
• Published: 05 May 2024

The learning curve in endoscopic transsphenoidal skull-base surgery: a systematic review

• Abdulraheem Alomari 1 ,
• Mazin Alsarraj 2 &
• Sarah Alqarni 3  

BMC Surgery volume  24 , Article number:  135 ( 2024 ) Cite this article

The endoscopic endonasal transsphenoidal approach (EETA) has revolutionized skull-base surgery; however, it is associated with a steep learning curve (LC), necessitating additional attention from surgeons to ensure patient safety and surgical efficacy. The current literature is constrained by the small sample sizes of studies and their observational nature. This systematic review aims to evaluate the literature and identify strengths and weaknesses related to the assessment of EETA-LC.

A systematic review was conducted following the PRISMA guidelines. PubMed and Google Scholar were searched for clinical studies on EETA-LC using detailed search strategies, including pertinent keywords and Medical Subject Headings. The selection criteria included studies comparing the outcomes of skull-base surgeries involving pure EETA in the early and late stages of surgeons’ experience, studies that assessed the learning curve of at least one surgical parameter, and articles published in English.

The systematic review identified 34 studies encompassing 5,648 patients published between 2002 and 2022, focusing on the EETA learning curve. Most studies were retrospective cohort designs (88%). Various patient assortment methods were noted, including group-based and case-based analyses. Statistical analyses included descriptive and comparative methods, along with regression analyses and curve modeling techniques. Pituitary adenoma (PA) being the most studied pathology (82%). Among the evaluated variables, improvements in outcomes across variables like EC, OT, postoperative CSF leak, and GTR. Overcoming the initial EETA learning curve was associated with sustained outcome improvements, with a median estimated case requirement of 32, ranging from 9 to 120 cases. These findings underscore the complexity of EETA-LC assessment and the importance of sustained outcome improvement as a marker of proficiency.

Conclusions

The review highlights the complexity of assessing the learning curve in EETA and underscores the need for standardized reporting and prospective studies to enhance the reliability of findings and guide clinical practice effectively.

Peer Review reports

With the advent of endoscopic techniques, skull-base surgery has significantly advanced. The modern history of neuro-endoscopy began in the early 1900s with an innovation by Lespinasse and Dandy, involving intraventricular endoscopy to coagulate the choroid plexus for treating communicating hydrocephalus [ 1 ]. In 1963, Guiot first reported an endoscopic approach via the transsphenoidal route as an adjunct to procedures performed under microscopy [ 2 , 3 ]. In 1992, Jankowski et al. described a purely endoscopic approach for pituitary adenoma resection [ 1 ].

The advantages of endoscopy have encouraged skull-base surgeons to adopt this technique, which provides a panoramic view of critical anatomical landmarks and improved access to the corners and deep surgical areas while inducing only minor trauma to the nasal structures, thereby enhancing postoperative patient comfort [ 4 ]. Compared with procedures involving microscopy, the endoscopic approach results in a shorter operating time (OT), a reduced hospitalization period, a lower rate of complications, and a higher endocrinological cure rate [ 5 , 6 ]. Despite these benefits, the endoscopic approach is hindered by a two-dimensional view, instrument interference, difficulties in achieving homeostasis, and a steep learning curve (LC) [ 4 ].

Since its inception, pioneers in the field have recognized the steep LC associated with the endoscopic technique [ 7 ]. The safety and efficacy of the endoscopic endonasal transsphenoidal approach (EETA), as an alternative to the gold-standard microscopic technique, have been established. However, the steep LC associated with the endoscopic approach may affect short-term outcomes post-procedure [ 5 , 6 ]. Additionally, as the skull-base endoscopic technique constantly evolves and expands, a thorough understanding of the associated LC is critical.

The results of existing publications on the EETA-LC are challenging to interpret due to small sample sizes, observational study designs, and a lack of standardization in assessment methodologies. In this systematic review aims to elucidate the EETA-LC from the literature by addressing the following questions: How was EETA LC evaluated? Which set of variables was used to assess the LC? What is the influence of the LC on the examined variables?

A systematic review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines [ 8 ]. The review was registered on PROSPERO (CRD42023494731). We searched different databases for articles that assessed the learning curve of EETA without date restriction (PubMed, and Google Scholar). We used a particular equation for each database using a combination of the following keywords and Medical Subject Headings: (Endoscopy OR endoscopic skull base OR endoscopic endonasal transsphenoidal approach) AND (Skull Base Neoplasms OR Pituitary OR pituitary adenoma) AND (Learning Curve OR endoscopic learning curve OR surgical learning curve).

First, two authors (AA, MA) independently screened the titles and abstracts of articles in the databases for learning curve analysis of EETA, either for a single surgeon or a team, by directly comparing outcomes between early and late cases performed. The full texts of the relevant articles were reviewed. When there was a disagreement, the articles were thoroughly discussed before their inclusion in the review. The bibliographies of the selected studies were also screened for relevant citations, which turned up studies that were already selected from the database search.

Studies were included according to the following inclusion criteria: 1) Comparison of outcomes between initial and advanced experiences with the endonasal endoscopic transsphenoidal approach to treat skull-base pathology, defined as "early experience" and "late experience," respectively; 2) Assessment of at least one parameter based on early and late experiences; 3) Randomized controlled trials, prospective cohort studies, retrospective cohort studies, case–control studies, and case series studies were included; and 4) English-language publications.

The study’s exclusion criteria included the following: 1) Studies not performing learning curve analysis; 2) Studies comparing the outcomes of microscopic and endoscopic transsphenoidal approaches without providing separate data for the endoscopic approach; 3) Studies comparing the learning curve between two EETA techniques, using simulated models or questionnaire-based analysis; 4) Studies comparing the microscopic vs. endoscopic approach without separate data available specifically for the endoscopic arm. Additionally, case reports, reviews, animal studies, technical notes, comments, and correspondence were excluded.

Data collection and analysis

The following data were extracted directly from the articles: 1) author names; 2) the year of publication; 3) Time interval of performed procedures; 4) study design; 5) the sample size; 6) techniques used for learning curve analysis (methods used to assort the patients for the analysis); (conducting statistical analysis vs. simple comparison of outcomes); 7) the sample size in each study arm when group splitting performed (early experience vs. late experience); 8) detailed information about surgeon experience at the time of LC assessment (including or omitting the first few EETA cases); 9) single vs. multiple pathologies; 10) team vs. single-surgeon experiences; 11) evaluated set of variables; 12) Variables that improved with experience; and 13) the number of cases required to overcome the initial LC or other methods to identify overcoming the learning curve.

Study quality assessment and risk of bias

Two reviewers conducted a quality assessment and evaluated the risk of bias in the included articles. We utilized the Newcastle–Ottawa Scale (NOS) [ 9 ] and the GRADE system [ 10 ].

Heterogeneity Analysis: Due to substantial heterogeneity observed among the included studies, which encompassed variations in study design, included pathologies, and outcome measures, a formal meta-analysis was not feasible. Therefore, we opted for a qualitative synthesis instead of a formal meta-analysis. Heterogeneity analysis and sensitivity analyses were not explicitly conducted.

Based on the inclusion and exclusion criteria, a total of 34 studies were identified (6 articles excluded after reviewing the full articles), including 5,648 patients [ 7 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 ] (Fig.  1 ). The included studies were published between 2002 and 2022, and the evaluated procedures were performed between 1990 and 2018. The majority of the included articles comprised retrospective cohort studies (88%), with two being prospective studies, and two articles presenting data from both prospective and retrospective study designs. Assessing a surgical learning curve involves various methods and techniques documented within the included articles. We observed various methods for patient assortment in conducting learning curve analyses across the literature, with group-based learning curve analysis noticeable in a significant proportion of articles (68%). Within these studies, there was an unclear rationale behind patient grouping. Nonetheless, patients were categorized into either equal group, segmented based on arbitrary time periods, or separated based on improvements in outcomes observed retrospectively after data analysis. Eleven articles (32%) utilize case-based analysis, where individual surgical cases serve as distinct data points, and their outcomes are monitored over time.

PRISMA flow diagram. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses

* The bibliographies of the selected studies were also screened for relevant citations which turned up studies already included from databases search

Our systematic review encompasses a wide range of statistical tests employed in the included studies to analyze various data types and address multifaceted research inquiries. The primary statistical methodologies utilized encompass descriptive statistical analysis, which includes metrics such as mean, median, frequency, and standard deviation, along with comparative statistical analysis, which includes techniques such as Chi-square analysis, analysis of variance (ANOVA), and t-tests. Descriptive statistical analysis alone was evident in 10 articles (29%), whereas comparative statistical analysis was present in 24 articles (71%). Noteworthy examples include Leach et al. [ 16 ], who conducted analysis of variance (ANOVA) with post hoc Bonferroni tests for parametric data, Chi-Square Test, or Mann–Whitney tests for nonparametric data, and regression analysis to explore the relationship between surgical duration and relevant factors. Smeth et al. [ 17 ] undertook analyses using chi-square, Fisher exact, Student t-test, Mann–Whitney U test, and analysis of variance, aligning with their examination of categorical and continuous variables across distinct groups. Similarly, Sonnenburg et al. [ 12 ] applied a one-way ANOVA to discern variations between groups, highlighting the importance of understanding differences in means across categorical variables or treatment cohorts.

Regression analyses, scatterplots, McNemar tests, ROC curve analysis, and logistic regression models were integral across various studies, serving multiple purposes. Regression analyses, such as linear regression models, facilitated the exploration of intricate relationships among variables like age, tumor size, and surgical duration, identifying potential risk factors in surgical contexts [ 22 ]. Scatterplots visually depicted these relationships, offering intuitive insights into temporal variations, notably in the examination of surgery date versus duration [ 22 ]. McNemar tests were instrumental in evaluating changes in hormone levels, crucial for understanding postoperative outcomes and hormonal dynamics [ 37 ]. Additionally, ROC curve analysis provided a robust method for determining the level of surgical experience necessary to achieve gross total resection (GTR), offering actionable insights into surgical proficiency and patient outcomes [ 37 ]. Binary logistic regression models were utilized to identify prognostic factors contributing to the attainment of Gross Total Resection (GTR), hormonal recuperation, and visual restoration. For instance, variables such as surgical experience (≤ 100 vs. > 100 cases) were examined within this analytical framework [ 37 ].

In our examination of the included articles, we noted a lack of thorough description regarding the experience of surgeons or surgical teams with the endoscopic endonasal transsphenoidal approach (EETA), the extent of the approach undertaken, and the level of involvement of individual surgeons or surgical teams during procedures. Thirteen articles (38%) reported including the initial cases of EETA, which may indicate a lack of prior experience with the approach. Additionally, seven articles (21%) detailed the experience of a single surgeon, while the majority (79%) evaluated team experiences. There was a wide range of pathologies included in all the studies. Twenty articles (59%) focused on a single pathology, while fourteen studies (41%) examined multiple pathologies. Pituitary adenoma (PA) was the most frequently reported pathology (82%), followed by craniopharyngioma (CP) (44%). Three studies assessed the learning curve of cerebrospinal fluid (CSF) leak repair following treatment of multiple pathologies. Descriptions of the surgical approach, particularly distinguishing between simple and extended techniques, were notably lacking across all articles. However, seventeen articles (50%) did mention pathologies that often require an extended approach, such as meningioma, chordoma, and CP. A number of studies have investigated the variations in tumor type and size among the examined groups, particularly between early and late groups. Notably, findings from studies such as [ 7 , 16 , 17 , 22 , 23 , 26 , 38 ] indicated that no statistical differences were observed between these groups. The characteristics of the included studies [ 7 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 ] are summarized in Table  1 .

The EETA-LC was evaluated based on a diverse set of variables. The most frequently analyzed variables were postoperative cerebrospinal fluid (CSF) leak in 28 articles (82%) [ 7 , 12 , 13 , 15 , 16 , 17 , 19 , 20 , 21 , 22 , 23 , 25 , 27 , 28 , 29 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 ], gross total resection (GTR) in 21 articles (62%) [ 7 , 13 , 14 , 16 , 19 , 21 , 22 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 36 , 37 , 38 , 39 , 40 ], post operative diabetes insipidus (DI) in 15 articles (44%) [ 12 , 13 , 16 , 17 , 19 , 21 , 22 , 29 , 30 , 31 , 32 , 34 , 36 , 37 , 41 ], operative time (OT) in 12 articles (35%) [ 7 , 13 , 14 , 16 , 17 , 22 , 29 , 32 , 34 , 35 , 36 , 38 ] and visual improvement in 12 articles (35%) [ 13 , 14 , 16 , 21 , 22 , 28 , 31 , 32 , 34 , 36 , 37 , 41 ]. (Fig.  2 ).

Frequency at which certain variables were evaluated in the literature to assess the EETA learning curve. EETA, endoscopic endonasal transsphenoidal approach; post-op, postoperative; CSF, cerebrospinal fluid; GTR, gross total resection; DI, diabetes insipidus; LOS, length of stay; IOP, intraoperative; ICA, internal carotid artery; SIADH, syndrome of inappropriate antidiuretic hormone secretion; LD, lumbar drain; CNS, central nervous system; CN, cranial nerve; EBL, estimated blood loss; DVT, deep vein thrombosis

In all the studies included, improvements were observed between early and late-experience stages [ 7 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 ]. Among the evaluated variables, the following improvements were noted: the endocrinological cure rate (EC) showed improvement in all 7 articles out of 7 evaluated [ 13 , 16 , 18 , 21 , 24 , 30 , 33 ], operative time (OT) improved in 11 out of 12 articles (91%) [ 13 , 14 , 16 , 17 , 22 , 29 , 32 , 34 , 35 , 36 , 38 ], postoperative cerebrospinal fluid leak (CSF) improved in 23 out of 28 articles (82%) [ 12 , 15 , 17 , 19 , 20 , 22 , 23 , 25 , 27 , 28 , 29 , 31 , 32 , 33 , 34 , 35 , 37 , 38 , 39 , 40 , 41 , 42 , 43 ], visual improvement was observed in 9 out of 12 articles (75%) [ 13 , 14 , 16 , 22 , 28 , 31 , 34 , 37 , 41 ], gross total resection (GTR) improved in 14 out of 21 articles (67%) [ 7 , 13 , 14 , 19 , 21 , 22 , 26 , 27 , 28 , 29 , 30 , 38 , 39 , 40 ], hospital length of stay (LOS) decreased in five out of 10 studies (50%) [ 11 , 12 , 16 , 17 , 22 ], and postoperative diabetes insipidus (DI) decreased in 7 out of 15 articles (47%) [ 3 , 14 , 16 , 17 , 21 , 22 , 33 ] (Fig.  3 ).

Proportion of main improved variables with experiences. EC, Endocrinological cure; OT, Operative time; post-op: postoperative; CSF, cerebrospinal fluid; GTR, gross total resection; hLOS, hospital length of stay; DI, diabetes insipidus

Moreover, 12 articles (35%) reported both significant and non-significant improvements in outcomes [ 7 , 13 , 14 , 16 , 17 , 21 , 22 , 31 , 32 , 34 , 38 , 41 ]. In 10 studies (29%), solely a trend of improvement was observed [ 11 , 15 , 19 , 20 , 23 , 26 , 27 , 29 , 30 , 40 ], while 8 articles (23%) reported solely significant improvements [ 18 , 24 , 25 , 35 , 36 , 37 , 42 , 43 ]. However, in four studies, despite observing a tendency towards better outcomes, no statistical disparities were identified among all assessed variables [ 12 , 28 , 33 , 39 ]. None of the included studies reported a deterioration in any of the assessed outcomes over time, except for one study where a significant decline in GTR was observed in the late group [ 33 ]. This decline was attributed to the inclusion of more invasive and complex tumors in the late group. Nevertheless, Younus et al. documented ongoing improvement in GTR even after surpassing the initial learning curve [ 7 ].

In this systematic review, the primary technique employed to determine the transition point indicating the overcoming of the initial learning curve involved observing sustained and consistent improvement in outcomes over time. In almost half of the included articles, overcoming the initial learning curve (observing improvement of outcomes) was linked to the number of cases performed. Out of the 34 analyzed studies, 16 (47%) estimated the number of cases needed to overcome the initial learning curve of EETA. Reported cases ranged widely from 9 to 120, with a mode of 50. Considering both the median and the Interquartile Range (IQR) provides a comprehensive understanding of the reported case distribution and central tendency for overcoming the initial EETA learning curve. The median number of cases needed is 32, with an IQR of 20. These numbers are estimates and require careful interpretation [ 16 , 17 , 20 , 21 , 22 , 23 , 24 , 25 , 29 , 31 , 32 , 33 , 35 , 36 , 37 , 38 , 42 ].

Regarding the quality of included studies, the NOS quality assessment scale was used. 21 studies graded as fair quality while the remaining 13 articles rated as poor quality [ 9 ]. The risk of bias was evaluated according to the GRADE system. All included studies are observational cohort study and graded either as low or very low grade [ 10 ]. This reflects the great heterogeneity and high risk of bias due to the study design of the current EETA-LC literature.

Endoscopic techniques have drastically improved skull-base surgery. Unlike procedures involving a microscope, many neurosurgeons have acquired experience in endoscopic techniques later in their careers, and the level of exposure to these techniques during training years has varied among surgeons. The LC is a critical factor in the acquisition of new surgical skills. Understanding the link between the EETA-LC and surgical outcomes will enable surgeons to better understand what to expect and what measures to apply as those surgical skills develop. Many studies in other surgical domains have reported on the LC during the acquisition of new surgical techniques [ 44 , 45 , 46 , 47 ]. Most minimally invasive surgeries are associated with a challenging LC, and EETA is no exception [ 7 , 46 ].

The concept of the LC was first established in the field of aircraft manufacturing and refers to an improvement in performance over time [ 48 ]. Smith et al. [ 17 ] have defined it as the number of procedures that must be performed for the outcomes to approach a long-term mean rate. Typically, an LC is characterized by an S-shaped curve with three stages: an early phase, during which new skill sets are acquired; a middle phase, in which the speed of learning rapidly increases; and an expert phase in which the performance reaches a plateau [ 49 ]. However, other curves have been proposed that involve a dip in the LC following the initial acceleration of the learning rate; this occurs especially with handling more challenging cases. Another potential decline may emerge after a long period of experience. Despite having reached a plateau in the learning curve after an extended period, declines in manual dexterity, eyesight, memory, and cognition may overshadow the benefits of accumulated experience, leading to diminished performance levels [ 50 ].

The absence of consensus on the best applicable methods to describe and assess the learning curve may explain the diversity of analysis methods observed in this systematic review. In their large systematic review regarding learning curve assessment in healthcare technologies, Ramsay et al. [ 51 ] reported that group splitting was the most frequent method. They defined group splitting as dividing the data by experience levels and conducting testing on discrete groups, often halves or thirds. The statistical methods applied included t-tests, chi-squared tests, Mann–Whitney U tests, and simple ANOVA.

In our review, we reached a similar conclusion. We observed that a substantial portion of articles (68%) utilized group-based learning curve analysis [ 7 , 11 , 12 , 13 , 16 , 17 , 19 , 21 , 22 , 23 , 26 , 27 , 28 , 30 , 31 , 32 , 33 , 34 , 36 , 37 , 38 , 42 , 43 ]. Additionally, we similarly noted that papers frequently lacked explanations for the selection of cut points, raising concerns about potential bias resulting from data-dependent splitting. It is important to acknowledge that this method of group categorization has inherent drawbacks, including challenges related to small sample sizes, the use of arbitrary cutoff points, and the inability to eliminate all potential confounding variables [ 52 ].

Descriptive analysis was found in 10 articles (29%) within this review [ 11 , 15 , 19 , 20 , 23 , 26 , 27 , 29 , 30 , 40 ]. While providing an initial grasp of data distribution and characteristics, descriptive analysis may fall short in capturing the intricate dynamics of the learning curve over time or the factors affecting its impact [ 51 ]. Alternatively, conducting rigorous statistical analyses afterward offers better insight and interpretation of the results. This approach aims to mitigate the influence of confounding factors on outcome assessments over time [ 51 , 52 ].

In our review, 24 articles (71%) conducted a wide variety of statistical analyses [ 7 , 12 , 13 , 14 , 16 , 17 , 18 , 21 , 22 , 24 , 25 , 28 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 41 , 42 , 43 ], including but not limited to the following tests: Chi-square Test, Fischer exact test, Student's t-test, Analysis of Variance (ANOVA), Mann–Whitney U Test, McNemar tests, Multivariate linear regression model, Cumulative Sum (CUSUM), and ROC Curve Analysis [ 13 , 16 , 22 , 32 , 37 , 38 , 39 ]. Four studies indicated that there was no statistically significant difference observed among the variables under evaluation. The lack of significance was attributed to several factors including small sample sizes, meticulous case selection, involvement of an otolaryngology team throughout the procedure, an increase in the number of invasive tumors in the late-experience study group, previous surgical experience, intensive training, level of supervision, and gradual inclusion of residents [ 12 , 28 , 33 , 39 ]. These efforts should be regarded as beneficial strategies aimed at reducing the steepness of the EETA learning curve.

To obtain more accurate results, it is crucial to eliminate confounding factors, such as the level of supervision, prior experience, the heterogeneity of cases being treated, and their complexity when evaluating the LC. Thus, it is essential to incorporate multivariate logistic regression analysis to mitigate the impact of these potential confounding factors [ 51 ]. Chi et al. [ 22 ] divided their patients into equal groups of 40 cases each. They then compared potential confounding variables to minimize their influence on learning curve assessment. This comparison includes demographic and clinical factors between the two groups, such as sex distribution, mean age, tumor size (microadenomas vs. macroadenomas), visual field defects, and tumor types (non-functioning, functioning adenomas, etc.). By conducting these comparisons, the researchers sought to identify discrepancies in demographic and clinical features between the groups.

The description of a surgeon's extensive prior experience is crucial for accurately quantifying the assessment of the learning curve, a point reported to be neglected during the assessment in various types of learning assessments related to healthcare procedures [ 49 ]. In our review, we observed the same conclusion in all included studies. However, the inclusion of the initial first few cases was mentioned in 13 (38%) articles, which might be used as a surrogate for no prior experience with EETA. Furthermore, five articles did not include the initial few cases. Among these, four studies examined the learning curve of more complex cases such as meningioma, craniopharyngioma, and growth hormone pituitary adenoma, employing an extended approach. Conversely, Younus et al. [ 7 ] deliberately excluded these cases to assess various stages of the learning curve.

Assessing multiple pathologies with varying complexities could significantly impact learning curve assessments. In our review, 59% of articles focused on a single pathology, while 41% explored multiple pathologies. Pituitary adenoma (PA) was the most evaluated (82%), followed by craniopharyngioma (CP) (44%). Controlling confounding variables like tumor type and size may yield more reliable results. Some studies used statistical analyses to compare early and late cases, while others relied on descriptive analyses. Shou et al. noted a drop in GTR over time due to late involvement of complex cases [ 33 ]. Conversely, studies analyzing tumor size and type found GTR improvement with experience [ 7 , 23 ]. Thorough multivariable analysis of confounding factors is crucial for representative LC analysis.

The LC is often assessed based on two main categories of variables: those related to the surgical procedure (OT, estimated blood loss, and extent of resection) and those related to patient outcomes (duration of hospitalization, the incidence of complications, and the mortality rate) [ 50 ]. In this systematic review, OT was one of the most frequent parameters that significantly reduced as one gained experience. Although OT is commonly utilized as an outcome measure, it is only a surrogate means of evaluating the LC and may not always accurately represent patient outcomes [ 52 ]. Another point to consider is the lack of standardized variables for assessing the LC, and the included studies evaluated more than 45 distinct variables. Khan et al. highlighted the importance of using consistent variable definitions across studies to derive accurate conclusions from aggregated LC data [ 52 ].

A dynamic relationship exists between surgical outcomes and the LC, and each phase of the LC influences a distinct set of variables differently. One study, which included data from 1,000 EETA cases after purposely eliminating the first 200 cases, showed that variables such as GTR and the endocrinological cure rate continued to improve after the first 200 cases, whereas other parameters remained unchanged. Authors concluded that some variables will continue to improve after passing the initial LC phase [ 7 ]. Determining the precise number of cases needed to surpass the initial learning curve (LC) has proven challenging. Shikary et al. observed a notable decrease in postoperative CSF leaks after 100 surgeries, while a reduction in operative time was evident after 120 cases [ 35 ]. However, specifying a definitive number to overcome the learning curve of the Endoscopic Endonasal Transsphenoidal Approach (EETA) remains challenging due to individual variability, diverse pathologies, and evolving surgical techniques.

Assessing the learning curve of the Endoscopic Endonasal Transsphenoidal Approach (EETA-LC) faces notable challenges due to its intricate techniques and the wide array of pathologies it addresses. The diversity across specialties makes standardizing studies difficult. To understand the dynamic learning process in EETA-LC, influenced by individual surgeon skill, patient nuances, and procedural complexities, longitudinal studies and advanced analytical methods are essential. Moreover, the complexity of statistical analysis adds another layer of challenge, highlighting the necessity for interdisciplinary collaboration and innovative methodologies.

To address the current limitations in the literature regarding the EETA LC, we propose several key strategies for future studies. Firstly, we advocate for multicenter collaboration, coupled with standardized processes, to comprehensively assess the EETA LC. This collaborative approach will facilitate the aggregation of data from diverse surgical settings, enhancing the generalizability of findings and minimizing bias. Furthermore, rigorous documentation of the previous and current experience of involved surgeons is paramount. We suggest categorizing surgeons based on their levels of experience to accurately elucidate the impact of proficiency on surgical outcomes. Secondly, given the wide variety of complexities of skull base pathologies encountered, we recommend further categorization of cases based on their levels of complexity. This stratification will enable a more nuanced analysis of the learning curve across different levels of surgical challenge. Thirdly, standardization of outcome measures used to assess the learning curve is imperative, with specific definitions provided for each outcome. This ensures consistency and comparability across studies, facilitating meaningful interpretation of results. Finally, conducting prospective study designs with sufficient follow-up periods, along with rigorous multivariate statistical analyses among these categorized groups, is essential to mitigate the influence of confounding variables and strengthen the validity of findings. Implementing these strategies will help future studies to overcome the current limitations in the literature, leading to a deeper understanding of the EETA learning curve and ultimately improving patient outcomes.

This systematic review identified 34 studies that reported a relationship between improvements in surgical outcomes and a surgeon’s level of experience with EETA. There is notable significant heterogeneity in the current literature on EETA-LC regarding the techniques used to assess the LC, variables assessed, types of pathology included, and insufficient reporting of the surgeon or team's current and previous experience with EETA. The main variables improved with experience were EC, postoperative CSF leak, OT, GTR visual improvement, and hospital LOS. Future studies with multicenter collaboration and standardized processes for assessing the EETA LC will enhance generalizability and minimize bias. Rigorous documentation of surgeons' experience levels, categorization of cases by complexity, and standardized outcome measures are essential. Additionally, rigorous statistical analyses will strengthen validity and mitigate confounding variables. Implementing these strategies will deepen our understanding of the EETA learning curve, ultimately leading to improved patient outcomes.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Endoscopic endonasal transsphenoidal approach

• Learning curve

Cerebrospinal fluid

Diabetes insipidus

Gross total resection

Cappabianca P, de Divitiis E. Endoscopy and transsphenoidal surgery. Neurosurgery. 2004;54:1043–50.

Article   PubMed   Google Scholar  

Gandhi CD, Post KD. Historical movements in transsphenoidal surgery. Neurosurg Focus. 2001;11:1–4.

Article   Google Scholar  

Gandhi CD, Christiano LD, Eloy JA, Prestigiacomo CJ, Post KD. The historical evolution of transsphenoidal surgery: facilitation by technological advances. Neurosurg Focus. 2009;27:E8.

de Divitiis E. Endoscopic transsphenoidal surgery: stone-in-the-pond effect. Neurosurgery. 2006;59:512–20.

Rotenberg B, Tam S, Ryu WHA, Duggal N. Microscopic versus endoscopic pituitary surgery: a systematic review. Laryngoscope. 2010;120:1292–7.

Tabaee A, Anand VK, Barrón Y, Hiltzik DH, Brown SM, Kacker A, et al. Endoscopic pituitary surgery: a systematic review and meta-analysis. J Neurosurg. 2009;111:545–54.

Younus I, Gerges MM, Uribe-Cardenas R, Morgenstern PF, Eljalby M, Tabaee A, et al. How long is the tail end of the learning curve? Results from 1000 consecutive endoscopic endonasal skull base cases following the initial 200 cases. J Neurosurg. 2020;134:750–60.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Int J Surg. 2021;88:105906.

Wells G, Shea B, O'Connell D, Peterson J, Welch V, Losos M, Tugwell P. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. 2013. Retrieved from http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp .

Balshem H, Helfand M, Schünemann HJ, Oxman AD, Kunz R, Brozek J, Vist GE, Falck-Ytter Y, Meerpohl J, Norris S, Guyatt GH. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol. 2011;64(4):401–6. https://doi.org/10.1016/j.jclinepi.2010.07.015 .

Cappabianca P, Cavallo L, Colao A, Del Basso De Caro M, Esposito F, Cirillo S, et al. Endoscopic endonasal transsphenoidal approach: outcome analysis of 100 consecutive procedures. Minim Invasive Neurosurg. 2002;45:193–200.

Article   CAS   PubMed   Google Scholar  

Sonnenburg RE, White D, Ewend MG, Senior B. The learning curve in minimally invasive pituitary surgery. Am J Rhinol. 2004;18:259–63.

Kenan K, İhsan A, Dilek O, Burak C, Gurkan K, Savas C. The learning curve in endoscopic pituitary surgery and our experience. Neurosurg Rev. 2006;29:298–305.

Yano S, Kawano T, Kudo M, Makino K, Nakamura H, Kai Y, et al. Endoscopic endonasal transsphenoidal approach through the bilateral nostrils for pituitary adenomas. Neurol Med Chir (Tokyo). 2009;49:1–7.

Gondim JA, Schops M, de Almeida JPC, de Albuquerque LAF, Gomes E, Ferraz T, et al. Endoscopic endonasal transsphenoidal surgery: surgical results of 228 pituitary adenomas treated in a pituitary center. Pituitary. 2010;13:68–77.

Leach P, Abou-Zeid AH, Kearney T, Davis J, Trainer PJ, Gnanalingham KK. Endoscopic transsphenoidal pituitary surgery: evidence of an operative learning curve. Neurosurgery. 2010;67:1205–12.

Smith SJ, Eralil G, Woon K, Sama A, Dow G, Robertson I. Light at the end of the tunnel: the learning curve associated with endoscopic transsphenoidal skull base surgery. Skull Base. 2010;20:69–74.

Article   PubMed   PubMed Central   Google Scholar  

Wagenmakers MAE, Netea-Maier RT, van Lindert EJ, Pieters GF, Grotenhuis AJ, Hermus AR. Results of endoscopic transsphenoidal pituitary surgery in 40 patients with a growth hormone-secreting macroadenoma. Acta Neurochir (Wien). 2011;153:1391–9.

Kumar S, Darr A, Hobbs C, Carlin W. Endoscopic, endonasal, trans-sphenoidal hypophysectomy: retrospective analysis of 171 procedures. J Laryngol Otol. 2012;126:1033–40.

Snyderman CH, Pant H, Kassam AB, Carrau RL, Prevedello DM, Gardner PA. The learning curve for endonasal surgery of the cranial base: A systematic approach to training. In: Kassam AB, Gardner PA, editors. Endoscopic approaches to the skull base. Ettlingen: Karger Publishers; 2012. p. 222–31.

Chapter   Google Scholar  

Bokhari AR, Davies MA, Diamond T. Endoscopic transsphenoidal pituitary surgery: a single surgeon experience and the learning curve. Br J Neurosurg. 2013;27:449.

Chi F, Wang Y, Lin Y, Ge J, Qiu Y, Guo L. A learning curve of endoscopic transsphenoidal surgery for pituitary adenoma. J Craniofac Surg. 2013;24:2064–7.

de los Santos G, Fragola C, Del Castillo R, Rodríguez V, D’oleo C, Reyes P. Endoscopic approaches to pituitary lesions: difficulties and challenges. Acta Otorrinolaringol Esp. 2013;64(258):64.

Google Scholar  

Hazer DB, Işık S, Berker D, Güler S, Gürlek A, Yücel T, et al. Treatment of acromegaly by endoscopic transsphenoidal surgery: surgical experience in 214 cases and cure rates according to current consensus criteria. J Neurosurg. 2013;119:1467–77.

Jakimovski D, Bonci G, Attia M, Shao H, Hofstetter C, Tsiouris AJ, et al. Incidence and significance of intraoperative cerebrospinal fluid leak in endoscopic pituitary surgery using intrathecal fluorescein. World Neurosurg. 2014;82:e513–23.

Koutourousiou M, Fernandez-Miranda JC, Wang EW, Snyderman CH, Gardner PA. Endoscopic endonasal surgery for olfactory groove meningiomas: outcomes and limitations in 50 patients. Neurosurg Focus. 2014;37:E8.

Mascarenhas L, Moshel YA, Bayad F, Szentirmai O, Salek AA, Leng LZ, et al. The transplanum transtuberculum approaches for suprasellar and sellar-suprasellar lesions: avoidance of cerebrospinal fluid leak and lessons learned. World Neurosurg. 2014;82:186–95.

Ottenhausen M, Banu MA, Placantonakis DG, Tsiouris AJ, Khan OH, Anand VK, et al. Endoscopic endonasal resection of suprasellar meningiomas: the importance of case selection and experience in determining extent of resection, visual improvement, and complications. World Neurosurg. 2014;82:442–9.

Ananth G, Hosmath AV, Varadaraju DN, Patil SR, Usman MM, Patil RP, et al. Learning curve in endoscopic transnasal sellar region surgery. J Evid Based Med Healthc. 2016;3:3166–72.

Jang JH, Kim KH, Lee YM, Kim JS, Kim YZ. Surgical results of pure endoscopic endonasal transsphenoidal surgery for 331 pituitary adenomas: a 15-year experience from a single institution. World Neurosurg. 2016;96:545–55.

Kshettry VR, Do H, Elshazly K, Farrell CJ, Nyquist G, Rosen M, et al. The learning curve in endoscopic endonasal resection of craniopharyngiomas. Neurosurg Focus. 2016;41:E9.

Qureshi T, Chaus F, Fogg L, Dasgupta M, Straus D, Byrne RW. Learning curve for the transsphenoidal endoscopic endonasal approach to pituitary tumors. Br J Neurosurg. 2016;30:637–42.

Shou X, Shen M, Zhang Q, Zhang Y, He W, Ma Z, et al. Endoscopic endonasal pituitary adenomas surgery: the surgical experience of 178 consecutive patients and learning curve of two neurosurgeons. BMC Neurol. 2016;16:1–8.

Ding H, Gu Y, Zhang X, Xie T, Liu T, Hu F, et al. Learning curve for the endoscopic endonasal approach for suprasellar craniopharyngiomas. J Clin Neurosci. 2017;42:209–16.

Shikary T, Andaluz N, Meinzen-Derr J, Edwards C, Theodosopoulos P, Zimmer LA. Operative learning curve after transition to endoscopic transsphenoidal pituitary surgery. World Neurosurg. 2017;102:608–12.

Eseonu CI, ReFaey K, Pamias-Portalatin E, Asensio J, Garcia O, Boahene KD, et al. Three-hand endoscopic endonasal transsphenoidal surgery: experience with an anatomy-preserving mononostril approach technique. Oper Neurosurg (Hagerstown). 2018;14:158–65.

Kim JH, Lee JH, Lee JH, Hong AR, Kim YJ, Kim YH. Endoscopic transsphenoidal surgery outcomes in 331 nonfunctioning pituitary adenoma cases after a single surgeon learning curve. World Neurosurg. 2018;109:e409–16.

Lofrese G, Vigo V, Rigante M, Grieco DL, Maresca M, Anile C, et al. Learning curve of endoscopic pituitary surgery: experience of a neurosurgery/ENT collaboration. J Clin Neurosci. 2018;47:299–303.

Robins JM, Alavi SA, Tyagi AK, Nix PA, Wilson TM, Phillips NI. The learning curve for endoscopic trans-sphenoidal resection of pituitary macroadenomas. A single institution experience, Leeds, UK. Acta Neurochir (Wien). 2018;160:39–47.

Algattas H, Setty P, Goldschmidt E, Wang EW, Tyler-Kabara EC, Snyderman CH, et al. Endoscopic endonasal approach for craniopharyngiomas with intraventricular extension: case series, long-term outcomes, and review. World Neurosurg. 2020;144:e447–59.

Soliman MA, Eaton S, Quint E, Alkhamees AF, Shahab S, O’Connor A, et al. Challenges, learning curve, and safety of endoscopic endonasal surgery of sellar-suprasellar lesions in a community hospital. World Neurosurg. 2020;138:e940–54.

Nix P, Alavi SA, Tyagi A, Phillips N. Endoscopic repair of the anterior skull base-is there a learning curve? Br J Neurosurg. 2018;32:407–11.

Park W, Nam D-H, Kong D-S, Lee KE, Park SI, Kim HY, et al. Learning curve and technical nuances of endoscopic skull base reconstruction with nasoseptal flap to control high-flow cerebrospinal fluid leakage: reconstruction after endoscopic skull base surgery other than pituitary surgery. Eur Arch Otorhinolaryngol. 2022;279:1335–40.

Lubowitz JH, Sahasrabudhe A, Appleby D. Minimally invasive surgery in total knee arthroplasty: the learning curve. Orthopedics. 2007;30:80.

PubMed   Google Scholar  

Hoppe DJ, Simunovic N, Bhandari M, Safran MR, Larson CM, Ayeni OR. The learning curve for hip arthroscopy: a systematic review. Arthroscopy. 2014;30:389–97.

Sclafani JA, Kim CW. Complications associated with the initial learning curve of minimally invasive spine surgery: a systematic review. Clin Orthop Relat Res. 2014;472:1711–7.

Pernar LI, Robertson FC, Tavakkoli A, Sheu EG, Brooks DC, Smink DS. An appraisal of the learning curve in robotic general surgery. Surg Endosc. 2017;31:4583–96.

Wright TP. Factors affecting the cost of airplanes. J Aeronaut Sci. 1936;3:122–8.

Cook JA, Ramsay CR, Fayers P. Using the literature to quantify the learning curve: a case study. Int J Technol Assess Health Care. 2007;23:255–60.

Hopper A, Jamison M, Lewis W. Learning curves in surgical practice. Postgrad Med J. 2007;83:777–9.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Ramsay CR, Grant AM, Wallace SA, Garthwaite PH, Monk AF, Russell IT. Assessment of the learning curve in health technologies: a systematic review. Int J Technol Assess Health Care. 2000;16:1095–108.

Khan N, Abboudi H, Khan MS, Dasgupta P, Ahmed K. Measuring the surgical ‘learning curve’: methods, variables and competency. BJU Int. 2014;113:504–8.

Download references

Acknowledgements

Not applicable

No funding was received for this study.

Author information

Authors and affiliations.

Neurosurgery Department, East Jeddah Hospital, 2277 King Abdullah Rd, Al Sulaymaniyah, 22253, Jeddah, Saudi Arabia

Abdulraheem Alomari

Otolaryngology and Head and Neck Surgery Department, King Abdullah Medical Complex, Prince Nayef Street, Northern Abhor, 23816, Jeddah, Saudi Arabia

Mazin Alsarraj

Neurosurgery Department, King Abdulaziz Medical City, 21423, Jeddah, Saudi Arabia

Sarah Alqarni

You can also search for this author in PubMed   Google Scholar

Contributions

AA: Acquisition of Data, Analysis and Interpretation of Data, Drafting the Article, Critically Revising the Article, Drafting the Article, Reviewed submitted version of manuscript, Approved the final version of the manuscript on behalf of all authors. Study supervision; MA: Conception and Design, Analysis and Interpretation of Data, Reviewed submitted version of manuscript, Analysis and Interpretation of Data; SA: Acquisition of Data, Conception and Design, Critically Revising the Article, Analysis and Interpretation of Data, Administrative / technical / material support.

Corresponding author

Correspondence to Abdulraheem Alomari .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Alomari, A., Alsarraj, M. & Alqarni, S. The learning curve in endoscopic transsphenoidal skull-base surgery: a systematic review. BMC Surg 24 , 135 (2024). https://doi.org/10.1186/s12893-024-02418-y

Download citation

Received : 21 September 2023

Accepted : 20 April 2024

Published : 05 May 2024

DOI : https://doi.org/10.1186/s12893-024-02418-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Endoscopic skull base
• Transsphenoidal surgery

BMC Surgery

ISSN: 1471-2482

• Submission enquiries: [email protected]
• General enquiries: [email protected]