slr literature review

How to Write a Systematic Review of the Literature

Affiliations.

• 1 1 Texas Tech University, Lubbock, TX, USA.
• 2 2 University of Florida, Gainesville, FL, USA.
• PMID: 29283007
• DOI: 10.1177/1937586717747384

This article provides a step-by-step approach to conducting and reporting systematic literature reviews (SLRs) in the domain of healthcare design and discusses some of the key quality issues associated with SLRs. SLR, as the name implies, is a systematic way of collecting, critically evaluating, integrating, and presenting findings from across multiple research studies on a research question or topic of interest. SLR provides a way to assess the quality level and magnitude of existing evidence on a question or topic of interest. It offers a broader and more accurate level of understanding than a traditional literature review. A systematic review adheres to standardized methodologies/guidelines in systematic searching, filtering, reviewing, critiquing, interpreting, synthesizing, and reporting of findings from multiple publications on a topic/domain of interest. The Cochrane Collaboration is the most well-known and widely respected global organization producing SLRs within the healthcare field and a standard to follow for any researcher seeking to write a transparent and methodologically sound SLR. Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA), like the Cochrane Collaboration, was created by an international network of health-based collaborators and provides the framework for SLR to ensure methodological rigor and quality. The PRISMA statement is an evidence-based guide consisting of a checklist and flowchart intended to be used as tools for authors seeking to write SLR and meta-analyses.

Keywords: evidence based design; healthcare design; systematic literature review.

• Evidence-Based Medicine* / organization & administration
• Research Design*
• Systematic Reviews as Topic*

What is a Systematic Literature Review?

A systematic literature review (SLR) is an independent academic method that aims to identify and evaluate all relevant literature on a topic in order to derive conclusions about the question under consideration. "Systematic reviews are undertaken to clarify the state of existing research and the implications that should be drawn from this." (Feak & Swales, 2009, p. 3) An SLR can demonstrate the current state of research on a topic, while identifying gaps and areas requiring further research with regard to a given research question. A formal methodological approach is pursued in order to reduce distortions caused by an overly restrictive selection of the available literature and to increase the reliability of the literature selected (Tranfield, Denyer & Smart, 2003). A special aspect in this regard is the fact that a research objective is defined for the search itself and the criteria for determining what is to be included and excluded are defined prior to conducting the search. The search is mainly performed in electronic literature databases (such as Business Source Complete or Web of Science), but also includes manual searches (reviews of reference lists in relevant sources) and the identification of literature not yet published in order to obtain a comprehensive overview of a research topic.

An SLR protocol documents all the information gathered and the steps taken as part of an SLR in order to make the selection process transparent and reproducible. The PRISMA flow-diagram support you in making the selection process visible.

In an ideal scenario, experts from the respective research discipline, as well as experts working in the relevant field and in libraries, should be involved in setting the search terms . As a rule, the literature is selected by two or more reviewers working independently of one another. Both measures serve the purpose of increasing the objectivity of the literature selection. An SLR must, then, be more than merely a summary of a topic (Briner & Denyer, 2012). As such, it also distinguishes itself from “ordinary” surveys of the available literature. The following table shows the differences between an SLR and an “ordinary” literature review.

• Charts of BSWL workshop (pdf, 2.88 MB)
• Listen to the interview (mp4, 12.35 MB)

Differences to "common" literature reviews

What are the objectives of slrs.

• Avoidance of research redundancies despite a growing amount of publications
• Identification of research areas, gaps and methods
• Input for evidence-based management, which allows to base management decisions on scientific methods and findings
• Identification of links between different areas of researc

Process steps of an SLR

A SLR has several process steps which are defined differently in the literature (Fink 2014, p. 4; Guba 2008, Transfield et al. 2003). We distinguish the following steps which are adapted to the economics and management research area:

1. Defining research questions

Briner & Denyer (2009, p. 347ff.) have developed the CIMO scheme to establish clearly formulated and answerable research questions in the field of economic sciences:

C – CONTEXT:  Which individuals, relationships, institutional frameworks and systems are being investigated?

I – Intervention:  The effects of which event, action or activity are being investigated?

M – Mechanisms:  Which mechanisms can explain the relationship between interventions and results? Under what conditions do these mechanisms take effect?

O – Outcomes:  What are the effects of the intervention? How are the results measured? What are intended and unintended effects?

The objective of the systematic literature review is used to formulate research questions such as “How can a project team be led effectively?”. Since there are numerous interpretations and constructs for “effective”, “leadership” and “project team”, these terms must be particularized.

With the aid of the scheme, the following concrete research questions can be derived with regard to this example:

Under what conditions (C) does leadership style (I) influence the performance of project teams (O)?

Which constructs have an effect upon the influence of leadership style (I) on a project team’s performance (O)?          

Research questions do not necessarily need to follow the CIMO scheme, but they should:

• ... be formulated in a clear, focused and comprehensible manner and be answerable;
• ... have been determined prior to carrying out the SLR;
• ... consist of general and specific questions.

As early as this stage, the criteria for inclusion and exclusion are also defined. The selection of the criteria must be well-grounded. This may include conceptual factors such as a geographical or temporal restrictions, congruent definitions of constructs, as well as quality criteria (journal impact factor > x).

2. Selecting databases and other research sources

The selection of sources must be described and explained in detail. The aim is to find a balance between the relevance of the sources (content-related fit) and the scope of the sources.

In the field of economic sciences, there are a number of literature databases that can be searched as part of an SLR. Some examples in this regard are:

• Business Source Complete
• ProQuest One Business
• Web of Science
• EconBiz        

Our video " Selecting the right databases " explains how to find relevant databases for your topic.

Literature databases are an important source of research for SLRs, as they can minimize distortions caused by an individual literature selection (selection bias), while offering advantages for a systematic search due to their data structure. The aim is to find all database entries on a topic and thus keep the retrieval bias low (tutorial on retrieval bias ).  Besides articles from scientific journals, it is important to inlcude working papers, conference proceedings, etc to reduce the publication bias ( tutorial on publication bias ).

Our online self-study course " Searching economic databases " explains step 2 und 3.

3. Defining search terms

Once the literature databases and other research sources have been selected, search terms are defined. For this purpose, the research topic/questions is/are divided into blocks of terms of equal ranking. This approach is called the block-building method (Guba 2008, p. 63). The so-called document-term matrix, which lists topic blocks and search terms according to a scheme, is helpful in this regard. The aim is to identify as many different synonyms as possible for the partial terms. A precisely formulated research question facilitates the identification of relevant search terms. In addition, keywords from particularly relevant articles support the formulation of search terms.

A document-term matrix for the topic “The influence of management style on the performance of project teams” is shown in this example .

Identification of headwords and keywords

When setting search terms, a distinction must be made between subject headings and keywords, both of which are described below:

• appear in the title, abstract and/or text
• sometimes specified by the author, but in most cases automatically generated
• non-standardized
• different spellings and forms (singular/plural) must be searched separately

Subject headings

• describe the content
• are generated by an editorial team
• are listed in a standardized list (thesaurus)
• may comprise various keywords
• include different spellings
• database-specific

Subject headings are a standardized list of words that are generated by the specialists in charge of some databases. This so-called index of subject headings (thesaurus) helps searchers find relevant articles, since the headwords indicate the content of a publication. By contrast, an ordinary keyword search does not necessarily result in a content-related fit, since the database also displays articles in which, for example, a word appears once in the abstract, even though the article’s content does not cover the topic.

Nevertheless, searches using both headwords and keywords should be conducted, since some articles may not yet have been assigned headwords, or errors may have occurred during the assignment of headwords. 

To add headwords to your search in the Business Source Complete database, please select the Thesaurus tab at the top. Here you can find headwords in a new search field and integrate them into your search query. In the search history, headwords are marked with the addition DE (descriptor).

The EconBiz database of the German National Library of Economics (ZBW – Leibniz Information Centre for Economics), which also contains German-language literature, has created its own index of subject headings with the STW Thesaurus for Economics . Headwords are integrated into the search by being used in the search query.

Since the indexes of subject headings divide terms into synonyms, generic terms and sub-aspects, they facilitate the creation of a document-term matrix. For this purpose it is advisable to specify in the document-term matrix the origin of the search terms (STW Thesaurus for Economics, Business Source Complete, etc.).

Searching in literature databases

Once the document-term matrix has been defined, the search in literature databases begins. It is recommended to enter each word of the document-term matrix individually into the database in order to obtain a good overview of the number of hits per word. Finally, all the words contained in a block of terms are linked with the Boolean operator OR and thereby a union of all the words is formed. The latter are then linked with each other using the Boolean operator AND. In doing so, each block should be added individually in order to see to what degree the number of hits decreases.

Since the search query must be set up separately for each database, tools such as  LitSonar  have been developed to enable a systematic search across different databases. LitSonar was created by  Professor Dr. Ali Sunyaev (Institute of Applied Informatics and Formal Description Methods – AIFB) at the Karlsruhe Institute of Technology.

Advanced search

Certain database-specific commands can be used to refine a search, for example, by taking variable word endings into account (*) or specifying the distance between two words, etc. Our overview shows the most important search commands for our top databases.

Additional searches in sources other than literature databases

In addition to literature databases, other sources should also be searched. Fink (2014, p. 27) lists the following reasons for this:

• the topic is new and not yet included in indexes of subject headings;
• search terms are not used congruently in articles because uniform definitions do not exist;
• some studies are still in the process of being published, or have been completed, but not published.

Therefore, further search strategies are manual search, bibliographic analysis, personal contacts and academic networks (Briner & Denyer, p. 349). Manual search means that you go through the source information of relevant articles and supplement your hit list accordingly. In addition, you should conduct a targeted search for so-called gray literature, that is, literature not distributed via the book trade, such as working papers from specialist areas and conference reports. By including different types of publications, the so-called publication bias (DBWM video “Understanding publication bias” ) – that is, distortions due to exclusive use of articles from peer-reviewed journals – should be kept to a minimum.

The PRESS-Checklist can support you to check the correctness of your search terms.

4. Merging hits from different databases

In principle, large amounts of data can be easily collected, structured and sorted with data processing programs such as Excel. Another option is to use literature management programs such as EndNote, Citavi or Zotero. The Saxon State and University Library Dresden (SLUB Dresden) provides an  overview of current literature management programs  . Software for qualitative data analysis such as NVivo is equally suited for data processing. A comprehensive overview of the features of different tools that support the SLR process can be found in Bandara et al. (2015).

Our online-self study course "Managing literature with Citavi" shows you how to use the reference management software Citavi.

When conducting an SLR, you should specify for each hit the database from which it originates and the date on which the query was made. In addition, you should always indicate how many hits you have identified in the various databases or, for example, by manual search.

Exporting data from literature databases

Exporting from literature databases is very easy. In  Business Source Complete  , you must first click on the “Share” button in the hit list, then “Email a link to download exported results” at the very bottom and then select the appropriate format for the respective literature program.

In the  Web of Science  database, you must select “Export” and select the relevant format. Tip: You can adjust the extracted data fields. Since for example the abstract is not automatically exported, decide which data fields are of interest for you.

Exporting data from the literature database  EconBiz  is somewhat more complex. Here you must first create a marked list and then select each hit individually and add it to the marked list. Afterwards, articles on the list can be exported.

After merging all hits from the various databases, duplicate entries (duplicates) are deleted.

5. Applying inclusion and exclusion criteria

All publications are evaluated in the literature management program applying the previously defined criteria for inclusion and exclusion. Only those sources that survive this selection process will subsequently be analyzed. The review process and inclusion criteria should be tested with a small sample and adjustments made if necessary before applying it to all articles. In the ideal case, even this selection would be carried out by more than one person, with each working independently of one another. It needs to be made clear how discrepancies between reviewers are dealt with. 

The review of the criteria for inclusion and exclusion is primarily based on the title, abstract and subject headings in the databases, as well as on the keywords provided by the authors of a publication in the first step. In a second step the whole article / source will be read.

Within the Citavi literature-management program, you can supplement title data by adding your own fields. In this regard, the criteria for inclusion can be listed individually and marked with 0 in the free text field for being “not fulfilled” and with 1 for being “fulfilled”. In the table view of all titles, you can use the column function to select which columns should be displayed. Here you can include the criteria for inclusion. By exporting the title list to Excel, it is easy to calculate how many titles remain when applying the criteria for inclusion and exclusion.

In addition to the common literature management tools, you can also use software tools that have been developed to support SLRs. The central library of the university in Zurich has published an overview and evaluation of different tools based on a survey among researchers. --> View SLR tools

The selection process needs to be made transparent. The PRISMA flow diagram supports the visualization of the number of included / excluded studies.

Forward and backward search

Should it become apparent that the number of sources found is relatively small, or if you wish to proceed with particular thoroughness, a forward-and-backward search based on the sources found is recommendable (Webster & Watson 2002, p. xvi). A backward search means going through the bibliographies of the sources found. A forward search, by contrast, identifies articles that have cited the relevant publications. The Web of Science and Scopus databases can be used to perform citation analyses.

6. Perform the review

As the next step, the remaining titles are analyzed as to their content by reading them several times in full. Information is extracted according to defined criteria and the quality of the publications is evaluated. If the data extraction is carried out by more than one person, a training ensures that there will be no differences between the reviewers.

Depending on the research questions there exist diffent methods for data abstraction (content analysis, concept matrix etc.). A so-called concept matrix can be used to structure the content of information (Webster & Watson 2002, p. xvii). The image to the right gives an example of a concept matrix according to Becker (2014).

Particularly in the field of economic sciences, the evaluation of a study’s quality cannot be performed according to a generally valid scheme, such as those existing in the field of medicine, for instance. Quality assessment therefore depends largely on the research questions.

Based on the findings of individual studies, a meta-level is then applied to try to understand what similarities and differences exist between the publications, what research gaps exist, etc. This may also result in the development of a theoretical model or reference framework.

Example concept matrix (Becker 2013) on the topic Business Process Management

7. synthesizing results.

Once the review has been conducted, the results must be compiled and, on the basis of these, conclusions derived with regard to the research question (Fink 2014, p. 199ff.). This includes, for example, the following aspects:

• historical development of topics (histogram, time series: when, and how frequently, did publications on the research topic appear?);
• overview of journals, authors or specialist disciplines dealing with the topic;
• comparison of applied statistical methods;
• topics covered by research;
• identifying research gaps;
• developing a reference framework;
• developing constructs;
• performing a meta-analysis: comparison of the correlations of the results of different empirical studies (see for example Fink 2014, p. 203 on conducting meta-analyses)

Publications about the method

Bandara, W., Furtmueller, E., Miskon, S., Gorbacheva, E., & Beekhuyzen, J. (2015). Achieving Rigor in Literature Reviews: Insights from Qualitative Data Analysis and Tool-Support.  Communications of the Association for Information Systems . 34(8), 154-204.

Booth, A., Papaioannou, D., and Sutton, A. (2012)  Systematic approaches to a successful literature review.  London: Sage.

Briner, R. B., & Denyer, D. (2012). Systematic Review and Evidence Synthesis as a Practice and Scholarship Tool. In Rousseau, D. M. (Hrsg.),  The Oxford Handbook of Evidenence Based Management . (S. 112-129). Oxford: Oxford University Press.

Durach, C. F., Wieland, A., & Machuca, Jose A. D. (2015). Antecedents and dimensions of supply chain robustness: a systematic literature review . International Journal of Physical Distribution & Logistic Management , 46 (1/2), 118-137. doi:  https://doi.org/10.1108/IJPDLM-05-2013-0133

Feak, C. B., & Swales, J. M. (2009). Telling a Research Story: Writing a Literature Review.  English in Today's Research World 2.  Ann Arbor: University of Michigan Press. doi:  10.3998/mpub.309338

Fink, A. (2014).  Conducting Research Literature Reviews: From the Internet to Paper  (4. Aufl.). Los Angeles, London, New Delhi, Singapore, Washington DC: Sage Publication.

Fisch, C., & Block, J. (2018). Six tips for your (systematic) literature review in business and management research.  Management Review Quarterly,  68, 103–106 (2018).  doi.org/10.1007/s11301-018-0142-x

Guba, B. (2008). Systematische Literaturrecherche.  Wiener Medizinische Wochenschrift , 158 (1-2), S. 62-69. doi:  doi.org/10.1007/s10354-007-0500-0  Hart, C.  Doing a literature review: releasing the social science research imagination.  London: Sage.

Jesson, J. K., Metheson, L. & Lacey, F. (2011).  Doing your Literature Review - traditional and Systematic Techniques . Los Angeles, London, New Delhi, Singapore, Washington DC: Sage Publication.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71. doi: 10.1136/bmj.n71.

Petticrew, M. and Roberts, H. (2006).  Systematic Reviews in the Social Sciences: A Practical Guide . Oxford:Blackwell. Ridley, D. (2012).  The literature review: A step-by-step guide . 2nd edn. London: Sage. 

Chang, W. and Taylor, S.A. (2016), The Effectiveness of Customer Participation in New Product Development: A Meta-Analysis,  Journal of Marketing , American Marketing Association, Los Angeles, CA, Vol. 80 No. 1, pp. 47–64.

Tranfield, D., Denyer, D. & Smart, P. (2003). Towards a methodology for developing evidence-informed management knowledge by means of systematic review.  British Journal of Management , 14 (3), S. 207-222. doi:  https://doi.org/10.1111/1467-8551.00375

Webster, J., & Watson, R. T. (2002). Analyzing the Past to Prepare for the Future: Writing a Literature Review.  Management Information Systems Quarterly , 26(2), xiii-xxiii.  http://www.jstor.org/stable/4132319

Durach, C. F., Wieland, A. & Machuca, Jose. A. D. (2015). Antecedents and dimensions of supply chain robustness: a systematic literature review. International Journal of Physical Distribution & Logistics Management, 45(1/2), 118 – 137.

What is particularly good about this example is that search terms were defined by a number of experts and the review was conducted by three researchers working independently of one another. Furthermore, the search terms used have been very well extracted and the procedure of the literature selection very well described.

On the downside, the restriction to English-language literature brings the language bias into play, even though the authors consider it to be insignificant for the subject area.

Bos-Nehles, A., Renkema, M. & Janssen, M. (2017). HRM and innovative work behaviour: a systematic literature review. Personnel Review, 46(7), pp. 1228-1253

• Only very specific keywords used
• No precise information on how the review process was carried out (who reviewed articles?)
• Only journals with impact factor (publication bias)

Jia, F., Orzes, G., Sartor, M. & Nassimbeni, G. (2017). Global sourcing strategy and structure: towards a conceptual framework. International Journal of Operations & Production Management, 37(7), 840-864

• Research questions are explicitly presented
• Search string very detailed
• Exact description of the review process
• 2 persons conducted the review independently of each other

Franziska Klatt

[email protected]

+49 30 314-29778

Privacy notice: The TU Berlin offers a chat information service. If you enable it, your IP address and chat messages will be transmitted to external EU servers. more information

The chat is currently unavailable.

Please use our alternative contact options.

• Systematic Review
• Open access
• Published: 23 May 2024

Systematic literature review of real-world evidence for treatments in HR+/HER2- second-line LABC/mBC after first-line treatment with CDK4/6i

• Veronique Lambert   ORCID: orcid.org/0000-0002-6984-0038 1 ,
• Sarah Kane   ORCID: orcid.org/0009-0006-9341-4836 2   na1 ,
• Belal Howidi   ORCID: orcid.org/0000-0002-1166-7631 2   na1 ,
• Bao-Ngoc Nguyen   ORCID: orcid.org/0000-0001-6026-2270 2   na1 ,
• David Chandiwana   ORCID: orcid.org/0009-0002-3499-2565 3 ,
• Yan Wu   ORCID: orcid.org/0009-0008-3348-9232 1 ,
• Michelle Edwards   ORCID: orcid.org/0009-0001-4292-3140 3 &
• Imtiaz A. Samjoo   ORCID: orcid.org/0000-0003-1415-8055 2   na1  

BMC Cancer volume  24 , Article number:  631 ( 2024 ) Cite this article

366 Accesses

1 Altmetric

Metrics details

Cyclin-dependent kinase 4 and 6 inhibitors (CDK4/6i) combined with endocrine therapy (ET) are currently recommended by the National Comprehensive Cancer Network (NCCN) guidelines and the European Society for Medical Oncology (ESMO) guidelines as the first-line (1 L) treatment for patients with hormone receptor-positive, human epidermal growth factor receptor 2-negative, locally advanced/metastatic breast cancer (HR+/HER2- LABC/mBC). Although there are many treatment options, there is no clear standard of care for patients following 1 L CDK4/6i. Understanding the real-world effectiveness of subsequent therapies may help to identify an unmet need in this patient population. This systematic literature review qualitatively synthesized effectiveness and safety outcomes for treatments received in the real-world setting after 1 L CDK4/6i therapy in patients with HR+/ HER2- LABC/mBC.

MEDLINE®, Embase, and Cochrane were searched using the Ovid® platform for real-world evidence studies published between 2015 and 2022. Grey literature was searched to identify relevant conference abstracts published from 2019 to 2022. The review was conducted in accordance with PRISMA guidelines (PROSPERO registration: CRD42023383914). Data were qualitatively synthesized and weighted average median real-world progression-free survival (rwPFS) was calculated for NCCN/ESMO-recommended post-1 L CDK4/6i treatment regimens.

Twenty records (9 full-text articles and 11 conference abstracts) encompassing 18 unique studies met the eligibility criteria and reported outcomes for second-line (2 L) treatments after 1 L CDK4/6i; no studies reported disaggregated outcomes in the third-line setting or beyond. Sixteen studies included NCCN/ESMO guideline-recommended treatments with the majority evaluating endocrine-based therapy; five studies on single-agent ET, six studies on mammalian target of rapamycin inhibitors (mTORi) ± ET, and three studies with a mix of ET and/or mTORi. Chemotherapy outcomes were reported in 11 studies. The most assessed outcome was median rwPFS; the weighted average median rwPFS was calculated as 3.9 months (3.3-6.0 months) for single-agent ET, 3.6 months (2.5–4.9 months) for mTORi ± ET, 3.7 months for a mix of ET and/or mTORi (3.0–4.0 months), and 6.1 months (3.7–9.7 months) for chemotherapy. Very few studies reported other effectiveness outcomes and only two studies reported safety outcomes. Most studies had heterogeneity in patient- and disease-related characteristics.

Conclusions

The real-world effectiveness of current 2 L treatments post-1 L CDK4/6i are suboptimal, highlighting an unmet need for this patient population.

Peer Review reports

Introduction

Breast cancer (BC) is the most diagnosed form of cancer in women with an estimated 2.3 million new cases diagnosed worldwide each year [ 1 ]. BC is the second leading cause of cancer death, accounting for 685,000 deaths worldwide per year [ 2 ]. By 2040, the global burden associated with BC is expected to surpass three million new cases and one million deaths annually (due to population growth and aging) [ 3 ]. Numerous factors contribute to global disparities in BC-related mortality rates, including delayed diagnosis, resulting in a high number of BC cases that have progressed to locally advanced BC (LABC) or metastatic BC (mBC) [ 4 , 5 , 6 ]. In the United States (US), the five-year survival rate for patients who progress to mBC is three times lower (31%) than the overall five-year survival rate for all stages (91%) [ 6 , 7 ].

Hormone receptor (HR) positive (i.e., estrogen receptor and/or progesterone receptor positive) coupled with negative human epidermal growth factor 2 (HER2) expression is the most common subtype of BC, accounting for ∼ 60–70% of all BC cases [ 8 , 9 ]. Historically, endocrine therapy (ET) through estrogen receptor modulation and/or estrogen deprivation has been the standard of care for first-line (1 L) treatment of HR-positive/HER2-negative (HR+/HER2-) mBC [ 10 ]. However, with the approval of the cyclin-dependent kinase 4/6 inhibitor (CDK4/6i) palbociclib in combination with the aromatase inhibitor (AI) letrozole in 2015 by the US Food and Drug Administration (FDA), 1 L treatment practice patterns have evolved such that CDK4/6i (either in combination with AIs or with fulvestrant) are currently considered the standard of care [ 11 , 12 , 13 , 14 , 15 , 16 , 17 ]. Other CDK4/6i (ribociclib and abemaciclib) in combination with ET are approved for the treatment of HR+/HER2- LABC/mBC; 1 L use of ribociclib in combination with an AI was granted FDA approval in March 2017 for postmenopausal women (with expanded approval in July 2018 for pre/perimenopausal women and for use in 1 L with fulvestrant for patients with disease progression on ET as well as for postmenopausal women), and abemaciclib in combination with fulvestrant was granted FDA approval in September 2017 for patients with disease progression following ET and as monotherapy in cases where disease progression occurs following ET and prior chemotherapy in mBC (with expanded approval in February 2018 for use in 1 L in combination with an AI for postmenopausal women) [ 18 , 19 , 20 , 21 ].

Clinical trials investigating the addition of CDK4/6i to ET have demonstrated significant improvement in progression-free survival (PFS) and significant (ribociclib) or numerical (palbociclib and abemaciclib) improvement in overall survival (OS) compared to ET alone in patients with HR+/HER2- advanced or mBC, making this combination treatment the recommended option in the 1 L setting [ 22 , 23 , 24 , 25 , 26 , 27 ]. However, disease progression occurs in a significant portion of patients after 1 L CDK4/6i treatment [ 28 ] and the optimal treatment sequence after progression on CDK4/6i remains unclear [ 29 ]. At the time of this review (literature search conducted December 14, 2022), guidelines by the National Comprehensive Cancer Network (NCCN) and the European Society for Medical Oncology (ESMO) recommend various options for the treatment of HR+/HER2- advanced BC in the second-line (2 L) setting, including fulvestrant monotherapy, mammalian target of rapamycin inhibitors (mTORi; e.g., everolimus) ± ET, alpelisib + fulvestrant (if phosphatidylinositol-4,5-bisphosphate 3-kinase catalytic subunit alpha mutation positive [PIK3CA-m+]), poly-ADP ribose polymerase inhibitors (PARPi) including olaparib or talazoparib (if breast cancer gene/partner and localizer of BRCA2 positive [BRCA/PALB2m+]), and chemotherapy (in cases when a visceral crisis is present) [ 15 , 16 ]. CDK4/6i can also be used in 2 L [ 16 , 30 ]; however, limited data are available to support CDK4/6i rechallenge after its use in the 1 L setting [ 15 ]. Depending on treatments used in the 1 L and 2 L settings, treatment in the third-line setting is individualized based on the patient’s response to prior treatments, tumor load, duration of response, and patient preference [ 9 , 15 ]. Understanding subsequent treatments after 1 L CDK4/6i, and their associated effectiveness, is an important focus in BC research.

Treatment options for HR+/HER2- LABC/mBC continue to evolve, with ongoing research in both clinical trials and in the real-world setting. Real-world evidence (RWE) offers important insights into novel therapeutic regimens and the effectiveness of treatments for HR+/HER2- LABC/mBC. The effectiveness of the current treatment options following 1 L CDK4/6i therapy in the real-world setting highlights the unmet need in this patient population and may help to drive further research and drug development. In this study, we conducted a systematic literature review (SLR) to qualitatively summarize the effectiveness and safety of treatment regimens in the real-world setting after 1 L treatment with CDK4/6i in patients with HR+/HER2- LABC/mBC.

Literature search

An SLR was performed in accordance with the Cochrane Handbook for Systematic Reviews of Interventions [ 31 ] and reported in alignment with the Preferred Reporting Items for Systematic Literature Reviews and Meta-Analyses (PRISMA) statement [ 32 ] to identify all RWE studies assessing the effectiveness and safety of treatments used for patients with HR+/HER2- LABC/mBC following 1 L CDK4/6i therapy and received subsequent treatment in 2 L and beyond (2 L+). The Ovid® platform was used to search MEDLINE® (including Epub Ahead of Print and In-Process, In-Data-Review & Other Non-Indexed Citations), Ovid MEDLINE® Daily, Embase, Cochrane Central Register of Controlled Trials, and Cochrane Database of Systematic Reviews by an experienced medical information specialist. The MEDLINE® search strategy was peer-reviewed independently by a senior medical information specialist before execution using the Peer Review of Electronic Search Strategies (PRESS) checklist [ 33 ]. Searches were conducted on December 14, 2022. The review protocol was developed a priori and registered with the International Prospective Register of Systematic Review (PROSPERO; CRD42023383914) which outlined the population, intervention, comparator, outcome, and study design (PICOS) criteria and methodology used to conduct the review (Table  1 ).

Search strategies utilized a combination of controlled vocabulary (e.g., “HER2 Breast Cancer” or “HR Breast Cancer”) and keywords (e.g., “Retrospective studies”). Vocabulary and syntax were adjusted across databases. Published and validated filters were used to select for study design and were supplemented using additional medical subject headings (MeSH) terms and keywords to select for RWE and nonrandomized studies [ 34 ]. No language restrictions were included in the search strategy. Animal-only and opinion pieces were removed from the results. The search was limited to studies published between January 2015 and December 2022 to reflect the time at which FDA approval was granted for the first CDK4/6i agent (palbociclib) in combination with AI for the treatment of LABC/mBC [ 35 ]. Further search details are presented in Supplementary Material 1 .

Grey literature sources were also searched to identify relevant abstracts and posters published from January 2019 to December 2022 for prespecified relevant conferences including ESMO, San Antonio Breast Cancer Symposium (SABCS), American Society of Clinical Oncology (ASCO), the International Society for Pharmacoeconomics and Outcomes Research (ISPOR US), and the American Association for Cancer Research (AACR). A search of ClinicalTrials.gov was conducted to validate the findings from the database and grey literature searches.

Study selection, data extraction & weighted average calculation

Studies were screened for inclusion using DistillerSR Version 2.35 and 2.41 (DistillerSR Inc. 2021, Ottawa, Canada) by two independent reviewers based on the prespecified PICOS criteria (Table  1 ). A third reviewer was consulted to resolve any discrepancies during the screening process. Studies were included if they reported RWE on patients aged ≥ 18 years with HR+/HER2- LABC/mBC who received 1 L CDK4/6i treatment and received subsequent treatment in 2 L+. Studies were excluded if they reported the results of clinical trials (i.e., non-RWE), were published in any language other than English, and/or were published prior to 2015 (or prior to 2019 for conference abstracts and posters). For studies that met the eligibility criteria, data relating to study design and methodology, details of interventions, patient eligibility criteria and baseline characteristics, and outcome measures such as efficacy, safety, tolerability, and patient-reported outcomes (PROs), were extracted (as available) using a Microsoft Excel®-based data extraction form (Microsoft Corporation, WA, USA). Data extraction was performed by a single reviewer and was confirmed by a second reviewer. Multiple publications identified for the same RWE study, patient population, and setting that reported data for the same intervention were linked and extracted as a single publication. Weighted average median real-world progression-free survival (rwPFS) values were calculated by considering the contribution to the median rwPFS of each study proportional to its respective sample size. These weighted values were then used to compute the overall median rwPFS estimate.

Quality assessment

The Newcastle-Ottawa scale (NOS) for nonrandomized (cohort) studies was used to assess the risk of bias for published, full-text studies [ 36 ]. The NOS allocates a maximum of nine points for the least risk of bias across three domains: (1) Formation of study groups (four points), (2) Comparability between study groups (two points), (3) Outcome ascertainment (three points). NOS scores can be categorized in three groups: very high risk of bias (0 to 3 points), high risk of bias (4 to 6), and low risk of bias (7 to 9) [ 37 ]. Risk of bias assessment was performed by one reviewer and validated by a second independent reviewer to verify accuracy. Due to limited methodological data by which to assess study quality, risk of bias assessment was not performed on conference abstracts or posters. An amendment to the PROSPERO record (CRD42023383914) for this study was submitted in relation to the quality assessment method (specifying usage of the NOS).

The database search identified 3,377 records; after removal of duplicates, 2,759 were screened at the title and abstract stage of which 2,553 were excluded. Out of the 206 reports retrieved and assessed for eligibility, an additional 187 records were excluded after full-text review; most of these studies were excluded for having patients with mixed lines of CDK4/6i treatment (i.e., did not receive CDK4/6i exclusively in 1 L) (Fig.  1 and Table S1 ). The grey literature search identified 753 records which were assessed for eligibility; of which 752 were excluded mainly due to the population not meeting the eligibility criteria (Fig.  1 ). In total, the literature searches identified 20 records (9 published full-text articles and 11 conference abstracts/posters) representing 18 unique RWE studies that met the inclusion criteria. The NOS quality scores for the included full-text articles are provided in Table S2 . The scores ranged from four to six points (out of a total score of nine) and the median score was five, indicating that all the studies suffered from a high risk of bias [ 37 ].

Most studies were retrospective analyses of chart reviews or medical registries, and all studies were published between 2017 and 2022 (Table S3 ). Nearly half of the RWE studies (8 out of 18 studies) were conducted in the US [ 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 ], while the remaining studies included sites in Canada, China, Germany, Italy, Japan, and the United Kingdom [ 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 ]. Sample sizes ranged from as few as 4 to as many as 839 patients across included studies, with patient age ranging from 26 to 86 years old.

Although treatment characteristics in the 1 L setting were not the focus of the present review, these details are captured in Table S3 . Briefly, several RWE studies reported 1 L CDK4/6i use in combination with ET (8 out of 18 studies) or as monotherapy (2 out of 18 studies) (Table S3 ). Treatments used in combination with 1 L CDK4/6i included letrozole, fulvestrant, exemestane, and anastrozole. Where reported (4 out of 18 studies), palbociclib was the most common 1 L CDK4/6i treatment. Many studies (8 out of 18 studies) did not report which specific CDK4/6i treatment(s) were used in 1 L or if its administration was in combination or monotherapy.

Characteristics of treatments after 1 L CDK4/6i therapy

Across all studies included in this review, effectiveness and safety data were only available for treatments administered in the 2 L setting after 1 L CDK4/6i treatment. No studies were identified that reported outcomes for patients treated in the third-line setting or beyond after 1 L CDK4/6i treatment. All 18 studies reported effectiveness outcomes in 2 L, with only two of these studies also describing 2 L safety outcomes. The distribution of outcomes reported in these studies is provided in Table S4 . Studies varied in their reporting of outcomes for 2 L treatments; some studies reported outcomes for a group of 2 L treatments while others described independent outcomes for specific 2 L treatments (i.e., everolimus, fulvestrant, or chemotherapy agents such as eribulin mesylate) [ 42 , 45 , 50 , 54 , 55 ]. Due to the heterogeneity in treatment classes reported in these studies, this data was categorized (as described below) to align with the guidelines provided by NCCN and ESMO [ 15 , 16 ]. The treatment class categorizations for the purpose of this review are: single-agent ET (patients who exclusively received a single-agent ET after 1 L CDK4/6i treatment), mTORi ± ET (patients who exclusively received an mTORi with or without ET after 1 L CDK4/6i treatment), mix of ET and/or mTORi (patients who may have received only ET, only mTORi, and/or both treatments but the studies in this group lacked sufficient information to categorize these patients in the “single-agent ET” or “mTOR ± ET” categories), and chemotherapy (patients who exclusively received chemotherapy after 1 L CDK4/6i treatment). Despite ESMO and NCCN guidelines indicating that limited evidence exists to support rechallenge with CDK4/6i after 1 L CDK4/6i treatment [ 15 , 16 ], two studies reported outcomes for this treatment approach. Data for such patients were categorized as “ CDK4/6i ± ET ” as it was unclear how many patients receiving CDK4/6i rechallenge received concurrent ET. All other patient groups that lacked sufficient information or did not report outcome/safety data independently (i.e., grouped patients with mixed treatments) to categorize as one of the treatment classes described above were grouped as “ other ”.

The majority of studies reported effectiveness outcomes for endocrine-based therapy after 1 L CDK4/6i treatment; five studies for single-agent ET, six studies for mTORi ± ET, and three studies for a mix of ET and/or mTORi (Fig.  2 ). Eleven studies reported effectiveness outcomes for chemotherapy after 1 L CDK4/6i treatment, and only two studies reported effectiveness outcomes for CDK4/6i rechallenge ± ET. Eight studies that described effectiveness outcomes were grouped into the “other” category. Safety data was only reported in two studies: one study evaluating the chemotherapy agent eribulin mesylate and one evaluating the mTORi everolimus.

Effectiveness outcomes

Real-world progression-free survival

Median rwPFS was described in 13 studies (Tables  2 and Table S5 ). Across the 13 studies, the median rwPFS ranged from 2.5 months [ 49 ] to 17.3 months [ 39 ]. Out of the 13 studies reporting median rwPFS, 10 studies reported median rwPFS for a 2 L treatment recommended by ESMO and NCCN guidelines, which ranged from 2.5 months [ 49 ] to 9.7 months [ 45 ].

Weighted average median rwPFS was calculated for 2 L treatments recommended by both ESMO and NCCN guidelines (Fig.  3 ). The weighted average median rwPFS for single-agent ET was 3.9 months ( n  = 92 total patients) and was derived using data from two studies reporting median rwPFS values of 3.3 months ( n  = 70) [ 38 ] and 6.0 months ( n  = 22) [ 40 ]. For one study ( n  = 7) that reported outcomes for single agent ET, median rwPFS was not reached during the follow-up period; as such, this study was excluded from the weighted average median rwPFS calculation [ 49 ].

The weighted average median rwPFS for mTORi ± ET was 3.6 months ( n  = 128 total patients) and was derived based on data from 3 studies with median rwPFS ranging from 2.5 months ( n  = 4) [ 49 ] to 4.9 months ( n  = 25) [ 54 ] (Fig.  3 ). For patients who received a mix of ET and/or mTORi but could not be classified into the single-agent ET or mTORi ± ET treatment classes, the weighted average median rwPFS was calculated to be 3.7 months ( n  = 17 total patients). This was calculated based on data from two studies reporting median rwPFS values of 3.0 months ( n  = 5) [ 46 ] and 4.0 months ( n  = 12) [ 49 ]. Notably, one study of patients receiving ET and/or everolimus reported a median rwPFS duration of 3.0 months; however, this study was excluded from the weighted average median rwPFS calculation for the ET and/or mTORi class as the sample size was not reported [ 53 ].

The weighted average median rwPFS for chemotherapy was 6.1 months ( n  = 499 total patients), calculated using data from 7 studies reporting median rwPFS values ranging from 3.7 months ( n  = 249) [ 38 ] to 9.7 months ( n  = 121) [ 45 ] (Fig.  3 ). One study with a median rwPFS duration of 5.6 months was not included in the weighted average median rwPFS calculation as the study did not report the sample size [ 53 ]. A second study was excluded from the calculation since the reported median rwPFS was not reached during the study period ( n  = 7) [ 41 ].

Although 2 L CDK4/6i ± ET rechallenge lacks sufficient information to support recommendation by ESMO and NCCN guidelines, the limited data currently available for this treatment have shown promising results. Briefly, two studies reported median rwPFS for CDK4/6i ± ET with values of 8.3 months ( n  = 302) [ 38 ] and 17.3 months ( n  = 165) (Table  2 ) [ 39 ]. The remaining median rwPFS studies reported data for patients classified as “Other” (Table S5 ). The “Other” category included median rwPFS outcomes from seven studies, and included a myriad of treatments (e.g., ET, mTOR + ET, chemotherapy, CDK4/6i + ET, alpelisib + fulvestrant, chidamide + ET) for which disaggregated median rwPFS values were not reported.

Overall survival

Median OS for 2 L treatment was reported in only three studies (Table  2 ) [ 38 , 42 , 43 ]. Across the three studies, the 2 L median OS ranged from 5.2 months ( n  = 3) [ 43 ] to 35.7 months ( n  = 302) [ 38 ]. Due to the lack of OS data in most of the studies, weighted averages could not be calculated. No median OS data was reported for the single-agent ET treatment class whereas two studies reported median OS for the mTORi ± ET treatment class, ranging from 5.2 months ( n  = 3) [ 43 ] to 21.8 months ( n  = 54) [ 42 ]. One study reported 2 L median OS of 24.8 months for a single patient treated with chemotherapy [ 43 ]. The median OS data in the CDK4/6i ± ET rechallenge group was 35.7 months ( n  = 302) [ 38 ].

Patient mortality was reported in three studies [ 43 , 44 , 45 ]. No studies reported mortality for the single-agent ET treatment class and only one study reported this outcome for the mTORi ± ET treatment class, where 100% of patients died ( n  = 3) as a result of rapid disease progression [ 43 ]. For the chemotherapy class, one study reported mortality for one patient receiving 2 L capecitabine [ 43 ]. An additional study reported eight deaths (21.7%) following 1 L CDK4/6i treatment; however, this study did not disclose the 2 L treatments administered to these patients [ 44 ].

Other clinical endpoints

The studies included limited information on additional clinical endpoints; two studies reported on time-to-discontinuation (TTD), two reported on duration of response (DOR), and one each on time-to-next-treatment (TTNT), time-to-progression (TTP), objective response rate (ORR), clinical benefit rate (CBR), and stable disease (Tables  2 and Table S5 ).

Safety, tolerability, and patient-reported outcomes

Safety and tolerability data were reported in two studies [ 40 , 45 ]. One study investigating 2 L administration of the chemotherapy agent eribulin mesylate reported 27 patients (22.3%) with neutropenia, 3 patients (2.5%) with febrile neutropenia, 10 patients (8.3%) with peripheral neuropathy, and 14 patients (11.6%) with diarrhea [ 45 ]. Of these, neutropenia of grade 3–4 severity occurred in 9 patients (33.3%) [ 45 ]. A total of 55 patients (45.5%) discontinued eribulin mesylate treatment; 1 patient (0.83%) discontinued treatment due to adverse events [ 45 ]. Another study reported that 5 out of the 22 patients receiving the mTORi everolimus combined with ET in 2 L (22.7%) discontinued treatment due to toxicity [ 40 ]. PROs were not reported in any of the studies included in the SLR.

The objective of this study was to summarize the existing RWE on the effectiveness and safety of therapies for patients with HR+/HER2- LABC/mBC after 1 L CDK4/6i treatment. We identified 18 unique studies reporting specifically on 2 L treatment regimens after 1 L CDK4/6i treatment. The weighted average median rwPFS for NCCN- and ESMO- guideline recommended 2 L treatments ranged from 3.6 to 3.9 months for ET-based treatments and was 6.1 months when including chemotherapy-based regimens. Treatment selection following 1 L CDK4/6i therapy remains challenging primarily due to the suboptimal effectiveness or significant toxicities (e.g., chemotherapy) associated with currently available options [ 56 ]. These results highlight that currently available 2 L treatments for patients with HR+/HER2- LABC/mBC who have received 1 L CDK4/6i are suboptimal, as evidenced by the brief median rwPFS duration associated with ET-based treatments, or notable side effects and toxicity linked to chemotherapy. This conclusion is aligned with a recent review highlighting the limited effectiveness of treatment options for HR+/HER2- LABC/mBC patients post-CDK4/6i treatment [ 56 , 57 ]. Registrational trials which have also shed light on the short median PFS of 2–3 months achieved by ET (i.e., fulvestrant) after 1 L CDK4/6i therapy emphasize the need to develop improved treatment strategies aimed at prolonging the duration of effective ET-based treatment [ 56 ].

The results of this review reveal a paucity of additional real-world effectiveness and safety evidence after 1 L CDK4/6i treatment in HR+/HER2- LABC/mBC. OS and DOR were only reported in two studies while other clinical endpoints (i.e., TTD, TTNT, TTP, ORR, CBR, and stable disease) were only reported in one study each. Similarly, safety and tolerability data were only reported in two studies each, and PROs were not reported in any study. This hindered our ability to provide a comprehensive assessment of real-world treatment effectiveness and safety following 1 L CDK4/6i treatment. The limited evidence may be due to the relatively short period of time that has elapsed since CDK4/6i first received US FDA approval for 1 L treatment of HR+/HER2- LABC/mBC (2015) [ 35 ]. As such, almost half of our evidence was informed by conference abstracts. Similarly, no real-world studies were identified in our review that reported outcomes for treatments in the third- or later-lines of therapy after 1 L CDK4/6i treatment. The lack of data in this patient population highlights a significant gap which limits our understanding of the effectiveness and safety for patients receiving later lines of therapy. As more patients receive CDK4/6i therapy in the 1 L setting, the number of patients requiring subsequent lines of therapy will continue to grow. Addressing this data gap over time will be critical to improve outcomes for patients with HR+/HER2- LABC/mBC following 1 L CDK4/6i therapy.

There are several strengths of this study, including adherence to the guidelines outlined in the Cochrane Handbook to ensure a standardized and reliable approach to the SLR [ 58 ] and reporting of the SLR following PRISMA guidelines to ensure transparency and reproducibility [ 59 ]. Furthermore, the inclusion of only RWE studies allowed us to assess the effectiveness of current standard of care treatments outside of a controlled environment and enabled us to identify an unmet need in this patient population.

This study had some notable limitations, including the lack of safety and additional effectiveness outcomes reported. In addition, the dearth of studies reporting PROs is a limitation, as PROs provide valuable insight into the patient experience and are an important aspect of assessing the impact of 2 L treatments on patients’ quality of life. The studies included in this review also lacked consistent reporting of clinical characteristics (e.g., menopausal status, sites of metastasis, prior surgery) making it challenging to draw comprehensive conclusions or comparisons based on these factors across the studies. Taken together, there exists an important gap in our understanding of the long-term management of patients with HR+/HER2- LABC/mBC. Additionally, the effectiveness results reported in our evidence base were informed by small sample sizes; many of the included studies reported median rwPFS based on less than 30 patients [ 39 , 40 , 41 , 46 , 49 , 51 , 60 ], with two studies not reporting the sample size at all [ 47 , 53 ]. This may impact the generalizability and robustness of the results. Relatedly, the SLR database search was conducted in December 2022; as such, novel agents (e.g., elacestrant and capivasertib + fulvestrant) that have since received FDA approval for the treatment of HR+/HER2- LABC/mBC may impact current 2 L rwPFS outcomes [ 61 , 62 ]. Finally, relative to the number of peer-reviewed full-text articles, this SLR identified eight abstracts and one poster presentation, comprising half (50%) of the included unique studies. As conference abstracts are inherently limited by how much content that can be described due to word limit constraints, this likely had implications on the present synthesis whereby we identified a dearth of real-world effectiveness outcomes in patients with HR+/HER2- LABC/mBC treated with 1 L CDK4/6i therapy.

Future research in this area should aim to address the limitations of the current literature and provide a more comprehensive understanding of optimal sequencing of effective and safe treatment for patients following 1 L CDK4/6i therapy. Specifically, future studies should strive to report robust data related to effectiveness, safety, and PROs for patients receiving 2 L treatment after 1 L CDK4/6i therapy. Future studies should also aim to understand the mechanism underlying CDK4/6i resistance. Addressing these gaps in knowledge may improve the long-term real-world management of patients with HR+/HER2- LABC/mBC. A future update of this synthesis may serve to capture a wider breadth of full-text, peer-reviewed articles to gain a more robust understanding of the safety, effectiveness, and real-world treatment patterns for patients with HR+/HER2- LABC/mBC. This SLR underscores the necessity for ongoing investigation and the development of innovative therapeutic approaches to address these gaps and improve patient outcomes.

This SLR qualitatively summarized the existing real-world effectiveness data for patients with HR+/HER2- LABC/mBC after 1 L CDK4/6i treatment. Results of this study highlight the limited available data and the suboptimal effectiveness of treatments employed in the 2 L setting and underscore the unmet need in this patient population. Additional studies reporting effectiveness and safety outcomes, in addition to PROs, for this patient population are necessary and should be the focus of future research.

PRISMA flow diagram. *Two included conference abstracts reported the same information as already included full-text reports, hence both conference abstracts were not identified as unique. Abbreviations: 1 L = first-line; AACR = American Association of Cancer Research; ASCO = American Society of Clinical Oncology; CDK4/6i = cyclin-dependent kinase 4/6 inhibitor; ESMO = European Society for Medical Oncology; ISPOR = Professional Society for Health Economics and Outcomes Research; n = number of studies; NMA = network meta-analysis; pts = participants; SABCS = San Antonio Breast Cancer Symposium; SLR = systematic literature review.

Number of studies reporting effectiveness outcomes exclusively for each treatment class. *Studies that lack sufficient information on effectiveness outcomes to classify based on the treatment classes outlined in the legend above. Abbreviations: CDK4/6i = cyclin-dependent kinase 4/6 inhibitor; ET = endocrine therapy; mTORi = mammalian target of rapamycin inhibitor.

Weighted average median rwPFS for 2 L treatments (recommended in ESMO/NCCN guidelines) after 1 L CDK4/6i treatment. Circular dot represents weighted average median across studies. Horizontal bars represent the range of values reported in these studies. Abbreviations: CDK4/6i = cyclin-dependent kinase 4/6 inhibitor; ESMO = European Society for Medical Oncology; ET = endocrine therapy, mTORi = mammalian target of rapamycin inhibitor; n = number of patients; NCCN = National Comprehensive Cancer Network; rwPFS = real-world progression-free survival.

Data availability

All data generated or analyzed during this study are included in this published article [and its supplementary information files]. This study is registered with PROSPERO (CRD42023383914).

Abbreviations

Second-line

Second-line treatment setting and beyond

American Association of Cancer Research

Aromatase inhibitor

American Society of Clinical Oncology

• Breast cancer

breast cancer gene/partner and localizer of BRCA2 positive

Clinical benefit rate

Cyclin-dependent kinase 4/6 inhibitor

Complete response

Duration of response

European Society for Medical Oncology

Food and Drug Administration

Human epidermal growth factor receptor 2

Human epidermal growth factor receptor 2 negative

Hormone receptor

Hormone receptor positive

Professional Society for Health Economics and Outcomes Research

Locally advanced breast cancer

Metastatic breast cancer

Medical Literature Analysis and Retrieval System Online

Medical subject headings

Mammalian target of rapamycin inhibitor

National Comprehensive Cancer Network

Newcastle Ottawa Scale

Objective response rate

Poly-ADP ribose polymerase inhibitor

Progression-free survival

Population, Intervention, Comparator, Outcome, Study Design

Partial response

Preferred Reporting Items for Systematic Literature Reviews and Meta-Analyses

Patient-reported outcomes

• Real-world evidence

San Antonio Breast Cancer Symposium

• Systematic literature review

Time-to-discontinuation

Time-to-next-treatment

Time-to-progression

United States

Łukasiewicz S, Czeczelewski M, Forma A, Baj J, Sitarz R, Stanisławek A, Breast, Cancer—Epidemiology. Risk factors, classification, prognostic markers, and current treatment Strategies—An. Updated Rev Cancers. 2021;13(17):4287.

Google Scholar  

World Health Organization (WHO). Breast Cancer Facts Sheet [updated July 12 2023. https://www.who.int/news-room/fact-sheets/detail/breast-cancer .

Arnold M, Morgan E, Rumgay H, Mafra A, Singh D, Laversanne M, et al. Current and future burden of breast cancer: global statistics for 2020 and 2040. Breast. 2022;66:15–23.

Article   PubMed   PubMed Central   Google Scholar  

Wilkinson L, Gathani T. Understanding breast cancer as a global health concern. Br J Radiol. 2022;95(1130):20211033.

Article   PubMed   Google Scholar  

Giaquinto AN, Sung H, Miller KD, Kramer JL, Newman LA, Minihan A et al. Breast Cancer Statistics, 2022. CA: A Cancer Journal for Clinicians. 2022;72(6):524– 41.

National Cancer Institute (NIH). Cancer Stat Facts: Female Breast Cancer [updated 2020. https://seer.cancer.gov/statfacts/html/breast.html .

American Cancer Society. Key Statistics for Breast Cancer [ https://www.cancer.org/cancer/types/breast-cancer/about/how-common-is-breast-cancer.html .

Zagami P, Carey LA. Triple negative breast cancer: pitfalls and progress. npj Breast Cancer. 2022;8(1):95.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Matutino A, Joy AA, Brezden-Masley C, Chia S, Verma S. Hormone receptor-positive, HER2-negative metastatic breast cancer: redrawing the lines. Curr Oncol. 2018;25(Suppl 1):S131–41.

Lloyd MR, Wander SA, Hamilton E, Razavi P, Bardia A. Next-generation selective estrogen receptor degraders and other novel endocrine therapies for management of metastatic hormone receptor-positive breast cancer: current and emerging role. Ther Adv Med Oncol. 2022;14:17588359221113694.

Cardoso F, Senkus E, Costa A, Papadopoulos E, Aapro M, André F, et al. 4th ESO-ESMO International Consensus guidelines for advanced breast Cancer (ABC 4)†. Ann Oncol. 2018;29(8):1634–57.

Article   CAS   PubMed   Google Scholar  

US Food Drug Administration. Palbociclib (Ibrance) 2017 [updated March 31, 2017. https://www.fda.gov/drugs/resources-information-approved-drugs/palbociclib-ibrance .

US Food Drug Administration. FDA expands ribociclib indication in HR-positive, HER2-negative advanced or metastatic breast cancer 2018 [updated July 18. 2018. https://www.fda.gov/drugs/resources-information-approved-drugs/fda-expands-ribociclib-indication-hr-positive-her2-negative-advanced-or-metastatic-breast-cancer .

US Food Drug Administration. FDA approves abemaciclib for HR positive, HER2-negative breast cancer 2017 [updated Sept 28. 2017. https://www.fda.gov/drugs/resources-information-approved-drugs/fda-approves-abemaciclib-hr-positive-her2-negative-breast-cancer .

NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines®). Breast Cancer 2022 [ https://www.nccn.org/professionals/physician_gls/pdf/breast.pdf .

Gennari A, André F, Barrios CH, Cortés J, de Azambuja E, DeMichele A, et al. ESMO Clinical Practice Guideline for the diagnosis, staging and treatment of patients with metastatic breast cancer. Ann Oncol. 2021;32(12):1475–95.

Beaver JA, Amiri-Kordestani L, Charlab R, Chen W, Palmby T, Tilley A, et al. FDA approval: Palbociclib for the Treatment of Postmenopausal Patients with estrogen Receptor-Positive, HER2-Negative metastatic breast Cancer. Clin Cancer Res. 2015;21(21):4760–6.

US Food Drug Administration. Ribociclib (Kisqali) [ https://www.fda.gov/drugs/resources-information-approved-drugs/ribociclib-kisqali#:~:text=On%20March%2013%2C%202017%2C%20the,hormone%20receptor%20(HR)%2Dpositive%2C .

US Food Drug Administration. FDA approves new treatment for certain advanced or metastatic breast cancers [ https://www.fda.gov/news-events/press-announcements/fda-approves-new-treatment-certain-advanced-or-metastatic-breast-cancers .

US Food Drug Administration. FDA expands ribociclib indication in HR-positive, HER2-negative advanced or metastatic breast cancer. 2018 [ https://www.fda.gov/drugs/resources-information-approved-drugs/fda-expands-ribociclib-indication-hr-positive-her2-negative-advanced-or-metastatic-breast-cancer .

US Food Drug Administration. FDA approves abemaciclib as initial therapy for HR-positive, HER2-negative metastatic breast cancer [ https://www.fda.gov/drugs/resources-information-approved-drugs/fda-approves-abemaciclib-initial-therapy-hr-positive-her2-negative-metastatic-breast-cancer .

Turner NC, Slamon DJ, Ro J, Bondarenko I, Im S-A, Masuda N, et al. Overall survival with Palbociclib and fulvestrant in advanced breast Cancer. N Engl J Med. 2018;379(20):1926–36.

Slamon DJ, Neven P, Chia S, Fasching PA, De Laurentiis M, Im SA, et al. Phase III randomized study of Ribociclib and Fulvestrant in hormone Receptor-Positive, human epidermal growth factor receptor 2-Negative advanced breast Cancer: MONALEESA-3. J Clin Oncol. 2018;36(24):2465–72.

Goetz MP, Toi M, Campone M, Sohn J, Paluch-Shimon S, Huober J, et al. MONARCH 3: Abemaciclib as initial therapy for advanced breast Cancer. J Clin Oncol. 2017;35(32):3638–46.

Gopalan PK, Villegas AG, Cao C, Pinder-Schenck M, Chiappori A, Hou W, et al. CDK4/6 inhibition stabilizes disease in patients with p16-null non-small cell lung cancer and is synergistic with mTOR inhibition. Oncotarget. 2018;9(100):37352–66.

Watt AC, Goel S. Cellular mechanisms underlying response and resistance to CDK4/6 inhibitors in the treatment of hormone receptor-positive breast cancer. Breast Cancer Res. 2022;24(1):17.

Goetz M. MONARCH 3: final overall survival results of abemaciclib plus a nonsteroidal aromatase inhibitor as first-line therapy for HR+, HER2- advanced breast cancer. SABCS; 2023.

Munzone E, Pagan E, Bagnardi V, Montagna E, Cancello G, Dellapasqua S, et al. Systematic review and meta-analysis of post-progression outcomes in ER+/HER2– metastatic breast cancer after CDK4/6 inhibitors within randomized clinical trials. ESMO Open. 2021;6(6):100332.

Gennari A, André F, Barrios CH, Cortés J, de Azambuja E, DeMichele A, et al. ESMO Clinical Practice Guideline for the diagnosis, staging and treatment of patients with metastatic breast cancer. Annals of Oncology. 2021;32(12):1475-95.

European Society for Medical Oncology (ESMO). ESMO ​Metastatic Breast Cancer Living Guideline: ER-positive HER2-negative​ Breast Cancer​ [updated May 2023. https://www.esmo.org/living-guidelines/esmo-metastatic-breast-cancer-living-guideline/er-positive-her2-negative-breast-cancer .

Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Welch PM VA, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.2 (updated February 2021). www.training.cochrane.org/handbook : Cochrane; 2021.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. PLoS Med. 2021;18(3):e1003583.

McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS peer review of electronic search strategies: 2015 Guideline Statement. J Clin Epidemiol. 2016;75:40–6.

Fraser C, Murray A, Burr J. Identifying observational studies of surgical interventions in MEDLINE and EMBASE. BMC Med Res Methodol. 2006;6(1):41.

US Food Drug Administration. Palbociclib (Ibrance). Silver Spring, MD: US Food and Drug Administration; 2017.

Book   Google Scholar  

GA Wells BS, D O’Connell J, Peterson V, Welch M, Losos PT. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses [ https://www.ohri.ca/programs/clinical_epidemiology/oxford.asp .

Lo CK-L, Mertz D, Loeb M. Newcastle-Ottawa Scale: comparing reviewers’ to authors’ assessments. BMC Med Res Methodol. 2014;14(1):45.

Martin JM, Handorf EA, Montero AJ, Goldstein LJ. Systemic therapies following progression on first-line CDK4/6-inhibitor treatment: analysis of real-world data. Oncologist. 2022;27(6):441–6.

Kalinsky KM, Kruse M, Smyth EN, Guimaraes CM, Gautam S, Nisbett AR et al. Abstract P1-18-37: Treatment patterns and outcomes associated with sequential and non-sequential use of CDK4 and 6i for HR+, HER2- MBC in the real world. Cancer Research. 2022;82(4_Supplement):P1-18-37-P1-18-37.

Choong GM, Liddell S, Ferre RAL, O’Sullivan CC, Ruddy KJ, Haddad TC, et al. Clinical management of metastatic hormone receptor-positive, HER2-negative breast cancer (MBC) after CDK 4/6 inhibitors: a retrospective single-institution study. Breast Cancer Res Treat. 2022;196(1):229–37.

Xi J, Oza A, Thomas S, Ademuyiwa F, Weilbaecher K, Suresh R, et al. Retrospective Analysis of Treatment Patterns and effectiveness of Palbociclib and subsequent regimens in metastatic breast Cancer. J Natl Compr Canc Netw. 2019;17(2):141–7.

Rozenblit M, Mun S, Soulos P, Adelson K, Pusztai L, Mougalian S. Patterns of treatment with everolimus exemestane in hormone receptor-positive HER2-negative metastatic breast cancer in the era of targeted therapy. Breast Cancer Res. 2021;23(1):14.

Bashour SI, Doostan I, Keyomarsi K, Valero V, Ueno NT, Brown PH, et al. Rapid breast Cancer Disease Progression following cyclin dependent kinase 4 and 6 inhibitor discontinuation. J Cancer. 2017;8(11):2004–9.

Giridhar KV, Choong GM, Leon-Ferre R, O’Sullivan CC, Ruddy K, Haddad T, et al. Abstract P6-18-09: clinical management of metastatic breast cancer (MBC) after CDK 4/6 inhibitors: a retrospective single-institution study. Cancer Res. 2019;79:P6–18.

Article   Google Scholar  

Mougalian SS, Feinberg BA, Wang E, Alexis K, Chatterjee D, Knoth RL, et al. Observational study of clinical outcomes of eribulin mesylate in metastatic breast cancer after cyclin-dependent kinase 4/6 inhibitor therapy. Future Oncol. 2019;15(34):3935–44.

Moscetti LML, Riggi L, Sperduti I, Piacentini FOC, Toss A, Barbieri E, Cortesi L, Canino FMA, Zoppoli G, Frassoldati A, Schirone A, Dominici MECF. SEQUENCE OF TREATMENTS AFTER CDK4/6 THERAPY IN ADVANCED BREAST CANCER (ABC), A GOIRC MULTICENTER RETRO/ PROSPECTIVE STUDY. PRELIMINARY RESULTS IN THE RETROSPECTIVE SERIES OF 116 PATIENTS. Tumori. 2022;108(4S):80.

Menichetti AZE, Giorgi CA, Bottosso M, Leporati R, Giarratano T, Barbieri C, Ligorio F, Mioranza E, Miglietta F, Lobefaro R, Faggioni G, Falci C, Vernaci G, Di Liso E, Girardi F, Griguolo G, Vernieri C, Guarneri V, Dieci MV. CDK 4/6 INHIBITORS FOR METASTATIC BREAST CANCER: A MULTICENTER REALWORLD STUDY. Tumori. 2022;108(4S):70.

Marschner NW, Harbeck N, Thill M, Stickeler E, Zaiss M, Nusch A, et al. 232P Second-line therapies of patients with early progression under CDK4/6-inhibitor in first-line– data from the registry platform OPAL. Annals of Oncology. 2022;33:S643-S4

Gousis C, Lowe KMH, Kapiris M. V. Angelis. Beyond First Line CDK4/6 Inhibitors (CDK4/6i) and Aromatase Inhibitors (AI) in Patients with Oestrogen Receptor Positive Metastatic Breast Cancer (ERD MBC): The Guy’s Cancer Centre Experience. Clinical Oncology2022. p. e178.

Endo Y, Yoshimura A, Sawaki M, Hattori M, Kotani H, Kataoka A, et al. Time to chemotherapy for patients with estrogen receptor-positive breast Cancer and cyclin-dependent kinase 4 and 6 inhibitor use. J Breast Cancer. 2022;25(4):296–306.

Li Y, Li W, Gong C, Zheng Y, Ouyang Q, Xie N, et al. A multicenter analysis of treatment patterns and clinical outcomes of subsequent therapies after progression on palbociclib in HR+/HER2- metastatic breast cancer. Ther Adv Med Oncol. 2021;13:17588359211022890.

Amaro CP, Batra A, Lupichuk S. First-line treatment with a cyclin-dependent kinase 4/6 inhibitor plus an aromatase inhibitor for metastatic breast Cancer in Alberta. Curr Oncol. 2021;28(3):2270–80.

Crocetti SPM, Tassone L, Marcantognini G, Bastianelli L, Della Mora A, Merloni F, Cantini L, Scortichini L, Agostinelli V, Ballatore Z, Savini A, Maccaroni E. Berardi R. What is the best therapeutic sequence for ER-Positive/HER2- Negative metastatic breast cancer in the era of CDK4/6 inhibitors? A single center experience. Tumori. 2020;106(2S).

Nichetti F, Marra A, Giorgi CA, Randon G, Scagnoli S, De Angelis C, et al. 337P Efficacy of everolimus plus exemestane in CDK 4/6 inhibitors-pretreated or naïve HR-positive/HER2-negative breast cancer patients: A secondary analysis of the EVERMET study. Annals of Oncology. 2020;31:S382

Luhn P, O’Hear C, Ton T, Sanglier T, Hsieh A, Oliveri D, et al. Abstract P4-13-08: time to treatment discontinuation of second-line fulvestrant monotherapy for HR+/HER2– metastatic breast cancer in the real-world setting. Cancer Res. 2019;79(4Supplement):P4–13.

Mittal A, Molto Valiente C, Tamimi F, Schlam I, Sammons S, Tolaney SM et al. Filling the gap after CDK4/6 inhibitors: Novel Endocrine and Biologic Treatment options for metastatic hormone receptor positive breast Cancer. Cancers (Basel). 2023;15(7).

Ashai N, Swain SM. Post-CDK 4/6 inhibitor therapy: current agents and novel targets. Cancers (Basel). 2023;15(6).

Higgins JPTTJ, Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated February 2022). www.training.cochrane.org/handbook : Cochrane; 2022.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71.

Serdar CC, Cihan M, Yücel D, Serdar MA. Sample size, power and effect size revisited: simplified and practical approaches in pre-clinical, clinical and laboratory studies. Biochem Med (Zagreb). 2021;31(1):010502.

US Food Drug Administration. FDA approves elacestrant for ER-positive, HER2-negative, ESR1-mutated advanced or metastatic breast cancer [updated January 27 2023. https://www.fda.gov/drugs/resources-information-approved-drugs/fda-approves-elacestrant-er-positive-her2-negative-esr1-mutated-advanced-or-metastatic-breast-cancer .

US Food Drug Administration. FDA approves capivasertib with fulvestrant for breast cancer [updated November 16 2023. https://www.fda.gov/drugs/resources-information-approved-drugs/fda-approves-capivasertib-fulvestrant-breast-cancer .

Download references

Acknowledgements

The authors would like to acknowledge Joanna Bielecki who developed, conducted, and documented the database searches.

This study was funded by Pfizer Inc. (New York, NY, USA) and Arvinas (New Haven, CT, USA).

Author information

Sarah Kane, Belal Howidi, Bao-Ngoc Nguyen and Imtiaz A. Samjoo contributed equally to this work.

Authors and Affiliations

Pfizer, 10017, New York, NY, USA

Veronique Lambert & Yan Wu

EVERSANA, Burlington, ON, Canada

Sarah Kane, Belal Howidi, Bao-Ngoc Nguyen & Imtiaz A. Samjoo

Arvinas, 06511, New Haven, CT, USA

David Chandiwana & Michelle Edwards

You can also search for this author in PubMed   Google Scholar

Contributions

VL, IAS, SK, BH, BN, DC, YW, and ME participated in the conception and design of the study. IAS, SK, BH and BN contributed to the literature review, data collection, analysis, and interpretation of the data. VL, IAS, SK, BH, BN, DC, YW, and ME contributed to the interpretation of the data and critically reviewed for the importance of intellectual content for the work. VL, IAS, SK, BH, BN, DC, YW, and ME were responsible for drafting or reviewing the manuscript and for providing final approval. VL, IAS, SK, BH, BN, DC, YW, and ME meet the International Committee of Medical Journal Editors (ICMJE) criteria for authorship for this article, take responsibility for the integrity of the work, and have given their approval for this version to be published.

Corresponding author

Correspondence to Imtiaz A. Samjoo .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors of this manuscript declare that the research presented was funded by Pfizer Inc. and Arvinas. While the support from Pfizer Inc. and Arvinas was instrumental in facilitating this research, the authors affirm that their interpretation of the data and the content of this manuscript were conducted independently and without bias to maintain the transparency and integrity of the research. IAS, SK, BH, and BN are employees of EVERSANA, Canada, which was a paid consultant to Pfizer in connection with the development of this manuscript.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Lambert, V., Kane, S., Howidi, B. et al. Systematic literature review of real-world evidence for treatments in HR+/HER2- second-line LABC/mBC after first-line treatment with CDK4/6i. BMC Cancer 24 , 631 (2024). https://doi.org/10.1186/s12885-024-12269-8

Download citation

Received : 26 January 2024

Accepted : 16 April 2024

Published : 23 May 2024

DOI : https://doi.org/10.1186/s12885-024-12269-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• First-line CDK4/6i

ISSN: 1471-2407

• Submission enquiries: [email protected]
• General enquiries: [email protected]

 Bibliometric-Systematic Literature Review (B-SLR) Method

The Bibliometric-Systematic Literature Review (B-SLR) is a method for designing systematic literature reviews grounded on the findings of bibliometric analyses. The B-SLR method is anchored upon three core tenets: transparency in the research process, reproducibility of the findings, and the potential for making a novel scholarly contribution.

Presenting the B-SLR Method

The Bibliometric-Systematic Literature Review (B-SLR) is a novel toolbox consisting of a 10-step process for conducting rigorous literature reviews in the management field.

Via the B-SLR method, researchers could receive guidance on the critical choices, possible challenges, and best practices at every phase of the review procedure, from crafting the idea to refining the final document.

Why use the B-SLR method?

The exponential growth of academic literature presents challenges for scholars seeking to keep up-to-date with the latest developments. The Bibliometric-Systematic Literature Review (B-SLR) method provides a solution, combining bibliometric techniques with systematic literature review practices. It strikes a balance between breadth and depth, accommodating the multidisciplinary nature of contemporary research. Based on VOSviewer software, the B-SLR method is a versatile toolkit crafted to meet various research aims, from comprehensive reviews to formulating theories, plotting future directions in various fields, bridging knowledge gaps, or integrating findings from different disciplines. 

T he B-SLR method is grounded in the core principles of critical analysis, timeliness, broad coverage, rigour, coherence, and contributions to knowledge advancement.

Resources available for the B-SLR m ethod

Please find below additional resources available for the B-SLR m e thod.

Resources for Authors

Publications adopting the B-SLR method

Youtube Channel

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List

How-to conduct a systematic literature review: A quick guide for computer science research

Angela carrera-rivera.

a Faculty of Engineering, Mondragon University

William Ochoa

Felix larrinaga.

b Design Innovation Center(DBZ), Mondragon University

Associated Data

• No data was used for the research described in the article.

Performing a literature review is a critical first step in research to understanding the state-of-the-art and identifying gaps and challenges in the field. A systematic literature review is a method which sets out a series of steps to methodically organize the review. In this paper, we present a guide designed for researchers and in particular early-stage researchers in the computer-science field. The contribution of the article is the following:

• • Clearly defined strategies to follow for a systematic literature review in computer science research, and
• • Algorithmic method to tackle a systematic literature review.

Graphical abstract

Specifications table

Method details

A Systematic Literature Review (SLR) is a research methodology to collect, identify, and critically analyze the available research studies (e.g., articles, conference proceedings, books, dissertations) through a systematic procedure [12] . An SLR updates the reader with current literature about a subject [6] . The goal is to review critical points of current knowledge on a topic about research questions to suggest areas for further examination [5] . Defining an “Initial Idea” or interest in a subject to be studied is the first step before starting the SLR. An early search of the relevant literature can help determine whether the topic is too broad to adequately cover in the time frame and whether it is necessary to narrow the focus. Reading some articles can assist in setting the direction for a formal review., and formulating a potential research question (e.g., how is semantics involved in Industry 4.0?) can further facilitate this process. Once the focus has been established, an SLR can be undertaken to find more specific studies related to the variables in this question. Although there are multiple approaches for performing an SLR ( [5] , [26] , [27] ), this work aims to provide a step-by-step and practical guide while citing useful examples for computer-science research. The methodology presented in this paper comprises two main phases: “Planning” described in section 2, and “Conducting” described in section 3, following the depiction of the graphical abstract.

Defining the protocol is the first step of an SLR since it describes the procedures involved in the review and acts as a log of the activities to be performed. Obtaining opinions from peers while developing the protocol, is encouraged to ensure the review's consistency and validity, and helps identify when modifications are necessary [20] . One final goal of the protocol is to ensure the replicability of the review.

Define PICOC and synonyms

The PICOC (Population, Intervention, Comparison, Outcome, and Context) criteria break down the SLR's objectives into searchable keywords and help formulate research questions [ 27 ]. PICOC is widely used in the medical and social sciences fields to encourage researchers to consider the components of the research questions [14] . Kitchenham & Charters [6] compiled the list of PICOC elements and their corresponding terms in computer science, as presented in Table 1 , which includes keywords derived from the PICOC elements. From that point on, it is essential to think of synonyms or “alike” terms that later can be used for building queries in the selected digital libraries. For instance, the keyword “context awareness” can also be linked to “context-aware”.

Planning Step 1 “Defining PICOC keywords and synonyms”.

Formulate research questions

Clearly defined research question(s) are the key elements which set the focus for study identification and data extraction [21] . These questions are formulated based on the PICOC criteria as presented in the example in Table 2 (PICOC keywords are underlined).

Research questions examples.

Select digital library sources

The validity of a study will depend on the proper selection of a database since it must adequately cover the area under investigation [19] . The Web of Science (WoS) is an international and multidisciplinary tool for accessing literature in science, technology, biomedicine, and other disciplines. Scopus is a database that today indexes 40,562 peer-reviewed journals, compared to 24,831 for WoS. Thus, Scopus is currently the largest existing multidisciplinary database. However, it may also be necessary to include sources relevant to computer science, such as EI Compendex, IEEE Xplore, and ACM. Table 3 compares the area of expertise of a selection of databases.

Planning Step 3 “Select digital libraries”. Description of digital libraries in computer science and software engineering.

Define inclusion and exclusion criteria

Authors should define the inclusion and exclusion criteria before conducting the review to prevent bias, although these can be adjusted later, if necessary. The selection of primary studies will depend on these criteria. Articles are included or excluded in this first selection based on abstract and primary bibliographic data. When unsure, the article is skimmed to further decide the relevance for the review. Table 4 sets out some criteria types with descriptions and examples.

Planning Step 4 “Define inclusion and exclusion criteria”. Examples of criteria type.

Define the Quality Assessment (QA) checklist

Assessing the quality of an article requires an artifact which describes how to perform a detailed assessment. A typical quality assessment is a checklist that contains multiple factors to evaluate. A numerical scale is used to assess the criteria and quantify the QA [22] . Zhou et al. [25] presented a detailed description of assessment criteria in software engineering, classified into four main aspects of study quality: Reporting, Rigor, Credibility, and Relevance. Each of these criteria can be evaluated using, for instance, a Likert-type scale [17] , as shown in Table 5 . It is essential to select the same scale for all criteria established on the quality assessment.

Planning Step 5 “Define QA assessment checklist”. Examples of QA scales and questions.

Define the “Data Extraction” form

The data extraction form represents the information necessary to answer the research questions established for the review. Synthesizing the articles is a crucial step when conducting research. Ramesh et al. [15] presented a classification scheme for computer science research, based on topics, research methods, and levels of analysis that can be used to categorize the articles selected. Classification methods and fields to consider when conducting a review are presented in Table 6 .

Planning Step 6 “Define data extraction form”. Examples of fields.

The data extraction must be relevant to the research questions, and the relationship to each of the questions should be included in the form. Kitchenham & Charters [6] presented more pertinent data that can be captured, such as conclusions, recommendations, strengths, and weaknesses. Although the data extraction form can be updated if more information is needed, this should be treated with caution since it can be time-consuming. It can therefore be helpful to first have a general background in the research topic to determine better data extraction criteria.

After defining the protocol, conducting the review requires following each of the steps previously described. Using tools can help simplify the performance of this task. Standard tools such as Excel or Google sheets allow multiple researchers to work collaboratively. Another online tool specifically designed for performing SLRs is Parsif.al 1 . This tool allows researchers, especially in the context of software engineering, to define goals and objectives, import articles using BibTeX files, eliminate duplicates, define selection criteria, and generate reports.

Build digital library search strings

Search strings are built considering the PICOC elements and synonyms to execute the search in each database library. A search string should separate the synonyms with the boolean operator OR. In comparison, the PICOC elements are separated with parentheses and the boolean operator AND. An example is presented next:

(“Smart Manufacturing” OR “Digital Manufacturing” OR “Smart Factory”) AND (“Business Process Management” OR “BPEL” OR “BPM” OR “BPMN”) AND (“Semantic Web” OR “Ontology” OR “Semantic” OR “Semantic Web Service”) AND (“Framework” OR “Extension” OR “Plugin” OR “Tool”

Gather studies

Databases that feature advanced searches enable researchers to perform search queries based on titles, abstracts, and keywords, as well as for years or areas of research. Fig. 1 presents the example of an advanced search in Scopus, using titles, abstracts, and keywords (TITLE-ABS-KEY). Most of the databases allow the use of logical operators (i.e., AND, OR). In the example, the search is for “BIG DATA” and “USER EXPERIENCE” or “UX” as a synonym.

Example of Advanced search on Scopus.

In general, bibliometric data of articles can be exported from the databases as a comma-separated-value file (CSV) or BibTeX file, which is helpful for data extraction and quantitative and qualitative analysis. In addition, researchers should take advantage of reference-management software such as Zotero, Mendeley, Endnote, or Jabref, which import bibliographic information onto the software easily.

Study Selection and Refinement

The first step in this stage is to identify any duplicates that appear in the different searches in the selected databases. Some automatic procedures, tools like Excel formulas, or programming languages (i.e., Python) can be convenient here.

In the second step, articles are included or excluded according to the selection criteria, mainly by reading titles and abstracts. Finally, the quality is assessed using the predefined scale. Fig. 2 shows an example of an article QA evaluation in Parsif.al, using a simple scale. In this scenario, the scoring procedure is the following YES= 1, PARTIALLY= 0.5, and NO or UNKNOWN = 0 . A cut-off score should be defined to filter those articles that do not pass the QA. The QA will require a light review of the full text of the article.

Performing quality assessment (QA) in Parsif.al.

Data extraction

Those articles that pass the study selection are then thoroughly and critically read. Next, the researcher completes the information required using the “data extraction” form, as illustrated in Fig. 3 , in this scenario using Parsif.al tool.

Example of data extraction form using Parsif.al.

The information required (study characteristics and findings) from each included study must be acquired and documented through careful reading. Data extraction is valuable, especially if the data requires manipulation or assumptions and inferences. Thus, information can be synthesized from the extracted data for qualitative or quantitative analysis [16] . This documentation supports clarity, precise reporting, and the ability to scrutinize and replicate the examination.

Analysis and Report

The analysis phase examines the synthesized data and extracts meaningful information from the selected articles [10] . There are two main goals in this phase.

The first goal is to analyze the literature in terms of leading authors, journals, countries, and organizations. Furthermore, it helps identify correlations among topic s . Even when not mandatory, this activity can be constructive for researchers to position their work, find trends, and find collaboration opportunities. Next, data from the selected articles can be analyzed using bibliometric analysis (BA). BA summarizes large amounts of bibliometric data to present the state of intellectual structure and emerging trends in a topic or field of research [4] . Table 7 sets out some of the most common bibliometric analysis representations.

Techniques for bibliometric analysis and examples.

Several tools can perform this type of analysis, such as Excel and Google Sheets for statistical graphs or using programming languages such as Python that has available multiple  data visualization libraries (i.e. Matplotlib, Seaborn). Cluster maps based on bibliographic data(i.e keywords, authors) can be developed in VosViewer which makes it easy to identify clusters of related items [18] . In Fig. 4 , node size is representative of the number of papers related to the keyword, and lines represent the links among keyword terms.

[1] Keyword co-relationship analysis using clusterization in vos viewer.

This second and most important goal is to answer the formulated research questions, which should include a quantitative and qualitative analysis. The quantitative analysis can make use of data categorized, labelled, or coded in the extraction form (see Section 1.6). This data can be transformed into numerical values to perform statistical analysis. One of the most widely employed method is frequency analysis, which shows the recurrence of an event, and can also represent the percental distribution of the population (i.e., percentage by technology type, frequency of use of different frameworks, etc.). Q ualitative analysis includes the narration of the results, the discussion indicating the way forward in future research work, and inferring a conclusion.

Finally, the literature review report should state the protocol to ensure others researchers can replicate the process and understand how the analysis was performed. In the protocol, it is essential to present the inclusion and exclusion criteria, quality assessment, and rationality beyond these aspects.

The presentation and reporting of results will depend on the structure of the review given by the researchers conducting the SLR, there is no one answer. This structure should tie the studies together into key themes, characteristics, or subgroups [ 28 ].

SLR can be an extensive and demanding task, however the results are beneficial in providing a comprehensive overview of the available evidence on a given topic. For this reason, researchers should keep in mind that the entire process of the SLR is tailored to answer the research question(s). This article has detailed a practical guide with the essential steps to conducting an SLR in the context of computer science and software engineering while citing multiple helpful examples and tools. It is envisaged that this method will assist researchers, and particularly early-stage researchers, in following an algorithmic approach to fulfill this task. Finally, a quick checklist is presented in Appendix A as a companion of this article.

CRediT author statement

Angela Carrera-Rivera: Conceptualization, Methodology, Writing-Original. William Ochoa-Agurto : Methodology, Writing-Original. Felix Larrinaga : Reviewing and Supervision Ganix Lasa: Reviewing and Supervision.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

Funding : This project has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie Grant No. 814078.

Carrera-Rivera, A., Larrinaga, F., & Lasa, G. (2022). Context-awareness for the design of Smart-product service systems: Literature review. Computers in Industry, 142, 103730.

1 https://parsif.al/

Data Availability

Advertisement

The ABC of systematic literature review: the basic methodological guidance for beginners

• Published: 23 October 2020
• Volume 55 , pages 1319–1346, ( 2021 )

Cite this article

• Hayrol Azril Mohamed Shaffril 1 ,
• Samsul Farid Samsuddin 2 &
• Asnarulkhadi Abu Samah 1 , 3  

18k Accesses

131 Citations

2 Altmetric

Explore all metrics

There is a need for more methodological-based articles on systematic literature review (SLR) for non-health researchers to address issues related to the lack of methodological references in SLR and less suitability of existing methodological guidance. With that, this study presented a beginner's guide to basic methodological guides and key points to perform SLR, especially for those from non-health related background. For that, a total of 75 articles that passed the minimum quality were retrieved using systematic searching strategies. Seven main points of SLR were discussed, namely (1) the development and validation of the review protocol/publication standard/reporting standard/guidelines, (2) the formulation of research questions, (3) systematic searching strategies, (4) quality appraisal, (5) data extraction, (6) data synthesis, and (7) data demonstration.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach

What is qualitative in qualitative research.

Criteria for Good Qualitative Research: A Comprehensive Review

Athukorala, K., Głowacka, D., Jacucci, G., Oulasvirta, A., Vreeken, J.: Is exploratory search different?: A comparison of information search behavior for exploratory and lookup tasks. J. Assoc. Inf. Sci. Technol. 67 (11), 2635–2651 (2016). https://doi.org/10.1002/asi.23617

Article   Google Scholar  

Barnett-Page, E., Thomas, J.: Methods for the synthesis of qualitative research: a critical review. BMC Med. Res. Methodol. (2009). https://doi.org/10.1186/1471-2288-9-59

Bates, J., Best, P., McQuilkin, J., Taylor, B.: Will web search engines replace bibliographic databases in the systematic identification of research? J. Acad. Librariansh. 43 (1), 8–17 (2017)

Google Scholar  

Berrang-Ford, L., Pearce, T., Ford, J.D.: Systematic review approaches for climate change adaptation research. Reg. Environ. Change 15 (5), 755–769 (2015). https://doi.org/10.1007/s10113-014-0708-7

Braun, V., Clarke, V.: Using thematic analysis in psychology. Qual. Res. Psychol. 3 (2), 77–101 (2006). https://doi.org/10.1191/1478088706qp063oa

Burgers, C., Brugman, B.C., Boeynaems, A.: Systematic literature reviews: four applications for interdisciplinary research. J. Pragmat. 145 , 102–109 (2019)

Brunton, G., Oliver, S., Oliver, K., Lorenc, T.: A Synthesis of Research Addressing Children’s, Young People’s and Parents’ Views of Walking and Cycling for Transport. EPPI-Centre, Social. Science Research Unit, Institute of Education, University of London, London (2006)

Cañón, M., Buitrago-Gómez, Q.: The research question in clinical practice: a guideline for its formulation. Rev Colomb Psiquiatr. 47 (3), 193–200 (2018). https://doi.org/10.1016/j.rcp.2016.06.004

Centre for Reviews and Dissemination.: Systematic Reviews: CRD’s Guidance for Undertaking Reviews in Health Care. University of York, York (2006)

Charrois, T.L.: Systematic reviews: What do you get to know to get started? Can. J. Hosp. Pharm. 68 (2), 144–148 (2015)

Cooke, A., Smith, D., Booth, A.: Beyond PICO: the SPIDER tool for qualitative evidence synthesis. Qual. Health Res. 22 (10), 1435–1443 (2012)

Cooper, C., Booth, A., Campbell, J., Britten, N., Garside, R.: Defining the process to literature searching in systematic reviews: a literature review of guidance and supporting studies. BMC Med. Res. Methodol. 18 , 85 (2018)

Creswell, J.: Qualitative Inquiry & Research Design: Choosing Among Five Approaches, 3rd edn. Sage Publications, Inc., Thousand Oaks, CA (2013)

del Amo, I.F., Erkoyuncu, J.A., Roy, R., Palmarini, R., Onoufriou, D.: A systematic review of augmented reality content-related techniques for knowledge transfer in maintenance applications. Comput. Ind. 103 , 47–71 (2018)

Delaney, A., Tamás, P.A.: Searching for evidence or approval? A commentary on database search in systematic reviews and alternative information retrieval methodologies. Res. Synth. Method 9 (1), 124–131 (2018)

Dixon-Woods, M., Agarwal, S., Jones, D., Young, B., Sutton, A.: Synthesising qualitative and quantitative evidence: a review of possible methods. J. Health Serv. Res. Policy 10 (1), 45–53 (2005)

Doody, O., Bailey, M.E.: Setting a research question, aim and objective. Nurse Res. 23 (4), 19–23 (2016)

Durach, C.F., Kembro, J., Wieland, A.: A new paradigm for systematic literature reviews in supply chain management. J. Supply Chain Manag. 53 (4), 67–85 (2017)

Fagan, J.C.: An evidence-based review of academic web search engines, 2014–2016: implications for Librarians’ Practice and Research Agenda. Inf. Technol. Libr. 36 (2), 7–47 (2017)

Flemming, K., Booth, A., Garside, R., Tunc¸alp, O., Noyes J.: Qualitative evidence synthesis for complex interventions and guideline development: clarification of the purpose, designs and relevant methods. BMJ Global Health (2018)

Gomersall, J.S., Jadotte, Y.T., Xue, Y., Lockwood, S., Riddle, D., Preda, A.: Conducting systematic reviews of economic evaluations. Int. J. Evid.-Based Healthc. 13 (3), 170–178 (2015). https://doi.org/10.1097/XEB.0000000000000063

Green, B.N., Johnson, C.D., Adams, A.: Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. J. Chiropr. Med. 5 (3), 101–117 (2006). https://doi.org/10.1016/S0899-3467(07)60142-6

Greyson, D., Rafferty, E., Slater, L., MacDonald, N., Bettinger, J.A., Dubé, È., MacDonald, S.E.: Systematic review searches must be systematic, comprehensive, and transparent: a critique of Perman et al. BMC Public Health 19 (1), 1–6 (2019). https://doi.org/10.1186/s12889-018-6275-y

Gusenbauer, M.: Google Scholar to overshadow them all? Comparing the sizes of 12 academic search engines and bibliographic databases. Sciencetometrics 118 (1), 177–214 (2019)

Gusenbauer M, Haddaway NR (2020) Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources. Res. Synth. Methods. 11(2):181–217. https://doi.org/10.1002/jrsm.1378

Haddaway, N.R., Collins, A.M., Coughlin, D., Kirk, S.: The role of google scholar in evidence reviews and its applicability to grey literature searching. PLoS ONE 10 (9), e0138237 (2015). https://doi.org/10.1371/journal.pone.0138237

Haddaway, N.R., Macura, B., Whaley, P., Pulin, A.S.: ROSES Reporting standards for Systematic Evidence Syntheses: pro forma, flow-diagram and descriptive summary of the plan and conduct of environmental systematic reviews and systematic maps. Environ. Evid 7 , 7 (2018). https://doi.org/10.1186/s13750-018-0121-7

Halevi, G., Moed, H., Bar-Illan, J.: Suitability of Google Scholar as a source of scientific information and as a source of data for scientific evaluation. Revi Lit 11 (3), 823–834 (2017)

Hannes, K.: Critical appraisal of qualitative research. In: Noyes, J., Hannes, K., Harden, A., Harris, J., Lewin, S., Lockwood, C. (eds.) Supplementary Guidance for Inclusion of Qualitative Research in Cochrane Systematic Reviews of Interventions. Cochrane Collaboration Qualitative Methods Group, London (2011)

Higgins, J.P.T., Altman, D.G., Gotzsche, P.C., Juni, P., Moher, D., Oxman, A.D., Savovic, J., Schulz, K.F., Weeks, L., Sterne, J.A.C.: The cochrane collaboration's tool for assessing risk of bias in randomised trials. BMJ 343 (7829), 1–9 (2011). https://doi.org/10.1136/bmj.d5928

Higgins, J.P.T., Thomas, J., Chandler, J., Cumpston, M., Li, T., Page, M.J., Welch, V.A. (eds.): Cochrane Handbook for Systematic Reviews of Interventions, 2nd edn. Wiley, Chichester (UK) (2019)

Hong, Q.N., Pluye, P., Fàbregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P., Gagnon, M-P., Griffiths, F., Nicolau, B., O’Cathain, A., Rousseau, M-C., Vedel, I.: Mixed Methods Appraisal Tool (MMAT), version 2018. Registration of Copyright (#1148552), Canadian Intellectual Property Office, Industry Canada (2018)

Housyar, M., Sotudeh, H.: A reflection on the applicability of Google Scholar as a tool for comprehensive retrieval in bibliometric research and systematic reviews. Int. J. Inf. Sci. Manag. 16 (2), 1–17 (2018)

Hopia, H., Latvala, E., Liimatainen, L.: Reviewing the methodology of an integrative review. Scand. J. Caring Sci. 30 (4), 662–669 (2016)

Johnson, B.T., Hennessy, E.A.: Systematic reviews and meta-analyses in the health sciences: best practice methods for research syntheses. Soc. Sci. Med. 233 , 237–251 (2019)

Kastner, M., Straus, S., Goldsmith, C.H.: Estimating the horizon of articles to decide when to stop searching in systematic reviews: an example using a systematic review of RCTs evaluating osteoporosis clinical decision support tools. AMIA Annu. Symp. Proc. Arch. 2007 , 389–393 (2007)

Kitchenham, B.A., Charters, S.M.: Guidelines for performing systematic literature reviews in software engineering. EBSE Technical Report (2007)

Kraus, S., Breier, M., Dasí-Rodríguez, S.: The art of crafting a systematic literature review in entrepreneurship research. Int. Entrep. Manag. J. 16 (3), 1023–1042 (2020). https://doi.org/10.1007/s11365-020-00635-4

Kushwah, S., Dhir, A., Sagar, M., Gupta, B.: Determinants of organic food consumption. A systematic literature review on motives and barriers. Appetite 143 , 104402 (2019)

Levy, Y., Ellis, T.J.: A systems approach to conduct an effective literature review in supports of information system research. Inf. Sci. J. 9 , 181–212 (2006)

Liberati, A., Altman, D.G., Tetzlaff, J., Mulrow, C., Gøtzsche, P.C., Ioannidis, J.P., Clarke, M., Devereaux, P.J., Kleijnen, J., Moher, D.: The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med 6 (7), e1000100 (2009)

Linares-Espinós, E., Hernández, V., Domínguez-Escrig, J.L., Fernández-Pello, S., Hevia, V., Mayor, J., Padilla-Fernández, B., Ribal, M.J.: Methodology of systematic review. Actas urologicas españolas 42 (8), 499–506 (2018)

Lockwood, C., Munn, Z., Porritt, K.: Qualitative research synthesis: methodological guidance for systematic reviewers utilizing meta-aggregation. Int. J. Evid. Based Healthc. 13(3), 179–187 (2015). https://doi.org/10.1097/XEB.0000000000000062

Long, A.F., Godfrey, M.: An evaluation tool to assess the quality of qualitative research studies. Int. J. Soc. Res. Methodol. 7 (2), 181–196 (2004)

Mallet, R., Hagen-Zanker, J., Slater, R., Duvendack, M.: The benefits and challenges of using systematic reviews in international development research. J. Dev. Eff. 4 , 445–455 (2012)

Mantzoukas, S.: Facilitating research students in formulating qualitative research questions. Nurse Educ. Today 28 (3), 371–377 (2008)

Mays, N., Pope, C., Popay, J.: Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J. Health Serv. Res. Policy 10 (1), 6–20 (2005)

Mengist, W., Soromessa, T., Legese, G.: Method for conducting systematic literature review and meta-analysis for environmental science research. MethodsX 7 , 100777 (2020). https://doi.org/10.1016/j.mex.2019.100777

Methley, A.M., Campbell, S., Chew-Graham, C., McNally, R., Cheraghi-Sohi, S.: PICO, PICOS and SPIDER: a comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews. BMC Health Serv. Res. (2014). https://doi.org/10.1186/s12913-014-0579-0

Moher, D., Liberati, A., Tetzlaff, J., Altman, D.G., PRISMA Group: Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 21;6 (7), e1000097 (2009)

Morton SC, Murad MH, O’Connor E, Lee CS, Booth M, Vandermeer BW, Snowden JM, D’Anci KE, Fu R, Gartlehner G, Wang Z, Steele DW (2018) Quantitative synthesis—an update. methods guide for comparative effectiveness reviews. (Prepared by the Scientific Resource Center under Contract No. 290-2012-0004-C). AHRQ Publication No. 18-EHC007- EF. Rockville, MD: Agency for Healthcare Research and Quality. Posted final reports are located on the Effective Health Care Program search page. https://doi.org/10.23970/AHRQEPCMETHGUIDE3

Noble, H., Mitchell, M.: What is grounded theory? Evid. Based Nurs. 19 (2), 34–35 (2016). https://doi.org/10.1136/eb-2016-102306

Okoli, C.: A guide to conducting a standalone systematic literature review. Commun. Assoc. Inf. Syst. 37 , 879–910 (2015)

Okoli, C., Schabram, K.: A guide to conducting a systematic literature review of information systems research. Philos. Methodol. Econ. eJ. (2010). https://doi.org/10.2139/ssrn.1954824

Onwuegbuzie, A.J., Leech, N.L.: Sampling designs in qualitative research: making the sampling process more public. Qual. Rep. 12 (2), 238–254 (2007)

Pace, R., Pluye, P., Bartlett, G., Macaulay, A.C., Salsberg, J., Jagosh, J., Seller, R.: Testing the reliability and efficiency of the pilot Mixed Methods Appraisal Tool (MMAT) for systematic mixed studies review. Int. J. Nurs Stud. 49 (1), 47–53 (2012). https://doi.org/10.1016/j.ijnurstu.2011.07.002

Page, M.J., McKenzie, J.E., Bossuyt, P.M., Boutron, I., Hoffmann, T., Mulrow, C.D., Shamseer, L., Moher, D.: Mapping of reporting guidance for systematic reviews and meta-analyses generated a comprehensive item bank for future reporting guidelines. J. Clin. Epidemiol. 118 , 60–68 (2020)

Paterson, B.L., Thorne, S.E., Canam, C., Jillings, C.: Meta-Study of Qualitative Health Research. A Practical Guide to Meta-Analysis and Meta-Synthesis. Sage Publications, Thousand Oaks (2001)

Palaskar, J.N.: Framing the research question using PICO strategy. J. Dent. Allied Sci. 6 (2), 55 (2017)

Patino, C.M., Ferreira, J.C.: Inclusion and exclusion criteria in research studies: definitions and why they matter. J. Bras. Pneumol. 44 (2), 84 (2018)

Peters, M.D., Godfrey, C.M., Khalil, H., McInerney, P., Parker, D., Soares, C.B.: Guidance for conducting systematic scoping reviews. Int. J. Evid.-Based Healthc. 13 (3), 141–146 (2015). https://doi.org/10.1097/XEB.0000000000000050

Petticrew, M., Roberts, H.: Systematic Reviews in the Social Sciences: A Practical Guide. Blackwell Publishing Ltd, Oxford (2006)

Prieto and Rumbo-Prieto: The systematic review: plurality of approaches and methodologies. Enferm. Clín. (English Edition) 28 (6), 387–393 (2018)

Reim, W., Parida, V., Örtqvist, D.: Product-Service Systems (PSS) business models and tactics – a systematic literature review. J. Clean. Prod. 97 , 61–75 (2015). https://doi.org/10.1016/j.jclepro.2014.07.003

Robinson, P., Lowe, J.: Literature reviews vs systematic reviews. Aust. N. Z. J. Public Health 39 (2), 103 (2015)

Rousseau, D. M., Manning, J., Denyer, D.: Evidence in management and organizational science: assembling the field’s full weight of scientific knowledge through syntheses. In: AIM Research Working Paper Series: Advanced Institute of Management Research (2008)

Sandelowski, M., Barroso, J., Voils, C.I.: Using qualitative metasummary to synthesize qualitative and quantitative descriptive findings. Res. Nuurs. Health 30 (1), 99–111 (2007). https://doi.org/10.1002/nur.20176

Sandelowski, M., Voils, C.I., Barroso, J.: Defining and designing mixed research synthesis studies. Res. Schools: Nat. Ref. J. Spons. Mid-South Educ. Res. Assoc. Univ. Alabama 13 (1), 29 (2006)

Schardt, C., Adams, M.B., Owens, T., Keitz, S., Fontelo, P.: Utilization of the PICO framework to improve searching PubMed for clinical questions. BMC Med. Inf. Decis. Mak. 7 , 16 (2007). https://doi.org/10.1186/1472-6947-7-16

Seehra, J., Pandis, N., Koletsi, D.: Use of quality assessment tools in systematic reviews was varied and inconsistent. J. Clin. Epidemiol. 69 , 179–184 (2016)

Shaffril, H.A.M., Krauss, S.E., Samsuddin, S.F.: A systematic review on Asian’s farmers’ adaptation practices towards climate change. Sci. Total Environ. 644 , 683–695 (2018)

Shaffril, H.A.M., Abu Samah, A., Samsuddin, S.F., Ali, Z.: Mirror-mirror on the wall, what climate change adaptation strategies are practiced by the Asian’s fishermen of all? J. Clean. Prod. 232 , 104–117 (2019)

Shorten, A., Shorten, B.: What is meta-analysis. Evid. Based Nurs. 16 (1), 3–4 (2013)

Siering, U., Eikermann, M., Hausner, E., Hoffmann-Eßer, W., Neugebauer, E.A.: Appraisal tools for clinical practice guidelines: a systematic review. PLoS ONE 8 (12), e82915 (2013)

Soares, C.B., Hoga, L., Sangaleti, C., Yonekura, T., Peduzzi, M., Silva, D.: Integrative review in nursing research and EBP: a type of systematic review? Int. J.Evid.-Based Healthcare 11 (3), 246–247 (2013)

Thomas, J., Noel-Storrb, A., Marshall, I., Wallace, B., McDonald, S., Mavergames, C., Glasziou, P., Shemilta, I., Synnote, A., Turnere, T., Elliott, J.: Living systematic reviews: 2. Combining human and machine effort. J. Clin. Epidemiol. 91 , 31–37 (2017)

Thomas, J., Kneale, D., McKenzie, J.E., Brennan, S.E., Bhaumik, S.: Chapter 2: Determining the scope of the review and the questions it will address. In: Higgins, J.P.T., Thomas, J., Chandler, J., Cumpston, M., Li, T., Page, M.J., Welch, V.A. (eds.) Cochrane Handbook for Systematic Reviews of Interventions version 6.1 (2020). Cochrane (2020). https://www.training.cochrane.org/handbook

Wanden-Berghe, C., Sanz-Valero, J.: Systematic reviews in nutrition: standardized methodology. Br. J. Nutr. 107 , S3–S7 (2012)

Whittemore, R., Knafl, K.: The integrative review: updated methodology. J. Adv. Nurs. 52 (5), 546–553 (2005)

Wong, G., Greenhalgh, T., Westhorp, G., Buckingham, J., Pawson, R.: RAMESES publication standards: realist syntheses. BMC Med. 11 , 21 (2013). https://doi.org/10.1186/1741-7015-11-21

Xiao, Y., Watson, M.: Guidance on conducting a systematic literature review. J. Plan. Educ. 39 (1), 93–112 (2019)

Younger, P.: Using Google Scholar to conduct a literature search. Nurs. Stand. 24 (45), 40–46 (2010)

Download references

Author information

Authors and affiliations.

Institute for Social Science Studies, Universiti Putra Malaysia, Putra Infoport, 43400, Serdang, Selangor, Malaysia

Hayrol Azril Mohamed Shaffril & Asnarulkhadi Abu Samah

Faculty of Computer Science and Information Technology, University of Malaya, 50603, Kuala Lumpur, Malaysia

Samsul Farid Samsuddin

Faculty of Human Ecology, Universiti Putra Malaysia, 43400, Serdang, Selangor, Malaysia

Asnarulkhadi Abu Samah

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Hayrol Azril Mohamed Shaffril .

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Mohamed Shaffril, H.A., Samsuddin, S.F. & Abu Samah, A. The ABC of systematic literature review: the basic methodological guidance for beginners. Qual Quant 55 , 1319–1346 (2021). https://doi.org/10.1007/s11135-020-01059-6

Download citation

Accepted : 12 October 2020

Published : 23 October 2020

Issue Date : August 2021

DOI : https://doi.org/10.1007/s11135-020-01059-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Systematic literature review
• Basic methodology
• Non-health related studies
• Find a journal
• Publish with us
• Track your research

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Online First
• Management of systemic lupus erythematosus: a systematic literature review informing the 2023 update of the EULAR recommendations
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0002-8832-7475 Myrto Kostopoulou 1 ,
• http://orcid.org/0000-0002-9771-6667 Chetan B Mukhtyar 2 ,
• http://orcid.org/0000-0001-5299-1406 George Bertsias 3 , 4 ,
• http://orcid.org/0000-0002-9812-4671 Dimitrios T Boumpas 1 , 5 ,
• http://orcid.org/0000-0003-2696-031X Antonis Fanouriakis 1
• 1 Rheumatology and Clinical Immunology Unit, Attikon University Hospital , National and Kapodistrian University of Athens School of Medicine , Athens , Greece
• 2 Vasculitis Service, Rheumatology Department , Norfolk and Norwich University Hospital NHS Trust , Norwich , UK
• 3 Rheumatology and Clinical Immunology , University of Crete, School of Medicine , Heraklion , Greece
• 4 Laboratory of Autoimmunity and Inflammation , Institute of Molecular Biology and Biotechnology , Heraklion , Greece
• 5 Laboratory of Autoimmunity and Inflammation , Biomedical Research Foundation of the Academy of Athens , Athens , Greece
• Correspondence to Dr Antonis Fanouriakis, Rheumatology and Clinical Immunology Unit, "Attikon" University Hospital, National and Kapodistrian University of Athens School of Medicine, Athens, Greece; afanour{at}med.uoa.gr

Objectives To analyse the new evidence (2018–2022) for the management of systemic lupus erythematosus (SLE) to inform the 2023 update of the European League Against Rheumatism (EULAR) recommendations.

Methods Systematic literature reviews were performed in the Medline and the Cochrane Library databases capturing publications from 1 January 2018 through 31 December 2022, according to the EULAR standardised operating procedures. The research questions focused on five different domains, namely the benefit/harm of SLE treatments, the benefits from the attainment of remission/low disease activity, the risk/benefit from treatment tapering/withdrawal, the management of SLE with antiphospholipid syndrome and the safety of immunisations against varicella zoster virus and SARS-CoV2 infection. A Population, Intervention, Comparison and Outcome framework was used to develop search strings for each research topic.

Results We identified 439 relevant articles, the majority being observational studies of low or moderate quality. High-quality randomised controlled trials (RCTs) documented the efficacy of the type 1 interferon receptor inhibitor, anifrolumab, in non-renal SLE, and belimumab and voclosporin, a novel calcineurin inhibitor, in lupus nephritis (LN), when compared with standard of care. For the treatment of specific organ manifestations outside LN, a lack of high-quality data was documented. Multiple observational studies confirmed the beneficial effects of attaining clinical remission or low disease activity, reducing the risk for multiple adverse outcomes. Two randomised trials with some concerns regarding risk of bias found higher rates of relapse in patients who discontinued glucocorticoids (GC) or immunosuppressants in SLE and LN, respectively, yet observational cohort studies suggest that treatment withdrawal might be feasible in a subset of patients.

Conclusion Anifrolumab and belimumab achieve better disease control than standard of care in extrarenal SLE, while combination therapies with belimumab and voclosporin attained higher response rates in high-quality RCTs in LN. Remission and low disease activity are associated with favourable long-term outcomes. In patients achieving these targets, GC and immunosuppressive therapy may gradually be tapered. Cite Now

• Systemic Lupus Erythematosus
• Lupus Nephritis

Data availability statement

All data relevant to the study are included in the article or uploaded as online supplemental information.

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:  http://creativecommons.org/licenses/by-nc/4.0/ .

https://doi.org/10.1136/ard-2023-225319

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

WHAT IS ALREADY KNOWN ON THIS TOPIC

Since the 2019 European League Against Rheumatism (EULAR) recommendations for the management of systemic lupus erythematosus (SLE), several studies have been published providing data on alternative therapeutic options and treatment targets. A systematic literature review (SLR) focusing on recent advances was performed to inform the 2023 update of EULAR recommendations for the management of SLE.

WHAT THIS STUDY ADDS

In extrarenal disease, anifrolumab and belimumab were superior to standard of care treatment in a number of high-quality randomised controlled trials.

High-quality evidence points towards better efficacy of combination treatments with belimumab or voclosporin compared with standard of care in patients with lupus nephritis.

Both remission and low disease activity have been associated with lower risk of adverse outcomes in observational studies.

Although treatment discontinuation increases the risk of flares, successful glucocorticoid withdrawal was accomplished in patients with SLE in remission in several cohort studies.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

This SLR provided a systematic update of current evidence regarding the management of patients with SLE, to inform the 2023 update of the EULAR recommendations.

Introduction

Management of systemic lupus erythematosus (SLE) is challenging, owing to the heterogeneity of disease phenotype, the variable severity of involvement even within the same organ manifestation, and the different efficacy of drugs in different patient subgroups and disease manifestations. 1 Patients with SLE will frequently require multiple drugs during the course of their disease to achieve and maintain sufficient control. To this end, it is important that recent years have witnessed significant progress in the form of introduction of new drugs to treat the disease. Anifrolumab, an anti-type 1 interferon receptor inhibitor, was approved in 2021 for the treatment of moderate-to-severe extrarenal SLE. 2 3 Belimumab and voclosporin (a novel calcineurin inhibitor (CNI)) were also approved by the European Medicines Agency in 2021 and 2022, respectively, for the treatment of lupus nephritis (LN), a cardinal manifestation of the disease affecting up to 40%–50% of patients, with significant impact on morbidity and survival. 4 5

These important advances provided the ground for an update of the European League Against Rheumatism (EULAR) recommendations for the management of SLE, which was published recently. 6 To this end, we performed structured systematic literature reviews (SLRs), aiming to update the evidence for the efficacy and safety of different therapies, as well as try to define the optimal therapy of different organ manifestations of the disease. The results of these SLRs were presented to the Task Force members during dedicated meetings to form the current evidence base, on which the formulation of the current recommendations was based. The current manuscript presents in detail the results of these SLRs.

We followed the standardised operating procedures for the development of EULAR-endorsed recommendations and employed the Appraisal of Guidelines Research and Evaluation instrument. Following assembly of the Task Force, the convenor (DTB), one methodologist (GB), one co-methodologist (CBM), and two fellows responsible for the SLR (AF and MK) created an outline of the proposed methodology, as well as the main research questions in the form of Population, Intervention, Comparison and Outcomes (PICOs), which were circulated among Task Force members. A Delphi-based methodology within the Task Force finally identified five research questions: (1) management of general and organ-specific SLE (divided in six subquestions regarding drug efficacy and safety in patients with active SLE, active mucocutaneous, musculoskeletal, haematological, neuropsychiatric and kidney involvement, respectively), (2) targets of treatment, (3) management of patients with SLE and antiphospholipid syndrome, (4) tapering/withdrawal of treatment in SLE and (5) efficacy and safety of vaccination against varicella zoster virus (VZV) reactivation and SARS-CoV2 infection (a generic SLR for infection risk and prevention in SLE was not performed, because there are specific EULAR recommendations on this topic). 7 Separate search strings were developed for each PICO (1–5), resulting in five separate SLRs (the six subquestions of PICO 1 (PICO 1a–f) were examined with a single search string) ( online supplemental file 1 and 2, tables S1.1–S1.10 ).

Supplemental material

Under the supervision of the methodologists, AF and MK performed the SLRs independently in two different databases (MEDLINE through PubMed and the Cochrane Library), with additional inclusion of Lancet Rheumatology (due to non-inclusion of the latter in PubMed). Since this was an update of the 2019 recommendations on general SLE, the current SLRs evaluated all English language publications published between January 2018 and December 2022. All study designs were included (meta-analyses, randomised controlled trials (RCTs), quasi-RCTs, cohort studies, case–control studies, cross–sectional studies) while narrative reviews, case series, case reports, conference abstracts, animal studies, trials in non-English language, trials with population<20 and trials on paediatric populations were excluded. In case a study was captured as an original publication and was also included in a meta-analysis, then only the meta-analysis data were used, to avoid duplicating the evidence from that particular study. Eligible studies were reviewed for snowball references and relevant articles, identified by manual search within the reference list of the originally retrieved publications, were also included. For each research question, a predefined extraction form was used to capture the population set, all relevant interventions, their duration of use, route of administration, dosage, follow-up time and the respective effect estimates, including incidence rate, mean difference, risk difference, correlation coefficient, odds ratio (OR) and relative risk. For each research question, results were synthesised and presented according to the interventions used and the respective outcomes.

Risk of bias (RoB) was assessed using the revised Cochrane Risk of Bias Assessment Tool for RCTs (ROB V.2), the Newcastle-Ottawa scale for observational studies, and the AMSTAR V.2 tool for meta-analyses ( online supplemental file 3 ). In case of disagreements, these were internally discussed until achievement of consensus, and one methodologist was involved when deemed necessary. A Preferred Reporting Items for Systematic Reviews and Meta-Analyses checklist was completed and has been submitted along with the manuscript.

We screened a total of 10 889 articles, of which 578 were selected for full-text review, and 439 were finally included for data extraction (see figure 1 for a detailed flow diagram of the selection process). The results below are presented in terms of general efficacy of drugs in SLE, followed by treatment of specific manifestations, with a focus on LN.

• Download figure
• Open in new tab
• Download powerpoint

Flow diagram of the study selection process.

Efficacy and safety of hydroxychloroquine (HCQ) in SLE

Between January 2018 and December 2022, a total of 39 studies (all observational) evaluated and confirmed the association of HCQ with various favourable outcomes ( online supplemental file 4, table S4.1 ). A total of 10 studies reported a negative association between HCQ use and mortality in SLE; a meta-analysis of 21 studies (26 037 patients) found a pooled HR 0.46 for death in patients with SLE receiving HCQ (consistent results in all geographic regions). 8 Fewer (or individual) studies showed a positive effect of HCQ on various outcomes (reduced rate of disease flares, thrombosis, osteonecrosis, infections, among others). Regarding safety of HCQ, the focus was on retinal toxicity. 9 10 The current SLRs identified 10 studies (mostly of poor or fair quality) ( table 1 ); two retrospective cohort studies of good quality (ie, lower RoB) reported retinopathy rates of 0.8% and 4.3%, respectively. Longer duration of HCQ intake and a higher cumulative dose were confirmed as risk factors for retinal toxicity. Regarding other safety issues, a concern for corrected QT (QTc) prolongation was raised when HCQ was used during the early phases of the COVID-19 pandemic; however, a total of six studies found no clinically relevant QTc prolongation with HCQ use.

• View inline

Prevalence of HCQ retinopathy in observational studies and associations

The recommended dose of 5 mg/kg in the 2023 recommendations was based on (1) an observational study of good quality, which calculated the threshold for an increased risk of flares near 5 mg/kg/day of HCQ dose, 11 (2) older evidence of good quality, suggesting that risk of toxicity is low for doses below 5 mg/kg real body weight 10 and (3) indirect evidence for a slightly increased risk of flares in patients who taper HCQ versus those who continue (see below, Safety of treatment tapering in SLE).

Efficacy and safety of glucocorticoids (GC) in SLE

Although GC are widely used in SLE, high-quality RCTs assessing the efficacy of different schemes and tapering strategies are still lacking. A single, retrospective study of good quality in 206 patients with LN found higher rates of 1-year complete response in patients who started with ≥40 mg/day compared with those who started with ≤30 mg/day, without increased risk for GC-related damage. 12 Two small RCTs (one with 32 and one with 20 patients, both with high RoB) compared different doses of GC with same background immunosuppression (cyclophosphamide (CYC) and mycophenolate mofetil (MMF), respectively) and found discordant results; one showed equal response rates and the other higher rates in the high-dose GC arm. 13 14

For safety, the SLRs identified a large number of studies examining different cut-offs of average prednisone doses in association with different adverse effects ( online supplemental file 4, tableS4.2 for association with infections and online supplemental table S4.3 for associations with other harms). Most studies pointed towards thresholds of mean 5–7.5 mg/day prednisone, associated with a variety of GC-related side effects in multivariable associations.

Efficacy and safety of immunosuppressive drugs in extrarenal SLE

Immunosuppressive therapies used to treat extrarenal manifestations of SLE include both conventional drugs (azathioprine (AZA), methotrexate (MTX), MMF, CNIs, among others), as well as biologic agents (approved therapies belimumab and anifrolumab, and drugs used off-label, such as rituximab (RTX)). During the period captured by the SLRs, no new head-to-head comparisons between conventional immunosuppressive drugs were identified, rather only limited observational studies (mainly of-low quality) reporting efficacy in selected manifestations (mainly LN). To this end, this part will focus on new data regarding approved biologics.

We retrieved a total of 53 publications of belimumab in SLE, published between 2018 and 2022 (among them, 6 RCTs, 7 open-label extensions of previous RCTs, 11 post hoc analyses of previously published RCTs, 7 meta-analyses and 18 real-world observational studies), overall confirming efficacy of the drug in extrarenal lupus. A Cochrane SLR including 6 RCTs of belimumab in SLE found belimumab to be associated with a pooled risk ratio of 1.33 (95% CI 1.22 to 1.45) and 1.59 (95% CI 1.17 to 2.15) for Safety of Estrogen in Lupus National Assessment—Systemic Lupus Erythematosus Disease Activity Index (SELENA-SLEDAI) reduction by four points and reduction of GC dose by 50%, respectively. 15 Importantly, after the publication of the 2019 recommendations, belimumab has been tested in phase III RCTs in specific ethnic/racial populations, the Efficacy and Safety of Belimumab in Black Race Patients with SLE (EMBRACE) RCT in 448 African-Americans, 16 and the Belimumab in Subjects with SLE-North East Asia (BLISS-NEA) in 707 patients from North-East Asia. 17 Although in both studies, SLE Responder Index (SRI)-4 responses at 52 weeks were higher with belimumab versus placebo, the EMBRACE did not reach statistical significance (SRI response at week 52 48.7% with belimumab versus 41.6% with placebo (OR 1.40, 95% CI 0.93 to 2.11)). On the contrary, in BLISS-NEA, more patients treated with belimumab were SRI-4 responders at week 52 (53.8% vs 40.1% with placebo, OR 1.99, 95% CI 1.40 to 2.82). Regarding safety of belimumab, a phase IV RCT (BASE, 4003 patients) designed to test safety issues, found slightly higher rates of serious depression (0.35% vs 0.05%; Δ 0.15%, 95% CI 0.02% to 0.58%), treatment-emergent suicidality (1.42% vs 1.16%; Δ 0.26%, 95% CI −0.44% to 0.96%) and sponsor-adjudicated serious suicide or self-injury (0.75% vs 0.25%; post hoc Δ 0.50%, 95% CI 0.06% to 0.94%) with belimumab compared with placebo. 18 Similarly, a pooled post hoc analysis of one phase II and five phase III RCTs of belimumab (total 4170 patients) reported that serious depression was more common with belimumab (0.2% vs 0.1%) although suicide/self-injury was similar (0.3% in each group). 19 Incidence of all other adverse events and mortality was also similar between belimumab and placebo.

In addition to the Treatment of Uncontrolled Lupus via the Interferon Pathway (TULIP) trials, the SLR retrieved a total of 17 publications related to the use of anifrolumab in SLE: 2 phase II RCTs (one was in LN), 2 open-label extension studies, 7 post hoc analyses of previous RCTs, and 4 meta-analyses. Despite the discordant SRI-4 data of the two TULIP trials, both studies found significantly greater British Isles Lupus Assessment Group (BILAG)-based Composite Lupus Assessment (BICLA) response rates with anifrolumab compared with placebo (pooled OR 2.25, 95% CI 1.72 to 2.95, in a meta-analysis). 20 A post hoc analysis of the TULIP trials found that anifrolumab was associated with lower annualised disease flare rates (rate ratio 0.75, 95% CI 0.60 to 0.95), prolonged time to first flare (HR 0.70, 95% CI 0.55 to 0.89) and fewer patients with ≥1 flare (Δ −9.3%, 95% CI −16.3% to −2.3%), compared to placebo. 21 Regarding GC-sparing potential, another post hoc analysis of both TULIP trials reported sustained reduction to ≤7.5 mg/day prednisone in patients on ≥10 mg/day at baseline in 50.5% for anifrolumab versus 31.8% for placebo (Δ 18.7%, p<0.001), 22 while the above-mentioned meta-analysis (including also the MUSE phase II study of the drug) calculated the respective pooled OR at 2.45 (95% CI 1.69 to 3.54) compared to placebo. 20 In terms of safety, in general, adverse events and serious adverse events were similar between anifrolumab and placebo in RCTs, with the exception of VZV infection; analysis of the TULIP trials found a higher incidence of VZV in anifrolumab-treated patients versus placebo (6.4% vs 1.4%), evident in both interferon-high and interferon-low patients, 22 and confirmed in meta-analyses. 20 23 On the other hand, in the long-term extension of the TULIP studies (placebo controlled, 369 patients), VZV rates by year decreased over time and were lower during the long-term extension period than during the first year of TULIP (6.8 for year 1, dropping to 2.9 in year 4). 24

In RCTs, both belimumab and anifrolumab showed better clinical responses in patients who had abnormal serological markers at baseline (low C3/C4 levels and/or high antidouble-stranded DNA antibodies). 22 25 26

Treatment of specific extrarenal manifestations of SLE

Subquestions 1b–1f of PICO 1 were focused on the efficacy of different immunosuppressive treatments in various organ manifestations of SLE (mucocutaneous, musculoskeletal, haematological, neuropsychiatric and kidney involvement). The results on LN are presented in a separate section. Regarding other manifestations, the SLRs confirmed the paucity of high-quality data for their treatment. For skin disease, belimumab and anifrolumab have documented efficacy in RCTs of their clinical programme; however, belimumab has used the skin component from BILAG, while the more recent TULIP trials of anifrolumab have used the skin-specific Cutaneous Lupus Activity and Damage Index (CLASI) ( table 2 ).

Efficacy of belimumab and anifrolumab on skin disease in SLE

A meta-analysis of six RCTs focusing on skin efficacy of belimumab found a pooled OR of clinical response (BILAG defined) at 52 weeks of 1.44 (95% CI 1.20 to 1.74, I 2 =0%). 27 Clinical response was first noted after 20 weeks of treatment (OR 1.35, 95% CI 1.01 to 1.81, I 2 =0%), sustained through 1 year. In addition, CLASI data for belimumab have been reported in three observational studies (including 62, 67 and 466 patients, respectively), all showing significant reductions from baseline, ranging from 4 to 6 units ( table 2 ). 28–30 Anifrolumab RCTs have used CLASI to assess response; post hoc analyses of both TULIP phase III and the phase II MUSE trial have shown percentage differences in CLASI-A 50 (ie, 50% reduction from baseline) response more than 20% from placebo, almost reaching 30% in MUSE. 22 31 32

Efficacy data on arthritis were more scarce, available only from RCT of belimumab and anifrolumab. The post hoc analysis of the TULIP studies found that anifrolumab was associated with greater percentage of patients achieving ≥50% reduction in active swollen and tender joints (treatment Δ: 12.6% (95% CI 2.4% to 22.9%)). 22 Significant reduction was also noted in a similar analysis of the MUSE phase II study (mean (SD) swollen and tender joint reductions –5.5 (6.3) vs –3.4 (5.9) for placebo, p=0.004). 32 For belimumab, only two small observational, uncontrolled studies (n=81 and 20, respectively) specifically reported a reduction in the number of swollen and tender joints. 33 34

The SLR retrieved very few studies regarding haematological and neuropsychiatric manifestations. For neuropsychiatric SLE (NPSLE), a single meta-analysis on the efficacy of RTX in refractory SLE (including NPSLE) reported a pooled complete response rate of 90% for neuropsychiatric manifestations (95% CI 53% to 99%). 35 No other relevant studies were identified. For immune cytopenias, post hoc analysis of the TULIP trials found a 25% difference in response rate in haematological manifestations, in favour of anifrolumab (56% vs 31% for placebo), but with no further details. 31 A similar analysis of the BLISS trials (published in 2012, thus not included in the current SLR) had not found a difference of belimumab over placebo for haematological manifestations.

Treatment of LN

The SLR identified 98 studies evaluating the efficacy and safety of various treatments in LN. These included 14 meta-analyses (1 of high quality, 9 of low or critically low quality and 4 network meta-analyses), 15 RCTs (5 of low RoB, 6 with some concerns and 4 with high RoB) and 69 studies with other study designs (2 open-label extension studies of RCTs, 2 post hoc studies, 1 integrated analysis and 64 observational studies including 8 prospective cohorts, 53 retrospective cohorts, 2 cross-sectional and 1 case–control study) and varied quality.

14 RCTs (5 head-to-head, 2 dose-comparison and 7 add-on vs placebo trials) involving 2099 LN patients evaluated the efficacy and safety of various drugs as initial treatments for LN ( table 3 ).

Efficacy of initial treatments for LN in RCTs 2018–2022

Regarding comparison of standard of care therapies (CYC and MMF), only two new RCTs, both in Asian LN populations, were identified from the SLR (one with high and one with low RoB). One small RCT of 49 LN patients with impaired kidney function (mean±SD baseline serum creatinine 1.58±1.38 mg/dL) showed similar efficacy between CYC (monthly pulses of 0.5–1 g/m 2 for 6 months) and low-dose MMF (1.5 g/day) after 24 weeks of treatment (19.0% vs 28.6%, p=0.572). 36 In a second RCT, a low versus high dose of intravenous CYC (low dose: six fortnightly intravenous CYC pulses of 500 mg, high dose: 4 weekly six cycles of 750 mg/m 2 ), both followed by AZA, were administered in 38 and 37 patients, respectively. After 52 weeks, patients in the high-dose group had significantly increased rates of complete/partial response (50% vs 73%, p=0.04) and fewer relapses (3% vs 24%, p=0.01) compared with the low-dose group, with no difference in infection rates and death. 37 Although this study was designated as low RoB, it was nevertheless open-label and the sample size was relatively small.

Five RCTs (2 with low RoB, 2 with some concerns and 1 with high RoB) explored the effect of CNIs, either as monotherapy or in combination with MMF, against CYC/MMF. 5 38–41 In an open-label non-inferiority (margin 15%) RCT of 299 LN patients, tacrolimus (TAC) was non-inferior to CYC in terms of complete and partial response after 24 weeks of treatment. When the individual components of response were investigated, TAC was associated with a significant decrease in estimated glomerular filtration rate (eGFR), counterbalanced by greater reductions in proteinuria compared with CYC. 38 Similarly, in another RCT of 83 patients with proliferative LN who received 1:1 TAC or MMF followed by AZA, both arms had comparable remission rates at 12 months (46.3% vs 57.1% p=0.3). 39 Regarding long-term outcomes, TAC was non-inferior to MMF in a study of 150 patients who were previously randomised to TAC or MMF as induction treatment and AZA as maintenance. 42 After 10 years, the TAC group had similar relapse rates compared with MMF and there was also no difference in a composite outcome (reduction in eGFR≥30%, chronic kidney disease stage 4/5 or death). As in the previous SLR, no RCT was identified assessing the role of CNI as monotherapy in proliferative LN in non-Asian populations. In a meta-analysis of trials in Asian populations, TAC outperformed CYC in terms of complete response (OR 2.41 95% CI 1.46 to 3.99, based on seven studies), but had a similar effect when compared with MMF (OR 0.95 95% CI 0.54 to 1.64, based on three studies). 43 Similar results were reported in two recent network meta-analyses. 44 45

Three RCTs investigated the efficacy of multitarget therapy (CNI in combination with MMF, two using voclosporin and one using TAC) compared with MMF or CYC, all pointing towards better response rates with the multitarget treatment. 5 40 41 In AURA-LV, a phase II multicentre RCT, 267 patients were randomised 1:1:1 to receive either voclosporin (23.7 or 39.5 mg, each two times per day) or placebo, in combination with MMF (2 g/day) and low dose GC. At 24 weeks, patients on low-dose voclosporin had significantly increased complete response rates (defined as urine protein-to-creatinine ratio (UPCr) <0.5 mg/mg, an eGFR>60 mL/min/1.73 m 2 or no decrease of ≥20% of baseline eGFR, no administration of rescue medication and no more than 10 mg prednisone equivalent per day for 3 or more consecutive days or for 7 or more days during weeks 44–52) compared with placebo (OR 2.03, 95% CI 1.01 to 4.05); in terms of safety, voclosporin was associated with higher rates of adverse events and death. 41 The AURORA trial was a phase III multicentre RCT involving 357 LN patients with class III, IV, V or mixed classes. Patients were randomly assigned to voclosporin (23.7 mg two times per day) or placebo in addition to 2 g/day of MMF and low-dose GCs and were followed for 52 weeks. Complete renal response (defined as in AURA-LV) was achieved in significantly more patients in the voclosporin group than placebo (41% vs 23%, OR 2.65 95% CI 1.64 to 4.27), while both groups had similar eGFR and safety profile during follow-up. Importantly, subgroup analysis showed no benefit from the introduction of voclosporin in class V or when the dose of MMF exceeded 2 g/day. 5 An integrated analysis of pooled data from phases II and III voclosporin trials, as well as a long-term extension study of the AURORA trial (the latter published after the completion of the present SLR) corroborated the previous findings in efficacy and safety. 46 47 In another small (n=56), open-label RCT with longer follow-up (72 weeks), combination treatment with TAC (0.06–0.08 mg/kg/day) and MMF (20–30 mg/kg/day) was superior to intravenous CYC (0.5–0.75 g/m monthly for 6 months) in terms of renal response (81.5% vs 57.7%, p<0.05) and kidney function (mean ± SD serum creatinine 56.7±32.1 vs 72.5±32.5, p 0.019). 40

Four RCTs evaluated the efficacy and safety of biologic agents added to background immunosuppressive therapy. Two phase III trials investigated the add-on effect of belimumab (one of low RoB and the other with some concerns), one phase II trial investigated the add-on effect of anifrolumab (RoB with some concerns) and another phase II RCT investigated the add-on effect of obinutuzumab (low RoB). In the Belimumab International Study in Lupus Nephritis (BLISS-LN), a phase III, double-blind, placebo-controlled trial, 448 patients were randomly assigned to intravenous belimumab (10 mg/kg/month) or placebo added to standard therapy (ie, six pulses of intravenous CYC 500 mg every 2 weeks followed by AZA, or MMF (3 g/day) plus GC 0.5–1 mg/kg/day as initial dose). 4 Patients were stratified according to induction treatment and race. The primary endpoint assessed at 104 weeks was the primary efficacy renal response (PERR) defined as UPCr≤0.7 g/g, eGFR no worse than 20% below the preflare value or at least 60 mL/min/1.73 m 2 and no use of rescue therapy. More patients in the belimumab group achieved PERR compared with placebo at 104 weeks (43% vs 32% OR 1.6 95%CI 1.0 to 2.3). 4 In a secondary analysis, patients with class 5 or with a UPCr>3 g/g did not benefit from the addition of belimumab, in terms of PERR. However, the risk of a 30% and 40% decline in eGFR and the risk of flare were significantly less in patients receiving belimumab. 48 The CALIBRATE study was a phase II open-label RCT in patients with refractory or relapsing LN, assessing the safety and potential benefit from the addition of belimumab to a background treatment of RTX and intravenous CYC. 49 Although the addition of belimumab did not increase adverse events, patients on belimumab and placebo had similar response rates (52% vs 41%, p=0.4). The phase II double-blinded TULIP-LN study randomised 147 patients with biopsy-proven proliferative LN in a 1:1:1 ratio to receive either monthly 300 mg of intravenous anifrolumab (basic regimen), 900 mg of intravenous anifrolumab for 3 doses and 300 mg thereafter (intensified regimen (IR)) or placebo on top of MMF (2 g/day) and GC. 50 The primary endpoint (change in UPCr at week 52 for combined anifrolumab vs placebo) was not met; however, when the two anifrolumab arms were analysed separately, more patients in the IR achieved complete response compared with placebo (45.5% and 31.1% respectively). Importantly, safety concerns were raised due to an increased incidence of VZV infection in the combined anifrolumab groups versus placebo (16.7% vs 8.2%). In another phase II RCT, 125 LN patients were randomly assigned to obinutuzumab, a humanised type 2 anti-CD20 monoclonal antibody, or placebo in addition to MMF and GC. 51 After 52 and 104 weeks significantly more patients in the obinutuzumab group achieved complete response (UPCr<0.5, normal renal function without worsening of baseline serum creatinine by >15% and inactive urinary sediment) compared with placebo (35% vs 23%, p=0.1 and 41% vs 23%, p=0.026, respectively).

This SLR identified only one trial (RoB with some concerns) that was specifically designed to compare different drugs as maintenance treatments. In this RCT, 215 patients with biopsy-proven LN who had previously received intravenous CYC plus GC and achieved remission were randomised 1:1 to leflunomide (20 mg/day) or AZA (100 mg/day) for 36 months. The primary endpoint, time to kidney flare, was similar between groups (16 vs 14 months, p=0.67), and there was no difference in safety profile. 52

Remission, low disease activity and associations with favourable outcomes in SLE

PICO 2 focused on the short-term and long-term benefits of attainment of treatment targets, both in extrarenal SLE and LN. The current SLR identified observational studies in which both remission (defined either per the recent Definition of Remission in SLE (DORIS) definition 53 or earlier definitions) and low disease activity (mainly defined as the lupus low disease activity state (LLDAS) 54 ) are associated with reduced risk for damage accrual ( table 4 ), as well as disease flares and other adverse sequelae (death, serious infections and hospitalisations, online supplemental table S4.4 ). In studies of good quality, range of OR for an increase in SDI were 0.49–0.75 for remission and 0.19–0.88 for LLDAS, versus patients not attaining these targets. Similarly, observational studies in LN examining the association between complete remission at variable time-points and favourable long-term kidney outcomes are shown in online supplemental table S4.5

Association of attainment of remission or LLDAS with risk for damage accrual

Safety of treatment tapering in SLE

PICO 4 addressed the issue of safety of tapering and/or withdrawal of immunosuppressive treatment in patients with SLE who have quiescent disease. Studies were categorised according to tapering of (1) GC, (2) immunosuppressive drugs and (3) antimalarials. For GC, a randomised study (CORTICOLUP) found higher rate of flares in patients with SLE on chronic prednisone 5 mg/day who discontinued GC, versus those who continued this dose. 55 A meta-analysis reported a pooled incidence of 24% (95% CI 21 to 27) and 13% (95% CI 8 to 18) for global and major flares, respectively, following GC withdrawal 56 ; a different meta-analysis focusing on risk factors found an increased risk for flare in serologically active, clinically quiescent disease after GC withdrawal (OR 1.78, 95% CI 1.00 to 3.15), while HCQ use trended towards decreased risk of flare, however results were not statistically significant (OR 0.50, 95% CI 0.23 to 1.07). Individual observational studies of the current SLR are shown in table 5 and support that gradual tapering to discontinuation of GC may be achieved without increasing the risk for flares, especially with slow tapering and long-standing remission prior to complete withdrawal (although most of these did not have a control patient group which did not discontinue GC).

Studies evaluating tapering and withdrawal of glucocorticoids in patients with SLE

Contrary to GC, although a similar RCT of withdrawal versus continuation has not been performed, discontinuation of antimalarials is more frequently associated with increased risk of flares. Four observational studies addressed this issue. Large observational studies from the multicentre Systemic Lupus International Collaborating Clinics (SLICC) cohort, 57 the Toronto Lupus cohort, 58 as well as five other SLE cohorts in Canada, 59 reported higher rates of disease flares in patients with SLE who stopped HCQ compared with patients who continued, with HR ranging from 1.56 57 to 2.30. 58 Tapering HCQ to a lower dose seems to be associated with a lower risk for flare, as patients in the Toronto cohort who tapered had significantly fewer flares versus abrupt discontinuation (45.9% vs 72.6%; p=0.01), 58 while the respective risk for flare in the SLICC study for those with HCQ dose reduction was 1.20 (95% CI 1.04 to 1.38) compared with patients who continued. 57

Finally, regarding withdrawal of synthetic immunosuppressive drugs, a limited number of studies have been published, mainly in LN. The Weaning of Immunosuppressive Therapy in Lupus Nephritis (WIN-Lupus) study randomised 96 patients with proliferative LN in remission after 2–3 years of immunosuppression to treatment discontinuation versus maintenance. 60 Relapses of LN (27.3% vs 12.5%), as well as severe disease flares (31.8% vs 12.5%), were significantly more common in the discontinuation group. An Italian uncontrolled observational study reported a 22.9% relapse rate (19/83 patients) in LN patients who discontinued immunosuppression. Antimalarial treatment and longer duration of remission (>3 years) at the time of therapy withdrawal were associated with lower risk of LN relapse. 61

Safety of herpes zoster and SARS-CoV2 vaccination in SLE

The final PICO focused on prevention of specific infections in SLE, namely VZV and COVID-19, rather than on general preventive measures for infections (vaccinations, etc), for which specific EULAR recommendations exist and are regularly updated. 7 These particular infections were chosen, because of the impact of zoster on patients with SLE (in view also of the potential increased risk with new therapies, such as interferon inhibitors), 62 and the public health problem imposed by the COVID pandemic, most obvious in populations with immunosuppression. 63

Regarding efficacy and safety of the zoster vaccine in patients with SLE, we identified three studies assessing the newer recombinant, adjuvanted vaccine (Shingrix) in patients with systemic autoimmune diseases, which also included a small subset with SLE. A study in 403 patients (16 with SLE) found a flare rate of 7.1% in the SLE group (all were mild), as well as one zoster breakthrough case. 64 Another study on 622 patients (24 with SLE) reported mild flares in 4/24 patients with SLE (17%), all treated only with GC. 65 The third, larger study, using two claims databases from the USA to estimate recombinant zoster vaccination among adults aged≥50 years with systemic autoimmune diseases and possible vaccine-related flares, found no statistically significant increase in flares for any autoimmune disease following either dose of recombinant vaccine (more than 4500 patients with SLE in the two databases, risk ratio for flare in the risk window vs control window 0.9–1.0 in this group). 66 Formerly, the live attenuated vaccine (Zostavax) was tested in a single, high-quality RCT in 90 quiescent patients with SLE (plus 10 healthy controls), testing VZV IgG reactivity and safety at 6 weeks. 67 Both anti-VZV IgG and T-cell spots increased significantly in herpes zoster-vaccinated patients, in a similar magnitude to healthy controls, while only two patients experienced a mild/moderate flare.

Regarding the immunogenicity and safety of SARS-CoV2 vaccination in patients with SLE, the SLR identified a significant number of studies ( online supplemental table S4.6 ). A meta-analysis, including 32 studies and 8269 patients in total, tested clinical effectiveness (ie, prevention from COVID-19), immunogenicity and safety, and found a pooled seropositivity rate 81.1% following various anti-SARS-CoV2 vaccine formulations (higher with mRNA vaccines), very rare severe adverse events (<1%), as well as a cumulative flare rate 5.5% 68 ; however, moderate or severe flares were reported only in 0%–2% of patients in all but one studies ( online supplemental table S4.6 ). Additionally, seven studies addressed the influence of concomitant or background immunosuppression on vaccine immunogenicity ( online supplemental table S4.7 ). As shown in these studies, concomitant use of MMF, RTX and possible GC was associated with lower patient ability to mount immune responses to SARS-CoV2 vaccination.

For the recent update of the EULAR recommendations for the management of SLE, we performed five different SLRs based on respective PICOs, to cover the most important aspects in the treatment of this challenging disease.

HCQ is the backbone treatment for all patients with SLE, while GC are still used in the majority of patients. The current SLR confirmed the beneficial effects of HCQ in lupus, ranging from prevention of infections or thrombosis to improved survival. Regarding retinal toxicity, although studies seem to converge to longer duration of use and higher cumulative dose as major risk factors for this complication, the actual rate of this complication had wide variation among studies, possibly in part due to different screening techniques used and definitions applied. We did not document other major safety signals. On the contrary, the current SLR confirmed the correlation of chronic GC use with multiple adverse outcomes in SLE (eg, susceptibility to infections, osteonecrosis, irreversible damage, among others). It should be noted that the recommended lowering of the maximum maintenance dose to 5 mg/day (instead of 7.5 mg/day) was not based on a randomised trial comparing the safety of these two different maintenance doses. Nevertheless, most observational studies that tested threshold daily prednisone doses in relation to adverse events pointed to the 5 mg/day, as well as to a stronger association with increasing doses (see table 1 ).

For the use of conventional and biologic immunosuppressive drugs in extrarenal SLE, the approved biologics anifrolumab and belimumab have proven efficacy in the form of high-quality RCTs with low RoB, compared with standard of care. Importantly, RCTs have become more elaborate in recent years, because in the anifrolumab studies, organ-specific endpoints, such as the CLASI and tender/swollen joint counts, were applied (belimumab studies had used SLEDAI and BILAG domains). RCTs are not available for conventional immunosuppressive agents in extrarenal SLE and are unlikely to be performed in the future due to the long experience with the everyday use of the drugs. Additionally, there are very few comparative studies between different immunosuppressive agents, (MTX, AZA, MMF, etc) all prior to the starting date of the current SLR.

Regarding the treatment of LN, equal efficacy of standard of care treatment, MMF and CYC, was again confirmed in additional comparative studies, mainly of low quality. More importantly, two high-quality RCTs with low RoB led to the approval of belimumab and voclosporin for the treatment of active LN. 4 5 These RCTs were the largest that have been performed in LN to date, and the BLISS-LN additionally used a novel response definition (PERR) and used an extended time-point at 2 years (all other RCTs of ‘induction’ therapies in LN have tested efficacy at 6 or 12 months). Post hoc analyses of both BLISS-LN and AURORA did not find a statistically significant benefit of any of the drugs in class 5 LN, but patients with this histologic class represented less than 20% of the study population in both studies; belimumab was also found to perform better in patients with baseline proteinuria less than 3 g/day.

For treatment targets of SLE, our SLR provided robust evidence for the positive association of remission and LLDAS with lower risk for multiple adverse outcomes, including damage ( table 4 ), flares, mortality and hospitalisation. Although the two states are comparable in terms of prognosis, data point towards slightly lower odds for damage accrual for remission over LLDAS; on the other hand, LLDAS is achieved more frequently than DORIS remission. The prognostic significance of both conditions has been tested in longitudinal cohorts of patients receiving routine care. Interestingly, a randomised trial has been designed to test whether a ‘treat-to-target’ approach aiming at remission or LLDAS confers additional benefit over standard of care. 69

Two randomised studies, CORTICOLUP and WIN-LUPUS, tested the discontinuation of prednisone (CORTICOLUP) and immunosuppressive agents (WIN-LUPUS) in extrarenal SLE and LN, respectively. 55 60 Although both studies found higher rates of relapse in patients that discontinued treatment, and withstanding their limitations (eg, CORTICOLUP was criticised for the abrupt—rather than more gradual—stopping of prednisone from 5 mg/day), they have opened the way for similar trials in SLE. A number of cohort studies have been reported with successful discontinuation, especially of GC, without an increased risk for flare in the majority of patients.

Some methodological considerations of our work merit explanation. Since high-quality studies are lacking for most organ manifestations of SLE, we adopted an inclusive approach during article screening and selection, in order to capture evidence from observational and non-controlled studies for topics where RCTs are absent or scarce. This led to inclusion of a large number of studies (n=439), many of which had limited contribution to the conclusions regarding drug efficacy for specific manifestations. This issue is particularly relevant for conventional immunosuppressive drugs, which are often used to treat extrarenal lupus manifestations, but lack support from randomised evidence. With improved trial design and approval of new drugs (mainly biologics), we anticipate that SLR for future updates of SLE recommendations will focus more on RCTs and high-quality observational studies with low RoB. Additionally, our SLR did not include the EMBASE database, and Medline was partially captured through PubMed. We acknowledge that this may have led to omission of some studies, nevertheless the multiple sources used for our SLR (PubMed, Cochrane, hand search of references of included studies) has reduced the possibility of leaving out significant studies.

In conclusion, the dedicated SLRs that supported the update of the EULAR recommendations for the management of SLE found high-quality data for the efficacy of biologic agents in treating the disease (anifrolumab and belimumab) and for the new treatment options in LN (RCT with low RoB for belimumab and voclosporin), but low-to-moderate quality concerning most other aspects of the disease. Additionally, treatment targets, such as remission and low disease activity, show a robust and consistent association with several favourable outcomes, supporting their establishment as the goal of therapy in SLE. Studies (some of them randomised) addressing the issue of treatment tapering in lupus patients in remission have also been published since the previous recommendations, following the paradigm of rheumatoid arthritis and spondylarthritis.

Ethics statements

Patient consent for publication.

Not applicable.

Ethics approval

Acknowledgments.

We wish to acknowledge the support of the EULAR Quality of Care Committee and express our sincere appreciation and gratitude to the EULAR Secretariat, especially Simona Lupatin, executive assistant and to Dora Togia for the outstanding organisation and coordination.

• Fanouriakis A ,
• Tziolos N ,
• Bertsias G , et al
• Morand EF ,
• Tanaka Y , et al
• Bruce IN , et al
• Houssiau F , et al
• Ginzler EM , et al
• Kostopoulou M ,
• Andersen J , et al
• Rondaan C ,
• Heijstek MW , et al
• Yang Y , et al
• Alunno A , et al
• Melles RB ,
• Mancini C ,
• Zhou B , et al
• Tselios K ,
• Gladman DD ,
• Al‐Sheikh H , et al
• Bandhan IH ,
• Ahmad HI , et al
• Bharati J ,
• Ramachandran R , et al
• Ginzler E ,
• Guedes Barbosa LS ,
• D’Cruz D , et al
• Bass D , et al
• Sheikh SZ ,
• Scheinberg MA ,
• Wei J-C , et al
• Wallace DJ ,
• Daniels M , et al
• Askanase AD , et al
• Merrill JT ,
• Morand EF , et al
• Kalunian KC ,
• van Vollenhoven RF ,
• Cervera R , et al
• Kneeland R ,
• Endo J , et al
• Zen M , et al
• Parodis I ,
• Sjöwall C ,
• Jönsen A , et al
• Iaccarino L ,
• Reggia R , et al
• Werth VP , et al
• Cappuccio AM ,
• Caprarulo C , et al
• Ceccarelli F ,
• Cipriano E ,
• Natalucci F , et al
• Alshaiki F ,
• Almuallim A , et al
• Sedhain A ,
• Agrawal RK , et al
• Usdadiya JB ,
• Jain VK , et al
• Peng X , et al
• Kamanamool N ,
• Ingsathit A ,
• Rattanasiri S , et al
• Wang M , et al
• Solomons N ,
• Pendergraft WF , et al
• Ying SKY , et al
• Yang S , et al
• Gao Y , et al
• Arriens C ,
• Ginzler EM ,
• Gibson K , et al
• Teng YKO , et al
• Atisha-Fregoso Y ,
• Malkiel S ,
• Harris KM , et al
• Mysler EF , et al
• Cascino MD , et al
• Dai M , et al
• Bertsias G ,
• Doria A , et al
• Franklyn K ,
• Navarra SV , et al
• Mathian A ,
• Haroche J , et al
• Almeida-Brasil CC ,
• Urowitz M , et al
• Papachristos DA ,
• Su J , et al
• Almeida‐Brasil CC ,
• Pineau CA ,
• Vinet E , et al
• Jourde-Chiche N ,
• Costedoat-Chalumeau N ,
• Baumstarck K , et al
• Loredo Martinez M , et al
• Lazova T , et al
• Panagiotopoulos A ,
• Thomas K , et al
• Stevens E ,
• Weinblatt ME ,
• Massarotti E , et al
• Lenfant T ,
• Kirchner E , et al
• Anderson TC ,
• Dooling K , et al
• Ho LY , et al
• Sim JJL , et al
• Brinks R , et al
• Abdelbaky MSE ,
• El Mamoun TA ,
• Mabrouk FI , et al
• Elkhalifa M ,
• Li J , et al
• Ototake Y ,
• Yamaguchi Y ,
• Kanaoka M , et al
• Martín-Iglesias D ,
• Artaraz J ,
• Fonollosa A , et al
• Leroux G , et al
• Lu C-H , et al
• Hernández-Rodríguez J ,
• Pelegrín L , et al
• Park J-H , et al
• Spinelli FR ,
• Moscarelli E ,
• Ceccarelli F , et al
• Zha Y , et al
• Kikuchi J ,
• Hanaoka H ,
• Saito S , et al
• Ji L , et al
• Alarcón GS ,
• Ugarte-Gil MF ,
• Pons-Estel G , et al
• Malek Mahdavi A ,
• Khabbazi A , et al
• Jakez-Ocampo J ,
• Rodriguez-Armida M ,
• Fragoso-Loyo H , et al
• Kandane-Rathnayake R ,
• Huq M , et al
• Choi S-E , et al
• Raymond W ,
• Eilertsen G , et al
• Gamboa-Cardenas RV ,
• Reátegui-Sokolova C , et al
• Perra D , et al
• Vagelli R ,
• Stagnaro C , et al
• Louthrenoo W ,
• Hoi A , et al
• Touma Z , et al
• Sebastiani GD , et al
• Ikeda Y , et al
• Hao Y , et al
• Coscia MA ,
• Pierro L , et al
• Elefante E ,
• Signorini V , et al
• Goswami RP ,
• Ghosh P , et al
• Kiyokawa T , et al

Supplementary materials

Supplementary data.

This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

• Data supplement 1

Handling editor Kimme L Hyrich

X @george_bertsias

Contributors MK and AF drafted the manuscript, CBM, GB and DTB edited the manuscript, and AF provided the final version. AF acts as the guarantor responsible for the overall content of the manuscript.

Funding This study was supported by the European League Against Rheumatism (project number QoC015).

Competing interests AF reports honoraria and/or consulting fees from Lilly, Boehringer, Novartis, AbbVie, AstraZeneca, GSK, MSD, Pfizer, UCB, Amgen, Aenorasis, support for attending meetings from UCB. MK reports honoraria and/or consulting fees from GSK, participation in advisory boards from GSK, AstraZeneca, Amgen. GB reports grants from GSK, AstraZeneca, Pfizer, honoraria and/or consulting fees from Lilly, Aenorasis, Novartis, AstraZeneca, GSK, SOBI, Pfizer, participation in advisory boards from Novartis. DTB reports unrestricted investigational grants from GSK, honoraria and/or consulting fees from GSK, AstraZeneca, Pfizer. CBM declares no conflict of interest.

Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Read the full text or download the PDF: