reporting guidelines for quantitative research

Writing Quantitative Research Studies

• Reference work entry
• First Online: 13 January 2019
• Cite this reference work entry

• Ankur Singh 2 ,
• Adyya Gupta 3 &
• Karen G. Peres 4  

1261 Accesses

1 Citations

Summarizing quantitative data and its effective presentation and discussion can be challenging for students and researchers. This chapter provides a framework for adequately reporting findings from quantitative analysis in a research study for those contemplating to write a research paper. The rationale underpinning the reporting methods to maintain the credibility and integrity of quantitative studies is outlined. Commonly used terminologies in empirical studies are defined and discussed with suitable examples. Key elements that build consistency between different sections (background, methods, results, and the discussion) of a research study using quantitative methods in a journal article are explicated. Specifically, recommended standard guidelines for randomized controlled trials and observational studies for reporting and discussion of findings from quantitative studies are elaborated. Key aspects of methodology that include describing the study population, sampling strategy, data collection methods, measurements/variables, and statistical analysis which informs the quality of a study from the reviewer’s perspective are described. Effective use of references in the methods section to strengthen the rationale behind specific statistical techniques and choice of measures has been highlighted with examples. Identifying ways in which data can be most succinctly and effectively summarized in tables and graphs according to their suitability and purpose of information is also detailed in this chapter. Strategies to present and discuss the quantitative findings in a structured discussion section are also provided. Overall, the chapter provides the readers with a comprehensive set of tools to identify key strategies to be considered when reporting quantitative research.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Bhaumik S, Arora M, Singh A, Sargent JD. Impact of entertainment media smoking on adolescent smoking behaviours. Cochrane Database Syst Rev. 2015;6:1–12. https://doi.org/10.1002/14651858.CD011720 .

Article   Google Scholar  

Dickersin K, Manheimer E, Wieland S, Robinson KA, Lefebvre C, McDonald S. Development of the Cochrane Collaboration’s CENTRAL register of controlled clinical trials. Eval Health Prof. 2002;25(1):38–64.

Google Scholar  

Docherty M, Smith R. The case for structuring the discussion of scientific papers: much the same as that for structuring abstracts. Br Med J. 1999;318(7193):1224–5.

Greenland S, Pearl J, Robins JM. Causal diagrams for epidemiologic research. Epidemiology. 1999;10(1):37–48.

Horton R. The rhetoric of research. Br Med J. 1995;310(6985):985–7.

Kool B, Ziersch A, Robinson P, Wolfenden L, Lowe JB. The ‘Seven deadly sins’ of rejected papers. Aust N Z J Public Health. 2016;40(1):3–4.

Mannocci A, Saulle R, Colamesta V, D’Aguanno S, Giraldi G, Maffongelli E, et al. What is the impact of reporting guidelines on public health journals in Europe? The case of STROBE, CONSORT and PRISMA. J Public Health. 2015;37(4):737–40.

Rothwell PM. External validity of randomised controlled trials: “to whom do the results of this trial apply?”. Lancet. 2005;365(9453):82–93.

Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. PLoS Med. 2010;7(3):e1000251.

Szklo M. Quality of scientific articles. Rev Saude Publica. 2006;40 Spec no:30–5.

Vandenbroucke JP, von Elm E, Altman DG, Gotzsche PC, Mulrow CD, Pocock SJ, et al. Strengthening the reporting of observational studies in epidemiology (STROBE): explanation and elaboration. PLoS Med. 2007;4(10):e297.

Weiss NS, Koepsell TD, Psaty BM. Generalizability of the results of randomized trials. Arch Intern Med. 2008;168(2):133–5.

Singh A, Gupta A, Peres MA, Watt RG, Tsakos G, Mathur MR. Association between tooth loss and hypertension among a primarily rural middle aged and older Indian adult population. J Public Health Dent. 2016;76:198–205.

Download references

Author information

Authors and affiliations.

Centre for Health Equity, Melbourne School of Population and Global Health, The University of Melbourne, Melbourne, VIC, Australia

Ankur Singh

School of Public Health, The University of Adelaide, Adelaide, SA, Australia

Adyya Gupta

Australian Research Centre for Population Oral Health (ARCPOH), Adelaide Dental School, The University of Adelaide, Adelaide, SA, Australia

Karen G. Peres

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Ankur Singh .

Editor information

Editors and affiliations.

School of Science and Health, Western Sydney University, Penrith, NSW, Australia

Pranee Liamputtong

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Singapore Pte Ltd.

About this entry

Cite this entry.

Singh, A., Gupta, A., Peres, K.G. (2019). Writing Quantitative Research Studies. In: Liamputtong, P. (eds) Handbook of Research Methods in Health Social Sciences. Springer, Singapore. https://doi.org/10.1007/978-981-10-5251-4_117

Download citation

DOI : https://doi.org/10.1007/978-981-10-5251-4_117

Published : 13 January 2019

Publisher Name : Springer, Singapore

Print ISBN : 978-981-10-5250-7

Online ISBN : 978-981-10-5251-4

eBook Packages : Social Sciences Reference Module Humanities and Social Sciences Reference Module Business, Economics and Social Sciences

Share this entry

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Korean Med Sci
• v.37(16); 2022 Apr 25

A Practical Guide to Writing Quantitative and Qualitative Research Questions and Hypotheses in Scholarly Articles

Edward barroga.

1 Department of General Education, Graduate School of Nursing Science, St. Luke’s International University, Tokyo, Japan.

Glafera Janet Matanguihan

2 Department of Biological Sciences, Messiah University, Mechanicsburg, PA, USA.

The development of research questions and the subsequent hypotheses are prerequisites to defining the main research purpose and specific objectives of a study. Consequently, these objectives determine the study design and research outcome. The development of research questions is a process based on knowledge of current trends, cutting-edge studies, and technological advances in the research field. Excellent research questions are focused and require a comprehensive literature search and in-depth understanding of the problem being investigated. Initially, research questions may be written as descriptive questions which could be developed into inferential questions. These questions must be specific and concise to provide a clear foundation for developing hypotheses. Hypotheses are more formal predictions about the research outcomes. These specify the possible results that may or may not be expected regarding the relationship between groups. Thus, research questions and hypotheses clarify the main purpose and specific objectives of the study, which in turn dictate the design of the study, its direction, and outcome. Studies developed from good research questions and hypotheses will have trustworthy outcomes with wide-ranging social and health implications.

INTRODUCTION

Scientific research is usually initiated by posing evidenced-based research questions which are then explicitly restated as hypotheses. 1 , 2 The hypotheses provide directions to guide the study, solutions, explanations, and expected results. 3 , 4 Both research questions and hypotheses are essentially formulated based on conventional theories and real-world processes, which allow the inception of novel studies and the ethical testing of ideas. 5 , 6

It is crucial to have knowledge of both quantitative and qualitative research 2 as both types of research involve writing research questions and hypotheses. 7 However, these crucial elements of research are sometimes overlooked; if not overlooked, then framed without the forethought and meticulous attention it needs. Planning and careful consideration are needed when developing quantitative or qualitative research, particularly when conceptualizing research questions and hypotheses. 4

There is a continuing need to support researchers in the creation of innovative research questions and hypotheses, as well as for journal articles that carefully review these elements. 1 When research questions and hypotheses are not carefully thought of, unethical studies and poor outcomes usually ensue. Carefully formulated research questions and hypotheses define well-founded objectives, which in turn determine the appropriate design, course, and outcome of the study. This article then aims to discuss in detail the various aspects of crafting research questions and hypotheses, with the goal of guiding researchers as they develop their own. Examples from the authors and peer-reviewed scientific articles in the healthcare field are provided to illustrate key points.

DEFINITIONS AND RELATIONSHIP OF RESEARCH QUESTIONS AND HYPOTHESES

A research question is what a study aims to answer after data analysis and interpretation. The answer is written in length in the discussion section of the paper. Thus, the research question gives a preview of the different parts and variables of the study meant to address the problem posed in the research question. 1 An excellent research question clarifies the research writing while facilitating understanding of the research topic, objective, scope, and limitations of the study. 5

On the other hand, a research hypothesis is an educated statement of an expected outcome. This statement is based on background research and current knowledge. 8 , 9 The research hypothesis makes a specific prediction about a new phenomenon 10 or a formal statement on the expected relationship between an independent variable and a dependent variable. 3 , 11 It provides a tentative answer to the research question to be tested or explored. 4

Hypotheses employ reasoning to predict a theory-based outcome. 10 These can also be developed from theories by focusing on components of theories that have not yet been observed. 10 The validity of hypotheses is often based on the testability of the prediction made in a reproducible experiment. 8

Conversely, hypotheses can also be rephrased as research questions. Several hypotheses based on existing theories and knowledge may be needed to answer a research question. Developing ethical research questions and hypotheses creates a research design that has logical relationships among variables. These relationships serve as a solid foundation for the conduct of the study. 4 , 11 Haphazardly constructed research questions can result in poorly formulated hypotheses and improper study designs, leading to unreliable results. Thus, the formulations of relevant research questions and verifiable hypotheses are crucial when beginning research. 12

CHARACTERISTICS OF GOOD RESEARCH QUESTIONS AND HYPOTHESES

Excellent research questions are specific and focused. These integrate collective data and observations to confirm or refute the subsequent hypotheses. Well-constructed hypotheses are based on previous reports and verify the research context. These are realistic, in-depth, sufficiently complex, and reproducible. More importantly, these hypotheses can be addressed and tested. 13

There are several characteristics of well-developed hypotheses. Good hypotheses are 1) empirically testable 7 , 10 , 11 , 13 ; 2) backed by preliminary evidence 9 ; 3) testable by ethical research 7 , 9 ; 4) based on original ideas 9 ; 5) have evidenced-based logical reasoning 10 ; and 6) can be predicted. 11 Good hypotheses can infer ethical and positive implications, indicating the presence of a relationship or effect relevant to the research theme. 7 , 11 These are initially developed from a general theory and branch into specific hypotheses by deductive reasoning. In the absence of a theory to base the hypotheses, inductive reasoning based on specific observations or findings form more general hypotheses. 10

TYPES OF RESEARCH QUESTIONS AND HYPOTHESES

Research questions and hypotheses are developed according to the type of research, which can be broadly classified into quantitative and qualitative research. We provide a summary of the types of research questions and hypotheses under quantitative and qualitative research categories in Table 1 .

Research questions in quantitative research

In quantitative research, research questions inquire about the relationships among variables being investigated and are usually framed at the start of the study. These are precise and typically linked to the subject population, dependent and independent variables, and research design. 1 Research questions may also attempt to describe the behavior of a population in relation to one or more variables, or describe the characteristics of variables to be measured ( descriptive research questions ). 1 , 5 , 14 These questions may also aim to discover differences between groups within the context of an outcome variable ( comparative research questions ), 1 , 5 , 14 or elucidate trends and interactions among variables ( relationship research questions ). 1 , 5 We provide examples of descriptive, comparative, and relationship research questions in quantitative research in Table 2 .

Hypotheses in quantitative research

In quantitative research, hypotheses predict the expected relationships among variables. 15 Relationships among variables that can be predicted include 1) between a single dependent variable and a single independent variable ( simple hypothesis ) or 2) between two or more independent and dependent variables ( complex hypothesis ). 4 , 11 Hypotheses may also specify the expected direction to be followed and imply an intellectual commitment to a particular outcome ( directional hypothesis ) 4 . On the other hand, hypotheses may not predict the exact direction and are used in the absence of a theory, or when findings contradict previous studies ( non-directional hypothesis ). 4 In addition, hypotheses can 1) define interdependency between variables ( associative hypothesis ), 4 2) propose an effect on the dependent variable from manipulation of the independent variable ( causal hypothesis ), 4 3) state a negative relationship between two variables ( null hypothesis ), 4 , 11 , 15 4) replace the working hypothesis if rejected ( alternative hypothesis ), 15 explain the relationship of phenomena to possibly generate a theory ( working hypothesis ), 11 5) involve quantifiable variables that can be tested statistically ( statistical hypothesis ), 11 6) or express a relationship whose interlinks can be verified logically ( logical hypothesis ). 11 We provide examples of simple, complex, directional, non-directional, associative, causal, null, alternative, working, statistical, and logical hypotheses in quantitative research, as well as the definition of quantitative hypothesis-testing research in Table 3 .

Research questions in qualitative research

Unlike research questions in quantitative research, research questions in qualitative research are usually continuously reviewed and reformulated. The central question and associated subquestions are stated more than the hypotheses. 15 The central question broadly explores a complex set of factors surrounding the central phenomenon, aiming to present the varied perspectives of participants. 15

There are varied goals for which qualitative research questions are developed. These questions can function in several ways, such as to 1) identify and describe existing conditions ( contextual research question s); 2) describe a phenomenon ( descriptive research questions ); 3) assess the effectiveness of existing methods, protocols, theories, or procedures ( evaluation research questions ); 4) examine a phenomenon or analyze the reasons or relationships between subjects or phenomena ( explanatory research questions ); or 5) focus on unknown aspects of a particular topic ( exploratory research questions ). 5 In addition, some qualitative research questions provide new ideas for the development of theories and actions ( generative research questions ) or advance specific ideologies of a position ( ideological research questions ). 1 Other qualitative research questions may build on a body of existing literature and become working guidelines ( ethnographic research questions ). Research questions may also be broadly stated without specific reference to the existing literature or a typology of questions ( phenomenological research questions ), may be directed towards generating a theory of some process ( grounded theory questions ), or may address a description of the case and the emerging themes ( qualitative case study questions ). 15 We provide examples of contextual, descriptive, evaluation, explanatory, exploratory, generative, ideological, ethnographic, phenomenological, grounded theory, and qualitative case study research questions in qualitative research in Table 4 , and the definition of qualitative hypothesis-generating research in Table 5 .

Qualitative studies usually pose at least one central research question and several subquestions starting with How or What . These research questions use exploratory verbs such as explore or describe . These also focus on one central phenomenon of interest, and may mention the participants and research site. 15

Hypotheses in qualitative research

Hypotheses in qualitative research are stated in the form of a clear statement concerning the problem to be investigated. Unlike in quantitative research where hypotheses are usually developed to be tested, qualitative research can lead to both hypothesis-testing and hypothesis-generating outcomes. 2 When studies require both quantitative and qualitative research questions, this suggests an integrative process between both research methods wherein a single mixed-methods research question can be developed. 1

FRAMEWORKS FOR DEVELOPING RESEARCH QUESTIONS AND HYPOTHESES

Research questions followed by hypotheses should be developed before the start of the study. 1 , 12 , 14 It is crucial to develop feasible research questions on a topic that is interesting to both the researcher and the scientific community. This can be achieved by a meticulous review of previous and current studies to establish a novel topic. Specific areas are subsequently focused on to generate ethical research questions. The relevance of the research questions is evaluated in terms of clarity of the resulting data, specificity of the methodology, objectivity of the outcome, depth of the research, and impact of the study. 1 , 5 These aspects constitute the FINER criteria (i.e., Feasible, Interesting, Novel, Ethical, and Relevant). 1 Clarity and effectiveness are achieved if research questions meet the FINER criteria. In addition to the FINER criteria, Ratan et al. described focus, complexity, novelty, feasibility, and measurability for evaluating the effectiveness of research questions. 14

The PICOT and PEO frameworks are also used when developing research questions. 1 The following elements are addressed in these frameworks, PICOT: P-population/patients/problem, I-intervention or indicator being studied, C-comparison group, O-outcome of interest, and T-timeframe of the study; PEO: P-population being studied, E-exposure to preexisting conditions, and O-outcome of interest. 1 Research questions are also considered good if these meet the “FINERMAPS” framework: Feasible, Interesting, Novel, Ethical, Relevant, Manageable, Appropriate, Potential value/publishable, and Systematic. 14

As we indicated earlier, research questions and hypotheses that are not carefully formulated result in unethical studies or poor outcomes. To illustrate this, we provide some examples of ambiguous research question and hypotheses that result in unclear and weak research objectives in quantitative research ( Table 6 ) 16 and qualitative research ( Table 7 ) 17 , and how to transform these ambiguous research question(s) and hypothesis(es) into clear and good statements.

a These statements were composed for comparison and illustrative purposes only.

b These statements are direct quotes from Higashihara and Horiuchi. 16

a This statement is a direct quote from Shimoda et al. 17

The other statements were composed for comparison and illustrative purposes only.

CONSTRUCTING RESEARCH QUESTIONS AND HYPOTHESES

To construct effective research questions and hypotheses, it is very important to 1) clarify the background and 2) identify the research problem at the outset of the research, within a specific timeframe. 9 Then, 3) review or conduct preliminary research to collect all available knowledge about the possible research questions by studying theories and previous studies. 18 Afterwards, 4) construct research questions to investigate the research problem. Identify variables to be accessed from the research questions 4 and make operational definitions of constructs from the research problem and questions. Thereafter, 5) construct specific deductive or inductive predictions in the form of hypotheses. 4 Finally, 6) state the study aims . This general flow for constructing effective research questions and hypotheses prior to conducting research is shown in Fig. 1 .

Research questions are used more frequently in qualitative research than objectives or hypotheses. 3 These questions seek to discover, understand, explore or describe experiences by asking “What” or “How.” The questions are open-ended to elicit a description rather than to relate variables or compare groups. The questions are continually reviewed, reformulated, and changed during the qualitative study. 3 Research questions are also used more frequently in survey projects than hypotheses in experiments in quantitative research to compare variables and their relationships.

Hypotheses are constructed based on the variables identified and as an if-then statement, following the template, ‘If a specific action is taken, then a certain outcome is expected.’ At this stage, some ideas regarding expectations from the research to be conducted must be drawn. 18 Then, the variables to be manipulated (independent) and influenced (dependent) are defined. 4 Thereafter, the hypothesis is stated and refined, and reproducible data tailored to the hypothesis are identified, collected, and analyzed. 4 The hypotheses must be testable and specific, 18 and should describe the variables and their relationships, the specific group being studied, and the predicted research outcome. 18 Hypotheses construction involves a testable proposition to be deduced from theory, and independent and dependent variables to be separated and measured separately. 3 Therefore, good hypotheses must be based on good research questions constructed at the start of a study or trial. 12

In summary, research questions are constructed after establishing the background of the study. Hypotheses are then developed based on the research questions. Thus, it is crucial to have excellent research questions to generate superior hypotheses. In turn, these would determine the research objectives and the design of the study, and ultimately, the outcome of the research. 12 Algorithms for building research questions and hypotheses are shown in Fig. 2 for quantitative research and in Fig. 3 for qualitative research.

EXAMPLES OF RESEARCH QUESTIONS FROM PUBLISHED ARTICLES

• EXAMPLE 1. Descriptive research question (quantitative research)
• - Presents research variables to be assessed (distinct phenotypes and subphenotypes)
• “BACKGROUND: Since COVID-19 was identified, its clinical and biological heterogeneity has been recognized. Identifying COVID-19 phenotypes might help guide basic, clinical, and translational research efforts.
• RESEARCH QUESTION: Does the clinical spectrum of patients with COVID-19 contain distinct phenotypes and subphenotypes? ” 19
• EXAMPLE 2. Relationship research question (quantitative research)
• - Shows interactions between dependent variable (static postural control) and independent variable (peripheral visual field loss)
• “Background: Integration of visual, vestibular, and proprioceptive sensations contributes to postural control. People with peripheral visual field loss have serious postural instability. However, the directional specificity of postural stability and sensory reweighting caused by gradual peripheral visual field loss remain unclear.
• Research question: What are the effects of peripheral visual field loss on static postural control ?” 20
• EXAMPLE 3. Comparative research question (quantitative research)
• - Clarifies the difference among groups with an outcome variable (patients enrolled in COMPERA with moderate PH or severe PH in COPD) and another group without the outcome variable (patients with idiopathic pulmonary arterial hypertension (IPAH))
• “BACKGROUND: Pulmonary hypertension (PH) in COPD is a poorly investigated clinical condition.
• RESEARCH QUESTION: Which factors determine the outcome of PH in COPD?
• STUDY DESIGN AND METHODS: We analyzed the characteristics and outcome of patients enrolled in the Comparative, Prospective Registry of Newly Initiated Therapies for Pulmonary Hypertension (COMPERA) with moderate or severe PH in COPD as defined during the 6th PH World Symposium who received medical therapy for PH and compared them with patients with idiopathic pulmonary arterial hypertension (IPAH) .” 21
• EXAMPLE 4. Exploratory research question (qualitative research)
• - Explores areas that have not been fully investigated (perspectives of families and children who receive care in clinic-based child obesity treatment) to have a deeper understanding of the research problem
• “Problem: Interventions for children with obesity lead to only modest improvements in BMI and long-term outcomes, and data are limited on the perspectives of families of children with obesity in clinic-based treatment. This scoping review seeks to answer the question: What is known about the perspectives of families and children who receive care in clinic-based child obesity treatment? This review aims to explore the scope of perspectives reported by families of children with obesity who have received individualized outpatient clinic-based obesity treatment.” 22
• EXAMPLE 5. Relationship research question (quantitative research)
• - Defines interactions between dependent variable (use of ankle strategies) and independent variable (changes in muscle tone)
• “Background: To maintain an upright standing posture against external disturbances, the human body mainly employs two types of postural control strategies: “ankle strategy” and “hip strategy.” While it has been reported that the magnitude of the disturbance alters the use of postural control strategies, it has not been elucidated how the level of muscle tone, one of the crucial parameters of bodily function, determines the use of each strategy. We have previously confirmed using forward dynamics simulations of human musculoskeletal models that an increased muscle tone promotes the use of ankle strategies. The objective of the present study was to experimentally evaluate a hypothesis: an increased muscle tone promotes the use of ankle strategies. Research question: Do changes in the muscle tone affect the use of ankle strategies ?” 23

EXAMPLES OF HYPOTHESES IN PUBLISHED ARTICLES

• EXAMPLE 1. Working hypothesis (quantitative research)
• - A hypothesis that is initially accepted for further research to produce a feasible theory
• “As fever may have benefit in shortening the duration of viral illness, it is plausible to hypothesize that the antipyretic efficacy of ibuprofen may be hindering the benefits of a fever response when taken during the early stages of COVID-19 illness .” 24
• “In conclusion, it is plausible to hypothesize that the antipyretic efficacy of ibuprofen may be hindering the benefits of a fever response . The difference in perceived safety of these agents in COVID-19 illness could be related to the more potent efficacy to reduce fever with ibuprofen compared to acetaminophen. Compelling data on the benefit of fever warrant further research and review to determine when to treat or withhold ibuprofen for early stage fever for COVID-19 and other related viral illnesses .” 24
• EXAMPLE 2. Exploratory hypothesis (qualitative research)
• - Explores particular areas deeper to clarify subjective experience and develop a formal hypothesis potentially testable in a future quantitative approach
• “We hypothesized that when thinking about a past experience of help-seeking, a self distancing prompt would cause increased help-seeking intentions and more favorable help-seeking outcome expectations .” 25
• “Conclusion
• Although a priori hypotheses were not supported, further research is warranted as results indicate the potential for using self-distancing approaches to increasing help-seeking among some people with depressive symptomatology.” 25
• EXAMPLE 3. Hypothesis-generating research to establish a framework for hypothesis testing (qualitative research)
• “We hypothesize that compassionate care is beneficial for patients (better outcomes), healthcare systems and payers (lower costs), and healthcare providers (lower burnout). ” 26
• Compassionomics is the branch of knowledge and scientific study of the effects of compassionate healthcare. Our main hypotheses are that compassionate healthcare is beneficial for (1) patients, by improving clinical outcomes, (2) healthcare systems and payers, by supporting financial sustainability, and (3) HCPs, by lowering burnout and promoting resilience and well-being. The purpose of this paper is to establish a scientific framework for testing the hypotheses above . If these hypotheses are confirmed through rigorous research, compassionomics will belong in the science of evidence-based medicine, with major implications for all healthcare domains.” 26
• EXAMPLE 4. Statistical hypothesis (quantitative research)
• - An assumption is made about the relationship among several population characteristics ( gender differences in sociodemographic and clinical characteristics of adults with ADHD ). Validity is tested by statistical experiment or analysis ( chi-square test, Students t-test, and logistic regression analysis)
• “Our research investigated gender differences in sociodemographic and clinical characteristics of adults with ADHD in a Japanese clinical sample. Due to unique Japanese cultural ideals and expectations of women's behavior that are in opposition to ADHD symptoms, we hypothesized that women with ADHD experience more difficulties and present more dysfunctions than men . We tested the following hypotheses: first, women with ADHD have more comorbidities than men with ADHD; second, women with ADHD experience more social hardships than men, such as having less full-time employment and being more likely to be divorced.” 27
• “Statistical Analysis
• ( text omitted ) Between-gender comparisons were made using the chi-squared test for categorical variables and Students t-test for continuous variables…( text omitted ). A logistic regression analysis was performed for employment status, marital status, and comorbidity to evaluate the independent effects of gender on these dependent variables.” 27

EXAMPLES OF HYPOTHESIS AS WRITTEN IN PUBLISHED ARTICLES IN RELATION TO OTHER PARTS

• EXAMPLE 1. Background, hypotheses, and aims are provided
• “Pregnant women need skilled care during pregnancy and childbirth, but that skilled care is often delayed in some countries …( text omitted ). The focused antenatal care (FANC) model of WHO recommends that nurses provide information or counseling to all pregnant women …( text omitted ). Job aids are visual support materials that provide the right kind of information using graphics and words in a simple and yet effective manner. When nurses are not highly trained or have many work details to attend to, these job aids can serve as a content reminder for the nurses and can be used for educating their patients (Jennings, Yebadokpo, Affo, & Agbogbe, 2010) ( text omitted ). Importantly, additional evidence is needed to confirm how job aids can further improve the quality of ANC counseling by health workers in maternal care …( text omitted )” 28
• “ This has led us to hypothesize that the quality of ANC counseling would be better if supported by job aids. Consequently, a better quality of ANC counseling is expected to produce higher levels of awareness concerning the danger signs of pregnancy and a more favorable impression of the caring behavior of nurses .” 28
• “This study aimed to examine the differences in the responses of pregnant women to a job aid-supported intervention during ANC visit in terms of 1) their understanding of the danger signs of pregnancy and 2) their impression of the caring behaviors of nurses to pregnant women in rural Tanzania.” 28
• EXAMPLE 2. Background, hypotheses, and aims are provided
• “We conducted a two-arm randomized controlled trial (RCT) to evaluate and compare changes in salivary cortisol and oxytocin levels of first-time pregnant women between experimental and control groups. The women in the experimental group touched and held an infant for 30 min (experimental intervention protocol), whereas those in the control group watched a DVD movie of an infant (control intervention protocol). The primary outcome was salivary cortisol level and the secondary outcome was salivary oxytocin level.” 29
• “ We hypothesize that at 30 min after touching and holding an infant, the salivary cortisol level will significantly decrease and the salivary oxytocin level will increase in the experimental group compared with the control group .” 29
• EXAMPLE 3. Background, aim, and hypothesis are provided
• “In countries where the maternal mortality ratio remains high, antenatal education to increase Birth Preparedness and Complication Readiness (BPCR) is considered one of the top priorities [1]. BPCR includes birth plans during the antenatal period, such as the birthplace, birth attendant, transportation, health facility for complications, expenses, and birth materials, as well as family coordination to achieve such birth plans. In Tanzania, although increasing, only about half of all pregnant women attend an antenatal clinic more than four times [4]. Moreover, the information provided during antenatal care (ANC) is insufficient. In the resource-poor settings, antenatal group education is a potential approach because of the limited time for individual counseling at antenatal clinics.” 30
• “This study aimed to evaluate an antenatal group education program among pregnant women and their families with respect to birth-preparedness and maternal and infant outcomes in rural villages of Tanzania.” 30
• “ The study hypothesis was if Tanzanian pregnant women and their families received a family-oriented antenatal group education, they would (1) have a higher level of BPCR, (2) attend antenatal clinic four or more times, (3) give birth in a health facility, (4) have less complications of women at birth, and (5) have less complications and deaths of infants than those who did not receive the education .” 30

Research questions and hypotheses are crucial components to any type of research, whether quantitative or qualitative. These questions should be developed at the very beginning of the study. Excellent research questions lead to superior hypotheses, which, like a compass, set the direction of research, and can often determine the successful conduct of the study. Many research studies have floundered because the development of research questions and subsequent hypotheses was not given the thought and meticulous attention needed. The development of research questions and hypotheses is an iterative process based on extensive knowledge of the literature and insightful grasp of the knowledge gap. Focused, concise, and specific research questions provide a strong foundation for constructing hypotheses which serve as formal predictions about the research outcomes. Research questions and hypotheses are crucial elements of research that should not be overlooked. They should be carefully thought of and constructed when planning research. This avoids unethical studies and poor outcomes by defining well-founded objectives that determine the design, course, and outcome of the study.

Disclosure: The authors have no potential conflicts of interest to disclose.

Author Contributions:

• Conceptualization: Barroga E, Matanguihan GJ.
• Methodology: Barroga E, Matanguihan GJ.
• Writing - original draft: Barroga E, Matanguihan GJ.
• Writing - review & editing: Barroga E, Matanguihan GJ.

• Open access
• Published: 29 March 2021

The PRISMA 2020 statement: an updated guideline for reporting systematic reviews

• Matthew J. Page   ORCID: orcid.org/0000-0002-4242-7526 1 ,
• Joanne E. McKenzie 1 ,
• Patrick M. Bossuyt 2 ,
• Isabelle Boutron 3 ,
• Tammy C. Hoffmann 4 ,
• Cynthia D. Mulrow 5 ,
• Larissa Shamseer 6 ,
• Jennifer M. Tetzlaff 7 ,
• Elie A. Akl 8 ,
• Sue E. Brennan 1 ,
• Roger Chou 9 ,
• Julie Glanville 10 ,
• Jeremy M. Grimshaw 11 ,
• Asbjørn Hróbjartsson 12 ,
• Manoj M. Lalu 13 ,
• Tianjing Li 14 ,
• Elizabeth W. Loder 15 ,
• Evan Mayo-Wilson 16 ,
• Steve McDonald 1 ,
• Luke A. McGuinness 17 ,
• Lesley A. Stewart 18 ,
• James Thomas 19 ,
• Andrea C. Tricco 20 ,
• Vivian A. Welch 21 ,
• Penny Whiting 17 &
• David Moher 22  

Systematic Reviews volume  10 , Article number:  89 ( 2021 ) Cite this article

302k Accesses

78k Citations

101 Altmetric

Metrics details

An Editorial to this article was published on 19 April 2021

The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement, published in 2009, was designed to help systematic reviewers transparently report why the review was done, what the authors did, and what they found. Over the past decade, advances in systematic review methodology and terminology have necessitated an update to the guideline. The PRISMA 2020 statement replaces the 2009 statement and includes new reporting guidance that reflects advances in methods to identify, select, appraise, and synthesise studies. The structure and presentation of the items have been modified to facilitate implementation. In this article, we present the PRISMA 2020 27-item checklist, an expanded checklist that details reporting recommendations for each item, the PRISMA 2020 abstract checklist, and the revised flow diagrams for original and updated reviews. In order to encourage its wide dissemination this article is freely accessible on BMJ, PLOS Medicine, Journal of Clinical Epidemiology and International Journal of Surgery journal websites.

Systematic reviews serve many critical roles. They can provide syntheses of the state of knowledge in a field, from which future research priorities can be identified; they can address questions that otherwise could not be answered by individual studies; they can identify problems in primary research that should be rectified in future studies; and they can generate or evaluate theories about how or why phenomena occur. Systematic reviews therefore generate various types of knowledge for different users of reviews (such as patients, healthcare providers, researchers, and policy makers) [ 1 , 2 ]. To ensure a systematic review is valuable to users, authors should prepare a transparent, complete, and accurate account of why the review was done, what they did (such as how studies were identified and selected) and what they found (such as characteristics of contributing studies and results of meta-analyses). Up-to-date reporting guidance facilitates authors achieving this [ 3 ].

The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement published in 2009 (hereafter referred to as PRISMA 2009) [ 4 , 5 , 6 , 7 , 8 , 9 , 10 ] is a reporting guideline designed to address poor reporting of systematic reviews [ 11 ]. The PRISMA 2009 statement comprised a checklist of 27 items recommended for reporting in systematic reviews and an “explanation and elaboration” paper [ 12 , 13 , 14 , 15 , 16 ] providing additional reporting guidance for each item, along with exemplars of reporting. The recommendations have been widely endorsed and adopted, as evidenced by its co-publication in multiple journals, citation in over 60,000 reports (Scopus, August 2020), endorsement from almost 200 journals and systematic review organisations, and adoption in various disciplines. Evidence from observational studies suggests that use of the PRISMA 2009 statement is associated with more complete reporting of systematic reviews [ 17 , 18 , 19 , 20 ], although more could be done to improve adherence to the guideline [ 21 ].

Many innovations in the conduct of systematic reviews have occurred since publication of the PRISMA 2009 statement. For example, technological advances have enabled the use of natural language processing and machine learning to identify relevant evidence [ 22 , 23 , 24 ], methods have been proposed to synthesise and present findings when meta-analysis is not possible or appropriate [ 25 , 26 , 27 ], and new methods have been developed to assess the risk of bias in results of included studies [ 28 , 29 ]. Evidence on sources of bias in systematic reviews has accrued, culminating in the development of new tools to appraise the conduct of systematic reviews [ 30 , 31 ]. Terminology used to describe particular review processes has also evolved, as in the shift from assessing “quality” to assessing “certainty” in the body of evidence [ 32 ]. In addition, the publishing landscape has transformed, with multiple avenues now available for registering and disseminating systematic review protocols [ 33 , 34 ], disseminating reports of systematic reviews, and sharing data and materials, such as preprint servers and publicly accessible repositories. To capture these advances in the reporting of systematic reviews necessitated an update to the PRISMA 2009 statement.

Development of PRISMA 2020

A complete description of the methods used to develop PRISMA 2020 is available elsewhere [ 35 ]. We identified PRISMA 2009 items that were often reported incompletely by examining the results of studies investigating the transparency of reporting of published reviews [ 17 , 21 , 36 , 37 ]. We identified possible modifications to the PRISMA 2009 statement by reviewing 60 documents providing reporting guidance for systematic reviews (including reporting guidelines, handbooks, tools, and meta-research studies) [ 38 ]. These reviews of the literature were used to inform the content of a survey with suggested possible modifications to the 27 items in PRISMA 2009 and possible additional items. Respondents were asked whether they believed we should keep each PRISMA 2009 item as is, modify it, or remove it, and whether we should add each additional item. Systematic review methodologists and journal editors were invited to complete the online survey (110 of 220 invited responded). We discussed proposed content and wording of the PRISMA 2020 statement, as informed by the review and survey results, at a 21-member, two-day, in-person meeting in September 2018 in Edinburgh, Scotland. Throughout 2019 and 2020, we circulated an initial draft and five revisions of the checklist and explanation and elaboration paper to co-authors for feedback. In April 2020, we invited 22 systematic reviewers who had expressed interest in providing feedback on the PRISMA 2020 checklist to share their views (via an online survey) on the layout and terminology used in a preliminary version of the checklist. Feedback was received from 15 individuals and considered by the first author, and any revisions deemed necessary were incorporated before the final version was approved and endorsed by all co-authors.

The PRISMA 2020 statement

Scope of the guideline.

The PRISMA 2020 statement has been designed primarily for systematic reviews of studies that evaluate the effects of health interventions, irrespective of the design of the included studies. However, the checklist items are applicable to reports of systematic reviews evaluating other interventions (such as social or educational interventions), and many items are applicable to systematic reviews with objectives other than evaluating interventions (such as evaluating aetiology, prevalence, or prognosis). PRISMA 2020 is intended for use in systematic reviews that include synthesis (such as pairwise meta-analysis or other statistical synthesis methods) or do not include synthesis (for example, because only one eligible study is identified). The PRISMA 2020 items are relevant for mixed-methods systematic reviews (which include quantitative and qualitative studies), but reporting guidelines addressing the presentation and synthesis of qualitative data should also be consulted [ 39 , 40 ]. PRISMA 2020 can be used for original systematic reviews, updated systematic reviews, or continually updated (“living”) systematic reviews. However, for updated and living systematic reviews, there may be some additional considerations that need to be addressed. Where there is relevant content from other reporting guidelines, we reference these guidelines within the items in the explanation and elaboration paper [ 41 ] (such as PRISMA-Search [ 42 ] in items 6 and 7, Synthesis without meta-analysis (SWiM) reporting guideline [ 27 ] in item 13d). Box 1 includes a glossary of terms used throughout the PRISMA 2020 statement.

PRISMA 2020 is not intended to guide systematic review conduct, for which comprehensive resources are available [ 43 , 44 , 45 , 46 ]. However, familiarity with PRISMA 2020 is useful when planning and conducting systematic reviews to ensure that all recommended information is captured. PRISMA 2020 should not be used to assess the conduct or methodological quality of systematic reviews; other tools exist for this purpose [ 30 , 31 ]. Furthermore, PRISMA 2020 is not intended to inform the reporting of systematic review protocols, for which a separate statement is available (PRISMA for Protocols (PRISMA-P) 2015 statement [ 47 , 48 ]). Finally, extensions to the PRISMA 2009 statement have been developed to guide reporting of network meta-analyses [ 49 ], meta-analyses of individual participant data [ 50 ], systematic reviews of harms [ 51 ], systematic reviews of diagnostic test accuracy studies [ 52 ], and scoping reviews [ 53 ]; for these types of reviews we recommend authors report their review in accordance with the recommendations in PRISMA 2020 along with the guidance specific to the extension.

How to use PRISMA 2020

The PRISMA 2020 statement (including the checklists, explanation and elaboration, and flow diagram) replaces the PRISMA 2009 statement, which should no longer be used. Box  2 summarises noteworthy changes from the PRISMA 2009 statement. The PRISMA 2020 checklist includes seven sections with 27 items, some of which include sub-items (Table  1 ). A checklist for journal and conference abstracts for systematic reviews is included in PRISMA 2020. This abstract checklist is an update of the 2013 PRISMA for Abstracts statement [ 54 ], reflecting new and modified content in PRISMA 2020 (Table  2 ). A template PRISMA flow diagram is provided, which can be modified depending on whether the systematic review is original or updated (Fig.  1 ).

 PRISMA 2020 flow diagram template for systematic reviews. The new design is adapted from flow diagrams proposed by Boers [ 55 ], Mayo-Wilson et al. [ 56 ] and Stovold et al. [ 57 ] The boxes in grey should only be completed if applicable; otherwise they should be removed from the flow diagram. Note that a “report” could be a journal article, preprint, conference abstract, study register entry, clinical study report, dissertation, unpublished manuscript, government report or any other document providing relevant information

We recommend authors refer to PRISMA 2020 early in the writing process, because prospective consideration of the items may help to ensure that all the items are addressed. To help keep track of which items have been reported, the PRISMA statement website ( http://www.prisma-statement.org/ ) includes fillable templates of the checklists to download and complete (also available in Additional file 1 ). We have also created a web application that allows users to complete the checklist via a user-friendly interface [ 58 ] (available at https://prisma.shinyapps.io/checklist/ and adapted from the Transparency Checklist app [ 59 ]). The completed checklist can be exported to Word or PDF. Editable templates of the flow diagram can also be downloaded from the PRISMA statement website.

We have prepared an updated explanation and elaboration paper, in which we explain why reporting of each item is recommended and present bullet points that detail the reporting recommendations (which we refer to as elements) [ 41 ]. The bullet-point structure is new to PRISMA 2020 and has been adopted to facilitate implementation of the guidance [ 60 , 61 ]. An expanded checklist, which comprises an abridged version of the elements presented in the explanation and elaboration paper, with references and some examples removed, is available in Additional file 2 . Consulting the explanation and elaboration paper is recommended if further clarity or information is required.

Journals and publishers might impose word and section limits, and limits on the number of tables and figures allowed in the main report. In such cases, if the relevant information for some items already appears in a publicly accessible review protocol, referring to the protocol may suffice. Alternatively, placing detailed descriptions of the methods used or additional results (such as for less critical outcomes) in supplementary files is recommended. Ideally, supplementary files should be deposited to a general-purpose or institutional open-access repository that provides free and permanent access to the material (such as Open Science Framework, Dryad, figshare). A reference or link to the additional information should be included in the main report. Finally, although PRISMA 2020 provides a template for where information might be located, the suggested location should not be seen as prescriptive; the guiding principle is to ensure the information is reported.

Use of PRISMA 2020 has the potential to benefit many stakeholders. Complete reporting allows readers to assess the appropriateness of the methods, and therefore the trustworthiness of the findings. Presenting and summarising characteristics of studies contributing to a synthesis allows healthcare providers and policy makers to evaluate the applicability of the findings to their setting. Describing the certainty in the body of evidence for an outcome and the implications of findings should help policy makers, managers, and other decision makers formulate appropriate recommendations for practice or policy. Complete reporting of all PRISMA 2020 items also facilitates replication and review updates, as well as inclusion of systematic reviews in overviews (of systematic reviews) and guidelines, so teams can leverage work that is already done and decrease research waste [ 36 , 62 , 63 ].

We updated the PRISMA 2009 statement by adapting the EQUATOR Network’s guidance for developing health research reporting guidelines [ 64 ]. We evaluated the reporting completeness of published systematic reviews [ 17 , 21 , 36 , 37 ], reviewed the items included in other documents providing guidance for systematic reviews [ 38 ], surveyed systematic review methodologists and journal editors for their views on how to revise the original PRISMA statement [ 35 ], discussed the findings at an in-person meeting, and prepared this document through an iterative process. Our recommendations are informed by the reviews and survey conducted before the in-person meeting, theoretical considerations about which items facilitate replication and help users assess the risk of bias and applicability of systematic reviews, and co-authors’ experience with authoring and using systematic reviews.

Various strategies to increase the use of reporting guidelines and improve reporting have been proposed. They include educators introducing reporting guidelines into graduate curricula to promote good reporting habits of early career scientists [ 65 ]; journal editors and regulators endorsing use of reporting guidelines [ 18 ]; peer reviewers evaluating adherence to reporting guidelines [ 61 , 66 ]; journals requiring authors to indicate where in their manuscript they have adhered to each reporting item [ 67 ]; and authors using online writing tools that prompt complete reporting at the writing stage [ 60 ]. Multi-pronged interventions, where more than one of these strategies are combined, may be more effective (such as completion of checklists coupled with editorial checks) [ 68 ]. However, of 31 interventions proposed to increase adherence to reporting guidelines, the effects of only 11 have been evaluated, mostly in observational studies at high risk of bias due to confounding [ 69 ]. It is therefore unclear which strategies should be used. Future research might explore barriers and facilitators to the use of PRISMA 2020 by authors, editors, and peer reviewers, designing interventions that address the identified barriers, and evaluating those interventions using randomised trials. To inform possible revisions to the guideline, it would also be valuable to conduct think-aloud studies [ 70 ] to understand how systematic reviewers interpret the items, and reliability studies to identify items where there is varied interpretation of the items.

We encourage readers to submit evidence that informs any of the recommendations in PRISMA 2020 (via the PRISMA statement website: http://www.prisma-statement.org/ ). To enhance accessibility of PRISMA 2020, several translations of the guideline are under way (see available translations at the PRISMA statement website). We encourage journal editors and publishers to raise awareness of PRISMA 2020 (for example, by referring to it in journal “Instructions to authors”), endorsing its use, advising editors and peer reviewers to evaluate submitted systematic reviews against the PRISMA 2020 checklists, and making changes to journal policies to accommodate the new reporting recommendations. We recommend existing PRISMA extensions [ 47 , 49 , 50 , 51 , 52 , 53 , 71 , 72 ] be updated to reflect PRISMA 2020 and advise developers of new PRISMA extensions to use PRISMA 2020 as the foundation document.

We anticipate that the PRISMA 2020 statement will benefit authors, editors, and peer reviewers of systematic reviews, and different users of reviews, including guideline developers, policy makers, healthcare providers, patients, and other stakeholders. Ultimately, we hope that uptake of the guideline will lead to more transparent, complete, and accurate reporting of systematic reviews, thus facilitating evidence based decision making.

Box 1 Glossary of terms

Systematic review —A review that uses explicit, systematic methods to collate and synthesise findings of studies that address a clearly formulated question [ 43 ]

Statistical synthesis —The combination of quantitative results of two or more studies. This encompasses meta-analysis of effect estimates (described below) and other methods, such as combining P values, calculating the range and distribution of observed effects, and vote counting based on the direction of effect (see McKenzie and Brennan [ 25 ] for a description of each method)

Meta-analysis of effect estimates —A statistical technique used to synthesise results when study effect estimates and their variances are available, yielding a quantitative summary of results [ 25 ]

Outcome —An event or measurement collected for participants in a study (such as quality of life, mortality)

Result —The combination of a point estimate (such as a mean difference, risk ratio, or proportion) and a measure of its precision (such as a confidence/credible interval) for a particular outcome

Report —A document (paper or electronic) supplying information about a particular study. It could be a journal article, preprint, conference abstract, study register entry, clinical study report, dissertation, unpublished manuscript, government report, or any other document providing relevant information

Record —The title or abstract (or both) of a report indexed in a database or website (such as a title or abstract for an article indexed in Medline). Records that refer to the same report (such as the same journal article) are “duplicates”; however, records that refer to reports that are merely similar (such as a similar abstract submitted to two different conferences) should be considered unique.

Study —An investigation, such as a clinical trial, that includes a defined group of participants and one or more interventions and outcomes. A “study” might have multiple reports. For example, reports could include the protocol, statistical analysis plan, baseline characteristics, results for the primary outcome, results for harms, results for secondary outcomes, and results for additional mediator and moderator analyses

Box 2 Noteworthy changes to the PRISMA 2009 statement

• Inclusion of the abstract reporting checklist within PRISMA 2020 (see item #2 and Box 2 ).

• Movement of the ‘Protocol and registration’ item from the start of the Methods section of the checklist to a new Other section, with addition of a sub-item recommending authors describe amendments to information provided at registration or in the protocol (see item #24a-24c).

• Modification of the ‘Search’ item to recommend authors present full search strategies for all databases, registers and websites searched, not just at least one database (see item #7).

• Modification of the ‘Study selection’ item in the Methods section to emphasise the reporting of how many reviewers screened each record and each report retrieved, whether they worked independently, and if applicable, details of automation tools used in the process (see item #8).

• Addition of a sub-item to the ‘Data items’ item recommending authors report how outcomes were defined, which results were sought, and methods for selecting a subset of results from included studies (see item #10a).

• Splitting of the ‘Synthesis of results’ item in the Methods section into six sub-items recommending authors describe: the processes used to decide which studies were eligible for each synthesis; any methods required to prepare the data for synthesis; any methods used to tabulate or visually display results of individual studies and syntheses; any methods used to synthesise results; any methods used to explore possible causes of heterogeneity among study results (such as subgroup analysis, meta-regression); and any sensitivity analyses used to assess robustness of the synthesised results (see item #13a-13f).

• Addition of a sub-item to the ‘Study selection’ item in the Results section recommending authors cite studies that might appear to meet the inclusion criteria, but which were excluded, and explain why they were excluded (see item #16b).

• Splitting of the ‘Synthesis of results’ item in the Results section into four sub-items recommending authors: briefly summarise the characteristics and risk of bias among studies contributing to the synthesis; present results of all statistical syntheses conducted; present results of any investigations of possible causes of heterogeneity among study results; and present results of any sensitivity analyses (see item #20a-20d).

• Addition of new items recommending authors report methods for and results of an assessment of certainty (or confidence) in the body of evidence for an outcome (see items #15 and #22).

• Addition of a new item recommending authors declare any competing interests (see item #26).

• Addition of a new item recommending authors indicate whether data, analytic code and other materials used in the review are publicly available and if so, where they can be found (see item #27).

Gurevitch J, Koricheva J, Nakagawa S, Stewart G. Meta-analysis and the science of research synthesis. Nature. 2018;555:175–82. https://doi.org/10.1038/nature25753 .

Article   CAS   PubMed   Google Scholar  

Gough D, Thomas J, Oliver S. Clarifying differences between reviews within evidence ecosystems. Syst Rev. 2019;8:170. https://doi.org/10.1186/s13643-019-1089-2 .

Article   PubMed   PubMed Central   Google Scholar  

Moher D. Reporting guidelines: doing better for readers. BMC Med. 2018;16:233. https://doi.org/10.1186/s12916-018-1226-0 .

Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151:264–9, W64. https://doi.org/10.7326/0003-4819-151-4-200908180-00135 .

Article   PubMed   Google Scholar  

Moher D, Liberati A, Tetzlaff J, Altman DG. PRISMA Group Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:b2535. https://doi.org/10.1136/bmj.b2535 .

Moher D, Liberati A, Tetzlaff J, Altman DG. PRISMA Group Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6:e1000097. https://doi.org/10.1371/journal.pmed.1000097 .

Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol. 2009;62:1006–12. https://doi.org/10.1016/j.jclinepi.2009.06.005 .

Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Int J Surg. 2010;8:336–41. https://doi.org/10.1016/j.ijsu.2010.02.007 .

Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Open Med. 2009;3:e123–30.

PubMed   PubMed Central   Google Scholar  

Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Reprint--preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Phys Ther. 2009;89:873–80. https://doi.org/10.1093/ptj/89.9.873 .

Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG. Epidemiology and reporting characteristics of systematic reviews. PLoS Med. 2007;4:e78. https://doi.org/10.1371/journal.pmed.0040078 .

Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol. 2009;62:e1–34. https://doi.org/10.1016/j.jclinepi.2009.06.006 .

Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009;339:b2700. https://doi.org/10.1136/bmj.b2700 .

Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Ann Intern Med. 2009;151:W65–94. https://doi.org/10.7326/0003-4819-151-4-200908180-00136 .

Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009;6:e1000100. https://doi.org/10.1371/journal.pmed.1000100 .

Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting. systematic reviews and meta-analyses of studies that evaluate health care. interventions: explanation and elaboration. PLoS Med. 2009;6:e1000100. https://doi.org/10.1371/journal.pmed.1000100 .

Page MJ, Shamseer L, Altman DG, et al. Epidemiology and reporting characteristics of systematic reviews of biomedical research: a cross-sectional study. PLoS Med. 2016;13:e1002028. https://doi.org/10.1371/journal.pmed.1002028 .

Panic N, Leoncini E, de Belvis G, Ricciardi W, Boccia S. Evaluation of the endorsement of the preferred reporting items for systematic reviews and meta-analysis (PRISMA) statement on the quality of published systematic review and meta-analyses. PLoS One. 2013;8:e83138. https://doi.org/10.1371/journal.pone.0083138 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Agha RA, Fowler AJ, Limb C, et al. Impact of the mandatory implementation of reporting guidelines on reporting quality in a surgical journal: a before and after study. Int J Surg. 2016;30:169–72. https://doi.org/10.1016/j.ijsu.2016.04.032 .

Leclercq V, Beaudart C, Ajamieh S, Rabenda V, Tirelli E, Bruyère O. Meta-analyses indexed in PsycINFO had a better completeness of reporting when they mention PRISMA. J Clin Epidemiol. 2019;115:46–54. https://doi.org/10.1016/j.jclinepi.2019.06.014 .

Page MJ, Moher D. Evaluations of the uptake and impact of the preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement and extensions: a scoping review. Syst Rev. 2017;6:263. https://doi.org/10.1186/s13643-017-0663-8 .

O’Mara-Eves A, Thomas J, McNaught J, Miwa M, Ananiadou S. Using text mining for study identification in systematic reviews: a systematic review of current approaches. Syst Rev. 2015;4:5. https://doi.org/10.1186/2046-4053-4-5 .

Marshall IJ, Noel-Storr A, Kuiper J, Thomas J, Wallace BC. Machine learning for identifying randomized controlled trials: an evaluation and practitioner’s guide. Res Synth Methods. 2018;9:602–14. https://doi.org/10.1002/jrsm.1287 .

Marshall IJ, Wallace BC. Toward systematic review automation: a practical guide to using machine learning tools in research synthesis. Syst Rev. 2019;8:163. https://doi.org/10.1186/s13643-019-1074-9 .

McKenzie JE, Brennan SE. Synthesizing and presenting findings using other methods. In: Higgins JPT, Thomas J, Chandler J, et al., editors. Cochrane handbook for systematic reviews of interventions. London: Cochrane; 2019. https://doi.org/10.1002/9781119536604.ch12 .

Chapter   Google Scholar  

Higgins JPT, López-López JA, Becker BJ, et al. Synthesising quantitative evidence in systematic reviews of complex health interventions. BMJ Glob Health. 2019;4(Suppl 1):e000858. https://doi.org/10.1136/bmjgh-2018-000858 .

Campbell M, McKenzie JE, Sowden A, et al. Synthesis without meta-analysis (SWiM) in systematic reviews: reporting guideline. BMJ. 2020;368:l6890. https://doi.org/10.1136/bmj.l6890 .

Sterne JAC, Savović J, Page MJ, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019;366:l4898. https://doi.org/10.1136/bmj.l4898 .

Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016;355:i4919. https://doi.org/10.1136/bmj.i4919 .

Whiting P, Savović J, Higgins JP, ROBIS group, et al. ROBIS: a new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol. 2016;69:225–34. https://doi.org/10.1016/j.jclinepi.2015.06.005 .

Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008. https://doi.org/10.1136/bmj.j4008 .

Hultcrantz M, Rind D, Akl EA, et al. The GRADE working group clarifies the construct of certainty of evidence. J Clin Epidemiol. 2017;87:4–13. https://doi.org/10.1016/j.jclinepi.2017.05.006 .

Booth A, Clarke M, Dooley G, et al. The nuts and bolts of PROSPERO: an international prospective register of systematic reviews. Syst Rev. 2012;1:2. https://doi.org/10.1186/2046-4053-1-2 .

Moher D, Stewart L, Shekelle P. Establishing a new journal for systematic review products. Syst Rev. 2012;1:1. https://doi.org/10.1186/2046-4053-1-1 .

Page MJ, McKenzie JE, Bossuyt PM, et al. Updating guidance for reporting systematic reviews: development of the PRISMA 2020 statement. J Clin Epidemiol 2021;134:103–112. https://doi.org/10.1016/j.jclinepi.2021.02.003 .

Page MJ, Altman DG, Shamseer L, et al. Reproducible research practices are underused in systematic reviews of biomedical interventions. J Clin Epidemiol. 2018;94:8–18. https://doi.org/10.1016/j.jclinepi.2017.10.017 .

Page MJ, Altman DG, McKenzie JE, et al. Flaws in the application and interpretation of statistical analyses in systematic reviews of therapeutic interventions were common: a cross-sectional analysis. J Clin Epidemiol. 2018;95:7–18. https://doi.org/10.1016/j.jclinepi.2017.11.022 .

Page MJ, McKenzie JE, Bossuyt PM, et al. Mapping of reporting guidance for systematic reviews and meta-analyses generated a comprehensive item bank for future reporting guidelines. J Clin Epidemiol. 2020;118:60–8. https://doi.org/10.1016/j.jclinepi.2019.11.010 .

Tong A, Flemming K, McInnes E, Oliver S, Craig J. Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC Med Res Methodol. 2012;12:181. https://doi.org/10.1186/1471-2288-12-181 .

France EF, Cunningham M, Ring N, et al. Improving reporting of meta-ethnography: the eMERGe reporting guidance. BMC Med Res Methodol. 2019;19:25. https://doi.org/10.1186/s12874-018-0600-0 .

Page MJ, Moher D, Bossuyt PM, et al. PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. BMJ. 2021;372:n160. https://doi.org/10.1136/bmj.n160 .

Rethlefsen ML, Kirtley S, Waffenschmidt S, et al. PRISMA-S Group PRISMA-S: an extension to the PRISMA statement for reporting literature searches in systematic reviews. Syst Rev. 2021;10:39. https://doi.org/10.1186/s13643-020-01542-z .

Higgins JPT, Thomas J, Chandler J, et al. Cochrane handbook for systematic reviews of interventions: version 6.0. London: Cochrane; 2019. Available from https://training.cochrane.org/handbook

Book   Google Scholar  

Dekkers OM, Vandenbroucke JP, Cevallos M, Renehan AG, Altman DG, Egger M. COSMOS-E: guidance on conducting systematic reviews and meta-analyses of observational studies of etiology. PLoS Med. 2019;16:e1002742. https://doi.org/10.1371/journal.pmed.1002742 .

Cooper H, Hedges LV, Valentine JV. The handbook of research synthesis and meta-analysis. New York: Russell Sage Foundation; 2019.

IOM (Institute of Medicine). Finding what works in health care: standards for systematic reviews. Washington, D.C.: The National Academies Press; 2011.

Google Scholar  

Moher D, Shamseer L, Clarke M, PRISMA-P Group, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1. https://doi.org/10.1186/2046-4053-4-1 .

Shamseer L, Moher D, Clarke M, PRISMA-P Group, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ. 2015;350:g7647. https://doi.org/10.1136/bmj.g7647 .

Hutton B, Salanti G, Caldwell DM, et al. The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations. Ann Intern Med. 2015;162:777–84. https://doi.org/10.7326/M14-2385 .

Stewart LA, Clarke M, Rovers M, PRISMA-IPD Development Group, et al. Preferred reporting items for systematic review and meta-analyses of individual participant data: the PRISMA-IPD statement. JAMA. 2015;313:1657–65. https://doi.org/10.1001/jama.2015.3656 .

Zorzela L, Loke YK, Ioannidis JP, et al. PRISMAHarms Group PRISMA harms checklist: improving harms reporting in systematic reviews. BMJ. 2016;352:i157. https://doi.org/10.1136/bmj.i157 .

McInnes MDF, Moher D, Thombs BD, the PRISMA-DTA Group, et al. Preferred reporting items for a systematic review and meta-analysis of diagnostic test accuracy studies: the PRISMA-DTA statement. JAMA. 2018;319:388–96. https://doi.org/10.1001/jama.2017.19163 .

Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-SCR): checklist and explanation. Ann Intern Med. 2018;169:467–73. https://doi.org/10.7326/M18-0850 .

Beller EM, Glasziou PP, Altman DG, et al. PRISMA for Abstracts Group PRISMA for Abstracts: reporting systematic reviews in journal and conference abstracts. PLoS Med. 2013;10:e1001419. https://doi.org/10.1371/journal.pmed.1001419 .

Boers M. Graphics and statistics for cardiology: designing effective tables for presentation and publication. Heart. 2018;104:192–200. https://doi.org/10.1136/heartjnl-2017-311581 .

Mayo-Wilson E, Li T, Fusco N, Dickersin K, MUDS investigators. Practical guidance for using multiple data sources in systematic reviews and meta-analyses (with examples from the MUDS study). Res Synth Methods. 2018;9:2–12. https://doi.org/10.1002/jrsm.1277 .

Stovold E, Beecher D, Foxlee R, Noel-Storr A. Study flow diagrams in Cochrane systematic review updates: an adapted PRISMA flow diagram. Syst Rev. 2014;3:54. https://doi.org/10.1186/2046-4053-3-54 .

McGuinness LA. mcguinlu/PRISMA-Checklist: Initial release for manuscript submission (Version v1.0.0). Geneva: Zenodo; 2020. https://doi.org/10.5281/zenodo.3994319 .

Aczel B, Szaszi B, Sarafoglou A, et al. A consensus-based transparency checklist. Nat Hum Behav. 2020;4:4–6. https://doi.org/10.1038/s41562-019-0772-6 .

Barnes C, Boutron I, Giraudeau B, Porcher R, Altman DG, Ravaud P. Impact of an online writing aid tool for writing a randomized trial report: the COBWEB (Consort-based WEB tool) randomized controlled trial. BMC Med. 2015;13:221. https://doi.org/10.1186/s12916-015-0460-y .

Chauvin A, Ravaud P, Moher D, et al. Accuracy in detecting inadequate research reporting by early career peer reviewers using an online CONSORT-based peer-review tool (COBPeer) versus the usual peer-review process: a cross-sectional diagnostic study. BMC Med. 2019;17:205. https://doi.org/10.1186/s12916-019-1436-0 .

Wayant C, Page MJ, Vassar M. Evaluation of reproducible research practices in oncology systematic reviews with meta-analyses referenced by national comprehensive cancer network guidelines. JAMA Oncol. 2019;5:1550–5. https://doi.org/10.1001/jamaoncol.2019.2564 .

Article   PubMed Central   PubMed   Google Scholar  

McKenzie JE, Brennan SE. Overviews of systematic reviews: great promise, greater challenge. Syst Rev. 2017;6:185. https://doi.org/10.1186/s13643-017-0582-8 .

Moher D, Schulz KF, Simera I, Altman DG. Guidance for developers of health research reporting guidelines. PLoS Med. 2010;7:e1000217. https://doi.org/10.1371/journal.pmed.1000217 .

Simera I, Moher D, Hirst A, Hoey J, Schulz KF, Altman DG. Transparent and accurate reporting increases reliability, utility, and impact of your research: reporting guidelines and the EQUATOR Network. BMC Med. 2010;8:24. https://doi.org/10.1186/1741-7015-8-24 .

Speich B, Schroter S, Briel M, et al. Impact of a short version of the CONSORT checklist for peer reviewers to improve the reporting of randomised controlled trials published in biomedical journals: study protocol for a randomised controlled trial. BMJ Open. 2020;10:e035114. https://doi.org/10.1136/bmjopen-2019-035114 .

Stevens A, Shamseer L, Weinstein E, et al. Relation of completeness of reporting of health research to journals’ endorsement of reporting guidelines: systematic review. BMJ. 2014;348:g3804. https://doi.org/10.1136/bmj.g3804 .

Hair K, Macleod MR, Sena ES, IICARus Collaboration. A randomised controlled trial of an Intervention to Improve Compliance with the ARRIVE guidelines (IICARus). Res Integr Peer Rev. 2019;4:12. https://doi.org/10.1186/s41073-019-0069-3 .

Blanco D, Altman D, Moher D, Boutron I, Kirkham JJ, Cobo E. Scoping review on interventions to improve adherence to reporting guidelines in health research. BMJ Open. 2019;9:e026589. https://doi.org/10.1136/bmjopen-2018-026589 .

Charters E. The use of think-aloud methods in qualitative research: an introduction to think-aloud methods. Brock Educ J. 2003;12:68–82. https://doi.org/10.26522/brocked.v12i2.38 .

Article   Google Scholar  

Welch V, Petticrew M, Tugwell P, PRISMA-Equity Bellagio group, et al. PRISMA-equity 2012 extension: reporting guidelines for systematic reviews with a focus on health equity. PLoS Med. 2012;9:e1001333. https://doi.org/10.1371/journal.pmed.1001333 .

Wang X, Chen Y, Liu Y, et al. Reporting items for systematic reviews and meta-analyses of acupuncture: the PRISMA for acupuncture checklist. BMC Complement Altern Med. 2019;19:208. https://doi.org/10.1186/s12906-019-2624-3 .

Download references

Acknowledgements

We dedicate this paper to the late Douglas G Altman and Alessandro Liberati, whose contributions were fundamental to the development and implementation of the original PRISMA statement.

We thank the following contributors who completed the survey to inform discussions at the development meeting: Xavier Armoiry, Edoardo Aromataris, Ana Patricia Ayala, Ethan M Balk, Virginia Barbour, Elaine Beller, Jesse A Berlin, Lisa Bero, Zhao-Xiang Bian, Jean Joel Bigna, Ferrán Catalá-López, Anna Chaimani, Mike Clarke, Tammy Clifford, Ioana A Cristea, Miranda Cumpston, Sofia Dias, Corinna Dressler, Ivan D Florez, Joel J Gagnier, Chantelle Garritty, Long Ge, Davina Ghersi, Sean Grant, Gordon Guyatt, Neal R Haddaway, Julian PT Higgins, Sally Hopewell, Brian Hutton, Jamie J Kirkham, Jos Kleijnen, Julia Koricheva, Joey SW Kwong, Toby J Lasserson, Julia H Littell, Yoon K Loke, Malcolm R Macleod, Chris G Maher, Ana Marušic, Dimitris Mavridis, Jessie McGowan, Matthew DF McInnes, Philippa Middleton, Karel G Moons, Zachary Munn, Jane Noyes, Barbara Nußbaumer-Streit, Donald L Patrick, Tatiana Pereira-Cenci, Ba′ Pham, Bob Phillips, Dawid Pieper, Michelle Pollock, Daniel S Quintana, Drummond Rennie, Melissa L Rethlefsen, Hannah R Rothstein, Maroeska M Rovers, Rebecca Ryan, Georgia Salanti, Ian J Saldanha, Margaret Sampson, Nancy Santesso, Rafael Sarkis-Onofre, Jelena Savović, Christopher H Schmid, Kenneth F Schulz, Guido Schwarzer, Beverley J Shea, Paul G Shekelle, Farhad Shokraneh, Mark Simmonds, Nicole Skoetz, Sharon E Straus, Anneliese Synnot, Emily E Tanner-Smith, Brett D Thombs, Hilary Thomson, Alexander Tsertsvadze, Peter Tugwell, Tari Turner, Lesley Uttley, Jeffrey C Valentine, Matt Vassar, Areti Angeliki Veroniki, Meera Viswanathan, Cole Wayant, Paul Whaley, and Kehu Yang. We thank the following contributors who provided feedback on a preliminary version of the PRISMA 2020 checklist: Jo Abbott, Fionn Büttner, Patricia Correia-Santos, Victoria Freeman, Emily A Hennessy, Rakibul Islam, Amalia (Emily) Karahalios, Kasper Krommes, Andreas Lundh, Dafne Port Nascimento, Davina Robson, Catherine Schenck-Yglesias, Mary M Scott, Sarah Tanveer and Pavel Zhelnov. We thank Abigail H Goben, Melissa L Rethlefsen, Tanja Rombey, Anna Scott, and Farhad Shokraneh for their helpful comments on the preprints of the PRISMA 2020 papers. We thank Edoardo Aromataris, Stephanie Chang, Toby Lasserson and David Schriger for their helpful peer review comments on the PRISMA 2020 papers.

Provenance and peer review

Not commissioned; externally peer reviewed.

Patient and public involvement

Patients and the public were not involved in this methodological research. We plan to disseminate the research widely, including to community participants in evidence synthesis organisations.

There was no direct funding for this research. MJP is supported by an Australian Research Council Discovery Early Career Researcher Award (DE200101618) and was previously supported by an Australian National Health and Medical Research Council (NHMRC) Early Career Fellowship (1088535) during the conduct of this research. JEM is supported by an Australian NHMRC Career Development Fellowship (1143429). TCH is supported by an Australian NHMRC Senior Research Fellowship (1154607). JMT is supported by Evidence Partners Inc. JMG is supported by a Tier 1 Canada Research Chair in Health Knowledge Transfer and Uptake. MML is supported by The Ottawa Hospital Anaesthesia Alternate Funds Association and a Faculty of Medicine Junior Research Chair. TL is supported by funding from the National Eye Institute (UG1EY020522), National Institutes of Health, United States. LAM is supported by a National Institute for Health Research Doctoral Research Fellowship (DRF-2018-11-ST2–048). ACT is supported by a Tier 2 Canada Research Chair in Knowledge Synthesis. DM is supported in part by a University Research Chair, University of Ottawa. The funders had no role in considering the study design or in the collection, analysis, interpretation of data, writing of the report, or decision to submit the article for publication.

Author information

Authors and affiliations.

School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia

Matthew J. Page, Joanne E. McKenzie, Sue E. Brennan & Steve McDonald

Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Amsterdam University Medical Centres, University of Amsterdam, Amsterdam, Netherlands

Patrick M. Bossuyt

Université de Paris, Centre of Epidemiology and Statistics (CRESS), Inserm, F 75004, Paris, France

Isabelle Boutron

Institute for Evidence-Based Healthcare, Faculty of Health Sciences and Medicine, Bond University, Gold Coast, Australia

Tammy C. Hoffmann

Annals of Internal Medicine, University of Texas Health Science Center at San Antonio, San Antonio, Texas, USA

Cynthia D. Mulrow

Knowledge Translation Program, Li Ka Shing Knowledge Institute, Toronto, Canada; School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Canada

Larissa Shamseer

Evidence Partners, Ottawa, Canada

Jennifer M. Tetzlaff

Clinical Research Institute, American University of Beirut, Beirut, Lebanon; Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada

Elie A. Akl

Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, OR, USA

York Health Economics Consortium (YHEC Ltd), University of York, York, UK

Julie Glanville

Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada; School of Epidemiology and Public Health, University of Ottawa, Ottawa, Canada; Department of Medicine, University of Ottawa, Ottawa, Canada

Jeremy M. Grimshaw

Centre for Evidence-Based Medicine Odense (CEBMO) and Cochrane Denmark, Department of Clinical Research, University of Southern Denmark, JB Winsløwsvej 9b, 3rd Floor, 5000 Odense, Denmark; Open Patient data Exploratory Network (OPEN), Odense University Hospital, Odense, Denmark

Asbjørn Hróbjartsson

Department of Anesthesiology and Pain Medicine, The Ottawa Hospital, Ottawa, Canada; Clinical Epidemiology Program, Blueprint Translational Research Group, Ottawa Hospital Research Institute, Ottawa, Canada; Regenerative Medicine Program, Ottawa Hospital Research Institute, Ottawa, Canada

Manoj M. Lalu

Department of Ophthalmology, School of Medicine, University of Colorado Denver, Denver, Colorado, United States; Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, USA

Tianjing Li

Division of Headache, Department of Neurology, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts, USA; Head of Research, The BMJ, London, UK

Elizabeth W. Loder

Department of Epidemiology and Biostatistics, Indiana University School of Public Health-Bloomington, Bloomington, Indiana, USA

Evan Mayo-Wilson

Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK

Luke A. McGuinness & Penny Whiting

Centre for Reviews and Dissemination, University of York, York, UK

Lesley A. Stewart

EPPI-Centre, UCL Social Research Institute, University College London, London, UK

James Thomas

Li Ka Shing Knowledge Institute of St. Michael’s Hospital, Unity Health Toronto, Toronto, Canada; Epidemiology Division of the Dalla Lana School of Public Health and the Institute of Health Management, Policy, and Evaluation, University of Toronto, Toronto, Canada; Queen’s Collaboration for Health Care Quality Joanna Briggs Institute Centre of Excellence, Queen’s University, Kingston, Canada

Andrea C. Tricco

Methods Centre, Bruyère Research Institute, Ottawa, Ontario, Canada; School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Canada

Vivian A. Welch

Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada; School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Canada

David Moher

You can also search for this author in PubMed   Google Scholar

Contributions

JEM and DM are joint senior authors. MJP, JEM, PMB, IB, TCH, CDM, LS, and DM conceived this paper and designed the literature review and survey conducted to inform the guideline content. MJP conducted the literature review, administered the survey and analysed the data for both. MJP prepared all materials for the development meeting. MJP and JEM presented proposals at the development meeting. All authors except for TCH, JMT, EAA, SEB, and LAM attended the development meeting. MJP and JEM took and consolidated notes from the development meeting. MJP and JEM led the drafting and editing of the article. JEM, PMB, IB, TCH, LS, JMT, EAA, SEB, RC, JG, AH, TL, EMW, SM, LAM, LAS, JT, ACT, PW, and DM drafted particular sections of the article. All authors were involved in revising the article critically for important intellectual content. All authors approved the final version of the article. MJP is the guarantor of this work. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

Corresponding author

Correspondence to Matthew J. Page .

Ethics declarations

Competing interests.

All authors have completed the ICMJE uniform disclosure form at http://www.icmje.org/conflicts-of-interest/ and declare: EL is head of research for the BMJ ; MJP is an editorial board member for PLOS Medicine ; ACT is an associate editor and MJP, TL, EMW, and DM are editorial board members for the Journal of Clinical Epidemiology ; DM and LAS were editors in chief, LS, JMT, and ACT are associate editors, and JG is an editorial board member for Systematic Reviews . None of these authors were involved in the peer review process or decision to publish. TCH has received personal fees from Elsevier outside the submitted work. EMW has received personal fees from the American Journal for Public Health , for which he is the editor for systematic reviews. VW is editor in chief of the Campbell Collaboration, which produces systematic reviews, and co-convenor of the Campbell and Cochrane equity methods group. DM is chair of the EQUATOR Network, IB is adjunct director of the French EQUATOR Centre and TCH is co-director of the Australasian EQUATOR Centre, which advocates for the use of reporting guidelines to improve the quality of reporting in research articles. JMT received salary from Evidence Partners, creator of DistillerSR software for systematic reviews; Evidence Partners was not involved in the design or outcomes of the statement, and the views expressed solely represent those of the author.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1..

PRISMA 2020 checklist.

Additional file 2.

PRISMA 2020 expanded checklist.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Page, M.J., McKenzie, J.E., Bossuyt, P.M. et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Syst Rev 10 , 89 (2021). https://doi.org/10.1186/s13643-021-01626-4

Download citation

Accepted : 04 January 2021

Published : 29 March 2021

DOI : https://doi.org/10.1186/s13643-021-01626-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Systematic Reviews

ISSN: 2046-4053

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Reporting Guidelines

It is important that your manuscript gives a clear and complete account of the research that you have done. Well reported research is more useful and complete reporting allows editors, peer reviewers and readers to understand what you did and how.

Poorly reported research can distort the literature, and leads to research that cannot be replicated or used in future meta-analyses or systematic reviews.

You should make sure that you manuscript is written in a way that the reader knows exactly what you did and could repeat your study if they wanted to with no additional information. It is particularly important that you give enough information in the methods section of your manuscript.

To help with reporting your research, there are reporting guidelines available for many different study designs. These contain a checklist of minimum points that you should cover in your manuscript. You should use these guidelines when you are preparing and writing your manuscript, and you may be required to provide a completed version of the checklist when you submit your manuscript. 

The EQUATOR (Enhancing the Quality and Transparency Of health Research) Network is an international initiative that aims to improve the quality of research publications. It provides a comprehensive list of reporting guidelines and other material to help improve reporting. 

A list full of all of the reporting guidelines endorsed by the EQUATOR Network can be found here . Some of the reporting guidelines for common study designs are:

• Randomized controlled trials – CONSORT
• Systematic reviews – PRISMA
• Observational studies – STROBE
• Case reports – CARE
• Qualitative research – COREQ
• Pre-clinical animal studies – ARRIVE

Peer reviewers may be asked to use these checklists when assessing your manuscript. If you follow these guidelines, editors and peer reviewers will be able to assess your manuscript better as they will more easily understand what you did. It may also mean that they ask you for fewer revisions.

Enhancing the QUAlity and Transparency Of health Research

• Courses & events
• Librarian Network
• Search for reporting guidelines

Browse for reporting guidelines by selecting one or more of these drop-downs:

Displaying 19 reporting guidelines found.

Key reporting guidelines, shaded green, are displayed first. Show the most recently added records first .

• Improving the Reporting of Primary Care Research: Consensus Reporting Items for Studies in Primary Care-the CRISP Statement
• Systematic Development of Standards for Mixed Methods Reporting in Rehabilitation Health Sciences Research
• Initial Standardized Framework for Reporting Social Media Analytics in Emergency Care Research
• CONFERD-HP : recommendations for reporting COmpeteNcy FramEwoRk Development in health professions
• Development of the ASSESS tool: a comprehenSive tool to Support rEporting and critical appraiSal of qualitative, quantitative, and mixed methods implementation reSearch outcomes
• Guiding document analyses in health professions education research
• Social Accountability Reporting for Research (SAR 4Research): checklist to strengthen reporting on studies on social accountability in the literature
• Application of Mixed Methods in Health Services Management Research: A Systematic Review
• Six practical recommendations for improved implementation outcomes reporting
• Ensuring best practice in genomics education and evaluation: reporting item standards for education and its evaluation in genomics (RISE 2 Genomics)
• Journal article reporting standards for qualitative primary, qualitative meta-analytic, and mixed methods research in psychology: The APA Publications and Communications Board task force report
• Consolidated criteria for strengthening reporting of health research involving indigenous peoples: the CONSIDER statement
• Standards for Reporting Implementation Studies (StaRI) Statement
• RAMESES II reporting standards for realist evaluations
• Developing a methodological framework for organisational case studies: a rapid review and consensus development process
• A checklist to improve reporting of group-based behaviour-change interventions
• Evaluating complex interventions in end of life care: the MORECare statement on good practice generated by a synthesis of transparent expert consultations and systematic reviews
• The quality of mixed methods studies in health services research
• Guidelines for conducting and reporting mixed research in the field of counseling and beyond

Reporting guidelines for main study types

Translations.

Some reporting guidelines are also available in languages other than English. Find out more in our Translations section .

• About the Library

For information about Library scope and content, identification of reporting guidelines and inclusion/exclusion criteria please visit About the Library .

Visit our Help page for information about searching for reporting guidelines and for general information about using our website.

Library index

• What is a reporting guideline?
• Browse reporting guidelines by specialty
• Reporting guidelines under development
• Translations of reporting guidelines
• EQUATOR Network reporting guideline manual
• Reporting guidelines for animal research
• Guidance on scientific writing
• Guidance developed by editorial groups
• Research funders’ guidance on reporting requirements
• Professional medical writing support
• Research ethics, publication ethics and good practice guidelines
• Links to other resources

• Systematic Review
• Open access
• Published: 26 April 2024

Systematic review on the frequency and quality of reporting patient and public involvement in patient safety research

• Sahar Hammoud   ORCID: orcid.org/0000-0003-4682-9001 1 ,
• Laith Alsabek 1 , 2 ,
• Lisa Rogers 1 &
• Eilish McAuliffe 1  

BMC Health Services Research volume  24 , Article number:  532 ( 2024 ) Cite this article

Metrics details

In recent years, patient and public involvement (PPI) in research has significantly increased; however, the reporting of PPI remains poor. The Guidance for Reporting Involvement of Patients and the Public (GRIPP2) was developed to enhance the quality and consistency of PPI reporting. The objective of this systematic review is to identify the frequency and quality of PPI reporting in patient safety (PS) research using the GRIPP2 checklist.

Searches were performed in Ovid MEDLINE, EMBASE, PsycINFO, and CINAHL from 2018 to December, 2023. Studies on PPI in PS research were included. We included empirical qualitative, quantitative, mixed methods, and case studies. Only articles published in peer-reviewed journals in English were included. The quality of PPI reporting was assessed using the short form of the (GRIPP2-SF) checklist.

A total of 8561 studies were retrieved from database searches, updates, and reference checks, of which 82 met the eligibility criteria and were included in this review. Major PS topics were related to medication safety, general PS, and fall prevention. Patient representatives, advocates, patient advisory groups, patients, service users, and health consumers were the most involved. The main involvement across the studies was in commenting on or developing research materials. Only 6.1% ( n  = 5) of the studies reported PPI as per the GRIPP2 checklist. Regarding the quality of reporting following the GRIPP2-SF criteria, our findings show sub-optimal reporting mainly due to failures in: critically reflecting on PPI in the study; reporting the aim of PPI in the study; and reporting the extent to which PPI influenced the study overall.

Conclusions

Our review shows a low frequency of PPI reporting in PS research using the GRIPP2 checklist. Furthermore, it reveals a sub-optimal quality in PPI reporting following GRIPP2-SF items. Researchers, funders, publishers, and journals need to promote consistent and transparent PPI reporting following internationally developed reporting guidelines such as the GRIPP2. Evidence-based guidelines for reporting PPI should be encouraged and supported as it helps future researchers to plan and report PPI more effectively.

Trial registration

The review protocol is registered with PROSPERO (CRD42023450715).

Peer Review reports

Patient safety (PS) is defined as “the absence of preventable harm to a patient and reduction of risk of unnecessary harm associated with healthcare to an acceptable minimum” [ 1 ]. It is estimated that one in 10 patients are harmed in healthcare settings due to unsafe care, resulting in over three million deaths annually [ 2 ]. More than 50% of adverse events are preventable, and half of these events are related to medications [ 3 , 4 ]. There are various types of adverse events that patients can experience such as medication errors, patient falls, healthcare-associated infections, diagnostic errors, pressure ulcers, unsafe surgical procedures, patient misidentification, and others [ 1 ].

Over the last few decades, the approach of PS management has shifted toward actively involving patients and their families in managing PS. This innovative approach has surpassed the traditional model where healthcare providers were the sole managers of PS [ 5 ]. Recent research has shown that patients have a vital role in promoting their safety and decreasing the occurrence of adverse events [ 6 ]. Hence, there is a growing recognition of patient and family involvement as a promising method to enhance PS [ 7 ]. This approach includes involving patients in PS policy development, research, and shared decision making [ 1 ].

In the last decade, research involving patients and the public has significantly increased. In the United Kingdom (U.K), the National Institute for Health Research (NIHR) has played a critical role in providing strategic and infrastructure support to integrate Public and Patient Involvement (PPI) throughout publicly funded research [ 8 ]. This has established a context where PPI is recognised as an essential element in research [ 9 ]. In Ireland, the national government agency responsible for the management and delivery of all public health and social services; the National Health Service Executive (HSE) emphasise the importance of PPI in research and provide guidance for researchers on how to involve patients and public in all parts of the research cycle and knowledge translation process [ 10 ]. Similar initiatives are also developing among other European countries, North America, and Australia. However, despite this significant expansion of PPI research, the reporting of PPI in research articles continues to be sub-optimal, inconsistent, and lacks essential information on the context, process, and impact of PPI [ 9 ]. To address this problem, the Guidance for Reporting Involvement of Patients and the Public (GRIPP) was developed in 2011 following the EQUATOR methodology to enhance the quality, consistency, and transparency of PPI reporting. Additionally, to provide guidance for researchers, patients, and the public to advance the quality of the international PPI evidence-base [ 11 ]. The first GRIPP checklist was a significant start in producing higher-quality PPI reporting; however, it was developed following a systematic review, and did not include any input from the international PPI research community. Given the importance of reaching consensus in generating current reporting guidelines, a second version of the GRIPP checklist (GRIPP2) was developed to tackle this problem by involving the international PPI community in its development [ 9 ]. There are two versions of the GRIPP2 checklist, a long form (GRIPP2-LF) for studies with PPI as the primary focus, and a short form (GRIPP2-SF) for studies with PPI as secondary or tertiary focus.

Since the publication of the GRIPP2 checklist, several systematic reviews have been conducted to assess the quality of PPI reporting on various topics. For instance, Bergin et al. in their review to investigate the nature and impact of PPI in cancer research, reported a sub-optimal quality of PPI reporting using the GRIPP2-SF, mainly due to failure to address PPI challenges [ 12 ]. Similarly, Owyang et al. in their systematic review to assess the prevalence, extent, and quality of PPI in orthopaedic practice, described a poor PPI reporting following the GRIPP2-SF checklist criteria [ 13 ]. While a few systematic reviews have been conducted to assess theories, strategies, types of interventions, and barriers and enablers of PPI in PS [ 5 , 14 , 15 , 16 ], no previous review has assessed the quality of PPI reporting in PS research. Thus, our systematic review aims to address this knowledge gap. The objective of this review is to identify the frequency PPI reporting in PS research using the GRIPP2 checklist from 2018 (the year after GRIPP2 was published) and the quality of reporting following the GRIPP2-SF. The GRIPP2 checklist was chosen as the benchmark as it is the first international, evidence-based, community consensus informed guideline for the reporting of PPI in research and more specifically in health and social care research [ 9 ]. Additionally, it is the most recent report-focused framework and the most recommended by several leading journals [ 17 ].

We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines to plan and report this review [ 18 ]. The review protocol was published on PROSPERO the International Database of Prospectively Registered Systematic Reviews in August 2023 (CRD42023450715).

Search strategy

For this review, we used the PICo framework to define the key elements in our research. These included articles on patients and public (P-Population) involvement (I- phenomenon of Interest) in PS (C-context). Details are presented in Table  1 . Four databases were searched including Ovid MEDLINE, EMBASE, PsycINFO, and CINAHL to identify papers on PPI in PS research. A systematic search strategy was initially developed using MEDLINE. MeSH terms and keywords relevant to specific categories (e.g., patient safety) were combined using the “OR” Boolean term (i.e. patient safety OR adverse event OR medical error OR surgical error) and categories were then combined using the “AND” Boolean term. (i.e. “patient and public involvement” AND “patient safety”). The search strategy was adapted for the other three databases. Full search strategies are provided in Supplementary file 1 . The search was conducted on July 27th, 2023, and was limited to papers published from 2018. As the GRIPP2 tool was published in 2017, this limit ensured the retrieval of relevant studies. An alert system was set on the four databases to receive all new published studies until December 2023, prior to the final analysis. The search was conducted without restrictions on study type, research design, and language. To reduce selection bias, hand searching was carried out on the reference lists of all the eligible articles in the later stages of the review. This was done by the first author. The search strategy was developed by the first author and confirmed by the research team and a Librarian. The database search was conducted by the first author.

Inclusion and exclusion criteria

Studies on PPI in PS research with a focus on health/healthcare were included in this review. We defined PPI as active involvement which is in line with the NIHR INVOLVE definition as “research being carried out ‘with’ or ‘by’ members of the public rather than ‘to’, ‘about’ or ‘for’ them” [ 19 ]. This includes any PPI including, being a co-applicant on a research project or grant application, identifying research priorities, being a member of an advisory or steering group, participating in developing research materials or giving feedback on them, conducting interviews with study participants, participating in recruitment, data collection, data analysis, drafting manuscripts and/or dissemination of results. Accordingly, we excluded studies where patients or the public were only involved as research participants.

We defined patients and public to include patients, relatives, carers, caregivers and community, which is also in line with the NIHR PPI involvement in National Health Service [ 19 ].

Patient safety included topics on medication safety, adverse events, communication, safety culture, diagnostic errors, and others. A full list of the used terms for PPI and PS is provided in Supplementary file 1 . Regarding the research type and design, we included empirical qualitative, quantitative, mixed methods, and case studies. Only articles published in peer-reviewed journals and in English were included.

Any article that did not meet the inclusion criteria was excluded. Studies not reporting outcomes were excluded. Furthermore, review papers, conference abstracts, letters to editor, commentary, viewpoints, and short communications were excluded. Finally, papers published prior to 2018 were excluded.

Study selection

The selection of eligible studies was done by the first and the second authors independently, starting with title and abstracts screening to eliminate papers that failed to meet our inclusion criteria. Then, full text screening was conducted to decide on the final included papers in this review. Covidence, an online data management system supported the review process, ensuring reviewers were blinded to each other’s decisions. Disagreements between reviewers were discussed first, in cases where the disagreement was not resolved, the fourth author was consulted.

Data extraction and analysis

A data extraction sheet was developed using excel then piloted, discussed with the research team and modified as appropriate. The following data were extracted: citation and year of publication, objective of the study, country, PS topic, design, setting, PPI participants, PPI stages (identifying research priorities, being a member of an advisory or steering group, etc.…), frequency of PPI reporting as per the GRIPP2 checklist, and the availability of a plain language summary. Additionally, data against the five items of GRIPP2-SF (aim of PPI in the study, methods used for PPI, outcomes of PPI including the results and the extent to which PPI influenced the study overall, and reflections on PPI) were extracted. To avoid multiple publication bias and missing outcomes, data extraction was done by the first and the second authors independently and then compared. Disagreements between reviewers were first discussed, and then resolved by the third and fourth authors if needed.

Quality assessment

The quality of PPI reporting was assessed using GRIPP2-SF developed by Staniszewska et al. [ 9 ] as it is developed to improve the quality, consistency, and reporting of PPI in social and healthcare research. Additionally the GRIPP2-SF is suitable for all studies regardless of whether PPI is the primary, secondary, or tertiary focus, whereas the GRIPP2-LF is not suitable for studies where PPI serves as a secondary or tertiary focus. The checklist includes five items (mentioned above) that authors should include in their studies. It is important to mention that Staniszewska et al. noted that “while GRIPP2-SF aims to guide consistent reporting, it is not possible to be prescriptive about the exact content of each item, as the current evidence-base is not advanced enough to make this possible” ([ 9 ] p5). For that reason, we had to develop criteria for scoring the five reporting items. We used three scoring as Yes, No, and partial for each of the five items of the GRIPP2-SF. Yes, was given when authors presented PPI information on the item clearly in the paper. No, when no information was provided, and partial when the information partially met the item requirement. For example, as per GRIPP2-SF authors should provide a clear description of the methods used for PPI in the study. In the example given by Staniszewska et al., information on patient/public partners and how many of them were provided, as well as the stages of the study they were involved in (i.e. refining the focus of the research questions, developing the search strategy, interpreting results). Thus, in our evaluation of the included studies, we gave a yes if information on PPI participants (i.e. patient partners, community partners, or family members etc..) and how many of them were involved was provided, and information on the stages or actions of their involvement in the study was provided. However, we gave a “partial” if information was not fully provided (i.e. information on patient/public partners and how many were involved in the study without describing in what stages or actions they were involved, and vice versa), and a “No” if no information was presented at all.

The quality of PPI reporting was done by the first and the second authors independently and then compared. Disagreements between reviewers were first discussed, and then resolved by the third and fourth author when needed.

Assessing the quality or risk of bias of the included studies was omitted, as the focus in this review was on appraising the quality of PPI reporting rather than assessing the quality of each research article.

Data synthesis

After data extraction, a table summarising the included studies was developed. Studies were compared according to the main outcomes of the review; frequency of PPI reporting following the GRIPP2 checklist and the quality of reporting as per GRIPP2-SF five items, and the availability of a plain language summary.

Search results and study selection

The database searches yielded a total of 8491 studies. First, 2496 were removed as duplicates. Then, after title and abstract screening, 5785 articles were excluded leaving 210 articles eligible for the full text review. After a careful examination, 68 of these studies were included in this review. A further 38 studies were identified from the alert system that was set on the four databases and 32 studies from the reference check of the included studies. Of these 70 articles, 56 were further excluded and 14 were added to the previous 68 included studies. Thus, 82 studies met the inclusion criteria and were included in this review. A summary of the database search results and the study selection process are presented in Fig.  1 .

PRISMA flow diagram of the study selection process. The PRISMA flow diagram details the review search results and selection process

Overview of included studies

Details of the study characteristics including first author and year of publication, objective, country, study design, setting, PS topic, PPI participants and involvement stages are presented in Supplementary file 2 . The majority of the studies were conducted in the U.K ( n  = 24) and the United States of America ( n  = 18), with the remaining 39 conducted in other high income countries, the exception being one study in Haiti. A range of study designs were identified, the most common being qualitative ( n  = 31), mixed methods ( n  = 13), interventional ( n  = 5), and quality improvement projects ( n  = 4). Most PS topics concerned medication safety ( n  = 17), PS in general (e.g., developing a PS survey or PS management application) ( n  = 14), fall prevention ( n  = 13), communication ( n  = 11), and adverse events ( n  = 10), with the remaining PS topics listed in Supplementary file 2 .

Patient representatives, advocates, and patient advisory groups ( n  = 33) and patients, service users, and health consumers ( n  = 32) were the main groups involved. The remaining, included community members/ organisations. Concerning PPI stages, the main involvement across the studies was in commenting on or developing research materials ( n  = 74) including, patient leaflets, interventional tools, mobile applications, and survey instruments. Following this stage, involvement in data analysis, drafting manuscripts, and disseminating results ( n  = 30), and being a member of a project advisory or steering group ( n  = 18) were the most common PPI evident in included studies. Whereas the least involvement was in identifying research priorities ( n  = 5), and being a co-applicant on a research project or grant application ( n  = 6).

Regarding plain language summary, only one out of the 82 studies (1.22%) provided a plain language summary in their paper [ 20 ].

Frequency and quality of PPI reporting

The frequency of PPI reporting following the GRIPP2 checklist was 6.1%, where only five of the 82 included studies reported PPI in their papers following the GRIPP2 checklist. The quality of PPI reporting in those studies is presented in Table  2 . Of these five studies, one study (20%) did not report the aim of PPI in the study and one (20%) did not comment on the extent to which PPI influenced the study overall.

The quality of PPI reporting of the remaining 77 studies is presented in Table  3 . The aim of PPI in the study was reported in 62.3% of articles ( n  = 48), while 3.9% ( n  = 3) partially reported this. A clear description of the methods used for PPI in the study was reported in 79.2% of papers ( n  = 61) and partially in 20.8% ( n  = 16). Concerning the outcomes, 81.8% of papers ( n  = 63) reported the results of PPI in the study, while 10.4% ( n  = 8) partially did. Of the 77 studies, 68.8% ( n  = 53) reported the extent to which PPI influenced the study overall and 3.9% ( n  = 3) partially reported this. Finally, 57.1% ( n  = 44) of papers critically reflected on the things that went well and those that did not and 2.6% ( n  = 2) partially reflected on this.

Summary of main findings

This systematic review assessed the frequency of reporting PPI in PS research using the GRIPP2 checklist and quality of reporting using the GRIPP2-SF. In total, 82 studies were included in this review. Major PS topics were related to medication safety, general PS, and fall prevention. Patient representatives, advocates, patient advisory groups, patients, service users, and health consumers were the most involved. The main involvement across the studies was in commenting on or developing research materials such as educational and interventional tools, survey instruments, and applications while the least was in identifying research priorities and being a co-applicant on a research project or grant application. Thus, significant effort is still needed to involve patients and the public in the earlier stages of the research process given the fundamental impact of PS on their lives.

Overall completeness and applicability of evidence

A low frequency of reporting PPI in PS research following the GRIPP2 guidelines was revealed in this review, where only five of the 82 studies included mentioned that PPI was reported as per the GRIPP2 checklist. This is despite it being the most recent report-focused framework and the most recommended by several leading journals [ 17 ]. This was not surprising as similar results were reported in recent reviews in other healthcare topics. For instance, Musbahi et al. in their systematic review on PPI reporting in bariatric research reported that none of the 90 papers identified in their review mentioned or utilised the GRIPP2 checklist [ 102 ]. Similarly, a study on PPI in orthodontic research found that none of the 363 included articles reported PPI against the GRIPP2 checklist [ 103 ].

In relation to the quality of reporting following the GRIPP2-SF criteria, our findings show sub-optimal reporting within the 77 studies that did not use GRIPP2 as a guide/checklist to report their PPI. Similarly, Bergin et al. in their systematic review to investigate the nature and impact of PPI in cancer research concluded that substandard reporting was evident [ 12 ]. In our review, this was mainly due to failure to meet three criteria. First, the lowest percentage of reporting (57.1%, n  = 44) was related to critical reflection on PPI in the study (i.e., what went well and what did not). In total, 31 studies (42.9%) did not provide any information on this, and two studies were scored as partial. The first study mentioned that only involving one patient was a limitation [ 27 ] and the other stated that including three patients in the design of the tool was a strength [ 83 ]. Both studies did not critically comment or reflect on these points so that future researchers are able to avoid such problems and enhance PPI opportunities. For instance, providing the reasons/challenges behind the exclusive inclusion of a single patient and explaining how this limits the study findings and conclusion would help future researchers to address these challenges. Likewise, commenting on why incorporating three patients in the design of the study tool could be seen as a strength would have been beneficial. This could be, fostering diverse perspectives and generating novel ideas for developing the tool. Similar to our findings, Bergin et al. in their systematic review reported that 40% of the studies failed to meet this criterion [ 12 ].

Second, only 48 out of 77 articles (62.3%) reported the aim of PPI in their study, which is unlike the results of Bergin et al. where most of the studies (93.1%) in their review met this criterion [ 12 ]. Of the 29 studies which did not meet this criterion in our review, few mentioned in their objective developing a consensus-based instrument [ 41 ], reaching a consensus on the patient-reported outcomes [ 32 ], obtaining international consensus on a set of core outcome measures [ 98 ], and facilitating a multi-stakeholder dialogue [ 71 ] yet, without indicating anything in relation to patients, patient representatives, community members, or any other PPI participants. Thus, the lack of reporting the aim of PPI was clearly evident in this review. Reporting the aim of PPI in the study is crucial for promoting transparency, methodological rigor, reproducibility, and impact assessment of the PPI.

Third, 68.8% ( n  = 53) of the studies reported the extent to which PPI influenced the study overall including positive and negative effects if any. This was again similar to the findings of Bergin et al., where 38% of the studies did not meet this criterion mainly due to a failure to address PPI challenges in their respective studies [ 12 ]. Additionally, Owyang et al. in their review on the extent, and quality of PPI in orthopaedic practice, also described a poor reporting of PPI impact on research [ 13 ]. As per the GRIPP2 guidelines, both positive and negative effects of PPI on the study should be reported when applicable. Providing such information is essential as it enhances future research on PPI in terms of both practice and reporting.

Reporting a clear description of the methods used for PPI in the study was acceptable, with 79.2% of the papers meeting this criterion. Most studies provided information in the methods section of their papers on the PPI participants, their number, stages of their involvement and how they were involved. Providing clear information on the methods used for PPI is vital to give the reader a clear understanding of the steps taken to involve patients, and for other researchers to replicate these methods in future research. Additionally, reporting the results of PPI in the study was also acceptable with 81.8% of the papers reporting the outcomes of PPI in the results section. Reporting the results of PPI is important for enhancing methodological transparency, providing a more accurate interpretation for the study findings, contributing to the overall accountability and credibility of the research, and informing decision making.

Out of the 82 studies included in this review, only one study provided a plain language summary. We understand that PS research or health and medical research in general is difficult for patients and the public to understand given their diverse health literacy and educational backgrounds. However, if we expect patients and the public to be involved in research then, it is crucial to translate this research that has a huge impact on their lives into an easily accessible format. Failing to translate the benefits that such research may have on patient and public lives may result in them underestimating the value of this research and losing interest in being involved in the planning or implementation of future research [ 103 ]. Thus, providing a plain language summary for research is one way to tackle this problem. To our knowledge, only a few health and social care journals (i.e. Cochrane and BMC Research Involvement and Engagement) necessitate a plain language summary as a submission requirement. Having this as a requirement for submission is crucial in bringing the importance of this issue to researchers’ attention.

Research from recent years suggests that poor PPI reporting in articles relates to a lack of submission requirements for PPI reporting in journals and difficulties with word limits for submitted manuscripts [ 13 ]. Price et al. assessed the frequency of PPI reporting in published papers before and after the introduction of PPI reporting obligations by the British Medical Journal (BMJ) [ 104 ]. The authors identified an increase in PPI reporting in papers published by BMJ from 0.5% to 11% between the periods of 2013–2014 and 2015–2016. The study findings demonstrate the impact of journal guidelines in shaping higher quality research outputs [ 13 ]. In our review, we found a low frequency of PPI reporting in PS research using the GRIPP2 checklist, alongside sub-optimal quality of reporting following GRIPP2-SF. This could potentially be attributed to the absence of submission requirements for PPI reporting in journals following the GRIPP2 checklist, as well as challenges posed by word limits.

Strengths and limitations

This systematic review presents an overview on the frequency of PPI reporting in PS research using the GRIPP2 checklist, as well as an evaluation of the quality of reporting following the GRIPP2-SF. As the first review to focus on PS research, it provides useful knowledge on the status of PPI reporting in this field, and the extent to which researchers are adopting and adhering to PPI reporting guidelines. Despite these strengths, our review has some limitations that should be mentioned. First, only English language papers were included in this review due to being the main language of the researchers. Thus, there is a possibility that relevant articles on PPI in PS research may have been omitted. Another limitation is related to our search which was limited to papers published starting 2018 as the GRIPP2 guidelines were published in 2017. Thus it is probable that the protocols of some of these studies were developed earlier than the publication of the GRIPP2 checklist, meaning that PPI reporting following GRIPP2 was not common practice and thus not adopted by these studies. This might limit the conclusions we can draw from this review. Finally, the use of GRIPP2 to assess the quality of PPI reporting might be a limitation as usability testing has not yet been conducted to understand how the checklist works in practice with various types of research designs. However, the GRIPP2 is the first international, evidence-based, community consensus informed guideline for the reporting of PPI in health and social care research. Reflections and comments from researchers using the GRIPP2 will help improve its use in future studies.

Implications for research and practice

Lack of PPI reporting not only affects the quality of research but also implies that others cannot learn from previous research experience. Additionally, without consistent and transparent reporting it is difficult to evaluate the impact of various PPI in research [ 9 ]: “if it is not reported it cannot be assessed” ([ 105 ] p19). Enhanced PPI reporting will result in a wider range and richer high-quality evidence-based PPI research, leading to a better understanding of PPI use and effectiveness [ 103 ]. GRIPP2 reporting guidelines were developed to provide guidance for researchers, patients, and the public to enhance the quality of PPI reporting and improve the quality of the international PPI evidence-base. The guidance can be used prospectively to plan PPI or retrospectively to guide the structure or PPI reporting in research [ 9 ]. To enhance PPI reporting, we recommend the following;

Publishers and journals

First, we encourage publishers and journals to require researchers to report PPI following the GRIPP2 checklist. Utilising the short or the long version should depend on the primary focus of the study (i.e., if PPI is within the primary focus of the research then the GRIPP2-LF is recommended). Second, we recommend that journals and editorial members advise reviewers to evaluate PPI reporting within research articles following the GRIPP2 tool and make suggestions accordingly. Finally, we encourage journals to add a plain language summary as a submission requirement to increase research dissemination and improve the accessibility of research for patients and the public.

Researchers

Though there is greater evidence of PPI in research, it is still primarily the researchers that are setting the research agenda and deciding on the research questions to be addressed. Thus, significant effort is still needed to involve patients and the public in the earlier stages of the research process given the fundamental impact of PS on their lives. To enhance future PPI reporting, perhaps adding a criterion following the GRIPP2 tool to existing EQUATOR checklists for reporting research papers such as STROBE, PRISMA, CONSORT, may support higher quality research. Additionally, currently, there is no detailed explanation paper for the GRIPP2 where each criterion is explained in detail with examples. Addressing this gap would be of great benefit to guide the structure of PPI reporting and to explore the applicability of each criterion in relation to different stages of PPI in research. For instance, having a detailed explanation for each criterion across different research studies having various PPI stages would be of high value to improve future PPI reporting given the growing interest in PPI research in recent years and the relatively small PPI evidence base in health and medical research.

Funding bodies can also enhance PPI reporting by adding a requirement for researchers to report PPI following the GRIPP2 checklist. In Ireland, the National HSE has already initiated this by requiring all PPI in HSE research in Ireland to be reported following the GRIPP2 guidelines [ 10 ].

This study represents the first systematic review on the frequency and quality of PPI reporting in PS research using the GRIPP2 checklist. Most PS topics were related to medication safety, general PS, and fall prevention. The main involvement across the studies was in commenting on or developing research materials. Thus, efforts are still needed to involve patients and the public across all aspects of the research process, especially earlier stages of the research cycle. The frequency of PPI reporting following the GRIPP2 guidelines was low, and the quality of reporting following the GRIPP2-SF criteria was sub-optimal. The lowest percentages of reporting were on critically reflecting on PPI in the study so future research can learn from this experience and work to improve it, reporting the aim of the PPI in the study, and reporting the extent to which PPI influenced the study overall including positive and negative effects. Researchers, funders, publishers, journals, editorial members and reviewers have a responsibility to promote consistent and transparent PPI reporting following internationally developed reporting guidelines such as the GRIPP2. Evidence-based guidelines for reporting PPI should be supported to help future researchers plan and report PPI more effectively, which may ultimately improve the quality and relevance of research.

Availability of data and materials

All data generated or analysed during this study are included in this published article and its Supplementary information files.

Abbreviations

• Patient safety

United Kingdom

National Institute for Health Research

Public and Patient Involvement

Health Service Executive

Guidance for Reporting Involvement of Patients and the Public

Second version of the GRIPP checklist

Long form of GRIPP2

Short form of GRIPP2

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

The International Database of Prospectively Registered Systematic Reviews

British Medical Journal

Patient saftey: World Health Organisation. 2023. Available from: https://www.who.int/news-room/fact-sheets/detail/patient-safety . Updated 11 Sept 2023.

Slawomirski L, Klazinga N. The economics of patient safety: from analysis to action. Paris: Organisation for Economic Co-operation and Development; 2020.

Google Scholar  

Panagioti M, Khan K, Keers RN, Abuzour A, Phipps D, Kontopantelis E, et al. Prevalence, severity, and nature of preventable patient harm across medical care settings: systematic review and meta-analysis. Bmj-Brit Med J. 2019;366:l4185.

Article   Google Scholar  

Hodkinson A, Tyler N, Ashcroft DM, Keers RN, Khan K, Phipps D, et al. Preventable medication harm across health care settings: a systematic review and meta-analysis. Bmc Medicine. 2020;18(1):313.

Article   PubMed   PubMed Central   Google Scholar  

Park M, Giap TTT. Patient and family engagement as a potential approach for improving patient safety: A systematic review. J Adv Nurs. 2020;76(1):62–80.

Article   PubMed   Google Scholar  

Chegini Z, Janati A, Bababie J, Pouraghaei M. The role of patients in the delivery of safe care in hospital: Study protocol. J Adv Nurs. 2019;75(9):2015–23.

Chegini Z, Arab-Zozani M, Islam SMS, Tobiano G, Rahimi SA. Barriers and facilitators to patient engagement in patient safety from patients and healthcare professionals’ perspectives: A systematic review and meta-synthesis. Nurs Forum. 2021;56(4):938–49.

Going the extra mile: improving the nation’s health and wellbeing through public involvement in research. London: National Institute for Health; 2015.

Staniszewska S, Brett J, Simera I, Seers K, Mockford C, Goodlad S, et al. GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. Bmj-Brit Med J. 2017;358:j3453.

Article   CAS   Google Scholar  

Minogue V. Knowledge translation, dissemination, and impact: a practical guide for researchers. Guide No 8: patient and public involvement in HSE research. Ireland: Health Service Executive Research and Development; 2021.

Staniszewska S, Brett J, Mockford C, Barber R. The GRIPP checklist: Strengthening the quality of patient and public involvement reporting in research. Int J Technol Assess Health Care. 2011;27(4):391–9.

Bergin RJ, Short CE, Davis N, Marker J, Dawson MT, Milton S, et al. The nature and impact of patient and public involvement in cancer prevention, screening and early detection research: A systematic review. Prev Med. 2023;167:107412.

Owyang D, Bakhsh A, Brewer D, Boughton OR, Cobb JP. Patient and public involvement within orthopaedic research a systematic review. J Bone Joint Surg Am. 2021;103(13):e51.

Busch IM, Saxena A, Wu AW. Putting the patient in patient safety investigations: barriers and strategies for involvement. J Patient Saf. 2021;17(5):358–62.

Lee M, Lee NJ, Seo HJ, Jang H, Kim SM. Interventions to engage patients and families in patient safety: a systematic review. West J Nurs Res. 2021;43(10):972–83.

Ocloo J, Garfield S, Franklin BD, Dawson S. Exploring the theory, barriers and enablers for patient and public involvement across health, social care and patient safety: a systematic review of reviews. Health Res Policy Syst. 2021;19(1):8.

Greenhalgh T, Hinton L, Finlay T, Macfarlane A, Fahy N, Clyde B, et al. Frameworks for supporting patient and public involvement in research: Systematic review and co-design pilot. Health Expect. 2019;22(4):785–801.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. Plos Medicine. 2021;18(3):372.

INVOLVE. What is public involvement in research? NIHR; 2019. Available from: https://www.invo.org.uk/find-out-more/what-is-public-involvement-in-research-2/ .

Shahid A, Sept B, Kupsch S, Brundin-Mather R, Piskulic D, Soo A, et al. Development and pilot implementation of a patient-oriented discharge summary for critically Ill patients. World J Crit Care Med. 2022;11(4):255–68.

Bisset CN, Dames N, Oliphant R, Alasadi A, Anderson D, Parson S, et al. Exploring shared surgical decision-making from the patient’s perspective: is the personality of the surgeon important? Colorectal Dis. 2020;22(12):2214–21.

Article   CAS   PubMed   Google Scholar  

Morris RL, Ruddock A, Gallacher K, Rolfe C, Giles S, Campbell S. Developing a patient safety guide for primary care: A co-design approach involving patients, carers and clinicians. Health Expect. 2021;24(1):42–52.

Tobiano G, Marshall AP, Gardiner T, Jenkinson K, Shapiro M, Ireland M. Development and psychometric testing of the patient participation in bedside handover survey. Health Expect. 2022;25(5):2492–502.

Francis-Coad J, Farlie MK, Haines T, Black L, Weselman T, Cummings P, et al. Revising and evaluating falls prevention education for older adults in hospital. Health Educ J. 2023;82(8):878–91.

Troya MI, Chew-Graham CA, Babatunde O, Bartlam B, Higginbottom A, Dikomitis L. Patient and public involvement and engagement in a doctoral research project exploring self-harm in older adults. Health Expect. 2019;22(4):617–31.

Aharaz A, Kejser CL, Poulsen MW, Jeftic S, Ulstrup-Hansen AI, Jorgensen LM, et al. Optimization of the Danish National Electronic Prescribing System to improve patient safety: Development of a user-friendly prototype of the digital platform shared medication record. Pharmacy (Basel, Switzerland). 2023;11(2):41.

PubMed   Google Scholar  

Aho-Glele U, Bouabida K, Kooijman A, Popescu IC, Pomey MP, Hawthornthwaite L, et al. Developing the first pan-Canadian survey on patient engagement in patient safety. BMC Health Serv Res. 2021;21(1):1099.

Albutt A, O’Hara J, Conner M, Lawton R. Involving patients in recognising clinical deterioration in hospital using the patient wellness questionnaire: A mixed-methods study. J Res Nurs. 2020;25(1):68–86.

Bell SK, Bourgeois F, DesRoches CM, Dong J, Harcourt K, Liu SK, et al. Filling a gap in safety metrics: development of a patient-centred framework to identify and categorise patient-reported breakdowns related to the diagnostic process in ambulatory care. BMJ Qual Saf. 2022;31(7):526–40.

Boet S, Etherington N, Lam S, Lê M, Proulx L, Britton M, et al. Implementation of the Operating Room Black Box research program at the Ottawa Hospital through patient, clinical, and organizational engagement: Case study. J Med Internet Res. 2021;23(3):e15443.

Carter J, Tribe RM, Shennan AH, Sandall J. Threatened preterm labour: Women’s experiences of risk and care management: A qualitative study. Midwifery. 2018;64:85–92.

Da Silva Lopes AM, Colomer-Lahiguera S, Mederos Alfonso N, Aedo-Lopez V, Spurrier-Bernard G, Tolstrup LK, et al. Patient-reported outcomes for monitoring symptomatic toxicities in cancer patients treated with immune-checkpoint inhibitors: A Delphi study. Eur J Cancer. 2021;157:225–37.

de Jong LD, Lavender AP, Wortham C, Skelton DA, Haines TP, Hill AM. Exploring purpose-designed audio-visual falls prevention messages on older people’s capability and motivation to prevent falls. Health Soc Care Community. 2019;27(4):e471–82.

Doucette L, Kiely BT, Gierisch JM, Marion E, Nadler L, Heflin MT, et al. Participatory research to improve medication reconciliation for older adults in the community. J Am Geriatr Soc. 2023;71(2):620–31.

Elrod CS, Pappa ST, Heyn PC, Wong RA. Using an academic-community partnership model to deliver evidence-based falls prevention programs in a metropolitan setting: A community case study. Front Public Health. 2023;11:1073520.

Feldman E, Pos FJ, Smeenk RJ, van der Poel H, van Leeuwen P, de Feijter JM, et al. Selecting a PRO-CTCAE-based subset for patient-reported symptom monitoring in prostate cancer patients: a modified Delphi procedure. ESMO Open. 2023;8(1):100775.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Francis-Coad J, Watts T, Bulsara C, Hill A-M. Designing and evaluating falls prevention education with residents and staff in aged care homes: a feasibility study. Health Educ (0965-4283). 2022;122(5):546–63.

Fuller TE, Pong DD, Piniella N, Pardo M, Bessa N, Yoon C, et al. Interactive digital health tools to engage patients and caregivers in discharge preparation: implementation study. J Med Internet Res. 2020;22(4):e15573.

Gibson B, Butler J, Schnock K, Bates D, Classen D. Design of a safety dashboard for patients. Patient Educ Couns. 2020;103(4):741–7.

Giles SJ, Lewis PJ, Phipps DL, Mann F, Avery AJ, Ashcroft DM. Capturing patients’ perspectives on medication safety: the development of a patient-centered medication safety framework. J Patient Saf. 2020;16(4):e324–39.

Gnagi R, Zuniga F, Brunkert T, Meyer-Massetti C. Development of a medication literacy assessment instrument (MELIA) for older people receiving home care. J Adv Nurs. 2022;78(12):4210–20.

Goodsmith N, Zhang L, Ong MK, Ngo VK, Miranda J, Hirsch S, et al. Implementation of a community-partnered research suicide-risk management protocol: case study from community partners in care. Psychiatr Serv (Washington, DC). 2021;72(3):281–7.

Gorman LS, Littlewood DL, Quinlivan L, Monaghan E, Smith J, Barlow S, et al. Family involvement, patient safety and suicide prevention in mental healthcare: ethnographic study. BJPsych open. 2023;9(2):e54.

Green MM, Meyer C, Hutchinson AM, Sutherland F, Lowthian JA. Co‐designing being your best program—a holistic approach to frailty in older community dwelling australians. Health Soc Care Community. 2021;30(5):e2022–32.

Guo X, Wang Y, Wang L, Yang X, Yang W, Lu Z, et al. Effect of a fall prevention strategy for the older patients: A quasi-experimental study. Nurs Open. 2023;10(2):1116–24.

Hahn-Goldberg S, Chaput A, Rosenberg-Yunger Z, Lunsky Y, Okrainec K, Guilcher S, et al. Tool development to improve medication information transfer to patients during transitions of care: A participatory action research and design thinking methodology approach. Res Social Adm Pharm. 2022;18(1):2170–7.

Harrington A, Darke H, Ennis G, Sundram S. Evaluation of an alternative model for the management of clinical risk in an adult acute psychiatric inpatient unit. Int J Ment Health Nurs. 2019;28(5):1099–109.

Harris K, Softeland E, Moi AL, Harthug S, Ravnoy M, Storesund A, et al. Development and validation of patients’ surgical safety checklist. BMC Health Serv Res. 2022;22(1):259.

Hawley-Hague H, Tacconi C, Mellone S, Martinez E, Ford C, Chiari L, et al. Smartphone apps to support falls rehabilitation exercise: app development and usability and acceptability study. JMIR Mhealth Uhealth. 2020;8(9):e15460.

Holmqvist M, Ros A, Lindenfalk B, Thor J, Johansson L. How older persons and health care professionals co-designed a medication plan prototype remotely to promote patient safety: case study. JMIR aging. 2023;6:e41950.

Jayesinghe R, Moriarty F, Khatter A, Durbaba S, Ashworth M, Redmond P. Cost outcomes of potentially inappropriate prescribing in middle-aged adults: A Delphi consensus and cross-sectional study. Br J Clin Pharmacol. 2022;88(7):3404–20.

Johannessen T, Ree E, Stromme T, Aase I, Bal R, Wiig S. Designing and pilot testing of a leadership intervention to improve quality and safety in nursing homes and home care (the SAFE-LEAD intervention). BMJ Open. 2019;9(6):e027790.

Joseph K, Newman B, Manias E, Walpola R, Seale H, Walton M, et al. Engaging with ethnic minority consumers to improve safety in cancer services: A national stakeholder analysis. Patient Educ Couns. 2022;105(8):2778–84.

Khan A, Spector ND, Baird JD, Ashland M, Starmer AJ, Rosenbluth G, et al. Patient safety after implementation of a coproduced family centered communication programme: multicenter before and after intervention study. BMJ. 2018;363:k4764.

Khazen M, Mirica M, Carlile N, Groisser A, Schiff GD. Developing a framework and electronic tool for communicating diagnostic uncertainty in primary care: a qualitative study. JAMA Network Open. 2023;6(3):e232218-e.

Knight SW, Trinkle J, Tschannen D. Hospital-to-homecare videoconference handoff: improved communication, coordination of care, and patient/family engagement. Home Healthc Now. 2019;37(4):198–207.

Lawrence V, Kimona K, Howard RJ, Serfaty MA, Wetherell JL, Livingston G, et al. Optimising the acceptability and feasibility of acceptance and commitment therapy for treatment-resistant generalised anxiety disorder in older adults. Age Ageing. 2019;48(5):741–50.

Louch G, Reynolds C, Moore S, Marsh C, Heyhoe J, Albutt A, et al. Validation of revised patient measures of safety: PMOS-30 and PMOS-10. BMJ Open. 2019;9(11):e031355.

MacDonald T, Jackson S, Charles M-C, Periel M, Jean-Baptiste M-V, Salomon A, et al. The fourth delay and community-driven solutions to reduce maternal mortality in rural Haiti: a community-based action research study. BMC Pregnancy Childbirth. 2018;18(1):254.

Mackintosh N, Sandall J, Collison C, Carter W, Harris J. Employing the arts for knowledge production and translation: Visualizing new possibilities for women speaking up about safety concerns in maternity. Health Expect. 2018;21(3):647–58.

Marchand K, Turuba R, Katan C, Brasset C, Fogarty O, Tallon C, et al. Becoming our young people’s case managers: caregivers’ experiences, needs, and ideas for improving opioid use treatments for young people using opioids. Subst Abuse Treat Prev Policy. 2022;17(1):1–15.

Mazuz K, Biswas S. Co-designing technology and aging in a service setting: Developing an interpretive framework of how to interact with older age users. Gerontechnology. 2022;21(1):1–13.

McCahon D, Duncan P, Payne R, Horwood J. Patient perceptions and experiences of medication review: qualitative study in general practice. BMC Prim Care. 2022;23(1):293.

McMullen S, Panagioti M, Planner C, Giles S, Angelakis I, Keers RN, et al. Supporting carers to improve patient safety and maintain their well-being in transitions from mental health hospitals to the community: A prioritisation nominal group technique. Health Expect. 2023;26(5):2064–74.

Morris RL, Giles S, Campbell S. Involving patients and carers in patient safety in primary care: A qualitative study of a co-designed patient safety guide. Health Expect. 2023;26(2):630–9.

Morris RL, Stocks SJ, Alam R, Taylor S, Rolfe C, Glover SW, et al. Identifying primary care patient safety research priorities in the UK: a James Lind Alliance Priority Setting Partnership. BMJ Open. 2018;8(2):e020870.

Nether KG, Thomas EJ, Khan A, Ottosen MJ, Yager L. Implementing a robust process improvement program in the neonatal intensive care unit to reduce harm. J Healthc Qual. 2022;44(1):23–30.

Powell C, Ismail H, Cleverley R, Taylor A, Breen L, Fylan B, et al. Patients as qualitative data analysts: Developing a method for a process evaluation of the “Improving the Safety and Continuity of Medicines management at care Transitions” (ISCOMAT) cluster randomised control trial. Health Expect. 2021;24(4):1254–62.

Article   PubMed Central   Google Scholar  

Powell C, Ismail H, Davis M, Taylor A, Breen L, Fylan B, et al. Experiences of patients with heart failure with medicines at transition intervention: Findings from the process evaluation of the Improving the Safety and Continuity of Medicines management at Transitions of care (ISCOMAT) programme. Health Expect. 2022;25(5):2503–14.

Radecki B, Keen A, Miller J, McClure JK, Kara A. Innovating fall safety: engaging patients as experts. J Nurs Care Qual. 2020;35(3):220–6.

Rosgen BK, Plotnikoff KM, Krewulak KD, Shahid A, Hernandez L, Sept BG, et al. Co-development of a transitions in care bundle for patient transitions from the intensive care unit: a mixed-methods analysis of a stakeholder consensus meeting. BMC Health Serv Res. 2022;22(1):10.

Schenk EC, Bryant RA, Van Son CR, Odom-Maryon T. Developing an intervention to reduce harm in hospitalized patients: patients and families in research. J Nurs Care Qual. 2019;34(3):273–8.

Spazzapan M, Vijayakumar B, Stewart CE. A bit about me: Bedside boards to create a culture of patient-centered care in pediatric intensive care units (PICUs). J Healthc Risk Manag. 2020;39(3):11–9.

Stoll JA, Ranahan M, Richbart MT, Brennan-Taylor MK, Taylor JS, Brady L, et al. Development of video animations to encourage patient-driven deprescribing: A team alice study. Patient Educ Couns. 2021;104(11):2716–23.

Subbe CP, Tomos H, Jones GM, Barach P. Express check-in: developing a personal health record for patients admitted to hospital with medical emergencies: a mixed-method feasibility study. Int J Qual Health Care. 2021;33(3):121.

Tai D, Li E, Liu-Ambrose T, Bansback N, Sadatsafavi M, Davis JC. Patient-Reported Outcome Measures (PROMs) to support adherence to falls prevention clinic recommendations: a qualitative study. Patient Prefer Adherence. 2020;14:2105–21.

Thakur T, Chewning B, Zetes N, Lee JTY. Involving caregivers in design and assessment of opioid risk and safety communication intervention in children. Patient Educ Couns. 2021;104(10):2432–6.

Thomas J, Dahm MR, Li J, Georgiou A. Can patients contribute to enhancing the safety and effectiveness of test-result follow-up? Qualitative outcomes from a health consumer workshop. Health Expect. 2021;24(2):222–33.

Tremblay MC, Bradette-Laplante M, Witteman HO, Dogba MJ, Breault P, Paquette JS, et al. Providing culturally safe care to indigenous people living with diabetes: Identifying barriers and enablers from different perspectives. Health Expect. 2021;24(2):296–306.

Troya MI, Dikomitis L, Babatunde OO, Bartlam B, Chew-Graham CA. Understanding self-harm in older adults: A qualitative study. EClinicalMedicine. 2019;12:52–61.

Tyler N, Giles S, Daker-White G, McManus BC, Panagioti M. A patient and public involvement workshop using visual art and priority setting to provide patients with a voice to describe quality and safety concerns: Vitamin B12 deficiency and pernicious anaemia. Health Expect. 2021;24(1):87–94.

Tyler N, Planner C, Shears B, Hernan A, Panagioti M, Giles S. Developing the Resident Measure of Safety in Care Homes (RMOS): A Delphi and think aloud study. Health Expect. 2023;26(3):1149–58.

Van den Bulck SA, Vankrunkelsven P, Goderis G, Van Pottelbergh G, Swerts J, Panis K, et al. Developing quality indicators for Chronic Kidney Disease in primary care, extractable from the Electronic Medical Record. A Rand-modified Delphi method. BMC Nephrol. 2020;21(1):161.

Van Strien-Knippenberg IS, Boshuizen MCS, Determann D, de Boer JH, Damman OC. Cocreation with Dutch patients of decision-relevant information to support shared decision-making about adjuvant treatment in breast cancer care. Health Expect. 2022;25(4):1664–77.

Wilson NA, Reich AJ, Graham J, Bhatt DL, Nguyen LL, Weissman JS. Patient perspectives on the need for implanted device information: Implications for a post-procedural communication framework. Health Expect. 2021;24(4):1391–402.

Winterberg AV, Lane B, Hill LM, Varughese AM. Optimizing Pediatric Induction Experiences Using Human-centered Design. J Perianesth Nurs. 2022;37(1):48–52.

Yang R, Donaldson GW, Edelman LS, Cloyes KG, Sanders NA, Pepper GA. Fear of older adult falling questionnaire for caregivers (FOAFQ-CG): Evidence from content validity and item-response theory graded-response modelling. J Adv Nurs. 2020;76(10):2768–80.

Young A, Menon D, Street J, Al-Hertani W, Stafinski T. A checklist for managed access programmes for reimbursement co-designed by Canadian patients and caregivers. Health Expect. 2018;21(6):973–80.

Yuen EYN, Street M, Abdelrazek M, Blencowe P, Etienne G, Liskaser R, et al. Evaluating the efficacy of a digital App to enhance patient-centred nursing handover: A simulation study. J Clin Nurs. 2023;32(19–20):7626–37.

Jo S, Nabatchi T. Coproducing healthcare: individual-level impacts of engaging citizens to develop recommendations for reducing diagnostic error. Public Manag Rev. 2019;21(3):354–75.

O’Hara JK, Reynolds C, Moore S, Armitage G, Sheard L, Marsh C, et al. What can patients tell us about the quality and safety of hospital care? Findings from a UK multicentre survey study. BMJ Qual Saf. 2018;27(9):673–82.

de Jong LD, Francis-Coad J, Wortham C, Haines TP, Skelton DA, Weselman T, et al. Evaluating audio-visual falls prevention messages with community-dwelling older people using a World Cafe forum approach. BMC Geriatrics. 2019;19(1):345.

O’Donnell D, Shé ÉN, McCarthy M, Thornton S, Doran T, Smith F, et al. Enabling public, patient and practitioner involvement in co-designing frailty pathways in the acute care setting. BMC Health Serv Res. 2019;19(1):797.

Russ S, Latif Z, Hazell A, Ogunmuyiwa H, Tapper J, Wachuku-King S, et al. A Smartphone app designed to empower patients to contribute toward safer surgical care: community-based evaluation using a participatory approach. Jmir Mhealth Uhealth. 2020;8(1):e12859.

Mazuz K, Biswas S, Lindner U. Developing self-management application of fall prevention among older adults: a content and usability evaluation. Front Digital Health. 2020;2:11.

Hjelmfors L, Strömberg A, Friedrichsen M, Sandgren A, Mårtensson J, Jaarsma T. Using co-design to develop an intervention to improve communication about the heart failure trajectory and end-of-life care. Bmc Palliat Care. 2018;17:17.

Horgan S, Hegarty J, Andrews E, Hooton C, Drennan J. Impact of a quality improvement intervention on the incidence of surgical site infection in patients undergoing colorectal surgery: Pre-test-post-test design. J Clin Nurs. 2023;32(15–16):4932–46.

Tyler N, Wright N, Grundy A, Waring J. Developing a core outcome set for interventions to improve discharge from mental health inpatient services: a survey, Delphi and consensus meeting with key stakeholder groups. BMJ Open. 2020;10(5):e034215.

Ward ME, De Brún A, Beirne D, Conway C, Cunningham U, English A, et al. Using Co-Design to Develop a Collective Leadership Intervention for Healthcare Teams to Improve Safety Culture. Int J Environ Res Public Health. 2018;15(6):1182.

Berthelsen DB, Simon LS, Ioannidis JPA, Voshaar M, Richards P, Goel N, et al. Stakeholder endorsement advancing the implementation of a patient-reported domain for harms in rheumatology clinical trials: outcome of the OMERACT safety working group. Semin Arthritis Rheum. 2023;63:152288.

Okkenhaug A, Tritter JQ, Landstad BJ. Developing a research tool to detect iatrogenic adverse events in psychiatric health care by involving service users and health professionals. J Psychiatr Ment Health Nurs. 2023;00:1–12.

Musbahi A, Clyde D, Small P, Courtney M, Mahawar K, Lamb PJ, et al. A systematic review of patient and public involvement (PPI) in bariatric research trials: the need for more work. Obes Surg. 2022;32(11):3740–51.

Patel VA, Shelswell J, Hillyard N, Pavitt S, Barber SK. A study of the reporting of patient and public involvement and engagement (PPIE) in orthodontic research. J Orthod. 2021;48(1):42–51.

Price A, Schroter S, Snow R, Hicks M, Harmston R, Staniszewska S, et al. Frequency of reporting on patient and public involvement (PPI) in research studies published in a general medical journal: a descriptive study. BMJ Open. 2018;8:e020452.

Amadea T, Anne-Marie B, Louise L. A researcher’s guide to patient and public involvement. 2017.

Download references

Acknowledgements

This research is funded as part of the Collective Leadership and Safety Cultures (Co-Lead) research programme which is funded by the Irish Health Research Board, grant reference number RL-2015–1588 and the Health Service Executive. The funders had no role in the study conceptualisation, design, data collection, analysis, decision to publish, or preparation of the manuscript.

Author information

Authors and affiliations.

UCD Centre for Interdisciplinary Research, Education and Innovation in Health Systems (UCD IRIS), School of Nursing, Midwifery and Health Systems, Health Sciences Centre, University College Dublin, Dublin, Ireland

Sahar Hammoud, Laith Alsabek, Lisa Rogers & Eilish McAuliffe

Department of Oral and Maxillofacial Surgery, University Hospital Galway, Galway, Ireland

Laith Alsabek

You can also search for this author in PubMed   Google Scholar

Contributions

S.H and E.M.A designed the study. S.H developed the search strategies with feedback from L.A, L.R, and E.M.A. S.H conducted all searches. S.H and L.A screened the studies, extracted the data, and assessed the quality of PPI reporting. S.H analysed the data with feedback from E.M.A. S.H drafted the manuscript. All authors revised and approved the submitted manuscript. All authors agreed to be personally accountable for the author's own contributions and to ensure that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Authors’ information

Corresponding author.

Correspondence to Sahar Hammoud .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Hammoud, S., Alsabek, L., Rogers, L. et al. Systematic review on the frequency and quality of reporting patient and public involvement in patient safety research. BMC Health Serv Res 24 , 532 (2024). https://doi.org/10.1186/s12913-024-11021-z

Download citation

Received : 10 January 2024

Accepted : 21 April 2024

Published : 26 April 2024

DOI : https://doi.org/10.1186/s12913-024-11021-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Patient and public involvement
• Patient participation
• Research reporting
• Research involvement

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

An official website of the United States government

Here’s how you know

Official websites use .gov A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS A lock ( Lock A locked padlock ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

JavaScript appears to be disabled on this computer. Please click here to see any active alerts .

PFAS Enforcement Discretion and Settlement Policy Under CERCLA

Memorandum provides direction about how the EPA will exercise its enforcement discretion under the Comprehensive Environmental Response, Compensation, and Liability Act (CERCLA) in matters involving per- and polyfluoroalkyl substances (PFAS). The memorandum describes how EPA will focus on holding responsible entities who significantly contributed to the release of PFAS contamination into the environment, including parties that have manufactured PFAS or used PFAS in the manufacturing process, federal facilities, and other industrial parties.

The policy also reflects that EPA does not intend to pursue entities where equitable factors do not support seeking response actions or costs under CERCLA, including farmers, municipal landfills, water utilities, municipal airports, and local fire departments. Issued: April 19, 2024

PFAS Enforcement Discretion Settlement Policy Under CERCLA (pdf) (372.5 KB, 4/19/2024)

On April 19, 2024, the EPA announced that it was designating two types of PFAS, perfluorooctanoic acid (PFOA) and perfluorooctanesulfonic acid (PFOS), as CERCLA hazardous substances. Information regarding the rulemaking is available from the Agency's  Designation of Perfluorooctanoic Acid (PFOA) and Perfluorooctanesulfonic Acid (PFOS) as CERCLA Hazardous Substances web page. 

• Enforcement Home
• Enforcement Basics
• Enforcement and Compliance Annual Results for FY 2023
• Air Enforcement
• Water Enforcement
• Waste, Chemical and Cleanup Enforcement
• Criminal Enforcement
• Enforcement at Federal Facilities
• Data and Results
• Policy, Guidance and Publications