is peer review the same as literature review

Peer Reviewed Literature

What is peer review, terminology, peer review what does that mean, what types of articles are peer-reviewed, what information is not peer-reviewed, what about google scholar.

• How do I find peer-reviewed articles?
• Scholarly vs. Popular Sources

Research Librarian

For more help on this topic, please contact our Research Help Desk: [email protected] or 781-768-7303. Stay up-to-date on our current hours . Note: all hours are EST.

This Guide was created by Carolyn Swidrak (retired).

Research findings are communicated in many ways.  One of the most important ways is through publication in scholarly, peer-reviewed journals.

Research published in scholarly journals is held to a high standard.  It must make a credible and significant contribution to the discipline.  To ensure a very high level of quality, articles that are submitted to scholarly journals undergo a process called peer-review.

Once an article has been submitted for publication, it is reviewed by other independent, academic experts (at least two) in the same field as the authors.  These are the peers.  The peers evaluate the research and decide if it is good enough and important enough to publish.  Usually there is a back-and-forth exchange between the reviewers and the authors, including requests for revisions, before an article is published. 

Peer review is a rigorous process but the intensity varies by journal.  Some journals are very prestigious and receive many submissions for publication.  They publish only the very best, most highly regarded research. 

The terms scholarly, academic, peer-reviewed and refereed are sometimes used interchangeably, although there are slight differences.

Scholarly and academic may refer to peer-reviewed articles, but not all scholarly and academic journals are peer-reviewed (although most are.)  For example, the Harvard Business Review is an academic journal but it is editorially reviewed, not peer-reviewed.

Peer-reviewed and refereed are identical terms.

From  Peer Review in 3 Minutes  [Video], by the North Carolina State University Library, 2014, YouTube (https://youtu.be/rOCQZ7QnoN0).

Peer reviewed articles can include:

• Original research (empirical studies)
• Review articles
• Systematic reviews
• Meta-analyses

There is much excellent, credible information in existence that is NOT peer-reviewed.  Peer-review is simply ONE MEASURE of quality. 

Much of this information is referred to as "gray literature."

Government Agencies

Government websites such as the Centers for Disease Control (CDC) publish high level, trustworthy information.  However, most of it is not peer-reviewed.  (Some of their publications are peer-reviewed, however. The journal Emerging Infectious Diseases, published by the CDC is one example.)

Conference Proceedings

Papers from conference proceedings are not usually peer-reviewed.  They may go on to become published articles in a peer-reviewed journal. 

Dissertations

Dissertations are written by doctoral candidates, and while they are academic they are not peer-reviewed.

Many students like Google Scholar because it is easy to use.  While the results from Google Scholar are generally academic they are not necessarily peer-reviewed.  Typically, you will find:

• Peer reviewed journal articles (although they are not identified as peer-reviewed)
• Unpublished scholarly articles (not peer-reviewed)
• Masters theses, doctoral dissertations and other degree publications (not peer-reviewed)
• Book citations and links to some books (not necessarily peer-reviewed)
• Next: How do I find peer-reviewed articles? >>
• Last Updated: Feb 12, 2024 9:39 AM
• URL: https://libguides.regiscollege.edu/peer_review

• Maps & Floorplans
• Libraries A-Z

• Ellis Library (main)
• Engineering Library
• Geological Sciences
• Journalism Library
• Law Library
• Mathematical Sciences
• MU Digital Collections
• Veterinary Medical
• More Libraries...
• Instructional Services
• Course Reserves
• Course Guides
• Schedule a Library Class
• Class Assessment Forms
• Recordings & Tutorials
• Research & Writing Help
• More class resources
• Places to Study
• Borrow, Request & Renew
• Call Numbers
• Computers, Printers, Scanners & Software
• Digital Media Lab
• Equipment Lending: Laptops, cameras, etc.
• Subject Librarians
• Writing Tutors
• More In the Library...
• Undergraduate Students
• Graduate Students
• Faculty & Staff
• Researcher Support
• Distance Learners
• International Students
• More Services for...
• View my MU Libraries Account (login & click on My Library Account)
• View my MOBIUS Checkouts
• Renew my Books (login & click on My Loans)
• Place a Hold on a Book
• Request Books from Depository
• View my ILL@MU Account
• Set Up Alerts in Databases
• More Account Information...

Clinical Psychology Capstone: Literature Review & Peer Review

• Articles & Guidelines
• Facts & Figures
• Laws & Policies
• Managing Citations
• Literature Review & Peer Review

Literature Review

What are the differences between literature reviews?

• Literature Review - A general summary, or overview of the topic that is typically qualitative and subjective
• Systematic Review - A type of literature review that answers a focused clinical question
• Meta-Analysis - A type of systematic review using statistical methods to combine data from systematic reviews

What is the best way to find literature reviews?

• PsycINFO - A psychology database with the capacity to limit by Methodology

How do I know a journal is peer reviewed?

• If searching in a database (eg: Social Work Abstracts,GenderWatch), select Peer Review from the Refine/Limit Results options.
• Check the journal's website:  look for the 'about' or 'about this journal' section.
• Check Ulrich's Periodicals Directory ; Search by journal name and look for the little black referee's jersey icon.
• Ask us : Call, text, email, or chat

What does Peer Review mean anyway?

When you submit an article to a journal, someone has to determine if it's worth printing.  Peer review was developed as a way to screen articles and determine the quality of your article. 

At a peer reviewed journal, the editor sends your article out to several reviewers (usually three) who are in the same field, or 'peers'.  Generally, your name will be taken off of the article so personalities don't interfer with the process.  The reviewers read through your article looking to see if:  the topic is unique or novel, if the data or research is sound, and if it's well written.  The reviewers can: reject the article; accept it with revisions; accept it as is.  

Benefits of peer review is that multiple people decide vs just the editor and the review process weeds out poorly written or researched articles.

Drawbacks of peer review is that it's only as good as the reviewers so poorly written or researched articles have gotten published.  Also, peer review was established as a way to check quality not catch fraud.

For more on peer review (I know that someone is interested...), check out Nature Peer Review Debate

• << Previous: Managing Citations
• Last Updated: Apr 24, 2024 1:00 PM
• URL: https://libraryguides.missouri.edu/clinicalpsych

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 12 November 2021

Demystifying the process of scholarly peer-review: an autoethnographic investigation of feedback literacy of two award-winning peer reviewers

• Sin Wang Chong   ORCID: orcid.org/0000-0002-4519-0544 1 &
• Shannon Mason 2  

Humanities and Social Sciences Communications volume  8 , Article number:  266 ( 2021 ) Cite this article

2729 Accesses

6 Citations

18 Altmetric

Metrics details

• Language and linguistics

A Correction to this article was published on 26 November 2021

This article has been updated

Peer reviewers serve a vital role in assessing the value of published scholarship and improving the quality of submitted manuscripts. To provide more appropriate and systematic support to peer reviewers, especially those new to the role, this study documents the feedback practices and experiences of two award-winning peer reviewers in the field of education. Adopting a conceptual framework of feedback literacy and an autoethnographic-ecological lens, findings shed light on how the two authors design opportunities for feedback uptake, navigate responsibilities, reflect on their feedback experiences, and understand journal standards. Informed by ecological systems theory, the reflective narratives reveal how they unravel the five layers of contextual influences on their feedback practices as peer reviewers (micro, meso, exo, macro, chrono). Implications related to peer reviewer support are discussed and future research directions are proposed.

Similar content being viewed by others

The transformative power of values-enacted scholarship

What matters in the cultivation of student feedback literacy: exploring university efl teachers’ perceptions and practices.

Writing impact case studies: a comparative study of high-scoring and low-scoring case studies from REF2014

Introduction.

The peer-review process is the longstanding method by which research quality is assured. On the one hand, it aims to assess the quality of a manuscript, with the desired outcome being (in theory if not always in practice) that only research that has been conducted according to methodological and ethical principles be published in reputable journals and other dissemination outlets (Starck, 2017 ). On the other hand, it is seen as an opportunity to improve the quality of manuscripts, as peers identify errors and areas of weakness, and offer suggestions for improvement (Kelly et al., 2014 ). Whether or not peer review is actually successful in these areas is open to considerable debate, but in any case it is the “critical juncture where scientific work is accepted for publication or rejected” (Heesen and Bright, 2020 , p. 2). In contemporary academia, where higher education systems across the world are contending with decreasing levels of public funding, there is increasing pressure on researchers to be ‘productive’, which is largely measured by the number of papers published, and of funding grants awarded (Kandiko, 2010 ), both of which involve peer review.

Researchers are generally invited to review manuscripts once they have established themselves in their disciplinary field through publication of their own research. This means that for early career researchers (ECRs), their first exposure to the peer-review process is generally as an author. These early experiences influence the ways ECRs themselves conduct peer review. However, negative experiences can have a profound and lasting impact on researchers’ professional identity. This appears to be particularly true when feedback is perceived to be unfair, with feedback tone largely shaping author experience (Horn, 2016 ). In most fields, reviewers remain anonymous to ensure freedom to give honest and critical feedback, although there are concerns that a lack of accountability can result in ‘bad’ and ‘rude’ reviews (Mavrogenis et al., 2020 ). Such reviews can negatively impact all researchers, but disproportionately impact underrepresented researchers (Silbiger and Stubler, 2019 ). Regardless of career phase, no one is served well by unprofessional reviews, which contribute to the ongoing problem of bullying and toxicity prevalent in academia, with serious implications on the health and well-being of researchers (Keashly and Neuman, 2010 ).

Because of its position as the central process through which research is vetted and refined, peer review should play a similarly central role in researcher training, although it rarely features. In surveying almost 3000 researchers, Warne ( 2016 ) found that support for reviewers was mostly received “in the form of journal guidelines or informally as advice from supervisors or colleagues” (p. 41), with very few engaging in formal training. Among more than 1600 reviewers of 41 nursing journals, only one third received any form of support (Freda et al., 2009 ), with participants across both of these studies calling for further training. In light of the lack of widespread formal training, most researchers learn ‘on the job’, and little is known about how researchers develop their knowledge and skills in providing effective assessment feedback to their peers. In this study, we undertake such an investigation, by drawing on our first-hand experiences. Through a collaborative and reflective process, we look to identify the forms and forces of our feedback literacy development, and seek to answer specifically the following research questions:

What are the exhibited features of peer reviewer feedback literacy?

What are the forces at work that affect the development of feedback literacy?

Literature review

Conceptualisation of feedback literacy.

The notion of feedback literacy originates from the research base of new literacy studies, which examines ‘literacies’ from a sociocultural perspective (Gee, 1999 ; Street, 1997 ). In the educational context, one of the most notable types of literacy is assessment literacy (Stiggins, 1999 ). Traditionally, assessment literacy is perceived as one of the indispensable qualities of a successful educator, which refers to the skills and knowledge for teachers “to deal with the new world of assessment” (Fulcher, 2012 , p. 115). Following this line of teacher-oriented assessment literacy, recent attempts have been made to develop more subject-specific assessment literacy constructs (e.g., Levi and Inbar-Lourie, 2019 ). Given the rise of student-centred approaches and formative assessment in higher education, researchers began to make the case for students to be ‘assessment literate’; comprising of such knowledge and skills as understanding of assessment standards, the relationship between assessment and learning, peer assessment, and self-assessment skills (Price et al., 2012 ). Feedback literacy, as argued by Winstone and Carless ( 2019 ), is essentially a subset of assessment literacy because “part of learning through assessment is using feedback to calibrate evaluative judgement” (p. 24). The notion of feedback literacy was first extensively discussed by Sutton ( 2012 ) and more recently by Carless and Boud ( 2018 ). Focusing on students’ feedback literacy, Sutton ( 2012 ) conceptualised feedback literacy as a three-dimensional construct—an epistemological dimension (what do I know about feedback?), an ontological dimension (How capable am I to understand feedback?), and a practical dimension (How can I engage with feedback?). In close alignment with Sutton’s construct, the seminal conceptual paper by Carless and Boud ( 2018 ) further illustrated the four distinctive abilities of feedback literate students: the abilities to (1) understand the formative role of feedback, (2) make informed and accurate evaluative judgement against standards, (3) manage emotions especially in the face of critical and harsh feedback, and (4) take action based on feedback. Since the publication of Carless and Boud ( 2018 ), student and teacher feedback literacy has been in the limelight of assessment research in higher education (e.g., Chong 2021b ; Carless and Winstone 2020 ). These conceptual contributions expand the notion of feedback literacy to consider not only the manifestations of various forms of effective student engagement with feedback but also the confluence of contexts and individual differences of students in developing students’ feedback literacy by drawing upon various theoretical perspectives (e.g., ecological systems theory; sociomaterial perspective) and disciplines (e.g., business and human resource management). Others address practicalities of feedback literacy; for example, how teachers and students can work in synergy to develop feedback literacy (Carless and Winstone, 2020 ) and ways to maximise student engagement with feedback at a curricular level (Malecka et al., 2020). In addition to conceptualisation, advancement of the notion of feedback literacy is evident in the recent proliferation of primary studies. The majority of these studies are conducted in the field of higher education, focusing mostly on student feedback literacy in classrooms (e.g., Molloy et al., 2019 ; Winstone et al., 2019 ) and in the workplace (Noble et al., 2020 ), with a handful focused on teacher feedback literacy (e.g., Xu and Carless 2016 ). Some studies focusing on student feedback literacy adopt a qualitative case study research design to delve into individual students’ experience of engaging with various forms of feedback. For example, Han and Xu ( 2019 ) analysed the profiles of feedback literacy of two Chinese undergraduate students. Findings uncovered students’ resistance to engagement with feedback, which relates to the misalignment between the cognitive, social, and affective components of individual students’ feedback literacy profiles. Others reported interventions designed to facilitate students’ uptake of feedback, focusing on their effectiveness and students’ perceptions. Specifically, affordances and constraints of educational technology such as electronic feedback portfolio (Chong, 2019 ; Winstone et al., 2019 ) are investigated. Of particular interest is a recent study by Noble et al. ( 2020 ), which looked into student feedback literacy in the workplace by probing into the perceptions of a group of Australian healthcare students towards a feedback literacy training programme conducted prior to their placement. There is, however, a dearth of primary research in other areas where elicitation, process, and enactment of feedback are vital; for instance, academics’ feedback literacy. In the ‘publish or perish’ culture of higher education, academics, especially ECRs, face immense pressure to publish in top-tiered journals in their fields and face the daunting peer-review process, while juggling other teaching and administrative responsibilities (Hollywood et al., 2019 ; Tynan and Garbett 2007 ). Taking up the role of authors and reviewers, researchers have to possess the capacity and disposition to engage meaningfully with feedback provided by peer reviewers and to provide constructive comments to authors. Similar to students, researchers have to learn how to manage their emotions in the face of critical feedback, to understand the formative values of feedback, and to make informed judgements about the quality of feedback (Gravett et al., 2019 ). At the same time, feedback literacy of academics also resembles that of teachers. When considering the kind of feedback given to authors, academics who serve as peer reviewers have to (1) design opportunities for feedback uptake, (2) maintain a professional and supportive relationship with authors, and (3) take into account the practical dimension of giving feedback (e.g., how to strike a balance between quality of feedback and time constraints due to multiple commitments) (Carless and Winstone 2020 ). To address the above, one of the aims of the present study is to expand the application of feedback literacy as a useful analytical lens to areas outside the classroom, that is, scholarly peer-review activities in academia, by presenting, analysing, and synthesising the personal experiences of the authors as successful peer reviewers for academic journals.

Conceptual framework

We adopt a feedback literacy of peer reviewers framework (Chong 2021a ) as an analytical lens to analyse, systemise, and synthesise our own experiences and practices as scholarly peer reviewers (Fig. 1 ). This two-tier framework includes a dimension on the manifestation of feedback literacy, which categorises five features of feedback literacy of peer reviewers, informed by student and teacher feedback literacy frameworks by Carless and Boud ( 2018 ) and Carless and Winstone ( 2020 ). When engaging in scholarly peer review, reviewers are expected to be able to provide constructive and formative feedback, which authors can act on in their revisions ( engineer feedback uptake ). Besides, peer reviewers who are usually full-time researchers or academics lead hectic professional lives; thus, when writing reviewers’ reports, it is important for them to consider practically and realistically the time they can invest and how their various degrees of commitment may have an impact on the feedback they provide ( navigate responsibilities ). Furthermore, peer reviewers should consider the emotional and relational influences their feedback exert on the authors. It is crucial for feedback to be not only informative but also supportive and professional (Chong, 2018 ) ( maintain relationships ). Equally important, it is imperative for peer reviewers to critically reflect on their own experience in the scholarly peer-review process, including their experience of receiving and giving feedback to academic peers, as well as the ways authors and editors respond to their feedback ( reflect on feedback experienc e). Lastly, acting as gatekeepers of journals to assess the quality of manuscripts, peer reviewers have to demonstrate an accurate understanding of the journals’ aims, remit, guidelines and standards, and reflect those in their written assessments of submitted manuscripts ( understand standards ). Situated in the context of scholarly peer review, this collaborative autoethnographic study conceptualises feedback literacy not only as a set of abilities but also orientations (London and Smither, 2002 ; Steelman and Wolfeld, 2016 ), which refers to academics’ tendency, beliefs, and habits in relation to engaging with feedback (London and Smither, 2002 ). According to Cheung ( 2000 ), orientations are influenced by a plethora of factors, namely experiences, cultures, and politics. It is important to understand feedback literacy as orientations because it takes into account that feedback is a convoluted process and is influenced by a plethora of contextual and personal factors. Informed by ecological systems theory (Bronfenbrenner, 1986 ; Neal and Neal, 2013 ) and synthesising existing feedback literacy models (Carless and Boud, 2018 ; Carless and Winstone, 2020 ; Chong, 2021a , 2021b ), we consider feedback literacy as a malleable, situated, and emergent construct, which is influenced by the interplay of various networked layers of ecological systems (Neal and Neal, 2013 ) (Fig. 1 ). Also important is that conceptualising feedback literacy as orientations avoids dichotomisation (feedback literate vs. feedback illiterate), emphasises the developmental nature of feedback literacy, and better captures the multifaceted manifestations of feedback engagement.

The outer ring of the figure shows the components of feedback literacy while the inner ring concerns the layers of contexts (ecosystems) which influence the manifestation of feedback literacy of peer reviewers.

Echoing recent conceptual papers on feedback literacy which emphasises the indispensable role of contexts (Chong 2021b ; Boud and Dawson, 2021 ; Gravett et al., 2019 ), our conceptual framework includes an underlying dimension of networked ecological systems (micro, meso, exo, macro, and chrono), which portrays the contextual forces shaping our feedback orientations. Informed by the networked ecological system theory of Neal and Neal ( 2013 ), we postulate that there are five systems of contextual influence, which affect the feedback experience and development of feedback literacy of peer reviewers. The five ecological systems refer to ‘settings’, which is defined by Bronfenbrenner ( 1986 ) as “place[s] where people can readily engage in social interactions” (p. 22). Even though Bronfenbrenner’s ( 1986 ) somewhat dated definition of ‘place’ is limited to ‘physical space’, we believe that ‘places’ should be more broadly defined in the 21st century to encompass physical and virtual, recent and dated, closed and distanced locations where people engage; as for ‘interactions’, from a sociocultural perspective, we understand that ‘interactions’ can include not only social, but also cognitive and emotional exchanges (Vygotsky, 1978 ). Microsystem refers to a setting where people, including the focal individual, interact. Mesosystem , on the other hand, means the interactions between people from different settings and the influence they exert on the focal individual. An exosystem , similar to a microsystem, is understood as a single setting but this setting excludes the focal individual but it is likely that participants in this setting would interact with the focal individual. The remaining two systems, macrosystem and chronosystem, refer not only to ‘settings’ but ‘forces that shape the patterns of social interactions that define settings’ (Neal and Neal, 2013 , p. 729). Macrosystem is “the set of social patterns that govern the formation and dissolution of… interactions… and thus the relationship among ecological systems” (ibid). Some examples of macrosystems given by Neal and Neal ( 2013 ) include political and cultural systems. Finally, chronosystem is “the observation that patterns of social interactions between individuals change over time, and that such changes impact on the focal individual” (ibid, p. 729). Figure 2 illustrates this networked ecological systems theory using a hypothetical example of an early career researcher who is involved in scholarly peer review for Journal A; at the same time, they are completing a PhD and are working as a faculty member at a university.

This is a hypothetical example of an early career researcher who is involved in scholarly peer review for Journal A.

From the reviewed literature on the construct of feedback literacy, the investigation of feedback literacy as a personal, situated, and unfolding process is best done through an autoethnographic lens, which underscores critical self-reflection. Autoethnography refers to “an approach to research and writing that seeks to describe and systematically analyse (graphy) personal experience (auto) in order to understand cultural experience (ethno)” (Ellis et al., 2011 , p. 273). Autoethnography stems from research in the field of anthropology and is later introduced to the fields of education by Ellis and Bochner ( 1996 ). In higher education research, autoethnographic studies are conducted to illuminate on topics related to identity and teaching practices (e.g., Abedi Asante and Abubakari, 2020 ; Hains-Wesson and Young 2016 ; Kumar, 2020 ). In this article, a collaborative approach to autoethnography is adopted. Based on Chang et al. ( 2013 ), Lapadat ( 2017 ) defines collaborative autoethnography (CAE) as follows:

… an autobiographic qualitative research method that combines the autobiographic study of self with ethnographic analysis of the sociocultural milieu within which the researchers are situated, and in which the collaborating researchers interact dialogically to analyse and interpret the collection of autobiographic data. (p. 598)

CAE is not only a product but a worldview and process (Wall, 2006 ). CAE is a discrete view about the world and research, which straddles between paradigmatic boundaries of scientific and literary studies. Similar to traditional scientific research, CAE advocates systematicity in the research process and consideration is given to such crucial research issues as reliability, validity, generalisability, and ethics (Lapadat, 2017 ). In closer alignment with studies on humanities and literature, the goal of CAE is not to uncover irrefutable universal truths and generate theories; instead, researchers of CAE are interested in co-constructing and analysing their own personal narratives or ‘stories’ to enrich and/or challenge mainstream beliefs and ideas, embracing diverse rather than canonical ways of behaviour, experience, and thinking (Ellis et al., 2011 ). Regarding the role of researchers, CAE researchers openly acknowledge the influence (and also vulnerability) of researchers throughout the research process and interpret this juxtaposition of identities between researchers and participants of research as conducive to offering an insider’s perspective to illustrate sociocultural phenomena (Sughrua, 2019 ). For our CAE on the scholarly peer-review experiences of two ECRs, the purpose is to reconstruct, analyse, and publicise our lived experience as peer reviewers and how multiple forces (i.e., ecological systems) interact to shape our identity, experience, and feedback practice. As a research process, CAE is a collaborative and dynamic reflective journey towards self-discovery, resulting in narratives, which connect with and add to the existing literature base in a personalised manner (Ellis et al., 2011 ). The collaborators should go beyond personal reflection to engage in dialogues to identify similarities and differences in experiences to throw new light on sociocultural phenomena (Merga et al., 2018 ). The iterative process of self- and collective reflections takes place when CAE researchers write about their own “remembered moments perceived to have significantly impacted the trajectory of a person’s life” and read each other’s stories (Ellis et al., 2011 , p. 275). These ‘moments’ or vignettes are usually written retrospectively, selectively, and systematically to shed light on facets of personal experience (Hughes et al., 2012 ). In addition to personal stories, some autoethnographies and CAEs utilise multiple data sources (e.g., reflective essays, diaries, photographs, interviews with co-researchers) and various ways of expressions (e.g., metaphors) to achieve some sort of triangulation and to present evidence in a ‘systematic’ yet evocative manner (Kumar, 2020 ). One could easily notice that overarching methodological principles are discussed in lieu of a set of rigid and linear steps because the process of reconstructing experience through storytelling can be messy and emergent, and certain degree of flexibility is necessary. However, autoethnographic studies, like other primary studies, address core research issues including reliability (reader’s judgement of the credibility of the narrator), validity (reader’s judgement that the narratives are believable), and generalisability (resemblance between the reader’s experience and the narrative, or enlightenment of the reader regarding unfamiliar cultural practices) (Ellis et al., 2011 ). Ethical issues also need to be considered. For example, authors are expected to be honest in reporting their experiences; to protect the privacy of the people who ‘participated’ in our stories, pseudonyms need to be used (Wilkinson, 2019 ). For the current study, we follow the suggested CAE process outlined by Chang et al. ( 2013 ), which includes four stages: deciding on topic and method , collecting materials , making meaning , and writing . When deciding on the topic, we decided to focus on our experience as scholarly peer reviewers because doing peer review and having our work reviewed are an indispensable part of our academic lives. The next is to collect relevant autoethnographic materials. In this study, we follow Kumar ( 2020 ) to focus on multiple data sources: (1) reflective essays which were written separately through ‘recalling’, which is referred to by Chang et al. ( 2013 ) as ‘a free-spirited way of bringing out memories about critical events, people, place, behaviours, talks, thoughts, perspectives, opinions, and emotions pertaining to the research topic’ (p. 113), and (2) discussion meetings. In our reflective essays, we included written records of reflection and excerpts of feedback in our peer-review reports. Following material collection is meaning making. CAE, as opposed to autoethnography, emphasises the importance of engaging in dialogues with collaborators and through this process we identify similarities and differences in our experiences (Sughrua, 2019 ). To do so, we exchanged our reflective essays; we read each other’s reflections and added questions or comments on the margins. Then, we met online twice to share our experiences and exchange views regarding the two reflective essays we wrote. Both meetings lasted for approximately 90 min, were audio-recorded and transcribed. After each meeting, we coded our stories and experiences with reference to the two dimensions of the ecological framework of feedback literacy (Fig. 1 ). With regards to coding our data, we followed the model of Miles and Huberman ( 1994 ), which comprises four stages: data reduction (abstracting data), data display (visualising data in tabular form), conclusion-drawing, and verification. The coding and writing processes were done collaboratively on Google Docs and care was taken to address the aforesaid ethical (e.g., honesty, privacy) and methodological issues (e.g., validity, reliability, generalisability). As a CAE study, the participants are the researchers themselves, that is, the two authors of this paper. We acknowledge that research data are collected from human subjects (from the two authors), such data are collected in accordance with the standards and guidelines of the School Research Ethics Committee at the School of Social Sciences, Education and Social Work, Queen’s University Belfast (Ref: 005_2021). Despite our different experiences in our unique training and employment contexts, we share some common characteristics, both being ECRs (<5 years post-PhD), working in the field of education, active in the scholarly publication process as both authors and peer reviewers. Importantly for this study, we were both recipients of Reviewer of the Year Award 2019 awarded jointly by the journal, Higher Education Research & Development and the publisher , Taylor & Francis. This award in recognition of the quality of our reviewing efforts, as determined by the editorial board of a prestigious higher education journal, provided a strong impetus for this study, providing an opportunity to reflect on our own experiences and practices. The extent of our peer-review activities during our early career leading up to the time of data collection is summarised in Table 1 .

Findings and discussion

Analysis of the four individual essays (E1 and E2 for each participant) and transcripts of the two subsequent discussions (D1 and D2) resulted in the identification of multiple descriptive codes and in turn a number of overarching themes (Supplementary Appendix 1). Our reporting of these themes is guided by our conceptual framework, where we first focus on the five manifestations of feedback literacy to highlight the experiences that contribute to our growth as effective and confident peer reviewers. Then, we report on the five ecological systems to unravel how each contextual layer develops our feedback literacy as peer reviewers. (Note that the discussion of the chronosystem has been necessarily incorporated into each of the four others dimensions: microsystem , mesosystem , exosystem , and macrosystem in order to demonstrate temporal changes). In particular, similarities and differences will be underscored, and connections with manifested feedback beliefs and behaviours will be made. We include quotes from both Author 1 (A1) and Author 2 (A2), in order to illustrate our findings, and to show the richness and depth of the data collected (Corden and Sainsbury, 2006 ). Transcribed quotes may be lightly edited while retaining meaning, for example through the removal of fillers and repetitions, which is generally accepted practice to ensure readability ( ibid ).

Manifestations of feedback literacy

Engineering feedback uptake.

The two authors have a strong sense of the purpose of peer review as promoting not only research quality, but the growth of researchers. One way that we engineer author uptake is to ensure that feedback is ‘clear’ (A2,E1), ‘explicit’ (A2,E1), ‘specific’ (A1,E1), and importantly ‘actionable… to ensure that authors can act on this feedback so that their manuscripts can be improved and ultimately accepted for publication’ (A1,E1). In less than favourable author outcomes, we ensure that there is reference to the role of the feedback in promoting the development of the manuscript, which A1 refers to as ‘promotion of a growth mindset’ (A1,E1). For example, after requesting a second round of major revisions, A2 ‘acknowledged the frustration that the author might have felt on getting further revisions by noting how much improvement was made to the paper, but also making clear the justification for sending it off for more work’ (A2,E1). We both note that we tend to write longer reviews when a rejection is the recommended outcome, as our ultimate goal is to aid in the development of a manuscript.

Rejections doesn’t mean a paper is beyond repair. It can still be fixed and improved; a rejection simply means that the fix may be too extensive even for multiple review cycles. It is crucial to let the authors whose manuscripts are rejected know that they can still act on the feedback to improve their work; they should not give up on their own work. I think this message is especially important to first-time authors or early career researchers. (A1,E1)

In promoting a growth mindset and in providing actionable feedback, we hope to ‘show the authors that I’m not targeting them, but their work’ (A1,D1). We particularly draw on our own experiences as ECRs, with first-hand understanding that ‘everyone takes it personally when they get rejected. Yeah. Moreover, it is hard to separate (yourself from the paper)’ (A2,D1).

Navigating responsibilities

As with most academics, the two authors have multiple pressures on their time, and there ‘isn’t much formal recognition or reward’ (A1,E1) and ‘little extrinsic incentive for me to review’ (A2,E1). Nevertheless we both view our roles as peer reviewers as ‘an important part of the process’ (A2,E1), ‘a modest way for me to give back to the academic community’ (A1,E1). Through peer review we have built a sense of ‘identity as an academic’ (A1,D1), through ‘being a member of the academic community’ (A2,D1). While A1 commits to ‘review as many papers as possible’ (A1,E1) and A2 will usually accept offers to review, there are still limits on our time and therefore we consider the topic and methods employed when deciding whether or not to accept an invitation, as well as the journal itself, as we feel we can review more efficiently for journals with which we are more familiar. A1 and A2 have different processes for conducting their review that are most efficient for their own situations. For A1, the process begins with reading the whole manuscript in one go, adding notes to the pdf document along the way, which he then reviews, and makes a tentative decision, including ‘a few reasons why I have come to this decision’ (A1,E1). After waiting at least one day, he reviews all of the notes and begins writing the report, which is divided into the sections of the paper. He notes it ‘usually takes me 30–45 min to write a report. I then proofread this report and submit it to the system. So it usually takes me no more than three hours to complete a review’ (A1,E1). For A2, the process for reviewing and structuring the report is quite different, with a need to ‘just find small but regular opportunities to work on the review’ (A2,E1). As was the case during her Ph.D, which involved juggling research and raising two babies, ‘I’ve trained myself to be able to do things in bits’ (A2,D1). So A2 also begins by reading the paper once through, although generally without making initial comments. The next phase involves going through the paper at various points in time whenever possible, and at the same time building up the report, making the report structurally slightly different to that of A1.

What my reviews look like are bullet points, basically. And they’re not really in a particular order. They generally… follow the flow (of the paper). But I mean, I might think of something, looking at the methods and realise, hey, you haven’t defined this concept in the literature review so I’ll just add you haven’t done this. And so I will usually preface (the review)… Here’s a list of suggestions. Some of them are minor, some of them are serious, but they’re in no particular order. (A1,D1)

As such, both reviewers engage in personalised strategies to make more effective use of their time. Both A1 and A2 give explicit but not exhaustive examples of an area of concern, and they also pose questions for the author to consider, in both cases placing the onus back on the author to take action. As A1 notes, ‘I’m not going to do a summary of that reference for you. I’m just going to include that there. If you’d like you can check it out’ (A1,D1). For A2, a lack of adequate reporting of the methods employed in a study makes it difficult to proceed, and in such cases will not invest further time, sending it back to the editor, because ‘I can’t even comment on the findings… I can’t go on. I’m not gonna waste my time’ (A2,D1). In cases where the authors may be ‘on the fence’ about a particular review, they will use the confidential comments to the editor to help work through difficult cases as ‘they are obviously very experienced reviewers’ (A1,D1). Delegating tasks to the expertise of the editorial teams when appropriate also ensures time is used more prudently.

Maintaining relationships

Except in a few cases where A2 has reviewed for journals with a single-blind model, the vast majority of the reviews that we have completed have been double-blind. This means that we are unaware of the identity of the author/s, and we are unknown to them. However, ‘even with blind-reviews I tend to think of it as a conversation with a person’ (A2,E1). A1 talks about the need to have respect for the author and their expertise and effort ‘regardless of the quality of the submission (which can be in some cases subjective)’ (A1,E1). A2 writes similarly about the ‘privilege’ and ‘responsibility’ of being able to review manuscripts that authors ‘have put so much time and energy into possibly over an extended period’ (A2,E1). In this way it is possible to develop a sort of relationship with an author even without knowing their identity. In trying to articulate the nature of that relationship (which we struggle to do so definitively), we note that it is more than just a reviewer, and A2 reflected on a recent review, which went through a number of rounds of resubmission where ‘it felt like we were developing a relationship, more like a mentor than a reviewer’ (A2,E1).

I consider this role as a peer reviewer more than giving helpful and actionable feedback; I would like to be a supporter and critical friend to the authors, even though in most cases I don’t even know who they are or what career stage they are at (A1,E1).

In any case, as A1 notes, ‘we don’t even need to know who that person is because we know that people like encouragement’ (A1,D1), and we are very conscious of the emotional impact that feedback can have on authors, and the inherent power imbalance in the relationship. For this reason, A1 is ‘cautious about the way I write so that I don’t accidentally make the authors the target of my feedback’. As A2 notes ‘I don’t want authors feeling depressed after reading a review’ (A2,E1). While we note that we try to deliver our feedback with ‘respect’ (A1,E1; A1,E2; A2,D1) ‘empathy’ (A1,E1), and ‘kindness’ (A2,D1), we both noted that we do not ‘sugar coat’ our feedback and A1 describes himself as ‘harsh’ and ‘critical’ (A1,E1) while A2 describes herself as ‘pretty direct’ (A2,E1). In our discussion, we tried to delve into this seeming contradiction:… the encouragement, hopefully is to the researcher, but the directness it should be, I hope, is related directly to whatever it is, the methods or the reporting or the scope of the literature review. It’s something specific about the manuscript itself. And I know myself, being an ECR and being reviewed, that it’s hard to separate yourself from your work… And I want to make it really explicit. If it’s critical, it’s not about the person. It’s about the work, you know, the weakness of the work, but not the person. (A2,D1)

A1 explains that at times his initial report may be highly critical, and at times he will ‘sit back and rethink… With empathy, I will write feedback, which is more constructive’ (A1,E1). However, he adds that ‘I will never try to overrate a piece or sugar-coat my comments just to sound “friendly”’ (A1,E1), with the ultimate goal being to uphold academic rigour. Thus, honesty is seen as the best strategy to maintain a strong, professional relationship with reviewers. Another strategy employed by A2 is showing explicit commitment to the review process. One way this is communicated is by prefacing a review with a summary of the paper, not only ‘to confirm with the author that I am interpreting the findings in the way that they intended, but also importantly to show that I have engaged with the paper’ (A2,E1). Further, if the recommendation is for a further round of review, she will state directly to the authors ‘that I would be happy to review a revised manuscript’ (A2,E1).

Reflecting on feedback experience

As ECRs we have engaged in the scholarly publishing process initially as authors, subsequently as reviewers, and most recently as Associate Editors. Insights gained in each of these roles have influenced our feedback practices, and have interacted to ‘develop a more holistic understanding of the whole review process’ (A1,E1).

We reflect on our experiences as authors beginning in our doctoral candidatures, with reviews that ranged from ‘the most helpful to the most cynical’ (A1,E1). A2 reflected on two particular experiences both of which resulted in rejection, one being ‘snarky’ and ‘unprofessional’ with ‘no substance’, the other providing ‘strong encouragement … the focus was clearly on the paper and not me personally’ (A2,E1). It was this experience that showed the divergence between the tone and content of review despite the same outcome, and as result A2 committed to being ‘ the amazing one’. A1 also drew from a negative experience noting that ‘I remember the least useful feedback as much as I do with the most constructive one’ (A1,E1). This was particularly the case when a reviewer made apparently politically-motivated judgements that A1 ‘felt very uncomfortable with’ and flagged with the editor (A1,E1). Through these experiences both authors wrote in their essays about the need to focus on the work and not on the individual, with an understanding that a review ‘can have a really serious impact’ (A2,D1) on an author.

It is important to note that neither authors have been involved in any formal or informal training on how to conduct peer review, although A1 expresses appreciation of the regular practice of one journal for which he reviews, where ‘the editor would write an email to the reviewers giving feedback on the feedback we have given’ (A1,E1). For A2, an important source of learning is in comparing her reviews with that of others who have reviewed the same manuscript, the norm for some journals being to send all reports to all reviewers along with the final decision.

I’m always interested to see how [my] review compares with others. Have I given the same recommendation? Have I identified the same areas of weakness? Have I formatted my review in the same way? How does the tone of delivery differ? I generally find that I give a similar if not the same response to other reviews, and I’m happy to see that I often pick up the same issues with methodology. (A2,E1)

For A2 there is comfort in seeing reviews that are similar to others, although we both draw on experiences where our recommendation diverged from others, with a source of assurance being the ultimate decision of the editor.

So it’s like, I don’t think it can be published and that [other] reviewer thinks it’s excellent. So usually, what the editor would do in this instance is invite the third one. Right, yeah. But then this editor told me… that they decided to go with my decision to reject because they find that my comments are more convincing. (A1,D1)

A2 also was surprised to read another report of the same manuscript she reviewed, that raised similar concerns and gave the same recommendation for major revisions, but noted the ‘wording is soooo snarky. What need?’ (A2,E1). In one case that A1 detailed in our first discussion, significant but improbable changes made to the methodology section of a resubmitted paper caused him to question the honesty of the reporting, making him ‘uncomfortable’ and as a result reported his concerns to the editor. In this case the review took some time to craft, trying to balance the ‘fine line between catering for the emotion [of the author], right, and upholding the academic standards’ (A1,D1). While he conceded initially his report was ‘kind of too harsh… later I think I rephrased it a little bit, I kind of softened (it)’.

While the role of Associate Editor is very new to A2 and thus was yet unable to comment, for A1 the ‘opportunity to read various kinds of comments given by reviewers’ (A1,E1) is viewed favourably. This includes not only how reviewers structure their feedback, but also how they use the confidential comments to the editors to express their thoughts more openly, providing important insights into the process that are largely hidden.

Understanding standards

While our reviewing practices are informed more broadly ‘according to more general academic standards of the study itself, and the clarity and fullness of the reporting’ (A2,E1), we look in the first instance to advice and guidelines from journals to develop an understanding of journal-specific standards, although A2 notes that a lack of review guidelines for one of the earliest journals she reviewed led her to ‘searching Google for standard criteria’ (A2,E1). However, our development in this area seems to come from developing a familiarity with a journal, particularly through engagement with the journal as an author.

In addition to reading the scope and instructions for authors to obtain such basic information as readership, length of submissions, citation style, the best way for me to understand the requirements and preferences of the journals is my own experience as an author. I review for journals which I have published in and for those which I have not. I always find it easier to make a judgement about whether the manuscripts I review meet the journal’s standards if I have published there before. (A1,E1)

Indeed, it seems that journal familiarity is connected closely to our confidence in reviewing, and while both authors ‘review for journals which I have published in and for those which I have not’ (A1,E1), A2 states that she is reluctant to ‘readily accept an offer to review for a journal that I’m not familiar with’, and A1 takes extra time to ‘do more preparatory work before I begin reading the manuscript and writing the review’ when reviewing for an unfamiliar journal.

Ecological systems

Microsystem.

Three microsystems exert influence on A1’s and A2’s development of feedback literacy: university, journal community, and Twitter.

In regards to the university, we are full-time academics in research-intensive universities in the UK and Japan where expectations for academics include publishing research in high-impact journals ‘which is vital to promotion’ (A1,E2). It is especially true in A2’s context where the national higher education agenda is to increase world rankings of universities. Thus, ‘there is little value placed on peer review, as it is not directly related to the broader agenda’ (A2,E2). When considering his recent relocation to the UK together with the current pandemic, A1 navigated his responsibilities within the university context and decided to allocate more time to his university-related responsibilities, especially providing learning and pastoral support to his students, who are mostly international students. Besides, A2 observed that there is a dearth of institution-wide support on conducting peer review although ‘there are a lot of training opportunities related to how to write academic papers in English, how to present at international conferences, how to write grant applications’, etc. (A2,E2). As a result, she ‘struggled for a couple of years’ because of the lack of institutional support for her development as a peer reviewer’ (A2,D2); but this helplessness also motivated her to seek her own ways to learn how to give feedback, such as ‘seeing through glimpses of other reviews, how others approach it, in terms of length, structure, tone, foci etc.’ (A2,E2). A1 shares the same view that no training is available at his institution to support his development as a peer reviewer. However, his postgraduate supervision experiences enabled him to reflect on how his feedback can benefit researchers. In our second online discussion, A1 shared that he held individual advising sessions with some postgraduate students, which made him realise that it is important for feedback to serve the function to inspire rather than to ‘give them right answers’ (A1,D2).

Because of the lack of formal training provided by universities, both authors searched for other professional communities to help us develop our expertise in giving feedback as peer reviewers, with journal communities being the next microsystem. We found that international journals provide valuable opportunities for us to understand more about the whole peer-review process, in particular the role of feedback. For A1, the training which he received from the editor-in-chief when he took up the associate editorship of a language education journal two years ago was particularly useful. A1 benefited greatly from meetings with the editor who walked him through every stage in the review process and provided ‘hands-on experience on how to handle delicate scenarios’ (A1,E2). Since then, A1 has had plenty of opportunities to oversee various stages of peer review and read a large number of reviewers’ reports which helped him gain ‘a holistic understanding of the peer-review process’ (A1,E2) and gradually made him become more cognizant of how he wants to give feedback. Although there was no explicit instruction on the technical aspect of giving feedback, A1 found that being an associate editor has developed his ‘consciousness’ and ‘awareness’ of giving feedback as a peer reviewer (A1,D2). Further, he felt that his editorial experiences provided him the awareness to constantly refine and improve his ways of giving feedback, especially ways to make his feedback ‘more structured, evidence-based, and objective’ (A1,E2). Despite not reflecting from the perspective of an editor, A2 recalled her experience as an author who received in-depth and constructive feedback from a reviewer, which really impacted the way she viewed the whole review process. She understood from this experience that even though the paper under review may not be particularly strong, peer reviewers should always aim to provide formative feedback which helps the authors to improve their work. These positive experiences of the two authors are impactful on the ways they give feedback as peer reviewers. In addition, close engagement with a specific journal has helped A2 to develop a sense of belonging, making it ‘much more than a journal, but also a way to become part of an academic community’ (A2,E2). With such a sense of belonging, it is more likely for her to be ‘pulled towards that journal than others’ when she can only review a limited number of manuscripts (A2,D2).

Another professional community in which we are both involved is Twitter. We regard Twitter as a platform for self-learning, reflection, and inspiration. We perceive Twitter as a space where we get to learn from others’ peer-review experiences and disciplinary practices. For example, A1 found the tweets on peer-review informative ‘because they are written by different stakeholders in the process—the authors, editors, reviewers’ and offer ‘different perspectives and sometimes different versions of the same story’ (A1,E2). A2 recalled a tweet she came across about the ‘infamous Reviewer 2’ and how she learned to not make the same mistakes (A2,D2). Reading other people’s experiences helps us reconsider our own feedback practices and, more broadly, the whole peer-review system because we ‘get a glimpse of the do’s and don’ts for peer reviewers’ (A1,E2).

Further to our three common microsystems, A2 also draws on a unique microsystem, that of her former profession as a teacher, which shapes her feedback practices in three ways. First, in her four years of teacher training, a lot of emphasis was placed on assessment and feedback such as ‘error correction’; this understanding related to giving feedback to students and was solidified through ‘learning on the job’ (A2,D2). Second, A2 acknowledges that as a teacher, she has a passion to ‘guide others in their knowledge and skill development… and continue this in our review practices’ (A2,E2). Finally, her teaching experience prepared her to consider the authors’ emotional responses in her peer-review feedback practices, constantly ‘thinking there’s a person there who’s going to be shattered getting a rejection’ (A2,D2).

Mesosystem considers the confluence of our interactions in various microsystems. Particularly, we experienced a lack of support from our institutions, which pushed us to seek alternative paths to acquire the art of giving feedback. This has made us realise the importance of self-learning in developing feedback literacy as peer reviewers, especially in how to develop constructive and actionable feedback. Both authors self-learn how to give feedback by reading others’ feedback. A1 felt ‘fortunate to be involved in journal editing and Twitter’ because he gets ‘a glimpse of how other peer reviewers give feedback to authors’ (A1,E2). A2, on the other hand, learned through her correspondences with a journal editor who made her stop ‘looking for every word’ and move away from ‘over proofreading and over editing’ (A2,D2).

Focusing on the chronosystem, it is noticed that both authors adjusted how they give feedback over time because of the aggregated influence of their microsystems. What stands out is that they have become more strategic in giving feedback. One way this is achieved is through focusing their comments on the arguments of the manuscripts instead of burning the midnight oil with error-correcting.

Exosystem concerns the environment where the focal individuals do not have direct interactions with the people in it but have access to information about. In his case, A1’s understanding of advising techniques promoted by a self-access language learning centre is conducive to the cultivation of his feedback literacy. Although A1 is not a part of the language advising team, he has a working relationship with the director. A1 was especially impressed by the learner-centeredness of an advising process:

The primary duty of the language advisor is not to be confused with that of a language teacher. Language teachers may teach a lecture on a linguistic feature or correct errors on an essay, but language advisors focus on designing activities and engaging students in dialogues to help them reflect on their own learning needs… The advisors may also suggest useful resources to the students which cater to their needs. In short, language advisors work in partnership with the students to help them improve their language while language teachers are often perceived as more authoritative figures (A1, E2).

His understanding of advising has affected how A1 provides feedback as a peer reviewer in a number of ways. First, A1 places much more emphasis on humanising his feedback, for example, by considering ‘ways to work in partnership with the authors and making this “partnership mindset” explicit to the authors through writing’ (A1,E2). One way to operationalise this ‘partnership mindset’ in peer review is to ‘ask a lot of questions’ and provide ‘multiple suggestions’ for the authors to choose from (A1,E2). Furthermore, his knowledge of the difference between feedback as giving advice and feedback as instruction has led him to include feedback, which points authors to additional resources. Below is a feedback point A1 gave in one of his reviews:

The description of the data analysis process was very brief. While we are not aiming at validity and reliability in qualitative studies, it is important for qualitative researchers to describe in detail how the data collected were analysed (e.g. iterative coding, inductive/deductive coding, thematic analysis) in order to ascertain that the findings were credible and trustworthy. See Johnny Saldaña’s ‘The Coding Manual for Qualitative Researchers’.

Another exosystem that we have knowledge about is formal peer-review training courses provided by publishers. These online courses are usually run asynchronously. Even though we did not enrol in these courses, our interest in peer review has led us to skim the content of these courses. Both of us questioned the value of formal peer-review training in developing feedback literacy of peer reviewers. For example, A2 felt that opportunities to review are more important because they ‘put you in that position where you have responsibility and have to think critically about how you are going to respond’ (A2,D2). To A1, formal peer-review training mostly focuses on developing peer reviewers’ ‘understanding of the whole mechanism’ but not providing ‘training on how to give feedback… For example, do you always ask a question without giving the answers you know? What is a good suggestion?’ (A1,D2).

Macrosystem

The two authors have diverse sociocultural experiences because of their family backgrounds and work contexts. When reflecting on their sociocultural experiences, A1 focused on his upbringing in Hong Kong where both of his parents are school teachers and his professional experience as a language teacher in secondary and tertiary education in Hong Kong while A2 discussed her experience of working in academia in Japan as an anglophone.

Observing his parents’ interactions with their students in schools, A1 was immersed in an Asian educational discourse characterised by ‘mutual respect and all sorts of formality’ (A1,E2). After he finished university, A1 became a school teacher and then a university lecturer (equivalent to a teaching fellow in the UK), getting immersed continuously in the etiquette of educational discourse in Hong Kong. Because of this, A1 knows that being professional means to be ‘formal and objective’ and there is a constant expectation to ‘treat people with respect’ (A1,E2). At the same time, his parents are unlike typical Asian parents; they are ‘more open-minded’, which made him more willing to listen and ‘consider different perspectives’ (A1,D2). Additionally, social hierarchy also impacted his approach to giving feedback as a peer reviewer. A1 started his career as a school teacher and then a university lecturer in Hong Kong with no formal research training. After obtaining his BA and MA, it is not until recently that A1 obtained his PhD by Prior Publication. Perhaps because of his background as a frontline teacher, A1 did not regard himself as ‘a formally trained researcher’ and perceived himself as not ‘elite enough to give feedback to other researchers’ (A1,E2). Both his childhood and his self-perceived identity have led to the formation of two feedback strategies: asking questions and providing a structured report mimicking the sections in the manuscript. A1 frequently asks questions in his reports ‘in a bid to offset some of the responsibilities to the authors’ (A1,E2). A1 struggles to decide whether to address authors using second- or third-person pronouns. A1 consistently uses third-person pronouns in his feedback because he wants to sound ‘very formal’ (A1,D2). However, A1 shared that he has recently started using second-person pronouns to make his feedback more interactive.

A2, on the other hand, pondered upon her sociocultural experiences as a school teacher in Australia, her position as an anglophone in a Japanese university, and her status as first-generation high school graduate. Reflecting on her career as a school teacher, A2 shared that her students had high expectations on her feedback:

So if you give feedback that seems unfair, you know … they’ll turn around and say, ‘What are you talking about’? They’re going to react back if your feedback is not clear. I think a lot of them [the students] appreciate the honesty. (A2,D2)

A2 acknowledges that her identity as a native English speaker has given her the advantage to publish extensively in international journals because of her high level of English proficiency and her access to ‘data from the US and from Australia which are more marketable’ (A2,D2). At the same time, as a native English speaker, she has empathy for her Japanese colleagues who struggle to write proficiently in English and some who even ‘pay thousands of dollars to have their work translated’ (A2,D2). Therefore, when giving feedback as a peer reviewer, she tries not to make a judgement on an author’s English proficiency and will not reject a paper based on the standard of English alone. Finally, as a first-generation scholar without any previous connections to academia, she struggles with belonging and self-confidence. As a result she notes that it usually takes her a long time to complete a review because she would like to be sure what she is saying is ‘right or constructive and is not on the wrong track’ (A2,D2).

Implications and future directions

In investigating the manifestations of the authors’ feedback literacy development, and the ecological systems in which this development occurs, this study unpacks the various sources of influence behind our feedback behaviours as two relatively new but highly commended peer reviewers. The findings show that our feedback literacy development is highly personalised and contextualised, and the sources of influence are diverse and interconnected, albeit largely informal. Our peer-review practices are influenced by our experiences within academia, but influences are much broader and begin much earlier. Peer-review skills were enhanced through direct experience not only in peer review but also in other activities related to the peer-review process, and as such more hands-on, on-site feedback training for peer reviewers may be more appropriate than knowledge-based training. The authors gain valuable insights from seeing the reviews of others, and as this is often not possible until scholars take on more senior roles within journals, co-reviewing is a potential way for ECRs to gain experience (McDowell et al., 2019 ). We draw practical and moral support from various communities, particularly online to promote “intellectual candour”, which refers to honest expressions of vulnerability for learning and trust building (Molloy and Bearman, 2019 , p. 32); in response to this finding we have developed an online community of practice, specifically as a space for discussing issues related to peer review (a Twitter account called “Scholarly Peers”). Importantly, our review practices are a product not only of how we review, but why we review, and as such training should not focus solely on the mechanics of review, but extend to its role within academia, and its impact not only on the quality of scholarship, but on the growth of researchers.

The significance of this study is its insider perspective, and the multifaceted framework that allows the capturing of the complexity of factors that influence individual feedback literacy development of two recognised peer reviewers. It must be stressed that the findings of this study are highly idiosyncratic, focusing on the experiences of only two peer reviewers and the educational research discipline. While the research design is such that it is not an attempt to describe a ‘typical’ or ‘expected’ experience, the scope of the study is a limitation, and future research could be expanded to studies of larger cohorts in order to identify broader trends. In this study, we have not included the reviewer reports themselves, and these reports provide a potentially rich source of data, which will be a focus in our continued investigation in this area. Further research could also investigate the role that peer-review training courses play in the feedback literacy development and practices of new and experienced peer reviewers. Since journal peer review is a communication process, it is equally important to investigate authors’ perspectives and experiences, especially pertaining to how authors interpret reviewers’ feedback based on the ways that it is written.

Data availability

Because of the sensitive nature of the data these are not made available.

Change history

26 november 2021.

A Correction to this paper has been published: https://doi.org/10.1057/s41599-021-00996-3

Abedi Asante L, Abubakari Z (2020) Pursuing PhD by publication in geography: a collaborative autoethnography of two African doctoral researchers. J Geogr High Educ 45(1):87–107. https://doi.org/10.1080/03098265.2020.1803817

Article   Google Scholar  

Boud D, Dawson P (2021). What feedback literate teachers do: An empirically-derived competency framework. Assess Eval High Educ. Advanced online publication. https://doi.org/10.1080/02602938.2021.1910928

Bronfenbrenner U (1986) Ecology of the family as a context for human development. Res Perspect Dev Psychol 22:723–742. https://doi.org/10.1037/0012-1649.22.6.723

Carless D, Boud D (2018) The development of student feedback literacy: enabling uptake of feedback. Assess Eval High Educ 43(8):1315–1325. https://doi.org/10.1080/02602938.2018.1463354

Carless D, Winstone N (2020) Teacher feedback literacy and its interplay with student feedback literacy. Teach High Educ, 1–14. https://doi.org/10.1080/13562517.2020.1782372

Chang H, Ngunjiri FW, Hernandez KC (2013) Collaborative autoethnography. Left Coast Press

Cheung D (2000) Measuring teachers’ meta-orientations to curriculum: application of hierarchical confirmatory factor analysis. The J Exp Educ 68(2):149–165. https://doi.org/10.1080/00220970009598500

Chong SW (2021a) Improving peer-review by developing peer reviewers’ feedback literacy. Learn Publ 34(3):461–467. https://doi.org/10.1002/leap.1378

Chong SW (2021b) Reconsidering student feedback literacy from an ecological perspective. Assess Eval High Educ 46(1):92–104. https://doi.org/10.1080/02602938.2020.1730765

Chong SW (2019) College students’ perception of e-feedback: a grounded theory perspective. Assess Eval High Educ 44(7):1090–1105. https://doi.org/10.1080/02602938.2019.1572067

Chong SW (2018) Interpersonal aspect of written feedback: a community college students’ perspective. Res Post-Compul Educ 23(4):499–519. https://doi.org/10.1080/13596748.2018.1526906

Corden A, Sainsbury R (2006) Using verbatim quotations in reporting qualitative social research: the views of research users. University of York Social Policy Research Unit

Ellis C, Adams TE, Bochner AP (2011) Autoethnography: An Overview. Historical Soc Res, 12:273–290

Ellis C, Bochner A (1996) Composing ethnography: Alternative forms of qualitative writing. Sage

Freda MC, Kearney MH, Baggs JG, Broome ME, Dougherty M (2009) Peer reviewer training and editor support: results from an international survey of nursing peer reviewers. J Profession Nurs 25(2):101–108. https://doi.org/10.1016/j.profnurs.2008.08.007

Fulcher G (2012) Assessment literacy for the language classroom. Lang Assess Quart 9(2):113–132. https://doi.org/10.1080/15434303.2011.642041

Gee JP (1999) Reading and the new literacy studies: reframing the national academy of sciences report on reading. J Liter Res 3(3):355–374. https://doi.org/10.1080/10862969909548052

Gravett K, Kinchin IM, Winstone NE, Balloo K, Heron M, Hosein A, Lygo-Baker S, Medland E (2019) The development of academics’ feedback literacy: experiences of learning from critical feedback via scholarly peer review. Assess Eval High Educ 45(5):651–665. https://doi.org/10.1080/02602938.2019.1686749

Hains-Wesson R, Young K (2016) A collaborative autoethnography study to inform the teaching of reflective practice in STEM. High Educ Res Dev 36(2):297–310. https://doi.org/10.1080/07294360.2016.1196653

Han Y, Xu Y (2019) Student feedback literacy and engagement with feedback: a case study of Chinese undergraduate students. Teach High Educ, https://doi.org/10.1080/13562517.2019.1648410

Heesen R, Bright LK (2020) Is Peer Review a Good Idea? Br J Philos Sci, https://doi.org/10.1093/bjps/axz029

Hollywood A, McCarthy D, Spencely C, Winstone N (2019) ‘Overwhelmed at first’: the experience of career development in early career academics. J Furth High Educ 44(7):998–1012. https://doi.org/10.1080/0309877X.2019.1636213

Horn SA (2016) The social and psychological costs of peer review: stress and coping with manuscript rejection. J Manage Inquiry 25(1):11–26. https://doi.org/10.1177/1056492615586597

Hughes S, Pennington JL, Makris S (2012) Translating Autoethnography Across the AERA Standards: Toward Understanding Autoethnographic Scholarship as Empirical Research. Educ Researcher, 41(6):209–219

Kandiko CB(2010) Neoliberalism in higher education: a comparative approach. Int J Art Sci 3(14):153–175. http://www.openaccesslibrary.org/images/BGS220_Camille_B._Kandiko.pdf

Keashly L, Neuman JH (2010) Faculty experiences with bullying in higher education-causes, consequences, and management. Adm Theory Prax 32(1):48–70. https://doi.org/10.2753/ATP1084-1806320103

Kelly J, Sadegieh T, Adeli K (2014) Peer review in scientific publications: benefits, critiques, & a survival guide. J Int Fed Clin Chem Labor Med 25(3):227–243. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4975196/

Google Scholar  

Kumar KL (2020) Understanding and expressing academic identity through systematic autoethnography. High Educ Res Dev, https://doi.org/10.1080/07294360.2020.1799950

Lapadat JC (2017) Ethics in autoethnography and collaborative autoethnography. Qual Inquiry 23(8):589–603. https://doi.org/10.1177/1077800417704462

Levi T, Inbar-Lourie O (2019) Assessment literacy or language assessment literacy: learning from the teachers. Lang Assess Quarter 17(2):168–182. https://doi.org/10.1080/15434303.2019.1692347

London MS, Smither JW (2002) Feedback orientation, feedback culture, and the longitudinal performance management process. Hum Res Manage Rev 12(1):81–100. https://doi.org/10.1016/S1053-4822(01)00043-2

Malecka B, Boud D, Carless D (2020) Eliciting, processing and enacting feedback: mechanisms for embedding student feedback literacy within the curriculum. Teach High Educ, 1–15. https://doi.org/10.1080/13562517.2020.1754784

Mavrogenis AF, Quaile A, Scarlat MM (2020) The good, the bad and the rude peer-review. Int Orthopaed 44(3):413–415. https://doi.org/10.1007/s00264-020-04504-1

McDowell GS, Knutsen JD, Graham JM, Oelker SK, Lijek RS (2019) Co-reviewing and ghostwriting by early-career researchers in the peer review of manuscripts. ELife 8:e48425. https://doi.org/10.7554/eLife.48425

Article   CAS   PubMed   PubMed Central   Google Scholar  

Merga MK, Mason S, Morris JE (2018) Early career experiences of navigating journal article publication: lessons learned using an autoethnographic approach. Learn Publ 31(4):381–389. https://doi.org/10.1002/leap.1192

Miles MB, Huberman AM (1994) Qualitative data analysis: An expanded sourcebook (2nd edn.). Sage

Molloy E, Bearman M (2019) Embracing the tension between vulnerability and credibility: ‘Intellectual candour’ in health professions education. Med Educ 53(1):32–41. https://doi.org/10.1111/medu.13649

Article   PubMed   Google Scholar  

Molloy E, Boud D, Henderson M (2019) Developing a learning-centred framework for feedback literacy. Assess Eval High Educ 45(4):527–540. https://doi.org/10.1080/02602938.2019.1667955

Neal JW, Neal ZP (2013) Nested or networked? Future directions for ecological systems theory. Soc Dev 22(4):722–737. https://doi.org/10.1111/sode.12018

Noble C, Billett S, Armit L, Collier L, Hilder J, Sly C, Molloy E (2020) “It’s yours to take”: generating learner feedback literacy in the workplace. Adv Health Sci Educ Theory Pract 25(1):55–74. https://doi.org/10.1007/s10459-019-09905-5

Price M, Rust C, O’Donovan B, Handley K, Bryant R (2012) Assessment literacy: the foundation for improving student learning. Oxford Centre for Staff and Learning Development

Silbiger NJ, Stubler AD (2019) Unprofessional peer reviews disproportionately harm underrepresented groups in STEM. PeerJ 7:e8247. https://doi.org/10.7717/peerj.8247

Article   PubMed   PubMed Central   Google Scholar  

Starck JM (2017) Scientific peer review: guidelines for informative peer review. Springer Spektrum

Steelman LA, Wolfeld L (2016) The manager as coach: the role of feedback orientation. J Busi Psychol 33(1):41–53. https://doi.org/10.1007/s10869-016-9473-6

Stiggins RJ (1999) Evaluating classroom assessment training in teacher education programs. Educ Meas: Issue Pract 18(1):23–27. https://doi.org/10.1111/j.1745-3992.1999.tb00004.x

Street B (1997) The implications of the ‘new literacy studies’ for literacy Education. Engl Educ 31(3):45–59. https://doi.org/10.1111/j.1754-8845.1997.tb00133.x

Sughrua WM (2019) A nomenclature for critical autoethnography in the arena of disciplinary atomization. Cult Stud Crit Methodol 19(6):429–465. https://doi.org/10.1177/1532708619863459

Sutton P (2012) Conceptualizing feedback literacy: knowing, being, and acting. Innov Educ Teach Int 49(1):31–40. https://doi.org/10.1080/14703297.2012.647781

Article   MathSciNet   Google Scholar  

Tynan BR, Garbett DL (2007) Negotiating the university research culture: collaborative voices of new academics. High Educ Res Dev 26(4):411–424. https://doi.org/10.1080/07294360701658617

Vygotsky LS (1978) Mind in society: The development of higher psychological processes. Harvard University Press

Wall S (2006) An autoethnography on learning about autoethnography. Int J Qual Methods 5(2):146–160. https://doi.org/10.1177/160940690600500205

Article   ADS   MathSciNet   Google Scholar  

Warne V (2016) Rewarding reviewers-sense or sensibility? A Wiley study explained. Learn Publ 29:41–40. https://doi.org/10.1002/leap.1002

Wilkinson S (2019) The story of Samantha: the teaching performances and inauthenticities of an early career human geography lecturer. High Educ Res Dev 38(2):398–410. https://doi.org/10.1080/07294360.2018.1517731

Winstone N, Carless D (2019) Designing effective feedback processes in higher education: a learning-focused approach. Routledge

Winstone NE, Mathlin G, Nash RA (2019) Building feedback literacy: students’ perceptions of the developing engagement with feedback toolkit. Front Educ 4:1–11. https://doi.org/10.3389/feduc.2019.00039

Xu Y, Carless D (2016) ‘Only true friends could be cruelly honest’: cognitive scaffolding and social-affective support in teacher feedback literacy. Assess Eval High Educ 42(7):1082–1094. https://doi.org/10.1080/02602938.2016.1226759

Download references

Author information

Authors and affiliations.

Queen’s University Belfast, Belfast, UK

Sin Wang Chong

Nagasaki University, Nagasaki, Japan

Shannon Mason

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Sin Wang Chong .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Ethical approval

We acknowledge that research data are collected from human subjects (from the two authors), such data are collected in accordance with the standards and guidelines of the School Research Ethics Committee at the School of Social Sciences, Education and Social Work, Queen’s University Belfast (Ref: 005_2021).

Informed consent

Since the participants are the two authors, there is no informed consent form.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplemental material file #1, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Chong, S.W., Mason, S. Demystifying the process of scholarly peer-review: an autoethnographic investigation of feedback literacy of two award-winning peer reviewers. Humanit Soc Sci Commun 8 , 266 (2021). https://doi.org/10.1057/s41599-021-00951-2

Download citation

Received : 02 August 2021

Accepted : 12 October 2021

Published : 12 November 2021

DOI : https://doi.org/10.1057/s41599-021-00951-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

• Science Notes Posts
• Contact Science Notes
• Todd Helmenstine Biography
• Anne Helmenstine Biography
• Free Printable Periodic Tables (PDF and PNG)
• Periodic Table Wallpapers
• Interactive Periodic Table
• Periodic Table Posters
• How to Grow Crystals
• Chemistry Projects
• Fire and Flames Projects
• Holiday Science
• Chemistry Problems With Answers
• Physics Problems
• Unit Conversion Example Problems
• Chemistry Worksheets
• Biology Worksheets
• Periodic Table Worksheets
• Physical Science Worksheets
• Science Lab Worksheets
• My Amazon Books

Understanding Peer Review in Science

Peer review is an essential element of the scientific publishing process that helps ensure that research articles are evaluated, critiqued, and improved before release into the academic community. Take a look at the significance of peer review in scientific publications, the typical steps of the process, and and how to approach peer review if you are asked to assess a manuscript.

What Is Peer Review?

Peer review is the evaluation of work by peers, who are people with comparable experience and competency. Peers assess each others’ work in educational settings, in professional settings, and in the publishing world. The goal of peer review is improving quality, defining and maintaining standards, and helping people learn from one another.

In the context of scientific publication, peer review helps editors determine which submissions merit publication and improves the quality of manuscripts prior to their final release.

Types of Peer Review for Manuscripts

There are three main types of peer review:

• Single-blind review: The reviewers know the identities of the authors, but the authors do not know the identities of the reviewers.
• Double-blind review: Both the authors and reviewers remain anonymous to each other.
• Open peer review: The identities of both the authors and reviewers are disclosed, promoting transparency and collaboration.

There are advantages and disadvantages of each method. Anonymous reviews reduce bias but reduce collaboration, while open reviews are more transparent, but increase bias.

Key Elements of Peer Review

Proper selection of a peer group improves the outcome of the process:

• Expertise : Reviewers should possess adequate knowledge and experience in the relevant field to provide constructive feedback.
• Objectivity : Reviewers assess the manuscript impartially and without personal bias.
• Confidentiality : The peer review process maintains confidentiality to protect intellectual property and encourage honest feedback.
• Timeliness : Reviewers provide feedback within a reasonable timeframe to ensure timely publication.

Steps of the Peer Review Process

The typical peer review process for scientific publications involves the following steps:

• Submission : Authors submit their manuscript to a journal that aligns with their research topic.
• Editorial assessment : The journal editor examines the manuscript and determines whether or not it is suitable for publication. If it is not, the manuscript is rejected.
• Peer review : If it is suitable, the editor sends the article to peer reviewers who are experts in the relevant field.
• Reviewer feedback : Reviewers provide feedback, critique, and suggestions for improvement.
• Revision and resubmission : Authors address the feedback and make necessary revisions before resubmitting the manuscript.
• Final decision : The editor makes a final decision on whether to accept or reject the manuscript based on the revised version and reviewer comments.
• Publication : If accepted, the manuscript undergoes copyediting and formatting before being published in the journal.

Pros and Cons

While the goal of peer review is improving the quality of published research, the process isn’t without its drawbacks.

• Quality assurance : Peer review helps ensure the quality and reliability of published research.
• Error detection : The process identifies errors and flaws that the authors may have overlooked.
• Credibility : The scientific community generally considers peer-reviewed articles to be more credible.
• Professional development : Reviewers can learn from the work of others and enhance their own knowledge and understanding.
• Time-consuming : The peer review process can be lengthy, delaying the publication of potentially valuable research.
• Bias : Personal biases of reviews impact their evaluation of the manuscript.
• Inconsistency : Different reviewers may provide conflicting feedback, making it challenging for authors to address all concerns.
• Limited effectiveness : Peer review does not always detect significant errors or misconduct.
• Poaching : Some reviewers take an idea from a submission and gain publication before the authors of the original research.

Steps for Conducting Peer Review of an Article

Generally, an editor provides guidance when you are asked to provide peer review of a manuscript. Here are typical steps of the process.

• Accept the right assignment: Accept invitations to review articles that align with your area of expertise to ensure you can provide well-informed feedback.
• Manage your time: Allocate sufficient time to thoroughly read and evaluate the manuscript, while adhering to the journal’s deadline for providing feedback.
• Read the manuscript multiple times: First, read the manuscript for an overall understanding of the research. Then, read it more closely to assess the details, methodology, results, and conclusions.
• Evaluate the structure and organization: Check if the manuscript follows the journal’s guidelines and is structured logically, with clear headings, subheadings, and a coherent flow of information.
• Assess the quality of the research: Evaluate the research question, study design, methodology, data collection, analysis, and interpretation. Consider whether the methods are appropriate, the results are valid, and the conclusions are supported by the data.
• Examine the originality and relevance: Determine if the research offers new insights, builds on existing knowledge, and is relevant to the field.
• Check for clarity and consistency: Review the manuscript for clarity of writing, consistent terminology, and proper formatting of figures, tables, and references.
• Identify ethical issues: Look for potential ethical concerns, such as plagiarism, data fabrication, or conflicts of interest.
• Provide constructive feedback: Offer specific, actionable, and objective suggestions for improvement, highlighting both the strengths and weaknesses of the manuscript. Don’t be mean.
• Organize your review: Structure your review with an overview of your evaluation, followed by detailed comments and suggestions organized by section (e.g., introduction, methods, results, discussion, and conclusion).
• Be professional and respectful: Maintain a respectful tone in your feedback, avoiding personal criticism or derogatory language.
• Proofread your review: Before submitting your review, proofread it for typos, grammar, and clarity.
• Couzin-Frankel J (September 2013). “Biomedical publishing. Secretive and subjective, peer review proves resistant to study”. Science . 341 (6152): 1331. doi: 10.1126/science.341.6152.1331
• Lee, Carole J.; Sugimoto, Cassidy R.; Zhang, Guo; Cronin, Blaise (2013). “Bias in peer review”. Journal of the American Society for Information Science and Technology. 64 (1): 2–17. doi: 10.1002/asi.22784
• Slavov, Nikolai (2015). “Making the most of peer review”. eLife . 4: e12708. doi: 10.7554/eLife.12708
• Spier, Ray (2002). “The history of the peer-review process”. Trends in Biotechnology . 20 (8): 357–8. doi: 10.1016/S0167-7799(02)01985-6
• Squazzoni, Flaminio; Brezis, Elise; Marušić, Ana (2017). “Scientometrics of peer review”. Scientometrics . 113 (1): 501–502. doi: 10.1007/s11192-017-2518-4

Related Posts

The Literature Review Step by Step: Overview

Searching tips.

• Use Keywords
• Brainstorm Synonyms
• Use Search Tools
• Evaluating Resources using ESCAPE
• Peer-Review: What is it and How to Tell

Example Topic:  How do stereotypes affect female politicians during elections?

Keywords:  stereotypes, female politicians, elections

Most words have synonyms that mean the same, or very similar, things. For each keyword in your topic, try to come up with at least one synonym. Not all keywords will have synonyms, but many do!

Example: 

Keep an Eye Out

Sometimes scholars use terms that you might not be familiar with, or which might mean something very specific within the discipline. While searching, look for unfamiliar terms or words that show up a lot. Try searching for those and see if you find more relevant sources.

Most library databases have search tools built in. Try some of these:

• Date Range:  Limit your search to sources published between specific years.
• Peer Reviewed:  Limit your search to scholarly journal articles.
• Full Text:  Make sure all of the results are available to read in full.

Or look for an "advanced search" page to find these tools - and more!

You can evaluate any source using the ESCAPE plan.

Find out more about ESCAPE in our Evaluating Information Tutorial .

• Peer Review Process (PDF)
• Peer Review Clues (PDF)

Advanced Search Tips

• Too Many Results?
• Too Few Results?
• Wrong Results?

Use the operator AND to find only sources that mention both keywords.

This search will bring back fewer results than searching either keyword on its own.

Use the OR operator to expand your search with additional keywords.

This will find sources that include either word, so you'll see more results than by searching for just one keyword.

Use the “QUOTES” strategy to search for several words in a phrase.

This will bring back results that only use that exact phrase.

Search the Library Databases

Credo Reference provides full-text online versions of over 3,500 published reference works from more than 100 publishers in a variety of major subjects.

• Next: Step 1 >>
• Last Updated: Apr 25, 2024 10:50 AM
• URL: https://libguides.brenau.edu/c.php?g=1375806

Tutorial: Evaluating Information: Peer Review

• Evaluating Information
• Scholarly Literature Types
• Primary vs. Secondary Articles
• Peer Review
• Systematic Reviews & Meta-Analysis
• Gray Literature
• Evaluating Like a Boss
• Evaluating AV

Types of scholarly literature

You will encounter many types of articles and it is important to distinguish between these different categories of scholarly literature. Keep in mind the following definitions.

Peer-reviewed (or refereed):  Refers to articles that have undergone a rigorous review process, often including revisions to the original manuscript, by peers in their discipline, before publication in a scholarly journal. This can include empirical studies, review articles, meta-analyses among others.

Empirical study (or primary article): An empirical study is one that aims to gain new knowledge on a topic through direct or indirect observation and research. These include quantitative or qualitative data and analysis. In science, an empirical article will often include the following sections: Introduction, Methods, Results, and Discussion.

Review article:  In the scientific literature, this is a type of article that provides a synthesis of existing research on a particular topic. These are useful when you want to get an idea of a body of research that you are not yet familiar with. It differs from a systematic review in that it does not aim to capture ALL of the research on a particular topic.

Systematic review:  This is a methodical and thorough literature review focused on a particular research question. It's aim is to identify and synthesize all of the scholarly research on a particular topic in an unbiased, reproducible way to provide evidence for practice and policy-making. It may involve a meta-analysis (see below). 

Meta-analysis:  This is a type of research study that combines or contrasts data from different independent studies in a new analysis in order to strengthen the understanding of a particular topic. There are many methods, some complex, applied to performing this type of analysis.

Peer Reviewed Article vs. Review Article

TIP :  Review articles and Peer-reviewed articles are not the same thing!  Review articles synthesize and analyze the results of multiple studies on a topic; peer-reviewed articles are articles of any kind that have been vetted for quality by an expert or number of experts in the field. The bibliographies of review articles can be a great source of original, peer-reviewed empirical articles.

Peer Review in Three Minutes

Watch this 3 minute intro to peer review video by North Carolina State University or read this short introduction by the University of Texas at Austin Library.

Is It Peer-Reviewed & How Can I Tell?

There are a couple of ways you can tell if a journal is peer-reviewed:

• If it's online, go to the journal home page and check under About This Journal. Often the brief description of the journal will note that it is peer-reviewed or refereed or will list the Editors or Editorial Board.
• Go to the database Ulrich's and do a Title Keyword search for the journal. If it is peer-reviewed or refereed, the title will have a little umpire shirt symbol by it.
• BE CAREFUL! A journal can be refereed/peer-reviewed and still have non-peer reviewed articles in it. Generally if the article is an editorial, brief news item or short communication, it's not been through the full peer-review process. Databases like Web of Knowledge will let you restrict your search only to articles (and not editorials, conference proceedings, etc).

Checking Peer Review in Ulrich's

Using Ulrich's Periodicals Directory to Verify Peer Review of Your Journal Title

To use this, click on the Ulrich's link to enter the database (or search for it on the main library website).

Change the Quick Search dropdown menu to Title (Keyword) and type your  full   journal title  (not your article title or keywords) into the Quick Search box, then click Search. 

This will give you a list of journal titles which includes the title you typed in. Check the Legend in the upper right corner to view the Refereed symbol ("refereed" is another term for peer-reviewed.) Then check your journal title to make sure it has the refereed symbol next to it.

NOTE: Though a journal can be peer-reviewed, letters to the editor and news reports in those same scientific journals are not! Make sure your article is a primary research article.

• Ulrich's Periodicals Directory Contains information on currently published as well as discontinued periodicals. Includes magazines, journals, newsletters, newspapers, conference proceedings, and electronic publications, together with search and browse indexes. It also contains complete names and addresses of journal publishers.

Peer reviewed articles

They both come out once every month. They're both in English. Both published in the United States. Both of them are "factual".They both have pictures. They even cover some of the same topics.

The difference is that one--Journal of Geography--is peer-reviewed, whereas  National Geographic  is a popular-press title. 

Peer review  is scientists' and other scholars' best effort to publish accurate information. Each article has been submitted by a researcher, and then  reviewed by other scholars in the same field to ensure that it is sound science.  What they are looking for is that:

• The methodology has been fully described (and the study can thus be replicated by another researcher)
• There are no obvious errors of calculation or statistical analysis
• Crucially: The findings support the conclusions. That is, do the results of the research support what the researcher has said about them? The classic problematic example is a scientist claiming that hair growth causes time to pass: The correlation is clearly not causation.

Things to know about peer review:

• It isn't a perfect system:  Scientists make errors (or commit fraud) as often as any other human being and sometimes bad articles slip through. But in general, peer-review ensures that many trained eyes have seen an article before it appears in print.
• Peer-reviewed journals are generally considered "primary source" material:  When a new scientific discovery is made, a peer-reviewed journal is often--but not always--the first place it appears.
• Popular and trade publications  are not peer-reviewed, they are simply edited. That does not mean they are any less potentially truthful or informative--most popular and trade publications take pride in careful fact-checking.* But when the topic is scientific research, the information is generally  "secondary" : It has already appeared elsewhere (usually in a peer-reviewed journal) and has now been "digested" for a broader audience.
• Peer-reviewed journals will always identify themselves as such. If you want to verify that a journal is peer-reviewed, check Ulrich's Periodical Directory .

Some sources of peer-reviewed articles:

• Cornell University Library homepage In particular, check the Articles & Full-Text search and then choose "Limit to articles from scholarly publications, including peer-review" at the top-left.
• Google Scholar Google Scholar takes Google's PageRank algorithm and runs it on a pre-selected set of tables-of-contents and metadata from a preselected set of scholarly journals and papers. These are largely--though not entirely--peer-reviewed. It is a much better option than a regular Google search for scholarly information.
• Web of Science Choosing "All Databases" allows you to search an index of journal articles, conference proceedings, data sets, and other resources in the sciences, social sciences, arts, and humanities.
• << Previous: Primary vs. Secondary Articles
• Next: Systematic Reviews & Meta-Analysis >>
• Last Updated: Oct 20, 2021 11:11 AM
• URL: https://guides.library.cornell.edu/evaluate

Understanding peer review

The peer review process.

Peer review is a formal quality control process completed before an  academic work  is published. Not all academic literature is peer reviewed, but many academic journal articles and books will be. Peer-reviewed literature is sometimes also called “refereed literature”.

“Peer assessment”, where peers and colleagues give general feedback on your work, is sometimes also called peer review. However, “peer review” in published research refers specifically to the process described below.

Researchers who are experts in the same field review the submitted work to see if the research and arguments are sound, original, and of high enough quality to be published. Peer reviewers will provide feedback for the author to incorporate before the article is published, or they may advise that the work is not published at all. 

If the peer review process is conducted to a high standard by relevant, respected experts, it can improve the quality of published academic literature and encourage original, high-quality research. However, peer review takes time, which affects how quickly an article can be published. The quality of peer review processes can also vary across publishers and reviewers.           

Find peer-reviewed literature

In most cases, you can identify whether an academic work is peer reviewed by looking at the publisher’s website or using a “peer reviewed” filter when searching a database. 

Academic journals

Some journals require all research articles to be peer reviewed. You can look up the journal’s website to find out their peer review process. Be aware that other content in these journals, like reviews and editorial pieces, may not be peer reviewed. 

Another way to check if a journal has a peer review process is to look it up in a journal directory.

• Search a journal’s International Standard Serial Number (ISSN) or title ( not the article title) and see if it is listed with this black ”refereed” icon.  
• Click the journal title for more details and a link to the journal website.

Directory of Open Access Journals (DOAJ)

• Search for open access journals. Click the journal title in the search results to check for links to their peer review process.

Some academic journal databases will include an option to only search peer-reviewed journals. This option will appear either in their “advanced search” function or as a filter for their results page once you’ve searched. The  Library catalogue also has a “peer-reviewed journals” filter on its search results page.

Academic books 

Check the book publisher’s website for their review processes. University presses are highly likely to publish peer-reviewed academic books.

If the book is open access, check to see if it’s listed in the Directory of Open Access Books (DOAB) , which only includes scholarly, peer-reviewed books. 

Academic books are less likely than academic journals to be clearly identified as peer reviewed in databases. Searching an ebook collection that focuses on research will increase your chances of finding peer-reviewed books.

You can also search for a book’s title in the Library catalogue or an academic journal database to see if other researchers have reviewed it and commented on the book’s peer review process. 

Related information

We're here to help, online or in person.

• Research Guides

Literature Review: A Self-Guided Tutorial

Peer review.

• Literature Reviews: A Recap
• Reading the Literature
• Using Concept Maps
• Developing Research Questions
• Considering Strong Opinions
• 2. Review discipline styles
• Super Searching
• Finding the Full Text
• Citation Searching This link opens in a new window
• When to stop searching
• Citation Management
• Annotating Articles Tip
• 5. Critically analyze and evaluate
• How to Review the Literature
• Using a Synthesis Matrix
• 7. Write literature review

As part of the scholarly publishing process, authors' manuscripts often go through peer review before they are published. Watch the video below to learn about the peer review process. As you watch the video, consider these questions:

• What is the purpose of peer review?
• What are the advantages of this process?
• What are the limitations?

Credit: CSU Dominguez Hills ( CC BY-NC 4.0 ) Run Time: 2:58

• << Previous: Literature Reviews: A Recap
• Next: Reading the Literature >>
• Last Updated: Feb 22, 2024 10:53 AM
• URL: https://libguides.williams.edu/literature-review

Main Navigation Menu

Peer-review and primary research, what is peer-reviewed literature, identifying peer-reviewed journal articles, related research guides.

• What is a Primary Study
• Finding Peer-Reviewed Journal Articles
• Finding Randomized Controlled Trials (RCTs)
• Evaluating Scholarly Articles
• Google Scholar
• Tips for Reading Journal Articles

STEM Librarian

Peer-reviewed journals are also called “refereed” or “juried” journals. They are sometimes called "scholarly" or "academic" journals. The peer review process means that a manuscript is reviewed by others in the same field. These individuals (peers) read and review the manuscript, offering their comments and judgment as to its value. The process is intended to enhance the quality of the publications.

Note: You might find different terminology used to refer to peer-reviewed articles. A professor might ask you to find primary studies, primary research, scholarly articles or peer-reviewed articles. These are all generally referred as the same thing, however, if you need further help ask your professor or a librarian!

Below is an example of a peer-reviewed journal for Wildlife Management:

Characteristics of Peer Reviewed Journals

• Journal articles are written by experts in the field. 
• Journal articles are often intended for a person with knowledge in a specific discipline: a medical journal is written for doctors, a legal journal for attorneys, etc.
• The author of a journal article is always listed—usually, along with his or her qualifications or brief information about the author.
• Journal articles include a list of references. This allows you to see what the sources are and to check them if you wish, providing you with other possible resources.
• Scholarly journals are often published by a professional organization or society.
• Often, the word “journal” appears in the title. However, this is not always a good clue: Ladies Home Journal, for instance, is a popular magazine.
• Often, a journal article is preceded by an abstract, or summary of the content.
• Journals do not include advertisements.
• Titles of articles in journals are very revealing of content, not just clever or catchy, as is often the case with popular magazines.
• Scholarly journal articles often report on research; they may include theoretical assumptions, methodology, hypotheses, results, and conclusions. 

Example of a Peer-Reviewed Journal Article

Arizona State University. (n.d.). Anatomy of an Article. Retrieved from https://askabiologist.asu.edu/explore/anatomy-of-an-article

• Writing a Literature Review by Sara Evans Last Updated Mar 13, 2024 1321 views this year
• Next: What is a Primary Study >>
• Last Updated: Feb 15, 2024 2:45 PM
• URL: https://guides.library.ucmo.edu/peerreview

Kinesiology: Peer Review vs Literature Review

• Peer Review vs Literature Review

What is Peer Review?

Peer Review is a critical part of evaluating information. It is a process that journals use to ensure the articles they publish represent the best scholarship currently available, and articles from peer reviewed journal are often grounded in empirical research. When an article is submitted to a peer reviewed journal, the editors send it out to other scholars in the same field (the author's peers) to get their opinion on the quality of the scholarship, its relevance to the field, its appropriateness for the journal, etc. Sometimes, you'll see this referred to as "refereed." 

Publications that don't use peer review (Time, Cosmo, Salon) rely on an editor to determine the value of an article. Their goal is mainly to educate or entertain the general public, not to support scholarly research.

Some of the library’s databases will allow you to limit your search to scholarly journals. When you enter your search term(s), you can check the  Peer Reviewed  boxes and run your search with confidence that results will be from peer reviewed journals.  The image below is from an EBSCO database, CINAHL. 

KEEP IN MIND THAT PEER REVIEWED JOURNALS ALSO CONTAIN INFORMATION THAT IS NOT PEER REVIEWED  -- book reviews, letters to the editor, short commentary and editorial commentary are not peer reviewed. 

You can access  Ulrich's Web  to see if a particular journal is peer reviewed. (highlighted stars mean that those titles are peer reviewed).

If you are unsure, please reach out to your Librarian and we will assist you. 

Search Tips

Interfaces and search options vary across databases, but best practices for searching are relatively consistent across interfaces. No matter which database you choose, remember these important tips...

1) Don't search wth your topic as a single phrase!  Determine the key concepts of your topic. Then place each concept in its own search bar.  For example...

2) Use ORs  to string together synonyms or related terms for core concepts...

3) Use truncation,  when appropriate. Adding an * to the end of a word will catch all forms of that word. For example,  teach*  will return  teach, teachers, teaching , etc.

4) Look for the "peer reviewed" limit  in each database. You can set this limit on the main search screen (before you search) or narrow your results after you've started your search. Typically, this option is available in the Advanced Search settings.

5) Set Date and Full Text limits  as appropriate for your topic..

APA Resources

Basics of Seventh Edition APA Style Free Tutorial from the American Psychological Association

Purdue OWL APA Formatting and Style

• Student Paper Checklist (Concise Guide Version) Use this checklist while writing your paper to make sure it is consistent with seventh edition APA Style. This checklist corresponds to the writing and formatting guidelines described in full in the Concise Guide to APA Style (7th ed.).

What is a Lit Review?

Think of scholarly papers like a conversation. A paper takes a look at what people are saying on a particular topic and then adds something new to the conversation based on their own research. A literature review is how scholars get caught up on the conversation so they will know what to say or ask next.

A literature review can be just a simple summary of the sources, but it usually has an organizational pattern and combines both summary and synthesis.

A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information. It might give a new interpretation of old material or combine new with old interpretations. Or it might trace the intellectual progression of the field, including major debates.

Literature review

A literature review is a critical summary of what the scientific literature says about your specific topic or question. Often student research in APA fields falls into this category. Your professor might ask you to write this kind of paper to demonstrate your familiarity with work in the field pertinent to the research you hope to conduct. 

While the APA Publication Manual does not require a specific order for a literature review, a good literature review typically contains the following components:

• Introduction
• Thesis statement
• Summary and synthesis of sources
• List of references

Some instructors may also want you to write an abstract for a literature review, so be sure to check with them when given an assignment. Also, the length of a literature review and the required number of sources will vary based on course and instructor preferences.

NOTE:  A literature review and an annotated bibliography are  not  synonymous. While both types of writing involve examining sources, the literature review seeks to synthesize the information and draw connections between sources. If you are asked to write an annotated bibliography, you should consult the  Publication Manual of the American Psychological Association  for the APA Format for Annotated Bibliographies.

• << Previous: Home
• Last Updated: Apr 26, 2024 11:11 PM
• URL: https://librarysubjectguides.svsu.edu/Kinesiology

About Systematic Reviews

A Peer Review vs a Systematic Review

Automate every stage of your literature review to produce evidence-based research faster and more accurately.

Learn More In a sea of existing scientific literature, the rate at which new articles and reviews are being published is skyrocketing. So how does one know with all this available literature which studies are credible and relevant? This is where the different review types come into play. Each of the review types addresses scientific questions in its own unique manner. With evidence-based medicine gaining popularity, there is a need to produce high-quality scientific articles to guide clinical practice. This has led to the emergence of evidence synthesis which attempts to identify, collect and analyze results from multiple sources. Evidence synthesis is described as the interpretation of information contained in individual studies within the scope of the research topic. Systematic reviews are the most effective forms used to conduct evidence synthesis since they use rigorous, methodical, and reproducible processes. But there is a difference between evidence synthesis and a systematic review , so be sure you have a clear understanding of them. Evidence synthesis methods also comprise other types of reviews such as scoping reviews, narrative reviews, meta-analyses, and rapid reviews. If you’d like to know more about the differences between a rapid review vs systematic review , you can read on at the link.

In this article, we will look at what a systematic review is , and how it is different from a peer review.

What Is A Systematic Review?

A systematic review uses explicit methods to identify, select, and critically appraise relevant primary research. It attempts to review all the available evidence to answer a clearly formulated research question. Systematic reviews use inclusion or eligibility criteria to filter out studies irrelevant to the search topic. They are hallmarks of the evidence synthesis process in scientific research since they use all eligible existing research on a topic. Unlike research papers like RCTs that report primary data, systematic reviews report on the findings by combing primary information extracted from eligible studies. This makes systematic reviews a secondary report of data. The methodology involved is rigorous, transparent, and reproducible, this makes systematic reviews a high-quality source of information.

The Systematic review methodology involves the following steps.

• Formulating and stating a clear research question to answer (PICO approach)
• Developing a protocol (with strict inclusion and exclusion criteria for the selection of primary studies)
• Performing a detailed and broad literature search
• Critical appraisal of the selected studies
• Data extraction from the primary studies included in the review
• Data synthesis and analysis using qualitative or quantitative methods [1].

Learn More About DistillerSR

(Article continues below)

What Is A Peer Review?

Peer review is a system to assess the quality of a manuscript submitted by the author before it is published. Independent researchers in the same discipline assess the manuscripts for originality, quality, validity, and significance to help editors determine if they can be published in their journal. Based on the quality score awarded by the reviewers, the manuscript may be forwarded to a more fitting journal. For example, high-quality manuscripts are forwarded to high-quality journals, this leads to categories called journal classes. These independent reviewers may or may not have competencies similar to the authors of the manuscript.

How Does A Peer Review Work?

When a manuscript or an article is submitted to a journal, it is assessed by the editor to see if it meets the criteria for submission. Once it is confirmed that it satisfies the criteria, the editorial team then selects potential peer reviewers within the same field to evaluate the manuscript.

Single-blind, double-blind, or open reviewing, are techniques used to assess the originality, quality, validity, and significance of the manuscript. A variation from these standard approaches is the transparent, collaborative, and post-publication peer reviews.

Different journals use any of these methods to evaluate the quality of the content they publish. You can find out which peer-review system is used by a particular journal on their “about” page.

Why Do We Need A Peer Review?

Peer reviews are designed to assess the validity, quality, and originality of scientific articles that are published. This helps in maintaining the integrity of scientific literature by filtering out poor articles. Peer reviewers are experts in a certain field who volunteer their time to help improve the quality of manuscripts that are published. They help point out gaps in articles that may require further explanation and suggest changes to make a paper easier to read, and more useful in the field.

Researchers mustn’t confuse these two types of reviews. A systematic review involves reviewing all the available eligible literature to find credible, and reliable evidence to answer a specific research question. A peer review, on the other hand, is done by other authors in the same discipline and evaluates the quality, validity, and originality of a single article before it is published. The process of peer review establishes confidence in the articles that are published in a journal. Systematic reviews then search for and collect studies that have been peer-reviewed, to synthesize evidence found in them. Systematic reviews, after their completion by the author, are also peer-reviewed before publication.

3 Reasons to Connect

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

• An introductory paragraph that explains what your working topic and thesis is
• A forecast of key topics or texts that will appear in the review
• Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
• Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically Evaluate: Mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

• Summarize the key findings you have taken from the literature and emphasize their significance
• Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

• Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
• Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
• Qualitative versus quantitative research
• Empirical versus theoretical scholarship
• Divide the research by sociological, historical, or cultural sources
• Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

• It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
• Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
• Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
• Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

• Interlibrary Loan and Scan & Deliver
• Course Reserves
• Purchase Request
• Collection Development & Maintenance
• Current Negotiations
• Ask a Librarian
• Instructor Support
• Library How-To
• Research Guides
• Research Support
• Study Rooms
• Research Rooms
• Partner Spaces
• Loanable Equipment
• Print, Scan, Copy
• 3D Printers
• Poster Printing
• OSULP Leadership
• Strategic Plan

Scholarly Articles: How can I tell?

• Journal Information

Literature Review

• Author and affiliation
• Introduction
• Specialized Vocabulary
• Methodology
• Research sponsors
• Peer-review

The literature review section of an article is a summary or analysis of all the research the author read before doing his/her own research. This section may be part of the introduction or in a section called Background. It provides the background on who has done related research, what that research has or has not uncovered and how the current research contributes to the conversation on the topic. When you read the lit review ask:

• Does the review of the literature logically lead up to the research questions?
• Do the authors review articles relevant to their research study?
• Do the authors show where there are gaps in the literature?

The lit review is also a good place to find other sources you may want to read on this topic to help you get the bigger picture.

• << Previous: Journal Information
• Next: Author and affiliation >>
• Last Updated: Apr 15, 2024 3:26 PM
• URL: https://guides.library.oregonstate.edu/ScholarlyArticle

Contact Info

121 The Valley Library Corvallis OR 97331–4501

Phone: 541-737-3331

Services for Persons with Disabilities

In the Valley Library

• Oregon State University Press
• Special Collections and Archives Research Center
• Undergrad Research & Writing Studio
• Graduate Student Commons
• Tutoring Services
• Northwest Art Collection

Digital Projects

• Oregon Explorer
• Oregon Digital
• ScholarsArchive@OSU
• Digital Publishing Initiatives
• Atlas of the Pacific Northwest
• Marilyn Potts Guin Library  
• Cascades Campus Library
• McDowell Library of Vet Medicine

• About WordPress
• Get Involved
• WordPress.org
• Documentation
• Learn WordPress

SRJ Student Resource

Literature review vs research articles: how are they different.

Unlock the secrets of academic writing with our guide to the key differences between a literature review and a research paper! 📚 Dive into the world of scholarly exploration as we break down how a literature review illuminates existing knowledge, identifies gaps, and sets the stage for further research. 🌐 Then, gear up for the adventure of crafting a research paper, where you become the explorer, presenting your unique insights and discoveries through independent research. 🚀 Join us on this academic journey and discover the art of synthesizing existing wisdom and creating your own scholarly masterpiece! 🎓✨

We are always accepting submissions!  Submit work within  SRJ’s  scope  anytime while you’re a graduate student.

Leave a Reply Cancel reply

The act of commenting on this site is an opt-in action and San Jose State University may not be held liable for the information provided by participating in the activity.

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

Your go-to destination for graduate student research support

• Student Services
• Faculty Services

Peer Review and Primary Literature: An Introduction: Peer Review: What is it?

• Scholarly Journal vs. Magazine
• Peer Review: What is it?
• Finding Peer-Reviewed Articles
• Primary Journal Literature
• Is it Primary Research? How Do I Know?

What is Peer Review?

What Does "Peer Reviewed" Mean? When a journal is peer-reviewed (also called " Refereed "), it means that all articles submitted for publication have gone through a rigorous evaluation.  To ensure that each article meets the highest standards of scholarship, it is evaluated by the editor(s) of the journal.  This is sometimes called " internal review ." In some cases multiple in-house research editors will evaluate the scholarly strength of the article before even considering it for further peer-review.

The editor(s) then enlists the services of other scholars in the same field as the manuscript's author--in other words, that author's peers. This peer-review process is sometimes called " external review ."  These peer scholars offer their view on the quality of the article and its research.  How appropriate and exacting was the research method?  Were the results presented in the best way possible?  Was the literature review thorough?  Does the article make a significant contribution to the scholarship of that field?  Does it meet the scope of this particular journal? There are many criteria for judgement!

The exact peer-review process will vary between publishers.  Additionally, the whole value of peer review, as it now exists, is often hotly debated. Some believe that the Open Access (OA) movement of publishing research on the web and inviting scrutiny and comment will eventually eliminate the need for publisher driven peer-review.

However, as of now, peer-reviewed journal literature is still considered the highest form of scholarship.  Also, your professors will likely say that they want you to use peer-reviewed articles in your paper--sometimes exclusively.

Links to Additional Pages That Discuss the Peer Review Process

Here are a few additional web links for discussions of specific processes and the current controversies:

• View a PowerPoint presentation on the process at the BMJ (British Medical Journal) Group.
• One major publisher of scholarly journals is Elsevier (the publisher behind ScienceDirect).  Here is a page on their peer review policies.  They link to other related articles, blog entries and the like on the bottom half of that page.
• Even to receive a grant to conduct field research, scientists often must go through a peer review.  Here is an example of that process from the National Institutes of Health.

Subject Guide

• << Previous: Scholarly Journal vs. Magazine
• Next: Finding Peer-Reviewed Articles >>
• Last Updated: Nov 16, 2022 12:46 PM
• URL: https://suffolk.libguides.com/PeerandPrimary

• HKS Library & Research Services

Q. What's the difference between peer-reviewed literature and material like white papers and policy briefs?

• 1 Academic Integrity
• 1 Alerts & Feeds
• 9 Borrowing
• 6 Business Research
• 1 Campaigns and Elections
• 6 Citation Styles
• 8 Citation Tools
• 3 Copyright & Licensing
• 4 Country Research
• 5 Course Readings
• 14 Data & Statistics
• 13 Databases
• 2 Directories
• 2 Faculty Publications & Testimony
• 1 Faculty Services
• 2 Google Scholar
• 8 Government Documents
• 11 Harvard Libraries (non-HKS)
• 3 Hours and Locations
• 1 Job Search
• 9 Legal Research
• 4 Library Access
• 1 Literature Review
• 2 Mobile Apps
• 3 Multimedia
• 3 Open Access
• 3 Policy Analysis Exercise
• 2 Publishing
• 2 Recommendations
• 1 Research Consultations
• 3 Research Tools
• 1 Scanning & Faxing
• 6 Search Tips
• 3 Second Year Paper
• 1 Study Rooms
• 6 Technology
• 1 Workshops

Answered By: James Adams Last Updated: Mar 14, 2024     Views: 3405

Articles in a peer-reviewed or "refereed" journal are reviewed prior to publication by experts (usually faculty members) with credentials in the article's field of study. Many Harvard article databases allow you to limit your search results to only include peer-reviewed journals.

Other material like white papers and policy briefs that are not peer-reviewed can still be valuable for your research. Relevant preprints that will eventually be peer-reviewed or gray literature unlikely to be published in a journal could be included as long as they pass the C.R.A.A.P. test, evaluating Currency , Relevancy , Authority , Accuracy , and Purpose . 

A good strategy for finding gray literature is to conduct a search in the HKS Library Customized Google Think Tank search , which includes over 700 think tank sites from across the world. If you are uncertain about whether to include grey literature in your research project, check with your class instructor.

Return to Search and Browse All FAQs

Understanding the influence of different proxy perspectives in explaining the difference between self-rated and proxy-rated quality of life in people living with dementia: a systematic literature review and meta-analysis

• Open access
• Published: 24 April 2024

Cite this article

You have full access to this open access article

• Lidia Engel   ORCID: orcid.org/0000-0002-7959-3149 1 ,
• Valeriia Sokolova 1 ,
• Ekaterina Bogatyreva 2 &
• Anna Leuenberger 2  

125 Accesses

Explore all metrics

Proxy assessment can be elicited via the proxy-patient perspective (i.e., asking proxies to assess the patient’s quality of life (QoL) as they think the patient would respond) or proxy-proxy perspective (i.e., asking proxies to provide their own perspective on the patient’s QoL). This review aimed to identify the role of the proxy perspective in explaining the differences between self-rated and proxy-rated QoL in people living with dementia.

A systematic literate review was conducted by sourcing articles from a previously published review, supplemented by an update of the review in four bibliographic databases. Peer-reviewed studies that reported both self-reported and proxy-reported mean QoL estimates using the same standardized QoL instrument, published in English, and focused on the QoL of people with dementia were included. A meta-analysis was conducted to synthesize the mean differences between self- and proxy-report across different proxy perspectives.

The review included 96 articles from which 635 observations were extracted. Most observations extracted used the proxy-proxy perspective (79%) compared with the proxy-patient perspective (10%); with 11% of the studies not stating the perspective. The QOL-AD was the most commonly used measure, followed by the EQ-5D and DEMQOL. The standardized mean difference (SMD) between the self- and proxy-report was lower for the proxy-patient perspective (SMD: 0.250; 95% CI 0.116; 0.384) compared to the proxy-proxy perspective (SMD: 0.532; 95% CI 0.456; 0.609).

Different proxy perspectives affect the ratings of QoL, whereby adopting a proxy-proxy QoL perspective has a higher inter-rater gap in comparison with the proxy-patient perspective.

Similar content being viewed by others

Exploring self-report and proxy-report quality-of-life measures for people living with dementia in care homes

Convergent validity of EQ-5D with core outcomes in dementia: a systematic review

Proxy reporting of health-related quality of life for people with dementia: a psychometric solution

Avoid common mistakes on your manuscript.

Quality of life (QoL) has become an important outcome for research and practice but obtaining reliable and valid estimates remains a challenge in people living with dementia [ 1 ]. According to the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) criteria [ 2 ], dementia, termed as Major Neurocognitive Disorder (MND), involves a significant decline in at least one cognitive domain (executive function, complex attention, language, learning, memory, perceptual-motor, or social cognition), where the decline represents a change from a patient's prior level of cognitive ability, is persistent and progressive over time, is not associated exclusively with an episode of delirium, and reduces a person’s ability to perform everyday activities. Since dementia is one of the most pressing challenges for healthcare systems nowadays [ 3 ], it is critical to study its impact on QoL. The World Health Organization defines the concept of QoL as “individuals' perceptions of their position in life in the context of the culture and value systems in which they live and in relation to their goals, expectations, standards, and concerns” [ 4 ]. It is a broad ranging concept incorporating in a complex way the persons' physical health, psychological state, level of independence, social relationships, personal beliefs, and their relationships to salient features of the environment.

Although there is evidence that people with mild to moderate dementia can reliably rate their own QoL [ 5 ], as the disease progresses, there is typically a decline in memory, attention, judgment, insight, and communication that may compromise self-reporting of QoL [ 6 ]. Additionally, behavioral symptoms, such as agitation, and affective symptoms, such as depression, may present another challenge in obtaining self-reported QoL ratings due to emotional shifts and unwillingness to complete the assessment [ 7 ]. Although QoL is subjective and should ideally be assessed from an individual’s own perspective [ 8 ], the decline in cognitive function emphasizes the need for proxy-reporting by family members, health professionals, or care staff who are asked to report on behalf of the person with dementia. However, proxy-reports are not substitutable for self-reports from people with dementia, as they offer supplementary insights, reflecting the perceptions and viewpoints of people surrounding the person with dementia [ 9 ].

Previous research has consistently highlighted a disagreement between self-rated and proxy-rated QoL in people living with dementia, with proxies generally providing lower ratings (indicating poorer QoL) compared with person’s own ratings [ 8 , 10 , 11 , 12 ]. Impairment in cognition associated with greater dementia severity has been found to be associated with larger difference between self-rating and proxy-rating obtained from family caregivers, as it becomes increasingly difficult for severely cognitively impaired individuals to respond to questions that require contemplation, introspection, and sustained attention [ 13 , 14 ]. Moreover, non-cognitive factors, such as awareness of disease and depressive symptoms play an important role when comparing QoL ratings between individuals with dementia and their proxies [ 15 ]. Qualitative evidence has also shown that people with dementia tend to compare themselves with their peers, whereas carers make comparisons with how the person used to be in the past [ 9 ]. The disagreement between self-reported QoL and carer proxy-rated QoL could be modulated by some personal, cognitive or relational factors, for example, the type of relationship or the frequency of contact maintained, person’s cognitive status, carer’s own feeling about dementia, carer’s mood, and perceived burden of caregiving [ 14 , 16 ]. Disagreement may also arise from the person with dementia’s problems to communicate symptoms, and proxies’ inability to recognize certain symptoms, like pain [ 17 ], or be impacted by the amount of time spent with the person with dementia [ 18 ]. This may also prevent proxies to rate accurately certain domains of QoL, with previous evidence showing higher level of agreement for observable domains, such as mobility, compared with less observable domains like emotional wellbeing [ 8 ]. Finally, agreement also depends on the type of proxy (i.e., informal/family carers or professional staff) and the nature of their relationship, for instance, proxy QoL scores provided by formal carers tend to be higher (reflecting better QoL) compared to the scores supplied by family members [ 19 , 20 ]. Staff members might associate residents’ QoL with the quality of care delivered or the stage of their cognitive impairment, whereas relatives often focus on comparison with the person’s QoL when they were younger, lived in their own home and did not have dementia [ 20 ].

What has been not been fully examined to date is the role of different proxy perspectives employed in QoL questionnaires in explaining disagreement between self-rated and proxy-rated scores in people with dementia. Pickard et al. (2005) have proposed a conceptual framework for proxy assessments that distinguish between the proxy-patient perspective (i.e., asking proxies to assess the patient’s QoL as they think the patient would respond) or proxy-proxy perspective (i.e., asking proxies to provide their own perspective on the patient’s QoL) [ 21 ]. In this context, the intra-proxy gap describes the differences between proxy-patient and proxy-proxy perspective, whereas the inter-rater gap is the difference between self-report and proxy-report [ 21 ].

Existing generic and dementia-specific QoL instruments specify the perspective explicitly in their instructions or imply the perspective indirectly in their wording. For example, the instructions of the Dementia Quality of Life Measure (DEMQOL) asks proxies to give the answer they think their relative would give (i.e., proxy-patient perspective) [ 22 ], whereas the family version of the Quality of Life in Alzheimer’s Disease (QOL-AD) instructs the proxies to rate their relative’s current situation as they (the proxy) see it (i.e., proxy-proxy perspective) [ 7 ]. Some instruments, like the EQ-5D measures, have two proxy versions for each respective perspective [ 23 , 24 ]. The Adult Social Care Outcome Toolkit (ASCOT) proxy version, on the other hand, asks proxies to complete the questions from both perspectives, from their own opinion and how they think the person would answer [ 25 ].

QoL scores generated using different perspectives are expected to differ, with qualitative evidence showing that carers rate the person with dementia’s QoL lower (worse) when instructed to comment from their own perspective than from the perspective of the person with dementia [ 26 ]. However, to our knowledge, no previous review has fully synthesized existing evidence in this area. Therefore, we aimed to undertake a systematic literature review to examine the role of different proxy-assessment perspectives in explaining differences between self-rated and proxy-rated QoL in people living with dementia. The review was conducted under the hypothesis that the difference in QoL estimates will be larger when adopting the proxy-proxy perspective compared with proxy-patient perspective.

The review was registered with the International Prospective Register of Systematic Reviews (CRD42022333542) and followed the Preferred Reporting Items System for Systematic Reviews and Meta-Analysis (PRISMA) guidelines (see Appendix 1 ) [ 27 ].

Search strategy

This review used two approaches to obtain literature. First, primary articles from an existing review by Roydhouse et al. were retrieved [ 28 ]. The review included studies published from inception to February 2018 that compared self- and proxy-reports. Studies that focused explicitly on Alzheimer’s Disease or dementia were retrieved for the current review. Two reviewers conducted a full-text review to assess whether the eligibility criteria listed below for the respective study were met. An update of the Roydhouse et al. review was undertaken to capture more recent studies. The search strategy by Roydhouse et al. was amended and covered studies published after January 1, 2018, and was limited to studies within the context of dementia. The original search was undertaken over a three-week period (17/11/2021–9/12/2021) and then updated on July 3, 2023. Peer-reviewed literature was sourced from MEDLINE, CINAHL, and PsycINFO databases via EBSCOHost as well as EMBASE. Four main search term categories were used: (1) proxy terms (i.e., care*-report*), (2) QoL/ outcome terms (i.e., ‘quality of life’), (3) disease terms (i.e., ‘dementia’), and (4) pediatric terms (i.e., ‘pediatric*’) (for exclusion). Keywords were limited to appear in titles and abstracts only, and MeSH terms were included for all databases. A list of search strategy can be found in Appendix 2 . The first three search term categories were searched with AND, and the NOT function was used to exclude pediatric terms. A limiter was applied in all database searches to only include studies with human participants and articles published in English.

Selection criteria

Studies from all geographical locations were included in the review if they (1) were published in English in a peer-reviewed journal (conference abstracts, dissertations, a gray literature were excluded); (2) were primary studies (reviews were excluded); (3) clearly defined the disease of participants, which were limited to Alzheimer’s disease or dementia; (4) reported separate QoL scores for people with dementia (studies that included mixed populations had to report a separate QoL score for people with dementia to be considered); (5) were using a standardized and existing QoL instrument for assessment; and (6) provided a mean self-reported and proxy-reported QoL score for the same dyads sample (studies that reported means for non-matched samples were excluded) using the same QoL instrument.

Four reviewers (LE, VS, KB, AL) were grouped into two groups who independently screened the 179 full texts from the Roydhouse et. al (2022) study that included Alzheimer’s disease or dementia patients. If a discrepancy within the inclusion selection occurred, articles were discussed among all the reviewers until a consensus was reached. Studies identified from the database search were imported into EndNote [ 29 ]. Duplicates were removed through EndNote and then uploaded to Rayyan [ 30 ]. Each abstract was reviewed by two independent reviewers (any two from four reviewers). Disagreements regarding study inclusions were discussed between all reviewers until a consensus was reached. Full-text screening of each eligible article was completed by two independent reviewers (any two from four reviewers). Again, a discussion between all reviewers was used in case of disagreements.

Data extraction

A data extraction template was created in Microsoft Excel. The following information were extracted if available: country, study design, study sample, study setting, dementia type, disease severity, Mini-Mental Health State Exam (MMSE) score details, proxy type, perspective, living arrangements, QoL assessment measure/instrument, self-reported scores (mean, SD), proxy-reported scores (mean, SD), and agreement statistics. If a study reported the mean (SD) for the total score as well as for specific QoL domains of the measure, we extracted both. If studies reported multiple scores across different time points or subgroups, we extracted all scores. For interventional studies, scores from both the intervention group and the control group were recorded. In determining the proxy perspective, we relied on authors’ description in the article. If the perspective was not explicitly stated, we adopted the perspective of the instrument developers; where more perspectives were possible (e.g., in the case of the EQ-5D measures) and the perspective was not explicitly stated, it was categorized as ‘undefined.’ For agreement, we extracted the Intraclass Correlation Coefficient (ICC), a reliability index that reflects both degree of correlation and agreement between measurements of continuous variables. While there are different forms of ICC based on the model (1-way random effects, 2-wy random effects, or 2-way fixed effects), the type (single rater/measurement or the mean k raters/measurements), and definition of relationship [ 31 ], this level of information was not extracted due to insufficient information provided in the original studies. Values for ICC range between 0 and 1, with values interpreted as poor (less than 0.5), moderate (0.5–0.75), good (0.75–0.9), and excellent (greater than 0.9) reliability between raters [ 31 ].

Data synthesis and analysis

Characteristics of studies were summarized descriptively. Self-reported and proxy-reported means and SD were extracted from the full texts and the mean difference was calculated (or extracted if available) for each pair. Studies that reported median values instead of mean values were converted using the approach outlined by Wan et al. (2014) [ 32 ]. Missing SDs (5 studies, 20 observations) were obtained from standard errors or confidence intervals reported following the Cochrane guidelines [ 33 ]. Missing SDs (6 studies, 29 observations) in studies that only presented the mean value without any additional summary statistics were imputed using the prognostic method [ 34 ]. Thereby, we predicted the missing SDs by calculating the average SDs of observed studies with full information by the respective measure and source (self-report versus proxy-report).

A meta-analysis was performed in Stata (17.1 Stata Corp LLC, College Station, TX) to synthesize mean differences between self- and proxy-reported scores across different proxy perspectives. First, the pooled raw mean differences were calculated for each QoL measure separately, given differences in scales between measures. Secondly, we calculated the pooled standardized mean difference (SMD) for all studies stratified by proxy type (family carer, formal carers, mixed), dementia severity (mild, moderate, severe), and living arrangement (residential/institutional care, mixed). SMD accounts for the use of different measurement scales, where effect sizes were estimated using Cohen’s d. Random-effects models were used to allow for unexplained between-study variability based on the restricted maximum-likelihood (REML) estimator. The percentage of variability attributed to heterogeneity between the studies was assessed using the I 2 statistic; an I 2 of 0%-40% represents possibly unimportant heterogeneity, 30–60% moderate heterogeneity, 50–90% substantial heterogeneity, and 75%-100% considerable heterogeneity [ 35 ]. Chi-squared statistics (χ 2 ) provided evidence of heterogeneity, where a p -value of 0.1 was used as significance level. For studies that reported agreement statistics, based on ICC, we also ran a forest plot stratified by the study perspective. We also calculated Q statistic (Cochran’s test of homogeneity), which assesses whether observed differences in results are compatible with chance alone.

Risk of bias and quality assessment

The quality of studies was assessed using the using a checklist for assessing the quality of quantitative studies developed by Kmet et al. (2004) [ 36 ]. The checklist consists of 14 items and items are scored as ‘2’ (yes, item sufficiently addressed), ‘1’ (item partially addressed), ‘0’ (no, not addressed), or ‘not applicable.’ A summary score was calculated for each study by summing the total score obtained across relevant items and dividing by the total possible score. Scores were adjusted by excluding items that were not applicable from the total score. Quality assessment was undertaken by one reviewer, with 25% of the papers assessed independently by a second reviewer.

The PRISMA diagram in Fig.  1 shows that after the abstract and full-text screening, 38 studies from the database search and 58 studies from the Roydhouse et al. (2022) review were included in this review—a total of 96 studies. A list of all studies included and their characteristics can be found in Appendix 3.

PRISMA 2020 flow diagram

General study characteristics

The 96 articles included in the review were published between 1999 and 2023 from across the globe; most studies (36%) were conducted in Europe. People with dementia in these studies were living in the community (67%), residential/institutional care (15%), as well as mixed dwelling settings (18%). Most proxy-reports were provided by family carers (85%) and only 8 studies (8%) included formal carers. The mean MMSE score for dementia and Alzheimer’s participants was 18.77 (SD = 4.34; N  = 85 studies), which corresponds to moderate cognitive impairment [ 37 ]. Further characteristics of studies included are provided in Table  1 . The quality of studies included (see Appendix 4) was generally very good, scoring on average 91% (SD: 9.1) with scores ranging from 50 to 100%.

Quality of life measure and proxy perspective used

A total of 635 observations were recorded from the 96 studies. The majority of studies and observations extracted assumed the proxy-proxy perspective (77 studies, 501 observations), followed by the proxy-patient perspective (18 studies, 62 observations), with 18 studies (72 observations) not clearly defining the perspective. Table 2 provides a detailed overview of number of studies and observations across the respective QoL measures and proxy perspectives. Two studies (14 observations) adopted both perspectives within the same study design: one using the QOL-AD measure [ 5 ] and the second study exploring the EQ-5D-3L and EQ VAS [ 38 ]. Overall, the QOL-AD was the most often used QoL measure, followed by the EQ-5D and DEMQOL. Mean scores for specific QoL domains were accessible for the DEMQOL and QOL-AD. However, only the QOL-AD provided domain-specific mean scores from both proxy perspectives.

Mean scores and mean differences by proxy perspective and QoL measure

The raw mean scores for self-reported and proxy-reported QoL scores are provided in the Supplementary file 2. The pooled raw mean difference by proxy perspective and measure is shown in Table  3 . Regardless of the perspective adopted and the QoL instrument used, self-reported scores were higher (indicating better QoL) compared with proxy-reported scores, except for the DEMQOL, where proxies reported better QoL than people with dementia themselves. Most instruments were explored from one perspective, except for the EQ-5D-3L, EQ VAS, and QOL-AD, for which mean differences were available for both perspectives. For these three measures, mean differences were smaller when adopting the proxy-patient perspective compared with proxy-proxy perspective, although mean scores for the QOL-AD were slightly lower from the proxy-proxy perspective. I 2 statistics indicate considerable heterogeneity (I 2  > 75%) between studies. Mean differences by specific QoL domains are provided in Appendix 5, but only for the QOL-AD measure that was explored from both perspectives. Generally, mean differences appeared to be smaller for the proxy-proxy perspective than the proxy-patient perspective across all domains, except for ‘physical health’ and ‘doing chores around the house.’ However, results need to be interpreted carefully as proxy-patient perspective scores were derived from only one study.

Standardized mean differences by proxy perspective, stratified by proxy type, dementia severity, and living arrangement

Table 4 provides the SMD by proxy perspective, which adjusts for the different QoL measurement scales. Findings suggest that adopting the proxy-patient perspective results in lower SMDs (SMD: 0.250; 95% CI 0.116; 0.384) compared with the proxy-proxy perspective (SMD: 0.532; 95% CI 0.456; 0.609). The largest SMD was recorded for studies that did not define the study perspective (SMD: 0.594; 95% CI 0.469; 0.718). A comparison by different proxy types (formal carers, family carers, and mixed proxies) revealed some mixed results. When adopting the proxy-proxy perspective, the largest SMD was found for family carers (SMD: 0.556; 95% CI 0.465; 0.646) compared with formal carers (SMD: 0.446; 95% CI 0.305; 0.586) or mixed proxies (SMD: 0.335; 95% CI 0.211; 0.459). However, the opposite relationship was found when the proxy-patient perspective was used, where the smallest SMD was found for family carers compared with formal carers and mixed proxies. The SMD increased with greater level of dementia severity, suggesting a greater disagreement. However, compared with the proxy-proxy perspective, where self-reported scores were greater (i.e., better QoL) than proxy-reported scores across all dementia severity levels, the opposite was found when adopting the proxy-patient perspective, where proxies reported better QoL than people with dementia themselves, except for the severe subgroup. No clear trend was observed for different living settings, although the SMD appeared to be smaller for people with dementia living in residential care compared with those living in the community.

Direct proxy perspectives comparison studies

Two studies assessed both proxy perspectives within the same study design. Bosboom et al. (2012) found that compared with self-reported scores (mean: 34.7; SD: 5.3) using the QOL-AD, proxy scores using the proxy-patient perspective were closer to the self-reported scores (mean: 32.1; SD: 6.1) compared with the proxy-proxy perspective (mean: 29.5; SD: 5.4) [ 5 ]. Similar findings were reported by Leontjevas et al. (2016) using the EQ-5D-3L, including the EQ VAS, showing that the inter-proxy gap between self-report (EQ-5D-3L: 0.609; EQ VAS: 65.37) and proxy-report was smaller when adopting the proxy-patient perspective (EQ-5D-3L: 0.555; EQ VAS: 65.15) compared with the proxy-proxy perspective (EQ-5D-3L: 0.492; EQ VAS: 64.42) [ 38 ].

Inter-rater agreement (ICC) statistics

Six studies reported agreement statistics based on ICC, from which we extracted 17 observations that were included in the meta-analysis. Figure  2 shows the study-specific and overall estimates of ICC by the respective study perspective. The heterogeneity between studies was high ( I 2  = 88.20%), with a Q test score of 135.49 ( p  < 0.001). While the overall ICC for the 17 observations was 0.3 (95% CI 0.22; 0.38), indicating low agreement, the level of agreement was slightly better when adopting a proxy-patient perspective (ICC: 0.36, 95% CI 0.23; 0.49) than a proxy-proxy perspective (ICC: 0.26, 95% CI 0.17; 0.35).

Forest plot depicting study-specific and overall ICC estimates by study perspective

While previous studies highlighted a disagreement between self-rated and proxy-rated QoL in people living with dementia, this review, for the first time, assessed the role of different proxy perspectives in explaining the inter-rater gap. Our findings align with the baseline hypothesis and indicate that QoL scores reported from the proxy-patient perspective are closer to self-reported QoL scores than the proxy-proxy perspective, suggesting that the proxy perspective does impact the inter-rater gap and should not be ignored. This finding was observed across different analyses conducted in this review (i.e., pooled raw mean difference, SMD, ICC analysis), which also confirms the results of two previous primary studies that adopted both proxy perspectives within the same study design [ 5 , 38 ]. Our findings emphasize the need for transparency in reporting the proxy perspective used in future studies, as it can impact results and interpretation. This was also noted by the recent ISPOR Proxy Task Force that developed a checklist of considerations when using proxy-reporting [ 39 ]. While consistency in proxy-reports is desirable, it is crucial to acknowledge that each proxy perspective holds significance in future research, depending on study objectives. It is evident that both proxy perspectives offer distinct insights—one encapsulating the perspectives of people with dementia, and the other reflecting the viewpoints of proxies. Therefore, in situations where self-report is unattainable due to advanced disease severity and the person’s perspective on their own QoL assessment is sought, it is recommended to use the proxy-patient perspective. Conversely, if the objective of future research is to encompass the viewpoints of proxies, opting for the proxy-proxy perspective is advisable. However, it is important to note that proxies may deviate from instructed perspectives, requiring future qualitative research to examine the adherence to proxy perspectives. Additionally, others have argued that proxy-reports should not substitute self-reports, and only serve as supplementary sources alongside patient self-reports whenever possible [ 9 ].

This review considered various QoL instruments, but most instruments adopted one specific proxy perspective, limiting detailed analyses. QoL instruments differ in their scope (generic versus disease-specific) as well as coverage of QoL domains. The QOL-AD, an Alzheimer's Disease-specific measure, was commonly used. Surprisingly, for this measure, the mean differences between self-reported and proxy-reported scores were smaller using the proxy-proxy perspective, contrary to the patterns observed with all other instruments. This may be due to the lack of studies reporting QOL-AD proxy scores from the proxy-patient perspective, as the study by Bosboom et al. (2012) found the opposite [ 5 ]. Previous research has also suggested that the inter-rater gap is dependent on the QoL domains and that the risk of bias is greater for more ‘subjective’ (less observable) domains such as emotions, feelings, and moods in comparison with observable, objective areas such as physical domains [ 8 , 40 ]. However, this review lacks sufficient observations for definitive results on QoL dimensions and their impact on self-proxy differences, emphasizing the need for future research in this area.

With regard to proxy type, there is an observable trend suggesting a wider inter-rater gap when family proxies are employed using the proxy-proxy perspective, in contrast to formal proxies. This variance might be attributed to the use of distinct anchoring points; family proxies tend to assess the individual's QoL in relation to their past self before having dementia, while formal caregivers may draw comparisons with other individuals with dementia under their care [ 41 ]. However, the opposite was found when the proxy-patient perspective was used, where family proxies scores seemed to align more closely with self-reported scores, resulting in lower SMD scores. This suggests that family proxies might possess a better ability to empathize with the perspective of the person with dementia compared to formal proxies. Nonetheless, it is important to interpret these findings cautiously, given the relatively small number of observations for formal caregiver reports. Additionally, other factors such as emotional connection, caregiver burden, and caregiver QoL may also impact proxy-reports by family proxies [ 14 , 16 ] that have not been explored in this review.

Our review found that the SMD between proxy and self-report increased with greater level of dementia severity, contrasting a previous study, which showed that cognitive impairment was not the primary factor that accounted for the differences in the QoL assessments between family proxies and the person with dementia [ 15 ]. However, it is noteworthy that different interpretations and classifications were used across studies to define mild, moderate, and severe dementia, which needs to be considered. Most studies used MMSE to define dementia severity levels. Given the MMSE’s role as a standard measure of cognitive function, the study findings are considered generalizable and clinically relevant for people with dementia across different dementia severity levels. When examining the role of the proxy perspective by level of severity, we found that compared with the proxy-proxy perspective, where self-reported scores were greater than proxy-reported scores across all dementia severity levels, the proxy-patient perspective yielded the opposite results, and proxies reported better QoL than people with dementia themselves, except for the severe subgroup. It is possible that in the early stages of dementia, the person with dementia has a greater awareness of increasing deficits, coupled with denial and lack of acceptance, leading to a more critical view of their own QoL than how proxies think they would rate their QoL. However, future studies are warranted, given the small number of observations adopting the proxy-patient perspective in our review.

The heterogeneity observed in the studies included was high, supporting the use of random-effects meta-analysis. This is not surprising given the diverse nature of studies included (i.e., RCTs, cross-sectional studies), differences in the population (i.e., people living in residential care versus community-dwelling people), mixed levels of dementia severity, and differences between instruments. While similar heterogeneity was observed in another review on a similar topic [ 42 ], our presentation of findings stratified by proxy type, dementia severity, and living arrangement attempted to account for such differences across studies.

Limitations and recommendations for future studies

Our review has some limitations. Firstly, proxy perspectives were categorized based on the authors' descriptions, but many papers did not explicitly state the perspective, which led to the use of assumptions based on instrument developers. Some studies may have modified the perspective's wording without reporting it. Due to lack of resources, we did not contact the authors of the original studies directly to seek clarification around the proxy perspective adopted. Regarding studies using the EQ-5D, which has two proxy perspectives, some studies did not specify which proxy version was used, suggesting the potential use of self-reported versions for proxies. In such cases, the proxy perspective was categorized as undefined. Despite accounting for factors like QoL measure, proxy type, setting, and dementia severity, we could not assess the impact of proxy characteristics (e.g., carer burden) or dementia type due to limited information provided in the studies. We also faced limitations in exploring the proxy perspective by QoL domains due to limited information. Further, not all studies outlined the data collection process in full detail. For example, it is possible that the proxy also assisted the person with dementia with their self-report, which could have resulted in biased estimates and the need for future studies applying blinding. Although we assessed the risk of bias of included studies, the checklist was not directly reflecting the purpose of our study that looked into inter-rater agreement. No checklist for this purpose currently exists. Finally, quality appraisal by a second reviewer was only conducted for the first 25% of the studies due to resource constraints and a low rate of disagreement between the two assessors. However, an agreement index between reviewers regarding the concordance in selecting full texts for inclusion and conducting risk of bias assessments was not calculated.

This review demonstrates that the choice of proxy perspective impacts the inter-rater gap. QoL scores from the proxy-patient perspective align more closely with self-reported scores than the proxy-proxy perspective. These findings contribute to the broader literature investigating factors influencing differences in QoL scores between proxies and individuals with dementia. While self-reported QoL is the gold standard, proxy-reports should be viewed as complements rather than substitutes. Both proxy perspectives offer unique insights, yet QoL assessments in people with dementia are complex. The difference in self- and proxy-reports can be influenced by various factors, necessitating further research before presenting definitive results that inform care provision and policy.

Data availability

All data associated with the systematic literature review are available in the supplementary file.

Moyle, W., & Murfield, J. E. (2013). Health-related quality of life in older people with severe dementia: Challenges for measurement and management. Expert Review of Pharmacoeconomics & Outcomes, 13 (1), 109–122. https://doi.org/10.1586/erp.12.84

Article   Google Scholar  

Sachdev, P. S., Blacker, D., Blazer, D. G., Ganguli, M., Jeste, D. V., Paulsen, J. S., & Petersen, R. C. (2014). Classifying neurocognitive disorders: The DSM-5 approach. Nature reviews Neurology, 10 (11), 634–642. https://doi.org/10.1038/nrneurol.2014.181

Article   PubMed   Google Scholar  

Health, The Lancet Regional., & – Europe. (2022). Challenges for addressing dementia. The Lancet Regional Health . https://doi.org/10.1016/j.lanepe.2022.100504

The WHOQOL Group. (1995). The World Health Organization quality of life assessment (WHOQOL): Position paper from the World Health Organization. Social Science & Medicine, 41 (10), 1403–1409. https://doi.org/10.1016/0277-9536(95)00112-K

Bosboom, P. R., Alfonso, H., Eaton, J., & Almeida, O. P. (2012). Quality of life in Alzheimer’s disease: Different factors associated with complementary ratings by patients and family carers. International Psychogeriatrics, 24 (5), 708–721. https://doi.org/10.1017/S1041610211002493

Scholzel-Dorenbos, C. J., Rikkert, M. G., Adang, E. M., & Krabbe, P. F. (2009). The challenges of accurate measurement of health-related quality of life in frail elderly people and dementia. Journal of the American Geriatrics Society, 57 (12), 2356–2357. https://doi.org/10.1111/j.1532-5415.2009.02586.x

Logsdon, R. G., Gibbons, L. E., McCurry, S. M., & Teri, L. (2002). Assessing quality of life in older adults with cognitive impairment. Psychosomatic Medicine, 64 (3), 510–519. https://doi.org/10.1097/00006842-200205000-00016

Hutchinson, C., Worley, A., Khadka, J., Milte, R., Cleland, J., & Ratcliffe, J. (2022). Do we agree or disagree? A systematic review of the application of preference-based instruments in self and proxy reporting of quality of life in older people. Social Science & Medicine, 305 , 115046. https://doi.org/10.1016/j.socscimed.2022.115046

Smith, S. C., Hendriks, A. A. J., Cano, S. J., & Black, N. (2020). Proxy reporting of health-related quality of life for people with dementia: A psychometric solution. Health and Quality of Life Outcomes, 18 (1), 148. https://doi.org/10.1186/s12955-020-01396-y

Article   CAS   PubMed   PubMed Central   Google Scholar  

Andrieu, S., Coley, N., Rolland, Y., Cantet, C., Arnaud, C., Guyonnet, S., Nourhashemi, F., Grand, A., Vellas, B., & group, P. (2016). Assessing Alzheimer’s disease patients’ quality of life: Discrepancies between patient and caregiver perspectives. Alzheimer’s & Dementia, 12 (4), 427–437. https://doi.org/10.1016/j.jalz.2015.09.003

Jönsson, L., Andreasen, N., Kilander, L., Soininen, H., Waldemar, G., Nygaard, H., Winblad, B., Jonhagen, M. E., Hallikainen, M., & Wimo, A. (2006). Patient- and proxy-reported utility in Alzheimer disease using the EuroQoL. Alzheimer Disease & Associated Disorders, 20 (1), 49–55. https://doi.org/10.1097/01.wad.0000201851.52707.c9

Zucchella, C., Bartolo, M., Bernini, S., Picascia, M., & Sinforiani, E. (2015). Quality of life in Alzheimer disease: A comparison of patients’ and caregivers’ points of view. Alzheimer Disease & Associated Disorders, 29 (1), 50–54. https://doi.org/10.1097/WAD.0000000000000050

Article   CAS   Google Scholar  

Buckley, T., Fauth, E. B., Morrison, A., Tschanz, J., Rabins, P. V., Piercy, K. W., Norton, M., & Lyketsos, C. G. (2012). Predictors of quality of life ratings for persons with dementia simultaneously reported by patients and their caregivers: The Cache County (Utah) study. International Psychogeriatrics, 24 (7), 1094–1102. https://doi.org/10.1017/S1041610212000063

Article   PubMed   PubMed Central   Google Scholar  

Schiffczyk, C., Romero, B., Jonas, C., Lahmeyer, C., Muller, F., & Riepe, M. W. (2010). Generic quality of life assessment in dementia patients: A prospective cohort study. BMC Neurology, 10 , 48. https://doi.org/10.1186/1471-2377-10-48

Sousa, M. F., Santos, R. L., Arcoverde, C., Simoes, P., Belfort, T., Adler, I., Leal, C., & Dourado, M. C. (2013). Quality of life in dementia: The role of non-cognitive factors in the ratings of people with dementia and family caregivers. International Psychogeriatrics, 25 (7), 1097–1105. https://doi.org/10.1017/S1041610213000410

Arons, A. M., Krabbe, P. F., Scholzel-Dorenbos, C. J., van der Wilt, G. J., & Rikkert, M. G. (2013). Quality of life in dementia: A study on proxy bias. BMC Medical Research Methodology, 13 , 110. https://doi.org/10.1186/1471-2288-13-110

Gomez-Gallego, M., Gomez-Garcia, J., & Ato-Lozano, E. (2015). Addressing the bias problem in the assessment of the quality of life of patients with dementia: Determinants of the accuracy and precision of the proxy ratings. The Journal of Nutrition, Health & Aging, 19 (3), 365–372. https://doi.org/10.1007/s12603-014-0564-7

Moon, H., Townsend, A. L., Dilworth-Anderson, P., & Whitlatch, C. J. (2016). Predictors of discrepancy between care recipients with mild-to-moderate dementia and their caregivers on perceptions of the care recipients’ quality of life. American Journal of Alzheimer’s Disease & Other Dementias, 31 (6), 508–515. https://doi.org/10.1177/1533317516653819

Crespo, M., Bernaldo de Quiros, M., Gomez, M. M., & Hornillos, C. (2012). Quality of life of nursing home residents with dementia: A comparison of perspectives of residents, family, and staff. The Gerontologist, 52 (1), 56–65. https://doi.org/10.1093/geront/gnr080

Griffiths, A. W., Smith, S. J., Martin, A., Meads, D., Kelley, R., & Surr, C. A. (2020). Exploring self-report and proxy-report quality-of-life measures for people living with dementia in care homes. Quality of Life Research, 29 (2), 463–472. https://doi.org/10.1007/s11136-019-02333-3

Pickard, A. S., & Knight, S. J. (2005). Proxy evaluation of health-related quality of life: A conceptual framework for understanding multiple proxy perspectives. Medical Care, 43 (5), 493–499. https://doi.org/10.1097/01.mlr.0000160419.27642.a8

Smith, S. C., Lamping, D. L., Banerjee, S., Harwood, R. H., Foley, B., Smith, P., Cook, J. C., Murray, J., Prince, M., Levin, E., Mann, A., & Knapp, M. (2007). Development of a new measure of health-related quality of life for people with dementia: DEMQOL. Psychological Medicine, 37 (5), 737–746. https://doi.org/10.1017/S0033291706009469

Article   CAS   PubMed   Google Scholar  

Brooks, R. (1996). EuroQol: The current state of play. Health Policy, 37 (1), 53–72. https://doi.org/10.1016/0168-8510(96)00822-6

Herdman, M., Gudex, C., Lloyd, A., Janssen, M., Kind, P., Parkin, D., Bonsel, G., & Badia, X. (2011). Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Quality of Life Research, 20 (10), 1727–1736. https://doi.org/10.1007/s11136-011-9903-x

Rand, S., Caiels, J., Collins, G., & Forder, J. (2017). Developing a proxy version of the adult social care outcome toolkit (ASCOT). Health and Quality of Life Outcomes, 15 (1), 108. https://doi.org/10.1186/s12955-017-0682-0

Engel, L., Bucholc, J., Mihalopoulos, C., Mulhern, B., Ratcliffe, J., Yates, M., & Hanna, L. (2020). A qualitative exploration of the content and face validity of preference-based measures within the context of dementia. Health and Quality of Life Outcomes, 18 (1), 178. https://doi.org/10.1186/s12955-020-01425-w

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hrobjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., … Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. British Medical Journal, 372 , n71. https://doi.org/10.1136/bmj.n71

Roydhouse, J. K., Cohen, M. L., Eshoj, H. R., Corsini, N., Yucel, E., Rutherford, C., Wac, K., Berrocal, A., Lanzi, A., Nowinski, C., Roberts, N., Kassianos, A. P., Sebille, V., King, M. T., Mercieca-Bebber, R., Force, I. P. T., & the, I. B. o. D. (2022). The use of proxies and proxy-reported measures: A report of the international society for quality of life research (ISOQOL) proxy task force. Quality of Life Research, 31 (2), 317–327. https://doi.org/10.1007/s11136-021-02937-8

The EndNote Team. (2013). EndNote (Version EndNote X9) [64 bit]. Philadelphia, PA: Clarivate.

Ouzzani, M., Hammady, H., Fedorowicz, Z., & Elmagarmid, A. (2016). Rayyan—A web and mobile app for systematic reviews. Systematic Reviews, 5 (1), 210. https://doi.org/10.1186/s13643-016-0384-4

Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15 (2), 155–163. https://doi.org/10.1016/j.jcm.2016.02.012

Wan, X., Wang, W., Liu, J., & Tong, T. (2014). Estimating the sample mean and standard deviation from the sample size, median, range and/or interquartile range. BMC Medical Research Methodology, 14 , 135. https://doi.org/10.1186/1471-2288-14-135

Higgins JPT and Green S (editors). (2011). Cochrane handbook for systematic reviews of interventions.Version 5.1.0 [updated March 2011]. Retrieved 20 Jan 2023, from https://handbook-5-1.cochrane.org/chapter_7/7_7_3_2_obtaining_standard_deviations_from_standard_errors_and.htm

Ma, J., Liu, W., Hunter, A., & Zhang, W. (2008). Performing meta-analysis with incomplete statistical information in clinical trials. BMC Medical Research Methodology, 8 , 56. https://doi.org/10.1186/1471-2288-8-56

Deeks, J. J., Higgins, J. P. T., Altman, D. G., & on behalf of the Cochrane Statistical Methods Group. (2023). Chapter 10: Analysing data and undertaking meta-analyses. In J. Higgins & J. Thomas (Eds.), Cochrane Handbook for Systematic Reviews of Interventions. Version 6.4.

Kmet, L. M., Cook, L. S., & Lee, R. C. (2004). Standard quality assessment criteria for evaluating primary research papers from a variety of fields: Health and technology assessment unit. Alberta Heritage Foundation for Medical Research.

Lewis, T. J., & Trempe, C. L. (2017). Diagnosis of Alzheimer’s: Standard-of-care . USA: Elsevier Science & Technology.

Google Scholar  

Leontjevas, R., Teerenstra, S., Smalbrugge, M., Koopmans, R. T., & Gerritsen, D. L. (2016). Quality of life assessments in nursing homes revealed a tendency of proxies to moderate patients’ self-reports. Journal of Clinical Epidemiology, 80 , 123–133. https://doi.org/10.1016/j.jclinepi.2016.07.009

Lapin, B., Cohen, M. L., Corsini, N., Lanzi, A., Smith, S. C., Bennett, A. V., Mayo, N., Mercieca-Bebber, R., Mitchell, S. A., Rutherford, C., & Roydhouse, J. (2023). Development of consensus-based considerations for use of adult proxy reporting: An ISOQOL task force initiative. Journal of Patient-Reported Outcomes, 7 (1), 52. https://doi.org/10.1186/s41687-023-00588-6

Li, M., Harris, I., & Lu, Z. K. (2015). Differences in proxy-reported and patient-reported outcomes: assessing health and functional status among medicare beneficiaries. BMC Medical Research Methodology . https://doi.org/10.1186/s12874-015-0053-7

Robertson, S., Cooper, C., Hoe, J., Lord, K., Rapaport, P., Marston, L., Cousins, S., Lyketsos, C. G., & Livingston, G. (2020). Comparing proxy rated quality of life of people living with dementia in care homes. Psychological Medicine, 50 (1), 86–95. https://doi.org/10.1017/S0033291718003987

Khanna, D., Khadka, J., Mpundu-Kaambwa, C., Lay, K., Russo, R., Ratcliffe, J., & Quality of Life in Kids: Key Evidence to Strengthen Decisions in Australia Project, T. (2022). Are We Agreed? Self- versus proxy-reporting of paediatric health-related quality of life (HRQoL) Using generic preference-based measures: A systematic review and meta-analysis. PharmacoEconomics, 40 (11), 1043–1067. https://doi.org/10.1007/s40273-022-01177-z

Download references

Open Access funding enabled and organized by CAUL and its Member Institutions. This study was conducted without financial support.

Author information

Authors and affiliations.

Monash University Health Economics Group, School of Public Health and Preventive Medicine, Monash University, Level 4, 553 St. Kilda Road, Melbourne, VIC, 3004, Australia

Lidia Engel & Valeriia Sokolova

School of Health and Social Development, Deakin University, Burwood, VIC, Australia

Ekaterina Bogatyreva & Anna Leuenberger

You can also search for this author in PubMed   Google Scholar

Contributions

LE contributed to the study conception and design. The original database search was performed by AL and later updated by VS. All authors were involved in the screening process, data extraction, and data analyses. Quality assessment was conducted by VS and LE. The first draft of the manuscript was written by LE and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Lidia Engel .

Ethics declarations

Competing interests.

Lidia Engel is a member of the EuroQol Group.

Ethical approval

Not applicable.

Consent to participate

Consent to publish, additional information, publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (XLSX 67 KB)

Supplementary file2 (docx 234 kb), rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Engel, L., Sokolova, V., Bogatyreva, E. et al. Understanding the influence of different proxy perspectives in explaining the difference between self-rated and proxy-rated quality of life in people living with dementia: a systematic literature review and meta-analysis. Qual Life Res (2024). https://doi.org/10.1007/s11136-024-03660-w

Download citation

Accepted : 27 March 2024

Published : 24 April 2024

DOI : https://doi.org/10.1007/s11136-024-03660-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Quality of Life
• Outcome measurement
• Find a journal
• Publish with us
• Track your research

• Open access
• Published: 23 April 2024

Designing feedback processes in the workplace-based learning of undergraduate health professions education: a scoping review

• Javiera Fuentes-Cimma 1 , 2 ,
• Dominique Sluijsmans 3 ,
• Arnoldo Riquelme 4 ,
• Ignacio Villagran   ORCID: orcid.org/0000-0003-3130-8326 1 ,
• Lorena Isbej   ORCID: orcid.org/0000-0002-4272-8484 2 , 5 ,
• María Teresa Olivares-Labbe 6 &
• Sylvia Heeneman 7  

BMC Medical Education volume  24 , Article number:  440 ( 2024 ) Cite this article

240 Accesses

Metrics details

Feedback processes are crucial for learning, guiding improvement, and enhancing performance. In workplace-based learning settings, diverse teaching and assessment activities are advocated to be designed and implemented, generating feedback that students use, with proper guidance, to close the gap between current and desired performance levels. Since productive feedback processes rely on observed information regarding a student's performance, it is imperative to establish structured feedback activities within undergraduate workplace-based learning settings. However, these settings are characterized by their unpredictable nature, which can either promote learning or present challenges in offering structured learning opportunities for students. This scoping review maps literature on how feedback processes are organised in undergraduate clinical workplace-based learning settings, providing insight into the design and use of feedback.

A scoping review was conducted. Studies were identified from seven databases and ten relevant journals in medical education. The screening process was performed independently in duplicate with the support of the StArt program. Data were organized in a data chart and analyzed using thematic analysis. The feedback loop with a sociocultural perspective was used as a theoretical framework.

The search yielded 4,877 papers, and 61 were included in the review. Two themes were identified in the qualitative analysis: (1) The organization of the feedback processes in workplace-based learning settings, and (2) Sociocultural factors influencing the organization of feedback processes. The literature describes multiple teaching and assessment activities that generate feedback information. Most papers described experiences and perceptions of diverse teaching and assessment feedback activities. Few studies described how feedback processes improve performance. Sociocultural factors such as establishing a feedback culture, enabling stable and trustworthy relationships, and enhancing student feedback agency are crucial for productive feedback processes.

Conclusions

This review identified concrete ideas regarding how feedback could be organized within the clinical workplace to promote feedback processes. The feedback encounter should be organized to allow follow-up of the feedback, i.e., working on required learning and performance goals at the next occasion. The educational programs should design feedback processes by appropriately planning subsequent tasks and activities. More insight is needed in designing a full-loop feedback process, in which specific attention is needed in effective feedforward practices.

Peer Review reports

The design of effective feedback processes in higher education has been important for educators and researchers and has prompted numerous publications discussing potential mechanisms, theoretical frameworks, and best practice examples over the past few decades. Initially, research on feedback primarily focused more on teachers and feedback delivery, and students were depicted as passive feedback recipients [ 1 , 2 , 3 ]. The feedback conversation has recently evolved to a more dynamic emphasis on interaction, sense-making, outcomes in actions, and engagement with learners [ 2 ]. This shift aligns with utilizing the feedback process as a form of social interaction or dialogue to enhance performance [ 4 ]. Henderson et al. (2019) defined feedback processes as "where the learner makes sense of performance-relevant information to promote their learning." (p. 17). When a student grasps the information concerning their performance in connection to the desired learning outcome and subsequently takes suitable action, a feedback loop is closed so the process can be regarded as successful [ 5 , 6 ].

Hattie and Timperley (2007) proposed a comprehensive perspective on feedback, the so-called feedback loop, to answer three key questions: “Where am I going? “How am I going?” and “Where to next?” [ 7 ]. Each question represents a key dimension of the feedback loop. The first is the feed-up, which consists of setting learning goals and sharing clear objectives of learners' performance expectations. While the concept of the feed-up might not be consistently included in the literature, it is considered to be related to principles of effective feedback and goal setting within educational contexts [ 7 , 8 ]. Goal setting allows students to focus on tasks and learning, and teachers to have clear intended learning outcomes to enable the design of aligned activities and tasks in which feedback processes can be embedded [ 9 ]. Teachers can improve the feed-up dimension by proposing clear, challenging, but achievable goals [ 7 ]. The second dimension of the feedback loop focuses on feedback and aims to answer the second question by obtaining information about students' current performance. Different teaching and assessment activities can be used to obtain feedback information, and it can be provided by a teacher or tutor, a peer, oneself, a patient, or another coworker. The last dimension of the feedback loop is the feedforward, which is specifically associated with using feedback to improve performance or change behaviors [ 10 ]. Feedforward is crucial in closing the loop because it refers to those specific actions students must take to reduce the gap between current and desired performance [ 7 ].

From a sociocultural perspective, feedback processes involve a social practice consisting of intricate relationships within a learning context [ 11 ]. The main feature of this approach is that students learn from feedback only when the feedback encounter includes generating, making sense of, and acting upon the information given [ 11 ]. In the context of workplace-based learning (WBL), actionable feedback plays a crucial role in enabling learners to leverage specific feedback to enhance their performance, skills, and conceptual understandings. The WBL environment provides students with a valuable opportunity to gain hands-on experience in authentic clinical settings, in which students work more independently on real-world tasks, allowing them to develop and exhibit their competencies [ 3 ]. However, WBL settings are characterized by their unpredictable nature, which can either promote self-directed learning or present challenges in offering structured learning opportunities for students [ 12 ]. Consequently, designing purposive feedback opportunities within WBL settings is a significant challenge for clinical teachers and faculty.

In undergraduate clinical education, feedback opportunities are often constrained due to the emphasis on clinical work and the absence of dedicated time for teaching [ 13 ]. Students are expected to perform autonomously under supervision, ideally achieved by giving them space to practice progressively and providing continuous instances of constructive feedback [ 14 ]. However, the hierarchy often present in clinical settings places undergraduate students in a dependent position, below residents and specialists [ 15 ]. Undergraduate or junior students may have different approaches to receiving and using feedback. If their priority is meeting the minimum standards given pass-fail consequences and acting merely as feedback recipients, other incentives may be needed to engage with the feedback processes because they will need more learning support [ 16 , 17 ]. Adequate supervision and feedback have been recognized as vital educational support in encouraging students to adopt a constructive learning approach [ 18 ]. Given that productive feedback processes rely on observed information regarding a student's performance, it is imperative to establish structured teaching and learning feedback activities within undergraduate WBL settings.

Despite the extensive research on feedback, a significant proportion of published studies involve residents or postgraduate students [ 19 , 20 ]. Recent reviews focusing on feedback interventions within medical education have clearly distinguished between undergraduate medical students and residents or fellows [ 21 ]. To gain a comprehensive understanding of initiatives related to actionable feedback in the WBL environment for undergraduate health professions, a scoping review of the existing literature could provide insight into how feedback processes are designed in that context. Accordingly, the present scoping review aims to answer the following research question: How are the feedback processes designed in the undergraduate health professions' workplace-based learning environments?

A scoping review was conducted using the five-step methodological framework proposed by Arksey and O'Malley (2005) [ 22 ], intertwined with the PRISMA checklist extension for scoping reviews to provide reporting guidance for this specific type of knowledge synthesis [ 23 ]. Scoping reviews allow us to study the literature without restricting the methodological quality of the studies found, systematically and comprehensively map the literature, and identify gaps [ 24 ]. Furthermore, a scoping review was used because this topic is not suitable for a systematic review due to the varied approaches described and the large difference in the methodologies used [ 21 ].

Search strategy

With the collaboration of a medical librarian, the authors used the research question to guide the search strategy. An initial meeting was held to define keywords and search resources. The proposed search strategy was reviewed by the research team, and then the study selection was conducted in two steps:

An online database search included Medline/PubMed, Web of Science, CINAHL, Cochrane Library, Embase, ERIC, and PsycINFO.

A directed search of ten relevant journals in the health sciences education field (Academic Medicine, Medical Education, Advances in Health Sciences Education, Medical Teacher, Teaching and Learning in Medicine, Journal of Surgical Education, BMC Medical Education, Medical Education Online, Perspectives on Medical Education and The Clinical Teacher) was performed.

The research team conducted a pilot or initial search before the full search to identify if the topic was susceptible to a scoping review. The full search was conducted in November 2022. One team member (MO) identified the papers in the databases. JF searched in the selected journals. Authors included studies written in English due to feasibility issues, with no time span limitation. After eliminating duplicates, two research team members (JF and IV) independently reviewed all the titles and abstracts using the exclusion and inclusion criteria described in Table  2 and with the support of the screening application StArT [ 25 ]. A third team member (AR) reviewed the titles and abstracts when the first two disagreed. The reviewer team met again at a midpoint and final stage to discuss the challenges related to study selection. Articles included for full-text review were exported to Mendeley. JF independently screened all full-text papers, and AR verified 10% for inclusion. The authors did not analyze study quality or risk of bias during study selection, which is consistent with conducting a scoping review.

The analysis of the results incorporated a descriptive summary and a thematic analysis, which was carried out to clarify and give consistency to the results' reporting [ 22 , 24 , 26 ]. Quantitative data were analyzed to report the characteristics of the studies, populations, settings, methods, and outcomes. Qualitative data were labeled, coded, and categorized into themes by three team members (JF, SH, and DS). The feedback loop framework with a sociocultural perspective was used as the theoretical framework to analyze the results.

The keywords used for the search strategies were as follows:

Clinical clerkship; feedback; formative feedback; health professions; undergraduate medical education; workplace.

Definitions of the keywords used for the present review are available in Appendix 1 .

As an example, we included the search strategy that we used in the Medline/PubMed database when conducting the full search:

("Formative Feedback"[Mesh] OR feedback) AND ("Workplace"[Mesh] OR workplace OR "Clinical Clerkship"[Mesh] OR clerkship) AND (("Education, Medical, Undergraduate"[Mesh] OR undergraduate health profession*) OR (learner* medical education)).

Inclusion and exclusion criteria

The following inclusion and exclusion criteria were used (Table  1 ):

Data extraction

The research group developed a data-charting form to organize the information obtained from the studies. The process was iterative, as the data chart was continuously reviewed and improved as necessary. In addition, following Levac et al.'s recommendation (2010), the three members involved in the charting process (JF, LI, and IV) independently reviewed the first five selected studies to determine whether the data extraction was consistent with the objectives of this scoping review and to ensure consistency. Then, the team met using web-conferencing software (Zoom; CA, USA) to review the results and adjust any details in the chart. The same three members extracted data independently from all the selected studies, considering two members reviewing each paper [ 26 ]. A third team member was consulted if any conflict occurred when extracting data. The data chart identified demographic patterns and facilitated the data synthesis. To organize data, we used a shared Excel spreadsheet, considering the following headings: title, author(s), year of publication, journal/source, country/origin, aim of the study, research question (if any), population/sample size, participants, discipline, setting, methodology, study design, data collection, data analysis, intervention, outcomes, outcomes measure, key findings, and relation of findings to research question.

Additionally, all the included papers were uploaded to AtlasTi v19 to facilitate the qualitative analysis. Three team members (JF, SH, and DS) independently coded the first six papers to create a list of codes to ensure consistency and rigor. The group met several times to discuss and refine the list of codes. Then, one member of the team (JF) used the code list to code all the rest of the papers. Once all papers were coded, the team organized codes into descriptive themes aligned with the research question.

Preliminary results were shared with a number of stakeholders (six clinical teachers, ten students, six medical educators) to elicit their opinions as an opportunity to build on the evidence and offer a greater level of meaning, content expertise, and perspective to the preliminary findings [ 26 ]. No quality appraisal of the studies is considered for this scoping review, which aligns with the frameworks for guiding scoping reviews [ 27 ].

The datasets analyzed during the current study are available from the corresponding author upon request.

A database search resulted in 3,597 papers, and the directed search of the most relevant journals in the health sciences education field yielded 2,096 titles. An example of the results of one database is available in Appendix 2 . Of the titles obtained, 816 duplicates were eliminated, and the team reviewed the titles and abstracts of 4,877 papers. Of these, 120 were selected for full-text review. Finally, 61 papers were included in this scoping review (Fig.  1 ), as listed in Table  2 .

PRISMA flow diagram for included studies, incorporating records identified through the database and direct searching

The selected studies were published between 1986 and 2022, and seventy-five percent (46) were published during the last decade. Of all the articles included in this review, 13% (8) were literature reviews: one integrative review [ 28 ] and four scoping reviews [ 29 , 30 , 31 , 32 ]. Finally, fifty-three (87%) original or empirical papers were included (i.e., studies that answered a research question or achieved a research purpose through qualitative or quantitative methodologies) [ 15 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 , 60 , 61 , 62 , 63 , 64 , 65 , 66 , 67 , 68 , 69 , 70 , 71 , 72 , 73 , 74 , 75 , 76 , 77 , 78 , 79 , 80 , 81 , 82 , 83 , 84 , 85 ].

Table 2 summarizes the papers included in the present scoping review, and Table  3 describes the characteristics of the included studies.

The thematic analysis resulted in two themes: (1) the organization of feedback processes in WBL settings, and (2) sociocultural factors influencing the organization of feedback processes. Table 4 gives a summary of the themes and subthemes.

Organization of feedback processes in WBL settings.

Setting learning goals (i.e., feed-up dimension).

Feedback that focuses on students' learning needs and is based on known performance standards enhances student response and setting learning goals [ 30 ]. Discussing goals and agreements before starting clinical practice enhances students' feedback-seeking behavior [ 39 ] and responsiveness to feedback [ 83 ]. Farrell et al. (2017) found that teacher-learner co-constructed learning goals enhance feedback interactions and help establish educational alliances, improving the learning experience [ 50 ]. However, Kiger (2020) found that sharing individualized learning plans with teachers aligned feedback with learning goals but did not improve students' perceived use of feedback [ 64 ]

Two papers of this set pointed out the importance of goal-oriented feedback, a dynamic process that depends on discussion of goal setting between teachers and students [ 50 ] and influences how individuals experience, approach, and respond to upcoming learning activities [ 34 ]. Goal-oriented feedback should be embedded in the learning experience of the clinical workplace, as it can enhance students' engagement in safe feedback dialogues [ 50 ]. Ideally, each feedback encounter in the WBL context should conclude, in addition to setting a plan of action to achieve the desired goal, with a reflection on the next goal [ 50 ].

Feedback strategies within the WBL environment. (i.e., feedback dimension)

In undergraduate WBL environments, there are several tasks and feedback opportunities organized in the undergraduate clinical workplace that can enable feedback processes:

Questions from clinical teachers to students are a feedback strategy [ 74 ]. There are different types of questions that the teacher can use, either to clarify concepts, to reach the correct answer, or to facilitate self-correction [ 74 ]. Usually, questions can be used in conjunction with other communication strategies, such as pauses, which enable self-correction by the student [ 74 ]. Students can also ask questions to obtain feedback on their performance [ 54 ]. However, question-and-answer as a feedback strategy usually provides information on either correct or incorrect answers and fewer suggestions for improvement, rendering it less constructive as a feedback strategy [ 82 ].

Direct observation of performance by default is needed to be able to provide information to be used as input in the feedback process [ 33 , 46 , 49 , 86 ]. In the process of observation, teachers can include clarification of objectives (i.e., feed-up dimension) and suggestions for an action plan (i.e., feedforward) [ 50 ]. Accordingly, Schopper et al. (2016) showed that students valued being observed while interviewing patients, as they received feedback that helped them become more efficient and effective as interviewers and communicators [ 33 ]. Moreover, it is widely described that direct observation improves feedback credibility [ 33 , 40 , 84 ]. Ideally, observation should be deliberate [ 33 , 83 ], informal or spontaneous [ 33 ], conducted by a (clinical) expert [ 46 , 86 ], provided immediately after the observation, and clinical teacher if possible, should schedule or be alert on follow-up observations to promote closing the gap between current and desired performance [ 46 ].

Workplace-based assessments (WBAs), by definition, entail direct observation of performance during authentic task demonstration [ 39 , 46 , 56 , 87 ]. WBAs can significantly impact behavioral change in medical students [ 55 ]. Organizing and designing formative WBAs and embedding these in a feedback dialogue is essential for effective learning [ 31 ].

Summative organization of WBAs is a well described barrier for feedback uptake in the clinical workplace [ 35 , 46 ]. If feedback is perceived as summative, or organized as a pass-fail decision, students may be less inclined to use the feedback for future learning [ 52 ]. According to Schopper et al. (2016), using a scale within a WBA makes students shift their focus during the clinical interaction and see it as an assessment with consequences [ 33 ]. Harrison et al. (2016) pointed out that an environment that only contains assessments with a summative purpose will not lead to a culture of learning and improving performance [ 56 ]. The recommendation is to separate the formative and summative WBAs, as feedback in summative instances is often not recognized as a learning opportunity or an instance to seek feedback [ 54 ]. In terms of the design, an organizational format is needed to clarify to students how formative assessments can promote learning from feedback [ 56 ]. Harrison et al. (2016) identified that enabling students to have more control over their assessments, designing authentic assessments, and facilitating long-term mentoring could improve receptivity to formative assessment feedback [ 56 ].

Multiple WBA instruments and systems are reported in the literature. Sox et al. (2014) used a detailed evaluation form to help students improve their clinical case presentation skills. They found that feedback on oral presentations provided by supervisors using a detailed evaluation form improved clerkship students’ oral presentation skills [ 78 ]. Daelmans et al. (2006) suggested that a formal in-training assessment programme composed by 19 assessments that provided structured feedback, could promote observation and verbal feedback opportunities through frequent assessments [ 43 ]. However, in this setting, limited student-staff interactions still hindered feedback follow-up [ 43 ]. Designing frequent WBA improves feedback credibility [ 28 ]. Long et al. (2021) emphasized that students' responsiveness to assessment feedback hinges on its perceived credibility, underlining the importance of credibility for students to effectively engage and improve their performance [ 31 ].

The mini-CEX is one of the most widely described WBA instruments in the literature. Students perceive that the mini-CEX allows them to be observed and encourages the development of interviewing skills [ 33 ]. The mini-CEX can provide feedback that improves students' clinical skills [ 58 , 60 ], as it incorporates a structure for discussing the student's strengths and weaknesses and the design of a written action plan [ 39 , 80 ]. When mini-CEXs are incorporated as part of a system of WBA, such as programmatic assessment, students feel confident in seeking feedback after observation, and being systematic allows for follow-up [ 39 ]. Students suggested separating grading from observation and using the mini-CEX in more informal situations [ 33 ].

Clinical encounter cards allow students to receive weekly feedback and make them request more feedback as the clerkship progresses [ 65 ]. Moreover, encounter cards stimulate that feedback is given by supervisors, and students are more satisfied with the feedback process [ 72 ]. With encounter card feedback, students are responsible for asking a supervisor for feedback before a clinical encounter, and supervisors give students written and verbal comments about their performance after the encounter [ 42 , 72 ]. Encounter cards enhance the use of feedback and add approximately one minute to the length of the clinical encounter, so they are well accepted by students and supervisors [ 72 ]. Bennett (2006) identified that Instant Feedback Cards (IFC) facilitated mid-rotation feedback [ 38 ]. Feedback encounter card comments must be discussed between students and supervisors; otherwise, students may perceive it as impersonal, static, formulaic, and incomplete [ 59 ].

Self-assessments can change students' feedback orientation, transforming them into coproducers of learning [ 68 ]. Self-assessments promote the feedback process [ 68 ]. Some articles emphasize the importance of organizing self-assessments before receiving feedback from supervisors, for example, discussing their appraisal with the supervisor [ 46 , 52 ]. In designing a feedback encounter, starting with a self-assessment as feed-up, discussing with the supervisor, and identifying areas for improvement is recommended, as part of the feedback dialogue [ 68 ].

Peer feedback as an organized activity allows students to develop strategies to observe and give feedback to other peers [ 61 ]. Students can act as the feedback provider or receiver, fostering understanding of critical comments and promoting evaluative judgment for their clinical practice [ 61 ]. Within clerkships, enabling the sharing of feedback information among peers allows for a better understanding and acceptance of feedback [ 52 ]. However, students can find it challenging to take on the peer assessor/feedback provider role, as they prefer to avoid social conflicts [ 28 , 61 ]. Moreover, it has been described that they do not trust the judgment of their peers because they are not experts, although they know the procedures, tasks, and steps well and empathize with their peer status in the learning process [ 61 ].

Bedside-teaching encounters (BTEs) provide timely feedback and are an opportunity for verbal feedback during performance [ 74 ]. Rizan et al. (2014) explored timely feedback delivered within BTEs and determined that it promotes interaction that constructively enhances learner development through various corrective strategies (e.g., question and answers, pauses, etc.). However, if the feedback given during the BTEs was general, unspecific, or open-ended, it could go unnoticed [ 74 ]. Torre et al. (2005) investigated which integrated feedback activities and clinical tasks occurred on clerkship rotations and assessed students' perceived quality in each teaching encounter [ 81 ]. The feedback activities reported were feedback on written clinical history, physical examination, differential diagnosis, oral case presentation, a daily progress note, and bedside feedback. Students considered all these feedback activities high-quality learning opportunities, but they were more likely to receive feedback when teaching was at the bedside than at other teaching locations [ 81 ].

Case presentations are an opportunity for feedback within WBL contexts [ 67 , 73 ]. However, both students and supervisors struggled to identify them as feedback moments, and they often dismissed questions and clarifications around case presentations as feedback [ 73 ]. Joshi (2017) identified case presentations as a way for students to ask for informal or spontaneous supervisor feedback [ 63 ].

Organization of follow-up feedback and action plans (i.e., feedforward dimension).

Feedback that generates use and response from students is characterized by two-way communication and embedded in a dialogue [ 30 ]. Feedback must be future-focused [ 29 ], and a feedback encounter should be followed by planning the next observation [ 46 , 87 ]. Follow-up feedback could be organized as a future self-assessment, reflective practice by the student, and/or a discussion with the supervisor or coach [ 68 ]. The literature describes that a lack of student interaction with teachers makes follow-up difficult [ 43 ]. According to Haffling et al. (2011), follow-up feedback sessions improve students' satisfaction with feedback compared to students who do not have follow-up sessions. In addition, these same authors reported that a second follow-up session allows verification of improved performances or confirmation that the skill was acquired [ 55 ].

Although feedback encounter forms are a recognized way of obtaining information about performance (i.e., feedback dimension), the literature does not provide many clear examples of how they may impact the feedforward phase. For example, Joshi et al. (2016) consider a feedback form with four fields (i.e., what did you do well, advise the student on what could be done to improve performance, indicate the level of proficiency, and personal details of the tutor). In this case, the supervisor highlighted what the student could improve but not how, which is the missing phase of the co-constructed action plan [ 63 ]. Whichever WBA instrument is used in clerkships to provide feedback, it should include a "next steps" box [ 44 ], and it is recommended to organize a long-term use of the WBA instrument so that those involved get used to it and improve interaction and feedback uptake [ 55 ]. RIME-based feedback (Reporting, Interpreting, Managing, Educating) is considered an interesting example, as it is perceived as helpful to students in knowing what they need to improve in their performance [ 44 ]. Hochberg (2017) implemented formative mid-clerkship assessments to enhance face-to-face feedback conversations and co-create an improvement plan [ 59 ]. Apps for structuring and storing feedback improve the amount of verbal and written feedback. In the study of Joshi et al. (2016), a reasonable proportion of students (64%) perceived that these app tools help them improve their performance during rotations [ 63 ].

Several studies indicate that an action plan as part of the follow-up feedback is essential for performance improvement and learning [ 46 , 55 , 60 ]. An action plan corresponds to an agreed-upon strategy for improving, confirming, or correcting performance. Bing-You et al. (2017) determined that only 12% of the articles included in their scoping review incorporated an action plan for learners [ 32 ]. Holmboe et al. (2004) reported that only 11% of the feedback sessions following a mini-CEX included an action plan [ 60 ]. Suhoyo et al. (2017) also reported that only 55% of mini-CEX encounters contained an action plan [ 80 ]. Other authors reported that action plans are not commonly offered during feedback encounters [ 77 ]. Sokol-Hessner et al. (2010) implemented feedback card comments with a space to provide written feedback and a specific action plan. In their results, 96% contained positive comments, and only 5% contained constructive comments [ 77 ]. In summary, although the recommendation is to include a “next step” box in the feedback instruments, evidence shows these items are not often used for constructive comments or action plans.

Sociocultural factors influencing the organization of feedback processes.

Multiple sociocultural factors influence interaction in feedback encounters, promoting or hampering the productivity of the feedback processes.

Clinical learning culture

Context impacts feedback processes [ 30 , 82 ], and there are barriers to incorporating actionable feedback in the clinical learning context. The clinical learning culture is partly determined by the clinical context, which can be unpredictable [ 29 , 46 , 68 ], as the available patients determine learning opportunities. Supervisors are occupied by a high workload, which results in limited time or priority for teaching [ 35 , 46 , 48 , 55 , 68 , 83 ], hindering students’ feedback-seeking behavior [ 54 ], and creating a challenge for the balance between patient care and student mentoring [ 35 ].

Clinical workplace culture does not always purposefully prioritize instances for feedback processes [ 83 , 84 ]. This often leads to limited direct observation [ 55 , 68 ] and the provision of poorly informed feedback. It is also evident that this affects trust between clinical teachers and students [ 52 ]. Supervisors consider feedback a low priority in clinical contexts [ 35 ] due to low compensation and lack of protected time [ 83 ]. In particular, lack of time appears to be the most significant and well-known barrier to frequent observation and workplace feedback [ 35 , 43 , 48 , 62 , 67 , 83 ].

The clinical environment is hierarchical [ 68 , 80 ] and can make students not consider themselves part of the team and feel like a burden to their supervisor [ 68 ]. This hierarchical learning environment can lead to unidirectional feedback, limit dialogue during feedback processes, and hinder the seeking, uptake, and use of feedback [ 67 , 68 ]. In a learning culture where feedback is not supported, learners are less likely to want to seek it and feel motivated and engaged in their learning [ 83 ]. Furthermore, it has been identified that clinical supervisors lack the motivation to teach [ 48 ] and the intention to observe or reobserve performance [ 86 ].

In summary, the clinical context and WBL culture do not fully use the potential of a feedback process aimed at closing learning gaps. However, concrete actions shown in the literature can be taken to improve the effectiveness of feedback by organizing the learning context. For example, McGinness et al. (2022) identified that students felt more receptive to feedback when working in a safe, nonjudgmental environment [ 67 ]. Moreover, supervisors and trainees identified the learning culture as key to establishing an open feedback dialogue [ 73 ]. Students who perceive culture as supportive and formative can feel more comfortable performing tasks and more willing to receive feedback [ 73 ].

Relationships

There is a consensus in the literature that trusting and long-term relationships improve the chances of actionable feedback. However, relationships between supervisors and students in the clinical workplace are often brief and not organized as more longitudinally [ 68 , 83 ], leaving little time to establish a trustful relationship [ 68 ]. Supervisors change continuously, resulting in short interactions that limit the creation of lasting relationships over time [ 50 , 68 , 83 ]. In some contexts, it is common for a student to have several supervisors who have their own standards in the observation of performance [ 46 , 56 , 68 , 83 ]. A lack of stable relationships results in students having little engagement in feedback [ 68 ]. Furthermore, in case of summative assessment programmes, the dual role of supervisors (i.e., assessing and giving feedback) makes feedback interactions perceived as summative and can complicate the relationship [ 83 ].

Repeatedly, the articles considered in this review describe that long-term and stable relationships enable the development of trust and respect [ 35 , 62 ] and foster feedback-seeking behavior [ 35 , 67 ] and feedback-giver behavior [ 39 ]. Moreover, constructive and positive relationships enhance students´ use of and response to feedback [ 30 ]. For example, Longitudinal Integrated Clerkships (LICs) promote stable relationships, thus enhancing the impact of feedback [ 83 ]. In a long-term trusting relationship, feedback can be straightforward and credible [ 87 ], there are more opportunities for student observation, and the likelihood of follow-up and actionable feedback improves [ 83 ]. Johnson et al. (2020) pointed out that within a clinical teacher-student relationship, the focus must be on establishing psychological safety; thus, the feedback conversations might be transformed [ 62 ].

Stable relationships enhance feedback dialogues, which offer an opportunity to co-construct learning and propose and negotiate aspects of the design of learning strategies [ 62 ].

Students as active agents in the feedback processes

The feedback response learners generate depends on the type of feedback information they receive, how credible the source of feedback information is, the relationship between the receiver and the giver, and the relevance of the information delivered [ 49 ]. Garino (2020) noted that students who are most successful in using feedback are those who do not take criticism personally, who understand what they need to improve and know they can do so, who value and feel meaning in criticism, are not surprised to receive it, and who are motivated to seek new feedback and use effective learning strategies [ 52 ]. Successful users of feedback ask others for help, are intentional about their learning, know what resources to use and when to use them, listen to and understand a message, value advice, and use effective learning strategies. They regulate their emotions, find meaning in the message, and are willing to change [ 52 ].

Student self-efficacy influences the understanding and use of feedback in the clinical workplace. McGinness et al. (2022) described various positive examples of self-efficacy regarding feedback processes: planning feedback meetings with teachers, fostering good relationships with the clinical team, demonstrating interest in assigned tasks, persisting in seeking feedback despite the patient workload, and taking advantage of opportunities for feedback, e.g., case presentations [ 67 ].

When students are encouraged to seek feedback aligned with their own learning objectives, they promote feedback information specific to what they want to learn and improve and enhance the use of feedback [ 53 ]. McGinness et al. (2022) identified that the perceived relevance of feedback information influenced the use of feedback because students were more likely to ask for feedback if they perceived that the information was useful to them. For example, if students feel part of the clinical team and participate in patient care, they are more likely to seek feedback [ 17 ].

Learning-oriented students aim to seek feedback to achieve clinical competence at the expected level [ 75 ]; they focus on improving their knowledge and skills and on professional development [ 17 ]. Performance-oriented students aim not to fail and to avoid negative feedback [ 17 , 75 ].

For effective feedback processes, including feed-up, feedback, and feedforward, the student must be feedback-oriented, i.e., active, seeking, listening to, interpreting, and acting on feedback [ 68 ]. The literature shows that feedback-oriented students are coproducers of learning [ 68 ] and are more involved in the feedback process [ 51 ]. Additionally, students who are metacognitively aware of their learning process are more likely to use feedback to reduce gaps in learning and performance [ 52 ]. For this, students must recognize feedback when it occurs and understand it when they receive it. Thus, it is important to organize training and promote feedback literacy so that students understand what feedback is, act on it, and improve the quality of feedback and their learning plans [ 68 ].

Table 5 summarizes those feedback tasks, activities, and key features of organizational aspects that enable each phase of the feedback loop based on the literature review.

The present scoping review identified 61 papers that mapped the literature on feedback processes in the WBL environments of undergraduate health professions. This review explored how feedback processes are organized in these learning contexts using the feedback loop framework. Given the specific characteristics of feedback processes in undergraduate clinical learning, three main findings were identified on how feedback processes are being conducted in the clinical environment and how these processes could be organized to support feedback processes.

First, the literature lacks a balance between the three dimensions of the feedback loop. In this regard, most of the articles in this review focused on reporting experiences or strategies for delivering feedback information (i.e., feedback dimension). Credible and objective feedback information is based on direct observation [ 46 ] and occurs within an interaction or a dialogue [ 62 , 88 ]. However, only having credible and objective information does not ensure that it will be considered, understood, used, and put into practice by the student [ 89 ].

Feedback-supporting actions aligned with goals and priorities facilitate effective feedback processes [ 89 ] because goal-oriented feedback focuses on students' learning needs [ 7 ]. In contrast, this review showed that only a minority of the studies highlighted the importance of aligning learning objectives and feedback (i.e., the feed-up dimension). To overcome this, supervisors and students must establish goals and agreements before starting clinical practice, as it allows students to measure themselves on a defined basis [ 90 , 91 ] and enhances students' feedback-seeking behavior [ 39 , 92 ] and responsiveness to feedback [ 83 ]. In addition, learning goals should be shared, and co-constructed, through a dialogue [ 50 , 88 , 90 , 92 ]. In fact, relationship-based feedback models emphasize setting shared goals and plans as part of the feedback process [ 68 ].

Many of the studies acknowledge the importance of establishing an action plan and promoting the use of feedback (i.e., feedforward). However, there is yet limited insight on how to best implement strategies that support the use of action plans, improve performance and close learning gaps. In this regard, it is described that delivering feedback without perceiving changes, results in no effect or impact on learning [ 88 ]. To determine if a feedback loop is closed, observing a change in the student's response is necessary. In other words, feedback does not work without repeating the same task [ 68 ], so teachers need to observe subsequent tasks to notice changes [ 88 ]. While feedforward is fundamental to long-term performance, it is shown that more research is needed to determine effective actions to be implemented in the WBL environment to close feedback loops.

Second, there is a need for more knowledge about designing feedback activities in the WBL environment that will generate constructive feedback for learning. WBA is the most frequently reported feedback activity in clinical workplace contexts [ 39 , 46 , 56 , 87 ]. Despite the efforts of some authors to use WBAs as a formative assessment and feedback opportunity, in several studies, a summative component of the WBA was presented as a barrier to actionable feedback [ 33 , 56 ]. Students suggest separating grading from observation and using, for example, the mini-CEX in informal situations [ 33 ]. Several authors also recommend disconnecting the summative components of WBAs to avoid generating emotions that can limit the uptake and use of feedback [ 28 , 93 ]. Other literature recommends purposefully designing a system of assessment using low-stakes data points for feedback and learning. Accordingly, programmatic assessment is a framework that combines both the learning and the decision-making function of assessment [ 94 , 95 ]. Programmatic assessment is a practical approach for implementing low-stakes as a continuum, giving opportunities to close the gap between current and desired performance and having the student as an active agent [ 96 ]. This approach enables the incorporation of low-stakes data points that target student learning [ 93 ] and provide performance-relevant information (i.e., meaningful feedback) based on direct observations during authentic professional activities [ 46 ]. Using low-stakes data points, learners make sense of information about their performance and use it to enhance the quality of their work or performance [ 96 , 97 , 98 ]. Implementing multiple instances of feedback is more effective than providing it once because it promotes closing feedback loops by giving the student opportunities to understand the feedback, make changes, and see if those changes were effective [ 89 ].

Third, the support provided by the teacher is fundamental and should be built into a reliable and long-term relationship, where the teacher must take the role of coach rather than assessor, and students should develop feedback agency and be active in seeking and using feedback to improve performance. Although it is recognized that institutional efforts over the past decades have focused on training teachers to deliver feedback, clinical supervisors' lack of teaching skills is still identified as a barrier to workplace feedback [ 99 ]. In particular, research indicates that clinical teachers lack the skills to transform the information obtained from an observation into constructive feedback [ 100 ]. Students are more likely to use feedback if they consider it credible and constructive [ 93 ] and based on stable relationships [ 93 , 99 , 101 ]. In trusting relationships, feedback can be straightforward and credible, and the likelihood of follow-up and actionable feedback improves [ 83 , 88 ]. Coaching strategies can be enhanced by teachers building an educational alliance that allows for trustworthy relationships or having supervisors with an exclusive coaching role [ 14 , 93 , 102 ].

Last, from a sociocultural perspective, individuals are the main actors in the learning process. Therefore, feedback impacts learning only if students engage and interact with it [ 11 ]. Thus, feedback design and student agency appear to be the main features of effective feedback processes. Accordingly, the present review identified that feedback design is a key feature for effective learning in complex environments such as WBL. Feedback in the workplace must ideally be organized and implemented to align learning outcomes, learning activities, and assessments, allowing learners to learn, practice, and close feedback loops [ 88 ]. To guide students toward performances that reflect long-term learning, an intensive formative learning phase is needed, in which multiple feedback processes are included that shape students´ further learning [ 103 ]. This design would promote student uptake of feedback for subsequent performance [ 1 ].

Strengths and limitations

The strengths of this study are (1) the use of an established framework, the Arksey and O'Malley's framework [ 22 ]. We included the step of socializing the results with stakeholders, which allowed the team to better understand the results from another perspective and offer a realistic look. (2) Using the feedback loop as a theoretical framework strengthened the results and gave a more thorough explanation of the literature regarding feedback processes in the WBL context. (3) our team was diverse and included researchers from different disciplines as well as a librarian.

The present scoping review has several limitations. Although we adhered to the recommended protocols and methodologies, some relevant papers may have been omitted. The research team decided to select original studies and reviews of the literature for the present scoping review. This caused some articles, such as guidelines, perspectives, and narrative papers, to be excluded from the current study.

One of the inclusion criteria was a focus on undergraduate students. However, some papers that incorporated undergraduate and postgraduate participants were included, as these supported the results of this review. Most articles involved medical students. Although the authors did not limit the search to medicine, maybe some articles involving students from other health disciplines needed to be included, considering the search in other databases or journals.

The results give insight in how feedback could be organized within the clinical workplace to promote feedback processes. On a small scale, i.e., in the feedback encounter between a supervisor and a learner, feedback should be organized to allow for follow-up feedback, thus working on required learning and performance goals. On a larger level, i.e., in the clerkship programme or a placement rotation, feedback should be organized through appropriate planning of subsequent tasks and activities.

More insight is needed in designing a closed loop feedback process, in which specific attention is needed in effective feedforward practices. The feedback that stimulates further action and learning requires a safe and trustful work and learning environment. Understanding the relationship between an individual and his or her environment is a challenge for determining the impact of feedback and must be further investigated within clinical WBL environments. Aligning the dimensions of feed-up, feedback and feedforward includes careful attention to teachers’ and students’ feedback literacy to assure that students can act on feedback in a constructive way. In this line, how to develop students' feedback agency within these learning environments needs further research.

Boud D, Molloy E. Rethinking models of feedback for learning: The challenge of design. Assess Eval High Educ. 2013;38:698–712.

Article   Google Scholar  

Henderson M, Ajjawi R, Boud D, Molloy E. Identifying feedback that has impact. In: The Impact of Feedback in Higher Education. Springer International Publishing: Cham; 2019. p. 15–34.

Chapter   Google Scholar  

Winstone N, Carless D. Designing effective feedback processes in higher education: A learning-focused approach. 1st ed. New York: Routledge; 2020.

Google Scholar  

Ajjawi R, Boud D. Assessment & Evaluation in Higher Education Researching feedback dialogue: an interactional analysis approach. 2015. https://doi.org/10.1080/02602938.2015.1102863 .

Carless D. Feedback loops and the longer-term: towards feedback spirals. Assess Eval High Educ. 2019;44:705–14.

Sadler DR. Formative assessment and the design of instructional systems. Instr Sci. 1989;18:119–44.

Hattie J, Timperley H. The Power of Feedback The Meaning of Feedback. Rev Educ Res. 2007;77:81–112.

Zarrinabadi N, Rezazadeh M. Why only feedback? Including feed up and feed forward improves nonlinguistic aspects of L2 writing. Language Teaching Research. 2023;27(3):575–92.

Fisher D, Frey N. Feed up, back, forward. Educ Leadersh. 2009;67:20–5.

Reimann A, Sadler I, Sambell K. What’s in a word? Practices associated with ‘feedforward’ in higher education. Assessment evaluation in higher education. 2019;44:1279–90.

Esterhazy R. Re-conceptualizing Feedback Through a Sociocultural Lens. In: Henderson M, Ajjawi R, Boud D, Molloy E, editors. The Impact of Feedback in Higher Education. Cham: Palgrave Macmillan; 2019. https://doi.org/10.1007/978-3-030-25112-3_5 .

Bransen D, Govaerts MJB, Sluijsmans DMA, Driessen EW. Beyond the self: The role of co-regulation in medical students’ self-regulated learning. Med Educ. 2020;54:234–41.

Ramani S, Könings KD, Ginsburg S, Van Der Vleuten CP. Feedback Redefined: Principles and Practice. J Gen Intern Med. 2019;34:744–53.

Atkinson A, Watling CJ, Brand PL. Feedback and coaching. Eur J Pediatr. 2022;181(2):441–6.

Suhoyo Y, Schonrock-Adema J, Emilia O, Kuks JBM, Cohen-Schotanus JA. Clinical workplace learning: perceived learning value of individual and group feedback in a collectivistic culture. BMC Med Educ. 2018;18:79.

Bowen L, Marshall M, Murdoch-Eaton D. Medical Student Perceptions of Feedback and Feedback Behaviors Within the Context of the “Educational Alliance.” Acad Med. 2017;92:1303–12.

Bok HGJ, Teunissen PW, Spruijt A, Fokkema JPI, van Beukelen P, Jaarsma DADC, et al. Clarifying students’ feedback-seeking behaviour in clinical clerkships. Med Educ. 2013;47:282–91.

Al-Kadri HM, Al-Kadi MT, Van Der Vleuten CPM. Workplace-based assessment and students’ approaches to learning: A qualitative inquiry. Med Teach. 2013;35(SUPPL):1.

Dennis AA, Foy MJ, Monrouxe LV, Rees CE. Exploring trainer and trainee emotional talk in narratives about workplace-based feedback processes. Adv Health Sci Educ. 2018;23:75–93.

Watling C, LaDonna KA, Lingard L, Voyer S, Hatala R. ‘Sometimes the work just needs to be done’: socio-cultural influences on direct observation in medical training. Med Educ. 2016;50:1054–64.

Bing-You R, Hayes V, Varaklis K, Trowbridge R, Kemp H, McKelvy D. Feedback for Learners in Medical Education: What is Known? A Scoping Review Academic Medicine. 2017;92:1346–54.

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8:19–32.

Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Ann Intern Med. 2018;169:467–73.

Colquhoun HL, Levac D, O’brien KK, Straus S, Tricco AC, Perrier L, et al. Scoping reviews: time for clarity in definition methods and reporting. J Clin Epidemiol. 2014;67:1291–4.

StArt - State of Art through Systematic Review. 2013.

Levac D, Colquhoun H, O’Brien KK. Scoping studies: Advancing the methodology. Implementation Science. 2010;5:1–9.

Peters MDJ, BPharm CMGHK, Parker PMD, Soares CB. Guidance for conducting systematic scoping reviews. Int J Evid Based Healthc. 2015;13:141–6.

Bing-You R, Varaklis K, Hayes V, Trowbridge R, Kemp H, McKelvy D, et al. The Feedback Tango: An Integrative Review and Analysis of the Content of the Teacher-Learner Feedback Exchange. Acad Med. 2018;93:657–63.

Ossenberg C, Henderson A, Mitchell M. What attributes guide best practice for effective feedback? A scoping review. Adv Health Sci Educ. 2019;24:383–401.

Spooner M, Duane C, Uygur J, Smyth E, Marron B, Murphy PJ, et al. Self -regulatory learning theory as a lens on how undergraduate and postgraduate learners respond to feedback: A BEME scoping review: BEME Guide No. 66. Med Teach. 2022;44:3–18.

Long S, Rodriguez C, St-Onge C, Tellier PP, Torabi N, Young M. Factors affecting perceived credibility of assessment in medical education: A scoping review. Adv Health Sci Educ. 2022;27:229–62.

Bing-You R, Hayes V, Varaklis K, Trowbridge R, Kemp H, McKelvy D. Feedback for Learners in Medical Education: What is Known? A Scoping Review: Lippincott Williams and Wilkins; 2017.

Schopper H, Rosenbaum M, Axelson R. “I wish someone watched me interview:” medical student insight into observation and feedback as a method for teaching communication skills during the clinical years. BMC Med Educ. 2016;16:286.

Crommelinck M, Anseel F. Understanding and encouraging feedback-seeking behaviour: a literature review. Med Educ. 2013;47:232–41.

Adamson E, Kinga L, Foy L, McLeodb M, Traynor J, Watson W, et al. Feedback in clinical practice: Enhancing the students’ experience through action research. Nurse Educ Pract. 2018;31:48–53.

Al-Mously N, Nabil NM, Al-Babtain SA, et al. Undergraduate medical students’ perceptions on the quality of feedback received during clinical rotations. Med Teach. 2014;36(Supplement 1):S17-23.

Bates J, Konkin J, Suddards C, Dobson S, Pratt D. Student perceptions of assessment and feedback in longitudinal integrated clerkships. Med Educ. 2013;47:362–74.

Bennett AJ, Goldenhar LM, Stanford K. Utilization of a Formative Evaluation Card in a Psychiatry Clerkship. Acad Psychiatry. 2006;30:319–24.

Bok HG, Jaarsma DA, Spruijt A, Van Beukelen P, Van Der Vleuten CP, Teunissen PW, et al. Feedback-giving behaviour in performance evaluations during clinical clerkships. Med Teach. 2016;38:88–95.

Bok HG, Teunissen PW, Spruijt A, Fokkema JP, van Beukelen P, Jaarsma DA, et al. Clarifying students’ feedback-seeking behaviour in clinical clerkships. Med Educ. 2013;47:282–91.

Calleja P, Harvey T, Fox A, Carmichael M, et al. Feedback and clinical practice improvement: A tool to assist workplace supervisors and students. Nurse Educ Pract. 2016;17:167–73.

Carey EG, Wu C, Hur ES, Hasday SJ, Rosculet NP, Kemp MT, et al. Evaluation of Feedback Systems for the Third-Year Surgical Clerkship. J Surg Educ. 2017;74:787–93.

Daelmans HE, Overmeer RM, Van der Hem-Stokroos HH, Scherpbier AJ, Stehouwer CD, van der Vleuten CP. In-training assessment: qualitative study of effects on supervision and feedback in an undergraduate clinical rotation. Medical education. 2006;40(1):51–8.

DeWitt D, Carline J, Paauw D, Pangaro L. Pilot study of a ’RIME’-based tool for giving feedback in a multi-specialty longitudinal clerkship. Med Educ. 2008;42:1205–9.

Dolan BM, O’Brien CL, Green MM. Including Entrustment Language in an Assessment Form May Improve Constructive Feedback for Student Clinical Skills. Med Sci Educ. 2017;27:461–4.

Duijn CC, Welink LS, Mandoki M, Ten Cate OT, Kremer WD, Bok HG. Am I ready for it? Students’ perceptions of meaningful feedback on entrustable professional activities. Perspectives on medical education. 2017;6:256–64.

Elnicki DM, Zalenski D. Integrating medical students’ goals, self-assessment and preceptor feedback in an ambulatory clerkship. Teach Learn Med. 2013;25:285–91.

Embo MP, Driessen EW, Valcke M, Van der Vleuten CP. Assessment and feedback to facilitate self-directed learning in clinical practice of Midwifery students. Medical teacher. 2010;32(7):e263-9.

Eva KW, Armson H, Holmboe E, Lockyer J, Loney E, Mann K, et al. Factors influencing responsiveness to feedback: On the interplay between fear, confidence, and reasoning processes. Adv Health Sci Educ. 2012;17:15–26.

Farrell L, Bourgeois-Law G, Ajjawi R, Regehr G. An autoethnographic exploration of the use of goal oriented feedback to enhance brief clinical teaching encounters. Adv Health Sci Educ. 2017;22:91–104.

Fernando N, Cleland J, McKenzie H, Cassar K. Identifying the factors that determine feedback given to undergraduate medical students following formative mini-CEX assessments. Med Educ. 2008;42:89–95.

Garino A. Ready, willing and able: a model to explain successful use of feedback. Adv Health Sci Educ. 2020;25:337–61.

Garner MS, Gusberg RJ, Kim AW. The positive effect of immediate feedback on medical student education during the surgical clerkship. J Surg Educ. 2014;71:391–7.

Bing-You R, Hayes V, Palka T, Ford M, Trowbridge R. The Art (and Artifice) of Seeking Feedback: Clerkship Students’ Approaches to Asking for Feedback. Acad Med. 2018;93:1218–26.

Haffling AC, Beckman A, Edgren G. Structured feedback to undergraduate medical students: 3 years’ experience of an assessment tool. Medical teacher. 2011;33(7):e349-57.

Harrison CJ, Könings KD, Dannefer EF, Schuwirth LWTT, Wass V, van der Vleuten CPMM. Factors influencing students’ receptivity to formative feedback emerging from different assessment cultures. Perspect Med Educ. 2016;5:276–84.

Harrison CJ, Könings KD, Schuwirth LW, Wass V, Van der Vleuten CP, Konings KD, et al. Changing the culture of assessment: the dominance of the summative assessment paradigm. BMC medical education. 2017;17:1–4.

Harvey P, Radomski N, O’Connor D. Written feedback and continuity of learning in a geographically distributed medical education program. Medical teacher. 2013;35(12):1009–13.

Hochberg M, Berman R, Ogilvie J, Yingling S, Lee S, Pusic M, et al. Midclerkship feedback in the surgical clerkship: the “Professionalism, Reporting, Interpreting, Managing, Educating, and Procedural Skills” application utilizing learner self-assessment. Am J Surg. 2017;213:212–6.

Holmboe ES, Yepes M, Williams F, Huot SJ. Feedback and the mini clinical evaluation exercise. Journal of general internal medicine. 2004;19(5):558–61.

Tai JHM, Canny BJ, Haines TP, Molloy EK. The role of peer-assisted learning in building evaluative judgement: opportunities in clinical medical education. Adv Health Sci Educ. 2016;21:659–76.

Johnson CE, Keating JL, Molloy EK. Psychological safety in feedback: What does it look like and how can educators work with learners to foster it? Med Educ. 2020;54:559–70.

Joshi A, Generalla J, Thompson B, Haidet P. Facilitating the Feedback Process on a Clinical Clerkship Using a Smartphone Application. Acad Psychiatry. 2017;41:651–5.

Kiger ME, Riley C, Stolfi A, Morrison S, Burke A, Lockspeiser T. Use of Individualized Learning Plans to Facilitate Feedback Among Medical Students. Teach Learn Med. 2020;32:399–409.

Kogan J, Shea J. Implementing feedback cards in core clerkships. Med Educ. 2008;42:1071–9.

Lefroy J, Walters B, Molyneux A, Smithson S. Can learning from workplace feedback be enhanced by reflective writing? A realist evaluation in UK undergraduate medical education. Educ Prim Care. 2021;32:326–35.

McGinness HT, Caldwell PHY, Gunasekera H, Scott KM. ‘Every Human Interaction Requires a Bit of Give and Take’: Medical Students’ Approaches to Pursuing Feedback in the Clinical Setting. Teach Learn Med. 2022. https://doi.org/10.1080/10401334.2022.2084401 .

Noble C, Billett S, Armit L, Collier L, Hilder J, Sly C, et al. ``It’s yours to take{’’}: generating learner feedback literacy in the workplace. Adv Health Sci Educ. 2020;25:55–74.

Ogburn T, Espey E. The R-I-M-E method for evaluation of medical students on an obstetrics and gynecology clerkship. Am J Obstet Gynecol. 2003;189:666–9.

Po O, Reznik M, Greenberg L. Improving a medical student feedback with a clinical encounter card. Ambul Pediatr. 2007;7:449–52.

Parkes J, Abercrombie S, McCarty T, Parkes J, Abercrombie S, McCarty T. Feedback sandwiches affect perceptions but not performance. Adv Health Sci Educ. 2013;18:397–407.

Paukert JL, Richards ML, Olney C. An encounter card system for increasing feedback to students. Am J Surg. 2002;183:300–4.

Rassos J, Melvin LJ, Panisko D, Kulasegaram K, Kuper A. Unearthing Faculty and Trainee Perspectives of Feedback in Internal Medicine: the Oral Case Presentation as a Model. J Gen Intern Med. 2019;34:2107–13.

Rizan C, Elsey C, Lemon T, Grant A, Monrouxe L. Feedback in action within bedside teaching encounters: a video ethnographic study. Med Educ. 2014;48:902–20.

Robertson AC, Fowler LC. Medical student perceptions of learner-initiated feedback using a mobile web application. Journal of medical education and curricular development. 2017;4:2382120517746384.

Scheidt PC, Lazoritz S, Ebbeling WL, Figelman AR, Moessner HF, Singer JE. Evaluation of system providing feedback to students on videotaped patient encounters. J Med Educ. 1986;61(7):585–90.

Sokol-Hessner L, Shea J, Kogan J. The open-ended comment space for action plans on core clerkship students’ encounter cards: what gets written? Acad Med. 2010;85:S110–4.

Sox CM, Dell M, Phillipi CA, Cabral HJ, Vargas G, Lewin LO. Feedback on oral presentations during pediatric clerkships: a randomized controlled trial. Pediatrics. 2014;134:965–71.

Spickard A, Gigante J, Stein G, Denny JC. Automatic capture of student notes to augment mentor feedback and student performance on patient write-ups. J Gen Intern Med. 2008;23:979–84.

Suhoyo Y, Van Hell EA, Kerdijk W, Emilia O, Schönrock-Adema J, Kuks JB, et al. nfluence of feedback characteristics on perceived learning value of feedback in clerkships: does culture matter? BMC medical education. 2017;17:1–7.

Torre DM, Simpson D, Sebastian JL, Elnicki DM. Learning/feedback activities and high-quality teaching: perceptions of third-year medical students during an inpatient rotation. Acad Med. 2005;80:950–4.

Urquhart LM, Ker JS, Rees CE. Exploring the influence of context on feedback at medical school: a video-ethnography study. Adv Health Sci Educ. 2018;23:159–86.

Watling C, Driessen E, van der Vleuten C, Lingard L. Learning culture and feedback: an international study of medical athletes and musicians. Med Educ. 2014;48:713–23.

Watling C, Driessen E, van der Vleuten C, Vanstone M, Lingard L. Beyond individualism: Professional culture and its influence on feedback. Med Educ. 2013;47:585–94.

Soemantri D, Dodds A, Mccoll G. Examining the nature of feedback within the Mini Clinical Evaluation Exercise (Mini-CEX): an analysis of 1427 Mini-CEX assessment forms. GMS J Med Educ. 2018;35:Doc47.

Van De Ridder JMMM, Stokking KM, McGaghie WC, ten Cate OTJ, van der Ridder JM, Stokking KM, et al. What is feedback in clinical education? Med Educ. 2008;42:189–97.

van de Ridder JMMM, McGaghie WC, Stokking KM, ten Cate OTJJ. Variables that affect the process and outcome of feedback, relevant for medical training: a meta-review. Med Educ. 2015;49:658–73.

Boud D. Feedback: ensuring that it leads to enhanced learning. Clin Teach. 2015. https://doi.org/10.1111/tct.12345 .

Brehaut J, Colquhoun H, Eva K, Carrol K, Sales A, Michie S, et al. Practice feedback interventions: 15 suggestions for optimizing effectiveness. Ann Intern Med. 2016;164:435–41.

Ende J. Feedback in clinical medical education. J Am Med Assoc. 1983;250:777–81.

Cantillon P, Sargeant J. Giving feedback in clinical settings. Br Med J. 2008;337(7681):1292–4.

Norcini J, Burch V. Workplace-based assessment as an educational tool: AMEE Guide No. 31. Med Teach. 2007;29:855–71.

Watling CJ, Ginsburg S. Assessment, feedback and the alchemy of learning. Med Educ. 2019;53:76–85.

van der Vleuten CPM, Schuwirth LWT, Driessen EW, Dijkstra J, Tigelaar D, Baartman LKJ, et al. A model for programmatic assessment fit for purpose. Med Teach. 2012;34:205–14.

Schuwirth LWT, der Vleuten CPM. Programmatic assessment: from assessment of learning to assessment for learning. Med Teach. 2011;33:478–85.

Schut S, Driessen E, van Tartwijk J, van der Vleuten C, Heeneman S. Stakes in the eye of the beholder: an international study of learners’ perceptions within programmatic assessment. Med Educ. 2018;52:654–63.

Henderson M, Boud D, Molloy E, Dawson P, Phillips M, Ryan T, Mahoney MP. Feedback for learning. Closing the assessment loop. Framework for effective learning. Canberra, Australia: Australian Government, Department for Education and Training; 2018.

Heeneman S, Pool AO, Schuwirth LWT, van der Vleuten CPM, Driessen EW, Oudkerk A, et al. The impact of programmatic assessment on student learning: theory versus practice. Med Educ. 2015;49:487–98.

Lefroy J, Watling C, Teunissen P, Brand P, Watling C. Guidelines: the do’s, don’ts and don’t knows of feedback for clinical education. Perspect Med Educ. 2015;4:284–99.

Ramani S, Krackov SK. Twelve tips for giving feedback effectively in the clinical environment. Med Teach. 2012;34:787–91.

Telio S, Ajjawi R, Regehr G. The, “Educational Alliance” as a Framework for Reconceptualizing Feedback in Medical Education. Acad Med. 2015;90:609–14.

Lockyer J, Armson H, Könings KD, Lee-Krueger RC, des Ordons AR, Ramani S, et al. In-the-Moment Feedback and Coaching: Improving R2C2 for a New Context. J Grad Med Educ. 2020;12:27–35.

Black P, Wiliam D. Developing the theory of formative assessment. Educ Assess Eval Account. 2009;21:5–31.

Download references

Author information

Authors and affiliations.

Department of Health Sciences, Faculty of Medicine, Pontificia Universidad Católica de Chile, Avenida Vicuña Mackenna 4860, Macul, Santiago, Chile

Javiera Fuentes-Cimma & Ignacio Villagran

School of Health Professions Education, Maastricht University, Maastricht, Netherlands

Javiera Fuentes-Cimma & Lorena Isbej

Rotterdam University of Applied Sciences, Rotterdam, Netherlands

Dominique Sluijsmans

Centre for Medical and Health Profession Education, Department of Gastroenterology, Faculty of Medicine, Pontificia Universidad Católica de Chile, Santiago, Chile

Arnoldo Riquelme

School of Dentistry, Faculty of Medicine, Pontificia Universidad Católica de Chile, Santiago, Chile

Lorena Isbej

Sistema de Bibliotecas UC (SIBUC), Pontificia Universidad Católica de Chile, Santiago, Chile

María Teresa Olivares-Labbe

Department of Pathology, Faculty of Health, Medicine and Health Sciences, Maastricht University, Maastricht, Netherlands

Sylvia Heeneman

You can also search for this author in PubMed   Google Scholar

Contributions

J.F-C, D.S, and S.H. made substantial contributions to the conception and design of the work. M.O-L contributed to the identification of studies. J.F-C, I.V, A.R, and L.I. made substantial contributions to the screening, reliability, and data analysis. J.F-C. wrote th e main manuscript text. All authors reviewed the manuscript.

Corresponding author

Correspondence to Javiera Fuentes-Cimma .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1, supplementary material 2., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Fuentes-Cimma, J., Sluijsmans, D., Riquelme, A. et al. Designing feedback processes in the workplace-based learning of undergraduate health professions education: a scoping review. BMC Med Educ 24 , 440 (2024). https://doi.org/10.1186/s12909-024-05439-6

Download citation

Received : 25 September 2023

Accepted : 17 April 2024

Published : 23 April 2024

DOI : https://doi.org/10.1186/s12909-024-05439-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Clinical clerkship
• Feedback processes
• Feedforward
• Formative feedback
• Health professions
• Undergraduate medical education
• Undergraduate healthcare education
• Workplace learning

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Open access
• Published: 25 April 2024

A scoping review of academic and grey literature on migrant health research conducted in Scotland

• G. Petrie 1 ,
• K. Angus 2 &
• R. O’Donnell 2  

BMC Public Health volume  24 , Article number:  1156 ( 2024 ) Cite this article

151 Accesses

8 Altmetric

Metrics details

Migration to Scotland has increased since 2002 with an increase in European residents and participation in the Asylum dispersal scheme. Scotland has become more ethnically diverse, and 10% of the current population were born abroad. Migration and ethnicity are determinants of health, and information on the health status of migrants to Scotland and their access to and barriers to care facilitates the planning and delivery of equitable health services. This study aimed to scope existing peer-reviewed research and grey literature to identify gaps in evidence regarding the health of migrants in Scotland.

A scoping review on the health of migrants in Scotland was carried out for dates January 2002 to March 2023, inclusive of peer-reviewed journals and grey literature. CINAHL/ Web of Science/SocIndex and Medline databases were systematically searched along with government and third-sector websites. The searches identified 2166 journal articles and 170 grey literature documents for screening. Included articles were categorised according to the World Health Organisation’s 2016 Strategy and Action Plan for Refugee and Migrant Health in the European region. This approach builds on a previously published literature review on Migrant Health in the Republic of Ireland.

Seventy-one peer reviewed journal articles and 29 grey literature documents were included in the review. 66% were carried out from 2013 onwards and the majority focused on asylum seekers or unspecified migrant groups. Most research identified was on the World Health Organisation’s strategic areas of right to health of refugees, social determinants of health and public health planning and strengthening health systems. There were fewer studies on the strategic areas of frameworks for collaborative action, preventing communicable disease, preventing non-communicable disease, health screening and assessment and improving health information and communication.

While research on migrant health in Scotland has increased in recent years significant gaps remain. Future priorities should include studies of undocumented migrants, migrant workers, and additional research is required on the issue of improving health information and communication.

Peer Review reports

The term migrant is defined by the International Organisation for Migration as “ a person who moves away from his or her place of usual residence, whether within a country or across an international border, temporarily or permanently, and for a variety of reasons. The term includes several well-defined legal categories of people, including migrant workers; persons whose particular types of movements are legally-defined, such as smuggled migrants; as well as those whose status are not specifically defined under international law, such as international students.” [ 1 ] Internationally there are an estimated 281 million migrants – 3.6% of the world population, including 26.4 million refugees and 4.1 million asylum seekers – the highest number ever recorded [ 2 ]. The UN Refugee Society defines the term refugee as “ someone who has been forced to flee his or her country because of persecution, war or violence…most likely, they cannot return home or are afraid to do so .” The term asylum-seeker is defined as “someone whose request for sanctuary has yet to be processed.” [ 3 ].

Net-migration to Europe was negative in the 19th century due to higher levels of emigration, however in the mid-20th century immigration began to rise, because of an increase in migrant workers and following conflicts in the Middle East and North Africa [ 4 ]. Current migration drivers include conflicts alongside world-wide economic instability, exacerbated by the Covid-19 pandemic [ 5 ]. Environmental damage due to climate change is expected to inflate the number of asylum seekers entering Europe in future [ 6 ]. The increase in migration to Europe is not a short-term influx but a long-term phenomenon, and European nations must adapt and find solutions to resulting financial, safeguarding and health challenges [ 7 ].

Data on healthcare use by migrants in Europe is variable, which means cross-country comparisons are inadequate [ 8 ]. Many countries do not record migration information within health records and all use disparate criteria to classify migrant status. The lack of comparative data hinders public health surveillance and effective interventions [ 9 ]. Even where information is available, results can be contradictory due to the multifarious migrant population. Migrants have a wide range of origin countries, socio-economic position, age and journeys undertaken which can affect health status [ 10 ].

Migrants initially may have better health than the general population, known as the ‘Healthy Migrant effect’ [ 11 ]. However, health declines with increasing length of residence [ 12 ] and over time to levels comparable with the general population [ 13 ]. Second generation immigrants may have higher mortality than average [ 14 ]. The process of acculturation to the host country, with adoption of unhealthy lifestyle and behaviours, increases the risk for chronic disease [ 15 ]. In addition, inequalities in health of migrants compared to host populations has been confirmed by wide-ranging research [ 16 ].

Host countries may limit healthcare access, with undocumented migrants sometimes only entitled to emergency care [ 17 ]. Even when access is granted, inequitable services can affect quality of care due to language barriers and cultural factors [ 18 ]. Poor working/living conditions and discrimination can exacerbate health inequalities [ 12 ]. Processing facilities for asylum seekers are frequently overpopulated, stressful environments [ 19 ] and threat of deportation, lack of citizenship rights and integration can negatively affect health and access to care [ 20 ]. Undocumented workers are unprotected by health and safety legislation leading to dangerous working conditions and injuries [ 15 ].

A systematic review of migrant health in the European Union (EU) found migrants have worse self-perceived health than the general population [ 21 ]. Research evidence indicates increased prevalence of cardiovascular disease, diabetes, mental health disorders and adverse pregnancy outcomes. Exposure to conflict, harsh travel conditions and suboptimal vaccine programmes can mean higher risk of communicable disease [ 22 ]. Scoping reviews have also been conducted to describe trends within migration health research in the United Kingdom (UK) [ 23 ] and identify gaps for future research agendas in the UK [ 23 ] and in the Republic of Ireland [ 24 ].

Almost three-quarters (73%) of published migration health research in the UK has been conducted in England, focusing primarily on infectious diseases and mental health. There is limited evidence on the social determinants of health, access to and use of healthcare and structural and behavioural factors behaviours that influence migrant health in the UK [ 23 ]. By contrast, a large amount of the migration research conducted in the Republic of Ireland has focused on the social determinants of health, and on health system adaptations, with a paucity of research focusing on improving health information systems [ 24 ].

Migration and Health in Scotland

Immigration to Scotland began to rise in 2003 with the expansion of the EU [ 25 ]. The population in Scotland increased from 5.11 million to 5.47 million between 2005 and 2020 and is predicted to continue rising until 2028 [ 26 ] despite low birth rates, with the increased population resulting from inward migration [ 27 ]. Scotland’s population is becoming more ethnically diverse [ 28 ] and susceptibility to different health conditions varies by ethnic group, which has implications for the planning and provision of health services [ 29 ]. 7% of the current Scottish population are non-UK nationals and 10% were born outside Britain. The commonest countries of origin were Poland, Ireland, Italy, Nigeria and India [ 30 ].

Within Scotland, linking health data to ethnicity is standard in order to monitor and improve health of minority groups [ 31 ]. Ethnic background can differ from country of birth which means migration status cannot be assumed [ 32 ], although health inequalities experienced by migrants often extend to affect all ethnic minority groups [ 33 ]. The Scottish Health and Ethnicity Linkage Study (SHELS) linked census data to health records of 91% of the population which has provided information on mortality and morbidity by ethnic group and country of birth [ 34 ]. SHELS research indicates that the white-Scottish population have a higher mortality rate than other ethnic groups. This may be consequent to the comparatively poor health of the Scottish population relative to other European nations: high mortality rates in the general population may cause a perception that the health of minorities is more advantageous than in reality [ 35 ].

Cezard et al’s [ 13 ] analysis of self-perceived health among people in Scotland found that being born abroad had a positive impact on health status. Health declined with increased length of residence, which may be explained by cultural convergence with the majority population. Allik et al. [ 36 ] compared health inequalities by ethnic background and found that with increasing age, health differences reduced thus people aged over 75 of all ethnicities had similar or worse health status than White-Scottish people. While working-age migrants appear to be healthier than the White Scottish population, it cannot be assumed that in future this would extend to older age groups.

Research has shown deprivation as a cause of heath inequalities among ethnic minority and migrant groups [ 37 ]. The socio-economic status of minority ethnic groups in Scotland is unusual, as most are of similar or higher status than the white-Scottish population [ 38 ]. Therefore, public health interventions targeting deprivation may not address risk-factors for ethnic minorities and migrants [ 36 ]. Further research on determinants of health in migrants can help with planning and design of inclusive policies.

The 2011 census indicated that 50% of immigrants lived in the cities of Edinburgh, Glasgow, and Aberdeen. Glasgow had a greater percentage of non-European immigrants due to participation in the Asylum dispersal programme [ 39 ]. 10% of UK asylum seekers are placed in Glasgow, but records are not kept following approval of asylum claims, therefore the size of the refugee population is unknown [ 40 ]. While immigration is controlled by the British government, in policy areas devolved to the Scottish government, refugees and asylum seekers have more rights than elsewhere in UK, including access to primary healthcare for undocumented migrants [ 40 ]. Despite the mitigating effect of Scottish policies, asylum seekers’ health is worsened by the asylum process and associated poverty, marginalisation, and discrimination [ 40 ]. Health deteriorates with increasing length of time in the asylum system [ 40 ] and asylum seekers and refugees have additional health needs and require enhanced support [ 41 ]. Research on the health needs of asylum seekers in Scotland is required to ensure adequate healthcare.

Aim and objectives

While scoping reviews on migrant health have been carried out in Europe [ 12 ], Ireland [ 24 ] and the UK [ 23 ] none are currently specific to the Scottish context. Given the devolved government of Scotland and demographics described above, a targeted review would help to clarify research priorities, with the aim of improving health and health care within the migrant community in Scotland. This work therefore builds on the published scoping review of migrant health in the Republic of Ireland [ 24 ]. The authors recommend replication of the study in other countries to facilitate cross-country comparison. Our aim was to scope peer-reviewed research and grey literature on migrant health conducted in Scotland and identify any gaps in the evidence. Our objectives were to: [1] understand the extent of the available research by topic area [2] summarise the types of research already conducted, populations studied, topics covered and approaches taken [3], map the existing research conducted in Scotland and [4] identify areas for future research based on any gaps in the evidence identified.

A scoping review was conducted as they can aid detection of evidence gaps [ 42 ] and allow incorporation of grey literature in topics with insufficient published research [ 43 ]. Arksey and O’Malley’s [ 44 ] five stage scoping review framework was used.

Stage 1: identifying the research question

Arskey and O’Malley [ 44 ] suggest maintaining a broad approach to identifying the research question, in order to generate breadth of coverage. On this basis, and in line with the research question identified in the Villarroel et al. [ 24 ] scoping review, our research question was framed as follows: What is the scope, main topics and gaps in evidence in the existing literature on health of international migrants living in Scotland? Arksey and O’Malley [ 44 ] highlight the importance of defining terminology at the outset of scoping reviews. For consistency, we used the broad definition of ‘migrant’ as per Villaroel et al. [ 24 ], from the International Organisation for Migration (IOM) [ 1 ]. References to refugees or asylum seekers followed the United Nations Refugee Agency definitions [ 3 ].

Stage 2: identifying relevant studies

Electronic database searches identified reports alongside a grey literature search, in line with Arskey and O’Malley’s [ 44 ] guidance to search for evidence via different sources. CINAHL, Web of Science, SocIndex and Medline academic databases were selected with input from co-authors. Search terms for the review were based upon those used by Villaroel et al. [ 24 ] with additional relevant terms from Hannigan et al. [ 9 ] The strategy combined three sets of terms for: Migrants (e.g., refugee, migrant, immigrant or newcomer), Scotland and Health. Both free text terms and index terms were used and adapted to the 4 academic databases and searches were run on 10th March 2023 (see Additional File 1 for database search strategies). Thirteen Government, University, and third-sector websites in Scotland were scoped for selection then hand-searched for grey literature (listed in Additional File 1 ).

Stage 3: study selection

Net-migration to Scotland increased in the 2000s [ 27 ] hence a date range of January 2002-March 2023 was used to identify evidence. The search was limited to English only. Inclusion/exclusion criteria for the studies were based on those used by Villaroel et al. [ 24 ] and expanded upon following discussion with co-authors (see Table  1 ). Reports were included if based on primary or secondary research on the health of international migrants in Scotland and used qualitative, quantitative or mixed methods research design. International or UK based reports were only included if Scottish results were documented separately. Reports on the health of ethnic minority groups in Scotland was included if place of birth was recorded. Research on internal (non-international) migrants within Scotland, either moving from one Scottish area to another or from another part of the United Kingdom to Scotland, were excluded.

Stage 4: data charting

All records were saved to RefWorks for screening. Records were first screened at title/abstract stage with 10% independently checked by the co-authors. The remaining reports were single screened using full text by the first author. Data from the included records was extracted and organised in tabular form under the following headings, which were agreed by team members: article type (peer-reviewed article or grey literature), publication date, geographical setting, study/intervention’s target population, funding, primary research focus on migrant health (y/n), study objective, data collection method, study design (qualitative/quantitative/mixed) and main finding. Reports were not critically appraised in this scoping review.

Stage 5: collating, summarising and reporting results

A report (either a peer-reviewed journal article or grey literature report) is used as our unit of analysis. In order to present the range of research identified, reports were grouped by the different headings in our data charting table and the outcomes considered for relevance to our scoping review’s aim. Our Results summarise the recency, focus, study designs and funding sources of the identified research, followed by the geographical settings and whether Scotland was included in international research reports. Reports were grouped by their study population and further sub-divided by publication type and geographical area for summarising. Finally, the WHO’s European strategy and action plan (SAAP) for refugee and migrant health [ 7 ] is a policy framework designed to help governments and other stakeholders monitor and improve migrant health in Europe. There are nine strategic areas in the WHO’s SAAP, which prioritise the most salient issues. In line with Villaroel et al’s [ 24 ] approach and in order to compare scoping review outcomes, these areas were used to categorise the findings of this review. Each report was matched to the most appropriate SAAP:

Establishing a Framework for Collaborative Action.

Advocating for the right to health of refugees.

Addressing the social determinants of health.

Achieving public health preparedness and ensuring an effective response.

Strengthening health systems and their resilience.

Preventing communicable disease.

Preventing and reducing the risks caused by non-communicable disease.

Ensuring ethical and effective health screening and assessment.

Improving health information and communication.

The primary focus (aims and objectives) of each report was used to identify the relevant SAAP area/areas. To improve reliability, results were compared using coding criteria used in Villaroel et al’s study (MacFarlane 2023, personal communication, 31st May). 10% of the reports were checked by one co-author to ensure consistent coding to SAAP categories. Any instances of uncertainty in mapping reports to the relevant SAAP area/areas were discussed and resolved by team members.

This scoping review of the literature on migrant health in Scotland identified 2166 records from academic literature databases, following duplicate removal, and 170 records from website searches (see Fig.  1 ). Following screening, a total of 71 peer-reviewed journal articles and 29 grey literature studies (totalling 100 reports) were included for analysis (Results table and reference list are presented in Additional File 2 ).

Flow chart illustrating the identification of sources of evidence included in the scoping review

Overall findings

The majority of reports were published between 2013 and 2022. Fifty-eight reports (58%) focused exclusively on migrant health [ 18 , 39 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 , 60 , 61 , 62 , 63 , 64 , 65 , 66 , 67 , 68 , 69 , 70 , 71 , 72 , 73 , 74 , 75 , 76 , 77 , 78 , 79 , 80 , 81 , 82 , 83 , 84 , 85 , 86 , 87 , 88 , 89 , 90 , 91 , 92 , 93 , 94 , 95 , 96 , 97 , 98 , 99 , 100 , 101 , 102 ]. 23 centred on health but included other populations in addition to migrants – for example research on ethnic minorities or other vulnerable groups [ 13 , 31 , 35 , 103 , 104 , 105 , 106 , 107 , 108 , 109 , 110 , 111 , 112 , 113 , 114 , 115 , 116 , 117 , 118 , 119 , 120 , 121 , 122 ]. Seventeen reports were included where the sample population were migrants, but the primary topic was not health – for example destitution, integration, and service needs [ 27 , 73 , 74 , 123 , 124 , 125 , 126 , 127 , 128 , 129 , 130 , 131 , 132 , 133 , 134 , 135 ]. Health data was reported as part of the wider subject matter. One report [ 136 ] looked at the social determinants of breastfeeding including migrant status and one [ 137 ] compared attitudes to aging and family support between countries.

Funding sources were not declared for 35 (35%) of reports. The Scottish Government funded 20 reports (20%) [ 13 , 27 , 32 , 39 , 45 , 46 , 47 , 66 , 77 , 88 , 99 , 100 , 101 , 102 , 113 , 116 , 119 , 121 , 129 , 134 ]. Other common sources of funding included Government funded public bodies ( n  = 13) [ 45 , 48 , 49 , 50 , 51 , 52 , 53 , 104 , 107 , 113 , 116 , 131 , 136 ], the Scottish Health Service ( n  = 18) (either the National Health Service (NHS) [ 13 , 54 , 56 , 57 , 58 , 59 , 102 , 113 , 116 ], local NHS trusts [ 45 , 60 , 61 , 77 , 102 , 103 , 112 ] or by Public Health Scotland [ 13 , 113 ]) Eleven reports (11%) were funded by Universities. The charity sector financed 15 (15%) reports [ 53 , 63 , 66 , 69 , 70 , 71 , 72 , 73 , 74 , 103 , 111 , 123 , 125 , 132 , 138 ] and the EU and Scottish local authorities funded four reports each [ 45 , 62 , 75 , 76 , 77 , 102 , 125 , 135 ]. Professional bodies financed one report [ 126 ] as did the Japanese government [ 64 ]. No reports received funding from the business sector. The biggest sources of funding for grey literature were Refugee charities (40%) and the Scottish government (30%) (see Fig. 2 ).

Sources of funding for migrant health research in Scotland

Research methods and data collection

52% of reports used qualitative research methods. Forty-five reports (86%) collected data using 1–1 interviews and 24 (46%) used focus groups. Other methods of data collection included questionnaires (six studies (11%)), workshops (two studies (3.85%)) and observation (two studies (3.85%)). Oral/written evidence, guided play sessions, family case studies and participatory activity sessions were used in one report each.

28% of reports used quantitative research methods, most commonly cross section design (ten studies (36%)) and cohort design (18 studies (64%)). Information was obtained from databases including medical records, Census data and national records in 21 reports (75%). Questionnaires were used in six reports (21%). Other methods including body measurements, food diaries, blood samples, interviews and case reviews were used in 1 report each.

20% of reports used mixed methods. The most common method of data collection was questionnaires in 14 reports (70%), interviews in ten reports (50%), focus groups in seven reports (35%), workshops in three reports (13.6%), and databases in three reports (13.6%). Other methods included literature review in two reports (10%), case note reviews in two reports (10%) and one reports each used mapping and school records.

Geographical areas of study

Ninety-one reports were situated in Scotland, of which 35 (38.5%) covered the whole country and 56 (61.5%) specified a city or area where research was undertaken. Some UK and international reports also specified the area of Scotland. The largest share of research within Scotland overall was in Glasgow with 36 reports, followed by Edinburgh with 16 reports, Lothian with six reports, Aberdeen with five reports and Grampian with three reports. The Northeast, Stirling, Highlands, Inverness, Lanarkshire, Motherwell and Selkirk had one report in each area.

There were seven international reports, three on mortality by country of birth [ 75 , 76 , 78 ], one on cross cultural communication [ 79 ], one on maternity care in Poland and Scotland [ 99 ], one comparing attitudes to aging in China and Scotland [ 137 ] and one on the link between birthweights and integration of migrants [ 64 ]. The remaining two reports were UK based, one on immunisation of Roma and traveller communities [ 117 ] and one on the link between ethnic diversity and mortality [ 104 ]. All the included international and UK reports documented the Scottish data separately within results.

Migrant population

Thirty-one reports included all migrants in the study population. The remaining reports included 30 studies on asylum seekers/refugees, 11 on Polish migrants, ten on Africans, six each on South Asians/Chinese/European, three on Arabs, and two on Roma populations (see Fig.  3 ). Most reports did not specify the country of origin for Asylum seekers and refugees - where country of birth was specified, reports were also included in the appropriate category.

Migrant populations studied in health research in Scotland

Grey literature and peer-reviewed reports differed in population focus. The most common populations of interest in grey literature were asylum seekers/refugees consisting of 18 reports (62%) [ 27 , 47 , 54 , 55 , 59 , 63 , 70 , 71 , 72 , 73 , 74 , 123 , 125 , 127 , 128 , 132 , 134 , 138 ] while for peer-reviewed journals 24 reports (34%) focused on all migrants [ 13 , 35 , 45 , 48 , 64 , 76 , 78 , 79 , 80 , 81 , 104 , 105 , 108 , 109 , 113 , 114 , 115 , 116 , 118 , 120 , 121 , 122 , 136 ].

Migrant study population also differed by local area; Glasgow city, where the majority of research occurred, had 18 reports of 36 (50%) on Asylum seekers/refugees [ 47 , 48 , 52 , 53 , 54 , 55 , 58 , 63 , 70 , 71 , 72 , 82 , 83 , 127 , 128 , 130 , 138 , 139 ] eight reports (22%) on Africans [ 52 , 53 , 84 , 85 , 86 , 87 , 106 , 107 ], seven reports (19%) on all migrants [ 45 , 48 , 80 , 102 , 104 , 105 , 121 ] and two reports (5.5%) on Roma migrants [ 103 , 117 ]. Other populations had one reports each. In Edinburgh five reports of 16 (31%) were on the Polish population [ 56 , 67 , 68 , 89 , 90 ], and two reports (12.5%) on Asylum seekers/refugees [ 60 , 133 ], Chinese [ 62 , 137 ], South Asian [ 46 , 119 ], all migrants [ 105 , 121 ] and Africans [ 87 , 107 ]. The remaining migrant groups had one report each. Other areas of Scotland show no clear pattern with studies in disparate migrant population groups.

Number of reports per Strategic and Action Plan (SAAP) Area

SAAP Area mapping

1. establishing a framework for collaborative action.

Nine reports had a primary focus on collaborative action and were categorised under SAAP area 1 (see Fig.  4 ) [ 66 , 70 , 72 , 73 , 103 , 125 , 129 , 132 , 134 ]. Four reports (33%) used a mixed methods study design, the remaining five reports (67%) used a qualitative design. One report [ 66 ] focused on the epidemiology of female genital mutilation and a proposed intervention strategy. One report [ 66 ] focused on the epidemiology of female genital mutilation and a proposed intervention strategy. One report [ 103 ] evaluated service provision to the Roma community in Glasgow. The remaining reports focused on refugees and asylum seekers: four [ 73 , 125 , 132 , 134 ] evaluations of refugee integration projects, one [ 70 ] on services available to pregnant women, and one [ 72 ] an assessment of a peer-education service. One report [ 129 ] was a review of service provisions for migrants during the Covid-19 pandemic. All reports in SAAP area 1 were grey literature and three (37.5%) had a primary focus on migrant health while four (50%) focused on integration, one (11%) included data on ethnic minorities and one (11%) on services during the covid-19 pandemic. The majority (seven reports (78%)) were also categorised to another SAAP area most commonly area 2 (five studies (55%)) or area 5 (four studies (44%)).

2. Advocating for the right to health of refugees

Nineteen reports focused on SAAP area 2, advocating for the right to health of refugees (see Fig.  4 ) [ 47 , 52 , 53 , 54 , 55 , 63 , 70 , 71 , 83 , 103 , 123 , 124 , 125 , 127 , 128 , 129 , 134 , 138 , 140 ]. Sixteen reports (84%) had a qualitative study design and the remaining three (16%) reports used mixed methods. Nine reports (47%) focused on the health impact of the asylum system [ 52 , 55 , 71 , 74 , 123 , 127 , 128 , 129 , 138 ], five (26%) on health and access to care [ 47 , 54 , 83 , 103 , 124 ], two (10.5%) on maternity care [ 63 , 70 ], two (10.5%) on integration services [ 125 , 134 ] and one report on mental health in HIV positive migrants [ 53 ]. Nine reports (47%) had a primary focus on migrant health while the remaining 10 (53%) also involved wider social issues. The majority (15 (79%)) of reports were grey literature. All the articles in this group overlapped with another SAAP area. Area 3 is the most common joint category with ten reports (53%) followed by area 5 with seven reports (37%), area 1 shares five reports (26%), while areas 4 and 8 share one report each (5%).

3. Addressing the social determinants of health

Twenty-nine reports were categorised to SAAP area 3 – addressing the social determinants of health (see Fig.  4 ) [ 13 , 27 , 45 , 50 , 52 , 55 , 60 , 62 , 63 , 65 , 68 , 71 , 74 , 80 , 81 , 82 , 91 , 92 , 93 , 102 , 112 , 123 , 124 , 127 , 128 , 136 , 137 , 138 ]. The majority (14 (48%)) used a qualitative study method, eight (28%) used quantitative methodology and the remaining seven reports (24%) used mixed methods. Nineteen reports (65.5%) were peer-reviewed journals [ 13 , 45 , 50 , 52 , 60 , 62 , 63 , 65 , 68 , 80 , 81 , 82 , 91 , 92 , 93 , 104 , 112 , 124 , 136 , 137 ] and ten (34.5%) were grey literature [ 27 , 55 , 63 , 71 , 74 , 102 , 123 , 127 , 128 , 138 ]. Ten reports (34.5%) discussed the effects of the asylum system on health [ 27 , 52 , 63 , 71 , 74 , 123 , 124 , 127 , 128 , 137 ] and one (3.5%) migration and health [ 50 ]. Six reports (21%) focused on culture and ethnicity [ 82 , 92 , 102 , 104 , 112 , 137 ], five reports (17%) discussed economic and environmental determinants of health [ 13 , 45 , 67 , 81 , 93 ] and five reports (17%) the health impact of social activities [ 55 , 60 , 62 , 80 , 91 ]. Of the remaining reports, one [ 65 ] discussed Brexit and mental health of European migrants and one discussed the effect of coping strategies on wellbeing in Polish migrants [ 68 ]. Most reports, 18 (62%) had a primary focus on migrant health [ 45 , 50 , 52 , 55 , 60 , 62 , 63 , 65 , 67 , 68 , 71 , 80 , 81 , 82 , 91 , 92 , 93 , 102 ], six reports (21%) discussed wider social factors in addition to health [ 74 , 123 , 124 , 127 , 128 , 138 ]. Of the remaining reports three (10%) looked at ethnic background and country of birth [ 13 , 112 , 136 ], one [ 27 ] included other vulnerable groups and one [ 137 ] included people living in China and Chinese migrants to Scotland. Thirteen reports were also categorised to one or more additional SAAP area - ten (34%) were also applicable to area 2 [ 52 , 55 , 63 , 71 , 74 , 123 , 124 , 127 , 128 , 138 ], three (10%) to area 5 [ 63 , 82 , 92 ] and one (7%) to area 4 [ 27 ].

4. Achieving public health preparedness and ensuring an effective response

Twenty-one reports were assigned to SAAP area 4 (see Fig.  4 ) [ 27 , 31 , 35 , 39 , 47 , 57 , 64 , 75 , 76 , 77 , 78 , 94 , 104 , 108 , 109 , 111 , 113 , 114 , 116 , 120 , 135 ] of which fourteen (67%) used quantitative research methods, four (19%) mixed methods and three (14%) qualitative methods. Thirteen (62%) reports were peer-reviewed journals [ 35 , 59 , 64 , 75 , 78 , 104 , 108 , 109 , 111 , 113 , 114 , 116 , 120 ] and eight (38%) grey literature [ 27 , 31 , 39 , 47 , 57 , 77 , 94 , 135 ]. Most reports (12 (57%)) focused on morbidity and mortality in migrant populations [ 31 , 35 , 64 , 75 , 76 , 78 , 104 , 108 , 109 , 113 , 114 , 116 ]. Six (29%) investigated health status and healthcare needs in migrant groups in Scotland [ 39 , 47 , 57 , 77 , 94 , 135 ]. Two reports (9.5%) analysed the epidemiology of HIV infections [ 111 , 120 ] and the remaining report focused on the health needs of young people during the covid-19 pandemic [ 27 ]. Nine reports (43%) had a primary focus on migrant health [ 39 , 47 , 55 , 64 , 75 , 76 , 77 , 78 , 94 ] while eight (38%) also analysed data by ethnicity [ 31 , 35 , 104 , 108 , 109 , 113 , 114 , 116 ]. Of the remaining reports, three (14%) included other populations within Scotland [ 27 , 111 , 120 ] and one (5%) included other characteristics in addition to health information [ 135 ]. Ten reports (48%) were also categorised to another SAAP area; one to area 2 [ 47 ], one to area 3 [ 27 ], four to area 5 [ 47 , 57 , 77 , 135 ], two to area 6 [ 111 , 120 ] and two to area 9 [ 31 , 108 ].

5. Strengthening health systems and their resilience

Twenty-nine reports were assigned to SAAP area 5 (see Fig.  4 ) [ 18 , 47 , 48 , 49 , 54 , 57 , 63 , 69 , 70 , 72 , 77 , 79 , 82 , 83 , 92 , 95 , 96 , 97 , 99 , 101 , 103 , 118 , 119 , 126 , 129 , 131 , 133 , 135 , 141 ] of which 23 (79%) used qualitative research methods. Three reports used quantitative methods (10.3%) and the remaining three used mixed methods (10.3%). Twelve reports (41%) examined migrants needs and experiences of health care [ 47 , 49 , 54 , 57 , 58 , 77 , 83 , 95 , 103 , 119 , 129 , 135 ], eight (24%) focused on pregnancy and childcare [ 63 , 70 , 92 , 96 , 97 , 99 , 101 , 118 ] and two (7%) on barriers to healthcare access [ 48 , 131 ]. Two reports (7%) evaluated healthcare programmes [ 72 , 133 ] and two focused on communication in primary care [ 79 ] and maternity services [ 69 ]. The remaining three reports (10%) covered sexual health [ 82 ], health information needs of Syrian refugees [ 126 ] and general practitioner training [ 18 ]. Nineteen (65.5%) were peer reviewed journals [ 18 , 48 , 49 , 58 , 69 , 79 , 82 , 83 , 92 , 95 , 96 , 97 , 99 , 101 , 118 , 119 , 125 , 131 , 133 ] and ten (34.5%) were grey literature [ 47 , 54 , 57 , 63 , 70 , 72 , 77 , 103 , 129 , 135 ]. Twenty-one (72%) had a primary focus on migrant health [ 18 , 47 , 48 , 49 , 54 , 57 , 58 , 63 , 69 , 70 , 72 , 77 , 79 , 82 , 83 , 92 , 95 , 96 , 97 , 99 , 101 ]. Six reports (21%) included research on other characteristics or services [ 103 , 126 , 129 , 131 , 133 , 135 ]. The remaining two reports (7%) included ethnic groups as well as migrants in the data [ 118 , 119 ]. Nineteen reports (65.5%) were also assigned to one or more other category areas: five reports (17%) to area 1 [ 47 , 70 , 72 , 103 , 129 ], five reports (17%) to area 2 [ 54 , 63 , 83 , 103 , 129 ], three reports (10%) to area 3 [ 63 , 82 , 92 ], four reports (14%) to area 4 [ 47 , 57 , 77 , 135 ], one (3.5%) to area 7 [ 119 ] and one (3.5%) to area 9 [ 48 ].

6. Preventing communicable diseases

Fourteen reports were assigned to SAAP area 6 (see Fig.  4 ) [ 56 , 61 , 87 , 88 , 89 , 90 , 105 , 106 , 107 , 111 , 115 , 117 , 120 , 122 ] of which four (31%) used quantitative methods, five (38%) used qualitative methods and five (38%) used mixed methods. Five reports (38.5%) examined immunisation behaviour [ 56 , 61 , 89 , 90 , 117 ], five (38%) on epidemiology and treatment of HIV [ 106 , 107 , 111 , 120 , 122 ]. The remaining four reports (31%) focused on tuberculosis in healthcare workers [ 115 ], malaria [ 105 ] and sexual health services [ 87 , 88 ]. Only one reports was grey literature [ 88 ], the remainder were peer-reviewed journals. Six reports (46%) had a primary focus on migrant health [ 56 , 61 , 87 , 88 , 89 , 90 ] while seven reports (54%) also included other at-risk groups in the analysis. Four reports (31%) were also assigned to another SAAP category, two (15%) to area 4 [ 111 , 120 ] and two (15%) to area 8 [ 88 , 115 ].

7. Preventing and reducing the risks posed by non-communicable diseases

Eight reports were categorised to SAAP area 7 (see Fig.  4 ) [ 46 , 51 , 59 , 84 , 85 , 86 , 98 , 119 ] of which six (75%) used qualitative research methods, one (12.5%) used quantitative methods and one (12.5%) used mixed methods. Only one report (12.5%) was grey literature [ 59 ] the remaining seven reports (87.5%) were peer-reviewed journals [ 48 , 87 , 92 , 126 , 127 , 128 , 140 ]. Three reports (37.5%) focused on health behaviours [ 51 , 85 , 98 ], two (25%) on mental health, two (25%) on diabetes and one (12.5%) on chronic disease. Seven reports(87.5%) had a primary focus on migrant health [ 46 , 51 , 59 , 84 , 85 , 86 , 98 ], with the remaining report (12.5%) including ethnic minority groups [ 119 ]. One report (12.5%) was also assigned to SAAP area number 5 [ 119 ].

8. Ensuring ethical and effective health screening and assessment

There were six reports assigned to category 8 (see Fig.  4 ) [ 53 , 88 , 100 , 110 , 115 , 121 ] of which two (33%) used a quantitative research method, three (50%) used a qualitative method and one used mixed methods. One report (14%) was grey literature [ 88 ] the remaining five reports (83%) were peer reviewed journals [ 53 , 100 , 110 , 115 , 121 ]. Three reports (50%) focused on cancer screening in migrant women [ 21 , 100 , 110 ], one (17%) analysed access to HIV testing among African migrants [ 53 ], one (17%) on T.B in healthcare workers [ 72 ] and one (17%) on sexual health [ 36 ]. Three reports (50%) had a primary focus on migrant health [ 53 , 88 , 100 ] while the remaining three reports (50%) included other at-risk groups in the analysis [ 110 , 115 , 121 ]. There were three reports which overlapped with other SAAP areas: one [ 53 ] (17%) was categorised to area 2 while two [ 88 , 115 ] (33%) were categorised to area 6.

9. Improving health information and communication

Three reports were assigned to SAAP area 9 (see Fig.  4 ) [ 31 , 108 , 130 ]. One of these (33%) used a qualitative approach, one (33%) used a quantitative approach and one (33%) used mixed methods. Two [ 108 , 130 ] (66%) were peer-reviewed journal articles and one [ 31 ] (33%) was grey literature. Two reports (66%) focused on improving migrant demographics and health information using databases [ 31 , 108 ] while one (33%) described an information-needs matrix for refugees and asylum seekers [ 130 ]. Two [ 31 , 108 ] included ethnicities in the data while one [ 130 ] had a primary focus on migrant health. Two reports [ 31 , 108 ] (66%) also applied to SAAP area 4 while one report [ 130 ] (33%) was in SAAP area 9 only.

To our knowledge this is the first scoping review conducted on migrant health in Scotland. A previous rapid literature review [ 94 ] found most research focused on health behaviours, mental health, communicable disease and use of and access to healthcare; however, the review limited migrant definition to those who had immigrated within five years and asylum seekers were not included.

In our review, the majority of reports were published from 2013 onwards, aligning with the expansion in migrant research internationally [ 142 ]. 52% used qualitative research methods, 28% used quantitative methods and 20% used mixed methods. 58% focused on migrant health: the remaining papers included other populations or health as part of a wider remit. Research funding was mostly provided by the Scottish Government, NHS, refugee charities and Universities. No studies received funding from the private sector, although this sector has the potential resource and capacity to play a key role in funding future research to improve migrant health in Scotland. Geographically, most studies took place in Glasgow (36%), nationwide (38.5%) or Edinburgh (16%) – other areas were under-represented including Aberdeen (5%), despite being the city with the largest migrant population [ 30 ]. There was a lack of studies in rural localities. These findings concur with a UK migrant health review by Burns et al. [ 23 ] where research was concentrated in larger cities and data was sparse in rural areas relative to the migrant population.

Half of the research identified that was conducted in Glasgow focused on asylum seekers/refugees. Glasgow was previously the only Scottish city to host asylum seekers [ 143 ] and currently supports the most asylum seekers of any local authority in the UK [ 29 ]. In April 2022, the UK government widened the Asylum dispersal scheme to all local authorities [ 144 ]. Around 70% of Scotland’s refugee support services are based in Glasgow and the South-west [ 145 ]. As reduced access to services may impact the health of asylum seekers, research in Glasgow may not be generalizable to other regions of Scotland.

Almost one-third (30%) of all reports focused on asylum seekers and refugees – an overrepresentation given that only 18% of migrants to the UK are asylum seekers [ 146 ] and as low as 2% of all migrants in Scotland [ 147 ]. Asylum seekers and refugees are at risk of poor health due to trauma, difficult journeys, overcrowded camps, poor nutrition and lack of access to healthcare [ 148 ]. They have worse maternity outcomes and increased rates of mental illness [ 149 ]. Increased research on health of asylum seekers and refugees is necessary due to their additional vulnerabilities [ 142 ]. However, asylum seeker’s country of origin was generally not specified. Asylum seekers have heterogenic backgrounds [ 150 ] and nationality and trauma experience affect health status [ 151 ]. Further research focused on specific nationalities of asylum seekers would enhance understanding of the health needs in this population.

Almost one-third (31%) of studies did not specify a migrant group. This concurs with a Norwegian migrant health study by Laue et al. [ 152 ] where 36% of research did not identify country of birth. Where nationality was identified, Polish, African and South Asian were most prevalent. Poles are the largest migrant group in Scotland, however for the other most common immigrant groups of Irish, Italian and Nigerian [ 30 ] there was an absence of research. No studies took place on Nigerian migrants – nine studies indicated African populations, but country of birth was not specified. Since March 2022, 23,000 Ukrainians have migrated to Scotland [ 153 ], however no studies on Ukrainians were identified currently. Research may be underway which is yet to be published.

Only one study explored the impact of Brexit on European migrants’ health despite 56% of migrants to Scotland being EU nationals [ 30 ]. Again, research may be taking place currently, which is yet to be published. No studies involved undocumented migrants despite this populations’ high rates of poor physical/mental health exacerbated by poor housing and working conditions [ 154 ]. An estimated 7.2–9.5% of the workforce in the UK are migrant workers who have higher risks of poor working conditions and injury [ 155 ]. Scotland depends on a migrant workforce for some industries such as agriculture [ 156 ] but only two research papers specified migrant workers.

Most research papers related to the right to health of refugees (SAAP 2), social determinants of health (SAAP 3), public health planning (SAAP 4) and strengthening health systems (SAAP 5). Areas with less research were frameworks for collaborative action (SAAP 1), preventing communicable disease (SAAP 6), preventing non-communicable disease (SAAP 7) and health screening and assessment (SAAP 8). Only three studies related to improving health information and communication (SAAP 9). Lebano et al. [ 12 ] conducted a literature review of migrant health in Europe and found data collection unreliable and disorganised. There is a lack of data on the numbers and types of migrants entering Scotland and research tends not to differentiate between ethnic minorities and migrants [ 94 ]. As poor-quality information hinders surveillance and planning of services SAAP area 9 is an important consideration for increased research.

Villarroel et al. [ 24 ] also found more research in SAAP areas 3 to 5 and less in areas 6 to 9. However, their study returned no results in category 1, collaborative action, or 2, the right to health of refugees, while this study assigned 9% of articles to category 1 and 19% to category 2. Most articles in our study relating to categories 1 and 2 were grey literature, which was excluded from the original Irish scoping review. This highlights a potential difference in the focus of peer-reviewed articles compared to government/refugee charity commissioned reports. Collaborative action and the right to health of refugees and asylum seekers are entwined in Scotland due to the complex policy environment; the social determinants of health such as housing, education, welfare rights and social integration are influenced by a variety of UK and Scottish statutory bodies as well as third sector organisations [ 157 ]. Despite this complexity, organisations work well together [ 158 ]. Further academic research in this area would enhance joint working practices and networks.

A scoping review in the UK [ 23 ] found similar quantities of research corresponding to SAAP areas 3, 2 and 9. However in Scotland areas 1, 5 and 8 were a combined 44% of included papers compared with 27.8% of results on health systems and structures in Burns et al’s [ 23 ] study. Almost half of the articles in SAAP areas 1,5 and 8 were grey literature, which was not included in Burns et al’s [ 23 ] review. Conversely, Burns et al. [ 23 ] found 81.9% of research in the UK related to epidemiology, equivalent to SAAP categories 4,6 and 7. In a Norwegian scoping review of migrant health [ 152 ] 65% of research was related to epidemiological data on health and disease. Only 42% of the research in this current study related to epidemiological data; the quantity of evidence was reduced by excluding combined research from the UK. As Scotland has higher mortality and morbidity than elsewhere in the UK [ 29 ] it is important to undertake further epidemiological research limited to Scotland.

Strengths and weaknesses

Strengths of this review include the use of the WHO’s SAAP categories [ 7 ] to classify data, in accordance with the Villarroel et al’s [ 24 ] study: this means results are linked to policy on migrant health and facilitates comparability to the Irish study results. Additionally results include data on migrant groups, locality, and funding of included papers; these highlight potential omissions for future research consideration. Results include diverse research methods and published and grey literature giving a wide overview of available evidence, reported using the Preferred Reporting Items for Systematic reviews and Meta-Analyses for Scoping Reviews (PRISMA-ScR) checklist (see Additional File 3 ) [ 159 ].

Limitations included the lack of an open-access protocol and search limitations of English language and selected databases. This means some relevant reports may be omitted. Due to time and resource limitations no quality appraisal was planned for included reports. Whilst we did not synthesise the findings for each topic area and migrant group, future systematic reviews could be undertaken to address this limitation and build on this work.

Conclusions

Immigration and ethnic diversity in Scotland have increased since 2002 which is reflected in the expansion of migrant health research. This review highlights evidence gaps including a lack of research in rural areas, undocumented migrants and migrant workers. There is a tendency to cluster asylum seekers together rather than differentiate between national groups. Within the SAAP areas there is less evidence relating to collaborative action, preventing communicable disease, preventing non-communicable disease and health screening and assessment. Further research is required on improving health information and communication for migrant populations in Scotland – a significant omission given the importance of accurate information for health service planning.

Availability of data and materials

All data analysed during this review comes from the papers listed in Additional file 2 .

Abbreviations

European Union

Human Immunodeficiency Virus

National Health Service

Strategy and Action Plan

The Scottish Health and Ethnicity Linkage Study

United Kingdom

World Health Organisation

International Organisation for Migration (IOM). IOM Definition of Migrant. 2024. Available from: https://www.iom.int/about-migration .Cited 2024 Feb 8.

International Organisation for Migration United Nations. World Migration Report. 2022. Available from: available: https://worldmigrationreport.iom.int/wmr-2022-interactive/ .

The United Nations Refugee Angency. Refugee facts: What is a refugee? 2024. Available from: https://www.unrefugees.org/refugee-facts/what-is-a-refugee/ . Cited 2024 Feb 8.

Migration Data Portal. Migration data in Europe. 2023. Available from: https://www.migrationdataportal.org/regional-data-overview/europe#past-and-present-trends . Cited 2023 Aug 22.

International Centre for Migration Policy Development. Migration Outlook 2022 Twelve migration issues to look out for in 2022 Origins, key events and priorities for Europe. 2022. Available from: https://www.icmpd.org/file/download/56783/file/ICMPD%2520Migration%2520Outlook%25202022.pdf .

European Parliament. Exploring migration causes: why people migrate. 2023. Available from: https://www.europarl.europa.eu/news/en/headlines/world/20200624STO81906/exploring-migration-causes-why-people-migrate .

World Health Organisation. Strategic plan: Strategy and Action Plan for Refugee and Migrant Health in the WHO European Region 2016–2022. 2016. Available from: https://www.who.int/publications/i/item/strategic-plan-strategy-and-action-plan-for-refugee-and-migrant-health-in-the-who-european-region-2016-2022 .

Graetz V, Rechel B, Groot W, Norredam M, Pavlova M. Utilization of health care services by migrants in Europe—a systematic literature review. Br Med Bull. 2017;121(1):5–18. Available from: https://www.academic.oup.com/bmb/article-lookup/doi/10.1093/bmb/ldw057 .

Article   CAS   PubMed   Google Scholar  

Hannigan A, O'Donnell P, O'Keeffe M, MacFarlane A. How do Variations in Definitions of “Migrant” and their Application Influence the Access of Migrants to Health Care Services? Copenhagen: WHO Regional Office for Europe; 2016. (Health Evidence Network Synthesis Report, No. 46.) Available from: https://www.ncbi.nlm.nih.gov/books/NBK391032/ .

Rechel B, Mladovsky P, Ingleby D, Mackenbach JP, McKee M. Migration and health in an increasingly diverse Europe. Lancet. 2013;381(9873):1235–45. Available from: https://linkinghub.elsevier.com/retrieve/pii/S0140673612620868 .

Article   PubMed   Google Scholar  

Giannoni M, Franzini L, Masiero G. Migrant integration policies and health inequalities in Europe. BMC Public Health. 2016;16(1):463. Available from:  http://www.bmcpublichealth.biomedcentral.com/articles/10.1186/s12889-016-3095-9 .

Article   PubMed   PubMed Central   Google Scholar  

Lebano A, Hamed S, Bradby H, Gil-Salmerón A, Durá-Ferrandis E, Garcés-Ferrer J, et al. Migrants’ and refugees’ health status and healthcare in Europe: a scoping literature review. BMC Public Health. 2020;20(1):1039. Available from: https://www.bmcpublichealth.biomedcentral.com/articles/10.1186/s12889-020-08749-8 .

Cézard G, Finney N, Kulu H, Marshall A. Ethnic differences in self-assessed health in Scotland: The role of socio-economic status and migrant generation. Popul Space Place. 2022;28(3):e2403. Available from: https://onlinelibrary.wiley.com/doi/10.1002/psp.2403 .

Article   Google Scholar  

Anson J. The migrant mortality advantage: a 70 month follow-up of the brussels population. Eur J Popul. 2004;20(3):191–218.

World Health Organisation. Health of refugees and migrants. WHO European Region. 2018. Available from: https://www.who.int/publications/i/item/health-of-refugees-and-migrants---who-european-region-(2018) .

Mladovsky P. A framework for analysing migrant health policies in Europe. Health Policy (New York). 2009;93(1):55–63. Available from:  https://www.linkinghub.elsevier.com/retrieve/pii/S0168851009001444 .

De Vito E, de Waure C, Specchia ML, Parente P, Azzolini E, Frisicale EM, et al. Are undocumented migrants’ entitlements and barriers to healthcare a public health challenge for the European Union? Public Health Rev. 2016;37(1):13. Available from: http://publichealthreviews.biomedcentral.com/articles/10.1186/s40985-016-0026-3 .

Katikireddi SV, Bhopal R, Quickfall JA. GPs need training and funding in caring for refugees and asylum seekers. BMJ. 2004;328(7442):770.1. Available from:  https://www.bmj.com/lookup/doi/10.1136/bmj.328.7442.770 .

Carballo M, Hargreaves S, Gudumac I, Maclean EC. Evolving migrant crisis in Europe: implications for health systems. Lancet Glob Heal. 2017;5(3):e252-253. Available from:  https://linkinghub.elsevier.com/retrieve/pii/S2214109X17300402 .

Juárez SP, Honkaniemi H, Dunlavy AC, Aldridge RW, Barreto ML, Katikireddi SV et al. Effects of non-health-targeted policies on migrant health: a systematic review and meta-analysis. Lancet Glob Heal. 2019;7(4):e420–35. Available from: https://linkinghub.elsevier.com/retrieve/pii/S2214109X18305606 .

Nielsen SS, Krasnik A. Poorer self-perceived health among migrants and ethnic minorities versus the majority population in Europe: a systematic review. Int J Public Health. 2010;55(5):357–71. Available from: ( http://link.springer.com/10.1007/s00038-010-0145-4 ).

World Health Organsation. World report on the health of refugees and migrants. 2022. Available from: https://www.who.int/publications/i/item/9789240054462 .

Burns R, Zhang CX, Patel P, Eley I, Campos-Matos I, Aldridge RW. Migration health research in the United Kingdom: a scoping review. J Migr Heal. 2021;4:100061. Available from: https://linkinghub.elsevier.com/retrieve/pii/S2666623521000283 .

Villarroel N, Hannigan A, Severoni S, Puthoopparambil S, MacFarlane A. Migrant health research in the Republic of Ireland: a scoping review. BMC Public Health. 2019;19(1):324. Available from: ( https://bmcpublichealth.biomedcentral.com/articles/10.1186/s12889-019-6651-2 ).

Scottish Government. Demographic Change in Scotland. 2010. Available from: https://www.gov.scot/binaries/content/documents/govscot/publications/research-and-analysis/2010/11/demographic-change-scotland/documents/0108163-pdf/0108163-pdf/govscot%3Adocument/0108163.pdf .

National Records of Scotland. Projected Population of Scotland (Interim) 2020-based. 2022. Available from: https://www.nrscotland.gov.uk/files/statistics/population-projections/2020-based/pop-proj-2020-scot-nat-pub.pdf .

Scottish Government. Coronavirus (COVID-19) - experiences of vulnerable children, young people, and parents: research. 2021. Available from: https://www.gov.scot/publications/experiences-vulnerable-children-young-people-parents-during-covid-19-pandemic/ .

Scotland’s Census. Scotland’s Census: Ethnicity. 2011. Available from: https://www.scotlandscensus.gov.uk/census-results/at-a-glance/ethnicity/#:~:text=Scotland’spopulationwas96.0%25 .

Walsh D. The changing ethnic profiles of Glasgow and Scotland, and the implications for population health. 2017. Available from: https://www.gcph.co.uk/assets/0000/6255/The_changing_ethnic_profiles_of_Glasgow_and_Scotland.pdf .

National Records of Scotland. Migration Statistics Quarterly Summary for Scotland. 2021. Available from: https://www.nrscotland.gov.uk/files/statistics/migration/quarterly-summary/miration-statistics-quarterly-summary-february-2021.pdf .

The Scottish Ethnicity and Health Research Strategy Working Group. Health in our Multi-ethnic Scotland Future Research Priorities. 2009. Available from: https://www.healthscotland.scot/media/1842/health-in-our-multi-ethnic-scotland-full-report.pdf  .

The Scottish Public Health Observatory. Ethnic minorities: defining ethnicity and race. 2023. Available from: https://www.scotpho.org.uk/population-groups/ethnic-minorities/defining-ethnicity-and-race/ . Cited 2023 Aug 22.

Krasnik A, Bhopal RS, Gruer L, Kumanyika SK. Advancing a unified, global effort to address health disadvantages associated with migration, ethnicity and race. Eur J Public Health. 2018;28(suppl_1). Available from: https://academic.oup.com/eurpub/article/doi/10.1093/eurpub/cky046/4956664 .

Bhopal R, Fischbacher C, Povey C, Chalmers J, Mueller G, Steiner M, et al. Cohort profile: scottish health and ethnicity linkage study of 4.65 million people exploring ethnic variations in disease in Scotland. Int J Epidemiol. 2011;40(5):1168–75. Available from: https://academic.oup.com/ije/article-lookup/doi/10.1093/ije/dyq118 .

Bhopal RS, Gruer L, Cezard G, Douglas A, Steiner MFC, Millard A, et al. Mortality, ethnicity, and country of birth on a national scale, 2001–2013: a retrospective cohort (Scottish Health and Ethnicity Linkage Study). Basu S, editor. Plos Med. 2018;15(3):e1002515. https://doi.org/10.1371/journal.pmed.1002515 . Basu S, editor.

Allik M, Brown D, Dundas R, Leyland AH. Differences in ill health and in socioeconomic inequalities in health by ethnic groups: a cross-sectional study using 2011 Scottish census. Ethn Health. 2022;27(1):190–208. https://doi.org/10.1080/13557858.2019.1643009 ( https://www.tandfonline.com/doi/full/ ).

Watkinson RE, Sutton M, Turner AJ. Ethnic inequalities in health-related quality of life among older adults in England: secondary analysis of a national cross-sectional survey. Lancet Public Hea. 2021;6(3):e145-154.

Fischbacher CM, Cezard G, Bhopal RS, Pearce J, Bansal N. Measures of socioeconomic position are not consistently associated with ethnic differences in cardiovascular disease in Scotland: methods from the Scottish Health and Ethnicity Linkage Study (SHELS). Int J Epidemiol. 2014;43(1):129–39. Available from: https://academic.oup.com/ije/article-lookup/doi/10.1093/ije/dyt237 .

Scottish Government. Characteristics of Recent and Established EEA and non-EEA migrants in Scotland: Analysis of the 2011 Census. 2015. Available from: https://www.gov.scot/publications/characteristics-recent-established-eea-non-eea-migrants-scotland-analysis-2011-census/ .

House of Lords Library. Refugees and asylum-seekers: UK policy. 2022. https://lordslibrary.parliament.uk/refugees-and-asylum-seekers-uk-policy/ .

British Medical Association. Refugee and asylum seeker patient health toolkit. Unique health challenges for refugees and asylum seekers. 2022. Available from: https://www.bma.org.uk/advice-and-support/ethics/refugees-overseas-visitors-and-vulnerable-migrants/refugee-and-asylum-seeker-patient-health-toolkit/unique-health-challenges-for-refugees-and-asylum-seekers .

Khalil H, Peters M, Godfrey CM, McInerney P, Soares CB, Parker D. An evidence-based approach to scoping reviews. Worldviews Evidence-Based Nurs. 2016;13(2):118–23. Available from: https://doi.org/10.1111/wvn.12144 .

Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing the methodology. Implement Sci. 2010;5(1):69. Available from: ( http://implementationscience.biomedcentral.com/articles/10.1186/1748-5908-5-69 ).

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.Available from: http://www.tandfonline.com/doi/abs/10.1080/1364557032000119616 ).

Kearns A, Whitley E, Egan M, Tabbner C, Tannahill C. Healthy migrants in an unhealthy city? The Effects of time on the health of migrants living in deprived areas of glasgow. J Int Migr Integr. 2017;18(3):675–98. Available from: http://link.springer.com/10.1007/s12134-016-0497-6 .

PubMed   Google Scholar  

Porqueddu T. Herbal medicines for diabetes control among Indian and Pakistani migrants with diabetes. Anthropol Med. 2017;24(1):17–31. Available from: https://www.tandfonline.com/doi/full/10.1080/13648470.2016.1249338 .

Roshan N. Supporting new communities: a qualitative study of health needs among asylum seekers and refugee communities in North Glasgow final report. 2005. Available from: https://www.stor.scot.nhs.uk/handle/11289/579930 .

Piacentini T, O’Donnell C, Phipps A, Jackson I, Stack N. Moving beyond the ‘language problem’: developing an understanding of the intersections of health, language and immigration status in interpreter-mediated health encounters. Lang Intercult Commun. 2019;19(3):256–71. Available from: https://linkinghub.elsevier.com/retrieve/pii/S1353829214001233 .

Sime D. ‘I think that Polish doctors are better’: Newly arrived migrant children and their parents׳ experiences and views of health services in Scotland. Health Place. 2014;30:86–93. Available from: https://linkinghub.elsevier.com/retrieve/pii/S1353829214001233 .

Steven K, Munoz S, Migrants, Matter. Report of a Peer Researched Project on EU Migrant Health in the Highlands of Scotland. University of the Highlands and Islands. 2016. Available from: https://www.spiritadvocacy.org.uk/assets/Birchwood-Highland-HUG-Migrants-Matter-study-2015-2016.pdf .

Anderson AS, Bush H, Lean M, Bradby H, Williams R, Lea E. Evolution of atherogenic diets in South Asian and Italian women after migration to a higher risk region. J Hum Nutr Diet. 2005;18(1):33–43. Available from: ( https://onlinelibrary.wiley.com/doi/10.1111/j.1365-277X.2004.00584.x ).

Isaacs A, Burns N, Macdonald S, O’Donnell CA. ‘I don’t think there’s anything I can do which can keep me healthy’: how the UK immigration and asylum system shapes the health and wellbeing of refugees and asylum seekers in Scotland. Crit Public Health. 2022;32(3):422–32. Available from: https://www.tandfonline.com/doi/full/10.1080/09581596.2020.1853058 .

Palattiyil G, Sidhva D. Caught in a web of multiple jeopardy: post-traumatic stress disorder and HIV-positive asylum seekers in Scotland. Clin Soc Work J. 2015;43(4):362–74. Available from: http://link.springer.com/10.1007/s10615-015-0542-5 ).

Abdulkadir J, Azzudin A, Buick A, Curtice L, Dzingisai M, Easton D, et al. What do you mean, I have a right to health? Participatory action research on health and human rights. 2016. Available from: https://strathprints.strath.ac.uk/58209/1/Abdulkadir_etal_IPPI_2016_What_do_you_mean_I_have_a_right_to_health.pdf .

Strang A, Quinn N. Integration or isolation? Mapping social connections and well-being amongst refugees in Glasgow. 2014. Available from: https://eresearch.qmu.ac.uk/bitstream/handle/20.500.12289/4139/eResearch%25204139.pdf?sequence=1&isAllowed=y .

Gorman DR, Bielecki K, Larson HJ, Willocks LJ, Craig J, Pollock KG. Comparing vaccination hesitancy in polish migrant parents who accept or refuse nasal flu vaccination for their children. Vaccine. 2020;38(13):2795–9. Available from: https://linkinghub.elsevier.com/retrieve/pii/S0264410X20302255 .

Love J, Vertigans S, Domaszk E, Zdeb K, Love A, Sutton P. Health & ethnicity in Aberdeenshire: a study of Polish in-migrants; a report for the Scottish Health Council. 2007. Available from: https://rgu-repository.worktribe.com/output/247667 .

O’Donnell CA, Higgins M, Chauhan R, Mullen K. Asylum seekers’ expectations of and trust in general practice: a qualitative study. Br J Gen Pract. 2008;58(557):e1-11. Available from: https://bjgp.org/lookup/doi/10.3399/bjgp08X376104 .

Quinn N, Shirjeel S, Siebelt L, Donnelly R, Pietka E. An evaluation of the sanctuary community conversation programme to address mental health stigma with asylum seekers and refugees in Glasgow. 2011. Available from: https://www.healthscotland.com/uploads/documents/5584-SanctuaryCommunityConversationEvaluation.pdf .

Ager A. Community contact and mental health amongst socially isolated refugees in Edinburgh. J Refug Stud. 2002;15(1):71–80. Available from: https://academic.oup.com/jrs/article-lookup/doi/10.1093/jrs/15.1.71 .

Sim JA, Ulanika AA, Katikireddi SV, Gorman D. Out of two bad choices, I took the slightly better one’: Vaccination dilemmas for Scottish and Polish migrant women during the H1N1 influenza pandemic. Public Health. 2011;125(8):505–11. Available from: https://linkinghub.elsevier.com/retrieve/pii/S0033350611001697 .

Zhao S, Patuano A. International Chinese Students in the UK: association between use of green spaces and lower stress levels. Sustainability. 2021;4(1):89. Available from: https://www.mdpi.com/2071-1050/14/1/89 .

Da Lomba S, Murray N. Women and Children First? Refused asylum seekers’ access to and experiences of maternity care in Glasgow. 2014. Available from: https://strathprints.strath.ac.uk/58655/1/Lomba_Murray_SRC_2014_Women_and_Children_First_Refused_Asylum_Seekers_Access_to_and_Experiences.pdf .

Sørbye IK, Vangen S, Juarez SP, Bolumar F, Morisaki N, Gissler M, et al. Birthweight of babies born to migrant mothers - What role do integration policies play? SSM - Popul Heal. 2019;9:100503. Available from: https://linkinghub.elsevier.com/retrieve/pii/S2352827319301971 .

Teodorowski P, Woods R, Czarnecka M, Kennedy C. Brexit, acculturative stress and mental health among EU citizens in Scotland. Popul Space Place. 2021;27(6). Available from: https://onlinelibrary.wiley.com/doi/10.1002/psp.2436 .

Baillot H, Murray N, Connelly E, Howard N. Tackling Female Genital Mutilation in Scotland: A Scottish model of intervention. 2014. Available from: https://www.celcis.org/application/files/8116/2185/5421/Tackling_Female_Genital_Mutilation_-_A_Scottish_Model_of_Intervention.pdf .

Weishaar HB. Consequences of international migration: a qualitative study on stress among Polish migrant workers in Scotland. Public Health. 2008;122(11):1250–6. Available from: https://linkinghub.elsevier.com/retrieve/pii/S0033350608000942 .

Weishaar HB. You have to be flexible—coping among polish migrant workers in Scotland. Health Place. 2010;16(5):820–7. Available from: https://linkinghub.elsevier.com/retrieve/pii/S1353829210000432 .

Crowther S, Lau A. Migrant polish women overcoming communication challenges in scottish maternity services: a qualitative descriptive study. Midwifery. 2019;72:30–8. Available from: https://linkinghub.elsevier.com/retrieve/pii/S0266613819300361 .

Fassetta G, Da Lomba S, Quinn N. A healthy start? Experiences of pregnant refugee and asylum seeking women in Scotland. 2016. Available from: https://www.redcross.org.uk/-/media/documents/about-us/research-publications/refugee-support/a-healthy-start-report.pdf .

Positive Action in Housing. 12 months since the Park Inn Tragedy in Glasgow, one in three hotel asylum seekers say their mental health has deteriorated. 2021. Available from: https://www.paih.org/one-in-three-glasgow-asylum-seekers-suffering-depression-and-anxiety .

Strang A. Refugee Peer Education for Health and Well-being. Evaluation Report. 2015. Available from: https://www.scottishrefugeecouncil.org.uk/wp-content/uploads/2019/10/Peer-Education-Evaluation-Report.pdf .

Strang A, Marsden R, Mignard E. The Holistic Integration Service: Learning and Evaluation Year 1. 2014. Available from: https://www.scottishrefugeecouncil.org.uk/wp-content/uploads/2019/10/Holistic-Integration-Service-year-1-evaluation-report.pdf .

British Red Cross. How will we survive? Steps to preventing destitution in the asylum system. 2021. Available from: https://www.redcross.org.uk/-/media/documents/about-us/how-will-we-survive-preventing-destitution-in-the-asylum-system.pdf .

Bhopal RS, Rafnsson SB, Agyemang C, Fagot-Campagna A, Giampaoli S, Hammar N, et al. Mortality from circulatory diseases by specific country of birth across six European countries: test of concept. Eur J Public Health. 2012;22(3):353–9. Available from: https://academic.oup.com/eurpub/article-lookup/doi/10.1093/eurpub/ckr062 .

Rafnsson SB, Bhopal RS, Agyemang C, Fagot-Campagna A, Harding S, Hammar N, et al. Sizable variations in circulatory disease mortality by region and country of birth in six European countries. Eur J Public Health. 2013;23(4):594–605. Available from: https://academic.oup.com/eurpub/article-lookup/doi/10.1093/eurpub/ckt023 ).

de Lima P, Masud Chaudhry M, Whelton R, Arshad R. A study of migrant workers in Grampian. 2007. Available from: . http://www.communitiesscotland.gov.uk/stellent/groups/public/%0Adocuments/webpages/pubcs_019731.pdff .

Ikram UZ, Mackenbach JP, Harding S, Rey G, Bhopal RS, Regidor E, et al. All-cause and cause-specific mortality of different migrant populations in Europe. Eur J Epidemiol. 2016;31(7):655–65. Available from: http://link.springer.com/10.1007/s10654-015-0083-9 .

de Brún T, De-Brún MO, van Weel-Baumgarten E, van Weel C, Dowrick C, Lionis C, et al. Guidelines and training initiatives that support communication in cross-cultural primary-care settings: appraising their implementability using Normalization Process Theory. Fam Pract. 2015;cmv022. Available from: https://academic.oup.com/fampra/article-lookup/doi/10.1093/fampra/cmv022 .

García-Medrano S, Panhofer H. Improving migrant well-being: spontaneous movement as a way to increase the creativity, spontaneity and welfare of migrants in Glasgow. Body Mov Danc Psychother. 2020;15(3):189–203. Available from: https://www.tandfonline.com/doi/full/10.1080/17432979.2020.1767208 .

Jamil NA, Gray SR, Fraser WD, Fielding S, Macdonald HM. The relationship between vitamin D status and muscle strength in young healthy adults from sunny climate countries currently living in the northeast of Scotland. Osteoporos Int. 2017;28(4):1433–43. Available from: http://link.springer.com/10.1007/s00198-016-3901-3 .

Kaneoka M, Spence W. The cultural context of sexual and reproductive health support: an exploration of sexual and reproductive health literacy among female Asylum Seekers and Refugees in Glasgow. Int J Migr Heal Soc Care. 2019;16(1):46–64. Available from: https://www.emerald.com/insight/content/doi/10.1108/IJMHSC-01-2019-0002/full/html .

O’Donnell CA, Higgins M, Chauhan R, Mullen K. They think we’re OK and we know we’re not. A qualitative study of asylum seekers’ access, knowledge and views to health care in the UK. BMC Health Serv Res. 2007;7(1):75. Available from: https://bmchealthservres.biomedcentral.com/articles/10.1186/1472-6963-7-75 .

Cooper M, Harding S, Mullen K, O’Donnell C. ‘A chronic disease is a disease which keeps coming back … it is like the flu’: chronic disease risk perception and explanatory models among French- and Swahili-speaking African migrants. Ethn Health. 2012;17(6):597–613. Available from: http://www.tandfonline.com/doi/abs/10.1080/13557858.2012.740003 .

Ezika EA. An exploration of smoking behavior of african male immigrants living in glasgow. Tob Use Insights. 2014;7:TUI .S13262. Available from: http://journals.sagepub.com/doi/10.4137/TUI.S13262 .

Karadzhov D, White R. Between the whispers of the devil and the revelation of the word : christian clergy’s mental health literacy and pastoral support for BME congregants. J Spiritual Ment Heal. 2020;22(2):147–72. Available from: https://www.tandfonline.com/doi/full/10.1080/19349637.2018.1537755 ).

Yakubu BD, Simkhada P, van Teijlingen E, Eboh W. Sexual health information and uptake of sexual health services by African women in Scotland: a pilot study. Int J Heal Promot Educ. 2010;48(3):79–84. Available from: http://www.tandfonline.com/doi/abs/10.1080/14635240.2010.10708186 .

Goff J, Kay K, Lima M, Shallangwa S, We All Have A. Different Consciousness About It: Exploring the Sexual Health Needs of People From African Communities in Scotland. 2021. Available from: https://www.waverleycare.org/wp-content/uploads/2023/05/We_All_Have_Different_Consciousness_About_It_Report.pdf .

Bielecki K, Craig J, Willocks LJ, Pollock KG, Gorman DR. Impact of an influenza information pamphlet on vaccination uptake among Polish pupils in Edinburgh, Scotland and the role of social media in parental decision making. BMC Public Health. 2020;20(1):1381. Available from: https://bmcpublichealth.biomedcentral.com/articles/10.1186/s12889-020-09481-z .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Gorman DR, Bielecki K, Willocks LJ, Pollock KG. A qualitative study of vaccination behaviour amongst female Polish migrants in Edinburgh, Scotland. Vaccine. 2019;37(20):2741–7. Available from: https://linkinghub.elsevier.com/retrieve/pii/S0264410X19304220 .

Bak-Klimek A, Karatzias T, Elliott L, MacLean R. The determinants of well-being among polish economic immigrants. Testing the sustainable happiness model in migrant population. J Happiness Stud. 2018;19(6):1565–88. Available from: http://link.springer.com/10.1007/s10902-017-9877-7 .

Cheung NF. The cultural and social meanings of childbearing for Chinese and Scottish women in Scotland. Midwifery. 2002;18(4):279–95. Available from: https://linkinghub.elsevier.com/retrieve/pii/S0266613802903281 .

Papadaki A, Scott J. The impact on eating habits of temporary translocation from a Mediterranean to a Northern European environment. Eur J Clin Nutr. 2002;56(5):455–61. Available from: https://www.nature.com/articles/1601337 .

McCann A, Mackie P. Improving the Health of Migrants to Scotland: An update for Scottish Directors of Public Health. 2016. Available from: https://www.scotphn.net/wp-content/uploads/2016/04/2016_03_23-Migrant-Health-Report-FINAL-1.pdf .

Ahmed A, Cameron S, Dickson C, Mountain K. Arabic-speaking students’ primary care experiences in Scotland. Community Pract J Community Pract Heal Visit Assoc. 2010;83(2):23–6.

Google Scholar  

Bray J, Gorman D, Dundas K, Sim J. Obstetric care of New European migrants in Scotland: an audit of antenatal care, obstetric outcomes and communication. Scott Med J. 2010;55(3):26–31. Available from: ( http://journals.sagepub.com/doi/10.1258/rsmsmj.55.3.26 .

Cheung NF. Choice and control as experienced by Chinese and Scottish childbearing women in Scotland. Midwifery. 2002;18(3):200–13. Available from: https://linkinghub.elsevier.com/retrieve/pii/S0266613802903153 .

Spence W, Zhu L. Perceptions of smoking cessation among Glasgow’s Chinese community. Tob Prev Cessat. 2017;3(October). Available from: http://www.journalssystem.com/tpc/Perceptions-of-smoking-cessation-among-Glasgow-s-Chinese-community,77942,0,2.html .

Gorman DR, Katikireddi SV, Morris C, Chalmers JWT, Sim J, Szamotulska K, et al. Ethnic variation in maternity care: a comparison of Polish and Scottish women delivering in Scotland 2004–2009. Public Health. 2014;128(3):262–7. Available from: https://linkinghub.elsevier.com/retrieve/pii/S0033350613003910 .

Gorman DR, Porteous LA. Influences on Polish migrants’ breast screening uptake in Lothian, Scotland. Public Health. 2018;158:86–92. Available from: https://linkinghub.elsevier.com/retrieve/pii/S0033350617304018 .

Hogg R, de Kok B, Netto G, Hanley J, Haycock-Stuart E. Supporting Pakistani and Chinese families with young children: perspectives of mothers and health visitors. Child Care Health Dev. 2015;41(3):416–23. Available from: https://onlinelibrary.wiley.com/doi/10.1111/cch.12154 .

Kearns A, Whitley E. Health, Wellbeing and Social Inclusion of Migrants in North Glasgow. 2010. Available from: https://www.gowellonline.com/assets/0000/0521/Health_Wellbeing_and_Social_Inclusion_of_Migrants_in_North_Glasgow.pdf .

Poole L, Adamson K. Report on the Situation of the Roma Community in Govanhill, Glasgow. 2008. Available from: https://www.bemis.org.uk/resources/gt/scotland/reportonthesituationoftheromacommunityingovanhill,Glasgow.pdf .

Schofield L, Walsh D, Feng Z, Buchanan D, Dibben C, Fischbacher C, et al. Does ethnic diversity explain intra-UK variation in mortality? A longitudinal cohort study. BMJ Open. 2019;9(3):e024563. Available from: https://bmjopen.bmj.com/lookup/doi/10.1136/bmjopen-2018-024563 .

Unger HW, McCallum AD, Ukachukwu V, McGoldrick C, Perrow K, Latin G, et al. Imported malaria in Scotland – an overview of surveillance, reporting and trends. Travel Med Infect Dis. 2011;9(6):289–97. Available from: https://linkinghub.elsevier.com/retrieve/pii/S1477893911001074 .

Young I, Flowers P, McDaid LM. Barriers to uptake and use of pre-exposure prophylaxis (PrEP) among communities most affected by HIV in the UK: findings from a qualitative study in Scotland. BMJ Open. 2014;4(11):e005717. Available from: https://bmjopen.bmj.com/lookup/doi/10.1136/bmjopen-2014-005717 .

Young I, Flowers P, McDaid LM. Key factors in the acceptability of treatment as prevention (TasP) in Scotland: a qualitative study with communities affected by HIV. Sex Transm Infect. 2015;91(4):269–74. Available from: https://sti.bmj.com/lookup/doi/10.1136/sextrans-2014-051711 .

Bhopal R, Cm FI, Teiner SM, Halmers CJ, Ovey PC, Amieson J. Ethnicity and health in Scotland: Can we fill the information gap ? A demonstration project focusing on coronary heart disease and linkage of census and health records. Ethics. 2005. Available from: http://www.cphs.mvm.ed.ac.uk/docs/Retrocodingfinalreport.pdf .

Cezard GI, Bhopal RS, Ward HJT, Bansal N, Bhala N. Ethnic variations in upper gastrointestinal hospitalizations and deaths: the Scottish Health and Ethnicity Linkage Study. Eur J Public Health. 2016;26(2):254–60. Available from: https://academic.oup.com/eurpub/article-lookup/doi/10.1093/eurpub/ckv182 .

Christie-de Jong F, Kotzur M, Amiri R, Ling J, Mooney JD, Robb KA. Qualitative evaluation of a codesigned faith-based intervention for muslim women in Scotland to encourage uptake of breast, colorectal and cervical cancer screening. BMJ Open. 2022;12(5):e058739. Available from: https://bmjopen.bmj.com/lookup/doi/10.1136/bmjopen-2021-058739 .

Cree VE, Sidhva D. Children and HIV in Scotland: findings from a cross-sector needs assessment of children and young people infected and affected by HIV in Scotland. Br J Soc Work. 2011;41(8):1586–603. Available from: https://academic.oup.com/bjsw/article-lookup/doi/10.1093/bjsw/bcr036 .

Gallimore A, Irshad T, Cooper M, Cameron S. Influence of culture, religion and experience on the decision of Pakistani women in Lothian, Scotland to use postnatal contraception: a qualitative study. BMJ Sex Reprod Heal. 2021;47(1):43–8. Available from: https://jfprhc.bmj.com/lookup/doi/10.1136/bmjsrh-2019-200497 .

Gruer LD, Cézard GI, Wallace LA, Hutchinson SJ, Douglas AF, Buchanan D, et al. Complex differences in infection rates between ethnic groups in Scotland: a retrospective, national census-linked cohort study of 1.65 million cases. J Public Health (Bangkok). 2022;44(1):60–9. Available from: https://academic.oup.com/jpubhealth/article/44/1/60/6106111 .

Article   CAS   Google Scholar  

Bhala N, Cézard G, Ward HJT, Bansal N, Bhopal R. Ethnic variations in liver- and alcohol-related disease hospitalisations and mortality: the Scottish health and ethnicity linkage study. Alcohol Alcohol. 2016;51(5):593–601. Available from: https://academic.oup.com/alcalc/article-lookup/doi/10.1093/alcalc/agw018 .

Pollock KG, McDonald E, Smith-Palmer A, Johnston F, Ahmed S. Tuberculosis in healthcare workers, Scotland. Scott Med J. 2017;62(3):101–3. Available from: http://journals.sagepub.com/doi/10.1177/0036933017727963 .

Gruer LD, Millard AD, Williams LJ, Bhopal RS, Katikireddi SV, Cézard GI, et al. Differences in all-cause hospitalisation by ethnic group: a data linkage cohort study of 4.62 million people in Scotland, 2001–2013. Public Health. 2018;161:5–11. Available from: https://linkinghub.elsevier.com/retrieve/pii/S0033350618301501 .

Jackson C, Bedford H, Cheater FM, Condon L, Emslie C, Ireland L, et al. Needles, Jabs and Jags: a qualitative exploration of barriers and facilitators to child and adult immunisation uptake among Gypsies, Travellers and Roma. BMC Public Health. 2017;17(1):254. Available from: http://bmcpublichealth.biomedcentral.com/articles/10.1186/s12889-017-4178-y .

John JR, Curry G, Cunningham-Burley S. Exploring ethnic minority women’s experiences of maternity care during the SARS-CoV-2 pandemic: a qualitative study. BMJ Open. 2021;11(9):e050666. Available from: https://bmjopen.bmj.com/lookup/doi/10.1136/bmjopen-2021-050666 .

Lawton J, Ahmad N, Hanna L, Douglas M, Hallowell N. Diabetes service provision: a qualitative study of the experiences and views of Pakistani and Indian patients with Type 2 diabetes. Diabet Med. 2006;23(9):1003–7. Available from: https://onlinelibrary.wiley.com/doi/10.1111/j.1464-5491.2006.01922.x .

Livingston MR, Shaw LE, Codere G, Goldberg DJ. Human immunodeficiency virus acquired heterosexually abroad: expert panel assessment of the indigenous/nonindigenous to the united kingdom status of cases. J Travel Med. 2006;12(1):19–25. Available from: https://academic.oup.com/jtm/article-lookup/doi/10.2310/7060.2005.00005 .

Nelson M, Patton A, Robb K, Weller D, Sheikh A, Ragupathy K, et al. Experiences of cervical screening participation and non-participation in women from minority ethnic populations in Scotland. Heal Expect. 2021;24(4):1459–72. Available from: https://onlinelibrary.wiley.com/doi/10.1111/hex.13287 .

Noble G, Okpo E, Tonna I, Fielding S. Factors associated with late HIV diagnosis in North-East Scotland: a six-year retrospective study. Public Health. 2016;139:36–43. Available from: https://linkinghub.elsevier.com/retrieve/pii/S0033350616301020 .

Gillespie M. Trapped: Destitution and Asylum in Scotland. 2012. Available from: http://www.rst.org.uk/wp-content/uploads/2012/11/Trapped-destitution-and-asylum-summary-final-compressed-pictures.pdf .

Hopkins P, Hill M. The needs and strengths of unaccompanied asylum-seeking children and young people in Scotland. Child Fam Soc Work. 2010;15(4):399–408. Available from: https://onlinelibrary.wiley.com/doi/10.1111/j.1365-2206.2010.00687.x .

Marsden R, Harris C. “We started life again”: Integration experiences of refugee families reuniting in Glasgow. 2015. Available from: https://www.refworld.org/docid/560cde294.html .

Martzoukou K, Burnett S. Exploring the everyday life information needs and the socio-cultural adaptation barriers of Syrian refugees in Scotland. J Doc. 2018;74(5):1104–32. Available from: https://www.emerald.com/insight/content/doi/10.1108/JD-10-2017-0142/full/html .

McKenna R. From pillar to post: Destitution among people refused asylum in Scotland. 2019; Available from: https://www.rst.org.uk/wp-content/uploads/2019/03/From-Pillar-to-Post-Feb-2019.pdf .

Independent Commission of Inquiry. Failings in the provision of care to New Scots during the Covid pandemic: Part 2. 2022. Available from: https://static1.squarespace.com/static/62af1289a666c80e00b17253/t/636b9190408f81778746eaa7/1667994032702/AIS+Phase+2+Report+Full.pdf .

Trevena P, Gawlewicz A, Wright S. Addressing the needs of Scotland’s migrant and minority ethnic populations under Covid-19: lessons for the future. 2022. Available from: https://migrantessentialworkers.com/wp-content/uploads/2022/11/SC-Migrant-C19-Innovations.pdf .

Oduntan O, Ruthven I. The information needs matrix: a navigational guide for refugee integration. Inf Process Manag. 2019;56(3):791–808. Available from: https://linkinghub.elsevier.com/retrieve/pii/S0306457318306939 .

Sime D, Fox R, Migrant C. Social capital and access to services post-migration: transitions, negotiations and complex agencies. Child Soc. 2015;29(6):524–34. Available from: https://onlinelibrary.wiley.com/doi/10.1111/chso.12092 .

Strang A, Baillot H, Mignard E. Insights into integration pathways. New Scots and the Holistic Integration Service. A report drawing on year two of the Holistic Integration Service. 2015. Available from: https://scottishrefugeecouncil.org.uk/wp-content/uploads/2019/10/Holistic-Integration-Service-Year-2-report.pdf .

Weir KEA, Wilson SJ, Gorman DR. The Syrian vulnerable person resettlement programme: evaluation of Edinburgh’s reception arrangements. J Public Health (Bangkok). 2018;40(3):451–60. Available from: https://academic.oup.com/jpubhealth/article/40/3/451/4600209 .

Hammond CN. Scots 2- Engagement analysis of the New Scot Refugee Integration Strategy 2018–2022. 2018. Available from: https://www.gov.scot/binaries/content/documents/govscot/publications/research-and-analysis/2018/06/news-scots-2-engagement-analysis-new-scots-refugee-integration-strategy/documents/00537019-pdf/00537019-pdf/govscot%3Adocument/00537019.pdf .

Blake Stevenson. A8 Nationals in Glasgow. 2007. Available from: http://crosshillandgovanhill.org.uk/grindocs/A8NationalsinGlasgow.pdf .

Ajetunmobi O, Whyte B, Chalmers J, Fleming M, Stockton D, Wood R. Informing the ‘early years’ agenda in Scotland: understanding infant feeding patterns using linked datasets. J Epidemiol Community Health. 2014;68(1):83–92. Available from: https://jech.bmj.com/lookup/doi/10.1136/jech-2013-202718 .

Laidlaw K, Wang D, Coelho C, Power M. Attitudes to ageing and expectations for filial piety across Chinese and British cultures: a pilot exploratory evaluation. Aging Ment Health. 2010;14(3):283–92. Available from: http://www.tandfonline.com/doi/abs/10.1080/13607860903483060 .

Marsden R, Aldegheri E, Khan A, Lowe M, Strang A, Salinas E, et al. “What’s going on?” A study into destitution and poverty faced by asylum seekers and refugees in Scotland. 2005. Available from: http://www.rst.org.uk/wp-content/uploads/2012/11/Whats_going_on_A_study.pdf .

Quinn N. Participatory action research with asylum seekers and refugees experiencing stigma and discrimination: the experience from Scotland. Disabil Soc. 2014;29(1):58–70. Available from: http://www.tandfonline.com/doi/abs/10.1080/09687599.2013.769863 .

British Red Cross, Refugee Survival Trust. How will we survive? Steps to preventing destitution in the asylum system. 2021. Available from: https://mcusercontent.com/c17c136fc126588cb51e5471d/files/a35dd0e1-d785-f962-6a41-01e928493775/DASS_Research_Report_2021.pdf .

O’Donnell R, Angus K, McCulloch P, Amos A, Greaves L, Semple S. Fathers’ views and experiences of creating a smoke-free home: a scoping review. Int J Environ Res Public Health. 2019;16(24):5164. Available from: https://www.mdpi.com/1660-4601/16/24/5164 .

Sweileh WM, Wickramage K, Pottie K, Hui C, Roberts B, Sawalha AF, et al. Bibliometric analysis of global migration health research in peer-reviewed literature (2000–2016). BMC Public Health. 2018;18(1):777. Available from: https://bmcpublichealth.biomedcentral.com/articles/10.1186/s12889-018-5689-x .

Wren K. Supporting asylum seekers and refugees in glasgow: the role of multi-agency networks. J Refug Stud. 2007;20(3):391–413. Available from: https://academic.oup.com/jrs/article-lookup/doi/10.1093/jrs/fem006 .

UK Government Home Office. A Fairer Asylum Accommodation System. 2022. Available from: https://www.emcouncils.gov.uk/write/Migration/Asylum_Dispersal_Factsheet_PDF.pdf .

Scottish Refugee Council. Scotland’s Welcome: an analysis of community support for refugee integration. 2020. Available from https://scottishrefugeecouncil.org.uk/wp-content/uploads/2021/01/Community-support-analysis-2020.pdf .

Sturge G, UK Parliament House of Commons Library Asylum statistics Research Briefing. 2023. Available from: https://commonslibrary.parliament.uk/research-briefings/sn01403/#:~:text=IntheyearendingJune,ofimmigrantstotheUK .

The Migration Observatory. Where do migrants live in the UK? The Migration Observatory at the University of Oxford. 2022. Available from: https://migrationobservatory.ox.ac.uk/resources/briefings/where-do-migrants-live-in-the-uk .

Pavli A, Maltezou H. Health problems of newly arrived migrants and refugees in Europe. J Travel Med. 2017;24(4). Available from: http://academic.oup.com/jtm/article/doi/10.1093/jtm/tax016/3095987/Health-problems-of-newly-arrived-migrants-and .

Humphris R, Bradby H. Health Status of Refugees and Asylum Seekers in Europe. In: Oxford Research Encyclopedia of Global Public Health. Oxford University Press; 2017. Available from: https://oxfordre.com/publichealth/view/10.1093/acrefore/9780190632366.001.0001/acrefore-9780190632366-e-8 .

Bradby H, Humphris R, Newall D, Phillimore J. Public Health Aspects of Migrant Health: A Review of the Evidence on Health Status for Refugees and Asylum Seekers in the European Region. (Health Evidence Network Synthesis Report, No. 44.) ANNEX 2, DEFINITIONS OF REFUGEES, ASYLUM SEEKERS AND MIGRANTS IN THE LITERATURE. Copenhagen: Eerat; 2015. Available from:  https://www.ncbi.nlm.nih.gov/books/NBK379415/ .

Gerritsen AAM, Bramsen I, Devillé W, van Willigen LHM, Hovens JE, van der Ploeg HM. Physical and mental health of Afghan, Iranian and Somali asylum seekers and refugees living in the Netherlands. Soc Psychiatry Psychiatr Epidemiol. 2006;41(1):18–26. Available from: http://link.springer.com/10.1007/s00127-005-0003-5 .

Laue J, Diaz E, Eriksen L, Risør T. Migration health research in Norway: a scoping review. Scand J Public Health. 2023;51(3):381–90. Available from: http://journals.sagepub.com/doi/10.1177/14034948211032494 .

Scottish Refugee Council. Ukraine response one year on. 2023. Available from: https://scottishrefugeecouncil.org.uk/ukraine-response-one-year-on/ . Cited 2023 Aug 26.

Woodward A, Howard N, Wolffers I. Health and access to care for undocumented migrants living in the European Union: a scoping review. Health Policy Plan. 2014;29(7):818–30. Available from: https://academic.oup.com/heapol/article-lookup/doi/10.1093/heapol/czt061 .

Simon J, Kiss N, Laszewska A, Mayer S. Public health aspects of migrant health: a review of the evidence on health status for labour migrants in the European Region. Health Evidence Network Synthesis Report 43. 2015. Available from: http://www.epgencms.europarl.europa.eu/cmsdata/upload/114f16b6-1667-44ab-802b-a5a83dd50af0/WHO-HEN-Report-A5-1-Labour-FINAL_EN.pdf .

Scottish Government. Seasonal migrant workers in Scottish agriculture: research report. 2023. Available from: https://www.gov.scot/publications/seasonal-migrant-workers-scottish-agriculture/pages/10/ .

Scottish Government. New Scots: refugee integration strategy 2018–2022. 2018. Available from: https://www.gov.scot/publications/new-scots-refugee-integration-strategy-2018-2022/pages/11/ .

Oliva A, Palavra V, Caloun J. Refugees in Scotland: understanding the policy domain. 2016. Available from: https://www.academia.edu/34097718/REFUGEES_IN_SCOTLAND_UNDERSTANDING_THE_POLICY_DOMAIN .

Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73. Available from: https://www.acpjournals.org/doi/10.7326/M18-0850 .

Download references

Acknowledgements

Thank-you to Professor Anne MacFarlane and PHD student Anne Cronin, of the University of Limerick, Ireland for sharing the coding guidelines currently used in an update to Villarroel et. al’s 2019 study on Migrant Health in the Republic of Ireland.

No funding was received for this work, which was undertaken as G. Petrie’s Master of Public Health dissertation module at the University of Stirling.

Author information

Authors and affiliations.

Caledonia House, NHS Greater Glasgow and Clyde, Glasgow, Scotland, UK

Institute for Social Marketing and Health, Faculty of Health Sciences and Sport, University of Stirling, Stirling, FK9 4LA, Scotland, UK

K. Angus & R. O’Donnell

You can also search for this author in PubMed   Google Scholar

Contributions

KA, RO and GP finalised the study design collectively. GP conducted the searches, analysis and write up, with support from KA and RO. All three authors read and approved the manuscript prior to submission.

Corresponding author

Correspondence to R. O’Donnell .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., supplementary material 3., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Petrie, G., Angus, K. & O’Donnell, R. A scoping review of academic and grey literature on migrant health research conducted in Scotland. BMC Public Health 24 , 1156 (2024). https://doi.org/10.1186/s12889-024-18628-1

Download citation

Received : 04 September 2023

Accepted : 16 April 2024

Published : 25 April 2024

DOI : https://doi.org/10.1186/s12889-024-18628-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Asylum seekers
• Scoping review
• Research funding
• Immigration

BMC Public Health

ISSN: 1471-2458

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Open access
• Published: 21 June 2023

Anxiety increased among children and adolescents during pandemic-related school closures in Europe: a systematic review and meta-analysis

• Helena Ludwig-Walz   ORCID: orcid.org/0000-0003-1558-8241 1 ,
• Indra Dannheim   ORCID: orcid.org/0000-0002-4478-7808 2 , 3 ,
• Lisa M. Pfadenhauer   ORCID: orcid.org/0000-0001-5038-8072 4 , 5 ,
• Jörg M. Fegert   ORCID: orcid.org/0000-0001-6070-4323 6 &
• Martin Bujard   ORCID: orcid.org/0000-0002-3603-4160 1 , 7  

Child and Adolescent Psychiatry and Mental Health volume  17 , Article number:  74 ( 2023 ) Cite this article

2065 Accesses

11 Citations

4 Altmetric

Metrics details

Considering the heterogenous evidence, a systematic review of the change in anxiety in European children and adolescents associated with the COVID-19 pandemic is lacking. We therefore assessed the change compared with pre-pandemic baselines stratified by gender and age as well as evaluated the impact of country-specific restriction policies.

A registration on the ‘International Prospective Register of Systematic Reviews’ (PROSPERO) occurred and an a priori protocol was published. We searched six databases (PubMed, Embase, PsycINFO, Cochrane Central Register of Controlled Trials, Web of Science, WHO COVID-19) using a peer-reviewed search string with citation tracking and grey literature screening. Primary outcomes were: (1) general anxiety symptoms; and (2) clinically relevant anxiety rates. We used the Oxford COVID-19 Stringency Index as an indicator of pandemic-related restrictions. Screening of title/abstract and full text as well as assessing risk of bias (using the ‘Risk of Bias in Non-randomized Studies of Exposure’ [ROBINS-E]) and certainty of evidence (using the ‘Grading of Recommendations Assessment, Development and Evaluation’ [GRADE]) was done in duplicate. We pooled data using a random effects model. Reporting is in accordance with the Preferred Reporting Items for Systematic review and Meta-Analysis (PRISMA) statement.

Of 7,422 non-duplicate records, 18 studies with data from 752,532 pre-pandemic and 763,582 pandemic participants met full inclusion criteria. For general anxiety symptoms the total change effect estimate yielded a standardised mean difference (SMD) of 0.34 (95% confidence interval [CI], 0.17–0.51) and for clinically relevant anxiety rates we observed an odds ratio of 1.08 (95%-CI, 0.98–1.19). Increase in general anxiety symptoms was highest in the 11–15 years age group. Effect estimates were higher when pandemic-related restrictions were more stringent (Oxford Stringency Index > 60: SMD, 0.52 [95%-CI, 0.30–0.73]) and when school closures (School Closure Index ≥ 2: SMD, 0.44 [95%-CI, 0.23–0.65]) occurred.

General anxiety symptoms among children and adolescents in Europe increased in a pre/during comparison of the COVID-19 pandemic; particularly for males aged 11–15 years. In periods of stringent pandemic-related restrictions and/or school closures a considerable increase in general anxiety symptoms could be documented.

PROSPERO registration: CRD42022303714.

Mental disorders are important causes of disease burden among children and adolescents [ 1 , 2 ]. Even before the COVID-19 pandemic, the burden of disease study highlighted that anxiety disorders were the most prevalent condition in 2019 among young people in Europe. Among mental health conditions, such disorders represented a leading cause of years lived with disability [ 2 , 3 ]. In this regard, a link can be drawn between the non-treatment or undertreatment of anxiety disorders in childhood and adolescence and mental illnesses in adulthood, such as anxiety, depression and substance use disorders [ 4 , 5 ]. Anxiety is generally defined as feelings of concern that appear to have no obvious cause, but are sufficiently persistent and severe to affect daily life [ 6 ]. With the onset of the COVID-19 pandemic, the implementation of a broad range of public health and social measures (PHSM) [ 7 ] served to exacerbate many determinants of poor mental health. In particular, the environment of children and adolescents has been changed considerably by PHSM, which comprise school and leisure facilities closing, fewer peer interactions, changes in the family system as a result of the requirement to work from home, and quarantine orders [ 7 , 8 , 9 ]. As already known from previous studies [ 10 , 11 , 12 , 13 ], such changes can lead to serious impairments in young people’s mental health. To date, the impact of the COVID-19 pandemic on anxiety has been assessed primarily for the adult population [ 14 , 15 , 16 ] or its global prevalence for children and adolescents [ 17 , 18 , 19 ]. Existing European studies with a pre-pandemic baseline showed heterogeneous results [ 20 , 21 , 22 , 23 ]. However, a deeper understanding of changes in anxiety symptoms in the young population group is lacking, especially for the European continent.

An up-to-date examination of changes in anxiety symptoms among children and adolescents is therefore imperative and of great public health (PH) relevance in order to counteract suboptimal developments [ 2 ]. An analysis of the changes in the European continent means that the lack of an evidence base in the subgroup-stratified summary among children and adolescents can be rectified. It also allows for the use of a quasi-experimental design by analysing the impact of heterogeneous pandemic-related interventions in the European countries. Hence, the aim of this systematic review and meta-analysis is to identify, critically assess, summarise, and determine the certainty of evidence (CoE) regarding the impact of the COVID-19 pandemic on anxiety among children and adolescents in Europe compared with the pre-pandemic baseline. Thereby, it aims to provide information about the relevance of pandemic-related restrictions which will contribute to the analysis and the lessons learned from the immediate restrictions taken to safeguard the population in various European countries.

This systematic review and meta-analysis is reported according to the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) [ 24 ] statement (Additional file 1 : Table S1). Our protocol is registered on the ‘International Prospective Register of Systematic Reviews’ (PROSPERO; CRD42022303714) [ 25 ] and was published a priori [ 26 ]; any deviations from the original review protocol are presented in Additional file 1 : Table S2.

Data sources, search strategy and eligibility criteria

We searched for published articles in six electronic databases (PubMed, Embase, PsycINFO, Cochrane Central Register of Controlled Trials, Web of Science, WHO COVID-19 database [including pre-prints]), up to 18 March, 2022. Additionally, we enlarged our searches by examining previous systematic reviews and meta-analysis on the same topic, checking reference lists in included studies and searching relevant grey literature sources such as reports issued by key organisations and abstracts of relevant conferences up to 16 April, 2022; more information on the screened key organisations and conferences is provided in Additional file 1 : Table S3.

We developed the search strategy according to the Population–Exposure–Comparison–Outcome (PECO) [ 27 ] scheme and included the following key search terms: children and adolescents (population), COVID-19 (exposure) and anxiety (outcome). The availability of a pre-pandemic baseline (comparison) was assessed manually. The six tailored search strategies can be found in Additional file 1 : Table S4. The search strategy was reviewed by a search specialist using the evidence-based checklist ‘Peer Review of Electronic Search Strategies’ (PRESS) [ 28 ].

Our pre-defined eligibility criteria were equally defined according to the PECO [ 27 ] scheme:

Population: Children and adolescents ≤ 19 years, living in the WHO European region [ 29 ].

Exposure: Participation in survey during the COVID-19 pandemic.

Comparison: Pre-pandemic baseline.

Primary outcomes: Measurements of general anxiety symptoms or clinically relevant anxiety rates; no secondary outcomes were considered.

We excluded studies undertaken in children and adolescents with pre-existing psychiatric diagnoses. No limits regarding language and effect measurement were applied, however our search strategy was designed and run in English. Publications drawing upon the same study population and measurement time points were included as one item. When measurement time points varied during the COVID-19 pandemic, each measurement time point was considered individually.

Selection process and data extraction

After deduplication, two reviewers (HLW, ID) used the recommended EPPI reviewer software [ 30 ] to independently screen first titles and abstracts, and second full texts, in accordance with the above eligibility criteria. Disagreements or uncertainty about eligibility were resolved through discussion. Reasons for exclusion after full text screening were recorded and are reported in a separate table (Additional file 1 : Table S5).

Further, two reviewers (HLW, ID) used piloted extraction forms to independently extract data from one third of the published studies and unpublished data requested from study authors. Remaining data extraction was completed by one reviewer (HLW) and verified by the other (ID). Differences in data extraction were discussed and resolved between the two reviewers. Our data extraction forms, in accordance with a former systematic review [ 31 ], included the following items: study information (first author, year of publication, country, study type), population and setting (sample size, % female, age of CA), COVID-19 determinants (time point of data measurement), pre-pandemic baseline (time point of data measurement, link between pre-pandemic population and the population during the pandemic) and outcomes (type of outcome, diagnostic instrument, psychometric properties of the diagnostic instrument, symptom reporter). We defined general anxiety symptoms and clinically relevant anxiety rates as primary outcomes. General self-reported measurements of anxiety were summarised as general anxiety symptoms. Since the measurement instruments and scales used varied considerably, the measurement data was standardised to standardised mean difference (SMD) with a 95% confidence interval (CI); this standardisation is also recommended by the Cochrane Handbook [ 32 ]. Measurements with a clinical cut-off or with a clinical diagnostic (International Statistical Classification of Diseases and Related Health Problems [ICD]) were summarised as clinically relevant anxiety rates and reported as odds ratio (OR) with a 95% CI. To describe PHSM restrictions in the measurement time frame of the studies and make them comparable, we used the Oxford COVID-19 Stringency Index [ 8 ] and the School Closure Index [ 8 ] as indicators. The Oxford COVID-19 Stringency Index consists of nine metrics including school closures, workplace closures and stay-at-home requirements. The index ranges from 0 (no restrictions) to 100 (most stringent restrictions) and was validated [ 8 ]. In accordance with the COVIDSurg Collaborative [ 33 ], we defined three categories: light restrictions (index < 20), moderate lockdowns (index 20–60) and full lockdowns (index > 60). The School Closure Index represents the handling of school closures and is an incorporated measurement in the Oxford COVID-19 Stringency Index, which was considered separately in our analyses. The index ranges from 0 to 3: 0 describes no restrictions; 1 contains recommended closure or all schools open with alterations resulting in significant differences compared with non-COVID-19 operations; 2 involves closure (only some levels or categories, e.g. just high school, or just public schools); and 3 requires closures at all levels [ 8 ]. We defined the cut-offs as ‘no or few alterations compared with a pre-COVID-19 situation’ (index < 2) and ‘partial or full school closure’ (index ≥ 2) [ 31 ]. We contacted nearly all study authors and asked to provide further unpublished data on age or gender-stratified data.

Risk of Bias assessment

Three reviewers (HLW, LMP, ID) independently assessed the risk of bias (RoB) in teams of two using the ‘Risk of Bias in Non-randomized Studies of Exposure’ (ROBINS-E) instrument [ 34 ]. For each study, the seven bias domains and a whole RoB assessment was revealed as either low, some concerns, high RoB, or very high RoB [ 34 ].

Data synthesis and statistical analyses

For the meta-analysis, we pooled effect estimates for general anxiety symptoms and clinically relevant anxiety rates in total and analysed different subgroups: gender (female/male), age (11–15, 16–19 years), Oxford Stringency Index (> 60/ ≤ 60) [ 8 ] and School Closure Index (≥ 2/ < 2) [ 8 ]. We used, where possible, results from adjusted analysis for pooling. If necessary, dichotomous data were transferred to SMD, using the formula recommended by Chinn [ 35 ]. Where multiple pre-pandemic measurements were available, the last measurement was used for calculation purposes. We excluded measurements, with combined anxiety/depression scores, from the meta-analysis. Where parent and self-reported data were presented [ 36 ], we gave preference to the self-reported data. Furthermore, within the meta-analysis, we grouped the studies according to their RoB rating; low/some concerns (= low) RoB studies and high RoB/very high RoB (= high) RoB studies were summarised both separately and in total. In particular, the pooled effect of the low RoB studies was taken for further interpretation. We used Review Manager 5.4.1 [ 37 ] and R Studio 4.2.1 [ 38 ] for data entry, statistical analysis, and graph creation. In all meta-analyses, random-effect models and the inverse-variance method with the ‘DerSimonian and Laird’ approach were used.

We investigated heterogeneity by using visual inspection of the forest plots as well as the Chi 2 test and I 2 index [ 39 ]. If I 2  > 50%, substantial heterogeneity was presumed. We conducted sensitivity analyses and meta-regression (if ≥ 10 studies per examined variable) to explain substantial heterogeneity [ 40 ]. Publication bias was analysed by visually interpreting funnel plots for signs of asymmetry [ 41 ] and statistically by calculating the Egger’s test (if ≥ 10 studies) [ 42 ].

Certainty of evidence

We assessed the overall CoE for each outcome using the ‘Grading of Recommendations Assessment, Development and Evaluation’ (GRADE) system and presented it along with the main findings of the review in a ‘Summary of findings’ table, based on a transparent format with defined applied criteria (Additional file 1 : Table S6) and a generated evidence profile (Additional file 1 : Table S7) [ 43 ]. The GRADE tool covers five categories for downgrading (RoB, imprecision, inconsistency, indirectness, publication bias) and three categories for upgrading (magnitude of effects, dose–response relationships, impact of residual confounding). The CoE could be rated as high, moderate, low or very low.

Our electronic search identified 7,420 non-duplicate records from database searches and additional two grey literature publications. Of these, 51 studies entered full-text screening. After a comprehensive screening process, detailed in the PRISMA flow diagram (Additional file 1 : Figure S1), we included 18 studies with 22 effect measures, comprising 16 peer-reviewed studies [ 20 , 22 , 23 , 36 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 ], one report [ 56 ], and one pre-print [ 21 ]. Reasons for exclusion after full-text screening are described in Additional file 1 : Table S5.

Study characteristics

The characteristics of each of the studies that were included are described in Table 1 . The total population sample included data from 752,532 pre-pandemic and 763,582 pandemic participants (broken down into general anxiety symptoms: 11,425 pre-pandemic and 13,387 pandemic participants; clinically relevant anxiety rates: 741,107 pre-pandemic and 750,195 pandemic participants). Studies were carried out in a range of countries: four in Germany [ 21 , 44 , 45 , 56 ], four in the United Kingdom [ 22 , 23 , 36 , 55 ], three in Italy [ 47 , 48 , 49 ], two in Spain [ 51 , 52 ], two in Switzerland [ 53 , 54 ], and one in Israel [ 46 ], one in the Netherlands [ 20 ], and one in Norway [ 50 ], respectively. Most of the studies measured general anxiety symptoms in spring/summer 2020 (14 effect measures) [ 20 , 21 , 22 , 23 , 36 , 45 , 46 , 47 , 49 , 50 , 51 , 53 , 54 , 55 ], while two effect measurements were conducted in autumn 2020 [ 21 , 22 ] and three in winter 2020/spring 2021 [ 21 , 51 , 52 ]. Clinically relevant anxiety rates were analysed in four studies [ 44 , 45 , 48 , 56 ]. Of the included studies, 17 [ 20 , 21 , 22 , 23 , 36 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 52 , 53 , 54 , 55 , 56 ] reported data for children and adolescents over the age of 11 and 11 studies [ 20 , 36 , 44 , 45 , 47 , 48 , 49 , 51 , 54 , 55 , 56 ] for children and adolescents under the age of 11. The measurement time point was rated as ‘full lockdown’ (Oxford Stringency Index > 60) in 14 studies [ 20 , 22 , 23 , 36 , 44 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 ] and partial or full school closure occured in 11 studies (School Closure Index ≥ 2) [ 20 , 21 , 22 , 23 , 36 , 46 , 47 , 48 , 49 , 51 , 55 ]. In addition, 12 studies [ 20 , 21 , 22 , 23 , 36 , 44 , 45 , 47 , 49 , 53 , 55 , 56 ] provided further study data (generally unpublished gender-stratified and age-stratified data). The effect estimates of the 18 studies that were included are summarised in Additional file 1 : Table S8. The RoB assessment revealed a ‘some concerns’ rating for six studies [ 20 , 21 , 44 , 46 , 53 , 56 ], a ‘high RoB’ rating for eight studies [ 22 , 23 , 36 , 45 , 47 , 50 , 51 , 55 ] and a ‘very high RoB’ rating for four studies [ 48 , 49 , 52 , 54 ]. Detailed rating information is provided in Additional file 1 : Figure S2 (traffic-light plot) and Additional file 1 : Figure S3 (weighted-bar plot).

Meta-analysis of general anxiety symptoms

For general anxiety symptoms, 12 studies [ 20 , 21 , 22 , 23 , 36 , 45 , 46 , 48 , 49 , 52 , 53 , 55 ] were pooled and CoE was graded as ‘very low’ (Table 2 ; further information in Additional file 1 : Table S7). In a pooling of four low RoB studies with six measures, a total change of a SMD of 0.34 (95% CI, 0.17 to 0.51, I 2  = 96%; Fig.  1 ) was calculated. Following gender stratification, a SMD of 0.30 (95% CI, 0.12 to 0.49, I 2  = 90%; Additional file 1 : Figure S4) for females and 0.34 (95% CI, 0.07 to 0.60, I 2  = 95%; Additional file 1 : Figure S5) for males in low RoB studies was revealed. Age-stratified pooling was possible for the 11–15 years age category with three studies [ 20 , 21 , 53 ] and five effect measures, and for the 16–19 years age category with two studies [ 20 , 21 ] and four effect measures. For the 11–15 years age category, the total change effect estimate yielded a SMD of 0.39 (95% CI, 0.18 to 0.60, I 2  = 93%; Additional file 1 : Figure S6). Change effect estimates were also evident for females (SMD, 0.34; 95% CI, 0.19 to 0.49; I 2  = 71%; Additional file 1 : Figure S7) and males (SMD, 0.45; 95% CI, 0.15 to 0.74; I 2  = 93%; Additional file 1 : Figure S8). Pooling within the 16–19 years age category revealed a SMD of 0.24 (95% CI, -0.01 to 0.49, I 2  = 92%; Additional file 1 : Figure S9) in total, a SMD of 0.18 (95% CI, -0.01 to 0.37; I 2  = 75%; Additional file 1 : Figure S10) for females and a SMD of 0.31 (95% CI, -0.02 to 0.63; I 2  = 92%; Additional file 1 : Figure S11) for males.

Forest plot of changes in youth general anxiety symptoms comparing before and during COVID-19 pandemic. SE, standard error; SMD, standardized mean differences; RoB, risk of bias; 95%-CI, 95%-confidence interval

To estimate the extent to which the stringency of PHSM has an impact on anxiety symptoms, low RoB studies were pooled by the Oxford COVID-19 Stringency Index (> 60 vs ≤ 60) and the School Closure Index (≥ 2 and < 2). An increase in general anxiety symptoms was observed for the Oxford COVID-19 Stringency Index > 60 (SMD, 0.52; 95% CI, 0.30 to 0.73; I 2  = 96%; Fig.  2 ) and the School Closure Index ≥ 2 (SMD, 0.44; 95% CI, 0.23 to 0.65; I 2  = 96%; Fig.  3 ).

Forest plot of changes in youth general anxiety symptoms comparing Oxford Stringency Index. SE, standard error; SMD, standardized mean differences; SI, stringency index; 95%-CI, 95%-confidence interval

Forest plot of changes in youth general anxiety symptoms comparing School Closure Index. SE, standard error; SMD, standardized mean differences; SL, School Closure Index; 95%-CI, 95%-confidence interval

Meta-analysis of clinically relevant anxiety rates

For clinically relevant anxiety rates, four studies [ 44 , 45 , 48 , 56 ] were pooled and CoE was graded as ‘very low’ (Table 2 ; further information in Additional file 1 : Table S7). Total change yielded an OR of 1.08 (95% CI, 0.98 to 1.19, I 2  = 82%; Fig.  4 ) in two low RoB studies [ 44 , 56 ]. Clinically relevant anxiety rates increased significantly in females in low RoB studies (OR, 1.10 [95% CI, 1.02 to 1.19], I 2  = 52%; Additional file 1 : Figure S12), but not for males (OR, 1.04 [95% CI, 0.92 to 1.17], I 2  = 76%; Additional file 1 : Figure S13).

Forest plot of changes in youth clinically relevant anxiety symptoms comparing before and during COVID-19 pandemic. OR, Odds Ratio; RoB, risk of bias; 95%-CI, 95%-confidence interval

Heterogeneity, publication bias and sensitivity analysis

As heterogeneity was substantial in all meta-analyses (I 2  > 50%), meta-regression analyses were conducted for the total population, female and male children and adolescents. In every meta-regression analysis, ‘RoB’ and ‘study design’ represent positive covariates (Additional file 1 : Tables S9-14). The covariate ‘RoB’ was addressed by the aforementioned stratification of low vs high RoB studies. Effect direction and significance did not change after removing the study with cross-sectional design. Sensitivity analyses (Additional file 1 : Table S15) revealed significant differences for study design and effect conversion. However, only one cross-sectional study and one study with converted measurements were included in the analyses. Effect direction and significance did not alter after removing these studies from meta-analyses. Visual analysis of the (contour-enhanced) funnel plots implied asymmetry (Additional file 1 : Figures S14–S19), but was discarded by applying Egger’s test (Additional file 1 : Table S16).

This systematic review and meta-analysis provides insights into the changes in general anxiety symptoms and clinically relevant anxiety rates in European children and adolescents after the onset of the COVID-19 pandemic when compared with the pre-pandemic baseline. We included 18 studies that assessed changes in over 750,000 children and adolescents (for several measurement points) across Europe. The pooled effect estimates of low RoB studies revealed an increase in general anxiety symptoms overall, and particularly for males in the 11–15 years age category. A significant increase in clinically relevant anxiety rates was also observed among female children and adolescents.

Considering the various different restriction policies in European countries, this systematic review and meta-analysis is the first that assessed the association between PHSM and higher general anxiety symptoms. For children above six years of age, school closures have been a major disruptor as these measures radically changed their life [ 9 ]. Instead of having social contact five days a week, often for six or eight hours a day with their class, peers and friends, they were homebound and unable to socialise properly. These full or partial school closures affected approximately 105 million pupils and students in Europe [ 57 ]. Our meta-analyses revealed particularly high general anxiety symptoms during periods of school closure (SMD, 0.44; 95% CI, 0.23 to 0.65) and other restriction measures (SMD, 0.52; 95% CI, 0.30 to 0.73); these effect increases outlined a potential impact of school closures and PHSM on anxiety symptoms. However, the evidence rating of "very low" have to be considered here; therefore, further reseach is needed. Both effect estimates were higher than in a previous meta-analysis on depression [ 31 ]. As social anxiety can be reduced through exposure to social interactions, the non-exposure to social contacts and social challenges in the school environment as a result of PHSM may explain the stronger correlation with the symptomatology. Further research will allow a comparison of the reduction in anxiety symptoms between subgroups and countries following the acute pandemic phase. Our results suggest that the higher association with restrictive measures could lead to a more rapid reduction in symptoms once life returns to normal. Nevertheless, social exclusion of children and adolescents during the pandemic could lead to life-long mental and physical health consequences [ 3 , 58 , 59 ]. However, a clearcut separation of the effects on anxiety due to school closure from those due to other pandemic related restrictions—like worries about (elderly) relatives, fear of long-lasting health effects (long COVID), and also closure of recreational and sports facilities—was not possible. This limitation was already found in a previous review [ 19 ]. Therefore, our results must be interpreted indicative regarding the possible drivers for the increased anxiety.

Regarding different subgroups, our analyses first showed strong differences between studies with low and high RoB. While the increase in general anxiety symptoms is clearly evident for studies with a low RoB, the pooling effects of high RoB studies were indistinct and non-significant. The heterogeneous evidence in literature can partly be attributed to the different quality of existing studies; this underlines the importance of strictly assessing the RoB. Second, age-specific analyses found considerably higher effect estimates for children and adolescents aged 11–15 years, in particular among males, but lower and more imprecise estimates for those aged 16–19 years. Taking into account the fact that the risk of anxiety disorders among children aged 10–14 years had already been reported as being high three decades before the COVID-19 pandemic [ 2 ], our findings showed that this age group was also more vulnerable to increases during the COVID-19 pandemic. The imprecise results for general anxiety symptoms among males in the 16–19 year age category are in contrast to findings on depression [ 31 ]. This underlines the necessity to differentiate between different mental health diagnoses in specific age groups in the COVID-19 pandemic. Third, for clinically relevant anxiety rates, the pooled associations were based on two low RoB studies from Germany and should be interpreted with caution; further empirical evidence is needed here.

This paper has strong implications for both policy and clinical practice. Policy-makers should consider the unintended consequences before imposing PHSM such as school closures on the mental health of children and adolescents. Psychiatrists, psychotherapists and other public health experts for children and adolescents should therefore be included in pandemic crisis task forces [ 60 , 61 ]. The increase in general anxiety symptoms and the variation between specific groups and countries requires children and adolescents to be closely monitored over the next few years. This monitoring should cover a broad range of age groups, similar to the recommendation of the U.S. Preventive Services Task Force to screen all children and adolescents aged 8–18, regardless of whether they have symptoms [ 62 ]. Based on our study, children and adolescents born in 2005 to 2010 (aged 11–15 years in 2020 to 2021) should be monitored henceforth. While our study indicates a strong need for anxiety disorder therapies (like previous research for depression symptoms [ 31 ]), these professionals were understaffed even before the pandemic [ 6 ]. Policy makers should therefore strengthen availability and capacity of these professional groups.

Screening and adequate diagnoses are important for identifying children and adolescents with anxiety disorders and the need for therapy. The gap between studies measuring general anxiety symptoms and those measuring clinically relevant anxiety rates in our systematic review might indicate a lack of clinical evidence and diagnoses. Parents, teachers, health care professionals and sports trainers should be made aware of risk factors and symptoms of anxiety disorders as well as mental health services. The negative consequences in later life of a failure to address anxiety symptoms on children and adolescents are well documented [ 3 , 58 , 59 ]. Moreover, even before the COVID-19 pandemic, anxiety and depression disorders were two of the top five causes of overall disease burden for children and adolescents in Europe, and suicide was a leading cause of death among 10–19-year-olds in the WHO European region [ 3 ]. It is therefore important to implement evidence-based interventions that can help address mental health issues in children. Targeted interventions and longer programmes in particular seemed to be more effective [ 3 ]. In addition, protective factors should be communicated and supported; including parent–child dialogue [ 63 ], a predictable home environment [ 64 ], peer-to-peer social contact [ 65 ] and physical activity [ 66 ]. Further, increased resilience among child and adolescents could be a predictor of fewer anxiety symptoms [ 67 , 68 ].

There are several research gaps regarding anxiety symptoms in the COVID-19 pandemic in Europe, including evidence for children aged below 10 years, differentiation by social status or education, and clinically relevant anxiety rates. Generally, there are only a very small number of studies on anxiety with a pre-pandemic baseline in Europe, although no such studies were able to be included for Eastern European countries and hardly any evidence from southern Europe. To improve this, representative longitudinal cohort or panel studies on CA should be conducted in all European countries so as to have a pre-crisis baseline and to monitor changes over time. Such a cohort or panel should include validated anxiety measures for general symptoms and for a clinically relevant cut-off, as well as demographic, socioeconomic and health-related confounders. These criteria are necessary in order to reduce the RoB and to allow subgroup-specific analyses.

Strength and limitations

There are several limitations to this review. First, RoB was high for 12 studies (66% of the studies included), mainly based on bias due to participant selection, missing data and insufficient adjustment of important confounders. This limitation was addressed by downgrading for RoB in GRADE and we stratified our meta-analyses by RoB. Second, the instruments that were used differed greatly in their scales. To unify them, we transformed the effect estimates to SMD or OR. Third, there was a high level of heterogeneity in the meta-analyses (I 2  > 50%), which we tried to explain by conducting meta-regression analyses. Fourth, no country pooling and visualisation over time were possible due to the low study quality. There were only a small number of available studies within our strict inclusion criteria with age-group-specific data. Fifth, there is a lack of longitudinal studies. Sixth, more subgroup analyses were not feasible. Seventh, the Oxford Stringency Index [ 8 ] and the School Closure Index [ 8 ] were used as proxies for PHSM and cannot cover all facets of the COVID-19 pandemic.

The strengths of this review are that it largely follows the methodological guidelines recommended by the Cochrane Handbook for systematic reviews [ 32 ], such as systematic search in several databases with a peer-reviewed search strategy and consideration of pre-prints, grey literature, and conference abstracts. In addition, literature screening, data extraction and RoB rating were performed independently and unpublished data was requested from study authors. In addition, the assessment of the RoB and the CoE was conducted using recommended tools. Thus, an assessment of evidence based on high quality studies was possible, allowing contradictory findings from previous studies to be properly interpreted.

This systematic review and meta-analysis showed an increase in general anxiety symptoms among European children and adolescents during the first two years of the COVID-19 pandemics compared with a pre-pandemic baseline. The 11–15 years male age group was particularly affected. Social distancing policies implemented in European countries, and in particular school closures, might be associated with a considerable increase in the effect of general anxiety symptoms. Therefore, school closures should be implemented only with the greatest caution and with consideration of the evidence available regarding the mental health of children and adolescents. At present, the need is huge to monitor anxiety symptoms in children and adolescents on a long-term basis and to identify which of the 105 million children and adolescents in Europe have disorders that require professional management and treatment. Due to long-term consequences of anxiety disorders and the risk of suicidality, those affected have to be clinically addressed through early identification and therapy.

Availability of data and materials

All data are included in the manuscript and appendix.

Reiner RC, Olsen HE, Ikeda CT, et al. Diseases, Injuries, and Risk Factors in Child and Adolescent Health, 1990 to 2017: findings From the Global Burden of Diseases, Injuries, and Risk Factors 2017 Study. JAMA Pediatr. 2019;173:e190337. https://doi.org/10.1001/jamapediatrics.2019.0337 .

Article   PubMed   PubMed Central   Google Scholar  

Castelpietra G, Knudsen AKS, Agardh EE, et al. The burden of mental disorders, substance use disorders and self-harm among young people in Europe, 1990–2019: Findings from the Global Burden of Disease Study 2019. Lancet Reg Health Eur. 2022;16:100341. https://doi.org/10.1016/j.lanepe.2022.100341 .

Bruckmayer M, Phillips W. Children and mental health: Preventive approaches to anxiety and depression : European platform for investing in children. Luxembourg: Publications Office of the European Union; 2021.

Google Scholar  

Copeland WE, Angold A, Shanahan L, et al. Longitudinal patterns of anxiety from childhood to adulthood: the Great Smoky Mountains Study. J Am Acad Child Adolesc Psychiatry. 2014;53:21–33. https://doi.org/10.1016/j.jaac.2013.09.017 .

Article   PubMed   Google Scholar  

Monk NJ, McLeod GFH, Mulder RT, et al. Childhood anxious/withdrawn behaviour and later anxiety disorder: a network outcome analysis of a population cohort. Psychol Med. 2021;1:1–12. https://doi.org/10.1017/S0033291721002889 .

Article   Google Scholar  

Liu J, Chen X, Lewis G. Childhood internalizing behaviour: analysis and implications. J Psychiatr Ment Health Nurs. 2011;18:884–94. https://doi.org/10.1111/j.1365-2850.2011.01743.x .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Rehfuess EA, Movsisyan A, Pfadenhauer LM, et al. Public health and social measures during health emergencies such as the COVID-19 pandemic: an initial framework to conceptualize and classify measures. Influenza Other Respir Viruses. 2023;17:e13110. https://doi.org/10.1111/irv.13110 .

Hale T, Angrist N, Goldszmidt R, et al. A global panel database of pandemic policies (Oxford COVID-19 Government Response Tracker). Nat Hum Behav. 2021;5:529–38. https://doi.org/10.1038/s41562-021-01079-8 .

Fegert JM, Vitiello B, Plener PL, et al. Challenges and burden of the Coronavirus 2019 (COVID-19) pandemic for child and adolescent mental health: a narrative review to highlight clinical and research needs in the acute phase and the long return to normality. Child Adolesc Psychiatry Ment Health. 2020;14:20. https://doi.org/10.1186/s13034-020-00329-3 .

Brooks SK, Webster RK, Smith LE, et al. The psychological impact of quarantine and how to reduce it: rapid review of the evidence. The Lancet. 2020;395:912–20. https://doi.org/10.1016/S0140-6736(20)30460-8 .

Article   CAS   Google Scholar  

Almeida ILdL, Rego JF, Teixeira ACG, et al. Social isolation and its impact on child and adolescent development: a systematic review. Rev Paul Pediatr. 2021;40:e2020385. https://doi.org/10.1590/1984-0462/2022/40/2020385 .

Lacey RE, Kumari M, Bartley M. Social isolation in childhood and adult inflammation: evidence from the National Child Development Study. Psychoneuroendocrinology. 2014;50:85–94. https://doi.org/10.1016/j.psyneuen.2014.08.007 .

Kratzer S, Pfadenhauer LM, Biallas RL, et al. Unintended consequences of measures implemented in the school setting to contain the COVID-19 pandemic: a scoping review. Cochrane Database Syst Rev. 2022;6:CD15397. https://doi.org/10.1002/14651858.CD015397 .

Santomauro DF, Mantilla Herrera AM, Shadid J, et al. Global prevalence and burden of depressive and anxiety disorders in 204 countries and territories in 2020 due to the COVID-19 pandemic. The Lancet. 2021;398:1700–12. https://doi.org/10.1016/S0140-6736(21)02143-7 .

Hajek A, Neumann-Böhme S, Sabat I, et al. Depression and anxiety in later COVID-19 waves across Europe: New evidence from the European COvid Survey (ECOS). Psychiatry Res. 2022;317:114902. https://doi.org/10.1016/j.psychres.2022.114902 .

Article   PubMed Central   Google Scholar  

Thompson EJ, Stafford J, Moltrecht B, et al. Psychological distress, depression, anxiety, and life satisfaction following COVID-19 infection: evidence from 11 UK longitudinal population studies. The Lancet Psychiatry. 2022;9:894–906. https://doi.org/10.1016/S2215-0366(22)00307-8 .

Racine N, McArthur BA, Cooke JE, et al. Global Prevalence of Depressive and Anxiety Symptoms in Children and Adolescents During COVID-19: A Meta-analysis. JAMA Pediatr. 2021;175:1142–50. https://doi.org/10.1001/jamapediatrics.2021.2482 .

Viner RM, Mytton OT, Bonell C, et al. Susceptibility to SARS-CoV-2 infection among children and adolescents compared with adults: a systematic review and meta-analysis. JAMA Pediatr. 2021;175:143–56. https://doi.org/10.1001/jamapediatrics.2020.4573 .

Viner R, Russell S, Saulle R, et al. School closures during social lockdown and mental health, health behaviors, and well-being among children and adolescents during the first COVID-19 wave: a systematic review. JAMA Pediatr. 2022;176:400–9. https://doi.org/10.1001/jamapediatrics.2021.5840 .

Luijten MAJ, van Muilekom MM, Teela L, et al. The impact of lockdown during the COVID-19 pandemic on mental and social health of children and adolescents. Qual Life Res. 2021;30:2795–804. https://doi.org/10.1007/s11136-021-02861-x .

Ravens-Sieberer U, Erhart M, Devine J, et al. Child and adolescent mental health during the COVID-19 pandemic: results of the three-wave longitudinal COPSY study. SSRN J. 2022. https://doi.org/10.2139/ssrn.4024489 .

Widnall E, Winstone L, Plackett R, et al. Impact of school and peer connectedness on adolescent mental health and well-being outcomes during the COVID-19 pandemic: a longitudinal panel survey. Int J Environ Res Public Health. 2022. https://doi.org/10.3390/ijerph19116768 .

Knowles G, Gayer-Anderson C, Turner A, et al. Covid-19, social restrictions, and mental distress among young people: a UK longitudinal, population-based study. J Child Psychol Psychiatry. 2022. https://doi.org/10.1111/jcpp.13586 .

Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:71. https://doi.org/10.1136/bmj.n71 .

Ludwig-Walz H, Fegert JM, Bujard M Depression and anxiety among children and adolescents during the COVID-19 pandemic in Europe: A systematic review: PROSPERO 2022. CRD42022303714

Ludwig-Walz H, Dannheim I, Pfadenhauer LM, et al. Anxiety among children and adolescents during the COVID-19 pandemic in Europe: a systematic review protocol. Syst Rev. 2023. https://doi.org/10.1186/s13643-023-02225-1 .

Morgan RL, Whaley P, Thayer KA, et al. Identifying the PECO: A framework for formulating good questions to explore the association of environmental and other exposures with health outcomes. Environ Int. 2018;121:1027–31. https://doi.org/10.1016/j.envint.2018.07.015 .

McGowan J, Sampson M, Salzwedel DM, et al. PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. J Clin Epidemiol. 2016;75:40–6. https://doi.org/10.1016/j.jclinepi.2016.01.021 .

WHO Regional Office for Europe (2022) Countries. https://www.euro.who.int/en/countries . Accessed 18 May 2022

Thomas J, Graziosi S, Brunton J et al. (2020) EPPI-Reviewer: advanced software for systematic reviews, maps and evidence synthesis.: EPPI-Centre Software., London: UCL Social Research Institute

Ludwig-Walz H, Dannheim I, Pfadenhauer LM, et al. Increase of depression among children and adolescents after the onset of the COVID-19 pandemic in Europe: a systematic review and meta-analysis. Child Adolesc Psychiatry Ment Health. 2022;16:109. https://doi.org/10.1186/s13034-022-00546-y .

Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (ed) (2022) Cochrane Handbook for Systematic Reviews of Interventions: version 6.3. updated February 2022

COVIDSurg Collaborative,. Effect of COVID-19 pandemic lockdowns on planned cancer surgery for 15 tumour types in 61 countries: an international, prospective, cohort study. Lancet Oncol. 2021;22:1507–17. https://doi.org/10.1016/S1470-2045(21)00493-9 .

Higgins J, Morgan R, Rooney A, Taylor K, Thayer K, Silva R, Lemeris C, Akl A, Arroyave W, Bateson T, Berkman N, Demers P, Forastiere F, Glenn B, Hróbjartsson A, Kirrane E, LaKind J, Luben T, Lunn R, McAleenan A, McGuinness L, Meerpohl J, Mehta S, Nachman R, Obbagy J, O'Connor A, Radke E, Savović J, Schubauer-Berigan M, Schwingl P, Schunemann H, Shea B, Steenland K, Stewart T, Straif K, Tilling K, Verbeek V, Vermeulen R, Viswanathan M, Zahm S, Sterne J (2022) Risk Of Bias In Non-randomized Studies - of Exposure (ROBINS-E): Launch version. https://www.riskofbias.info/welcome/robins-e-tool

Chinn S. A simple method for converting an odds ratio to effect size for use in meta-analysis. Statist Med. 2000;19:3127–31. https://doi.org/10.1002/1097-0258(20001130)19:22%3C3127:AID-SIM784%3E3.0.CO;2-M .

Wright N, Hill J, Sharp H, et al. Interplay between long-term vulnerability and new risk: Young adolescent and maternal mental health immediately before and during the COVID-19 pandemic. JCPP Adv. 2021;1:e12008. https://doi.org/10.1111/jcv2.12008 .

The Cochrane Collaboration (2020) Review Manager (RevMan) [Computer program]

RStudio: Integrated Development Environment for R (2022). RStudio Team, Boston

Higgins JPT, Thompson SG, Deeks JJ, et al. Measuring inconsistency in meta-analyses. BMJ. 2003;327:557–60. https://doi.org/10.1136/bmj.327.7414.557 .

Deeks JJ, Higgins JP, Altman DG (2022) Chapter 10: Analysing data and undertaking meta-analyses. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (ed) Cochrane Handbook for Systematic Reviews of Interventions: version 6.3. updated February 2022

Sterne JAC, Sutton AJ, Ioannidis JPA, et al. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ. 2011;343:d4002. https://doi.org/10.1136/bmj.d4002 .

Egger M, Davey Smith G, Schneider M, et al. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315:629–34. https://doi.org/10.1136/bmj.315.7109.629 .

Schünemann HJ, Cuello C, Akl EA, et al. GRADE guidelines: 18. How ROBINS-I and other tools to assess risk of bias in nonrandomized studies should be used to rate the certainty of a body of evidence. J Clin Epidemiol. 2019;111:105–14. https://doi.org/10.1016/j.jclinepi.2018.01.012 .

Kostev K, Weber K, Riedel-Heller S, et al. Increase in depression and anxiety disorder diagnoses during the COVID-19 pandemic in children and adolescents followed in pediatric practices in Germany. Eur Child Adolesc Psychiatry. 2021. https://doi.org/10.1007/s00787-021-01924-1 .

Rau L-M, Grothus S, Sommer A, et al. Chronic pain in schoolchildren and its association with psychological wellbeing before and during the COVID-19 pandemic. J Adolesc Health. 2021;69:721–8. https://doi.org/10.1016/j.jadohealth.2021.07.027 .

Shoshani A, Kor A. The mental health effects of the COVID-19 pandemic on children and adolescents: risk and protective factors. Psychol Trauma. 2021. https://doi.org/10.1037/tra0001188 .

Frigerio A, Nettuno F, Nazzari S. Maternal mood moderates the trajectory of emotional and behavioural problems from pre- to during the COVID-19 lockdown in preschool children. Eur Child Adolesc Psychiatry. 2022. https://doi.org/10.1007/s00787-021-01925-0 .

Davico C, Marcotulli D, Lux C, et al. Impact of the COVID-19 pandemic on child and adolescent psychiatric emergencies. J Clin Psychiatry. 2021. https://doi.org/10.4088/JCP.20m13467 .

Crescentini C, Feruglio S, Matiz A, et al. Stuck Outside and Inside: An Exploratory Study on the Effects of the COVID-19 Outbreak on Italian Parents and Children’s Internalizing Symptoms. Front Psychol. 2020;11:586074. https://doi.org/10.3389/fpsyg.2020.586074 .

Hafstad GS, Sætren SS, Wentzel-Larsen T et al. (2021) Adolescents' symptoms of anxiety and depression before and during the Covid-19 outbreak - A prospective population-based study of teenagers in Norway. Lancet Reg Health Eur 5:100093. https://doi.org/10.1016/j.lanepe.2021.100093

Giménez-Dasí M, Quintanilla L, Fernández-Sánchez M. Longitudinal Effects of the Pandemic and Confinement on the Anxiety Levels of a Sample of Spanish Children in Primary Education. Int J Environ Res Public Health. 2021. https://doi.org/10.3390/ijerph182413063 .

Carrillo-Diaz M, Ortega-Martínez AR, Romero-Maroto M, et al. Lockdown impact on lifestyle and its association with oral parafunctional habits and bruxism in a Spanish adolescent population. Int J Paediatr Dent. 2022;32:185–93. https://doi.org/10.1111/ipd.12843 .

Ertanir B, Kassis W, Garrote A. Longitudinal Changes in Swiss Adolescent’s Mental Health Outcomes from before and during the COVID-19 Pandemic. Int J Environ Res Public Health. 2021. https://doi.org/10.3390/ijerph182312734 .

Borbás R, Fehlbaum LV, Dimanova P, et al. Mental well-being during the first months of Covid-19 in adults and children: behavioral evidence and neural precursors. Sci Rep. 2021;11:17595. https://doi.org/10.1038/s41598-021-96852-0 .

Bignardi G, Dalmaijer ES, Anwyl-Irvine AL, et al. Longitudinal increases in childhood depression symptoms during the COVID-19 lockdown. Arch Dis Child. 2020. https://doi.org/10.1136/archdischild-2020-320372 .

Witte J, Zeitler A, Hasemann L (2022) Krankenhausversorgung von Kindern und Jugendlichen während der Pandemie: Fokus: Psychische Erkrankungen. Ergebnisse des DAK Kinder und Jugendreports 2022 / Datenbasis: 2019 bis 2021, Bielefeld

United Nations, Department of Economic and Social Affairs, Population Division. World Population Prospects 2022, Online Edition; 2022

Lahey BB. Why are children who exhibit psychopathology at high risk for psychopathology and dysfunction in adulthood? JAMA Psychiat. 2015;72:865–6. https://doi.org/10.1001/jamapsychiatry.2015.0798 .

Viswanathan M, Wallace I, Middleton JC et al. (2022) Screening for Depression, Anxiety, and Suicide Risk in Children and Adolescents: An Evidence Review for the U.S. Preventive Services Task Force, Rockville (MD)

(2014) Health in All Policies (HiAP) framework for country action. Health Promot Int 29(Suppl 1):19–28. Doi: https://doi.org/10.1093/heapro/dau035

Moradian N, Moallemian M, Delavari F, et al. Interdisciplinary Approaches to COVID-19. Adv Exp Med Biol. 2021;1318:923–36. https://doi.org/10.1007/978-3-030-63761-3_52 .

Article   CAS   PubMed   Google Scholar  

Mangione CM, Barry MJ, Nicholson WK, et al. Screening for Anxiety in Children and Adolescents: US Preventive Services Task Force Recommendation Statement. JAMA. 2022;328:1438–44. https://doi.org/10.1001/jama.2022.16936 .

Ioffe M, Pittman LD, Kochanova K, et al. Parent-adolescent communication influences on anxious and depressive symptoms in early adolescence. J Youth Adolesc. 2020;49:1716–30. https://doi.org/10.1007/s10964-020-01259-1 .

Glynn LM, Davis EP, Luby JL, et al. A predictable home environment may protect child mental health during the COVID-19 pandemic. Neurobiol Stress. 2021;14:100291. https://doi.org/10.1016/j.ynstr.2020.100291 .

Espinoza G, Hernandez HL. Adolescent loneliness, stress and depressive symptoms during the COVID-19 pandemic: the protective role of friends. Infant Child Dev. 2022;31:e2305. https://doi.org/10.1002/icd.2305 .

Wolf S, Seiffer B, Zeibig J-M, et al. Is physical activity associated with less depression and anxiety during the COVID-19 pandemic? a rapid systematic review. Sports Med. 2021;51:1771–83. https://doi.org/10.1007/s40279-021-01468-z .

McGuinn LA, Rivera NR, Osorio-Valencia E, et al. Changes in depressive and anxiety symptoms during COVID-19 in children from the PROGRESS cohort. Pediatr Res. 2022. https://doi.org/10.1038/s41390-022-02379-z .

Masten AS, Lucke CM, Nelson KM, et al. Resilience in development and psychopathology: multisystem perspectives. Annu Rev Clin Psychol. 2021;17:521–49. https://doi.org/10.1146/annurev-clinpsy-081219-120307 .

Birmaher B, Brent DA, Chiappetta L, et al. Psychometric properties of the Screen for Child Anxiety Related Emotional Disorders (SCARED): a replication study. J Am Acad Child Adolesc Psychiatry. 1999;38:1230–6. https://doi.org/10.1097/00004583-199910000-00011 .

Chorpita BF, Yim L, Moffitt C, et al. Assessment of symptoms of DSM-IV anxiety and depression in children: a revised child anxiety and depression scale. Behav Res Ther. 2000;38:835–55. https://doi.org/10.1016/s0005-7967(99)00130-8 .

Achenbach TM, Rescorla L (2001) Manual for the ASEBA Preschool forms & profiles. Publicaciones de Psicología Aplicada. Serie Menor, núm. 80. ASEBA, Burlington, VT

Achenbach TM, Rescorla L (2001) Manual for the ASEBA school-age forms & profiles: An integrated system of multi-informant assessment. Publicaciones de Psicología Aplicada. Serie Menor, núm. 80. ASEBA, Burlington, VT

Howell CR, Gross HE, Reeve BB, et al. Known-groups validity of the Patient-Reported Outcomes Measurement Information System (PROMIS(®)) in adolescents and young adults with special healthcare needs. Qual Life Res. 2016;25:1815–23. https://doi.org/10.1007/s11136-016-1237-2 .

Finbråten HS, Kleppang AL, Steigen AM. Using the Rasch measurement theory to assess the psychometric properties of the Hopkins Symptom Checklist-10 in adolescents. Health Qual Life Outcomes. 2021;19:248. https://doi.org/10.1186/s12955-021-01884-9 .

Kleppang AL, Hagquist C. The psychometric properties of the Hopkins Symptom Checklist-10: a Rasch analysis based on adolescent data from Norway. Fam Pract. 2016;33:740–5. https://doi.org/10.1093/fampra/cmw091 .

Burks DB (2020) The SENA: an effective clinical evaluation system in the child and adolescent population. https://virtualpsychcentre.com/the-sena-an-effective-clinical-evaluation-system-in-the-child-and-adolescent-population/ . Accessed 25 May 2022

Buela-Casal G, Guillén-Riquelme A. Short form of the Spanish adaptation of the State-Trait Anxiety Inventory. Int J Clin Health Psychol. 2017;17:261–8. https://doi.org/10.1016/j.ijchp.2017.07.003 .

Walter R, Remschmidt H. Untersuchungen zur Reliabilität, Validität und Faktorenstruktur einer deutschsprachigen Version der Child Behavior Checklist. Z Klin Psychol Psychother. 1999;28:177–84. https://doi.org/10.1026//0084-5345.28.3.177 .

Mossman SA, Luft MJ, Schroeder HK, et al. The Generalized Anxiety Disorder 7-item (GAD-7) scale in adolescents with generalized anxiety disorder: signal detection and validation. Ann Clin Psychiatry. 2017;29:227-234A.

PubMed   PubMed Central   Google Scholar  

White D, Leach C, Sims R, et al. Validation of the Hospital Anxiety and Depression Scale for use with adolescents. Br J Psychiatry. 1999;175:452–4. https://doi.org/10.1192/bjp.175.5.452 .

Reardon T, Spence SH, Hesse J, et al. Identifying children with anxiety disorders using brief versions of the Spence Children’s Anxiety Scale for children, parents, and teachers. Psychol Assess. 2018;30:1342–55. https://doi.org/10.1037/pas0000570 .

Download references

Acknowledgements

We would like to acknowledge Dr Sabrina Schlesinger (Head of Research Group Systematic Reviews; German Diabetes Center) for her peer-review of the search strategy according to the Peer Review of Electronic Search Strategies (PRESS) Evidence-Based Checklist.

Open Access funding enabled and organized by Projekt DEAL. This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Author information

Authors and affiliations.

Federal Institute for Population Research (BiB), Wiesbaden, Germany

Helena Ludwig-Walz & Martin Bujard

Regional Innovative Centre of Health and Quality of Live Fulda (RIGL), Fulda University of Applied Sciences, Fulda, Germany

Indra Dannheim

Department of Nutritional, Food and Consumer Sciences, Fulda University of Applied Sciences, Fulda, Germany

Chair of Public Health and Health Services Research, IBE, Faculty of Medicine, LMU Munich, Munich, Germany

Lisa M. Pfadenhauer

Pettenkofer School of Public Health, Munich, Germany

Department for Child and Adolescent Psychiatry and Psychotherapy, University Medical Center, Competence Domain Mental Health Prevention, Ulm, Germany

Jörg M. Fegert

Institute of Medical Psychology, Medical Faculty, University Heidelberg, Heidelberg, Germany

Martin Bujard

You can also search for this author in PubMed   Google Scholar

Contributions

HLW and MB formulated the research question, with clinical feedback from JMF. All authors contributed to the study concept and design. HLW and ID screened titles, abstracts, full text and extracted data. HLW, LMP and ID assessed risk of bias. HLW and LMP conducted the GRADE assessment. HLW and MB prepared the first draft of the manuscript. HLW and ID accessed and verified all data. The corresponding author had final responsibility for deciding to submit for publication. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Helena Ludwig-Walz .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

Dr Lisa M. Pfadenhauer (last five years): Co-author of the Cochrane reviews ‘Measures implemented in the school setting to contain the COVID‐19 pandemic: a rapid scoping review’, ‘Measures implemented in the school setting to contain the COVID‐19 pandemic’ and ‘Unintended consequences of measures implemented in the school setting to contain the COVID‐19 pandemic: a scoping review’. Prof Dr Martin Bujard (last five years): Research funding from European Union and BMBF (German Ministry of Education and Research). Travel grants and honoraria from universities, federal and state parliaments, federal and state ministries, Evangelical-Lutheran Church, Federal Agency for Civic Education. Consultant for BMFSFJ (Federal Ministry of Family, Senior Citizens, Women and Youth). Every grant and every honorarium were declared to the law office of the Federal Institute for Population Research (BiB). Prof Dr Jörg M. Fegert (last five years): Research funding from European Union, BMG (Federal Ministry of Health), BMBF (Federal Ministry of Education and Research), BMFSFJ (Federal Ministry of Family, Senior Citizens, Women and Youth), DFG (German Research Foundation), G-BA Innovation Fund, State Ministries of Baden-Württemberg and Saarland, State Foundation Baden-Württemberg, Porticus Foundation, Evangelical-Lutheran Church in Württemberg. Travel grants, honoraria, sponsorship for conferences and medical educational purposes from APK, Adenauer- and Ebertstiftung, Deutschlandfunk, DFG, DJI, DKSB, Infectopharm, med update, UNICEF, professional associations, universities and federal and state ministries. Consultant for APK, federal and state ministries. No industry-sponsored lecture series, no shareholdings, no participation in pharmaceutical companies. Every grant and every honorarium were declared to the law office of the University Hospital Ulm. No other disclosures were reported.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1..

Additional Tables, Tables S1-S16 and additional Figures, Figures S1-S19.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Ludwig-Walz, H., Dannheim, I., Pfadenhauer, L.M. et al. Anxiety increased among children and adolescents during pandemic-related school closures in Europe: a systematic review and meta-analysis. Child Adolesc Psychiatry Ment Health 17 , 74 (2023). https://doi.org/10.1186/s13034-023-00612-z

Download citation

Received : 05 April 2023

Accepted : 10 May 2023

Published : 21 June 2023

DOI : https://doi.org/10.1186/s13034-023-00612-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Meta-analysis
• Communicable disease control
• Evidence-informed decision-making
• Pandemic preparedness
• Health policy

Child and Adolescent Psychiatry and Mental Health

ISSN: 1753-2000

• Submission enquiries: [email protected]

• Case Report
• Open access
• Published: 27 April 2024

Minimally invasive treatment of uterine necrosis with favorable outcomes: an uncommon case presentation and literature review

• Tengge Yu 1  

BMC Women's Health volume  24 , Article number:  267 ( 2024 ) Cite this article

21 Accesses

Metrics details

Uterine necrosis is a rare condition and is considered a life-threatening complication. However, cases of uterine necrosis were rarely reported, particularly those caused by infection. In terms of treatment, no minimally invasive treatment for uterine necrosis has been reported, and total hysterectomy is mostly considered as the treatment option.

The article specifically focuses on minimally invasive treatments and provides a summary of recent cases of uterine necrosis.

Case presentation

We report the case of a 28-year-old patient gravid 1, para 0 underwent a cesarean section after unsuccessful induction due to fetal death. She presented with recurrent fever and vaginal discharge. The blood inflammation markers were elevated, and a CT scan revealed irregular lumps with low signal intensity in the uterine cavity. The gynecological examination revealed the presence of gray and white soft tissue, approximately 5 cm in length, exuding from the cervix. The secretions were found to contain Fusobacterium necrophorum, Escherichia coli, and Proteus upon culturing. Given the patient’s sepsis and uterine necrosis caused by infection, laparoscopic exploration uncovered white pus and necrotic tissue openings in the anterior wall of the uterus. The necrotic tissue was removed during the operation, and the uterus was repaired. Postoperative pathological findings revealed complete degeneration and necrosis of fusiform cell-like tissue. Severe uterine necrosis caused by a multi-drug resistant bacterial infection was considered after the operation. She was treated with antibiotics for three weeks and was discharged after the infection was brought under control. The patient expressed satisfaction with the treatment plan, which preserved her uterus, maintained reproductive function, and minimized the extent of surgery.

Based on the literature review of uterine necrosis, we found that it presents a potential risk of death, emphasizing the importance of managing the progression of the condition. Most treatment options involve a total hysterectomy. A partial hysterectomy reduces the extent of the operation, preserves fertility function, and can also yield positive outcomes in the treatment of uterine necrosis, serving as a complement to the overall treatment of this condition.

Peer Review reports

Uterine necrosis is a rare complication. Several cases of uterine necrosis have been reported following embolization of the uterine arteries for postpartum hemorrhage or uterine fibroids, or as a result of severe endometritis [ 1 ]. Symptoms of uterine necrosis caused by infection typically include lower abdominal pain, fever, and foul-smelling vaginal discharge. When the infection affects the tissue surrounding the uterus, the uterus becomes enlarged and tender, and the edema of the inflamed tissue holds the uterus in place. Some complications may occur infrequently, including peritonitis, pelvic vein thrombosis, pulmonary embolism, pelvic abscess, sepsis, kidney damage, and even death. Diagnosis is usually based on clinical symptoms and physical examination. Inflammatory markers, imaging studies, and secretion cultures can also assist in the diagnosis. Hysteronecrosis is typically treated with a total hysterectomy. Most patients recover, and only a small number of patients do not survive. We reported a case of uterine necrosis caused by infection. We removed part of the uterus instead of performing a total hysterectomy. The patient recovered well. Few cases of uterine necrosis have been reported, and no one has reported minimally invasive treatment for it. Given the rarity of the case and the lack of minimally invasive treatment options for uterine necrosis, this report was written in conjunction with a literature review summarizing similar cases of uterine necrosis.

We report the case of a 28-year-old patient who was gravid 1, para 0, with no significant medical history. The patient is Asian, from the Han ethnic group, China’s largest ethnic group. When she was 32 weeks pregnant, intrauterine fetal demise was confirmed by ultrasound. A cesarean section was performed due to the difficulty of vaginal trial labor following a lateral perineal incision, which was necessary because the fetus’s shoulder was exposed. After the operation, the patient continued to experience a high fever, with a maximum temperature of 39.5 degrees Celsius, and the fat around the abdominal incision has become liquefied. The number of patient’s pulses was 140, respiratory rate was 22 times per minute, and blood pressure was 131/87mmhg. After receiving treatment with medications such as Tienam and Piperacillin, the patient’s body temperature and inflammation returned to normal, and she was discharged from Municipal integrated traditional Chinese and Western medicine hospital. The type of antibiotic Tienam is Carbapenem antibiotics, and the dose is 500 mg by injection three times a day. The type of antibiotic Piperacillin is semi-synthetic penicillin antibiotics, and the dose is 1.5 g by injection three times a day. The disease subside after 7 days treatment.

A week later, she was admitted to Municipal integrated traditional Chinese and Western medicine hospital for the second time due to fever and pain in her lower left abdomen. The patient’s heart rate was normal, respiratory rate was 20 times per minute, and blood pressure was 121/80mmhg. The blood inflammation index was elevated, indicated by a C-reactive protein level of 52.01 mg/L. Brain CT and lung CT scans revealed no significant abnormalities. She was discharged after two weeks of treatment with medications such as Tienam and Piperacillin with the same dose as last time. The disease subside after 5 days treatment.

Five days later, she was admitted to our hospital for the third time due to a recurring fever, accompanied by vaginal purulent discharge and odor. The patient’s heart rate was 110 times per minute, respiratory rate was 23 times per minute, and blood pressure was 132/85mmhg. There was no increase in β-HCG, white cell count was 12.3 × 10^9 /L in the differential blood count, hemoglobin was 104 g/L, and procalcitonin was 0.12 ng/ml. She felt feverish and lethargic, with mild nausea. The patient was treated orally with Moxifloxacin by 1 tablet once a day for 3 days. A vaginal color ultrasound revealed a hypoechoic area in front of the uterus, indicating encapsulated effusion. The ultrasound also revealed an abnormal uterine echo, uneven uterine enlargement with abundant blood supply, trace effusion of the cervical canal, and pelvic effusion. The enhanced CT scan revealed swelling and adhesion of the anterior wall of the uterus and the adjacent anterior abdominal wall, along with changes in the surrounding exudate. Additionally, a lumpy, uneven low signal shadow was observed in the uterine cavity, along with visible pelvic fluid (Fig.  1 ). The histopathological analysis of intrauterine effluents revealed degenerative smooth muscle tissue accompanied by pus. Anaerobic culture of cervical secretions suggested the presence of Fusobacterium necrophorum. Biopsy of cervical and vaginal lesions revealed complete necrosis of fusiform cell-like tissue, with increased infiltration of inflammatory cells, and no identifiable endometrial tissue. Due to the presence of pus in the uterus, morinidazole was administered, and uterine drainage was performed. However, the result was not favorable. A gynecological examination revealed the presence of necrotic tissue in the vagina, extending approximately 5 cm from the cervical opening. The tissue appeared white and emitted a foul odor. It was recommended to undergo a laparoscopic surgery.

Sagittal computed tomography images. The uterus shows heterogeneous spongiform enlargement with multiple air locules, measuring 4 × 2.6 cm and extending over 5 cm. We have observed a difficulty in distinguishing between the myometrium and the endometrial cavity

Given that the patient had strong fertility requirements, the surgical procedure included laparoscopic necrotic tissue excision, uterine reconstruction, and the uterine drainage tube placement. During the laparoscopy, adhesion of the uterus to the anterior wall of the abdomen was observed. An opening with white pus and necrotic tissue was visible in the anterior wall of the uterus. The necrotic tissue in the cavity was removed during the operation (Fig.  2 Fig.  3 ). Cefoxitin (1.5 g tid ivgtt) and ornidazole (500 mg bid oral) were administered postoperatively to prevent infection for 2 days. After the surgery, the patient developed a fever with a peak body temperature of 39.3 degrees Celsius, which prompted a switch to cefoperazone-sulbactam sodium (2 g bid ivgtt) and ornidazole antibiotics (500 mg bid oral) for 7 days. After the body temperature returned to normal, the antibiotics were downgraded, the uterine drainage tube was removed, and oral antibiotics were continued after discharge. The results of the vaginal secretion culture indicated the presence of Escherichia coli and Proteus bacteria. The pathological results revealed extensively denatured necrotic tissue with calcification and heightened inflammatory cell infiltration.

The images seen during the surgery. The necrotic tissue of the uterus, attached to the anterior wall of the abdomen, is clearly visible in gray and white colors. It is situated in the anterior wall of the uterus and is connected to the uterine cavity

The gross specimen appeared gray in color, with an extremely soft texture, measuring about 5 cm in length, and accompanied by a foul odor

Uterine necrosis is a rare and serious complication. Cases of uterine necrosis have been reported in the literature as a complication of cesarean section, embolization for postpartum hemorrhage, or for a leiomyomatous uterus, as well as in cases of severe endometritis [ 2 , 3 , 4 , 5 , 6 , 7 ] (Table  1 ). Several authors have described cases of uterine necrosis associated with intrapartum or postpartum complications that increase the risk of infection. The literature reports cases of uterine necrosis resulting from the placement of B-Lynch compression sutures, uterine artery embolization, or surgical ligation techniques used to treat postpartum hemorrhage. These procedures may increase the risk of infection as the tissue becomes devascularized. A. Fouad et al. described a case similar to ours in which a patient underwent a cesarean section due to fetal death, followed by postoperative sepsis and purulent uterine necrosis. Despite undergoing a hysterectomy, the disease continued to progress and ultimately resulted in death due to septic shock and multiple organ failure.

The case we report has identified pathogenic bacteria in the culture, which is significant for diagnosing infection-induced uterine necrosis. Fusobacterium necrophorum is a pleomorphic, Gram-negative, non-spore-forming obligate anaerobic coccobacillus. It is associated with localized abscesses, throat infections, and life-threatening systemic diseases. It is a common resident of the oral cavity and the vagina. Of the two subspecies of Fusobacterium necrophorum, biovar B is the primary pathogen for humans. Potential virulence factors include cell wall endotoxin lipopolysaccharide, hemagglutinin, and hemolysin. Most reported cases related to gynaecology occurred in the postpartum or post-abortion period, in addition to a few reports associated with the use of intrauterine devices, tubo-ovarian abscesses, and gynecological Lemierre’s syndrome [ 8 , 9 , 10 ]. Although infected with the same pathogen, the case reported by T. Widelock et al. developed more severe symptoms, including lung abscesses and kidney failure, as a result of hematoplasm infection [ 11 , 12 , 13 ].

A pelvic ultrasound is the initial diagnostic test that can reveal signs of uterine necrosis. The uterine cavity is typically expanded and exhibits multiple echogenic foci with accompanying dirty acoustic shadowing. Little or no vascularity is observed [ 14 ]. The diagnosis requires further exploration through a CT scan or MRI, as these are the preferred methods of investigation. The CT scan is highly useful for diagnosis as it demonstrates the presence of gas in the myometrium, the lack of enhancement of the myometrium after contrast injection associated with uterine enlargement, and the presence of free fluid in the peritoneal space [ 15 , 16 ].

Since uterine necrosis is described as a life-threatening complication, it is suggested to manage it with hysterectomy and broad-spectrum antibiotic therapy [ 17 , 18 ]. But sometimes it’s a case-by-case situation.

Avoid the chances of associated infections by systematic vaginal sampling in the third trimester, and promote good asepsis during surgery and antibiotic coverage in case of doubt about any undiagnosed prepartum infection, which may potentiate hypoxia and the risk of necrosis. Uterine necrosis may be secondary to all these intertwined factors and could be potentiated by an environment of hypoxia, hypoperfusion, hypovolemia secondary to hemorrhage, massive transfusions with disadvantages in a patient who is immunocompromised by pregnancy, and possibly, by other vitamin and iron deficiencies.

Given the limited number of reported cases of uterine necrosis in the past, there is no standardized treatment protocol. However, due to the potential fatality of uterine necrosis, most treatment options involve total hysterectomy. In our case, only the necrotic tissue of the uterus was removed in young women who had not given birth, and the prognosis for the patient is good. This study also has limitations, including the short follow-up time and the small number of cases collected. It needs to be complemented by subsequent case reports related to uterine necrosis.

Data availability

No datasets were generated or analysed during the current study.

Dharia S, Shah S, Kissinger M, Sanders A, Singh G. Group A Streptococcal endometritis and toxic shock causing septic pelvic thrombophlebitis and Septic Pulmonary Emboli. BMJ Case Rep. 2023;16(11):e255455. https://doi.org/10.1136/bcr-2023-255455 . PMID: 37996149; PMCID: PMC10668199.

Article   PubMed   Google Scholar  

Fouad A, Bouab M, Youssouf N, Lamrissi A, Fichtali K, Bouhya S. Uterine necrosis simulating a textiloma: a case report. Int J Surg Case Rep. 2022;91:106754. https://doi.org/10.1016/j.ijscr.2021.106754 . Epub 2022 Jan 5. PMID: 35051885; PMCID: PMC8777381.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Luo L, Wan J, Chen X, Zhang H, Zhang M, Chen Q. Uterine necrosis, infection, and subinvolution: complications observed after combined application of modified B-Lynch suture and vascular ligation. J Int Med Res. 2021;49(5):3000605211010730. https://doi.org/10.1177/03000605211010730 . PMID: 33947256; PMCID: PMC8113939.

Tymon-Rosario J, Chuang M. Multidrug-resistant Escherichia coli resulting in Postpartum Necrotizing endomyometritis. Case Rep Obstet Gynecol. 2019;2019:6715974. https://doi.org/10.1155/2019/6715974 . PMID: 31139481; PMCID: PMC6500617.

Article   PubMed   PubMed Central   Google Scholar  

Tanaka K, Muraoka Y, Toda Y, Kiyomoto C, Okubo Y, Nagashima T, Furukawa S, Fujiwara M, Mochizuki M, Kobayashi Y, Iwashita M. Finegoldia magna myometritis with uterine necrosis after uterine artery embolisation. J Obstet Gynaecol. 2017;37(5):688–9. Epub 2017 Mar 20. PMID: 28319430.

Kashan D, Muthu N, Chaucer B, Davalos F, Bernstein M, Chendrasekhar A. Uterine perforation with Intra-abdominal Clostridium perfringens Gas Gangrene: a rare and fatal infection. J Gynecol Surg. 2016;32(3):182–4. PMID: 27274183; PMCID: PMC4876544.

Widelock T, Elkattah R, Gibbs S, Mashak Z, Mohling S, DePasquale S. Uterine Necrosis Associated with Fusobacterium necrophorum infection. Case Rep Obstet Gynecol. 2015;2015:934913. https://doi.org/10.1155/2015/934913 . Epub 2015 Apr 27. PMID: 26000185; PMCID: PMC4426659.

Koshy KM, Malik W, Roberts SC. Myometritis with pelvic septic vein thrombophlebitis secondary to Fusobacterium necrophorum sepsis. BMJ Case Rep. 2022;15(10):e250097. https://doi.org/10.1136/bcr-2022-250097 . PMID: 36229077; PMCID: PMC9562719.

Tamura S, Jwa SC, Tarumoto N, Ishihara O. Septic shock caused by Fusobacterium Necrophorum after sexual intercourse during recovery from infectious mononucleosis in an adolescent: a Case Report. J Pediatr Adolesc Gynecol. 2020;33(5):566–9. https://doi.org/10.1016/j.jpag.2020.06.018 . Epub 2020 Jun 26. PMID: 32599171.

Reymond B, Huette P, Roger PA, Tredez E, Gagneur O, Sanguin S, Guihéneuf R, Dupont H, Mahjoub Y, Abou Arab O. Fatal Fusobacterium necrophorum infection with gynecological Lemierre’s syndrome. Med Mal Infect. 2019;49(1):72–4. Epub 2018 Oct 9. PMID: 30309662.

Brazier JS. Human infections with Fusobacterium necrophorum. Anaerobe. 2006;12(4):165–72. https://doi.org/10.1016/j.anaerobe.2005.11.003 . Epub 2005 Dec 22. PMID: 16962962.

Hagelskjaer Kristensen L, Prag J. Lemierre’s syndrome and other disseminated Fusobacterium necrophorum infections in Denmark: a prospective epidemiological and clinical survey. Eur J Clin Microbiol Infect Dis. 2008;27(9):779–89. https://doi.org/10.1007/s10096-008-0496-4 . Epub 2008 Mar 11. PMID: 18330604; PMCID: PMC7102232.

Treszezamsky AD, Molina Boero MF, Mehta I. Cervical conization complicated by sepsis with lung and liver abscesses. J Low Genit Tract Dis. 2010;14(2):130-3. https://doi.org/10.1097/LGT.0b013e3181c7104e . PMID: 20354422.

Ruiz Sánchez E, Peinado Rodenas J, Gil Martínez-Acacio L, Arones Collantes M, Villar García M, García de la Torre JP, Amezcua Recover AN. Uterine necrosis. A rare complication of embolisation due to post-partum haemorrhage. J Gynecol Obstet Hum Reprod. 2021;50(2):101773. Epub 2020 Apr 20. PMID: 32325270.

Benkirane S, Saadi H, Serji B, Mimouni A. Uterine necrosis following a combination of uterine compression sutures and vascular ligation during a postpartum hemorrhage: a case report. Int J Surg Case Rep. 2017;38:5–7. https://doi.org/10.1016/j.ijscr.2017.07.005 . Epub 2017 Jul 10. PMID: 28728102; PMCID: PMC5516088.

Pirard C, Squifflet J, Gilles A, Donnez J. Uterine necrosis and sepsis after vascular embolization and surgical ligation in a patient with postpartum hemorrhage. Fertil Steril. 2002;78(2):412-3. https://doi.org/10.1016/s0015-0282(02)03229-6 . PMID: 12137882.

Gupta S, Buxi TBS, Rastogi D, Ghuman S, Mittal S, Sud S. Imaging of uterine necrosis: a rare complication of uterine artery embolization for post partum hemorrhage. Curr Med Res Pract. 2017;7(1):20–3.

Article   Google Scholar  

Melenhorst M, Hehenkamp W, de Heer K, Herger F. CT features in uterine necrosis of unknown cause: a case report. Clin Imaging. 2014;38(4):543–6.

Download references

Acknowledgements

Not applicable.

All authors declare that no fundings were received concerning the published case or the decision to publish it.

Author information

Authors and affiliations.

Department of Gynecology and Obstetrics, West China Xiamen Hospital of Sichuan University, Xiamen, 361000, China

You can also search for this author in PubMed   Google Scholar

Contributions

Tengge Yu have drafted the work and substantively revised it.

Corresponding author

Correspondence to Tengge Yu .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Ethics approval and consent to participate

Patient consent for publication.

Consent for publication

Written Informed consent for publication obtained from the participant.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Yu, T. Minimally invasive treatment of uterine necrosis with favorable outcomes: an uncommon case presentation and literature review. BMC Women's Health 24 , 267 (2024). https://doi.org/10.1186/s12905-024-03089-w

Download citation

Received : 21 December 2023

Accepted : 12 April 2024

Published : 27 April 2024

DOI : https://doi.org/10.1186/s12905-024-03089-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Minimally invasive surgical procedures
• Fertility preservation
• Bacterial infections

BMC Women's Health

ISSN: 1472-6874

• Submission enquiries: [email protected]
• General enquiries: [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• v.2(6); 2022 Nov

Effective Peer Review: Who, Where, or What?

Peer review is widely viewed as one of the most critical elements in assuring the integrity of scientific literature ( Baldwin, 2018 ; Smith, 2006 ). Despite the widespread acceptance and utilization of peer review, many difficulties with the process have been identified ( Hames, 2014 ; Horrobin, 2001 ; Smith, 2006 ). One of the primary goals of the peer review process is to identify flaws in the work and, by so doing, help editors choose which manuscripts to publish. It is surprising that one of the persistent problems in peer review is assessing the quality of the reviews. Both authors and journal editors expect peer review to detect errors in experimental design and methodology and to ensure that the interpretation of the findings is presented in an objective and thoughtful manner. In traditional peer review, two or more reviewers are asked to evaluate a manuscript on the basis of the expectation that if the two reviewers agree on the quality of the submission, the likelihood of a high-quality review is increased. Unfortunately, studies have not consistently confirmed a high degree of agreement among reviewers. Rothwell and Martynn (2000) evaluated the reproducibility of peer review in neuroscience journals and meeting abstracts and found that agreement was approximately what would be expected by chance. Similarly, Scharschmidt et al. (1994) found similar results in the evaluation of 1,000 manuscripts submitted to the Journal of Clinical Investigation, where clustering of grades in the middle resulted in an agreement being “…only marginally…” better than chance. These observations suggest that we cannot rely on the agreement of reviewers to be an indication of the quality of the reviews. Another potential way to evaluate the quality of reviews would be to assess the ability of reviewers to detect errors in submissions. It is generally accepted that detection of intentional fraud is beyond the scope of typical peer review, but we do expect reviewers to detect major and minor errors as a primary function of the traditional peer review system ( Hwang, 2006 ; Weissman, 2006 ). Schroter et al. (2008) evaluated the ability of reviewers to detect major and minor errors by introducing errors into three previously published papers describing randomized controlled clinical trials. Reviewers detected approximately three of the nine errors introduced in each manuscript. Unfortunately, reviewers who had undergone training in how to conduct a high-quality peer review were not significantly better than untrained reviewers. Similar results have been reported by Godlee et al. (1998) and Baxt et al. (1998) . Baxt et al. (1998) did report that reviewers who rejected or suggested revision of a manuscript identified more errors than those who accepted the manuscript (decision: 17.3% of major errors detected [accept], 29.6% of major errors detected [revise], and 39.1% of major errors detected [reject]). It is almost certainly true that the extent of the failure to recognize errors in submitted manuscripts may differ among scientific disciplines and journals. It also however seems likely that these observations do have some applicability to journals such as JID Innovations . It is critical that both authors and editors are cognizant of these limitations of peer review in their assessment of reviews. These findings compel journals to continue to work to develop new strategies to train and evaluate reviewers. The findings also suggest that factors beyond the failure to detect objective mistakes in a manuscript may be playing a role in the discrepancy in reviewers’ evaluations. One area of ongoing concern in the peer review process is the role of reviewer bias in assessing the scientific work of colleagues ( Kuehn, 2017 ; Lee et al, 2013 ; Tvina et al, 2019 ).

Bias in the peer review process can take many forms, including collaborator/competitor bias, affiliation bias based on an investigator’s institution or department, geographical bias based on the region or country of origin, racial bias, and gender or sex bias ( Kuehn, 2017 ; Lee et al, 2013 ; Tvina et al, 2019 ). All of these forms of bias present the risk that a decision of the reviewer will not be based solely on the quality or merit of the work but rather be influenced by a bias of the reviewer. We and other journals routinely seek to avoid selecting individuals to review work from their own institutions and ask all reviewers to declare any potential personal conflicts of interest. All these methods require either the editor or the reviewer to identify a bias and fail to address the issue of implicit or unconscious reviewer bias. The dominant method currently utilized for peer review is the so-called single-blind review, in which the identity and affiliations of the authors are known to the reviewers, whereas the identity of reviewers remains unknown to the authors. This has led to concern that knowledge of the identity of the authors and their institutions may be the source of significant reviewer bias, especially implicit bias, in the evaluation of manuscripts. Double anonymized peer review (DAPR), also known as double-blind peer review, has been suggested as a way to address this issue ( Bazi, 2020 ; Lee et al, 2013 ). Studies have compared single-blind with double-blind reviewing and reported that there is no significant difference in the quality of the reviews ( Alam et al, 2011 ; Godlee et al, 1998 ; Justice et al, 1998 ; van Rooyen et al, 1998 ). Although these studies looked at measures such as the number of errors detected, acceptance rate, and distribution of initial reviewer scores, they were not designed to address specific sources of bias such as authors’ gender, institution, or geographic location. Other studies have been undertaken to directly address the issue of bias in the peer review process. Ross et al (2006) compared the acceptance of abstracts submitted to the American Heart Association’s annual scientific meeting during a period when the reviewers knew the identity and origin of the authors (i.e., single-blind review) with when this information was not known by the reviewers (i.e., double-anonymized peer review). They found a significant increase in acceptance of non‒United States abstracts and abstracts from non-English speaking countries when the reviewers were unaware of the country of origin of the abstracts ( Ross et al, 2006 ). They also found a significant decrease in the acceptance of abstracts from prestigious institutions when the reviewers were unaware of the institutions where the work was done. In a similar study, Tomkins et al. (2017) found that papers submitted to a prestigious computer science meeting were more likely to be accepted if they were from famous authors, top universities, and top companies. Okike et al. (2016) documented similar results for manuscripts submitted to the orthopedic literature. They submitted a fabricated manuscript that was presented as being written by two prominent orthopedic surgeons (past Presidents of the American Academy of Orthopedic Surgeons) from prestigious institutions. When reviewed in the traditional single-blind fashion, which included the identity of the authors, the manuscript was accepted by 87% of the reviewers. By contrast, when the identity of the authors was unknown, the manuscript was accepted by 68% of the reviewers ( P  = 0.02) ( Blank, 1991 ). A study conducted at The American Economic Review found that authors at near-top-ranked universities experienced lower acceptance rates when authorship was anonymized ( Blank, 1991 ). Of interest, they also found that for women, there was no difference in the acceptance rate between the double-anonymized and single-blinded reviews; however, for men, the acceptance rate was lower with double-anonymized reviews.

These studies provide strong evidence that knowledge of who and where the study was performed can impact the acceptance of abstracts and manuscripts. This conflicts with the goal of the review process to base our judgments on the quality of what the results demonstrate. It is difficult to estimate how much this may affect the fate of a manuscript at JID Innovations . We do not have evidence that our review process has been impacted by bias as is reported in the studies discussed. However, neither can we state with certainty that such bias is not a factor in the reviews we receive. One of the goals of JID Innovations is to be a truly open-access journal available to all investigators in skin science from around the world. We have sought to be an outlet for studies that challenge existing paradigms or that may report negative results. We want to be seen as providing fair and objective reviews for all authors, regardless of where they work or who they are. If we are to achieve this goal, it is imperative that the who and where of a specific manuscript do not negatively impact the evaluation of the what. We want young investigators, investigators at less prestigious institutions or from less well-known laboratories, and investigators from any country around the world to be confident that their work will be judged by what they report and not by the who and the where.

To be true to this mission, JID Innovations will be initiating DAPR starting in October 2022. This is not being done because we are aware of any issues of bias with our current process of peer review but because we realize that the absence of proof is not proof of absence. As a part of this process, authors will be asked to remove identifying material from manuscripts at the time of submission in preparation for the review process ( https://www.jidinnovations.org/content/authorinfo ). As a result, primary reviewers will see only the what of the manuscript. We realize that this process involves extra work for both the authors and our staff, but we feel the benefits will outweigh this small cost. Indeed, in other journals that have taken this step, surveys have shown that both authors and reviewers ultimately prefer double-anonymized reviews ( Bennett et al, 2018 ; Moylan et al, 2014 ). We realize that achieving 100% anonymization of a manuscript is nearly impossible. Studies have shown that the rate of successful anonymizing, where the reviewers cannot discern the authorship of a manuscript, ranged from 47 to 73%. It is however interesting that even with this rate of success in the anonymizing process, a meta-analysis of trials of double- versus that of single-blind peer review has suggested an impact, with lower acceptance rates with double-anonymized peer review ( Ucci et al, 2022 ). More work clearly needs to be done to assess the value of the DAPR process, and we will be monitoring our results carefully.

The institution of DAPR in JID Innovations will assure our authors that the what of their manuscript is our focus. It does not matter who you are or where you are from. It will also emphasize to our reviewers that our focus is on the what. We will be carefully monitoring the results of this new policy and plan to report back on our experience. We also welcome your feedback on your experience as a reviewer and author for JID Innovations ; send your comments to us at [email protected] .

Finally, this decision should be seen not as the end of our efforts to improve the peer review process but merely as a first step. We will continue to work to improve all aspects of the peer review process for JID Innovations . We firmly believe that the use of double-blind -anonymized peer review will bring us closer to ensuring to our authors and readers that the work that is published by JID Innovations has been selected on the basis of what the paper reports and not on who performed the studies or where they were located.

Conflict of Interest

The author states no conflicts of interest.

• Alam M., Kim N.A., Havey J., Rademaker A., Ratner D., Tregre B., et al. Blinded vs. unblinded peer review of manuscripts submitted to a dermatology journal: a randomized multi-rater study. Br J Dermatol. 2011; 165 563‒7. [ PubMed ] [ Google Scholar ]
• Baldwin M. Scientific autonomy, public accountability, and the rise of “peer review” in the Cold War United States. Isis. 2018; 109 :538–558. [ Google Scholar ]
• Baxt W.G., Waeckerle J.F., Berlin J.A., Callaham M.L. Who reviews the reviewers? Feasibility of using a fictitious manuscript to evaluate peer reviewer performance. Ann Emerg Med. 1998; 32 310‒7. [ PubMed ] [ Google Scholar ]
• Bazi T. Peer review: single-blind, double-blind, or all the way-blind? Int Urogynecol J. 2020; 31 :481–483. [ PubMed ] [ Google Scholar ]
• Bennett K.E., Jagsi R., Zietman A. Radiation oncology authors and reviewers prefer double-blind peer review. Proc Natl Acad Sci USA. 2018; 115 :E1940. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Blank R.M. The effects of double-blind versus single-blind reviewing: experimental evidence from the American Economic Review. Am Econ Rev. 1991; 81 :1041–1067. [ Google Scholar ]
• Godlee F., Gale C.R., Martyn C.N. Effect on the quality of peer review of blinding reviewers and asking them to sign their reports: a randomized controlled trial. JAMA. 1998 280237‒40. [ PubMed ] [ Google Scholar ]
• Hames I. Peer review at the beginning of the 21st century. Sci Ed. 2014; 1 :4–8. [ Google Scholar ]
• Horrobin D.F. Something rotten at the core of science? Trends Pharmacol Sci. 2001; 22 51‒2. [ PubMed ] [ Google Scholar ]
• Hwang W.S. Can peer review police fraud? Nat Neurosci. 2006; 9 :149. [ PubMed ] [ Google Scholar ]
• Justice A.C., Cho M.K., Winker M.A., Berlin J.A., Rennie D. Does masking author identity improve peer review quality? A randomized controlled trial. PEER Investigators [published correction appears in JAMA 1998;280:968. JAMA. 1998; 280 240‒2. [ PubMed ] [ Google Scholar ]
• Kuehn B.M. Rooting out bias. ELife. 2017; 6 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Lee C.J., Sugimoto C.R., Zhang G., Cronin B. Bias in peer review. JASIST. 2013; 64 :2–17. [ Google Scholar ]
• Moylan E.C., Harold S., O'Neill C., Kowalczuk M.K. Open, single-blind, double-blind: which peer review process do you prefer? BMC Pharmacol Toxicol. 2014; 15 :55. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Okike K., Hug K.T., Kocher M.S., Leopold S.S. Single-blind vs double-blind peer review in the setting of author prestige. JAMA. 2016; 316 1315‒6. [ PubMed ] [ Google Scholar ]
• Ross J.S., Gross C.P., Desai M.M., Hong Y., Grant A.O., Daniels S.R., et al. Effect of blinded peer review on abstract acceptance. JAMA. 2006; 295 1675‒80. [ PubMed ] [ Google Scholar ]
• Rothwell P.M., Martyn C.N. Reproducibility of peer review in clinical neuroscience. Is agreement between reviewers any greater than would be expected by chance alone? Brain. 2000; 123 1964‒9. [ PubMed ] [ Google Scholar ]
• Scharschmidt B.F., DeAmicis A., Bacchetti P., Held M.J. Chance, concurrence, and clustering. Analysis of reviewers' recommendations on 1,000 submissions to the Journal of Clinical Investigation. J Clin Invest. 1994; 93 1877‒80. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Schroter S., Black N., Evans S., Godlee F., Osorio L., Smith R. What errors do peer reviewers detect, and does training improve their ability to detect them? J R Soc Med. 2008; 101 507‒14. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Smith R. Peer review: a flawed process at the heart of science and journals. J R Soc Med. 2006; 99 178‒82. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Tomkins A., Zhang M., Heavlin W.D. Reviewer bias in single- versus double-blind peer review. Proc Natl Acad Sci USA. 2017; 114 12708‒13. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Tvina A., Spellecy R., Palatnik A. Bias in the peer review process: can we do better? Obstet Gynecol. 2019; 133 1081‒3. [ PubMed ] [ Google Scholar ]
• Ucci M.A., D'Antonio F., Berghella V. Double- vs single-blind peer review effect on acceptance rates: a systematic review and meta-analysis of randomized trials. Am J Obstet Gynecol MFM. 2022; 4 [ PubMed ] [ Google Scholar ]
• van Rooyen S., Godlee F., Evans S., Smith R., Black N. Effect of blinding and unmasking on the quality of peer review: a randomized trial. JAMA. 1998; 280 234‒7. [ PubMed ] [ Google Scholar ]
• Weissmann G. Science fraud: from patchwork mouse to patchwork data. FASEB J. 2006; 20 587‒90. [ PubMed ] [ Google Scholar ]