qualitative research article in education

Qualitative research in education : Journals

• Journal of ethnographic & qualitative research "Journal of Ethnographic & Qualitative Research (JEQR) is a quarterly, peer-reviewed periodical, publishing scholarly articles that address topics relating directly to empirical qualitative research and conceptual articles addressing topics related to qualitative."
• Qualitative research : QR "Qualitative Research (QRJ) is a bimonthly peer reviewed journal that publishes original research and review articles on the methodological diversity and multi-disciplinary focus of qualitative research."
• << Previous: Recent print books
• Next: Databases >>
• Background information
• Recent e-books
• Recent print books
• Connect to Stanford e-resources
• Last Updated: Jan 23, 2024 12:45 PM
• URL: https://guides.library.stanford.edu/qualitative_research_in_ed

• Tools and Resources
• Customer Services
• Original Language Spotlight
• Alternative and Non-formal Education 
• Cognition, Emotion, and Learning
• Curriculum and Pedagogy
• Education and Society
• Education, Change, and Development
• Education, Cultures, and Ethnicities
• Education, Gender, and Sexualities
• Education, Health, and Social Services
• Educational Administration and Leadership
• Educational History
• Educational Politics and Policy
• Educational Purposes and Ideals
• Educational Systems
• Educational Theories and Philosophies
• Globalization, Economics, and Education
• Languages and Literacies
• Professional Learning and Development
• Research and Assessment Methods
• Technology and Education
• Share This Facebook LinkedIn Twitter

Article contents

Digital culture and qualitative methodologies in education.

• Eliane Schlemmer Eliane Schlemmer Universidade do Vale do Rio dos Sinos - UNISOS
• https://doi.org/10.1093/acrefore/9780190264093.013.508
• Published online: 30 September 2019

From a digital culture perspective, this article has as main objective to assess two contemporary qualitative research methods in the field of education with distinct theoretical orientations: the cartographic method as a way of tracing trajectories in research-intervention with a theoretical basis in the biology of knowledge, enactive cognition and inventive cognition; and the cartographic method as a means of identifying and mapping the controversies linked to the different associations between human and non-human actors with a theoretical basis in actor-network theory (ANT). With their own specificities, both methods have been fruitful in the development of qualitative research in the field of education, in the context of digital culture, and more recently, in the hybrid culture of atopic habitation, mainly because they also relate to equally consistent theories and aspects of human cognition, making it possible to detect traces and clues in the fluid associations between actors enhanced by different digital technologies (DT), including data mining and learning analytics. From the Brazilian perspective on the topic, this article approaches the experience of the cartographic method of research intervention as well as the cartography of controversies as tools for developing qualitative research in education. These different forms of the cartographic method have inspired the construction of didactic-pedagogical experiences based on theoretical approaches linked to cognition, producing inventive methodologies and interventionist pedagogical practices. These methodologies and practices, which will be discussed at length in this article, have been developed and validated by the Research Group in Digital Education at Unisinos University at different levels and in varied educational settings.

• qualitative research
• cartography
• digital culture
• multimodality
• pervasiveness

Introduction

A version of this article in its original language.

A social group’s socialization is made visible by a distinctive way of acting, which develops rituals representing emotions, common values, and norms for coexistence, all of which contribute to constructing a culture. This makes it possible to refer, for example, to a pre-digital, digital, or gamer culture, and, more recently, a hybrid culture in atopic habitation.

Research itself is also embedded in a culture, which is evident in the understanding of what science is and how to do science in different areas of knowledge. This way of doing science, of researching, has faced new challenges and implications arising mainly from exponential digital technological growth.

The production of knowledge, as well as its almost instantaneous dissemination, produces broad access to ever greater amounts of information in a very short time, providing direct interaction with the researcher or research group responsible for a given discovery and/or innovation, as well as the constitution of research networks. This movement, so characteristic of contemporary scientific inquiry, instigates dialogue among fields, allowing distinct areas of knowledge to overlap and establish new research contexts. Thus we have seen the emergence of biomedicine, biotechnology, food engineering, and informatics in digital education/education, among many others, posing new methodological research challenges.

Lopes and Schlemmer ( 2017 ), point out that research in education in the context of digital culture has sought theoretical and methodological references that can support the complexity of knowledge production in this context, provoking dialogue and the problematization of the very theoretical and epistemological field that sustains it, in addition to the simple problematization related to the use of specific digital technologies (DT) and their effects on education. In this sense, theorists such as Pierre Lévy, Manuel Castells, Bruno Latour, Michel Maffesoli, Massimo Di Felice, Lúcia Santaella, and André Lemos, among others, problematize the sociocultural dimension of digital technological development, allowing us to broaden the focus of research into the contexts of emerging digital culture and, according to Schlemmer ( 2018 ), hybrid culture in atopic habitation, as well as to understand how this phenomenon relates to the field of education. It is therefore a matter of researching the broader meanings that these cultures produce in the field of teaching and learning, that is, the broader field of education.

With this context in mind, the following is a presentation of the research we have conducted as part of the Research Group in Digital Education, which highlights some of the concerns, strategies, and theoretical and methodological approaches that we have adopted in our research in the field of education in digital culture, and more recently in the hybrid culture in atopic habitation, which has enabled us to engage in broader and deeper dialogue. The objective is to consider the cartographic method of intervention research so as to highlight the research process, showing how we gradually adopted digital culture research methods before arriving at the cartographic method of intervention research and the cartography of controversies, which are discussed at length in this article.

From Digital Culture to Hybrid Culture in Atopic Habitation: Challenges for Qualitative Research in Education

In the field of research on education and digital culture in Brazil, Schlemmer, Lopes, and Molina ( 2012 ) outline some epistemological and methodological challenges on experiments in providing guidance by master’s and doctoral students in education. These challenges involve the construction of research objects and problems in the field of education and digital culture in DT contexts, including virtual learning environments, digital social media, and 3D digital virtual worlds. The authors propose rethinking the term virtual as a demarcation of an epistemological and methodological frontier in the ethnographic research of education and digital culture. They argue that the term digital would be more appropriate in distinguishing this border. They also suggest that the term netnography would be the most appropriate in a theoretical-methodological ethnographic research design in education and digital culture.

With this perspective, Lopes, Schlemmer, and Molina ( 2014b ) present some approaches to developing research and new procedures involving the use of applications (apps) and mobile devices (netbooks, tablets, and smartphones) in ethnographic research based on the project “Escola aumentada: Cartografia digital e mobilidade para a aprendizagem e a cidadania.” For the authors, the development of mobile technologies, geolocation (GPS and locative media), and distributed online databases (cloud computing) created new possibilities for the production of records and interactions in ethnographic research. While the diversity of digital records (text, photo, audio, and video) coupled with indexing and tagging mechanisms (tags, hashtags, geotags, etc.) opens up new possibilities for research, it also requires participants to understand the relevant syntax and new technological procedures for the production, recording, and sharing of information. Some programs and applications for desktop and mobile devices such as NVivo and Evernote, among others, can facilitate this process, contributing to the planning and organization of such production. The authors also refer to participatory research methodologies capable of involving researchers and subjects in the production and analysis of data. The current technological context, they note, can consolidate a new scenario in the field of research in which scholars and subjects act as co-producers of knowledge. In this sense, cartography mediated by digital devices emerges as a rich methodological possibility, involving social actors in the context of local problems. This engagement is a necessary condition for the production of meaning through the shared and mediated use of DT in the process of knowledge construction. This scenario can reconfigure both the research itself and the contract between researchers and subjects, since action and participation become instances of authorship from which all sides speak and produce.

Expanding their experimentation along with the discussion over the cartographic method, Lopes, Schlemmer, and Molina ( 2014a ) carried out an early attempt to map actions linked to a digital inclusion program—Província de São Pedro (PSP)—to distribute netbooks and tablets to teachers and students, initially prioritizing schools in cities participating in the Territories of Peace Program (Programa Territórios de Paz, PTP) to address the following research problem: What is the reasoning for linking an educational program of digital inclusion to a public safety program? The objective of the research was to understand the ties between such programs. Thus, the authors present digital cartography as a methodological proposition for online research and indicate the digital map of culture in Rio Grande do Sul (RS) and digital cartography as technological possibilities to promote greater connectivity between such programs.

Schlemmer and Lopes ( 2016 ) and Schlemmer ( 2016a ) also analyze the potential of the method to inspire new practices in line with the need to understand the phenomenon of learning in all its complexity—social, political, cognitive, affective, and technological—precisely because of its interventionist nature. In this context, the authors present a theoretical, methodological, and technological experiment developed in higher education and inspired by the cartographic method as a way of monitoring and evaluating learning in gamified processes and games from an interventionist perspective developed in a hybrid, multimodal, pervasive, and ubiquitous context.

Lopes and Schlemmer ( 2017 ) problematize the ethical, epistemological, and methodological aspects related to the field of education in digital culture, reflecting on how ethics can dialogue with the choices scholars make when conducting research. The authors present the paths adopted in two surveys conducted between 2010 and 2015 with a state public school in the metropolitan region of Porto Alegre participating in government programs for digital inclusion. Founded on the intervention research cartographic method, they present some of the results of discussions with teachers and students, based on the experience of producing and publishing information on the Internet. They problematize the ethical dilemma of research intervention based on the idea of technological appropriation, as a process that is established from the changes of practices that take place in contexts of analogical school culture and digital culture. They discuss and propose, based on the results of the research, overcoming the ethical dilemma of children and young students participating in surveys involving the publication of online content and the fears regarding media exposure—namely, the production and access to inappropriate content—and inattention in the classroom.

This research demonstrates that the cartographic method has been relevant, especially when the research tries to understand phenomena related to learning in the digital culture or, more recently, in the hybrid culture in an atopic habitation. Associated with this perspective, in the digital technological context are systems of data mining and learning analytics that make it possible to more effectively follow the tracks left by the subjects in the different spaces in which they interact.

Hybrid, in this context, refers to the mix between different elements resulting in a new element composed of the previous ones. For Latour ( 1994 ), the hybrid consists of multiple matrices, mixtures of nature and culture, which is therefore contrary to the separation between culture/nature, human/non human, among other things.

By atopic habitation, Di Felice ( 2009 ) refers to a relationship, a form of communication, characterized by the network interactions between different human and non human collectives, digital and territorial technologies. Atopic habitation “is thus the transient and fluid hybridization of bodies, technologies and landscapes, and as the advent of a new typology of ecosystem, neither organic, nor inorganic, nor static, nor delimitable, but informative and immaterial” (p. 291).

Thus, according to Schlemmer ( 2013 , 2014 , 2015 , 2016a , 2016b , 2016c , 2017 ), the hybrid is understood to be a mixture of space (geographic and digital), presence (physical and digital), technologies (analogue and digital), and culture (pre-digital and digital). It is in this context that the term multimodal is used, which includes the different imbricated educational modalities: the presential-physical modality and online modality and, while online, being able to combine elements of electronic learning, mobile learning, pervasive learning, ubiquitous learning, immersive learning, gamification learning, and game-based learning.

When scholars refer to hybridism in atopic habitation, in multimodality, pervasiveness, and ubiquity, they mean actions and interactions between human actors (HA) and non-human actors (NHA), in geographic and digital spaces, in interactions of different cultures (digital and pre-digital), constituting inseparable networks linking interconnected natures, techniques, and cultures. This suggests that a new understanding of culture and society may be emerging, one that embraces coexistence, co-engendering, mutual respect, solidarity, and the recognition of the other as a legitimate interlocutor. Thus, it is worth seeking to understand what these changes might mean in the area of education and, by extension, for research in education.

It is in this context that two contemporary methodological approaches are presented and discussed within the scope of qualitative research in education, linked to specific theoretical orientations.

The Cartographic Method of Research Intervention in Relation to Cognition Sciences

The cartographic method of intervention research is based on the cartographic method proposed by Deleuze and Guattari ( 1995 ). This method has been developed in Brazil by Kastrup ( 2007 , 2008 ), Passos, Kastrup, and Escóssia ( 2009 ), and Passos, Kastrup, and Tedesco ( 2014 ) as a means of interventionist research.

According to Passos et al. ( 2009 ), this approach originated from concerns about research methodology, which requires more open and, at the same time, inventive procedures. Thus, the theme of cartography emerged as a methodological issue in the face of impasses in cognition research, developed by the research group Cognição e Subjetividades. 1 The method began taking shape when the members of the research group questioned the assumption that knowledge means representing or recognizing reality by configuring the importance of the binomial cognition/creation and calling for a more detailed investigation process into the temporal dimension of knowledge production processes. Thus, the authors defined the concept of cognition as creative, autopoietic (Humberto Maturana and Francisco Varela, 2001 ), or enactive (Francisco Varela, 1988 ).

Maturana and Rezepka ( 2000 ) suggest that the way people attribute meaning and learn has a distinctly human quality, since people are autonomous and autopoietic, in congruence with the environment in which they are inserted. This congruence can cause disturbances in the structure of human beings, promoting learning processes insofar as the structure self-produces to compensate for the disturbance. Thus, for Maturana ( 1993a , 1993b ), learning is the act of transforming within a particular environment of recurrent interactions, and happens as the behavior of an organism varies during its ontogenesis in a manner congruent with the variations of the environment (Maturana, 1998 ). Therefore, when I refer to a medium or a hybrid, it means that the congruence of the subject with this environment causes disruptions in the structure of this subject, which allows him or her to attribute meanings that originate from the action and interaction in that space, thus promoting learning processes as the structure reproduces itself to compensate for the disturbance, and in so doing, compounds its ontogeny.

The process of cognition consists of creating a range of behaviors through conduct within a field of interactions. Knowledge, from this perspective, is not simply representation, but implies a permanent interpretation of action. For Varela ( 1990 ), interpretation and knowledge are emergent (in the sense of emerging) results of action or acting in the world. Thus, greater capacity for cognition consists, to a large extent, of asking the pertinent questions that arise at every moment of our life. These are not predefined but rather enactive: they emerge from action in the world, the relevant aspect being what our common sense deems appropriate within a given context. Thus, knower and what is known, subject and object, determine each other and arise simultaneously. The enactive orientation proposes an intermediate way of transcending both extremes: subject and object define each other and are correlative.

The central point of cognition for Maturana and Varela ( 1997 ) is its ability to elicit meaning: knowledge is not predetermined or established a priori, but implicit in regular processes of cognitive activities themselves. In this way, cognition is not the representation of a world that exists independently, but rather the “production” of a world through the process of living, the “continuous coincidence of our being, our doing and our knowing” (p. 20).

According to Passos ( 2015 ), “to know is to enter a structural coupling with the environment, to interact” (p. 85). However, the understanding of interaction changes in this perspective because it no longer assumes the preexistence of the two terms (organism and environment, subject and object) that interact. To interact in this perspective means to construct oneself and the environment, being, therefore, the “act of knowing reality, an act of affirmation of self, self-surrendering, of autopoiesis. By redefining the cognitive act, its representational sense disappears” (p. 86). In this way, transgression lies in imbuing knowledge with pragmatic value that makes it a performative act, in which to know is to do and vice versa. This delegitimizes the understanding of knowledge through transparency or indifference of the cognitive act: “The whole act of knowing is a form of engagement in the world, of commitment to the world that constitutes itself in this act” (p. 86).

An important element that marks the difference between the enactive approach and any form of biological constructivism or neo-Kantianism is the emphasis on codetermination. In this context, the understanding that conduct is potentially unpredictable marks a departure from the theoretical approach of Maturana and Varela, from other behaviorist and Piagetian approaches.

In this perspective, according to Passos et al. ( 2009 ), subject and object—the poles of cognoscence—are outcomes of cognitive activity rather than conditions. By broadening the concept of cognition and understanding it as linked to creation, the production of knowledge pragmatically and reciprocally shapes the self and the cognitive domain such that cognitive practice engenders subjectivity, overcoming an understanding of dependence of a cognitive subject and a given world, understood as invariant fundamentals. Understanding of cognition as an act of creation brings with it “the problem of the ethical commitment of the cognitive act to the created reality. Production of knowledge, production of subjectivity” (p. 13). The methodological problem is set as follows: “How to study the plane of reality production? What method allows us to follow these processes?” (p. 13). Instead of rules to be applied in the method, the authors offer clues to guide the researcher, since it is not always possible to predetermine every methodological procedure: “The clues that guide the cartographer are like references that contribute to maintaining an attitude of openness to what is happening and of calibrating the course of the research—the meta-hodos of research” (p. 13).

Initially, eight points were proposed to guide the practice of the cartographic method. These were not laid out in hierarchical order but as a rhizome (based on Deleuze & Guattari, 1995 ), referring to each other and forming a set of connections and references to order, develop, and collectivize the cartographer’s experience.

According to Passos et al. ( 2009 ), cartography is an intervention research method (point 1) that aims to track the process, through clues, guiding the course of the research, without establishing a linear path to an end. In this way, it considers the effects of the research process on the research object, the researcher, and the results, and does not simply represent an object: “Cartography seeks to ensure the accuracy of the method without giving up the unpredictability of the process of knowledge production, which is a positive requirement of the ad hoc investigation process” (Kastrup, 2007 , p. 19). What sets it apart from other approaches is the focus on the process and not the end result. With the aim of tracking the process (point 3), clues may arise, which might help to describe, discuss, and, above all, to collect the experience of the cartographer.

In this sense, the cartographer needs to keep in mind that the action of researching his object in motion constitutes a practice in which his path establishes links with the participants inserted into the context of what is being investigated. This composition of agency between heterogeneous actors is expressed by Barros and Kastrup ( 2009 ), drawing from Caiafa ( 2007 ). For these authors, agency implies a relationship of cooperation, a kind of sympathy, which, in addition to a simple feeling of esteem, refers to a composition of bodies implying mutual affection that enable the ethnographer to effectively “enter into relationship with the heterogeneous ones that surround it, to act with them, to write with them” (Barros & Kastrup, 2009 , p. 57).

According to Passos et al. ( 2009 ), cartography as a methodological orientation needs to be articulated using three ideas that make up a plan of action or a research plan: transversality, implication, and the dissolution of the observer’s point of view (point 6). In traditional third- and first-person methodologies, there is always the imposition of a point of view capable of representing or signifying the object at hand. There must be an observer, which implies the “subject–object separation or duality, as well as the imposition of an interpretative reference frame separate from experience” (Passos & Eirado, 2009 , p. 121). These authors discuss the work of Varela, Thompson, and Rosch ( 2003 ), who point out that third-person methodology does not work when studying cognition or the mind, because there is a circularity between knowledge and the known world that is fundamental but often overlooked. This is more evident in studies of cognition because it is not possible to separate the structure that is known from the experience of knowing. This understanding of cognitive experience as its own creation, that is, of both the known object and the subject it knows, which occurs in circular motion, is called “enactive action or approach, modulating the notion of autopoiesis formulated by Maturana and Varela in the 1970s” (Passos & Eirado, 2009 , p. 121).

For Passos and Eirado ( 2009 ), the biology of knowledge, autopoiesis, accepts the challenge of thinking without a foundation, since third-person methodology needs to be complemented by first-person methodology. Woven together, these make it possible to penetrate the circularity that arises in the experience of acquiring knowledge. The cartographer has to avoid merely seeking solutions and testing hypotheses, for “he does not take the self as an object, but the self-emergence processes as the destabilization of the points of view that collapse the experience in the (‘internal’) self” (p. 123). He must inhabit the experience without being bound by any point of view, his main task being to dissolve the observer’s point of view without neglecting observation. Enaction assumes that all experience emerges from experimentation, since it does not refer to what is already a given but rather the emergence of change. Data does not exist a priori, waiting to be gathered, instead, it is constituted in the experience itself. In this way, it is up to the cartographer to accompany this emergence of himself and the world in the experience, and for this it is imperative to be immersed and never immune to the process.

Kastrup and Barros ( 2009 ) argue that the method is not a research model developed through clues, strategies, and procedures. The procedures are embodied in apparatuses (dispositif) that perform important and distinct functions in the operation of cartography. 2 Grounded in Deleuze’s work, they understand apparatuses as “machines that make it possible to see and speak,” composed of lines of visibility, enunciation, force, and subjectification. These apparatuses are aligned with the process of creation, and the work of the researcher-cartographer is to unravel these lines and monitor their effects. The purpose of the apparatus involves three movement functions (point 4): reference (more or less regular apparatus, in which repetition and variation are articulated); explicitation (research territory to be explored, explicitation of the lines that participate in the ongoing production process, inseparable dimensions, research and intervention); and transformation production (“transformation of the relations between the elements/lines/affective, cognitive, institutional, micro and macropolitical vectors, activating movements and sustaining processes of production” [Kastrup & Barros, 2009 , p. 80]). Cartography can produce and transform the reality to be analyzed. In this way, mapping implies intervention.

The cartographer, the person using the cartographic method, does so through “cartographic attention.” Cartographic attention (point 2), according to Kastrup ( 2007 , p. 15), is based on Freud’s concept of “free-floating attention” and Bergson’s concept of “attentive recognition.” It is concentrated and open with four varieties (movements): tracing, touch, landing, and attentive recognition. The cartographer’s work begins with tracing , which involves scanning/sweeping the field, an overview with open and unfocused attention. It is a broader look at something that touches it, beyond the search for information. Touch triggers the selection process, the first meaning, the first analysis performed on the selection process. It is characterized by a quick sense of focus on attention, when something touches, it draws attention, causing it to become alert, but that does not yet define what the cartographer will focus on. The movement that refers to a defined point of attention and focus is the landing , which is to stop, zoom in, choose/define, and indicate that the selected element needs to be inspected more closely for analysis. That is, “the landing gesture indicates that the perception, whether it is visual, auditory or otherwise, makes a stop and the field closes in a kind of zoom. A new territory is formed, the field of observation is reconfigured” (Kastrup, 2009 , p. 43). The fourth and final movement is of Bergson’s attentive recognition , characterized by an investigative attitude about the landing, to which the cartographer’s attention is drawn. It represents analysis itself.

The cartographer’s objective is to map a territory that he/she did not previously inhabit (point 7), to understand the planes of power (point 5)—a moving plane of the reality of things at work in it—and to produce knowledge over the course of research, which involves attention and, with it, the very creation of the field of observation (Escóssia & Tedesco, 2009 ).

Because it is a form of intervention research, the analysis occurs in the process, in the movement of cartography, which makes it possible to carry out the intervention while the process is taking place. Thus, Escóssia and Tedesco ( 2009 ) point to the double direction of the nature of cartography: as a knowledge process that is not restricted to describing or classifying the formal contours of the objects of the world, but in tracing the movement itself that animates them, and as a practice of intervention, where access to the plane of power implies inhabiting it, so that the acts of the cartographer, also a collective of forces, participate and intervene in the changes and in the transformations that occur.

This inhabiting of an existential territory is significantly different from the “application of a theory or the execution of a prescriptive methodological planning, since it implies receiving and being welcomed in the difference that is expressed between the terms of the relation: subject and object, researcher and researched, I and the world” (Alvarez & Passos, 2009 , p. 148). In cartography, one does not “separate theory and practice, spaces of reflection and action. To know, to act and to inhabit a territory are no longer experiences distant from each other” (p. 149).

As far as the cartographic method of research intervention is concerned, where the data are produced Passos and Barros ( 2009 ) emphasize the question of narrativity, that is, it is always narratives that we deal with, being that sometimes the research participants also are cartographers. What each one says, what the situation says implies taking a position in a certain narrativity politics (point 8). 3 This narrative position (ethos of research) is embedded in other policies that are at stake, such as research policies, subjectivity, or cognitive policies. So all production of knowledge comes from an implicitly political position. According to the authors, narrativity politics refers to the position we take in facing the world and ourselves. In this way, “the knowledge we express about ourselves and the world is not only a theoretical problem, but a policy-related problem” (Passos & Barros, 2009 , p. 151).

According to Passos and Barros ( 2009 ), narrativity politics involves two methods and two ways of speaking—extensivism and intensivism—and also two narrative procedures: redundancy (“organizing what in this case is abundance, generating a circulation of meaning that reinforces the clarity of the case, its unity and identity” [p. 158]) and disassembly (“extracting from the larger case the agitation of microcases as microstruggles brought into the scene” [p.161]). In this dismantling process, three characteristics stand out: (1) the procedure to narrate the “case” is due to an increase in the coefficient of deterritorialization; (2) “everything is political”; and (3) everything acquires collective value.

Continuing their elaboration of the eight points of the cartographic method, Passos et al. ( 2014 ) cite thinkers besides Deleuze, Guattari, Maturana, and Varela, introducing Latour, among others, into the discussion regarding the research experience. According to Passos et al. ( 2014 ), the “importance of the research experience points to its inscription on the plane of powers, which constitutes the production plan of both knowledge and known reality” (p. 8). Researchers are immersed in the experience, which distances the cartographic method from other approaches guided by processes such as the “collection,” processing, and analysis of the data, taken as information. In this way, the cartographic method is based on inventive cognition and creative cognition, thus differentiating itself from the idea of the representation of a preexisting world. Therefore, “the cartographic method is not defined by the procedures it adopts, but it is an activity guided by a directive of a nature that is not strictly epistemological, but ethical-aesthetic-political” (p. 9).

Kastrup, Tedesco, and Passos ( 2015 ) point out that the cartographic method is compatible and can be used alongside different techniques, strategies, and research approaches, among them interviews, data analysis, and qualitative or quantitative strategies. In this way, the method is fluid, distinct from methodological models guided by the assumptions of representation. However, research that investigates the experience of research itself must make clear the “firm position of the cartographer with regard to the guideline of research: access/production of the plane of powers that responds to the creation/transformation of experience” (p. 9).

Inventive cognition emerges from the biology of knowledge (Maturana and Varela), of enunciative cognition (Varela), and includes elements of Bergson, Nietzsche, Foucault, Deleuze, and Guattari. According to Kastrup ( 2015 ), thinking stems from stimuli that make you think and does not happen spontaneously, from nothing. The stimuli are, therefore, forces of the present, of a world in movement and accelerated transformation constituting “the unique ground of emergence of thought and novelty” (p. 96). This understanding differs from that held by those who understand cognition from a perspective that Maturana and Varela ( 2001 ) call environmentalist (a realistic assumption of a given world), which “does not allow us to think about the invention of the world itself and above all the world in transformation” (p. 96). Thus, Kastrup ( 2015 ) seeks to understand the “shifts of cognition in the contemporary,” from the encounter of two intercessors for the psychology of cognition: (a) Maturana and Varela, who promote the idea of the biology of knowledge (autopoiesis), by refusing the model of representation and promoting the understanding of cognition as an invention of oneself and of the world; and (b) Deleuze and Guattari, who focus on the transformations currently taking place in cognition. To this understanding, the author adds that, in order to be able to understand the new ways of knowing and living emerging today, it is necessary to affirm the present as a movement of virtualization of currently constituted cognitive forms. The conditions of cognition have in themselves tension between constituted forms and forces of instability: “Forces of the present, which problematize the old forms, placing cognition on the route of experimentation” (Kastrup, 2015 , p. 97).

In this context, Kastrup ( 2015 ) refers to DTs, stating that they cannot be understood as mere objects or as solutions to old problems but as a basis “for creating new problems, new relationships with information, in time, with space, with oneself and with others” (p. 97). 4 Thus, the relation between the constituted forms and the present is not of rupture or of discontinuity, but of coexistence, the conditions of cognition being polytemporal and not invariant or historical. The problem of cognitive functioning is in how the present can provoke “cracks in historical strata, in old mental habits, in established structural couplings and produce novelty,” in addition to understanding it as historically produced. “It is the living gift that coexists with the history of structural couplings. Through this notion, Varela introduces in the studies of cognition the possibility of thinking it into becoming, becoming that makes the history bifurcate” (pp. 98–99).

In the biology of knowledge, with the concept of autopoiesis, Varela resignifies the understanding of learning by, in approaching the problem, placing the actor as the prototypical apprentice. Learning is not, as previous theories proposed, adapting to a given environment, or obtaining knowledge, but experimentation, invention of self and the world. The invention of a work of art is correlated with the production of the artist him or herself. As a novelty comes a theory of action, since for Maturana and Varela the living system is a constantly moving cognitive system in a process of permanent self-production, that is, autopoietic, which can be understood, according to Kastrup ( 2015 ), by the formula BE = DO = KNOW. 5 In this sense, the functioning of the living being is confused with the process of self-creation; according to Varela ( 1990 , p. 99), “doing is ontological.” Understanding cognition as action or practice leads to its permanent modification and not to invariant structures. In addition to the logic of action, cognition refers to flows in conduct (Maturana & Mpodozis, 1992 , p. 18).

Paradoxically, according to Kastrup ( 2015 ), what ensures the flow of the conduct is precisely the crack, the break, the notion of breakdown, described in Varela ( 1990 ) and Varela, Thompson, and Rosch ( 2003 ), as perturbation, a “problematization” of the structures of the living, ranging from engagements with the world, without it being possible to determine a principle that guides this drift toward the pursuit of a superior equilibrium. Breakdowns are the source of the autonomous and creative side of living cognition and arise as a theoretical-scientific formulation for an understanding of cognition that is not restricted to solving problems but is, first of all, the invention of problems. Thus, Varela ( 1990 ) explains the rooting of cognition in the “concrete,” dealing with earlier conceptions that approach cognition from the point of view of logic, general mechanisms, or representation, grouped under the denomination of “abstract” approaches to cognition (p. 102). The breakdown is a cognitive activity that happens in the immediate present and in that concrete actually lives. This “concrete” for Varela ( 1990 ) is not a step for something different, but how we arrived and where we are.

In the perspective of reconciling cognition with concrete Varela presents the notion of enaction (actuation), previously explained. For Kastrup ( 2015 ), this notion refers first to an embodied cognition distinct from the understanding of cognition as a mental process, for it is “tributary to action, resulting from experiences that are not mentally inscribed, but in the body” (p. 103). It is an action guided by local sensory processes and not by the perception of objects or forms. These sensorimotor attachments are not separate from the lived cognition (biological, psychological, and cultural couplings). Thus, the embodiment of knowledge implies social couplings, including linguistic ones, so that the body, in addition to a biological entity, is able to register and mark itself historically and culturally.

In order to exemplify the concept of enaction, linked to the embodiment of knowledge, Varela et al. ( 2003 ) refer to learning a musical instrument, where the musician is taken as a prototype of the learner. In this process of learning, initially the body functions are commanded by the mind, because the process begins with a representation, with symbolic instructions. However, to learn to play an instrument is not to follow rules, and learning truly only happens “when the symbolic relation is transformed into direct coupling of the body with the instrument, eliminating the intermediary of representation” (p. 103). That is, therein is the enaction, actuation, incarnation, or embodiment of knowledge. Thus, for the author, cognition begins to function outside the register of representation, in direct coupling with the matter that the world provides. To learn is not to adapt to the musical instrument, but to act with it. Thus, the notion of acting refers to a collective dimension that appears in the body, at the same time as it indicates the participation of the body in the configuration of the world that is shared by the collective.

This understanding of coupling as agency allows Kastrup ( 2015 ) to move forward in a second sense of the notion of acting— cognition as invention of a world—constructed in the interface between Varela’s cognitive studies and Deleuze and Guattari’s subjectivity production, from whom it takes the concept of agency as “direct communication, without mediation of representation” (p. 104). Communication without subordination, hierarchy, or determinism does not operate by causality, but by reciprocal implication between movements, processes, or heterogeneous flows, by double capture. According to the authors still referring to the learning of a musical instrument, agency refers to the production of a complex apprentice–instrument unit, which produces a process of reciprocal differentiation. The mechanical relationship occurs between previous elements (having the same elements and the same relationships, we will have the same product behavior repeated in the same way) whereas machinic agency, on the other hand, connects flows or processes and creates forms.

Kastrup ( 2015 ) uses again the example of learning a musical instrument to demonstrate that if we understand flute learning, for example, as a machinic agency, “learning is eliminating distances, because one learns between mouth and flute, learns in the middle, on the surface of its coupling, outside the field of representation” (p. 104). That is to say, in this adaptation with the medium, “the blowing motion is able to interact with the arrangement of the instrument and at the same time generate the sound and the apprentice” (p. 105). Thus, coupling should be thought of as a machinic agency or a product of learning, a creative activity always focused on becoming and not a mechanical representation or repetition. This understanding puts an end to the supposed determinism of the object or the environment. The best learner is the one who permanently creates a relationship with the instrument, incessantly reinventing himself as a musician.

The best student, for Kastrup ( 2015 ), is not the one who approaches the world through crystallized habits, but who can always remain in the process of learning, which can also be understood as permanent unlearning. That is, learn is to experience incessantly in order to evade the control of representation, preventing crystallized habits from forming, that is, be alert to continuous variations and rapid resonances, implying, at the same time, a certain lack of attention to the practical schemes of recognition.

Bergson ( 1934 ) theorizes this relation between certain attention and correlative inattention. For this author, there is a pragmatic, utilitarian life that assures learning while solving problems, but there is also an additional attention, which is attention to duration, which ensures learning as the invention of problems.

In this context, it is fundamental to consider that the contemporary world has provoked the emergence of new forms of subjectivity, mainly by the ceaseless and almost omnipresent presence of all kinds of DT, which has accelerated processes of transformation and innovation in the ways of living and engaging, which are more and more open and in flux. In this whirlwind of uncertainty, subjectivity is called upon to reconfigure itself and must learn to deal with breakdowns, with the disturbances that present themselves. On the other hand, this same reality, coupled with the online approach to the most diverse cultures, according to Kastrup ( 2015 ), reveals the precariousness of any supposed foundation that can be provided by the world (p. 108), that is, if we are affected on all sides by disturbances of all nature, solutions are not assured. Therefore, if we want to create new ways of knowing and living, we must invent a world, for learning to live in a world without fundamentals is to invent it by living, remembering that invention of self cannot be achieved without the invention of a related world.

If interpretation and knowledge are emergent results (in the sense of emerging) of action in the world or acting, when spaces are hybrid, multimodal, pervasive, and ubiquitous, and dwelling is atopical, what are the relevant issues that emerge concerning the action and performance of the subjects in these spaces? How do meanings emerge? What world do we produce and invent?

The Cartographic Method as a Means of Identifying and Mapping Controversies and Actor–Network Theory

Recently, actor–network theory (ANT), developed by Latour, Law, and Callon, also recognized subject–object codetermination by emphasizing the participation of non-humans—objects and quasi-objects—in social relations, thus presenting itself as an alternative to the binaries of modernity by eschewing a compartmentalized view of reality. 6

In this perspective, ANT (Latour, 2012 ) provides a new understanding of what is social, presenting the idea that humans establish a social network not only to interact with other people, but with non human elements as well. According to the author, the social is not simply made up of people, but also machines, animals, texts, money, architecture, laboratories, institutions, among other elements. By the principle of connectivity, everything is linked in a network, with multiple inputs, which is always in continuous movement and open to new elements. For the author, social refers to the network of HA and NHA, where the actor is any person, thing, (quasi-)object, or institution that produces agency, that is, something with the ability to produce effects on the network (although indirectly), of being actant. The understanding of agency, therefore, is related to the human and non human actors (actants), similarly, who participate in the actions and provoke transformations in the network, in movement. Thus, in ANT, or sociology of associations, the non human is no longer considered only as an artifact, whose meaning is attributed by the human, but as having agency, because it participates in actions in everyday situations and causes transformations. Non human actors also shape events in the creation of meanings, acting in the reflective and symbolic sphere.

In this context, network is understood from the perspective of a rhizome (based on Deleuze & Guattari, 1995 ), that is, seen as something alive, changeable—as flows, circulations, alliances, and movements of a series of animate and inanimate elements—and not as fixed to a set of actors. It refers to transformations, translations, displacements, therefore, quite distinct from the traditional understanding of a network as a form or structure. The network is the associative movement that forms the social, being rather an instrument of analysis or its object. The actor–network binomial perspective proposes that the actor never acts alone. In acting, it is influenced (constituted) by the networks in which it has connections and, at the same time, it can represent these networks, as well as influence them. In this way, it is never quite clear who is acting. The actor is, at the same time, the builder and receiver of the networks.

For Latour ( 2012 ), the social has no predefined locus, but is understood as provisional, performative, as processes of aggregations, associations, and reassociations between HA and NHA. In order to understand the social, which, therefore, is not what explains but rather what needs to be explained, the author recommends that the actors be followed in their associations and reassociations (cartography). I understand, therefore, that nowadays these associations and reassociations are increasingly constituted in nomadic movements, in an atopic habitation that takes place in hybrid, multimodal, pervasive, and ubiquitous spaces. 7

With regard to science, Latour ( 2016 ) says that every idea only proceeds from multiple deviations and compositions. It is the attribution of science to understand this process and not only the result. For this, it is necessary to retrace the entire chain of deviations and compositions, and what matters in this process is what emerges and forms in the course of the process of composition and deviations of courses of action. Linked to this question, Latour ( 2016 ) in the second letter of Cogitamus, raises the problem of method: How is it possible to analyze deviations and compositions if, in general, they are invisible?

In this context, the concept of proof becomes the protagonist, because, according to the author, it is at the moment of proof that the blunting of deviations and compositions is revealed. This evidence, although there are other forms of evidence, materializes in the panel: everything works well, until it stops working. This is more pedagogical form of expression of proof. The computer, initially understood as a technical object, is now presented as a sociotechnical project: “From simple, my computer has become multiple; of unified, has become disunited; it immediately became mediate; of fast, it became slow” (Latour, 2016 , p. 47). The network, or part of it, with the different elements that together kept it working, now fails, becoming visible. It is at this moment that it is necessary to analyze the links, the relationships, the networks that integrate it.

This perspective, presented by Latour ( 2016 ), although linked to the method he calls “cartography of controversies,” could also be linked to the cartographic method of intervention research, proposed by Kastrup ( 2007 , 2008 ), Passos et al. ( 2009 ), and Passos, Kastrup, and Tedesco ( 2014 ).

While Latour ( 2016 ) refers to pane and understood as a proof, bringing sociotechnical network analysis (HA and ANH) to the context; we could think from the point of view of the subject’s cognition (HA) as a cognitive imbalance (Piaget) or as a breakdown (Varela) understood as a proof, later assumed by Kastrup, Tedesco, and Passos ( 2015 ) from the perspective of the inventive cognition.

With regard to the interactions that take place in this sociotechnical network, the deviations and compositions are visible by the traces that the different actants produce in the movement of associations, which can be accompanied by “another” sociotechnical network formed by HA (teachers) and NHA (mining and data-based systems and learning analytics), providing elements that allow us to trace the cosmogram and understand the process under construction.

Turning to the question of the pane for Latour ( 2016 )—in the context of a sociotechnical network; cognitive imbalance for Piaget and breakdown for Varela—in the context of cognition; a process of investigation begins, so that the initial indetermination begins to be deciphered, the source of the disturbance is found, and a problem is revealed progressively. 8 Hypotheses and solutions are tested and verified, until the problem is solved.

In the case of the computer (NHA), in the context of sociotechnical network (formed by HA and NHA), the pane is repaired and put back to use. In the case of cognition, what in the comprehension of Varela ( 1990 ) and Kastrup, Tedesco, and Passos ( 2015 ) refers to an enactuated, self-engendered process of agency, caused by a breakdown, takes knowledge to a superior equilibrium, not in the perspective of problem-solving but, above all, in the invention of problems. It is not the result, but the process by which the result is achieved. In a way, this is related to the concept of debugging, a result of a metacognitive process (Piaget, 1976 , 1978a , 1978b , 1995 ) that is necessary for computational thinking, which was very present in research related to language programming in the 1980s and 1990s.

From the proof concept, Latour ( 2016 ), in his third letter in Cogitamus, goes on to discuss scientific controversies, stating that the statements pass between two poles: radical doubt and unquestionable certainty: “At the beginning of the exercise, the statement floats; in the end, one must find it solidly anchored in a precise landscape . . .” (p. 81). According to the author, this is where the importance of controversies lies.

The term controversy, according to Latour ( 2016 ), “designates all possible positions, ranging from absolute doubt . . . to indisputable certainty” (p. 79). The word “controversy” describes a shared uncertainty about aspects of science and/or technology that are not yet stabilized. It occurs when there is a disagreement between the actors, that is, “when actors discover that they cannot ignore each other and controversies end when actors manage to work out the solid commitment to live together” (Venturini, 2010 , p. 260). The cartography of controversies consists in mapping the actions of human and non human actors involved in contemporary sociotechnical questions, without, however, assuming an a priori frame or an order to be followed. In this context, according to Latour ( 2012 ), the Actor-Network Theory (ANT) achieves a better understanding of the order after the actors explain all the controversies in which they were involved, that is, “We [social scientists] will not try to discipline, to frame you [the actors] in our categories; we will allow them to stick to their own worlds and only then will we ask for their explanation of how they were established” (p. 44). It is not up to the analyst to define and order the social, but rather to the actors present in the context. If the goal is to restore order, it is best to go through the associations, tracing the connections between the controversies themselves. The pursuit of order, rigor, and pattern is by no means abandoned, just repositioned one step further in the form of abstraction, so that actors can unfold their own and various cosmos, no matter how irrational they may seem.

Latour ( 2012 ) notes that it is possible to trace stronger relationships and discover more revealing patterns when we find a way to record the links between unstable and mutable frames of reference rather than trying to stabilize one. What makes one expand, relate, compare, and organize is what one has to describe. The important thing is not to stop the flow of controversy, because if the actors do not act, they will leave no clues: “No clue, no information, no description—and therefore no conversation” (p. 217). It is in the flow of controversies that one has to find the “firm ground: on shifting sands. Contrary to what is commonly said, relativism is a way of floating in the data, not plunging into it” (p. 46).

The main idea is to give visibility to the different understandings about situations, movements, representativities, influences, and interests. For this, it is necessary to explore, visualize polemics, the movement of action and motion, that is, where mediation flows (Lemos, 2013 ). Controversies are those spaces of dialogue, conflict, negotiation, and action, which the actors reveal by leaving traces.

For Latour ( 2016 ), mapping a controversy is learning to locate all these movements. This implies following the statements from doubts permeated by intermediate states (rumor, opinion, idea, proposition), pros and cons, until they become the

final result, where clear and well-defined inscriptions are evident. However, with the condition of taking it in its motion, and not frozen in an object. Remembering that nothing is definitive, both techniques and sciences do not exist by the simple force of inertia. To exist is to always be in that front line. What we now understand as a certain statement is only the final stage of a controversy and in no way its beginning. (Latour, 2016 , pp. 80–81)

Latour ( 2016 ) states that the two extremes must be considered: fact and opinion, which correspond to two moments in the controversy. Accompanying the controversies is then to describe the ways in which the actors construct and modify the evidence.

Venturini ( 2012 , p. 800) proposes a script to subsidize the creation of cartographies of controversies, which is summarized by Lemos ( 2013 , p. 118): (1) to define the best possible controversy; (2) observe, describe, and maintain that the object is controversial; (3) identify whether the controversy is: cold/hot, present/past, secret/public, difficult to access/accessible, limited/unlimited; (4) apply the lenses to the collection of information (gather statements, opinions, read the specialized literature); (5) identify human and non human actants and sketch the network that connects them; (6) identify cosmogram, ideologies, and worldviews. The cartographer must then identify the representativity, influence, and interest of the actors in the networks.

If there is evidence, Latour ( 2016 ) proposes that two conditions are imposed on the analysis: (1) that it comes from the actors, and (2) that it results from joint activity, therefore, not more than a cogito, but a Cogitamus.

In this context, the author refers to the new passage from the infinite world (modern and post-scientific revolution) to the complicated multiverse or pluriverse, term coined by William James. Thus, if there is no radical discontinuity, the meaning of the word revolution changes. Latour ( 2016 ), based on Sloterdijk, stresses that it is not a matter of revolution or emancipation, but of explicitness, since “history never breaks with the past, but permanently makes more and more explicit with which we have to learn to live, elements that will be compatible or incompatible with existing ones” (p. 115). Thus, the author inaugurates a new “epistemological policy,” in which the researcher’s function is to describe “the agency of all beings that a particular culture links with practical forms of life” (p. 166). This refers to a movement of associations between the different parties participating in a controversy, which can be represented by what the author calls a cosmogram. In order to retract the different parties, Latour ( 2016 ) proposes the description of the associations of convenience, coexistence, opposition, and exclusion between HA and NHA, whose conditions of existence become explicit in the course of the trials submitted by the disputes: “to become sensitive to these lists of associations and logical duels without resorting to the distinction between the rational and the irrational, the modern and the archaic, the systematic and the unsystematic” (Latour, 2016 , pp. 116–117).

Mapping the cosmograms means working with the movement, with the agency distribution drawing of mobility, unlike the paradigm, which works with frames (stabilized theoretical framework). When designing a cosmogram, the analyst does not need to resort to structures, systems, or frames, which limit or even render unviable the monitoring of the connections in formation in the sociotechnical networks. He needs to go through the network, follow the actants in their associations, identifying the controversies, their different visions or world versions (multiverses or pluriverses), showing how the whole world (a complicated and complex pluriverse), a cosmos, reveals itself, emerges, and reconfigures itself in the confrontation of ontologies.

It is within the scope of interactions that occur in this sociotechnical network that the different actants (human and non human) produce traces in the movement of associations. These traces, in the scope of education research, can be accompanied by another sociotechnical network formed by HA (teachers-researchers) and NHA (diverse systems, including those based on mining and data and learning analytics) which, when mapped, provide elements that make it possible to draw the cosmogram (diagram of mediations, motion, mobility), a multiverse, and understand the process under construction—the movement of these associations happening. The aim is to map the controversies: “the study of innovations and controversies is one of the first privileged places where objects can be held longer as visible, disseminated and recognized mediators before they become invisible, non-social intermediaries” (Latour, 2012 , p. 120).

It is important to consider that for Latour ( 1994 ), the logic of thinking of the sciences is vitiated by a need for purification by division, which results in exclusion. According to Melo ( 2011 ), this logic, in a way, freezes the possibility of transformation, because it does not consider the continuous mixture, which produces hybrids incessantly and indefinitely. Instead of imposing one part(s) on another, replacing one or the other(s), as the modern project intended, the parts become the contact with heterogeneous versions, through the adjustment of hybrid practices and interests, through which these parties receive from each other the chance of mutual transformation (Despret, 2002 ). TAR presents the notion of hybrids and generalized symmetry.

It is possible to bring elements of this vision presented by Latour to reflect on the question of culture, since it is not a question of dividing, of classifying, into analogical (pre-digital) culture, digital culture, or cyberculture, or, but to consider the mixture, that is to say, the hybrids that are produced in that mixture, which is resignified and transformed in that coexistence.

Conclusion: Education Qualitative Research in the Hybrid and Multimodal Culture Context in an Atopic Dwelling

Both methods, the cartographic method of intervention research and the cartography of controversies, with their specificities, have proved robust for the development of qualitative research in the field of education, in the context of digital culture and hybrid culture in an atopic dwelling, mainly because they are also related to equally consistent and contemporary theories in terms of human cognition aspects, enabling us to track the traces and clues in the mobility of associations between actants, which are enhanced by different Digital Technologies (DT), including data mining and learning analytics.

These methods have been used in the research developed by the Grupo de Pesquisa Educação Digital (GPe-dU Unisinos/CNPq), in which I am involved at the Universidade do Vale do Rio dos Sinos (UNISINOS), since 2010 . In particular, the cartographic method of intervention research, as well as being appropriate as a research method, has been investigated for its potency with regard to the development of new methodologies and pedagogical practices, due to its interventionist characteristic in accompanying the process (along the course), aligned with the need to understand the phenomenon of learning in its complexity—social, political, cognitive, affective, and technological (Schlemmer, Lopes, & Molina, 2012 ), in contexts of hybridism, multimodality, pervasiveness, and ubiquity. Its power to accompany learning processes has also been investigated as well as the possibility of it being appropriated by teachers and students in their own learning pathways.

Among the research projects that have used the cartographic method of intervention research are: “Escola aumentada: Cartografias digitais para as aprendizagens e a cidadania,” “Gamificação em Espaços de Convivência Híbridos e Multimodais: Uma experiência no ensino superior,” and “Gamificação em Espaços de Convivência Híbridos e Multimodais: A educação na cultura digital.” Under different approaches and in varying contexts, research explores the perspectives of hybridism, multimodality, pervasiveness, and ubiquity, as well as cartographic attention. It was in the context of these projects that the cartographic method of intervention research began to become the object of study, inspiring studies carried out on the formation and qualification of teachers and students (Lopes & Schlemmer, 2017 ; Lopes & Valentini, 2012 ; Schlemmer, 2014 , 2015 ; Schlemmer & Lopes, 2016 ) in the scope of elementary education, graduation in pedagogy and digital games, post-graduation stricto sensu , and continuing teacher training. Our focus has been to explore some elements related to the hybrid, multimodal, pervasive, and ubiquitous culture and the new regimes of action, participation, and socialization of experience.

In the context of elementary education schools, we have outlined some experiences involving cartography with the support of digital media, geolocation, and digital marking (quick response [QR] codes), to provoke experiences of local mapping of people, objects, and places in the public space. In the undergraduate, and graduate continuing teacher education context, as well as in elementary school, we also designed inventive methodologies and pedagogical interventionist, aggregative, and gammatical practices with a cartographic bent, mainly the cartographer attention through the four movements (tracing, touching, landing and attentive recognition) , linked to elements of gamification, with the support of digital media, geolocation, digital coding (QR codes), and augmented reality, in order to provoke learning experiences in a hybrid, multimodal, pervasive, and ubiquitous context. According to Schlemmer and Lopes ( 2016 ), unlike classic cartography (mapping), the idea was to provoke aesthetic and/or informational experiences for the production of meanings about our environment in the case of schools, and on the concepts present in undergraduate and undergraduate academic activities in the case of higher education. The purpose of these experiments is to activate sensibility and cognition as functions of intelligence, as well as registration and sharing as functions of sociability.

Schlemmer ( 2018 ) states that inspiration in the cartographic method of intervention research to develop inventive methodologies and interventionist, aggregative, and gamified pedagogical practices comes from the following elements:

the intervention research perspective, and, during the development of games and gamified processes, at different times, both the teacher and the subjects themselves act as interventionist mediators

the idea of working with learning as an invention of problems (“supplementary” attention, duration, attentive recognition of a context), besides learning how to solve problems (attention to pragmatic utilitarian life)

the proposal to follow the course

the clue metaphor

movements of the cartographer’s attention (tracing, touching, landing, and attentive recognition).

In the context of tracks, according to Schlemmer ( 2018 ), the proposal is to work with the concept of geographic tracks (local/specific points in the community/city), live tracks (people from local community who hold certain knowledge necessary for the development of the missions), online tracks (specialists that do not belong to the local community, but that can appear in video), as characters in Mixed Reality (MR) or in Augmented Reality (AR) (iotized objects).

In the scope of a cartographer’s attention movements, tracing is characterized by the exploration/scanning of the field—in the case of this research, geographic and online spaces in search of clues (information) to understand the processes; the touch triggers the selection process which consists, in this case from Schlemmer ( 2016a , 2017 ), in the selection of geographic clues, online clues, and live clues, to direct the research; and landing refers to stopping, zooming in on lanes, choice/definition, and attentive recognition in the perception of the global context.

The metaphor of the clue, as well as the changing focus of the cartographer’s attention, can also serve as inspiration to understand the composition of tracks, as well as the progression itself in the context of a game or gamified process—gaining achievements—at the same time as achievements can be understood as skills for the own gameplay and sociability. 9

It is important to point out that as a result of this process the inventive methodology |Gamified Learning Processes (GLP) is created (Schlemmer, 2018 ).

The Grupo de Pesquisa em Educação Digital (GPe-dU) has also investigated and developed theoretical, methodological, and technological experiences inspired by the cartographic method of research, as methodology for the monitoring and evaluation of learning in games and gamified processes from an interventionist perspective and developed in a hybrid, multimodal, pervasive, and ubiquitous context. According to Schlemmer and Lopes ( 2016 ) and Schlemmer ( 2016a ) such contexts are more easily subject to the pulverization of the spaces of participation and registration and, therefore, can hinder the exercise of teaching and of the discourse regarding both pedagogical mediation and the evaluation. Thus, the proposal developed by the authors, inspired by the cartographic method of intervention research as a methodology of monitoring and evaluation, allows accompanying the subjects in their different learning pathways, involving analogue technologies and DT, physical and online face-to-face interactions to develop their own missions and projects that, from the perspective of bring your own device (BYOD), can extend beyond the time set for formal education. The fact that the subject possesses a mobile device and is connected creates conditions of possibility for him or her to remain engaged in the process, regardless of time and space. Thus, the processes of monitoring and evaluation can, at different times, be “situated” and still intertwined. By means of clues, designed and planned to provide the hybridization of analogue and digital spaces, it is possible to establish a multimodal context, which is desirable when talking about immersion, agency, and engagement.

More specifically, with regard to the movements proposed by Kastrup ( 2007 , 2008 ) and others in the cartographic method, the development of gamification and game experiments allowed us to evaluate the power of the method and the inadequacies we perceive in our own experience of building the game or gamification. The question that seemed to us most challenging was to guarantee the unpredictability and rhizomatic opening of the cartographic method and attention. If we work from a perspective of “use of” rather than inventiveness, the design of phases of a gamified game or process, for example, which was designed by someone to be “applied” in education or simply played by players, cannot always guarantee the rhizomatic opening that underlies the two proposals of methods previously carried out, because it has a limited context and whose control is not complete developed in the field reconfiguration of the students learning. The possible reconfigurations take place in the circular or linear dimension of success itself when completing the missions. In this sense, an a priori objective always seems limited from the point of view of cartography, but not limiting from the point of view of learning. What is learned opens possibilities, but in the dynamics of the game these are not necessarily unpredictable, since it is a condition that games advance in phases.

The clue metaphor to the tracks conception, according to Schlemmer and Lopes ( 2016 ), was inspiring for game designers or gamification, but, to become inspirational to gaming, we identified the need for players to leave “traces” which may become clues for other players. An interesting strategy would be to insert notebooks as an object/item that loads and can be left somewhere—as in some online and offline role-playing games—into the dynamics of the game or gamification. Another strategy would be to insert HA and/or bots equipped with AI to record and report events, producing clues for the route of the tracks to grow and insert, at continuously, new challenges based on the reconfiguration of the field of knowledge produced by the players themselves. This perspective of valuing, in the context of game or gamification, ways of recording and sharing personal narratives (dynamic clue production) would be an interesting possibility for both teachers and students to map their learning processes—after all, mapping, from the perspective here presented, is a means to track processes.

In this case, it seems important to consider that it is necessary to invest in game dynamics that strengthen and value the narratives of the players (as in the case of Role Playing Game [RPG] or, better still, to work at the level of inventiveness, in which the entire process of the conception and development of the game or gamification is co-constructed by the learning subjects themselves).

It is important to mention that, in the case of the researches we develop, because they are situated within the scope of enactive and inventive cognition, the creation of the game or process is a result of a co-creation process between and with the learning subjects themselves, where the teacher acts as an interventionist, conducting pedagogical mediation. Thus, everything that composes the game or gamified process—from the pre-concept, concept, and development; what is implied in the definition of mechanics and dynamics; to being able to include tracks and progression—are defined by the subjects themselves who, therefore, extrapolate the perspective of knowledge as representation and learning as problem-solving, working on the level of knowledge as interpretation and learning through the invention of problems, from the notion of the rhizome, which is at the base of the development of both the cartographic method of intervention research, as well as the cartography of the controversies. This approach is qualitatively different from the traditional approaches found in games or gamified processes, where the subjects are only users, players, whose learning is by solving problems and their evolution by linear stages or phases.

Most recently, the projects “The City as a Learning Space: Games and Gamification in the Constitution of Hybrid, Multimodal, Pervasive and Ubiquitous Spaces for the Development of Citizenship,” and “The City as a Learning Space: Education for Citizenship in Hybrid, Multimodal, Pervasive and Ubiquitous Contexts,” also developed on the basis of the cartographic method of intervention research, we have more appropriately called the cartography of controversies, in order to better understand, in addition to what occurs at the micro level (enative and inventive cognition), the associations that occur between HA and NHA, which may be due to convenience, coexistence, opposition, and exclusion. These associations also evidence planes of forces and, therefore, political expression of cognition, in the sense that “knowing involves a position in relation to the world and itself, an attitude, an ethos” (Kastrup, Tedesco, & Passos, 2015 , p. 12). These associations can be evidenced in a cosmogram, which makes it possible to draw the distribution of the agency, the diagram of the mediations, in short, the design of the movement, its formation. In this way, the cosmogram works with the empirical and in motion, unlike the paradigm that works with the frame, with the theoretical framework stabilized with the model as structure.

Schlemmer and Lopes ( 2016 ) and Schlemmer ( 2018 ) emphasize that the proposal does not consist in a transposition of the method or methods, but rather an experimentation with the logic behind them, as well as some of its elements, which are linked to others, in this case, present in games, gamification, and PAGs, allowing us to develop inventive methodologies and pedagogical interventionist, aggregative, and gamified practices in the educational context.

Thus, relating the cartographic method of controversies and, consequently, elements present in the ANT with the cartographic method of intervention research and with the perspective of enactive cognition, in its two senses of action—corporate cognition and inventive cognition—allows us to understand that the process of invention or innovation, both in development and research, advances from multiple breakdowns, deviations, compositions, associations, and reassociations that occur in the empirical movement. In this way, understanding an invention, an innovation, implies tracking the traces and retracing the whole network of deviations and compositions, associations and reassociations that constitute the path.

From this context questions that inspire future investigations arise:

How can Latour’s concept of symmetry or flat ontology, in which HA and NHA are on the same plane (which eliminates the anthropocentric view of cognition), contribute to research in education?

How can the Latour cosmogram contribute to the intervention research cartographic method as a means of understanding the paths made by the different actors, as well as the controversies established in a network?

How can the intervention research cartographic method, comprising cognition as the invention of problems, associated with the cartography of controversies, be part of the methodology to help us understand the place of the human in the context of intelligent cities?

To sum up, “we go, we hear, we learn, we practice, we become competent, we change our minds. Very simple indeed: this is called research. Good research always produces copious new descriptions . . . There is no in-formation, just trans-formation” (Latour, 2012 , pp. 212–216).

• Alvarez, J. , & Passos, E. (2009). Cartografar é habitar um território existencial. In E. Passos , V. Kastrup , & L. Escóssia (Eds.), Pistas do método da cartografia: Pesquisa-intervenção e produção de subjetividade (pp. 131–149). Porto Alegre, Brazil: Sulina.
• Barros, L. P. de , & Kastrup, V. (2009). Cartografar é acompanhar processos. In E. Passos , V. Kastrup , & L. Escóssia (Eds.), Pistas do método da cartografia: Pesquisa-intervenção e produção de subjetividade (pp. 52–75). Porto Alegre, Brazil: Sulina.
• Bergson, H. (1934). La pensée et le mouvant: Essais et conférences. In Œuvres: Édition du Centenaire . Paris, France: Presses Universitaires de France.
• Caiafa, J. (2007). Aventura das cidades . Rio de Janeiro, Brazil: Ed. FGV.
• Deleuze, G. , & Guattari, F. (1995). Mil platôs–capitalismo e esquizofrenia , Vol. 1 ( A. Guerra Neto and C. Pinto Costa , Trans.). Rio de Janeiro, Brazil: Editora 34.
• Despret, V. (2002). Quand le loup habitera avec l’agneau . Paris, France: Les empecheurs de penser em rond.
• Dewey, J. (1938). Experience and education . New York, NY: Macmillan.
• Di Felice, M. (2009). Paisagens pós-urbanas: o fim da experiência urbana e as formas comunicativas do habitar . São Paulo, Brazil: Annablume.
• Escóssia, L. , & Tedesco, S. (2009). O coletivo de forças como plano de experiência cartográfica. In E. Passos , V. Kastrup , & L. Escóssia (Eds.), Pistas do método da cartografia: Pesquisa-intervenção e produção de subjetividade . Porto Alegre, Brazil: Sulina.
• Foucault, M. (1979). Microfísica do poder (Organized and translated by Roberto Machado ). Rio de Janeiro, Brazil: Edições Graal.
• Kastrup, V. (2007). O funcionamento da atenção no trabalho do cartógrafo . Psicologia & Sociedade , 19 (1), 15–22.
• Kastrup, V. (2008). O método cartográfico e os quatro níveis da pesquisa-intervenção. In L. R. Castro & V. Besset (Eds.), Pesquisa-intervenção na infância e adolescência . Rio de Janeiro, Brazil: Nau Editora.
• Kastrup, V. (2009). O funcionamento da atenção no trabalho do cartógrafo. In E. Passos , V. Kastrup , & L. Escóssia (Eds.), Pistas do método da cartografia: Pesquisa-intervenção e produção de subjetividade . Porto Alegre, Brazil: Sulina.
• Kastrup, V. (2015). A Cognição Contemporânea e a Aprendizagem Inventiva. In V. Kastrup , S. Tedesco , & E. Passos , Políticas da Cognição . Porto Alegre, Brazil: Sulina.
• Kastrup, V. , & Barros, R. B. (2009). Movimentos-funções do dispositivo na prática da cartografia. In E. Passos , V. Kastrup , & L. Escóssia (Eds.), Pistas do método da cartografia: Pesquisa-intervenção e produção de subjetividade . Porto Alegre, Brazil: Sulina.
• Kastrup, V. , Tedesco, S. , & Passos, E. (2015). Políticas da Cognição . Porto Alegre, Brazil: Sulina.
• Latour, B. (1994). Jamais fomos modernos . Rio de Janeiro, Brazil: Editora 34.
• Latour, B. (2012). Reagregando o social: Uma introdução à teoria do ator-rede . São Paulo, Brazil: EDUSC.
• Latour, B. (2016). Cogitamus: Seis cartas sobre as humanidades científicas . São Paulo, Brazil: Editora 34.
• Lemos, A. (2013). A comunicação das coisas: Teoria ator-rede e cibercultura . São Paulo, Brazil: Annablume.
• Lopes, D. Q. , & Schlemmer, E. (2017). Considerações éticas, epistemológicas e metodológicas sobre o fazer pesquisa em educação e cultura digital. Revista Edapeci: Educação a Distância e Práticas Educativas Comunicacionais e Interculturais , 17 , 1–15.
• Lopes, D. Q. , Schlemmer, E. , & Molina, R. (2014a). Atenção Cartográfica em Pesquisas Online sobre Políticas de Inclusão Digital. Polise Psique , 4 , 150–169.
• Lopes, D. Q. , Schlemmer, E. , & Molina, R. (2014b). Cartography mediated by digital technologies: New perspectives for ethnographic research. In P. Landri , A. Maccarini , & R. DeRosa (Eds.), Networked together: Designing participatory research in online ethnography (pp. 24–38). Naples: University of Naples.
• Lopes, D. Q. , & Valentini, C. B. (2012). Mídias locativas e realidade mixada: A produção de sen dos sobre o digital-virtual a par r da cartografia com suporte das tecnologias digitais. Educação Unisinos , 16 (3), 205–214.
• Maturana, H. R. (1993a). As bases biológicas do aprendizado. Dois Pontos, Belo Horizonte , 2 (16), 64–70.
• Maturana, H. R. (1993b). Uma nova concepção de aprendizagem. Dois Pontos, Belo Horizonte , 2 (15), 28–35.
• Maturana, H. R. (1998). Da Biologia à Psicologia . Porto Alegre, Brazil: Artes Médicas.
• Maturana, H. R. , & Mpodozis, J. (1992). Origen de las especies por medio de la deriva natural . Publicacion ocasional No 46/1992. Museo Natural de Historia Natural. Santiago, Direccion de Bibliotecas. Archivos y Museos.
• Maturana H. , & Rezepka, S. N., de . (2000). Formação Humana e Capacitação . Petrópolis, Brazil: Vozes.
• Maturana, H. R. , & Varela, F. J. G. (1997). De Máquinas e Seres Vivos: Autopoiese—a Organização do Vivo . Porto Alegre, Brazil: Artes Médicas.
• Maturana, H. R. , & Varela, F. J. (2001). A Árvore do Conhecimento: As bases biológicas da compreensão humana . São Paulo, Brazil: Pala Athenas.
• Melo, M. F. A. Q. (2011). Discutindo a aprendizagem sob a perspectiva da teoria ator-rede. In Educar em Revista (vol. 39, pp. 177–190). Curitiba, Brazil: UFPR.
• Passos, E. (2015). Pensando a subjetividade com conceitos híbridos: A psicologia em interface com a filosofia e a biologia. In V. Kastrup , S. Tedesco , & E. Passos , Políticas da Cognição . Porto Alegre, Brazil: Sulina.
• Passos, E. , & Barros, R. B. (2009). Por uma política da narratividade. In E. Passos , V. Kastrup , & L. Escóssia (Eds.), Pistas do método da cartografia: Pesquisa-intervenção e produção de subjetividade . Porto Alegre, Brazil: Sulina.
• Passos, E. , & Eirado, A. (2009). Cartografia como dissolução do ponto de vista do observador. In E. Passos , V. Kastrup , & L. Escóssia (Eds.), Pistas do método da cartografia: Pesquisa-intervenção e produção de subjetividade . Porto Alegre, Brazil: Sulina.
• Passos, E. , Kastrup, V. , & Escóssia, L. (Eds.). (2009). Pistas do método da cartografia: Pesquisa-intervenção e produção de subjetividade . Porto Alegre, Brazil: Sulina.
• Passos, E. , Kastrup, V. , & Tedesco, S. (2014). Pistas do método da cartografia: A experiência da pesquisa e o plano comum . Porto Alegre, Brazil: Sulina.
• Piaget, J. (1976). A Equilibração das Estruturas Cognitivas . Rio de Janeiro, Brazil: Zahar Editores.
• Piaget, J. (1978a). Fazer e compreender . São Paulo, Brazil: Melhoramentos.
• Piaget, J. (1978b). A Tomada de consciência . São Paulo, Brazil: Melhoramentos.
• Piaget, J. (1995). Abstração reflexionante: Relações lógico-aritméticas e ordem das relações espaciais . Porto Alegre, Brazil: Artes Médicas.

• Schlemmer, E. (2013). Anatomia no metaverso Second Life: Uma proposta em i-Learning . Relatório de Pesquisa. São Leopoldo, Brazil: Unisinos.
• Schlemmer, E. (2014). Gamificação em espaços de convivência híbridos e multimodais: Design e cognição em discussão. Revista da FAEEBA-Educação e Contemporaneidade , 23 (42), 73–89.
• Schlemmer, E. (2015). Mídia social em contexto de hibridismo e multimodalidade: O percurso da experiência na formação de mestres e doutores. Revista Diálogo Educacional (PUCPR. Impresso) , 15 , 399–421.
• Schlemmer, E. (2016a). Games e Gamificação: Uma alternativa aos modelos de EaD. Revista Iberoamericana de Educación a Distancia , 19 , 1–12.
• Schlemmer, E. (2016b). Gamificação em Espaços de Convivência Híbridos e Multimodais: Uma experiência no ensino superior . Research Report. Processo v. 408336, pp. 2013–2027.
• Schlemmer, E. (2016c). Hibridismo, Multimodalidade e Nomadismo: Codeterminação e coexistência para uma Educação em contexto de ubiquidade. In D. Mill & A. Reali (Eds.), Educação a distância, qualidade e convergências: sujeitos, conhecimentos, práticas e tecnologias (Vol. 1 (pp. 1–24). São Carlos, Brazil: EdUFSCar.
• Schlemmer, E. (2017). Gamificação em espaços de convivência híbridos e multimodais: a educação na cultura digital . Research Report. São Leopoldo, Brazil: Unisinos.
• Schlemmer, E. (2018). Projetos de aprendizagem gamificados: Uma metodologia inventiva para a educação na cultura híbrida e multimodal. MOMENTO–Diálogos em Educação , 27 , 41–69.
• Schlemmer, E. , Chagas, W. S. , & Schuster, B. E. (2015). Games e Gamificação na modalidade EAD: Da prática pedagógica na formação Inicial em Pedagogia à prática pedagógica no Ensino Fundamental. In IV Seminário Web Currículo e XII Encontro de Pesquisadores em Currículo (vol. 1, pp. 728–736). São Paulo, Brazil: PUC-SP.
• Schlemmer, E. , & Lopes, D. Q. (2016). Avaliação da Aprendizagem em Processos Gamificados: Desafios para Apropriação do Método Cartográfico. In L. Alves & I. de J. Coutinho (Eds.), Jogos Digitais e Aprendizagem (Vol. 1, pp. 179–208). Campinas, Brazil: Papirus Editora.
• Schlemmer, E. , Lopes, D. Q. , & Molina, R. (2012). Epistemological and methodological challenges in the field of research in education and digital culture. In Rethinking educational ethnography: Researching online communities and interactions Proceedings of the 2nd Annual Rethinking Educational Ethnography conference in Barcelona (Vol. 1, pp. 74–81). Barcelona, Spain: Universitat de Barcelona.
• Varela, F. (1988). Conhecer: As ciências cognitivas tendências e perspectivas . Lisbon, Portugal: Instituto Piaget.
• Varela, F. (1990). Conhecer: As ciências cognitivas: Tendências e perspectivas . Lisbon, Portugal: Instituto Piaget.
• Varela, F. , Thompson, E. , & Rosch, E. (2003). A mente Incorporada: Ciências Cognitivas e Experiência Humana . Porto Alegre, Brazil: Artmed.
• Venturini, T. (2010). Diving in magma . Public Understanding of Science , 19 (3), 258–273.
• Venturini, T. (2012). Building on faults: How to represent controversies with digital methods . Public Understanding of Science , 21 (7), 796–812.

1. A group composed of researchers from the Universidade Federal Fluminense and Universidade Federal do Rio de Janeiro.

2. The idea of an apparatus (dispositif) is based on Foucault ( 1979 ), who understood it as a decidedly heterogeneous agglomeration involving discourses, institutions, architectural organizations, regulatory decisions, laws, administrative measures, scientific statements, and philosophical, moral, and philanthropic propositions. In short, the apparatus is made up of that which is said and unsaid. The apparatus is the network that can be established among these elements (p. 244). The relationship among these elements indicates the existence of change of positions and modification of functions. An apparatus always responds to an urgent need, made clear by its strategic or dominant function.

3. Passos, Kastrup, and Escóssia ( 2009 ) understand politics in a broad sense as the form of human activity that, linked to power, relates to subjects, articulating them according to rules or norms that are not necessarily only legal in nature. Politics is also done through local arrangements, that is, micro relations, indicating this micropolitical dimension of power relations (Foucault, 1979 ).

4. See conception of non human actor, present in Latour’s ANT.

5. It is important to emphasize that the perspective of “doing and understanding” is also described as a theory in the work of Jean Piaget ( 1978a ). It is, however, necessary to establish differences and similarities.

6. ANT emerged from an interdisciplinary perspective, with contributions from different areas, and is still defining itself as a methodological tool.

7. The author of this article.

8. In the sense put forth by Dewey ( 1938 ) and taken up by Latour ( 2016 ).

9. In gamer lingo, achievements are goals that a subject can complete during the game. They can be explicit or secret, that is, that the subject discovers during the process of play.

Printed from Oxford Research Encyclopedias, Education. Under the terms of the licence agreement, an individual user may print out a single article for personal use (for details see Privacy Policy and Legal Notice).

date: 03 June 2024

• Cookie Policy
• Privacy Policy
• Legal Notice
• Accessibility
• [66.249.64.20|185.126.86.119]
• 185.126.86.119

Character limit 500 /500

Graduate Teacher Education Students Use and Evaluate ChatGPT as an Essay-Writing Tool

• Anthony G Picciano City University of New York, Hunter College

Artificial intelligence (AI) has been evolving since the mid-20 th Century when luminaries such as Alan Turing, Herbert Simon and Marvin Minsky began developing rudimentary AI applications. For decades, AI programs remained pretty much in the realm of computer science and experimental game playing.  This changed radically in the 2020s when commercial vendors such as OpenAI  and Google developed generative AI programs (ChatGPT) and (Bard) using large, language modelling (LLM).  As a result, generative AI  is now being considered for use in all walks of life including education.

     In Spring 2023, when ChatGPT burst into the public psyche, twenty-five education students in the author’s graduate seminar were invited to participate in a qualitative study using ChatGPT as a tool for completing an essay assignment.  Fifteen (N=15) accepted the offer. The purpose in doing this was to give students in this seminar the opportunity to use ChatGPT in a supportive environment and to collect qualitative data from them on their experiences using ChatGPT.

     All of these students have master’s degrees in education and experience as teachers in New York City schools. Their training and experience give them keen insights into pedagogical practice making them ideally suited to evaluate ChatGPT as an instructional tool. This article reports on the results of this study.

Keywords:  Artificial Intelligence, AI, ChatGPT,  Graduate Teacher Education, Qualitative Research

Author Biography

Anthony g picciano, city university of new york, hunter college.

Copyright (c) 2024 Anthony G Picciano

This work is licensed under a Creative Commons Attribution 4.0 International License .

As a condition of publication, the author agrees to apply the Creative Commons – Attribution International 4.0 (CC-BY) License to OLJ articles. See: https://creativecommons.org/licenses/by/4.0/ .

This licence allows anyone to reproduce OLJ articles at no cost and without further permission as long as they attribute the author and the journal. This permission includes printing, sharing and other forms of distribution.

Author(s) hold copyright in their work, and retain publishing rights without restrictions

The DOAJ Seal is awarded to journals that demonstrate best practice in open access publishing

OLC Membership

OLC Research Center

Information

• For Readers
• For Authors
• For Librarians

Chemistry Education Research and Practice

Reconstructing perspectives: investigating how molecular geometry cards (mgcards) and molecular model building (mmb) disrupt students' alternative notions of molecular structure – a qualitative study.

* Corresponding authors

a Chemistry Education Study Program, Faculty of Teacher Training and Education, Tanjungpura University, Kalimantan Barat, Indonesia E-mail: [email protected]

b School of Chemistry, University of Birmingham, UK

c Department of Genetics and Genome Biology, University of Leicester, UK

The range of abstract concepts encountered when learning chemistry and the inability of students to make connections between the macroscopic, sub-microscopic, and symbolic representations, used in chemistry teaching, are believed to be the main reasons for students’ difficulty when learning chemistry. Prediction and determination of molecular geometry using the theory of valence shell electron pair repulsion (VSEPR) is a sample of the abstract concept that is hard to understand by students who learn chemistry. Students may comprehend these ideas better if the learning process is supplemented with cutting-edge, interactive learning aids. To address the conceptual difficulties that students encounter when learning how to predict the shapes of molecules, a card game (MGCards) has been developed which is supported by simple molecular model building (MMB). The card game allows students to work through the steps required to predict the shape of a molecule in an engaging format. The student learning process is supported by feedback at all stages (if students make a mistake, they receive hints that will help them in the next step of the game). Action research with qualitative methods has been used to design, develop, and evaluate the MGCards. The MGCards and MMB were piloted at the University of Leicester with year one Natural Sciences students and modified based on the feedback received. Both MGCards and MMB were then used as part of the first-year chemistry education programme at Tanjungpura University in Indonesia. The findings of students’ answer analysis (pre- and post-test) in both cycles showed that students had a better understanding after learning with MGCards and MMB. The positive feedback for MGCards and MMB confirmed that these resources were effective in delivering an engaging learning experience. The results suggest that MGCards and MMB play a significant role in enhancing students’ understanding while also keeping them engaged.

Article information

Download citation, permissions.

Reconstructing perspectives: investigating how molecular geometry cards (MGCards) and molecular model building (MMB) disrupt students' alternative notions of molecular structure – a qualitative study

Erlina, D. P. Williams, C. Cane, Hairida, M. Ulfah and A. F. Wafiq, Chem. Educ. Res. Pract. , 2024, Advance Article , DOI: 10.1039/D3RP00038A

To request permission to reproduce material from this article, please go to the Copyright Clearance Center request page .

If you are an author contributing to an RSC publication, you do not need to request permission provided correct acknowledgement is given.

If you are the author of this article, you do not need to request permission to reproduce figures and diagrams provided correct acknowledgement is given. If you want to reproduce the whole article in a third-party publication (excluding your thesis/dissertation for which permission is not required) please go to the Copyright Clearance Center request page .

Read more about how to correctly acknowledge RSC content .

Social activity

Search articles by author.

This article has not yet been cited.

Advertisements

• Open access
• Published: 29 May 2024

The Retrospective Stressor Analysis (RSA): a novel qualitative tool for identifying causes of burnout and mitigation strategies during residency

• Kristin L. Chrouser 1 ,
• Laura Zebib 1 ,
• Blake F. Webb 2 ,
• Tandi Bagian 2 &
• Timothy Arnold 3 , 4  

BMC Medical Education volume  24 , Article number:  591 ( 2024 ) Cite this article

102 Accesses

3 Altmetric

Metrics details

Resident physicians are at an increased risk of burnout due to their high-pressure work environments and busy schedules which can lead to poor mental health outcomes and decreased performance quality. Given variability among training programs and institutions across the United States, stressors likely differ, and interventions must be tailored to the local context, but few tools exist to assist in this process.

A tool commonly used in adverse event analysis was adapted into a “retrospective stressor analysis” (RSA) for burnout prevention. The RSA was tested in a group of chief residents studying quality improvement and patient safety in veteran’s hospitals across the United States. The RSA prompted them to identify stressors experienced during their residencies across four domains (clinical practice, career development, personal life, and personal health), perceived causes of the stressors, and potential mitigation strategies.

Fifty-eight chief residents completed the RSA. Within the clinical domain, they describe the stress of striving for efficiency and clinical skills acquisition, all while struggling to provide quality care in high pressure environments. In the career domain, identifying mentors and opportunities for research engagement was stressful. Within their personal lives, a lack of time-constrained their ability to maintain hobbies, relationships, and attend meaningful social events while also reducing their engagement in healthy behaviors such as exercise, optimal nutrition, and attending medical appointments. Within each of these domains, they identified and described stress mitigation strategies at the individual, departmental, and national levels.

The RSA is a novel tool that can identify national trends in burnout drivers while simultaneously providing tailored prevention strategies for residents and their training sites.

Peer Review reports

Resident physicians are vital to the US healthcare system, but burnout rates among residents range from 17–94%, with variation by specialty and program [ 1 , 2 ]. Burnout is characterized by emotional exhaustion, depersonalization, and a reduced sense of personal accomplishment.[ 3 ] External factors like demanding work environments, high patient care standards, long hours, poor work-life balance, lack of mental health support, and mistreatment in the workplace, combined with internal factors such as perfectionist personality, neuroticism, and previous mental health diagnosis, heighten the risk of burnout [ 1 , 4 , 5 , 6 ].

Burnout can be detrimental to resident physicians’ personal and professional well-being, leading to decreased job satisfaction, increasing attrition, depression, substance use, and suicide [ 7 , 8 , 9 ]. Burnout can also have clinical implications, negatively affecting patient access and quality of care. Burned-out physicians are more likely to make medical errors, exhibit increased implicit and explicit biases, and become less productive [ 9 , 10 , 11 , 12 ]. Meta-analyses find that burnout among healthcare providers is associated with reduced patient satisfaction, quality indicators, and perceived patient safety [ 13 , 14 ]. Thus, patient safety, quality of care, and physician wellbeing are inextricably linked. Consequently, the Accreditation Council for Graduate Medical Education (ACGME) has enhanced requirements for residency program accreditation, emphasizing monitoring and maintaining well-being during residency training [ 15 ].

A recent review of interventions to reduce resident burnout notes the current literature is of marginal quality and results are inconclusive [ 16 ]. While self-care initiatives have been shown to alleviate burnout in some small samples, such interventions emphasize modification of internal factors. This shifts the responsibility onto residents and does not address the systemic and environmental factors that promote burnout. Studies suggest that interventions addressing external factors such as work-hour limitations, structured mentorship programs, and access to mental health programs are associated with decreased burnout among residents [ 16 , 17 , 18 , 19 ]. Bakker’s Job Demands-Job Resources Conceptual Model illustrates that burnout is a consequence of chronic work-related stress, when job demands exceed job resources and individuals can no longer cope [ 20 ]. Therefore, effective burnout prevention requires reduction of job demands and/or augmentation of job resources, and should address both internal and external risk factors unique to each specialty and residency program.

Given the ACGME’s interest in burnout prevention, many residency programs use standardized surveys to monitor resident burnout rates. Tailored information can be gleaned from measures such as the Mini ReZ, which assesses the impact of several common residency stressors (e.g., electronic health record, interruptions, sleep impairment) [ 21 , 22 ]. However, causes of burnout will likely change rapidly over time as institutions adopt new technologies, face novel challenges (like COVID-19), or respond to regulatory changes.[ 23 ] For example, the advent of the electronic health record (EHR) rapidly changed documentation processes and created new stressors that increased physician burnout [ 24 ]. In the context of a constantly changing environment, surveys that identify sources of burnout based on the existing literature may fail to promptly capture ever-shifting stressors. Surveys are confined to capturing trends in explicitly asked topics. This limits our ability to capture emerging or unknown stressors. Furthermore, there is a paucity of data on residents’ perceptions of the causes of burnout. Therefore, we need tools that can elucidate burnout drivers and potential prevention strategies in rapidly changing environments from the perspective of impacted individuals. This will aid resource allocation for high-yield improvements.

The adaptation of an adverse event analysis tool can leverage methods that are already familiar to residents in order to generate an understanding of burnout drivers and potential interventions. With growing evidence of the negative impact of burnout on providers, trainees, and patient care, institutions need innovative tools to monitor for new causes of burnout in changing environments. This will allow them to rapidly shift burnout prevention strategies when appropriate. This study aims to 1) characterize recent residency graduates’ perceptions of the drivers of burnout, 2) identify potential interventions for mitigating resident burnout, and 3) assess the utility of the RSA (retrospective stressor analysis) as a novel tool to identify a wider breadth of sources of resident burnout than found in the current literature as well as generate practical strategies to mitigate these causes.

Root-cause analysis (RCA) is a methodology to identify underlying causes of an adverse event and has been used in healthcare to characterize and help prevent future adverse outcomes [ 25 ]. Residents are typically familiar with RCA methodology, including the “five whys” and fishbone diagrams from their patient safety and quality improvement training. The final deliverable of an RCA is a list of “action items” to address or eliminate these underlying causes and prevent similar future problems. Similar to an RCA, this Retrospective Stressor Analysis (RSA) was designed to identify potential underlying causes of stressors and list possible corrective actions/prevention strategies (see Appendix A). The RSA has dual utility to 1) be used by institutions to explore resident perceptions of causes as a cohort and implement resident-derived interventions and 2) be used by residents as an opportunity for self-reflection on their own individual perceived burnout causes and identify actions they can personally take to mitigate burnout and improve resilience.

In March 2022, 87 VA Chief Residents in Quality Improvement and Patient Safety (CRQS) across 67 Veterans Affairs Healthcare Centers were given a homework assignment on Building Resilience/Preventing Burnout. Participants were instructed to recall stressors experienced during their residencies across four domains (clinical practice, career development, personal life, and personal health) that they felt increased their risk of burnout. Then they listed perceived causes of these stressors and potential prevention and mitigation strategies. After aggregating the deidentified data, we coded stressors and mitigation strategies and identified themes within each of the four domains (clinical practice, career development, personal life, and personal health). Evaluating each entry’s content and context, one author (KC) developed codes using thematic analysis. After compiling the initial codebook, a second coder (TA) coded 20% of the entries in each domain. Co-analysis agreement was > 80%, and disagreements were resolved by discussion. Given the large dataset, codes for stressors were then ranked by frequency within each domain and the top 15 illustrated as word clouds. Conceptual themes were identified within each domain.

Preventive strategies for each domain were compiled and categorized by intervention level (personal, departmental, national) and themes were identified. Participants received a deidentified compilation of prevention strategies as a resource to share with their medical education community. The University of Michigan Institutional Review Board reviewed this study and determined it to be exempt and waived ethical approval and consent to participate. The data are available from the corresponding author, Dr. Kristin Chrouser, upon request.

In 2022, 58 chief residents (67%) completed the RSA assignment. All responses were deidentified, so demographic information is not available. Participants identified 1020 stressors (306 clinical, 262 career, 247 personal life, and 205 personal health) and 569 mitigation strategies (165 clinical, 136 career, 133 personal life, and 135 personal health). Qualitative analysis of stressors and mitigation strategies revealed several themes within the four domains.

Themes from clinical practice domain stressors

Participants describe the stresses related to their clinical work (Fig.  1 ), such as high patient volume, patient acuity, challenging patient interactions, poor outcomes, and systems issues, including EHR frustrations, documentation hassles, administrative burden, and lack of backup. They highlighted challenges regarding the management of clinical work, such as striving for efficiency, admitting a lack of knowledge/experience, and asking for help. They were stressed by their adjustments to gaining seniority over the course of training related to role transition, acquiring leadership and teaching skills, and delegation challenges. They describe challenges related to their role as learners, such as time to study, gaining clinical knowledge, and learning procedures. They also describe their emotional experience/response to the stresses of their clinical role, including experiencing imposter syndrome, worry, the weight of responsibilities, emotion management, coping with mistakes, and facing inadequacies.

Clinical Practice Domain: Stressors that increased burnout risk*

*Size of word correlates with frequency of theme

Themes from the career development domain stressors

Participants describe various aspects of career development they considered stressful—such as research, publishing, presentations, teaching, committees, and professional relationships (Fig.  2 ). Many find career planning and career choices difficult, including fellowship decisions. They also recognize challenges in finding and becoming good mentors. Balancing academic and personal priorities and time management were common struggles. They also describe the additional stress of learning to cope with bias, competitiveness, failure, burnout, and performance anxiety.

Career Domain: Stressors that increased burnout risk*

Themes from the personal life domain stressors

Participants describe challenges in maintaining their personal lives as residents (Fig.  3 ). Lack of time is a common complaint, leading to difficulty separating work and home lives while describing concerns with a lack of opportunity to unwind from the stressors of residency. This includes inadequate time to invest in social life and maintain relationships with family and friends, eventually leading to erosion of social support. Social isolation was exacerbated by geographic separation from family support, moving to a new city, and COVID-19 restrictions.

Personal Life Domain: Stressors that increased burnout risk*

Many felt that long work hours led to difficulty coping with home stressors such as caregiving for children, family, and pets. Schedule inflexibility during residency led to missing important social events and being unavailable to manage family emergencies. They also recognized the difficulty of home maintenance, finances, and chores due to a lack of time. Residents described their emotional experience/response to these stresses in their personal lives as leading to guilt and feeling overwhelmed.

Themes from the personal health domain stressors

Participants described various challenges in maintaining their personal health as residents (Fig.  4 ). They described that a lack of time led to an inability to maintain healthy habits such as exercise, nutritious meals, proper hydration, and adequate sleep. Accessing physical and mental healthcare for themselves was difficult due to their schedules and social pressure to prioritize work over healthcare needs. Similarly, due to the demands and expectations of residency, many found it difficult to take a day off when ill.

Personal Health Domain: Stressors that increased burnout risk*

The struggle to cope and manage stress was a common complaint. Many participants noted this was exacerbated by the physical and mental stress of pregnancy and parenthood. They also described various emotional experiences related to their health: fear of COVID-19, feeling out of control, anxiety, and guilt for taking a sick day.

Mitigation strategies at the personal level

Table 1 outlines potential interventions to reduce burnout at various levels. For mitigation strategies on a personal level, many emphasized the importance of maintaining productivity through intentional organization within all domains. For example, within the clinical and career domains, their self-identified need for ongoing clinical learning could be achieved through setting clear goals and creating consistent study schedules, and within the personal domain, by scheduling designated time for relaxation, vacation, hobbies, and quality time with family. Interestingly, this also included creating time to prioritize one’s own health and attend medical appointments. Residents stressed the importance of determining clear personal goals, priorities, and setting expectations both at work and with family members. Many described the need to outsource home tasks, including house cleaning, grocery shopping, and childcare. Also, they described behaviors to automate healthy choices such as meal prepping, not purchasing unhealthy snacks, tracking water intake, and organizing resident meals with healthy options to maintain personal wellness.

Many residents commented on the importance of creating a team-like atmosphere in the work environment, including learning what tasks to delegate and consistently coordinating debrief sessions after adverse patient outcomes. They also discussed the importance of leveraging specific relationships, such as nursing staff, specialists, hospital resources, and asking for help from senior residents or faculty. Many advised the importance of adapting one’s mindset, such as adopting a reflective mindfulness practice, being vulnerable with peers and mentors, and reframing success and failure. A common theme was the importance of peer-to-peer relationships in discussing potential hurdles such as imposter syndrome and creating a culture where open discussion was encouraged.

Mitigation strategies at the departmental level

At the departmental level, participants emphasized the importance of mentorship and coaching. While some encouraged the importance of individually reaching out to potential mentors early and the utility of building peer relationships, others described the role institutions can play in creating mentorship programming. They desired mentors who would discuss imposter syndrome and failure and guide mentees through career and personal decisions. Many felt a need for significant shifts in culture to encourage open communication, sharing failures, and enhancing feedback mechanisms.

Some advised significant changes to patient care responsibilities, such as reduced work hours, capping the number of patients, and reduced note writing. Others described a need for greater standardization of clinical expectations such as templates for best practices, patient handoffs, consults, checklists, and “guides” for workflows in different clinic settings. Many described the need for augmenting the curriculum to include robust mechanisms for research support and increased training during orientation on communication skills, efficiency in the workplace, teaching, navigating difficult cases, managing imposter syndrome, and coping with failure.

Mitigation strategies at the national level

The most cited mitigation strategy across all domains was reducing resident duty hours. While many emphasized the role of institutions in complying with duty hour restrictions, further adjustments to duty hours require top-down implementation by the ACGME. Residents stated that there was a need for greater flexibility to utilize wellness days or sick leave. Given the stressors associated with family planning, many felt the ACGME and/or institutions should provide information and resources for cryopreservation, parental leave, and lactation. Lastly, increasing resident income was suggested as an effective strategy to alleviate resident budgetary stressors, accurately reflect work hours, and compensate some for the added stress of challenging work schedules such as jeopardy call schedules and night-float.

Burnout’s impact on physician well-being and quality of care is well established, and current rates are concerning [ 1 , 5 ]. Therefore, we need tools for ongoing assessment of the underlying causes of resident burnout and identification of potential interventions within local work environments. In this study, we describe the successful use of a novel tool, the retrospective stressor analysis (RSA), informed by the familiar RCA process, to characterize residents’ perceptions of burnout causes and potential mitigation strategies. To our knowledge, the field currently lacks alternatives to survey-based tools that will identify new sources of burnout and provide individuals and institutions with intervention strategies.

Participants in our study highlight how lack of time impacts their well-being in all four domains. This is not surprising as previous studies have shown a significant difference in the burnout rates of residents based on adherence to work-hour restrictions [ 26 ]. Similar to our findings, Mian et al., identified several common stressors among trainees that lead to burnout, such as overwork/sleep deprivation, emotional drain of caring for sick patients, lack of time for personal life outside training, and residency coinciding with major life events such as parenthood [ 27 ]. Related, Linzer et al. found burnout correlated with work-related conditions such as value alignment, teamwork, work control, and time pressures [ 22 ]. The RSA identified similar themes among participants. In addition, the RSA also provided greater context across the four domains to elucidate previously unacknowledged sources of stress, such as career decision-making, acquisition of leadership roles, and coping with adverse patient outcomes. To our knowledge, these have not been previously identified as potential sources of burnout within the resident population. Moreover, participants provided highly specific stressors, such as “variability in clinical preferences among attendings”, and potential mitigation strategies that program directors might find useful when restructuring expectations or generating standardized workflows.

While the RSA may be a novel resource for understanding burnout, there were some challenges with using it in practice. Despite clear instructions to list multiple causes, some participants did not identify more than one potential cause of their stressors, even though this is a common step used in root cause analysis. Failure to identify a variety of causes can reduce the diversity of proposed interventions. Despite this potential limitation, our cohort of residents still generated a large range of interventions for burnout prevention and mitigation. However, if RSA is used in smaller resident samples in the future without encouraging participants to provide a range of causes, this might generate a reduced range of interventions, thus limiting impact. The “personal life” domain was listed prior to “personal health,” so often participants included many factors in the personal life domain that would have been more appropriate under personal health, which made analysis of frequency by domain more challenging.

Moreover, proposed preventive strategies overwhelmingly focused on personal actions, although some participants suggested departmental/institutional/national policy reforms. The Institute for Healthcare Improvement’s Action Hierarchy Tool is used to assist RCA teams in identifying interventions with the strongest effect for sustained and successful system improvement [ 28 ]. Stronger actions are those that do not rely on human memory, such as architectural changes, forced functioning, removing unnecessary steps, and tangible involvement of leadership; while weaker actions, such as trainings and new procedures, rely on humans to remember to perform an action. Many of the recommended interventions identified using the RSA would be classified as weaker actions, as they rely on residents to remember and make time to perform specific tasks such as exercise, mindfulness, studying, and delegating in the clinical space. Future iterations of the tool should encourage the development of stronger intervention actions.

Although these challenges with RSA had minimal effect on data and analysis, our group refined the RSA tool for future data collection. Refinements included adjusting domain order to reduce categorization errors, adding reminders of QI tools useful in collecting a broader range of potential causes (5 whys, fishbone diagram), and providing examples of systemic preventive strategies (e.g., programs, policies). The revised RSA is available in Appendix B. Although our sample size was more than adequate to reach saturation for a qualitative study, the RSA’s usability, generalizability, and utility of our findings may vary among residents. The Chief Residents in Quality Improvement and Patient Safety were already familiar with RCA tools and methods, but this might not be the case for all residents, and future iterations may be informed by piloting the tool in varied resident populations. Participants also provided this data via a homework assignment, and even though they were assured their responses were confidential, fear of being identified by course directors may have influenced their responses. Demographics of participants were not collected, precluding any analysis based on specialty or gender. This is a limitation as burnout causes and mitigation recommendations might differ based on demographic categories.

The RSA provided findings consistent with factors known to contribute to burnout in the literature while generating a broader range of stressors than previously reported. RSA utilization can allow residency programs to identify emerging burnout drivers as medicine changes rapidly and provides a wealth of intervention ideas appropriate to the local context. Engaging residents in developing implementation strategies can serve the dual purpose of reinforcing skills applicable to adverse event analysis techniques and helping prevent resident burnout. Qualitative data assessment from the RSA could also be used by national associations to identify novel stressors and then generate new quantitative survey questions more appropriate for measurement within a larger population.

We adapted a familiar patient safety tool, root cause analysis (RCA), to create the retrospective stressor analysis (RSA) for burnout prevention. This novel tool allowed recent residency graduates to identify stressors they believe increased their risk of burnout and generate practical preventive strategies at personal, institutional, and national levels. Common themes highlighted the difficulty of inflexible schedules and lack of time invested in protective factors such as social support, mentorship, and healthy habits. The RSA is a novel tool that can identify national trends in the drivers of burnout while providing tailored prevention strategies for individuals, training sites, and the ACGME to consider for future implementation.

Disclaimers

The opinions expressed in this presentation are the authors’ own and do not necessarily reflect the view of the Department of Veterans Affairs or the United States government.

Availability of data and materials

The data are available from the corresponding author, Dr. Kristin Chrouser, upon request.

Abbreviations

Retrospective Stressor Analysis

American Council for Graduate Medical Education

Kratzke IM, Woods LC, Adapa K, et al. The Sociotechnical Factors Associated With Burnout in Residents in Surgical Specialties: A Qualitative Systematic Review. J Surg Educ. 2022May 1;79(3):614–23.

Article   Google Scholar  

Rodrigues H, Cobucci R, Oliveira A, et al. Burnout syndrome among medical residents: A systematic review and meta-analysis. PLoS One. 2018;13(11):e0206840.

Nene Y, Tadi P. Resident Burnout. StatPearls Publishing. 2022. Available from: https://www.ncbi.nlm.nih.gov/books/NBK553176/ . Cited 2023 Feb 24.

Zhou AY, Panagioti M, Esmail A, et al. Factors associated with burnout and stress in trainee physicians: a systematic review and meta-analysis. JAMA Netw Open. 2020;3(8):e2013761.

Ferguson C, Low G, Shiau G. Resident physician burnout: insights from a Canadian multispecialty survey. Postgrad Med J. 2020Jun 1;96(1136):331–8.

McManus I, Keeling A, Paice E. Stress, burnout and doctors’ attitudes to work are determined by personality and learning style: A twelve year longitudinal study of UK medical graduates. BMC Med. 2004Dec;2(1):29.

Chaukos D, Chad-Friedman E, Mehta DH, et al. Risk and resilience factors associated with resident burnout. Acad Psychiatry. 2017;41(2):189–94.

van der Heijden F, Dillingh G, Bakker A, et al. Suicidal thoughts among medical residents with burnout. Arch Suicide Res. 2008;12(4):344–6.

West CP, Dyrbye LN, Shanafelt TD. Physician burnout: contributors, consequences and solutions. J Intern Med. 2018Jun;283(6):516–29.

Dewa CS, Loong D, Bonato S, Trojanowski L, et al. The relationship between resident burnout and safety-related and acceptability-related quality of healthcare: a systematic literature review. BMC Med Educ. 2017Nov 9;17(1):195.

Scheepers RA, Boerebach BCM, Arah OA, et al. A systematic review of the impact of physicians’ occupational well-being on the quality of patient care. Int J Behav Med. 2015;22(6):683–98.

Dyrbye L, Herrin J, West CP, Wittlin NM, et al. Association of racial bias with burnout among resident physicians. JAMA Netw Open. 2019J 3;2(7):e197457.

Salyers MP, Bonfils KA, Luther L, et al. The Relationship Between Professional Burnout and Quality and Safety in Healthcare: A Meta-Analysis. J Gen Intern Med. 2017Apr 1;32(4):475–82.

Tawfik DS, Scheid A, Profit J, Shanafelt T, et al. Evidence relating health care provider burnout and quality of care. Ann Intern Med. 2019;171(8):555–67.

American Medical Association. ACGME seeks to transform residency to foster wellness. 2015. Available from: https://www.ama-assn.org/medical-residents/medical-resident-wellness/acgme-seeks-transform-residency-foster-wellness . Cited 2023 Feb 24.

Busireddy KR, Miller JA, Ellison K, et al. Efficacy of Interventions to Reduce Resident Physician Burnout: A Systematic Review. J Grad Med Educ. 2017Jun 1;9(3):294–301.

Mullins CH, Gleason F, Wood T, et al. Do internal or external characteristics more reliably predict burnout in resident physicians: a multi-institutional study. J Surg Educ. 2020;77(6):e86-93.

Leung VWY, Konci X, Meterissian S. Is there a role for formal mentorship programs in reducing burnout in surgical residency?: a literature review. Available from: https://www.ijsed.com/article/25104.pdf . Cited 2023 Feb 25.

Clough BA, March S, Chan RJ, et al. Psychosocial interventions for managing occupational stress and burnout among medical doctors: a systematic review. Syst Rev. 2017Jul 17;6(1):144.

Bakker AB, Demerouti E. The job demands-resources model: state of the art. J Manag Psychol. 2007;22(3):309–28.

Maslach C, Jackson SE, Leiter MP. Maslach Burnout Inventory: Third edition. In: Zalaquett CP, editor. Evaluating stress: A book of resources. Lanham, MD, US: Scarecrow Education, xvii; 1997. p. 191–218. Available from: https://psycnet.apa.org/fulltext/1997-09146-011.pdf .

Linzer M, Shah P, Nankivil N, et al. The Mini Z Resident (Mini ReZ): Psychometric Assessment of a Brief Burnout Reduction Measure. J Gen Intern Med. 2023Feb;38(2):545–8.

Cravero AL, Kim NJ, Feld LD, et al. Impact of exposure to patients with COVID-19 on residents and fellows: an international survey of 1420 trainees. Postgrad Med J. 2021;97(1153):706–15.

Yan Q, Jiang Z, Harbin Z, et al. Exploring the relationship between electronic health records and provider burnout: A systematic review. J Am Med Inform Assoc. 2021Apr 23;28(5):1009–21.

Williams PM. Techniques for root cause analysis. Proc. 2001;14(2):154–7.

Google Scholar  

Marchalik D, Brems J, Rodriguez A, et al. The Impact of Institutional Factors on Physician Burnout: A National Study of Urology Trainees. Urology. 2019Sep;1(131):27–35.

Mian A, Dahye K, Duane C, et al. Medical student and resident burnout: a review of causes, effects, and prevention. J Fam Med Dis Prev. 2018;31:4.

RCA2: Improving root cause analyses and actions to prevent harm. National Patient Safety Foundation. Available from: https://www.ashp.org/-/media/assets/policy-guidelines/docs/endorsed-documents/endorsed-documents-improving-root-cause-analyses-actions-prevent-harm.ashx . Cited 2023 Jun 19.

Download references

Acknowledgements

Maria Lennox, MS assisted with collating the raw data.

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Author information

Authors and affiliations.

Department of Urology, University of Michigan, Ann Arbor, MI, USA

Kristin L. Chrouser & Laura Zebib

VHA National Center for Patient Safety, Ann Arbor, MI, USA

Blake F. Webb & Tandi Bagian

Human Factors Engineering Division, Clinical Informatics and Data Management, Office of Health Informatics, Veterans Health Administration, Washington, DC, USA

Timothy Arnold

College of Pharmacy, University of Michigan, Ann Arbor, MI, USA

You can also search for this author in PubMed   Google Scholar

Contributions

KC, TB, and TA conceptualized, developed, and designed the research study and tools. KC, TA, LZ, and BW analyzed and organized data. KC and LZ wrote the first draft. All authors edited manuscript drafts, read, and approved the final manuscript.

Corresponding author

Correspondence to Kristin L. Chrouser .

Ethics declarations

Ethical approval and consent to participate.

The University of Michigan Institutional Review Board reviewed this study and determined it to be exempt from ethical approval and consent to participate.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Chrouser, K.L., Zebib, L., Webb, B.F. et al. The Retrospective Stressor Analysis (RSA): a novel qualitative tool for identifying causes of burnout and mitigation strategies during residency. BMC Med Educ 24 , 591 (2024). https://doi.org/10.1186/s12909-024-05571-3

Download citation

Received : 09 February 2024

Accepted : 17 May 2024

Published : 29 May 2024

DOI : https://doi.org/10.1186/s12909-024-05571-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Quality improvement

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Open access
• Published: 03 June 2024

The use of evidence to guide decision-making during the COVID-19 pandemic: divergent perspectives from a qualitative case study in British Columbia, Canada

• Laura Jane Brubacher   ORCID: orcid.org/0000-0003-2806-9539 1 , 2 ,
• Chris Y. Lovato 1 ,
• Veena Sriram 1 , 3 ,
• Michael Cheng 1 &
• Peter Berman 1  

Health Research Policy and Systems volume  22 , Article number:  66 ( 2024 ) Cite this article

Metrics details

The challenges of evidence-informed decision-making in a public health emergency have never been so notable as during the COVID-19 pandemic. Questions about the decision-making process, including what forms of evidence were used, and how evidence informed—or did not inform—policy have been debated.

We examined decision-makers' observations on evidence-use in early COVID-19 policy-making in British Columbia (BC), Canada through a qualitative case study. From July 2021- January 2022, we conducted 18 semi-structured key informant interviews with BC elected officials, provincial and regional-level health officials, and civil society actors involved in the public health response. The questions focused on: (1) the use of evidence in policy-making; (2) the interface between researchers and policy-makers; and (3) key challenges perceived by respondents as barriers to applying evidence to COVID-19 policy decisions. Data were analyzed thematically, using a constant comparative method. Framework analysis was also employed to generate analytic insights across stakeholder perspectives.

Overall, while many actors’ impressions were that BC's early COVID-19 policy response was evidence-informed, an overarching theme was a lack of clarity and uncertainty as to what evidence was used and how it flowed into decision-making processes. Perspectives diverged on the relationship between 'government' and public health expertise, and whether or not public health actors had an independent voice in articulating evidence to inform pandemic governance. Respondents perceived a lack of coordination and continuity across data sources, and a lack of explicit guidelines on evidence-use in the decision-making process, which resulted in a sense of fragmentation. The tension between the processes involved in research and the need for rapid decision-making was perceived as a barrier to using evidence to inform policy.

Conclusions

Areas to be considered in planning for future emergencies include: information flow between policy-makers and researchers, coordination of data collection and use, and transparency as to how decisions are made—all of which reflect a need to improve communication. Based on our findings, clear mechanisms and processes for channeling varied forms of evidence into decision-making need to be identified, and doing so will strengthen preparedness for future public health crises.

Peer Review reports

The challenges of evidence-informed decision-making Footnote 1 in a public health emergency have never been so salient as during the COVID-19 pandemic, given its unprecedented scale, rapidly evolving virology, and multitude of global information systems to gather, synthesize, and disseminate evidence on the SARS-CoV-2 virus and associated public health and social measures [ 1 , 2 , 3 ]. Early in the COVID-19 pandemic, rapid decision-making became central for governments globally as they grappled with crucial decisions for which there was limited evidence. Critical questions exist, in looking retrospectively at these decision-making processes and with an eye to strengthening future preparedness: Were decisions informed by 'evidence'? What forms of evidence were used, and how, by decision-makers? [ 4 , 5 , 6 ].

Scientific evidence, including primary research, epidemiologic research, and knowledge synthesis, is one among multiple competing influences that inform decision-making processes in an outbreak such as COVID-19 [ 7 ]. Indeed, the use of multiple forms of evidence has been particularly notable as it applies to COVID-19 policy-making. Emerging research has also documented the important influence of ‘non-scientific’ evidence such as specialized expertise and experience, contextual information, and level of available resources [ 8 , 9 , 10 ]. The COVID-19 pandemic has underscored the politics of evidence-use in policy-making [ 11 ]; what evidence is used and how can be unclear, and shaped by political bias [ 4 , 5 ]. Moreover, while many governments have established scientific advisory boards, the perspectives of these advisors were reportedly largely absent from COVID-19 policy processes [ 6 ]. How evidence and public health policy interface—and intersect—is a complex question, particularly in the dynamic context of a public health emergency.

Within Canada, a hallmark of the public health system and endorsed by government is evidence-informed decision-making [ 12 ]. In British Columbia (BC), Canada, during the early phases of COVID-19 (March—June 2020), provincial public health communication focused primarily on voluntary compliance with recommended public health and social measures, and on supporting those most affected by the pandemic. Later, the response shifted from voluntary compliance to mandatory enforceable government orders [ 13 ]. Like many other jurisdictions, the government’s public messaging in BC asserted that the province took an approach to managing the COVID-19 pandemic and developing related policy that was based on scientific evidence, specifically. For example, in March 2021, in announcing changes to vaccination plans, Dr. Bonnie Henry, the Provincial Health Officer, stated, " This is science in action " [ 14 ]. As a public health expert with scientific voice, the Provincial Health Officer has been empowered to speak on behalf of the BC government across the COVID-19 pandemic progression. While this suggests BC is a jurisdiction which has institutionalized scientifically-informed decision-making as a core tenet of effective public health crisis response, it remains unclear as to whether BC’s COVID-19 response could, in fact, be considered evidence-informed—particularly from the perspectives of those involved in pandemic decision-making and action. Moreover, if evidence-informed, what types of evidence were utilized and through what mechanisms, how did this evidence shape decision-making, and what challenges existed in moving evidence to policy and praxis in BC’s COVID-19 response?

The objectives of this study were: (1) to explore and characterize the perspectives of BC actors involved in the COVID-19 response with respect to evidence-use in COVID-19 decision-making; and (2) to identify opportunities for and barriers to evidence-informed decision-making in BC’s COVID-19 response, and more broadly. This inquiry may contribute to identifying opportunities for further strengthening the synthesis and application of evidence (considered broadly) to public health policy and decision-making, particularly in the context of future public health emergencies, both in British Columbia and other jurisdictions.

Study context

This qualitative study was conducted in the province of British Columbia (BC), Canada, a jurisdiction with a population of approximately five million people [ 15 ]. Within BC’s health sector, key actors involved in the policy response to COVID-19 included: elected officials, the BC Government’s Ministry of Health (MOH), the Provincial Health Services Authority (PHSA), Footnote 2 the Office of the Provincial Health Officer (PHO), Footnote 3 the BC Centre for Disease Control (BCCDC), Footnote 4 and Medical Health Officers (MHOs) and Chief MHOs at regional and local levels.

Health research infrastructure within the province includes Michael Smith Health Research BC [ 16 ] and multiple post-secondary research and education institutions (e.g., The University of British Columbia). Unlike other provincial (e.g., Ontario) and international (e.g., UK) jurisdictions, BC did not establish an independent, formal scientific advisory panel or separate organizational structure for public health intelligence in COVID-19. That said, a Strategic Research Advisory Council was established, reporting to the MOH and PHO, to identify COVID-19 research gaps and commission needed research for use within the COVID-19 response [ 17 ].

This research was part of a multidisciplinary UBC case study investigating the upstream determinants of the COVID-19 response in British Columbia, particularly related to institutions, politics, and organizations and how these interfaced with, and affected, pandemic governance [ 18 ]. Ethics approval for this study was provided by the University of British Columbia (UBC)’s Institutional Research Ethics Board (Certificate #: H20-02136).

Data collection

From July 2021 to January 2022, 18 semi-structured key informant interviews were conducted with BC elected officials, provincial and regional-level health officials, and civil society actors (e.g., within non-profit research organizations, unions) (Table 1 ). Initially, respondents were purposively sampled, based on their involvement in the COVID-19 response and their positioning within the health system organizational structure. Snowball sampling was used to identify additional respondents, with the intent of representing a range of organizational roles and actor perspectives. Participants were recruited via email invitation and provided written informed consent to participate.

Interviews were conducted virtually using Zoom® videoconferencing, with the exception of one hybrid in-person/Zoom® interview. Each interview was approximately one hour in duration. One to two research team members led each interview. The full interview protocol focused on actors’ descriptions of decision-making processes across the COVID-19 pandemic progression, from January 2020 to the date of the interviews, and they were asked to identify key decision points (e.g., emergency declaration, business closures) [see Additional File 1 for the full semi-structured interview guide]. For this study, we used a subset of interview questions focused on evidence-use in the decision-making process, and the organizational structures or actors involved, in BC's early COVID-19 pandemic response (March–August 2020). Questions were adapted to be relevant to a respondent’s expertise and particular involvement in the response. ‘Evidence’ was left undefined and considered broadly by the research team (i.e., both ‘scientific’/research-based and ‘non-scientific’ inputs) within interview questions, and therefore at the discretion of the participant as to what inputs they perceived and described as ‘evidence’ that informed or did not inform pandemic decision-making. Interviews were audio-recorded over Zoom® with permission and transcribed using NVivo Release 1.5© software. Each transcript was then manually verified for accuracy by 1–2 members of the research team.

Data analysis

An inductive thematic analysis was conducted, using a constant comparative method, to explore points of divergence and convergence across interviews and stakeholder perspectives [ 19 ]. Transcripts were inductively coded in NVivo Release 1.5© software, which was used to further organize and consolidate codes, generate a parsimonious codebook to fit the data, and retrieve interview excerpts [ 20 ]. Framework analysis was also employed as an additional method for generating analytic insights across stakeholder perspectives and contributed to refining the overall coding [ 21 ]. Triangulation across respondents and analytic methods, as well as team collaboration in reviewing and refining the codebook, contributed to validity of the analysis [ 22 ].

How did evidence inform early COVID-19 policy-making in BC?

Decision-makers described their perceptions on the use of evidence in policy-making; the interface between researchers and policy-makers; and specific barriers to evidence-use in policy-making within BC’s COVID-19 response. In discussing the use of evidence, respondents focused on ‘scientific’ evidence; however, they noted a lack of clarity as to how and what evidence flowed into decision-making. They also acknowledged that ‘scientific’ evidence was one of multiple factors influencing decisions. The themes described below reflect the narrative underlying their perspectives.

Perceptions of evidence-use

Multiple provincial actors generally expressed confidence or had an overall impression that decisions were evidence-based (IDI5,9), stating definitively that, "I don’t think there was a decision we made that wasn’t evidence-informed" (IDI9) and that "the science became a driver of decisions that were made" (IDI5). However, at the regional health authority level, one actor voiced skepticism that policy decisions were consistently informed by scientific evidence specifically, stating, "a lot of decisions [the PHO] made were in contrast to science and then shifted to be by the science" ( IDI6). The evolving nature of the available evidence and scientific understanding of the virus throughout the pandemic was acknowledged. For instance, one actor stated that, "I’ll say the response has been driven by the science; the science has been changing…from what I’ve seen, [it] has been a very science-based response" (IDI3).

Some actors narrowed in on certain policy decisions they believed were or were not evidence-informed. Policy decisions in 2020 that actors believed were directly informed by scientific data included the early decision to restrict informal, household gatherings; to keep schools open for in-person learning; to implement a business safety plan requirement across the province; and to delay the second vaccine dose for maximum efficacy. One provincial public health actor noted that an early 2020 decision made, within local jurisdictions, to close playgrounds was not based on scientific evidence. Further, the decision prompted public health decision-makers to centralize some decision-making to the provincial level, to address decisions being made 'on the ground' that were not based on scientific evidence (IDI16). Similarly, they added that the policy decision to require masking in schools was not based on scientific evidence; rather, "it's policy informed by the noise of your community." As parents and other groups within the community pushed for masking, this was "a policy decision to help schools stay open."

Early in the pandemic response, case data in local jurisdictions were reportedly used for monitoring and planning. These "numerator data" (IDI1), for instance case or hospitalization counts, were identified as being the primary mode of evidence used to inform decisions related to the implementation or easing of public health and social measures. The ability to generate epidemiological count data early in the pandemic due to efficient scaling up of PCR testing for COVID-19 was noted as a key advantage (IDI16). As the pandemic evolved in 2020, however, perspectives diverged in relation to the type of data that decision-makers relied on. For example, it was noted that BCCDC administered an online, voluntary survey to monitor unintended consequences of public health and social measures and inform targeted interventions. Opinions varied on whether this evidence was successfully applied in decision-making. One respondent emphasized this lack of application of evidence and perceived that public health orders were not informed by the level and type of evidence available, beyond case counts: "[In] a communicable disease crisis like a pandemic, the collateral impact slash damage is important and if you're going to be a public health institute, you actually have to bring those to the front, not just count cases" (IDI1).

There also existed some uncertainty and a perceived lack of transparency or clarity as to how or whether data analytic ‘entities’, such as BCCDC or research institutions, fed directly into decision-making. As a research actor shared, "I’m not sure that I know quite what all those channels really look like…I’m sure that there’s a lot of improvement that could be driven in terms of how we bring strong evidence to actual policy and practice" (IDI14). Another actor explicitly named the way information flowed into decision-making in the province as "organic" (IDI7). They also noted the lack of a formal, independent science advisory panel for BC’s COVID-19 response, which existed in other provincial and international jurisdictions. Relatedly, one regional health authority actor perceived that the committee that was convened to advise the province on research, and established for the purpose of applying research to the COVID-19 response, "should have focused more on knowledge translation, but too much time was spent commissioning research and asking what kinds of questions we needed to ask rather than looking at what was happening in other jurisdictions" (IDI6). Overall, multiple actors noted a lack of clarity around application of evidence and who is responsible for ensuring evidence is applied. As a BCCDC actor expressed, in relation to how to prevent transmission of COVID-19:

We probably knew most of the things that we needed to know about May of last year [2020]. So, to me, it’s not even what evidence you need to know about, but who’s responsible for making sure that you actually apply the evidence to the intervention? Because so many of our interventions have been driven by peer pressure and public expectation rather than what we know to be the case [scientifically] (IDI1).

Some described the significance of predictive disease modelling to understand the COVID-19 trajectory and inform decisions, as well as to demonstrate to the public the effectiveness of particular measures, which "help[ed] sustain our response" (IDI2). Others, however, perceived that "mathematical models were vastly overused [and] overvalued in decision-making around this pandemic" (IDI1) and that modellers stepped outside their realm of expertise in providing models and policy recommendations through the public media.

Overall, while many actors’ impressions were that the response was evidence-informed, an overarching theme was a lack of clarity and uncertainty with respect to how evidence actually flowed into decision-making processes, as well as what specific evidence was used and how. Participants noted various mechanisms created or already in place prior to COVID-19 that fed data into, and facilitated, decision-making. There was an acknowledgement that multiple forms of evidence—including scientific data, data on public perceptions, as well as public pressure—appeared to have influenced decision-making.

Interface between researchers and policy-makers

There was a general sense that the Ministry supported the use of scientific and research-based evidence specifically. Some actors identified particular Ministry personnel as being especially amenable to research and focused on data to inform decisions and implementation. More broadly, the government-research interface was characterized by one actor as an amicable one, a "research-friendly government", and that the Ministry of Health (MOH), specifically, has a research strategy whereby, "it’s literally within their bureaucracy to become a more evidence-informed organization" (IDI11). The MOH was noted to have funded a research network intended to channel evidence into health policy and practice, and which reported to the research side of the MOH.

Other actors perceived relatively limited engagement with the broader scientific community. Some perceived an overreliance on 'in-house expertise' or a "we can do that [ourselves] mentality" within government that precluded academic researchers’ involvement, as well as a sense of "not really always wanting to engage with academics to answer policy questions because they don’t necessarily see the value that comes" (IDI14). With respect to the role of research, an actor stated:

There needs to be a provincial dialogue around what evidence is and how it gets situated, because there’s been some tension around evidence being produced and not used or at least not used in the way that researchers think that it should be (IDI11).

Those involved in data analytics within the MOH acknowledged a challenge in making epidemiological data available to academic researchers, because "at the time, you’re just trying to get decisions made" (IDI7). Relatedly, a research actor described the rapid instigation of COVID-19 research and pivoting of academic research programs to respond to the pandemic, but perceived a slow uptake of these research efforts from the MOH and PHSA for decision-making and action. Nevertheless, they too acknowledged the challenge of using research evidence, specifically, in an evolving and dynamic pandemic:

I think we’ve got to be realistic about what research in a pandemic situation can realistically contribute within very short timelines. I mean, some of these decisions have to be made very quickly...they were intuitive decisions, I think some of them, rather than necessarily evidence-based decisions (IDI14).

Relatedly, perspectives diverged on the relationship between 'government' and public health expertise, and whether or not public health actors had an independent voice in articulating evidence to inform governance during the pandemic. Largely from Ministry stakeholders, and those within the PHSA, the impressions were that Ministry actors were relying on public health advice and scientific expertise. As one actor articulated, "[the] government actually respected and acknowledged and supported public health expertise" (IDI9). Others emphasized a "trust of the people who understood the problem" (IDI3)—namely, those within public health—and perceived that public health experts were enabled "to take a lead role in the health system, over politics" (IDI12). This perspective was not as widely held by those in the public health sector, as one public health actor expressed, "politicians and bureaucrats waded into public health practice in a way that I don't think was appropriate" and that, "in the context of a pandemic, it’s actually relatively challenging to bring true expert advice because there’s too many right now. Suddenly, everybody’s a public health expert, but especially bureaucrats and politicians." They went on to share that the independence of public health to speak and act—and for politicians to accept independent public health advice—needs to be protected and institutionalized as "core to good governance" (IDI1). Relatedly, an elected official linked this to the absence of a formal, independent science table to advise government and stated that, "I think we should have one established permanently. I think we need to recognize that politicians aren't always the best at discerning scientific evidence and how that should play into decision-making" (IDI15).

These results highlight the divergent perspectives participants had as to the interface between research and policy-making and a lack of understanding regarding process and roles.

Challenges in applying evidence to policy decisions

Perspectives converged with respect to the existence of numerous challenges with and barriers to applying evidence to health policy and decision-making. These related to the quality and breadth of available data, both in terms of absence and abundance. For instance, as one public health actor noted in relation to health policy-making, "you never have enough information. You always have an information shortage, so you're trying to make the best decisions you can in the absence of usually really clear information" (IDI8). On the other hand, as evidence emerged en masse across jurisdictions in the pandemic, there were challenges with synthesizing evidence in a timely fashion for 'real-time' decision-making. A regional health authority actor highlighted this challenge early in the COVID-19 pandemic and perceived that there was not a provincial group bringing new synthesized information to decision-makers on a daily basis (IDI6). Other challenges related to the complexity of the political-public health interface with respect to data and scientific expertise, which "gets debated and needs to be digested by the political process. And then decisions are made" (IDI5). This actor further expressed that debate among experts needs to be balanced with efficient crisis response, that one has to "cut the debate short. For the sake of expediency, you need to react."

It was observed that, in BC’s COVID-19 response, data was gathered from multiple sources with differing data collection procedures, and sometimes with conflicting results—for instance, 'health system data' analyzed by the PHSA and 'public health data' analyzed by the BCCDC. This was observed to present challenges from a political perspective in discerning "who’s actually getting the 'right' answers" (IDI7). An added layer of complexity was reportedly rooted in how to communicate such evidence to the public and "public trust in the numbers" (IDI7), particularly as public understanding of what evidence is, how it is developed, and why it changes, can influence public perceptions of governance.

Finally, as one actor from within the research sector noted, organizationally and governance-wise, the system was "not very well set up to actually use research evidence…if we need to do better at using evidence in practice, we need to fix some of those things. And we actually know what a lot of those things are." For example , "there’s no science framework for how organizations work within that" and " governments shy away from setting science policy " (IDI11). This challenge was framed as having a macro-level dimension, as higher-level leadership structures were observed to not incentivize the development and effective use of research among constituent organizations, and also micro-level implications. From their perspective, researchers will struggle without such policy frameworks to obtain necessary data-sharing agreements with health authorities, nor will they be able to successfully navigate other barriers to conducting action-oriented research that informs policy and practice.

Similarly, a research actor perceived that the COVID-19 pandemic highlighted pre-existing fragmentation, "a pretty disjointed sort of enterprise" in how research is organized in the province:

I think pandemics need strong leadership and I think pandemic research response needed probably stronger leadership than it had. And I think that’s to do with [how] no one really knew who was in charge because no one really was given the role of being truly in charge of the research response (IDI14).

This individual underscored that, at the time of the interview, there were nearly 600 separate research projects being conducted in BC that focused on COVID-19. From their perspective, this reflected the need for more centralized direction to provide leadership, coordinate research efforts, and catalyze collaborations.

Overall, respondents perceived a lack of coordination and continuity across data sources, and a lack of explicit guidelines on evidence-use in the decision-making process, which resulted in a sense of fragmentation. The tension between the processes involved in research and the need for rapid decision-making was perceived as a barrier to using evidence to inform policy.

This study explored the use of evidence to inform early COVID-19 decision-making within British Columbia, Canada, from the perspectives of decision-makers themselves. Findings underscore the complexity of synthesizing and applying evidence (i.e., ‘scientific’ or research-based evidence most commonly discussed) to support public health policy in 'real-time', particularly in the context of public health crisis response. Despite a substantial and long-established literature on evidence-based clinical decision-making [ 23 , 24 ], understanding is more limited as to how public health crisis decision-making can be evidence-informed or evidence-based. By contributing to a growing global scholarship of retrospective examinations of COVID-19 decision-making processes [ 25 , 26 , 27 , 28 ], our study aimed to broaden this understanding and, thus, support the strengthening of public health emergency preparedness in Canada, and globally.

Specifically, based on our findings on evidence-based public health practice, we found that decision-makers clearly emphasized ‘evidence-based’ or ‘evidence-informed’ as meaning ‘scientific’ evidence. They acknowledged other forms of evidence such as professional expertise and contextual information as influencing factors. We identified four key points related to the process of evidence-use in BC's COVID-19 decision-making, with broader implications as well:

Role Differences: The tensions we observed primarily related to a lack of clarity among the various agencies involved as to their respective roles and responsibilities in a public health emergency, a finding that aligns with research on evidence-use in prior pandemics in Canada [ 29 ]. Relatedly, scientists and policy-makers experienced challenges with communication and information-flow between one another and the public, which may reflect their different values and standards, framing of issues and goals, and language [ 30 ].

Barriers to Evidence-Use: Coordination and consistency in how data are collected across jurisdictions reportedly impeded efficiency and timeliness of decision-making. Lancaster and Rhodes (2020) suggest that evidence itself should be treated as a process, rather than a commodity, in evidence-based practice [ 31 ]. Thus, shifting the dialogue from 'barriers to evidence use' to an approach that fosters dialogue across different forms of evidence and different actors in the process may be beneficial.

Use of Evidence in Public Health versus Medicine: Evidence-based public health can be conflated with the concept of evidence-based medicine, though these are distinct in the type of information that needs to be considered. While ‘research evidence’ was the primary type of evidence used, other important types of evidence informed policy decisions in the COVID-19 public health emergency—for example, previous experience, public values, and preferences. This concurs with Brownson’s (2009) framework of factors driving decision-making in evidence-based public health [ 32 ]. Namely, that a balance between multiple factors, situated in particular environmental and organizational context, shapes decision-making: 1) best available research evidence; 2) clients'/population characteristics, state, needs, values, and preferences; and 3) resources, including a practitioner’s expertise. Thus, any evaluation of evidence-use in public health policy must take into consideration this multiplicity of factors at play, and draw on frameworks specific to public health [ 33 ]. Moreover, public health decision-making requires much more attention to behavioural factors and non-clinical impacts, which is distinct from the largely biology-focused lens of evidence-based medicine.

Transparency: Many participants emphasized a lack of explanation about why certain decisions were made and a lack of understanding about who was involved in decisions and how those decisions were made. This point was confirmed by a recent report on lessons learned in BC during the COVID-19 pandemic in which the authors describe " the desire to know more about the reasons why decisions were taken " as a " recurring theme " (13:66). These findings point to a need for clear and transparent mechanisms for channeling evidence, irrespective of the form used, into public health crisis decision-making.

Our findings also pointed to challenges associated with the infrastructure for utilizing research evidence in BC policy-making, specifically a need for more centralized authority on the research side of the public health emergency response to avoid duplication of efforts and more effectively synthesize findings for efficient use. Yet, as a participant questioned, what is the realistic role of research in a public health crisis response? Generally, most evidence used to inform crisis response measures is local epidemiological data or modelling data [ 7 ]. As corroborated by our findings, challenges exist in coordinating data collection and synthesis of these local data across jurisdictions to inform 'real-time' decision-making, let alone to feed into primary research studies [ 34 ].

On the other hand, as was the case in the COVID-19 pandemic, a 'high noise' research environment soon became another challenge as data became available to researchers. Various mechanisms have been established to try and address these challenges amid the COVID-19 pandemic, both to synthesize scientific evidence globally and to create channels for research evidence to support timely decision-making. For instance: 1) research networks and collaborations are working to coordinate research efforts (e.g., COVID-END network [ 35 ]); 2) independent research panels or committees within jurisdictions provide scientific advice to inform decision-making; and 3) research foundations, funding agencies, and platforms for knowledge mobilization (e.g., academic journals) continue to streamline funding through targeted calls for COVID-19 research grant proposals, or for publication of COVID-19 research articles. While our findings describe the varied forms of evidence used in COVID-19 policy-making—beyond scientific evidence—they also point to the opportunity for further investments in infrastructure that coordinates, streamlines, and strengthens collaborations between health researchers and decision-makers that results in timely uptake of results into policy decisions.

Finally, in considering these findings, it is important to note the study's scope and limitations: We focused on evidence use in a single public health emergency, in a single province. Future research could expand this inquiry to a multi-site analysis of evidence-use in pandemic policy-making, with an eye to synthesizing lessons learned and best practices. Additionally, our sample of participants included only one elected official, so perspectives were limited from this type of role. The majority of participants were health officials who primarily referred to and discussed evidence as ‘scientific’ or research-based evidence. Further work could explore the facilitators and barriers to evidence-use from the perspectives of elected officials and Ministry personnel, particularly with respect to the forms of evidence—considered broadly—and other varied inputs, that shape decision-making in the public sphere. This could include a more in-depth examination of policy implementation and how the potential societal consequences of implementation factor into public health decision-making.

We found that the policy decisions made during the initial stages of the COVID-19 pandemic were perceived by actors in BC's response as informed by—not always based on—scientific evidence, specifically; however, decision-makers also considered other contextual factors and drew on prior pandemic-related experience to inform decision-making, as is common in evidence-based public health practice [ 32 ]. The respondents' experiences point to specific areas that need to be considered in planning for future public health emergencies, including information flow between policy-makers and researchers, coordination in how data are collected, and transparency in how decisions are made—all of which reflect a need to improve communication. Furthermore, shifting the discourse from evidence as a commodity to evidence-use as a process will be helpful in addressing barriers to evidence-use, as well as increasing understanding about the public health decision-making process as distinct from clinical medicine. Finally, there is a critical need for clear mechanisms that channel evidence (whether ‘scientific’, research-based, or otherwise) into health crisis decision-making, including identifying and communicating the decision-making process to those producing and synthesizing evidence. The COVID-19 pandemic experience is an opportunity to reflect on what needs to be done to guild our public health systems for the future [ 36 , 37 ]. Understanding and responding to the complexities of decision-making as we move forward, particularly with respect to the synthesis and use of evidence, can contribute to strengthening preparedness for future public health emergencies.

Availability of data and materials

The data that support the findings of this study are not publicly available to maintain the confidentiality of research participants.

The terms 'evidence-informed' and 'evidence-based' decision-making are used throughout this paper, though are distinct. The term 'evidence-informed' suggests that evidence is used and considered, though not necessarily solely determinative in decision-making [ 38 ].

The Provincial Health Services Authority (PHSA) works with the Ministry of Health (MOH) and regional health authorities to oversee the coordination and delivery of programs.

The Office of the Provincial Health Officer (PHO) has binding legal authority in the case of an emergency, and responsibility to monitor the health of BC’s population and provide independent advice to Ministers and public offices on public health issues.

The British Columbia Centre for Disease Control (BCCDC) is a program of the PHSA and provides provincial and national disease surveillance, detection, treatment, prevention, and consultation.

Abbreviations

British Columbia

British Columbia Centre for Disease Control

Coronavirus Disease 2019

Medical Health Officer

Ministry of Health

Provincial Health Officer

Provincial Health Services Authority

Severe Acute Respiratory Syndrome Coronavirus—2

University of British Columbia

Rubin O, Errett NA, Upshur R, Baekkeskov E. The challenges facing evidence-based decision making in the initial response to COVID-19. Scand J Public Health. 2021;49(7):790–6.

Article   PubMed   Google Scholar  

Williams GA, Ulla Díez SM, Figueras J, Lessof S, Ulla SM. Translating evidence into policy during the COVID-19 pandemic: bridging science and policy (and politics). Eurohealth (Lond). 2020;26(2):29–48.

Google Scholar  

Vickery J, Atkinson P, Lin L, Rubin O, Upshur R, Yeoh EK, et al. Challenges to evidence-informed decision-making in the context of pandemics: qualitative study of COVID-19 policy advisor perspectives. BMJ Glob Heal. 2022;7(4):1–10.

Piper J, Gomis B, Lee K. “Guided by science and evidence”? The politics of border management in Canada’s response to the COVID-19 pandemic. Front Polit Sci. 2022;4

Cairney P. The UK government’s COVID-19 policy: what does “Guided by the science” mean in practice? Front Polit Sci. 2021;3(March):1–14.

Colman E, Wanat M, Goossens H, Tonkin-Crine S, Anthierens S. Following the science? Views from scientists on government advisory boards during the COVID-19 pandemic: a qualitative interview study in five European countries. BMJ Glob Heal. 2021;6(9):1–11.

Salajan A, Tsolova S, Ciotti M, Suk JE. To what extent does evidence support decision making during infectious disease outbreaks? A scoping literature review. Evid Policy. 2020;16(3):453–75.

Article   Google Scholar  

Cairney P. The UK government’s COVID-19 policy: assessing evidence-informed policy analysis in real time. Br Polit. 2021;16(1):90–116.

Lancaster K, Rhodes T, Rosengarten M. Making evidence and policy in public health emergencies: lessons from COVID-19 for adaptive evidence-making and intervention. Evid Policy. 2020;16(3):477–90.

Yang K. What can COVID-19 tell us about evidence-based management? Am Rev Public Adm. 2020;50(6–7):706–12.

Parkhurst J. The politics of evidence: from evidence-based policy to the good governance of evidence. Abingdon: Routledge; 2017.

Office of the Prime Minister. Minister of Health Mandate Letter [Internet]. 2021. https://pm.gc.ca/en/mandate-letters/2021/12/16/minister-health-mandate-letter

de Faye B, Perrin D, Trumpy C. COVID-19 lessons learned review: Final report. Victoria, BC; 2022.

First Nations Health Authority. Evolving vaccination plans is science in action: Dr. Bonnie Henry. First Nations Health Authority. 2021.

BC Stats. 2021 Sub-provincial population estimates highlights. Vol. 2021. Victoria, BC; 2022.

Michael Smith Health Research BC [Internet]. 2023. healthresearchbc.ca. Accessed 25 Jan 2023.

Michael Smith Health Research BC. SRAC [Internet]. 2023. https://healthresearchbc.ca/strategic-provincial-advisory-committee-srac/ . Accessed 25 Jan 2023.

Brubacher LJ, Hasan MZ, Sriram V, Keidar S, Wu A, Cheng M, et al. Investigating the influence of institutions, politics, organizations, and governance on the COVID-19 response in British Columbia, Canada: a jurisdictional case study protocol. Heal Res Policy Syst. 2022;20(1):1–10.

Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3:77–101.

DeCuir-Gunby JT, Marshall PL, McCulloch AW. Developing and using a codebook for the analysis of interview data: an example from a professional development research project. Field Methods. 2011;23(2):136–55.

Gale NK, Heath G, Cameron E, Rashid S, Redwood S. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol. 2013;13(117):1–8.

Creswell JW, Miller DL. Determining validity in qualitative inquiry. Theory Pract. 2000;39(3):124–30.

Sackett D. How to read clinical journals: I. Why to read them and how to start reading them critically. Can Med Assoc J. 1981;1245:555–8.

Evidence Based Medicine Working Group. Evidence-based medicine: a new approach to teaching the practice of medicine. JAMA Netw. 1992;268(17):2420–5.

Allin S, Fitzpatrick T, Marchildon GP, Quesnel-Vallée A. The federal government and Canada’s COVID-19 responses: from “we’re ready, we’re prepared” to “fires are burning.” Heal Econ Policy Law. 2022;17(1):76–94.

Bollyky TJ, Hulland EN, Barber RM, Collins JK, Kiernan S, Moses M, et al. Pandemic preparedness and COVID-19: an exploratory analysis of infection and fatality rates, and contextual factors associated with preparedness in 177 countries, from Jan 1, 2020, to Sept 30, 2021. Lancet. 2022;6736(22):1–24.

Kuhlmann S, Hellström M, Ramberg U, Reiter R. Tracing divergence in crisis governance: responses to the COVID-19 pandemic in France, Germany and Sweden compared. Int Rev Adm Sci. 2021;87(3):556–75.

Haldane V, De Foo C, Abdalla SM, Jung AS, Tan M, Wu S, et al. Health systems resilience in managing the COVID-19 pandemic: lessons from 28 countries. Nat Med. 2021;27(6):964–80.

Article   CAS   PubMed   Google Scholar  

Rosella LC, Wilson K, Crowcroft NS, Chu A, Upshur R, Willison D, et al. Pandemic H1N1 in Canada and the use of evidence in developing public health policies—a policy analysis. Soc Sci Med. 2013;83:1–9.

Article   PubMed   PubMed Central   Google Scholar  

Saner M. A map of the interface between science & policy. Ottawa, Ontario; 2007. Report No.: January 1.

Lancaster K, Rhodes T. What prevents health policy being “evidence-based”? New ways to think about evidence, policy and interventions in health. Br Med Bull. 2020;135(1):38–49.

Brownson RC, Fielding JE, Maylahn CM. Evidence-based public health: a fundamental concept for public health practice. Annu Rev Public Health. 2009;30:175–201.

Rychetnik L, Frommer M, Hawe P, Shiell A. Criteria for evaluating evidence on public health interventions. J Epidemiol Community Health. 2002;56:119–27.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Khan Y, Brown A, Shannon T, Gibson J, Généreux M, Henry B, et al. Public health emergency preparedness: a framework to promote resilience. BMC Public Health. 2018;18(1):1–16.

COVID-19 Evidence Network to Support Decision-Making. COVID-END [Internet]. 2023. https://www.mcmasterforum.org/networks/covid-end . Accessed 25 Jan 2023.

Canadian Institutes of Health Research. Moving forward from the COVID-19 pandemic: 10 opportunities for strengthening Canada’s public health systems. 2022.

Di Ruggiero E, Bhatia D, Umar I, Arpin E, Champagne C, Clavier C, et al. Governing for the public’s health: Governance options for a strengthened and renewed public health system in Canada. 2022.

Adjoa Kumah E, McSherry R, Bettany-Saltikov J, Hamilton S, Hogg J, Whittaker V, et al. Evidence-informed practice versus evidence-based practice educational interventions for improving knowledge, attitudes, understanding, and behavior toward the application of evidence into practice: a comprehensive systematic review of undergraduate studen. Campbell Syst Rev. 2019;15(e1015):1–19.

Download references

Acknowledgements

We would like to extend our gratitude to current and former members of the University of British Columbia Working Group on Health Systems Response to COVID-19 who contributed to various aspects of this study, including Shelly Keidar, Kristina Jenei, Sydney Whiteford, Dr. Md Zabir Hasan, Dr. David M. Patrick, Dr. Maxwell Cameron, Mahrukh Zahid, Dr. Yoel Kornreich, Dr. Tammi Whelan, Austin Wu, Shivangi Khanna, and Candice Ruck.

Financial support for this work was generously provided by the University of British Columbia's Faculty of Medicine (Grant No. GR004683) and Peter Wall Institute for Advanced Studies (Grant No. GR016648), as well as a Canadian Institutes of Health Research Operating Grant (Grant No. GR019157). These funding bodies were not involved in the design of the study, the collection, analysis or interpretation of data, or in the writing of this manuscript.

Author information

Authors and affiliations.

School of Population and Public Health, University of British Columbia, Vancouver, Canada

Laura Jane Brubacher, Chris Y. Lovato, Veena Sriram, Michael Cheng & Peter Berman

School of Public Health Sciences, University of Waterloo, Waterloo, Canada

Laura Jane Brubacher

School of Public Policy and Global Affairs, University of British Columbia, Vancouver, Canada

Veena Sriram

You can also search for this author in PubMed   Google Scholar

Contributions

CYL, PB, and VS obtained funding for and designed the study. LJB, MC, and PB conducted data collection. LJB and VS analyzed the qualitative data. CYL and LJB collaboratively wrote the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Laura Jane Brubacher .

Ethics declarations

Ethics approval and consent to participate.

This case study received the approval of the UBC Behavioural Research Ethics Board (Certificate # H20-02136). Participants provided written informed consent.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1..

Semi-structured interview guide [* = questions used for this specific study]

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Brubacher, L.J., Lovato, C.Y., Sriram, V. et al. The use of evidence to guide decision-making during the COVID-19 pandemic: divergent perspectives from a qualitative case study in British Columbia, Canada. Health Res Policy Sys 22 , 66 (2024). https://doi.org/10.1186/s12961-024-01146-2

Download citation

Received : 08 February 2023

Accepted : 29 April 2024

Published : 03 June 2024

DOI : https://doi.org/10.1186/s12961-024-01146-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Decision-making
• Public health
• Policy-making
• Qualitative

Health Research Policy and Systems

ISSN: 1478-4505

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Understanding the Stigma Experience of Men Living with HIV in Sub-Saharan Africa: A Qualitative Meta-synthesis

• Substantive Review
• Published: 22 May 2024

Cite this article

• Sarah E. Janek   ORCID: orcid.org/0009-0002-1213-2791 1 ,
• Sandy Hatoum   ORCID: orcid.org/0009-0002-3618-9733 2 ,
• Leila Ledbetter   ORCID: orcid.org/0000-0002-5206-8002 3 &
• Michael V. Relf   ORCID: orcid.org/0000-0002-4951-8869 1 , 2  

51 Accesses

Explore all metrics

Men living with HIV (MLWH) in sub-Saharan Africa experience poor health outcomes and increased AIDS-related deaths due to stigma influencing testing and treatment uptake and adherence. PRISMA 2020 was used to report a meta-synthesis of the stigma experiences of MLWH in SSA. With the help of an expert librarian, a search of six databases was formulated and performed to examine the available qualitative and mixed method studies with qualitative results relevant to the research question. Studies focused on adult men living with HIV, with five studies specifically examining the HIV experience of men who have sex with men. Study themes were synthesized to describe MLWH’s perceived, internalized, anticipated, enacted, and intersectional stigma experiences. Most studies included masculinity as a key theme that affected both testing and treatment adherence upon diagnosis. Future research is needed to better understand subpopulations, such as men who have sex with men living with HIV, and what interventions may be beneficial to mitigate the disparities among MLWH in SSA.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

“It is not an acceptable disease”: A qualitative study of HIV-related stigma and discrimination and impacts on health and wellbeing for people from ethnically diverse backgrounds in Australia

Overcoming hiv stigma a qualitative analysis of hiv cure research and stigma among men who have sex with men living with hiv, understanding hiv-related stigma among women in the southern united states: a literature review.

Kaiser Family Foundation. The global HIV/AIDS epidemic. KFF. 2022. https://www.kff.org/global-health-policy/fact-sheet/the-global-hivaids-epidemic/ . Accessed 13 Nov 2022.

UNAIDS. Fact sheet—latest global and regional statistics on the status of the AIDS epidemic. Joint United Nations Programme on HIV/AIDS (UNAIDS). 2022. https://www.unaids.org/sites/default/files/media_asset/UNAIDS_FactSheet_en.pdf . Accessed 23 Oct 2022.

West CA, Chang GC, Currie DW, Bray R, Kinchen S, Behel S, McCullough-Sanden R, Low A, Bissek A, Shang JD, Ndongmo CB, Dokubo EK, Balachandra S, Lobognon LR, Dube L, Nuwagaba-Biribonwoha H, Li M, Pasipamire M, Getaneh Y, Lulseged S, Eshetu F, Kingwara L, Zielinski-Gutierrez E, Tlhomola M, Ramphalla P, Kalua T, Auld AF, Williams DB, Remera E, Rwibasira GN, Mugisha V, Malamba SS, Mushi J, Jalloh MF, Mgomella GS, Kirungi WL, Biraro S, Awor AC, Barradas DT, Mugurungi O, Rogers JH, Bronson M, Bodika SM, Ajiboye A, Gaffga N, Moore C, Patel HK, Voetsch AC. Unawareness of HIV infection among men aged 15–59 years in 13 sub-Saharan African countries: findings from the population-based HIV impact assessments, 2015–2019. JAIDS. 2021;87:S97.

PubMed   Google Scholar  

UNAIDS. Executive summary—in danger: UNAIDS global AIDS update 2022 [Internet]. Joint United Nations Programme on HIV/AIDS (UNAIDS). 2022. https://www.unaids.org/sites/default/files/media_asset/2022-global-aids-update-summary_en.pdf . Accessed 23 Oct 2022.

UNAIDS. Addressing a blind spot in the response to HIV—reaching out to men and boys. Joint United Nations Programme on HIV/AIDS (UNAIDS). 2017. https://www.unaids.org/sites/default/files/media_asset/blind_spot_en.pdf . Accessed 23 Oct 2022.

Hatzenbuehler ML, Phelan JC, Link BG. Stigma as a fundamental cause of population health inequalities. Am J Public Health. 2013;103:813–21. https://doi.org/10.2105/AJPH.2012.301069 .

Article   PubMed   PubMed Central   Google Scholar  

Relf MV, Holzemer WL, Holt L, Nyblade L, Caiola CE. A review of the state of the science of HIV and stigma: context, conceptualization, measurement, interventions, gaps, and future priorities. J Assoc Nurses AIDS Care. 2021;32:392–407. https://doi.org/10.1097/JNC.0000000000000237 .

Herek GM. AIDS and stigma. Am Behav Sci. 1999;42:1106–16. https://doi.org/10.1177/00027649921954787 .

Article   Google Scholar  

Earnshaw VA, Smith LR, Chaudoir SR, Amico KR, Copenhaver MM. HIV stigma mechanisms and well-being among PLWH: a test of the HIV stigma framework. AIDS Behav. 2013;17:1785–95. https://doi.org/10.1007/s10461-013-0437-9 .

Derlega VJ, Winstead BA, Greene K, Serovich J, Elwood WN. Perceived HIV-related stigma and HIV disclosure to relationship partners after finding out about the seropositive diagnosis. J Health Psychol. 2002;7:415–32. https://doi.org/10.1177/1359105302007004330 .

Article   PubMed   Google Scholar  

Arinaitwe I, Amutuhaire H, Atwongyeire D, Tusingwire E, Kawungezi PC, Rukundo GZ, Ashaba S. Social support, food insecurity, and HIV stigma among men living with HIV in rural southwestern Uganda: a cross-sectional analysis. HIV AIDS (Auckl). 2021;13:657–66. https://doi.org/10.2147/HIV.S316174 .

Mburu G, Ram M, Siu G, Bitira D, Skovdal M, Holland P. Intersectionality of HIV stigma and masculinity in eastern Uganda: implications for involving men in HIV programmes. BMC Public Health. 2014;14:1061. https://doi.org/10.1186/1471-2458-14-1061 .

Sileo KM, Fielding-Miller R, Dworkin SL, Fleming PJ. A scoping review on the role of masculine norms in men’s engagement in the HIV care continuum in sub-Saharan Africa. AIDS Care. 2019;31:1435–46. https://doi.org/10.1080/09540121.2019.1595509 .

Abara WE, Garba I. HIV epidemic and human rights among men who have sex with men in sub-Saharan Africa: implications for HIV prevention, care, and surveillance. Glob Public Health. 2017;12:469–82. https://doi.org/10.1080/17441692.2015.1094107 .

Joshi K, Lessler J, Olawore O, Loevinsohn G, Bushey S, Tobian AAR, Grabowski MK. Declining HIV incidence in sub-Saharan Africa: a systematic review and meta-analysis of empiric data. J Int AIDS Soc. 2021. https://doi.org/10.1002/jia2.25818 .

Nyato D, Kuringe E, Drake M, Casalini C, Nnko S, Shao A, Komba A, Baral SD, Wambura M, Changalucha J. Participants’ accrual and delivery of HIV prevention interventions among men who have sex with men in sub-Saharan Africa: a systematic review. BMC Public Health. 2018. https://doi.org/10.1186/s12889-018-5303-2 .

Hamilton A, Thompson N, Choko AT, Hlongwa M, Jolly P, Korte JE, Conserve DF. HIV self-testing uptake and intervention strategies among men in sub-Saharan Africa: a systematic review. Front Public Health. 2021. https://doi.org/10.3389/fpubh.2021.594298 .

Hlongwa M, Hlongwana K, Makhunga S, Choko AT, Dzinamarira T, Conserve D, Tsai AC. Linkage to HIV care following HIV self-testing among men: systematic review of quantitative and qualitative studies from six countries in sub-Saharan Africa. AIDS Behav. 2022. https://doi.org/10.1007/s10461-022-03800-8 .

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, Shamseer L, Tetzlaff JM, Akl EA, Brennan SE, Chou R, Glanville J, Grimshaw JM, Hróbjartsson A, Lalu MM, Li T, Loder EW, Mayo-Wilson E, McDonald S, McGuinness LA, Stewart LA, Thomas J, Tricco AC, Welch VA, Whiting P, Moher D. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021. https://doi.org/10.1136/bmj.n71 .

Turan B, Hatcher AM, Weiser SD, Johnson MO, Rice WS, Turan JM. Framing mechanisms linking HIV-related stigma, adherence to treatment, and health outcomes. Am J Public Health. 2017;107:863–9. https://doi.org/10.2105/AJPH.2017.303744 .

McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol. 2016;75:40–6. https://doi.org/10.1016/j.jclinepi.2016.01.021 .

UNAIDS. 90-90-90: an ambitious treatment target to help end the AIDS epidemic. Joint United Nations Programme on HIV/AIDS (UNAIDS). 2014. https://www.unaids.org/sites/default/files/media_asset/90-90-90_en.pdf . Accessed 10 June 2023.

UNAIDS. AIDS by the numbers: AIDS is not over, but it can be. Joint United Nations Programme on HIV/AIDS (UNAIDS). 2016. https://www.unaids.org/sites/default/files/media_asset/AIDS-by-the-numbers-2016_en.pdf . Accessed 10 June 2023.

Covidence systematic review software. Veritas Health Innovation. 2022. https://www.covidence.org/ . Accessed 8 Mar 2022.

Kmet LM, Lee RC, Cook LS. HTA initiative #13: standard quality assessment criteria for evaluating primary research papers from a variety of fields. Alberta Heritage Foundation for Medical Research. 2004. https://www.ihe.ca/download/standard_quality_assessment_criteria_for_evaluating_primary_research_papers_from_a_variety_of_fields.pdf . Accessed 24 Oct 2022.

Sandelowski M, Barroso J. Handbook for synthesizing qualitative research. New York: Springer Publishing Company; 2007.

Google Scholar  

Belay YA, Yitayal M, Atnafu A, Taye FA. Patient experiences and preferences for antiretroviral therapy service provision: implications for differentiated service delivery in Northwest Ethiopia. AIDS Res Ther. 2022;19:1–16. https://doi.org/10.1186/s12981-022-00452-5 .

Hlongwa M, Jama NA, Mehlomakulu V, Pass D, Basera W, Nicol E. Barriers and facilitating factors to HIV treatment among men in a high-HIV-burdened district in KwaZulu-Natal, South Africa: a qualitative study. Am J Mens Health. 2022. https://doi.org/10.1177/15579883221120987 .

Lofgren SM, Tsui S, Atuyambe L, Ankunda L, Komuhendo R, Wamala N, Sadiq A, Kirumira P, Srishyla D, Flynn A, Pastick KA, Meya DB, Nakasujja N, Porta C. Barriers to HIV care in Uganda and implications for universal test-and-treat: a qualitative study. AIDS Care. 2022;34:597–605. https://doi.org/10.1080/09540121.2021.1946000 .

Mandawa MB, Mahiti GR. Factors contributing to loss to follow-up from HIV care among men living with HIV/AIDS in Kibaha District, Tanzania. HIV AIDS (Auckl). 2022;14:503–16. https://doi.org/10.2147/hiv.S381204 .

Mange T, Henderson N, Lukelelo N. ‘After 25 years of democracy we are still stigmatised and discriminated against’ healthcare experiences of HIV-positive older black gay men in a township in South Africa. J Pract Teach Learn. 2022;9:87–100. https://doi.org/10.1921/jpts.v19i1-2.1674 .

Mathenjwa M, Khidir H, Milford C, Mosery N, Rambally Greener L, Pratt MC, O’Neil K, Harrison A, Bangsberg DR, Safren SA, Smit JA, Psaros C, Matthews LT. Acceptability of an intervention to promote viral suppression and serostatus disclosure for men living with HIV in South Africa: qualitative findings. AIDS Behav. 2022;26:1–12. https://doi.org/10.1007/s10461-021-03278-w .

Muwanguzi PA, Nelson LE, Ngabirano TD, Kiwanuka N, Osingada CP, Sewankambo NK. Linkage to care and treatment among men with reactive HIV self-tests after workplace-based testing in Uganda: a qualitative study. Front Public Health. 2022. https://doi.org/10.3389/fpubh.2022.650719 .

Nabikande S, Namutundu J, Nangendo J, Okello T, Agwang W, Tusabe J, Kabwama SN, Katahoire AR. Men’s late presentation for HIV care in Eastern Uganda: the role of masculinity norms. PLoS ONE. 2022. https://doi.org/10.1371/journal.pone.0277534 .

Ndione AG, Procureur F, Senne JN, Cornaglia F, Gueye K, Ndour CT, Lépine A. Sexuality-based stigma and access to care: intersecting perspectives between health care providers and men who have sex with men in HIV care centres in Senegal. Health Policy Plan. 2022;37:587–96. https://doi.org/10.1093/heapol/czac010 .

Rich C, Mavhu W, France NF, Munatsi V, Byrne E, Willis N, Nolan A. Exploring the beliefs, experiences and impacts of HIV-related self-stigma amongst adolescents and young adults living with HIV in Harare, Zimbabwe: a qualitative study. PLoS ONE. 2022. https://doi.org/10.1371/journal.pone.0268498 .

Hendrickson ZM, Naugle DA, Tibbels N, Dosso A, Van Lith LM, Mallalieu EC, Kamara D, Dailly-Ajavon P, Cisse A, Ahanda KS, Thaddeus S, Babalola S, Hoffman CJ. “You take medications, you live normally”: the role of antiretroviral therapy in mitigating men’s perceived threats of HIV in Côte d’Ivoire. AIDS Behav. 2019;23:2600–9. https://doi.org/10.1007/s10461-019-02614-5 .

Naugle DA, Tibbels NJ, Hendrickson ZM, Dosso A, Van Lith LM, Mallalieu EC, Kouadio AM, Kra W, Kamara D, Dailly-Ajavon P, Cissé A, Seifert-Ahanda K, Thaddeus S, Babalola S, Hoffman CJ. Bringing fear into focus: the intersections of HIV and masculine gender norms in Côte d’Ivoire. PLoS ONE. 2019. https://doi.org/10.1371/journal.pone.0223414 .

Tibbels NJ, Hendrickson ZM, Naugle DA, Dosso A, Van Lith LM, Mallalieu EC, Kouadio AM, Kra W, Kamara D, Dailly-Ajavon P, Cisse A, Seifert-Ahanda K, Thaddeus S, Babalola S, Hoffmann CJ. Men’s perceptions of HIV care engagement at the facility- and provider-levels: experiences in Cote d’Ivoire. PLoS ONE. 2019. https://doi.org/10.1371/journal.pone.0211385 .

Balogun A, Bissell P, Saddiq M. Negotiating access to the Nigerian healthcare system: the experiences of HIV-positive men who have sex with men. Cult Health Sex. 2020;22:233–46. https://doi.org/10.1080/13691058.2019.1582802 .

Mukumbang FC. Leaving no man behind: how differentiated service delivery models increase men’s engagement in HIV care. Int J Health Policy Manag. 2021;10:129–40. https://doi.org/10.34172/ijhpm.2020.32 .

Ogunbajo A, Kershaw T, Kushwaha S, Boakye F, Wallace-Atiapah N-D, Nelson LE. Barriers, motivators, and facilitators to engagement in HIV care among HIV-infected Ghanaian men who have sex with men (MSM). AIDS Behav. 2018;22:829–39. https://doi.org/10.1007/s10461-017-1806-6 .

Okal J, Lango D, Matheka J, Obare F, Ngunu-Gituathi C, Mugambi M, Avina S. “It is always better for a man to know his HIV status”—a qualitative study exploring the context, barriers and facilitators of HIV testing among men in Nairobi, Kenya. PLoS ONE. 2020. https://doi.org/10.1371/journal.pone.0231645 .

Okoror TA, Falade CO, Walker EM, Olorunlana A, Anaele A. Social context surrounding HIV diagnosis and construction of masculinity: a qualitative study of stigma experiences of heterosexual HIV positive men in southwest Nigeria. BMC Public Health. 2016;16:507. https://doi.org/10.1186/s12889-016-3165-z .

Sileo KM, Reed E, Kizito W, Wagman JA, Stockman JK, Wanyenze RK, Chemusto H, Musoke W, Mukasa B, Kiene SM. Masculinity and engagement in HIV care among male fisherfolk on HIV treatment in Uganda. Cult Health Sex. 2019;21:774–88. https://doi.org/10.1080/13691058.2018.1516299 .

Zissette S, Watt MH, Prose NS, Mntambo N, Moshabela M. “If you don’t take a stand for your life, who will help you?”: men’s engagement in HIV care in KwaZulu-Natal, South Africa. Psychol Men Masc. 2016;17:265–73. https://doi.org/10.1037/men0000025 .

Berner-Rodoreda A, Ngwira E, Alhassan Y, Chione B, Dambe R, Bärnighausen T, Phiri S, Taegtmeyer M, Neuhann F. “Deadly”, “fierce”, “shameful”: notions of antiretroviral therapy, stigma and masculinities intersecting men’s life-course in Blantyre, Malawi. BMC Public Health. 2021. https://doi.org/10.1186/s12889-021-12314-2 .

Mantell JE, Masvawure TB, Mapingure M, Apollo T, Gwanzura C, Block L, Bennett E, Preko P, Musuka G, Rabkin M. Engaging men in HIV programmes: a qualitative study of male engagement in community-based antiretroviral refill groups in Zimbabwe. J Int AIDS Soc. 2019. https://doi.org/10.1002/jia2.25403 .

Misra S, Mehta HT, Eschliman EL, Rampa S, Poku OB, Wang W-Q, Ho-Foster AR, Mosepele M, Becker TD, Entaile P, Arscott-Mills T, Opondo PR, Blank MB, Yang LH. Identifying “what matters most” to men in Botswana to promote resistance to HIV-related stigma. Qual Health Res. 2021;31:1680–96. https://doi.org/10.1177/10497323211001361 .

Mooney AC, Gottert A, Khoza N, Rebombo D, Hove J, Suárez AJ, Twine R, MacPhail C, Treves-Kagan S, Kahn K, Pettifor A, Lippman SA. Men’s perceptions of treatment as prevention in South Africa: implications for engagement in HIV care and treatment. AIDS Educ Prev. 2017;29:274–87. https://doi.org/10.1521/aeap.2017.29.3.274 .

Moyo I, Macherera M, Mavhandu-Mudzusi AH. The lived experiences of men who have sex with men when accessing HIV care services in Zimbabwe. Health SA. 2021. https://doi.org/10.4102/hsag.v26i0.1462 .

Meskele M, Khuzwayo N, Taylor M. Mapping the evidence of intimate partner violence among women living with HIV/AIDS in sub-Saharan Africa: a scoping review. BMJ Open. 2021. https://doi.org/10.1136/bmjopen-2020-041326 .

Tenkorang EY, Asamoah-Boaheng M, Owusu AY. Intimate partner violence (IPV) against HIV-positive women in sub-Saharan Africa: a mixed-method systematic review and meta-analysis. Trauma Violence Abuse. 2021;22:1104–28. https://doi.org/10.1177/1524838020906560 .

Sileo KM, Fielding-Miller R, Dworkin SL, Fleming PJ. What role do masculine norms play in men’s HIV testing in sub-Saharan Africa?: a scoping review. AIDS Behav. 2018;22:2468–79. https://doi.org/10.1007/s10461-018-2160-z .

Naanyu V, Ruff J, Goodrich S, Spira T, Bateganya M, Toroitich-Ruto C, Otieno-Nyunya B, Siika AM, Wools-Kaloustian K. Qualitative exploration of perceived benefits of care and barriers influencing HIV care in trans Nzoia, Kenya. BMC Health Serv Res. 2020. https://doi.org/10.1186/s12913-020-05236-z .

Makoae LN, Seboni NM, Molosiwa K, Moleko M, Human S, Sukati NA, Holzemer WL. The symptom experience of people living with HIV/AIDS in southern Africa. J Assoc Nurses AIDS Care. 2005;16:22–32. https://doi.org/10.1016/j.jana.2005.03.005 .

McGuinness LA, Higgins JPT. Risk-of-bias VISualization (robvis): an R package and Shiny web app for visualizing risk-of-bias assessments. Res Synth Methods. 2020. https://doi.org/10.1002/jrsm.1411 .

Download references

Acknowledgements

The manuscript was supported by the Fogarty International Center/National Institutes of Health through Award Number R21TW011247 (M. Relf, Contact MPI/L. Nyblade, MPI) and the Duke University Center for AIDS Research (CFAR), an NIH funded program (5P30AI064518). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Research reported in this publication was supported by the Fogarty International Center of the National Institutes for Health under award R21TW012007 and by the Duke Center for AIDS Research, a National Institutes of Health funded program under award number 5P30AI064518. The content is solely the responsibility of the authors and does not represent the official views of the National Institutes of Health.

Author information

Authors and affiliations.

School of Nursing, Duke University, 307 Trent Drive, Box 3322, Durham, NC, 27710, USA

Sarah E. Janek & Michael V. Relf

Duke Global Health Institute, Durham, NC, USA

Sandy Hatoum & Michael V. Relf

Medical Center Library, Duke University, Durham, NC, USA

Leila Ledbetter

You can also search for this author in PubMed   Google Scholar

Contributions

All authors on this paper meet the four criteria for authorship as identified by the International Committee of Medical Journal Editors; all authors have contributed to the drafting or been involved in revising it, reviewed the final version of this manuscript before submission, and agree to be accountable for all aspects of the work. Specifically, using the CRediT taxonomy, the specific contribution of each author is as follows: Conceptualization & Methodology: S. Janek, L. Ledbetter, M. Relf. Formal Analysis: S. Hatoum, S. Janek, M. Relf. Funding Acquisition: M. Relf. Investigation: S. Hatoum, S. Janek, L. Ledbetter, M. Relf. Methodology: S. Janek, L. Ledbetter, M. Relf. Project administration: M. Relf. Supervision: M. Relf. Verification: S. Hatoum, S. Janek, M. Relf. Writing—manuscript draft: S. Janek. Writing—review & editing: S. Hatoum, S. Janek, L. Ledbetter, M. Relf.

Corresponding authors

Correspondence to Sarah E. Janek or Michael V. Relf .

Ethics declarations

Conflict of interest.

The authors report no real or perceived vested interests related to this article that could be construed as a conflict of interest.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Registration The protocol that guided this meta-synthesis was prospectively registered with Prospero (registration ID: CRD42022315871).

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 35 KB)

Rights and permissions.

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Janek, S.E., Hatoum, S., Ledbetter, L. et al. Understanding the Stigma Experience of Men Living with HIV in Sub-Saharan Africa: A Qualitative Meta-synthesis. AIDS Behav (2024). https://doi.org/10.1007/s10461-024-04329-8

Download citation

Accepted : 20 March 2024

Published : 22 May 2024

DOI : https://doi.org/10.1007/s10461-024-04329-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Meta-synthesis
• People with HIV
• Sub-Saharan Africa
• Qualitative methodology
• Find a journal
• Publish with us
• Track your research