literature review building services

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates, and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

• Quick Run-through
• Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and its scholarly context
• Develop a theoretical framework and methodology for your research
• Position your work in relation to other researchers and theorists
• Show how your research addresses a gap or contributes to a debate
• Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models, and methods?
• Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

The only proofreading tool specialized in correcting academic writing - try for free!

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Try for free

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

• Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically evaluate: mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarize yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved April 9, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, unlimited academic ai-proofreading.

✔ Document error-free in 5minutes ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

Smart Building: A Literature Review

Ieee account.

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

Co-benefits as a rationale and co-benefits as a factor for urban climate action: linking air quality and emission reductions in Moscow, Paris, and Montreal

• Open access
• Published: 14 December 2023
• Volume 176 , article number  179 , ( 2023 )

Cite this article

You have full access to this open access article

• Matteo Roggero   ORCID: orcid.org/0000-0003-1297-9706 1 ,
• Anastasiia Gotgelf 1 &
• Klaus Eisenack 1  

812 Accesses

12 Altmetric

Explore all metrics

If local governments reduce greenhouse gas emissions, they will not see effects unless a very large number of other actors do the same. However, reducing greenhouse gas emissions can have multiple local “co-benefits” (improved air quality, energy savings, even energy security), creating incentives for local governments to reduce emissions—if just for the local side-effects of doing so. Available empirical research yet shows a large gap between co-benefits as a rationale and an explanatory factor for climate mitigation by local governments: co-benefits are seemingly very large, but do not seem to drive local mitigation efforts. Relying on policy documents, available research, and other written sources, the present paper consists of a multiple case study addressing the link between co-benefits and climate mitigation in Moscow, Paris, and Montreal. Air quality plays a very different role in each case, ranging from a key driver of mitigation to a liability for local climate action. This heterogeneity of mechanisms in place emerges as a likely explanation for the lack of a clear empirical link between co-benefits and local mitigation in the literature. We finally discuss implications for urban climate action policy and research.

Similar content being viewed by others

Strategies to achieve a carbon neutral society: a review

Lin Chen, Goodluck Msigwa, … Pow-Seng Yap

The costs and benefits of environmental sustainability

Paul Ekins & Dimitri Zenghelis

Advancing green finance: a review of sustainable development

Chengbo Fu, Lei Lu & Mansoor Pirabi

Avoid common mistakes on your manuscript.

1 Introduction

The importance of cities for achieving climate mitigation targets is increasingly recognized, and many cities all over the world are committed to climate action. This goes beyond official commitments: cities are actually delivering on emission reductions (Hsu et al. 2020 ; Khan and Sovacool 2016 ; Kona et al. 2018 ). Decreasing emissions in one city does not make a big difference for the global climate. However, that does not seem to halt local governments from taking action on climate change. Much to the contrary, Chapter 8 of the last Assessment Report of the Intergovernmental Panel on Climate Change reports that local governments which reduce emissions and adapt to climate impacts also experience a broader range of further local benefits (Dodman et al. 2022 ): local benefits might thus be more evident for them than global ones. Policy and academic circles have multiple names for those benefits, synergies, or trade-offs related to climate action (including both mitigation and adaptation). The most popular one is arguably the “co-benefits” concept (Floater et al. 2016 ; Mayrhofer and Gupta 2016 ; Ürge-Vorsatz et al. 2014 ). Research has shown how co-benefits of climate policy go way beyond “cost and carbon,” covering issues as diverse as national pride, industry innovation, improved international relations, rural development, domestic comfort, noise reduction, greater mobility, and many more (Sovacool et al. 2020 ). More importantly, some of these co-benefits are substantial, out-weighting the costs of climate action policies. This is the case for the health benefits from improved air quality through mitigation action (Karlsson et al. 2020 ; Ürge-Vorsatz et al. 2014 ).

The interest in co-benefits is linked to their “advocacy potential” (Mayrhofer and Gupta 2016 ; Workman et al. 2018 ): co-benefits represent a strong rationale for climate action, providing a compelling argument for local governments to make climate action more acceptable to local constituencies. If so, those cities with the greatest potential to reap co-benefits should be among the cities with more advanced climate action. Surprisingly, empirical research on local climate action (Castán-Broto and Westman 2020 ; van der Heijden 2019 ) does not list co-benefits among the factors explaining climate action. Climate action by local governments is typically explained in terms of national policies (Andonova et al. 2017 ; Domorenok 2019 ; Heidrich et al. 2016 ), local capacities (Homsy 2018 ; Krause et al. 2021 ), and the role of transnational municipal networks (Bansard et al. 2017 ; Michaelowa and Michaelowa 2017 ). Co-benefits, instead, do not seem to drive local climate action (Krause 2011 ; Pablo-Romero et al. 2015 ; Pitt 2010 ; Rashidi and Patt 2018 ). There is, in other words, a large gap between 1) co-benefits as an argumentative rationale for climate action and 2) co-benefits as an empirical, explanatory factor for climate action.

If research has so far struggled to establish a clear empirical link between co-benefits and actual climate action, we probably know too little about the role of co-benefits in climate action processes. With that in mind, the present paper focuses on the largest and most studied co-benefit of climate action: public health benefits through improved air quality from mitigation policies (Karlsson et al. 2020 ; Ürge-Vorsatz et al. 2014 ). We explore the link between air quality and climate mitigation in Moscow, Paris, and Montreal, three large cities with a capable local government, a substantial degree of autonomy, an affinity to the technology sector, an explicit green growth agenda, and membership in multiple transnational municipal networks addressing urban climate action. All experienced air quality problems in the 1990s, and all achieved significant greenhouse gas (“GHG”) emission reductions since. Relying on policy documents and available research in the frame of a multiple case study research design, the present paper provides a qualitative account of the processes through which climate action unfolded in the three cities from the 1990s to the present, tracking the role of air quality therein. The goal is to answer the following, overarching research question: how do air quality considerations affect climate mitigation processes?

Looking at the same co-benefit in three remarkably similar contexts, we observed remarkably different processes. Air quality considerations clearly drove emission reductions in Moscow. They played only a minor role in Paris, though, where air quality rather relates to climate adaptation. In Montreal, finally, air quality and emission reductions represent completely separate processes, with little interaction. The complexity and heterogeneity of these processes likely explain why local governments seldom exploit co-benefits as a rationale for climate action to make mitigation measures more acceptable to the local constituencies. It is therefore not surprising that previous empirical studies struggle to establish an empirical link between co-benefits and climate action. Furthermore, if process complexity and heterogeneity pose challenges to exploiting the largest and most widespread type of co-benefit, the lesson arguably holds for co-benefits more generally. We then discuss implications for policy and research.

2 Literature review

2.1 co-benefits as a rationale for climate action by local governments.

Co-benefits of climate action (mitigation or adaptation) have attracted a substantial research interest in the last decade (Floater et al. 2016 ; Mayrhofer and Gupta 2016 ; Workman et al. 2018 ). They represent the biophysical and technological interdependencies between climate action measures and other policy areas (health, air quality, housing, transport, waste, etc.): promoting low-emissions vehicles with high fuel efficiency standards reduces energy consumption (a mitigation measure) but also contributes to reducing particulate matter concentrations, lowering the costs connected to respiratory diseases and premature deaths (a public health co-benefit). Co-benefits are not exclusive to mitigation: recent studies highlight adaptation co-benefits as well (Hennessey et al. 2017 ; Luyten et al. 2023 ), and address synergies and trade-offs between adaptation and mitigation (Dodman et al. 2022 ; Sharifi 2021 ). Recent reviews report a broad range of benefits from climate action (both mitigation and adaptation), including healthier diets and physical activity, soil and water quality improvements, public savings, the reduction of energy poverty, biodiversity, and, most importantly, air quality (Dodman et al. 2022 ; Karlsson et al. 2020 ). Air quality co-benefits represent the largest portion of the co-benefits literature, focusing on the public health dimension of climate mitigation (Maibach et al. 2010 ; Nowak et al. 2018 ; Sabel et al. 2016 ).

Early debates brought up the idea of co-benefits as a rationale for local climate mitigation. However simplistic, the reasoning is the following. Climate mitigation constitutes a collective action problem: the costs of reducing GHG emissions stand against benefits (a reduction in future climate impacts) that depend from many other actors also reducing their emissions. Overcoming collective action problems requires “selective incentives” (Brody et al. 2008 ; Olson 1965 ; Rübbelke 2002 ). Co-benefits are good candidates for that. Among those listed above, air quality stands out as a key concern in urban areas (Strosnider 2017 ), with manufacturing, transport, energy supply, waste, and the residential/commercial sector being both key sources of air pollution (EEA 2020 ) and key areas of local government regulation. Many have therefore argued that mitigation action is worthwhile for local governments because of the improved air quality, independently of the expected reduction in future climate impacts (Kousky and Schneider 2003 ; Lee and Koski 2014 ; Sippel and Jenssen 2009 ).

Whether co-benefits constitute a rationale for local governments to engage with climate mitigation depends on their size in relation to the costs of mitigation. A substantial amount of research has thus attempted to quantify air quality co-benefits and estimate their magnitude in monetary terms. The World Health Organization reports 6.7 million deaths globally attributed to the joint effects of ambient and household air pollution for 2019—a predominantly (although not exclusively) urban phenomenon (WHO 2023 ). A study of nine countries (Brazil, China, Germany, India, Indonesia, Nigeria, South Africa, the UK, and the USA) assesses between 1.2 and 1.6 million deaths per year avoided because of the air quality co-benefits of meeting climate mitigation targets (Hamilton et al. 2021 ). A study of premature mortality in 969 European cities and 47 greater cities assesses preventable deaths between 51,000 and 125,000 yearly (Khomenko et al. 2021 ). A similar study of 161 Chinese cities estimated 652,000 deaths in 2015 alone (Maji et al. 2018 ). For 2020, implementing Madrid’s integrated air quality and climate action plan should postpone 88 and 519 deaths respectively for PM 2.5 and NO 2 (Izquierdo et al. 2020 ).

Once the impacts of air pollution on health are assessed, it becomes possible to quantify the health care expenditures related to such effects. These have been estimated at an order of magnitude similar to or even higher than the costs of mitigation (Pearce 2000 ; Ščasný et al. 2015 ; Schucht et al. 2015 ; West et al. 2013 ; Zhang et al. 2017 ). To the best of our knowledge, no other type of co-benefit from either mitigation or adaptation has been shown to out-weight mitigation costs. Air quality co-benefits from mitigation thus represent the ideal case to explore the role of co-benefits as a rationale for climate mitigation by local governments, whether, in particular, local governments exploit the large air quality co-benefits to make climate mitigation efforts more acceptable to local constituencies.

2.2 Co-benefits as a factor for local climate mitigation

Framed in terms of air quality and public health, co-benefits arguably provide a strong argument for local governments to make climate mitigation efforts more acceptable to local constituencies. If so, one would then expect an empirical link between air quality co-benefits and actual mitigation in cities. Empirical research does not confirm that. A large amount of literature is available that tries to link various aspects of urban climate action (mitigation or adaptation; planning rather than measures or actual achievements; the degree of vertical and horizontal integration, etc.) to a plurality of potential explanatory factors (Castán-Broto and Westman 2020 ; Heidrich et al. 2016 ; Hsu et al. 2020 ; Khan and Sovacool 2016 ; Kona et al. 2018 ; Reckien et al. 2018 ; Roggero et al. 2023 ; van der Heijden 2019 ). Results rather point at factors such as local government capacity (Araos et al. 2016 ; Bansard et al. 2017 ; Heikkinen et al. 2020 ; Krause et al. 2021 ; Reckien et al. 2018 ; Steffen et al. 2019 ), national policies (Boussalis et al. 2018 ; Domorenok 2019 ; Eisenack and Roggero 2022 ; Krause 2011 ; Lee and Koski 2014 ; van der Heijden et al. 2019 ), and membership in transnational municipal networks (Domorenok 2019 ; Eisenack and Roggero 2022 ; Heikkinen et al. 2019 ; Rashidi and Patt 2018 ; Steffen et al. 2019 ).

There is no similarly clear consensus on the role of co-benefits (air quality or otherwise) for local climate mitigation. Pitt ( 2010 ) surveys 255 US cities and finds a slightly positive correlation between non-attainment of air quality standards and the number of climate-related policies. For Rashidi and Patt ( 2018 ), air pollution is a weakly significant and positive factor for climate policy adoption in 127 cities reporting to the Carbon Disclosure Project (CDP). Pablo-Romero et al. ( 2015 ) use data from 7586 Spanish municipalities to show weakly significant, small effects of co-benefits on membership in the Global Covenant of Mayors (GCoM). Zhou et al. ( 2022 ) find specific patterns of local, mitigation-related policy instruments in Chinese cities with lower air quality. Other studies do not find a significant relation between air quality and climate mitigation by local governments (Krause 2011 ). Lack of adequate data is a common issue in the study of urban climate action (Gurney et al. 2021 ; Parvez et al. 2019 ) and may account for the above. Qualitative case studies, which are less sensitive to data quality issues, show similar findings. Zimmermann ( 2018 ) compares the emergence of climate policy in Munich, Stuttgart, and Frankfurt, Germany, and finds some role for co-benefits (in the form of energy savings) only in the case of Frankfurt. Jones ( 2012 , 2013 , 2018 ) provides in-depth comparative accounts of climate action in Stockholm, Copenhagen, Tokyo, New York City, Vancouver, and Melbourne. References to air quality and public health do not feature in any of them.

2.3 Wrap-up: analytical framework and operative research questions

We hereby focus on air quality co-benefits from mitigation. Given the above, two main arguments call for that. First, the widespread character of air quality co-benefits makes them suitable for cross-case analysis. Second, if the largest type of co-benefits can be ignored, all other co-benefits can be ignored too—providing lessons for co-benefits more generally. We thus move from the premise that poor air quality can fuel public health concerns within a local constituency, and that local governments might leverage that to make mitigation efforts more acceptable. Local constituencies, voting mayors in and out of office, may or may not perceive the direct benefits of local climate mitigation measures (that is, a reduced contribution to climate change) as worth the costs. However, they may be more inclined to do so if confronted with the additional public health benefits of mitigation, next to those from a reduced contribution to climate change. We may thus expect local governments to highlight such co-benefits when articulating their mitigation policies—that is, to exploit the “advocacy potential” of co-benefits as illustrated in Fig.  1 . Crucially, this should leave traces in the policy documents (plans) and broader narrative (news articles, assessments, reports) characterizing climate action in the three cities at stake.

Link between air quality and greenhouse gas emissions as implied by the co-benefits concept

It is an open question whether Moscow, Paris, and Montreal actually exploit the advocacy potential of air quality co-benefits. Literature has shown that local governments tend to ignore air quality co-benefits while formulating their mitigation strategies: institutional barriers, lack of technical or financial capacity, and lack of political will can hamper the consideration for co-benefits in urban climate action, however large they may be (Boyd et al. 2022 ; Dale et al. 2020 ; Gotgelf et al. 2020 ; Shimamoto and McCormick 2017 ; Ürge-Vorsatz et al. 2014 ). With this in mind, we analyze the policy documents articulating climate mitigation in Moscow, Paris, and Montreal; reconstruct the processes through which climate mitigation in the three cities unfolds over the years; and search for references to public health-related air quality considerations. This leads us to a set of operational research questions: First, do local governments exploit the advocacy potential of air quality co-benefits to raise acceptance for local climate mitigation action? Second, do local governments take air quality co-benefits into account in their mitigation policies and strategies? And third, does air quality represent a key factor for local climate mitigation?

3 Materials and methods

Our paper constitutes an in-depth case study of climate action in three cities. The rationale for a case study lies in the need for a better understanding of the processes at play: the possibility that a link between air quality co-benefits and mitigation by local governments does exist, but has so far gone undetected because some important elements of it are yet unknown. The case study method represents the method of choice in such a situation, being designed to address complex, poorly understood phenomena in their natural setting (Yin 1994 ). A well-designed case study achieves this by choosing cases that are paradigmatic of a particular phenomenon and by relying on a diversity of sources in order to contextualize evidence ( ibid. ). The rationale for a multiple case study rests on two considerations. First, the same phenomenon (here: the link between co-benefits and mitigation by local governments) may play out differently in different contexts. Observing multiple instances of the same phenomenon allows to better disentangle the role of context. Second, there may be qualitatively different (heterogeneous) processes linking co-benefits and mitigation. Multiple cases are a key precondition for observing heterogeneity. As a result, the three cases do not constitute counterfactuals of one another (as in a comparative case study) but rather address the same phenomenon in different contexts.

Case selection proceeded as follows. The case study method is geared towards analytical representativeness, not statistical representativeness (Bhattacherjee 2012 ; Yin 1994 ): cases need to be representative of a specific phenomenon (here: air quality co-benefits from local climate mitigation), not of a specific population (“all cities,” “all European cities,” “all cities with a green mayor”). To this aim, we chose Moscow, Paris, and Montreal in light of (1) their progress in climate mitigation and (2) their well-documented history of poor air quality. Furthermore, we opted for a “most similar” research design (Seawright and Gerring 2008 ), with the goal of ideally ruling out expected confounders. By design, all the three cities bear those attributes that are well-known factors for urban climate action: all represent the main center of their respective politico-administrative systems, enjoy a wide degree of autonomy and a supportive institutional environment, are members of transnational municipal networks, and can pursue a green growth agenda relying on a local economy that has successfully transitioned from industry to services, with a strong role of research and innovation. All three cities also face challenges linked to a growing population and the resulting congestion problems.

More importantly, the most similar, multiple case study research design also calls for a narrow focus on air quality co-benefits: it requires all case studies to address the same phenomenon in different contexts. Addressing multiple and/or different co-benefits in the three cities implies analyzing different processes: health co-benefits in Paris, energy security in Montreal, or international recognition in Moscow are very unlikely to unfold through the same processes. If processes differ, it becomes impossible to disentangle them from the contexts.

With this in mind, the case study draws on written sources to reconstruct mitigation from 1990 to present. These encompass official documents articulating local mitigation and air quality policies, a broader range of related policy documents (e.g., assessments, by-laws), journalistic sources, and research contributions (see Supplementary Materials for an overview). As usual in case study research, multiple sources were drawn upon for both analysis (that is, answering the research question) and contextualization (that is, taking additional, contextual information into account while interpreting the evidence for analysis). Documents addressing climate action and air quality were retrieved through internet searches until the saturation point was reached, that is, when additional documents stopped providing new elements to the narrative that emerged thus far. After consolidating a corpus of written documents for all three cities, a coherent narrative for each case was developed through a process of meaning condensation (Kvale and Brinkman 2015 ) based on the conceptual framework presented in Fig.  1 . Specifically, documents were systematically reviewed to determine whether and how arguments regarding the biophysical linkages between reducing GHG emissions and reducing air pollution (e.g., in terms of public health) were incorporated into strategies and policies of the respective local governments. The output of this process is presented in the following section, providing a qualitative account of climate mitigation in the three cities.

4.1 City of Moscow

Moscow is the main political, economic, and business center of the Russian Federation. With over 12 million residents and a metropolitan area of over 20 million, it is Russia’s only megacity and one of the very few megacities on the European continent next to Paris, London, and Istanbul. Over the last 20 years, Moscow’s economy has experienced a growth in the service sector, with particular emphasis on trade (wholesale and retail), real estate transactions (renting and service provision), and finance [mos_11]. Footnote 1 Featuring more than 100 public universities and a broad range of research and development centers, the city has an affinity with technology and innovation, further fueling its transition towards a full-service economy. Historically, however, the city has relied on manufacturing and heavy industries for its prosperity: mechanical engineering, food industry, metallurgy, chemical industry, and building materials industry still play a key role for the city’s economy today and have done so for the last 20 years. With its mix of industries and services, Moscow represents the economically most developed region in the country. However, it faces social inequality issues, mainly due to the shortcomings of local distribution policies [mos_06].

Air quality considerations in Moscow’s development are best understood in the context of the overall poor environmental conditions characterizing the city up to the 1990s. The heavy industry shaping Moscow’s economy during Soviet times and the reliance on private means of transport for the mobility of the increasing population left a legacy of extremely poor air quality [mos_02]. By then, air pollution had reached levels that authorities perceived as a threat to public health [mos_01]. The fall of the Soviet Union was followed by a drop in industrial production, providing some relief to Moscow’s air. Yet, air quality remained an issue: in the early days of the new Russian Federation, Moscow saw the introduction of improved traffic management, emission standards for motor vehicles, the electrification of public transport, and a monitoring system for air quality [mos_03]. Since 2010, Moscow’s new mayor has made improving urban transport and reducing traffic congestion, as well as the associated improvements in air quality, part of his political legacy [mos_10]. In the years that followed, air quality improved dramatically, making Moscow one of the cities with the cleanest air by international comparison [mos_12].

As of 2021, the city of Moscow does not have a stand-alone climate plan. Rather, climate considerations feature in both sectoral plans and in the city’s overall environmental planning—Moscow’s climate policy, in other words, is implicit in the city’s urban development processes. Climate change features explicitly in Moscow’s planning only from 2010 onward, and mainly in relation to adaptation, not mitigation [mos_04; mos_05]. Yet, a broad range of mitigation-relevant, greenhouse gas-reducing measures can be found in the city’s planning [mos_04; mos_05; mos_08], including the new “Environmental Strategy of Moscow until 2030” [mos_09]. These documents emphasize the electrification of the public transport system, the introduction of fuel standards, and the expansion of the city’s urban green. This way, green areas in the city reached 54.5% in 2014 and are expected to grow to 61% in 2030, while by 2021, greenhouse gas emissions have decreased 25% since 1990 [mos_12; mos_13].

In addition, since 2011, Moscow has been actively involved in Transnational Municipal Networks on climate change and international emission reporting activities such as, respectively, C40 and CDP [mos_09]. The Moscow government has made a priority of furthering international cooperation on climate action [mos_12]. While drawing benefits from the common infrastructure of city networks, the city could also achieve greater international recognition, raising its attractiveness as a site for investments and business. The development of the city of Moscow as a global international center has indeed been among the strategic objectives of the Government of the Russian Federation and the most important activities of the Moscow City Government. Environmental protection and greenhouse gas emission reductions are seen as functional to that aim [mos_07].

4.2 City of Paris

The capital and largest city of France, Paris, represents the country’s main economic, cultural, and administrative center. Arguably due to the centralized nature of the French state, the national government has invested heavily in the urban development of Paris and its surroundings (specifically, the Île de France region), managing the city directly through state officials until the 1990s. As a result, the Paris region hosts a large portion of the nation’s industrial development, including the headquarters and research centers of France’s largest firms [par_05]. This has led to urban sprawl, suburbanization, and the subsequent congestion problems related to the high number of commuters flowing in and out of the city every day, many of which by car [par_14]. The historical industrial development of the city has now transitioned to services (in particular finance and IT) and high-tech (electronics, optics, aerospace [par_12]). This has made Paris increasingly wealthy, although at a slower pace than cities in other advanced economies. The city still faces social inequality issues, making housing and access to basic services a regular item on the local government’s agenda [par_09; par_14].

Paris faces a severe air quality problem: values of PM 10 , PM 2.5 , and other pollutants (NO 2 ) are above legal thresholds and have already led to infraction procedures at EU level in 2015 and 2017. Air quality has improved since the 1990s but remains a major public health issue [par_13]: the 2014–2016 period has seen the “worst pollution in a decade” [par_07; par_08], not as severe as back in the 1990s, but enough to trigger temporary restrictions on car use [par_06]. Most importantly, air pollution is mostly emphasized in connection with the increasingly frequent, climate change-induced heatwaves. Heatwaves have played a key role in both French and Paris climate planning: the heatwave that hit Europe in 2003 caused 15,000 excess deaths in France alone [par_01; par_02]. An extensive stakeholder involvement process followed, captured in the “Livre Blanc des Parisiens sur le Climat” (the “White book of Parisians about the Climate” [par_03]), and paving the way for the city’s first community-wide climate plan (the “Plan Climat de Paris” [par_04]) to be updated every 5 years.

A clear continuity of objectives, themes, and approaches emerges from the various iterations of the Plan Climat: a holistic approach considering “the city as an ecosystem”; emphasis on energy production within the city; a strong role of environmental justice and broader social considerations; an emphasis on the economic benefits of the energy transition as a way to address inequality and social tensions in the city; and the “exemplary” role of the local administration both in relation to the local community and on the international stage. Against this background, the city aims at reducing emissions by a “factor 4” by 2050, corresponding to a 75% reduction against a 2004 baseline. The 2017 iteration of the plan [par_10] raises ambition further by aiming at carbon neutrality by 2050, which is understood as reducing energy consumption by 50%, and meeting the remaining energy needs through 100% renewables within the city limits and through compensation otherwise.

Within the Plan Climat, the above goes under the header of the “Energy Transition,” capturing the climate mitigation aspects of the plan. “Resilience” represents instead climate adaptation. Arguably in light of the events portrayed above, adaptation is no afterthought in the Plan Climat. Rather, it is a main thematic cluster, at the same level of mitigation. Most importantly, this distinction plays a role in relation to air quality: the Plan Climat puts much more emphasis on air quality in relation to adaptation than it does in relation to mitigation. Under “Energy Transition,” air quality is referenced (barely) in relation to mobility (“managing and limiting the role of cars” in the city). Under “Resilience,” it is articulated in terms of mobility, construction, heating, and waste. The rationale for that is that the lack of precipitation can significantly increase pollutant concentrations, amplifying the impact of heatwaves [par_10; par_13].

Furthermore, an increased attention to air quality and broader environmental health can be observed over time. Later iterations of the plan put increasing emphasis on air quality, paralleled by additional plans issued around this time: the “Environmental Health Plan” [par_11] and the “Resilience Strategy” [par_09]. The planning document most explicitly addressing the link between air quality and climate change is the “Plan Climat Air Énergie Métropolitain” [par_13], a plan produced at the level of Greater Paris Footnote 2 articulating air quality predominantly in adaptation terms: reducing pollution makes the city less vulnerable to climate impacts. Formally, air quality and emission reductions represent “converging objectives” (p. 269) to be tackled with the same cross-cutting measures. Yet, the document devotes considerably more space to air quality in relation to climate vulnerability than in the context of mobility and greenhouse gas emission reductions.

4.3 City of Montreal

A city of about two million within a metropolitan area of about four, the City of Montreal represents the main economic center of Quebec and the second most populous city in Canada. Its location on the Saint Lawrence River has made Montreal a logistics hub of international importance at the confluence of road, rail, and waterway infrastructure [mon_10]. Next to the port and the overall logistics and transportation sector, the city’s economy is driven by services and high-tech: aeronautics, bio-pharmaceutics, IT, and the media industry are among its most important employers [mon_03; mon_10]. The city’s socioeconomic structure represents an instance of the “doughnut effect” with high levels of unemployment in the mostly francophone city center, a wealthier middle class commuting into the city from the surrounding suburban areas (hence from outside the city’s tax base), and an impoverished rural hinterland [mon_11]. As a result, the City of Montreal is not affluent: it faces persistent problems of unemployment and poverty [mon_16; mon_08], while struggling with congestion and suburbanization [mon_14].

During 2005 UNFCCC Conference of the Parties (“COP 11”), taking place in Montreal and hosted by the Quebec government, Montreal’s local government signed the “World Mayors and Municipal Leaders Declaration on Climate Change” and pledged to reduce emissions by 30% by 2020 and 80% by 2050, both against 1990 levels. At the same time, it released its “First strategic plan for sustainable development of the Montreal community” [mon_01]. The plan lays down the key characteristics of Montreal’s climate governance to come: community and business mobilization, exemplary role of the administration, ecological transition as a way to stimulate the local industry. Crucially, air quality is presented as a key issue, both per se and with reference to climate mitigation. In the years that followed, four further plans have defined Montreal’s climate action: (1) the “Sustainable Development Plan 2010–2015” [mon_02]; (2) the “2013–2020 Citywide GHG Emission Reduction Strategy” [mon_04]; (3) “Sustainable Montreal 2016–2020” [mon_05]; and (4) the “2020–2030 Climate Plan” [mon_12]. In line with the First Strategic Plan, they all show a broad and encompassing approach, with emphasis on mobility electrification, energy efficiency in the housing sector, and (to a lower degree) circular economy. In terms of emission reduction targets, the plans consistently reflected the commitments made with the World Mayor Declaration above. Recently, the city has increased its ambition further, aiming at carbon neutrality by 2050 [mon_12].

Montreal achieved 24% greenhouse gas emission reductions against 1990 levels already in 2014 [mon_09]. Air quality, instead, remained relatively stable, with some improvements in terms of pollutant concentrations but stable counts of smog days per year [mon_13]. This detachment between greenhouse gas emissions and air quality trends is reflected in the climate plans. The link between air quality and greenhouse gas emissions is still present in the “Sustainable Development Plan 2010–2015” [mon_02]. It disappears thereafter: later plans justify the emphasis on energy efficiency in the housing sector and sustainable mobility entirely with the aim of reducing greenhouse gas emissions, without any reference to air quality considerations. Rather, reference is made to Quebec’s large hydropower capacities and the goal to become a global leader in sustainable mobility.

These aspirations combine with the issue of Québec’s autonomy and francophone identity: Québec has the highest levels of environmental consciousness [mon_06] in a country, Canada, that had no climate policy until 2015. The resulting policy void made climate change a topic in which Quebec could distinguish itself from the rest of the country, show affinity to European values, and join the world stage as a nation [mon_07]. This provided a favorable institutional environment for Montreal’s climate action. Parallel to that, air quality becomes an autonomous topic related to wood-burning—a culturally embedded and therefore politically sensitive practice linked to home heating (fireplaces) and commercial processes (e.g., bakeries). The local government has put conspicuous efforts in limiting wood-burning in the city [mon_13; mon_15] justified on air quality and subsequent health considerations, with no link to climate-related discourses (emission reductions, resource efficiency, etc.).

4.4 Comparison

Table 1 provides an overview of the main characteristics of the three cases. As expected in a “most similar” multiple case study design, the three cities show considerable similarities: all represent the main center of their politico-administrative systems, enjoy a wide degree of autonomy and a supportive institutional environment, are members of transnational municipal networks, and can pursue a green growth agenda relying on a local economy that has successfully transitioned from industry to services, with a strong role of research and innovation. All three cities face challenges linked to a growing population and the resulting congestion problems. Despite these similarities, the link between air quality and climate policies takes a quite different shape in each city.

Figure  2 provides a graphical representation of the links between public health, air quality, and emission reductions in the three cities. The three diagrams summarize the process emerging from the respective case description, focusing on those issues and policies that the documents at stake explicitly and directly link to the concepts of interest (public health, air quality, and emission reductions). The figure shows how the three local governments articulate the link between public health, air quality, and emission reductions very differently. In Moscow, air quality represents a key concern, driving those efforts that led to the city’s emission reductions—although that was not the main aim. Later on, these achievements were reframed and communicated as climate mitigation, providing the city the international recognition it aimed at. For Paris, a direct link between air quality and emission reductions is barely observed. Parisians, in a nutshell, did not reduce their greenhouse gas emissions because they value air quality; rather, they addressed air quality in order to be less vulnerable to heatwaves. In Montreal, finally, an initial link between air quality and climate mitigation faded out over the years: leadership in sustainable mobility took an increasingly prominent role in the city’s overall planning, but air quality considerations became a separate issue.

Link between air quality and greenhouse gas emissions as observed empirically in the three cases

5 Discussion

We can now turn to our research questions. First, do local governments exploit the advocacy potential of air quality co-benefits to make local climate mitigation more acceptable to the local constituencies? Moscow does so, but in an effort to present itself as an attractive global city, not to increase the acceptance of some emission reduction measures. Paris also does, but with way more emphasis in the context of adaptation than mitigation. Montreal used to do so, but stopped. Thus, none of the three cities really exploits the advocacy potential of air quality co-benefits to make local climate mitigation more acceptable to the local constituencies. Second, do local governments take air quality co-benefits into account in their mitigation policies and strategies? The answer is still negative, albeit with some qualifications. In Moscow, there is no actual climate mitigation to begin with: emission reductions are an unintended by-product of air quality efforts, re-packaged at a later stage as climate action for city-marketing purposes. In Paris, adaptation policies are those that really take air quality co-benefits into account, while mitigation policies do so to a much more limited extent. In Montreal, air quality and emission reductions represent separate issues.

More generally, the question was posed whether air quality co-benefits represent a key factor for climate mitigation by local governments. Despite the above, the answer here is (almost) yes. For Moscow, climate mitigation is an unintended by-product of air quality control efforts. As a result, air quality is the key factor driving local climate mitigation. Similar considerations hold for Paris, where climate adaptation triggered air quality measures, contributing to climate mitigation in the process. Here, we observe air quality as an adaptation co-benefit for mitigation, not air quality co-benefits as an argumentative rationale to promote emission reductions—a testament to the importance of adaptation co-benefits (more on that below). For Montreal, the efforts related to the wood-burning issue make it unlikely that the local government is blind to air quality co-benefits while reducing emissions. Hence, it must have reasons for not emphasizing that. Overall, a link between co-benefits and mitigation is observed empirically, but it is more complex and heterogeneous than implied by the literature.

The above results have implications for policy and academia, calling for a closer look at the potential limitations of our analysis. The absence of interview materials is worth highlighting. Interviews would have provided access to insights and perspectives that do not leave traces on written sources. Given the long time span of the processes at stake, though, ethnographic approaches would have been extremely resource intensive, beyond the resources available for this analysis. From this angle, focusing on a variety of written materials (from policy documents to journalistic sources) seems like a good compromise between accuracy, diversity, and resource intensity. A second limit relates to case selection, which relies on well-known drivers of urban climate action from the literature. This way, our analysis inherits the same limitations of the extant literature: a bias towards large cities in the Global North. To our defense, it would be difficult to disentangle the role of co-benefits for urban climate action in contexts (e.g., small cities in the Global South) where knowledge of urban climate action is scarce to begin with. Keeping those limitations in mind, three main lessons can be drawn from our case study.

First, the Moscow case raises questions on the understanding of co-benefits as selective incentives. Relying on selective incentives (air quality co-benefits) to explain the private provision of a public good (emission reductions) suggests the presence of strategic interactions, implying that actors value a certain public good, but are held back by the collective action problem. This does not seem to reflect the action situation in Moscow, where the local government values a certain private good (local air quality) and provides it accordingly. The fact that a by-product of doing so (emission reductions) has public good characteristics does not seem to credibly play a role in the decision-making—the more so in light of Moscow’s efforts to gain international recognition as a global city. In the absence of the strategic interaction, the link between air quality and climate mitigation is a mere positive externality.

Second, the Paris case questions the model of causality implicit in the concept of co-benefits. The key idea is that local government actions reflect a plurality of motives because of their multiple implications—which is plausible. Viewing such motives as “ancillary,” however, masks the possibility that such motives lead to action together (conjunctural causation). Consider the interplay of adaptation, air quality, and mitigation in Paris: air quality does not seem to play a role for mitigation, but rather in conjunction with adaptation. Our reading of the documents found no strong evidence of an explicit link between air quality co-benefits and mitigation in Paris, but only because it sought for a direct link between the two, excluding a priori the possibility of a more complex interaction. This is a shortcoming both per se and in light of the growing role of adaptation co-benefits and their synergies/trade-offs with mitigation (Luyten et al. 2023 ; Sharifi 2021 ).

Third, the Montreal case raises the question of strategically underplaying specific co-benefits, detaching them from mitigation because of potential controversies. Research has shown how, regardless of the public health benefits, addressing air quality issues may be politically costly, with effects on the type and ambition of local policies (Eckersley et al. 2023 ; Maltby 2022 ). If tackling air quality is unpopular among constituents, it won’t be effective as an additional motivation in support of mitigation. Montreal’s wood-burning issue is a point in case, but similar situations can be found elsewhere: the city of Milan is phasing out wood-fueled ovens in pizzerias (Comune di Milano 2020 ), while summer barbeques bans in Beijing and San Francisco (BAAQMD 2008 ; BBC 2014 ; The Guardian 2013 ) impose limits on culturally important practices for air quality reasons. It would not be surprising if future climate plans in these cities underplay air quality considerations, focusing instead on other co-benefits. By the same token, the advocacy potential of air quality co-benefits grows with the awareness of air quality issues among constituents, possibly opening avenues for a more systematic linkage between local climate mitigation and its air quality co-benefits.

The above has implications for policy. The same co-benefit can range from being the driving factor for local emission reductions (as in Moscow) to being a liability for local climate action (as in Montreal). If so, decision-makers at higher levels seeking to provide favorable enabling conditions for climate action by local governments should prioritize flexibility over standardization: not making the integration of specific co-benefits (e.g., air quality) into local climate plans mandatory, but providing local governments with the means (e.g., personnel, know-how, reliable data) to deliver on integrated climate planning overall, prioritizing the areas (and hence the co-benefits) they see as most appropriate in their context. Furthermore, transnational municipal networks could translate the diversity of processes through which co-benefits affect local climate action into dedicated city-matching processes (Kern et al. 2021 ), promoting knowledge sharing between local governments that do or intend to exploit the same co-benefit in similar contexts.

The above has also implications for research. Air quality did arguably play a role in each of the three cases, but the role changed in every instance, and never really reflected the concept of co-benefit portrayed in Fig.  1 , raising questions on its analytical merit. Several authors critique the emphasis on the advocacy potential of co-benefits (Mayrhofer and Gupta 2016 ; Workman et al. 2018 ) as a way of re-framing climate action in more positive terms, away from the trade-off between environment and economic development. Actual implementations of the concept, however, proved to be challenging (Mendez 2015 ; Shimamoto and McCormick 2017 ) and highly context-dependent (Slovic and Ribeiro 2018 ; Ürge-Vorsatz et al. 2014 ). Our cases are consistent with all that, showing in addition how political realities and contextual factors may lead to a substantial departure from the ideal type of air quality improvements as an “ancillary” benefit to local mitigation.

From a similar angle, critical contributions argue that the overly optimistic perspective underlying the co-benefits concept may mask the presence of trade-offs between mitigation and other policy areas (Grafakos et al. 2020 ; Mayrhofer and Gupta 2016 ; Newell et al. 2018 ), including adaptation (Eisenack 2016 ; Luyten et al. 2023 ; Sharifi 2021 ). The Montreal case brings that thought one step further: controversies may undermine the advocacy potential of a given co-benefit, regardless of actual biophysical trade-offs. In the end, co-benefits cover a much broader spectrum than economic and environmental considerations (Karlsson et al. 2020 ; Sovacool et al. 2020 )—and so do the trade-offs they may imply. Capacity considerations are often brought up as a reason for a sub-standard inclusion of co-benefits in real-world processes (Boyd et al. 2022 ; Dale et al. 2020 ; Mendez 2015 ). Moscow, Paris, and Montreal certainly do not lack the capacity for integrated planning, yet they show different levels in the emphasis they put on the link between air quality and greenhouse gas emission reductions, mirroring prior studies (Shimamoto and McCormick 2017 ) and raising the question whether the literature’s emphasis on capacity may mask political costs. Tellingly, Paris emphasizes the same co-benefits (air quality) in very different ways in different contexts (adaptation vs. mitigation), suggesting a conscious, strategic choice.

Finally, the complexity of the link between co-benefits and climate action calls for analyses that move away from single factors and rather explore patterns of multiple attributes recurring across cases. Doing so would serve the double purpose of better understanding how certain co-benefits interact with other contextual factors, and which configurations of factors (including possibly multiple co-benefits) are systematically associated with climate action. First empirical contributions are available that identify specific archetypical configurations of attributes recurring across cases (Eisenack and Roggero 2022 ; van der Heijden 2017 ). Future research could further expand the palette of configurations so far identified. To do so, it could take clues from the growing body of research on archetype analysis, discussing methodological options and conceptual implications of searching for recurring patterns across multiple cases in the context of socio-ecological research (Eisenack et al. 2019 , 2021 ; Sietz et al. 2019 ).

6 Conclusion

The extant research on urban climate action shows a large gap between co-benefits as an argumentative rationale for emission reductions and co-benefits as an empirical factor explaining actual mitigation in cities. Motivated by this gap, the present paper has focused on the link between air quality—the largest and most studied source of co-benefits according to the literature—and climate action in Moscow, Paris, and Montreal, three cities that faced air quality problems in the 1990s and have substantially reduced their greenhouse gas emissions since. A multiple case study was carried out, relying on the analysis of policy documents, research articles, and other written sources. The analysis showed a very different link between air quality and climate mitigation in the three cities: an unintended effect exploited for the sake of international recognition (Moscow); a central element of local adaptation, not mitigation (Paris); and a potential source of controversy (Montreal).

On the surface, the three local governments do not seem to bring up air quality co-benefits to justify local climate mitigation, nor do they seem to take co-benefits into account in their mitigation policies and strategies. Yet, we argue that air quality does play a key role in all three cases—just not the one the co-benefits literature would expect. The link between air quality and greenhouse gas emission reductions, in other words, appears more complex and heterogeneous than expected: the case studies are illustrative of how co-benefits may incorrectly cast a collective action perspective on specific action situations (Moscow), obfuscating interactions with other contextual factors (Paris), and masking important trade-offs (Montreal).

In light of this complexity and heterogeneity, it is no wonder that the extant literature has so far struggled to establish an empirical link between co-benefits and urban climate action. Concepts such as conjunctural causation and equifinality have only recently started to make their way into urban climate action research and socio-ecological research more in general. Embedding them into future research at both a conceptual and methodological level may provide a better understanding of both their interaction with contextual factors and the many, different ways they contribute (or not) to emission reductions.

Ultimately, the advocacy potential of the co-benefits concept—co-benefits as a rationale for urban climate action—is possibly only one of the many ways biophysical and technical interdependencies can lead to emission reductions. Urban climate action research and climate governance overall will benefit from a better understanding of these many ways. Failing to recognize this means putting the same square peg of co-benefits into the many round and oddly shaped holes of local politics and institutions—a likely frustrating experience that the climate community should rather avoid.

The labels in square brackets correspond to the different pieces of evidence used; see Supplementary Materials for more detail on each piece.

Greater Paris (“Grand Paris” in French) is an administrative level created in 2016 between the municipal level (including the City of Paris) and the district level (the “Île de France” district). It is governed by councilors representing the municipalities involved and thus not elected directly. For the purposes of this paper, it represents an arena in which the City of Paris can act rather than an autonomous level with its own political system. Similar considerations hold for the Montreal Agglomeration.

Andonova LB, Hale TN, Roger CB (2017) National policy and transnational governance of climate change: substitutes or complements? Int Stud Quart 61(2):253–268. https://doi.org/10.1093/isq/sqx014

Article   Google Scholar  

Araos M, Berrang-Ford L, Ford JD, Austin SE, Biesbroek R, Lesnikowski A (2016) Climate change adaptation planning in large cities: a systematic global assessment. Environ Sci Policy 66:375–382. https://doi.org/10.1016/j.envsci.2016.06.009

BAAQMD (2008) Wood Burning Rule. Bay area air quality management district. https://www.sparetheair.org/understanding-air-quality/wood-burning-rule . Accessed 1 Nov 2023

Bansard JS, Pattberg PH, Widerberg O (2017) Cities to the rescue? Assessing the performance of transnational municipal networks in global climate governance. Int Environ Agreements 17(2):229–246. https://doi.org/10.1007/s10784-016-9318-9

BBC (2014) China: outdoor grills banned in Beijing. https://www.bbc.com/news/blogs-news-from-elsewhere-27219976 . Accessed 1 Nov 2023

Bhattacherjee A (2012) Social science research: principles, methods, and practices. University of South Florida. https://digitalcommons.usf.edu/oa_textbooks/3/ . Accessed 1 Nov 2023

Boussalis C, Coan TG, Holman MR (2018) Climate change communication from cities in the USA. Clim Change 149(2):173–187. https://doi.org/10.1007/s10584-018-2223-1

Boyd D, Pathak M, van Diemen R, Skea J (2022) Mitigation co-benefits of climate change adaptation: a case-study analysis of eight cities. Sustain Cities Soc 77:103563. https://doi.org/10.1016/j.scs.2021.103563

Brody SD, Zahran S, Grover H, Vedlitz A (2008) A spatial analysis of local climate change policy in the United States: risk, stress, and opportunity. Landsc Urban Plan 87(1):33–41. https://doi.org/10.1016/j.landurbplan.2008.04.003

Castán-Broto V, Westman LK (2020) Ten years after Copenhagen: reimagining climate change governance in urban areas. Wiley Interdiscip Rev: Clim Change 11(4):e643. https://doi.org/10.1002/wcc.643

Comune di Milano (2020) Regolamento per la qualità dell’aria. Comune di Milano. https://www.comune.milano.it/documents/20126/44754/Regolamento+per+la+qualità+dell%27aria+da+pubblicare+nel+sito.pdf/904e47cb-ce27-b754-9d24-8ef298cdc2c2 . Accessed 1 Nov 2023

Dale A, Robinson J, King L, Burch S, Newell R, Shaw A, Jost F (2020) Meeting the climate change challenge: local government climate action in British Columbia, Canada. Clim Policy 20(7):866–880. https://doi.org/10.1080/14693062.2019.1651244

del P. Pablo‐Romero M, Sánchez‐Braza A, González‐Limón JM (2015) Covenant of mayors: reasons for being an environmentally and energy friendly municipality: covenant of mayors. Rev Policy Res 32(5):576–599. https://doi.org/10.1111/ropr.12135

Dodman D, Hayward B, Pelling M, Castán Broto V, Chow W, Dawson R, Khirfan L, McPhearson T, Prakash A, Zheng Y, Ziervogel G (2022) Chapter 6: Cities, settlements and key infrastructure. In: Climate change 2022: impacts, adaptation and vulnerability. Contribution of Working Group II to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press, pp 907–1040

Domorenok E (2019) Voluntary instruments for ambitious objectives? The experience of the EU Covenant of Mayors. Environ Polit 28(2):293–314. https://doi.org/10.1080/09644016.2019.1549777

Eckersley P, Harrison O, Poberezhskaya M (2023) A new framework to understand the drivers of policy mixes in multilevel contexts: the case of urban air pollution. Environ Policy Gov 33(2):178–190. https://doi.org/10.1002/eet.2010

EEA (2020) Air quality in Europe—2020 report—European Environment Agency (09/2020). European Environment Agency. https://www.eea.europa.eu/publications/air-quality-in-europe-2020-report . Accessed 1 Nov 2023

Eisenack K (2016) Institutional adaptation to cooling water scarcity in the electricity sector under global warming. Ecol Econ 124:153–163

Eisenack K, Roggero M (2022) Many roads to Paris: explaining urban climate action in 885 European cities. Glob Environ Chang 72:102439. https://doi.org/10.1016/j.gloenvcha.2021.102439

Eisenack K, Villamayor-Tomas S, Epstein G, Kimmich C, Magliocca NR, Manuel-Navarrete D, Oberlack C, Roggero M, Sietz D (2019) Design and quality criteria for archetype analysis. Ecol Soc 24(3):6. https://doi.org/10.5751/ES-10855-240306

Eisenack K, Oberlack C, Sietz D (2021) Avenues of archetype analysis: roots, achievements, and next steps in sustainability research. Ecol Soc 26(2):art31. https://doi.org/10.5751/ES-12484-260231

Floater G, Heeckt C, Ulterino M, Mackie L, Rode P, Bhardwaj A, Carvalho M, Gill D, Bailey T, Huxley R (2016) Co-benefits of urban climate action: a framework for cities [Monograph]. https://lsecities.net/ . Accessed 1 Nov 2023

Gotgelf A, Roggero M, Eisenack K (2020) Archetypical opportunities for water governance adaptation to climate change. Ecol Soc 25(5):6

Grafakos S, Viero G, Reckien D, Trigg K, Viguie V, Sudmant A, Graves C, Foley A, Heidrich O, Mirailles JM, Carter J, Chang LH, Nador C, Liseri M, Chelleri L, Orru H, Orru K, Aelenei R, Bilska A, … Dawson R (2020) Integration of mitigation and adaptation in urban climate change action plans in Europe: a systematic assessment. Renew Sustain Energy Rev 121:109623. https://doi.org/10.1016/j.rser.2019.109623

Gurney KR, Liang J, Roest G, Song Y, Mueller K, Lauvaux T (2021) Under-reporting of greenhouse gas emissions in U.S. cities. Nat Commun 12(1):553. https://doi.org/10.1038/s41467-020-20871-0

Hamilton I, Kennard H, McGushin A, Höglund-Isaksson L, Kiesewetter G, Lott M, Milner J, Purohit P, Rafaj P, Sharma R, Springmann M, Woodcock J, Watts N (2021) The public health implications of the Paris Agreement: a modelling study. Lancet Planet Health 5(2):e74–e83. https://doi.org/10.1016/S2542-5196(20)30249-7

Heidrich O, Reckien D, Olazabal M, Foley A, Salvia M, de Gregorio Hurtado S, Orru H, Flacke J, Geneletti D, Pietrapertosa F (2016) National climate policies across Europe and their impacts on cities strategies. J Environ Manage 168:36–45

Heikkinen M, Ylä-Anttila T, Juhola S (2019) Incremental, reformistic or transformational: What kind of change do C40 cities advocate to deal with climate change? J Environ Planning Policy Manage 21(1):90–103. https://doi.org/10.1080/1523908X.2018.1473151

Heikkinen M, Karimo A, Klein J, Juhola S, Ylä-Anttila T (2020) Transnational municipal networks and climate change adaptation: a study of 377 cities. J Clean Prod 257:120474. https://doi.org/10.1016/j.jclepro.2020.120474

Hennessey R, Pittman J, Morand A, Douglas A (2017) Co-benefits of integrating climate change adaptation and mitigation in the Canadian energy sector. Energy Policy 111:214–221. https://doi.org/10.1016/j.enpol.2017.09.025

Homsy GC (2018) Size, sustainability, and urban climate planning in a multilevel governance framework. In: Hughes S, Chu EK, Mason SG (eds) Climate Change in Cities. Springer, pp 19–38

Chapter   Google Scholar  

Hsu A, Tan J, Ng YM, Toh W, Vanda R, Goyal N (2020) Performance determinants show European cities are delivering on climate mitigation. Nat Clim Chang 10(11):1015–1022. https://doi.org/10.1038/s41558-020-0879-9

Izquierdo R, García Dos Santos S, Borge R, Paz DDL, Sarigiannis D, Gotti A, Boldo E (2020) Health impact assessment by the implementation of Madrid City air-quality plan in 2020. Environ Res 183:109021. https://doi.org/10.1016/j.envres.2019.109021

Jones S (2012) A tale of two cities: climate change policies in Vancouver and Melbourne - barometers of cooperative federalism?: climate change policies in Vancouver and Melbourne. Int J Urban Reg Res 36(6):1242–1267. https://doi.org/10.1111/j.1468-2427.2011.01083.x

Jones S (2013) Climate change policies of city governments in federal systems: an analysis of Vancouver, Melbourne and New York City. Reg Stud 47(6):974–992. https://doi.org/10.1080/00343404.2011.585150

Jones S (2018) Cities Responding to Climate Change. Springer Int Publ. https://doi.org/10.1007/978-3-319-64810-1

Karlsson M, Alfredsson E, Westling N (2020) Climate policy co-benefits: a review. Clim Policy 20(3):292–316. https://doi.org/10.1080/14693062.2020.1724070

Kern K, Grönholm S, Haupt W, Hopman L, Tynkkynen N, Kettunen P (2021) Matching forerunner cities: assessing Turku’s climate policy in comparison with Malmö, Groningen and Rostock. Publisher: City of Turku, Central administration, Turku Urban Research Programme. https://www.turku.fi/sites/default/files/atoms/files/kto-tutkimusraportteja_1-2021.pdf . Accessed 1 Nov 2023

Khan F, Sovacool BK (2016) Testing the efficacy of voluntary urban greenhouse gas emissions inventories. Clim Change 139(2):141–154. https://doi.org/10.1007/s10584-016-1793-z

Khomenko S, Cirach M, Pereira-Barboza E, Mueller N, Barrera-Gómez J, Rojas-Rueda D, De Hoogh K, Hoek G, Nieuwenhuijsen M (2021) Premature mortality due to air pollution in European cities: a health impact assessment. Lancet Planet Health 5(3):e121–e134. https://doi.org/10.1016/S2542-5196(20)30272-2

Kona A, Bertoldi P, Monforti-Ferrario F, Rivas S, Dallemand JF (2018) Covenant of mayors signatories leading the way towards 1.5 degree global warming pathway. Sustain Cities Soc 41:568–575. https://doi.org/10.1016/j.scs.2018.05.017

Kousky C, Schneider SH (2003) Global climate policy: will cities lead the way? Clim Policy 3(4):359–372. https://doi.org/10.1016/j.clipol.2003.08.002

Krause RM (2011) Symbolic or substantive policy? Measuring the extent of local commitment to climate protection. Eviron Plann C Gov Policy 29(1):46–62. https://doi.org/10.1068/c09185

Krause RM, Hawkins CV, Park AYS (2021) The perfect amount of help: an examination of the relationship between capacity and collaboration in urban energy and climate initiatives. Urban Aff Rev 57(2):583–608. https://doi.org/10.1177/1078087419884650

Kvale S, Brinkman S (2015) InterViews: learning the craft of qualitative interviewing, 3rd edn. Sage Publications, Thousand Oaks

Lee T, Koski C (2014) Mitigating global warming in global cities: comparing participation and climate change policies of C40 cities. J Comp Policy Anal: Res Pract 16(5):475–492. https://doi.org/10.1080/13876988.2014.910938

Luyten A, Winkler MS, Ammann P, Dietler D (2023) Health impact studies of climate change adaptation and mitigation measures – a scoping review. J Clim Change Health 9. Scopus. https://doi.org/10.1016/j.joclim.2022.100186

Maibach EW, Nisbet M, Baldwin P, Akerlof K, Diao G (2010) Reframing climate change as a public health issue: an exploratory study of public reactions. BMC Public Health 10(1):299. https://doi.org/10.1186/1471-2458-10-299

Maji KJ, Dikshit AK, Arora M, Deshpande A (2018) Estimating premature mortality attributable to PM2.5 exposure and benefit of air pollution control policies in China for 2020. Sci Total Environ 612:683–693. https://doi.org/10.1016/j.scitotenv.2017.08.254

Maltby T (2022) Consensus and entrepreneurship: the contrasting local and national politics of UK air pollution. Environ Plan c: Polit Space 40(3):685–704. https://doi.org/10.1177/2399654420981609

Mayrhofer JP, Gupta J (2016) The science and politics of co-benefits in climate policy. Environ Sci Policy 57:22–30. https://doi.org/10.1016/j.envsci.2015.11.005

Mendez MA (2015) Assessing local climate action plans for public health co-benefits in environmental justice communities. Local Environ 20(6):637–663. https://doi.org/10.1080/13549839.2015.1038227

Michaelowa K, Michaelowa A (2017) Transnational climate governance initiatives: designed for effective climate change mitigation? Int Interact 43(1):129–155. https://doi.org/10.1080/03050629.2017.1256110

Newell R, Dale A, Roseland M (2018) Climate action co-benefits and integrated community planning: uncovering the synergies and trade-offs. Int J Clim Chang: Impacts Response 10(4):1–23. https://doi.org/10.18848/1835-7156/CGP/v10i04/1-23

Nowak DJ, Hirabayashi S, Doyle M, McGovern M, Pasher J (2018) Air pollution removal by urban forests in Canada and its effect on air quality and human health. Urban For Urban Green 29:40–48. https://doi.org/10.1016/j.ufug.2017.10.019

Olson M (1965) The Logic of Collective Action. Harvard University Press

Google Scholar  

Parvez M, Hazelton J, James G (2019) Greenhouse gas emissions disclosure by cities: the expectation gap. Sustain Acc, Manag Policy J, SAMPJ-11–2017–0138. https://doi.org/10.1108/SAMPJ-11-2017-0138

Pearce DW (2000) Policy framework for the ancillary benefits of climate change policies. Ancillary benefits and costs of greenhouse gas mitigation. Centre for Social and Economic Research on the Global Environment, London

Pitt D (2010) The impact of internal and external characteristics on the adoption of climate mitigation policies by US municipalities. Eviron Plann C Gov Policy 28(5):851–871. https://doi.org/10.1068/c09175

Rashidi K, Patt A (2018) Subsistence over symbolism: the role of transnational municipal networks on cities’ climate policy innovation and adoption. Mitig Adapt Strat Glob Change 23(4):507–523. https://doi.org/10.1007/s11027-017-9747-y

Reckien D, Salvia M, Heidrich O, Church JM, Pietrapertosa F, De Gregorio-Hurtado S, D’Alonzo V, Foley A, Simoes SG, Krkoška Lorencová E, Orru H, Orru K, Wejs A, Flacke J, Olazabal M, Geneletti D, Feliu E, Vasilie S, Nador C, … Dawson R (2018) How are cities planning to respond to climate change? Assessment of local climate plans from 885 cities in the EU-28. J Clean Prod 191:207–219. https://doi.org/10.1016/j.jclepro.2018.03.220

Roggero M, Fjornes J, Eisenack K (2023) Ambitious climate targets and emission reductions in cities: a configurational analysis. Clim Policy 1–15. https://doi.org/10.1080/14693062.2023.2282488

Rübbelke DT (2002) International climate policy to combat global warming: an analysis of the ancillary benefits of reducing carbon emissions. Edward Elgar Publishing, Cheltenham

Sabel CE, Hiscock R, Asikainen A, Bi J, Depledge M, van den Elshout S, Friedrich R, Huang G, Hurley F, Jantunen M, Karakitsios SP, Keuken M, Kingham S, Kontoroupis P, Kuenzli N, Liu M, Martuzzi M, Morton K, Mudu P, … Willers S (2016). Public health impacts of city policies to reduce climate change: findings from the URGENCHE EU-China project. Environ Health 15(S1):S25. https://doi.org/10.1186/s12940-016-0097-0

Ščasný M, Massetti E, Melichar J, Carrara S (2015) Quantifying the ancillary benefits of the representative concentration pathways on air quality in Europe. Environ Resource Econ 62(2):383–415. https://doi.org/10.1007/s10640-015-9969-y

Schucht S, Colette A, Rao S, Holland M, Schöpp W, Kolp P, Klimont Z, Bessagnet B, Szopa S, Vautard R, Brignon J-M, Rouïl L (2015) Moving towards ambitious climate policies: monetised health benefits from improved air quality could offset mitigation costs in Europe. Environ Sci Policy 50:252–269. https://doi.org/10.1016/j.envsci.2015.03.001

Seawright J, Gerring J (2008) Case selection techniques in case study research: a menu of qualitative and quantitative options. Polit Res Q 61(2):294–308. https://doi.org/10.1177/1065912907313077

Sharifi A (2021) Co-benefits and synergies between urban climate change mitigation and adaptation measures: a literature review. Sci Total Environ 750:141642. https://doi.org/10.1016/j.scitotenv.2020.141642

Shimamoto MM, McCormick S (2017) The role of health in urban climate adaptation: an analysis of six U.S. cities. Weather, Clim, Soc 9(4):777–785. https://doi.org/10.1175/WCAS-D-16-0142.1

Sietz D, Frey U, Roggero M, Gong Y, Magliocca N, Tan R, Janssen P, Václavík T (2019) Archetype analysis in sustainability research: methodological portfolio and analytical frontiers. Ecol Soc 24(3). https://doi.org/10.5751/ES-11103-240334

Sippel M, Jenssen T (2009) What about local governance? A review of promise and problems. MPRA Paper No. 20987. https://mpra.ub.uni-muenchen.de/20987

Slovic AD, Ribeiro H (2018) Policy instruments surrounding urban air quality: the cases of São Paulo, New York City and Paris. Environ Sci Policy 81:1–9. https://doi.org/10.1016/j.envsci.2017.12.001

Sovacool BK, Martiskainen M, Hook A, Baker L (2020) Beyond cost and carbon: the multidimensional co-benefits of low carbon transitions in Europe. Ecol Econ 169:106529. https://doi.org/10.1016/j.ecolecon.2019.106529

Steffen B, Schmidt TS, Tautorat P (2019) Measuring whether municipal climate networks make a difference: the case of utility-scale solar PV investment in large global cities. Clim Policy 19(7):908–922. https://doi.org/10.1080/14693062.2019.1599804

Strosnider H (2017) Rural and urban differences in air quality, 2008–2012, and community drinking water quality, 2010–2015—United States. Morb Mortaliy Wkly Rep 66. https://doi.org/10.15585/mmwr.ss6613a1

The Guardian (2013) Beijing confiscates barbecues in drive to cut air pollution. The Guardian. https://www.theguardian.com/world/2013/nov/27/china-barbecues-air-pollution . Accessed 1 Nov 2023

Ürge-Vorsatz D, Herrero ST, Dubash NK, Lecocq F (2014) Measuring the co-benefits of climate change mitigation. Annu Rev Environ Resour 39(1):549–582. https://doi.org/10.1146/annurev-environ-031312-125456

van der Heijden J (2019) Studying urban climate governance: where to begin, what to look for, and how to make a meaningful contribution to scholarship and practice. Earth Syst Gov 1:100005. https://doi.org/10.1016/j.esg.2019.100005

van der Heijden J, Bulkeley H, Certomà C (Eds.) (2019) Urban climate politics: agency and empowerment (1st ed.). Cambridge University Press. https://doi.org/10.1017/9781108632157

van der Heijden J (2017) Innovations in urban climate governance: voluntary programs for low-carbon buildings and cities (1st ed.). Cambridge University Press. https://doi.org/10.1017/9781108233293

West JJ, Smith SJ, Silva RA, Naik V, Zhang Y, Adelman Z, Fry MM, Anenberg S, Horowitz LW, Lamarque J-F (2013) Co-benefits of mitigating global greenhouse gas emissions for future air quality and human health. Nat Clim Chang 3(10):885–889. https://doi.org/10.1038/nclimate2009

WHO (2023) World health statistics 2023: monitoring health for the SDGs. World Health Organization, Sustainable Development Goals

Workman A, Blashki G, Bowen K, Karoly D, Wiseman J (2018) The political economy of health co-benefits: embedding health in the climate change agenda. Int J Environ Res Public Health 15(4):674. https://doi.org/10.3390/ijerph15040674

Yin RK (1994) Case study research: design and methods, 2nd edn. Sage Publications, Thousand Oaks

Zhang Y, Smith SJ, Bowden JH, Adelman Z, West JJ (2017) Co-benefits of global, domestic, and sectoral greenhouse gas mitigation for US air quality and human health in 2050. Environ Res Lett 12(11):114033. https://doi.org/10.1088/1748-9326/aa8f76

Zhou B, Wang Q, Zhang C (2022) Central–local governance gaps: the evolving differentiation of climate policies in China. Sustain Sci. https://doi.org/10.1007/s11625-022-01132-z

Zimmermann K (2018) Local climate policies in Germany. Challenges of governance and knowledge. Cogent Soc Sci 4(1):1482985. https://doi.org/10.1080/23311886.2018.1482985

Download references

Open Access funding enabled and organized by Projekt DEAL.

Author information

Authors and affiliations.

Resource Economics Group, Humboldt University of Berlin, Unter Den Linden 6, 10099, Berlin, Germany

Matteo Roggero, Anastasiia Gotgelf & Klaus Eisenack

You can also search for this author in PubMed   Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation, data collection, and analysis were performed by MR, AG, and KE. The first draft of the manuscript was written by MR and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Matteo Roggero .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 176 KB)

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Roggero, M., Gotgelf, A. & Eisenack, K. Co-benefits as a rationale and co-benefits as a factor for urban climate action: linking air quality and emission reductions in Moscow, Paris, and Montreal. Climatic Change 176 , 179 (2023). https://doi.org/10.1007/s10584-023-03662-6

Download citation

Received : 17 January 2023

Accepted : 05 December 2023

Published : 14 December 2023

DOI : https://doi.org/10.1007/s10584-023-03662-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Urban climate governance
• Climate mitigation
• Climate adaptation
• Find a journal
• Publish with us
• Track your research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• v.112(1); Jan-Feb 2015

Systematically Reviewing the Literature: Building the Evidence for Health Care Quality

There are important research and non-research reasons to systematically review the literature. This article describes a step-by-step process to systematically review the literature along with links to key resources. An example of a graduate program using systematic literature reviews to link research and quality improvement practices is also provided.

Introduction

Systematic reviews that summarize the available information on a topic are an important part of evidence-based health care. There are both research and non-research reasons for undertaking a literature review. It is important to systematically review the literature when one would like to justify the need for a study, to update personal knowledge and practice, to evaluate current practices, to develop and update guidelines for practice, and to develop work related policies. 1 A systematic review draws upon the best health services research principles and methods to address: What is the state of the evidence on the selected topic? The systematic process enables others to reproduce the methods and to make a rational determination of whether to accept the results of the review. An abundance of articles on systematic reviews exist focusing on different aspects of systematic reviews. 2 – 9 The purpose of this article is to describe a step by step process of systematically reviewing the health care literature and provide links to key resources.

Systematic Review Process: Six Key Steps

Six key steps to systematically review the literature are outlined in Table 1 and discussed here.

Systematic Review Steps

1. Formulate the Question and Refine the Topic

When preparing a topic to conduct a systematic review, it is important to ask at the outset, “What exactly am I looking for?” Hopefully it seems like an obvious step, but explicitly writing a one or two sentence statement of the topic before you begin to search is often overlooked. It is important for several reasons; in particular because, although we usually think we know what we are searching for, in truth our mental image of a topic is often quite fuzzy. The act of writing something concise and intelligible to a reader, even if you are the only one who will read it, clarifies your thoughts and can inspire you to ask key questions. In addition, in subsequent steps of the review process, when you begin to develop a strategy for searching the literature, your topic statement is the ready raw material from which you can extract the key concepts and terminology for your strategies. The medical and related health literature is massive, so the more precise and specific your understanding of your information need, the better your results will be when you search.

2. Search, Retrieve, and Select Relevant Articles

The retrieval tools chosen to search the literature should be determined by the purpose of the search. Questions to ask include: For what and by whom will the information be used? A topical expert or a novice? Am I looking for a simple fact? A comprehensive overview on the topic? Exploration of a new topic? A systematic review? For the purpose of a systematic review of journal research in the area of health care, PubMed or Medline is the most appropriate retrieval tool to start with, however other databases may be useful ( Table 2 ). In particular, Google Scholar allows one to search the same set of articles as PubMed/MEDLINE, in addition to some from other disciplines, but it lacks a number of key advanced search features that a skilled searcher can exploit in PubMed/MEDLINE.

Examples of Electronic Bibliographic Databases Specific to Health Care

Note: These databases may be available through university or hospital library systems.

An effective way to search the literature is to break the topic into different “building blocks.” The building blocks approach is the most systematic and works the best in periodical databases such as PubMed/MEDLINE. The “blocks” in a “building blocks” strategy consist of the key concepts in the search topic. For example, let’s say we are interested in researching about mobile phone-based interventions for monitoring of patient status or disease management. We could break the topic into the following concepts or blocks: 1. Mobile phones, 2. patient monitoring, and 3. Disease management. Gather synonyms and related terms to represent each concept and match to available subject headings in databases that offer them. Organize the resulting concepts into individual queries. Run the queries and examine your results to find relevant items and suggest query modifications to improve your results. Revise and re-run your strategy based on your observations. Repeat this process until you are satisfied or further modifications produce no improvements. For example in Medline, these terms would be used in this search and combined as follows: cellular phone AND (ambulatory monitoring OR disease management), where each of the key word phrases is an official subject heading in the MEDLINE vocabulary. Keep detailed notes on the literature search, as it will need to be reported in the methods section of the systematic review paper. Careful noting of search strategies also allows you to revisit a topic in the future and confidently replicate the same results, with the addition of those subsequently published on your topic.

3. Assess Quality

There is no consensus on the best way to assess study quality. Many quality assessment tools include issues such as: appropriateness of study design to the research objective, risk of bias, generalizability, statistical issues, quality of the intervention, and quality of reporting. Reporting guidelines for most literature types are available at the EQUATOR Network website ( http://www.equator-network.org/ ). These guidelines are a useful starting point; however they should not be used for assessing study quality.

4. Extract Data and Information

Extract information from each eligible article into a standardized format to permit the findings to be summarized. This will involve building one or more tables. When making tables each row should represent an article and each column a variable. Not all of the information that is extracted into the tables will end up in the paper. All of the information that is extracted from the eligible articles will help you obtain an overview of the topic, however you will want to reserve the use of tables in the literature review paper for the more complex information. All tables should be introduced and discussed in the narrative of the literature review. An example of an evidence summary table is presented in Table 3 .

Example of an evidence summary table

Notes: BP = blood pressure, HbA1c = Hemoglobin A1c, Hypo = hypoglycemic, I = Internet, NS = not significant, PDA = personal digital assistant, QOL = quality of life, SMBG = self-monitored blood glucose, SMS = short message service, V = voice

5. Analyze and Synthesize Data and information

The findings from individual studies are analyzed and synthesized so that the overall effectiveness of the intervention can be determined. It should also be observed at this time if the effect of an intervention is comparable in different studies, participants, and settings.

6. Write the Systematic Review

The PRISMA 12 and ENTREQ 13 checklists can be useful resources when writing a systematic review. These uniform reporting tools focus on how to write coherent and comprehensive reviews that facilitate readers and reviewers in evaluating the relative strengths and weaknesses. A systematic literature review has the same structure as an original research article:

TITLE : The systematic review title should indicate the content. The title should reflect the research question, however it should be a statement and not a question. The research question and the title should have similar key words.

STRUCTURED ABSTRACT: The structured abstract recaps the background, methods, results and conclusion in usually 250 words or less.

INTRODUCTION: The introduction summarizes the topic or problem and specifies the practical significance for the systematic review. The first paragraph or two of the paper should capture the attention of the reader. It might be dramatic, statistical, or descriptive, but above all, it should be interesting and very relevant to the research question. The topic or problem is linked with earlier research through previous attempts to solve the problem. Gaps in the literature regarding research and practice should also be noted. The final sentence of the introduction should clearly state the purpose of the systematic review.

METHODS: The methods provide a specification of the study protocol with enough information so that others can reproduce the results. It is important to include information on the:

• Eligibility criteria for studies: Who are the patients or subjects? What are the study characteristics, interventions, and outcomes? Were there language restrictions?
• Literature search: What databases were searched? Which key search terms were used? Which years were searched?
• Study selection: What was the study selection method? Was the title screened first, followed by the abstract, and finally the full text of the article?
• Data extraction: What data and information will be extracted from the articles?
• Data analysis: What are the statistical methods for handling any quantitative data?

RESULTS: The results should also be well-organized. One way to approach the results is to include information on the:

• Search results: What are the numbers of articles identified, excluded, and ultimately eligible?
• Study characteristics: What are the type and number of subjects? What are the methodological features of the studies?
• Study quality score: What is the overall quality of included studies? Does the quality of the included studies affect the outcome of the results?
• Results of the study: What are the overall results and outcomes? Could the literature be divided into themes or categories?

DISCUSSION: The discussion begins with a nonnumeric summary of the results. Next, gaps in the literature as well as limitations of the included articles are discussed with respect to the impact that they have on the reliability of the results. The final paragraph provides conclusions as well as implications for future research and current practice. For example, questions for future research on this topic are revealed, as well as whether or not practice should change as a result of the review.

REFERENCES: A complete bibliographical list of all journal articles, reports, books, and other media referred to in the systematic review should be included at the end of the paper. Referencing software can facilitate the compilation of citations and is useful in terms of ensuring the reference list is accurate and complete.

The following resources may be helpful when writing a systematic review:

CEBM: Centre for Evidence-based Medicine. Dedicated to the practice, teaching and dissemination of high quality evidence based medicine to improve health care Available at: http://www.cebm.net/ .

CITING MEDICINE: The National Library of Medicine Style Guide for Authors, Editors, and Publishers. This resource provides guidance in compiling, revising, formatting, and setting reference standards. Available at http://www.ncbi.nlm.nih.gov/books/NBK7265/ .

EQUATOR NETWORK: Enhancing the QUAlity and Transparency Of health Research. The EQUATOR Network promotes the transparent and accurate reporting of research studies. Available at: http://www.equator-network.org/ .

ICMJE RECOMMENDATIONS: International Committee of Medical Journal Editors Recommendations for the Conduct, Reporting, Editing and Publication of Scholarly Work in Medical Journals. The ICJME recommendations are followed by a large number of journals. Available at: http://www.icmje.org/about-icmje/faqs/icmje-recommendations/ .

PRISMA STATEMENT: Preferred Reporting Items for Systematic Reviews and Meta-Analyses. Authors can utilize the PRISMA Statement checklist to improve the reporting of systematic reviews and meta-analyses. Available at: http://prisma-statement.org .

THE COCHRANE COLLABORATION: A reliable source for making evidence generated through research useful for informing decisions about health. Available at: http://www.cochrane.org/ .

Examples of Systematic Reviews To Link Research and Quality Improvement

Over the past 17 years more than 300 learners, including physicians, nurses, and health administrators have completed a course as part of a Master of Health Administration or a Master of Science in Health Informatics degree at the University of Missouri. An objective of the course is to educate health informatics and health administration professionals about how to utilize a systematic, scientific, and evidence-based approach to literature searching, appraisal, and synthesis. Learners in the course conduct a systematic review of the literature on a health care topic of their choosing that could suggest quality improvement in their organization. Students select topics that make sense in terms of their core educational competencies and are related to their work. The categories of topics include public health, leadership, information management, health information technology, electronic medical records, telehealth, patient/clinician safety, treatment/screening evaluation cost/finance, human resources, planning and marketing, supply chain, education/training, policies and regulations, access, and satisfaction. Some learners have published their systematic literature reviews 14 – 15 . Qualitative comments from the students indicate that the course is well received and the skills learned in the course are applicable to a variety of health care settings.

Undertaking a literature review includes identification of a topic of interest, searching and retrieving the appropriate literature, assessing quality, extracting data and information, analyzing and synthesizing the findings, and writing a report. A structured step-by-step approach facilitates the development of a complete and informed literature review.

Suzanne Austin Boren, PhD, MHA, (above) is Associate Professor and Director of Academic Programs, and David Moxley, MLIS, is Clinical Instructor and Associate Director of Executive Programs. Both are in the Department of Health Management and Informatics at the University of Missouri School of Medicine.

Contact: ude.iruossim.htlaeh@snerob

None reported.

This paper is in the following e-collection/theme issue:

Published on 11.4.2024 in Vol 26 (2024)

This is a member publication of Imperial College London (Jisc)

Regulatory Standards and Guidance for the Use of Health Apps for Self-Management in Sub-Saharan Africa: Scoping Review

Authors of this article:

• Benard Ayaka Bene 1, 2 , MBBS, MPH   ; 
• Sunny Ibeneme 3 , MD, PhD   ; 
• Kayode Philip Fadahunsi 1 , MBBS, MPH   ; 
• Bala Isa Harri 4 , MBBS, MPH, MSc   ; 
• Nkiruka Ukor 5 , MSc   ; 
• Nikolaos Mastellos 1 , BSc, PhD   ; 
• Azeem Majeed 1 , MD   ; 
• Josip Car 1, 6 , MSc, MD, PhD  

1 Department of Primary Care and Public Health, School of Public Health, Imperial College London, London, United Kingdom

2 Department of Public Health, Federal Ministry of Health, Abuja, Nigeria

3 Digital Health Specialist, UNICEF East Asia Pacific Regional Office, Bangkok, Thailand

4 Department of Health Planning, Research and Statistics, Federal Ministry of Health, Abuja, Nigeria

5 Strategic Health Information Cluster, World Health Organization, Abuja, Nigeria

6 School of Life Course & Population Sciences, King’s College London, London, United Kingdom

Corresponding Author:

Benard Ayaka Bene, MBBS, MPH

Department of Primary Care and Public Health

School of Public Health

Imperial College London

The Reynolds Building

St Dunstan’s Road

London, W6 8RP

United Kingdom

Phone: 44 7598439185

Email: [email protected]

Background: Health apps are increasingly recognized as crucial tools for enhancing health care delivery. Many countries, particularly those in sub-Saharan Africa, can substantially benefit from using health apps to support self-management and thus help to achieve universal health coverage and the third sustainable development goal. However, most health apps published in app stores are of unknown or poor quality, which poses a risk to patient safety. Regulatory standards and guidance can help address this risk and promote patient safety.

Objective: This review aims to assess the regulatory standards and guidance for health apps supporting evidence-based best practices in sub-Saharan Africa with a focus on self-management.

Methods: A methodological framework for scoping reviews was applied. A search strategy was built and applied across the following databases, gray literature sources, and institutional websites: PubMed, Scopus, World Health Organization (WHO) African Index Medicus, OpenGrey, WHO Regional Office for Africa Library, ICTworks, WHO Directory of eHealth policies, HIS Strengthening Resource Center, International Telecommunication Union, Ministry of Health websites, and Google. The search covered the period between January 2005 and January 2024. The findings were analyzed using a deductive descriptive content analysis. The policy analysis framework was adapted and used to organize the findings. The Reporting Items for Stakeholder Analysis tool guided the identification and mapping of key stakeholders based on their roles in regulating health apps for self-management.

Results: The study included 49 documents from 31 sub-Saharan African countries. While all the documents were relevant for stakeholder identification and mapping, only 3 regulatory standards and guidance contained relevant information on regulation of health apps. These standards and guidance primarily aimed to build mutual trust; promote integration, inclusion, and equitable access to services; and address implementation issues and poor coordination. They provided guidance on systems quality, software acquisition and maintenance, security measures, data exchange, interoperability and integration, involvement of relevant stakeholders, and equitable access to services. To enhance implementation, the standards highlight that legal authority, coordination of activities, building capacity, and monitoring and evaluation are required. A number of stakeholders, including governments, regulatory bodies, funders, intergovernmental and nongovernmental organizations, academia, and the health care community, were identified to play key roles in regulating health apps.

Conclusions: Health apps have huge potential to support self-management in sub-Saharan Africa, but the lack of regulatory standards and guidance constitutes a major barrier. Hence, for these apps to be safely and effectively integrated into health care, more attention should be given to regulation. Learning from countries with effective regulations can help sub-Saharan Africa build a more robust and responsive regulatory system, ensuring the safe and beneficial use of health apps across the region.

International Registered Report Identifier (IRRID): RR2-10.1136/bmjopen-2018-025714

Introduction

Health apps are the most widely used digital health products globally [ 1 , 2 ]. Harnessing the potential of health apps creates a huge opportunity in providing support for health care delivery, including patient communication, patient education, and decision support for self-management [ 3 - 8 ]. Health apps can be an effective tool to strengthen health systems worldwide, especially in low- and middle-income countries including those in sub-Saharan Africa [ 4 , 5 , 9 ]. As a result, the attainment of universal health coverage (UHC) and sustainable development goal (SDG) 3, good health and well-being, can be accelerated [ 8 , 10 ].

Many health apps fall below the expected quality threshold [ 11 ]. Several studies have found that widely used health apps are often technically unreliable and clinically unsafe [ 12 - 14 ] and do not comply with ethical standards and the principles of confidentiality of information and data privacy [ 15 , 16 ]. In addition, many commercially available health apps were not developed using interoperability standards that are widely accepted in sub-Saharan Africa (eg, Fast Healthcare Interoperability Resources [FHIR]) [ 17 - 20 ]. Consequently, it becomes difficult to integrate these apps into a clinical workflow.

Hence, regulation through robust mechanisms is crucial to enhance the development, implementation, and adoption of health apps. Regulatory standards and guidance are essential for the safety of patients as they ensure quality assurance of any new technology in health care and contribute to building mutual trust while promoting the optimal use of the technology [ 21 - 23 ]. Therefore, to ensure that health apps that are used to support the self-management of patients are technically reliable and clinically safe, interoperable across systems, and compliant with the principles of confidentiality of information and data privacy, there is a need for effective regulatory standards. Furthermore, effective regulation can help ensure that health apps for self-management are culturally functional and competent and are accessible to those who need them regardless of gender, ethnicity, geographical location, or financial status [ 24 - 31 ].

Since 2005, there have been ongoing efforts to strengthen digital health governance at both the national and international levels [ 32 , 33 ]. In 2018, the World Health Organization (WHO) member states renewed their commitment to using digital health technologies (DHTs) to advance UHC and SDG 3 [ 33 ]. However, to date, the extent to which the use of health apps for self-management is regulated across countries within the WHO African Region (also known as sub-Saharan Africa) remains unclear. Therefore, this review was conducted to identify available regulatory standards and guidance and assess the extent to which they regulate health apps for self-management in sub-Saharan Africa. The review also mapped out the key stakeholders and their roles in regulating health apps for self-management across sub-Saharan Africa.

Review Questions

The review attempted to answer the following questions: (1) What regulatory standards and guidance are available for regulating health apps for self-management across sub-Saharan Africa? (2) To what extent do regulatory standards and guidance regulate health apps for self-management in terms of what aspects are regulated; why, how, and for whom; and what aspects are not regulated? (3) Who are the key stakeholders and what are their roles in regulating health apps for self-management?

Study Design

The process of this scoping review followed the methodological framework for conducting a scoping study originally described by Arksey and O’Malley [ 34 ] and the updated methodological guidance for conducting a Joanna Briggs Institute scoping review [ 34 - 37 ]. The reporting of the review was guided by the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist [ 38 ]. A completed PRISMA-ScR checklist is provided in Multimedia Appendix 1 . The protocol of this scoping review was published in BMJ Open [ 30 ].

Identifying Relevant Documents

Two reviewers (BAB and SI) developed the search strategy with the assistance of a librarian and in consultation with other research team members (KPF, BIH, NU, NM, AM, and JC). The following key terms were included: policy, legislation, strategy, regulation, standard, criterion, framework, guidance, guideline, digital health, eHealth, app, WHO African Region, and sub-Saharan Africa, and the names of all sub-Saharan African countries.

Owing to the absence of regulatory standards and guidance in scientific databases, the search focus was narrowed down to gray literature sources and institutional websites, including OpenGrey, WHO Regional Office for Africa (AFRO) Library, repositories for digital health policies (ICTworks, WHO’s Directory of eHealth Policies, and Health Information System Strengthening Resource Center), as well as the websites of WHO, International Telecommunication Union (ITU), and Ministries of Health (MOHs). The only scientific databases searched were PubMed, Scopus, and WHO AIM. PubMed was not included in the protocol. We also conducted a systematic search on Google. We used truncation to increase the yield of the results. The search strategy was then applied across PubMed, Scopus, and WHO AIM databases using Boolean terms (mainly OR and AND ) to combine search results. Gray literature sources and institutional websites were searched using phrases containing ≥2 keywords such as “eHealth regulation,” “digital health regulatory standard,” “eHealth regulatory standard,” “digital health regulation,” “digital health policy,” “eHealth policy,” “digital health strategy,” and “eHealth strategy.” For Google search, we added the names of the country to the phrases (eg, “digital health regulation Nigeria”). The reference lists of the included documents were also searched, and key individuals at the MOHs, WHO Country Offices, and the WHO AFRO were contacted for related documents. When our search was conducted, the WHO Directory of eHealth policies website was unavailable, and the WHO AFRO Library was undergoing reconstruction. The search strategies for PubMed, Scopus, and WHO AIM are provided in Multimedia Appendix 2 . The search was conducted between 2005 and January 2024.

Study Selection

The search results obtained from PubMed, Scopus, and WHO AIM were imported into Mendeley (Elsevier) [ 39 ] to remove duplicates. The search conducted on OpenGrey did not yield any results, whereas relevant records obtained from institutional websites, repositories, and Google were downloaded as PDF copies and uploaded to Mendeley. After removing duplicates, the remaining results were imported into Covidence (Veritas Health Innovation) [ 40 ] for screening. Two reviewers (BAB and SI) applied the predefined eligibility criteria ( Textbox 1 ) to screen the documents in 2 stages (title and abstract or executive summary). All discrepancies were discussed until the reviewers reached agreement.

Inclusion criteria

• Type of document: Regulatory standards, guidance, policies, strategies, and committee or government reports that address regulatory issues related to the use of health apps for self-management
• Location: Documents developed and implemented in countries within sub-Saharan Africa
• Date of publication: Documents developed since 2005; the global efforts toward promoting standards to minimize variability and potential harms that could arise from poorly regulated use of digital health began in 2005 [ 33 ]
• Language: Documents written in English language and other official languages of sub-Saharan African countries (Portuguese and French)

Exclusion criteria

• Type of document: Standards, guidance, policies, strategies, and reports not related to regulation of health apps
• Location: Documents from countries outside sub-Saharan Africa
• Date of publication: Documents developed before 2005
• Language: None

Data Charting (Extraction)

Two reviewers (BAB and SI), in consultation with the other members of the research team, developed the data extraction forms using an iterative process that included piloting data extraction and refinement until a consensus was reached.

We proposed in the study protocol [ 30 ] that data extraction would be conducted by the 2 reviewers independently. However, owing to the approach adopted for data extraction (deductive qualitative content analysis), 1 reviewer, rather than 2, initially extracted data from the included documents, and any concerns were discussed with a second reviewer [ 41 ]. Unresolved issues were then discussed and resolved with a third reviewer in a steering group meeting.

Collating, Summarizing, and Reporting Results

To address the research questions (particularly question 2), we adopted a deductive descriptive qualitative content analysis method to analyze and report the key findings. The policy analysis framework by Walt and Gilson [ 42 ] was adapted and applied to ensure that there was a consistent way of organizing the key findings: (1) Content (which aspects are regulated and which aspects are not?)—these are the components that directly or indirectly address regulatory issues related to the use of health apps for self-management, including areas that have not been addressed. (2) Context (why are those aspects regulated?)—this characterizes the rationale indicated for addressing regulatory issues related to the use of health apps for self-management. (3) Process (how are the regulatory standards developed and implemented?)—this describes the methods or approaches used to develop and implement regulatory standards. (4) Actors (who are the regulatory standards targeted toward?)—these are the key actors targeted by the standards.

Using a deductive descriptive qualitative content analysis approach, we examined each included document to systematically identify texts for concepts, patterns, and other relevant information. We then categorized them under content, context, process, or actors in relation to regulating health apps for self-management. The findings under content and context were further organized based on 4 predefined regulatory categories or themes as documented in the literature, namely (1) technical and clinical safety [ 12 - 14 ], (2) data protection and security [ 15 , 16 ], (3) standards and interoperability [ 28 , 31 ], and (4) inclusion and equitable access [ 24 - 29 ].

To address the third research question, the Reporting Items for Stakeholder Analysis (RISA) tool [ 41 ] was used as a guide to group key stakeholders based on role categorization as recognized globally by the WHO, the ITU, and UNESCO [ 32 , 33 , 43 ].

Ethical Considerations

Primary data were not collected in this study. Therefore, no ethics approval was required.

Search Results

A total of 2900 records were obtained after removing duplicates. Although the literature search was conducted in English, the search also yielded documents written in French and Portuguese from the ICTworks repository [ 44 ]. Following the initial screening of the title and abstract (or executive summaries), 73 documents were retrieved for full-text assessment. After applying the inclusion criteria for the full-text assessment, 49 documents were found eligible for inclusion in the review.

The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram [ 45 ] showing the study selection process is presented in Figure 1 .

Types of Documents

On the basis of the inclusion criteria, 3 categories of documents were considered for this review, namely “stand-alone regulatory standards and guidance that potentially regulate health apps for self-management,” “national policies and strategies on digital health,” and “other national documents that relate to the regulation of health apps for self-management.” Table 1 presents the types of documents obtained for each country within sub-Saharan Africa.

Characteristics of the Included Documents

Stand-alone regulatory standards and guidance.

We identified and included 6 stand-alone regulatory standards [ 18 , 19 , 46 - 49 ] from 3 countries (Ethiopia, Kenya, and Nigeria). All 6 documents were written in English. The years of development ranged between 2013 and 2021, as indicated in Multimedia Appendix 3 . The years of implementation were not specifically stated.

Although none of the included regulatory standards were exclusively developed to regulate health apps for self-management, 3 of them (Kenya Standards and Guidelines for mHealth Systems [ 18 ], Kenya Standards and Guidelines for E-Health Systems Interoperability [ 47 ], and Health Sector Information and Communications Technology Standards and Guidelines [ 48 ]) provided concept and information relevant to the regulation of health apps and were included in the qualitative content analysis. The Kenya Standards and Guidelines for mHealth Systems [ 18 ] provides standards and guidelines on the design, development, and implementation of mobile health (mHealth) solutions to ensure they are interoperable, scalable, and sustainable. The Kenya Standards and Guidelines for E-Health Systems Interoperability [ 47 ] outlines the principles, requirements, and standards for eHealth systems interoperability in Kenya. The Health Sector Information and Communications Technology Standards and Guidelines [ 48 ] provide guidance and a consistent approach across the health sector in Kenya for establishing, acquiring, and maintaining current and future information systems and information and communications technology (ICT) infrastructure that foster interoperability across systems. These 3 documents are a good combination of regulatory standards and guidance that provide content and context relevant to the regulation of health apps in sub-Saharan Africa.

The remaining 3 standards (standard for electronic health record [EHR] system in Ethiopia [ 19 ], standards and guidelines for electronic medical record systems in Kenya [ 46 ], and the health information exchange standard operating procedure and guideline [ 49 ]) were exclusively developed for EHRs or electronic medical records. However, they contain information relevant for mapping stakeholders with potential roles in regulating health apps for supporting self-management.

National Policies and Strategies on Digital Health

This review includes 35 national policies and strategies that are related to digital health (potentially covering health apps) [ 50 - 84 ] from 31 countries written in English, French, and Portuguese (Benin, Botswana, Burkina Faso, Burundi, Cameroon, Comoros, Côte d’Ivoire [Ivory Coast], Democratic Republic of the Congo, Eswatini, Ethiopia, Gabon, Ghana, Kenya, Liberia, Madagascar, Malawi, Mali, Mauritius, Mozambique, Namibia, Niger, Nigeria, Rwanda, Senegal, Sierra Leone, South Africa, Tanzania, Togo, Uganda, Zambia, and Zimbabwe). Although the literature search was conducted in English, it also yielded documents written in French and Portuguese from the ICTworks repository. The years of development and implementation range between 2005 and 2030. Policies and strategies written in French and Portuguese were translated into English using Google Translate. Documents labeled as national development plans, strategic plans, and strategic development plans were considered as national strategies.

National policies and strategies do not offer specific standards or guidance, but rather outline the country’s vision, policy directions, and strategies for using digital technologies in health care. They provide useful information for identifying digital health stakeholders who can play a role in regulating health apps for self-management. For example, Nigeria has a separate National Digital Health Policy [ 72 ] and a National Digital Health Strategy [ 71 ]. Both documents were developed by building on the lessons learned from the end-term evaluation of the previous National Health ICT Strategic Framework [ 85 ]. They describe Nigeria’s renewed vision, mission, goals, objectives, and strategies for the development and implementation of digital health with the aim to improve the quality, efficiency, and effectiveness of health service delivery and health outcomes.

It is worth noting that for countries with >1 policy or strategy, we included only the most recent versions. For instance, as mentioned earlier, Nigeria now has both a national digital health policy and a national digital health strategy. These 2 documents supersede and thus replace the old National Health ICT Strategic Framework [ 86 ]. Details of included documents are presented in Multimedia Appendix 3 .

Other Related National Documents

We included 8 other documents [ 20 , 85 , 87 - 92 ] from 6 countries (Ethiopia, Kenya, Liberia, Nigeria, South Africa, and Tanzania) that did not fall under either stand-alone regulatory standards and guidance or national policies and strategies. These were mostly frameworks, road maps, and reports that potentially provide information relevant to the use of health apps. The years of development and implementation range from 2016 to 2025. These documents do not provide standards or guidance, but they contain information that can help map the digital health stakeholders that potentially play a role in regulating health apps for self-management. When multiple versions of a document exist, only the latest version was taken into consideration. Multimedia Appendix 3 provides details of the included documents.

Content: Aspects That Are Regulated and Aspects That Are Not

Technical and clinical safety.

Technical and clinical safety standards are required to prevent or minimize the harm that may arise from the use of the health ICT systems (including mHealth systems) as well as to improve the health outcomes and user satisfaction. As shown in Figure 2 , two subthemes were generated from included standards [ 18 , 47 , 48 ] as content under technical and clinical safety: v(1) guidance on system quality and (2) guidance on software or app development, acquisition, support, and maintenance.

Notably, 2 of the included standards [ 18 , 47 ] provide guidance on system quality to ensure the quality, security, reliability, performance, and maintenance of eHealth and mHealth systems. The Kenya Standards and Guidelines for E-Health Systems Interoperability [ 47 ] recommend the implementation of a data quality protocol to ensure that the data collection, collation, analysis, interpretation, dissemination, and use are managed in accordance with the quality standards. Similarly, the Kenya Standards and Guidelines for mHealth Systems [ 18 ] recommends the inclusion of the following requirements in the technical manual: (1) minimum hardware requirements that should incorporate the preferred hardware architecture, (2) minimum software requirements that should include the minimum version of the underlying operating system as well as acceptable versions of related software, and (3) a detailed list of software dependencies (external libraries) necessary for the system to function properly.

The included standards [ 18 , 48 ] cover guidance on software or app development, acquisition, support, and maintenance, which aim to ensure the efficiency and effectiveness of eHealth and mHealth systems. The Kenya Standards and Guidelines for mHealth Systems [ 18 ] recommends a technical manual to provide a detailed description of the system’s installation and maintenance processes for system administrators and implementers; a developer’s guide for software developers and programmers to provide them with an overview of the system, description of the software design methodologies, description of the system architecture, and technical design diagrams; and a user manual to aid users in understanding how the system works and how each feature operates; in addition, the technical manual contains instructions for operating the software; entering and updating data; and generating, saving, and printing reports.

Although the contents generated here provide guidance that is relevant to health apps, they are not specific to health apps. Moreover, there are no clear measures to enable individuals or organizations that use health apps to manage clinical risk appropriately.

Data Protection and Security

Data protection and security are crucial aspects of managing patient information, thus ensuring the confidentiality, integrity, and availability of data as well as the rights and interests of the patient. Two subthemes related to data protection and security are (1) security measures for adequate protection of patients’ digital records and (2) guidance on data exchange.

The included standards [ 18 , 48 ] provide security measures for eHealth or mHealth systems to ensure the adequate protection of digitally accessible patient records. These measures include authentication, accountability, identification, authorization, integrity, confidentiality, availability, security, administration, and audit. This will help to achieve confidentiality, integrity, availability, and nonrepudiation of patient data or health records. Additional levels of security such as data encryption are required when there is a need to store sensitive information on removable devices or media or outside the MOH premises.

The Kenya Standards and Guidelines for mHealth Systems [ 18 ] provide the following guidance on data exchange to ensure privacy: (1) anonymize client data as much as possible before they can be shared; (2) where possible, use pseudonyms for the client data before they can be shared; (3) aggregate client data before they can be shared to reduce possibilities of tracing the data back to the client; and (4) minimize data so that access is available only to the data set required for that particular use. With regard to privacy rules, the Kenya Standards and Guidelines for E-Health Systems Interoperability [ 47 ] propose that a notice of privacy practices should be given to patients describing how their information may be used or shared while also specifying their legal rights.

Standards and Interoperability

Standards and interoperability are essential concepts in the field of IT, especially for systems that need to communicate and exchange data, as seen in the use of health apps for self-management. Two subthemes related to standards and interoperability are (1) interoperability as a basic requirement and (2) minimum standards to enable integration.

All the regulatory standards [ 18 , 47 , 48 ] highlight the importance of having interoperability as a basic requirement when selecting software products or services for use within the health system. This facilitates interaction across systems. For instance, to facilitate seamless interaction between mHealth systems and primary information systems for data capture, reporting, and decision support in various domains of the health system, the Kenya Standards and Guidelines for mHealth Systems [ 18 ] recommends the incorporation of at least 3 types of interoperability, namely, technical interoperability, semantic interoperability, and process interoperability.

Furthermore, 2 regulatory standards [ 18 , 47 ] proposed minimum interoperability standards to enable the integration of services and data exchange between various systems in health care. For instance, the Kenya Standards and Guidelines for mHealth Systems [ 18 ] suggests standards (for interoperability) for mHealth systems that are consistent with the recommendations in internationally accepted standards. They include the following: (1) clinical messaging—ensuring mHealth systems conform to Health Level 7 (HL7) version 3 standards and corresponding implementation guideline; (2) clinical terminology—ensuring terminologies and classifications for clinical concepts (eg, International Classification of Diseases, tenth revision—for diseases; Systemized Nomenclature of Medicine—for clinical data coding; Logical Observation Identifiers Names and Codes—for laboratories; and RxNorm—for Pharmacies); (3) the mHealth system must use the latest versions of international standards, such as HL7 Clinical Document Architecture for electronic sharing of clinical documents; (4) concepts—mHealth systems will use the idea of “concepts” so that information can be transmitted between systems without losing meaning or context, and HL7 Reference Implementation Model or other appropriate standards are recommended for implementing concepts; (5) architecture—to develop mHealth systems, developers should define the system architecture that should include data elements and business logic. Furthermore, to define how mHealth systems interact with other systems, developers of mHealth solutions must provide application programming interfaces. FHIR is the preferred application programming interface interoperability standard.

Inclusion and Equitable Access

Inclusion and equitable access are essential principles to ensure that health apps are culturally appropriate and relevant and accessible to everyone, regardless of gender, ethnicity, location, or economic status.

All the included regulatory standards [ 18 , 47 , 48 ] indicate that they were developed based on a combination of participatory and consultative approaches involving multiple actors or stakeholders, thus promoting inclusion. However, there are no specific measures or guidance to ensure adequate engagement and representation of all the relevant stakeholders and to sustain that engagement.

The Kenya Standards and Guidelines for mHealth Systems [ 18 ] proposes the following systems attributes to ensure equitable access to mHealth services at all times and from anywhere: (1) allocation of adequate storage and bandwidth capacity; (2) fast response time; (3) fast recovery capabilities; (4) performance monitoring; (5) business continuity processes, for example, backups; and (6) redundant sites and links. Furthermore, the Kenya Standards and Guidelines for mHealth Systems [ 18 ] prescribes the following metrics for measuring system availability: (1) downtime per year, (2) mean time between failure, (3) mean time to repair, and (4) failure in time.

Although the abovementioned systems attributes and metrics for measuring system availability are important, the included standards do not offer any concrete guidance or model for achieving a sustainable funding mechanism for health apps to ensure that they are readily available and accessible to those who need them.

Context: Reasons Why Those Aspects Are Regulated

The 3 standards [ 18 , 47 , 48 ] were developed to address unsafe, isolated, and inconsistent implementation. The Health Sector ICT Standards and Guidelines [ 48 ] suggest that although there has been a lot of ICT investment in the health sector leading to improvement in service delivery and information exchange, there remains the challenge of inconsistency in ICT implementation and harmonization of the health sector system requirements. Hence, there is a need to adopt global best practices for software development, acquisition, support, and maintenance by the MOH. In addition, the Kenya Standards and Guidelines for mHealth Systems [ 18 ] indicates that standards and guidelines are necessary to ensure a consistent approach to the development of ICT systems. Similarly, the Kenya Standards and Guidelines for E-Health Systems Interoperability [ 47 ] recognize the need to ensure that the processes of collecting, collating, analyzing, interpreting, disseminating, and using data are consistent with data quality standards.

To build mutual trust and maximize the benefits of eHealth information exchange, the Kenya Standards and Guidelines for E-Health Systems Interoperability [ 47 ] reiterate that as health data are constantly being exchanged across health information systems, robust security standards are required to maintain their integrity and confidentiality. This will build the trust of service users and consequently help to maximize the benefits of eHealth information exchange such as in self-management.

Two of the included regulatory standards [ 47 , 48 ] indicate that the context for standards and interoperability was (1) to address poor coordination, duplication of efforts, and inefficient use of resources and (2) to promote the integration of ICT systems.

The Kenya Standards and Guidelines for E-Health Systems Interoperability [ 47 ] acknowledge that the absence of interoperability standards over the years has led to the duplication of efforts and the inefficient use of ICT resources in health care. Now that ICT has become increasingly relevant in improving efficiency in health service delivery, the Kenya MOH recognizes the need to adopt a standardized approach, hence the development of interoperability standards for eHealth systems. In addition, the Health Sector ICT Standards and Guidelines [ 48 ] emphasize the relevance of interoperability as a requirement for addressing the inconsistency in implementing ICT in the health sector.

The Health Sector ICT Standards and Guidelines [ 48 ] consider “integration of ICT systems” as one of its key guiding principles, acknowledging the lack of information systems integration as a challenge experienced by ICT services across Kenya.

The contexts for inclusion and equitable access as generated from included standards [ 18 , 47 , 48 ] were (1) to promote inclusion and (2) to promote equitable access to services.

To promote inclusion, the standards [ 18 , 47 , 48 ] highlight the importance of involving and engaging multiple actors and stakeholders during the development process. However, no emphasis was placed on the need to sustain stakeholder engagement during the implementation process.

Pertaining to equitable access, the Kenya Standards and Guidelines for mHealth Systems [ 18 ] acknowledges that the public health care system is largely unavailable to most of the population in many developing countries because of geographical location, resource constraints, inefficiencies, and lack of awareness. Hence, it recognizes the importance of ensuring that mHealth services are always accessible by users and from anywhere as well as the need to put in place mechanisms to make this happen.

Process: How the Regulations Are Developed and Implemented

Two themes were generated from the included standards: development and implementation processes [ 18 , 47 , 48 ].

Development Process

All the included standards [ 18 , 47 , 48 ] indicate that they were developed through a participatory process and in consultation with a range of subject experts and interest groups. In addition, the standards [ 18 , 47 , 48 ] adopted a multisectoral approach to engage health-related stakeholders from government ministries or agencies and development partners and a range of subject experts and interest groups. It has also been reported that these standards [ 18 , 47 , 48 ] were developed based on international best practices and with reference to international standards. However, there is no indication that a stakeholder engagement strategy was adopted to sustain the engagement of stakeholders through the entire development and implementation process.

Implementation Process

The 3 regulatory standards [ 18 , 47 , 48 ] identify the key requirements to ensure effective implementation of IT services in the health sector. These are (1) legal authority, (2) coordination, (3) building capacity, and (4) monitoring and evaluation.

The included standards [ 18 , 47 , 48 ] were established based on the legal provisions enshrined in the health and other related acts and laws of Kenya as well as the relevant policies and strategies. Hence, it is expected that their implementation will comply with and be backed by those legal provisions. For example, the Health Sector ICT Standards and Guidelines [ 48 ] indicate that its implementation will be supported by the authority from the Kenya Communications Act 2009, E-Government Strategy, and National ICT Policy. Similarly, the Kenya Standards and Guidelines for mHealth Systems [ 18 ] asserts that it will be implemented by complying with existing and relevant national policies, legal frameworks, strategies, and standards, including the Health Information Policy, ICT Standards, and System Interoperability Principles.

The included standards [ 18 , 47 , 48 ] report that the implementation of regulations will require robust coordination mechanisms. For instance, the Health Sector ICT Standards and Guidelines [ 48 ] indicate that, as the Ministry’s ICT resource manager, the principal secretary (also the head of ICT), in collaboration with the ICT Governance Committee, is responsible for coordinating the implementation of the standard. The ICT Governance Committee comprises representatives from the heads of departments and ICT development partners in the health sector. The committee’s responsibilities include overseeing, enforcing, and reviewing standards as well as initiating ICT projects.

The Health Sector ICT Standards and Guidelines [ 48 ] highlight the need for capacity building or training of the MOH staff and stakeholders who are the primary users of the Ministry’s ICT services. This will enhance their capacity to implement the guidelines provided in the document in line with the ministry’s human resource development policies, regulations, and rules. However, it is acknowledged that building capacity for health ICT is a challenge given that there is low adoption of ICT among health providers, and ICT is not routinely included in the course content of most training programs. The Kenya Standards and Guidelines for mHealth Systems [ 18 ] listed the “number of mHealth practitioners trained on the standards and guidelines” as one of the indicators for monitoring and evaluating mHealth interventions.

The Health Sector ICT Standards and Guidelines [ 48 ] assert that monitoring and evaluation is an essential role of the MOH to ensure efficiency, accountability, and transparency throughout the implementation period. It further stresses that all those who use the Ministry’s ICT services are required to adhere to the provisions in the standard as the MOH will carry out quarterly monitoring exercises on the use of the standard to ensure compliance based on clear indicators. Furthermore, the ICT Governance Committee will periodically review and amend the standard to keep it relevant and effective. Similarly, the Kenya Standards and Guidelines for mHealth Systems [ 18 ] establishes the following key indicators for effectively monitoring and evaluating the implementation of the standards and guidelines: (1) the number of counties in which the MOH has disseminated the standards and guidelines, (2) the number of counties successfully implementing the standards and guidelines, (3) the number of mHealth practitioners trained on the standards and guidelines, (4) the number of mHealth practitioners accessing the standards and guidelines, (5) the number of mHealth practitioners who correctly understand the standards and guidelines, (6) the number of stakeholders who adhere to the standards and guidelines, (7) the number of mHealth systems that follow the required development steps, and (8) the number of mHealth practitioners who have implemented their systems by using the standards and guidelines. In addition, the Kenya Standards and Guidelines for mHealth Systems [ 18 ] indicates that the outlined standards will be reviewed every 3 years to ensure they are up to date with new changes including the changes in policies and systems upgrades.

Although all the abovementioned indicators are relevant, the implementation process is not explicit on the approach for regulating health apps and ensuring compliance with regulatory standards and guidance.

Actors: Those the Regulations Are Targeted at

The included standards [ 18 , 47 , 48 ] identified 2 main groups of actors for whom the regulations and guidance were targeted. They included (1) those who provide digital health services and (2) those who use the ICT infrastructure of the MOH.

Two of the standards [ 47 , 48 ] indicated that the regulations should be implemented by all individuals and organizations that provide ICT-related health care services to the public. Similarly, the Health Sector ICT Standards and Guidelines [ 48 ] state that all those who access or use the MOH ICT infrastructure are expected to adhere to the guidelines outlined in the document.

Mapping of Stakeholders

To address the third research question, we conducted a stakeholder mapping guided by the RISA tool [ 41 ].

A total of 11 categories of key stakeholders were identified from all 49 included documents (6 stand-alone regulatory standards and guidance, 35 national policies or strategies, and 8 other related documents). These categories are consistent with the digital health stakeholders recognized by the WHO, ITU, and UNESCO [ 32 , 33 , 43 ]. Table 2 presents the mapping of stakeholders according to their role categorization. A more detailed table with a potential role description with regard to regulating health apps for self-management is presented in Multimedia Appendix 4 .

a WHO: World Health Organization.

This paper presents the findings of a scoping review of regulatory standards and guidance for the use of health apps for self-management in sub-Saharan Africa. To the best of our knowledge, this is the first study that attempted to identify and assess the extent to which regulatory standards and guidance regulate and guide the use of health apps for self-management in sub-Saharan Africa as well as map out the key stakeholders and their potential roles.

Our findings reveal that only 1 country (Kenya) in sub-Saharan Africa currently has national regulatory standards that could potentially regulate the use of health apps for self-management. The included standards failed to adequately address adequate attention to inclusion and equitable access. This is concerning given the growing need to promote the adoption of culturally appropriate and relevant health apps and to ensure that they are available to those who need them regardless of gender, ethnicity, geographical location, or financial status [ 24 - 29 ]. Consequently, this review provides insights into the regulation of health apps for self-management in sub-Saharan Africa, which needs to be given more attention if the potential of these apps is to be harnessed in the region.

Principal Findings

We identified 49 documents from 31 countries in sub-Saharan Africa. Although none of the included standards provided a specific set of regulations on health apps for self-management, we identified 3 standards [ 18 , 47 , 48 ] that provided relevant information regarding the regulation of health apps. The included national policies and strategies, in contrast, only outline the goals and commitments made by national governments to promote the adoption of digital technologies in the health sector and the plans and paths set forth to achieve these goals. However, the information they provided was relevant for identifying and mapping digital health stakeholders who potentially have vital roles in regulating the use of health apps for self-management.

The policy analysis framework (content, context, process, and actors) [ 42 ] was adapted and applied to organize the key findings. The content covered the following areas: guidance on systems quality; guidance on software and app development, acquisition, support, and maintenance; security measures for adequate protection of patients’ digital records; guidance on data exchange; interoperability as a basic requirement; minimum standards to enable integration; involvement and engagement of relevant stakeholders; and system attributes for equitable access to services. Meanwhile, the context was to address unsafe, isolated, and inconsistent implementation; to build mutual trust and maximize the benefits of eHealth information exchange; to address poor coordination, duplication of efforts, and inefficient use of resources; to promote the integration of ICT systems; and to promote inclusion and equitable access to services. The process involved the development process (which covers participatory and consultative processes and multisectoral approach, with reference to international standards and best practices) and the implementation process (which covers legal authority, coordination, capacity building, and monitoring and evaluation). The targeted actors were those who provided digital health services and those who used the ICT infrastructure of the MOH.

Furthermore, key stakeholders with potential roles in regulating health apps for self-management were identified. They include the government, regulatory bodies, funders, intergovernmental and nongovernmental organizations, academia, and the health care community.

Implications of the Study Findings for Practice

Regulatory standards and guidance act as a bridge between technological innovation and its safe and effective use in health care. They ensure that while technology continues to advance, the safety and trust of patients are never compromised. Among the plethora of health apps on the market, the over-the-counter, nonregulated apps such as wellness and fitness apps are the most mainstream [ 93 - 95 ]. On the other side of the spectrum, there are regulated health apps that are classified under medical devices or software as medical device products [ 94 , 95 ]. Some of these are prescription-only apps, such as digital therapeutics (DTx) apps for managing substance dependence [ 95 , 96 ].

Although some high-income countries have made significant strides in ensuring the safety, effectiveness, and accessibility of health apps, the journey has indeed not been without challenges and hurdles. Sub-Saharan Africa, although dealing with its own unique set of challenges, has the opportunity to learn from the experiences of these high-income countries. This could potentially allow the region to bypass some of the hurdles encountered by high-income countries in their journeys.

Technical and clinical safety are essential requirements that health apps must meet before they can be considered for use for self-management to minimize the risk of harm to patients. It is well documented that health apps that function poorly pose a serious threat to the safety of patients. An example illustrating how health apps used for self-management can threaten patient safety is evident in a study [ 12 ]. This study [ 12 ] revealed that widely used health apps designed to calculate and estimate insulin doses could endanger patients by providing incorrect or inappropriate dose recommendations. Similarly, 2 successive studies that assessed the contents and tools of apps for asthma discovered that none of the apps in the first study offered comprehensive information or adequate tools for asthma self-management, whereas the follow-up study, which was conducted 2 years later, showed a 2-fold increase in the number of asthma apps, yet there was no improvement in the content and tools offered by the newer apps. In fact, many apps recommended self-management procedures that were not supported by evidence [ 13 , 14 ]. Accordingly, some health apps that support the self-management of long-term conditions do not adhere to evidence-based guidelines and are unresponsive to the evolving health needs of patients.

Although the context of included regulatory standards with regard to technical and clinical safety was to address unsafe, isolated, and inconsistent implementation, the guidance provided by these regulatory standards is not specific to health apps, and they do not provide appropriate guidance and standards for health organizations and other key stakeholders to establish a framework for managing the clinical risks associated with deploying and implementing self-management health apps. Considering the rapid advancements in digital health (including artificial intelligence [AI] or machine learning and big data), health apps will increasingly play a crucial role in supporting self-management through digitally enabled care pathways that will improve personalized care and health outcomes [ 97 , 98 ]. Therefore, it is imperative to ensure the technical reliability and clinical safety of health apps for self-management through robust regulatory standards and guidance. For instance, a guide on the criteria for health app assessment, developed by the UK government, includes technical stability and clinical safety as criteria for deciding whether health apps should be considered for use in the National Health Service (NHS) [ 99 ]. In addition, medical device apps are required to conform to the NHS clinical risk management standards as part of the clinical safety requirements [ 99 , 100 ]. In the event of any concerns regarding the safety of a medical device app, the Yellow Card reporting system can be used by a responsible clinical safety officer or any other individual to notify the Medicines and Healthcare products Regulatory Agency (MHRA) [ 101 , 102 ].

To adequately manage patient information when health apps are used for self-management, data protection and security standards and guidance are required. They guarantee that data are kept and handled safely and responsibly within the provisions of the law and that patients’ rights and interests are respected.

There have been ongoing concerns about compliance with ethical standards, the principles of confidentiality of information, and data privacy. For example, an assessment of apps that had previously been endorsed by the former UK NHS Apps Library revealed substantial gaps in compliance with data protection principles regarding the collection, storage, and transmission of personal information. This has raised a fundamental concern about the credibility of developer disclosures and whether these disclosures can be trusted by certification programs [ 15 ]. A study assessed the privacy practices of the 36 most popular apps for depression and smoking cessation for Android and iOS in the United States and Australia [ 16 ]. The findings revealed that although only 69% (25/36) of the apps included a privacy policy, 92% (33/36) of the apps shared data with a third party, and only 92% (23/25 with privacy policy) of the apps disclosed sharing data with a third party in their policy. Although 81% (29/36) of the apps shared data with Google and Facebook for the purposes of advertising, marketing, or analytics, only 43% (12/28) of the apps that shared data with Google and 50% (6/12) of the apps that shared data with Facebook disclosed this in their policy [ 16 ].

In this regard, health app developers and providers in the United Kingdom are required to conduct a data protection risk assessment before they launch or update their apps to ensure compliance with the United Kingdom General Data Protection Regulation (GDPR) and other relevant regulations, including the Data Protection Act 2018 [ 103 ]. By conducting a data protection risk assessment, health app developers and providers can demonstrate that they are accountable; they respect the privacy and dignity of their users; and that they deliver safe, effective, and ethical solutions [ 104 ].

Health apps are expected to play an increasingly important role in supporting self-management. However, this ambition can only be achieved if citizens trust that these apps are collecting and analyzing data safely and in accordance with robust regulatory standards and guidance. It is also crucial that these apps provide reliable information that clinicians can act on [ 98 ]. The context of the standards included in this study regarding data protection and security was to build mutual trust and maximize the benefits of eHealth information exchange. Trust is a key factor in the successful adoption and use of health apps, and transparency in data handling and clinical decision-making is essential to build and maintain that trust. This is also paramount for the widespread acceptance and impact of health apps on health care outcomes in sub-Saharan Africa.

We acknowledge the existence of numerous national laws related to data protection and security outside the health sector. Hence, guidelines that link these legislations together must be provided to ensure compliance with all relevant laws and guidance when using patient data. An example of how to achieve this is the United Kingdome’s guide to good practice for digital and data-driven health technologies that provides guidelines on how to abide by the laws and principles that govern data security and protection in the United Kingdom, including the GDPR, Data Protection Act 2018, and Caldicott Principles [ 105 ].

Standards and interoperability are essential for effectively developing, deploying, and implementing health apps to support self-management in sub-Saharan Africa. Interoperability is the ability of different systems, devices, or applications to communicate and exchange data with each other in a coordinated manner, thus providing timely and seamless portable information across organizational, regional, and national boundaries and optimizing both individual and population health [ 106 ]. In the same vein, standards enable interoperability between systems or devices through a common language and a common set of expectations [ 106 ].

Interoperability is crucial in improving the quality, safety, and efficiency of care delivery as well as empowering patients and providers with access to relevant and timely information [ 99 ]. One of the most widely used and accepted interoperability standards for health care data exchange is FHIR [ 106 , 107 ]. FHIR is a global industry standard developed by HL7 International. FHIR is designed to be quick to learn and implement and to support a variety of use cases, including self-management [ 108 ]. By using apps that are based on an FHIR standard, patients can benefit from data analytics that show how their health data relate to their chronic conditions or wellness goals [ 109 ]. They could also access all their health information from one place, even if they visit different health professionals who use different electronic medical records or EHR, thus promoting integrated care [ 28 , 31 , 33 , 109 - 115 ]. As a result, patient care can easily be coordinated.

The context of the included regulatory standards with regard to standards and interoperability was to address poor coordination, duplication of efforts, and inefficient use of resources and to promote the integration of ICT systems. However, in sub-Saharan Africa, there are many challenges and barriers to the adoption and implementation of interoperability standards, such as the lack of awareness or knowledge of the benefits and requirements of interoperability standards among stakeholders; lack of incentives or regulations to encourage or enforce the adoption of interoperability standards by app developers and vendors; lack of resources or capacity to implement interoperability standards, including technical expertise, infrastructure, funding, or governance; and lack of alignment or coordination among the different actors and initiatives involved in developing, deploying, and implementing the digital health interventions [ 30 , 116 - 119 ]. To address these challenges, some possible solutions may include raising awareness and education on the importance and value of interoperability standards for health apps among all relevant actors; developing and implementing policies and guidelines that promote or mandate the use of interoperability standards by app developers and vendors; providing technical assistance and support for app developers and vendors to adopt and implement interoperability standards, such as tools, frameworks, testing, certification, or accreditation; and establishing and strengthening collaboration and coordination among the different stakeholders and initiatives involved in health app development, deployment, and implementation in sub-Saharan Africa. In addition, the Digital Health Platform Handbook, a toolkit developed by the collaborative efforts of the WHO and ITU [ 120 ], can help countries in sub-Saharan Africa to develop and implement digital health platforms as the underlying infrastructure for interoperable and integrated national digital health systems. The digital health platform is a system-wide approach to developing digital health solutions with the aim to overcome the problems of siloed, vertical, and isolated applications and systems that hamper data management, innovation, efficiency, and impact in the health sector.

Inclusion and equitable access are crucial to ensuring that health apps and related services are culturally appropriate and relevant as well as accessible to all who need them, regardless of gender, ethnicity, geographical location, ability, or financial status [ 24 - 29 ]. This is the key to promoting a “sense of belonging” and “ownership” and thus underscoring the importance of stakeholder mapping and involvement or engagement through the development and implementation process [ 22 ].

In this study, the included regulatory standards demonstrate the importance of inclusion by adopting both a participatory and consultative approach involving multiple stakeholders from different sectors. However, the standards do not provide clear guidance to ensure the adequate participation and sustained engagement of all relevant stakeholders. The lack of concise guidance to ensure the adequate participation and engagement of all relevant stakeholders, especially the susceptible and disadvantaged groups, can increase the risk of tokenistic tendencies, which can undermine the cultural appropriateness of health apps [ 25 , 121 ]. Some susceptible groups, such as women and people with low socioeconomic status, may face additional barriers to accessing and using health apps, such as lack of digital literacy, privacy concerns, cultural norms, or stigma [ 25 ]. Similarly, the cost of developing, maintaining, and updating health apps may not be covered by public or private health insurance schemes, which could limit their affordability and availability for low-income or uninsured populations [ 95 ]. However, there is no specific guidance or model for an effective funding mechanism for health apps in the included regulatory standards.

To address these challenges and ensure equitable access to health apps for self-management in sub-Saharan Africa, possible measures may include developing policies and regulations that support integrating health app interventions into existing health systems and financing mechanisms and engaging with stakeholders from different sectors and backgrounds (including health professionals, patients, communities, governments, civil society, academia, and industry) to co-develop and co-implement frameworks or models that promote the use of health apps for self-management in ways that are responsive to the local context and needs. Moreover, establishing regulations that provide appropriate financing or reimbursement options will reduce the risk of developers of good quality health apps turning to data mining for revenue, thus increasing privacy concerns [ 95 ]. For instance, in Germany, the reimbursement of health apps classified as medical devices (Digitale Gesundheitsanwendungen) was introduced in 2021 under the statutory health insurance [ 122 , 123 ]. When a medical device is prescribed by a physician or a physiotherapist, the manufacturer must submit an application to the German Federal Institute for Drugs and Medical Devices (Bundesinstitut für Arzneimittel und Medizinprodukte) for approval [ 123 ]. The Federal Association of the Statutory Health Insurance Funds (Spitzenverband Bund der Krankenkassen) determines and negotiates the reimbursement thresholds following approval. However, the manufacturer must demonstrate that the app is safe, functional, and of good quality; complies with data protection requirements; and benefits patient care [ 123 ].

The process of regulating health apps essentially involves the development and implementation of regulatory standards and guidance. According to our study, the development process comprises a participatory and consultative process, a multisectoral approach, and a reference to international standards and best practices. In contrast, the implementation process is ongoing and requires appropriate legal authority, coordination, capacity building, and monitoring and evaluation.

We recognize that health apps can be accessed and used by patients from different parts of the world, and this means that countries need to carefully consider whether health apps that are accessed and used by their citizens meet the national or regional legal and ethical requirements, including their cultural and linguistic needs [ 23 ]. For countries in sub-Saharan Africa, a cross-border or regional collaboration between national legal authorities through the coordination of agencies such as the African Medicines Regulatory Harmonization (AMRH) may help to ensure that health apps built for the region are safe, effective, and user-friendly for everyone, considering the contextual differences of the countries [ 23 ]. For instance, all medical device companies that want to sell their products in the European market must obtain a Conformité Européenne (CE) mark for their devices, which indicates that they meet the legal requirements and can be freely circulated within the European Union [ 124 ]. Although the European Union member states regulate medical devices, the European Medicines Agency is involved in the regulatory process.

The regulation of health apps is extremely complex and involves a wide range of stakeholders. Therefore, a robust coordination mechanism is essential to reduce the risk of fragmentation and duplication of efforts and to promote the efficient use of resources. Most countries in sub-Saharan Africa have units in health ministries that coordinate and oversee the regulation of medical products. These units should be autonomous, full-fledged departments with legal authority (boards or commissions) to ensure independent, transparent, and accountable decision-making, but this is often not the case [ 125 ]. These units are recognized by the national authorities as regulators (eg, the National Medicines Regulatory Authority [NMRA]) [ 126 ]. Such organizational structures hinder the effectiveness of the national regulatory authorities in fulfilling their mandate and prevent the establishment of quality management systems to ensure transparent and accountable decision-making [ 125 ].

Furthermore, Essén et al [ 23 ] analyzed health app policy or regulation in 9 high-income countries (Sweden, Norway, Denmark, Netherlands, Belgium, Germany, England, the United States, and Singapore) and found that most of these countries adopted centralized approaches to app evaluation. Although centralized approaches might have advantages over self-evaluation, they may create bottlenecks and limit the availability of high-quality health apps for users. As suggested by Essén et al [ 23 ], a decentralized approach, such as the accreditation of evaluation agencies, maybe a worthwhile solution. However, this will require adequate coordination to ensure the consistency and reliability of the evaluation criteria and methods across different agencies as well as the transparency and accountability of the accreditation process. A possible way to achieve this is to adopt a common framework that can guide the evaluation and accreditation of health apps.

Similarly, the postmarket surveillance (PMS) system, which is a new regulation for medical devices in Europe, is a process of collecting and analyzing data on medical devices after they have been launched into the market to ensure their safety and performance and to identify any problems or need for improvements [ 127 , 128 ]. The PMS system is important because premarket data, which are obtained from testing a medical device before it is launched, have limitations in capturing the long-term performance and risks of the device [ 128 ]. Currently, the PMS system does not cover fitness and wellness apps, which are commonly used in self-management. Hence, Yu [ 93 ] proposed that the PMS system should also be applied to DHTs, such as fitness and wellness apps. They argue that the postmarket data would help regulators periodically review and adjust the regulatory standards for these groups of health apps based on their risks and benefits.

Drawing on the experience of the United Kingdom, it can be clearly demonstrated that the regulation of health apps is a complex, a multifaceted, and an evolving process that involves different regulators and criteria depending on the nature and function of the app. For instance, a centralized NHS Apps Library was launched as a beta site in April 2017 to provide patients with a collection of trusted and easy-to-use digital health tools [ 129 ]. The library provided access to a range of health apps that were reviewed and approved by the NHS, including apps that could help patients manage conditions such as diabetes, mental health, and chronic obstructive pulmonary disease [ 130 ]. However, the library was closed in December 2021 [ 131 ]. Although no reason for the closure was provided on the website, it is likely because of persistent concerns regarding the safety of patients and data privacy involving multiple apps including those listed in the library [ 12 , 14 - 16 , 131 , 132 ]. The NHS App was introduced in January 2019 before the closure of the NHS Apps Library to serve as the gateway for accessing NHS services including ordering repeat prescriptions and booking or managing appointments [ 133 ].

Furthermore, the United Kingdom Health Security Agency, formerly known as Public Health England, issued a guidance on criteria for health app assessment in October 2017 [ 99 ]. The purpose of this guidance was to ensure that all health apps built for the UK population work well and provide clear information about their functions, benefits, and intended outcomes for patients and health care professionals. On the basis of this guidance, those intending to build an app are required to conform to certain regulations before being considered for the app assessment process. The 2 main regulations are the medical device regulation and the Care Quality Commission (CQC) registration. Apps that are considered as medical devices must register with the MHRA and have a CE mark. Apps providing health or social care that fit into 1 of 14 regulated activities are required to register with the CQC before they can be assessed [ 134 ]. CQC is an independent regulator of health and social care services in England.

Similarly, the Organisation for the Review of Care and Health Apps (ORCHA) is a UK-based organization that independently evaluates and distributes health apps. It provides services such as app review, accreditation, curation, and recommendation within the United Kingdom and across the world [ 135 ]. ORCHA also enables organizations (including the NHS) to build a decentralized web-based digital health library of consumer-friendly over-the-counter apps [ 135 - 137 ]. These apps are continuously assessed by ORCHA against the standards and regulations in clinical and professional assurance, data quality and privacy, and usability and accessibility [ 137 ].

In addition, the Digital Technology Assessment Criteria (DTAC) were introduced in beta in October 2020, and its first official version was subsequently launched in February 2021 [ 138 ]. The DTAC plays a crucial role in ensuring that digital health tools meet the necessary standards in areas such as clinical safety, data protection, technical security, interoperability, usability, and accessibility. By serving as the national baseline criteria for DHTs in the NHS and social care, it provides a valuable framework for health care organizations during procurement. It also offers guidance for developers on the expectations for their digital technologies within the NHS and social care. This is an example of how a harmonized framework can help ensure the quality and safety of DHTs, including health apps.

In addition, the National Institute for Health and Care Excellence Evidence Standards Framework is a set of evidence standards for a wide range of DHTs designed to help evaluators and decision makers in the health care system to consistently identify DHTs that are likely to offer benefits to the users and the health care system [ 139 ]. The Evidence Standards Framework was first published in March 2019 and is ideally used before DHTs (including health apps) are considered for commissioning or procurement by the NHS [ 140 ]. It is a crucial tool for ensuring that DHTs are clinically effective and offer value to the health and care system in the United Kingdom. In August 2022, the framework was updated to include AI and data-driven technologies with adaptive algorithms [ 140 ].

Furthermore, DTx apps, which are a type of medical device, are not allowed into the UK market unless they comply with the UK GDPR and meet the requirements of DTAC. In addition, they must bear the CE or UK Conformity Assessed marks [ 141 ]. This means that DTx apps must demonstrate their safety and efficacy through clinical trials and comply with the relevant regulations for data protection and quality standards as regulated by the MHRA. DTx products are also recognized as DHTs under the National Institute for Health and Care Excellence Evidence Standards Framework [ 142 ]. DTx incorporates software to treat, prevent, or manage specific diseases or conditions [ 143 , 144 ]. The fact that DTx products typically focus on a narrow clinical indication and generate evidence of clinical efficacy underscores their potential to make a substantial contribution to self-management and health care delivery in general. The increasing recognition of the role of DTx in patient care by regulators is also noteworthy, and the creation of regulatory and reimbursement pathways for approved apps further enables DTx products to continue to play an important role in impacting health care delivery [ 1 , 143 ]. This is a testament to the potential of regulated health apps to revolutionize health care and improve patient outcomes.

Among the many lessons to learn from the experience of the United Kingdom is that the regulation of health apps must evolve to keep pace with advances in DHTs and adapt to the changing needs and demands of digital health. Moreover, efforts are being made to streamline the multifaceted approaches to simplify app regulation and access in the United Kingdom [ 23 ]. Therefore, a robust and dynamic coordination mechanism, along with political will, skilled personnel, reliable funding, and a robust framework for monitoring and evaluating progress and aligning key performance indicators, is essential for countries in sub-Saharan Africa to keep pace with the advancement in the regulation of health apps. There is also a need to strengthen collaboration and ensure regulatory harmonization among national regulatory authorities and continental bodies such as the regional economic communities, AMRH, and the WHO AFRO [ 126 ].

Capacity building and monitoring and evaluation are important factors for ensuring effective regulation of health apps given the complex nature of the process. The regulation of medical products (including health apps) in sub-Saharan Africa generally includes licensing and accreditation, evaluation, inspection, quality control, information dissemination and promotion, and monitoring of adverse events [ 125 ]. Therefore, high-level skills as well as effective monitoring and evaluation will be required to ensure the success of the process. For most countries in sub-Saharan Africa, the NMRA is responsible for coordinating and overseeing the regulatory system of medical products [ 125 , 126 ]. However, in most cases, NMRAs are unable to perform the core regulatory functions expected of them [ 145 ]. More than 90% of African countries have limited or no capacity to regulate medical products, with only 7% having moderately developed capabilities [ 145 ]. The lack of effective NMRAs in Africa exposes the citizens to potential harm by allowing unsafe, low-quality, and fake medical products to circulate and be used [ 145 ].

Although it is the responsibility of governments to establish functional regulatory systems and ensure effective monitoring and evaluation of the regulatory process, the involvement of international and continental organizations to support sub-Saharan African countries improve the regulatory capacity of their national regulatory agencies would be extremely beneficial. For instance, the African Medicines Agency (AMA) was established in November 2019 as a treaty adopted by the African Union Member States to help address the concerns arising from weak regulatory systems on the continent. At present, 37 countries have signed the AMA treaty, including 26 countries that have ratified it [ 146 ]. The main objective of the AMA is to enhance the capacity of States Parties and regional economic communities to regulate medical products to improve the quality, safety, and efficacy of medical products on the continent [ 147 ]. The AMA, in collaboration with other existing capacity building initiatives or organizations, such as the WHO Global Initiative on Digital Health, ITU, AMRH, WHO AFRO, and United Nations Children’s Fund, can assist sub-Saharan African countries in aligning their regulatory requirements with available resources and support them to acquire the necessary tools and skills to build effective and sustainable regulatory systems for health apps. This can be achieved by adopting a decentralized approach to engage a network of technical experts across the African Union similar to the model of the European Medicines Agency [ 148 ].

Actors or Stakeholders

The regulation of health apps often requires working with a wide range of actors or stakeholders. However, in this review, we identified only 2 main actor groups (those who provide digital health services and those who use the ICT infrastructure of the health ministry). These are the groups that are targeted by the included regulatory standards.

From a broader perspective, 12 categories of stakeholders according to their potential role in regulating health apps for the self-management were mapped in this study. The potential contribution of these stakeholders to the regulation of health apps for self-management in sub-Saharan Africa not only depends on their roles and responsibilities but also on their interests, needs, expectations, and influence [ 41 , 149 - 151 ]. Thus, a robust stakeholder analysis is paramount as it can help define the scope of the regulatory process, prioritize the requirements, manage the expectations, and ensure the engagement and participation of stakeholders throughout the regulatory process [ 41 , 152 - 156 ]. Our stakeholder mapping, as presented in Table 2 (refer to Multimedia Appendix 4 for more details), lays the foundation for national governments to conduct a robust stakeholder analysis and to adopt an all-inclusive stakeholder engagement strategy to manage and sustain the engagement and participation of all relevant stakeholders [ 157 , 158 ].

Recommendations

Our review found that the regulation of health apps in sub-Saharan Africa is especially poor and almost nonexistent, as only Kenya has national standards that could address some of the regulatory issues related to health apps. Therefore, we recommend the following actions to help sub-Saharan African countries improve the regulation of health apps to support self-management:

• Establish a clear and consistent definition of what constitutes a health app (considering AI or machine learning) and what level of regulation is required for different types of apps.
• Develop and implement criteria and guidelines that ensure the quality, safety, and usability of health apps.
• Engage with independent app evaluators, such as ORCHA, to adopt a common framework that can guide the evaluation and accreditation of health apps and use the framework to create and maintain decentralized and transparent platforms that showcase and evaluate health apps for users and health care professionals.
• Develop and implement policies and regulations that enable sustainable funding for health apps such as integrating the use of health apps for self-management into existing health systems and financing pathways or mechanisms.
• Support and facilitate innovation and collaboration across the sub-Saharan Africa region, especially in areas including but not limited to data security and privacy, interoperability standards, usability, accessibility, funding, capacity building, and monitoring and evaluation of the regulatory process.
• Manage and sustain the engagement, involvement, and participation of all relevant stakeholders in the regulatory process by conducting a robust stakeholder analysis and adopting an all-inclusive stakeholder engagement strategy.

Strengths and Limitations of the Study

This study has several strengths, which include an extensive search of gray literature and repositories, contact with key individuals, and the use of a systematic approach. Given that regulatory standards and guidance are unavailable in scientific databases, a wide range of gray literature and repositories were searched. In addition, contact was made with key staff members to obtain relevant documents, including those at the MOHs, the WHO country offices, and the WHO AFRO. Second, to enhance the strength of the study, a policy analysis framework was adapted and used to systematically organize the key study findings, whereas a deductive descriptive qualitative content analysis approach was used to identify and analyze texts that contained relevant concepts and other related information based on the 4 predefined themes. Third, the RISA tool was used to guide the mapping of key stakeholders. This has further increased the robustness of the study findings.

The limitations of this study include the fact that our literature search was conducted in English. Although the literature search was conducted in English, it yielded documents written in French and Portuguese from the ICTworks repository. Second, regulatory standards and guidance are not readily available on scientific databases; hence, it is possible that some relevant documents might have been missed. However, efforts were made to obtain these documents by contacting key stakeholders including key contact persons at the WHO AFRO, WHO country offices, and MOHs. In addition, contacting key individuals only for the purposes of requesting documents rather than conducting direct interviews was one of the limitations of this study. Interviewing key contact persons and stakeholders to obtain additional information could have strengthened the review; however, we did not interview any key individuals or stakeholders because it was beyond the scope of this review. Nonetheless, we recommend that future studies consider incorporating interviews to explore the perspectives of key stakeholders.

Conclusions

Health apps are increasingly being used by patients to manage their health, and sub-Saharan African countries can leverage these apps to advance their progress toward achieving SDG 3 (good health and well-being) and UHC, especially given the rapid advancement of AI and big data. However, our study has established that the regulation of health apps in sub-Saharan Africa is inadequate to ensure that health apps are technically reliable and clinically safe; interoperable across systems; compliant with the principles of confidentiality of information and data privacy; culturally appropriate and relevant; and accessible to everyone regardless of gender, ethnicity, location, or income. Therefore, the region can learn from the experiences of some high-income countries such as the United Kingdom and Germany to develop and implement a robust and responsive regulatory system that supports the widespread adoption of safe, effective, and beneficial health apps for its population.

Following the publication of this review, a summary of the findings will be disseminated to the relevant organizations. In addition, the key findings will be summarized and presented at national, regional, and international conferences.

Acknowledgments

The authors would like to thank Rebecca Jones, the Library Manager and Liaison Librarian at Charing Cross Library, who advised and assisted with the search strategy for this study. This work is part of the PhD research of BAB, which is sponsored by the government of Nigeria. AM and JC were supported by the National Institute for Health and Care Research (NIHR) Applied Research Collaboration Northwest London (NIHR200180). The views expressed in this publication are those of the authors and not necessarily those of the government of Nigeria or the NIHR or the Department of Health and Social Care. In the Results and Discussion sections, Microsoft Copilot in Bing [ 159 ] was used to help summarize and modify a few texts as well as suggest some citations.

Data Availability

The search strategy for PubMed, Scopus, and the World Health Organization AIM is presented in Multimedia Appendix 1 . All data generated or analyzed during this study are included in this published article (and its supplementary information files). The documents analyzed are available directly from the relevant institutional websites, ICTworks repository [ 44 ] or upon request from the relevant government departments in each country. Additionally, documents in the list of references that are not accessible on the web can be solicited from the corresponding author on reasonable request.

Authors' Contributions

BAB and JC conceived the study. BAB designed the study with contributions from JC and NM. BAB drafted the manuscript, and JC, NM, AM, SI, KPF, BIH, and NU read and contributed to it. AM was the clinical lead, and JC acted as a guarantor for this study. The final manuscript was read and approved by all the authors.

Conflicts of Interest

None declared.

PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist.

Database search strategy.

Details of included documents.

Mapping of the stakeholders according to their potential role in regulating health apps for self-management.

• Aitken M, Nass D. Digital health trends 2021: innovation, evidence, regulation, and adoption. IQVIA Institute for Human Data Science. 2021. URL: https:/​/www.​iqvia.com/​-/​media/​iqvia/​pdfs/​institute-reports/​digital-health-trends-2021/​iqvia-institute-digital-health-trends-2021.​pdf?&_=1669449368070 [accessed 2022-11-26]
• Mobile app threat landscape report. RiskIQ. 2020. URL: https://www.riskiq.com/2020-mobile-threat-landscape-report-thank-you/ [accessed 2021-07-19]
• El-Sappagh S, Ali F, Hendawi A, Jang JH, Kwak KS. A mobile health monitoring-and-treatment system based on integration of the SSN sensor ontology and the HL7 FHIR standard. BMC Med Inform Decis Mak. May 10, 2019;19(1):97. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Labrique AB, Vasudevan L, Kochi E, Fabricant R, Mehl G. mHealth innovations as health system strengthening tools: 12 common applications and a visual framework. Glob Health Sci Pract. Aug 06, 2013;1(2):160-171. [ FREE Full text ] [ CrossRef ]
• Adepoju IOO, Albersen BJA, De Brouwere V, van Roosmalen J, Zweekhorst M. mHealth for clinical decision-making in sub-Saharan Africa: a scoping review. JMIR Mhealth Uhealth. Mar 23, 2017;5(3):e38. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Vegesna A, Tran M, Angelaccio M, Arcona S. Remote patient monitoring via non-invasive digital technologies: a systematic review. Telemed J E Health. Jan 2017;23(1):3-17. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Use of appropriate digital technologies for public health: Report by the Director-General. World Health Organization. 2016. URL: https://iris.who.int/handle/10665/274134 [accessed 2023-05-06]
• El-Osta A, Rowe C, Majeed A. Developing a shared definition of self-driven healthcare to enhance the current healthcare delivery paradigm. J R Soc Med. Nov 2022;115(11):424-428. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hussein R. A review of realizing the universal health coverage (UHC) goals by 2030: part 2- what is the role of eHealth and technology? J Med Syst. Jul 2015;39(7):72. [ CrossRef ] [ Medline ]
• Sustainable development goal 3: Ensure healthy lives and promote well-being for all at all ages. United Nations. URL: https://sdgs.un.org/goals/goal3 [accessed 2023-05-07]
• Coronavirus: apps to help the elderly. Organisation for the Review of Care and Health Apps. 2020. URL: https://orchahealth.com/coronavirus-apps-to-help-the-elderly/ [accessed 2021-07-19]
• Huckvale K, Adomaviciute S, Prieto JT, Leow MKS, Car J. Smartphone apps for calculating insulin dose: a systematic assessment. BMC Med. May 06, 2015;13:106. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Huckvale K, Car M, Morrison C, Car J. Apps for asthma self-management: a systematic assessment of content and tools. BMC Med. Nov 22, 2012;10:144. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Huckvale K, Morrison C, Ouyang J, Ghaghda A, Car J. The evolution of mobile apps for asthma: an updated systematic assessment of content and tools. BMC Med. Mar 23, 2015;13:58. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Huckvale K, Prieto JT, Tilney M, Benghozi PJ, Car J. Unaddressed privacy risks in accredited health and wellness apps: a cross-sectional systematic assessment. BMC Med. Sep 07, 2015;13:214. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Huckvale K, Torous J, Larsen ME. Assessment of the data sharing and privacy practices of smartphone apps for depression and smoking cessation. JAMA Netw Open. Apr 05, 2019;2(4):e192542. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ndlovu K, Mars M, Scott RE. Interoperability frameworks linking mHealth applications to electronic record systems. BMC Health Serv Res. May 13, 2021;21(1):459. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kenya standards and guidelines for mHealth systems. Kenya Ministry of Health. 2017. URL: https:/​/www.​health.go.ke/​wp-content/​uploads/​2020/​02/​Revised-Guidelines-For-Mhealth-Systems-May-Version.​pdf [accessed 2023-03-21]
• Standard for electronic health record system (EHRs) in Ethiopia. Ethiopia Minister of Health. 2021. URL: https:/​/registry.​betterehealth.eu/​ehealth-policies/​standard-electronic-health-record-system-ehrs-ethiopia [accessed 2023-04-21]
• National health normative standards framework for digital health interoperability in South Africa. South Africa Department of Health. 2021. URL: https://www.health.gov.za/wp-content/uploads/2022/10/HNSF_Gazette_21_October_2022.pdf [accessed 2023-05-15]
• Ferretti A, Ronchi E, Vayena E. From principles to practice: benchmarking government guidance on health apps. Lancet Digit Health. Jun 2019;1(2):e55-e57. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Diao JA, Venkatesh KP, Raza MM, Kvedar JC. Multinational landscape of health app policy: toward regulatory consensus on digital health. NPJ Digit Med. May 11, 2022;5(1):61. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Essén A, Stern AD, Haase CB, Car J, Greaves F, Paparova D, et al. Health app policy: international comparison of nine countries' approaches. NPJ Digit Med. Mar 18, 2022;5(1):31. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Brown SA, Garcia AA, Kouzekanani K, Hanis CL. Culturally competent diabetes self-management education for Mexican Americans: the Starr County border health initiative. Diabetes Care. Feb 2002;25(2):259-268. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Chaney SC, Mechael P. Self-Care Trailblazer Group. 2020. URL: https://media.psi.org/wp-content/uploads/2020/09/31000510/Digital-Self-Care-Final.pdf [accessed 2021-05-20]
• Kanzaveli T. Healthcare: shiftingfrom “one size fits all” to “one size fits one”. Medium. 2017. URL: https:/​/tkanzaveli.​medium.com/​healthcare-shifting-from-one-size-fits-all-to-one-size-fits-one-d56136ded705 [accessed 2022-03-04]
• Myth 1 – one app will fit all!. Organisation for the Review of Care and Health Apps. URL: https://orchahealth.com/myth-1-one-app-will-fit-all/ [accessed 2022-03-04]
• Aitken M, Lyle J. Patient adoption of mHealth: use, evidence and remaining barriers to mainstream acceptance. IQVIA Institute for Human Data Science. Sep 2015. URL: https://www.iqvia.com/-/media/iqvia/pdfs/institute-reports/patient-adoption-of-mhealth.pdf [accessed 2021-05-21]
• Mechael P, Batavia H, Kaonga N. Barriers and gaps affecting mhealth in low and middle income countries: policy white paper. Center for Global Health and Economic Development Earth Institute, Columbia University. 2010. URL: http://www.globalproblems-globalsolutions-files.org/pdfs/mHealth_Barriers_White_Paper.pdf [accessed 2021-03-24]
• Bene BA, Ibeneme S, Fadahunsi KP, Harri BI, Ukor N, Mastellos N, et al. Regulatory standards and guidance for the use of health applications for self-management in Africa: scoping review protocol. BMJ Open. Feb 11, 2022;12(2):e058067. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Aitken M, Gauntlett C. Patient apps for improved healthcare: from novelty to mainstream. IMS Institute for Healthcare Informatics. 2013. URL: https://ignacioriesgo.es/wp-content/uploads/2014/03/iihi_patient_apps_report_editora_39_2_1.pdf [accessed 2024-03-10]
• National eHealth Strategy Toolkit. World Health Organization, International Telecommunication Union. 2012. URL: https://www.itu.int/pub/D-STR-E_HEALTH.05-2012 [accessed 2021-06-28]
• Global strategy on digital health 2020-2025. World Health Organization. 2021. URL: https://www.who.int/docs/default-source/documents/gs4dhdaa2a9f352b0445bafbc79ca799dce4d.pdf [accessed 2021-06-23]
• Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. Feb 2005;8(1):19-32. [ FREE Full text ] [ CrossRef ]
• Anderson S, Allen P, Peckham S, Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Res Policy Syst. Jul 09, 2008;6:7. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Levac D, Colquhoun H, O'Brien KK. Scoping studies: advancing the methodology. Implement Sci. Sep 20, 2010;5:69. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Peters MDJ, Marnie C, Tricco AC, Pollock D, Munn Z, Alexander L, et al. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synth. Oct 2020;18(10):2119-2126. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. Oct 02, 2018;169(7):467-473. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Leitner C, Potenziani D. Mendeley reference manager. Mendeley. 2022. URL: https://www.mendeley.com/reference-management/reference-manager [accessed 2022-08-03]
• Better systematic review management. Covidence. URL: https://www.covidence.org/ [accessed 2023-02-13]
• Franco-Trigo L, Fernandez-Llimos F, Martínez-Martínez F, Benrimoj SI, Sabater-Hernández D. Stakeholder analysis in health innovation planning processes: A systematic scoping review. Health Policy. Oct 2020;124(10):1083-1099. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Walt G, Gilson L. Reforming the health sector in developing countries: the central role of policy analysis. Health Policy Plan. Dec 1994;9(4):353-370. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Digital health: a call for government leadership and cooperation between ICT and health. Broadband Commission. 2017. URL: https://broadbandcommission.org/wp-content/uploads/2021/09/WGHealth_Report2017-.pdf [accessed 2021-06-28]
• Vota W. Every African country’s national eHealth strategy or digital health policy. ICT works. 2019. URL: https://www.ictworks.org/african-national-ehealth-strategy-policy/ [accessed 2023-12-10]
• Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. Jul 21, 2009;6(7):e1000097. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Standards and guidelines for electronic medical record systems in Kenya. Kenya Ministry of Medical Services, Kenya Ministry of Public Health and Sanitation. 2010. URL: http://guidelines.health.go.ke:8000/media/Standards_and_Guidelines_for_EMR_Systems.pdf [accessed 2023-04-21]
• Kenya standards and guidelines for E-health systems interoperability. Kenya Ministry of Health, AfyaInfo Project. 2014. URL: https://pdf.usaid.gov/pdf_docs/PA00TB2K.pdf [accessed 2023-03-21]
• Health sector ICT standards and guidelines. Kenya Ministry of Health. 2013. URL: https://www.medbox.org/pdf/5e148832db60a2044c2d2895 [accessed 2023-03-21]
• Health information exchange standard operating procedure (SOP) and guideline. Nigeria Federal Ministry of Health. Jul 2020.
• National eHealth strategy 2018-2022. Benin Ministry of Health. 2017.
• The eHealth strategy of botswana 2020-2024. Botswana Ministry of Health. URL: https://ehealth.ub.bw/bhdc/Docs/MOH%20ehealth%20Strategy%20Book%20A4.pdf [accessed 2023-04-22]
• Health sector digital strategy 2016-2020. Burkina Faso Ministry of Health.
• National health informatics development plan of Burundi. Burundi Ministry of Public Health. 2015.
• The 2020-2024 national digital health strategic plan. Cameroon Ministry of Public Health. 2020.
• National eHealth strategy 2017-2021. Comoros Ministry of Health. 2016.
• eHealth strategic plan. Cote d’Ivoire Minister of Health and Public Hygiene. 2011.
• National development plan for health informatics. Democratic Republic of Congo Ministry of Public Health. 2014.
• Kingdom of Swaziland eHealth strategy 2016 - 2020. Kingdom of Swaziland Ministry Of Health. 2016.
• Information revolution strategic plan (2018-2025). Ethiopia Ministry of Health. 2018.
• Strategic master plan of the health information system of the Gabon. Gabon Ministry of Public Health and Population. 2017.
• National e-Health strategy. Ghana Ministry of Health. 2010.
• Kenya national e-Health strategy. Kenya Ministry of Medical Services, Kenya Ministry of Public Health & Sanitation. 2011.
• Kenya national eHealth policy 2016-2030. Kenya Ministry of Health. 2016.
• National strategy - Liberia - 2016-2021. Liberia Ministry of Health. 2016.
• Strategic plan for strengthening the health information system of Madagascar 2018–2022. Madagascar Ministry of Public Health. 2017.
• National digital health strategy 2020-2025. Malawi Ministry of Health. 2020.
• National eHealth policy in Mali. Mali Ministry of Health and Public Hygiene. 2013.
• Health 2015: seamless continuity of care. Mauritius Ministry of Health and Quality of Life. 2015.
• Strategic plan of information system for health 2009-2014. Mozambique Ministry of Health. 2009.
• National eHealth strategy 2019-2023. Niger Ministry of Public Health. 2018.
• National digital health strategy 2021-2025. Nigeria Federal Ministry of Health. 2021.
• National digital health policy. Nigeria Federal Ministry of Health. 2021.
• National digital health strategic plan 2018-2023. Rwanda Ministry of Health. 2018. URL: https://elearning.helinanet.org/mod/resource/view.php?id=890 [accessed 2023-05-09]
• Strategic plan for digital health 2018-2023. Senegal Ministry of Health and Social Action. 2018.
• National digital health strategy 2018-2023. Sierra Leone Ministry of Health and Sanitation, Sierra Leone Ministry of Information and Communication. 2018.
• The national digital health strategy 2019 – 2024. Tanzania Ministry of Health, Community Development, Gender, Elderly and Children. 2019.
• National digital health strategy for South Africa 2019 - 2024. South Africa Department of Health. 2019.
• Strategic plan for the development of eHealth in Togo 2013-2015. Togo Ministry of Health. 2012.
• Uganda national eHealth policy. Uganda Ministry of Health. 2016.
• Uganda national eHealth strategy 2017 - 2021. Uganda Ministry of Health. URL: https://health.go.ug/sites/default/files/National%20e_Health%20Strategy_0.pdf [accessed 2023-05-16]
• National eHealth strategy 2017-2021. Zambia Ministry of Health. 2017.
• Zimbabwe’s E-Health strategy 2012-2017. Ministry of Health and Child Welfare. 2012.
• National eHealth strategy 2021-2025. Namibia Ministry of Health & Social Services. 2021. URL: https://www.scribd.com/document/639371316/eHealth-Strategy-Namibia-2021# [accessed 2023-05-13]
• Health sector ICT policy and strategy. Ghana Ministry of Health. 2005. URL: https://www.moh.gov.gh/wp-content/uploads/2016/02/Health-Sector-ICT-Policy-and-Strategy.pdf [accessed 2023-05-08]
• Adebola OJ. Beyond national digital health strategy: final report of end term evaluation for the National Health ICT Strategic Framework 2015-2020. Nigeria Federal Ministry of Health. May 2021.
• National Health ICT Strategic Framework 2015 - 2020. Nigeria Federal Ministry of Health. 2016. URL: https://www.health.gov.ng/doc/HealthICTStrategicFramework.pdf [accessed 2023-05-16]
• Digital health blueprint. Ethiopia Ministry of Health. 2021. URL: http:/​/repository.​iifphc.org/​bitstream/​handle/​123456789/​1658/​Ethiopian-Digital-Health-Blueprint.​pdf?sequence=1&isAllowed=y [accessed 2023-05-16]
• Kenya health information systems interoperability framework. Kenya Ministry of Health. 2020. URL: https:/​/www.​data4sdgs.org/​sites/​default/​files/​services_files/​Kenya%20Health%20Information%20Systems%20Interoperability%20Framework.​pdf [accessed 2023-05-16]
• National community health digitization strategy 2020-2025. Kenya Ministry of Health, Division of Community Health Services. 2021. URL: https:/​/www.​eahealth.org/​sites/​www.eahealth.org/​files/​content/​attachments/​2021-08-02/​eCHIS-Strategy-2020-2025.​pdf [accessed 2023-05-16]
• Leitner C, Potenziani D. Health information systems interoperability in Liberia. IntraHealth International. 2016. URL: https://elearning.helinanet.org/mod/resource/view.php?id=938 [accessed 2023-05-16]
• Narrative for 2022 national digital health annual operational plan (AOP). Nigeria Federal Ministry of Health. 2022.
• Tanzania digital health investment road map 2017-2023. Tanzania Ministry of Health, Community Development, Gender, Elderly and Children, President’s Office Regional Administration and Local Government. 2017.
• Yu H. Regulation of digital health technologies in the European Union: intended versus actual use. In: Cohen GI, Minssen T, Price II NW, Robertson C, Shachar C, editors. The Future of Medical Device Regulation: Innovation and Protection. Cambridge. Cambridge University Press; Mar 31, 2022;103-114.
• Policy for device software functions and mobile medical applications: guidance for industry and Food and Drug Administration staff. U.S. Food and Drug Administration. 2022. URL: https://www.fda.gov/media/80958/download [accessed 2023-10-10]
• Gordon WJ, Landman A, Zhang H, Bates DW. Beyond validation: getting health apps into clinical practice. NPJ Digit Med. 2020;3:14. [ FREE Full text ] [ CrossRef ] [ Medline ]
• FDA clears mobile medical app to help those with opioid use disorder stay in recovery programs. U.S. Food and Drug Administration. 2018. URL: https:/​/www.​fda.gov/​news-events/​press-announcements/​fda-clears-mobile-medical-app-help-those-opioid-use-disorder-stay-recovery-programs [accessed 2021-01-27]
• Digital maturity model: achieving digital maturity to drive growth. Deloitte. 2018. URL: https:/​/www2.​deloitte.com/​content/​dam/​Deloitte/​global/​Documents/​Technology-Media-Telecommunications/​deloitte-digital-maturity-model.​pdf [accessed 2021-10-20]
• May E. How digital health apps are empowering patients. Deloitte. 2021. URL: https:/​/www2.​deloitte.com/​us/​en/​blog/​health-care-blog/​2021/​how-digital-health-apps-are-empowering-patients.​html [accessed 2023-10-06]
• Guidance: criteria for health app assessment. Public Health England. 2017. URL: https:/​/www.​gov.uk/​government/​publications/​health-app-assessment-criteria/​criteria-for-health-app-assessment [accessed 2023-10-16]
• Clinical risk management standards. National Health Service Digital. 2020. URL: https://digital.nhs.uk/services/clinical-safety/clinical-risk-management-standards [accessed 2023-10-28]
• Report a problem with a medicine or medical device. Gov.uk. URL: https://www.gov.uk/report-problem-medicine-medical-device [accessed 2023-11-07]
• Digital technology assessment criteria for health and social care (DTAC) - Version 1.0. National Health Service X. 2021. URL: https:/​/view.​officeapps.live.com/​op/​view.​aspx?src=https%3A%2F%2Ftransform.​england.​nhs.​uk%2Fmedia%2Fdocuments%2FDTAC_version_1.​0_FINAL_updated_16.​04.​odt&wdOrigin=BROWSELINK [accessed 2023-11-07]
• Data protection impact assessment: NHS login - formerly Citizen Identity. National Health Service Digital. 2022. URL: https://digital.nhs.uk/services/nhs-login/data-protection-impact-assessment [accessed 2023-11-07]
• Risks and data protection impact assessments (DPIAs). Information Commissioner’s Office. URL: https:/​/ico.​org.uk/​for-organisations/​uk-gdpr-guidance-and-resources/​accountability-and-governance/​accountability-framework/​risks-and-data-protection-impact-assessments-dpias/​ [accessed 2023-11-07]
• A guide to good practice for digital and data-driven health technologies. Department of Health and Social Care. 2021. URL: https:/​/www.​gov.uk/​government/​publications/​code-of-conduct-for-data-driven-health-and-care-technology/​initial-code-of-conduct-for-data-driven-health-and-care-technology [accessed 2023-10-30]
• Interoperability in healthcare. Healthcare Information and Management Systems Society (HIMSS). 2023. URL: https://www.himss.org/resources/interoperability-healthcare [accessed 2023-10-17]
• DAPB4020: UK core Fast Healthcare Interoperability Resources (FHIR) release 4 (R4) governance. National Health Service Digital. 2022. URL: https:/​/digital.​nhs.uk/​data-and-information/​information-standards/​information-standards-and-data-collections-including-extractions/​publications-and-notifications/​standards-and-collections/​dapb4020-uk-core-fhir-r4-governance [accessed 2023-10-17]
• Fast Healthcare Interoperability Resources (FHIR). National Health Service Digital. 2022. URL: https://digital.nhs.uk/services/fhir-apis [accessed 2023-10-17]
• FHIR Interoperability Basics: 4 things to know. Health IT Analytics. 2022. URL: https://healthitanalytics.com/news/4-basics-to-know-about-the-role-of-fhir-in-interoperability [accessed 2023-11-07]
• Giordanengo A, Bradway M, Pedersen R, Grøttland A, Hartvigsen G, Årsand E. Integrating data from apps, wearables and personal electronic health record (pEHR) systems with clinicians’ electronic health records (EHR) systems. Int J Integr Care. Nov 09, 2016;16(5):16. [ FREE Full text ] [ CrossRef ]
• A plan for digital health and social care. Department of Health & Social Care. 2022. URL: https:/​/www.​gov.uk/​government/​publications/​a-plan-for-digital-health-and-social-care/​a-plan-for-digital-health-and-social-care [accessed 2022-12-01]
• Ryu B, Kim N, Heo E, Yoo S, Lee K, Hwang H, et al. Impact of an electronic health record-integrated personal health record on patient participation in health care: development and randomized controlled trial of MyHealthKeeper. J Med Internet Res. Dec 07, 2017;19(12):e401. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Winter A, Takabayashi K, Jahn F, Kimura E, Engelbrecht R, Haux R, et al. Quality requirements for electronic health record systems*. A Japanese-German information management perspective. Methods Inf Med. Aug 07, 2017;56(7):e92-e104. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Wachter RM. Making IT work: harnessing the power of health information technology to improve care in England. National Advisory Group on Health Information Technology. 2016. URL: https:/​/assets.​publishing.service.gov.uk/​government/​uploads/​system/​uploads/​attachment_data/​file/​550866/​Wachter_Review_Accessible.​pdf [accessed 2021-07-22]
• Framework on integrated people-centred health services (IPCHS). World Health Organisation. 2023. URL: https:/​/www.​who.int/​teams/​integrated-health-services/​clinical-services-and-systems/​service-organizations-and-integration [accessed 2023-06-05]
• Ibeneme S, Karamagi H, Muneene D, Goswami K, Chisaka N, Okeibunor J. Strengthening health systems using innovative digital health technologies in Africa. Front Digit Health. 2022;4:854339. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ibeneme S, Ukor N, Ongom M, Dasa T, Muneene D, Okeibunor J. Strengthening capacities among digital health leaders for the development and implementation of national digital health programs in Nigeria. BMC Proc. 2020;14(Suppl 10):9. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Delivering safe digital health. Organisation for the Review of Care and Health Apps. URL: https://orchahealth.com/ [accessed 2023-10-22]
• Mamuye AL, Yilma TM, Abdulwahab A, Broomhead S, Zondo P, Kyeng M, et al. Health information exchange policy and standards for digital health systems in Africa: a systematic review. PLOS Digit Health. Oct 2022;1(10):e0000118. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Digital health platform handbook: Building a digital information infrastructure (infostructure) for health. World Health Organization, International Telecommunication Union. 2022. URL: https://www.itu.int/dms_pub/itu-d/opb/str/D-STR-E_HEALTH.10-2020-PDF-E.pdf [accessed 2021-05-22]
• Framework for involving patients in patient safety 2021. National Health Service England. 2021. URL: https://www.england.nhs.uk/patient-safety/framework-for-involving-patients-in-patient-safety/ [accessed 2023-03-23]
• Olesch A. Towards harmonised EU landscape for digital health: summary of the roundtable discussions in selected EIT Health InnoStars countries. EIT Health InnoStars. Jan 2023. URL: https://eithealth.eu/wp-content/uploads/2023/02/EIT_Health_DiGA_report_Jan2023.pdf [accessed 2023-10-10]
• Grieb J, Tschammler D, Färber C, Woitz S. Digital health laws and regulations germany. Global Legal Group. 2023. URL: https://iclg.com/practice-areas/digital-health-laws-and-regulations/germany [accessed 2023-11-03]
• Human regulatory: medical devices. European Medicines Agency. URL: https://www.ema.europa.eu/en/human-regulatory/overview/medical-devices [accessed 2023-10-12]
• Strengthening the capacity for regulation of medical products in the African region. World Health Organization Regional Office for Africa. 2013. URL: https://iris.who.int/bitstream/handle/10665/94308/AFR_RC63_7.pdf?sequence=1 [accessed 2023-10-17]
• Ncube BM, Dube A, Ward K. Establishment of the African Medicines Agency: progress, challenges and regulatory readiness. J Pharm Policy Pract. Mar 08, 2021;14(1):29. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Post market surveillance system. European Union Medical Device Regulation. 2023. URL: https://eumdr.com/post-market-surveillance-system/ [accessed 2023-10-31]
• Dayal R. Effective post-market surveillance for medical devices: An essential part of medical devices regulation (MDR). Capgemini. 2020. URL: https:/​/www.​capgemini.com/​insights/​expert-perspectives/​effective-post-market-surveillance-for-medical-devices-an-essential-part-of-mdr/​ [accessed 2023-10-31]
• NHS app library reaches 70 apps in honour of the NHS birthday. Northampton General Hospital NHS Trust. 2018. URL: https:/​/www.​northamptongeneral.nhs.uk/​News/​News-Archive/​2018/​NHS-App-Library-reaches-70-apps-in-honour-of-the-NHS-birthday.​aspx [accessed 2023-09-21]
• Developers invited to add to NHS apps library. National Health Service Digital. 2018. URL: https://digital.nhs.uk/news/2018/developers-invited-to-add-to-nhs-apps-library [accessed 2023-09-22]
• The NHS apps library has closed. National Health Service Digital. 2021. URL: https:/​/digital.​nhs.uk/​services/​nhs-apps-library#:~:text=The%20NHS%20Apps%20Library%20was%20decommissioned%20in%20December%202021.​&text=Further%20information%20can%20be%20found%20on%20the%20NHS.​UK%20website [accessed 2023-09-22]
• Larsen ME, Huckvale K, Nicholas J, Torous J, Birrell L, Li E, et al. Using science to sell apps: evaluation of mental health app store quality claims. NPJ Digit Med. 2019;2:18. [ FREE Full text ] [ CrossRef ] [ Medline ]
• About the NHS app. National Health Service. Dec 4, 2023. URL: https://www.nhs.uk/nhs-app/about-the-nhs-app/ [accessed 2023-09-22]
• Scope of registration: regulated activities. Care Quality Commission. 2022. URL: https://www.cqc.org.uk/guidance-providers/scope-registration-regulated-activities [accessed 2023-11-05]
• Distributing great apps into health and care services across the world. Organisation for the Review of Care and Health Apps. 2020. URL: https://orchahealth.com/wp-content/uploads/2020/12/Health-and-Care-1.pdf [accessed 2023-10-09]
• Our founder, our story and our values: we exist to make digital health healthy. Organisation for the Review of Care and Health Apps. URL: https://orchahealth.com/about-us/ [accessed 2023-10-09]
• Health app library: empower your community with safe access to health apps and digital health products. Organisation for the Review of Care and Health Apps. URL: https:/​/orchahealth.​com/​our-products/​health-app-library/​#:~:text=A%20Health%20App%20Library%20is,on%20the%20Health%20App%20Library [accessed 2023-10-09]
• Digital technology assessment criteria (DTAC). National Health Service X. URL: https://www.nhsx.nhs.uk/key-tools-and-info/digital-technology-assessment-criteria-dtac/ [accessed 2023-10-09]
• Evidence standards framework (ESF) for digital health technologies. National Institute for Health and Care Excellence. 2023. URL: https:/​/www.​nice.org.uk/​about/​what-we-do/​our-programmes/​evidence-standards-framework-for-digital-health-technologies [accessed 2023-10-08]
• Tsang L, Kerr-Peterson H. UK NICE updates its evidence standards framework for data-driven digital health technologies. Ropes & Gray. 2022. URL: https:/​/www.​ropesgray.com/​en/​insights/​alerts/​2022/​10/​uk-nice-updates-its-evidence-standards-framework-for-data-driven-digital-health-technologies [accessed 2023-10-09]
• Guidance: medical device stand-alone software including apps (including IVDMDs). Medicines and healthcare products regulatory agency. 2023. URL: https:/​/assets.​publishing.service.gov.uk/​government/​uploads/​system/​uploads/​attachment_data/​file/​1168485/​Medical_device_stand-alone_software_including_apps__including_IVDMDs_.​pdf [accessed 2023-10-09]
• Digital therapeutics in the United Kingdom. Digital Therapeutics Alliance. 2021. URL: https://dtxalliance.org/wp-content/uploads/2021/06/DTA_DTx-Overview_UK.pdf [accessed 2023-10-09]
• Transforming global healthcare by advancing digital therapeutics. Digital Therapeutics Alliance. 2023. URL: https://dtxalliance.org/ [accessed 2023-10-10]
• International Organization for Standardization (ISO) digital therapeutic definition. Digital Therapeutic Alliance. Jun 2023. URL: https://dtxalliance.org/wp-content/uploads/2023/06/DTA_FS_ISO-Definition.pdf [accessed 2023-10-09]
• Ndomondo-Sigonda M, Miot J, Naidoo S, Dodoo A, Kaale E. Medicines regulation in Africa: current state and opportunities. Pharmaceut Med. 2017;31(6):383-397. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Chinele J. East Africa shows solid support for African Medicines Agency treaty. Health Policy Watch. Aug 16, 2023. URL: https://healthpolicy-watch.news/east-africa-shows-solid-support-for-african-medicines-agency-treaty/ [accessed 2023-10-09]
• Treaty for the establishment of the African Medicines Agency 2019. African Union. 2019. URL: https://au.int/sites/default/files/treaties/36892-treaty-0069_-_ama_treaty_e.pdf [accessed 2023-10-17]
• European Medicines Agency: about us. European Medicines Agency. Mar 1, 2023. URL: https://www.ema.europa.eu/en/documents/other/about-us-european-medicines-agency-ema_en.pdf [accessed 2023-10-18]
• Bryson JM. What to do when stakeholders matter. Public Adm Rev. Mar 2004;6(1):21-53. [ FREE Full text ] [ CrossRef ]
• Iyawa G, Herselman M, Botha A. Potential stakeholders and perceived benefits of a digital health innovation ecosystem for the Namibian context. Procedia Computer Science. 2017;121:431-438. [ CrossRef ]
• Ferretti V. From stakeholders analysis to cognitive mapping and multi-attribute value theory: an integrated approach for policy support. European Journal of Operational Research. Sep 2016;253(2):524-541. [ CrossRef ]
• Brugha R, Varvasovszky Z. Stakeholder analysis: a review. Health Policy Plan. Sep 2000;15(3):239-246. [ CrossRef ] [ Medline ]
• Schmeer K. Guidelines for conducting a stakeholder analysis 1999. Partnerships for Health Reform, Abt Associates. 1999. URL: https://www.ktecop.ca/wordpress/wp-content/uploads/guidelines-stakeholder-analysis-PHR-1999.pdf [accessed 2023-10-17]
• Gilmour J, Beilin R. Stakeholder mapping for effective risk assessment and communication. Australian Centre of Excellence for Risk Analysis, University of Melbourne. Apr 2007. URL: https://cebra.unimelb.edu.au/__data/assets/pdf_file/0006/2220990/gilmour0609.pdf [accessed 2023-10-17]
• Quality, service improvement and redesign tools: stakeholder analysis. National Health Service England, National Health Service Improvement. 2022. URL: https://www.england.nhs.uk/wp-content/uploads/2022/02/qsir-stakeholder-analysis.pdf [accessed 2023-10-20]
• Craven MP, Lang AR, Martin JL. Developing mHealth apps with researchers: multi-stakeholder design considerations. Springer; 2014. Presented at: Third International Conference, DUXU 2014, held as a part of HCI International; June 22-27, 2014;15-24; Heraklion, Greece. URL: https://doi.org/10.1007/978-3-319-07635-5_2 [ CrossRef ]
• How to encourage stakeholder participation. SustaiNet Software International. URL: https://sustainet.com/how-to-encourage-stakeholder-participation/ [accessed 2023-10-20]
• Stakeholder engagement. Organisation for Economic Cooperation and Development. URL: https:/​/www.​oecd.org/​governance/​better-international-rulemaking/​compendium/​keyprinciples/​stakeholderengagement.​htm [accessed 2023-10-20]
• Microsoft Copilot in Bing. Microsoft. URL: https://www.bing.com/chat?form=NTPCHB [accessed 2023-03-15]

Abbreviations

Edited by A Mavragani; submitted 19.05.23; peer-reviewed by N O'Brien, A Essén; comments to author 07.09.23; revised version received 08.12.23; accepted 23.02.24; published 11.04.24.

©Benard Ayaka Bene, Sunny Ibeneme, Kayode Philip Fadahunsi, Bala Isa Harri, Nkiruka Ukor, Nikolaos Mastellos, Azeem Majeed, Josip Car. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 11.04.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

New building powers for universities unveiled in proposed Ontario housing legislation

Province suggests legislation will focus on cutting 'red tape' to build more homes.

Social Sharing

The Ford government is tabling new housing legislation aimed at building homes faster in the province and achieving its goal of having 1.5 million new homes in Ontario by 2031.

Housing Minister Paul Calandra says the government's "Cutting Red Tape to Build More Homes Act" includes a number of changes, such as giving universities more power to build student housing on its land, giving municipalities the ability to address stalled developments and will prioritize ready-to-go housing projects.

The act will, if passed, require ministries to develop new business service standards for building permits. The government says it would reduce regulatory burden, make processes more transparent and cut down on project delays.

"These measures recognize the struggles that our municipal partners have faced in building homes and are targeted at removing those obstacles," said Housing Minister Paul Calandra in a statement. "We're not going to micromanage and dictate a 'one-size-fits-all' approach across the province. Municipalities know their communities best — they know where it makes sense to build homes."

The legislation would also exempt public universities from the Planning Act to speed up the building of new student housing units.

"Removing barriers faced by universities when building student housing shows how we can find innovative ways to get shovels in the ground sooner," said Jill Dunlop, minister of colleges and universities, in a statement.

"By requiring postsecondary institutions to publish their student housing policies, we're also ensuring students have access to and are aware of student housing options that are safe, affordable and within an easy commute to campus," she added.

New legislation is 'weak': opposition

The government says by exempting universities from the Planning Act, schools could build higher density student residences. Universities would also be able to avoid planning application fees.

Opposition parties said Tuesday they were concerned about the government inflating its housing start targets by including student housing. The government confirmed Wednesday this legislation will not include student housing toward its housing numbers at this point.

Commenting Wednesday, NDP Leader Marit Stiles says the legislation doesn't go far enough to build homes quickly and efficiently.

"Here in Ontario, I'm sorry to say today's legislation by the Conservative government lets all of those people down again," she told reporters. "This is a weak bill. It lacks ambition, it lacks boldness and it lacks urgency."

"A chunk of what this bill presents is just reversals to bad government policies," Stiles added.

The bill reverses some changes brought in by the Ford government through Bill 23, its "More Homes Built Faster Act", in 2022. It gets rid of an approach to development charges to only offer savings for developers building affordable housing.

Ontario Liberal MPP Adil Shamji says this is becoming common practice by this government.

"Every few months we have a new housing bill, and it usually walks back something that was in the last one," he said.

• Ontario considering counting student homes in housing goal
• Ontario home construction up, but still off pace for target

The government set a goal of constructing 1.5 million new homes by 2031 to alleviate a severe housing crunch that has contributed to high real estate prices and rents.

Ontario's spring budget shows the pace of new home construction is picking up in the province, with 88,000 housing starts projected in 2024, but is still far off the levels needed to get to the government's target. Calandra has previously indicated that Ontario needed to be building at least 125,000 homes this year, ramping up to at least 175,000 per year in the near future.

Building closer to transit

The government said it is also doing away with parking minimums for new builds near transit stations. The housing minister says it will allow for more mixed-use communities with housing, retail and office space close to public transit.

"We're building homes faster and at a lower cost," he said. "Including by letting homeowners and builders decide the right number of parking spaces in new residential developments near transit."

Mass timber construction will also be amended through this housing bill, which will soon allow developers to pursue 18-storey mass timber buildings. Currently, those developments are limited to 12 storeys high. The housing minister says it's just one type of standardized housing the province wants to increase access to.

"It can be all forms of housing," Calandra said. "Single detached, bungalows, townhomes — basically anything you can think of. We can work to ensure it is a design that is approved and then provide that to our municipal partners."

The legislation also includes a "use it or lose it" tool, which gives municipalities more power to address stalled developments. The government has previously said forcing developers to act on building permits would put an end to sluggish building.

Municipalities are also able to provide incentives to certain businesses through the legislation to help attract investment and enact the government's reversal of dissolving Peel Region.

This bill would amend the law that would have broken up the upper-tier region to instead task the transition board initially responsible for overseeing that municipal divorce with considering how to make Peel Region more efficient.

With files from The Canadian Press and CBC News

Related Stories

• Ford doubles down on refusal to allow fourplexes provincewide
• Ontario announces more than $1.8B in new funding to help municipalities build homes

Moscow, Russia skyline: the 50 most iconic buildings and best views in Moscow

Navigate forward to interact with the calendar and select a date. Press the question mark key to get the keyboard shortcuts for changing dates.

Navigate backward to interact with the calendar and select a date. Press the question mark key to get the keyboard shortcuts for changing dates.

1 The Moscow Kremlin

3 Bolshoi Theatre

4 gorky central park of culture and leisure, 5 krasnaya ploshchad'.

Track your travel spending and split costs with friends

Plan your trip. Keep your budget organized. Split the cost between tripmates. Wanderlog does it all.

6 St. Basil's Cathedral

7 the state tretyakov gallery, 8 cathedral of christ the saviour, 9 red square, 10 kolomenskoye, 11 museum of cosmonautics, 12 all-russian exhibition center, 13 the pushkin state museum of fine arts, 14 novodevichy convent, 15 state historical museum.

Don’t forget to pack anything

Stay organized with a to-do list, packing list, shopping list, any kind of list.

All travel reservations in 1 place

Never dig through your emails again — access all your flights, lodging, and any reservations in 1 place.

26 Central Armed Forces Museum

27 afimall city, 28 winzavod, 29 mini hotel rest on paveletsky train station, 30 catherine palace, 31 st. peter and paul orthodox church, 32 mayakovskaya, 34 kuskovo estate museum.

Perfect for road trips

See time and distance between places, and optimize your route to get the most of your day.

Collaborate with friends in real time

Plan along with your friends with live syncing and collaborative editing.

46 Grand Kremlin Palace

47 moscow art theatre, 48 moscow cathedral mosque, 49 zachatyevsky monastery, 50 house on the embankment, top searches in moscow, popular road trips from moscow, what's the weather like in moscow.

It depends on when you visit! We've compiled data from NASA on what the weather is like in Moscow for each month of the year: see the links below for more information.

• Weather in Moscow in January
• Weather in Moscow in February
• Weather in Moscow in March
• Weather in Moscow in April
• Weather in Moscow in May
• Weather in Moscow in June
• Weather in Moscow in July
• Weather in Moscow in August
• Weather in Moscow in September
• Weather in Moscow in October
• Weather in Moscow in November
• Weather in Moscow in December

All road trips from Moscow

• Moscow to London drive
• Moscow to Paris drive
• Moscow to St. Petersburg drive
• Moscow to Berlin drive
• Moscow to Prague drive
• Moscow to Amsterdam drive
• Moscow to Budapest drive
• Moscow to Vienna drive
• Moscow to Istanbul drive
• Moscow to Florence drive
• Moscow to Venice drive
• Moscow to Stockholm drive
• Moscow to Milan drive
• Moscow to Krakow drive
• Moscow to Copenhagen drive
• Moscow to Warsaw drive
• Moscow to Helsinki drive
• Moscow to Munich drive
• Moscow to Brussels drive
• Moscow to Tallinn drive
• Moscow to Riga drive
• Moscow to Oslo drive
• Moscow to Turin drive
• Moscow to Hamburg drive
• Moscow to Vilnius drive
• Moscow to Yaroslavl drive
• Moscow to Nizhny Novgorod drive
• Moscow to Kyiv drive
• Moscow to Tula drive
• Moscow to Bruges drive

Explore nearby places

• Likino-Dulevo
• Ivanteyevka
• Orekhovo-Zuevo
• Semyonovskoye
• Ivanovskoye
• Rumyantsevo
• Dzerzhinsky
• Sovkhoz Imeni Lenina
• Dolgoprudny

All related maps of Moscow

• Map of Moscow
• Map of Danki
• Map of Shatura
• Map of Likino-Dulevo
• Map of Uspenskoye
• Map of Gorskoye
• Map of Ivanteyevka
• Map of Reutov
• Map of Domodedovo
• Map of Peresvet
• Map of Vorobyovo
• Map of Bronnitsy
• Map of Orekhovo-Zuevo
• Map of Moskovsky
• Map of Semyonovskoye
• Map of Izmaylovo
• Map of Nikolskoye
• Map of Ivanovskoye
• Map of Marfino
• Map of Govorovo
• Map of Nagornoye
• Map of Mosrentgen
• Map of Bratsevo
• Map of Rumyantsevo
• Map of Mytishchi
• Map of Putilkovo
• Map of Razvilka
• Map of Khimki
• Map of Dzerzhinsky
• Map of Sovkhoz Imeni Lenina
• Map of Dolgoprudny

Moscow throughout the year

• Moscow in January
• Moscow in February
• Moscow in March
• Moscow in April
• Moscow in May
• Moscow in June
• Moscow in July
• Moscow in August
• Moscow in September
• Moscow in October
• Moscow in November
• Moscow in December

Looking for day-by-day itineraries in Moscow?

Get inspired for your trip to Moscow with our curated itineraries that are jam-packed with popular attractions everyday! Check them out here:

• 1-Day Moscow Itinerary
• 2-Day Moscow Itinerary
• 3-Day Moscow Itinerary
• 4-Day Moscow Itinerary
• 5-Day Moscow Itinerary

Best attractions in nearby cities

• Top things to do and attractions in Khimki

Best restaurants in nearby cities

• Where to eat: the best restaurants in Mytishchi
• Where to eat: the best restaurants in Khimki

• Itinerary + map in one view
• Live collaboration
• Auto-import hotels and reservations
• Optimize your route
• Offline access on mobile
• See time and distance between all your places

• About Our Authors
• SRAS: Study Abroad or Online
• GeoHistory: Geopolitics, History, News
• PopKult: Popular Culture
• Museum Studies: Museum Science, High Culture, City Planning
• Students Abroad: Travel Guides

Popular articles

View of the Kremlin and surrounding area from 1825. From The Album of Ancient Views of the Moscow Kremlin , by Ivan Zabelin. Available online from the Yeltsin Library .

The Kremlin: Moscow’s Historical Heart Through the Ages

Published: December 3, 2021

The Moscow Kremlin has long been the main symbol of Moscow and Russia – and for good reason. It was with the Kremlin that city of Moscow officially began and from which it grew. The Prince of Moscow, ruling from the Kremlin and drawing on the growing power of his city, united and conquered the cities and lands around him to create Russia.

While the Kremlin can be seen as something of a constant in Russian history, the Kremlin itself has seen major changes within its walls and to its own status. It has lost and gained buildings. It has changed from the seat of government to an ancillary structure, back to the governmental seat, and finally to a museum complex.

The resource below unites the work of multiple SRAS students writing on Home and Abroad , Challenge Grants , and Online Research Internships to bring you an overarching view of this iconic complex.

The Kremlin Walls

By Hudson Dobbs

The Kremlin was first established in 1156 by Prince Yuri Dolgorukiy. This post-dates the first mention of Moscow, which dates back to 1147, when Prince Dolgorukiy invited Prince Sviatoslav of Chernigov to Moscow to celebrate their alliance.

The actual site of this stronghold has likely been occupied since the second millennium BCE. It likely had fortifications built there as early as the 10th century, by the then-resident Vyatichi, a tribe of Slavic peoples.

Eventually, Prince Dolgorukiy ordered the construction of what would become the Kremlin walls. These first walls were tall and expansive and built out of wood. Although this structure was built for protection, it also served as a symbol for the power and strength of the new city of Moscow.

While the first walls did their job well, they were eventually burnt down by Tatar-Mongol forces and later upgraded to more fire-resistant oak in 1339. As the city grew, the Kremlin also further developed, and with it the popularity of building fortresses in town centers. Cities such as Smolensk, Kazan, Novgorod, and Pskov all constructed a Kremlin of their own. In fact, the word “kremlin” simply means “a fortress within a city.”

By the 13th century, the Kremlin housed the political and spiritual power of the state, with residences, workshops, churches, and state buildings all residing within its walls. In the 1360’s, Prince Dmitry Donskoy rebuilt the walls in limestone and a gleaming white Kremlin soon became the iconic image of Moscow. These walls were credited in helping the city defend itself from sieges by Grand Duke Algirdas of Lithuania in the late 1360’s.

The walls and towers that exist today are still another iteration, and were built on the order of Grand Prince Ivan III, also known as Ivan the Great, from 1485 to 1495. Ivan wanted to build something grander and more worthy of being his residence – something that would be comparable to Constantinople in terms of size and importance.

Wanting what would be specifically a “Third Rome,” Ivan invited Italian architects such as Aristotele Fioravanti and Pietro Antonio Solari. Their involvement is why the current fortress closely resembles castles of Northern Italy. Its red brick made the Kremlin unique for the time, as it was the first structure in Russia built from such material.

These brick walls have stood, with minor adjustments, since that time. One noticeable change came in the late 1600s, when Tsar Fyodor Alekseevich ordered the red brick to be whitewashed in limestone, returning it to gleaming white the city had been hitherto known for. Eventually, the whitewash stopped being maintained and was allowed to wear off, a process that was complete by the 1900s.

Grand Kremlin Palace Tour

By Jack Fisher

Formerly the Moscow residence of the Russian tsars, the Grand Kremlin Palace (not to be confused with the State Kremlin Palace) is a complex inside the Kremlin. It now hosts diplomatic meetings and official state ceremonies including presidential inaugurations. It is also designated as a residence of the President of the Russian Federation, but is rarely used for that purpose.

When SRAS gave me the opportunity to take an exclusive tour of this complex, which is an exclusive tour that is normally off-limits to the general public, I had to take it.

This particular tour is different from those that cover the more public areas inside the Kremlin and requires signing up early and submitting your documents for a security check.

I met the tour group on a Friday afternoon in Aleksandrovski Sad, which borders the Kremlin walls. From there, we made our way towards the Kremlin grounds entrance. There was a huge line to get into the grounds through a first security checkpoint, but we were able to skip straight to the front of it since we had registered for our tour ahead of time. Once we were through the gate, the crowd thinned out significantly.

As we walked through the Kremlin grounds, we saw other tour groups taking photos of the landscaping, palace, and other historical buildings. Unlike us, they didn’t have the permission of the Russian government to enter the actual palace. When we got to the palace, we walked through the front doors, crossed a second security checkpoint, met our guide, and started the tour.

Our tour guide inside the building was a woman that worked in preservation. She only spoke Russian, so everything was translated for us by an SRAS-hired guide to English. We began on the first floor of the newer section of the palace and saw several ornate living rooms and guest rooms, followed by the empress’s and emperor’s chambers. Unfortunately, we weren’t allowed to see the emperor’s office and bathroom as President Putin had decided to use them as his personal study for the remainder of his time in office.

After the first floor, we headed upstairs to the second. From the outside the palace appeared to have three floors, but in reality the second floor just had massive, vaulted ceilings and two levels of windows. From what I saw, the second floor seemed to be where the fun happened. The first major room we walked into was the Hall of the Order of St. George, built to house major military meetings and balls and today used as a large conference room. There were names of famous military officers and soldiers inscribed on the walls, and the hall looked like it could hold hundreds of guests. Then it was on to the Hall of the Order of St. Vladimir, which was way less cool. It did, however, have the largest chandelier in the palace, for what that’s worth.

Next, we moved into the oldest section of the palace. It was built in the late 1400s and the newer sections of the palace were built out to connect with it. Our guide told us that by the time of the last czars, the older section was used strictly for ceremonial purposes. The walls were covered with paintings of historical rulers and religious figures. It was definitely my favorite room as there seemed to be an aura of timelessness hanging about the place.

Then we went back through the Hall of the Order of St. Vladimir and through another hall to the older bedchamber of the czar and an older, smaller meeting room for the czar and his nobles. This section was markedly different as there was none of the opulence of the newer palace. It had a utilitarian feel due to its practical layout with comfortable but plain looking chairs, reasonably sized paintings, low ceilings and large traditional Russian stoves.

Finally, we visited the throne room. It was massive, just like the Hall of the Order of St. George, and had polished stone and gilding everywhere. Unfortunately, it was a reconstruction. Our guide let us know that the soviets had torn it apart when they came to power, creating what looked like a massive classroom to house the first meetings of the Soviet Congress. The Russian government had restored it completely within the past decade. She also let us know that the current heir to the Russian throne is Prince Harry of England, which is an interesting fact I’ve been surprising Brits with lately.

On our way out, we exited through a portrait hall. Most of the portraits were typical Enlightenment and Victorian era paintings with stuffy looking people. However, one painting caught my eye: the portrait of Knyaz Sbyatoclav. The man looked absolutely hardcore (and you can see him below in a photo I took).

In my opinion, it was definitely worth $75. While I wouldn’t go twice, the fact of the matter is that you get to see the inside of a beautiful building and stand in rooms that very powerful people meet in and have met in for hundreds of years – which is an opportunity that few regular people are given. Don’t think that it’s too expensive, because you’ll have the experience and memory with you for the rest of your life.

The Kremlin Without a Capital (1712-1918)

By Lee Sullivan

The Kremlin has always been a symbol of Russian power and authority. It is often used interchangeably with the Russian state in journalism and academic literature. This is not surprising considering the Kremlin is situated in the heart of Moscow and has typically housed Russian rulers and their offices – and continues to contain an official residence and office for Russian president Vladimir Putin. However, not all of Russia’s leaders have always called the Moscow fortress home. This article covers the nearly 200 years of Kremlin history when Moscow was not the capital.

Peter the Great moved Russia’s capital from Moscow to St. Petersburg in 1712. Despite the continued crowning of tsars in the Annunciation Cathedral and symbols of power in the Kremlin vaults, Moscow’s role in state life was minimal compared to that of the new capital. This changed when a new stage of construction began under Catherine the Great. Even though St. Petersburg was the new capital, she was crowned in Moscow following ancient tradition. A commission to replace the code of laws from Tsar Alexey Mikhailovich’s time was called in Moscow and its session was held in the Kremlin’s Faceted Palace. This was a sign that under Catherine the Great the state would be ruled from both St. Petersburg and Moscow. Additionally, the Senate was divided into departments under Catherine. Four were in St. Petersburg and two were in the newly commissioned Senate Building, which still stands in the Moscow Kremlin.

Catherine additionally planned a grand reconstruction of the Kremlin interior, one that would have seen most of its buildings demolished, save for the historic cluster of churches, and replaced with modern imperial architecture built with long, straight roads, much like St. Petersburg itself. Demolition was started, including to parts of the original Kremlin walls, when cracks began to appear in one of the cathedral walls due to the resulting disruption of the soil. Because of this, and because of the project’s already enormous cost, it was cancelled, and the original walls re-built.

In September of 1812, French troops occupied Moscow. Napoleon, who led them, planned to occupy the Kremlin as his residence. It is widely thought that in defense against the French, the Moscow mayor ordered fires be set across the city. They raged for days so and were so intense that Napoleon was forced to leave the Kremlin due to the smoke. Upon returning he declared an intention to remain in the Kremlin for winter and ordered additional fortification of the Kremlin walls. However, the French army was weakening due to battle loss and poor supply.

Napoleon ordered his troops to retreat and blow up the Kremlin in the process. Mines were laid but their effectiveness was reduced by rain and prompt Muscovite response. Still, considerable damage was done, including to the Vodozvodnaya Tower, which was completely destroyed.

The Kremlin quickly underwent restoration under Tsar Alexander I and Nicolas I. Despite the war’s considerable drain on state funding, Tsar Alexander I prioritized restoring many parts of the Kremlin including towers, walls, palaces, and cathedrals. He often traveled to Moscow to observe the restoration progress. Many of Russia’s best architects were included in the restoration efforts. Order was progressively restored to the Kremlin and new gardens, now called the Alexander Gardens, were laid out along its exterior. Buildings like the Senate were brought back to their original appearance.

Restoration was completed under Nicholas I, who gave special attention to the restoration of ancient Kremlin churches and other old buildings. He also commissioned the construction of new buildings like the Great Kremlin Palace, after having the old one demolished. The entire imperial family attended the palace blessing during an official ceremony in April 1849. It was constructed and designed with techniques that were ahead of their time – vaulted construction for walls and ceilings, inlaid stone floors, and iron rafters.

Shortly after the revolution, the Communists restored Moscow as the official capital in 1918 when Moscow was reinstated as Russia’s capital. Construction and restoration were completed by the mid-19 th century. During the Soviet years, the Kremlin housed Soviet leaders and saw the development and then dissolution of the Soviet state. Today the Kremlin stands in Russia’s capital as a unique architectural ensemble.

The Kremlin Under the Soviets

The new Bolshevik government made sweeping changes to the historic Kremlin complex to, as they saw it, better represent the character of the new socialist state.

During the revolution of November 1917, the Kremlin was ransacked, leaving it with broken glass, destroyed icons, and parts of the complex in disrepair. Restoration of the walls and towers began in 1918, but further restoration stalled for lack of funding and because the communists had not yet decided on a plan for their changes to their seat of government.

The first targets were churches and royal symbols. Nuns and monks who had long lived in the Kremlin were removed. Churches had valuables removed and transferred to the new Commissariat of Finances to fund state projects. Many royal treasures and even crown jewels were similarly transferred. The double-headed eagles on the top of the buildings were promptly removed.

Many buildings were repurposed. Initially, many were converted to housing for Communist functionaries as the revolution and war had depleted Moscow’s housing stock while driving immigration from the countryside to the city. At one point, over two thousand people lived inside the Kremlin. By 1939, however, Kremlin residents consisted of only about three dozen high ranking officials.

Other notable repurposings included turning the Palace of Facets into a canteen with its kitchen inside the Tsarina’s Golden Chamber. The Ivan the Great Bell Tower was turned into a workshop, the Small Nicholas Palace became a worker’s club, and a gym was placed in the Church of St. Catherine. In 1932, the Andrew and Alexander Halls within the palace were gutted to make room for a party congress.

Many of the buildings and statues within the complex were destroyed, often to make way for new construction; only 26 of the original 54 buildings survived the Soviet period. The Chudov Monastery and Ascension Convent were both destroyed to make way for a military academy and eventually the Kremlin Presidium was built on the ground to house the Supreme Soviet, the supreme legislative body of the USSR.

In 1929, the Maly Nikolaevsky Palace, a former royal residence, was replaced by a new administrative building.

In the 1920s, the Russian royals buried in the Archangel Cathedral on the Kremlin’s Cathedral Square were exhumed and autopsied. They and the items in their sarcophagi were turned over to the Kremlin museum. Some valuable artifacts were requisitioned to the state treasury.

In 1935, five stars of rubied glass replaced the double-headed eagles that once topped the Kremlin gate towers.

Throughout WWII, the Kremlin was disguised under mock construction and painted roofs. Despite this, several bombs still fell on the Kremlin grounds, but did not cause major damage.

In 1947, Stalin painted the Kremlin walls red in an unmistakable ode to socialism, a drastic change from the traditional white that the walls had carried for centuries.

In 1955, the Kremlin opened to the public as an open air museum. In that same year, a ban on living in the Kremlin was introduced, lessening any security risk opening it to the public might create.

The last wave of demolitions came in 1958-1961, when the Palace of Congresses, built to house the congresses of Communist Party and cultural events, replaced the Old Amoury and part of the Patriarch’s Palace.

In part due to the outcry from this massive renovation, greater care of the Kremlin grounds began. The official Kremlin museum system was established in 1966, and Elena Gagarin, daughter of Yuri Gagarin, was hired as museum director. Today, that system includes the large armoury, several churches, and items outside of the Kremlin, such as St. Basil’s Cathedral.

The changes made during the Soviet period have left the Kremlin with a striking architectural contrast between traditional, tsarist-era architecture with Soviet-style buildings and the iconic, ancient red walls and remaining cathedrals. Despite the destruction and changes that were carried out, the compound still offers an unforgettable look into Russian and Soviet history that is impossible to get from anywhere else.

The Kremlin Stars

Translated by Caroline Barrow

The following was originally posted to the the Russian 7 website . It has been translated here by SRAS Home and Abroad Translation Scholar Caroline Barrow. Additional edits and updates were applied in 2021.

On October 24, 1935, two long-standing symbols of the Russian monarchy—the two-headed eagles which stood on top of the Kremlin towers, were ordered to be brought down and replaced with five-pointed stars.

Why a five-pointed star became the symbol of the Soviet regime is unknown, but what is known is that Lev Trotsky supported this symbol. Greatly fascinated by the esoteric, he knew that stars and pentagrams have a strong energetic potential and are one of the strongest symbols. The swastika could have easily become the symbol of the new government, since it had a strong following in Russia at the beginning of the twentieth century. Swastikas were displayed on the currency of the temporary government led by Alexander Kerensky, and swastikas were painted on the walls of Empress Alexandra Fedrovna’s Ipatiev House before the royal family was executed there. This swastika trend was stopped almost solely by Trotsky and the Bolsheviks, who opted for the five-pointed star. The history of the twentieth century even showed that stars are stronger than swastikas… Stars shone over the Kremlin, in the place of two-headed eagle.

Erecting the thousand-kilogram stars on the Kremlin towers was not a simple thing to do. The problem was that the needed technology did not exist in 1935. The smallest of the Kremlin Towers, Borovitskaya, rose to 52 meters, and the tallest tower, Troitskaya, reached a height of 72 meters. Throughout the country, there were no tower cranes capable of reaching these heights. However, for Russian engineers, the word “no” did not exist, only the phrase “we must.” Engineers designed and built special cranes that could be installed on the upper deck for each tower. A metal base, called the console, was mounted at the base of each turret window, and on each console the engineers mounted a lifting crane. Thus, the process occurred in several stages: first the two-headed eagles were dismantled, and second, the stars erected.

Each star weighs about one ton. Given the height at which the stars would be placed and the fact that each star has a surface area of 6.3 square meters (potentially excellent for catching the wind), there was a danger that the stars might be blown away along with the top of the towers. So, it was decided to stress test the towers and, it turns out, with good reason: the upper part of each tower and its console was completely destroyed in the process. So, builders reinforced the masonry at the upper levels of the towers, and for the Spasskaya, Troitskaya, and Borovitskaya Tower, metal bracing was added to the base of the tower. The console on Nikolskaya Tower was so damaged that it had to be completely rebuilt.

All the stars were not made identical; four stars differ from one another in their artistic forms. On the Spasskaya Tower star, rays go out from the center. However, on Troitskaya Tower’s star, the rays look like spikes. The star on Borovitskaya Tower is made up of two contours, one inscribed in the other, and, finally, the rays on Nikolskaya Tower’s star have no pattern. In terms of length, the Spasskaya and Nikolskaya Towers were similar, with the distance between the ends of the rays being about 4.5 meters. On Troitskaya and Borovitskaya Towers, the star rays were shorter, and the distance between the ends of the rays was less, measuring 4 and 3.4 meters, respectively.

A star is good, but a spinning star is twice as nice. Moscow is large, its people many, and all must see the Kremlin stars. For the base of each star, special bearings were produced by the First Bearing Plant. These special bearings allow the stars to rotate with the wind even despite their significant weight. Consequently, it is possible to know the direction of the wind given the position of the stars.

Installation of the Kremlin Stars was a true celebration for Muscovites. The stars were not carried under the cover of night to Red Square. The day before the stars were placed on the towers they were put on display in Gorky Park. District and City Secretaries of the Communist Party came together with the ordinary mortals below to see the stars. The stars were lit from the outside to make the Ural stones shine and the rays sparkle. The eagles, taken off the towers, were also displayed to visually demonstrate the dilapidation of the “old” world and the beauty of the “new” world.

The Kremlin stars were not always ruby glass. The first stars, installed in October, 1935, were made from high-alloy stainless steel and red copper. In the center of each star, on both sides, the stars were embedded with precious stones outlining the hammer and sickle emblem. Over the course of a year, the glitter of the gems dimmed. The stars were also found to be too big, not fitting well with the architectural ensemble. In May, 1937, it was decided to install new, illuminated glass ruby stars. Also, they added a star to a fifth tower, the Vodovzvodnaya Tower. The ruby glass was produced at a factory in the city of Konstantinov, according to the method of the Moscovite glassmaker, N. I. Kurochkina. It was necessary to prepare 500 square meters of ruby glass, and for that, a new type was invented—selenium ruby glass. Before that, gold was used to color the glass; selenium was cheaper and produced a deeper color.

The Kremlin stars don’t only rotate, they also light up. In order not to overheat and cause damage, about 600 cubic meters of air is blown through the stars per hour. The stars are not affected by power outages, because they have their own, independent generators.

For the original lighting, the Moscow Electrical Lamp Plant produced the lights for the stars. The stars on Spasskaya, Troitskaya, and Nikolskaya Towers all had 5000-watt bulbs, and the other two operated at 3700 watts. In each star, two parallel filaments were installed. That way, if one burned out, the other filament still shone and a control panel is was notified of the burnout.

To change a bulb, one need not need to climb up to the star. Rather, the bulb comes down on a special rod that runs straight through the bearing. The whole process takes 30-35 minutes. In the stars’ history, the stars stopped shining only twice—once during the war, and another time for the filming of the now-classic movie The Barber of Siberia .

Editorial Note: Update 2021. Starting in 2015, the lighting of the Kremlin stars was updated with one star’s lighting system replaced each year. The old incandescent lamps were replaced with modern metal halide lamps. These lamps are approximately four times more energy efficient than the old bulbs and provide a more intense, higher-quality light. Metal halide lamps are often used for sports stadiums and other places where strong, high-quality light is needed.

In preparation for this switch, Employees of the Central Scientific and Restoration Design Workshops (TsNRPM) measured the illumination of each arm of each star separately to make sure that each would still be lit evenly and brightly. They also created models of the stars lit with various methods including LED matrices and optical fiber. In the end, metal halide was determined to be the closest in historical appearance to the existing incandescent lamps.

Within this update, each star was also given its first compressive maintenance since 1946. Damaged panes were replaced, the stars were cleaned inside and out, and the lubricants within the rotation system were replaced with modern fluids.

The State Kremlin Palace

By Benjamin Bradley Mulick

Finished in 1961 after three years of work, the Palace of Congresses, later renamed as the State Kremlin Palace (not to be confused with the Grand Kremlin Palace), opened its doors for the first time for the 22nd Congress of the Communist Party of the Soviet Union, welcoming thousands of party delegates as well as communist leaders from around the world. Today, it is still the Kremlin’s newest building and a multipurpose facility, housing large conventions, cultural displays from around the world and even its own ballet troupe. With these functions giving it continued purpose, the Kremlin’s most modern and out of place building is also one of its most significant.

The Kremlin State Palace stands out from the gilded, pastel buildings around it with its hulking angular lines, and large windows divided by tall marble columns. It’s crowned by a glass banquet hall, which was the brainchild of Khrushchev himself.

It features three main halls: The Great Hall, the Small Hall, and the Diplomatic Hall. The Great Hall is the largest, featuring the palace’s main stage and hosting its most important events. With a seating capacity of six thousand, it is where party congresses were held, and where some of Russia’s most prominent cultural programs take place today. The Small Hall hosts smaller musical performances, and by virtue of having removable seating, also hosts dance events, such as the World Cup of Latin American Dance, as held in June of 2021. The Diplomatic Hall provides a smaller and more intimate setting in which to enjoy performances. Last but not least, the Diplomatic Hall often hosts lesser-known artists, often performing genres that do best in closer settings, such as jazz and folk.

The facility also holds many smaller meeting rooms, intended as breakout rooms for conventions, but also used for various purposes today.

The construction of the State Kremlin Palace came with considerable controversy. Not only is it stylistically wildly inconsistent with the rest of the Kremlin’s buildings, one of Russia’s most oldest and most important historical ensembles, but it also resulted in the destruction of several older buildings to make ways for the Palace’s massive presence.

The demolished buildings included the Old Kremlin Armory Building, originally built in 1851 to house the Kremlin’s ceremonial guard and a collection of state documents and treasure. The northern wing of the Patriarchal Chambers was torn down, formerly part of the private quarters of the head of the Russian Orthodox Church.

Because these were officially designated historic buildings, the legality of razing them was questionable and likely would not have taken place had not the decision been made from the office of Khrushchev himself.

Perhaps the real loss, however, came from underground. The original plans for the palace, before the Second World War, envisioned it as truly massive facility built where Christ the Savior Cathedral now stands. In the Khrushchev era, it was planned to build a smaller but still very large building near MGU, along the river, in what were then the still-developing outskirts of the city. When Khrushchev decided to place it inside the Kremlin, its footprint was again shrunk and it faced restrictions on its height so that the view of the Dormition Cathedral would not be entirely lost.

To make up for this, the bottom part of the building was sunk sixteen meters into the valuable archeological depths of the Kremlin’s soil. The buildings torn down to make room for the Palace were themselves built over much older foundations.

Archeologists were given a short window to explore the former Palace of Natalya Narishkina, the mother of Tsar Peter I, as well the former sites of churches, royal kitchens, workshops, and studios in what was once an economic center based within the historic Kremlin.

Teams of archaeologists were assigned to the area, who, in addition to expected finds, also found a number of secret tunnels. Unfortunately, while the archaeologists did their best to learn and preserve what they could, the limited timeframe allowed by the construction of the State Kremlin Palace meant that the archaeological potential of the site was, in large part, wasted. The tunnels were filled in, the old foundations built over, and the ruins lost to history.

Today, the Palace is perhaps best known as the home of The Kremlin Ballet, which was specifically formed in 1990 under esteemed Russian artist and choreographer Andrey Petrov with the purpose of performing there after the Bolshoi Ballet stopped performing at the palace and returned to the Bolshoi, then under renovations.

While the Kremlin Ballet was created with a strong basis in the classics, they have made more recent contributions to the ballet world with a number of their own classically-inspired modern works, including a ballet adaptation of Mark Twain’s The Adventures of Tom Sawyer .

The State Kremlin Palace also hosts the Moscow Classical Ballet, which has been dancing in Moscow since 1966. Demonstrations of this tradition in the upcoming year will include the Moscow Classical Ballet’s dancing reinvention of Romeo and Juliet (which was considered scandalous when it was first performed in 1972), and a performance of Swan Lake , one of Russia’s most important contributions to dance, as performed by the Kremlin Ballet.

Built to hold important political events, the State Kremlin Palace is more a cultural building than a political one. The stage’s relatively short history promises to be subsumed by its promising future. Whatever the next big musical or cultural phenomenon in Russia is, the State Kremlin Palace will be a part of it.

• Read a review of The Snow Maidan as performed at the State Kremlin Palace on this site.

A Tour of the Moscow Kremlin Today

Tour as reviewed by Helen McHenry, 2019

As part of our SRAS cultural program, we were given the opportunity to take a tour of the Kremlin, a historic complex and symbol of the Russian government. We met our guide outside of Red Square before walking along the Kremlin walls to the visitors’ entrance. She pointed out the swallowtail merlons bordering the wall, a design popular in 15th century Italian-style architecture, before we mounted the battlement. To travel behind the Kremlin walls, we crossed a bridge that used to span the Neglinnaya River but today acts as an archway covering part of the footpath.

Inside the Kremlin is an intriguing mix of old and new – from the 15th century walls to the 20th century block of modernism known as the State Kremlin Palace. Our guide informed us of the controversy over the palace’s design, which stands in such contrast to the more traditional styles surrounding it. The building, built under Khrushchev’s leadership primarily as a government meeting hall, has almost as many floors underground as it does above ground. Although many cried out against the building when it was built, it still stands today, where it is now used mainly to host concerts.

A brief walk along a path lined with cannons from the state artillery collection brought us to what appeared to be the mother of all cannons. Indeed, the Tsar Cannon is the largest bombard by caliber ever manufactured and has never been used due to its vast size. Just around the corner lay a similarly large but unused item – the Tsar Bell. Commissioned during the time of Empress Anna, niece of Peter the Great, an almost life-size image of her adorns the bell’s surface.

We then traveled to Cathedral Square, which, as its name suggests, features a number of beautiful cathedrals. The overcast day did nothing to accentuate the gold domes that capped their many towers, but no amount of gloom could dim their impressive stature – so immense that photographing them from my vantage point proved a challenge. Each cathedral was adorned with more stunning iconography than the last, overwhelming to the point of monotony as we shuffled through the throngs of tourists.

Our next visit was to the State Armoury, a neoclassical building resplendent with the wealth of the tsars. We traipsed through room after room of riches, from icons, dishware, and diplomatic gifts to clothing, carriages, and thrones. What stood out to me the most was the two distinct – and sometimes warring – natures of Russian identity on display at the Armoury, East and West. The contrast was particularly obvious amongst the collections of clothing, weaponry, and thrones. The older pieces hearkened back to the time before the Western pivot of Peter the Great. While these remained just as ornately decorated as their modern counterparts, they were, on the whole, a lot less outlandish than those done in the styles of the West.

The Armoury marked our last stop within the Kremlin, so we traveled across the city center to the Cathedral of Christ the Savior. Although the cathedral is the world’s largest Orthodox church, the current building is not the original. Christ the Saviour was demolished under the reign of Stalin and was only rebuilt in the late 1990s. Since then, the cathedral has gained fame as the site of Pussy Riot’s 2012 performance, which landed three members in jail for “hooliganism.”

Our guide let us explore the church on our own, as the church requires groups to be led by its own guides. Looking forward to lunch, we opted for a quick pass through the cathedral. Had I not been so hungry, I could have spent hours inside, as every surface held intricately-painted religious imagery intermixed with adornments heavily gilded with gold. Photographs were not allowed within the cathedral, reserving this spectacle to be seen first-hand.

The Kremlin in its entirety is a spot I recommend to all visiting Moscow, as four hours within its walls was not enough for our group to even scratch the surface of the wonders within.

Tour as Reviewed by Joseph Ozment, 2016

As part of SRAS’s Russian as a Second Language (RSL) program at Moscow State University, I had the opportunity to attend a guided walking tour of the Kremlin and its museums. We had a professional tour guide provided by SRAS who was very well informed about all aspects of the Kremlin’s sites and always willing to answer questions.

The tour, as offered by SRAS each session, can differ slightly based on availability and timing. We began our day’s tour not at the Kremlin, but at the nearby Cathedral of Christ the Savior, Russia’s largest Orthodox cathedral and one of the largest Christian structures in the world. Note that there are wardrobe requirements for entering the church (men and women both must have their shoulders covered, while men cannot wear shorts and women must wear skirts at least beneath the knee).

Before going inside, we were taken around the massive structure, and given a brief yet informative overview of its history. We learned that, despite the classical style of the building, it is actually only about 20 years old, having been constructed to resemble the church that once stood on the same ground.

During Communist times, the ground on which the Church now stands was a massive swimming pool, having been filled with water after the original Church was destroyed. The plans that the Communists originally had for the site were to construct the headquarters of the Communist Party of the Soviet Union, which would be one of the tallest structures in the world and house the office of the Soviet Union’s premier inside the head of a giant Vladimir Lenin statue adorning the top.

The Cathedral is a truly stunning structure. Comparable only to St. Peter’s Basilica in Rome in my mind, the sheer amount of open air is amazing when one considers how still and tranquil it is on the inside.

We then continued onwards to the Kremlin itself, which was teeming with guided tours from all over the world, just like ours. Seeing other groups from America, but also some from France, Italy, China, and several other European and Asian countries was very interesting, as people tend not to think of Russia as a popular tourist destination. However, tourism here has grown rapidly in recent years, particularly since the ruble lost about half its value on world markets, making Russia a much more affordable location.

Anna informed us of the purposes of all of the first structures we encountered within the Kremlin walls. First of all, though, she made sure that we were aware that the word “Kremlin” does not refer just the center of government in Moscow, but is a general word that means fortress. Most Russian cities and towns of reasonable size and with a medieval history possess a Kremlin.

We saw one of the offices in which President Putin occasionally works, as well as the large, semi-controversial event and concert hall that resides just inside the main entrance to the Kremlin. Despite its modern style that clashes somewhat with the comparatively ancient structures around it, the fact that the building is covered in glass at least ensures that it reflects the beauty and history that abounds within the Kremlin.

After seeing the aptly named Tsar Cannon and Tsar Bell, both of which are two of the largest objects of their kind in the world, and neither of which have been used for their structural purpose in their existence, we moved on to see several of the many churches that stand within the walls of the Kremlin.

Inside the Church of the Annunciation, we were informed of some of the basic components of any Russian Orthodox Church. For starters, every inch of wall is covered in some image or another, from icons of Saints to giant murals that depict judgment day and the people of earth being sent either to heaven or hell. We also learned that the altar in an Orthodox church is given its own room, to which only the priests are allowed entry. The mysticism that is native to Orthodoxy and inherent to its liturgy was embodied in all aspects of these churches.

After our tour of the Kremlin’s outside squares, we were taken on a tour of the Armory Museum, which houses outfits, household items, carriages, armor, weapons, and various sundry items that belonged to the Tsars and Tsarinas of Russia. Anna knowledgeably led us through the various styles worn by different Russian rulers, and explained the significance(s) behind the appearance of what they wore and the carriages in which they rode.

We were in awe of the beautiful jewels that encrusted everything the royals wore and every vessel out of which they drank or off of which they ate, not to mention of the thrones on which they sat. We saw gifts from foreign dignitaries and rulers, and even the museum’s collection of Faberge creations.

All in all, it was a day rich with history and made even more enjoyable by our friendly and incredibly knowledgeable tour guide, Anna. There is hardly a more essential Russian experience to have during your time in Moscow than a guided tour of the Kremlin.

Incidentally, Anna, a guide that SRAS has worked with for years, helps run a guiding collective in Moscow called Bridge to Moscow . They run many private tours and are available for custom tours and travel as well.

Latest Updates

By Josh Wilson

In addition to the changes to how the Kremlin stars are lit and renovations to the Kremlin bells in Spasskaya Tower , for instance, several recent events are of interest.

In the mid-2000s, the Russian Orthodox Church lobbied for the restoration of the Chudov Monetary and the Ascension Convent within the Kremlin walls. The idea was seriously considered and even discussed on television by President Vladimir Putin, although only in the sense of rebuilding them as cultural monuments and part of the museum complex, rather than as working religious institutions. In the end, however, the Kremlin Presidium was simply torn down in 2016 and the area left mostly open with fragments of the old foundations left under glass for viewing. The result is a Kremlin even more dominated by open space and gardens.

Wind has damaged the Kremlin walls on a few occasions. In June 1998, several of the iconic sparrow tail structures on the wall were damaged by strong winds. In April 2018, strong wind damaged the Senate Palace roof. In October 2021, scaffolding being used to restore a section of the inner wall was blown over the top of the wall, also damaging several of the iconic sparrow tail structures. In all cases, the damage was quickly repaired.

You Might Also Like

The Moscow Kremlin has long been the main symbol of Moscow and Russia – and for good reason. It was with the Kremlin that city of Moscow officially began and from which it grew. The Prince of Moscow, ruling from the Kremlin and drawing on the growing power of his city, united and conquered the […]

Lubyanka: Inside the Story of Moscow’s Infamous Building and District

There is always history surrounding us. In a city like Moscow, this can seem overwhelmingly apparent. Moscow has many imposing buildings from many eras – some are immediately recognizable and others only invite wonder as to what stories lay behind their beauty or grime. Lubyanka is the name commonly used to refer to the building […]

Supermetal: The Latest in Moscow’s Urban Redevelopment

By the end of 2020, the founders of Khlebozavod9 and The Brusov Ship will open a new public space near Baumanskaya metro station in Moscow. A former industrial zone will host the Supermetal Cultural and Business Complex. The team’s plans call for two architectural monuments, laboratories with panoramic windows, three courtyards, and some small manufacturing […]

Warsaw’s Old Town District

While I was on study abroad in Warsaw, Poland, the place I visited most behind the Palace of Culture and Science, where I had classes every day, was Old Town. Why? Simply put, a trip to Old Town in Warsaw is like taking a journey back in time. It is the gem of the city […]

A Day in Tashkent’s Old City: Travel from Bishkek with SRAS

As part of SRAS’s Central Asian Studies program, students had the opportunity to travel to Uzbekistan for a full week. The first day of this week-long expedition began with a half-day tour of Tashkent’s old part of town. We were accompanied by our guide, Donat, or “Don” for short. He had outstanding English, and even […]

About the authors

Serena Keenan

At the time she wrote for this site, Serena Keenan was a rising junior at Smith College in Northampton, MA. She was majoring in Russian, East European, and Eurasian Studies with a minor in Government and a concentration in Translation Studies. She hoped to study abroad in Moscow during the spring 2022 semester. After college, she hoped to go on to work in nuclear nonproliferation. In her free time, she likes to read and crochet.

Program attended: Online Interships

View all posts by: Serena Keenan

Caroline Barrow

Caroline Barrow is a graduate of Texas A&M University with a degree in International Studies and Russian. She loves traveling and hearing people’s stories. Out of the places she’s been able to visit, her favorite was Kiev, Ukraine for its beauty, history, and friendly people. She received a Fulbright English Teaching Assistantship and, at the time she wrote for this site, was spending year teaching English in Kostanay, Kazakhstan. Additionally, she was been named SRAS’s Home and Abroad Translation Scholar for the 2013-2014 cycle. Her contributions included mostly translations of articles and blog posts that will be of interest to students.

Program attended: Home and Abroad Scholar

View all posts by: Caroline Barrow

Lee Sullivan

Lee Sullivan is an undergraduate student at Stetson University. She is currently pursuing a BS in cybersecurity and a BA in Russian, East European, and Eurasian studies. Next semester Lee will be in Vladivostok, Russia – studying the Russian language and participating in the Home and Abroad internship with SRAS. She aspires to pursue a master’s degree upon graduating.

View all posts by: Lee Sullivan

Benjamin Mulick

Ben Mulick, at the time he wrote for this site, was a fourth year Global Studies major at the University of Wisconsin–Milwaukee.

View all posts by: Benjamin Mulick

Jack Fischer

Jack Fischer, at the time he wrote for this site, was majoring in Physics with Russian and Economics minors at Iowa State University of Science and Technology in Ames, Iowa. He is studied Russian as a Second Language with SRAS over the summer of 2016 to improve his command of the Russian language. In the future, he’d like to work for himself and run a business, preferably abroad.

Program attended: Challenge Grants

View all posts by: Jack Fischer

Joseph Ozment

Joseph Ozment is a fourth-year International Studies and Russian Studies major at Rhodes College in Memphis, TN. He is minoring in music minor and has spent a lot of free time on music projects. He is studying Russian as a Second Language and also working an internship with The Moscow Times. He hopes to increase his Russian skills and cultural awareness so as to use his knowledge of the country and language in a professional setting in the future.

View all posts by: Joseph Ozment

Helen McHenry

Helen McHenry, at the time she wrote for this site, was a double major in international relations and Russian at the Ohio State University, with minors in Spanish and public policy. She studied with Russian as a Second Language with SRAS at the Higher School of Economics in Moscow in order to advance her proficiency in Russian and appreciation for Russian culture. She hoped to use the knowledge gained during her time abroad to advocate for foreign policy that strengthens relations between East and West in her future career.

View all posts by: Helen McHenry

Julia Brock

Julie Brock, at the time she wrote for this site, had returned to University of Kentucky to pursue a Global Studies Certificate, with an emphasis on Russia and the travel industry. She earned prior BA and MA degrees in psychology. A few years ago, she traveled to Russia, Estonia, and Finland, and loves the culture and history of these areas. She works at the campus library, and enjoys reading, running 5Ks, and spending time with her dogs. She lived for five years in Minnesota and loved the snow, winter sports, and Museum of Russian Art.

View all posts by: Julia Brock

Hudson Dobbs

Hudson received his BA in Russian Studies with a minor in Corporate Communications from Baylor University. At the time he wrote for this site, he was serving as a Home and Abroad Scholar as part of a Spring, 2022 session of SRAS’s Language and Society program St. Petersburg. While abroad, Hudson will be researching the Russian coffee culture, as well as the evolution of specialty coffee in Russia. His goal is to open his own coffee shop in the US - taking his experiences abroad and applying them to his future business.

View all posts by: Hudson Dobbs