a multiple case study research protocol

To read this content please select one of the options below:

Please note you do not have access to teaching notes, developing a robust case study protocol.

Management Research Review

ISSN : 2040-8269

Article publication date: 4 July 2023

Issue publication date: 11 January 2024

Case study research has been applied across numerous fields and provides an established methodology for exploring and understanding various research contexts. This paper aims to aid in developing methodological rigor by investigating the approaches of establishing validity and reliability.

Design/methodology/approach

Based on a systematic review of relevant literature, this paper catalogs the use of validity and reliability measures within academic publications between 2008 and 2018. The review analyzes case study research across 15 peer-reviewed journals (total of 1,372 articles) and highlights the application of validity and reliability measures.

The evidence of the systematic literature review suggests that validity measures appear well established and widely reported within case study–based research articles. However, measures and test procedures related to research reliability appear underrepresented within analyzed articles.

Originality/value

As shown by the presented results, there is a need for more significant reporting of the procedures used related to research reliability. Toward this, the features of a robust case study protocol are defined and discussed.

• Case study research
• Systematic literature review

Burnard, K.J. (2024), "Developing a robust case study protocol", Management Research Review , Vol. 47 No. 2, pp. 204-225. https://doi.org/10.1108/MRR-11-2021-0821

Emerald Publishing Limited

Copyright © 2023, Emerald Publishing Limited

Related articles

We’re listening — tell us what you think, something didn’t work….

Report bugs here

All feedback is valuable

Please share your general feedback

Join us on our journey

Platform update page.

Visit emeraldpublishing.com/platformupdate to discover the latest news and updates

Questions & More Information

Answers to the most commonly asked questions here

Research Approach: Multiple-Case Study

• First Online: 13 October 2023

Cite this chapter

• Maximilian Perez Mengual 5  

Part of the book series: Markt- und Unternehmensentwicklung Markets and Organisations ((MAU))

85 Accesses

To investigate innovation and reconfiguration happening in brick-and-mortar retail during the COVID-19 crisis, a multiple-case comparative research strategy was applied (Eisenhardt, 1991). In general, case studies use different perspectives and data sources to illustrate complex phenomena in a real-world context. With the COVID-19 crisis and its impact on retail business models, a unique phenomenon is investigated, making a qualitative, case-based approach highly suitable (Siggelkow, 2007).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as EPUB and PDF
• Read on any device
• Instant download
• Own it forever
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Author information

Authors and affiliations.

Nürnberg, Germany

Maximilian Perez Mengual

You can also search for this author in PubMed   Google Scholar

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 353 kb)

Rights and permissions.

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Fachmedien Wiesbaden GmbH, part of Springer Nature

About this chapter

Perez Mengual, M. (2023). Research Approach: Multiple-Case Study. In: Designing Physical Interaction Platforms. Markt- und Unternehmensentwicklung Markets and Organisations. Springer Gabler, Wiesbaden. https://doi.org/10.1007/978-3-658-41920-2_25

Download citation

DOI : https://doi.org/10.1007/978-3-658-41920-2_25

Published : 13 October 2023

Publisher Name : Springer Gabler, Wiesbaden

Print ISBN : 978-3-658-41919-6

Online ISBN : 978-3-658-41920-2

eBook Packages : Business and Economics (German Language)

Article Contents

Introduction, contents of a research study protocol, conflict of interest statement, how to write a research study protocol.

• Article contents
• Figures & tables
• Supplementary Data

Julien Al Shakarchi, How to write a research study protocol, Journal of Surgical Protocols and Research Methodologies , Volume 2022, Issue 1, January 2022, snab008, https://doi.org/10.1093/jsprm/snab008

• Permissions Icon Permissions

A study protocol is an important document that specifies the research plan for a clinical study. Many funders such as the NHS Health Research Authority encourage researchers to publish their study protocols to create a record of the methodology and reduce duplication of research effort. In this paper, we will describe how to write a research study protocol.

A study protocol is an essential part of a research project. It describes the study in detail to allow all members of the team to know and adhere to the steps of the methodology. Most funders, such as the NHS Health Research Authority in the United Kingdom, encourage researchers to publish their study protocols to create a record of the methodology, help with publication of the study and reduce duplication of research effort. In this paper, we will explain how to write a research protocol by describing what should be included.

Introduction

The introduction is vital in setting the need for the planned research and the context of the current evidence. It should be supported by a background to the topic with appropriate references to the literature. A thorough review of the available evidence is expected to document the need for the planned research. This should be followed by a brief description of the study and the target population. A clear explanation for the rationale of the project is also expected to describe the research question and justify the need of the study.

Methods and analysis

A suitable study design and methodology should be chosen to reflect the aims of the research. This section should explain the study design: single centre or multicentre, retrospective or prospective, controlled or uncontrolled, randomised or not, and observational or experimental. Efforts should be made to explain why that particular design has been chosen. The studied population should be clearly defined with inclusion and exclusion criteria. These criteria will define the characteristics of the population the study is proposing to investigate and therefore outline the applicability to the reader. The size of the sample should be calculated with a power calculation if possible.

The protocol should describe the screening process about how, when and where patients will be recruited in the process. In the setting of a multicentre study, each participating unit should adhere to the same recruiting model or the differences should be described in the protocol. Informed consent must be obtained prior to any individual participating in the study. The protocol should fully describe the process of gaining informed consent that should include a patient information sheet and assessment of his or her capacity.

The intervention should be described in sufficient detail to allow an external individual or group to replicate the study. The differences in any changes of routine care should be explained. The primary and secondary outcomes should be clearly defined and an explanation of their clinical relevance is recommended. Data collection methods should be described in detail as well as where the data will be kept secured. Analysis of the data should be explained with clear statistical methods. There should also be plans on how any reported adverse events and other unintended effects of trial interventions or trial conduct will be reported, collected and managed.

Ethics and dissemination

A clear explanation of the risk and benefits to the participants should be included as well as addressing any specific ethical considerations. The protocol should clearly state the approvals the research has gained and the minimum expected would be ethical and local research approvals. For multicentre studies, the protocol should also include a statement of how the protocol is in line with requirements to gain approval to conduct the study at each proposed sites.

It is essential to comment on how personal information about potential and enrolled participants will be collected, shared and maintained in order to protect confidentiality. This part of the protocol should also state who owns the data arising from the study and for how long the data will be stored. It should explain that on completion of the study, the data will be analysed and a final study report will be written. We would advise to explain if there are any plans to notify the participants of the outcome of the study, either by provision of the publication or via another form of communication.

The authorship of any publication should have transparent and fair criteria, which should be described in this section of the protocol. By doing so, it will resolve any issues arising at the publication stage.

Funding statement

It is important to explain who are the sponsors and funders of the study. It should clarify the involvement and potential influence of any party. The sponsor is defined as the institution or organisation assuming overall responsibility for the study. Identification of the study sponsor provides transparency and accountability. The protocol should explicitly outline the roles and responsibilities of any funder(s) in study design, data analysis and interpretation, manuscript writing and dissemination of results. Any competing interests of the investigators should also be stated in this section.

A study protocol is an important document that specifies the research plan for a clinical study. It should be written in detail and researchers should aim to publish their study protocols as it is encouraged by many funders. The spirit 2013 statement provides a useful checklist on what should be included in a research protocol [ 1 ]. In this paper, we have explained a straightforward approach to writing a research study protocol.

None declared.

Chan   A-W , Tetzlaff   JM , Gøtzsche   PC , Altman   DG , Mann   H , Berlin   J , et al.    SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials . BMJ   2013 ; 346 : e7586 .

Google Scholar

• conflict of interest
• national health service (uk)

Email alerts

Citing articles via.

• Advertising and Corporate Services
• Journals Career Network
• JSPRM Twitter

Affiliations

• Online ISSN 2752-616X
• Copyright © 2024 Oxford University Press and JSCR Publishing Ltd
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

• Study Protocol
• Open access
• Published: 06 May 2021

Learning from public health and hospital resilience to the SARS-CoV-2 pandemic: protocol for a multiple case study (Brazil, Canada, China, France, Japan, and Mali)

• Valéry Ridde   ORCID: orcid.org/0000-0001-9299-8266 1 ,
• Lara Gautier 2 , 3 ,
• Christian Dagenais 4 ,
• Fanny Chabrol 1 ,
• Renyou Hou 1 , 5 ,
• Emmanuel Bonnet 6 ,
• Pierre-Marie David 7 , 8 ,
• Patrick Cloos 2 , 9 ,
• Arnaud Duhoux 8 , 10 ,
• Jean-Christophe Lucet 11 , 12 ,
• Lola Traverson 1 ,
• Sydia Rosana de Araujo Oliveira 13 ,
• Gisele Cazarin 13 ,
• Nathan Peiffer-Smadja 11 , 12 , 14 ,
• Laurence Touré 15 ,
• Abdourahmane Coulibaly 15 ,
• Ayako Honda 16 ,
• Shinichiro Noda 17 ,
• Toyomitsu Tamura 17 ,
• Hiroko Baba 17 ,
• Haruka Kodoi 18 &
• Kate Zinszer 2 , 3  

Health Research Policy and Systems volume  19 , Article number:  76 ( 2021 ) Cite this article

5494 Accesses

31 Citations

18 Altmetric

Metrics details

All prevention efforts currently being implemented for COVID-19 are aimed at reducing the burden on strained health systems and human resources. There has been little research conducted to understand how SARS-CoV-2 has affected health care systems and professionals in terms of their work. Finding effective ways to share the knowledge and insight between countries, including lessons learned, is paramount to the international containment and management of the COVID-19 pandemic. The aim of this project is to compare the pandemic response to COVID-19 in Brazil, Canada, China, France, Japan, and Mali. This comparison will be used to identify strengths and weaknesses in the response, including challenges for health professionals and health systems.

We will use a multiple case study approach with multiple levels of nested analysis. We have chosen these countries as they represent different continents and different stages of the pandemic. We will focus on several major hospitals and two public health interventions (contact tracing and testing). It will employ a multidisciplinary research approach that will use qualitative data through observations, document analysis, and interviews, as well as quantitative data based on disease surveillance data and other publicly available data. Given that the methodological approaches of the project will be largely qualitative, the ethical risks are minimal. For the quantitative component, the data being used will be made publicly available.

We will deliver lessons learned based on a rigorous process and on strong evidence to enable operational-level insight for national and international stakeholders.

Peer Review reports

The current approach to controlling COVID-19 pandemic has largely been a strategy aimed to flatten the epidemic curve and lower peak morbidity and mortality [ 1 ]. Reducing the intensity of COVID-19 transmission is crucial to avoid overloading health systems and to allow for a more manageable increase and treatment of hospitalized and severe patients. The resilience of health systems, including public health, in response to COVID-19 is under question [ 2 ], including in high-income countries such as the USA [ 3 ], Spain [ 3 ], Taiwan [ 4 ], and Italy [ 5 ]. At different points during the pandemic, health systems have been unable to meet the laboratory testing and other supply chain demands such as personal protective equipment. Contact tracing has overwhelmed public health departments, often with less than an optimal time delay [ 6 ]. In Italy, guidelines were issued for patient selection for intensive care, restricting it to those that stand to benefit the most [ 7 ], and globally, there have been bed shortages in intensive care units (ICUs) [ 8 ]. In resource-limited settings, such as in Africa or South America, the low performance and resilience of health systems is alarming [ 9 , 10 , 11 ].

Several studies have shown that during the COVID-19 pandemic, social adversities such as poor living and working conditions have accumulated for certain social categories [ 12 ]. Thus, if policies including public health measures do not take into account various precarious sociodemographic situations (e.g., migrant status, children, language, low income, overcrowded housing, and inability to isolate oneself or the difficulty of protecting oneself), they may contribute to accentuating social adversities and their deleterious effects, whether or not directly related to the transmission of the virus [ 13 , 14 , 15 ]. Therefore, to mitigate COVID-19 pandemic social impacts, public health practices must adapt to living environments.

Coordinated and collaborative evidence-based responses are critical for the successful control of a public health emergency and to maintain health system functioning. The many unknowns of COVID-19 have made the response efforts difficult and variable [ 7 , 16 ], while it is known that improving equitable access to COVID-19 interventions would be a vital step in reducing disease propagation [ 13 , 14 ]. As stated in the early stages of the pandemic by the Global Research Forum for COVID-19, there is an urgent need to understand the resiliency of health systems in the context of pandemic planning and response [ 17 ]. The preconditions of context have significant impact on the resiliency of health systems faced with the COVID-19 crisis [ 9 , 18 ]. The need to integrate social sciences, health staff, and system resilience into the pandemic response was also identified as one of the priorities [ 19 , 20 ]. How different hospitals in different countries respond during this pandemic in their preparation and implementation is essential to study and understand [ 21 ]. Regarding public health measures, it is vital to understand how social factors were (or not) taken into account in planning COVID-19 interventions.

Research objectives

The aim of this project is to compare the pandemic response to COVID-19 in locations of Brazil, Canada, China, France, Japan, and Mali during the first and second waves of the pandemic. This comparison will be used to identify strengths and weaknesses in the response, including challenges for health professionals and health systems. The research questions are:

Q1. How was the response planned, organized, and implemented in different COVID-19 referral hospitals?

Q2. What disruptions were encountered, what strategies were adopted, and what is the resiliency of professionals and hospitals?

Q3. How were social and health inequalities considered in the design and planning of public health interventions to prevent the spread of SARS-CoV-2?

Q4. What collective and practical lessons learned from the COVID-19 crisis can be developed for better preparation and response in the future?

Q5. What are the factors that facilitated or hindered the dissemination and the use of these lessons learned between the countries?

Q6. How does the COVID-19 burden differ between each country, and what are the similarities and differences in spatial and temporal trends?

Research design: multiple case study approach

In the field of health systems research, comparative approaches are recommended [ 22 ] and are essential to develop operational, transferable lessons. We will use a multiple case study approach with multiple levels of nested analysis [ 23 ]. Each hospital and public health intervention will be considered a single case.

For the hospital case studies (Q1 and Q2), the analysis will correspond to varying importance of different configurations. The configurations will be identified using a comparative perspective based on the conceptual framework (Fig. 1 ) [ 23 ]. We chose these six countries as they represent the diversity of continents, contexts, and COVID-19 burden, and we have longstanding scientific and practice collaborations. We will focus on eight major hospitals in Recife and Manaus (Brazil), Zhejiang (China), Paris (France), Bamako, (Mali), Montreal and Laval (Canada), and Tokyo (Japan), see Fig. 1 .

For the public health case studies (Q3), we will focus on understanding if and how inequalities have been taken into consideration during the planning of two major SARS-CoV-2 infection prevention interventions: contact tracing and testing in the general population. Each intervention at each site will be considered a case study.

For Q4, we will generate high-quality lessons learned (LL) from a systematic approach to collecting, compiling, and analysing data from multiple sources, and reflecting both positive and negative intervention experiences [ 24 ]. To develop this process of LL, we conducted a rapid review of the literature, which led to a 10-step guide: (1) identification and mobilization of stakeholders; (2) formulation of the aims of the process; (3) identification of the events targeted to develop the LL; (4) choosing the moment to start the process of developing the LL; (5) selecting the methods; (6) developing interview grids; (7) choosing the data source; (8) data verification and revisions of the aspects to be covered; (9) analysis and formulation of preliminary LL; and (10) verification of the quality of LL.

To share the research results and validate the preliminary LL (step 10), we will organize one deliberative workshop [ 25 ] in each country and one international deliberative workshop between the six countries with national institutions and international organizations (WHO, GloPID-R [Global Research Collaboration for Infectious Disease Preparedness], PAHO [Pan American Health Organization], WHO AFRO [WHO Regional Office for Africa], TDR [Special Programme for Research and Training in Tropical Diseases], PHAC [Public Health Agency of Canada], European/Africa CDC [European/Africa Centres for Disease Control and Prevention]). The aim of the workshops will be to discuss the practical implications of the various findings and recommendations, in terms of preparation, interventions, training, and communication. The workshops will be supported by preliminary drafts of policy briefs (PB) [ 26 ] to share the research results that led to the preliminary LL, in an accessible format and in multiple languages. This will be part of our an action-oriented approach for decision-makers. Other knowledge transfer (KT) tools (infographics, videos, etc.) will be developed to disseminate these lessons to different audiences, once the LL have been finalized. The project's website will be used for information dissemination and communication ( https://u-paris.fr/hospicovid ) in English and French.

To evaluate these KT activities (Q5), a mixed-methods design, combining quantitative and qualitative data, will be used.

For Q6, subnational (e.g., provincial, district, or department) COVID-19 portraits will be constructed for each site which will correspond, geographically, to the selected hospitals and public health organizations of Q1–Q3.

Data collection

For Q1 and Q2, we will describe how countries have planned, organized, and implemented hospital responses to COVID-19, to describe the resilience of hospitals and their staff. Several empirical data collection techniques will be used (observation, interviews, document analysis). For the observations, the researchers will conduct lengthy observation sessions, when it is safe to do so, over several weeks in some of the hospitals. The aim is to observe the functioning of services, meetings, interaction between professionals, and so on. While these sessions will provide empirical data through systematic note-taking [ 27 ], they will also be instrumental for the interviews and developing the interview guide. Qualitative interviews will be conducted with stakeholders using a diversification sampling strategy [ 28 ] within each stakeholder group (decision-makers, managers, medical staff, non-medical staff). We anticipate that we will conduct approximately 30 interviews per site/institution, until empirical saturation is reached. The conceptual framework will inform the development of interview guides, which are discussed below. Interview guides will be developed collaboratively and piloted in each jurisdiction prior to use.

For Q3, the two public health measures, contact tracing and SARS-CoV-2 testing, will first be described using the Template for Intervention Description and Replication reporting guideline for population health and policy interventions (TIDier-PHP) for each site [ 29 ]. The descriptions will inform the interview guide as well as the conceptual dimensions of the REFLEX-ISS tool, which enables stakeholders to consider the ways that social and health inequalities are taken into account in their interventions [ 30 ]. The interview guides will be drafted and tested prior to use for qualitative interviews. These interviews will also be conducted with stakeholders using a diversification sampling strategy [ 28 ] within each stakeholder group (decision-makers, managers, public health practitioners). We anticipate that we will conduct approximately 30 interviews per site/institution, until saturation is reached.

For Q4 and Q5, two questionnaires will be used (i) to assess the PB and their use, and (ii) to assess the participants' intention to use the LL. This questionnaire, adapted from the tool developed by Légaré et al.[ 31 ], is based on the theory of planned behaviour [ 32 ] and Triandis’s theory [ 33 ]. Approximately 30 semi-structured interviews will be conducted online 3 months after the national workshops. All members of the research team and at least three key informants in each country will be invited to participate. The interview grid is constructed in part according to the essential organizational components of a deliberative workshop as formulated by Boyko et al [ 34 ].

For Q6, publicly available COVID-19 disease surveillance data will be collated for each site along with other data sources such as census (e.g., population, sociodemographic characteristics) and mapping files.

Conceptual frameworks

Our empirical research for Q1 and Q2 will be supported by an original analytical framework on health system resilience (Box 1 ).

Box 1: Health system resilience definition

We will incorporate an analytical framework from the UK Department for International Development [ 35 ] as well as aspects of another conceptual frameworks that emphasizes the importance of interactions between the health system and the population in achieving access to care [ 36 , 37 , 38 , 39 ]. The rationale for our framework (Fig. 2 ) is that we need to first understand the disruptions encountered by COVID-19, by addressing the question "resilience to what?" The COVID-19 pandemic will be at the core of our analysis, and it represents a series of shocks to the system (1. External/internal events). All types of events (or “situations” to be dealt with) are possible, such as sudden shocks (e.g., COVID-19 pandemic), unique stresses and challenges (e.g., staff and inputs availability), and chronic stresses (e.g., drug shortages). Second, we wish to answer the question "resilience of what?". We wish to uncover the effects (positive or negative) of these events, then the strategies deployed to deal with them (by describing them and explaining their rationale) as well as their impacts on organizational routines and system dimensions (2. Effects and strategies). While the 10 dimensions at the centre of the figure (e.g., governance, human resource, logistics) have been identified in a scoping review, our empirical analysis will be adapted to each hospital’s context [ 38 ]. It is important to understand health system “resilience processes” at the individual/team level. We will focus on how managers, health and non-health staff have mobilized resources to respond to the disruptions in the hospital (1. External/internal events). Third, following our definition of resilience (Box 1 ), we will need to understand how the mobilization (or not) of these strategies in the face of events influences access to health care (3. Impacts on healthcare access) and in particular, the five health system abilities (approachability, acceptability, availability, affordability, appropriateness) of access to care [ 36 ]. We will distinguish between these five dimensions for COVID-19 patients from other patients present in the hospitals. We recognize that one of the limitations of our research, due to a lack of resources, will be the difficulty of taking into account the other five demand dimensions of access to care (approachability, acceptability, availability, affordability, appropriateness). Finally, we will examine the combined impacts of these different resilience processes (absorption, adaptation, transformation) and outcomes (improvement, recovery, deterioration, collapse) regarding four different scenarios of hospital resilience (4).

For Q3, we will use the conceptual reflections that guided and informed the development of the REFLEX-ISS tool [ 40 ], which are the following: (a) What is the philosophy of social inequalities in health (SIH) in the context at hand? Is there a shared vision of SIH (i.e., a shared vision for the respondent, its institution, and its partners) based on a common analysis of the context and supported by evidence? (b) Was the intervention planning done in consultation with major stakeholders, including community-based organizations (and ensuring their ongoing commitment), and to what extent does the consultation process effectively enact the intersectoral approach at the institutional level (e.g., formalization of consultation spaces, such as setting up partnerships and steering committees) versus only at the individual level (e.g., mobilization of one’s own network of contacts)?

Analytical approach

All interviews (for Q1, Q2, and Q3) will be transcribed and coded using computer-assisted (or aided) qualitative data processing software guided by the approach and principles of framework analysis, i.e., using a deductive-inductive approach to coding [ 41 ].

From data collected in the hospitals (i.e., interview transcripts and observation notes), descriptive accounts of hospitals’ configurations (10 to 15 for each hospital) will allow us to highlight how adapted dimensions of the framework below are revealed through empirical data. We will uncover the recurrence, according to the specific and historical contexts of hospitals, of configurations between problem situations, the effects on organizational routines, and the strategies deployed to deal with them and the impacts caused (Fig. 3 ). These configurations will be organized with regard to the dimensions usually analysed in the resilience of health systems using a comparative perspective [ 38 ]. The three dimensions of processes from a resilience perspective (effects/strategies/impacts) could give rise to the existence of three types of configurations, for example, reactions (effects/strategies/impacts), anticipations (strategies/impacts before any effects), or inactions (effects but no strategies).

The analytical approach of the multiple case studies will be carried out in two stages. In the first stage, an intra-case analysis will be carried out for each hospital in the study. This analysis will be global and exploratory using a framework analysis approach (Fig. 2 ) [ 41 ]. Throughout the analysis, we will crosscheck and validate the data collected and interpretations with key stakeholders involved in the response. A case study report will be written for each hospital following the plan in Appendix 1 . In addition, we will conduct several scoping reviews on the resilience of health systems during the COVID-19 pandemic to ensure that both the historical contexts and up-to-date evidence are incorporated into our analyses. The protocols will available online at ( https://www.protocols.io ). The second stage of the analysis will be a comparative analysis between the cases, using the configurations as a heuristic tool and to generalize the analysis (Fig. 4 ).

Based on findings at the hospital level and using our resilience framework (Fig. 2 ), we will then synthesize the results to make generalizations to similar situations in other locations [ 23 ]. This will inform a middle-range theory on the resilience of health care systems and hospitals, i.e., “theories that lie between the minor but necessary working hypotheses that evolve in abundance during day-to-day research and the all-inclusive systematic efforts to develop a unified theory” [ 42 ]. The goal is to analyse whether configuration patterns from the different case studies provide the same or different understanding of hospital resilience processes and outcomes. In other words, we will attempt to identify consistencies in the processes and configurations of hospital resilience in the context of a pandemic.

For public health analyses (Q3), we will use the conceptual reflections that guided and/or informed the development of the REFLEX-ISS tool [ 40 ].

For Q4 and Q5, due to the small number of workshop participants, quantitative data from the two questionnaires will be subject to descriptive statistics. This will provide a descriptive understanding of the reactions of the participants following their participation in the deliberative workshop as well as their intention to use the knowledge and LL. For the semi-structured interviews, all interviews will be recorded after the consent of the interviewee and then fully transcribed. The transcribed data will then be coded using QDA Miner software and then analysed using a content analysis [ 43 ]. During the content analysis, we will seek to understand general trends and discrepancies with an emphasis on comparison between different stakeholders. Triangulation of the results of the qualitative and quantitative analyses will be carried out through a triangulation-convergence approach [ 44 ], which involves comparing and contrasting the same object from both sources of data to increase the richness of the interpretation and conclusions.

For Q6, epidemiological curves will be constructed for each site, as will COVID-19 burden maps, at the highest spatial resolution possible (e.g., neighbourhoods). We will create descriptive tables that will include the characteristics of COVID-19 (when possible) as well as timelines of the sequence of events and public health measures for each site.

Our research will provide unique insight into how hospitals in six countries have adapted to the COVID-19 pandemic and how public health interventions have addressed social inequalities in health. Our study is innovative, as it will provide an international comparison of contrasting epidemiological contexts and situations in order to make the knowledge useful to decision-makers through the production of LL. The challenges will be numerous due to the nature of an international research collaboration, particularly in the context of COVID-19, and with a focus on a complex concept such as resilience, with the goal to compare and contrast between very different contexts and cultures. Through our collaborative approach, we anticipate that the challenges will be overcome and that our results will provide relevant information for decision-makers in improving hospital resiliency and for improving the consideration of social and health inequalities in public health interventions.

Map of case studies

Health system resilience conceptual framework

Ideal hospital configuration

Comparative analysis example

Availability of data and materials

The data will be available at https://dataverse.ird.fr .

Abbreviations

Coronavirus disease 2019

European/Africa Centres for Disease Control and Prevention

Global Research Collaboration for Infectious Disease Preparedness

Intensive care units

Knowledge transfer

• Lessons learned

Pan American Health Organization

Policy briefs

Public Health Agency of Canada

Reflection on social inequalities in health

Severe acute respiratory syndrome coronavirus 2

Social inequalities in health

Special Programme for Research and Training in Tropical Diseases

Template for Intervention Description and Replication reporting guideline for population health and policy interventions

WHO Regional Office for Africa

Anderson RM, Heesterbeek H, Klinkenberg D, Hollingsworth TD. How will country-based mitigation measures influence the course of the COVID-19 epidemic? Lancet. 2020;395:931–4.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Thomas S, Sagan A, Larkin J, Cylus J, Figueras J, Karanikolos M. Strengthening health systems resilience. Key concepts and strategies. Copenhagen: WHO Regional Office for Europe; 2020 p. 29. Report No.: Policy Brief 36.

Legido-Quigley H, Mateos-García JT, Campos VR, Gea-Sánchez M, Muntaner C, McKee M. The resilience of the Spanish health system against the COVID-19 pandemic. Lancet Public Health. 2020;1:S2468266720300608.

Google Scholar  

Han E, Chiou S-T, McKee M, Legido-Quigley H. The resilience of Taiwan’s health system to address the COVID-19 pandemic. EClinicalMedicine. 2020;24.

Remuzzi A, Remuzzi G. COVID-19 and Italy: what next? Lancet. 2020;1:S0140673620306279.

Lewis D. Why many countries failed at COVID contact-tracing — but some got it right. Nature. 2020;588:384–7. http://www.nature.com/articles/d41586-020-03518-4

Mounk Y. The Extraordinary Decisions Facing Italian Doctors. The Atlantic. 2020 [cited 2020 Mar 20]. https://www.theatlantic.com/ideas/archive/2020/03/who-gets-hospital-bed/607807/

Schnirring L. ECDC: COVID-19 not containable, set to overwhelm hospitals. 2020. http://www.cidrap.umn.edu/news-perspective/2020/03/ecdc-covid-19-not-containable-set-overwhelm-hospitals

Van Damme W, Dahake R, Délamou A, Ingelbeen B, Wouters E, Vanham G, et al. The COVID-19 pandemic: diverse contexts; different epidemics—how and why? BMJ Glob Health. 2020;5:e003098.

Article   PubMed   Google Scholar  

Nuwagira E, Muzoora C. Is Sub-Saharan Africa prepared for COVID-19? Trop Med Health. 2020;48:18. https://doi.org/10.1186/s41182-020-00206-x .

Article   PubMed   PubMed Central   Google Scholar  

Bonnet E, Bodson oriane, Marcis FL, Faye A, Sambieni E, Fournet F, et al. The COVID-19 pandemic in francophone West Africa: from the first cases to responses in seven countries. In Review; 2020. https://www.researchsquare.com/article/rs-50526/v1

Bambra C, Riordan R, Ford J, Matthews F. The COVID-19 pandemic and health inequalities. J Epidemiol Community Health. 2020. https://doi.org/10.1136/jech-2020-214401 .

Mukumbang FC, Ambe AN, Adebiyi BO. Unspoken inequality: how COVID-19 has exacerbated existing vulnerabilities of asylum-seekers, refugees, and undocumented migrants in South Africa. Int J Equity Health. 2020. https://doi.org/10.1186/s12939-020-01259-4 .

Bajos N, Warszawski J, Pailhé A, Counil E, Jusot F, Spire A, et al. Les inégalités sociales au temps du COVID-19. Quest Santé Publique. 2020;40:1–12.

Devakumar D, Shannon G, Bhopal SS, Abubakar I. Racism and discrimination in COVID-19 responses. Lancet Lond Engl. 2020;395:1194.

Article   CAS   Google Scholar  

Legido-Quigley H, Asgari N, Teo YY, Leung GM, Oshitani H, Fukuda K, et al. Are high-performing health systems resilient against the COVID-19 epidemic? Lancet. 2020;395:848–50.

WHO, GOLPID-R. 2019 Novel Coronavirus: Global Research and Innovation Forum: Towards a Research Roadmap/report. Geneva: WHO-GOLPID-R; 2020 p. 7.

Paul E, Brown GW, Ridde V. COVID-19: time for paradigm shift in the nexus between local, national and global health. BMJ Glob Health. 2020;5: e002622. https://doi.org/10.1136/bmjgh-2020-002622 .

Haldane V, Morgan GT. From resilient to transilient health systems: the deep transformation of health systems in response to the COVID-19 pandemic. Health Policy Plan. 2020. https://doi.org/10.1093/heapol/czaa169/6034156 .

Article   Google Scholar  

Gilson L, Marchal B, Ayepong I, Barasa E, Dossou J-P, George A, et al. What role can health policy and systems research play in supporting responses to COVID-19 that strengthen socially just health systems? Health Policy Plan. 2020;czaa112. https://academic.oup.com/heapol/advance-article/doi/ https://doi.org/10.1093/heapol/czaa112/5918196

Chabrol F, Albert L, Ridde V. 40 years after Alma-Ata, is building new hospitals in low-income and lower-middle-income countries beneficial? BMJ Glob Health. BMJ Specialist J; 2019;3:e001293. https://gh.bmj.com/content/3/Suppl_3/e001293

Gilson L, World Health Organization. Health policy and systems research: A methodology reader. Geneva: Alliance for Health Policy and Systems Research, World Health Organization; 2012. https://www.who.int/alliance-hpsr/alliancehpsr_reader.pdf

Yin RK. Applications of case study research. 3rd ed. Thousand Oaks, Calif: SAGE; 2012.

Patton MQ. Evaluation, knowledge management, best practices, and high quality lessons learned. Am J Eval. 2001;22:329–36.

Lavis JN, Boyko JA, Gauvin F-P. Evaluating deliberative dialogues focussed on healthy public policy. BMC Public Health. 2014;14:1287. https://doi.org/10.1186/1471-2458-14-1287 .

Dagenais C, Ridde V. Policy brief as a knowledge transfer tool: to “make a splash”, your policy brief must first be read. Gac Sanit. 2018;32:203–5.

Emerson R, Fretz R, Shaw L. Writing Ethnographic Fieldnotes, Second Edition. [cited 2020 Mar 20]. https://www.press.uchicago.edu/ucp/books/book/chicago/W/bo12182616.html

Palinkas LA, Horwitz SM, Green CA, Wisdom JP, Duan N, Hoagwood K. Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Adm Policy Ment Health. 2015;42:533–44.

Campbell M, Katikireddi SV, Hoffmann T, Armstrong R, Waters E, Craig P. TIDieR-PHP: a reporting guideline for population health and policy interventions. BMJ. 2018. https://doi.org/10.1136/bmj.k1079 .

Guichard A, Émilie T, Kareen N, Ginette L, Ridde V. Adapting a health equity tool to meet professional needs (Québec, Canada). Health Promot Int. 2019;34:e71–83. https://academic.oup.com/heapro/article/34/6/e71/5068642

Légaré F, Borduas F, Freitas A, Jacques A, Godin G, Luconi F, et al. Development of a simple 12-item theory-based instrument to assess the impact of continuing professional development on clinical behavioral intentions. PLOS ONE. 2014;9: e91013. https://doi.org/10.1371/journal.pone.0091013 .

Ajzen I. The theory of planned behavior. Organ Behav Hum Decis Process. 1991;50:179–211.

Triandis HC. Values, attitudes, and interpersonal behavior. Neb Symp Motiv Neb Symp Motiv. 1980;27:195–259.

CAS   Google Scholar  

Boyko JA, Lavis JN, Abelson J, Dobbins M, Carter N. Deliberative dialogues as a mechanism for knowledge translation and exchange in health systems decision-making. Soc Sci Med. 1982;2012(75):1938–45.

DFID. Defining Disaster Resilience A DFID Approach Paper. DFID; 2011 p. 20. https://www.fsnnetwork.org/sites/default/files/dfid_defining_disaster_resilience.pdf

Levesque J-F, Harris MF, Russell G. Patient-centred access to health care: conceptualising access at the interface of health systems and populations. Int J Equity Health. 2013;12:18. https://doi.org/10.1186/1475-9276-12-18 .

Gilson L, Barasa E, Nxumalo N, Cleary S, Goudge J, Molyneux S, et al. Everyday resilience in district health systems: emerging insights from the front lines in Kenya and South Africa. BMJ Glob Health. 2017;2: e000224. https://doi.org/10.1136/bmjgh-2016-000224 .

Turenne CP, Gautier L, Degroote S, Guillard E, Chabrol F, Ridde V. Conceptual analysis of health systems resilience: a scoping review. Soc Sci Med. 1982;2019(232):168–80.

Blanchet K, Diaconu K, Witter S. Understanding the Resilience of Health Systems. In: Bozorgmehr K, Roberts B, Razum O, Biddle L, editors. Health Policy Syst Responses Forced Migr. Cham: Springer International Publishing; 2020. p. 99–117. http://link.springer.com/ https://doi.org/10.1007/978-3-030-33812-1_6

Guichard A, Ridde V, Nour K, Lafontaine G. RÉFEX-ISS - Outil de réflexion pour mieux prendre en considération les inégalités sociales de santé. 2015. http://www.equitesante.org/wp-content/uploads/2016/10/Guide-d-utilisation-REFLEX-ISS.pdf

Ritchie J, Spencer L. Qualitative data analysis for applied policy research in A. Bryman and R. G. Burgess [eds.] ‘Analysing qualitative data’, (pp.173-194). London: Routledge. Anal Qual Data. London: Routledge; 1994. p. 173–94.

Merton RK. Social theory and social structure. New York: Free Press; 1968.

Gale NK, Heath G, Cameron E, Rashid S, Redwood S. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol. 2013;13:117. https://doi.org/10.1186/1471-2288-13-117 .

Creswell JW, Plano Clark V. Designing and conducting mixed methods research. Thousand Oaks: Sage Publications; 2006.

Ridde V, Hunt M, Dagenais C, Agier I, Nikiema A, Chiocchio F, et al. A Policy Regarding Data Generated by a Global Health Intervention Research Program. BioéthiqueOnline. 2018 [cited 2020 Apr 22];5. http://id.erudit.org/iderudit/1044267ar

Gautier L, Sieleunou I, Kalolo A. Deconstructing the notion of “global health research partnerships” across Northern and African contexts. BMC Med Ethics. 2018;19:49. https://doi.org/10.1186/s12910-018-0280-7 .

Download references

Acknowledgements

We would like to thank the colleagues, trainees, and students participating in the project: Mali (Seydou Diabaté, Yacouba Diarra), France (Zoé Richard, Isadora Mathevet, Isadora Mathevet), Canada (Monica Zahreddine, Ashley Savard-Lamothe, Katarina Ost, Rachel Mikanagu, Andréanne Robitaille, Casey Coleman-Marcil, Britt McKinnon), Brazil (Amanda Correia Paes Zacarias, Andréa Carla Reis Andrade, Karla Myrelle Paz de Sousa, Bruna Lins Ramos, Franciscleide Lauriano da Silva, Aletheia Soares Sampaio, Ana Lúcia Ribeiro de Vasconcelos, Betise Mery Alencar Sousa Macau Furtado, Stéphanie Gomes de Medeiros.).

This work was supported by Canadian Institute of Health Research grant number DC0190GP, the French National Research Agency (ANR Flash Covid 2019) grant number ANR-20-COVI-0001-01, and Japan Science and Technology Agency (JST J-RAPID) grant number JPMJJR2011.

Author information

Authors and affiliations.

Centre Population et Développement (Ceped), Institut de recherche pour le développement (IRD) Université de Paris, ERL INSERM SAGESUD, Paris, France

Valéry Ridde, Fanny Chabrol, Renyou Hou & Lola Traverson

School of Public Health, University of Montreal, Montréal, Québec, Canada

Lara Gautier, Patrick Cloos & Kate Zinszer

Centre de recherche en santé publique (CRePS), Université de Montréal et CIUSSS du Centre-Sud-de-l’Île-de-Montréal, Montréal, Quebec, Canada

Lara Gautier & Kate Zinszer

Faculté des arts et des sciences, University of Montreal, Montréal, Québec, Canada

Christian Dagenais

Laboratory of Ethnology and Comparative Sociology, Université Paris Nanterre/CNRS, Paris, France

UMR 215 Prodig, CNRS, Université Paris 1 Panthéon-Sorbonne, AgroParisTech, Institut de recherche pour le développement (IRD), Aubervilliers, France

Emmanuel Bonnet

Faculté de Pharmacie, Université de Montréal, Montréal, Canada

Pierre-Marie David

Pole 1 de recherche sur la transformation des pratiques cliniques et organisationnelles, CISSS de Laval, Montréal, Canada

Pierre-Marie David & Arnaud Duhoux

Centre de recherche en santé publique (CRePS), Université de Montréal, Montréal, Québec, Canada

Patrick Cloos

Faculté des sciences infirmières, Université de Montréal, Montréal, Québec, Canada

Arnaud Duhoux

AP-HP, Bichat-Claude Bernard Hospital, Paris, France

Jean-Christophe Lucet & Nathan Peiffer-Smadja

IAME, INSERM, Université de Paris, Paris, France

Institut Aggeu Magalhães, Oswaldo Cruz Fondacion, Recife, Brazil

Sydia Rosana de Araujo Oliveira & Gisele Cazarin

National Institute for Health Research Health Protection Research Unit in Healthcare Associated Infections and Antimicrobial Resistance, Imperial College London, London, UK

Nathan Peiffer-Smadja

MISELI, Bamako, Mali

Laurence Touré & Abdourahmane Coulibaly

Research Center for Health Policy and Economics, Hitotsubashi Institute for Advanced Study, Hitotsubashi University, Tokyo, Japan

Ayako Honda

Bureau of International Health Cooperation, National Center for Global Health and Medicine, Tokyo, Japan

Shinichiro Noda, Toyomitsu Tamura & Hiroko Baba

Center Hospital of the National Center for Global Health and Medicine, Tokyo, Japan

Haruka Kodoi

You can also search for this author in PubMed   Google Scholar

Contributions

The protocol is the result of a collective work in which all the authors participated under the leadership of VR and KZ with the support of LG. Each research question was developed under the scientific leadership and participation of several researchers: Q1 and Q2 (VR, KZ, LG, FC, RH, PMD, JCL, SO, GC, NPS, LT, AH, SN, HB), Q3 (VR, KZ, FC, PC, LG, SO, LT, AC), Q4 and Q5 (CD, VR), Q6 (KZ, EB). All authors read and approved the final manuscript.

Corresponding author

Correspondence to Valéry Ridde .

Ethics declarations

Ethics approval and consent to participate.

Institutional review board approvals have been given at each hospital as well as at the University of Montreal for the entire project by the Science and Health Research Ethics Board. For the quantitative component, the data being used are publicly available. There will be no risk of infection of researchers during data collection in hospitals as the observations are on operational processes (not with patient interactions). All study data will be stored on a secure cloud server with access restricted to only authorized study personnel. All analyses will be performed on anonymous and de-identified data, and a research data management plan will be created. As is the case in most social science research, we will be careful in our writing and knowledge transfer processes to respect the anonymity of participants. Our results will be fact-focused, and no denunciations or negative and stigmatizing judgements about health professionals or their structures will be made. We will be sensitive to gender representation within the research team and with participants. We will also adopt an approach respectful of North/South partnerships, including equity in project decision-making, travels, and authorship; open access for results and data dissemination; and translation of study outputs [ 45 , 46 ].

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix 1. Outline of case study reports for each hospital

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Ridde, V., Gautier, L., Dagenais, C. et al. Learning from public health and hospital resilience to the SARS-CoV-2 pandemic: protocol for a multiple case study (Brazil, Canada, China, France, Japan, and Mali). Health Res Policy Sys 19 , 76 (2021). https://doi.org/10.1186/s12961-021-00707-z

Download citation

Received : 28 January 2021

Accepted : 15 March 2021

Published : 06 May 2021

DOI : https://doi.org/10.1186/s12961-021-00707-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Public health

Health Research Policy and Systems

ISSN: 1478-4505

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

• Research article
• Open access
• Published: 04 January 2021

A mixed methods multiple case study to evaluate the implementation of a care pathway for colorectal cancer surgery using extended normalization process theory

• R. van Zelm   ORCID: orcid.org/0000-0003-0238-6392 1 ,
• E. Coeckelberghs 1 ,
• W. Sermeus 2 ,
• A. Wolthuis 2 ,
• L. Bruyneel 1 ,
• M. Panella 3 &
• K. Vanhaecht 1 , 4  

BMC Health Services Research volume  21 , Article number:  11 ( 2021 ) Cite this article

3000 Accesses

7 Citations

4 Altmetric

Metrics details

Specific factors that facilitate or prevent the implementation of enhanced recovery protocols for colorectal cancer surgery have been described in previous qualitative studies. This study aims to perform a concurrent qualitative and quantitative evaluation of factors associated with successful implementation of a care pathway (CP) for patients undergoing surgery for colorectal cancer.

This comparative mixed methods multiple case study was based on a sample of 10 hospitals in 4 European countries that implemented a specific CP and performed pre- and post-implementation measurements. In-depth post-implementation interviews were conducted with healthcare professionals who were directly involved. Primary outcomes included protocol adherence and improvement rate. Secondary outcomes included length of stay (LOS) and self-rated protocol adherence. The hospitals were ranked based on these quantitative findings, and those with the highest and lowest scores were included in this study. Qualitative data were summarized on a per-case basis using extended Normalization Process Theory (eNPT) as theoretical framework. The data were then combined and analyzed using joint display methodology.

Data from 381 patients and 30 healthcare professionals were included. Mean protocol adherence rate increased from 56 to 62% and mean LOS decreased by 2.1 days. Both measures varied greatly between hospitals. The two highest-ranking hospitals and the three lowest-ranking hospitals were included as cases. Factors which could explain the differences in pre- and post-implementation performance included the degree to which the CP was integrated into daily practice, the level of experience and support for CP methodology provided to the improvement team, the intrinsic motivation of the team, shared goals and the degree of management support, alignment of CP development and hospital strategy, and participation of relevant disciplines, most notably, physicians.

Conclusions

Overall improvement was achieved but was highly variable among the 5 hospitals evaluated. Specific factors involved in the implementation process that may be contributing to these differences were conceptualized using eNPT. Multidisciplinary teams intending to implement a CP should invest in shared goals and teamwork and focus on integration of the CP into daily processes. Support from hospital management directed specifically at quality improvement including audit may likewise facilitate the implementation process.

Trial registration

NCT02965794 .

US National Library of Medicine, ClinicalTrials.gov . Registered 4 August 2014.

Peer Review reports

Over the past 15 years, procedures for colorectal cancer surgery have been standardized with the introduction of enhanced recovery protocols (ERPs), which are also known as enhanced recovery after surgery (ERAS) protocols [ 1 ]. The fourth update of the internationally-recognized ERAS protocol for this indication was published in 2018 [ 2 ]. Efficacy and safety of these protocols have been studied extensively, leading to the are feasible, safe, and result in improved postoperative outcomes [ 3 ]. However, adherence to the interventions recommended by the ERPs seems to be challenging. Although several groups have already presented evidence suggesting a direct relationship between adherence rates (ARs) and patient outcomes [ 4 , 5 , 6 , 7 ], reported ARs vary greatly.

Several groups have explored the use of ERPs and have attempted to identify relevant processes, facilitators, and barriers to their implementation. Gotlib Conn et al. (2015) and Gramlich et al. (2017) suggested that the implementation of ERPs involves complex cognitive and social processes. Notably, the participation of an individual serving as a “local champion” and relationship-building capacity are perceived as important factors involved in the implementation of these protocols [ 8 , 9 ]. Other studies, including a systematic review of 53 studies that focused on the implementation of ERPs in multiple surgical specialties, identified adaptation of a given ERP to local circumstances as a critical facilitator, including its alignment with evidence-based practice, leadership, teamwork, staff education, monitoring, and feedback. Barriers to implementation included resistance to change, lack of stakeholder buy-in, lack of resources, and rotating residents [ 10 , 11 , 12 ].

Qualitative research approaches have provided detailed insight into the implementation process and have identified facilitators and barriers in routine clinical practice. In this study, we perform a combined quantitative and qualitative evaluation to generate comprehensive insight into the factors that promote and prevent the implementation of ERPs. This study is the final part of a series of connected studies [ 13 , 14 , 15 ] that together provide a process evaluation of pathways associated with colorectal cancer surgery.

Care pathways (CPs) have been introduced as a strategy to improve adherence to recommended care [ 16 , 17 ]. CPs are complex interventions that structure care around individual patient needs, combine evidence based key interventions, feedback on the current care process and strategies for improvement [ 18 ]. As reported in our previous publications, the hospitals participating in this evaluation received feedback on their care process via feedback meetings and a feedback report. As a next step, a model CP based on the ERAS protocol was delivered to all teams and was explained in an on-site quality improvement workshop. Subsequently, participating teams implemented the model CP or adapted their existing local CP. This intervention is described in detail in the study protocol [ 19 ]. An earlier, qualitative study was performed to explore this implementation process [ 15 ], and a quantitative effect study [ 14 ] generated several implications for further research. These implications are addressed in this study.

Our goal is to evaluate the implementation of a CP for colorectal cancer surgery in 10 European hospitals. A multiple case study design was used to interpret and to explain relationships between quantitative data, which focused on the improvement of protocol adherence and reduced lengths of stay (LOSs), and qualitative findings, which included the perspectives of the participating healthcare professionals. We anticipate that an analysis of combined quantitative and qualitative data will enhance our understanding of the implementation process. We are specifically interested in determining how the perspectives of healthcare professionals regarding the CP implementation process in different contexts correlate with the effects and success of its implementation. The research questions include:

Which factors explain the difference between pre- and post-implementation performance (LOS and protocol adherence) and thus can be used to promote its improvement?

What is the relationship between intended and measured ARs?

Study design and setting

This international mixed methods study was performed in a selected sample of 10 hospitals in Belgium, Germany, France, and the Netherlands. A comparative multiple case study design was used [ 20 ]. Fig.  1 presents a diagram of the study design.

Design of the comparative mixed methods case study (Based on Creswell et al., 2017)

Data collection and measures

Quantitative evaluation.

For step 1, pre- and post-implementation data were collected from patient records. A sample of 20 consecutive patients from each hospital was included retrospectively for evaluation of pre-pathway implementation (2014); another 20 patients were included for post-pathway implementation (December 2016) to study the impact of the CP on patient and implementation outcomes. Adult patients (≥18 years of age) undergoing elective colorectal cancer surgery (open/laparoscopic) were included. Patients diagnosed with severe dementia (Diagnostic and Statistical Manual of Mental Disorders [DSM] IV), major neurocognitive disorder (DSM V), or severe concomitant disease that might have an impact on short-term outcomes (e.g., life expectancy less than 3 months) were excluded because these patients were unlikely to be able to follow the CP. The local study coordinator was instructed to collect data retrospectively from the patient record using a standardized data extraction form [ 14 ].

For step 2, we hypothesized that hospitals that scored lower on pre-implementation ARs would achieve higher improvement scores. Primary outcome measures included median protocol adherence (hospital median of the proportions of relevant interventions as defined by the protocol that were received by each patient) and improvement rates (IRs, i.e., the difference between pre- and post-test ARs). Differences in IRs were analyzed using Mann-Whitney U-tests.

Secondary outcome measures included mean LOS and self-rated protocol adherence (SrA). To determine SrA, additional quantitative data were captured post-implementation with a questionnaire that featured a five-point anchored scale. Each hospital received one questionnaire that assessed the level of intended implementation (0–100%) of each intervention described in the model CP. SrA was determined based on these findings. We hypothesized that there would be positive correlations between SrA and post-test AR and that teams that were actively engaged with the CP with the intent to improve adherence would score higher on assessments of post-test adherence. The relationship between variables was quantified using Pearson’s R.

We generated hospital rankings based on both absolute values and differences in median protocol adherence and mean LOS. The hospital with the highest adherence ranked first and was scored with 1 point; the hospital with the lowest adherence ranked tenth (and received 10 points). Likewise, the hospital with the most substantial improvement in protocol adherence ranked first (1 point), and the hospital that improved least ranked tenth (10 points). The same method was used to score LOS. This resulted in four rankings for each hospital. The total points scored by each hospital contributed to the overall score, with a potential range of 4–40 points.

Qualitative evaluation

Step 1 focused on the collection of post-implementation data with in-depth interviews with 3 professionals per hospital. The interviews were based on a semi-structured interview guide and focused on the key elements of process evaluation [ 21 ]. A second researcher took field notes, captured non-verbal reactions, and provided reflection during the debriefing that was carried out after each interview. Finally, project notes from feedback and improvement sessions recorded during the project were used to complete the qualitative dataset, which resulted in a “thick description” of the intervention, context, implementation, mechanisms, and perceived outcomes. The methods for the interviews and questionnaires are described in detail in the study protocol [ 19 ]. The full topic guide is described in a previous study [ 15 ].

In step 2, we set criteria for the selection of cases based on the quantitative data. Since our research is focused on improvement, we included hospitals with the highest (≤10 points) and lowest (≥ 30 points) rankings based on the quantitative data collected as described above. With this method, we would be able to cover the entire spectrum of improvement with a focus on the two extremes. The cases selected were carefully reviewed using extended Normalization Process Theory (eNPT) as a framework. We chose eNPT because it defines, explains, and links key elements that facilitate or impede normalization (defined here as turning a new practice into one that is routine) of complex interventions in a social system [ 22 , 23 ]. A systematic review by May et al. (2018) focused on the use of NPT as part of the evaluation of a wide range of practices and complex interventions indicated that this framework provided a combination of the conceptual tools needed for the comprehension of implementation as a process [ 23 ]. Four core constructs were defined in the third update of eNPT, including two that were focused on context and two addressing the concept of agency, or “the ability to make things happen” (May 2013, p.1). Each core construct is operationalized based on its underlying components. The theory provides four propositions that can be used to explain the normalization of a complex intervention (Table  1 ) [ 22 ]. The original interview guide was based on several theoretical frameworks, including eNPT. As such, all components of eNPT were covered in the interviews.

Qualitative analysis was performed at the hospital level. As such, we combined data, field notes, and reflections on the interviews from different respondents at the same hospital. Cases are summarized in the descriptions included in Additional file  1 . The case descriptions were discussed and reviewed by the research team to ensure that they reflected the findings from the original data.

Data merging and analysis (steps 3 and 4)

The cases were analyzed using “side-by-side” joint displays, which are tables that provide a visual display of combined quantitative and qualitative data [ 20 , 24 ]. The cases are presented in four displays, one for each core construct of eNPT. Each row represents a case and columns represent quantitative outcome data and healthcare professional experience. This method enabled us to determine if and why any of the individual cases differed with respect to outcomes and experience for each of the eNPT constructs and facilitated the search for patterns and explanations.

We collected data from 10 independent hospitals including 381 patients and 30 healthcare professionals. The characteristics of each hospital, including the number of beds, the number of colorectal procedures performed each year, the number of patients included from each location, and the backgrounds of each of the healthcare professionals interviewed for this study are shown in Table  2 .

Our findings revealed an overall improvement in AR of 6%, varying from − 13 to + 22%, and a statistically significant average increase from 56 to 62% ( p  < 0.00001). No significant change in AR was observed at 3 of the hospitals. Findings from one hospital revealed a reduced AR in the post-test; however, six of the ten hospitals exhibited significantly higher ARs. Overall, LOS decreased significantly by 2.1 days ( p  = 0.0230). However, there was considerable variation, ranging from a decrease of 5.06 days to an increase of 2.15 days [ 18 ].

Table  3 includes results from the quantitative evaluation of the primary and secondary outcomes. Based on pre-test median adherence we compared results from the 5 top-scoring hospitals with those from the 5 lowest-scoring hospitals to test the hypothesis that those with lower pre-test scores would exhibit higher IRs. The mean IRs for the top 5 vs. the bottom 5 hospitals were 0.2% (range of − 13 to 12%) vs. 11.2% (range of − 1 to 22%), respectively, although this difference did not reach statistical significance ( p  = 0.17384).

The correlation between intended adherence (self-rated) and measured adherence (median adherence post-test) is shown in Fig.  2 . Our findings revealed a small positive correlation between SrA and median adherence that did not reach statistical significance (Pearson’s R = 0.5358, R 2  = 0.2871, p  = 0.13706).

Self-rated adherence (SrA) vs. post-test median adherence rate (AR)

Case studies

As shown in Table 3 , Hospitals 1 and 2, each with scores of 10 points, were included as high-performing cases (i.e., hospitals that scored the highest in the overall rankings as described). Hospitals 8, 9, and 10 were included as low-performing cases (i.e., hospitals that scored lowest in the overall rankings). For each case, we prepared a short description that reviews the main quantitative findings and the experiences of professionals involved with a focus on the four core constructs of eNPT (Additional file 1 ).

Joint display are presented that include data from the highest-ranking (Hospitals 1 and 2) and lowest-ranking (Hospitals 8, 9, and 10) cases, followed by a short explanatory text that includes illustrative quotes that facilitate further comparison.

Capability , the first construct of eNPT, includes the possibilities offered by the complex intervention in terms of workability and integration into a social system. As shown in Table  4 , the workability of the CP was perceived as positive in four of the cases. The CP was perceived as having a minimal impact on workload and served to increase both structure and patient safety.

“The doctors worked the model pathway in our treatment standards.” (Hospital 2).

“At the start, yes, in the beginning. Now maybe we profit. But at the start we had to explain and tell everyone … . Now, it is … when it works, it works. When the patient arrives and everything is clear, it is a positive effect.” (Hospital 10).

However, respondents from Hospital 9 expressed doubts regarding feasibility and standardization.

“So that’s what we decided. Okay, because they were the same, it was dubious to get them up the first day. But, what they do recommend is that they have an evaluation by physio, or … well with them it is a … well an evaluation at least.” (Hospital 9).

Hospital 1 and Hospital 10 integrated the CP within the existing patient record. In Hospital 2, although the CP was not integrated into the patient record, it was integrated into the work processes. However, we note that the respondents perceived the process of reaching consensus as somewhat difficult. By contrast, perioperative care provided by Hospital 8 was characterized as unstructured and the CP was not implemented. In Hospital 9, part of the CP was implemented, but it was not integrated into the overall program.

In summary, implementation of the CP and associated improvements in performance were facilitated by the overall workability and practical nature of the CP, its clarity and safety, and its integration into pre-existing work processes.

Capacity , the second construct of eNPT, is defined as the social-structural resources available to agents. Findings presented in Table  5 reveal that resources, including time for multidisciplinary team meetings and a data system, were available only in Hospital 1. All other hospitals reported both resource and time constraints. Most notably, the lack of an automated data system for monitoring performance served as a barrier to implementation of the CP. Interestingly, we note that the hospital with the highest IR also reported limitations with respect to resources. Furthermore, in all hospitals except for Hospital 9, teamwork and collaboration were perceived as strong.

“And in fact we have no departments-life. We are not meeting together, except in the corridor and so on, but we have no regular meeting for routine problems or so.” (Hospital 9).

The improvement team in Hospital 1 had no previous experience with the implementation of CPs but received support from trained CP facilitators. The team in Hospital 2 had experience with CP methodology and was supported by the quality management department. In Hospital 10, only the individual who was promoting the program (i.e., the “local champion”) had experience in CP methodology, and in Hospitals 8 and 9, no improvement team was formed. These observed differences suggest that experience with CP methodology and critical support correlate with performance.

The role of the individual responsible for promoting the program, otherwise known as the “local champion” was different in all of the hospitals evaluated. In Hospitals 1 and 10, there were clear local champions of the CP program among the medical and nursing staff (albeit on only one of two participating wards in Hospital 10). In Hospital 2 there was one local champion on the medical staff, but this individual was new to the hospital. No individuals were championing this program in Hospitals 8 and 9.

Thus, we conclude that successful implementation of the CP was hindered by the lack of an automated data system that could be used for feedback purposes but was facilitated by experienced improvement teams or teams that received support for implementing CP methodology. Interestingly, the lack of resources presented no barrier in Hospital 1 as opposed to what was reported by the other hospitals. The roles played by teamwork and the local champion remain ambiguous.

Potential , the third construct in eNPT, includes individual intentions and the collective commitment of all agents. Findings presented in Table  6 reveal that willingness to change was perceived as intrinsic among the staff at Hospitals 1, 2, and 10, and that feedback on the pre-test performance acted to trigger efforts toward improvement. At Hospital 8, individual ways of working were reported. However, the most striking differences between the hospitals that exhibited high vs. low improvement included the relative status of the personnel involved in the decision to join the project and specific CP strategy. For example, in Hospitals 8, 9, and 10, the decision to join the project was made by middle management or by the improvement team itself. By contrast, in Hospitals 1 and 2, this decision was made by higher-level management. Similarly, in Hospitals 1 and 2, CP development was an integral part of the hospital strategy, while this is not the case in any of the other hospitals. There was a remarkable contrast regarding the nature of “normal” quality improvement strategies when comparing Hospitals 1 and 2 to Hospitals 8, 9, and 10. Respondents from Hospitals 8 and 10 reported deep differences in the approaches taken by management and clinicians.

“ … always on the conflict between an administrative approach and a medical approach, huh. So it’s that gap and it’s been going on for years” (Hospital 8).

Hospitals 8 and 9 reported that external pressure worked to facilitate standardization of care. This was not mentioned in reports from any of the other hospitals. Finally, it was observed that the teams in Hospitals 1 and 2 have clear objectives and priorities.

“And that fine-tuning … we first looked to see where there is room for improvement. So we set a number of general goals, of which the most remarkable was, say, reducing the admission, the length of stay, but also reducing nausea. In our analysis, these sprang out.” (Hospital 1).

In short, implementation of the CP was facilitated by the team’s intrinsic motivation to work on specific goals and priorities and by the fact that CP development is part of the hospital’s overall strategy. Individualism, external pressure, perceptions of serious differences between the managerial and clinical approaches to patient care, and decisions to join the project made by the middle, as opposed to the upper-level management, were all barriers to implementation.

Contribution , the fourth and final construct of eNPT, refers to the role of various agents in the process of implementation of a complex intervention. This would include providing explanations, cognitive participation, actions, and reflexive monitoring. As shown in Table  7 , the intervention was seen as “making sense” in all cases. The model CP was practical and clear and was valued for its evidence base. Feedback as part of the intervention was also seen as important. Positive outcomes were expected at Hospitals 1 and 10, while the expectations were more ambivalent at Hospital 2. The improvement teams were critical to the content of the CP at Hospitals 1, 2, and 10. Interventions were scrutinized and in some cases adapted before implementation.

“Yes, I have seen that. Except … we already had everything [laughs]. So yes, it did not contain much news for us.” (Hospital 1).

The number of disciplines involved in the implementation process was considerably larger in Hospitals 1 and 2; by contrast, in Hospitals 8 and 9, the absence of physicians was noticeable. All cases except Hospital 8 described a variety of implementation activities, including training and updating the local protocol.

“And so the care pathway is explained step-by-step, with the intention to receive comments.” (Hospital 2).

“The care pathway is in the patient record, it is printed for the colleagues, and also available in intranet. And I try to make sure everybody knows that.” (Hospital 10).

Another noticeable difference was that team-training was not organized in Hospital 10. Reflexive monitoring and the use of feedback to improve performance was regarded as important at all hospitals. It was remarkable that one of the higher-performing hospitals reported the greatest struggles in collecting feedback data. International benchmarking with other hospitals was also valued in all cases, although it was not clear how the feedback was shared or how benchmarking was perceived at Hospitals 8 and 9.

“And to be able to compare ourselves to other hospitals, which we have never ever done before, you know we rarely have some benchmarking.” (Hospital 9).

“I thought that was a good thing, that really was thought-provoking. One should compare oneself with other hospitals.” (Hospital 10).

All teams that implemented the CP indicated they have ideas and plans for future development and suggested that the implementation might “carry...forward in time and space” [ 22 ].

In summary, implementation of the CP and high performance was facilitated by the fact the intervention made sense to the healthcare staff. However, positive expectations were not sufficient to achieve positive outcomes. Additional facilitators might include the use of international feedback data and involvement of all relevant disciplines, as the absence of physician involvement was observed to be a barrier to improved performance.

Main results

The primary outcomes of this study included median protocol adherence and IR. Protocol adherence improved overall. Among the 10 hospitals ranked in order of pre-test adherence, we observed a difference in mean IRs of 0.2 and 11.2% for the 5 highest-ranked and the 5 lowest-ranked hospitals, respectively, although this difference did not achieve statistical significance. The lack of statistical significance might be attributed to the small sample size and/or to the wide variation in IRs.

The secondary outcomes of our study were mean LOS and SrA. While mean LOS decreased by 2.1 days, a decrease in LOS was not observed in all participating hospitals. This is in contrast to the results presented by Larson et al. [ 25 ] that focused on the collaborative implementation of a colorectal cancer CP in which reductions in LOS were achieved by all participating teams. A possible explanation for this discrepancy could be that the focus of our study was protocol adherence, as opposed to LOS.

We were unable to establish a relationship between SrA and post-test AR. We did observe, however, that 7 of the 9 hospitals in our study overestimated their performance. A systematic review by Adams (1999) of self-reporting bias in guideline adherence revealed an absolute overestimation of 27% [ 26 ]. The difference between self-reported and measured adherence in our study was less than 27%, although it is clear that the overestimation of SrA remains a problem.

We also observed differences in improvements in both protocol adherence and LOS. LOS is used as a primary outcome measure in most studies focused on ERAS or fast-track protocols. Recently, Balvardi et al. (2018) suggested LOS could be used as a measure of in-hospital recovery with equal construct-validity as “readiness for discharge.” [ 27 ]. However, due to the small number of patients per hospital in this study (≤20), LOS (and ∆LOS) should be interpreted with caution. This is among the reasons underlying our decision to use both LOS and protocol adherence for ranking and hospital selection.

Implementation process

Implementation of the CP differed between the hospitals. While there were minor differences in capability , as the workability of the CP was perceived as positive by all, its integration into work processes was stronger in hospitals with higher IRs. Hospital 1 had already implemented a CP before the start of our project; as such, our project was used as a means to update the local CP. Nonetheless, Hospital 1 improved its AR by 10%. These results suggest that, although they already had a CP in place, they may have used it more effectively as a result of participating in our project. In Hospital 2, a CP was developed from the start, although some of the care recommended by the ERP was already provided (i.e., 43% pre-implementation adherence). As proposed by eNPT, the capability of working successfully with a complex intervention depends on both its workability and its integration [ 22 ]. Adapting the ERP to fit local circumstances has been identified as a key facilitator of implementation [ 12 ]. Furthermore, the importance of integrating new ways of working within given systems was previously described in a study that featured an earlier iteration of eNPT in colorectal surgery [ 8 ]. This information provides some explanation for observed differences between high- and low-performing hospitals.

For capacity , there were more noticeable differences between the hospitals that could explain differences in IRs. The level of experience and support for using CP methodology provided to the improvement team appears to be directly related to the IR. This result is consistent with findings reported in previously published studies for colorectal surgery [ 1 , 8 , 11 , 25 , 28 ] as well as in other settings [ 29 , 30 ]. High IRs were achieved in hospitals in which a trained facilitator or quality management officer provided support for the improvement team. A lack of resources is a well-documented barrier to implementation [ 10 , 11 , 12 , 31 , 32 , 33 , 34 ]. However, our data suggest both high- and low-performing hospitals experienced a lack of resources. Three of the 5 hospitals had a “local champion” who provided support for the initiative. Hospital 1 identified a clear and institutionally-sanctioned champion. By contrast, the champion in Hospital 2 was relatively new to the hospital; this was perceived as disadvantageous. In Hospital 10, the champion worked on only one of two wards participating in this initiative, and respondents indicated that implementation of the CP was less successful on the second of the two wards. Coxon et al. (2017) developed a program theory based on the concept of “change agency” in which the change agent is identified as a clinical local champion. The authors suggest that local champions of these initiatives should have strong clinical skills and know-how, need to be familiar with the local situation, and should have good management and people skills [ 28 ]. One proposition of the eNPT states that the incorporation of a complex intervention in a given social system depends on the users’ capacity to cooperate and coordinate their actions [ 22 ]. Teams in the high-performing hospitals had superior access to cognitive resources (experience, training, facilitation) which facilitated cooperation and coordination of their actions. However, the roles of the local champion, teamwork and material resources in our hospital cases remain ambiguous.

The first observable difference between high- and low-ranking hospitals in potential was that intrinsic motivation, shared goals, and overall commitment were reported in Hospitals 1 and 2, but were lacking at Hospitals 8 and 9. In Hospital 10, the team was motivated, although the team on the ward where the medical lead, or local champion, worked showed more commitment than did the team on the other ward. This could be one explanation for low IRs found at this hospital. Previous research supports the importance of staff morale and commitment when implementing CPs. For example, Jabbour et al. (2018) identified strong commitment as a facilitator for the implementation of CPs in a complex environment [ 35 ]. Other studies focusing on the implementation of ERAS also identified commitment as facilitator [ 10 , 11 , 12 , 28 ]. Lack of commitment observed in Hospitals 8 and 9 could explain their low performance. Second, the view of CP development as part of the overall hospital strategy (Hospitals 1 and 2) and the perceived differences with respect to quality improvement reported among clinicians and managers (Hospitals 8, 9, and 10) could contribute to the observed differences in performance. In eNPT, individual intentions and shared commitment are concepts that are used to operationalize potential. This theory proposes that translation of capacity into collective action depends on participants’ potential (and thus intentions and commitment) for the successful enactment of complex interventions [ 22 ]. Numerous papers have described the importance of management support for quality improvement, including the systematic review by Kringos et al. (2015), as well as individual studies that have examined the implementation of CP or ERAS protocols [ 8 , 11 , 12 , 25 , 31 , 36 , 37 , 38 ]. These studies suggest that management endorsement and support are key factors that promote success. Lack of management support in Hospitals 8, 9, and 10, including the management level at which the decision to join the project was made, together with a CP development program that was not aligned with hospital policy could explain the low performance.

A focus on the final core construct, contribution , revealed several interesting results. In all cases, the intervention was valued and made sense to the users, although in Hospital 1, implementation of the model CP and feedback were perceived as a routine practice. Coherence or sense-making in eNPT terms involves the assignment of meaning to a specific intervention [ 22 ]. This can be seen as the first important step towards normalization of a given intervention, as has been described in previous research. Banks et al. (2017) note that a “clear understanding and acceptance of the aims of the project, including the legitimacy of the research data and the process of pathway development” (p.109) can lead to agreement and implementation [ 39 ]. Both high- and low-ranking hospitals exhibited sense-making and expectations of positive outcomes. As such, our findings suggest that these attributes are not sufficient to achieve positive outcomes.

We observed no meaningful distinctions between the hospitals regarding implementation activities used, except for Hospital 8, where there were no implementation activities. In our previous research, we identified implementation activities focused on competence, behavior, or workplace [ 15 ]. We noticed that, in all cases, implementation involved activities from all three categories. However, we did observe differences with respect to the involvement of relevant disciplines. There was a noticeable absence of physician involvement at the low-performing hospitals (Hospitals 8 and 9). This relates to the concept of cognitive participation as defined by eNPT and the level to which users choose to participate in a complex intervention and become members of a community of practice [ 22 ]. The importance of building a community of practice was also discussed by Gotlib Conn et al. (2015), who identified this as a key component of successful implementation [ 8 ]. This was also considered in the study of Larson et al. [ 25 ] that focused on the collaborative implementation of a CP for colorectal cancer surgery.

All hospitals save for Hospital 8 used feedback as important implementation activity. A systematic review revealed that audit and feedback play a role inpromoting effective changes to current practice [ 40 ]. Audit and feedback were also identified as key facilitators of the implementation of an ERP [ 12 ]. Audit and feedback, also known as reflexive monitoring in eNPT terminology, are important for the reconfiguration of actions and social relations that are necessary to normalize a given intervention [ 22 ]. We observed no differences in the perceived importance and use of feedback that could explain the differences in performance among the hospitals in our study.

As proposed in eNPT, the core constructs capability, capacity, and potential, all have an impact on contribution, which are the actions taken that serve to implement the intervention. In the end, the implementation and normalization of a complex intervention depend on continuous contributions from all users [ 22 ]. Fig.  3 highlights factors that may explain the differences between pre- and post-implementation performance. This figure is based on the “resources and possibilities for agents’ contributions to implementation processes” as described by May et al. and links the four main constructs [ 22 ]. The findings provide a specific focus on the factors that were present in high-performing hospitals and that were absent among those that were low-performing. Other factors, including workability of the CP, availability of resources, sense-making, collective and diverse implementation activities, and the use of reflexive monitoring, have been reported as important factors in the implementation process and were present in both the high- and low-ranking hospitals.

Factors contributing to the differences observed in pre- and post-implementation of the CP (adapted from May, 2013)

Factors present in high-performing, but not in low performing hospitals.

Strengths and limitations

The study was performed over a period of 2 years. During this time, the participating teams had the opportunity to review their processes and to develop, improve, implement, and normalize their CPs. A major methodological strength of this study is that the interviews were performed and initially coded before the quantitative data were analyzed. This strategy serves to reduce interpretation bias [ 21 ]. The selection of hospitals from both the high and low ends of the performance spectrum ensured that information-rich cases would be included.

Our study has several limitations. Because it was not feasible to include all 10 hospitals in the analysis, we selected the top 2 and bottom 3 hospitals based on the ranking data presented in Table 3 . This was an arbitrary selection, and we recognize that other selection strategies were possible. The ranking shows that the hospitals ranked 1–4 exhibit total ranking scores between 10 and 15 points, while the hospitals ranked 5–7 have total scores between 23 and 26 points, and the hospitals ranked 8–10 scored between 30 to 33 points. These findings suggest that the 10 hospitals can be divided into high, intermediate, and low performers. To validate our selection of the top 2 and bottom 3 hospitals, we compared some of our findings with those from the intermediate group (Hospitals 3–7). The characteristics of Hospitals 3 and 4 were similar to those described in detail for Hospitals 1 and 2. This suggests that including data from Hospitals 3 and 4 would provide no additional insights and that we captured ample data on high performance in our analysis that included only Hospitals 1 and 2. The hospitals in the intermediate group showed a more diverse picture. Some characteristics were similar to those of the high-performing hospitals (e.g., motivation, the involvement of a local champion, and variety of implementation activities) while some characteristics were similar to those of the low-performing hospitals (e.g., few resources and lack of support from management, as well as low levels of collective commitment and support/training in CP methodology). These findings were anticipated and stand in support of our decision to limit our analysis to findings from the top and bottom hospitals based on our ranking profile.

Hospitals 8 and 9 had a joint quality management officer and project support. As such, it was not always clear how to assign applicable responses. Furthermore, interviews were conducted with only 3 or 4 professionals who were directly involved at each hospital and, as such, we may have only a limited account of the implementation process. To mitigate this, we used data triangulation methods and checked interview data with field and project notes, which is an established method to enhance the trustworthiness of data [ 41 ].

Given the importance of improving protocol adherence and reducing LOS, additional research is warranted to increase our understanding of the contributing factors identified in our study (Fig. 3 ). Further research might focus on the effort to achieve data saturation at a single hospital, as opposed to data saturation for the overall sample. Similarly, audits in those hospitals using the CP or a longitudinal quantitative study might help to determine whether the CP was normalized in one or more of these cases.

Our study combined quantitative and qualitative data and revealed that a change in protocol adherence does not automatically lead to a change in LOS. Overall improvement in both protocol adherence and LOS was achieved, although the findings were highly variable among the hospitals studied.

Multiple factors in the implementation process could contribute to the differences in the IRs observed here. Conceptualization of these factors using eNPT suggests that teams that can integrate the CP into their social system, those that have experience or that receive support for the implementation of CP methodology, as well as those that are intrinsically motivated, capable of working towards shared goals, receive active management support, and are employed in environments in which CP development is aligned with the hospital strategy are ultimately more successful at the implementation of a CP for colorectal cancer surgery.

Our conclusion implies that multidisciplinary teams intending to implement a CP should invest in shared goals and teamwork and should focus on the integration of the CP into daily processes. Support from hospital management directed specifically at quality improvement may likewise facilitate the implementation process.

Availability of data and materials

The datasets generated and/or analyzed during the current study are not publicly available due them containing information that could compromise research participant privacy/consent, but are available from the corresponding author (RvZ) on reasonable request.

Abbreviations

Adherence Rate

Care Pathway

Diagnostic and Statistical Manual of Mental Disorders

Enhanced Recovery After Surgery

Enhanced Recovery Protocol

Improvement Rate

Length Of Stay

(extended) Normalization Process Theory

Self-rated Adherence

Ljungqvist O, Scott M, Fearon KC. Enhanced recovery after surgery: a review. JAMA Surg. 2017;152(3):292–8.

Article   PubMed   Google Scholar  

Gustafsson UO, Scott MJ, Hubner M, Nygren J, Demartines N, Francis N, et al. Guidelines for perioperative Care in Elective Colorectal Surgery: enhanced recovery after surgery (ERAS((R))) society recommendations: 2018. World J Surg. 2019;43(3):659–95.

Article   CAS   PubMed   Google Scholar  

Lau CS, Chamberlain RS. Enhanced recovery after surgery programs improve patient outcomes and recovery: a meta-analysis. World J Surg. 2017;41(4):899–913.

Gustafsson UO, Oppelstrup H, Thorell A, Nygren J, Ljungqvist O. Adherence to the ERAS protocol is associated with 5-year survival after colorectal Cancer surgery: a retrospective cohort study. World J Surg. 2016;40(7):1741–7.

Martin L, Gillis C, Atkins M, Gillam M, Sheppard C, Buhler S, et al. Implementation of an enhanced recovery after surgery program can change nutrition care practice: a multicenter experience in elective colorectal surgery. JPEN J Parenter Enteral Nutr. 2019;43(2):206–19.

Pisarska M, Pedziwiatr M, Malczak P, Major P, Ochenduszko S, Zub-Pokrowiecka A, et al. Do we really need the full compliance with ERAS protocol in laparoscopic colorectal surgery? A prospective cohort study. Int J Surg (London, England). 2016;36(Pt A):377–82.

Article   Google Scholar  

Li L, Jin J, Min S, Liu D, Liu L. Compliance with the enhanced recovery after surgery protocol and prognosis after colorectal cancer surgery: a prospective cohort study. Oncotarget. 2017;8(32):53531–41.

Article   PubMed   PubMed Central   Google Scholar  

Gotlib Conn L, McKenzie M, Pearsall EA, McLeod RS. Successful implementation of an enhanced recovery after surgery programme for elective colorectal surgery: a process evaluation of champions' experiences. Implement Sci. 2015;10:99.

Gramlich LM, Sheppard CE, Wasylak T, Gilmour LE, Ljungqvist O, Basualdo-Hammond C, et al. Implementation of enhanced recovery after surgery: a strategy to transform surgical care across a health system. Implement Sci. 2017;12(1):67.

Alawadi ZM, Leal I, Phatak UR, Flores-Gonzalez JR, Holihan JL, Karanjawala BE, et al. Facilitators and barriers of implementing enhanced recovery in colorectal surgery at a safety net hospital: a provider and patient perspective. Surgery. 2016;159(3):700–12.

Herbert G, Sutton E, Burden S, Lewis S, Thomas S, Ness A, et al. Healthcare professionals’ views of the enhanced recovery after surgery programme: a qualitative investigation. BMC Health Serv Res. 2017;17(1):1–14.

Stone AB, Yuan CT, Rosen MA, Grant MC, Benishek LE, Hanahan E, et al. Barriers to and facilitators of implementing enhanced recovery pathways using an implementation framework: a systematic review. JAMA Surg. 2018;153(3):270–9.

van Zelm R, Coeckelberghs E, Sermeus W, De Buck van Overstraeten A, Weimann A, Seys D, Panella M, Vanhaecht K. Variation in care for surgical patients with colorectal cancer: protocol adherence in 12 European hospitals. Int J Color Dis. 2017;32(10):1471–8.

van Zelm R, Coeckelberghs E, Sermeus W, Wolthuis A, Bruyneel L, Panella M, et al. Effects of implementing a care pathway for colorectal cancer surgery in ten European hospitals: an international multicenter pre-post-test study. Updat Surg. 2020;72:61–71.

van Zelm R, Coeckelberghs E, Aeyels D, Sermeus W, Wolthuis A, Panella M, et al. Qualitative evaluation of the implementation of a care pathway for colorectal cancer surgery. Qual Health Res. 2020;26:1049732320965188. https://doi.org/10.1177/1049732320965188 (Epub ahead of print).

Seys D. Understanding the impact of care pathways on adherence to guidelines, patient outcomes and teamwork. Leuven: KU Leuven; 2017.

Google Scholar  

Rodriguez-Lopez JL, Ling DC, Heron DE, Beriwal S. Lag time between evidence and guidelines: can clinical pathways bridge the gap? J Oncol Pract. 2019;15(3):e195–201.

Vanhaecht K, Sermeus W, Van Zelm R, Panella M. Care pathways are defined as complex interventions. BMC Med. 2010;8:31.

van Zelm R, Coeckelberghs E, Sermeus W, Aeyels D, Panella M, Vanhaecht K. Protocol for process evaluation of evidence-based care pathways: the case of colorectal cancer surgery. Int J Evid Based Healthc. 2018;16(3):145–53.

Creswell JW, Plano Clark VL. Designing and conducting mixed methods research. 3rd ed. Los Angeles: Sage; 2018.

Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ (Clinical research ed). 2015;350:h1258.

May C. Towards a general theory of implementation. Implementation Sci. 2013;8:18.

May CR, Cummings A, Girling M, Bracher M, Mair FS, May CM, et al. Using normalization process theory in feasibility studies and process evaluations of complex healthcare interventions: a systematic review. Implement Sci. 2018;13(1):80.

Guetterman TC, Fetters MD, Creswell JW. Integrating quantitative and qualitative results in health science mixed methods research through joint displays. Ann Fam Med. 2015;13(6):554–61.

Larson DW, Lovely JK, Welsh J, Annaberdyev S, Coffey C, Corning C, et al. A collaborative for implementation of an evidence-based clinical pathway for enhanced recovery in colon and rectal surgery in an affiliated network of healthcare organizations. Jt Comm J Qual Patient Saf. 2018;44(4):204–11.

PubMed   Google Scholar  

Adams AS, Soumerai SB, Lomas J, Ross-Degnan D. Evidence of self-report bias in assessing adherence to guidelines. Int J Qual Health Care. 1999;11(3):187–92.

Balvardi S, Pecorelli N, Castelino T, Niculiseanu P, Liberman AS, Charlebois P, et al. Measuring in-hospital recovery after colorectal surgery within a well-established enhanced recovery pathway: a comparison between hospital length of stay and time to readiness for discharge. Dis Colon Rectum. 2018;61(7):854–60.

Coxon A, Nielsen K, Cross J, Fox C. Implementing enhanced recovery pathways: a literature review with realist synthesis. Hospital Pract (1995). 2017;45(4):165–74.

Drew S, Judge A, May C, Farmer A, Cooper C, Javaid MK, et al. Implementation of secondary fracture prevention services after hip fracture: a qualitative study using extended normalization process theory. Implement Sci. 2015;10:57.

Thomas K, Bendtsen P, Krevers B. Towards implementing coordinated healthy lifestyle promotion in primary care: a mixed method study. Int J Integr Care. 2015;15:e030.

Kringos DS, Sunol R, Wagner C, Mannion R, Michel P, Klazinga NS, et al. The influence of context on the effectiveness of hospital quality improvement strategies: a review of systematic reviews. BMC Health Serv Res. 2015;15(1):277.

Martin D, Roulin D, Grass F, Addor V, Ljungqvist O, Demartines N, et al. A multicentre qualitative study assessing implementation of an Enhanced Recovery After Surgery program. Clin Nutr (Edinburgh, Scotland). 2018;37(6 Pt A):2172–7.

Article   CAS   Google Scholar  

Darker CD, Nicolson GH, Carroll A, Barry JM. The barriers and facilitators to the implementation of National Clinical Programmes in Ireland: using the MRC framework for process evaluations. BMC Health Serv Res. 2018;18(1):1–10.

Bjurling-Sjoberg P, Wadensten B, Poder U, Jansson I, Nordgren L. Struggling for a feasible tool - the process of implementing a clinical pathway in intensive care: a grounded theory study. BMC Health Serv Res. 2018;18(1):831.

Jabbour M, Newton AS, Johnson D, Curran JA. Defining barriers and enablers for clinical pathway implementation in complex clinical settings. Implement Sci. 2018;13(1):139.

Sutton E, Herbert G, Burden S, Lewis S, Thomas S, Ness A, et al. Using the normalization process theory to qualitatively explore sense-making in implementation of the enhanced recovery after surgery programme: "It's not rocket science". PLoS One. 2018;13(4):e0195890.

Article   PubMed   PubMed Central   CAS   Google Scholar  

Evans-Lacko S, Jarrett M, McCrone P, Thornicroft G. Facilitators and barriers to implementing clinical care pathways. BMC Health Serv Res. 2010;10(1):1–6.

Liu VX, Rosas E, Hwang J, Cain E, Foss-Durant A, Clopp M, et al. Enhanced recovery after surgery program implementation in 2 surgical populations in an integrated health care delivery system. JAMA Surg. 2017;152(7):e171032.

Banks J, Wye L, Hall N, Rooney J, Walter FM, Hamilton W, et al. The researchers' role in knowledge translation: a realist evaluation of the development and implementation of diagnostic pathways for cancer in two United Kingdom localities. Health Res Policy Syst. 2017;15(1):103.

Ivers NM, Grimshaw JM, Jamtvedt G, Flottorp S, O'Brien MA, French SD, et al. Growing literature, stagnant science? Systematic review, meta-regression and cumulative analysis of audit and feedback interventions in health care. J Gen Intern Med. 2014;29(11):1534–41.

Ritchie J, Lewis J, McNaughton Nicholls C, Ormston R. Qualitative research practice. A guide for social science students and researchers. 2nd ed. London: Sage; 2014.

Download references

Acknowledgements

We thank the local coordinators and improvement teams in the participating hospitals:

Belgium: AZ Groeninge, Kortrijk (Dr. B. Van Geluwe /. K. Vandendriessche / D. Verhelst), Institute Bordet, Brussels (Dr. G. Liberale / S. Timmermans), University Hospital Leuven (Prof. Dr. A. D’Hoore / Prof Dr. A. Wolthuis / D. Michiels / K. Op de Beeck / I. Verhelst).

France: American Hospital of Paris Dr. J. Pitre / P. Broumalt / P. Ihout / J. Tuall) / Institute Hospitalier Franco-Britannique, Paris (Dr. S. Houhou / P. Ihout) Clinique Hartmann – Ambroise Parre, Paris (Dr. P. Wintringer / P. Ihout).

Germany: Kreisklinikum Ebersberg (Dr. D. Plecity / I. Schwarz), Klinikum St. Georg, Leipzig (Prof. Dr. A. Weimann / Dr. M. Braunert / Dr. M. Wobith).

The Netherlands: Onze Lieve Vrouwe Hospital, Amsterdam (H. Hiemstra / E. Gooskens), Wilhelmina Hospital Assen (Dr. W. Bleeker / G. Boekeloo / H. Bouwman).

Research made possible by an unconditional educational grant by Baxter SA, Baxter Belgium, Baxter France, Baxter Germany and Baxter The Netherlands to the European Pathway Association. Baxter had no role in the study design, data collection, data analysis, interpretation of results, or manuscript creation.

Author information

Authors and affiliations.

Leuven Institute for Healthcare Policy, Katholieke Universiteit Leuven, Leuven, Belgium

R. van Zelm, E. Coeckelberghs, L. Bruyneel & K. Vanhaecht

Depertment of Abdominal Surgery, University Hospital Leuven, Leuven, Belgium

W. Sermeus & A. Wolthuis

Department of Translational Medicine, University of Eastern Piemonte (UPO), Novarra, Italy

Department of Quality, Academic Policy Advisor, University Hospital Leuven, Leuven, Belgium

K. Vanhaecht

You can also search for this author in PubMed   Google Scholar

Contributions

RvZ made substantial contributions to conception and design of the study, acquisition, analysis and interpretation of data and drafted the manuscript. EC was a major contributor to acquisition, analysis and interpretation of data, and was involved in drafting the manuscript. WS was a major contributor to conception and design of the study, and to interpretation of data. AW made substantial contributions to interpretation of data. LB made substantial contributions to analysis and interpretation of data. MP made substantial contributions to conception and design of the study. KV made substantial contributions to conception and design of the study, analysis and interpretation of the data, and was involved in drafting the manuscript. All authors agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. All authors read and approved the manuscript.

Corresponding author

Correspondence to R. van Zelm .

Ethics declarations

Ethics approval and consent to participate.

Ethics approval for this study was obtained from the ethics committee of the University Hospital Leuven, Belgium (S57152 (ML11311), the Comité de Protection des Personnes, Ile de France IV, Hôptital Saint-Louis, France (2015/07NI), and the ethics committee of the Saxon State Medical Association, Germany (EK-BR-15/15-1). Ethics approval was not applied for in the Netherlands, since the participating hospitals indicated that the project is a quality improvement project, with retrospective patient record analysis, and as such ethics approval was not required.

Interview participants were given the opportunity to ask questions about the study and each participant provided written informed consent to participate in the study. The participants were informed that their participation was voluntary, and that it was possible to withdraw from the study at any time.

Consent for publication

Not Applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1..

Case descriptions

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

van Zelm, R., Coeckelberghs, E., Sermeus, W. et al. A mixed methods multiple case study to evaluate the implementation of a care pathway for colorectal cancer surgery using extended normalization process theory. BMC Health Serv Res 21 , 11 (2021). https://doi.org/10.1186/s12913-020-06011-w

Download citation

Received : 17 March 2020

Accepted : 10 December 2020

Published : 04 January 2021

DOI : https://doi.org/10.1186/s12913-020-06011-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Process evaluation
• Care pathway
• Enhanced recovery
• Mixed methods case study
• Implementation
• Extended normalization process theory (eNPT)

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

• Study protocol
• Open access
• Published: 04 March 2013

Implementing a knowledge application program for anxiety and depression in community-based primary mental health care: a multiple case study research protocol

• Pasquale Roberge 1 , 2 ,
• Louise Fournier 2 , 3 ,
• Hélène Brouillet 2 ,
• Catherine Hudon 1 ,
• Janie Houle 4 ,
• Martin D Provencher 5 &
• Jean-Frédéric Lévesque 2 , 3  

Implementation Science volume  8 , Article number:  26 ( 2013 ) Cite this article

9981 Accesses

6 Citations

4 Altmetric

Metrics details

Anxiety and depressive disorders are increasingly recognized as a health care policy priority. Reducing the treatment gap for common mental disorders requires strengthening the quality of primary mental health care. We developed a knowledge application program designed to improve the organization and delivery of care for anxiety and depression in community-based primary mental health care teams in Quebec, Canada. The principal objectives of the study are: to implement and evaluate this evidence-based knowledge application program; to examine the contextual factors associated with the selection of local quality improvement strategies; to explore barriers and facilitators associated with the implementation of local quality improvement plans; and to study the implementation of local quality monitoring strategies.

The research design is a mixed-methods prospective multiple case study. The main analysis unit (cases) is composed of the six multidisciplinary community-based primary mental health care teams, and each of the cases has identified at least one primary care medical clinic interested in collaborating with the implementation project. The training modules of the program are based on the Chronic Care Model, and the implementation strategies were developed according to the Promoting Action on Research Implementation in Health Services conceptual framework.

The implementation of an evidence-based knowledge application program for anxiety and depression in primary care aims to improve the organization and delivery of mental health services. The uptake of evidence to improve the quality of care for common mental disorders in primary care is a complex process that requires careful consideration of the context in which innovations are introduced. The project will provide a close examination of the interplay between evidence, context and facilitation, and contribute to the understanding of factors associated with the process of implementation of interventions in routine care. The implementation of the knowledge application program with a population health perspective is consistent with the priorities set forth in the current mental health care reform in Quebec. Strengthening primary mental health care will lead to a more efficient health care system.

Peer Review reports

Anxiety and depressive disorders are increasingly recognized as a health care policy priority as they are the most common mental disorders among the general population and in primary care[ 1 – 5 ]. Lifetime prevalence is approximately 6.7% for major depression[ 4 ] and 16.6% for anxiety disorders[ 3 ]. These disorders are associated with significant psychological distress, and functional and social impairment[ 5 , 6 ]. People living with these disorders present a high risk of comorbidity, as anxiety and depressive disorders are frequently occurring with other mental disorders as well as chronic physical illness[ 7 – 10 ]. This co-occurrence is also proportional to the severity of disability, persistence of symptoms and deterioration of individuals’ health[ 7 – 9 ]. Efficient management of anxiety and depressive disorders could lead to a reduction in the social and economic costs of mental disorders[ 11 – 14 ]. Global health policies are increasingly concerned with improving public mental health[ 15 , 16 ], and the role of primary care in the recognition and treatment of common mental disorders has significantly evolved since the 1990s[ 17 , 18 ]. While the integration of mental health into primary care is paramount to improving access to mental health care, the current reforms require support to strengthen the quality of primary mental health services across a wide variety of contexts[ 17 ].

Research on the treatment gap for common mental disorders has shown that clinical practices do not keep pace with the ever-growing knowledge regarding optimal anxiety and depression management. This gap is a key issue in mental health services research, and concerted effort is required to further knowledge on the implementation of evidence-based practices[ 19 ]. Although pharmacological and psychological treatments for anxiety and depression have existed for several years now[ 20 – 22 ], it has been established that only a minority of anxiety or depression sufferers are diagnosed and treated according to clinical practice guidelines’ recommendations[ 23 – 28 ]. Access and equity issues have also been raised, in which individual factors such as age, education and mental health insurance coverage have been associated with access to care and treatment adequacy[ 23 , 28 – 32 ]. Several individual, social, professional and systemic factors contribute to this situation, such as low help-seeking and utilization of mental health services for common mental health problems, under-detection of anxiety and depressive disorders in primary care, limited access to evidence-based treatments, particularly psychotherapy, and lack of treatment intensification when required[ 1 , 33 – 35 ].

Research has evaluated several interventions to improve the quality of primary mental health care for common mental disorders, predominantly for depression. The broad spectrum of strategies used to implement change in clinical practice includes simple and inexpensive professional interventions, financial interventions, and organizational interventions centred on patients, care providers, and the health care system[ 36 ]. It is well recognized that isolated educational or organizational strategies, such as passive dissemination of clinical practice guidelines or systematic screening, only minimally impact patient outcomes[ 37 , 38 ]. Furthermore, consultation-liaison models do not appear sufficient as stand-alone interventions to improve patient outcomes at the population level[ 39 ]. Numerous studies show that implementing complex multimodal intervention strategies that include both organizational and educational activities can result in improvements to primary care quality and patient health[ 37 , 38 , 40 – 42 ]. Among complex models of quality improvement, collaborative care has been shown to be more effective than usual care in numerous trials for depression, and in some trials for anxiety disorders[ 38 , 40 , 42 , 43 ]. Models of collaborative care typically include active collaboration between primary care providers and mental health specialists, predominantly psychiatrists, and the role of a case manager[ 41 , 44 , 45 ]. The key functions of the case manager may include patient education, self-management support, systematic follow-up, clinical management, low intensity interventions, and care coordination[ 41 , 46 ]. Principles of stepped care have also been introduced in recent years to improve efficiency by modulating services according to patients’ characteristics. Stepped care involves continuous and systematic assessment of patient outcomes, as well as collaboration between primary care and specialised mental health service[ 35 , 47 – 49 ]. Current knowledge of the most promising active ingredients in these complex strategies is mature enough to implement them into real world clinical practice settings.

There is growing interest in introducing the notion of long-term management in the treatment of depression[ 35 ]. The Chronic Care Model (CCM) is the leading model for the management of long-term chronic conditions, a framework for improvement with a population-based approach that aims to provide planned, proactive, patient-centred, and evidence-based care delivery[ 50 , 51 ]. It involves changes at the system, community, organization, professional and patient level. A growing number of evaluative studies indicate that quality improvement programs based on the CCM lead to improvements in both health care processes and patient health[ 52 , 53 ]. The CCM offers a framework for change that is compatible with complex interventions for common mental disorders. Thus, the model seems particularly useful to structure anxiety and depression care organization, as these are often relapsing and tend to become chronic.

Primary mental health care services in the province of Quebec, Canada

Primary care services have become the key pillar of the mental health care system in Quebec. As of 2005, Quebec’s ministerial policies for mental health have focused on improving the primary care management of mental disorders and achieving an optimal hierarchy of care[ 54 , 55 ]. The mental health strategy included the implementation of community-based primary mental health care teams (CMHTs) throughout the province in each of the 94 Health and Social Services Centres, established within local services networks in each region, with a population responsibility and the management of a gateway for mental health services to ensure access and continuity of care. The team typically comprises a health care administrator, clinical coordinators, psychologists, social workers, nurses, psycho-educators and, occasionally, general practitioners. It is expected as part of the reform that these CMHTs work in collaboration with their local services networks, including primary care clinics, as well as other health care professionals ( e.g. , psychologists in private practice, community resources, hospital’s emergency departments), to meet the mental health care needs of the population. From a population health perspective, we postulate that patients with common mental disorders should be a priority in the CMHTs’ organisation and delivery of care. Consequently, they need to be supported to improve the quality of care for anxiety and depression, to be informed about the best available research evidence, and to use the finite resources of the public health care system in the most effective and efficient manner.

Research on the implementation of evidence-based practice has typically been conducted in controlled conditions, in medical clinics. Considering that Quebec’s primary CMHTs are not established on a medical practice foundation, and considering the relative lack of knowledge on barriers and facilitators to the implementation of change in that innovative context, we developed a knowledge application program specifically designed to support CMHTs in the organization and delivery of care for anxiety and depression. We grounded the knowledge application program on the CCM to ensure that the implementation of change in clinical practice would be supported by rigorous evidence-based data at the patient, clinical and organisational levels. We also relied on the Promoting Action on Research Implementation in Health Services (PARiHS) conceptual framework developed by Kitson and her collaborators[ 56 , 57 ] to design our implementation strategies. According to this model, successful implementation stems from the dynamic and simultaneous relationship between three elements: evidence, context and facilitation[ 56 ]. The dimension of ‘evidence’ refers to scientific robustness, as well as clinical experience and patient preferences. The dimension of ‘context’ takes into account the role of the context, culture, leadership and evaluation in the implementation of interventions in clinical practice . ‘Facilitation’ refers to the process of supporting and enabling the implementation through internal and external facilitators. Concerted efforts are essential to facilitate the uptake of evidence-based health care interventions, and research must focus on the complex process of implementing health service organization innovations and ensuring their sustainability[ 58 ]. This pragmatic and participative approach aims to study the extent to which clinical and organisational interventions assessed in controlled conditions can be implemented in multiple clinical practice contexts in Quebec. Studies that offer a system assessment perspective on implementation are valuable to improve our comprehension of factors that foster or hinder the implementation of interventions in real world practice in primary care[ 59 , 60 ].

To implement and evaluate a knowledge application program for anxiety and depression in six community-based primary mental health care settings;

To examine the contextual factors associated with the selection of quality improvement strategies integrated in the local quality improvement plans;

To study barriers and facilitators associated with the level of implementation of the local quality improvement plans;

To assess the impact of the clinical information systems component of the knowledge application program on the development and implementation of local quality monitoring strategies.

Study design

The research design is a mixed-methods prospective multiple case study[ 59 , 61 ]. A case study design has been selected to thoroughly examine the implementation of the knowledge application program within each CMHT and associated primary care clinic, and to examine the complex interplay between context, evidence and facilitation. Numerous variables determine the successful implementation of organizational and clinical changes in mental health care, and the multiple cases allow for the replication of results among diverse contexts. Case study research could complement current quality improvement research by contributing to understanding of contextual factors associated with the success or failure of quality improvement projects[ 62 ].

Ethical approval

The Research Ethics Committee of the Montréal Health and Social Services Centres Agency, acting as the primary ethics committee for the multicentre project, has approved this study. The six local ethics authorities have endorsed the decision. All local committee members will be asked to sign a written consent form. The confidentiality of participating patients will be protected with coded and anonymous processing of data.

Community-based mental health care teams

The six CMHTs were selected from the 94 Health and Social Services Centres in the province of Québec, Canada, according to diversity in terms of their local health network’s size, resources, and geographic environment (urban, semi-urban, or rural). A purposeful sampling strategy was selected to examine the implementation process in a variety of contexts. The inclusion of a CMHT in the sample was conditional to: the identification of a primary care medical clinic, usually a family medicine group[ 63 ], that would agree to collaborate to the research project; and the commitment to collect patient data as part of the implementation of the quality improvement program. The main analysis unit (case) is composed of the multidisciplinary CMHT. The general characteristics of the six CMHTs included in the sample are presented in Table  1 . Two of the CMHTs had participated in a previous quality improvement project conducted from 2008 to 2010. For each of the six CMHTs, organisation and clinical participants to the quality improvement research project included an administrative leader mandated to ensure coordination of the implementation at the local level and an interdisciplinary local working group (see Table  1 ).

Description of the intervention

The study builds on a knowledge application program that was developed in a previous project for anxiety and depression in CMHTs[ 64 ]. We developed the knowledge application program based on the determinants of a successful implementation of change in clinical practice outlined in the PARiHS framework[ 56 , 57 ]. The program comprises two main phases. The first phase involves the presentation of the knowledge application modules and the development of local quality improvement plans in each local working group, while the second phase consists of the implementation of the quality improvement plans at the local level.

Phase 1: The knowledge application program

The meetings and training modules.

The 10 modules of the knowledge transfer phase are presented through six meetings and training sessions with local working groups. The objectives and overview of the modules of the knowledge application phase are presented in Table  2 . The knowledge application program is founded on evidence-based data for patient, professional and organizational interventions for common mental disorders. Strategies for improvement related to each of the six components of the CCM are addressed throughout the modules: Health System Organization; Community Resources and Policies; Delivery System Design; Decision Support; Clinical Information Systems; Self-Management Support. As an illustration of the contents related to each component of the model, the ‘Delivery System Design’ component includes educational and discussion material on collaborative care and stepped care, while ‘Decision Support’ presents a number of clinical practice guidelines and decision support tools. Considering that local working groups are required to develop and implement a strategy to collect patient data as part of the ‘Clinical Information Systems’ component of the CCM, an overview of that specific improvement strategy will be presented.

The clinical information systems strategy

The collection of clinical data on patients with mental health problems is considered essential to achieve success in quality improvement efforts from a population-based approach[ 66 , 67 ]. Establishing a basic clinical information system for patients with anxiety and depressive disorders is mandatory to the participation of each site in the research project, as patient data on processes of care and health outcomes help plan and coordinate interdisciplinary patient care and obtain feedback for specific population groups[ 67 ]. Due to the considerable time, efforts and resources required to implement a clinical information system, as well as contextual risk factors, the strategy preconized in this project is to actively support CMHTs in the implementation of a basic data collection approach[ 68 ]. A patient data collection procedure will be introduced at each CMHT to support the follow-up, practice evaluation, and quality of care for patients with anxiety or depressive disorders.

The training module will present a patient data collection tool as well as a procedure for the implementation of a basic clinical information system in each CMHT. Medical clinics will be provided with the same toolkit if clinicians are also interested in collecting patient data as part of their collaboration with the CMHTs. The data collection tool will be developed by the research team to gather information on processes of care and health outcomes for patients with anxiety and depressive disorders. We will recommend that health care providers use the data collection tool for their patients aged 18 years or older with a new episode of major depression, panic disorder with or without agoraphobia, social phobia, and/or general anxiety disorder according to the DSM-IV-TR (primary diagnosis)[ 69 ]. These common mental disorders have been selected due to their high prevalence in primary care, their adequate response to both psychological and pharmacological treatments[ 20 , 70 , 71 ], and because their symptomatology can be monitored with common standardized tools[ 72 , 73 ]. Data on the care process will include the professional consulted, diagnosis, intervention type, referral to a specialist, and patient education. Data on patient health status will include standardized symptom and functioning scales to be filled out by patients on a regular basis. The Patient Health Questionnaire (PHQ-9)[ 73 ] for major depression and the Generalised Anxiety Disorder–7 item scale (GAD-7)[ 72 ] will be recommended to monitor symptomatology due to their satisfactory psychometric properties, brevity and free access for clinicians. The PHQ-9 is a well-known measure used to evaluate the severity of depression in clinical practice, and the GAD-7 fulfils the same purpose for anxiety disorders. Patients will also complete the Sheehan Disability Scale to evaluate their functioning in three aspects of daily life (work, social life and recreation, family life); this scale has high internal consistency and sensitivity[ 74 , 75 ].

The research team will also provide a series of quality indicators regarding evidence-based practice for anxiety and depression as presented in the knowledge application program on the following topics: assessment; patient education; self-management support; low intensity interventions; pharmacological management; psychotherapy; systematic follow up; and collaborative care. A primary goal is to encourage the establishment of a local quality monitoring routine closely related to their improvement efforts for anxiety and depression. For instance, a quality indicator could be the ‘percentage of patients with depression whose symptoms are reassessed with the PHQ-9 within three months of initiating treatment’[ 76 ], and the local improvement target could be determined by the local working groups. Consistent with the overall approach of our knowledge application approach, the local working groups will have the opportunity to adapt the recommended strategy to their context in terms of patient selection, data collection tool, procedures, quality indicators, and monitoring.

The facilitation approach

Evidence encompasses various sources of knowledge, the assumption of which is reflected in the program by the presentation of a knowledge application program based on research evidence, as well as a facilitation process that allows for developing a shared vision of evidence with consideration of clinical experience and local context of each CMHT working group. Facilitation is the key strategy to guide the local CMHTs in the implementation of evidence-based practice for anxiety and depression in clinical practice, and several strategies are introduced throughout the knowledge application program to ensure the success of implementation of evidence in clinical practice. A knowledge broker (HB) with a helping and enabling role, as suggested by the PARiHS framework[ 56 , 57 ], assumes the primary responsibility of external facilitation with each of the local leaders and working groups. The role of the knowledge broker includes: the preparation of the knowledge application program, the presentation of the training modules to the local working groups, and the support of local working groups throughout the project’s planning and implementation stages. The following attributes were considered essential to the role of knowledge broker in the project: communication skills, problem solving skills, and understanding of the mental health service organization in the provincial primary care context. The knowledge broker is actively supported in her role by a team of researchers and collaborators, who help develop and implement the knowledge application program and attend particular meetings as content experts.

The internal facilitation is under the responsibility of a local leader working with a committee. The local leader is preferably a health care manager in the CMHT with project management skills and decision-making authority at the local level. The local working group is composed of seven to fourteen members with the decision-making authority or influence necessary to support the development and implementation of the local quality improvement plan (Table  1 ). Throughout the knowledge application program, group members work towards building a local quality improvement plan (continuously planned over the course of the local meetings). Centralized regular meetings with the local leaders and the research team complement the facilitation process and provide an opportunity to discuss local implementation strategies among group leaders and researchers.

The local quality improvement plans

Although the knowledge application program is structured, it is not a prescriptive approach with regard to the implementation of evidence. The program includes several strategies to improve the quality of primary mental health care and services for anxiety or depressive disorders. These strategies are not mutually exclusive; they are complementary. The program promotes an approach that is tailored to the specific context of each CMHT and collaborating primary care medical clinic. It translates into the various players taking an active role at each project phase. The local quality improvement plans are therefore tailored according to the interaction of evidence, context and facilitation in each CMHT, and local working groups have the responsibility to take into consideration these elements in order to decide which specific improvement strategies they will seek to implement in their local context. The minimal requirement for the local working groups is to include at least two improvement strategies in their local quality improvement plans, as well as the mandatory clinical information systems strategy. This quality improvement plan must include the intervention strategies selected, reasons behind these choices, resources and tools required, approaches to take, a realistic implementation timetable and budget, task distribution, and indicators to assess the success of intervention implementations.

Phase 2: The implementation of the local quality improvement plans

Following the knowledge transfer phase and preparation of local quality improvement plans, the six local working groups in the CMHTs begin the implementation of their local improvement strategies. The role of the knowledge broker during the implementation phase includes quarterly meetings with local working groups to promote clinical and organizational quality of care assessment, tools and material presented to the community to aid the process, and support to implement Plan-Do-Study-Act cycles[ 77 , 78 ] to encourage experimentation with small changes, gradually progressing towards more complex improvements in each of the exposed organizations throughout the two years of implementation. The knowledge broker will facilitate discussion meetings with local working groups on the local quality improvement targets based on patients’ data collected by each CMHT. A research assistant will be available to provide support, if required by the local working committees, with the aggregation of data and descriptive analysis for the CMHTs prior to each local quarterly meeting. Data will be de-identified before being transmitted to the research team.

Planning the study of the intervention

Data sources.

Data collection and analysis will be performed using concurrent mixed methods[ 79 , 80 ] that rely on multiple data sources to facilitate understanding of a complex phenomenon. The mixed data collection method is based on four main data sources. First, a daily log will be kept by a knowledge broker for the whole duration of the research project, which will include meeting reports with local working groups and other contacts with local leaders, summaries of telephone conversations, follow-up discussions with key actors, and field notes. Second, the written documents regarding the local quality improvement plans will be consigned for the analysis with all other relevant case documents. Third, the Assessment of Chronic Illness Care (ACIC)[ 81 ] will be used to document the level and nature of changes regarding the six components of the Chronic Care Model. The ACIC is a quality-improvement tool for improving care at the community, organisation, practice and patient levels. The scale is composed of 28 items assessing levels of implementation of quality improvement strategies divided into six sections that are related to the six elements of the Chronic Care Model. The ACIC has been shown to be responsive to organisational quality improvement efforts[ 81 ]. The content of the ACIC tool has been adapted by the research team to identify areas of improvement relating specifically to evidence-based care for common mental disorders. The ACIC questionnaire will be completed before and after the implementation of the quality improvement program in each CMHT by a health care manager and a clinician. Fourth, the patient data collection in each CMHT will gather information on processes of care and health outcomes for patients with anxiety and depressive disorders for each case, and will provide information on local quality indicators and improvement targets.

Data coding and analysis

The first objective of the study aims at evaluating the implementation of the knowledge application program for anxiety and depressive disorders in the six CMHTs. The second objective aims at examining the contextual factors associated with the selection of quality improvement strategies integrated in the local quality improvement plans, while the third objective aims at exploring barriers and facilitators associated with the level of implementation of the local quality improvement plans. Data coding and reduction for these objectives will be based on all relevant qualitative and quantitative data for each of the six sites. The data management and reduction for all written material will be carried out using NVivo 10 (QSR International) qualitative data analysis software. The coding strategy will be based on the three elements of the PARiHS conceptual framework (evidence, context, facilitation), as well as the variables associated with each of the six components of the Chronic Care Model, and emerging themes over the course of the project. Data will be sorted with the ‘case’ as main analysis unit to allow an in-depth analysis of each case and to compare cases within the continuous analysis process.

The qualitative analysis of the data will be conducted using Miles and Huberman’s iterative cyclical process[ 80 ]. Data triangulation involving the first three data sources and the different users will ensure the validity of the description on the cases and their evolution, as well as permit the integration of both data types in the thematic analyses, for a better understanding of the implementation and results in each context. A cross-case analysis will then be conducted toward the end of the study to discern the common themes across cases and examine the interaction of evidence, context and facilitation as described in the PARiHS framework. The systematic comparative analysis of the six study cases will allow for discriminating factors associated with the implementation’s success, and examining the degree of implementation of the knowledge application program components in various contexts. We need to identify which specific components of the program were chosen and which intervention strategies were selected in the local quality improvement plans, whether they were successfully implemented by the end of the implementation phase and what local working groups think about the usefulness of this program and of its different components. We will also examine the contextual factors that influenced the selection of quality improvement strategies in relation to each of the CCM components, as well as explore barriers and facilitators associated with the level of implementation of specific anxiety and depressive disorder management strategies for each of the CCM components as planned in the local quality improvement plans.

The fourth objective aims at assessing the impact of the Clinical Information Systems component of the knowledge application program on the development and implementation of local quality monitoring strategies. Data coding and analysis will be conducted in continuity with the procedure exposed for the previous objectives, with the addition of the fourth data source: data from local clinical information systems. We will examine whether cases successfully implemented a basic clinical information system and introduced a monitoring approach to assess the quality of care for anxiety and depression in their local contexts. We will also explore the use of quality indicators and improvement targets to monitor the implementation of quality improvement strategies as described in the local plans. Contextual factors associated with data collection, selection and adaptation of recommended quality indicators will also be examined, and barriers and enablers to the implementation of the basic clinical information system will also be addressed. The analysis will draw on all data sources, particularly the quality monitoring and quarterly facilitation meetings during the implementation phase in each CMHT. Data for each case will be used for a comprehensive examination of elements, including the coverage of the data collection at the local level, the patient profiles, as well as the types of care processes and patient outcomes data collected. Because the multiple case study aims to provide an in-depth assessment of the implementation process in each specific context, the achievement of quality monitoring goals will be integrated to each within-case analyses.

Time frame of the study

The study will be conducted from November 2011 to October 2014. The project’s preparation phase (November 2011 to August 2012) involves preparing the knowledge application program, organising an initial centralized meeting with local leaders and planning local working groups.

During the knowledge transfer phase (September 2012 to January 2013), the ten modules of the knowledge application program are presented to each of the six local working groups over the course of six three-hour meeting and training sessions. Each site then prepares their local quality improvement plan, which establishes their quality improvement strategies. In this plan, they must determine a project timeline, identify and prioritize the required means and resources to support the implementation of their selected strategies, appoint collaborators and prepare a budget. Between February 2013 and March 2013, the second centralized meeting with local leaders will be held in order to assist them in the finalization of their respective local quality improvement plan.

During the implementation phase (April 2013 to October 2014), the research team will provide tools and support as needed to the local working groups in the implementation of their local improvement strategies. PDSA cycles will be introduced, and experimentation with the patient follow-up worksheet will begin. Local meetings will be held at three-month intervals in order to provide an update, collect patient data, and offer feedback about program impacts. A centralized meeting with local working groups will be held at the beginning and at the end of the implementation phase. Data will be collected throughout the study phases. Results will be presented in conferences and scientific journals, as well as to an advisory board involved in an integrated knowledge transfer approach throughout the process (September 2013 to April 2014).

The implementation of an evidence-based knowledge application program for anxiety and depression in primary care aims to improve the organisation and delivery of mental health services from a population perspective. This project will provide in-depth understanding regarding the contextual factors that are associated with the selection of strategies for improvement, the barriers and facilitators throughout the implementation process, and the development of a quality monitoring routine at the local level. The project will allow a close examination of the interplay between evidence, context and facilitation as conceptualized in the PARiHS framework. The project’s integrated approach to knowledge application is at the core of this collaborative endeavour, one that involves clinicians, health care managers, researchers, and decision-makers alike. This approach allows all stakeholders involved to develop a shared perspective on mental health, which is a determining factor in regard to the impact of the program on primary mental health care practices in Quebec[ 82 ]. We believe this collaboration to be conducive to enhancing reflective practices focused on the quality of mental health care.

A set of limitations should be considered for this study protocol. First, the research design was conceived primarily to examine conceptual and instrumental knowledge use rather than to examine the effects of multifaceted interventions adopted and implemented by each case following the knowledge transfer phase[ 83 ]. Therefore, it will not be possible to draw conclusions on changes in patient outcomes from the implementation of the knowledge application program. However, the results of the multiple case study will provide critical information to guide the development of a structured tailored organisational and professional intervention that could be examined with a pragmatic randomized controlled trial in other CMHTs and medical clinics to evaluate the effects of the implementation of evidence-based practice. Second, since the study is conducted in CMHTs and medical clinics in the province of Quebec, Canada, caution will be required in the transferability of findings to other primary care settings or organizations. We attempted to maximize the generalizability of results for our provincial health care system by conducting the multiple case study in six different contexts and primary care settings. Third, the conduct of an implementation study in multiple routine clinical settings presents challenges for data collection and consequently may present a threat to validity, which we addressed by using a triangulation approach based on multiple quantitative and qualitative data sources.

The uptake of evidence to improve the quality of care for common mental disorders in primary care is a complex process that requires careful consideration of the context in which innovations are introduced. The implementation of the knowledge application program within a population health perspective is consistent with the priorities set forth in the current mental health care reform in Quebec. Strengthening primary mental health care will lead to a more efficient health care system.

Mechanic D: Barriers to help-seeking, detection, and adequate treatment for anxiety and mood disorders: implications for health care policy. J Clin Psychiatry. 2007, 68 (Suppl 2): 20-26.

PubMed   Google Scholar  

Moussavi S, Chatterji S, Verdes E, Tandon A, Patel V, Ustun B: Depression, chronic diseases, and decrements in health: results from the World Health Surveys. Lancet. 2007, 370 (9590): 851-858. 10.1016/S0140-6736(07)61415-9.

Article   PubMed   Google Scholar  

Somers JM, Goldner EM, Waraich P, Hsu L: Prevalence and incidence studies of anxiety disorders: a systematic review of the literature. Can J Psychiatry. 2006, 51 (2): 100-113.

Waraich P, Goldner EM, Somers JM, Hsu L: Prevalence and incidence studies of mood disorders: a systematic review of the literature. Can J Psychiatry. 2004, 49 (2): 124-138.

Kroenke K, Spitzer RL, Williams JB, Monahan PO, Lowe B: Anxiety disorders in primary care: prevalence, impairment, comorbidity, and detection. Ann Intern Med. 2007, 146 (5): 317-325.

World Health Organization: The global burden of disease: 2004 update. 2008, 146- http://www.who.int/healthinfo/global_burden_disease/GBD_report_2004update_full.pdf ,

Google Scholar  

Katon WJ: Living Well with Chronic Illness: A Call for Public Health Action . Improving recognition and quality of depression care in patients with common chronic medical illnesses. 2012, Washington, DC: The National Academies Press,

Roy-Byrne PP, Davidson KW, Kessler RC, Asmundson GJ, Goodwin RD, Kubzansky L, Lydiard RB, Massie MJ, Katon W, Laden SK, Stein MB: Anxiety disorders and comorbid medical illness. Gen Hosp Psychiatry. 2008, 30 (3): 208-225. 10.1016/j.genhosppsych.2007.12.006.

Lamers F, van Oppen P, Comijs HC, Smit JH, Spinhoven P, van Balkom AJ, Nolen WA, Zitman FG, Beekman AT, Penninx BW: Comorbidity patterns of anxiety and depressive disorders in a large cohort study: the Netherlands Study of Depression and Anxiety (NESDA). J Clin Psychiatry. 2011, 72 (3): 341-348. 10.4088/JCP.10m06176blu.

Conway KP, Compton W, Stinson FS, Grant BF: Lifetime comorbidity of DSM-IV mood and anxiety disorders and specific drug use disorders: results from the National Epidemiologic Survey on Alcohol and Related Conditions. J Clin Psychiatry. 2006, 67 (2): 247-257. 10.4088/JCP.v67n0211.

Article   CAS   PubMed   Google Scholar  

Vasiliadis HM, Dionne PA, Preville M, Gentil L, Berbiche D, Latimer E: The excess healthcare costs associated with depression and anxiety in elderly living in the community. Am J Geriatr Psychiatry. 2012, 10.1097/JGP.0b013e318248ae9e.

Roberge P, Marchand A, Reinharz D, Cloutier K, Mainguy N, Miller JM, Bégin J, Turcotte J: Healthcare utilization following cognitive-behavioral treatment for panic disorder with agoraphobia. Cogn Behav Ther. 2005, 34 (2): 79-88. 10.1080/16506070510008443.

Lim K, Jacobs P, Ohinmaa A, Schopflocher D, Dewa CS: A new population-based measure of the economic burden of mental illness in Canada. Chronic Dis Can. 2008, 28 (3): 92-98.

Jacob V, Chattopadhyay SK, Sipe TA, Thota AB, Byard GJ, Chapman DP, Community Preventive Services Task Force: Economics of collaborative care for management of depressive disorders: a community guide systematic review. Am J Prev Med. 2012, 42 (5): 539-549. 10.1016/j.amepre.2012.01.011.

Horton R: Launching a new movement for mental health. Lancet. 2007, 370 (9590): 806-10.1016/S0140-6736(07)61243-4.

Kirby M, Keon W: Out of the shadows at last. Transforming mental health, mental illness and addiction services in Canada. 2006, Senate of Canada: Standing Senate Committee on Social Affairs, Science and Technology

World Health Organization, World Organization of Family Doctors (Wonca): Integrating mental health into primary care: a global perspective. 2008, http://www.who.int/mental_health/policy/services/mentalhealthintoprimarycare/en/ ,

Gask L, Lester H, Kendrick T, Peveler R: Primary Care Mental Health. 2009, London, UK: The Royal College of Psychiatrists

Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B: Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Adm Policy Ment Health. 2009, 36 (1): 24-34. 10.1007/s10488-008-0197-4.

Canadian Psychiatric Association: Clinical practice guidelines: Management of anxiety disorders. Can J Psychiatry. 2006, 51 (Suppl 2): 1-93.

Lam RW, Kennedy SH, Grigoriadis S, McIntyre RS, Milev R, Ramasubbu R, Parikh SV, Patten SB, Ravindran AV, Canadian Network for Mood and Anxiety Treatments (CANMAT): Canadian Network for Mood and Anxiety Treatments (CANMAT) clinical guidelines for the management of major depressive disorder in adults. III. Pharmacotherapy. J Affect Disord. 2009, 117 (Suppl 1): 26-43.

Article   Google Scholar  

Parikh SV, Segal ZV, Grigoriadis S, Ravindran AV, Kennedy SH, Lam RW, Patten SB, Canadian Network for Mood and Anxiety Treatments (CANMAT): Canadian Network for Mood and Anxiety Treatments (CANMAT) clinical guidelines for the management of major depressive disorder in adults. II. Psychotherapy alone or in combination with antidepressant medication. J Affect Disord. 2009, 117 (Suppl 1): 15-25.

Roberge P, Fournier L, Duhoux A, Nguyen CT, Smolders M: Mental health service use and treatment adequacy for anxiety disorders in Canada. Soc Psychiatry Psychiatr Epidemiol. 2011, 46 (4): 321-330. 10.1007/s00127-010-0186-2.

Duhoux A, Fournier L, Nguyen CT, Roberge P, Beveridge R: Guideline concordance of treatment for depressive disorders in Canada. Soc Psychiatry Psychiatr Epidemiol. 2009, 44 (5): 385-392. 10.1007/s00127-008-0444-8.

Stein MB, Roy-Byrne P, Craske MG, Campbell-Sills L, Lang AJ, Golinelli D, Rose RD, Bystritsky A, Sullivan G, Sherbourne CD: Quality of and patient satisfaction with primary health care for anxiety disorders. J Clin Psychiatry. 2011, 72 (7): 970-976. 10.4088/JCP.09m05626blu.

Article   PubMed   PubMed Central   Google Scholar  

Wang PS, Angermeyer M, Borges G, Bruffaerts R, Tat CW, Fayyad J, Gureje O, Haro JM, Huang Y, Kessler RC, Kovess V, Levinson D, Nakane Y, Oakley Brown MA, Ormel JH, Posada-Villa J, Alonso J, Lee S, Heeringa S, Pennell BE, Chatterji S, Ustun TB, de GG: Delay and failure in treatment seeking after first onset of mental disorders in the World Health Organization’s World Mental Health Survey Initiative. World Psychiatry. 2007, 6 (3): 177-185.

PubMed   PubMed Central   Google Scholar  

Fernandez A, Haro JM, Martinez-Alonso M, Demyttenaere K, Brugha TS, Autonell J, de Girolamo G, Bernert S, Lepine JP, Alonso J: Treatment adequacy for anxiety and depressive disorders in six European countries. Br J Psychiatry. 2007, 190: 172-173. 10.1192/bjp.bp.106.023507.

Prins MA, Verhaak PF, Smolders M, Laurant MG, van der Meer K, Spreeuwenberg P, van Marwijk HW, Penninx BW, Bensing JM: Patient factors associated with guideline-concordant treatment of anxiety and depression in primary care. J Gen Intern Med. 2010, 25 (7): 648-655. 10.1007/s11606-009-1216-1.

Duhoux A, Fournier L, Gauvin L, Roberge P: Quality of care for major depression and its determinants: a multilevel analysis. BMC Psychiatry. 2012, 12 (1): 142-10.1186/1471-244X-12-142.

Houle J, Beaulieu MD, Lesperance F, Frasure-Smith N, Lambert J: Inequities in medical follow-up for depression: a population-based study in Montreal. Psychiatr Serv. 2010, 61 (3): 258-263. 10.1176/appi.ps.61.3.258.

Stein MB, Sherbourne CD, Craske MG, Means-Christensen A, Bystritsky A, Katon W, Sullivan G, Roy-Byrne PP: Quality of care for primary care patients with anxiety disorders. Am J Psychiatry. 2004, 161 (12): 2230-2237. 10.1176/appi.ajp.161.12.2230.

Wang PS, Lane M, Olfson M, Pincus HA, Wells KB, Kessler RC: Twelve-month use of mental health services in the United States: results from the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005, 62 (6): 629-640. 10.1001/archpsyc.62.6.629.

Roy-Byrne PP, Wagner AW, Schraufnagel TJ: Understanding and treating panic disorder in the primary care setting. J Clin Psychiatry. 2005, 6 (Suppl 4): 16-22.

Collins KA, Westra HA, Dozois DJA, Burns DD: Gaps in accessing treatment for anxiety and depression: challenges for the delivery of care. Clin Psychol Rev. 2004, 24 (5): 583-616. 10.1016/j.cpr.2004.06.001.

Katon WJ, Seelig M: Population-based care of depression: team care approaches to improving outcomes. J Occup Environ Med. 2008, 50 (4): 459-467. 10.1097/JOM.0b013e318168efb7.

Grol R, Wensing M, Eccles M: Improving Patient Care. The Implementation of Change in Clinical Practice. 2005, Oxford: Elsevier

Gilbody S, Whitty P, Grimshaw J, Thomas R: Educational and organizational interventions to improve the management of depression in primary care: a systematic review. JAMA. 2003, 289 (23): 3145-3151. 10.1001/jama.289.23.3145.

Smolders M, Laurant M, Roberge P, van Balkom A, van Rijswijk E, Bower P, Grol R: Knowledge transfer and improvement of primary and ambulatory care for patients with anxiety. Can J Psychiatry. 2008, 53 (5): 277-293.

Cape J, Whittington C, Bower P: What is the role of consultation-liaison psychiatry in the management of depression in primary care? A systematic review and meta-analysis. Gen Hosp Psychiatry. 2010, 32 (3): 246-254. 10.1016/j.genhosppsych.2010.02.003.

Gilbody S, Bower P, Fletcher J, Richards D, Sutton AJ: Collaborative care for depression: a cumulative meta-analysis and review of longer-term outcomes. Arch Intern Med. 2006, 166 (21): 2314-2321. 10.1001/archinte.166.21.2314.

Williams JW, Gerrity M, Holsinger T, Dobscha S, Gaynes B, Dietrich A: Systematic review of multifaceted interventions to improve depression care. Gen Hosp Psychiatry. 2007, 29 (2): 91-116. 10.1016/j.genhosppsych.2006.12.003.

Kates N, Mazowita G, Lemire F, Jayabarathan A, Bland R, Selby P, Isomura T, Craven M, Gervais M, Audet D: The evolution of collaborative mental health care in Canada: A shared vision for the future. Can J Psychiatry. 2011, 56 (5): 1-10.

Archer J, Bower P, Gilbody S, Lovell K, Richards D, Gask L, Dickens C, Coventry P: Collaborative care for depression and anxiety problems. Cochrane Database Syst Rev. 2012, 10: CD006525-

Christensen H, Griffiths KM, Gulliver A, Clack D, Kljakovic M, Wells L: Models in the delivery of depression care: a systematic review of randomised and controlled intervention trials. BMC Fam Pract. 2008, 9: 25-10.1186/1471-2296-9-25.

Katon W, Von Korff M, Lin E, Simon G: Rethinking practitioner roles in chronic illness: the specialist, primary care physician, and the practice nurse. Gen Hosp Psychiatry. 2001, 23 (3): 138-144. 10.1016/S0163-8343(01)00136-0.

Gensichen J, Beyer M, Muth C, Gerlach FM, Von KM, Ormel J: Case management to improve major depression in primary health care: a systematic review. Psychol Med. 2006, 36 (1): 7-14.

Franx G, Oud M, de Lange J, Wensing M, Grol R: Implementing a stepped-care approach in primary care: results of a qualitative study. Implement Sci. 2012, 7 (1): 8-10.1186/1748-5908-7-8.

Unützer J, Park M: Strategies to improve the management of depression in primary care. Prim Care. 2012, 39 (2): 415-431. 10.1016/j.pop.2012.03.010.

Von Korff M, Tiemens B: Individualized stepped care of chronic illness. West J Med. 2000, 172 (2): 133-137. 10.1136/ewjm.172.2.133.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Wagner EH: Chronic disease management: what will it take to improve care for chronic illness?. Eff Clin Pract. 1998, 1 (1): 2-4.

CAS   PubMed   Google Scholar  

Wagner EH, Austin BT, Davis C, Hindmarsh M, Schaefer J, Bonomi A: Improving chronic illness care: translating evidence into action. Health Aff (Millwood). 2001, 20 (6): 64-78. 10.1377/hlthaff.20.6.64.

Article   CAS   Google Scholar  

Tsai AC, Morton SC, Mangione CM, Keeler EB: A meta-analysis of interventions to improve care for chronic illnesses. Am J Manag Care. 2005, 11 (8): 478-488.

Woltmann E, Grogan-Kaylor A, Perron B, Georges H, Kilbourne AM, Bauer MS: Comparative effectiveness of collaborative chronic care models for mental health conditions across primary, specialty, and behavioral health care settings: systematic review and meta-analysis. Am J Psychiatry. 2012, 169 (8): 790-804. 10.1176/appi.ajp.2012.11111616.

Ministère de la Santé et des Services sociaux du Québec: Plan d’action en santé mentale 2005–2010 - La force des liens. 2005, Québec: Gouvernement du Québec

Ministère de la Santé et des Services sociaux: Orientations relatives à l’organisation des soins et des services offerts à la clientèle adulte par les équipes en santé mentale de première ligne en CSSS. 2011, Québec: Gouvernement du Québec

Kitson A, Harvey G, McCormack B: Enabling the implementation of evidence based practice: a conceptual framework. Qual Health Care. 1998, 7 (3): 149-158. 10.1136/qshc.7.3.149.

Rycroft-Malone J, Kitson A, Harvey G, McCormack B, Seers K, Titchen A, Estabrooks C: Ingredients for change: revisiting a conceptual framework. Qual Saf Health Care. 2002, 11 (2): 174-180. 10.1136/qhc.11.2.174.

Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O: Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004, 82 (4): 581-629. 10.1111/j.0887-378X.2004.00325.x.

VanDeusen LC, Engle RL, Holmes SK, Parker VA, Petzel RA, Nealon Seibert M, Shwartz M, Sullivan JL: Strengthening organizations to implement evidence-based clinical practices. Health Care Manage Rev. 2010, 35 (3): 235-245.

Alexander J, A., Hearld L, R: The science of quality improvement implementation: developing capacity to make a difference. Med Care. 2011, 49: S6-20.

Yin RK: Case Study Research: Design and Methods. 2009, Thousand Oaks, CA: Sage

Baker GR: The contribution of case study research to knowledge of how to improve quality of care. BMJ Qual Saf. 2011, 20 (Suppl 1): 30-35. 10.1136/bmjqs.2010.046490.

Breton M, Levesque JF, Pineault R, Hogg W: Primary care reform: Can Quebec’s family medicine group model benefit from the experience of Ontario’s family health teams?. Healthc Policy. 2011, 7 (2): e122-35.

Cible Qualité Project. http://www.qualaxia.org/ms/cible-qualite/index.php?lg=en ,

Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, Robinson N: Lost in knowledge translation: time for a map?. J Contin Educ Health Prof. 2006, 26 (1): 13-24. 10.1002/chp.47.

Kilbourne AM, Keyser D, Pincus HA: Challenges and opportunities in measuring the quality of mental health care. Can J Psychiatry. 2010, 55 (9): 549-557.

Dorr D, Bonner LM, Cohen AN, Shoai RS, Perrin R, Chaney E, Young AS: Informatics systems to promote improved care for chronic illness: a literature review. J Am Med Inform Assoc. 2007, 14 (2): 156-163. 10.1197/jamia.M2255.

Paré G, Sicotte C, Jaana M, Girouard D: Prioritizing the risk factors influencing the success of clinical information system projects. A Delphi study in Canada. Methods Inf Med. 2008, 47 (3): 251-259.

American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders. 4th edition, text revised. 2000, Washington, DC: American Psychiatric Association

Kennedy SH, Milev R, Giacobbe P, Ramasubbu R, Lam RW, Parikh SV, Patten SB, Ravindran AV, Canadian Network for Mood and Anxiety Treatments (CANMAT): Canadian Network for Mood and Anxiety Treatments (CANMAT) Clinical guidelines for the management of major depressive disorder in adults. J Affect Disord. 2009, 117 (Suppl 1): 44-53.

National Institute for Health and Clinical Excellence (NICE): Depression: the Treatment and Management of Depression in Adults (updated edition). National Clinical Practice Guideline 90. 2009, UK: The British Psychological Society & The Royal College of Psychiatrists

Spitzer RL, Kroenke K, Williams JB, Löwe B: A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006, 166 (10): 1092-1097. 10.1001/archinte.166.10.1092.

Kroenke K, Spitzer RL, Williams JB: The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001, 16 (9): 606-613. 10.1046/j.1525-1497.2001.016009606.x.

Sheehan DV, Harnett-Sheehan K, Spann ME, Thompson HF, Prakash A: Assessing remission in major depressive disorder and generalized anxiety disorder clinical trials with the discan metric of the Sheehan disability scale. Int Clin Psychopharmacol. 2011, 26 (2): 75-83. 10.1097/YIC.0b013e328341bb5f.

Sheehan KH, Sheehan DV: Assessing treatment effects in clinical trials with the discan metric of the Sheehan Disability Scale. Int Clin Psychopharmacol. 2008, 23 (2): 70-83. 10.1097/YIC.0b013e3282f2b4d6.

Agency for Healthcare Research and Quality. http://www.ahrq.gov/ ,

Speroff T, O’Connor G: Study designs for PDSA quality improvement research. Q Manage Health Care. 2004, 13 (1): 17-32.

Langley GL, Moen RD, Nolan KM, Nolan TW, Norman CL, Provost LP: The Improvement Guide: A Practical Approach to Enhancing Organizational Performance: 2nd ed. 2009, San Francisco: Jossey-Bass Publishers

Tashakkori A, Teddlie CE: Mixed Methodology: Combining Qualitative and Quantitative Approaches. 2003, Thousand Oaks: CA: Sage

Miles MB, Huberman AM: Qualitative Data Analysis. 2010, Thousand Oaks: CA: Sage

Bonomi AE, Wagner EH, Glasgow RE, VonKorff M: Assessment of chronic illness care (ACIC): a practical tool to measure quality improvement. Health Serv Res. 2002, 37 (3): 791-820. 10.1111/1475-6773.00049.

Gagnon M: Knowledge dissemination and exchange of knowledge. Knowledge Translation in Health Care: Moving from Evidence to Practice. Edited by: Strauss S, Tetroe J, Graham ID. 2009, UK: Wiley-Blackwell, 235-245.

Graham ID, Bick D, Tetroe J, Straus SE, Harrison MB: Measuring outcomes of evidence-based practice: Distinguishing between knowledge use and its impact. Evaluating the Impact of Implementing Evidence-Based Practice. Edited by: Bick D, Graham ID. 2010, UK: Wiley-Blackwell, 18-37.

Download references

Acknowledgements

This study is supported by grants from the following: Canadian Institutes of Health, Fonds de recherche du Québec – Santé, Ministère de la Santé et des Services Sociaux and Bell Canada. PR, JH, JFL and CH were supported by a research scholarship from the Fonds de recherche du Québec – Santé and LF by an applied public health chair from the Canadian Institutes of Health Research, the Fonds de recherche du Québec – Santé and the Ministère de la Santé et des Services Sociaux. We would like to thank Annie Benoit for her assistance in preparing this article.

Author information

Authors and affiliations.

Département de médecine de famille et de médecine d’urgence, Faculté de médecine et des sciences de la santé, Université de Sherbrooke, 3001, 12e Avenue Nord, Sherbrooke, Québec, Canada

Pasquale Roberge & Catherine Hudon

Institut national de santé publique du Québec, 190, boul. Crémazie Est, Montréal, Québec, Canada

Pasquale Roberge, Louise Fournier, Hélène Brouillet & Jean-Frédéric Lévesque

Centre de recherche du Centre hospitalier universitaire de Montréal (CRCHUM), Pavillon Édouard-Asselin, 264, boul. René-Lévesque Est, Montréal, Québec, Canada

Louise Fournier & Jean-Frédéric Lévesque

Département de psychologie, Université du Québec à Montréal, Pavillon J.A. De Sève, 305 rue Christin, Montréal, Québec, Canada

Janie Houle

École de psychologie, Université Laval, Pavillon Félix-Antoine-Savard, 2325, rue des Bibliothèques, Québec, Québec, Canada

Martin D Provencher

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Pasquale Roberge .

Additional information

Competing interests.

The authors declare that they have no competing interests.

Authors’ contributions

All authors read and approved the final manuscript.

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( http://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article.

Roberge, P., Fournier, L., Brouillet, H. et al. Implementing a knowledge application program for anxiety and depression in community-based primary mental health care: a multiple case study research protocol. Implementation Sci 8 , 26 (2013). https://doi.org/10.1186/1748-5908-8-26

Download citation

Received : 18 January 2013

Accepted : 26 February 2013

Published : 04 March 2013

DOI : https://doi.org/10.1186/1748-5908-8-26

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Anxiety disorders
• Primary care
• Quality of care
• Quality improvement
• Mental health care
• Quality indicators
• Knowledge application

Implementation Science

ISSN: 1748-5908

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

• No results found

THE CASE STUDY PROTOCOL

Research methodology.

The case study protocol defines the procedures and general rules to be followed using the protocol which is different from a survey questionnaire” (Yin, 2009, p. 79). The case study protocol and a survey questionnaire are both directed at a single data point, whether it’s a single case or a single respondent (Yin, 2009). A case study protocol is always needed when

performing a multiple-case study (Yin, 2009). The protocol is a major way of increasing the

reliability of case study research and is intended to guide the researcher in carrying out data

collection from a single case (Yin, 2009). A case study protocol should have at least the following sections (Yin, 2009, p. 81):

1. Overview of the case study project (project objectives and auspices, case study issues, and relevant readings about the topic being investigated).

2. Field procedures (presentation of credentials, access to the case study “sites”, language pertaining to the protection of human subjects, sources of data, and procedural reminders).

3. Field procedures (the specific questions that the case study must keep in mind in collecting data, “table shells” for specific arrays of data, and the potential sources of information for answering each question …).

4. Investigator guide for the case study report (outline, format of the data, use and presentation of other documentation, and bibliographical information).

The importance of the protocol helps the researcher to remain focused on the topic and problem areas. This intuitive knowledge of the context and perspective will guide the researcher in the search for supporting information. By writing an overview of the case study, the researcher allows potential knowledge seeker to capitalize on the products of the case study and understand beforehand, the intent and depth of the case study research. There are also potential guidelines for field procedure. A researcher’s “field procedure of the protocol need to emphasize the major tasks in collecting data, including gaining access to key organizations or interviewees” (Yin, 2009, p. 85):

1. Having sufficient resources while in the field – including a personal computer, writing instruments, paper, paper clips, and a pre-established, quiet place to write notes privately.

2. Developing a procedure for calling for assistance and guidance, if needed, from other case study investigators or colleagues.

3. Making a clear schedule of the data collection activates that are expected to be completed within specified periods of time.

4. Providing for unanticipated events, including changes in the availability of interviewees as well as changes in the mood and motivation of the case study investigator.

“The heart of the protocol is a set of substantive questions reflecting your actual line of inquiry” (Yin, 2009, p. 86). Each question should be “posed to you, the investigator, not to an

interviewee” and linked to a source of evidence (Yin, 2009, p. 86). Each question of this protocol should reflect a specific type/level potentially categorized by Yin’s five levels of questions below (Yin, 2009, p. 86):

1. Level 1: question asked of specific interviewees.

2. Level 2: questions asked of the individual case (these are the questions in the case study protocol to be answered by the investigator during a single case, even when the single case is part of a larger, multiple-case study).

3. Level 3: questions asked of the pattern of findings across multiple cases.

4. Level 4: questions asked of the entire study – for example, calling the information beyond the case study evidence and including other literature or published data that may have been reviewed.

5. Level 5: normative questions about policy recommendations and conclusions, going beyond the narrow scope of study.

“The questions should cater to the unit of analysis of the case study, which may be at a different level from the unit of data collection of the case study” (Yin, 2009, p. 88). “The common

confusion begins because the data collection sources may be individual people (e.g., interviews with individuals), whereas the unit of analysis of your case study may be a collective (e.g., the organization to which the individual belongs) - a frequent design when the case is about the organization, community, or social group” (Yin, 2009, p. 88). Table 6 below illustrates design verses data collection using different units of analysis:

Individual behavior Individual attitudes Individual perceptions

Individual employee records Interview with individual’s supervisor; other employees

How organization works

Why organization works Personnel policiesOrganization outcomes

Abo ut a n indi vidua l Abo ut a n or ga ni za tion De sig n

From an individual From an organization

Data Collection Source

Table 6: Design verses Data Collected

Table 6 above, Design verses Data Collection, helps the researcher to identify exactly what data is desired and ensures parallel information is collected from different sites as during a multiple case study (Yin, 2009, p. 89). The researcher should include an outline in the protocol to guide

in the collection, presentation, and formatting of data (Yin, 2009). This rigor allows other researchers to follow the case (Yin, 2009). The researcher may choose a pilot case to discover unforeseen issues or challenges (Yin, 2009). The protocol helps align the researcher’s data collection efforts.

The case study protocol defines the procedures and general rules to be followed using the protocol (Yin, 2009). Yin (2009) reminds the researcher that the protocol is a major way of increasing the reliability of case study research and is intended to guide the researcher in carrying out data collection from a single case. The case study protocol should contain at minimum the following sections (Yin, 2009): (1) Overview of the case study project; (2) Field procedures (credentials); (3) Field procedures (questions); and (4) a form of investigator guide for the case study report. The importance of the protocol helps the researcher to remain focused on the topic and problem areas. Design verses Data Collection helps the researcher to identify exactly what data is desired and ensures parallel information is collected from different (Yin, 2009). The case study protocol is used in the collection of case study evidence as described in the next section.

• PROJECT MANAGEMENT SYSTEMS (PMS)
• PMS: PMBOK PERSPECTIVE
• CYBERNETICS: PRELUDE TO THE VSM
• VIABLE SYSTEM MODEL (VSM)
• ORIGINS OF THE VSM
• SYSTEM FOUR
• CASE STUDY RESEARCH
• COMPONENTS OF RESEARCH DESIGN
• CASE STUDY DESIGNS
• THE CASE STUDY PROTOCOL (You are here)
• COLLECTING CASE STUDY EVIDENCE
• FRAMEWORK DEVELOPMENT
• DATA COLLECTION STRATEGIES
• ROLE OF THE RESEARCHER
• PROJECT Q: A CASE STUDY

Related documents

Nurses' protocol-based care decision-making: a multiple case study

Affiliations.

• 1 Area of Nursing Professional Development, Clínica Universidad de Navarra, Pamplona, Spain.
• 2 Faculty of Nursing, University of Navarra, Pamplona, Spain.
• 3 Navarra Institute for Health Research (IdiSNA), Pamplona, Spain.
• 4 Area of Research and Innovation, Clínica Universidad de Navarra, Pamplona, Spain.
• 5 Faculty of Nursing, Oakland University, Rochester, MI, USA.
• 6 ImPuLs Research Group, University of Navarra, Pamplona, Spain.
• PMID: 33007122
• DOI: 10.1111/jocn.15524

Aim: To describe and explain nurses' protocol-based care decision-making.

Background: Protocol-based care is a strategy to reduce variability in clinical practice. There are no studies looking at protocol-based care decision-making. Understand this process is key to successful implementation.

Method: A multiple embedded case study was carried out. Nurses' protocol-based care decision-making was studied in three inpatient wards (medical, surgical and medical-surgical) of a university hospital in northern Spain. Data collection was performed between 2015 and 2016 including documentary analysis, non-participant observations, participant observations and interviews. Analysis of quantitative data involved descriptive statistics and qualitative data was submitted to Burnard's method of content analysis (1996). The data integration comprised the integration of the data set of each case separately and the integration of the findings resulting from the comparison of the cases. The following the thread method of data integration was used for this purpose. The SRQR guideline was used for reporting.

Results: The multiple embedded case study revealed protocol-based care decision-making as a linear and variable process that depends on the context and consists of multiple interrelated elements, among which the risk perception is foremost.

Conclusion: This study has allowed progress in protocol-based care decision-making characterisation. This knowledge is crucial to support the design of educational and management strategies aimed at implementing protocol-based care.

Relevance to clinical practice: Strategies to promote protocol-based care should address the contexts of practice and the ability of professionals' to accurately assess the degree of risk of clinical activity. Hence, it will promote quality of care, patient safety and efficiency in healthcare cost.

Keywords: case study research; decision-making; nursing; protocol.

© 2020 John Wiley & Sons Ltd.

• Decision Making*
• Nursing Assessment*

Grants and funding

• 2014/BIL/14/S2/048/Maphre Foundation