types of case study method in research methodology

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Methodology
• Case Study | Definition, Examples & Methods

Case Study | Definition, Examples & Methods

Published on 5 May 2022 by Shona McCombes . Revised on 30 January 2023.

A case study is a detailed study of a specific subject, such as a person, group, place, event, organisation, or phenomenon. Case studies are commonly used in social, educational, clinical, and business research.

A case study research design usually involves qualitative methods , but quantitative methods are sometimes also used. Case studies are good for describing , comparing, evaluating, and understanding different aspects of a research problem .

Table of contents

When to do a case study, step 1: select a case, step 2: build a theoretical framework, step 3: collect your data, step 4: describe and analyse the case.

A case study is an appropriate research design when you want to gain concrete, contextual, in-depth knowledge about a specific real-world subject. It allows you to explore the key characteristics, meanings, and implications of the case.

Case studies are often a good choice in a thesis or dissertation . They keep your project focused and manageable when you don’t have the time or resources to do large-scale research.

You might use just one complex case study where you explore a single subject in depth, or conduct multiple case studies to compare and illuminate different aspects of your research problem.

Prevent plagiarism, run a free check.

Once you have developed your problem statement and research questions , you should be ready to choose the specific case that you want to focus on. A good case study should have the potential to:

• Provide new or unexpected insights into the subject
• Challenge or complicate existing assumptions and theories
• Propose practical courses of action to resolve a problem
• Open up new directions for future research

Unlike quantitative or experimental research, a strong case study does not require a random or representative sample. In fact, case studies often deliberately focus on unusual, neglected, or outlying cases which may shed new light on the research problem.

If you find yourself aiming to simultaneously investigate and solve an issue, consider conducting action research . As its name suggests, action research conducts research and takes action at the same time, and is highly iterative and flexible. 

However, you can also choose a more common or representative case to exemplify a particular category, experience, or phenomenon.

While case studies focus more on concrete details than general theories, they should usually have some connection with theory in the field. This way the case study is not just an isolated description, but is integrated into existing knowledge about the topic. It might aim to:

• Exemplify a theory by showing how it explains the case under investigation
• Expand on a theory by uncovering new concepts and ideas that need to be incorporated
• Challenge a theory by exploring an outlier case that doesn’t fit with established assumptions

To ensure that your analysis of the case has a solid academic grounding, you should conduct a literature review of sources related to the topic and develop a theoretical framework . This means identifying key concepts and theories to guide your analysis and interpretation.

There are many different research methods you can use to collect data on your subject. Case studies tend to focus on qualitative data using methods such as interviews, observations, and analysis of primary and secondary sources (e.g., newspaper articles, photographs, official records). Sometimes a case study will also collect quantitative data .

The aim is to gain as thorough an understanding as possible of the case and its context.

In writing up the case study, you need to bring together all the relevant aspects to give as complete a picture as possible of the subject.

How you report your findings depends on the type of research you are doing. Some case studies are structured like a standard scientific paper or thesis, with separate sections or chapters for the methods , results , and discussion .

Others are written in a more narrative style, aiming to explore the case from various angles and analyse its meanings and implications (for example, by using textual analysis or discourse analysis ).

In all cases, though, make sure to give contextual details about the case, connect it back to the literature and theory, and discuss how it fits into wider patterns or debates.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

McCombes, S. (2023, January 30). Case Study | Definition, Examples & Methods. Scribbr. Retrieved 13 May 2024, from https://www.scribbr.co.uk/research-methods/case-studies/

Is this article helpful?

Shona McCombes

Other students also liked, correlational research | guide, design & examples, a quick guide to experimental design | 5 steps & examples, descriptive research design | definition, methods & examples.

Understanding Case Study Method in Research: A Comprehensive Guide

Table of Contents

Have you ever wondered how researchers uncover the nuanced layers of individual experiences or the intricate workings of a particular event? One of the keys to unlocking these mysteries lies in the qualitative research focusing on a single subject in its real-life context.">case study method , a research strategy that might seem straightforward at first glance but is rich with complexity and insightful potential. Let’s dive into the world of case studies and discover why they are such a valuable tool in the arsenal of research methods.

What is a Case Study Method?

At its core, the case study method is a form of qualitative research that involves an in-depth, detailed examination of a single subject, such as an individual, group, organization, event, or phenomenon. It’s a method favored when the boundaries between phenomenon and context are not clearly evident, and where multiple sources of data are used to illuminate the case from various perspectives. This method’s strength lies in its ability to provide a comprehensive understanding of the case in its real-life context.

Historical Context and Evolution of Case Studies

Case studies have been around for centuries, with their roots in medical and psychological research. Over time, their application has spread to disciplines like sociology, anthropology, business, and education. The evolution of this method has been marked by a growing appreciation for qualitative data and the rich, contextual insights it can provide, which quantitative methods may overlook.

Characteristics of Case Study Research

What sets the case study method apart are its distinct characteristics:

• Intensive Examination: It provides a deep understanding of the case in question, considering the complexity and uniqueness of each case.
• Contextual Analysis: The researcher studies the case within its real-life context, recognizing that the context can significantly influence the phenomenon.
• Multiple Data Sources: Case studies often utilize various data sources like interviews, observations, documents, and reports, which provide multiple perspectives on the subject.
• Participant’s Perspective: This method often focuses on the perspectives of the participants within the case, giving voice to those directly involved.

Types of Case Studies

There are different types of case studies, each suited for specific research objectives:

• Exploratory: These are conducted before large-scale research projects to help identify questions, select measurement constructs, and develop hypotheses.
• Descriptive: These involve a detailed, in-depth description of the case, without attempting to determine cause and effect.
• Explanatory: These are used to investigate cause-and-effect relationships and understand underlying principles of certain phenomena.
• Intrinsic: This type is focused on the case itself because the case presents an unusual or unique issue.
• Instrumental: Here, the case is secondary to understanding a broader issue or phenomenon.
• Collective: These involve studying a group of cases collectively or comparably to understand a phenomenon, population, or general condition.

The Process of Conducting a Case Study

Conducting a case study involves several well-defined steps:

• Defining Your Case: What or who will you study? Define the case and ensure it aligns with your research objectives.
• Selecting Participants: If studying people, careful selection is crucial to ensure they fit the case criteria and can provide the necessary insights.
• Data Collection: Gather information through various methods like interviews, observations, and reviewing documents.
• Data Analysis: Analyze the collected data to identify patterns, themes, and insights related to your research question.
• Reporting Findings: Present your findings in a way that communicates the complexity and richness of the case study, often through narrative.

Case Studies in Practice: Real-world Examples

Case studies are not just academic exercises; they have practical applications in every field. For instance, in business, they can explore consumer behavior or organizational strategies. In psychology, they can provide detailed insight into individual behaviors or conditions. Education often uses case studies to explore teaching methods or learning difficulties.

Advantages of Case Study Research

While the case study method has its critics, it offers several undeniable advantages:

• Rich, Detailed Data: It captures data too complex for quantitative methods.
• Contextual Insights: It provides a better understanding of the phenomena in its natural setting.
• Contribution to Theory: It can generate and refine theory, offering a foundation for further research.

Limitations and Criticism

However, it’s important to acknowledge the limitations and criticisms:

• Generalizability : Findings from case studies may not be widely generalizable due to the focus on a single case.
• Subjectivity: The researcher’s perspective may influence the study, which requires careful reflection and transparency.
• Time-Consuming: They require a significant amount of time to conduct and analyze properly.

Concluding Thoughts on the Case Study Method

The case study method is a powerful tool that allows researchers to delve into the intricacies of a subject in its real-world environment. While not without its challenges, when executed correctly, the insights garnered can be incredibly valuable, offering depth and context that other methods may miss. Robert K\. Yin ’s advocacy for this method underscores its potential to illuminate and explain contemporary phenomena, making it an indispensable part of the researcher’s toolkit.

Reflecting on the case study method, how do you think its application could change with the advancements in technology and data analytics? Could such a traditional method be enhanced or even replaced in the future?

How useful was this post?

Click on a star to rate it!

Average rating 0 / 5. Vote count: 0

No votes so far! Be the first to rate this post.

We are sorry that this post was not useful for you!

Let us improve this post!

Tell us how we can improve this post?

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

Submit Comment

Research Methods in Psychology

1 Introduction to Psychological Research – Objectives and Goals, Problems, Hypothesis and Variables

• Nature of Psychological Research
• The Context of Discovery
• Context of Justification
• Characteristics of Psychological Research
• Goals and Objectives of Psychological Research

2 Introduction to Psychological Experiments and Tests

• Independent and Dependent Variables
• Extraneous Variables
• Experimental and Control Groups
• Introduction of Test
• Types of Psychological Test
• Uses of Psychological Tests

3 Steps in Research

• Research Process
• Identification of the Problem
• Review of Literature
• Formulating a Hypothesis
• Identifying Manipulating and Controlling Variables
• Formulating a Research Design
• Constructing Devices for Observation and Measurement
• Sample Selection and Data Collection
• Data Analysis and Interpretation
• Hypothesis Testing
• Drawing Conclusion

4 Types of Research and Methods of Research

• Historical Research
• Descriptive Research
• Correlational Research
• Qualitative Research
• Ex-Post Facto Research
• True Experimental Research
• Quasi-Experimental Research

5 Definition and Description Research Design, Quality of Research Design

• Research Design
• Purpose of Research Design
• Design Selection
• Criteria of Research Design
• Qualities of Research Design

6 Experimental Design (Control Group Design and Two Factor Design)

• Experimental Design
• Control Group Design
• Two Factor Design

7 Survey Design

• Survey Research Designs
• Steps in Survey Design
• Structuring and Designing the Questionnaire
• Interviewing Methodology
• Data Analysis
• Final Report

8 Single Subject Design

• Single Subject Design: Definition and Meaning
• Phases Within Single Subject Design
• Requirements of Single Subject Design
• Characteristics of Single Subject Design
• Types of Single Subject Design
• Advantages of Single Subject Design
• Disadvantages of Single Subject Design

9 Observation Method

• Definition and Meaning of Observation
• Characteristics of Observation
• Types of Observation
• Advantages and Disadvantages of Observation
• Guides for Observation Method

10 Interview and Interviewing

• Definition of Interview
• Types of Interview
• Aspects of Qualitative Research Interviews
• Interview Questions
• Convergent Interviewing as Action Research
• Research Team

11 Questionnaire Method

• Definition and Description of Questionnaires
• Types of Questionnaires
• Purpose of Questionnaire Studies
• Designing Research Questionnaires
• The Methods to Make a Questionnaire Efficient
• The Types of Questionnaire to be Included in the Questionnaire
• Advantages and Disadvantages of Questionnaire
• When to Use a Questionnaire?

12 Case Study

• Definition and Description of Case Study Method
• Historical Account of Case Study Method
• Designing Case Study
• Requirements for Case Studies
• Guideline to Follow in Case Study Method
• Other Important Measures in Case Study Method
• Case Reports

13 Report Writing

• Purpose of a Report
• Writing Style of the Report
• Report Writing – the Do’s and the Don’ts
• Format for Report in Psychology Area
• Major Sections in a Report

14 Review of Literature

• Purposes of Review of Literature
• Sources of Review of Literature
• Types of Literature
• Writing Process of the Review of Literature
• Preparation of Index Card for Reviewing and Abstracting

15 Methodology

• Definition and Purpose of Methodology
• Participants (Sample)
• Apparatus and Materials

16 Result, Analysis and Discussion of the Data

• Definition and Description of Results
• Statistical Presentation
• Tables and Figures

17 Summary and Conclusion

• Summary Definition and Description
• Guidelines for Writing a Summary
• Writing the Summary and Choosing Words
• A Process for Paraphrasing and Summarising
• Summary of a Report
• Writing Conclusions

18 References in Research Report

• Reference List (the Format)
• References (Process of Writing)
• Reference List and Print Sources
• Electronic Sources
• Book on CD Tape and Movie
• Reference Specifications
• General Guidelines to Write References

Share on Mastodon

Case Studies

Case studies are a popular research method in business area. Case studies aim to analyze specific issues within the boundaries of a specific environment, situation or organization.

According to its design, case studies in business research can be divided into three categories: explanatory, descriptive and exploratory.

Explanatory case studies aim to answer ‘how’ or ’why’ questions with little control on behalf of researcher over occurrence of events. This type of case studies focus on phenomena within the contexts of real-life situations. Example: “An investigation into the reasons of the global financial and economic crisis of 2008 – 2010.”

Descriptive case studies aim to analyze the sequence of interpersonal events after a certain amount of time has passed. Studies in business research belonging to this category usually describe culture or sub-culture, and they attempt to discover the key phenomena. Example: “Impact of increasing levels of multiculturalism on marketing practices: A case study of McDonald’s Indonesia.”

Exploratory case studies aim to find answers to the questions of ‘what’ or ‘who’. Exploratory case study data collection method is often accompanied by additional data collection method(s) such as interviews, questionnaires, experiments etc. Example: “A study into differences of leadership practices between private and public sector organizations in Atlanta, USA.”

Advantages of case study method include data collection and analysis within the context of phenomenon, integration of qualitative and quantitative data in data analysis, and the ability to capture complexities of real-life situations so that the phenomenon can be studied in greater levels of depth. Case studies do have certain disadvantages that may include lack of rigor, challenges associated with data analysis and very little basis for generalizations of findings and conclusions.

John Dudovskiy

Case Study Research

• First Online: 29 September 2022

Cite this chapter

• Robert E. White   ORCID: orcid.org/0000-0002-8045-164X 3 &
• Karyn Cooper 4  

1330 Accesses

As a footnote to the previous chapter, there is such a beast known as the ethnographic case study. Ethnographic case study has found its way into this chapter rather than into the previous one because of grammatical considerations. Simply put, the “case study” part of the phrase is the noun (with “case” as an adjective defining what kind of study it is), while the “ethnographic” part of the phrase is an adjective defining the type of case study that is being conducted. As such, the case study becomes the methodology, while the ethnography part refers to a method, mode or approach relating to the development of the study.

The experiential account that we get from a case study or qualitative research of a similar vein is just so necessary. How things happen over time and the degree to which they are subject to personality and how they are only gradually perceived as tolerable or intolerable by the communities and the groups that are involved is so important. Robert Stake, University of Illinois, Urbana-Champaign

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as EPUB and PDF
• Read on any device
• Instant download
• Own it forever
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Bartlett, L., & Vavrus, F. (2017). Rethinking case study research . Routledge.

Google Scholar  

Bauman, Z. (2000). Liquid modernity . Polity Press.

Bhaskar, R., & Danermark, B. (2006). Metatheory, interdisciplinarity and disability research: A critical realist perspective. Scandinavian Journal of Disability Research, 8 (4), 278–297.

Article   Google Scholar  

Bulmer, M. (1986). The Chicago School of sociology: Institutionalization, diversity, and the rise of sociological research . University of Chicago Press.

Campbell, D. T. (1975). Degrees of freedom and the case study. Comparative Political Studies, 8 (1), 178–191.

Campbell, D. T., & Stanley, J. C. (1966). Experimental and quasi-experimental designs for research . Houghton Mifflin.

Chua, W. F. (1986). Radical developments in accounting thought. The Accounting Review, 61 (4), 601–632.

Creswell, J. W. (2013). Research design: Qualitative, quantitative, and mixed methods approaches (4th ed.). Sage.

Creswell, J. W., & Poth, C. N. (2017). Qualitative inquiry and research design . Sage.

Davey, L. (1991). The application of case study evaluations. Practical Assessment, Research, & Evaluation 2 (9) . Retrieved May 28, 2018, from http://PAREonline.net/getvn.asp?v=2&n=9

Demetriou, H. (2017). The case study. In E. Wilson (Ed.), School-based research: A guide for education students (pp. 124–138). Sage.

Denzin, N. K., & Lincoln, Y. S. (2005). The Sage handbook of qualitative research . Sage.

Flyvbjerg, B. (2004). Five misunderstandings about case-study research. In C. Seale, G. Gobo, J. F. Gubrium, & D. Silverman (Eds.), Qualitative research practice (pp. 420–433). Sage.

Hamel, J., Dufour, S., & Fortin, D. (1993). Case study methods . Sage.

Book   Google Scholar  

Healy, M. E. (1947). Le Play’s contribution to sociology: His method. The American Catholic Sociological Review, 8 (2), 97–110.

Johansson, R. (2003). Case study methodology. [Keynote speech]. In International Conference “Methodologies in Housing Research.” Royal Institute of Technology, Stockholm, September 2003 (pp. 1–14).

Klonoski, R. (2013). The case for case studies: Deriving theory from evidence. Journal of Business Case Studies, 9 (31), 261–266.

McDonough, J., & McDonough, S. (1997). Research methods for English language teachers . Routledge.

Merriam, S. B. (1998). Qualitative research and case study applications in education . Jossey-Bass.

Miles, M. B. (1979). Qualitative data as an attractive nuisance: The problem of analysis. Administrative Science Quarterly, 24 (4), 590–601.

Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Sage.

Mills, A. J., Durepos, G. & E. Wiebe (Eds.) (2010). What is a case study? Encyclopedia of case study research, Volumes I and II. Sage.

National Film Board of Canada. (2012, April). Here at home: In search of the real cost of homelessness . [Web documentary]. Retrieved February 9, 2020, from http://athome.nfb.ca/#/athome/home

Popper, K. (2002). Conjectures and refutations: The growth of scientific knowledge . Routledge.

Ridder, H.-G. (2017). The theory contribution of case study research designs. Business Research, 10 (2), 281–305.

Rolls, G. (2005). Classic case studies in psychology . Hodder Education.

Seawright, J., & Gerring, J. (2008). Case-Selection techniques in case study research: A menu of qualitative and quantitative options. Political Research Quarterly, 61 , 294–308.

Stake, R. E. (1995). The art of case study research . Sage.

Stake, R. E. (2005). Multiple case study analysis . Guilford Press.

Swanborn, P. G. (2010). Case study research: What, why and how? Sage.

Thomas, W. I., & Znaniecki, F. (1996). The Polish peasant in Europe and America: A classic work in immigration history . University of Illinois Press.

Yin, R. K. (1981). The case study crisis: Some answers. Administrative Science Quarterly, 26 (1), 58–65.

Yin, R. K. (1991). Advancing rigorous methodologies : A Review of “Towards Rigor in Reviews of Multivocal Literatures….”. Review of Educational Research, 61 (3), 299–305.

Yin, R. K. (1999). Enhancing the quality of case studies in health services research. Health Services Research, 34 (5) Part II, 1209–1224.

Yin, R. K. (2012). Applications of case study research (3rd ed.). Sage.

Yin, R. K. (2014). Case study research: Design and methods (5th ed.). Sage.

Zaretsky, E. (1996). Introduction. In W. I. Thomas & F. Znaniecki (Eds.), The Polish peasant in Europe and America: A classic work in immigration history (pp. vii–xvii). University of Illinois Press.

Download references

Author information

Authors and affiliations.

Faculty of Education, St. Francis Xavier University, Antigonish, NS, Canada

Robert E. White

OISE, University of Toronto, Toronto, ON, Canada

Karyn Cooper

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Robert E. White .

A Case in Case Study Methodology

Christine Benedichte Meyer

Norwegian School of Economics and Business Administration

Meyer, C. B. (2001). A Case in Case Study Methodology. Field Methods 13 (4), 329-352.

The purpose of this article is to provide a comprehensive view of the case study process from the researcher’s perspective, emphasizing methodological considerations. As opposed to other qualitative or quantitative research strategies, such as grounded theory or surveys, there are virtually no specific requirements guiding case research. This is both the strength and the weakness of this approach. It is a strength because it allows tailoring the design and data collection procedures to the research questions. On the other hand, this approach has resulted in many poor case studies, leaving it open to criticism, especially from the quantitative field of research. This article argues that there is a particular need in case studies to be explicit about the methodological choices one makes. This implies discussing the wide range of decisions concerned with design requirements, data collection procedures, data analysis, and validity and reliability. The approach here is to illustrate these decisions through a particular case study of two mergers in the financial industry in Norway.

In the past few years, a number of books have been published that give useful guidance in conducting qualitative studies (Gummesson 1988; Cassell & Symon 1994; Miles & Huberman 1994; Creswell 1998; Flick 1998; Rossman & Rallis 1998; Bryman & Burgess 1999; Marshall & Rossman 1999; Denzin & Lincoln 2000). One approach often mentioned is the case study (Yin 1989). Case studies are widely used in organizational studies in the social science disciplines of sociology, industrial relations, and anthropology (Hartley 1994). Such a study consists of detailed investigation of one or more organizations, or groups within organizations, with a view to providing an analysis of the context and processes involved in the phenomenon under study.

As opposed to other qualitative or quantitative research strategies, such as grounded theory (Glaser and Strauss 1967) or surveys (Nachmias & Nachmias 1981), there are virtually no specific requirements guiding case research. Yin (1989) and Eisenhardt (1989) give useful insights into the case study as a research strategy, but leave most of the design decisions on the table. This is both the strength and the weakness of this approach. It is a strength because it allows tailoring the design and data collection procedures to the research questions. On the other hand, this approach has resulted in many poor case studies, leaving it open to criticism, especially from the quantitative field of research (Cook and Campbell 1979). The fact that the case study is a rather loose design implies that there are a number of choices that need to be addressed in a principled way.

Although case studies have become a common research strategy, the scope of methodology sections in articles published in journals is far too limited to give the readers a detailed and comprehensive view of the decisions taken in the particular studies, and, given the format of methodology sections, will remain so. The few books (Yin 1989, 1993; Hamel, Dufour, & Fortin 1993; Stake 1995) and book chapters on case studies (Hartley 1994; Silverman 2000) are, on the other hand, mainly normative and span a broad range of different kinds of case studies. One exception is Pettigrew (1990, 1992), who places the case study in the context of a research tradition (the Warwick process research).

Given the contextual nature of the case study and its strength in addressing contemporary phenomena in real-life contexts, I believe that there is a need for articles that provide a comprehensive overview of the case study process from the researcher’s perspective, emphasizing methodological considerations. This implies addressing the whole range of choices concerning specific design requirements, data collection procedures, data analysis, and validity and reliability.

WHY A CASE STUDY?

Case studies are tailor-made for exploring new processes or behaviors or ones that are little understood (Hartley 1994). Hence, the approach is particularly useful for responding to how and why questions about a contemporary set of events (Leonard-Barton 1990). Moreover, researchers have argued that certain kinds of information can be difficult or even impossible to tackle by means other than qualitative approaches such as the case study (Sykes 1990). Gummesson (1988:76) argues that an important advantage of case study research is the opportunity for a holistic view of the process: “The detailed observations entailed in the case study method enable us to study many different aspects, examine them in relation to each other, view the process within its total environment and also use the researchers’ capacity for ‘verstehen.’ ”

The contextual nature of the case study is illustrated in Yin’s (1993:59) definition of a case study as an empirical inquiry that “investigates a contemporary phenomenon within its real-life context and addresses a situation in which the boundaries between phenomenon and context are not clearly evident.”

The key difference between the case study and other qualitative designs such as grounded theory and ethnography (Glaser & Strauss 1967; Strauss & Corbin 1990; Gioia & Chittipeddi 1991) is that the case study is open to the use of theory or conceptual categories that guide the research and analysis of data. In contrast, grounded theory or ethnography presupposes that theoretical perspectives are grounded in and emerge from firsthand data. Hartley (1994) argues that without a theoretical framework, the researcher is in severe danger of providing description without meaning. Gummesson (1988) says that a lack of preunderstanding will cause the researcher to spend considerable time gathering basic information. This preunderstanding may arise from general knowledge such as theories, models, and concepts or from specific knowledge of institutional conditions and social patterns. According to Gummesson, the key is not to require researchers to have split but dual personalities: “Those who are able to balance on a razor’s edge using their pre-understanding without being its slave” (p. 58).

DESCRIPTION OF THE ILLUSTRATIVE STUDY

The study that will be used for illustrative purposes is a comparative and longitudinal case study of organizational integration in mergers and acquisitions taking place in Norway. The study had two purposes: (1) to identify contextual factors and features of integration that facilitated or impeded organizational integration, and (2) to study how the three dimensions of organizational integration (integration of tasks, unification of power, and integration of cultures and identities) interrelated and evolved over time. Examples of contextual factors were relative power, degree of friendliness, and economic climate. Integration features included factors such as participation, communication, and allocation of positions and functions.

Mergers and acquisitions are inherently complex. Researchers in the field have suggested that managers continuously underestimate the task of integrating the merging organizations in the postintegration process (Haspeslaph & Jemison 1991). The process of organizational integration can lead to sharp interorganizational conflict as the different top management styles, organizational and work unit cultures, systems, and other aspects of organizational life come into contact (Blake & Mounton 1985; Schweiger & Walsh 1990; Cartwright & Cooper 1993). Furthermore, cultural change in mergers and acquisitions is compounded by additional uncertainties, ambiguities, and stress inherent in the combination process (Buono & Bowditch 1989).

I focused on two combinations: one merger and one acquisition. The first case was a merger between two major Norwegian banks, Bergen Bank and DnC (to be named DnB), that started in the late 1980s. The second case was a study of a major acquisition in the insurance industry (i.e., Gjensidige’s acquisition of Forenede), that started in the early 1990s. Both combinations aimed to realize operational synergies though merging the two organizations into one entity. This implied disruption of organizational boundaries and threat to the existing power distribution and organizational cultures.

The study of integration processes in mergers and acquisitions illustrates the need to find a design that opens for exploration of sensitive issues such as power struggles between the two merging organizations. Furthermore, the inherent complexity in the integration process, involving integration of tasks, unification of power, and cultural integration stressed the need for in-depth study of the phenomenon over time. To understand the cultural integration process, the design also had to be linked to the past history of the two organizations.

DESIGN DECISIONS

In the introduction, I stressed that a case is a rather loose design that requires that a number of design choices be made. In this section, I go through the most important choices I faced in the study of organizational integration in mergers and acquisitions. These include: (1) selection of cases; (2) sampling time; (3) choosing business areas, divisions, and sites; and (4) selection of and choices regarding data collection procedures, interviews, documents, and observation.

Selection of Cases

There are several choices involved in selecting cases. First, there is the question of how many cases to include. Second, one must sample cases and decide on a unit of analysis. I will explore these issues subsequently.

Single or Multiple Cases

Case studies can involve single or multiple cases. The problem of single cases is limitations in generalizability and several information-processing biases (Eisenhardt 1989).

One way to respond to these biases is by applying a multi-case approach (Leonard-Barton 1990). Multiple cases augment external validity and help guard against observer biases. Moreover, multi-case sampling adds confidence to findings. By looking at a range of similar and contrasting cases, we can understand a single-case finding, grounding it by specifying how and where and, if possible, why it behaves as it does. (Miles & Huberman 1994)

Given these limitations of the single case study, it is desirable to include more than one case study in the study. However, the desire for depth and a pluralist perspective and tracking the cases over time implies that the number of cases must be fairly few. I chose two cases, which clearly does not support generalizability any more than does one case, but allows for comparison and contrast between the cases as well as a deeper and richer look at each case.

Originally, I planned to include a third case in the study. Due to changes in management during the initial integration process, my access to the case was limited and I left this case entirely. However, a positive side effect was that it allowed a deeper investigation of the two original cases and in hindsight turned out to be a good decision.

Sampling Cases

The logic of sampling cases is fundamentally different from statistical sampling. The logic in case studies involves theoretical sampling, in which the goal is to choose cases that are likely to replicate or extend the emergent theory or to fill theoretical categories and provide examples for polar types (Eisenhardt 1989). Hence, whereas quantitative sampling concerns itself with representativeness, qualitative sampling seeks information richness and selects the cases purposefully rather than randomly (Crabtree and Miller 1992).

The choice of cases was guided by George (1979) and Pettigrew’s (1990) recommendations. The aim was to find cases that matched the three dimensions in the dependent variable and provided variation in the contextual factors, thus representing polar cases.

To match the choice of outcome variable, organizational integration, I chose cases in which the purpose was to fully consolidate the merging parties’ operations. A full consolidation would imply considerable disruption in the organizational boundaries and would be expected to affect the task-related, political, and cultural features of the organizations. As for the contextual factors, the two cases varied in contextual factors such as relative power, friendliness, and economic climate. The DnB merger was a friendly combination between two equal partners in an unfriendly economic climate. Gjensidige’s acquisition of Forenede was, in contrast, an unfriendly and unbalanced acquisition in a friendly economic climate.

Unit of Analysis

Another way to respond to researchers’ and respondents’ biases is to have more than one unit of analysis in each case (Yin 1993). This implies that, in addition to developing contrasts between the cases, researchers can focus on contrasts within the cases (Hartley 1994). In case studies, there is a choice of a holistic or embedded design (Yin 1989). A holistic design examines the global nature of the phenomenon, whereas an embedded design also pays attention to subunit(s).

I used an embedded design to analyze the cases (i.e., within each case, I also gave attention to subunits and subprocesses). In both cases, I compared the combination processes in the various divisions and local networks. Moreover, I compared three distinct change processes in DnB: before the merger, during the initial combination, and two years after the merger. The overall and most important unit of analysis in the two cases was, however, the integration process.

Sampling Time

According to Pettigrew (1990), time sets a reference for what changes can be seen and how those changes are explained. When conducting a case study, there are several important issues to decide when sampling time. The first regards how many times data should be collected, while the second concerns when to enter the organizations. There is also a need to decide whether to collect data on a continuous basis or in distinct periods.

Number of data collections. I studied the process by collecting real time and retrospective data at two points in time, with one-and-a-half- and two-year intervals in the two cases. Collecting data twice had some interesting implications for the interpretations of the data. During the first data collection in the DnB study, for example, I collected retrospective data about the premerger and initial combination phase and real-time data about the second step in the combination process.

Although I gained a picture of how the employees experienced the second stage of the combination process, it was too early to assess the effects of this process at that stage. I entered the organization two years later and found interesting effects that I had not anticipated the first time. Moreover, it was interesting to observe how people’s attitudes toward the merger processes changed over time to be more positive and less emotional.

When to enter the organizations. It would be desirable to have had the opportunity to collect data in the precombination processes. However, researchers are rarely given access in this period due to secrecy. The emphasis in this study was to focus on the postcombination process. As such, the precombination events were classified as contextual factors. This implied that it was most important to collect real-time data after the parties had been given government approval to merge or acquire. What would have been desirable was to gain access earlier in the postcombination process. This was not possible because access had to be negotiated. Due to the change of CEO in the middle of the merger process and the need for renegotiating access, this took longer than expected.

Regarding the second case, I was restricted by the time frame of the study. In essence, I had to choose between entering the combination process as soon as governmental approval was given, or entering the organization at a later stage. In light of the previous studies in the field that have failed to go beyond the initial two years, and given the need to collect data about the cultural integration process, I chose the latter strategy. And I decided to enter the organizations at two distinct periods of time rather than on a continuous basis.

There were several reasons for this approach, some methodological and some practical. First, data collection on a continuous basis would have required use of extensive observation that I didn’t have access to, and getting access to two data collections in DnB was difficult in itself. Second, I had a stay abroad between the first and second data collection in Gjensidige. Collecting data on a continuous basis would probably have allowed for better mapping of the ongoing integration process, but the contrasts between the two different stages in the integration process that I wanted to elaborate would probably be more difficult to detect. In Table 1 I have listed the periods of time in which I collected data in the two combinations.

Sampling Business Areas, Divisions, and Sites

Even when the cases for a study have been chosen, it is often necessary to make further choices within each case to make the cases researchable. The most important criteria that set the boundaries for the study are importance or criticality, relevance, and representativeness. At the time of the data collection, my criteria for making these decisions were not as conscious as they may appear here. Rather, being restricted by time and my own capacity as a researcher, I had to limit the sites and act instinctively. In both cases, I decided to concentrate on the core businesses (criticality criterion) and left out the business units that were only mildly affected by the integration process (relevance criterion). In the choice of regional offices, I used the representativeness criterion as the number of offices widely exceeded the number of sites possible to study. In making these choices, I relied on key informants in the organizations.

SELECTION OF DATA COLLECTION PROCEDURES

The choice of data collection procedures should be guided by the research question and the choice of design. The case study approach typically combines data collection methods such as archives, interviews, questionnaires, and observations (Yin 1989). This triangulated methodology provides stronger substantiation of constructs and hypotheses. However, the choice of data collection methods is also subject to constraints in time, financial resources, and access.

I chose a combination of interviews, archives, and observation, with main emphasis on the first two. Conducting a survey was inappropriate due to the lack of established concepts and indicators. The reason for limited observation, on the other hand, was due to problems in obtaining access early in the study and time and resource constraints. In addition to choosing among several different data collection methods, there are a number of choices to be made for each individual method.

When relying on interviews as the primary data collection method, the issue of building trust between the researcher and the interviewees becomes very important. I addressed this issue by several means. First, I established a procedure of how to approach the interviewees. In most cases, I called them first, then sent out a letter explaining the key features of the project and outlining the broad issues to be addressed in the interview. In this letter, the support from the institution’s top management was also communicated. In most cases, the top management’s support of the project was an important prerequisite for the respondent’s input. Some interviewees did, however, fear that their input would be open to the top management without disguising the information source. Hence, it became important to communicate how I intended to use and store the information.

To establish trust, I also actively used my preunderstanding of the context in the first case and the phenomenon in the second case. As I built up an understanding of the cases, I used this information to gain confidence. The active use of my preunderstanding did, however, pose important challenges in not revealing too much of the research hypotheses and in balancing between asking open-ended questions and appearing knowledgeable.

There are two choices involved in conducting interviews. The first concerns the sampling of interviewees. The second is that you must decide on issues such as the structure of the interviews, use of tape recorder, and involvement of other researchers.

Sampling Interviewees

Following the desire for detailed knowledge of each case and for grasping different participant’s views the aim was, in line with Pettigrew (1990), to apply a pluralist view by describing and analyzing competing versions of reality as seen by actors in the combination processes.

I used four criteria for sampling informants. First, I drew informants from populations representing multiple perspectives. The first data collection in DnB was primarily focused on the top management level. Moreover, most middle managers in the first data collection were employed at the head offices, either in Bergen or Oslo. In the second data collection, I compensated for this skew by including eight local middle managers in the sample. The difference between the number of employees interviewed in DnB and Gjensidige was primarily due to the fact that Gjensidige has three unions, whereas DnB only has one. The distribution of interviewees is outlined in Table 2 .

The second criterion was to use multiple informants. According to Glick et al. (1990), an important advantage of using multiple informants is that the validity of information provided by one informant can be checked against that provided by other informants. Moreover, the validity of the data used by the researcher can be enhanced by resolving the discrepancies among different informants’ reports. Hence, I selected multiple respondents from each perspective.

Third, I focused on key informants who were expected to be knowledgeable about the combination process. These people included top management members, managers, and employees involved in the integration project. To validate the information from these informants, I also used a fourth criterion by selecting managers and employees who had been affected by the process but who were not involved in the project groups.

Structured versus unstructured. In line with the explorative nature of the study, the goal of the interviews was to see the research topic from the perspective of the interviewee, and to understand why he or she came to have this particular perspective. To meet this goal, King (1994:15) recommends that one have “a low degree of structure imposed on the interviewer, a preponderance of open questions, a focus on specific situations and action sequences in the world of the interviewee rather than abstractions and general opinions.” In line with these recommendations, the collection of primary data in this study consists of unstructured interviews.

Using tape recorders and involving other researchers. The majority of the interviews were tape-recorded, and I could thus concentrate fully on asking questions and responding to the interviewees’ answers. In the few interviews that were not tape-recorded, most of which were conducted in the first phase of the DnB-study, two researchers were present. This was useful as we were both able to discuss the interviews later and had feedback on the role of an interviewer.

In hindsight, however, I wish that these interviews had been tape-recorded to maintain the level of accuracy and richness of data. Hence, in the next phases of data collection, I tape-recorded all interviews, with two exceptions (people who strongly opposed the use of this device). All interviews that were tape-recorded were transcribed by me in full, which gave me closeness and a good grasp of the data.

When organizations merge or make acquisitions, there are often a vast number of documents to choose from to build up an understanding of what has happened and to use in the analyses. Furthermore, when firms make acquisitions or merge, they often hire external consultants, each of whom produces more documents. Due to time constraints, it is seldom possible to collect and analyze all these documents, and thus the researcher has to make a selection.

The choice of documentation was guided by my previous experience with merger and acquisition processes and the research question. Hence, obtaining information on the postintegration process was more important than gaining access to the due-diligence analysis. As I learned about the process, I obtained more documents on specific issues. I did not, however, gain access to all the documents I asked for, and, in some cases, documents had been lost or shredded.

The documents were helpful in a number of ways. First, and most important, they were used as inputs to the interview guide and saved me time, because I did not have to ask for facts in the interviews. They were also useful for tracing the history of the organizations and statements made by key people in the organizations. Third, the documents were helpful in counteracting the biases of the interviews. A list of the documents used in writing the cases is shown in Table 3 .

Observation

The major strength of direct observation is that it is unobtrusive and does not require direct interaction with participants (Adler and Adler 1994). Observation produces rigor when it is combined with other methods. When the researcher has access to group processes, direct observation can illuminate the discrepancies between what people said in the interviews and casual conversations and what they actually do (Pettigrew 1990).

As with interviews, there are a number of choices involved in conducting observations. Although I did some observations in the study, I used interviews as the key data collection source. Discussion in this article about observations will thus be somewhat limited. Nevertheless, I faced a number of choices in conducting observations, including type of observation, when to enter, how much observation to conduct, and which groups to observe.

The are four ways in which an observer may gather data: (1) the complete participant who operates covertly, concealing any intention to observe the setting; (2) the participant-as-observer, who forms relationships and participates in activities, but makes no secret of his or her intentions to observe events; (3) the observer-as-participant, who maintains only superficial contact with the people being studied; and (4) the complete observer, who merely stands back and eavesdrops on the proceedings (Waddington 1994).

In this study, I used the second and third ways of observing. The use of the participant-as-observer mode, on which much ethnographic research is based, was rather limited in the study. There were two reasons for this. First, I had limited time available for collecting data, and in my view interviews made more effective use of this limited time than extensive participant observation. Second, people were rather reluctant to let me observe these political and sensitive processes until they knew me better and felt I could be trusted. Indeed, I was dependent on starting the data collection before having built sufficient trust to observe key groups in the integration process. Nevertheless, Gjensidige allowed me to study two employee seminars to acquaint me with the organization. Here I admitted my role as an observer but participated fully in the activities. To achieve variation, I chose two seminars representing polar groups of employees.

As observer-as-participant, I attended a top management meeting at the end of the first data collection in Gjensidige and observed the respondents during interviews and in more informal meetings, such as lunches. All these observations gave me an opportunity to validate the data from the interviews. Observing the top management group was by far the most interesting and rewarding in terms of input.

Both DnB and Gjensidige started to open up for more extensive observation when I was about to finish the data collection. By then, I had built up the trust needed to undertake this approach. Unfortunately, this came a little late for me to take advantage of it.

DATA ANALYSIS

Published studies generally describe research sites and data-collection methods, but give little space to discuss the analysis (Eisenhardt 1989). Thus, one cannot follow how a researcher arrives at the final conclusions from a large volume of field notes (Miles and Huberman 1994).

In this study, I went through the stages by which the data were reduced and analyzed. This involved establishing the chronology, coding, writing up the data according to phases and themes, introducing organizational integration into the analysis, comparing the cases, and applying the theory. I will discuss these phases accordingly.

The first step in the analysis was to establish the chronology of the cases. To do this, I used internal and external documents. I wrote the chronologies up and included appendices in the final report.

The next step was to code the data into phases and themes reflecting the contextual factors and features of integration. For the interviews, this implied marking the text with a specific phase and a theme, and grouping the paragraphs on the same theme and phase together. I followed the same procedure in organizing the documents.

I then wrote up the cases using phases and themes to structure them. Before starting to write up the cases, I scanned the information on each theme, built up the facts and filled in with perceptions and reactions that were illustrative and representative of the data.

The documents were primarily useful in establishing the facts, but they also provided me with some perceptions and reactions that were validated in the interviews. The documents used included internal letters and newsletters as well as articles from the press. The interviews were less factual, as intended, and gave me input to assess perceptions and reactions. The limited observation was useful to validate the data from the interviews. The result of this step was two descriptive cases.

To make each case more analytical, I introduced the three dimensions of organizational integration—integration of tasks, unification of power, and cultural integration—into the analysis. This helped to focus the case and to develop a framework that could be used to compare the cases. The cases were thus structured according to phases, organizational integration, and themes reflecting the factors and features in the study.

I took all these steps to become more familiar with each case as an individual entity. According to Eisenhardt (1989:540), this is a process that “allows the unique patterns of each case to emerge before the investigators push to generalise patterns across cases. In addition it gives investigators a rich familiarity with each case which, in turn, accelerates cross-case comparison.”

The comparison between the cases constituted the next step in the analysis. Here, I used the categories from the case chapters, filled in the features and factors, and compared and contrasted the findings. The idea behind cross-case searching tactics is to force investigators to go beyond initial impressions, especially through the use of structural and diverse lenses on the data. These tactics improve the likelihood of accurate and reliable theory, that is, theory with a close fit to the data (Eisenhardt 1989).

As a result, I had a number of overall themes, concepts, and relationships that had emerged from the within-case analysis and cross-case comparisons. The next step was to compare these emergent findings with theory from the organizational field of mergers and acquisitions, as well as other relevant perspectives.

This method of generalization is known as analytical generalization. In this approach, a previously developed theory is used as a template with which to compare the empirical results of the case study (Yin 1989). This comparison of emergent concepts, theory, or hypotheses with the extant literature involves asking what it is similar to, what it contradicts, and why. The key to this process is to consider a broad range of theory (Eisenhardt 1989). On the whole, linking emergent theory to existent literature enhances the internal validity, generalizability, and theoretical level of theory-building from case research.

According to Eisenhardt (1989), examining literature that conflicts with the emergent literature is important for two reasons. First, the chance of neglecting conflicting findings is reduced. Second, “conflicting results forces researchers into a more creative, frame-breaking mode of thinking than they might otherwise be able to achieve” (p. 544). Similarly, Eisenhardt (1989) claims that literature discussing similar findings is important because it ties together underlying similarities in phenomena not normally associated with each other. The result is often a theory with a stronger internal validity, wider generalizability, and a higher conceptual level.

The analytical generalization in the study included exploring and developing the concepts and examining the relationships between the constructs. In carrying out this analytical generalization, I acted on Eisenhardt’s (1989) recommendation to use a broad range of theory. First, I compared and contrasted the findings with the organizational stream on mergers and acquisition literature. Then I discussed other relevant literatures, including strategic change, power and politics, social justice, and social identity theory to explore how these perspectives could contribute to the understanding of the findings. Finally, I discussed the findings that could not be explained either by the merger and acquisition literature or the four theoretical perspectives.

In every scientific study, questions are raised about whether the study is valid and reliable. The issues of validity and reliability in case studies are just as important as for more deductive designs, but the application is fundamentally different.

VALIDITY AND RELIABILITY

The problems of validity in qualitative studies are related to the fact that most qualitative researchers work alone in the field, they focus on the findings rather than describe how the results were reached, and they are limited in processing information (Miles and Huberman 1994).

Researchers writing about qualitative methods have questioned whether the same criteria can be used for qualitative and quantitative studies (Kirk & Miller 1986; Sykes 1990; Maxwell 1992). The problem with the validity criteria suggested in qualitative research is that there is little consistency across the articles as each author suggests a new set of criteria.

One approach in examining validity and reliability is to apply the criteria used in quantitative research. Hence, the criteria to be examined here are objectivity/intersubjectivity, construct validity, internal validity, external validity, and reliability.

Objectivity/Intersubjectivity

The basic issue of objectivity can be framed as one of relative neutrality and reasonable freedom from unacknowledged research biases (Miles & Huberman 1994). In a real-time longitudinal study, the researcher is in danger of losing objectivity and of becoming too involved with the organization, the people, and the process. Hence, Leonard-Barton (1990) claims that one may be perceived as, and may even become, an advocate rather than an observer.

According to King (1994), however, qualitative research, in seeking to describe and make sense of the world, does not require researchers to strive for objectivity and distance themselves from research participants. Indeed, to do so would make good qualitative research impossible, as the interviewer’s sensitivity to subjective aspects of his or her relationship with the interviewee is an essential part of the research process (King 1994:31).

This does not imply, however, that the issue of possible research bias can be ignored. It is just as important as in a structured quantitative interview that the findings are not simply the product of the researcher’s prejudices and prior experience. One way to guard against this bias is for the researcher to explicitly recognize his or her presuppositions and to make a conscious effort to set these aside in the analysis (Gummesson 1988). Furthermore, rival conclusions should be considered (Miles & Huberman 1994).

My experience from the first phase of the DnB study was that it was difficult to focus the questions and the analysis of the data when the research questions were too vague and broad. As such, developing a framework before collecting the data for the study was useful in guiding the collection and analysis of data. Nevertheless, it was important to be open-minded and receptive to new and surprising data. In the DnB study, for example, the positive effect of the reorganization process on the integration of cultures came as a complete surprise to me and thus needed further elaboration.

I also consciously searched for negative evidence and problems by interviewing outliers (Miles & Huberman 1994) and asking problem-oriented questions. In Gjensidige, the first interviews with the top management revealed a much more positive perception of the cultural integration process than I had expected. To explore whether this was a result of overreliance on elite informants, I continued posing problem-oriented questions to outliers and people at lower levels in the organization. Moreover, I told them about the DnB study to be explicit about my presuppositions.

Another important issue when assessing objectivity is whether other researchers can trace the interpretations made in the case studies, or what is called intersubjectivity. To deal with this issue, Miles & Huberman (1994) suggest that: (1) the study’s general methods and procedures should be described in detail, (2) one should be able to follow the process of analysis, (3) conclusions should be explicitly linked with exhibits of displayed data, and (4) the data from the study should be made available for reanalysis by others.

In response to these requirements, I described the study’s data collection procedures and processing in detail. Then, the primary data were displayed in the written report in the form of quotations and extracts from documents to support and illustrate the interpretations of the data. Because the study was written up in English, I included the Norwegian text in a separate appendix. Finally, all the primary data from the study were accessible for a small group of distinguished researchers.

Construct Validity

Construct validity refers to whether there is substantial evidence that the theoretical paradigm correctly corresponds to observation (Kirk & Miller 1986). In this form of validity, the issue is the legitimacy of the application of a given concept or theory to established facts.

The strength of qualitative research lies in the flexible and responsive interaction between the interviewer and the respondents (Sykes 1990). Thus, meaning can be probed, topics covered easily from a number of angles, and questions made clear for respondents. This is an advantage for exploring the concepts (construct or theoretical validity) and the relationships between them (internal validity). Similarly, Hakim (1987) says the great strength of qualitative research is the validity of data obtained because individuals are interviewed in sufficient detail for the results to be taken as true, correct, and believable reports of their views and experiences.

Construct validity can be strengthened by applying a longitudinal multicase approach, triangulation, and use of feedback loops. The advantage of applying a longitudinal approach is that one gets the opportunity to test sensitivity of construct measures to the passage of time. Leonard-Barton (1990), for example, found that one of her main constructs, communicability, varied across time and relative to different groups of users. Thus, the longitudinal study aided in defining the construct more precisely. By using more than one case study, one can validate stability of construct across situations (Leonard-Barton 1990). Since my study only consists of two case studies, the opportunity to test stability of constructs across cases is somewhat limited. However, the use of more than one unit of analysis helps to overcome this limitation.

Construct validity is strengthened by the use of multiple sources of evidence to build construct measures, which define the construct and distinguish it from other constructs. These multiple sources of evidence can include multiple viewpoints within and across the data sources. My study responds to these requirements in its sampling of interviewees and uses of multiple data sources.

Use of feedback loops implies returning to interviewees with interpretations and developing theory and actively seeking contradictions in data (Crabtree & Miller 1992; King 1994). In DnB, the written report had to be approved by the bank’s top management after the first data collection. Apart from one minor correction, the bank had no objections to the established facts. In their comments on my analysis, some of the top managers expressed the view that the political process had been overemphasized, and that the CEO’s role in initiating a strategic process was undervalued. Hence, an important objective in the second data collection was to explore these comments further. Moreover, the report was not as positive as the management had hoped for, and negotiations had to be conducted to publish the report. The result of these negotiations was that publication of the report was postponed one-and-a-half years.

The experiences from the first data collection in the DnB had some consequences. I was more cautious and brought up the problems of confidentiality and the need to publish at the outset of the Gjensidige study. Also, I had to struggle to get access to the DnB case for the second data collection and some of the information I asked for was not released. At Gjensidige, I sent a preliminary draft of the case chapter to the corporation’s top management for comments, in addition to having second interviews with a small number of people. Beside testing out the factual description, these sessions gave me the opportunity to test out the theoretical categories established as a result of the within-case analysis.

Internal Validity

Internal validity concerns the validity of the postulated relationships among the concepts. The main problem of internal validity as a criterion in qualitative research is that it is often not open to scrutiny. According to Sykes (1990), the researcher can always provide a plausible account and, with careful editing, may ensure its coherence. Recognition of this problem has led to calls for better documentation of the processes of data collection, the data itself, and the interpretative contribution of the researcher. The discussion of how I met these requirements was outlined in the section on objectivity/subjectivity above.

However, there are some advantages in using qualitative methods, too. First, the flexible and responsive methods of data collection allow cross-checking and amplification of information from individual units as it is generated. Respondents’ opinions and understandings can be thoroughly explored. The internal validity results from strategies that eliminate ambiguity and contradiction, filling in detail and establishing strong connections in data.

Second, the longitudinal study enables one to track cause and effect. Moreover, it can make one aware of intervening variables (Leonard-Barton 1990). Eisenhardt (1989:542) states, “Just as hypothesis testing research an apparent relationship may simply be a spurious correlation or may reflect the impact of some third variable on each of the other two. Therefore, it is important to discover the underlying reasons for why the relationship exists.”

Generalizability

According to Mitchell (1983), case studies are not based on statistical inference. Quite the contrary, the inferring process turns exclusively on the theoretically necessary links among the features in the case study. The validity of the extrapolation depends not on the typicality or representativeness of the case but on the cogency of the theoretical reasoning. Hartley (1994:225) claims, “The detailed knowledge of the organization and especially the knowledge about the processes underlying the behaviour and its context can help to specify the conditions under which behaviour can be expected to occur. In other words, the generalisation is about theoretical propositions not about populations.”

Generalizability is normally based on the assumption that this theory may be useful in making sense of similar persons or situations (Maxwell 1992). One way to increase the generalizability is to apply a multicase approach (Leonard-Barton 1990). The advantage of this approach is that one can replicate the findings from one case study to another. This replication logic is similar to that used on multiple experiments (Yin 1993).

Given the choice of two case studies, the generalizability criterion is not supported in this study. Through the discussion of my choices, I have tried to show that I had to strike a balance between the need for depth and mapping changes over time and the number of cases. In doing so, I deliberately chose to provide a deeper and richer look at each case, allowing the reader to make judgments about the applicability rather than making a case for generalizability.

Reliability

Reliability focuses on whether the process of the study is consistent and reasonably stable over time and across researchers and methods (Miles & Huberman 1994). In the context of qualitative research, reliability is concerned with two questions (Sykes 1990): Could the same study carried out by two researchers produce the same findings? and Could a study be repeated using the same researcher and respondents to yield the same findings?

The problem of reliability in qualitative research is that differences between replicated studies using different researchers are to be expected. However, while it may not be surprising that different researchers generate different findings and reach different conclusions, controlling for reliability may still be relevant. Kirk and Miller’s (1986:311) definition takes into account the particular relationship between the researcher’s orientation, the generation of data, and its interpretation:

For reliability to be calculated, it is incumbent on the scientific investigator to document his or her procedure. This must be accomplished at such a level of abstraction that the loci of decisions internal to the project are made apparent. The curious public deserves to know how the qualitative researcher prepares him or herself for the endeavour, and how the data is collected and analysed.

The study addresses these requirements by discussing my point of departure regarding experience and framework, the sampling and data collection procedures, and data analysis.

Case studies often lack academic rigor and are, as such, regarded as inferior to more rigorous methods where there are more specific guidelines for collecting and analyzing data. These criticisms stress that there is a need to be very explicit about the choices one makes and the need to justify them.

One reason why case studies are criticized may be that researchers disagree about the definition and the purpose of carrying out case studies. Case studies have been regarded as a design (Cook and Campbell 1979), as a qualitative methodology (Cassell and Symon 1994), as a particular data collection procedure (Andersen 1997), and as a research strategy (Yin 1989). Furthermore, the purpose for carrying out case studies is unclear. Some regard case studies as supplements to more rigorous qualitative studies to be carried out in the early stage of the research process; others claim that it can be used for multiple purposes and as a research strategy in its own right (Gummesson 1988; Yin 1989). Given this unclear status, researchers need to be very clear about their interpretation of the case study and the purpose of carrying out the study.

This article has taken Yin’s (1989) definition of the case study as a research strategy as a starting point and argued that the choice of the case study should be guided by the research question(s). In the illustrative study, I used a case study strategy because of a need to explore sensitive, ill-defined concepts in depth, over time, taking into account the context and history of the mergers and the existing knowledge about the phenomenon. However, the choice of a case study strategy extended rather than limited the number of decisions to be made. In Schramm’s (1971, cited in Yin 1989:22–23) words, “The essence of a case study, the central tendency among all types of case study, is that it tries to illuminate a decision or set of decisions, why they were taken, how they were implemented, and with what result.”

Hence, the purpose of this article has been to illustrate the wide range of decisions that need to be made in the context of a particular case study and to discuss the methodological considerations linked to these decisions. I argue that there is a particular need in case studies to be explicit about the methodological choices one makes and that these choices can be best illustrated through a case study of the case study strategy.

As in all case studies, however, there are limitations to the generalizability of using one particular case study for illustrative purposes. As such, the strength of linking the methodological considerations to a specific context and phenomenon also becomes a weakness. However, I would argue that the questions raised in this article are applicable to many case studies, but that the answers are very likely to vary. The design choices are shown in Table 4 . Hence, researchers choosing a longitudinal, comparative case study need to address the same set of questions with regard to design, data collection procedures, and analysis, but they are likely to come up with other conclusions, given their different research questions.

Adler, P. A., and P. Adler. 1994. Observational techniques. In Handbook of qualitative research, edited by N. K. Denzin and Y. S. Lincoln, 377–92. London: Sage.

Andersen, S. S. 1997. Case-studier og generalisering: Forskningsstrategi og design (Case studies and generalization: Research strategy and design). Bergen, Norway: Fagbokforlaget.

Blake, R. R., and J. S. Mounton. 1985. How to achieve integration on the human side of the merger. Organizational Dynamics 13 (3): 41–56.

Bryman, A., and R. G. Burgess. 1999. Qualitative research. London: Sage.

Buono, A. F., and J. L. Bowditch. 1989. The human side of mergers and acquisitions. San Francisco: Jossey-Bass.

Cartwright, S., and C. L. Cooper. 1993. The psychological impact of mergers and acquisitions on the individual: A study of building society managers. Human Relations 46 (3): 327–47.

Cassell, C., and G. Symon, eds. 1994. Qualitative methods in organizational research: A practical guide. London: Sage.

Cook, T. D., and D. T. Campbell. 1979. Quasi experimentation: Design & analysis issues for field settings. Boston: Houghton Mifflin.

Crabtree, B. F., and W. L. Miller. 1992. Primary care research: A multimethod typology and qualitative road map. In Doing qualitative research: Methods for primary care, edited by B. F. Crabtree and W. L. Miller, 3–28. Vol. 3. Thousand Oaks, CA: Sage.

Creswell, J. W. 1998. Qualitative inquiry and research design: Choosing among five traditions. Thousand Oaks, CA: Sage.

Denzin, N. K., and L. S. Lincoln. 2000. Handbook of qualitative research. London: Sage.

Eisenhardt, K. M. 1989. Building theories from case study research. Academy of Management Review 14 (4): 532–50.

Flick, U. 1998. An introduction to qualitative research. London: Sage.

George, A. L. 1979. Case studies and theory development: The method of structured, focused comparison. In Diplomacy: New approaches in history, theory, and policy, edited by P. G. Lauren, 43–68. New York: Free Press.

Gioia, D. A., and K. Chittipeddi. 1991. Sensemaking and sensegiving in strategic change initiation. Strategic Management Journal 12:433–48.

Glaser, B. G., and A. L. Strauss. 1967. The discovery of grounded theory: Strategies for qualitative research. Chicago: Aldine.

Glick, W. H, G. P. Huber, C. C. Miller, D. H. Doty, and K. M. Sutcliffe. 1990. Studying changes in organizational design and effectiveness: Retrospective event histories and periodic assessments. Organization Science 1 (3): 293–312.

Gummesson, E. 1988. Qualitative methods in management research. Lund, Norway: Studentlitteratur, Chartwell-Bratt.

Hakim, C. 1987. Research design. Strategies and choices in the design of social research. Boston: Unwin Hyman.

Hamel, J., S. Dufour, and D. Fortin. 1993. Case study methods. London: Sage.

Hartley, J. F. 1994. Case studies in organizational research. In Qualitative methods in organizational research: A practical guide, edited by C. Cassell and G. Symon, 209–29. London: Sage.

Haspeslaph, P., and D. B. Jemison. 1991. The challenge of renewal through acquisitions. Planning Review 19 (2): 27–32.

King, N. 1994. The qualitative research interview. In Qualitative methods in organizational research: A practical guide, edited by C. Cassell and G. Symon, 14–36. London: Sage.

Kirk, J., and M. L. Miller. 1986. Reliability and validity in qualitative research. Qualitative Research Methods Series 1. London: Sage.

Leonard-Barton, D. 1990.Adual methodology for case studies: Synergistic use of a longitudinal single site with replicated multiple sites. Organization Science 1 (3): 248–66.

Marshall, C., and G. B. Rossman. 1999. Designing qualitative research. London: Sage.

Maxwell, J. A. 1992. Understanding and validity in qualitative research. Harvard Educational Review 62 (3): 279–99.

Miles, M. B., and A. M. Huberman. 1994. Qualitative data analysis. 2d ed. London: Sage.

Mitchell, J. C. 1983. Case and situation analysis. Sociology Review 51 (2): 187–211.

Nachmias, C., and D. Nachmias. 1981. Research methods in the social sciences. London: Edward Arnhold.

Pettigrew, A. M. 1990. Longitudinal field research on change: Theory and practice. Organization Science 1 (3): 267–92.

___. (1992). The character and significance of strategic process research. Strategic Management Journal 13:5–16.

Rossman, G. B., and S. F. Rallis. 1998. Learning in the field: An introduction to qualitative research. Thousand Oaks, CA: Sage.

Schramm, W. 1971. Notes on case studies for instructional media projects. Working paper for Academy of Educational Development, Washington DC.

Schweiger, D. M., and J. P. Walsh. 1990. Mergers and acquisitions: An interdisciplinary view. In Research in personnel and human resource management, edited by G. R. Ferris and K. M. Rowland, 41–107. Greenwich, CT: JAI.

Silverman, D. 2000. Doing qualitative research: A practical handbook. London: Sage.

Stake, R. E. 1995. The art of case study research. London: Sage.

Strauss, A. L., and J. Corbin. 1990. Basics of qualitative research: Grounded theory procedures and techniques. Newbury Park, CA: Sage.

Sykes, W. 1990. Validity and reliability in qualitative market research: A review of the literature. Journal of the Market Research Society 32 (3): 289–328.

Waddington, D. 1994. Participant observation. In Qualitative methods in organizational research, edited by C. Cassell and G. Symon, 107–22. London: Sage.

Yin, R. K. 1989. Case study research: Design and methods. Applied Social Research Series, Vol. 5. London: Sage.

___. 1993. Applications of case study research. Applied Social Research Series, Vol. 34. London: Sage.

Christine Benedichte Meyer is an associate professor in the Department of Strategy and Management in the Norwegian School of Economics and Business Administration, Bergen-Sandviken, Norway. Her research interests are mergers and acquisitions, strategic change, and qualitative research. Recent publications include: “Allocation Processes in Mergers and Acquisitions: An Organisational Justice Perspective” (British Journal of Management 2001) and “Motives for Acquisitions in the Norwegian Financial Industry” (CEMS Business Review 1997).

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

White, R.E., Cooper, K. (2022). Case Study Research. In: Qualitative Research in the Post-Modern Era. Springer, Cham. https://doi.org/10.1007/978-3-030-85124-8_7

Download citation

DOI : https://doi.org/10.1007/978-3-030-85124-8_7

Published : 29 September 2022

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-85126-2

Online ISBN : 978-3-030-85124-8

eBook Packages : Education Education (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Writing A Case Study

Types Of Case Study

Understand the Types of Case Study Here

People also read

A Complete Case Study Writing Guide With Examples

Simple Case Study Format for Students to Follow

Brilliant Case Study Examples and Templates For Your Help

Case studies are effective research methods that focus on one specific case over time. This gives a detailed view that's great for learning.

Writing a case study is a very useful form of study in the educational process. With real-life examples, students can learn more effectively. 

A case study also has different types and forms. As a rule of thumb, all of them require a detailed and convincing answer based on a thorough analysis.

In this blog, we are going to discuss the different types of case study research methods in detail.

So, let’s dive right in!

• 1. Understanding Case Studies
• 2. What are the Types of Case Study?
• 3. Types of Subjects of Case Study 
• 4. Benefits of Case Study for Students

Understanding Case Studies

Case studies are a type of research methodology. Case study research designs examine subjects, projects, or organizations to provide an analysis based on the evidence.

It allows you to get insight into what causes any subject’s decisions and actions. This makes case studies a great way for students to develop their research skills.

A case study focuses on a single project for an extended period, which allows students to explore the topic in depth.

What are the Types of Case Study?

Multiple case studies are used for different purposes. The main purpose of case studies is to analyze problems within the boundaries of a specific organization, environment, or situation. 

Many aspects of a case study such as data collection and analysis, qualitative research questions, etc. are dependent on the researcher and what the study is looking to address. 

Case studies can be divided into the following categories:

Illustrative Case Study

Exploratory case study, cumulative case study, critical instance case study, descriptive case study, intrinsic case study, instrumental case study.

Let’s take a look at the detailed description of each type of case study with examples. 

An illustrative case study is used to examine a familiar case to help others understand it. It is one of the main types of case studies in research methodology and is primarily descriptive. 

In this type of case study, usually, one or two instances are used to explain what a situation is like. 

Here is an example to help you understand it better:

Illustrative Case Study Example

An exploratory case study is usually done before a larger-scale research. These types of case studies are very popular in the social sciences like political science and primarily focus on real-life contexts and situations.

This method is useful in identifying research questions and methods for a large and complex study. 

Let’s take a look at this example to help you have a better understanding:

Exploratory Case Study Example

A cumulative case study is one of the main types of case studies in qualitative research. It is used to collect information from different sources at different times.

This case study aims to summarize the past studies without spending additional cost and time on new investigations. 

Let’s take a look at the example below:

Cumulative Case Study Example

Critical instances case studies are used to determine the cause and consequence of an event. 

The main reason for this type of case study is to investigate one or more sources with unique interests and sometimes with no interest in general. 

Take a look at this example below:

Critical Instance Case Study Example

When you have a hypothesis, you can design a descriptive study. It aims to find connections between the subject being studied and a theory.

After making these connections, the study can be concluded. The results of the descriptive case study will usually suggest how to develop a theory further.

This example can help you understand the concept better:

Descriptive Case Study Example

Intrinsic studies are more commonly used in psychology, healthcare, or social work. So, if you were looking for types of case studies in sociology, or types of case studies in social research, this is it.

The focus of intrinsic studies is on the individual. The aim of such studies is not only to understand the subject better but also their history and how they interact with their environment.

Here is an example to help you understand;

Intrinsic Case Study Example

This type of case study is mostly used in qualitative research. In an instrumental case study, the specific case is selected to provide information about the research question.

It offers a lens through which researchers can explore complex concepts, theories, or generalizations.

Take a look at the example below to have a better understanding of the concepts:

Instrumental Case Study Example

Review some case study examples to help you understand how a specific case study is conducted.

Types of Subjects of Case Study 

In general, there are 5 types of subjects that case studies address. Every case study fits into the following subject categories. 

• Person: This type of study focuses on one subject or individual and can use several research methods to determine the outcome. 
• Group: This type of study takes into account a group of individuals. This could be a group of friends, coworkers, or family. 
• Location: The main focus of this type of study is the place. It also takes into account how and why people use the place. 
• Organization: This study focuses on an organization or company. This could also include the company employees or people who work in an event at the organization. 
• Event: This type of study focuses on a specific event. It could be societal or cultural and examines how it affects the surroundings. 

Benefits of Case Study for Students

Here's a closer look at the multitude of benefits students can have with case studies:

Real-world Application

Case studies serve as a crucial link between theory and practice. By immersing themselves in real-world scenarios, students can apply theoretical knowledge to practical situations.

Critical Thinking Skills

Analyzing case studies demands critical thinking and informed decision-making. Students cultivate the ability to evaluate information, identify key factors, and develop well-reasoned solutions – essential skills in both academic and professional contexts.

Enhanced Problem-solving Abilities

Case studies often present complex problems that require creative and strategic solutions. Engaging with these challenges refines students' problem-solving skills, encouraging them to think innovatively and develop effective approaches.

Holistic Understanding

Going beyond theoretical concepts, case studies provide a holistic view of a subject. Students gain insights into the multifaceted aspects of a situation, helping them connect the dots and understand the broader context.

Exposure to Diverse Perspectives

Case studies often encompass a variety of industries, cultures, and situations. This exposure broadens students' perspectives, fostering a more comprehensive understanding of the world and the challenges faced by different entities.

So there you have it!

We have explored different types of case studies and their examples. Case studies act as the tools to understand and deal with the many challenges and opportunities around us.

Case studies are being used more and more in colleges and universities to help students understand how a hypothetical event can influence a person, group, or organization in real life. 

Not everyone can handle the case study writing assignment easily. It is even scary to think that your time and work could be wasted if you don't do the case study paper right. 

Our essay writing service online  is here to make your academic journey easier. 

Let us worry about your essay and buy case study today to ease your stress and achieve academic success.

Write Essay Within 60 Seconds!

Dr. Barbara is a highly experienced writer and author who holds a Ph.D. degree in public health from an Ivy League school. She has worked in the medical field for many years, conducting extensive research on various health topics. Her writing has been featured in several top-tier publications.

Paper Due? Why Suffer? That’s our Job!

Keep reading

Case Study Research Method in Psychology

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Learn about our Editorial Process

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

Case studies are in-depth investigations of a person, group, event, or community. Typically, data is gathered from various sources using several methods (e.g., observations & interviews).

The case study research method originated in clinical medicine (the case history, i.e., the patient’s personal history). In psychology, case studies are often confined to the study of a particular individual.

The information is mainly biographical and relates to events in the individual’s past (i.e., retrospective), as well as to significant events that are currently occurring in his or her everyday life.

The case study is not a research method, but researchers select methods of data collection and analysis that will generate material suitable for case studies.

Freud (1909a, 1909b) conducted very detailed investigations into the private lives of his patients in an attempt to both understand and help them overcome their illnesses.

This makes it clear that the case study is a method that should only be used by a psychologist, therapist, or psychiatrist, i.e., someone with a professional qualification.

There is an ethical issue of competence. Only someone qualified to diagnose and treat a person can conduct a formal case study relating to atypical (i.e., abnormal) behavior or atypical development.

 Famous Case Studies

• Anna O – One of the most famous case studies, documenting psychoanalyst Josef Breuer’s treatment of “Anna O” (real name Bertha Pappenheim) for hysteria in the late 1800s using early psychoanalytic theory.
• Little Hans – A child psychoanalysis case study published by Sigmund Freud in 1909 analyzing his five-year-old patient Herbert Graf’s house phobia as related to the Oedipus complex.
• Bruce/Brenda – Gender identity case of the boy (Bruce) whose botched circumcision led psychologist John Money to advise gender reassignment and raise him as a girl (Brenda) in the 1960s.
• Genie Wiley – Linguistics/psychological development case of the victim of extreme isolation abuse who was studied in 1970s California for effects of early language deprivation on acquiring speech later in life.
• Phineas Gage – One of the most famous neuropsychology case studies analyzes personality changes in railroad worker Phineas Gage after an 1848 brain injury involving a tamping iron piercing his skull.

Clinical Case Studies

• Studying the effectiveness of psychotherapy approaches with an individual patient
• Assessing and treating mental illnesses like depression, anxiety disorders, PTSD
• Neuropsychological cases investigating brain injuries or disorders

Child Psychology Case Studies

• Studying psychological development from birth through adolescence
• Cases of learning disabilities, autism spectrum disorders, ADHD
• Effects of trauma, abuse, deprivation on development

Types of Case Studies

• Explanatory case studies : Used to explore causation in order to find underlying principles. Helpful for doing qualitative analysis to explain presumed causal links.
• Exploratory case studies : Used to explore situations where an intervention being evaluated has no clear set of outcomes. It helps define questions and hypotheses for future research.
• Descriptive case studies : Describe an intervention or phenomenon and the real-life context in which it occurred. It is helpful for illustrating certain topics within an evaluation.
• Multiple-case studies : Used to explore differences between cases and replicate findings across cases. Helpful for comparing and contrasting specific cases.
• Intrinsic : Used to gain a better understanding of a particular case. Helpful for capturing the complexity of a single case.
• Collective : Used to explore a general phenomenon using multiple case studies. Helpful for jointly studying a group of cases in order to inquire into the phenomenon.

Where Do You Find Data for a Case Study?

There are several places to find data for a case study. The key is to gather data from multiple sources to get a complete picture of the case and corroborate facts or findings through triangulation of evidence. Most of this information is likely qualitative (i.e., verbal description rather than measurement), but the psychologist might also collect numerical data.

1. Primary sources

• Interviews – Interviewing key people related to the case to get their perspectives and insights. The interview is an extremely effective procedure for obtaining information about an individual, and it may be used to collect comments from the person’s friends, parents, employer, workmates, and others who have a good knowledge of the person, as well as to obtain facts from the person him or herself.
• Observations – Observing behaviors, interactions, processes, etc., related to the case as they unfold in real-time.
• Documents & Records – Reviewing private documents, diaries, public records, correspondence, meeting minutes, etc., relevant to the case.

2. Secondary sources

• News/Media – News coverage of events related to the case study.
• Academic articles – Journal articles, dissertations etc. that discuss the case.
• Government reports – Official data and records related to the case context.
• Books/films – Books, documentaries or films discussing the case.

3. Archival records

Searching historical archives, museum collections and databases to find relevant documents, visual/audio records related to the case history and context.

Public archives like newspapers, organizational records, photographic collections could all include potentially relevant pieces of information to shed light on attitudes, cultural perspectives, common practices and historical contexts related to psychology.

4. Organizational records

Organizational records offer the advantage of often having large datasets collected over time that can reveal or confirm psychological insights.

Of course, privacy and ethical concerns regarding confidential data must be navigated carefully.

However, with proper protocols, organizational records can provide invaluable context and empirical depth to qualitative case studies exploring the intersection of psychology and organizations.

• Organizational/industrial psychology research : Organizational records like employee surveys, turnover/retention data, policies, incident reports etc. may provide insight into topics like job satisfaction, workplace culture and dynamics, leadership issues, employee behaviors etc.
• Clinical psychology : Therapists/hospitals may grant access to anonymized medical records to study aspects like assessments, diagnoses, treatment plans etc. This could shed light on clinical practices.
• School psychology : Studies could utilize anonymized student records like test scores, grades, disciplinary issues, and counseling referrals to study child development, learning barriers, effectiveness of support programs, and more.

How do I Write a Case Study in Psychology?

Follow specified case study guidelines provided by a journal or your psychology tutor. General components of clinical case studies include: background, symptoms, assessments, diagnosis, treatment, and outcomes. Interpreting the information means the researcher decides what to include or leave out. A good case study should always clarify which information is the factual description and which is an inference or the researcher’s opinion.

1. Introduction

• Provide background on the case context and why it is of interest, presenting background information like demographics, relevant history, and presenting problem.
• Compare briefly to similar published cases if applicable. Clearly state the focus/importance of the case.

2. Case Presentation

• Describe the presenting problem in detail, including symptoms, duration,and impact on daily life.
• Include client demographics like age and gender, information about social relationships, and mental health history.
• Describe all physical, emotional, and/or sensory symptoms reported by the client.
• Use patient quotes to describe the initial complaint verbatim. Follow with full-sentence summaries of relevant history details gathered, including key components that led to a working diagnosis.
• Summarize clinical exam results, namely orthopedic/neurological tests, imaging, lab tests, etc. Note actual results rather than subjective conclusions. Provide images if clearly reproducible/anonymized.
• Clearly state the working diagnosis or clinical impression before transitioning to management.

3. Management and Outcome

• Indicate the total duration of care and number of treatments given over what timeframe. Use specific names/descriptions for any therapies/interventions applied.
• Present the results of the intervention,including any quantitative or qualitative data collected.
• For outcomes, utilize visual analog scales for pain, medication usage logs, etc., if possible. Include patient self-reports of improvement/worsening of symptoms. Note the reason for discharge/end of care.

4. Discussion

• Analyze the case, exploring contributing factors, limitations of the study, and connections to existing research.
• Analyze the effectiveness of the intervention,considering factors like participant adherence, limitations of the study, and potential alternative explanations for the results.
• Identify any questions raised in the case analysis and relate insights to established theories and current research if applicable. Avoid definitive claims about physiological explanations.
• Offer clinical implications, and suggest future research directions.

5. Additional Items

• Thank specific assistants for writing support only. No patient acknowledgments.
• References should directly support any key claims or quotes included.
• Use tables/figures/images only if substantially informative. Include permissions and legends/explanatory notes.
• Provides detailed (rich qualitative) information.
• Provides insight for further research.
• Permitting investigation of otherwise impractical (or unethical) situations.

Case studies allow a researcher to investigate a topic in far more detail than might be possible if they were trying to deal with a large number of research participants (nomothetic approach) with the aim of ‘averaging’.

Because of their in-depth, multi-sided approach, case studies often shed light on aspects of human thinking and behavior that would be unethical or impractical to study in other ways.

Research that only looks into the measurable aspects of human behavior is not likely to give us insights into the subjective dimension of experience, which is important to psychoanalytic and humanistic psychologists.

Case studies are often used in exploratory research. They can help us generate new ideas (that might be tested by other methods). They are an important way of illustrating theories and can help show how different aspects of a person’s life are related to each other.

The method is, therefore, important for psychologists who adopt a holistic point of view (i.e., humanistic psychologists ).

Limitations

• Lacking scientific rigor and providing little basis for generalization of results to the wider population.
• Researchers’ own subjective feelings may influence the case study (researcher bias).
• Difficult to replicate.
• Time-consuming and expensive.
• The volume of data, together with the time restrictions in place, impacted the depth of analysis that was possible within the available resources.

Because a case study deals with only one person/event/group, we can never be sure if the case study investigated is representative of the wider body of “similar” instances. This means the conclusions drawn from a particular case may not be transferable to other settings.

Because case studies are based on the analysis of qualitative (i.e., descriptive) data , a lot depends on the psychologist’s interpretation of the information she has acquired.

This means that there is a lot of scope for Anna O , and it could be that the subjective opinions of the psychologist intrude in the assessment of what the data means.

For example, Freud has been criticized for producing case studies in which the information was sometimes distorted to fit particular behavioral theories (e.g., Little Hans ).

This is also true of Money’s interpretation of the Bruce/Brenda case study (Diamond, 1997) when he ignored evidence that went against his theory.

Breuer, J., & Freud, S. (1895).  Studies on hysteria . Standard Edition 2: London.

Curtiss, S. (1981). Genie: The case of a modern wild child .

Diamond, M., & Sigmundson, K. (1997). Sex Reassignment at Birth: Long-term Review and Clinical Implications. Archives of Pediatrics & Adolescent Medicine , 151(3), 298-304

Freud, S. (1909a). Analysis of a phobia of a five year old boy. In The Pelican Freud Library (1977), Vol 8, Case Histories 1, pages 169-306

Freud, S. (1909b). Bemerkungen über einen Fall von Zwangsneurose (Der “Rattenmann”). Jb. psychoanal. psychopathol. Forsch ., I, p. 357-421; GW, VII, p. 379-463; Notes upon a case of obsessional neurosis, SE , 10: 151-318.

Harlow J. M. (1848). Passage of an iron rod through the head.  Boston Medical and Surgical Journal, 39 , 389–393.

Harlow, J. M. (1868).  Recovery from the Passage of an Iron Bar through the Head .  Publications of the Massachusetts Medical Society. 2  (3), 327-347.

Money, J., & Ehrhardt, A. A. (1972).  Man & Woman, Boy & Girl : The Differentiation and Dimorphism of Gender Identity from Conception to Maturity. Baltimore, Maryland: Johns Hopkins University Press.

Money, J., & Tucker, P. (1975). Sexual signatures: On being a man or a woman.

Further Information

• Case Study Approach
• Case Study Method
• Enhancing the Quality of Case Studies in Health Services Research
• “We do things together” A case study of “couplehood” in dementia
• Using mixed methods for evaluating an integrative approach to cancer care: a case study

• Open access
• Published: 13 May 2024

Bridging the gap between research evidence and its implementation in public health practice: case studies of embedded research model

• Abisope Akintola 1 , 3 ,
• Dorothy Newbury-Birch 2 &
• Stephanie Kilinc 2  

BMC Public Health volume  24 , Article number:  1299 ( 2024 ) Cite this article

Metrics details

To investigate the potential of embedded research in bridging the gap between research evidence and its implementation in public health practice.

Using a case study methodology, semi-structured interviews were conducted with 4 embedded researchers, 9 public health practitioners, and 4 other stakeholders (2 teachers and 2 students) across four case study sites. Sites and individuals were purposively selected. Sites included two local authorities, one secondary school, and one sports organisation. Thematic data analysis was adopted to analyse the qualitative data.

Four themes were identified: (1) building and maintaining relationships, (2) working with stakeholders, (3) informing practice, and (4) critical reflection.

Conclusions

Embedded researchers build and maintain relationships with practitioners and other stakeholders to produce research. Evidence from the co-produced research informs future practice and research to improve service and delivery rendered to the public. Thus, embedded researchers use their role to bridge the research evidence - implementation gap in public health practice.

Peer Review reports

Implementation science is widely recognised as a study of methods to adopt and utilise evidence-based interventions in specific locations or settings to improve the health of the population [ 1 ]. However, the gap between research evidence and its implementation in public health practice is still globally recognised [ 2 ]. According to scholars, some of the factors associated with the problem of inadequate implementation of research evidence in practice could either originate from the researchers or the practitioners [ 3 , 4 , 5 ]. This implies that both researchers and practitioners could be responsible for the creation of the gap between research evidence and its implementation in public health practice.

Evidence suggests that lack of access to research evidence is one of the barriers to the implementation of research evidence in practice [ 6 , 7 , 8 ]. One report suggests that increased connectivity between researchers and practitioners would enhance the practitioners’ accessibility to research evidence [ 9 ]. The report explained further that creating some forums where practitioners and researchers could interact would not only bring about easy access to relevant research evidence, but also would serve as a means to share learning, and link researchers and practitioners who have a common interest. Similarly, other scholars report that increasing the interaction between researchers and practitioners among other factors could facilitate the use of research-based evidence in practice [ 10 , 11 ]. To that end, there is a need to increase the opportunities for practitioners and researchers to interact in order to facilitate the utilisation of research evidence in public health practice.

As there are many identified barriers to the use of research evidence in practice, the disparity between the context and the language by which researchers and practitioners operate has also been identified as one of the barriers. The incompatibility in the language spoken by the researchers with respect to the scientific methods and the evidence generated could be ambiguous for practitioners [ 12 ]. Therefore, to overcome this challenge, scholars advise that practitioners and researchers should work collaboratively from the onset of the research while putting into consideration each other’s differences [ 13 , 14 ]. Furthermore, it has been recommended that researchers need to present their research findings and explain the relevance to solving practical problems to the practitioners in a simple language without ambiguity [ 15 ]. This suggests a need for an approach that would involve practitioners and researchers undertaking the research agenda together, and also a need for effectively communicating research findings and their relevance in a simple language to the practitioners.

The context in which the researchers operate could also serve as a challenge to the utilisation of research evidence in practice [ 9 ]. As such, competing pressures such as teaching commitments and publishing academic papers [ 16 ] could pose a challenge to the researchers’ involvement in practical problems that could inform their research questions. Hence, there is a need for an approach for researchers to be more involved in practical problems to facilitate the conduction of research that is relevant and applicable to problem solving. It was noted that not all researchers have the relevant skills to conduct co-produced research [ 17 ]. There is a need to create opportunities for researchers who have relevant skills to co-produce research, to conduct research with suitable practitioners.

On the other hand, organisational factors such as time constraints are contributing factors to the gap between research evidence and practice as most practitioners do not have the skills nor the time needed to implement research outcomes in practice [ 18 ]. To tackle these challenges, some studies recommend continuous training and commitment to quality health delivery on the part of practitioners. They also recommended advancements in technological decision support systems as instruments to combat barriers between research evidence and practice [ 19 , 20 ]. There is an argument that achieving these may be difficult as a result of inadequate funds in health services [ 21 ]. Hence, there is a need for the adoption of a method that will bring about building the capacity of the practitioners towards conducting research that is achievable based on the available budget.

Furthermore, the disparity of influence and power between academics and practitioners could be responsible for the wide gap between research and practice [ 22 ]. This means the relationship between academic researchers and practitioners plays a vital role in the use of research evidence. Therefore, there is a need for a method that would enhance or build mutually beneficial relationships between academic researchers and practitioners to bridge the ‘research evidence-implementation’ gap.

The separation of the development of research evidence from the places it is to be used contributes to the challenges of using research evidence in practice [ 23 ]. This implies that the creation of research knowledge where it is to be utilised could bridge the ‘research evidence-implementation’ gap. As such, co-production has been recommended by scholars to bridge the ‘research evidence-implementation’ gap as co-production involves the collaborative working between the researchers and the practitioners [ 24 ]. Hence, the adoption of co-production to produce public health knowledge by researchers, practitioners, and other stakeholders in non-clinical settings [ 13 , 25 ]. This is essential in tackling the challenges of inadequate implementation of research evidence in public health settings.

Being involved in co-production could result in reputational risk for the researcher involved as the researcher could be used by politicians to enhance authenticity to their political stand [ 26 ]. Thus, being viewed to approve such a political stand can limit the researcher’s ability to work only with a certain political group – this can also impact the researcher’s personal safety [ 27 ]. Also, this can impact negatively on the credibility of the co-production findings as it might be viewed as biased and not a true representation but a narrative to back up a political viewpoint, thus generating “policy-based evidence” [ 28 ] rather than “research-based evidence”. On the other hand, policy-makers might be at risk of sharing sensitive information while participating in co-production work [ 29 ] such as disclosing political errors.

Also, co-production can be costly as it usually involves the stakeholders travelling to the co-production site. This could be viewed as challenging for those that are involved in the co-production project, as their presence at meetings for the co-production work is seen as crucial. Also, funding and sustainability of co-production can pose a great risk to the adoption of co-production [ 48 ]. However, the challenges associated with co-production can be overcome if stakeholders are involved and are carried along at every stage of co-production, from design to implementation [ 30 ]. The success of co-production depends on but is not limited to the following: the individuals involved; how clear the aims and objectives of the project are to all those involved, and how duties are allocated [ 31 ]. This also suggests a need to critically analyse the role of stakeholders involved in co-production to overcome the challenges associated with co-production, to achieve success.

Embedded research, also known as ‘researcher-in-residence’, is becoming popular as a type of co-production research [ 3 ]. Different authors used different terminologies for embedded researchers such as insider researcher [ 32 ], knowledge broker [ 33 , 34 ], or scholar-practitioner [ 35 ]. Within an embedded research model, one of the distinguishing features is that the researcher is located in the host organisation as a member of staff to carry out a research agenda with the host organisation’s staff, and at the same time maintaining affiliation with an academic institution [ 36 , 37 , 38 , 39 ]. In this paper we investigate how an embedded research model can help bridge the gap between research evidence and its implementation in public health practice.

We conducted qualitative case studies and drew data from semi-structured interviews with four embedded researchers, nine public health practitioners, and four other stakeholders (two teachers and two students) across four case study sites including two local authorities (Sites one and two), one secondary school (Site three), and one sports organisation (Site four) in the Northeast of England.

One of the advantages of qualitative research is the ability to generate rich in-depth data or knowledge that can serve as a basis for health and social practices being effective and relevant to the contexts they are applied to [ 40 ]. We adopted a qualitative multi-site case study to understand the context by providing in-depth description and analysis within sites and as well by comparing data between sites in order to identify the similarities and differences between the sites explored [ 41 ]. Thus, this will assist to maximise the applicability of the findings on how an embedded research model can help bridge the gap between research evidence and its implementation in other similar settings.

In site one, the embedded research project aimed to understand and make recommendations regarding population changes, and service needs, including health, education, housing, and social care, in the local communities. In site two, an embedded researcher works at the local authority to provide research support to the local authority’s public health team to secure their targets which include commissioning evidence-based services and interventions, and promotion of healthy lifestyles. Site three conducted an embedded research project to explore the academic and health impact of the recent changes to the General Certificate of Secondary Education (GCSE) system on both staff and students. Site four was established to encourage more people to engage in physical activities to improve their health and well-being. In order to improve the service rendered to the public, an embedded researcher was employed in site four to co-produce research with the sports organisation members of staff. All the embedded researchers across the four case study sites were PhD holders. The amount of time spent in their respective host organisations varied from one hour per fortnight to two and a half days a week to suit the embedded researchers and the host organisations. The embedded researchers’ positions were funded either by the University they are affiliated with, or their host organisation.

Purposive snowball sampling was used in this study. Requests for participants and sites who could volunteer to be part of the study were sent out via relevant professional contacts and networks. The participants and sites that volunteered to take part in this research were asked to assist in the search for participants and/or sites by circulating the study’s details to those who might meet the study’s criteria and would be willing to take part in the study. The inclusion criteria were: (1) being a public health embedded researcher, and (2) being a public health practitioner or stakeholder who is working or has worked with a public health embedded researcher. Potential participants were assessed for eligibility before being interviewed. A total of 17 participants were recruited for the interviews across the four case study sites. The sample size would have been larger than 17 but for the Covid-19 pandemic. Ethical approval was obtained from the Teesside University School of Health and Life Sciences Research Governance and Ethics Committee in November 2019. Data was collected between November 2019 and April 2020.

To facilitate participation, participants were offered alternative modes of interview for their convenience: face-to-face, telephone, and Skype-based interviews. The Covid-19 pandemic occurred during the interview period, but most interviews conducted before COVID-19 were face-to-face. All interviews conducted during the pandemic (March 2020 and onwards) were either Skype or telephone-based, as advised by the Ethics department at Teesside University and as per the requirements of the interviewees’ workplaces. Before each interview, oral and written informed consent was obtained from each participant. Each participant was asked to complete two copies of the consent form, one for their own records and one for the researcher.

Following each interview, a reflective note was taken to identify what went well and what could be done differently in the next interview. Since there were three categories of interview participants – embedded researchers (ERs), public health practitioners (PHPs), and other stakeholders (students (STs) and teachers (TRs)–three sets of interviews were prepared. Although the interview questions were nearly the same for each category of participants, some of the interview questions differed in the way they were structured. Here is an example of how a question was worded differently depending on the participant: (ERs) Can you cite an example where you have built practitioners and other stakeholders’ confidence to conduct their own research? (PHPs, TRs, and STs) Can you cite an example where an embedded researcher has built your confidence to conduct your own research? A full outline of the interview guide is in Appendix .

A summary of each interview was noted in a research diary for reference. Details noted included where each interview took place, the date of the interview, the length of the interview and how the interviewee responded to questions. Each interview lasted between 40 and 90 min. The interviews were recorded, and data was transcribed. We analysed data using inductive thematic analysis [ 42 ] to allow new themes besides the preconceived ones to emerge from the coding of the interviews. Trustworthiness of the analysis was assessed by triangulating between data sources.

Four themes emerged from the analysis of the interview data on the potential of embedded research in bridging the gap between research evidence and its implementation in public health practice: (1) building and maintaining relationships (2) working with stakeholders, (3) informing practice, and (4) critical reflection.

Building and maintaining relationships

All participants across the four case study sites, irrespective of their age, years of experience, or education, recounted the significance of this theme to the embedded research projects in their respective sites. They articulated the benefits of the role of the embedded researchers in building and maintaining relationships with the public health practitioners and other stakeholders to facilitate the co-production of research evidence. They all agreed that building and maintaining relationships played a vital role in the utilisation of the co-produced research evidence and in the closing of the gap between research evidence and its implementation. Overall, the strategies adopted by the embedded researchers to achieve this theme were identified as: (1) building internal/external relationships and sharing skills, and 2) maintaining regular contact with practitioners and other stakeholders.

Building internal/external relationships and sharing skills

Participants agreed that the embedded researchers’ role entails having diverse connections built on good relationships. These relationships assist the embedded researchers in connecting their partners to other relevant organisations such as academic institutions and third sector agencies.

“I think some of that is around having this kind of good grounding so sort of beginning the role with already having made, a lot of kind of contacts, a lot of sort of good relationships been built. [..] I have a line manager in the council, who was the project manager for the first phase so we’ve got that continuity there [..] I also have an academic supervisor who is also my kind of my line manager from the academic side” [ERsite1] .

“I can say that’s [having connections] actually key because they are straddling both worlds. [..] not somebody who sat in the academic institution who didn’t understand the wider context. I think these roles are really key in bridging the institutions” [PHP2site1] .

It was clear that building relationships and connecting the ‘two worlds’ is not only advantageous to both institutions but also assisted the embedded researchers to seek support from both their academic supervisor at the University they were employed and the local authority (LA) they are working with. Therefore, this enables the embedded researchers to be supported fully to carry out their role successfully. It was also recognised that while embedded researchers play their role in building relationships and connecting relevant organisations, the role assisted them to understand the context in which research evidence is to be utilised. Thus, the relevance of research evidence to the host organisation facilitates its use.

This relationship-building was seen as crucial to the success of the role, and it was felt that these relationships could determine the success of any work carried out.

“[..] I would go as far to say I think it’s the relationship that’s built with the individuals who developed that project was important. [..] are the most important elements of co-production” [ERsite2] .

This implies that lack of relationship-building between researchers and public health practitioners can serve as a barrier to embedded research project. Furthermore, it was evident that the relationship built with the stakeholders who were involved in the embedded research was crucial to the projects. For instance, an embedded researcher from site two used her skills to build relationships with the volunteers that participated in the project.

“She [embedded researcher] has been there longer, excellent relationships with the volunteers, that helped to build and shape this project, so she has a very useful experience in terms of relationship-building” [PHP6site2] .

Thus, this assisted in structuring the work which had a positive impact on the project. This two-way relationship with other organisations, including the local universities and research participants, was seen as a benefit of embedded research.

Findings showed that embedded researchers used their contacts and good relationships to facilitate the sharing of skills useful in carrying out embedded research projects and also enable working with other academics at the University.

“[..] even for me just working as an individual in that organisation, I don’t know everything about the research, but because you are linked with the University, that gives an avenue to ask questions and link up with people with expertise to then support an evaluation” [ERsite2] .

These connections and relationships, therefore, enable the sharing of skills useful to co-produce relevant high-quality research evidence useful to host organisations and policy makers.

Within this current study, it was clear that if the embedded researchers were not located or had spent time in the sites, they felt it would be difficult for them to build relationships, and understand the context in which the co-produced research is to be utilised.

“So, having the researcher embedded within in what we do, the researcher has the understanding of the project, and initially she has been with it from the start to finish, so she understands the journey that’s been on, and she understands why it’s been done, how it’s been done [..] So, I think, so the embedded researcher role in what we do is infallible resource really” [ PHP1site4 ] .

The ‘embeddedness’ gave the researchers an understanding of the projects they were involved in. As such, the embedded researchers were seen as ‘insiders’ and their ‘embeddedness’ was seen as key to the success of the work.

It is worth noting that the amount of time spent by the embedded researchers in their respective host organisation varied and was negotiated at the sites to suit the embedded researchers and the host organisations.

“[..] I was familiar with quite a lot of people but obviously kind of being there regularly I have got to know them much better basically. [..] I mean it really varies; I would say probably kind of at least a couple of days in a week” [ERsite1] .

“Being embedded within their team I spend half of the week working within the organisation. It’s been a real pleasure to work alongside them” [ERsite2] .

“ So, we tend to have meetings where I will go in for a few hours at a time. I would probably say, maybe an hour in a fortnight ” [ERsite3] .

“[..] I spend two and a half days working within the organisation. [..] you want to be seen as part of that team and not somebody who just pops up every now and again” [ERsite4] .

However, building relationships and sharing skills was not seen as without its challenges with some tension between roles and expectations.

“[..] it has become trickier splitting myself now between the organisations as they all have their roles and expectations on how they want things to be done” [ ERsite2] .

“The structure can be quite challenging as well, but probably [..] just having that balance in the relationships with the organisation you are working for and the organisation you are evaluating for. And I think yeah you have got to have that one, but that is a challenge of working in large organisation” [PHP6site2] .

The embedded researchers from sites one and two found there was some tension in working in both ‘worlds’ as a result of the responsibilities associated with it, such as building relationships, and balancing diverse responsibilities. This is due to their dual affiliation as such, they are expected to manage a large workload, managing both successfully. A practitioner from site two added that the structure of the organisations the embedded researcher works could also be a challenge, therefore, it is important for an embedded researcher to be able to discuss this with both sides in order that they balance the relationships between the host organisation and the academic institution.

Another notable challenge is having to manage diverse expectations including the ability to balance competing interests of the different organisations.

“There is sort of difference in expectations because I think from the academic point of view, [..] we want publications, we want things that give us an academic output, whereas someone who works in the school is not going to be bothered about that sort of things. They have to see where it positively affects their school, [..] so I think having that difference in agendas on what you want to achieve from this school research can be quite hard to manage. [..] you want different things from this piece of research is quite hard, and make sure that both sides are happy at the end of the day, and I think we did that quite well” [ERsite3] .

For instance, an embedded researcher from the school stated that the expectations from the embedded research project did differ. That is, while part of the aim of the academic input was to publish the outcome of the project to improve or boost their academic output, the school aimed for a practical positive impact of the project on the school, such as improvement in students’ engagement in academic activities. Hence, it was essential to balance the competing interests of the school and the academic side of the embedded research project.

Maintaining regular contact with practitioners and other stakeholders

Based on the participants’ experiences, the embedded researchers built relationships with the practitioners and other stakeholders by maintaining regular contact.

“I think what we did was to help build that relationship. It was not just a telephone conversation just to discuss. We actually worked side by side so there was time to actually do that embedded research. We spent time in the office, we spent like one or two days a week” [PHP1site2] .

“Yeah, but then we did send them emails and stuff, in between [..] yeah we did have time outside of the face to face sessions and sending stuff to the teachers to encourage them, ‘can you remind the students that we have got to do this week’, we have got to get this done by then, so I would say obviously we had the face to face sessions but then we had email correspondence as well” [ERsite3] .

The practitioners from site two reported that the embedded researcher maintained regular contact by face to face, or by telephone. They further explained that they worked side by side with the embedded researcher to build relationships. This implies that if the practitioners and the embedded researcher were not chanced to work together, which assisted in maintaining regular contact, it would have been difficult to build relationships. Thus, this widens the gap between academia and practice. The embedded researchers had similar experiences. For instance, an embedded researcher from site three (school) confirmed that she maintained regular contact to build relationships with the students and the teachers by email and face to face. This shows that it is important to develop project strategies in order to maintain regular contact with the practitioners and other stakeholders to build relationships.

According to the embedded researchers, building mutually beneficial relationships was achieved by maintaining regular contact not only with the stakeholders but also with their academic supervisors which enabled the embedded researchers to have the necessary support to achieve their role.

“I mean knowing that I do have kind of the support at the University to draw on and also have a kind of a good working relationship with my line manager in the council as well really. I don’t feel that I am lacking in any kind of support, which is a good kind of place to be in yeah. So I have monthly meetings in the University and that’s very much really useful in times of keeping track of some of the other parts of my roles so around kind of trying to ensure that we can get some like academic publications and things like that so yeah” [ERsite1] .

Another strategy that was mentioned regarding how the embedded researchers maintained regular contact to build relationships with the practitioners and other stakeholders was ‘attending formal meetings’.

“Interestingly, the researcher has always been on the co-production committee and she attends the meetings, so she is excellent, much better than me because she has been there longer, [..] that helped to build and shape this project [..]” [PHP6site2] .

“So, I have to go to all their team meetings that’s gonna help you form a lot of relationships. Meetings are where the real connection starts to happen. So, you have to invest that time ” [ERsite4] .

As well as making use of formal meeting, the embedded researchers adopted ‘informal conversations’ to maintain regular contact to build relationships with the public health practitioners and other stakeholders.

“For me, I am quite like a chatty person and I think that’s like the characteristics of an embedded researcher. You need somebody who is easy to get on with lots of different people. You need to have that ability to do that. Otherwise, you gonna struggle to form a relationship especially if you aren’t there as often as what you would be if it’s a full-time job” [ ERsite4] .

A practitioner from the sports organisation added that engaging in informal conversations also helped in building a trustworthy relationship with the embedded researcher.

“[..] We have that relationship and some other things you can visit, particularly when things get tough, it’s easy enough to fall back on different conversations on sport [..] These conversations increase our relationship and trust, we trust each other” [PHP1site4] .

The practitioner further explained that he has a good relationship with the embedded researcher and so they engage in informal conversations at difficult times thereby developing a relationship that is based on trust.

Working with stakeholders

Results showed that the embedded researchers build and maintain relationships with the practitioners, and with other stakeholders in order to effectively work together to produce research. This, therefore, facilitated the production and the use of the co-produced research evidence at the embedded sites and helped close the gap between research evidence and its implementation as results were shared quickly with all those that were involved. All participants across the four case study sites unanimously agreed that this theme is one of the primary roles of an embedded researcher, and the strategies identified include: (1) co-producing research, and (2) building research capacity.

Co-producing research

The participants confirmed that they worked together to identify, plan, and conduct research intended to help the host organisations improve their services and meet the needs of the communities with which they work.

“We liaise with the researcher to develop the initial kind of overview of that population [..] the researcher supports us in developing the initial questions, the questionnaire, and the initial research” [PHP1site4] .

“[..] embedding research into the public health team. [..] then helping us to explore the questionnaires. The embedded researcher helps us with the development of that work including the formulae and evaluation for the intervention. We design and develop and embed and undertake the research together. She is very much a part of the team and a core within the team” [PHP4site2] .

“[..] So, really it’s about giving us the exposure to that sort of research. Well, honestly, I have learnt how to conduct research” [ST1site3] .

The participants acknowledged that working together to co-produce research with the embedded researchers encouraged adjustments to and engagement with research-related activities. Furthermore, embedded research was considered a cost-effective research approach.

“ I have been out in a couple of beneficiary interviews with the researcher. Certainly, I would not normally get involved with going out to see clients, but I have gone out a couple of times with the researcher, so that was interesting” [PHP5site2] .

“[..] the embedded researcher worked alongside the public health practitioners [..] how to shape some of the evaluations, including how to be really clear about the methodology, the approach [..] And how to write protocol [..] So, I think that was the aim of it, it was to ensure that we have much more effective and cost-effective research ” [PHP2site1] .

One public health practitioner reported that she participated in several research activities with the embedded researcher at site two. She recognised that working with the researcher enabled her to do research work that she would not have ordinarily done. This suggests that not working together with practitioners to co-produce research may potentially prevent practitioners from being meaningfully involved in the research process. In such situations, the gap between the development and implementation of research evidence may actually become wider. One practitioner from site one explained that embedded research was adopted in the LA so that the authority could conduct cost-effective research. This only further indicates that having an embedded researcher on-site working collaboratively with practitioners and stakeholders to conduct cost-effective research can help bridge the research implementation gap.

However, it was noted that the process of co-producing research between the embedded researchers and the public health practitioners and other stakeholders also facilitated shared learning.

“Despite the fact that we went in obviously thinking of teaching them but the fact that we can learn from them about what was important to them, what was important to young pupils in schools, and how to speak to young pupils because that is schooling in itself. [..] and I think also you learn new skills [..] so I think you get sort of practical experience and learn new skills sort of more practical skills I suppose, not just research skills, so yeah that is why I think I say it’s the most important thing” [ERsite3] .

“[..] and when I have been out with staff members, they will ask questions that I would never have thought of asking, because of their knowledge at work. [..] I have been learning a lot as well from the staff, and that shows the importance of doing it together” [ ERsite2] .

One embedded researcher from site three (school) reported that although their aim was to teach the students how to conduct research, they were able to learn what was important to the young people among other things from the students. Another embedded researcher from site two shared a similar experience and confirmed that during the co-production work, the public health practitioners used their tacit knowledge of their field to ask relevant questions that had not occurred to her. Since the practitioners are more knowledgeable than the researcher regarding actual on-site practices, they added substantial value to the project. This indicates just how much learning is a two-way process, and demonstrates co-production of knowledge which involves the amalgamation of the practitioners’ tacit knowledge and the researchers’ explicit knowledge.

Researchers were explicitly recognised for their ability to co-produce research with the public health practitioners and other stakeholders. Thus, the co-produced research was jointly owned by those involved in the embedded research projects. As the research was co-produced with the intention to assist the organisations to improve the service they render to the public, thus, the embedded researchers’ role assisted in facilitating the utilisation of research evidence. In addition, given the embedded research projects focused on meeting the needs of the host organisations, there were no instances where there were conflicts related to the research emerged.

Building research capacity

The embedded researchers explained that they conducted training, and other developmental activities to help develop the practitioners’ and other stakeholders’ research skill-set.

“I have done a kind of number of training sessions with staff and actually with volunteers that will want to get involved in collecting data [..] so I have run workshops, training workshop, so that means that when I go out there for collection the staff can come and do it with me” [ERsite4] .

“[..] another element of my role is to deliver training to staff around the use of data around the benefits of collecting relevant information, how that information can be used to inform practice in decisions and planning and things like that, we just had a conference couple of weeks ago which was very much about kind of sharing the learning and then sort of getting people involved in the work that we do really, so they are my kind of key targets really” [ ERsite1] .

Research-based training were offered by the embedded researchers in a variety of forms, such as using workshop training, one to one training and through seminars and conferences. For instance, an embedded researcher from site four (sports organisation) reported that she taught the practitioners to collect data at a training workshop that she organised. She explained that this training assisted the embedded research project because it helped the practitioners to get involved in the data collection phase as they had the skills from the training. Similarly, another embedded researcher from site one reported that getting the practitioners involved in the embedded research work facilitated the sharing of learning, which was one of her main goals while working at the LA. This particular researcher trained the public health practitioners to collect data and taught them how research evidence can inform practical decision making.

The participants agreed that working together with the embedded researchers strengthened their ability to conduct high-quality research capable of benefiting their respective organisations.

“ It also allowed us to utilise and build the capacity of public health practitioners who would often not undertake any research for some time” [PHP2site1] .

“So, it’s more like continuous professional development [..] So, the research skills are learnt such that at the end of the day, next time the research could be conducted independently, even if we didn’t have somebody coming from the outside. That’s the whole approach [..] is for developing public health practitioners to the extent that research can be conducted in a rigorous manner” [PHP1site1] .

“I think probably when I attended two beneficiary interviews with her and just seeing how to speak to people when you are asking them questions so there is a way to ask the questions so that they understand, probably by listening to the researcher at that point I sort of learnt how” [ PHP5site2] .

As the above suggests, the embedded researchers encouraged some practitioners who would ordinarily not participate in research to engage in research activities. This implies that working together with researchers may be a significant facilitator to building practitioners’ research capacity and closing the research implementation gap. The absence of an embedded researcher may even serve to widen the gap. Indeed, the public health practitioners observed that working with embedded researchers could eventually build their research capacity to independently conduct high-quality research in the future.

Overall, it was clear that the participants were aware of the importance of working together with embedded researchers, and the researchers were acknowledged for their ability to assist greatly with research-related training and support to build their research capacity. It would have been difficult for these organisations to generate high-quality on-site research if the embedded researchers had not been present. Consequently, the embedded researchers helped work to close the research evidence implementation gap.

Informing practice

The embedded researchers built and maintained relationships with the practitioners and other stakeholders to work together with them to co-produce research. The participants from the four case study sites reflected upon how the embedded researchers informed the sites of relevant research-based evidence, which helped in the development of future practice and research. By doing so, the embedded researchers bridged the gap between the discovery and implementation of research-based evidence. The results showed that all participants across all the four case study sites, irrespective of age, years of experience, and education, agreed that the role of the embedded researchers includes this theme.

The strategies adopted by the embedded researchers include: (1) identifying challenges in the host organisations, (2) utilising research experience, (3) implementing research evidence, (4) disseminating findings, identifying future research areas, and applying for funding, (5) presenting and publishing findings.

Identifying challenges in the host organisations

Participants agreed that the research skills of the embedded researchers are essential to the process of identifying the practical challenges facing the research sites. For instance, an embedded researcher used their research skill to unravel the root cause of the challenges facing a school (site three) through a thorough investigation by developing and conducting relevant research with the students and the teachers.

“[…] the GSCE reforms of the time that was taking place, it was causing a significant amount of stress and pressure for the teachers. In the first instance, teachers were having to grasp new skills at work, they were having to understand the new curriculum and subject knowledge. Some of the teachers weren’t particularly strong, there was a level of undue pressure and stress being put on the students, so pupils nationally were having to learn lots of different contents, they were sort of taken away the security blankets of things like modular testing in course work and what that meant was that students will now have to recall so much more knowledge in exam conditions” [TR1site3] .

Following the identification of these challenges, research-based recommendations were offered through the co-production research. By using research evidence to help tackle the school’s challenges, the researcher bridged the gap between the discovery and implementation of research-based evidence.

Utilising research experience

It is worth noting that the embedded researchers used their research experience to inform their host organisations of relevant existing and newly co-produced research evidence. The embedded researchers’ research-related expertise and the time they spent searching for relevant evidence were both seen as useful to the public health practitioners and other stakeholders.

“The beauty is that because it is their bread and butter, doing reviews and searching for evidence […] one of the things the embedded researcher did to help me with it was to do that literature review [..] it would have taken me much longer [..], so that’s the benefit [..] it is their strength and their experience and skills which they have got and which we may not have and the time to do it which we may not also have because we are constantly under the treadmill” [ PHP1site1] .

It was evident that the practitioners’ busy work schedules often restrict their ability to develop and implement their own research skills. Thankfully, the embedded researchers were able to assist the practitioners by using their research skills to overcome research-related challenges, and in the process taught them how to look for research evidence effectively. This, therefore, facilitates the implementation of evidence-based practice. The implication of this is that practitioners’ lack of research skills and time would have served as a barrier for evidence-based practice in the research sites.

It was clear that the research-based evidence searched for, or co-produced by the embedded researchers and the public health practitioners including other stakeholders was used to inform practice and make positive changes. Evidence showed that the embedded researchers had informed the host organisations of relevant research evidence and had used their research experience and skills to make research-based recommendations. In other words, the embedded researchers made valuable research evidence, and knowledge accessible. As such, this brought about desirable changes that improved service and delivery in the research sites.

“ So the way this works here is that you do the final report which has the recommendations in form of what we feel there should be changes to in practice, and that goes to their public management team and then they will look at that” [ERsite2] .

Furthermore, the embedded researchers also discussed how they helped make positive on-site changes occur. For instance, an embedded researcher from site two reported that positive changes were made in practice after developing recommendations in the form of a report submitted for management’s approval. It was clear that the practitioners take evidence-based advice from the embedded researcher to improve the quality of the services being offered to the public. Thus, this closes the gap between research evidence and its implementation.

Implementing research evidence

The interviews inquired as to how research-based evidence was translated into practice at the four research sites. As the interview process continued, it became clear that desired changes and improvements were achieved through the on-site application of research-based evidence. The results showed that across the four research sites, this process did indeed happen.

“[..] as it is very much about kind of being a resource to implement the recommendations and embed kind of the key findings from the research, again my role is trying to get some of these things into practice really so its embedded research but the main one of the main things is around embedding the recommendations as well, so that’s sort of work my role is around doing” [ ERsite1] .

“ [..] at the same time, it also helps the researcher coming in to understand what goes on in practice so that you don’t just go and conduct a piece of research that goes on the shelves. [..] So we would then need to weigh the evidence and the circumstances under which we are going to implement an intervention but we still take advice from the researcher on the evidence of what works. They could advice on what works [..] It’s more about the outcome of research being used to influence practice for quality improvement” [ PHP1site1] “There are changes that are made with how they recruit their staff for the delivery staff […] that changes were made and that was in practice, and they also kind of put it in a set of recommendations as to the ones to be delivered in schools” [ERsite4] .

Participants reported that the embedded researchers recommended existing research evidence, co-produced research evidence with the intent of informing practice, and also used relevant evidence to help improve service and delivery. In other words, the role of embedded researchers provided accessibility to research-based evidence that was utilised to develop solutions to on-site challenges and create positive change.

Disseminating findings, identifying future research areas, and applying for funding

The embedded researchers reported that having to present reports to diverse audiences prompted them to produce easily understandable, user-friendly reports that did not rely heavily on academic language.

“[..] so I have quarterly reports that I have to produce which has to be user-friendly and appeal to a various range of agencies within the organisation [..] we had, basically we have had quite a few different presentations to different kind of groups or the senior management team and departmental teams and things which was about and sharing the results and recommendations, we have follow-ups sort of things from that” [ ERsite1] .

“[..] Yeah, just into writing report so she will do like verbal update or she provides like some blueprints in an email ” [PHP5site2] .

The reports created by the embedded researchers avoided scientific terms that might be difficult for public health practitioners and other stakeholders to understand. Furthermore, practitioners and other stakeholders were informed of relevant research evidence in an unambiguous way. It is important to add that it would have been difficult for the embedded researchers to appropriately simplify their language if they had not had the opportunity to spend time on-site becoming familiar with the language used by the practitioners and stakeholders.

The participants also reported that the embedded research projects effectively discovered potential areas for future research. By making suggestions regarding future research, the embedded researchers furthered each host organisation’s potential to engage in relevant, change-creating research.

“[..] then the research outcomes were used to inform the next phase, so obviously that was the first phase, which we felt was really successful and worked really well, so then we took those sort of the things we learnt to the next phase” [ERsite3] .

For example, an embedded researcher from site three (school) stated that the first phase of their embedded research project was such a success that the findings of the first phase informed the direction of the second phase, thereby ensuring continuous research activities in the school.

Furthermore, participants agreed that the outcomes of the embedded research projects assisted with the application for future funding.

“[..] the results of the work that we did has been kind of used in terms of future funding opportunities, for providing data, providing kind of context information that was used in sort of proposals and in bids pushing and for applying for future funding” [ERsite1] .

It was evident that the presence of the embedded researchers in their host organisations encouraged the push to apply for funding to develop projects. This, therefore, facilitates continuous engagement in research activities. The practitioners felt that the role of the embedded researchers is crucial to producing funding applications and program development.

Presenting and publishing findings

Once embedded researchers succeeded at co-producing relevant on-site research evidence with practitioners and other stakeholders, and offering practical solutions to on-site challenges, it became clear that it would be necessary to present and publish the outcomes of the projects. Consequently, embedded researchers used their academic skills to publish the findings with practitioners and other stakeholders as co-authors. One of the benefits of publication is that published research can inform the host organisation, and other organisations facing similar challenges. Another significance of the role of embedded research pertaining to this, is that as the embedded research project is co-produced by both the embedded researcher and the host organisation, the findings from the research are jointly owned by both parties. This also assisted in integrating research into the host organisations culture.

“We wrote a book chapter with their names on the published book chapter. We got all of them involved with the writing of the chapter [..] that makes a sort of massive difference ” [ERsite3] .

“We co-authored a chapter of a book. We used the findings to create a book chapter but all of us has input into it including the researchers” [ST2site3] .

For example, participants from site three (school) reported that a book chapter based on co-produced research that they had worked on with the embedded researcher had been published [ 43 ]. Co-produced and co-published research evidence informs the school and research community of the institutional value of embedded research projects. The embedded researcher from site three (school) added that the names of the students and staff involved in the research and writing processes were included in the book chapter. The book chapter was co-edited by both an academic and a public health consultant. This publication has made a tremendous positive difference to how a school labelled as ‘deprived’ views itself. Indeed, being involved in the co-production of valuable research has encouraged both students and teachers.

To further explore how embedded researchers can inform public health practice, the participants were asked whether any other evidence-sharing processes had been used by the embedded researchers. The embedded researchers in this study were connected to more than one organisation. Consequently, they have access to organisations with information that can benefit public health practitioners and other stakeholders. The participants felt that participating in other organisations helped the embedded researchers fulfil their role as the discoverers and sharers of information. The participants viewed this role of the embedded researcher in their sites important as it informs them of the latest research evidence and activities in the field. This could also be seen as a way to sustain evidence-based practice in the sites. As the practitioners are regularly informed of the latest relevant evidence by attending research-based programmes, it facilitates the integration of research into the host organisations’ culture.

“When I see opportunities for conferences or local events, I will send an email or circulating them, there might be public health conference, it might be a Fuse conference that’s linked in erm linked in heavily with the thing we have worked on and I circulate that to the staff member, to say here is an opportunity” [ERsite2] .

For instance, an embedded researcher from site two stated that she regularly informed the practitioners of programmes and events presenting research relevant to their practice. By attending such events, practitioners can stay informed and up to date and are more likely to make changes to their practice based on timely research evidence. Consequently, the findings of this study indicate that staying familiar with the latest relevant research is one of the ways to close the gap between the discovery and implementation of research-based evidence.

Overall, it was evident that the embedded researchers’ ability to inform the organisations with relevant co-produced research evidence, and the ability to identify relevant information and opportunities and then circulate these to public health practitioners and stakeholders helped to inform the sites in creating relevant, research-based changes to benefit their public health practices. The positive outcomes they generated indicate that the role of embedded researchers can seriously contribute to closing the gap between the discovery and implementation of research-based evidence in the research sites.

Critical reflection

Twelve out of seventeen participants across the four sites discussed this theme as part of the role of the embedded researcher in their respective organisations. Participants felt that critical reflection was an important process an embedded researcher must engage in throughout the ‘journey’ of becoming an agent of closing the gap between research evidence and its implementation in practice. The identified strategy adopted by the embedded researchers within this theme is continuous reflection.

“I constantly reflect on my role to know what I am doing right, and what can be done differently” [ERsite1] .

“I have to spend really more time reflecting” [ERsite2] .

“It might be while you drive home [..] might be in the shower [..] might be when I take the dog out for a walk and tea time to reflect because you do need time to reflect on your research, on your methodology [..] about what the findings need to show [..] at times my bag is full of paper everywhere, millions of notes in here and I have to open and jot down some questions so that I won’t forget them because they are so important” [ERsite4] .

“I think it’s always good to sort of like reflect on what we have done, how we do things I personally want to think about whether I could have done things better […] so I think it’s quite important to sort of reflect on how you have done things, and how you could do things in the future, like what lessons you have learnt, I think it’s important to sort of reflect, to sort of think more about how you have done things and whether it could be practiced in the future” [ERsite3] .

Overall, the participants agreed that reflection helps embedded researchers assess their roles and constantly improve their work. Therefore, reflection is crucial to successfully co-producing research and closing the research implementation gap.

All participants, irrespective of their age, working experience and education, acknowledged that the relationships between the people involved in an embedded research project are crucial to the project’s success. This is in keeping with those made in previous studies that have concluded that building and maintaining mutually beneficial relationships with practitioners and other stakeholders significantly helps embedded researchers co-produce public health knowledge in non-clinical settings [ 33 , 44 ]. The study participants were also unanimous in their view that the ‘embeddedness’ of the researchers, or the degree to which they become part of or spend time within the host organisation, is significant. A higher degree of embeddedness appears to lead to the development of beneficial relationships and also helps researchers develop a better understanding of organisational contexts, that in turn leads to the development of effective solutions and useful, co-produced research. Notably, becoming embedded to a significant degree helps others see the researchers as part of the team. Previous studies have also indicated it is the duty of the embedded researcher to become part of the host organisation by working collaboratively with practitioners and other stakeholders [ 17 , 45 ].

Although the amount of time each embedded researcher spent within their host organisation varied, the interview data gathered from all sites confirmed that embedded researchers felt they were able to develop meaningful relationships with the host organisation. The National Institute for Health Research (NIHR) embedded research team reported similar findings and observed that the amount of time spent within an organisation can depend on the intensity of a project [ 46 ].

Among other strategies, informal conversations with the practitioners and other stakeholders also assisted the embedded researchers to build relationships. This was confirmed only by the embedded researchers in case study sites two and four who had worked in the host organisations for more than three years. This might be because the embedded researchers from the local authority (site two) and the sports organisation (site four) had worked and familiarised themselves with the members of the host organisation staff. Consequently, this could have facilitated easier informal conversations, unlike the embedded researcher in site one who has just spent seven months in the site. This confirms that it takes time for embedded researchers to build trustworthy relationships in the host organisation and they recommend an ‘introductory period’ of a minimum of three months for familiarisation before an embedded research project starts [ 39 ]. This was beneficial to the three case studies explored in an earlier study as it allowed the embedded researchers to familiarise themselves with their host organisations and as well build relationships with the host organisations’ staff [ 39 ]. This also aligns with the view of other scholars that an ‘introductory period’ is important before the commencement of an embedded research project [ 44 ]. It is worth noting that the practicability of an ‘introductory period’ may depend on the agreement between the parties involved.

Furthermore, embedded researchers must build relationships not only with practitioners and other stakeholders, but also with their academic supervisors. Having a successful relationship with the academic supervisor can help the embedded researcher overcome the challenges that arise as a consequence of having a dual affiliation and needing to manage diverse expectations and competing interests. The embedded researchers interviewed in this study had the support of their academic supervisors. Thanks to the vast experience of their supervisors, they are often excellent at mitigating unforeseen challenges. Indeed, among other factors, the success of an embedded researcher depends on the relationship between the researcher and his or her academic supervisor [ 13 , 39 ].

The interview participants recounted that it is important to work together to co-produce relevant research which is useful to the organisations. Other scholars have similarly concluded that embedded researchers work with members of their host organisations to identify, plan, and conduct research that will meet the needs of the organisation [ 36 ]. By working collaboratively, embedded researchers were able to train the practitioners and other stakeholders and improve their ability to help co-produce meaningful and valuable research that can be used to implement evidence-based adjustments to on-site practices.

The findings of this study indicate that working together produces meaningful research and also teaches practitioners and other stakeholders who assist embedded researchers, how to conduct research. Similarly, an earlier study concluded that embedded researchers encourage practitioners and other stakeholders to participate in research activities and increase an organisation’s capacity to conduct research [ 17 ]. In other words, the collaborative work that accompanies embedded research helps close the research implementation gap. However, it was noted in this current qualitative inquiry that having the right researchers assisted in carrying out the projects successfully. This is similar to an earlier study that argue that having the right combination of researchers and practitioners in co-production is crucial to the success of such project [ 13 ]. Also, other scholars pointed out that not all researchers have the relevant skills to conduct co-produced research [ 17 ]. Therefore, it is essential to have the right combination of researchers, practitioners, and other stakeholders while working together to co-produce research to ensure its success.

Based on the current qualitative inquiry, the role of the embedded researchers includes informing practice by making recommendations and positive changes that utilise both existing and newly co-produced research evidence. Doing so makes research evidence more accessible to public health practitioners and other stakeholders and ultimately improves service and delivery. An earlier study similarly revealed that informing practice has been identified as a way by which embedded researchers communicate new and existing relevant research evidence and integrate research findings into practice [ 3 ].

As discussed earlier, two of the factors responsible for the gap between the discovery and implementation of research evidence are the disparity between the language spoken by the researchers and practitioners and the complexity of the language spoken by researchers, which is often include scientific jargon. Such complex language can be difficult for practitioners to understand or lead to ambiguities in interpretation [ 12 ]. To discover whether language differences was an issue in this study, the interviews included questions regarding how research evidence and recommendations were communicated to public health practitioners and other stakeholders. These questions were designed to create an understanding of how the embedded researchers had communicated. The interviews revealed that the embedded researchers communicated research outcomes and recommendations effectively to the practitioners by using simple, unambiguous language. Using such language helped make research evidence more accessible to the practitioners.

Providing evidence for reports and future funding applications was identified as an important part of the embedded researchers’ work within their host organisations [ 17 , 47 ]. The interview participants agreed that the researchers sometimes helped secure funds needed to conduct research at the host organisation. Doing so encouraged each host organisation’s staff to participate in research that could prove useful to the organisation in the future.

Critical reflection helps embedded researchers evaluate the role they play within their host organisation and keep track of their progress [ 33 , 48 ]. In other words, reflection helps researchers identify and improve upon the areas that are not meeting expectations and discover what approaches are working successfully. This corresponds with the findings from this current qualitative inquiry. The interview participants acknowledged that the embedded researchers continuously reflect on their role and their work in order to identify what is and is not working. This assists embedded researchers to think of ways to apply acquired learning to daily on-site practice to improve their role in the co-production of research to bridge the gap between research evidence and its implementation in public health practice.

Limitations of the study

One of the limitations of this study was the sample size. A total of 17 participants was recruited for this study, although the sample size would have been larger than 17 but for the COVID-19 pandemic. Another consideration of this piece of work, being qualitative research, was subjectivity. The information provided by the participants was based on their point of view. Hence, it might be difficult to objectively verify the qualitative information provided to ensure that accurate information was provided by the participant regarding the phenomenon of interest. Nevertheless, some practical measures were undertaken to ensure the credibility of this work. Data triangulation and site triangulation [ 49 ] were adopted in this study. These were done to increase the confidence in the outcome of the qualitative multi-site case study.

Overall, the success that the embedded researchers experienced, including building relationships, co-producing research, translating research into practical changes, evaluating projects, and informing future public health practices as well as future research, justifies increasing the amount of embedded research being conducted in public health practice. Embedded researchers also bring the tremendous benefit of strengthening the research capacities of public health practitioners and other stakeholders by providing research-based training and support. Such developments have the ability to prove the potential of embedded research projects. Finally, the relevant research-based recommendations made from the co-produced research guided by the embedded researchers are used to inform practice. The positive outcomes generated by the embedded research process indicate that embedded researchers can meaningfully contribute to closing the gap between the discovery and implementation of research evidence.

Availability of data and materials

The datasets generated and/or analysed during the current study are not publicly available. They are available from the corresponding author on reasonable request, subject to approval from the Teesside University School of Health and Life Sciences Research Governance and Ethics Committee.

Lobb R, Colditz GA. Implementation science and its application to population health. Annu Rev Public Health. 2013;34:235–51.

Article   PubMed   PubMed Central   Google Scholar  

Di Ruggiero E, Viehbeck S, Greyson D. Knowledge utilization and exchange. Oxford: Oxford University Press—Oxford Bibliographies in Public Health; 2017.

Google Scholar  

Marshall M, Pagel C, French C, Utley M, Allwood D, Fulop N, Pope C, Banks V, Goldman A. Moving improvement research closer to practice: the researcher in residence model. BMJ Qual Saf. 2014;23:801–5.

Chew S, Armstrong N, Martin G. Institutionalising knowledge brokering as a sustainable knowledge translation solution in healthcare: how can it work in practice? Evid Policy. 2013;9:335–51.

Article   Google Scholar  

Proudfoot A, Mcauley D, Hind M, Griffith M. Translational research: what does it means, what has it delivered and what it might deliver? Curr Opin Crit Care. 2011;17:495–503.

Article   PubMed   Google Scholar  

Albert MA, Fretheim A, Maiga D. Factors influencing the utilization of research findings by health policymakers in a developing country: the selection of mali’s essential medicines. Health Res Policy Syst. 2007;5:2.

Armstrong R, Doyle J, Lamb C, Waters E. Multi-sectoral health promotion and public health: the role of evidence. J Public Health. 2006;28(2):168–72.

Bunn F. Strategies to promote the impact of systematic reviews on healthcare policy: a systematic review of the literature. Evid Policy. 2011;7:428.

Allen T, Grace C, Martin S. From analysis to action: connecting research and local government in an age of austerity. Report of the local government knowledge navigator. London: Local Government Association; 2014.

Campbell D, Donald B, Moore G, Frew D. Evidence check: knowledge brokering to commission research reviews for policy. Evid Policy. 2011;7(1):97–107.

Dobbins M, Robeson P, Ciliska D, Hanna S, Cameron R, O’mara L, et al. A description of a Knowledge Broker Role implemented as part of a randomized controlled trial evaluating three knowledge translation strategies. Implement Sci. 2009;4:23.

Friese B, Bogenschneider K. The Voice of experience: how social scientists communicate family research to policymakers. Fam Relat. 2009;58(2):229–43.

Newbury-birch D, Allan K. Co-creating and co-producing Research evidence: a guide for practitioners and academics in Health, Social Care and Education Settings. London and New York: Routledge; 2019.

Book   Google Scholar  

Van der graaf P, Forrest L, Adams J, Shucksmith J, White M. How do public health professionals view and engage with research? A qualitative interview study and stakeholder workshop engaging public health professionals and researchers. BMC Public Health. 2017;17:892.

Hobin EP, Riley B, Hayward S, Ruggiero ED, Birdsell J. Maximising the use of evidence: exploring the Intersection between Population health intervention research and knowledge translation from a Canadian perspective. Evid Policy. 2012;8(1):97–115.

Oliver K, Kothari A, Mays N. The dark side of coproduction: do the costs outweigh the benefits for health research? Health Res Policy Syst. 2019;17:33.

Wong S. Tales from the frontline: the experiences of early childhood practitioners working with an ‘Embedded’ research team. Eval Program Plann. 2009;32:99–108.

Fathimath S, David E, Helen B. Nurses’ perceptions of barriers and facilitators to implement EBP in the Maldives. Adv Nurs. 2014;2014:7. Article ID 698604.

Aszkenasy OM, Dawson D, Gill M, Haines A, Patterson DLH. Audit of direct access cardiac investigations: experience in an inner london health district. J R Soc Med. 1994;87:588–90.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Davis DA, Thomson MA, Oxman AD, Haynes RB. Changing physician performance: a systematic review of the effect of educational strategies. J Am Med Assoc. 1995;274:700–5.

Article   CAS   Google Scholar  

Sutton M. How to get the best health outcome for a given amount of money. BMJ. 1997;1997(315):47–9.

Rycroft-malone J. From knowing to doing: from the academy to practice, comment on the many meanings of evidence: implications for the translational science agenda in healthcare. Int J Health Policy Manag. 2014;2:1–2.

Walshe K, Davies H. Health research, development and innovation in England from 1988 to 2013: from Research production to knowledge mobilization. J Health Serv Res. 2013;18:1–12.

Ryan B. Coproduction: option or obligation? Aust J Public Adm. 2012;71:314–24.

Groundwater-smith S, Mockler N. The knowledge building school: from the outside in, from the inside out. Change Transform Educ. 2002;5:15–24.

Himmrich J. How should academics interact with policy makers? Lessons on building a long-term advocacy strategy. LSE Impact Blog. 2016. Available at: How should academics interact with policy makers? Lessons on building a long-term advocacy strategy. | Impact of Social Sciences (lse.ac.uk). Accessed 15/01/2022.

Zevallos Z. Protecting activist academics against public harassment. Other Sociologist. 2017. Available at: Protecting Activist Academics Against Public Harassment – The Other Sociologist. Accessed 15/01/2022.

Davey SG, Ebrahim S, Frankel S. How policy informs the evidence. BMJ. 2001;322(7280):184–5.

Maybin J. How proximity and trust are key factors in getting research to feed into policymaking. LSE Impact Blog. 2016. Available at: How proximity and trust are key factors in getting research to feed into policymaking | Impact of Social Sciences (lse.ac.uk). Accessed 15/01/2022.

Rycroft-malone J, Burton CR, Bucknall T, Graham ID, Hutchinson AM, Stacey D. Collaboration and co-production of knowledge in healthcare: opportunities and challenges. Int J Health Policy Manage. 2016;5(4):221–3.

Hegger D, Dieperink C. Toward successful joint knowledge production for Climate Change Adaptation: lessons from six Regional projects in the Netherlands. Ecol Soc. 2014;19(2):34.

Brannick T, Coghlan D. Defense of being native: the case for insider academic research. Organ Res Methods. 2007;10:59–74.

Langeveld K, Stronks K, Harting J. Use of a Knowledge Broker to establish healthy public policies in a city district: a developmental evaluation. BMC Public Health. 2016;16:271.

Yost J, Dobbins M, Traynor R, Decorby K, Workentine S, Greco L. Tools to support evidence-informed public health decision making. BMC Public Health. 2014;14:728.

Smith LS, Wilkins N. Mind the gap: approaches to addressing the research-to-practice, practice-to-research chasm. J Public Health Manage Pract. 2018;24:S6–11.

Mcginity R, Salokangas M. Introduction: “Embedded Research” as an Approach into academia for emerging researchers. Management in Education. 2014;28:3–5.

Dixon-woods M, Martin G. Does quality improvement improve quality? Future. Hosp J. 2016;3:191–4.

Dixon-woods M, Mcnicol S, Martin G. Ten challenges in improving quality in healthcare: lessons from the Health foundation’s programme evaluations and relevant literature. BMJ Qual Saf. 2012;21:876–84.

Vindrola-padros C, Eyre L, Baxter H, et al. Addressing the challenges of knowledge co-production in quality improvement: learning from the implementation of the researcher-in-residence model. BMJ Qual Saf. 2019;28:67–73.

Polit D, Beck C. Generalization in quantitative and qualitative research: myths and strategies. Int J Nurs Stud. 2010;47:1451–8.

Jenkins EK, Slemon A, Haines-saah RJ, Oliffe J. A Guide to multisite qualitative analysis. Qual Health Res. 2018;28(12):1969–77.

Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Res Psychol. 2006;3(2):77–101.

Hayden MC, Waller G, Hodgson A, Brown S, Harris S, Miller K, Barber D, Hudson L, Newbury-Birch D. ‘Pupils, Teachers and Academics Working Together on a Research Project Examining How Students and Teachers Feel About the New GCSEs’, in Newbury-Birch and Allan. Co-creating and Co-producing Research Evidence: A Guide for Practitioners and Academics in Health, Social Care and Education Settings. London and New York: Routledge, Taylor & Francis Group; 2019.

Lewis S, Russell A. Being embedded: a way forward for ethnographic research. Ethnography. 2011;12:398–416.

Rowley H. Going beyond procedure: engaging with the ethical complexities of being an embedded researcher. Manage Educ. 2014;28:19–24.

Embedded research. https://www.embeddedresearch.org.uk . (No date). Accessed 22 Nov 2023.

Jenness V. Pluto, prisons, and plaintiffs: notes on systematic back-translation from an embedded researcher. Soc Probl. 2008;55:1–22.

Duggan J. Critical friendship and critical orphanship: embedded research of an english local authority initiative. Manage Educ. 2014;28(1):12–8.

Denzin N. The Research Act: A Theoretical Introduction to Sociological Methods (2nd edition). New York: McGraw Hill; 1978.

Download references

Acknowledgements

We thank the participants for sharing their expertise and time. We are grateful for the contribution of Ronnie Ramlogan who supported us in the preparation of this manuscript.

This research received no external funding.

Author information

Authors and affiliations.

School of Health and Life Science, Teesside University, Middlesbrough, UK

Abisope Akintola

School of School of Social Sciences, Humanities & Law, Teesside University, Middlesbrough, UK

Dorothy Newbury-Birch & Stephanie Kilinc

Manchester Institute of Innovation Research, Alliance Manchester Business School, University of Manchester, Manchester, UK

You can also search for this author in PubMed   Google Scholar

Contributions

This study is part of AA’s PhD work, as such, AA conducted this piece of work with the supervision of DNB and SK.

Corresponding author

Correspondence to Abisope Akintola .

Ethics declarations

Ethics approval and consent to participate.

Ethical approval was obtained from the Teesside University School of Health and Life Sciences Research Governance and Ethics Committee. All methods were carried out in accordance with relevant research guidelines and regulations. The participants were informed about the purpose of the research, the voluntary nature of participation, and their right to withdraw at any time without any consequences. The anonymity and privacy of the participants were guaranteed throughout the study. Informed written and verbal consent was obtained from the participants before the interviews.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Interview schedule for embedded researchers

Role identification and background information about the embedded research initiative.

What is your role in your organisation? Prompt - Job title, Daily task, Responsibilities. B) How long have you been in this role? C) Can you tell me about your background and what you do? Prompt -The journey so far- How do you get to where you are now?  D) As an embedded researcher where is your academic affiliation?

How long has your embedded research initiative been going on in your organisation? B) Do you know the rationale for employing an embedded researcher in your organisation? C) Who funds your project? D)What is the management arrangement?

Moving on to look at the embedded research initiative more specifically

What is the aim of the embedded research project you are involved in? B) How many hours/days do you spend in your host organisation in a week, and in the academic institution?  C) Why? D) How often do you contact your academic supervisor?

How has embedded research gone so far in your organisation?  B) How many people are involved in the co-production/embedded research you are involved in? or who do you work with? C) How many embedded researchers are involved in the project? Prompt - How many professionals/stakeholders?

What are your views and experience of embedded research? Prompt - what have you learnt? What, if anything, has helped?  (Why do you say that?) What, if anything, has been more difficult or challenging? (Why do you say that)? What difference has embedded research made in your organisation?  (so if embedded research has been useful, why and how?)

Looking more specifically at the role of the embedded researcher in the organisation

What is your role, as an embedded researcher in bridging the gap between research evidence and its implementation in practice? Prompts - How do you inform practice with research evidence?  How do you communicate research evidence to practitioners and other stakeholders to facilitate its use in practice? B) Does your role involve the translation of research evidence into practice? If yes, what is the process? can you please cite an example? What evidence-sharing methods or processes do you use?

Can you think of any changes in practice/policy as a result of research evidence being used? Prompt – What role did you play? Who was involved? What changed? How? For who?

Tell me what you think are the benefits of working as an embedded researcher? Why do you say that? B) How do you manage the dual affiliation? Prompt -what are the benefits (What has helped?) and also what are the challenges?

Tell me what you think are the challenges of working as an embedded researcher? Prompt - Why do you say that? B) What are the barriers to data sharing, if any?

Do you think building mutually beneficial relationships with the host organisation staff is important to the success of an embedded research project? If yes, Why? B) How do you build relationships with the host organisation’s staff?

Can you cite an example of where you have built practitioners and other stakeholders’ confidence in conducting their own research?

Does your role requires managing research funds? If yes, how do you manage this?

 How often do you reflect on your role? Prompt- To know what works and what needs to be improved?  Why is this important?

Do you think the development of a toolkit on the role of embedded research in bridging the gap between research evidence and its implementation in public health practice would be useful? If yes, Why and how do you think it could be used in practice?”

Any top tips for other researchers considering embedded research?

Please don’t mention names, but can you think of any potential participants- people you are working with or have worked with that you can pass on the details of this research?  B) Would you be happy to be contacted afterward to circulate details of this research to those you have identified, to see if they will be willing to participate in this research?

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Akintola, A., Newbury-Birch, D. & Kilinc, S. Bridging the gap between research evidence and its implementation in public health practice: case studies of embedded research model. BMC Public Health 24 , 1299 (2024). https://doi.org/10.1186/s12889-024-18727-z

Download citation

Received : 05 January 2024

Accepted : 28 April 2024

Published : 13 May 2024

DOI : https://doi.org/10.1186/s12889-024-18727-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Public health
• Embedded research
• Research-based evidence
• Co-production
• Research evidence- implementation gap

BMC Public Health

ISSN: 1471-2458

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Open access
• Published: 10 May 2024

An evaluation of computational methods for aggregate data meta-analyses of diagnostic test accuracy studies

• Yixin Zhao 1   na1 ,
• Bilal Khan 1   na1 &
• Zelalem F. Negeri 1  

BMC Medical Research Methodology volume  24 , Article number:  111 ( 2024 ) Cite this article

106 Accesses

9 Altmetric

Metrics details

A Generalized Linear Mixed Model (GLMM) is recommended to meta-analyze diagnostic test accuracy studies (DTAs) based on aggregate or individual participant data. Since a GLMM does not have a closed-form likelihood function or parameter solutions, computational methods are conventionally used to approximate the likelihoods and obtain parameter estimates. The most commonly used computational methods are the Iteratively Reweighted Least Squares (IRLS), the Laplace approximation (LA), and the Adaptive Gauss-Hermite quadrature (AGHQ). Despite being widely used, it has not been clear how these computational methods compare and perform in the context of an aggregate data meta-analysis (ADMA) of DTAs.

We compared and evaluated the performance of three commonly used computational methods for GLMM - the IRLS, the LA, and the AGHQ, via a comprehensive simulation study and real-life data examples, in the context of an ADMA of DTAs. By varying several parameters in our simulations, we assessed the performance of the three methods in terms of bias, root mean squared error, confidence interval (CI) width, coverage of the 95% CI, convergence rate, and computational speed.

For most of the scenarios, especially when the meta-analytic data were not sparse (i.e., there were no or negligible studies with perfect diagnosis), the three computational methods were comparable for the estimation of sensitivity and specificity. However, the LA had the largest bias and root mean squared error for pooled sensitivity and specificity when the meta-analytic data were sparse. Moreover, the AGHQ took a longer computational time to converge relative to the other two methods, although it had the best convergence rate.

Conclusions

We recommend practitioners and researchers carefully choose an appropriate computational algorithm when fitting a GLMM to an ADMA of DTAs. We do not recommend the LA for sparse meta-analytic data sets. However, either the AGHQ or the IRLS can be used regardless of the characteristics of the meta-analytic data.

Peer Review reports

Meta-analysis is a statistical technique used in research to combine and analyze the results of multiple independent studies on a particular topic or research question [ 1 ]. A meta-analysis of diagnostic test accuracy (DTA) is a specific type of meta-analysis that focuses on combining and analyzing data from multiple studies assessing the performance of diagnostic tests, allowing for synthesizing diagnostic test characteristics, such as sensitivity (Se) and specificity (Sp) across multiple independent studies [ 2 , 3 ]. In an aggregate data meta-analysis (ADMA) of DTAs, one gathers information on the true positive (TP), true negative (TN), false positive (FP), and false negative (FN) results for a specific diagnostic test across various studies. From these data, the study-specific observed Se, Sp, and other relevant measures of diagnostic accuracy can be calculated. By pooling the results from multiple studies, researchers aim to derive summary estimates of these test characteristics, while considering the variability and potential biases present in the individual studies.

Researchers and practitioners usually use generalized linear mixed models (GLMM) such as the bivariate random-effects model of Chu and Cole [ 4 ] to meta-analyze DTA data and obtain the maximum likelihood estimates (MLEs) of the model parameters. However, unlike the linear mixed model version of Reitsma et al. (2005) [ 5 ], since Chu and Cole’s GLMM does not have a closed-form solution for the log-likelihood due to the complex random effects variance components, one needs to use numerical methods to approximate the log-likelihood function and obtain MLEs of the model parameters. Commonly used computational methods in the context of an ADMA of DTAs include the Adaptive Gaussian Hermite quadrature (AGHQ) [ 6 ], the Laplace approximation (LA) [ 6 ], and the iteratively re-weighted least squares (IRLS) [ 7 , 8 ].

There have been some attempts at comparing and evaluating some of these numerical methods in different contexts. Ju et al. (2020) [ 9 ] compared the AGHQ, LA and the penalized quasi-likelihood (PQL) for meta-analyzing sparse binary data, and found that the AGHQ and PQL did not show improved performance compared to the LA. However, Ju et al. did not take the IRLS into account, and compared the numerical methods only in terms of the pooled odds ratio but not concerning the between-study variance and covariance. Additionally, their study was focused on a meta-analysis of sparse binary intervention studies outcomes, not on DTA data. Thomas, Platt & Benedetti [ 10 ] studied the performances of the PQL and AGHQ algorithm for meta-analysis of binary outcomes in the context of an individual participant data meta-analysis (IPDMA) of intervention studies. They found that there were no appreciable differences between the two computational methods. However, Thomas et al. did not consider the LA and meta-analysis of DTAs.

However, to the best of our knowledge, there was no evidence in the literature that describes the performance of these widely used computational algorithms for GLMM in the context of either IPDMA or ADMA of DTAs, partly because DTA meta-analysis is a relatively newer area of research compared to the widely studied meta-analysis of intervention studies. Additionally, since diagnosis precedes intervention, it is crucial to establish the accuracy of diagnostic tests using sound statistical methods or algorithms to minimize misdiagnosis of patients due to flawed evidence. Moreover, since meta-analytic methods for intervention or treatment studies cannot be used to meta-analyze DTA data because of differences in data characteristics and model assumptions [ 11 ], establishing evidence on the performance of computational methods for ADMA of DTA studies is needed. Therefore, this paper aims to fill this important research gap by comparing and evaluating the AGHQ, IRLS, and LA performances for GLMM to meta-analyze DTAs using aggregate data. We will compare the numerical methods using an extensive simulation study in terms of absolute bias, root mean squared error (RMSE), coverage probability, 95% confidence interval (CI) width, convergence rate, and computational speed. We will also illustrate the methods using real-life meta-analytic data.

The rest of this article is organized as follows. Motivating examples  section presents motivating examples using two real-life data, Methods  section introduces the statistical methods, including the GLMM model, the numerical algorithms and a simulation study. In Simulation study results  section, we discuss our simulation study results, and in Illustrative examples  section, we illustrate the computational methods using the motivating examples data. We conclude the manuscript with a discussion and concluding remarks in Discussion and Conclusions  sections.

Motivating examples

This Section describes two real-life data sets (see Appendix Tables A 1 and A 2 ) to motivate the statistical methods we present in Methods section.

First, consider an article by Vonasek et al. (2021) [ 12 ], which studied the accuracy of screening tests (e.g., visually identifying early signs and symptoms) for active pulmonary tuberculosis in children. Figure  1 depicts the forest plots of the sensitivity and specificity measurements.

Forest plots of sensitivity (left) and specificity (right) of the meta-analysis from Vonasek et al. (2021) [ 12 ]. The a and b in Schwoebel 2020 denote the two distinct screening tests, “One or more of cough, fever, or poor weight gain in tuberculosis contacts” and “One or more of cough, fever, or decreased playfulness in children aged under five years, inpatient or outpatient,” respectively, utilized in the study

The meta-analysis of Vonasek et al. [ 12 ] included 19 studies with no indication of sparsity in either Se or Sp; that is, none of the included primary studies had observed Se or Sp close to 0 or 1. The average number of diseased ( \(n_1\) ) and non-diseased ( \(n_2\) ) participants were about 99 and 11,058, respectively, where the average \(n_2\) was affected by four potentially outlier studies whose respective number of non-diseased participants were 1,903 [ 13 ], 1,903 [ 13 ], 1,336 [ 14 ], and 200,580 [ 15 ]. In Illustrative examples  section, we will demonstrate how the three computational algorithms deal with the data since the existence of such outlying studies may potentially distort the meta-analysis results.

In the second example, we present the study by Jullien et al. (2020) that studied the diagnosing characteristics of “Rapid diagnostic tests for plague” [ 16 ]. As can be seen from the forest plots presented in Fig.  2 , this meta-analysis contained only nine studies and the average number of diseased and non-diseased participants were 188 and 223, respectively, with no indication of potentially outlying studies.

Forest plots of sensitivity (left) and specificity (right) of the meta-analysis from Jullien et al. (2020) [ 16 ]

However, the second meta-analysis had some sparse data, particularly in the diseased group. There were 4/9 (44%) primary studies with 100% sensitivity (i.e., with \(FN=0\) ). Thus, we will revisit this data set in Illustrative examples  section to examine how the numerical methods perform in the context of sparse DTAs.

In this Section, we describe the commonly used conventional meta-analytic model for ADMA of DTAs, the three computational methods used to estimate the parameters of this model and methods for our simulation study.

The standard model

The bivariate binomial-normal (BBN) model is a bivariate random-effects model first developed by Chu and Cole [ 4 ]. The BBN model assumes the binomial distribution for modelling the within-study variability and the bivariate normal distribution for modelling the between-study variability in Se and Sp across studies. The BBN is generally accepted as the preferred model for ADMA of DTAs because it models the within-study variability using the exact Binomial distribution, instead of approximating it with the normal distribution, and it does not require an ad hoc continuity correction when any of the four cell frequencies in a DTA contain zero counts. If we let \(\textbf{y}_i = [\text {logit}(Se_i), \text {logit}(Sp_i)]'\) denote the study-specific logit-transformed sensitivity and specificity vector, \(\textbf{b}_i\) the study-specific random-effects, \(\varvec{\mu }\) the pooled sensitivity and specificity vector, and \(\varvec{\Sigma }\) the between-study heterogeneity parameter, the marginal likelihood function of the BBN model can be given as in equation 1 . However, since this likelihood does not have closed-form expression because the integral cannot be evaluated analytically in a closed-form [ 4 ], one needs to use numerical approximation methods to estimate the likelihood.

where \(i=1,...,k\) denotes the i -th study in the meta-analysis.

The AGHQ [ 6 ] is a numerical method used to approximate log-likelihoods by numerical integration to obtain the MLEs of model parameters. Although estimation becomes more precise as the number of quadrature points increases, it often gives rise to computational difficulties for high-dimension random effects and convergence problems where variances are close to zero or cluster sizes are small [ 6 ]. Most of the time, the AGHQ [ 6 ] is the default estimation method and is regarded as the most accurate. Nonetheless, the LA [ 6 ] which is the Gauss-Hermite quadrature of order one [ 17 ] and the IRLS [ 7 , 8 ] that aims to find the solution to a weighted least squares iteratively, can also be used to find MLEs and usually have lower computational difficulties and faster computational speed.

Simulation study design

Data simulation.

To compare the three computational methods for each combination of model parameter settings, we simulated data based on each simulation scenario and fitted the BBN model using the AGHQ, LA, and IRLS algorithms. To inform our simulations, we scraped the Cochrane Database of Systematic Reviews and selected 64 reviews containing meta-analyses data. Unwrapping these reviews and performing data cleaning gave us access to 393 meta-analyses covering a wide range of medical diagnosis tests. We fitted the BBN model to each of the 393 meta-analyses to obtain the empirical distribution of the model parameters. Based on these results, we defined our true parameter settings as shown in Table 1 . Following Ju et al. (2020) [ 9 ] and Jackson et al. (2018) [ 18 ], we introduced sparsity into the meta-analysis by considering large values of ( Se ,  Sp ).

Accordingly, we considered a total of \(3^4\times 4 = 324\) total scenarios in our simulation study. For each parameter combination, we conducted our simulation study by (1) simulating 1000 DTA data based on normal random effects following the steps described by Negeri and Beyene [ 19 ], (2) fitting the BBN model to each simulated data using the three computational methods, and (3) comparing the estimated results by each numerical method with the true values in terms of absolute bias, RMSE, CI width, coverage probability, convergence rate, and computing time.

We used the R statistical language [ 20 ] version 4.2.2 and RStudio [ 21 ] version 2023.09.0+463 for all data analyses. We utilized the glmer() function from the lme4 R package [ 22 ] to apply the IRLS and LA by setting nAGQ to 0 and 1, respectively. We fitted the BBN model with the AGHQ algorithm using the mixed_model() function from the GLMMadaptive R package [ 23 ] by setting the number of quadrature points used in the approximation (nAGQ) to 5.

Performance evaluation criteria

In our simulation study, we defined the convergence rate of the BBN model as the number of converged fits over the total number of fits in an iteration. We counted fits with non-positive semi-definite covariance matrices and fits that did not meet optimality conditions as non-converging. While assessing the convergence rate, we found that the “converged” message provided in the model summary from the glmer() function is sometimes non-trustable. For example, we saw a warning message such as “boundary (singular) fit: see help(’isSingular’)” when fitting the BBN model, which indicates a fit that did not converge, but the “converged” option wrongly provided convergence. Thus, we treated those singular fits as non-convergence to calculate the convergence rate. We measured the computing speed for each numerical method using R ’s built-in function system.time() . The remaining metrics, such as the absolute bias, RMSE, coverage probability, and CI width were calculated following Burton et al. (2006) [ 24 ] and Morris et al. (2019) [ 25 ].

Simulation study results

In this Section, we use the different metrics described in Methods  section to evaluate the performance of the three computational methods and summarize our simulation study findings by metrics. Note that the solid line is IRLS, the dashed line is LA, the dotted line is AGHQ, and that the lines might overlap for some scenarios when there is no difference in results between the three computational methods.

Absolute bias

Figure 3 depicts the bias of the three computational methods for sensitivity and specificity. We found that when the true Se and Sp were far from perfect, there was barely any difference among these three numerical methods as the three lines overlap for the first two columns. However, for all variance-covariance settings, the LA had the largest absolute bias compared to the AGHQ and the IRLS (Fig.  3 , third pane). Moreover, when data is sparse (i.e. large Se and Sp closer to 100%), the IRLS and AGHQ were comparable, although IRLS had a slightly larger absolute bias. We observed consistent results for the other scenarios considered in our simulations (see the Appendix figures).

Bias for sensitivity (Se) and specificity (Sp) based on the IRLS (solid line), Laplace approximation (dashed line) and Gauss-Hermite quadrature (dotted line) when \(\sigma _1^2=1.59\) , \(\sigma _2^2=1.83\) , \(\sigma _{12}=-0.34\) , \(n_1=300\) , and \(n_2=500\)

Similarly, the three computational methods had comparable performance when it comes to the bias of the between-study variances \(\sigma _{1}^2\) and \(\sigma _{2}^2\) for relatively small Se and Sp (Fig.  4 , first two panes). However, for sparse DTA data (large Se and Sp), the LA still had the largest absolute bias, and the AGHQ had the smallest bias for between-study variances. Similar results were found for the other scenarios examined in our simulations (see the Appendix figures).

Bias for between-study variances based on the IRLS (solid line), Laplace approximation (dashed line) and Gauss-Hermite quadrature (dotted line) when \(\sigma _1^2=1.59\) , \(\sigma _2^2=1.83\) , \(\sigma _{12}=-0.34\) , \(n_1=300\) , and \(n_2=500\)

Root mean squared error (RMSE)

Concerning RMSE (Fig.  5 ), we observed a similar trend to bias. That is, the three numerical methods were comparable when the DTA data was not sparse, but the LA yielded larger RMSE for all (Se, Sp) pairs. Furthermore, the IRLS and the AGHQ were comparable, although the AGHQ had a slightly larger RMSE. Consistent results were observed for the other scenarios considered in our simulations (see the Appendix figures).

RMSE for sensitivity (Se) and specificity (Sp) based on the IRLS (solid line), Laplace approximation (dashed line) and Gauss-Hermite quadrature (dotted line) when \(\sigma _1^2=1.59\) , \(\sigma _2^2=1.83\) , \(\sigma _{12}=-0.34\) , \(n_1=300\) , and \(n_2=500\)

Confidence interval (CI) width and coverage

For CI width (Fig.  6 ), the three numerical methods gave almost the same results when the true Se and Sp were small. However, there were marginal differences among the computational methods when DTA was sparse, as the IRLS had the smallest CI width for specificity and the LA yielded the smallest CI width for sensitivity. Moreover, as Se or Sp increased, the width of the CI decreased.

CI width for sensitivity (Se) and specificity (Sp) based on the IRLS (solid line), Laplace approximation (dashed line) and Gauss-Hermite quadrature (dotted line) when \(\sigma _1^2=1.59\) , \(\sigma _2^2=1.83\) , \(\sigma _{12}=-0.34\) , \(n_1=300\) , and \(n_2=500\)

Figure  7 presents the coverage probabilities of the three computational methods. Similar to the other metrics, the AGHQ, LA, and IRLS had comparable coverage probability when data were not sparse (i.e., small Se and Sp). However, the LA had the smallest coverage probability for sparse DTA data compared to the other two methods, and the AGHQ had a slightly larger coverage than the IRLS. Moreover, as the number of studies in a meta-analysis increased, the coverage probability of the methods decreased. We found similar results for the other simulation scenarios considered in our simulations (see the Appendix figures).

Coverage for sensitivity (Se) and specificity (Sp) based on the IRLS (solid line), Laplace approximation (dashed line) and Gauss-Hermite quadrature (dotted line) when \(\sigma _1^2=1.59\) , \(\sigma _2^2=1.83\) , \(\sigma _{12}=-0.34\) , \(n_1=300\) , and \(n_2=500\)

Convergence rate and computing time

Table 2 depicts the average convergence rate, average computing time, and the interquartile range (IQR) for computing time across all simulation scenarios for the three computational methods. Accordingly, on average, the AGHQ had the highest convergence rate but the longest computing time compared to the two methods. We also observed that longer computing times were associated with higher convergence rates. Moreover, the AGHQ also had the largest IQR of the three numerical methods.

Illustrative examples

This Section summarizes the results of fitting the BBN model to the two motivating examples presented in Motivating examples  section using the three computational algorithms.

Table 3 summarizes the results of applying the numerical algorithms to the Vonasek et al. (2021) [ 12 ] data. All three numerical algorithms converged to the MLEs. The AGHQ estimated both the pooled Se and pooled Sp very differently than the other two methods. The LA and IRLS approaches resulted in similar pooled Se and pooled Sp estimates, with their pooled Sp closer to the observed specificities of the outlying studies identified in Motivating examples  section than the non-outlying studies, indicating that the LA and IRLS estimates may be influenced by outlying studies [ 2 , 3 ]. These results suggest that the AGHQ yielded estimates that were less affected by the outlying studies in specificity. However, all three methods yielded comparable between-study variance-covariance estimates.

We present the results of fitting the BBN model to the meta-analysis of Jullien et al. (2020) [ 16 ] in Table 4 . The AGHQ algorithm failed to converge with its Hessian matrix being non-positive-definite. Despite that, all three methods produced comparable pooled Se and Sp estimates, \(\sigma _{12}\) and \(\sigma _2^2\) . However, the LA produced a very large between-study variance of logit-transformed sensitivity \((\sigma _1^2)\) , which could be attributed to the apparent data sparsity among the diseased participants, consistent with our simulation study results.

In this study, we compared three commonly used computational algorithms, the AGHQ, the LA, and the IRLS, that numerically approximate the log-likelihood function of a bivariate GLMM for ADMA of DTAs. To determine which method is more appropriate in practice, we compared the performance of these methods using extensive simulation studies and real-life data sets. Our simulation settings were informed after analyzing 393 real-life meta-analyses from the Cochrane Database of Systematic Reviews.

In almost all of our simulation scenarios, we observed no obvious difference among the three numerical methods when Se and Sp were relatively small and not close to 100%. However, when the DTA data were sparse or equivalently when Se and Sp were both large and close to 100%, there were appreciable differences among these three computational algorithms. The LA usually had the largest absolute bias and RMSE but the smallest coverage probability for Se and Sp compared to the IRLS and the AGHQ. The IRLS and AGHQ were comparable, but IRLS had the smallest convergence rate. Though the AGHQ had the largest convergence rate among the three algorithms, it had the longest computing time.

Unlike the results reported by Ju et al. (2020) [ 9 ] for meta-analysis of rare intervention studies, we found appreciable differences in bias and RMSE of the LA and the AGHQ for sparse data, albeit in the context of ADMA of DTAs. However, we were not able to make similar comparisons in terms of the between-study variances since it wasn’t reported in their study. Similarly, a comparison was impossible between our findings and those of Thomas et al. (2017) [ 10 ] since the latter study evaluated only the AGHQ, not the LA and IRLS algorithms.

Our real-life data analyses also revealed consistent results with our simulation studies. The AGHQ produced robust pooled Se and Sp estimates when applied to DTA data with a few outlying studies. The LA yielded the largest between-study variance estimates when a GLMM was fitted to sparse DTA data. Although the PQL approach has been discouraged by other researchers in the context of intervention studies meta-analysis with binary outcomes [ 9 ] and is not commonly used in the context of meta-analysis of DTA studies, following a Reviewer’s suggestion, we applied it to our motivating examples data sets (see Appendix Table C 3 ) and observed inferior results consistent with that of Ju et al. [ 9 ]. Thus, we opted not to investigate its performance in our simulation study. Moreover, it was not unexpected to find the LA and IRLS algorithms affected by outliers since they utilize methods known to be prone to unusual observations – the normal distribution and least squares, respectively. Whereas the LA works by approximating the integrand of the likelihood with the normal distribution, for example, the IRLS iteratively solves a system of score equations via weighted least squares. The AGHQ approximates the entire likelihood or integral via a numerical approach known as quadrature method, making it the least sensitive approach to outliers.

The strengths of our manuscript include being the first study to report on the evaluation and comparison of commonly used computational methods for ADMA of DTAs and considering several real-life scenarios by informing our simulation study with 393 meta-analysis results from the Cochrane Database of Systematic Reviews. Thus, our study has contributed to the literature by filling an existing gap in the body of knowledge and by producing results applicable to practical real-world situations. Although we considered only the frequently used numerical methods in ADMA of DTAs, not including more than three such computational algorithms can be considered a limitation of our study, which can be pursued in a future study. For example, it is worth evaluating and validating the performance of these numerical methods in comparison with the Newton-Raphson-based algorithms [ 26 ], the many procedures implemented in the metadta Stata tool [ 27 ], or in the context of IPDMA of DTA studies with or without multiple cut-offs [ 28 ]. Moreover, the LA and IRLS algorithms appeared to be impacted by outlying studies when applied to a real-life meta-analysis. Thus, it is worth a future study investigating this issue further via a simulation study to see if this property of the two algorithms repeats for different data settings.

In summary, the IRLS, AGHQ, and the LA had similar performances for non-sparse data, but the LA performed worse for sparse DTA data sets. Whereas the AGHQ had the best convergence rate but the longest computing time, the IRLS had the shortest computing time but the worst convergence rate. Therefore, we suggest practitioners and researchers use any of the three computational methods for conducting ADMA of DTAs without sparse data. However, the LA should be avoided and either the IRLS or the AGHQ should be used when sparsity is a concern.

Availability of data and materials

All data generated or analyzed during this study will be included in this published article and its supplementary information files.

Abbreviations

Aggregate Data Meta-Analysis

Adaptive Gaussian-Hermite Quadrature

Bivariate Binomial-Normal

Confidence Interval

Diagnostic Test Accuracy

Generalized Linear Mixed Models

Individual Participant Data Meta-Analysis

Interquartile Range

Iteratively Reweighted Least Squares

Laplace Approximation

Penalized Quasi-likelihood

Root Mean Squared Error

Sensitivity

Specificity

Glass GV. Primary, secondary, and meta-analysis of research. Educ Res. 1976;5(10):3–8.

Article   Google Scholar  

Negeri ZF, Beyene J. Statistical methods for detecting outlying and influential studies in meta-analysis of diagnostic test accuracy studies. Stat Methods Med Res. 2020;29(4):1227–42.

Article   PubMed   Google Scholar  

Negeri ZF, Beyene J. Robust bivariate random-effects model for accommodating outlying and influential studies in meta-analysis of diagnostic test accuracy studies. Stat Methods Med Res. 2020;29(11):3308–25.

Chu H, Cole SR. Bivariate meta-analysis of sensitivity and specificity with sparse data: a generalized linear mixed model approach. J Clin Epidemiol. 2006;59(12):1331–2.

Reitsma JB, Glas AS, Rutjes AW, Scholten RJ, Bossuyt PM, Zwinderman AH. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol. 2005;58(10):982–90.

Pinheiro JC, Bates DM. Approximations to the log-likelihood function in the nonlinear mixed-effects model. J Comput Graph Stat. 1995;4(1):12–35.

Jorgensen M. Iteratively reweighted least squares. Encycl Environmetrics. 2006.

Burrus CS. Iterative reweighted least squares. 2012;12. OpenStax CNX Available online: http://cnx.org/contents/92b90377-2b34-49e4-b26f-7fe572db78a1 . Accessed 15 Nov 2023.

Ju K, Lin L, Chu H, Cheng LL, Xu C. Laplace approximation, penalized quasi-likelihood, and adaptive Gauss-Hermite quadrature for generalized linear mixed models: Towards meta-analysis of binary outcome with sparse data. BMC Med Res Methodol. 2020;20(1):1–11.

Thomas D, Platt R, Benedetti A. A comparison of analytic approaches for individual patient data meta-analyses with binary outcomes. BMC Med Res Methodol. 2017;17:1–12.

Rücker G. Network Meta-Analysis of Diagnostic Test Accuracy Studies. In: Biondi-Zoccai G, editor. Diagnostic Meta-Analysis. Cham: Springer; 2018. pp. 183–97.

Vonasek B, Ness T, Takwoingi Y, Kay AW, van Wyk SS, Ouellette L, et al. Screening tests for active pulmonary tuberculosis in children. Cochrane Database Syst Rev. 2021;(6). Art. No.: CD013693.

Schwoebel V, Koura KG, Adjobimey M, Gnanou S, Wandji AG, Gody J-C, et al. Tuberculosis contact investigation and short-course preventive therapy among young children in Africa. Int J Tuberc Lung Dis. 2020;24(4):452–60.

Article   CAS   PubMed   Google Scholar  

Sawry S, Moultrie H, Van Rie A. Evaluation of the intensified tuberculosis case finding guidelines for children living with HIV. Int J Tuberc Lung Dis. 2018;22(11):1322–8.

Vonasek B, Kay A, Devezin T, Bacha JM, Kazembe P, Dhillon D, et al. Tuberculosis symptom screening for children and adolescents living with HIV in six high HIV/TB burden countries in Africa. AIDS. 2021;35(1):73–9.

Jullien S, Dissanayake HA, Chaplin M. Rapid diagnostic tests for plague. Cochrane Database Syst. Rev. 2020;(6). Art. No.: CD013459.

Liu Q, Pierce DA. A note on Gauss-Hermite quadrature. Biometrika. 1994;81(3):624–9.

Google Scholar  

Jackson D, Law M, Stijnen T, Viechtbauer W, White IR. A comparison of seven random-effects models for meta-analyses that estimate the summary odds ratio. Stat Med. 2018;37(7):1059–85.

Article   PubMed   PubMed Central   Google Scholar  

Negeri ZF, Beyene J. Skew-normal random-effects model for meta-analysis of diagnostic test accuracy (DTA) studies. Biom J. 2020;62(5):1223–44.

R Core Team. R: A Language and Environment for Statistical Computing. Vienna; 2022. https://www.R-project.org/ . Accessed 10 Sept 2023.

RStudio Team. RStudio: Integrated Development Environment for R. Boston; 2020. http://www.rstudio.com/ . Accessed 10 Sept 2023.

Bates D, Mächler M, Bolker B, Walker S. Fitting Linear Mixed-Effects Models Using lme4. J Stat Softw. 2015;67(1):1–48. https://doi.org/10.18637/jss.v067.i01 .

Rizopoulos D. GLMMadaptive: Generalized Linear Mixed Models using Adaptive Gaussian Quadrature. 2023. R package version 0.9-1. https://CRAN.R-project.org/package=GLMMadaptive . Accessed 10 Sept 2023.

Burton A, Altman DG, Royston P, Holder RL. The design of simulation studies in medical statistics. Stat Med. 2006;25(24):4279–92.

Morris TP, White IR, Crowther MJ. Using simulation studies to evaluate statistical methods. Stat Med. 2019;38(11):2074–102.

Willis BH, Baragilly M, Coomar D. Maximum likelihood estimation based on Newton-Raphson iteration for the bivariate random effects model in test accuracy meta-analysis. Stat Methods Med Res. 2020;29(4):1197–211.

Nyaga VN, Arbyn M. Comparison and validation of metadta for meta-analysis of diagnostic test accuracy studies. Res Synth Methods. 2023;14(3):544–62.

Negeri ZF, Levis B, Ioannidis JP, Thombs BD, Benedetti A. An empirical comparison of statistical methods for multiple cut-off diagnostic test accuracy meta-analysis of the Edinburgh postnatal depression scale (EPDS) depression screening tool using published results vs individual participant data. BMC Med Res Methodol. 2024;24(1):28.

Download references

Acknowledgements

We are grateful to the Faculty of Mathematics, University of Waterloo, for providing us with computing resources.

Dr. Negeri, Yixin Zhao (through Dr. Negeri) and Bilal Khan (through Dr. Negeri) were supported by the University of Waterloo’s New Faculty Start-Up Grant. Bilal Khan was also supported by the University of Waterloo’s NSERC USRA award.

Author information

Yixin Zhao and Bilal Khan contributed equally to this work.

Authors and Affiliations

Department of Statistics and Actuarial Science, University of Waterloo, 200 University Ave W, Waterloo, N2L 3G1, Ontario, Canada

Yixin Zhao, Bilal Khan & Zelalem F. Negeri

You can also search for this author in PubMed   Google Scholar

Contributions

ZN contributed to the conception and design of the study, participated in data analyses, and provided critical revisions to the manuscript. YZ contributed to the writing of R code for data analyses, running and summarizing of the simulation study, and drafting of the manuscript; BK contributed to the writing of R code for data analyses, scraping the Cochrane Database of Systematic Reviews and designing of the simulation study, and drafting of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Zelalem F. Negeri .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1..

Absolute bias, RMSE, CI width, and coverage probabilities of the three computational methods for additional simulation scenarios.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Zhao, Y., Khan, B. & Negeri, Z. An evaluation of computational methods for aggregate data meta-analyses of diagnostic test accuracy studies. BMC Med Res Methodol 24 , 111 (2024). https://doi.org/10.1186/s12874-024-02217-2

Download citation

Received : 01 January 2024

Accepted : 15 April 2024

Published : 10 May 2024

DOI : https://doi.org/10.1186/s12874-024-02217-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Meta-analysis
• Diagnostic test accuracy
• Generalized linear mixed models
• Computational methods
• Adaptive Gauss-Hermite
• Laplace approximation

BMC Medical Research Methodology

ISSN: 1471-2288

• General enquiries: [email protected]

• Open access
• Published: 13 May 2024

Patient medication management, understanding and adherence during the transition from hospital to outpatient care - a qualitative longitudinal study in polymorbid patients with type 2 diabetes

• Léa Solh Dost   ORCID: orcid.org/0000-0001-5767-1305 1 , 2 ,
• Giacomo Gastaldi   ORCID: orcid.org/0000-0001-6327-7451 3 &
• Marie P. Schneider   ORCID: orcid.org/0000-0002-7557-9278 1 , 2  

BMC Health Services Research volume  24 , Article number:  620 ( 2024 ) Cite this article

Metrics details

Continuity of care is under great pressure during the transition from hospital to outpatient care. Medication changes during hospitalization may be poorly communicated and understood, compromising patient safety during the transition from hospital to home. The main aims of this study were to investigate the perspectives of patients with type 2 diabetes and multimorbidities on their medications from hospital discharge to outpatient care, and their healthcare journey through the outpatient healthcare system. In this article, we present the results focusing on patients’ perspectives of their medications from hospital to two months after discharge.

Patients with type 2 diabetes, with at least two comorbidities and who returned home after discharge, were recruited during their hospitalization. A descriptive qualitative longitudinal research approach was adopted, with four in-depth semi-structured interviews per participant over a period of two months after discharge. Interviews were based on semi-structured guides, transcribed verbatim, and a thematic analysis was conducted.

Twenty-one participants were included from October 2020 to July 2021. Seventy-five interviews were conducted. Three main themes were identified: (A) Medication management, (B) Medication understanding, and (C) Medication adherence, during three periods: (1) Hospitalization, (2) Care transition, and (3) Outpatient care. Participants had varying levels of need for medication information and involvement in medication management during hospitalization and in outpatient care. The transition from hospital to autonomous medication management was difficult for most participants, who quickly returned to their routines with some participants experiencing difficulties in medication adherence.

Conclusions

The transition from hospital to outpatient care is a challenging process during which discharged patients are vulnerable and are willing to take steps to better manage, understand, and adhere to their medications. The resulting tension between patients’ difficulties with their medications and lack of standardized healthcare support calls for interprofessional guidelines to better address patients’ needs, increase their safety, and standardize physicians’, pharmacists’, and nurses’ roles and responsibilities.

Peer Review reports

Introduction

Continuity of patient care is characterized as the collaborative engagement between the patient and their physician-led care team in the ongoing management of healthcare, with the mutual objective of delivering high-quality and cost-effective medical care [ 1 ]. Continuity of care is under great pressure during the transition of care from hospital to outpatient care, with a risk of compromising patients’ safety [ 2 , 3 ]. The early post-discharge period is a high-risk and fragile transition: once discharged, one in five patients experience at least one adverse event during the first three weeks following discharge, and more than half of these adverse events are drug-related [ 4 , 5 ]. A retrospective study examining all discharged patients showed that adverse drug events (ADEs) account for up to 20% of 30-day hospital emergency readmissions [ 6 ]. During hospitalization, patients’ medications are generally modified, with an average of nearly four medication changes per patient [ 7 ]. Information regarding medications such as medication changes, the expected effect, side effects, and instructions for use are frequently poorly communicated to patients during hospitalization and at discharge [ 8 , 9 , 10 , 11 ]. Between 20 and 60% of discharged patients lack knowledge of their medications [ 12 , 13 ]. Consideration of patients’ needs and their active engagement in decision-making during hospitalization regarding their medications are often lacking [ 11 , 14 , 15 ]. This can lead to unsafe discharge and contribute to medication adherence difficulties, such as non-implementation of newly prescribed medications [ 16 , 17 ].

Patients with multiple comorbidities and polypharmacy are at higher risk of ADE [ 18 ]. Type 2 diabetes is one of the chronic health conditions most frequently associated with comorbidities and patients with type 2 diabetes often lack care continuum [ 19 , 20 , 21 ]. The prevalence of patients hospitalized with type 2 diabetes can exceed 40% [ 22 ] and these patients are at higher risk for readmission due to their comorbidities and their medications, such as insulin and oral hypoglycemic agents [ 23 , 24 , 25 ].

Interventions and strategies to improve patient care and safety at transition have shown mixed results worldwide in reducing cost, rehospitalization, ADE, and non-adherence [ 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 ]. However, interventions that are patient-centered, with a patient follow-up and led by interprofessional healthcare teams showed promising results [ 34 , 35 , 36 ]. Most of these interventions have not been implemented routinely due to the extensive time to translate research into practice and the lack of hybrid implementation studies [ 37 , 38 , 39 , 40 , 41 ]. In addition, patient-reported outcomes and perspectives have rarely been considered, yet patients’ involvement is essential for seamless and integrated care [ 42 , 43 ]. Interprofessional collaboration in which patients are full members of the interprofessional team, is still in its infancy in outpatient care [ 44 ]. Barriers and facilitators regarding medications at the transition of care have been explored in multiple qualitative studies at one given time in a given setting (e.g., at discharge, one-month post-discharge) [ 8 , 45 , 46 , 47 , 48 ]. However, few studies have adopted a holistic methodology from the hospital to the outpatient setting to explore changes in patients’ perspectives over time [ 49 , 50 , 51 ]. Finally, little is known about whether, how, and when patients return to their daily routine following hospitalization and the impact of hospitalization weeks after discharge.

In Switzerland, continuity of care after hospital discharge is still poorly documented, both in terms of contextual analysis and interventional studies, and is mainly conducted in the hospital setting [ 31 , 35 , 52 , 53 , 54 , 55 , 56 ]. The first step of an implementation science approach is to perform a contextual analysis to set up effective interventions adapted to patients’ needs and aligned to healthcare professionals’ activities in a specific context [ 41 , 57 ]. Therefore, the main aims of this study were to investigate the perspectives of patients with type 2 diabetes and multimorbidities on their medications from hospital discharge to outpatient care, and on their healthcare journey through the outpatient healthcare system. In this article, we present the results focusing on patients’ perspectives of their medications from hospital to two months after discharge.

Study design

This qualitative longitudinal study, conducted from October 2020 to July 2021, used a qualitative descriptive methodology through four consecutive in-depth semi-structured interviews per participant at three, 10-, 30- and 60-days post-discharge, as illustrated in Fig.  1 . Longitudinal qualitative research is characterized by qualitative data collection at different points in time and focuses on temporality, such as time and change [ 58 , 59 ]. Qualitative descriptive studies aim to explore and describe the depth and complexity of human experiences or phenomena [ 60 , 61 , 62 ]. We focused our qualitative study on the 60 first days after discharge as this period is considered highly vulnerable and because studies often use 30- or 60-days readmission as an outcome measure [ 5 , 63 ].

This qualitative study follows the Consolidated Criteria for Reporting Qualitative Research (COREQ). Ethics committee approval was sought and granted by the Cantonal Research Ethics Commission, Geneva (CCER) (2020 − 01779).

Recruitment took place during participants’ hospitalization in the general internal medicine divisions at the Geneva University Hospitals in the canton of Geneva (500 000 inhabitants), Switzerland. Interviews took place at participants’ homes, in a private office at the University of Geneva, by telephone or by secure video call, according to participants’ preference. Informal caregivers could also participate alongside the participants.

Study flowchart

Researcher characteristics

All the researchers were trained in qualitative studies. The diabetologist and researcher (GG) who enrolled the patients in the study was involved directly or indirectly (advice asked to the Geneva University Hospital diabetes team of which he was a part) for most participants’ care during hospitalization. LS (Ph.D. student and community pharmacist) was unknown to participants and presented herself during hospitalization as a “researcher” and not as a healthcare professional to avoid any risk of influencing participants’ answers. This study was not interventional, and the interviewer (LS) invited participants to contact a healthcare professional for any questions related to their medication or medical issues.

Population and sampling strategy

Patients with type 2 diabetes were chosen as an example population to describe polypharmacy patients as these patients usually have several health issues and polypharmacy [ 20 , 22 , 25 ]. Inclusions criteria for the study were: adult patients with type 2 diabetes, with at least two other comorbidities, hospitalized for at least three days in a general internal medicine ward, with a minimum of one medication change during hospital stay, and who self-managed their medications once discharged home. Exclusion criteria were patients not reachable by telephone following discharge, unable to give consent (patients with schizophrenia, dementia, brain damage, or drug/alcohol misuse), and who could not communicate in French. A purposive sampling methodology was applied aiming to include participants with different ages, genders, types, and numbers of health conditions by listing participants’ characteristics in a double-entry table, available in Supplementary Material 1 , until thematic saturation was reached. Thematic saturation was considered achieved when no new code or theme emerged and new data repeated previously coded information [ 64 ]. The participants were identified if they were hospitalized in the ward dedicated to diabetes care or when the diabetes team was contacted for advice. The senior ward physician (GG) screened eligible patients and the interviewer (LS) obtained written consent before hospital discharge.

Data collection and instruments

Sociodemographic (age, gender, educational level, living arrangement) and clinical characteristics (reason for hospitalization, date of admission, health conditions, diabetes diagnosis, medications before and during hospitalization) were collected by interviewing participants before their discharge and by extracting participants’ data from electronic hospital files by GG and LS. Participants’ pharmacies were contacted with the participant’s consent to obtain medication records from the last three months if information regarding medications before hospitalization was missing in the hospital files.

Semi-structured interview guides for each interview (at three, 10-, 30- and 60-days post-discharge) were developed based on different theories and components of health behavior and medication adherence: the World Health Organization’s (WHO) five dimensions for adherence, the Information-Motivation-Behavioral skills model and the Social Cognitive Theory [ 65 , 66 , 67 ]. Each interview explored participants’ itinerary in the healthcare system and their perspectives on their medications. Regarding medications, the following themes were mentioned at each interview: changes in medications, patients’ understanding and implication; information on their medications, self-management of their medications, and patients’ medication adherence. Other aspects were mentioned in specific interviews: patients’ hospitalization and experience on their return home (interview 1), motivation (interviews 2 and 4), and patient’s feedback on the past two months (interview 4). Interview guides translated from French are available in Supplementary Material 2 . The participants completed self-reported and self-administrated questionnaires at different interviews to obtain descriptive information on different factors that may affect medication management and adherence: self-report questionnaires on quality of life (EQ-5D-5 L) [ 68 ], literacy (Schooling-Opinion-Support questionnaire) [ 69 ], medication adherence (Adherence Visual Analogue Scale, A-VAS) [ 70 ] and Belief in Medication Questionnaire (BMQ) [ 71 ] were administered to each participant at the end of selected interviews to address the different factors that may affect medication management and adherence as well as to determine a trend of determinants over time. The BMQ contains two subscores: Specific-Necessity and Specific-Concerns, addressing respectively their perceived needs for their medications, and their concerns about adverse consequences associated with taking their medication [ 72 ].

Data management

Informed consent forms, including consent to obtain health data, were securely stored in a private office at the University of Geneva. The participants’ identification key was protected by a password known only by MS and LS. Confidentiality was guaranteed by pseudonymization of participants’ information and audio-recordings were destroyed once analyzed. Sociodemographic and clinical characteristics, medication changes, and answers to questionnaires were securely collected by electronic case report forms (eCRFs) on RedCap®. Interviews were double audio-recorded and field notes were taken during interviews. Recorded interviews were manually transcribed verbatim in MAXQDA® (2018.2) by research assistants and LS and transcripts were validated for accuracy by LS. A random sample of 20% of questionnaires was checked for accuracy for the transcription from the paper questionnaires to the eCRFs. Recorded sequences with no link to the discussed topics were not transcribed and this was noted in the transcripts.

Data analysis

A descriptive statistical analysis of sociodemographic, clinical characteristics and self-reported questionnaire data was carried out. A thematic analysis of transcripts was performed, as described by Braun and Clarke [ 73 ], by following six steps: raw data was read, text segments related to the study objectives were identified, text segments to create new categories were identified, similar or redundant categories were reduced and a model that integrated all significant categories was created. The analysis was conducted in parallel with patient enrolment to ensure data saturation. To ensure the validity of the coding method, transcripts were double coded independently and discussed by the research team until similar themes were obtained. The research group developed and validated an analysis grid, with which LS coded systematically the transcriptions and met regularly with the research team to discuss questions on data analysis and to ensure the quality of coding. The analysis was carried out in French, and the verbatims of interest cited in the manuscript were translated and validated by a native English-speaking researcher to preserve the meaning.

In this analysis, we used the term “healthcare professionals” when more than one profession could be involved in participants’ medication management. Otherwise, when a specific healthcare professional was involved, we used the designated profession (e.g. physicians, pharmacists).

Patient and public involvement

During the development phase of the study, interview guides and questionnaires were reviewed for clarity and validity and adapted by two patient partners, with multiple health conditions and who experienced previously a hospital discharge. They are part of the HUG Patients Partners + 3P platform for research and patient and public involvement.

Interviews and participants’ descriptions

A total of 75 interviews were conducted with 21 participants. In total, 31 patients were contacted, seven refused to participate (four at the project presentation and three at consent), two did not enter the selection criteria at discharge and one was unreachable after discharge. Among the 21 participants, 15 participated in all interviews, four in three interviews, one in two interviews, and one in one interview, due to scheduling constraints. Details regarding interviews and participants characteristics are presented in Tables  1 and 2 .

The median length of time between hospital discharge and interviews 1,2,3 and 4 was 5 (IQR: 4–7), 14 (13-20), 35 (22-38), and 63 days (61-68), respectively. On average, by comparing medications at hospital admission and discharge, a median of 7 medication changes (IQR: 6–9, range:2;17) occurred per participant during hospitalization and a median of 7 changes (5–12) during the two months following discharge. Details regarding participants’ medications are described in Table  3 .

Patient self-reported adherence over the past week for their three most challenging medications are available in Supplementary Material 3 .

Qualitative analysis

We defined care transition as the period from discharge until the first medical appointment post-discharge, and outpatient care as the period starting after the first medical appointment. Data was organized into three key themes (A. Medication management, B. Medication understanding, and C. Medication adherence) divided into subthemes at three time points (1. Hospitalization, 2. Care transition and 3. Outpatient care). Figure  2 summarizes and illustrates the themes and subthemes with their influencing factors as bullet points.

Participants’ medication management, understanding and adherence during hospitalization, care transition and outpatient care

A. Medication management

A.1 medication management during hospitalization: medication management by hospital staff.

Medications during hospitalization were mainly managed by hospital healthcare professionals (i.e. nurses and physicians) with varying degrees of patient involvement: “At the hospital, they prepared the medications for me. […] I didn’t even know what the packages looked like.” Participant 22; interview 1 (P22.1) Some participants reported having therapeutic education sessions with specialized nurses and physicians, such as the explanation and demonstration of insulin injection and glucose monitoring. A patient reported that he was given the choice of several treatments and was involved in shared decision-making. Other participants had an active role in managing and optimizing dosages, such as rapid insulin, due to prior knowledge and use of medications before hospitalization.

A.2 Medication management at transition: obtaining the medication and initiating self-management

Once discharged, some participants had difficulties obtaining their medications at the pharmacy because some medications were not stored and had to be ordered, delaying medication initiation. To counter this problem upstream, a few participants were provided a 24-to-48-hour supply of medications at discharge. It was sometimes requested by the patient or suggested by the healthcare professionals but was not systematic. The transition from medication management by hospital staff to self-management was exhausting for most participants who were faced with a large amount of new information and changes in their medications: “ When I was in the hospital, I didn’t even realize all the changes. When I came back home, I took away the old medication packages and got out the new ones. And then I thought : « my God, all this…I didn’t know I had all these changes » ” P2.1 Written documentation, such as the discharge prescription or dosage labels on medication packages, was helpful in managing their medication at home. Most participants used weekly pill organizers to manage their medications, which were either already used before hospitalization or were introduced post-discharge. The help of a family caregiver in managing and obtaining medications was reported as a facilitator.

A.3 Medication management in outpatient care: daily self-management and medication burden

A couple of days or weeks after discharge, most participants had acquired a routine so that medication management was less demanding, but the medication burden varied depending on the participants. For some, medication management became a simple action well implemented in their routine (“It has become automatic” , P23.4), while for others, the number of medications and the fact that the medications reminded them of the disease was a heavy burden to bear on a daily basis (“ During the first few days after getting out of the hospital, I thought I was going to do everything right. In the end, well [laughs] it’s complicated. I ended up not always taking the medication, not monitoring the blood sugar” P12.2) To support medication self-management, some participants had written documentation such as treatment plans, medication lists, and pictures of their medication packages on their phones. Some participants had difficulties obtaining medications weeks after discharge as discharge prescriptions were not renewable and participants did not see their physician in time. Others had to visit multiple physicians to have their prescriptions updated. A few participants were faced with prescription or dispensing errors, such as prescribing or dispensing the wrong dosage, which affected medication management and decreased trust in healthcare professionals. In most cases, according to participants, the pharmacy staff worked in an interprofessional collaboration with physicians to provide new and updated prescriptions.

B. Medication understanding

B.1 medication understanding during hospitalization: new information and instructions.

The amount of information received during hospitalization varied considerably among participants with some reporting having received too much, while others saying they received too little information regarding medication changes, the reason for changes, or for introducing new medications: “They told me I had to take this medication all my life, but they didn’t tell me what the effects were or why I was taking it.” P5.3

Hospitalization was seen by some participants as a vulnerable and tiring period during which they were less receptive to information. Information and explanations were generally given verbally, making it complicated for most participants to recall it. Some participants reported that hospital staff was attentive to their needs for information and used communication techniques such as teach-back (a way of checking understanding by asking participants to say in their own words what they need to know or do about their health or medications). Some participants were willing to be proactive in the understanding of their medications while others were more passive, had no specific needs for information, and did not see how they could be engaged more.

B.2 Medication understanding at transition: facing medication changes

At hospital discharge, the most challenging difficulty for participants was to understand the changes made regarding their medications. For newly diagnosed participants, the addition of new medications was more difficult to understand, whereas, for experienced participants, changes in known medications such as dosage modification, changes within a therapeutic class, and generic substitutions were the most difficult to understand. Not having been informed about changes caused confusion and misunderstanding. Therefore, medication reconciliation done by the patient was time-consuming, especially for participants with multiple medications: “ They didn’t tell me at all that they had changed my treatment completely. They just told me : « We’ve changed a few things. But it was the whole treatment ». ” P2.3 Written information, such as the discharge prescription, the discharge report (brief letter summarizing information about the hospitalization, given to the patient at discharge), or the label on the medication box (written by the pharmacist with instructions on dosage) helped them find or recall information about their medications and diagnoses. However, technical terms were used in hospital documentations and were not always understandable. For example, this participant said: “ On the prescription of valsartan, they wrote: ‘resume in the morning once profile…’[once hypertension profile allows]… I don’t know what that means.” P8.1 In addition, some documents were incomplete, as mentioned by a patient who did not have the insulin dosage mentioned on the hospital prescription. Some participants sought help from healthcare professionals, such as pharmacists, hospital physicians, or general practitioners a few days after discharge to review medications, answer questions, or obtain additional information.

B.3 Medication understanding in the outpatient care: concerns and knowledge

Weeks after discharge, most participants had concerns about the long-term use of their medications, their usefulness, and the possible risk of interactions or side effects. Some participants also reported having some lack of knowledge regarding indications, names, or how the medication worked: “I don’t even know what Brilique® [ticagrelor, antiplatelet agent] is for. It’s for blood pressure, isn’t it?. I don’t know.” P11.4 According to participants, the main reasons for the lack of understanding were the lack of information at the time of prescribing and the large number of medications, making it difficult to search for information and remember it. Participants sought information from different healthcare professionals or by themselves, on package inserts, through the internet, or from family and friends. Others reported having had all the information needed or were not interested in having more information. In addition, participants with low medication literacy, such as non-native speakers or elderly people, struggled more with medication understanding and sought help from family caregivers or healthcare professionals, even weeks after discharge: “ I don’t understand French very well […] [The doctor] explained it very quickly…[…] I didn’t understand everything he was saying” P16.2

C. Medication adherence

C.2 medication adherence at transition: adopting new behaviors.

Medication adherence was not mentioned as a concern during hospitalization and a few participants reported difficulties in medication initiation once back home: “I have an injection of Lantus® [insulin] in the morning, but obviously, the first day [after discharge], I forgot to do it because I was not used to it.” P23.1 Participants had to quickly adopt new behaviors in the first few days after discharge, especially for participants with few medications pre-hospitalization. The use of weekly pill organizers, alarms and specific storage space were reported as facilitators to support adherence. One patient did not initiate one of his medications because he did not understand the medication indication, and another patient took her old medications because she was used to them. Moreover, most participants experienced their hospitalization as a turning point, a time when they focused on their health, thought about the importance of their medications, and discussed any new lifestyle or dietary measures that might be implemented.

C.3 Medication adherence in outpatient care: ongoing medication adherence

More medication adherence difficulties appeared a few weeks after hospital discharge when most participants reported nonadherence behaviors, such as difficulties implementing the dosage regimen, or intentionally discontinuing the medication and modifying the medication regimen on their initiative. Determinants positively influencing medication adherence were the establishment of a routine; organizing medications in weekly pill-organizers; organizing pocket doses (medications for a short period that participants take with them when away from home); seeking support from family caregivers; using alarm clocks; and using specific storage places. Reasons for nonadherence were changes in daily routine; intake times that were not convenient for the patient; the large number of medications; and poor knowledge of the medication or side effects. Healthcare professionals’ assistance for medication management, such as the help of home nurses or pharmacists for the preparation of weekly pill-organizers, was requested by participants or offered by healthcare professionals to support medication adherence: “ I needed [a home nurse] to put my pills in the pillbox. […] I felt really weak […] and I was making mistakes. So, I’m very happy [the doctor] offered me [home care]. […] I have so many medications.” P22.3 Some participants who experienced prehospitalization non-adherence were more aware of their non-adherence and implemented strategies, such as modifying the timing of intake: “I said to my doctor : « I forget one time out of two […], can I take them in the morning? » We looked it up and yes, I can take it in the morning.” P11.2 In contrast, some participants were still struggling with adherence difficulties that they had before hospitalization. Motivations for taking medications two months after discharge were to improve health, avoid complications, reduce symptoms, reduce the number of medications in the future or out of obligation: “ I force myself to take them because I want to get to the end of my diabetes, I want to reduce the number of pills as much as possible.” P14.2 After a few weeks post-hospitalization, for some participants, health and illness were no longer the priority because of other life imperatives (e.g., family or financial situation).

This longitudinal study provided a multi-faceted representation of how patients manage, understand, and adhere to their medications from hospital discharge to two months after discharge. Our findings highlighted the varying degree of participants’ involvement in managing their medications during their hospitalization, the individualized needs for information during and after hospitalization, the complicated transition from hospital to autonomous medication management, the adaptation of daily routines around medication once back home, and the adherence difficulties that surfaced in the outpatient care, with nonadherence prior to hospitalization being an indicator of the behavior after discharge. Finally, our results confirmed the lack of continuity in care and showed the lack of patient care standardization experienced by the participants during the transition from hospital to outpatient care.

This in-depth analysis of patients’ experiences reinforces common challenges identified in the existing literature such as the lack of personalized information [ 9 , 10 , 11 ], loss of autonomy during hospitalization [ 14 , 74 , 75 ], difficulties in obtaining medication at discharge [ 11 , 45 , 76 ] and challenges in understanding treatment modifications and generics substitution [ 11 , 32 , 77 , 78 ]. Some of these studies were conducted during patients’ hospitalization [ 10 , 75 , 79 ] or up to 12 months after discharge [ 80 , 81 ], but most studies focused on the few days following hospital discharge [ 9 , 11 , 14 , 82 ]. Qualitative studies on medications at transition often focused on a specific topic, such as medication information, or a specific moment in time, and often included healthcare professionals, which muted patients’ voices [ 9 , 10 , 11 , 47 , 49 ]. Our qualitative longitudinal methodology was interested in capturing the temporal dynamics, in-depth narratives, and contextual nuances of patients’ medication experiences during transitions of care [ 59 , 83 ]. This approach provided a comprehensive understanding of how patients’ perspectives and behaviors evolved over time, offering insights into the complex interactions of medication management, understanding and adherence, and turning points within their medication journeys. A qualitative longitudinal design was used by Fylan et al. to underline patients’ resilience in medication management during and after discharge, by Brandberg et al. to show the dynamic process of self-management during the 4 weeks post-discharge and by Lawton et al. to examine how patients with type 2 diabetes perceived their care after discharge over a period of four years [ 49 , 50 , 51 ]. Our study focused on the first two months following hospitalization and future studies should focus on following discharged and at-risk patients over a longer period, as “transitions of care do not comprise linear trajectories of patients’ movements, with a starting and finishing point. Instead, they are endless loops of movements” [ 47 ].

Our results provide a particularly thorough description of how participants move from a state of total dependency during hospitalization regarding their medication management to a sudden and complete autonomy after hospital discharge impacting medication management, understanding, and adherence in the first days after discharge for some participants. Several qualitative studies have described the lack of shared decision-making and the loss of patient autonomy during hospitalization, which had an impact on self-management and created conflicts with healthcare professionals [ 75 , 81 , 84 ]. Our study also highlights nuanced patient experiences, including varying levels of patient needs, involvement, and proactivity during hospitalization and outpatient care, and our results contribute to capturing different perspectives that contrast with some literature that often portrays patients as more passive recipients of care [ 14 , 15 , 74 , 75 ]. Shared decision-making and proactive medication are key elements as they contribute to a smoother transition and better outcomes for patients post-discharge [ 85 , 86 , 87 ].

Consistent with the literature, the study identifies some challenges in medication initiation post-discharge [ 16 , 17 , 88 ] but our results also describe how daily routine rapidly takes over, either solidifying adherence behavior or generating barriers to medication adherence. Participants’ nonadherence prior to hospitalization was a factor influencing participants’ adherence post-hospitalization and this association should be further investigated, as literature showed that hospitalized patients have high scores of non-adherence [ 89 ]. Mortel et al. showed that more than 20% of discharged patients stopped their medications earlier than agreed with the physician and 25% adapted their medication intake [ 90 ]. Furthermore, patients who self-managed their medications had a lower perception of the necessity of their medication than patients who received help, which could negatively impact medication adherence [ 91 ]. Although participants in our study had high BMQ scores for necessity and lower scores for concerns, some participants expressed doubts about the need for their medications and a lack of motivation a few weeks after discharge. Targeted pharmacy interventions for newly prescribed medications have been shown to improve medication adherence, and hospital discharge is an opportune moment to implement this service [ 92 , 93 ].

Many medication changes were made during the transition of care (a median number of 7 changes during hospitalization and 7 changes during the two months after discharge), especially medication additions during hospitalization and interruptions after hospitalization. While medication changes during hospitalization are well described, the many changes following discharge are less discussed [ 7 , 94 ]. A Danish study showed that approximately 65% of changes made during hospitalization were accepted by primary healthcare professionals but only 43% of new medications initiated during hospitalization were continued after discharge [ 95 ]. The numerous changes after discharge may be caused by unnecessary intensification of medications during hospitalization, delayed discharge letters, lack of standardized procedures, miscommunication, patient self-management difficulties, or in response to an acute situation [ 96 , 97 , 98 ]. During the transition of care, in our study, both new and experienced participants were faced with difficulties in managing and understanding medication changes, either for newly prescribed medication or changes in previous medications. Such difficulties corroborate the findings of the literature [ 9 , 10 , 47 ] and our results showed that the lack of understanding during hospitalization led to participants having questions about their medications, even weeks after discharge. Particular attention should be given to patients’ understanding of medication changes jointly by physicians, nurses and pharmacists during the transition of care and in the months that follow as medications are likely to undergo as many changes as during hospitalization.

Implication for practice and future research

The patients’ perspectives in this study showed, at a system level, that there was a lack of standardization in healthcare professional practices regarding medication dispensing and follow-up. For now, in Switzerland, there are no official guidelines on medication prescription and dispensation during the transition of care although some international guidelines have been developed for outpatient healthcare professionals [ 3 , 99 , 100 , 101 , 102 ]. Here are some suggestions for improvement arising from our results. Patients should be included as partners and healthcare professionals should systematically assess (i) previous medication adherence, (ii) patients’ desired level of involvement and (iii) their needs for information during hospitalization. Hospital discharge processes should be routinely implemented to standardize hospital discharge preparation, medication prescribing, and dispensing. Discharge from the hospital should be planned with community pharmacies to ensure that all medications are available and, if necessary, doses of medications should be supplied by the hospital to bridge the gap. A partnership with outpatient healthcare professionals, such as general practitioners, community pharmacists, and homecare nurses, should be set up for effective asynchronous interprofessional collaboration to consolidate patients’ medication management, knowledge, and adherence, as well as to monitor signs of deterioration or adverse drug events.

Future research should consolidate our first attempt to develop a framework to better characterize medication at the transition of care, using Fig. 2   as a starting point. Contextualized interventions, co-designed by health professionals, patients and stakeholders, should be tested in a hybrid implementation study to test the implementation and effectiveness of the intervention for the health system [ 103 ].

Limitations

This study has some limitations. First, the transcripts were validated for accuracy by the interviewer but not by a third party, which could have increased the robustness of the transcription. Nevertheless, the interviewer followed all methodological recommendations for transcription. Second, patient inclusion took place during the COVID-19 pandemic, which may have had an impact on patient care and the availability of healthcare professionals. Third, we cannot guarantee the accuracy of some participants’ medication history before hospitalization, even though we contacted the participants’ main pharmacy, as participants could have gone to different pharmacies to obtain their medications. Fourth, our findings may not be generalizable to other populations and other healthcare systems because some issues may be specific to multimorbid patients with type 2 diabetes or to the Swiss healthcare setting. Nevertheless, issues encountered by our participants regarding their medications correlate with findings in the literature. Fifth, only 15 out of 21 participants took part in all the interviews, but most participants took part in at least three interviews and data saturation was reached. Lastly, by its qualitative and longitudinal design, it is possible that the discussion during interviews and participants’ reflections between interviews influenced participants’ management, knowledge, and adherence, even though this study was observational, and no advice or recommendations were given by the interviewer during interviews.

Discharged patients are willing to take steps to better manage, understand, and adhere to their medications, yet they are also faced with difficulties in the hospital and outpatient care. Furthermore, extensive changes in medications not only occur during hospitalization but also during the two months following hospital discharge, for which healthcare professionals should give particular attention. The different degrees of patients’ involvement, needs and resources should be carefully considered to enable them to better manage, understand and adhere to their medications. At a system level, patients’ experiences revealed a lack of standardization of medication practices during the transition of care. The healthcare system should provide the ecosystem needed for healthcare professionals responsible for or involved in the management of patients’ medications during the hospital stay, discharge, and outpatient care to standardize their practices while considering the patient as an active partner.

Data availability

The anonymized quantitative survey datasets and the qualitative codes are available in French from the corresponding author on reasonable request.

Abbreviations

adverse drug events

Adherence Visual Analogue Scale

Belief in Medication Questionnaire

Consolidated Criteria for Reporting Qualitative Research

case report form

standard deviation

World Health Organization

American Academy of Family Physician. Continuity of Care, Definition of 2020. Accessed 10 July 2022 https://www.aafp.org/about/policies/all/continuity-of-care-definition.html

Kripalani S, LeFevre F, Phillips CO, Williams MV, Basaviah P, Baker DW. Deficits in communication and information transfer between hospital-based and primary care physicians: implications for patient safety and continuity of care. JAMA. 2007;297(8):831–41.

Article   CAS   PubMed   Google Scholar  

World Health Organization (WHO). Medication Safety in Transitions of Care. 2019.

Forster AJ, Murff HJ, Peterson JF, Gandhi TK, Bates DW. The incidence and severity of adverse events affecting patients after discharge from the hospital. Ann Intern Med. 2003;138(3):161–7.

Article   PubMed   Google Scholar  

Krumholz HM. Post-hospital syndrome–an acquired, transient condition of generalized risk. N Engl J Med. 2013;368(2):100–2.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Banholzer S, Dunkelmann L, Haschke M, Derungs A, Exadaktylos A, Krähenbühl S, et al. Retrospective analysis of adverse drug reactions leading to short-term emergency hospital readmission. Swiss Med Wkly. 2021;151:w20400.

Blozik E, Signorell A, Reich O. How does hospitalization affect continuity of drug therapy: an exploratory study. Ther Clin Risk Manag. 2016;12:1277–83.

Article   PubMed   PubMed Central   Google Scholar  

Allen J, Hutchinson AM, Brown R, Livingston PM. User experience and care for older people transitioning from hospital to home: patients’ and carers’ perspectives. Health Expect. 2018;21(2):518–27.

Daliri S, Bekker CL, Buurman BM, Scholte Op Reimer WJM, van den Bemt BJF, Karapinar-Çarkit F. Barriers and facilitators with medication use during the transition from hospital to home: a qualitative study among patients. BMC Health Serv Res. 2019;19(1):204.

Bekker CL, Mohsenian Naghani S, Natsch S, Wartenberg NS, van den Bemt BJF. Information needs and patient perceptions of the quality of medication information available in hospitals: a mixed method study. Int J Clin Pharm. 2020;42(6):1396–404.

Foulon V, Wuyts J, Desplenter F, Spinewine A, Lacour V, Paulus D, et al. Problems in continuity of medication management upon transition between primary and secondary care: patients’ and professionals’ experiences. Acta Clin Belgica: Int J Clin Lab Med. 2019;74(4):263–71.

Article   Google Scholar  

Micheli P, Kossovsky MP, Gerstel E, Louis-Simonet M, Sigaud P, Perneger TV, et al. Patients’ knowledge of drug treatments after hospitalisation: the key role of information. Swiss Med Wkly. 2007;137(43–44):614–20.

PubMed   Google Scholar  

Ziaeian B, Araujo KL, Van Ness PH, Horwitz LI. Medication reconciliation accuracy and patient understanding of intended medication changes on hospital discharge. J Gen Intern Med. 2012;27(11):1513–20.

Allen J, Hutchinson AM, Brown R, Livingston PM. User experience and care integration in Transitional Care for older people from hospital to home: a Meta-synthesis. Qual Health Res. 2016;27(1):24–36.

Mackridge AJ, Rodgers R, Lee D, Morecroft CW, Krska J. Cross-sectional survey of patients’ need for information and support with medicines after discharge from hospital. Int J Pharm Pract. 2018;26(5):433–41.

Mulhem E, Lick D, Varughese J, Barton E, Ripley T, Haveman J. Adherence to medications after hospital discharge in the elderly. Int J Family Med. 2013;2013:901845.

Fallis BA, Dhalla IA, Klemensberg J, Bell CM. Primary medication non-adherence after discharge from a general internal medicine service. PLoS ONE. 2013;8(5):e61735.

Zhou L, Rupa AP. Categorization and association analysis of risk factors for adverse drug events. Eur J Clin Pharmacol. 2018;74(4):389–404.

Moreau-Gruet F. La multimorbidité chez les personnes de 50 ans et plus. Résultats basés sur l’enqête SHARE (Survey of Health, Ageing and Retirement in Europe. Obsan Bulletin 4/2013. 2013(Neuchâtel: OBservatoire suisse de la santé).

Iglay K, Hannachi H, Joseph Howie P, Xu J, Li X, Engel SS, et al. Prevalence and co-prevalence of comorbidities among patients with type 2 diabetes mellitus. Curr Med Res Opin. 2016;32(7):1243–52.

Sibounheuang P, Olson PS, Kittiboonyakun P. Patients’ and healthcare providers’ perspectives on diabetes management: a systematic review of qualitative studies. Res Social Adm Pharm. 2020;16(7):854–74.

Müller-Wieland D, Merkel M, Hamann A, Siegel E, Ottillinger B, Woker R, et al. Survey to estimate the prevalence of type 2 diabetes mellitus in hospital patients in Germany by systematic HbA1c measurement upon admission. Int J Clin Pract. 2018;72(12):e13273.

Blanc AL, Fumeaux T, Stirnemann J, Dupuis Lozeron E, Ourhamoune A, Desmeules J, et al. Development of a predictive score for potentially avoidable hospital readmissions for general internal medicine patients. PLoS ONE. 2019;14(7):e0219348.

Hansen LO, Greenwald JL, Budnitz T, Howell E, Halasyamani L, Maynard G, et al. Project BOOST: effectiveness of a multihospital effort to reduce rehospitalization. J Hosp Med. 2013;8(8):421–7.

Khalid JM, Raluy-Callado M, Curtis BH, Boye KS, Maguire A, Reaney M. Rates and risk of hospitalisation among patients with type 2 diabetes: retrospective cohort study using the UK General Practice Research Database linked to English Hospital Episode statistics. Int J Clin Pract. 2014;68(1):40–8.

Lussier ME, Evans HJ, Wright EA, Gionfriddo MR. The impact of community pharmacist involvement on transitions of care: a systematic review and meta-analysis. J Am Pharm Assoc. 2020;60(1):153–.

van der Heijden A, de Bruijne MC, Nijpels G, Hugtenburg JG. Cost-effectiveness of a clinical medication review in vulnerable older patients at hospital discharge, a randomized controlled trial. Int J Clin Pharm. 2019;41(4):963–71.

Bingham J, Campbell P, Schussel K, Taylor AM, Boesen K, Harrington A, et al. The Discharge Companion Program: an interprofessional collaboration in Transitional Care Model Delivery. Pharm (Basel). 2019;7(2):68.

Google Scholar  

Farris KB, Carter BL, Xu Y, Dawson JD, Shelsky C, Weetman DB, et al. Effect of a care transition intervention by pharmacists: an RCT. BMC Health Serv Res. 2014;14:406.

Meslot C, Gauchet A, Hagger MS, Chatzisarantis N, Lehmann A, Allenet B. A Randomised Controlled Trial to test the effectiveness of planning strategies to improve Medication Adherence in patients with Cardiovascular Disease. Appl Psychol Health Well Being. 2017;9(1):106–29.

Garnier A, Rouiller N, Gachoud D, Nachar C, Voirol P, Griesser AC, et al. Effectiveness of a transition plan at discharge of patients hospitalized with heart failure: a before-and-after study. ESC Heart Fail. 2018;5(4):657–67.

Daliri S, Bekker CL, Buurman BM, Scholte Op Reimer WJM, van den Bemt BJF, Karapinar-Çarkit F. Medication management during transitions from hospital to home: a focus group study with hospital and primary healthcare providers in the Netherlands. Int J Clin Pharm. 2020.

Hansen LO, Young RS, Hinami K, Leung A, Williams MV. Interventions to reduce 30-day rehospitalization: a systematic review. Ann Intern Med. 2011;155(8):520–8.

Leppin AL, Gionfriddo MR, Kessler M, Brito JP, Mair FS, Gallacher K, et al. Preventing 30-day hospital readmissions: a systematic review and meta-analysis of randomized trials. JAMA Intern Med. 2014;174(7):1095–107.

Donzé J, John G, Genné D, Mancinetti M, Gouveia A, Méan M et al. Effects of a Multimodal Transitional Care Intervention in patients at high risk of readmission: the TARGET-READ Randomized Clinical Trial. JAMA Intern Med. 2023.

Rodrigues CR, Harrington AR, Murdock N, Holmes JT, Borzadek EZ, Calabro K, et al. Effect of pharmacy-supported transition-of-care interventions on 30-Day readmissions: a systematic review and Meta-analysis. Ann Pharmacother. 2017;51(10):866–89.

Lam MYY, Dodds LJ, Corlett SA. Engaging patients to access the community pharmacy medicine review service after discharge from hospital: a cross-sectional study in England. Int J Clin Pharm. 2019;41(4):1110–7.

Hossain LN, Fernandez-Llimos F, Luckett T, Moullin JC, Durks D, Franco-Trigo L, et al. Qualitative meta-synthesis of barriers and facilitators that influence the implementation of community pharmacy services: perspectives of patients, nurses and general medical practitioners. BMJ Open. 2017;7(9):e015471.

En-Nasery-de Heer S, Uitvlugt EB, Bet PM, van den Bemt BJF, Alai A, van den Bemt P et al. Implementation of a pharmacist-led transitional pharmaceutical care programme: process evaluation of medication actions to reduce hospital admissions through a collaboration between Community and Hospital pharmacists (MARCH). J Clin Pharm Ther. 2022.

Morris ZS, Wooding S, Grant J. The answer is 17 years, what is the question: understanding time lags in translational research. J R Soc Med. 2011;104(12):510–20.

De Geest S, Zúñiga F, Brunkert T, Deschodt M, Zullig LL, Wyss K, et al. Powering Swiss health care for the future: implementation science to bridge the valley of death. Swiss Med Wkly. 2020;150:w20323.

Noonan VK, Lyddiatt A, Ware P, Jaglal SB, Riopelle RJ, Bingham CO 3, et al. Montreal Accord on patient-reported outcomes (PROs) use series - paper 3: patient-reported outcomes can facilitate shared decision-making and guide self-management. J Clin Epidemiol. 2017;89:125–35.

Hesselink G, Schoonhoven L, Barach P, Spijker A, Gademan P, Kalkman C, et al. Improving patient handovers from hospital to primary care: a systematic review. Ann Intern Med. 2012;157(6):417–28.

(OFSP) Interprofessionnalité dans le domaine de la santé Soins ambulatoire. Accessed 4 January 2024. https://www.bag.admin.ch/bag/fr/home/strategie-und-politik/nationale-gesundheitspolitik/foerderprogramme-der-fachkraefteinitiative-plus/foerderprogramme-interprofessionalitaet.html

Mitchell SE, Laurens V, Weigel GM, Hirschman KB, Scott AM, Nguyen HQ, et al. Care transitions from patient and caregiver perspectives. Ann Fam Med. 2018;16(3):225–31.

Davoody N, Koch S, Krakau I, Hägglund M. Post-discharge stroke patients’ information needs as input to proposing patient-centred eHealth services. BMC Med Inf Decis Mak. 2016;16:66.

Ozavci G, Bucknall T, Woodward-Kron R, Hughes C, Jorm C, Joseph K, et al. A systematic review of older patients’ experiences and perceptions of communication about managing medication across transitions of care. Res Social Adm Pharm. 2021;17(2):273–91.

Fylan B, Armitage G, Naylor D, Blenkinsopp A. A qualitative study of patient involvement in medicines management after hospital discharge: an under-recognised source of systems resilience. BMJ Qual Saf. 2018;27(7):539–46.

Fylan B, Marques I, Ismail H, Breen L, Gardner P, Armitage G, et al. Gaps, traps, bridges and props: a mixed-methods study of resilience in the medicines management system for patients with heart failure at hospital discharge. BMJ Open. 2019;9(2):e023440.

Brandberg C, Ekstedt M, Flink M. Self-management challenges following hospital discharge for patients with multimorbidity: a longitudinal qualitative study of a motivational interviewing intervention. BMJ Open. 2021;11(7):e046896.

Lawton J, Rankin D, Peel E, Douglas M. Patients’ perceptions and experiences of transitions in diabetes care: a longitudinal qualitative study. Health Expect. 2009;12(2):138–48.

Mabire C, Bachnick S, Ausserhofer D, Simon M. Patient readiness for hospital discharge and its relationship to discharge preparation and structural factors: a cross-sectional study. Int J Nurs Stud. 2019;90:13–20.

Meyers DC, Durlak JA, Wandersman A. The quality implementation framework: a synthesis of critical steps in the implementation process. Am J Community Psychol. 2012;50(3–4):462–80.

Meyer-Massetti C, Hofstetter V, Hedinger-Grogg B, Meier CR, Guglielmo BJ. Medication-related problems during transfer from hospital to home care: baseline data from Switzerland. Int J Clin Pharm. 2018;40(6):1614–20.

Neeman M, Dobrinas M, Maurer S, Tagan D, Sautebin A, Blanc AL, et al. Transition of care: a set of pharmaceutical interventions improves hospital discharge prescriptions from an internal medicine ward. Eur J Intern Med. 2017;38:30–7.

Geese F, Schmitt KU. Interprofessional Collaboration in Complex Patient Care Transition: a qualitative multi-perspective analysis. Healthc (Basel). 2023;11(3).

Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. Int J Nurs Stud. 2013;50(5):587–92.

Thomson R, Plumridge L, Holland J, Editorial. Int J Soc Res Methodol. 2003;6(3):185–7.

Audulv Å, Hall EOC, Kneck Å, Westergren T, Fegran L, Pedersen MK, et al. Qualitative longitudinal research in health research: a method study. BMC Med Res Methodol. 2022;22(1):255.

Kim H, Sefcik JS, Bradway C. Characteristics of qualitative descriptive studies: a systematic review. Res Nurs Health. 2017;40(1):23–42.

Sandelowski M. Whatever happened to qualitative description? Res Nurs Health. 2000;23(4):334–40.

Bradshaw C, Atkinson S, Doody O. Employing a qualitative description Approach in Health Care Research. Glob Qual Nurs Res. 2017;4:2333393617742282.

PubMed   PubMed Central   Google Scholar  

Bellone JM, Barner JC, Lopez DA. Postdischarge interventions by pharmacists and impact on hospital readmission rates. J Am Pharm Assoc (2003). 2012;52(3):358–62.

Hennink MM, Kaiser BN, Marconi VC. Code saturation versus meaning saturation: how many interviews are Enough? Qual Health Res. 2016;27(4):591–608.

World Health Organization. Adherence to long-term therapies: evidence for action. 2003.

Fisher JD, Fisher WA, Amico KR, Harman JJ. An information-motivation-behavioral skills model of adherence to antiretroviral therapy. Health Psychol. 2006;25(4):462–73.

Bandura A. Health promotion from the perspective of social cognitive theory. Psychol Health. 1998;13(4):623–49.

ShiftEUROQOL Research FOndation EQ 5D Instruments. Accessed 30 July 2022 https://euroqol.org/eq-5d-instruments/sample-demo/

Jeppesen KM, Coyle JD, Miser WF. Screening questions to predict limited health literacy: a cross-sectional study of patients with diabetes mellitus. Ann Fam Med. 2009;7(1):24–31.

Giordano TP, Guzman D, Clark R, Charlebois ED, Bangsberg DR. Measuring adherence to antiretroviral therapy in a diverse population using a visual analogue scale. HIV Clin Trials. 2004;5(2):74–9.

Horne R, Weinman J, Hankins M. The beliefs about medicines questionnaire: the development and evaluation of a new method for assessing the cognitive representation of medication. Psychol Health. 1999;14(1):1–24.

Horne R, Chapman SC, Parham R, Freemantle N, Forbes A, Cooper V. Understanding patients’ adherence-related beliefs about medicines prescribed for long-term conditions: a meta-analytic review of the necessity-concerns Framework. PLoS ONE. 2013;8(12):e80633.

Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Res Psychol. 2006;3(2):77–101.

Waibel S, Henao D, Aller M-B, Vargas I, Vázquez M-L. What do we know about patients’ perceptions of continuity of care? A meta-synthesis of qualitative studies. Int J Qual Health Care. 2011;24(1):39–48.

Rognan SE, Jørgensen MJ, Mathiesen L, Druedahl LC, Lie HB, Bengtsson K, et al. The way you talk, do I have a choice?’ Patient narratives of medication decision-making during hospitalization. Int J Qualitative Stud Health Well-being. 2023;18(1):2250084.

Michel B, Hemery M, Rybarczyk-Vigouret MC, Wehrle P, Beck M. Drug-dispensing problems community pharmacists face when patients are discharged from hospitals: a study about 537 prescriptions in Alsace. Int J Qual Health Care. 2016;28(6):779–84.

Bruhwiler LD, Hersberger KE, Lutters M. Hospital discharge: what are the problems, information needs and objectives of community pharmacists? A mixed method approach. Pharm Pract (Granada). 2017;15(3):1046.

Knight DA, Thompson D, Mathie E, Dickinson A. Seamless care? Just a list would have helped!’ Older people and their carer’s experiences of support with medication on discharge home from hospital. Health Expect. 2013;16(3):277–91.

Gualandi R, Masella C, Viglione D, Tartaglini D. Exploring the hospital patient journey: what does the patient experience? PLoS ONE. 2019;14(12):e0224899.

Norberg H, Håkansson Lindqvist M, Gustafsson M. Older individuals’ experiences of Medication Management and Care after Discharge from Hospital: an interview study. Patient Prefer Adherence. 2023;17:781–92.

Jones KC, Austad K, Silver S, Cordova-Ramos EG, Fantasia KL, Perez DC, et al. Patient perspectives of the hospital discharge process: a qualitative study. J Patient Exp. 2023;10:23743735231171564.

Hesselink G, Flink M, Olsson M, Barach P, Dudzik-Urbaniak E, Orrego C, et al. Are patients discharged with care? A qualitative study of perceptions and experiences of patients, family members and care providers. BMJ Qual Saf. 2012;21(Suppl 1):i39–49.

Murray SA, Kendall M, Carduff E, Worth A, Harris FM, Lloyd A, et al. Use of serial qualitative interviews to understand patients’ evolving experiences and needs. BMJ. 2009;339:b3702.

Berger ZD, Boss EF, Beach MC. Communication behaviors and patient autonomy in hospital care: a qualitative study. Patient Educ Couns. 2017;100(8):1473–81.

Davis RE, Jacklin R, Sevdalis N, Vincent CA. Patient involvement in patient safety: what factors influence patient participation and engagement? Health Expect. 2007;10(3):259–67.

Greene J, Hibbard JH. Why does patient activation matter? An examination of the relationships between patient activation and health-related outcomes. J Gen Intern Med. 2012;27(5):520–6.

Mitchell SE, Gardiner PM, Sadikova E, Martin JM, Jack BW, Hibbard JH, et al. Patient activation and 30-day post-discharge hospital utilization. J Gen Intern Med. 2014;29(2):349–55.

Weir DL, Motulsky A, Abrahamowicz M, Lee TC, Morgan S, Buckeridge DL, et al. Failure to follow medication changes made at hospital discharge is associated with adverse events in 30 days. Health Serv Res. 2020;55(4):512–23.

Kripalani S, Goggins K, Nwosu S, Schildcrout J, Mixon AS, McNaughton C, et al. Medication nonadherence before hospitalization for Acute Cardiac events. J Health Commun. 2015;20(Suppl 2):34–42.

Mortelmans L, De Baetselier E, Goossens E, Dilles T. What happens after Hospital Discharge? Deficiencies in Medication Management encountered by geriatric patients with polypharmacy. Int J Environ Res Public Health. 2021;18(13).

Mortelmans L, Goossens E, Dilles T. Beliefs about medication after hospital discharge in geriatric patients with polypharmacy. Geriatr Nurs. 2022;43:280–7.

Bandiera C, Ribaut J, Dima AL, Allemann SS, Molesworth K, Kalumiya K et al. Swiss Priority setting on implementing Medication Adherence interventions as Part of the European ENABLE COST action. Int J Public Health. 2022;67.

Elliott R, Boyd M, Nde S. at e. Supporting adherence for people starting a new medication for a long-term condition through community pharmacies: a pragmaticrandomised controlled trial of the New Medicine Service. 2015.

Grimmsmann T, Schwabe U, Himmel W. The influence of hospitalisation on drug prescription in primary care–a large-scale follow-up study. Eur J Clin Pharmacol. 2007;63(8):783–90.

Larsen MD, Rosholm JU, Hallas J. The influence of comprehensive geriatric assessment on drug therapy in elderly patients. Eur J Clin Pharmacol. 2014;70(2):233–9.

Viktil KK, Blix HS, Eek AK, Davies MN, Moger TA, Reikvam A. How are drug regimen changes during hospitalisation handled after discharge: a cohort study. BMJ Open. 2012;2(6):e001461.

Strehlau AG, Larsen MD, Søndergaard J, Almarsdóttir AB, Rosholm J-U. General practitioners’ continuation and acceptance of medication changes at sectorial transitions of geriatric patients - a qualitative interview study. BMC Fam Pract. 2018;19(1):168.

Anderson TS, Lee S, Jing B, Fung K, Ngo S, Silvestrini M, et al. Prevalence of diabetes medication intensifications in older adults discharged from US Veterans Health Administration Hospitals. JAMA Netw Open. 2020;3(3):e201511.

Royal Pharmaceutical Society. Keeping patients safewhen they transfer between care providers– getting the medicines right June 2012. Accessed 27 October 2023 https://www.rpharms.com/Portals/0/RPS%20document%20library/Open%20access/Publications/Keeping%20patients%20safe%20transfer%20of%20care%20report.pdf

International Pharmaceutical Federation (FIP). Medicines reconciliation: A toolkit for pharmacists. Accessed 23 September 2023 https://www.fip.org/file/4949

Californian Pharmacist Assiociation Transitions of Care Resource Guide. https://cdn.ymaws.com/www.cshp.org/resource/resmgr/Files/Practice-Policy/For_Pharmacists/transitions_of_care_final_10.pdf

Royal Collegue of Physicians. Medication safety at hospital discharge: Improvement guide and resource. Accessed 18 September 2023 https://www.rcplondon.ac.uk/file/33421/download

Douglas N, Campbell W, Hinckley J. Implementation science: buzzword or game changer. J Speech Lang Hear Res. 2015;58.

Download references

Acknowledgements

The authors would like to thank all the patients who took part in this study. We would also like to thank the Geneva University Hospitals Patients Partners + 3P platform as well as Mrs. Tourane Corbière and Mr. Joël Mermoud, patient partners, who reviewed interview guides for clarity and significance. We would like to thank Samuel Fabbi, Vitcoryavarman Koh, and Pierre Repiton for the transcriptions of the audio recordings.

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Open access funding provided by University of Geneva

Author information

Authors and affiliations.

School of Pharmaceutical Sciences, University of Geneva, Geneva, Switzerland

Léa Solh Dost & Marie P. Schneider

Institute of Pharmaceutical Sciences of Western Switzerland, University of Geneva, Geneva, Switzerland

Division of Endocrinology, Diabetes, Hypertension and Nutrition, Department of Medicine, Geneva University Hospitals, Geneva, Switzerland

Giacomo Gastaldi

You can also search for this author in PubMed   Google Scholar

Contributions

LS, GG, and MS conceptualized and designed the study. LS and GG screened and recruited participants. LS conducted the interviews. LS, GG, and MS performed data analysis and interpretation. LS drafted the manuscript and LS and MS worked on the different versions. MS and GG approved the final manuscript.

Corresponding authors

Correspondence to Léa Solh Dost or Marie P. Schneider .

Ethics declarations

Ethics approval and consent to participate.

Ethics approval was sought and granted by the Cantonal Research Ethics Commission, Geneva (CCER) (2020 − 01779), and informed consent to participate was obtained from all participants.

Consent for publication

Informed consent for publication was obtained from all participants.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, supplementary material 3, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Solh Dost, L., Gastaldi, G. & Schneider, M. Patient medication management, understanding and adherence during the transition from hospital to outpatient care - a qualitative longitudinal study in polymorbid patients with type 2 diabetes. BMC Health Serv Res 24 , 620 (2024). https://doi.org/10.1186/s12913-024-10784-9

Download citation

Received : 28 June 2023

Accepted : 26 February 2024

Published : 13 May 2024

DOI : https://doi.org/10.1186/s12913-024-10784-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Continuity of care
• Transition of care
• Patient discharge
• Medication management
• Medication adherence
• Qualitative research
• Longitudinal studies
• Patient-centered care
• Interprofessional collaboration
• Type 2 diabetes

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

New Content From Advances in Methods and Practices in Psychological Science

• Advances in Methods and Practices in Psychological Science
• Cognitive Dissonance
• Meta-Analysis
• Methodology
• Preregistration
• Reproducibility

A Practical Guide to Conversation Research: How to Study What People Say to Each Other Michael Yeomans, F. Katelynn Boland, Hanne Collins, Nicole Abi-Esber, and Alison Wood Brooks  

Conversation—a verbal interaction between two or more people—is a complex, pervasive, and consequential human behavior. Conversations have been studied across many academic disciplines. However, advances in recording and analysis techniques over the last decade have allowed researchers to more directly and precisely examine conversations in natural contexts and at a larger scale than ever before, and these advances open new paths to understand humanity and the social world. Existing reviews of text analysis and conversation research have focused on text generated by a single author (e.g., product reviews, news articles, and public speeches) and thus leave open questions about the unique challenges presented by interactive conversation data (i.e., dialogue). In this article, we suggest approaches to overcome common challenges in the workflow of conversation science, including recording and transcribing conversations, structuring data (to merge turn-level and speaker-level data sets), extracting and aggregating linguistic features, estimating effects, and sharing data. This practical guide is meant to shed light on current best practices and empower more researchers to study conversations more directly—to expand the community of conversation scholars and contribute to a greater cumulative scientific understanding of the social world. 

Open-Science Guidance for Qualitative Research: An Empirically Validated Approach for De-Identifying Sensitive Narrative Data Rebecca Campbell, McKenzie Javorka, Jasmine Engleton, Kathryn Fishwick, Katie Gregory, and Rachael Goodman-Williams  

The open-science movement seeks to make research more transparent and accessible. To that end, researchers are increasingly expected to share de-identified data with other scholars for review, reanalysis, and reuse. In psychology, open-science practices have been explored primarily within the context of quantitative data, but demands to share qualitative data are becoming more prevalent. Narrative data are far more challenging to de-identify fully, and because qualitative methods are often used in studies with marginalized, minoritized, and/or traumatized populations, data sharing may pose substantial risks for participants if their information can be later reidentified. To date, there has been little guidance in the literature on how to de-identify qualitative data. To address this gap, we developed a methodological framework for remediating sensitive narrative data. This multiphase process is modeled on common qualitative-coding strategies. The first phase includes consultations with diverse stakeholders and sources to understand reidentifiability risks and data-sharing concerns. The second phase outlines an iterative process for recognizing potentially identifiable information and constructing individualized remediation strategies through group review and consensus. The third phase includes multiple strategies for assessing the validity of the de-identification analyses (i.e., whether the remediated transcripts adequately protect participants’ privacy). We applied this framework to a set of 32 qualitative interviews with sexual-assault survivors. We provide case examples of how blurring and redaction techniques can be used to protect names, dates, locations, trauma histories, help-seeking experiences, and other information about dyadic interactions. 

Impossible Hypotheses and Effect-Size Limits Wijnand van Tilburg and Lennert van Tilburg

Psychological science is moving toward further specification of effect sizes when formulating hypotheses, performing power analyses, and considering the relevance of findings. This development has sparked an appreciation for the wider context in which such effect sizes are found because the importance assigned to specific sizes may vary from situation to situation. We add to this development a crucial but in psychology hitherto underappreciated contingency: There are mathematical limits to the magnitudes that population effect sizes can take within the common multivariate context in which psychology is situated, and these limits can be far more restrictive than typically assumed. The implication is that some hypothesized or preregistered effect sizes may be impossible. At the same time, these restrictions offer a way of statistically triangulating the plausible range of unknown effect sizes. We explain the reason for the existence of these limits, illustrate how to identify them, and offer recommendations and tools for improving hypothesized effect sizes by exploiting the broader multivariate context in which they occur. 

It’s All About Timing: Exploring Different Temporal Resolutions for Analyzing Digital-Phenotyping Data Anna Langener, Gert Stulp, Nicholas Jacobson, Andrea Costanzo, Raj Jagesar, Martien Kas, and Laura Bringmann  

The use of smartphones and wearable sensors to passively collect data on behavior has great potential for better understanding psychological well-being and mental disorders with minimal burden. However, there are important methodological challenges that may hinder the widespread adoption of these passive measures. A crucial one is the issue of timescale: The chosen temporal resolution for summarizing and analyzing the data may affect how results are interpreted. Despite its importance, the choice of temporal resolution is rarely justified. In this study, we aim to improve current standards for analyzing digital-phenotyping data by addressing the time-related decisions faced by researchers. For illustrative purposes, we use data from 10 students whose behavior (e.g., GPS, app usage) was recorded for 28 days through the Behapp application on their mobile phones. In parallel, the participants actively answered questionnaires on their phones about their mood several times a day. We provide a walk-through on how to study different timescales by doing individualized correlation analyses and random-forest prediction models. By doing so, we demonstrate how choosing different resolutions can lead to different conclusions. Therefore, we propose conducting a multiverse analysis to investigate the consequences of choosing different temporal resolutions. This will improve current standards for analyzing digital-phenotyping data and may help combat the replications crisis caused in part by researchers making implicit decisions. 

Calculating Repeated-Measures Meta-Analytic Effects for Continuous Outcomes: A Tutorial on Pretest–Posttest-Controlled Designs David R. Skvarc, Matthew Fuller-Tyszkiewicz  

Meta-analysis is a statistical technique that combines the results of multiple studies to arrive at a more robust and reliable estimate of an overall effect or estimate of the true effect. Within the context of experimental study designs, standard meta-analyses generally use between-groups differences at a single time point. This approach fails to adequately account for preexisting differences that are likely to threaten causal inference. Meta-analyses that take into account the repeated-measures nature of these data are uncommon, and so this article serves as an instructive methodology for increasing the precision of meta-analyses by attempting to estimate the repeated-measures effect sizes, with particular focus on contexts with two time points and two groups (a between-groups pretest–posttest design)—a common scenario for clinical trials and experiments. In this article, we summarize the concept of a between-groups pretest–posttest meta-analysis and its applications. We then explain the basic steps involved in conducting this meta-analysis, including the extraction of data and several alternative approaches for the calculation of effect sizes. We also highlight the importance of considering the presence of within-subjects correlations when conducting this form of meta-analysis.   

Reliability and Feasibility of Linear Mixed Models in Fully Crossed Experimental Designs Michele Scandola, Emmanuele Tidoni  

The use of linear mixed models (LMMs) is increasing in psychology and neuroscience research In this article, we focus on the implementation of LMMs in fully crossed experimental designs. A key aspect of LMMs is choosing a random-effects structure according to the experimental needs. To date, opposite suggestions are present in the literature, spanning from keeping all random effects (maximal models), which produces several singularity and convergence issues, to removing random effects until the best fit is found, with the risk of inflating Type I error (reduced models). However, defining the random structure to fit a nonsingular and convergent model is not straightforward. Moreover, the lack of a standard approach may lead the researcher to make decisions that potentially inflate Type I errors. After reviewing LMMs, we introduce a step-by-step approach to avoid convergence and singularity issues and control for Type I error inflation during model reduction of fully crossed experimental designs. Specifically, we propose the use of complex random intercepts (CRIs) when maximal models are overparametrized. CRIs are multiple random intercepts that represent the residual variance of categorical fixed effects within a given grouping factor. We validated CRIs and the proposed procedure by extensive simulations and a real-case application. We demonstrate that CRIs can produce reliable results and require less computational resources. Moreover, we outline a few criteria and recommendations on how and when scholars should reduce overparametrized models. Overall, the proposed procedure provides clear solutions to avoid overinflated results using LMMs in psychology and neuroscience.   

Understanding Meta-Analysis Through Data Simulation With Applications to Power Analysis Filippo Gambarota, Gianmarco Altoè  

Meta-analysis is a powerful tool to combine evidence from existing literature. Despite several introductory and advanced materials about organizing, conducting, and reporting a meta-analysis, to our knowledge, there are no introductive materials about simulating the most common meta-analysis models. Data simulation is essential for developing and validating new statistical models and procedures. Furthermore, data simulation is a powerful educational tool for understanding a statistical method. In this tutorial, we show how to simulate equal-effects, random-effects, and metaregression models and illustrate how to estimate statistical power. Simulations for multilevel and multivariate models are available in the Supplemental Material available online. All materials associated with this article can be accessed on OSF ( https://osf.io/54djn/ ).   

Feedback on this article? Email  [email protected]  or login to comment.

APS regularly opens certain online articles for discussion on our website. Effective February 2021, you must be a logged-in APS member to post comments. By posting a comment, you agree to our Community Guidelines and the display of your profile information, including your name and affiliation. Any opinions, findings, conclusions, or recommendations present in article comments are those of the writers and do not necessarily reflect the views of APS or the article’s author. For more information, please see our Community Guidelines .

Please login with your APS account to comment.

Privacy Overview

Loading metrics

Open Access

Peer-reviewed

Research Article

A novel method for multiple phenotype association studies based on genotype and phenotype network

Roles Data curation, Formal analysis, Methodology, Visualization, Writing – original draft, Writing – review & editing

Affiliation Department of Mathematical Sciences, Michigan Technological University, Houghton, Michigan, United States of America

Roles Formal analysis, Methodology, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

• Xuewei Cao, 
• Shuanglin Zhang, 
• Qiuying Sha

• Published: May 10, 2024
• https://doi.org/10.1371/journal.pgen.1011245
• Reader Comments

This is an uncorrected proof.

Joint analysis of multiple correlated phenotypes for genome-wide association studies (GWAS) can identify and interpret pleiotropic loci which are essential to understand pleiotropy in diseases and complex traits. Meanwhile, constructing a network based on associations between phenotypes and genotypes provides a new insight to analyze multiple phenotypes, which can explore whether phenotypes and genotypes might be related to each other at a higher level of cellular and organismal organization. In this paper, we first develop a bipartite signed network by linking phenotypes and genotypes into a Genotype and Phenotype Network (GPN). The GPN can be constructed by a mixture of quantitative and qualitative phenotypes and is applicable to binary phenotypes with extremely unbalanced case-control ratios in large-scale biobank datasets. We then apply a powerful community detection method to partition phenotypes into disjoint network modules based on GPN. Finally, we jointly test the association between multiple phenotypes in a network module and a single nucleotide polymorphism (SNP). Simulations and analyses of 72 complex traits in the UK Biobank show that multiple phenotype association tests based on network modules detected by GPN are much more powerful than those without considering network modules. The newly proposed GPN provides a new insight to investigate the genetic architecture among different types of phenotypes. Multiple phenotypes association studies based on GPN are improved by incorporating the genetic information into the phenotype clustering. Notably, it might broaden the understanding of genetic architecture that exists between diagnoses, genes, and pleiotropy.

Author summary

Biological pleiotropy refers to a SNP or gene that has a direct biological influence on more than one phenotypic trait, which can offer significant insights in understanding the complex genotype-phenotype relationships. Network analyses provide an integrative approach to characterize complex genomic associations by linking phenotypes and genotypes into a Genotype and Phenotype Network (GPN). Jointly analyzing multiple phenotypes and incorporating the genetic information into the phenotype clustering may increase the statistical power to discover the cross-phenotype association and pleiotropy. We evaluate our proposed multiple phenotype association tests based on network modules detected by GPN for 72 EHR-derived phenotypes in the diseases of the musculoskeletal system and connective tissue in the UK Biobank. From the post-GWAS analyses, we observe that the test based on GPN can identify more significantly enriched biological pathways than that without considering the network modules. Meanwhile, some of the uniquely identified SNPs by the test based on GPN are also colocalized in the eQTL study of the gene expression in the Muscle Skeletal tissue.

Citation: Cao X, Zhang S, Sha Q (2024) A novel method for multiple phenotype association studies based on genotype and phenotype network. PLoS Genet 20(5): e1011245. https://doi.org/10.1371/journal.pgen.1011245

Editor: Heather J. Cordell, Newcastle University, UNITED KINGDOM

Received: August 18, 2023; Accepted: March 29, 2024; Published: May 10, 2024

Copyright: © 2024 Cao et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: The UK Biobank data are accessed via https://www.ukbiobank.ac.uk/ . The GWAS catalog summary data are accessed via https://www.ebi.ac.uk/gwas/ . The SNP-gene associations in the Muscle Skeletal tissue are downloaded via https://gtexportal.org/home/ . Software The software for the proposed method is publicly available at https://github.com/xueweic/GPN . PLINK version 1.9 can be downloaded from https://www.cog-genomics.org/plink/1.9/ . LDSC: the command line tool for estimating heritability and genetic correlation from GWAS summary statistics can be downloaded from https://github.com/bulik/ldsc [ 27 ]. FUMA: the platform that can be used to annotate, prioritize, visualize and interpret GWAS results can be found from https://fuma.ctglab.nl/ [ 57 ]. DAVID: the functional tool can be found from https://david.ncifcrf.gov/ [ 65 , 66 ]. Cytoscape: an open source software platform for visualizing complex networks which can be accessed via https://cytoscape.org/ [ 71 ].

Funding: The work was in part funded by the Michigan Technological University Health Research Institute Fellowship program, the Portage Health Foundation Graduate Assistantship, and Graduate Dean Awards Advisory Panel. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Genome-wide association studies (GWAS) have successfully identified thousands of single nucleotide polymorphisms (SNPs) genetically associated with a wide range of complex human diseases and traits [ 1 , 2 ]. Over the past decade, more than 10,000 associations between SNPs and diseases/traits have been discovered [ 3 ]. Although GWAS have emerged as a common and powerful tool to detect the complexity of the genotype-phenotype associations, a common limitation of GWAS is that they focus on only a single phenotype at a time [ 4 – 7 ]. Joint analysis of multiple correlated phenotypes for GWAS may provide more power to identify and interpret pleiotropic loci, which are essential to understand pleiotropy in diseases and complex traits [ 4 , 8 , 9 ]. In brief, biological pleiotropy refers to a SNP or gene that has a direct biological influence on more than one phenotypic trait [ 10 ]. Biological pleiotropy can offer significant insights in understanding the complex genotype-phenotype relationships [ 2 ]. Therefore, multiple phenotypes are usually collected in many GWAS cohorts and jointly analyzing multiple phenotypes may increase statistical power to discover the cross-phenotype associations and pleiotropy [ 10 – 13 ].

Many statistical methods have been developed to jointly test the association between a SNP and multiple correlated phenotypes [ 14 ]. The most widely used methods for multiple phenotype association studies can be roughly classified into three categories: 1) statistical tests based on combining either the univariate test statistics or p-values, such as O’Brien’s method [ 15 ], adaptive Fisher’s combination (AFC) [ 16 ], aSPU [ 17 ], and others [ 18 ]; 2) multivariate analyses based on regression methods, such as multivariate analysis of variance (MANOVA) [ 19 ], reverse regression methods (MultiPhen) [ 20 ], linear mixed effect models (LMM) [ 21 ], and generalized estimating equations (GEE) [ 22 ]; and 3) dimension reduction methods, such as clustering linear combination (CLC) [ 12 ], canonical correlation analysis (CCA) [ 23 ], and principal components analysis (PCA) [ 24 , 25 ]. However, most phenotypes are influenced by many SNPs that act in concert to alter cellular function [ 26 ], the above mentioned methods are only based on phenotypic correlation without considering the genetic correlation among phenotypes. Therefore, these methods may loss statistical power to detect the true pleiotropic effects comparing the methods based on genetic architecture among complex diseases. To address this issue, numerous types of algorithms to investigate the genetic correlation among complex traits and diseases have been developed [ 27 – 29 ]. Many of these algorithms are often in conjunction with linkage disequilibrium (LD) information by using GWAS summary association data [ 28 ]. For example, cross-trait LD score regression has been developed to estimate genetic and phenotypic correlation that requires only GWAS summary statistics and is not biased by overlapping samples [ 27 ].

In 2007, a conceptually different approach based on the human disease network had been developed, exploring whether human complex traits and the corresponding genotypes might be related to each other at a higher level of cellular and organismal organization [ 30 ]. Network analyses provide an integrative approach to characterize complex genomic associations [ 31 ]. Over the past decade, network methodologies, particularly Weighted Gene Co-expression Network Analysis (WGCNA) [ 32 ], have become increasingly popular in genetic association studies. This popularity is due to their effectiveness in identifying complex patterns of gene expression and clarifying the relationships among genes. Researchers have applied WGCNA to unravel the genetic underpinnings of complex traits, focusing primarily on gene-gene interactions [ 33 , 34 ]. However, constructing networks that map the associations between phenotypes and genotypes can provide fresh perspectives, enabling the simultaneous analysis of multiple phenotypes and SNPs. Notably, it might broaden the understanding of genetic architecture that exists between diagnoses, genes, and pleiotropy [ 8 ]. Modules detected from human disease networks are useful in providing insights pertaining to biological functionality [ 35 ]. Therefore, community detection methods play a key role in understanding the global and local structures of disease interaction, in shedding light on association connections that may not be easily visible in the network topology [ 36 ]. Many community detection methods have been applied from social networks to human disease networks, such as Louvain’s method [ 8 ] with modularity as a measure and core module identification to identify small and structurally well-defined communities [ 35 ]. However, most community detection methods have been developed for unsigned networks [ 37 – 43 ].

To date, many biobanks, such as the UK Biobank [ 44 ], aggregate data across tens of thousands of phenotypes and provide a great opportunity to construct the human disease network and perform joint analyses of multiple correlated phenotypes. The electronic health record (EHR)-driven genomic research (EDGR) workflow is the most popular way to analyze multiple diagnosis codes in Biobank data, at its core, which is the use of EHR data for genomic research in the investigation of population-wide genomic characterization [ 45 ]. In most EHR systems, the whole phenome can be divided into numerous phenotypic categories according to the first few characters of the International Classification of Disease (ICD) billing codes [ 46 ]. However, the ICD-based categories are based on the underlying cause of death rather than on the shared genetic architecture among all complex diseases and traits. Meanwhile, the phenotypes in large biobanks usually have extremely unbalanced case-control ratios. Therefore, linking phenotypes, especially EHR-derived phenotypes, with genotypes in a network is also very important to examine the genetic architecture of complex diseases and traits.

Overview of methods

In this paper, we develop a bipartite signed network by linking phenotypes and genotypes into a Genotype and Phenotype Network (GPN; Fig 1A ). The GPN can be constructed by a mixture of quantitative and qualitative phenotypes and is applicable to phenotypes with extremely unbalanced case-control ratios for large-scale biobank datasets since the saddlepoint approximation [ 47 ] is used to test the association between genotype and phenotype with extremely unbalanced case-control ratio. After projecting genotypes into phenotypes, the genetic correlation of phenotypes can be calculated based on the shared associations among all genotypes ( Fig 1B ). We then apply a powerful community detection method to partition phenotypes into disjoint network modules using the hierarchical clustering method and the number of modules is determined by perturbation ( Fig 1C ) [ 48 ]. The phenotypes in each network module share the same genetic information. After partitioning phenotypes into disjoint network modules, a statistical method for multiple phenotype association studies can be applied to test the association between phenotypes in each module and a SNP, then a Bonferroni correction can be used to test if all phenotypes are associated with a SNP ( Fig 1D ). To jointly analyze the association between multiple phenotypes in each module with a SNP, we use six multiple phenotype association tests, including ceCLC [ 49 ], CLC [ 12 ], HCLC [ 50 ], MultiPhen [ 20 ], O’Brien [ 15 ], and Omnibus [ 12 ]. The advantage of the association test based on network modules detected by GPN is that phenotypes in a network module are highly correlated based on the genetic architecture, therefore, the association test is more powerful to identify pleiotropic SNPs. After we obtain the GWAS signals from the previous steps, post-GWAS analyses can be applied to understand the high level of biological mechanism, such as pathway/tissue enrichment analysis and colocalization of GWAS signals and eQTL analysis in the specific disease-associated tissue ( Fig 1E–1G ). The construction of GPN, community detection method, and six multiple phenotype association tests have been implemented in R, which is an open-source software and publicly available on GitHub: https://github.com/xueweic/GPN .

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

a . Construction of GPN. Each phenotype (yellow square) and each SNP form a directed edge which represents the strength of the association, where the red dashed line indicates that the minor allele of the SNP is a protective allele to the phenotype, and the blue dashed line indicates that the minor allele of the SNP is a risk allele to the phenotype. b . Construction of PPN, which is the one-mode projection of GPN on phenotypes. c . The community detection method is used to partition phenotypes into disjoint network modules. d . Multiple phenotype association tests based on the network modules detected by GPN. e . GWAS signals are identified by a multiple phenotype association test with or without considering network modules. f . Functional enrichment analysis based on the detected GWAS signals and the publicly available functional database. g . Colocalization of GWAS signals and eQTL analysis. (All networks are generated by an open source software platform, Cytoscape 0.9.2, which can be accessed via https://cytoscape.org/ [ 51 ]; Other figures are generated by an open source software, R 4.2.2, which can be accessed via https://www.r-project.org/ ).

https://doi.org/10.1371/journal.pgen.1011245.g001

Simulation studies

We first use extensive simulation studies to validate multiple phenotype association studies based on the newly proposed GPN. In the simulation studies, we assess the type I error rate and power with different numbers of phenotypes (60, 80, and 100), different types of phenotypes along with different sample sizes: (i) mixture phenotypes are half quantitative and half qualitative with balanced case-control ratios for sample sizes of 2,000 and 4,000, and (ii) binary phenotypes are all qualitative but with extremely unbalanced case-control ratios for sample sizes of 10,000 and 20,000. Similar to the simulation models introduced in Sha et al. [ 12 ], we generate six different models (see Data Simulation).

Type I error rates

Tables A-F in S1 Text summarize the estimated type I error rates of six multiple phenotype association tests for mixture phenotypes under models 1–6, respectively. “N.O.” represents the type I error rates of multiple phenotype association tests being calculated without considering network modules; “NET” presents the type I error rates of the tests being evaluated by considering network modules detected by GPN. Based on 500 Monte-Carlo (MC) runs which is the same as 10 6 replicates, the 95% confidence intervals (Cis) for type I error rates divided by nominal significance levels 0.001 and 0.0001 are (0.938, 1.062) and (0.804,1.196), respectively. The bold-faced values indicate that the values are beyond the upper bounds of the 95% Cis. We can see that almost all of the estimated type I error rates of ceCLC, CLC, HCLC, and Omnibus tests are within 95% Cis. However, O’Brien in NET has inflated type I error rates under model 6. MultiPhen has inflated type I error rates for the sample size of 2,000. If the sample size is 4000, MultiPhen in N.O. also inflates type I error rates, but MultiPhen in NET can control type I error rates for the significance level is 0.0001. Tables G-L in S1 Text summarize the estimated type I error rates of six multiple phenotype association tests for binary phenotypes with extremely unbalanced case-control ratios under models 1–6. Similar to Tables A-F in S1 Text , ceCLC, CLC, HCLC, and Omnibus have corrected type I error rates at almost all simulation settings. However, O’Brien in NET has inflated type I error rates and MultiPhen has inflated type I error rates at all scenarios. If there is no clusters of the phenotypes, we also see that only MultiPhen has inflated type I error rates and other five multiple phenotype association tests have well-controlled type I error rates ( Table M in S1 Text ).

Power comparisons

For power comparisons, we consider 100 causal SNPs for models 1–4 and 200 causal SNPs for models 5–6 (see Data Simulation). In each of the simulation models, the power is evaluated using 10 MC runs which is the same as 1,000 replicates for models 1–4 and 2,000 replicates for models 5–6. Meanwhile, the power is evaluated at the Bonferroni corrected significance level of 0.05 based on the number of causal SNPs in each MC run.

Fig 2 ( Fig A in S1 Text ) shows the power of six multiple phenotype association tests under six simulation models for different effect sizes with a total of 80 mixture phenotypes and a sample size of 4,000 (2,000). From Figs 2 and A in S1 Text , we can see that: (i) All tests in NET (filled by the dashed line) are much more powerful than those in N.O., indicating that tests based on network modules detected by GPN are more powerful than the tests without considering network modules. Since the community detection method can partition phenotypes into different network modules based on shared genetic architecture, the phenotypes can be clustered in the same module if they have higher genetic correlations. In particular, the power of O’Brien [ 15 ] increases a lot in the case of a SNP affecting phenotypes in different directions. (ii) ceCLC is more powerful than other tests in both N.O. and NET under the six simulation models. (iii) As sample size increases, the power of all multiple phenotype association tests increases. We also perform power comparisons for a total of 60 and 100 mixture phenotypes with 2,000 and 4,000 sample sizes for different effect sizes under the six simulation models ( Figs B-E in S1 Text ), respectively. We observe that the patterns of the power are similar to those observed in Figs 2 and A in S1 Text .

The number of mixture phenotypes (half continuous phenotypes and half binary phenotypes with balanced case-control ratios) is 80 and the sample size is 4,000. The power of all of the six tests is evaluated using 10 MC runs. (Figure is generated by an open source software, R 4.2.2, which can be accessed via https://www.r-project.org/ ).

https://doi.org/10.1371/journal.pgen.1011245.g002

To mimic phenotypes in the UK Biobank, we also consider the case with all phenotypes being binary with extremely unbalanced case-control ratios. The phenotypes are generated based on extremely unbalanced case-control ratios which are randomly selected from the set of case-control ratios with cases greater than 200 from UK Biobank ICD-10 code level 3 phenotypes (see Real Dataset; case-control ratios belong to [0.000658,0.03937]). In this simulation, we consider a total of 60, 80, and 100 phenotypes along with two sample sizes, 10,000 and 20,000. Figs F-K in S1 Text show the power comparisons of the six tests under six simulation models. Fig L in S1 Text shows the power comparisons of the six tests under the models that mimic real data cluster structures. The patterns of power comparisons for binary phenotypes and for the models that mimic real data cluster structure are similar to those observed in Figs 2 and A-E in S1 Text .

Real data analysis based on UK Biobank

Furthermore, we apply the newly proposed multiple phenotype association test based on network modules detected by GPN to a set of diseases of the musculoskeletal system and connective tissue across more than 300,000 individuals from the UK Biobank.

Network module detection

We construct GPN based on 72 EHR-derived phenotypes in the diseases of the musculoskeletal system and connective tissue with 288,647 SNPs in autosomal chromosomes in the UK Biobank. Due to all phenotypes in our analysis being extremely unbalanced, the strength of the association between phenotype and genotype is calculated by the saddlepoint approximation [ 47 ]. After the construction of GPN, we apply a powerful community detection method and these 72 phenotypes are partitioned into 8 disjoint network modules ( Fig 3 ). There are 2–37 phenotypes in each module.

The blocks with different color indicate different modules, where the values in the legend represent the number of phenotypes in each network module. The labels of phenotypes are listed in the form of ICD-10 code and the corresponding diseases can be found in the UK Biobank. The connection between two phenotypes represents the absolutely value of the weight greater than 40. (The graph was prepared by Cytoscape 0.9.2, which can be accessed via https://cytoscape.org/ )

https://doi.org/10.1371/journal.pgen.1011245.g003

We can see that the network modules are not consistent with the ICD-based categories which are based on the underlying cause of death rather than the shared genetic architecture among all complex diseases. For example, Fig 3 shows three phenotypes, M32.9 Systemic lupus erythematosus, M35.0 Sicca syndrome, and M65.3 Trigger finger, are detected in network module III (in red). However, these three phenotypes do not belong to the same ICD-category (Data-Field 41202 in UK Biobank), where M35.0 is one of the diseases in the other systemic involvement of connective tissue (M35) and M65.3 belongs to the synovitis and tenosynovitis (M65). To investigate the genetic correlation among these three phenotypes, we use the saddlepoint approximation to test the association between each phenotype and each SNP. As shown in Fig M in S1 Text , the Manhattan plots for the three phenotypes in network module III (M32.9, M35.0, and M65.3) have a similar pattern. Although the synovitis and tenosynovitis (M65.9) and M65.3 belong to the same ICD code category (M65), the Manhattan plot of M65.9 shows that there are no SNPs significantly associated with this phenotype and the genetic correlation between M65.9 and M65.3 is not strong. Therefore, we can conclude that the community detection method based on our proposed GPN can partition phenotypes into different categories based on the shared genetic architecture.

Furthermore, we apply the hierarchical clustering method to compare the genetic correlation of phenotypes obtained by our proposed GPN and that estimated by LDSC [ 27 ]. Figs N-O in S1 Text show that dendrograms of hierarchical clustering method based on the genetic correlation of phenotypes obtained by GPN, and the phenotypic or genetic correlation estimated by LDSC, respectively. In Fig N in S1 Text , the cluster results of the phenotypic correlation estimated by LDSC are similar to that of the genetic correlation based on GPN, but GPN can separately identify two highly genetic correlated phenotypes, ankylosing spondylitis (M45) and ankylosing spondylitis with site unspecified (M45.X9). However, the cluster results of the genetic correlation estimated by LDSC are different from those obtained by GPN. Some phenotypes in the same UK Biobank level 1 category can be clustered in the same group by GPN but not by LDSC ( Fig O in S1 Text ).

Interpretation of the association test.

We apply five multiple phenotype association tests (ceCLC, CLC, HCLC, O’Brien, and Omnibus) to test the association between 72 EHR-derived phenotypes and each of 288,647 SNPs in the UK Biobank. MultiPhen is not considered here since it has inflated type I error rates, especially for the phenotypes with extremely unbalanced case-control ratios.

First, we apply the five tests in N.O. to test the association between 72 phenotypes and each SNP. We use the commonly used genome-wide significance level 5×10 −8 . Fig 4A shows the Venn diagram of the number of SNPs identified by the five tests. There are 11 SNPs identified by all five tests. ceCLC identifies 647 SNPs with 32 unique SNPs not being identified by other four tests. Among the 32 novel SNPs, two SNPs, rs13107325 (p-value = 4.6×10 −10 ) and rs443198 (p-value = 1.73×10 −11 ), are significantly associated with at least one of the 72 phenotypes reported in the GWAS catalog ( Table N in S1 Text ). rs13107325 is reported to be associated with osteoarthritis (M19.9) [ 52 ] and rotator cuff syndrome (M75.1) [ 53 ]. Meanwhile, rs13107325 is mapped to gene SLC39A8 that is also reported to be significantly associated with multisite chronic pain (M25.5) [ 54 ]. rs443198 is mapped to gene NOTCH4 which is associated with systemic sclerosis (M34) [ 55 ]. Moreover, the mapped gene NOTCH4 is one of the most important genes reported to be associated with multiple diseases in the disease category of the musculoskeletal system and connective tissue, such as rheumatoid arthritis (M06.9) [ 56 ], psoriatic arthritis (M07.3) [ 57 ], Takayasu arteritis (M31.4) [ 58 ], systemic lupus erythematosus (M32.9) [ 59 ], and appendicular lean mass (M62.9) [ 60 ]. We map these 32 unique SNPs into genes with 20 kb upstream and 20 kb downstream regions. There are 27 out of 32 SNPs with corresponding mapped genes associated with 14 phenotypes reported in the GWAS catalog ( Table N in S1 Text ). These 14 phenotypes and corresponding ICD-10 codes are summarized in Table O in S1 Text .

( a ) and in NET ( b ). The number below each method indicates the total number of SNPs identified by the corresponding method. (Figure is generated by an open source software, R 4.2.2, which can be accessed via https://www.r-project.org/ ).

https://doi.org/10.1371/journal.pgen.1011245.g004

Next, we test the associations between phenotypes in each of the eight network modules detected by the GPN and each SNP. Then, we adjust the p-value of each method for testing the association between a SNP and all of the 72 phenotypes by Bonferroni correction. We adopt the commonly used genome-wide significance level 5×10 −8 . Fig 4B shows that all tests can identify more SNPs comparing with the number of SNPs identified in N.O. ceCLC in NET identifies 980 SNPs, where 647 SNPs are identified in N.O. Meanwhile, there are 950 SNPs identified by HCLC, 949 SNPs by CLC, and 891 SNPs by Omnibus, where the corresponding results in N.O. are 354 SNPs, 808 SNPs, and 634 SNPs, respectively. In particular, the number of SNPs identified by O’Brien in NET is increased a lot, where there are 948 SNPs identified in NET and only 57 SNPs identified in N.O. As the results shown in Fig 4B , there are 807 overlapped SNPs identified by all five tests in NET which is much larger than 11 overlapped SNPs identified in N.O.

To compare the difference between the tests in N.O. and in NET, we summarize the number of overlapping SNPs identified by each method in N.O. and NET in Fig P in S1 Text . We observe that most SNPs identified in N.O. can be identified in NET. Meanwhile, tests in NET can identify much more SNPs than those in N.O. As mentioned previously, the advantage of the tests based on the network modules detected by GPN is that we can identify potential pleiotropic SNPs and also interpret SNP effects on which network modules based on the shared genetic architecture. Notably, we also investigate the smallest p-value obtained by each of the eight phenotypic modules for each of the 980 SNPs identified by ceCLC. For example, 396 SNPs have the smallest p-values for testing the association with network module III. Based on the results of the univariate score test corrected for saddlepoint approximation (SPAtest) ( Fig M in S1 Text ), 104 SNPs are significantly associated with at least one phenotype in module III. All of these 104 SNPs can be identified by ceCLC, HCLC, and Omnibus in NET and 103 SNPs can be identified by CLC and O’Brien in NET. The results show that the tests based on network modules can detect potential pleiotropic loci which can not be detected by the univariate test. Fig Q in S1 Text shows the QQ plots and inflation factors in each of the eight network modules for six tests in the real data analysis. We see that the inflation factors for all approaches are close to 1.

Pathway enrichment analysis.

ceCLC is more powerful than the other four tests in simulations and also can identify more SNPs in real data analysis, therefore, we only perform the post-GWAS analyses of the SNPs identified by ceCLC. There are 191 mapped genes containing at least one of the 647 SNPs identified by ceCLC in N.O. and 252 mapped genes containing at least one of the 980 SNPs identified by ceCLC in NET. In this study, significantly enriched pathways are identified by those genes with false discovery rate (FDR) < 0.05.

From the pathway enrichment analyses, we observe that ceCLC based on the network modules identifies more significantly enriched pathways than that without considering network modules. Fig 5 shows that 16 pathways are significantly enriched by 191 mapped genes in N.O. and 29 pathways are significantly enriched by 252 mapped genes in NET, where all of the 16 pathways identified in N.O. are also identified in NET. Two pathways identified in N.O. and NET, rheumatoid arthritis (hsa05323; FDR = 8.72×10 −3 in N.O. and FDR = 6.48×10 −8 in NET) and systemic lupus erythematosus (hsa05322; FDR = 4.25×10 −19 in N.O. and FDR = 1.02×10 −40 in NET) showed in Fig 5 , are related to the diseases of the musculoskeletal system and connective tissue. For example, osteopetrosis (M19.9) and rheumatoid arthritis (M06.9) are related to the rheumatoid arthritis pathway. Meanwhile, the pathway related to at least one of the 72 phenotypes, hematopoietic cell lineage (hsa04640; FDR = 1.08×10 −5 ), is only identified in NET. Notably, DBGET system ( https://www.genome.jp/dbget-bin/www_bget?hsa05322 ) reports that there are two pathways related to systemic lupus erythematosus: antigen processing and presentation (hsa04612; FDR = 4.83×10 −3 in N.O. and FDR = 2.82×10 −16 in NET) identified in both N.O. and NET and cell adhesion molecule (hsa04514; FDR = 1.04×10 −5 ) only identified in NET.

( a ) and NET ( b ). The red marked pathways denote the pathways related to the diseases of the musculoskeletal system and connective tissue. There are 191 genes in N.O. and 252 genes in NET that are applied to the pathway enrichment analysis. (Figure is generated by an open source software, R 4.2.2, which can be accessed via https://www.r-project.org/ ).

https://doi.org/10.1371/journal.pgen.1011245.g005

Meanwhile, the above five pathways related to the diseases of the musculoskeletal system and connective tissue contain more enriched genes identified by ceCLC in NET than the enriched genes identified in N.O. For example, 43 SNPs within six mapped genes identified by ceCLC in N.O. are enriched in rheumatoid arthritis pathway, including ATP6V1G2 , HLA-DRA , LTB , TNF , HLA-DRB1 , and HLA-DQA1 ; and 111 SNPs within 12 mapped genes in NET are enriched in this pathway, including HLA-DMA , HLA-DMB , ATP6V1G2 , HLA-DRA , LTB , HLA-DOA , TNF , HLA-DOB , HLA-DQA2 , HLA-DRB1 , HLA-DQA1 , and HLA-DQB1 . Compared with the results of ceCLC in N.O., the test based on network modules identifies six more enriched genes, especially, gene HLA-DMB (including rs241458; p-value = 7.09×10 −9 ) and gene HLA-DOA (including rs3097646; p-value = 5.50×10 −9 ) that have not been reported in the GWAS catalog.

Tissue enrichment analysis.

To further investigate the biological mechanism, we use FUMA [ 61 ] to annotate 191 mapped genes in N.O. and 252 mapped genes in NET in terms of biological context. Due to these mapped genes associated with at least one phenotype in the diseases of the musculoskeletal system and connective tissue, we can test if these mapped genes are enriched in the relevant-tissue based on FUMA. Fig R in S1 Text shows the ordered enriched tissues based on the mapped genes identified by ceCLC in N.O. and NET. We observe that the mapped genes identified by ceCLC in N.O. are most enriched in brain-related tissue ( Fig R(a) in S1 Text ). Nevertheless, Fig R(b) in S1 Text shows that the mapped genes identified by ceCLC in NET are significantly enriched in the Muscle-Skeletal tissue with p-value < 0.05. The construction of GPN is benefit to multiple phenotype association studies by clustering the related phenotypes based on the genetic information. Notably, the identified SNPs are more likely to be within the same relevant biological context.

Colocalization of GWAS and eQTL analysis.

We perform the colocalization analysis on the 33 unique SNPs identified by ceCLC ( Table N in S1 Text ; one SNP in NET and 32 SNPs in N.O.) and all SNP-gene association pairs in the Muscle Skeletal tissue reported in GTEx. Fig S in S1 Text shows the colocalization signals with the uniquely identified SNPs by ceCLC that are selected to be the lead SNPs in the colocalization analysis. NET identifies one unique SNP, rs4148866, which is mapped to gene ABCB9 . Even if gene ABCB9 has no reported associations with any diseases of the musculoskeletal system and connective tissue in the GWAS Catalog, the Bayesian posterior probability of colocalization analysis for shared variant of significant SNPs identified by ceCLC and gene expression in the Muscle Skeletal tissue (PP H4 ) is 98.4%. The higher value of PP H4 indicates that gene ABCB9 and Muscle Skeletal tissue play an important role in the disease mechanism due to the same variant responsible for a GWAS locus and also affecting gene expression [ 62 ]. Among 32 unique SNPs identified by ceCLC in N.O., there are two SNPs, rs34333163 and rs6916921, selected to be the lead SNPs ( Fig S in S1 Text ). Both of them are reported in the GWAS Catalog that have associations with at least one of the diseases in the musculoskeletal system. However, the PP H4 values for the corresponding genes SLC38A8 and ATP6V1G2 are lower than 50%.

In this paper, we propose a novel method for multiple phenotype association studies based on genotype and phenotype network. The construction of a bipartite signed network, GPN, is to link genotypes with phenotypes using the evidence of associations. To understand pleiotropy in diseases and complex traits and explore the genetic correlation among phenotypes, we project genotypes into phenotypes based on the GPN. We also apply a powerful community detection method to detect the network modules based on the shared genetic architecture. In contrast to previous community detection methods for disease networks, the applied method benefits from exploring the biological functionality interactions of diseases based on the signed network. Furthermore, we apply several multiple phenotype association tests to test the association between phenotypes in each network module and a SNP. Extensive simulation studies show that all multiple phenotype association tests based on network modules have corrected type I error rates if the corresponding test is a valid test for testing the association between a SNP and phenotypes without considering network modules. Most tests in NET are much more powerful than those in N.O. Meanwhile, we evaluate the performance of the association tests based on network modules detected by GPN through a set of 72 EHR-derived phenotypes in the diseases of the musculoskeletal system and connective tissue across more than 300,000 samples from the UK Biobank. Compared with the tests in N.O., all tests based on network modules can identify more potentially pleiotropic SNPs and ceCLC can identify more SNPs than other methods.

In addition, the construction of GPN does not require access to individual-level genotypes and phenotypes data, which only requires association evidence between each genotype and each phenotype. Therefore, when individual-level data are not available, this evidence can be obtained from GWAS summary statistics, such as the effect sizes (odds ratios for binary phenotypes) and corresponding p-values. The development of GPN can also be applied to omics studies, such as constructing a GPN that incorporates expression Quantitative Trait Locus (eQTLs) and gene expressions. However, in the context of numerous omics studies, the sample sizes are not very large. We have broadened our simulation analysis to include the same six simulation models, specifically targeting scenarios with the number of phenotypes of 60 and the sample size of 1,000. We observe similar results as simulations with larger sample sizes: the tests in NET are much more powerful than those in N.O ( Fig T in S1 Text ). Meanwhile, the simulation studies show that the powerful network community detection method can correctly partition phenotypes into several disjoint network modules based on the shared genetic architecture. Since the determination of the number of network modules in community detection method is independent of the association tests [ 48 ], we only need to perform the perturbation procedure once in real data analyses. In our real data analysis with 72 phenotypes and 288,647 SNPs, it only takes 1.5 hour with 1,000 perturbations to obtain the optimal number of network modules on a macOS (2.7 GHz Quad-Core Intel Core i7, 16 GB memory).

In this paper, the multiple phenotype association test based on the network module uses association information twice. We first use association information to detect communities and to cluster phenotypes into different groups, then we use the association information to perform the multiple phenotype association test. One may doubt whether the multiple phenotype association test has inflated type I error rates by using the association information twice. However, the community detection uses association information between all SNPs and all phenotypes, while the multiple phenotype association test only considers one SNP. Based on our simulation studies, the first time use of association information only affects the multiple phenotype association test slightly and is not enough to affect the type I error rates.

In summary, the proposed GPN provides a new insight to investigate the genetic correlation among phenotypes. Especially when the phenotypes have extremely unbalanced case-control ratios, the weight of an edge in the signed bipartite network can be calculated based on the saddlepoint approximation. The power of multiple phenotype association tests based on network modules detected by GPN are improved by incorporating the genetic information into the phenotypic clustering. Therefore, the proposed method can be applied to large-scale data across multiple related traits and diseases (i.e., biobanks data set, etc.).

Consider a sample with n unrelated individuals, indexed by i = 1,⋯, n . Suppose each individual has a total of K phenotypes and M SNPs. Let Y = ( y ik ) be an n × K matrix of K phenotypes, where y ik denotes the phenotype value of the i th individual for the k th phenotype. The phenotypes can be both quantitative and qualitative, especially for phenotypes with extremely unbalanced case-control ratios. Let G = ( g im ) be an n × M matrix of genotypes, where g im represents the genotypic score of the i th individual at the m th SNP which is the number of minor alleles that the i th individual carries at the SNP.

Construction of the genotype and phenotype network

Here, W kl is the genetic correlation between the k th and l th phenotypes based on the association strengths T km for k = 1,⋯, K and m = 1,⋯, M . Thus, the PPN is also a signed network.

Community detection method

We can use the identified C network modules to further investigate the associations between phenotypes in each network module and SNPs.

Multiple phenotype association tests

After we obtain C network modules for the phenotypes, we apply a multiple phenotype association test to identify the association between phenotypes in each of the C network modules and a SNP. Any multiple phenotype association test can be applied here. In this article, we apply six commonly used multiple phenotype association tests to each network module, including ceCLC [ 49 ], CLC [ 12 ], HCLC [ 50 ], MultiPhen [ 20 ], O’Brien [ 15 ], and Omnibus [ 12 ] (see details in Text A in S1 Text ), then a Bonferroni correction is used to adjust for multiple testing for the C network modules to test if all phenotypes in the C network modules associated with a SNP.

Data simulation

To generate a qualitative disease affection status, we use a liability threshold model based on a quantitative phenotype and its case-control ratio. Let n a and n C denote the number of affected individuals and the number of non-affected individuals. For a given case-control ratio r and sample size N , n c = N /( r +1) and n a = rN /( r +1). An individual is defined to be affected if the individual’s phenotype is in the top n a of all phenotypes. For each phenotype, the case-control ratio is randomly chosen from a set S . The set S contains all case-control ratios with the number of cases greater than 200 from UK Biobank ICD-10 code level 3 phenotypes.

Based on the factor model, we consider different numbers of phenotypes, 60, 80, and 100, and different sample sizes. For mixture phenotypes, the sample sizes are 2,000 and 4,000; for binary phenotypes, the sample sizes are 10,000 and 20,000. We consider six simulation models ( Text B and Table P in S1 Text ) with M = 2,000 and MAF ∼ U (0.05,0.5). The calculations of the type I error rates and power of multiple phenotype association test in N.O. and in NET are summarized in Text C in S1 Text .

Real dataset

The UK Biobank is a population-based cohort study with a wide variety of genetic and phenotypic information [ 67 ]. It includes ~ 500K people from all around the United Kingdom who were aged between 40 and 69 when recruited in 2006–2010 [ 44 , 68 ]. Following the genotype and phenotype preprocess introduced in Liang et al. [ 50 ], there are 288,647 SNPs and 72 EHR-derived phenotypes in the diseases of the musculoskeletal system and connective tissue for 322,607 individuals are kept in our real data analysis [ 69 ] ( Text D and Fig U in S1 Text ). Among the 72 phenotypes, lumbar and other intervertebral disk disorders with myelopathy (M51.0) has the smallest case-control ratio 0.000658 with 212 cases and 322,395 controls; Gonarthrosis (M17.9) has the largest case-control ratio 0.03937 with 12,218 cases and 310,389 controls. Therefore, all of the phenotypes we considered in our analysis have extremely unbalanced case-control ratios. Furthermore, each phenotype is adjusted by 13 covariates, including age, sex, genotyping array, and the first 10 genetic principal components (PCs) [ 65 ]. The analysis is performed based on the adjusted phenotypes.

Correlation analysis

To compare the genetic and phenotypic correlations among the 72 EHR-derived phenotypes, we apply cross-triat LDSC regression [ 27 ] to obtain the genetic correlation and phenotypic correlation which can provide useful etiological insights [ 27 ]. GWAS summary statistics are generated from the association between phenotype and genotype which are calculated by the saddlepoint approximation. We use the precomputed LD scores of European individuals in the 1000 Genomes project for high-quality HapMap3 SNPs (‘eur_w_ld_chr’). For the phenotypic correlation, we consider 70 phenotypes excluding M79.6 (Enthesopathy of lower limb) and M67.8 (Other specified disorders of synovium and tendon), since the heritabilities of these two phenotypes estimated by LDSC are out of bounds. For the genetic correlation, we only consider 52 phenotypes exlcuding 20 phenotypes, where the heritabilities of these phenotypes are not significantly different from zero. We apply the K-means hierarchical clustering method to compare the correlations of phenotypes obtained by our proposed GPN and LDSC.

Post-GWAS analyses

To better understand the biological functions behind the SNPs identified by one multiple phenotype association test, we identify the pathways in which the identified SNPs are involved. We use the functional annotation tool named Database for Annotation, Visualization, and Integrated Discovery bioinformatics resource (DAVID: https://david.ncifcrf.gov/ ) [ 70 , 71 ] for the Kyoto Encyclopedia of Genes and Genomes (KEGG) pathway enrichment analysis. A mapped gene used in the pathway enrichment analysis denotes the gene that includes at least one identified SNPs with a 20kb window region. The biological pathways with FDR < 0.05 and enriched gene count > 2 are considered statistically significant [ 72 ].

To prioritize and interpret the GWAS signals and identify lead SNPs, tissue enrichment analyses are performed using the Functional Mapping and Annotation (FUMA: https://fuma.ctglab.nl/ ) [ 61 ] platform and the GWAS signals from one multiple phenotype association test in N.O. and in NET, respectively. FUMA first performs a genic aggregation analysis of GWAS association signals to calculate gene-wise association signals using MAGMA, which is a commonly used generalized gene-set analysis of GWAS summary statistics [ 73 ]. Then, it subsequently tests whether tissues and cell types are enriched for expression of the genes with gene-wise association signals. For tissue enrichment analysis, we use 30 general tissue types in GTEx v8 reference set ( https://gtexportal.org/home/ ).

Colocalization analysis.

As most associated variants are noncoding, it is expected that they influence disease risk through altering gene expression or splicing [ 74 ]. The colocalization analysis is a way to identify the association of a GWAS SNP and a gene expression QTL that are colocalized. We perform colocalization analysis using the ‘coloc’ package in R [ 62 ], a Bayesian statistical methodology that tests pairwise colocalization of eQTLs with unique identified SNPs by ceCLC in NET and N.O. from the UK Biobank dataset. The SNP-gene associations in the Muscle Skeletal tissue are downloaded from GTEx v7. We use the default setting of the prior probabilities, p 1 = p 2 = 10 −4 and p 12 = 10 −5 , for a causal variant in an eQTL or a GWAS SNP and a shared causal variant between eQTL and GWAS SNP, respectively.

Supporting information

S1 text. supplemental texts, tables, and figures..

https://doi.org/10.1371/journal.pgen.1011245.s001

Acknowledgments

Part of this research has been conducted using the UK Biobank Resource under application number 102999 and the NHGRI-EBI GWAS Catalog. High-Performance Computing Shared Facility (Superior) at Michigan Technological University was used in obtaining results presented in this publication. Some parts of this work used the Extreme Science and Engineering Discovery Environment (XSEDE), which is supported by National Science Foundation grant number ACI-1548562. Specifically, it used the Bridges-2 system, which is supported by NSF award number ACI-1445606, at the Pittsburgh Supercomputing Center (PSC).

• View Article
• PubMed/NCBI
• Google Scholar
• 36. Newman M. Networks: Oxford university press; 2018.