findings in mixed methods research

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Mixed Methods Research | Definition, Guide & Examples

Mixed Methods Research | Definition, Guide & Examples

Published on August 13, 2021 by Tegan George . Revised on June 22, 2023.

Mixed methods research combines elements of quantitative research and qualitative research in order to answer your research question . Mixed methods can help you gain a more complete picture than a standalone quantitative or qualitative study, as it integrates benefits of both methods.

Mixed methods research is often used in the behavioral, health, and social sciences, especially in multidisciplinary settings and complex situational or societal research.

• To what extent does the frequency of traffic accidents ( quantitative ) reflect cyclist perceptions of road safety ( qualitative ) in Amsterdam?
• How do student perceptions of their school environment ( qualitative ) relate to differences in test scores ( quantitative ) ?
• How do interviews about job satisfaction at Company X ( qualitative ) help explain year-over-year sales performance and other KPIs ( quantitative ) ?
• How can voter and non-voter beliefs about democracy ( qualitative ) help explain election turnout patterns ( quantitative ) in Town X?
• How do average hospital salary measurements over time (quantitative) help to explain nurse testimonials about job satisfaction (qualitative) ?

Table of contents

When to use mixed methods research, mixed methods research designs, advantages of mixed methods research, disadvantages of mixed methods research, other interesting articles, frequently asked questions.

Mixed methods research may be the right choice if your research process suggests that quantitative or qualitative data alone will not sufficiently answer your research question. There are several common reasons for using mixed methods research:

• Generalizability : Qualitative research usually has a smaller sample size , and thus is not generalizable. In mixed methods research, this comparative weakness is mitigated by the comparative strength of “large N,” externally valid quantitative research.
• Contextualization: Mixing methods allows you to put findings in context and add richer detail to your conclusions. Using qualitative data to illustrate quantitative findings can help “put meat on the bones” of your analysis.
• Credibility: Using different methods to collect data on the same subject can make your results more credible. If the qualitative and quantitative data converge, this strengthens the validity of your conclusions. This process is called triangulation .

As you formulate your research question , try to directly address how qualitative and quantitative methods will be combined in your study. If your research question can be sufficiently answered via standalone quantitative or qualitative analysis, a mixed methods approach may not be the right fit.

But mixed methods might be a good choice if you want to meaningfully integrate both of these questions in one research study.

Keep in mind that mixed methods research doesn’t just mean collecting both types of data; you need to carefully consider the relationship between the two and how you’ll integrate them into coherent conclusions.

Mixed methods can be very challenging to put into practice, and comes with the same risk of research biases as standalone studies, so it’s a less common choice than standalone qualitative or qualitative research.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

There are different types of mixed methods research designs . The differences between them relate to the aim of the research, the timing of the data collection , and the importance given to each data type.

As you design your mixed methods study, also keep in mind:

• Your research approach ( inductive vs deductive )
• Your research questions
• What kind of data is already available for you to use
• What kind of data you’re able to collect yourself.

Here are a few of the most common mixed methods designs.

Convergent parallel

In a convergent parallel design, you collect quantitative and qualitative data at the same time and analyze them separately. After both analyses are complete, compare your results to draw overall conclusions.

• On the qualitative side, you analyze cyclist complaints via the city’s database and on social media to find out which areas are perceived as dangerous and why.
• On the quantitative side, you analyze accident reports in the city’s database to find out how frequently accidents occur in different areas of the city.

In an embedded design, you collect and analyze both types of data at the same time, but within a larger quantitative or qualitative design. One type of data is secondary to the other.

This is a good approach to take if you have limited time or resources. You can use an embedded design to strengthen or supplement your conclusions from the primary type of research design.

Explanatory sequential

In an explanatory sequential design, your quantitative data collection and analysis occurs first, followed by qualitative data collection and analysis.

You should use this design if you think your qualitative data will explain and contextualize your quantitative findings.

Exploratory sequential

In an exploratory sequential design, qualitative data collection and analysis occurs first, followed by quantitative data collection and analysis.

You can use this design to first explore initial questions and develop hypotheses . Then you can use the quantitative data to test or confirm your qualitative findings.

“Best of both worlds” analysis

Combining the two types of data means you benefit from both the detailed, contextualized insights of qualitative data and the generalizable , externally valid insights of quantitative data. The strengths of one type of data often mitigate the weaknesses of the other.

For example, solely quantitative studies often struggle to incorporate the lived experiences of your participants, so adding qualitative data deepens and enriches your quantitative results.

Solely qualitative studies are often not very generalizable, only reflecting the experiences of your participants, so adding quantitative data can validate your qualitative findings.

Method flexibility

Mixed methods are less tied to disciplines and established research paradigms. They offer more flexibility in designing your research, allowing you to combine aspects of different types of studies to distill the most informative results.

Mixed methods research can also combine theory generation and hypothesis testing within a single study, which is unusual for standalone qualitative or quantitative studies.

Mixed methods research is very labor-intensive. Collecting, analyzing, and synthesizing two types of data into one research product takes a lot of time and effort, and often involves interdisciplinary teams of researchers rather than individuals. For this reason, mixed methods research has the potential to cost much more than standalone studies.

Differing or conflicting results

If your analysis yields conflicting results, it can be very challenging to know how to interpret them in a mixed methods study. If the quantitative and qualitative results do not agree or you are concerned you may have confounding variables , it can be unclear how to proceed.

Due to the fact that quantitative and qualitative data take two vastly different forms, it can also be difficult to find ways to systematically compare the results, putting your data at risk for bias in the interpretation stage.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Non-probability sampling
• Quantitative research
• Inclusion and exclusion criteria

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to systematically measure variables and test hypotheses . Qualitative methods allow you to explore concepts and experiences in more detail.

In mixed methods research , you use both qualitative and quantitative data collection and analysis methods to answer your research question .

Data collection is the systematic process by which observations or measurements are gathered in research. It is used in many different contexts by academics, governments, businesses, and other organizations.

Triangulation in research means using multiple datasets, methods, theories and/or investigators to address a research question. It’s a research strategy that can help you enhance the validity and credibility of your findings.

Triangulation is mainly used in qualitative research , but it’s also commonly applied in quantitative research . Mixed methods research always uses triangulation.

These are four of the most common mixed methods designs :

• Convergent parallel: Quantitative and qualitative data are collected at the same time and analyzed separately. After both analyses are complete, compare your results to draw overall conclusions. 
• Embedded: Quantitative and qualitative data are collected at the same time, but within a larger quantitative or qualitative design. One type of data is secondary to the other.
• Explanatory sequential: Quantitative data is collected and analyzed first, followed by qualitative data. You can use this design if you think your qualitative data will explain and contextualize your quantitative findings.
• Exploratory sequential: Qualitative data is collected and analyzed first, followed by quantitative data. You can use this design if you think the quantitative data will confirm or validate your qualitative findings.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

George, T. (2023, June 22). Mixed Methods Research | Definition, Guide & Examples. Scribbr. Retrieved April 9, 2024, from https://www.scribbr.com/methodology/mixed-methods-research/

Is this article helpful?

Tegan George

Other students also liked, writing strong research questions | criteria & examples, what is quantitative research | definition, uses & methods, what is qualitative research | methods & examples, what is your plagiarism score.

• - Google Chrome

Intended for healthcare professionals

• Access provided by Google Indexer
• My email alerts
• BMA member login
• Username * Password * Forgot your log in details? Need to activate BMA Member Log In Log in via OpenAthens Log in via your institution

Search form

• Advanced search
• Search responses
• Search blogs
• Three techniques for...

Three techniques for integrating data in mixed methods studies

• Related content
• Peer review
• Alicia O’Cathain , professor 1 ,
• Elizabeth Murphy , professor 2 ,
• Jon Nicholl , professor 1
• 1 Medical Care Research Unit, School of Health and Related Research, University of Sheffield, Sheffield S1 4DA, UK
• 2 University of Leicester, Leicester, UK
• Correspondence to: A O’Cathain a.ocathain{at}sheffield.ac.uk
• Accepted 8 June 2010

Techniques designed to combine the results of qualitative and quantitative studies can provide researchers with more knowledge than separate analysis

Health researchers are increasingly using designs that combine qualitative and quantitative methods, and this is often called mixed methods research. 1 Integration—the interaction or conversation between the qualitative and quantitative components of a study—is an important aspect of mixed methods research, and, indeed, is essential to some definitions. 2 Recent empirical studies of mixed methods research in health show, however, a lack of integration between components, 3 4 which limits the amount of knowledge that these types of studies generate. Without integration, the knowledge yield is equivalent to that from a qualitative study and a quantitative study undertaken independently, rather than achieving a “whole greater than the sum of the parts.” 5

Barriers to integration have been identified in both health and social research. 6 7 One barrier is the absence of formal education in mixed methods research. Fortunately, literature is rapidly expanding to fill this educational gap, including descriptions of how to integrate data and findings from qualitative and quantitative methods. 8 9 In this article we outline three techniques that may help health researchers to integrate data or findings in their mixed methods studies and show how these might enhance knowledge generated from this approach.

Triangulation protocol

Researchers will often use qualitative and quantitative methods to examine different aspects of an overall research question. For example, they might use a randomised controlled trial to assess the effectiveness of a healthcare intervention and semistructured interviews with patients and health professionals to consider the way in which the intervention was used in the real world. Alternatively, they might use a survey of service users to measure satisfaction with a service and focus groups to explore views of care in more depth. Data are collected and analysed separately for each component to produce two sets of findings. Researchers will then attempt to combine these findings, sometimes calling this process triangulation. The term triangulation can be confusing because it has two meanings. 10 It can be used to describe corroboration between two sets of findings or to describe a process of studying a problem using different methods to gain a more complete picture. The latter meaning is commonly used in mixed methods research and is the meaning used here.

The process of triangulating findings from different methods takes place at the interpretation stage of a study when both data sets have been analysed separately (figure ⇓ ). Several techniques have been described for triangulating findings. They require researchers to list the findings from each component of a study on the same page and consider where findings from each method agree (convergence), offer complementary information on the same issue (complementarity), or appear to contradict each other (discrepancy or dissonance). 11 12 13 Explicitly looking for disagreements between findings from different methods is an important part of this process. Disagreement is not a sign that something is wrong with a study. Exploration of any apparent “inter-method discrepancy” may lead to a better understanding of the research question, 14 and a range of approaches have been used within health services research to explore inter-method discrepancy. 15

Point of application for three techniques for integrating data in mixed methods research

• Download figure
• Open in new tab
• Download powerpoint

The most detailed description of how to carry out triangulation is the triangulation protocol, 11 which although developed for multiple qualitative methods, is relevant to mixed methods studies. This technique involves producing a “convergence coding matrix” to display findings emerging from each component of a study on the same page. This is followed by consideration of where there is agreement, partial agreement, silence, or dissonance between findings from different components. This technique for triangulation is the only one to include silence—where a theme or finding arises from one data set and not another. Silence might be expected because of the strengths of different methods to examine different aspects of a phenomenon, but surprise silences might also arise that help to increase understanding or lead to further investigations.

The triangulation protocol moves researchers from thinking about the findings related to each method, to what Farmer and colleagues call meta-themes that cut across the findings from different methods. 11 They show a worked example of triangulation protocol, but we could find no other published example. However, similar principles were used in an iterative mixed methods study to understand patient and carer satisfaction with a new primary angioplasty service. 16 Researchers conducted semistructured interviews with 16 users and carers to explore their experiences and views of the new service. These were used to develop a questionnaire for a survey of 595 patients (and 418 of their carers) receiving either the new service or usual care. Finally, 17 of the patients who expressed dissatisfaction with aftercare and rehabilitation were followed up to explore this further in semistructured interviews. A shift of thinking to meta-themes led the researchers away from reporting the findings from the interviews, survey, and follow-up interviews sequentially to consider the meta-themes of speed and efficiency, convenience of care, and discharge and after care. The survey identified that a higher percentage of carers of patients using the new service rated the convenience of visiting the hospital as poor than those using usual care. The interviews supported this concern about the new service, but also identified that the weight carers gave to this concern was low in the context of their family member’s life being saved.

Morgan describes this move as the “third effort” because it occurs after analysis of the qualitative and the quantitative components. 17 It requires time and energy that must be planned into the study timetable. It is also useful to consider who will carry out the integration process. Farmer and colleagues require two researchers to work together during triangulation, which can be particularly important in mixed methods studies if different researchers take responsibility for the qualitative and quantitative components. 11

Following a thread

Moran-Ellis and colleagues describe a different technique for integrating the findings from the qualitative and quantitative components of a study, called following a thread. 18 They state that this takes place at the analysis stage of the research process (figure ⇑ ). It begins with an initial analysis of each component to identify key themes and questions requiring further exploration. Then the researchers select a question or theme from one component and follow it across the other components—they call this the thread. The authors do not specify steps in this technique but offer a visual model for working between datasets. An approach similar to this has been undertaken in health services research, although the researchers did not label it as such, probably because the technique has not been used frequently in the literature (box)

An example of following a thread 19

Adamson and colleagues explored the effect of patient views on the appropriate use of services and help seeking using a survey of people registered at a general practice and semistructured interviews. The qualitative (22 interviews) and quantitative components (survey with 911 respondents) took place concurrently.

The researchers describe what they call an iterative or cyclical approach to analysis. Firstly, the preliminary findings from the interviews generated a hypothesis for testing in the survey data. A key theme from the interviews concerned the self rationing of services as a responsible way of using scarce health care. This theme was then explored in the survey data by testing the hypothesis that people’s views of the appropriate use of services would explain their help seeking behaviour. However, there was no support for this hypothesis in the quantitative analysis because the half of survey respondents who felt that health services were used inappropriately were as likely to report help seeking for a series of symptoms presented in standardised vignettes as were respondents who thought that services were not used inappropriately. The researchers then followed the thread back to the interview data to help interpret this finding.

After further analysis of the interview data the researchers understood that people considered the help seeking of other people to be inappropriate, rather than their own. They also noted that feeling anxious about symptoms was considered to be a good justification for seeking care. The researchers followed this thread back into the survey data and tested whether anxiety levels about the symptoms in the standardised vignettes predicted help seeking behaviour. This second hypothesis was supported by the survey data. Following a thread led the researchers to conclude that patients who seek health care for seemingly minor problems have exceeded their thresholds for the trade-off between not using services inappropriately and any anxiety caused by their symptoms.

Mixed methods matrix

A unique aspect of some mixed methods studies is the availability of both qualitative and quantitative data on the same cases. Data from the qualitative and quantitative components can be integrated at the analysis stage of a mixed methods study (figure ⇑ ). For example, in-depth interviews might be carried out with a sample of survey respondents, creating a subset of cases for which there is both a completed questionnaire and a transcript. Cases may be individuals, groups, organisations, or geographical areas. 9 All the data collected on a single case can be studied together, focusing attention on cases, rather than variables or themes, within a study. The data can be examined in detail for each case—for example, comparing people’s responses to a questionnaire with their interview transcript. Alternatively, data on each case can be summarised and displayed in a matrix 8 9 20 along the lines of Miles and Huberman’s meta-matrix. 21 Within a mixed methods matrix, the rows represent the cases for which there is both qualitative and quantitative data, and the columns display different data collected on each case. This allows researchers to pay attention to surprises and paradoxes between types of data on a single case and then look for patterns across all cases 20 in a qualitative cross case analysis. 21

We used a mixed methods matrix to study the relation between types of team working and the extent of integration in mixed methods studies in health services research (table ⇓ ). 22 Quantitative data were extracted from the proposals, reports, and peer reviewed publications of 75 mixed methods studies, and these were analysed to describe the proportion of studies with integrated outputs such as mixed methods journal articles. Two key variables in the quantitative component were whether the study was assessed as attempting to integrate qualitative or quantitative data or findings and the type of publications produced. We conducted qualitative interviews with 20 researchers who had worked on some of these studies to explore how mixed methods research was practised, including how the team worked together.

Example of a mixed methods matrix for a study exploring the relationship between types of teams and integration between qualitative and quantitative components of studies* 22

• View inline

The shared cases between the qualitative and quantitative components were 21 mixed methods studies (because one interviewee had worked on two studies in the quantitative component). A matrix was formed with each of the 21 studies as a row. The first column of the matrix contained the study identification, the second column indicated whether integration had occurred in that project, and the third column the score for integration of publications emerging from the study. The rows were then ordered to show the most integrated cases first. This ordering of rows helped us to see patterns across rows.

The next columns were themes from the qualitative interview with a researcher from that project. For example, the first theme was about the expertise in qualitative research within the team and whether the interviewee reported this as adequate for the study. The matrix was then used in the context of the qualitative analysis to explore the issues that affected integration. In particular, it helped to identify negative cases (when someone in the analysis doesn’t fit with the conclusions the analysis is coming to) within the qualitative analysis to facilitate understanding. Interviewees reported the need for experienced qualitative researchers on mixed methods studies to ensure that the qualitative component was published, yet two cases showed that this was neither necessary nor sufficient. This pushed us to explore other factors in a research team that helped generate outputs, and integrated outputs, from a mixed methods study.

Themes from a qualitative study can be summarised to the point where they are coded into quantitative data. In the matrix (table ⇑ ), the interviewee’s perception of the adequacy of qualitative expertise on the team could have been coded as adequate=1 or not=2. This is called “quantitising” of qualitative data 23 ; coded data can then be analysed with data from the quantitative component. This technique has been used to great effect in healthcare research to identify the discrepancy between health improvement assessed using quantitative measures and with in-depth interviews in a randomised controlled trial. 24

We have presented three techniques for integration in mixed methods research in the hope that they will inspire researchers to explore what can be learnt from bringing together data from the qualitative and quantitative components of their studies. Using these techniques may give the process of integration credibility rather than leaving researchers feeling that they have “made things up.” It may also encourage researchers to describe their approaches to integration, allowing them to be transparent and helping them to develop, critique, and improve on these techniques. Most importantly, we believe it may help researchers to generate further understanding from their research.

We have presented integration as unproblematic, but it is not. It may be easier for single researchers to use these techniques than a large research team. Large teams will need to pay attention to team dynamics, considering who will take responsibility for integration and who will be taking part in the process. In addition, we have taken a technical stance here rather than paying attention to different philosophical beliefs that may shape approaches to integration. We consider that these techniques would work in the context of a pragmatic or subtle realist stance adopted by some mixed methods researchers. 25 Finally, it is important to remember that these techniques are aids to integration and are helpful only when applied with expertise.

Summary points

Health researchers are increasingly using designs which combine qualitative and quantitative methods

However, there is often lack of integration between methods

Three techniques are described that can help researchers to integrate data from different components of a study: triangulation protocol, following a thread, and the mixed methods matrix

Use of these methods will allow researchers to learn more from the information they have collected

Cite this as: BMJ 2010;341:c4587

Funding: Medical Research Council grant reference G106/1116

Competing interests: All authors have completed the unified competing interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare financial support for the submitted work from the Medical Research Council; no financial relationships with commercial entities that might have an interest in the submitted work; no spouses, partners, or children with relationships with commercial entities that might have an interest in the submitted work; and no non-financial interests that may be relevant to the submitted work.

Contributors: AOC wrote the paper. JN and EM contributed to drafts and all authors agreed the final version. AOC is guarantor.

Provenance and peer review: Not commissioned; externally peer reviewed.

• ↵ Lingard L, Albert M, Levinson W. Grounded theory, mixed methods and action research. BMJ 2008 ; 337 : a567 . OpenUrl FREE Full Text
• ↵ Creswell JW, Fetters MD, Ivankova NV. Designing a mixed methods study in primary care. Ann Fam Med 2004 ; 2 : 7 -12. OpenUrl Abstract / FREE Full Text
• ↵ Lewin S, Glenton C, Oxman AD. Use of qualitative methods alongside randomised controlled trials of complex healthcare interventions: methodological study. BMJ 2009 ; 339 : b3496 . OpenUrl Abstract / FREE Full Text
• ↵ O’Cathain A, Murphy E, Nicholl J. Integration and publications as indicators of ‘yield’ from mixed methods studies. J Mix Methods Res 2007 ; 1 : 147 -63. OpenUrl CrossRef Web of Science
• ↵ Barbour RS. The case for combining qualitative and quantitative approaches in health services research. J Health Serv Res Policy 1999 ; 4 : 39 -43. OpenUrl PubMed
• ↵ O’Cathain A, Nicholl J, Murphy E. Structural issues affecting mixed methods studies in health research: a qualitative study. BMC Med Res Methodol 2009 ; 9 : 82 . OpenUrl CrossRef PubMed
• ↵ Bryman A. Barriers to integrating quantitative and qualitative research. J Mix Methods Res 2007 ; 1 : 8 -22. OpenUrl CrossRef
• ↵ Creswell JW, Plano-Clark V. Designing and conducting mixed methods research . Sage, 2007 .
• ↵ Bazeley P. Analysing mixed methods data. In: Andrew S, Halcomb EJ, eds. Mixed methods research for nursing and the health sciences . Wiley-Blackwell, 2009 :84-118.
• ↵ Sandelowski M. Triangles and crystals: on the geometry of qualitative research. Res Nurs Health 1995 ; 18 : 569 -74. OpenUrl CrossRef PubMed Web of Science
• ↵ Farmer T, Robinson K, Elliott SJ, Eyles J. Developing and implementing a triangulation protocol for qualitative health research. Qual Health Res 2006 ; 16 : 377 -94. OpenUrl Abstract / FREE Full Text
• ↵ Foster RL. Addressing the epistemologic and practical issues in multimethod research: a procedure for conceptual triangulation. Adv Nurs Sci 1997 ; 20 : 1 -12. OpenUrl PubMed
• ↵ Erzerberger C, Prein G. Triangulation: validity and empirically based hypothesis construction. Qual Quant 1997 ; 31 : 141 -54. OpenUrl CrossRef Web of Science
• ↵ Fielding NG, Fielding JL. Linking data . Sage, 1986 .
• ↵ Moffatt S, White M, Mackintosh J, Howel D. Using quantitative and qualitative data in health services research—what happens when mixed method findings conflict? BMC Health Serv Res 2006 ; 6 : 28 . OpenUrl CrossRef PubMed
• ↵ Sampson FC, O’Cathain A, Goodacre S. Is primary angioplasty an acceptable alternative to thrombolysis? Quantitative and qualitative study of patient and carer satisfaction. Health Expectations (forthcoming).
• ↵ Morgan DL. Practical strategies for combining qualitative and quantitative methods: applications to health research. Qual Health Res 1998 ; 8 : 362 -76. OpenUrl Abstract / FREE Full Text
• ↵ Moran-Ellis J, Alexander VD, Cronin A, Dickinson M, Fielding J, Sleney J, et al. Triangulation and integration: processes, claims and implications. Qualitative Research 2006 ; 6 : 45 -59. OpenUrl Abstract / FREE Full Text
• ↵ Adamson J, Ben-Shlomo Y, Chaturvedi N, Donovan J. Exploring the impact of patient views on ‘appropriate’ use of services and help seeking: a mixed method study. Br J Gen Pract 2009 ; 59 : 496 -502. OpenUrl Web of Science
• ↵ Wendler MC. Triangulation using a meta-matrix. J Adv Nurs 2001 ; 35 : 521 -5. OpenUrl CrossRef PubMed Web of Science
• ↵ Miles M, Huberman A. Qualitative data analysis: an expanded sourcebook . Sage, 1994 .
• ↵ O’Cathain A, Murphy E, Nicholl J. Multidisciplinary, interdisciplinary or dysfunctional? Team working in mixed methods research. Qual Health Res 2008 ; 18 : 1574 -85. OpenUrl Abstract / FREE Full Text
• ↵ Sandelowski M. Combining qualitative and quantitative sampling, data collection, and analysis techniques in mixed-method studies. Res Nurs Health 2000 ; 23 : 246 -55. OpenUrl CrossRef PubMed Web of Science
• ↵ Campbell R, Quilty B, Dieppe P. Discrepancies between patients’ assessments of outcome: qualitative study nested within a randomised controlled trial. BMJ 2003 ; 326 : 252 -3. OpenUrl FREE Full Text
• ↵ Mays N, Pope C. Assessing quality in qualitative research. BMJ 2000 ; 320 : 50 -2. OpenUrl FREE Full Text

• What is mixed methods research?

Last updated

20 February 2023

Reviewed by

Miroslav Damyanov

By blending both quantitative and qualitative data, mixed methods research allows for a more thorough exploration of a research question. It can answer complex research queries that cannot be solved with either qualitative or quantitative research .

Analyze your mixed methods research

Dovetail streamlines analysis to help you uncover and share actionable insights

Mixed methods research combines the elements of two types of research: quantitative and qualitative.

Quantitative data is collected through the use of surveys and experiments, for example, containing numerical measures such as ages, scores, and percentages. 

Qualitative data involves non-numerical measures like beliefs, motivations, attitudes, and experiences, often derived through interviews and focus group research to gain a deeper understanding of a research question or phenomenon.

Mixed methods research is often used in the behavioral, health, and social sciences, as it allows for the collection of numerical and non-numerical data.

• When to use mixed methods research

Mixed methods research is a great choice when quantitative or qualitative data alone will not sufficiently answer a research question. By collecting and analyzing both quantitative and qualitative data in the same study, you can draw more meaningful conclusions. 

There are several reasons why mixed methods research can be beneficial, including generalizability, contextualization, and credibility. 

For example, let's say you are conducting a survey about consumer preferences for a certain product. You could collect only quantitative data, such as how many people prefer each product and their demographics. Or you could supplement your quantitative data with qualitative data, such as interviews and focus groups , to get a better sense of why people prefer one product over another.

It is important to note that mixed methods research does not only mean collecting both types of data. Rather, it also requires carefully considering the relationship between the two and method flexibility.

You may find differing or even conflicting results by combining quantitative and qualitative data . It is up to the researcher to then carefully analyze the results and consider them in the context of the research question to draw meaningful conclusions.

When designing a mixed methods study, it is important to consider your research approach, research questions, and available data. Think about how you can use different techniques to integrate the data to provide an answer to your research question.

• Mixed methods research design

A mixed methods research design  is   an approach to collecting and analyzing both qualitative and quantitative data in a single study.

Mixed methods designs allow for method flexibility and can provide differing and even conflicting results. Examples of mixed methods research designs include convergent parallel, explanatory sequential, and exploratory sequential.

By integrating data from both quantitative and qualitative sources, researchers can gain valuable insights into their research topic . For example, a study looking into the impact of technology on learning could use surveys to measure quantitative data on students' use of technology in the classroom. At the same time, interviews or focus groups can provide qualitative data on students' experiences and opinions.

• Types of mixed method research designs

Researchers often struggle to put mixed methods research into practice, as it is challenging and can lead to research bias. Although mixed methods research can reveal differences or conflicting results between studies, it can also offer method flexibility.

Designing a mixed methods study can be broken down into four types: convergent parallel, embedded, explanatory sequential, and exploratory sequential.

Convergent parallel

The convergent parallel design is when data collection and analysis of both quantitative and qualitative data occur simultaneously and are analyzed separately. This design aims to create mutually exclusive sets of data that inform each other. 

For example, you might interview people who live in a certain neighborhood while also conducting a survey of the same people to determine their satisfaction with the area.

Embedded design

The embedded design is when the quantitative and qualitative data are collected simultaneously, but the qualitative data is embedded within the quantitative data. This design is best used when you want to focus on the quantitative data but still need to understand how the qualitative data further explains it.

For instance, you may survey students about their opinions of an online learning platform and conduct individual interviews to gain further insight into their responses.

Explanatory sequential design

In an explanatory sequential design, quantitative data is collected first, followed by qualitative data. This design is used when you want to further explain a set of quantitative data with additional qualitative information.

An example of this would be if you surveyed employees at a company about their satisfaction with their job and then conducted interviews to gain more information about why they responded the way they did.

Exploratory sequential design

The exploratory sequential design collects qualitative data first, followed by quantitative data. This type of mixed methods research is used when the goal is to explore a topic before collecting any quantitative data.

An example of this could be studying how parents interact with their children by conducting interviews and then using a survey to further explore and measure these interactions.

Integrating data in mixed methods studies can be challenging, but it can be done successfully with careful planning.

No matter which type of design you choose, understanding and applying these principles can help you draw meaningful conclusions from your research.

• Strengths of mixed methods research

Mixed methods research designs combine the strengths of qualitative and quantitative data, deepening and enriching qualitative results with quantitative data and validating quantitative findings with qualitative data. This method offers more flexibility in designing research, combining theory generation and hypothesis testing, and being less tied to disciplines and established research paradigms.

Take the example of a study examining the impact of exercise on mental health. Mixed methods research would allow for a comprehensive look at the issue from different angles. 

Researchers could begin by collecting quantitative data through surveys to get an overall view of the participants' levels of physical activity and mental health. Qualitative interviews would follow this to explore the underlying dynamics of participants' experiences of exercise, physical activity, and mental health in greater detail.

Through a mixed methods approach, researchers could more easily compare and contrast their results to better understand the phenomenon as a whole.  

Additionally, mixed methods research is useful when there are conflicting or differing results in different studies. By combining both quantitative and qualitative data, mixed methods research can offer insights into why those differences exist.

For example, if a quantitative survey yields one result while a qualitative interview yields another, mixed methods research can help identify what factors influence these differences by integrating data from both sources.

Overall, mixed methods research designs offer a range of advantages for studying complex phenomena. They can provide insight into different elements of a phenomenon in ways that are not possible with either qualitative or quantitative data alone. Additionally, they allow researchers to integrate data from multiple sources to gain a deeper understanding of the phenomenon in question.  

• Challenges of mixed methods research

Mixed methods research is labor-intensive and often requires interdisciplinary teams of researchers to collaborate. It also has the potential to cost more than conducting a stand alone qualitative or quantitative study . 

Interpreting the results of mixed methods research can be tricky, as it can involve conflicting or differing results. Researchers must find ways to systematically compare the results from different sources and methods to avoid bias.

For example, imagine a situation where a team of researchers has employed an explanatory sequential design for their mixed methods study. After collecting data from both the quantitative and qualitative stages, the team finds that the two sets of data provide differing results. This could be challenging for the team, as they must now decide how to effectively integrate the two types of data in order to reach meaningful conclusions. The team would need to identify method flexibility and be strategic when integrating data in order to draw meaningful conclusions from the conflicting results.

• Advanced frameworks in mixed methods research

Mixed methods research offers powerful tools for investigating complex processes and systems, such as in health and healthcare.

Besides the three basic mixed method designs—exploratory sequential, explanatory sequential, and convergent parallel—you can use one of the four advanced frameworks to extend mixed methods research designs. These include multistage, intervention, case study , and participatory. 

This framework mixes qualitative and quantitative data collection methods in stages to gather a more nuanced view of the research question. An example of this is a study that first has an online survey to collect initial data and is followed by in-depth interviews to gain further insights.

Intervention

This design involves collecting quantitative data and then taking action, usually in the form of an intervention or intervention program. An example of this could be a research team who collects data from a group of participants, evaluates it, and then implements an intervention program based on their findings .

This utilizes both qualitative and quantitative research methods to analyze a single case. The researcher will examine the specific case in detail to understand the factors influencing it. An example of this could be a study of a specific business organization to understand the organizational dynamics and culture within the organization.

Participatory

This type of research focuses on the involvement of participants in the research process. It involves the active participation of participants in formulating and developing research questions, data collection, and analysis.

An example of this could be a study that involves forming focus groups with participants who actively develop the research questions and then provide feedback during the data collection and analysis stages.

The flexibility of mixed methods research designs means that researchers can choose any combination of the four frameworks outlined above and other methodologies , such as convergent parallel, explanatory sequential, and exploratory sequential, to suit their particular needs.

Through this method's flexibility, researchers can gain multiple perspectives and uncover differing or even conflicting results when integrating data.

When it comes to integration at the methods level, there are four approaches.

Connecting involves collecting both qualitative and quantitative data during different phases of the research.

Building involves the collection of both quantitative and qualitative data within a single phase.

Merging involves the concurrent collection of both qualitative and quantitative data.

Embedding involves including qualitative data within a quantitative study or vice versa.

• Techniques for integrating data in mixed method studies

Integrating data is an important step in mixed methods research designs. It allows researchers to gain further understanding from their research and gives credibility to the integration process. There are three main techniques for integrating data in mixed methods studies: triangulation protocol, following a thread, and the mixed methods matrix.

Triangulation protocol

This integration method combines different methods with differing or conflicting results to generate one unified answer.

For example, if a researcher wanted to know what type of music teenagers enjoy listening to, they might employ a survey of 1,000 teenagers as well as five focus group interviews to investigate this. The results might differ; the survey may find that rap is the most popular genre, whereas the focus groups may suggest rock music is more widely listened to. 

The researcher can then use the triangulation protocol to come up with a unified answer—such as that both rap and rock music are popular genres for teenage listeners. 

Following a thread

This is another method of integration where the researcher follows the same theme or idea from one method of data collection to the next. 

A research design that follows a thread starts by collecting quantitative data on a specific issue, followed by collecting qualitative data to explain the results. This allows whoever is conducting the research to detect any conflicting information and further look into the conflicting information to understand what is really going on.

For example, a researcher who used this research method might collect quantitative data about how satisfied employees are with their jobs at a certain company, followed by qualitative interviews to investigate why job satisfaction levels are low. They could then use the results to explore any conflicting or differing results, allowing them to gain a deeper understanding of job satisfaction at the company. 

By following a thread, the researcher can explore various research topics related to the original issue and gain a more comprehensive view of the issue.

Mixed methods matrix

This technique is a visual representation of the different types of mixed methods research designs and the order in which they should be implemented. It enables researchers to quickly assess their research design and adjust it as needed. 

The matrix consists of four boxes with four different types of mixed methods research designs: convergent parallel, explanatory sequential, exploratory sequential, and method flexibility. 

For example, imagine a researcher who wanted to understand why people don't exercise regularly. To answer this question, they could use a convergent parallel design, collecting both quantitative (e.g., survey responses) and qualitative (e.g., interviews) data simultaneously.

If the researcher found conflicting results, they could switch to an explanatory sequential design and collect quantitative data first, then follow up with qualitative data if needed. This way, the researcher can make adjustments based on their findings and integrate their data more effectively.

Mixed methods research is a powerful tool for understanding complex research topics. Using qualitative and quantitative data in one study allows researchers to understand their subject more deeply. 

Mixed methods research designs such as convergent parallel, explanatory sequential, and exploratory sequential provide method flexibility, enabling researchers to collect both types of data while avoiding the limitations of either approach alone.

However, it's important to remember that mixed methods research can produce differing or even conflicting results, so it's important to be aware of the potential pitfalls and take steps to ensure that data is being correctly integrated. If used effectively, mixed methods research can offer valuable insight into topics that would otherwise remain largely unexplored.

What is an example of mixed methods research?

An example of mixed methods research is a study that combines quantitative and qualitative data. This type of research uses surveys, interviews, and observations to collect data from multiple sources.

Which sampling method is best for mixed methods?

It depends on the research objectives, but a few methods are often used in mixed methods research designs. These include snowball sampling, convenience sampling, and purposive sampling. Each method has its own advantages and disadvantages.

What is the difference between mixed methods and multiple methods?

Mixed methods research combines quantitative and qualitative data in a single study. Multiple methods involve collecting data from different sources, such as surveys and interviews, but not necessarily combining them into one analysis. Mixed methods offer greater flexibility but can lead to differing or conflicting results when integrating data.

Get started today

Go from raw data to valuable insights with a flexible research platform

Editor’s picks

Last updated: 21 December 2023

Last updated: 16 December 2023

Last updated: 6 October 2023

Last updated: 5 March 2024

Last updated: 25 November 2023

Last updated: 15 February 2024

Last updated: 11 March 2024

Last updated: 12 December 2023

Last updated: 6 March 2024

Last updated: 10 April 2023

Last updated: 20 December 2023

Latest articles

Related topics, log in or sign up.

Get started for free

• News & Highlights

• Publications and Documents
• Postgraduate Education
• Browse Our Courses
• C/T Research Academy
• K12 Investigator Training
• Translational Innovator
• SMART IRB Reliance Request
• Biostatistics Consulting
• Regulatory Support
• Pilot Funding
• Informatics Program
• Community Engagement
• Diversity Inclusion
• Research Enrollment and Diversity
• Harvard Catalyst Profiles

Community Engagement Program

Supporting bi-directional community engagement to improve the relevance, quality, and impact of research.

• Getting Started
• Resources for Equity in Research
• Community-Engaged Student Practice Placement
• Maternal Health Equity
• Youth Mental Health
• Leadership and Membership
• Past Members
• Study Review Rubric
• Community Ambassador Initiative
• Implementation Science Working Group
• Past Webinars & Podcasts
• Policy Atlas
• Community Advisory Board

For more information:

Mixed methods research.

According to the National Institutes of Health , mixed methods strategically integrates or combines rigorous quantitative and qualitative research methods to draw on the strengths of each. Mixed method approaches allow researchers to use a diversity of methods, combining inductive and deductive thinking, and offsetting limitations of exclusively quantitative and qualitative research through a complementary approach that maximizes strengths of each data type and facilitates a more comprehensive understanding of health issues and potential resolutions.¹ Mixed methods may be employed to produce a robust description and interpretation of the data, make quantitative results more understandable, or understand broader applicability of small-sample qualitative findings.

Integration

This refers to the ways in which qualitative and quantitative research activities are brought together to achieve greater insight. Mixed methods is not simply having quantitative and qualitative data available or analyzing and presenting data findings separately. The integration process can occur during data collection, analysis, or in the presentation of results.

¹ NIH Office of Behavioral and Social Sciences Research: Best Practices for Mixed Methods Research in the Health Sciences

Basic Mixed Methods Research Designs 

View image description .

Five Key Questions for Getting Started

• What do you want to know?
• What will be the detailed quantitative, qualitative, and mixed methods research questions that you hope to address?
• What quantitative and qualitative data will you collect and analyze?
• Which rigorous methods will you use to collect data and/or engage stakeholders?
• How will you integrate the data in a way that allows you to address the first question?

Rationale for Using Mixed Methods

• Obtain different, multiple perspectives: validation
• Build comprehensive understanding
• Explain statistical results in more depth
• Have better contextualized measures
• Track the process of program or intervention
• Study patient-centered outcomes and stakeholder engagement

Sample Mixed Methods Research Study

The EQUALITY study used an exploratory sequential design to identify the optimal patient-centered approach to collect sexual orientation data in the emergency department.

Qualitative Data Collection and Analysis : Semi-structured interviews with patients of different sexual orientation, age, race/ethnicity, as well as healthcare professionals of different roles, age, and race/ethnicity.

Builds Into : Themes identified in the interviews were used to develop questions for the national survey.

Quantitative Data Collection and Analysis : Representative national survey of patients and healthcare professionals on the topic of reporting gender identity and sexual orientation in healthcare.

Other Resources:

  Introduction to Mixed Methods Research : Harvard Catalyst’s eight-week online course offers an opportunity for investigators who want to understand and apply a mixed methods approach to their research.

Best Practices for Mixed Methods Research in the Health Sciences [PDF] : This guide provides a detailed overview of mixed methods designs, best practices, and application to various types of grants and projects.

Mixed Methods Research Training Program for the Health Sciences (MMRTP ): Selected scholars for this summer training program, hosted by Johns Hopkins’ Bloomberg School of Public Health, have access to webinars, resources, a retreat to discuss their research project with expert faculty, and are matched with mixed methods consultants for ongoing support.

Michigan Mixed Methods : University of Michigan Mixed Methods program offers a variety of resources, including short web videos and recommended reading.

To use a mixed methods approach, you may want to first brush up on your qualitative skills. Below are a few helpful resources specific to qualitative research:

• Qualitative Research Guidelines Project : A comprehensive guide for designing, writing, reviewing and reporting qualitative research.
• Fundamentals of Qualitative Research Methods – What is Qualitative Research : A six-module web video series covering essential topics in qualitative research, including what is qualitative research and how to use the most common methods, in-depth interviews, and focus groups.

View PDF of the above information.

• Privacy Policy

Buy Me a Coffee

Home » Mixed Methods Research – Types & Analysis

Mixed Methods Research – Types & Analysis

Table of Contents

Mixed Methods Research

Mixed methods research is an approach to research that combines both quantitative and qualitative research methods in a single study or research project. It is a methodological approach that involves collecting and analyzing both numerical (quantitative) and narrative (qualitative) data to gain a more comprehensive understanding of a research problem.

Types of Mixed Research

There are different types of mixed methods research designs that researchers can use, depending on the research question, the available data, and the resources available. Here are some common types:

Convergent Parallel Design

This design involves collecting both qualitative and quantitative data simultaneously, analyzing them separately, and then merging the findings to draw conclusions. The qualitative and quantitative data are given equal weight, and the findings are integrated during the interpretation phase.

Sequential Explanatory Design

In this design, the researcher collects and analyzes quantitative data first, and then uses qualitative data to explain or elaborate on the quantitative findings. The researcher may use the qualitative data to clarify unexpected or contradictory results from the quantitative analysis.

Sequential Exploratory Design

This design involves collecting qualitative data first, analyzing it, and then collecting and analyzing quantitative data to confirm or refute the qualitative findings. Qualitative data are used to generate hypotheses that are tested using quantitative data.

Concurrent Triangulation Design

This design involves collecting both qualitative and quantitative data concurrently and then comparing the results to find areas of agreement and disagreement. The findings are integrated during the interpretation phase to provide a more comprehensive understanding of the research question.

Concurrent Nested Design

This design involves collecting one type of data as the primary method and then using the other type of data to elaborate or clarify the primary data. For example, a researcher may use quantitative data as the primary method and qualitative data as a secondary method to provide more context and detail.

Transformative Design

This design involves using mixed methods research to not only understand the research question but also to bring about social change or transformation. The research is conducted in collaboration with stakeholders and aims to generate knowledge that can be used to improve policies, programs, and practices.

Concurrent Embedded Design

Concurrent embedded design is a type of mixed methods research design in which one type of data is embedded within another type of data. This design involves collecting both quantitative and qualitative data simultaneously, with one type of data being the primary method and the other type of data being the secondary method. The secondary method is embedded within the primary method, meaning that it is used to provide additional information or to clarify the primary data.

Data Collection Methods

Here are some common data collection methods used in mixed methods research:

Surveys are a common quantitative data collection method used in mixed methods research. Surveys involve collecting standardized responses to a set of questions from a sample of participants. Surveys can be conducted online, in person, or over the phone.

Interviews are a qualitative data collection method that involves asking open-ended questions to gather in-depth information about a participant’s experiences, perspectives, and opinions. Interviews can be conducted in person, over the phone, or online.

Focus groups

Focus groups are a qualitative data collection method that involves bringing together a small group of participants to discuss a topic or research question. The group is facilitated by a researcher, and the discussion is recorded and analyzed for themes and patterns.

Observations

Observations are a qualitative data collection method that involves systematically watching and recording behavior in a natural setting. Observations can be structured or unstructured and can be used to gather information about behavior, interactions, and context.

Document Analysis

Document analysis is a qualitative data collection method that involves analyzing existing documents, such as reports, policy documents, or media articles. Document analysis can be used to gather information about trends, policy changes, or public attitudes.

Experimentation

Experimentation is a quantitative data collection method that involves manipulating one or more variables and measuring their effects on an outcome. Experiments can be conducted in a laboratory or in a natural setting.

Data Analysis Methods

Mixed methods research involves using both quantitative and qualitative data analysis methods to analyze data collected through different methods. Here are some common data analysis methods used in mixed methods research:

Quantitative Data Analysis

Quantitative data collected through surveys or experiments can be analyzed using statistical methods. Statistical analysis can be used to identify relationships between variables, test hypotheses, and make predictions. Common statistical methods used in quantitative data analysis include regression analysis, t-tests, ANOVA, and correlation analysis.

Qualitative Data Analysis

Qualitative data collected through interviews, focus groups, or observations can be analyzed using a variety of qualitative data analysis methods. These methods include content analysis, thematic analysis, narrative analysis, and grounded theory. Qualitative data analysis involves identifying themes and patterns in the data, interpreting the meaning of the data, and drawing conclusions based on the findings.

Integration of Data

The integration of quantitative and qualitative data involves combining the results from both types of data analysis to gain a more comprehensive understanding of the research question. Integration can involve either a concurrent or sequential approach. Concurrent integration involves analyzing quantitative and qualitative data at the same time, while sequential integration involves analyzing one type of data first and then using the results to inform the analysis of the other type of data.

Triangulation

Triangulation involves using multiple sources or types of data to validate or corroborate findings. This can involve using both quantitative and qualitative data or multiple qualitative methods. Triangulation can enhance the credibility and validity of the research findings.

Mixed Methods Meta-analysis

Mixed methods meta-analysis involves the systematic review and synthesis of findings from multiple studies that use mixed methods designs. This involves combining quantitative and qualitative data from multiple studies to gain a broader understanding of a research question.

How to conduct Mixed Methods Research

Here are some general steps for conducting mixed methods research:

• Identify the research problem: The first step is to clearly define the research problem and determine if mixed methods research is appropriate for addressing it.
• Design the study: The research design should include both qualitative and quantitative data collection and analysis methods. The specific design will depend on the research question and the purpose of the study.
• Collect data : Data collection involves collecting both qualitative and quantitative data through various methods such as surveys, interviews, observations, and document analysis.
• Analyze data: Both qualitative and quantitative data need to be analyzed separately and then integrated. Analysis methods may include coding, statistical analysis, and thematic analysis.
• Interpret results: The results of the analysis should be interpreted, taking into account both the quantitative and qualitative findings. This involves integrating the results and identifying any patterns, themes, or discrepancies.
• Draw conclusions : Based on the interpretation of the results, conclusions should be drawn that address the research question and objectives.
• Report findings: Finally, the findings should be reported in a clear and concise manner, using both quantitative and qualitative data to support the conclusions.

Applications of Mixed Methods Research

Mixed methods research can be applied to a wide range of research fields and topics, including:

• Education : Mixed methods research can be used to evaluate educational programs, assess the effectiveness of teaching methods, and investigate student learning experiences.
• Health and social sciences: Mixed methods research can be used to study health interventions, understand the experiences of patients and their families, and assess the effectiveness of social programs.
• Business and management: Mixed methods research can be used to investigate customer satisfaction, assess the impact of marketing campaigns, and analyze the effectiveness of management strategies.
• Psychology : Mixed methods research can be used to explore the experiences and perspectives of individuals with mental health issues, investigate the impact of psychological interventions, and assess the effectiveness of therapy.
• Sociology : Mixed methods research can be used to study social phenomena, investigate the experiences and perspectives of marginalized groups, and assess the impact of social policies.
• Environmental studies: Mixed methods research can be used to assess the impact of environmental policies, investigate public perceptions of environmental issues, and analyze the effectiveness of conservation strategies.

Examples of Mixed Methods Research

Here are some examples of Mixed-Methods research:

• Evaluating a school-based mental health program: A researcher might use a concurrent embedded design to evaluate a school-based mental health program. The researcher might collect quantitative data through surveys and qualitative data through interviews with students and teachers. The quantitative data might be analyzed using statistical methods, while the qualitative data might be analyzed using thematic analysis. The results of the two types of data analysis could be integrated to provide a comprehensive evaluation of the program’s effectiveness.
• Understanding patient experiences of chronic illness: A researcher might use a sequential explanatory design to investigate patient experiences of chronic illness. The researcher might collect quantitative data through surveys and then use the results of the survey to inform the selection of participants for qualitative interviews. The qualitative data might be analyzed using content analysis to identify common themes in the patients’ experiences.
• Assessing the impact of a new public transportation system : A researcher might use a concurrent triangulation design to assess the impact of a new public transportation system. The researcher might collect quantitative data through surveys and qualitative data through focus groups with community members. The results of the two types of data analysis could be triangulated to provide a more comprehensive understanding of the impact of the new transportation system on the community.
• Exploring teacher perceptions of technology integration in the classroom: A researcher might use a sequential exploratory design to investigate teacher perceptions of technology integration in the classroom. The researcher might collect qualitative data through in-depth interviews with teachers and then use the results of the interviews to develop a survey. The quantitative data might be analyzed using descriptive statistics to identify trends in teacher perceptions.

When to use Mixed Methods Research

Mixed methods research is typically used when a research question cannot be fully answered by using only quantitative or qualitative methods. Here are some common situations where mixed methods research is appropriate:

• When the research question requires a more comprehensive understanding than can be achieved by using only quantitative or qualitative methods.
• When the research question requires both an exploration of individuals’ experiences, perspectives, and attitudes, as well as the measurement of objective outcomes and variables.
• When the research question requires the examination of a phenomenon in its natural setting and context, which can be achieved by collecting rich qualitative data, as well as the generalization of findings to a larger population, which can be achieved through the use of quantitative methods.
• When the research question requires the integration of different types of data or perspectives, such as combining data collected from participants with data collected from stakeholders or experts.
• When the research question requires the validation of findings obtained through one method by using another method.
• When the research question involves studying a complex phenomenon that cannot be understood by using only one method, such as studying the impact of a policy on a community’s well-being.
• When the research question involves studying a topic that has not been well-researched, and using mixed methods can help provide a more comprehensive understanding of the topic.

Purpose of Mixed Methods Research

The purpose of mixed methods research is to provide a more comprehensive understanding of a research problem than can be obtained through either quantitative or qualitative methods alone.

Mixed methods research is particularly useful when the research problem is complex and requires a deep understanding of the context and subjective experiences of participants, as well as the ability to generalize findings to a larger population. By combining both qualitative and quantitative methods, researchers can obtain a more complete picture of the research problem and its underlying mechanisms, as well as test hypotheses and identify patterns that may not be apparent with only one method.

Overall, mixed methods research aims to provide a more holistic and nuanced understanding of the research problem, allowing researchers to draw more valid and reliable conclusions, make more informed decisions, and develop more effective interventions and policies.

Advantages of Mixed Methods Research

Mixed methods research offers several advantages over using only qualitative or quantitative research methods. Here are some of the main advantages of mixed methods research:

• Comprehensive understanding: Mixed methods research provides a more comprehensive understanding of the research problem by combining both qualitative and quantitative data, which allows for a more nuanced interpretation of the data.
• Triangulation : Mixed methods research allows for triangulation, which is the use of multiple sources of data to verify findings. This improves the validity and reliability of the research.
• Addressing limitations: Mixed methods research can address the limitations of qualitative or quantitative research by compensating for the weaknesses of each method.
• Flexibility : Mixed methods research is flexible, allowing researchers to adapt the research design and methods as needed to best address the research question.
• Validity : Mixed methods research can increase the validity of the research by using multiple methods to measure the same concept.
• Generalizability : Mixed methods research can improve the generalizability of the findings by using quantitative data to test the applicability of qualitative findings to a larger population.
• Practical applications: Mixed methods research is useful for developing practical applications, such as interventions or policies, as it provides a more comprehensive understanding of the research problem.

Limitations of Mixed Methods Research

Here are some of the main limitations of mixed methods research:

• Time-consuming: Mixed methods research can be time-consuming and may require more resources than using only one research method.
• Complex data analysis: Integrating qualitative and quantitative data can be challenging and requires specialized skills for data analysis.
• Sampling bias: Mixed methods research can be subject to sampling bias, particularly if the sampling strategies for the qualitative and quantitative components are not aligned.
• Validity and reliability: Mixed methods research requires careful attention to the validity and reliability of both the qualitative and quantitative data, as well as the integration of the two data types.
• Difficulty in balancing the two methods: Mixed methods research can be difficult to balance the qualitative and quantitative methods effectively, particularly if one method dominates the other.
• Theoretical and philosophical issues: Mixed methods research raises theoretical and philosophical questions about the compatibility of qualitative and quantitative research methods and the underlying assumptions about the nature of reality and knowledge.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Questionnaire – Definition, Types, and Examples

Case Study – Methods, Examples and Guide

Observational Research – Methods and Guide

Quantitative Research – Methods, Types and...

Qualitative Research Methods

Explanatory Research – Types, Methods, Guide

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Methodology
• Mixed Methods Research | Definition, Guide, & Examples

Mixed Methods Research | Definition, Guide, & Examples

Published on 4 April 2022 by Tegan George . Revised on 25 October 2022.

Mixed methods research combines elements of quantitative research and qualitative research in order to answer your research question . Mixed methods can help you gain a more complete picture than a standalone quantitative or qualitative study, as it integrates benefits of both methods.

Mixed methods research is often used in the behavioral, health, and social sciences, especially in multidisciplinary settings and complex situational or societal research.

• To what extent does the frequency of traffic accidents ( quantitative ) reflect cyclist perceptions of road safety ( qualitative ) in Amsterdam?
• How do student perceptions of their school environment ( qualitative ) relate to differences in test scores ( quantitative ) ?
• How do interviews about job satisfaction at Company X ( qualitative ) help explain year-over-year sales performance and other KPIs ( quantitative ) ?
• How can voter and non-voter beliefs about democracy ( qualitative ) help explain election turnout patterns ( quantitative ) in Town X?
• How do average hospital salary measurements over time (quantitative) help to explain nurse testimonials about job satisfaction (qualitative) ?

Table of contents

When to use mixed methods research, mixed methods research designs, benefits of mixed methods research, disadvantages of mixed methods research, frequently asked questions about mixed methods research.

Mixed methods research may be the right choice if your research process suggests that quantitative or qualitative data alone will not sufficiently answer your research question. There are several common reasons for using mixed methods research:

• Generalisability : Qualitative research usually has a smaller sample size , and thus is not generalisable . In mixed methods research, this comparative weakness is mitigated by the comparative strength of ‘large N’, externally valid quantitative research.
• Contextualisation: Mixing methods allows you to put findings in context and add richer detail to your conclusions. Using qualitative data to illustrate quantitative findings can help ‘put meat on the bones’ of your analysis.
• Credibility: Using different methods to collect data on the same subject can make your results more credible. If the qualitative and quantitative data converge, this strengthens the validity of your conclusions. This process is called triangulation .

As you formulate your research question , try to directly address how qualitative and quantitative methods will be combined in your study. If your research question can be sufficiently answered via standalone quantitative or qualitative analysis, a mixed methods approach may not be the right fit.

Keep in mind that mixed methods research doesn’t just mean collecting both types of data; you need to carefully consider the relationship between the two and how you’ll integrate them into coherent conclusions. Mixed methods can be very challenging to put into practice, so it’s a less common choice than standalone qualitative or qualitative research.

Prevent plagiarism, run a free check.

There are different types of mixed methods research designs . The differences between them relate to the aim of the research, the timing of the data collection , and the importance given to each data type.

As you design your mixed methods study, also keep in mind:

• Your research approach ( inductive vs deductive )
• Your research questions
• What kind of data is already available for you to use
• What kind of data you’re able to collect yourself.

Here are a few of the most common mixed methods designs.

Convergent parallel

In a convergent parallel design, you collect quantitative and qualitative data at the same time and analyse them separately. After both analyses are complete, compare your results to draw overall conclusions.

• On the qualitative side, you analyse cyclist complaints via the city’s database and on social media to find out which areas are perceived as dangerous and why.
• On the quantitative side, you analyse accident reports in the city’s database to find out how frequently accidents occur in different areas of the city.

In an embedded design, you collect and analyse both types of data at the same time, but within a larger quantitative or qualitative design. One type of data is secondary to the other.

This is a good approach to take if you have limited time or resources. You can use an embedded design to strengthen or supplement your conclusions from the primary type of research design.

Explanatory sequential

In an explanatory sequential design, your quantitative data collection and analysis occurs first, followed by qualitative data collection and analysis.

You should use this design if you think your qualitative data will explain and contextualise your quantitative findings.

Exploratory sequential

In an exploratory sequential design, qualitative data collection and analysis occurs first, followed by quantitative data collection and analysis.

You can use this design to first explore initial questions and develop hypotheses. Then you can use the quantitative data to test or confirm your qualitative findings.

‘Best of both worlds’ analysis

Combining the two types of data means you benefit from both the detailed, contextualised insights of qualitative data and the generalisable, externally valid insights of quantitative data. The strengths of one type of data often mitigate the weaknesses of the other.

For example, solely quantitative studies often struggle to incorporate the lived experiences of your participants, so adding qualitative data deepens and enriches your quantitative results.

Solely qualitative studies are often not very generalisable, only reflecting the experiences of your participants, so adding quantitative data can validate your qualitative findings.

Method flexibility

Mixed methods are less tied to disciplines and established research paradigms. They offer more flexibility in designing your research, allowing you to combine aspects of different types of studies to distill the most informative results.

Mixed methods research can also combine theory generation and hypothesis testing within a single study, which is unusual for standalone qualitative or quantitative studies.

Mixed methods research is very labour-intensive. Collecting, analysing, and synthesising two types of data into one research product takes a lot of time and effort, and often involves interdisciplinary teams of researchers rather than individuals. For this reason, mixed methods research has the potential to cost much more than standalone studies.

Differing or conflicting results

If your analysis yields conflicting results, it can be very challenging to know how to interpret them in a mixed methods study. If the quantitative and qualitative results do not agree or you are concerned you may have confounding variables , it can be unclear how to proceed.

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to test a hypothesis by systematically collecting and analysing data, while qualitative methods allow you to explore ideas and experiences in depth.

In mixed methods research , you use both qualitative and quantitative data collection and analysis methods to answer your research question .

Data collection is the systematic process by which observations or measurements are gathered in research. It is used in many different contexts by academics, governments, businesses, and other organisations.

Triangulation in research means using multiple datasets, methods, theories and/or investigators to address a research question. It’s a research strategy that can help you enhance the validity and credibility of your findings.

Triangulation is mainly used in qualitative research , but it’s also commonly applied in quantitative research . Mixed methods research always uses triangulation.

These are four of the most common mixed methods designs :

• Convergent parallel: Quantitative and qualitative data are collected at the same time and analysed separately. After both analyses are complete, compare your results to draw overall conclusions. 
• Embedded: Quantitative and qualitative data are collected at the same time, but within a larger quantitative or qualitative design. One type of data is secondary to the other.
• Explanatory sequential: Quantitative data is collected and analysed first, followed by qualitative data. You can use this design if you think your qualitative data will explain and contextualise your quantitative findings.
• Exploratory sequential: Qualitative data is collected and analysed first, followed by quantitative data. You can use this design if you think the quantitative data will confirm or validate your qualitative findings.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

George, T. (2022, October 25). Mixed Methods Research | Definition, Guide, & Examples. Scribbr. Retrieved 9 April 2024, from https://www.scribbr.co.uk/research-methods/mixed-methods/

Is this article helpful?

Tegan George

• Open access
• Published: 12 April 2024

Evaluation of an HIV homecare program for lost-to-follow-up populations: a mixed methods study in Detroit, Michigan

• L. V. Bonadonna 1 , 3 ,
• E. Guerrero 2 ,
• T. McClendon 4 ,
• S. Union 4 ,
• D. Kabbani 3 ,
• D. Wittmann 4 ,
• J. Cohn 3 &
• J. Veltman 5  

AIDS Research and Therapy volume  21 , Article number:  21 ( 2024 ) Cite this article

Metrics details

Maintaining people living with HIV (PLWHIV) in clinical care is a global priority. In the Metro Detroit area of Michigan, approximately 30% of PLWHIV are out of care. To re-engage lost-to-follow-up patients, Wayne Health Infectious Disease clinic launched an innovative Homecare program in 2017. In addition to home healthcare delivery, the program included links to community resources and quarterly community meetings. We aimed to evaluate Homecare’s impact on participants’ ability to stay engaged in HIV care and reach viral suppression. We included data from PLWHIV and their healthcare workers.

We used a convergent mixed-methods design, including first year program record review, semi-structured interviews, and a validated Likert scale questionnaire rating illness perception before and after Homecare. Interview data were collected from 15 PLWHIV in Metro Detroit and two healthcare workers responsible for program delivery. Semi-structured interviews focused on obstacles to clinic-based care, support networks, and illness perceptions. Interview data were transcribed and analyzed using a thematic approach. A fully coded analysis was used to create a conceptual framework of factors contributing to Homecare’s success. Means in eight categories of the Brief Illness Perception (IPQ) were compared using paired T-tests.

In the first year of Homecare, 28 of 34 participants (82%) became virally suppressed at least once. The program offered (1) social support and stigma reduction through strong relationships with healthcare workers, (2) removal of physical and resource barriers such as transportation, and (3) positive changes in illness perceptions. PLWHIV worked towards functional coping strategies, including improvements in emotional regulation, acceptance of their diagnosis, and more positive perspectives of control. Brief-IPQ showed significant changes in six domains before and after Homecare.

Homecare offers an innovative system for successfully re-engaging and maintaining lost-to-follow-up PLWHIV in care. These findings have implications for HIV control efforts and could inform the development of future programs for difficult to reach populations.

Although steady advances have been made in reducing AIDS-related deaths, efforts to reach the 90-90-90 goals defined by the Joint United National Programme on HIV/ AIDS (UNAIDS) have not moved fast enough [ 1 ]. By 2020, UNAIDS aimed to diagnose 90% of all people living with human immunodeficiency virus (PLWHIV), treat 90% of all those diagnosed with antiretroviral therapy (ART), and achieve viral suppression for 90% of people taking ART worldwide [ 2 ]. This goal has not been met, but as the world looks to achieve the 95-95-95 goals by 2030, retention in HIV care is highlighted as a major global priority for controlling disease progression and reducing mortality [ 3 ].

In the United States, there are substantial numbers of PLWHIV not maintained in clinical care [ 4 ]. In the state of Michigan, specifically, 68% of PLWHIV were retained in care and only 60% achieved viral suppression in 2020 [ 5 ]. A variety of research trials have tried to improve retention in care, including the use of peer navigators, case management, peer counseling, and/or economic incentives. Unfortunately, these programs have only had modest effect [ 6 , 7 ]. Documented barriers to HIV clinical care have included poor patient-provider relationships [ 8 ], lack of social support [ 9 ], stigma [ 10 ], financial stress [ 11 ], and complex lifelong medication regimens [ 12 ] – especially in individuals with comorbid conditions [ 13 ]. There are also reports of mistrust in the healthcare system, particularly among African American communities [ 14 ].

To address gaps in the HIV care continuum, Wayne Health Infectious Disease (ID) clinic created the Homecare program to bring outpatient care to the homes of PLWHIV lost-to-follow-up in the Metro Detroit area of Michigan [ 15 ]. Launched in September 2017, Homecare focused on retention to care, and aimed to engage PLWHIV who had not visited a clinic or had HIV specific labs (CD4 or viral load) drawn in one year or longer. The program was offered to PLWHIV lost-to-follow-up at Wayne Health ID clinic and advertised to the Detroit Public Health Departments’ Data to Care Program and other community-based organizations (CBO). We conducted a convergent mixed-methods study to understand Homecare’s impact on participants’ ability to stay engaged in care and reach virologic suppression of HIV. More specifically, we aimed to investigate how Homecare engaged PLWHIV differently from clinic-based care, and if and why PLWHIV changed their perceptions of HIV during enrollment.

Study design

We conducted a convergent mixed methods study. We reviewed first-year Homecare Program data and then designed, collected, and analyzed qualitative interview data and quantitative survey data at the same time. The qualitative interview data and quantitative survey data represent our two data sets, embodying a convergent study design as described by Creswell [ 16 ]. This design is typically used in studies where phenomena are new or relatively unexplored [ 16 ]. We compared and merged qualitative interviews describing Homecare participants’ lived experiences with first year program data and a validated survey instrument after all individual quantitative and qualitative analyses were complete. We identified and evaluated important socioeconomic, behavioral, and structural factors mediating Homecare through: (1) semi-structured interviews and (2) a brief illness perception questionnaire.

Interview participants

From August to December 2019, we interviewed PLWHIV participating in Homecare who willingly gave informed, written consent to participate in research. We recruited PLWHIV ( N  = 15) by convenience sampling, whereby we telephoned or messaged PLWHIV and gauged interest in participation. In three cases, we approached PLWHIV after they had visited clinic. All Homecare participants were eligible to participate, however, four did not respond, three were hospitalized, two were not able to communicate verbally, and others declined for various reasons such as lack of time, and dislike of surveys. We continued to elicit interviews until our sample size reached theoretical saturation, described in qualitative methods as the point when no new ideas emerged from the interview [ 17 ]. Subsequent data collection activities were conducted in-clinic or the PLWHIV’s home or requested location. We conducted one interview with the medical assistant and nurse practitioner responsible for delivering Homecare to elicit their perspective on program effectiveness and compare their evaluation with PLWHIV responses.

Intervention

Once enrolled in Homecare, PLWHIV were visited by a medical assistant and nurse practitioner trained in trauma informed care and motivational interviewing. Under supervision by an ID attending physician, medical staff visited PLWHIV at their residence monthly until they reached viral suppression, defined as less than 200 HIV RNA copies in a milliliter (copies/mL) of blood. Afterwards, they were visited every three months to maintain clinical care. PLWHIV were considered “retained in care” if they completed two or more HIV medical visits in one calendar year. Homecare staff wore street clothes during home visits and arrived in an unmarked car to maintain privacy. They asked PLWHIV if they were alone prior to initiating visits, and if others in the home were aware of their HIV status. For those PLWHIV who chose to maintain complete confidentiality, Homecare staff would vocalize “your condition” instead of “HIV” during care. Additionally, staff remained careful of tone and visibility of documents and laptop while in the home. If Homecare staff encountered insect infestation or hostile environment, they would transition PLWHIV back to clinic appointments until the issue could be resolved. Some PLWHIV also requested clinic appointments if their living situation changed.

Available medical services through Homecare were comprehensive, including in-home breast, pelvic and rectal exams, blood draws, behavioral health screenings, counseling, STI treatment, immunizations, and injectable contraception. Home visits also included an environmental assessment of the home and links to CBOs for needed resources, such as houseware and other social assistance. Homecare workers had a dedicated cell phone for direct communication with patients. They used text message, video chat, and social media platforms between appointments to stay connected with patients. Homecare also provided quarterly community meetings where PLWHIV could come together to talk about shared experiences, enjoy a meal, and participate in creative activities. Homecare emphasized the role of cultural humility as a vehicle for effective social support through healthcare provider selection and training. Staff attended workshops on motivational interviewing and trauma-based care to empower PLWHIV to manage treatment with ART. To evaluate the effectiveness of the Homecare program, we performed a convergent mixed-methods study from August to December 2019.

Data collection

Quantitative data collection.

We reviewed first year program records with data from all PLWHIV enrolled in Homecare from September 2017 to September 2018 to evaluate retention to care and viral suppression. Afterwards and separately, we surveyed 15 PLWHIV using the Brief Illness Perception Questionnaire (IPQ) during qualitative interviews between August and December 2019. Validated by Broadbent et al. [ 18 ], the Brief-IPQ allowed us to measure cognitive representations of illness perceptions before and after participation in the HIV Homecare program. The eight-item questionnaire employs a 10-point Likert scale to measure domains of consequence, timeline, personal control, treatment control, identity, concern, understanding, and emotional response to living with HIV. A choice of 0 indicated the lowest perception of each domain, and a score of 10 indicated the highest. At the time of the interview, we asked PLWHIV to rank their perceptions before Homecare, and then asked them to rank their current perceptions.

Qualitative data collection

All PLWHIV were interviewed using a semi-structured interview guide, focusing on themes of (a) social supports/network, (b) economic environment, (c) trust in the healthcare system, and (d) motivation and personal agency. We focused on themes identified as key modifiers for retention to care by these health professionals, and supported by literature review [ 8 , 9 , 10 , 11 ]. We explored the differences between clinic-based care and Homecare by specifically prompting PLWHIV to describe their experiences in both systems. Our goal was to collect rich descriptions of how Homecare changes the experience and availability of biomedical and psychosocial care in comparison to a traditional clinic model. The interview guide was developed in collaboration with three health professionals with 30 years combined experience working with PLWHIV. The guide was piloted with one PLWHIV and then modified based on their responses. The final interview guide can be found in the appendix. All interviewers were conducted by the first author. Interviews lasted 14 to 58 min, with a median time of 35 min. Results of the pilot interview are included here. To triangulate data sources, we also interviewed Homecare program staff.

All interviews were recorded, transcribed, and uploaded into Atlas.ti qualitative data analysis software. We conducted a data-driven thematic analysis, as described by Braun et al. [ 19 ]. We proceeded inductively, reading and re-reading transcripts for deep familiarization, developing a codebook, and then applying codes to transcripts, making frequent comparisons between texts [ 19 ]. All transcripts were coded by the first author, and approximately 40% were independently coded by the fourth author, reviewed, and discussed for reliability. After revising codes, and reaching consensus, we grouped codes into three categories of factors influencing Homecare’s impact on PLWHIV and created a conceptual framework to display theoretical insights. Table  1 shows the codes we used to build each category. All quantitative analyses were performed using Microsoft Excel. We summarized continuous data by medians and interquartile ranges (IQR). We compared results from the Brief-IPQ before and after Homecare using paired T-tests. All p-values generated were two sided, with significance set at 5%. We integrated qualitative and quantitative data through a Joint Display Analysis showing box plots of the Brief-IPQ with representative quotes of each questionnaire domain shown beneath. We also used the Brief-IPQ to support the development of the conceptual framework of Homecare’s impact on retention in HIV care.

Quantitative results

In its first year, Homecare enrolled 34 PLWHIV previously lost-to-follow-up in clinic-based care. The majority identified as African American ( n  = 31) and male ( n  = 26). After one year of enrollment, 31 were retained in care with at least 2 medical visits. Additionally, 28 Homecare participants achieved viral loads < 200 copies/mL at least once during fifteen months of follow up.

Table  2 displays interview participants’ demographic characteristics and their perceived barriers to care. The participants’ median age was 42 and 80% were male. Participants had been living with HIV for a median of 10 years and had been in Homecare for a median of 631 days.

Results from eight categories of the Brief-IPQ before and after participation in Homecare are shown in Table  3 . Means were significantly decreased (all p  < 0.05) for the domains: emotional response, consequences and identity. PLWHIV reported HIV affecting them less emotionally, having less impact on their daily life, and experiencing fewer health sequelae after being enrolled in Homecare. Means were significantly increased (all p  < 0.005) for the domains: personal control, treatment control, and understanding of HIV. PLWHIV reported more personal control over their HIV treatment, greater appreciation for the benefits of treatment, and better understanding of HIV in general after Homecare.

The domains timeline and concern did not show statistically significant results. Almost every PLWHIV always knew that HIV was a life-long diagnosis that could not be cured. Concern over HIV status was variable – some PLWHIV coped through repression before Homecare and grew more concerned of HIV as they began to acknowledge, accept, and appreciate treatment management; and others became less concerned through Homecare because they gained a better understanding of their health status and concrete steps to becoming virally suppressed.

Qualitative results – interview findings

Interview findings were organized conceptually into three categories: (1) social support and stigma reduction, (2) removing physical and resource barriers, and (3) changing perceptions of illness. Categories one and two are represented below with quotes from PLWHIV and their healthcare workers. The third category is incorporated in the mixed methods results, with interview findings presented in Fig.  1 .

A and B : Joint display. Illness perceptions before and after enrollment in Homecare are presented in box plots, showing median (central line), interquartile range (box), range (whiskers), and outliers (open circles). Quotes displayed below illustrate Homecare participants’ explanation for changing perceptions. 0 indicates lowest perception, 10 indicates highest. See Appendix 1 for B-IPQ

Social support and stigma reduction

Upon receiving diagnosis of HIV, many PLWHIV enrolled in Homecare experienced some form of rejection from family and/or peers.

“There wasn’t anybody I could talk to. I tried to tell my family and they ostracized me.” (PLWHIV, Male, Age 71) .

Some chose not to disclose their status to anyone or to only very select people in their life for fear of causing their loved ones excessive worry or burden.

“I haven’t gotten to the point where I feel comfortable with exposing it because I haven’t fully 100% accepted it.… I don’t want my children to get something to worry about, give my ex-wife something to worry about. That’s just between me and God…. I don’t want that to be something that someone else has to go through…. That’s my actions, that’s my results…. So, nobody else has to share that debt but me.” (PLWHIV, Male, Age 46) .

This limited or nonexistent support network meant that PLWHIV were entirely responsible for enrolling, navigating, and maintaining their healthcare. Additionally, PLWHIV described feelings of shame associated with their status as they faced challenges with perceived stigma which made them want to push others away.

“I still felt like an outcast because…people try to hide it, but I see it…. This is something I have to deal with…. So, I just started using it as a weapon to keep people away…. People when they come up to me, especially guys, the first thing that come out my mouth, ‘I got HIV’…. It’s a defense mechanism for me. It just keeps a lot of people away.” (PLWHIV, Male, Age 46) .

Building relationships with healthcare providers was often complicated by a lack of continuity of care, and the strain of having to repeatedly explain their social and medical histories to new providers.

“When I was in the clinic, I had probably seen about four different doctors. I had to keep – it was like ‘so, what happened, what happened’ – I had to keep talking about a traumatic experience over and over and over!” (PLWHIV, Male, Age 21) .

Interactions with healthcare staff were, for many, one of the very few or only outlets PLWHIV had to discuss their HIV status openly. Some PLWHIV interviewed had good working relationships with their clinic-based providers, but others had difficulty forming a trusting bond.

“I had several doctors that I just did not like. They had poor bedside manner. It was just like a matter a fact thing for them. Like, dude I think I’m dying. You know one of those ‘should’ve-known-better’ attitudes.” (PLWHIV, Male, Age 57) .

In contrast to clinic-based care, relationships with Homecare workers were perceived as consistent and widely praised as uplifting, warm, and non-judgmental.

“I just felt like it was a routine [at the clinic]…. Just another guy with HIV. With them [Homecare workers], they make you feel a little more like they understand. They can talk to you. It’s about having that one on one, not being the next one.” (PLWHIV, Male, Age 46) .

Homecare workers were seen by participants as helping them focus on self-reflection and life goals not only for their health, but for their future as well.

“I took it a lot more serious, ‘cause it was a lot of conversation, not just a whole bunch of medical conversation. It was conversation about my life. Um, what do I want to do in the future. So having these types of conversations of what I want to do in the future, I had to be healthy to do these things.” (PLWHIV, Male, Age 29) .

The quarterly community meetings, where individuals in the program could gather for food, conversation, and a guest speaker, also contributed to this sense of shared experience, community, and support:

“It just keeps you motivated to, like, help other people. Encourage other people. When you sit around these people. You build relationships. I still talk to a few of the guys outside of it. And…you need a support system outside of it.” (PLWHIV, Male, Age 29) .

Homecare workers discussed the role of cultural humility in the program, and the need for empathy, understanding, and shared experience to foster strong relationships with PLWHIV.

“When people are even thinking about starting a program like this you have to consider the population that you’re dealing with. You have to get people who are culturally sensitive to that population, people who may be accepted….” (Homecare worker, Female, Age 43) .

There were negative opinions shared by three PLWHIV interviewed. Two individuals were disappointed there were not more opportunities for peer interaction through the Homecare program. One individual was disappointed with the length of time between appointments after he achieved viral suppression and hoped to see Homecare staff more frequently.

Removing physical and resource barriers

Lack of transportation was a barrier to care for 73% of PLWHIV. Additionally, 40% noted work obligations, and 13% reported childcare responsibilities as other barriers to care. Most PLWHIV interviewed could arrange for medical transportation services to their clinic appointment, but many described these services as unreliable.

“You have the medical transportation, you know. Sometimes they come, sometimes they don’t. Or sometimes they come after your appointment time….” (PLWHIV, Male, Age 41) .

Entering the clinic itself was a barrier, as worries about privacy and confidentiality were troubling for many PLWHIV. This was compounded by the fact that wait times were often long, and PLWHIV were left sitting in open spaces.

“Before I was always paranoid when I would go to the clinic. Who will see me? And if I see someone, I know I’ll try to hide and be isolated or I’d constantly be going to the bathroom….I remember telling the lady at the counter once, when y’all ready for me can you just call me on my cell phone. Don’t announce my name.” (PLWHIV, Male, Age 33) .

In contrast, Homecare workers were described as punctual; they came directly to the PLWHIV’s home – eliminating several barriers described above.

“They [Homecare workers] come to my house. They’re ready to go. If I go to the clinic… when I get there, they’re ready to see me or I might have to wait…. You know, a reasonable amount of time [would be] 5–10 minutes, but an hour? Come on…” (PLWHIV, Male, Age 57) .

Homecare providers commented on the value of additional time for environmental home assessments to connect PLWHIV to community resources. As described by a Homecare worker:

“ For a new patient [an appointment could] be like an hour and a half to two hours because you’re not just in there for the patient assessment. You’re in there doing a whole environmental assessment also because do they have water, do they have heat, do they have a bed, do they have the necessities that they need because that’s where I touch base with the community organizations, request a bed, request a refrigerator…” (Homecare worker, Female, Age 49) .

Mixed methods results

Figure  1  A and Fig.  1 B show a joint display analysis of the Brief-IPQ, with box plots displaying Likert scale results and representative quotations for each domain. We included the exact questions we asked PLWHIV and their responses. Both quantitative and qualitative findings provided evidence that participation in Homecare improved PLWHIV’s ability to reach virologic suppression, remain in care, and positively impact perceptions of illness, including emotional responses to HIV. Additionally, the impact of HIV on daily functioning was significantly reduced, including reductions in negative health consequences. This is shown through direct quotation and Brief-IPQ results. Through Homecare, PLWHIV changed their framework of control, shifting to a person-centered approach to managing their healthcare plans. PLWHIV improved their understanding of the scientific intricacies of HIV care, such as medication resistance and markers of immune function. Moreover, they accepted their treatment’s vital role for their physical health and survivorship. The conceptual framework in Fig.  2 displays obstacles and barriers that contribute to PLWHIV becoming lost-to-follow-up and how the Homecare program mediates their return to care. Quantitative and qualitative data were complimentary, showing significant results in the Brief-IPQ with concordant direct interview quotations, and both were considered to build this framework.

Conceptual framework. Based on qualitative and quantitative data presented in the results section, Homecare mediates return to and maintenance in care for lost-to-follow-up people living with HIV

Retaining PLWHIV in clinical care is a major global priority for reducing morbidity and transmission of HIV [ 2 ]. Our results show that Homecare can help PLWHIV remain in care and achieve viral suppression through (1) improved social support and stigma reduction, (2) removal of physical and resource barriers to care and (3) changed perceptions of illness. Understanding these three pillars may help strengthen health systems seeking to improve treatment adherence and viral suppression strategies.

Both qualitative and quantitative studies have repeatedly shown that stigma and discrimination impact people’s decision to access treatment for HIV [ 20 ]. In a systematic review of these effects across cultural contexts, there were multiple levels of influence, including intrapersonal, interpersonal, and structural stigmas enacted on PLWHIV [ 10 ]. We found participants in the current study faced these same circumstances – shame of their diagnosis, concealment, rejection from family, and, in some cases, judgement from clinic-based providers. This compromised available social support, and by extension, adaptive coping strategies. Enrollment in Homecare, however, grounded PLWHIV in trusting relationships with healthcare providers and provided a space for quarterly community meetings with other PLWHIV. The only negative opinions of the program shared were from individuals who hoped from more time with peers and Homecare staff, which speaks to the importance of these connections. The home-setting allowed people to open up and form closer bonds with their healthcare providers, without a sense of time-pressure, or uneasiness with the setting. Homecare staff were discreet and prioritized confidentiality during home visits. Strong patient-provider relationships and available social support have been documented as facilitators to medication adherence in different settings [ 8 , 21 , 22 ]. Serving a largely African American community, Homecare providers remained vigilant of racial disparities in HIV, and reported an understanding of unique barriers facing their communities. Other studies have shown that cultural competency significantly affects HIV care, especially among minority groups, in Detroit and other American cities [ 23 , 24 , 25 ]. Considering African Americans continue to face the highest burden of new HIV infections compared to other racial/ ethnic groups in the United States, Homecare exemplifies how cultural inclusivity can contribute to viral suppression in this group [ 26 ].

Like other studies, we found that many PLWHIV faced financial hardship, lack of transportation and work obligations that prevented them from attending clinic appointments [ 11 , 27 ]. To mitigate economic barriers to care, some research trials have offered conditional economic incentives to improve HIV treatment adherence. In the United States, this showed only modest effect in short-term adherence rates in a large, multi-city community based trial [ 7 ]. When evaluating long-term adherence, other studies have shown no significant gains in treatment adherence after the active intervention period [ 28 ]. Homecare strategized away from economic incentives, and instead focused on eliminating resource barriers through increased time, punctuality, and convenience with healthcare provider visits. In attempts to alleviate financial burdens, Homecare workers used the current resource network around Metro Detroit to connect PLWHIV with existing social services and benefits programs. Transportation problems were eliminated because Homecare workers came to PLWHIV’s residence and offered flexible scheduling and frequent appointment reminders. Worry about being “found-out” by physically entering a clinic associated with HIV was eliminated through Homecare. Concerns of confidentiality and unintended disclosure have been demonstrated as a barrier to care in other settings as well [ 29 , 30 ].

PLWHIV in our study showed significant changes in six domains of the Brief-IPQ. These illness perceptions were first described by Leventhal et al. as dynamic processes by which people attempt to understand their illness and then adjust their behavior to cope with health threats [ 31 ]. Our study showed that after participation in Homecare, PLWHIV changed the role of HIV in their life. With the help of Homecare workers, they worked towards functional coping strategies, including improvements in emotional regulation, acceptance of their diagnosis, and shifting perspectives of the control and power they possess. In accordance with our work, a multi-site, cross cultural study found that lower perceptions of consequence and greater controllability may improve PLWHIVIV’s ability to cope with illness-related stressors [ 32 ]. Additionally, other studies have shown that the perception of greater consequences on one’s life and greater emotional impact were related to higher viral loads [ 33 , 34 ]. This may highlight the way in which Homecare’s impact on illness perceptions led to viral suppression for many of the participants. Many Homecare participants also reported improvements in their understanding of HIV, associated symptoms, their medications and treatment plans.

There are several limitations to our study. Reporting and recall bias may have occurred because we asked PLWHIV to comment on events that occurred in the past, including ranking the effect HIV had on their life before Homecare. Additionally, selection bias may have occurred because we used convenience sampling to select participants. This is less likely, however, because most people we asked agreed to participate in the study. All study participants were offered a $25 gift card for their time, which may have influenced their interview responses. We tried to eliminate this by emphasizing that participation in the study would not affect clinical care, and all participants had the option of ending the interview at any time. The interview transcripts were fully coded by the first author and partially coded by the fourth author for data reliability purposes. Coders remained aware of their preconceived ideas and biases and used bracketing, described in qualitative methods [ 35 ], to prevent these ideas from influencing data analysis. Data were collected until the point of saturation when no new ideas emerged from interviews.

Future research might investigate the transition from Homecare program back to clinic-based care. Longitudinal analysis of viral load suppression is needed to evaluate the effectiveness of Homecare. Additionally, cost-effectiveness analysis may help show how Homecare compares to other clinical adherence interventions, including those with conditional and unconditional economic incentives. Reproducing and evaluating Homecare in other settings with a greater sample size may help improve reliability of the results.

Overall, this study demonstrated that it is possible to re-engage lost-to-follow-up PLWHIV, retain them in care, and reach viral suppression. Through work grounded in cultural humility, Homecare improved social support, reduced stigma, removed resource barriers, and helped change illness perceptions in PLWHIV in Detroit. These findings have significant implications for HIV control efforts and provide evidence for including lost-to-follow-up PLWHIV in consideration for new long-acting injectable HIV medications. These results could inform the development of future programs for difficult to reach populations.

Data availability

All available data is included within this published article. The S1 Appendix contains interview guides for people living with HIV and healthcare personnel.

Abbreviations

People Living with HIV

United National Programme on HIV/ AIDS

Antiretroviral therapy

Infectious Disease

Community-based organizations

Illness Perception Questionnaire

Interquartile ranges

Joint United Nations Programme on HIV/AIDS (UNAIDS). UNAIDS Data 2019. pp. 1–376. Available: https://www.unaids.org/sites/default/files/media_asset/2019-UNAIDS-data_en.pdf .

Joint United Nations Program on HIV/AIDS. 90-90-90 An ambitious treatment target to help end the AIDS epidemic. United Nations. 2014. Available: https://www.unaids.org/sites/default/files/media_asset/90-90-90_en.pdf .

Giordano TP, Gifford AL, White AC, Almazor MES, Rabeneck L, Hartman C, et al. Retention in Care: a challenge to survival with HIV infection. Clin Infect Dis. 2007;44:1493–9. https://doi.org/10.1086/516778 .

Article   PubMed   Google Scholar  

Centers for Disease Control and Prevention. Selected National HIV Prevention and Care Outcomes in the United States. Available: https://www.cdc.gov/hiv/pdf/library/factsheets/cdc-hiv-national-hiv-care-outcomes.pdf .

Michigan Department of Health & Human Services. HIV, Care Continuum, Report. 2020. 2021. Available: https://www.michigan.gov/documents/mdhhs/HIV_Care_Continuum_Report_2021_729583_7.pdf .

Shah M, Risher K, Berry S, Dowdy D. The epidemiologic and economic impact of improving HIV Testing, linkage, and Retention in Care in the United States. Clin Infect Dis. 2016;62:220–9. https://doi.org/10.1093/cid/civ801 .

El-Sadr WM, Donnell D, Beauchamp G, Irene Hall H, Torian LV, Zingman B, et al. Financial incentives for linkage to care and viral suppression among HIV-positive patients a randomized clinical trial (HPTN 065). JAMA Intern Med. 2017;177:1083–92. https://doi.org/10.1001/jamainternmed.2017.2158 .

Article   PubMed   PubMed Central   Google Scholar  

Beach MC, Keruly J, Moore RD. Is the quality of the patient-provider relationship associated with better adherence and health outcomes for patients with HIV? J Gen Intern Med. 2006;21:661–5. https://doi.org/10.1111/j.1525-1497.2006.00399.x .

Enriquez M, Mercier DA, Cheng AL, Banderas JW. Perceived social support among adults struggling with adherence to HIV Care and Treatment. J Assoc Nurses AIDS Care. 2019;30:362–71. https://doi.org/10.1097/JNC.0000000000000059 .

Katz IT, Ryu AE, Onuegbu AG, Psaros C, Weiser SD, Bangsberg DR et al. Impact of HIV-related stigma on treatment adherence: systematic review and meta-synthesis. J Int AIDS Soc. 2013;16.

Burch LS, Smith CJ, Phillips AN, Johnson MA, Lampe FC. Socioeconomic status and response to antiretroviral therapy in high-income countries: a literature review. Aids. 2016;30:1147–61. https://doi.org/10.1097/QAD.0000000000001068 .

Article   CAS   PubMed   Google Scholar  

De La Mata NL, Ly PS, Nguyen KV, Merati TP, Pham TT, Lee MP, et al. Loss to follow-up trends in HIV-positive patients receiving antiretroviral treatment in Asia from 2003 to 2013. J Acquir Immune Defic Syndr. 2017;74:555–62. https://doi.org/10.1097/QAI.0000000000001293 .

Monroe AK, Rowe TL, Moore RD, Chander G. Medication adherence in HIV-positive patients with diabetes or hypertension: a focus group study. BMC Health Serv Res. 2013;13. https://doi.org/10.1186/1472-6963-13-488 .

Gaston GB, Alleyne-Green B. The impact of African americans’ beliefs about HIV medical care on treatment adherence: a systematic review and recommendations for interventions. AIDS Behav. 2013;17:31–40. https://doi.org/10.1007/s10461-012-0323-x .

McLendon T, Union S, Cohn J, Carrington W, Veltman J. Home Care Reaching Our Lost to Follow Up Patients Overcoming Barriers to Care Utilizing Nontraditional Care Models. Poster abstract presentation at the 2018 Ryan White Conference on HIV Care and Treatment. Oxon Hill, MD; 2018.

Creswell J. Chapter 3: Core Mixed Methods Design. Designing and Conducting Mixed Methods Research, Third Edition. Sage Publications, Inc; 2018.

Glaser BG, Strauss AL. The Discovery of grounded theory: strategies for qualitative research. Chicago: Aldine Pub. Co; 1967.

Google Scholar  

Broadbent E, Petrie K, Main Jodie WJ. Brief illness perception questionnaire (brief IPQ). J Physiother. 2012;58:202. https://doi.org/10.1016/S1836-9553(12)70116-9 .

Article   Google Scholar  

Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3: 77–101. Available: http://www.ncbi.nlm.nih.gov/pubmed/11752478 .

Misir P. Structuration Theory: a conceptual Framework for HIV/AIDS Stigma. J Int Assoc Provid AIDS Care. 2015;14:328–34. https://doi.org/10.1177/2325957412463072 .

O’Laughlin KN, Wyatt MA, Kaaya S, Bangsberg DR, Ware NC. How treatment partners help: Social analysis of an African adherence support intervention. AIDS Behav. 2012;16:1308–15. https://doi.org/10.1007/s10461-011-0038-4 .

Bontempi JMB, Burleson L, Lopez MH. HIV medication adherence programs: the importance of social support. J Community Health Nurs. 2004;21:111–22. https://doi.org/10.1207/s15327655jchn2102_05 .

Saha S, Korthuis PT, Cohn JA, Sharp VL, Moore RD, Beach MC. Primary care provider cultural competence and racial disparities in HIV care and outcomes. J Gen Intern Med. 2013;28:622–9. https://doi.org/10.1007/s11606-012-2298-8 .

Saha S, Sanders DS, Korthuis PT, Cohn JA, Sharp VL, Haidet P, et al. The role of cultural distance between patient and provider in explaining racial/ethnic disparities in HIV care. Patient Educ Couns. 2011;85:e278–84. https://doi.org/10.1016/j.pec.2011.01.012 .

Beach MC, Saha S, Korthuis PT, Sharp V, Cohn J, Wilson IB, et al. Patient-provider communication differs for black compared to white HIV-infected patients. AIDS Behav. 2011;15:805–11. https://doi.org/10.1007/s10461-009-9664-5 .

Centers for Disease Control and Prevention (CDC). HIV Among African Americans, Atlanta GA. 2019. Available: https://www.cdc.gov/nchhstp/newsroom/docs/factsheets/cdc-hiv-aa-508.pdf .

Sagrestano LM, Clay J, Finerman R, Gooch J, Rapino M. Transportation vulnerability as a barrier to service utilization for HIV-positive individuals. AIDS Care - Psychol Socio-Medical Asp AIDS/HIV. 2014;26:314–9. https://doi.org/10.1080/09540121.2013.819403 .

Galarraga O, Genberg B, Martin R, Laws MB, Wilson I. Conditional economic incentive to improve HIV treatment adherence: literature review and theoretical considerations. AIDS Behav. 2014;17:2283–92. https://doi.org/10.1007/s10461-013-0415-2.Conditional .

Clair-Sullivan NS, Mwamba C, Whetham J, Moore CB, Darking M, Vera J. Barriers to HIV care and adherence for young people living with HIV in Zambia and mHealth. mHealth. 2019;5:45–45. https://doi.org/10.21037/mhealth.2019.09.02 .

Madiba S, Josiah U. Perceived Stigma and Fear of Unintended Disclosure are barriers in Medication Adherence in adolescents with Perinatal HIV in Botswana: a qualitative study. Biomed Res Int. 2019;2019. https://doi.org/10.1155/2019/9623159 .

Leventhal H, Meyer D, Nerenz D. The common sense representation of illness danger. Med Psychol. 1980;II:7–30.

Reynolds N, Eller L, Nicholas P, Corless I, Kirksey K, Hamilton M, et al. HIV illness representation as a predictor of Self-Care Management and Health outcomes: a Multi-site, cross-cultural study. AIDS Behav. 2008;23:1–7. https://doi.org/10.1038/jid.2014.371 .

Article   CAS   Google Scholar  

Norcini Pala A, Steca P. Illness perceptions and coping strategies among individuals diagnosed with HIV. J Behav Med. 2015;38:620–31. https://doi.org/10.1007/s10865-015-9639-0 .

Leone D, Borghi L, Lamiani G, Barlascini L, Bini T, d’Arminio Monforte A, et al. Illness representations of HIV positive patients are associated with virologic success. Front Psychol. 2016;7:1–10. https://doi.org/10.3389/fpsyg.2016.01991 .

Gibbs G. Chapter 4: thematic coding and categorizing. Analyzing qualitative data. Sage Publications, Inc; 2012. pp. 38–55. https://doi.org/10.4135/9781849208574.n4 .

Download references

Acknowledgements

Advice and support from the funders and advisers are gratefully acknowledged: Wayne State University School of Medicine Student Research Fellowship and the Infectious Disease Society of America Grant for Emerging Researchers/ Clinician Mentorship. The authors thank all the research participants and the organizations supporting people living with HIV in metro-Detroit.

This project received no funding.

Author information

Authors and affiliations.

Emory University School of Medicine, Atlanta, GA, USA

L. V. Bonadonna

Wayne State University, Detroit, MI, USA

E. Guerrero

Wayne State University School of Medicine, Detroit, MI, USA

L. V. Bonadonna, D. Kabbani & J. Cohn

Wayne Health, Detroit, MI, USA

T. McClendon, S. Union & D. Wittmann

Loma Linda University School of Medicine, Loma Linda, CA, USA

You can also search for this author in PubMed   Google Scholar

Contributions

LB developed the design of this research, and conducted all interviews in collaboration with EG, and DK. LB, JV, DK, DW interpreted results. All authors contributed to the writing of the manuscript and approved the final copy.

Corresponding author

Correspondence to L. V. Bonadonna .

Ethics declarations

Ethical approval.

This study received ethical approval from the Internal Review Board (IRB) of Wayne State University in Detroit, Michigan, USA. All participants signed written, informed consent to participate in this research.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Bonadonna, L., Guerrero, E., McClendon, T. et al. Evaluation of an HIV homecare program for lost-to-follow-up populations: a mixed methods study in Detroit, Michigan. AIDS Res Ther 21 , 21 (2024). https://doi.org/10.1186/s12981-024-00608-5

Download citation

Received : 23 February 2023

Accepted : 25 March 2024

Published : 12 April 2024

DOI : https://doi.org/10.1186/s12981-024-00608-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Lost-to-follow-up
• Stigma reduction
• Social support

AIDS Research and Therapy

ISSN: 1742-6405

• Submission enquiries: [email protected]

• Open access
• Published: 11 April 2024

The role of champions in the implementation of technology in healthcare services: a systematic mixed studies review

• Sissel Pettersen 1 ,
• Hilde Eide 2 &
• Anita Berg 1  

BMC Health Services Research volume  24 , Article number:  456 ( 2024 ) Cite this article

Metrics details

Champions play a critical role in implementing technology within healthcare services. While prior studies have explored the presence and characteristics of champions, this review delves into the experiences of healthcare personnel holding champion roles, as well as the experiences of healthcare personnel interacting with them. By synthesizing existing knowledge, this review aims to inform decisions regarding the inclusion of champions as a strategy in technology implementation and guide healthcare personnel in these roles.

A systematic mixed studies review, covering qualitative, quantitative, or mixed designs, was conducted from September 2022 to March 2023. The search spanned Medline, Embase, CINAHL, and Scopus, focusing on studies published from 2012 onwards. The review centered on health personnel serving as champions in technology implementation within healthcare services. Quality assessments utilized the Mixed Methods Appraisal Tool (MMAT).

From 1629 screened studies, 23 were included. The champion role was often examined within the broader context of technology implementation. Limited studies explicitly explored experiences related to the champion role from both champions’ and health personnel’s perspectives. Champions emerged as promoters of technology, supporting its adoption. Success factors included anchoring and selection processes, champions’ expertise, and effective role performance.

The specific tasks and responsibilities assigned to champions differed across reviewed studies, highlighting that the role of champion is a broad one, dependent on the technology being implemented and the site implementing it. Findings indicated a correlation between champion experiences and organizational characteristics. The role’s firm anchoring within the organization is crucial. Limited evidence suggests that volunteering, hiring newly graduated health personnel, and having multiple champions can facilitate technology implementation. Existing studies predominantly focused on client health records and hospitals, emphasizing the need for broader research across healthcare services.

Conclusions

With a clear mandate, dedicated time, and proper training, health personnel in champion roles can significantly contribute professional, technological, and personal competencies to facilitate technology adoption within healthcare services. The review finds that the concept of champions is a broad one and finds varied definitions of the champion role concept. This underscores the importance of describing organizational characteristics, and highlights areas for future research to enhance technology implementation strategies in different healthcare settings with support of a champion.

Peer Review reports

Digital health technologies play a transformative role in healthcare service systems [ 1 , 2 ]. The utilization of technology and digitalization is essential for ensuring patient safety, delivering high quality, cost-effective, and sustainable healthcare services [ 3 , 4 ]. The implementation of technology in healthcare services is a complex process that demands systematic changes in roles, workflows, and service provision [ 5 , 6 ].

The successful implementation of new technologies in healthcare services relies on the adaptability of health professionals [ 7 , 8 , 9 ]. Champions have been identified as a key factor in the successful implementation of technology among health personnel [ 10 , 11 , 12 ]. However, they have rarely been studied as an independent strategy; instead, they are often part of a broader array of strategies in implementation studies (e.g., Hudson [ 13 ], Gullslett and Bergmo [ 14 ]). Prior research has frequently focused on determining the presence or absence of champions [ 10 , 12 , 15 ], as well as investigating the characteristics of individuals assuming the champion role (e.g., George et al. [ 16 ], Shea and Belden [ 17 ]).

Recent reviews on champions [ 18 , 19 , 20 ] have studied their effects on adherence to guidelines, implementation of innovations and facilitation of evidence-based practice. While these reviews suggest that having champions yields positive effects, they underscore the importance for studies that offer detailed insights into the champion’s role concerning specific types of interventions.

There is limited understanding of the practical role requirements and the actual experiences of health personnel in performing the champion role in the context of technology implementation within healthcare services. Further, this knowledge is needed to guide future research on the practical, professional, and relational prerequisites for health personnel in this role and for organizations to successfully employ champions as a strategy in technology implementation processes.

This review seeks to synthesize the existing empirical knowledge concerning the experiences of those in the champion role and the perspectives of health personnel involved in technology implementation processes. The aim is to contribute valuable insights that enhance our understanding of practical role requirements, the execution of the champion role, and best practices in this domain.

The term of champions varies [ 10 , 19 ] and there is a lack of explicit conceptualization of the term ‘champion’ in the implementation literature [ 12 , 18 ]. Various terms for individuals with similar roles also exist in the literature, such as implementation leader, opinion leader, facilitator, change agent, superuser and facilitator. For the purpose of this study, we have adopted the terminology utilized in the recent review by Rigby, Redley and Hutchinson [ 21 ] collectively referring to these roles as ‘champions’. This review aims to explore the experiences of health personnel in their role as champions and the experiences of health personnel interacting with them in the implementation of technology in the healthcare services.

Prior review studies on champions in healthcare services have employed various designs [ 10 , 18 , 19 , 20 ]. In this review, we utilized a comprehensive mixed studies search to identify relevant empirical studies [ 22 ]. The search was conducted utilizing the Preferred Reporting Items for Systematic and Meta-Analysis (PRISMA) guidelines, ensuring a transparent and comprehensive overview that can be replicated or updated by others [ 23 ]. The study protocol is registered in PROSPERO (ID CRD42022335750), providing a more comprehensive description of the methods [ 24 ]. A systematic mixed studies review, examining research using diverse study designs, is well-suited for synthesizing existing knowledge and identifying gaps by harnessing the strengths of both qualitative and quantitative methods [ 22 ]. Our search encompassed qualitative, quantitative, and mixed methods design to capture experiences with the role of champions in technology implementation.

Search strategy and study selection

Search strategy.

The first author, in collaboration with a librarian, developed the search strategy based on initial searches to identify appropriate terms and truncations that align with the eligibility criteria. The search was constructed utilizing a combination of MeSH terms and keywords related to technology, implementation, champion, and attitudes/experiences. Conducted in August/September 2022, the search encompassed four databases: Medline, Embase, CINAHL, and Scopus, with an updated search conducted in March 2023. The full search strategy for Medline is provided in Appendix  1 . The searches in Embase, CINAHL and Scopus employed the same strategy, with adopted terms and phrases to meet the requirements of each respective database.

Eligibility criteria

We included all empirical studies employing qualitative, quantitative, and mixed methods designs that detailed the experiences and/or attitudes of health personnel regarding the champions role in the implementation of technology in healthcare services. Articles in the English language published between 2012 and 2023 were considered. The selected studies involved technology implemented or adapted within healthcare services.

Conference abstract and review articles were excluded from consideration. Articles published prior 2012 were excluded as a result of the rapid development of technology, which could impact the experiences reported. Furthermore, articles involving surgical technology and pre-implementation studies were also excluded, as the focus was on capturing experiences and attitudes from the adoption and daily use of technology. The study also excluded articles that involved champions without clinical health care positions.

Study selection

A total of 1629 studies were identified and downloaded from the selected databases, with Covidence [ 25 ] utilized as a software platform for screening. After removing 624 duplicate records, all team members collaborated to calibrate the screening process utilizing the eligibility criteria on the initial 50 studies. Subsequently, the remaining abstracts were independently screened by two researchers, blinded to each other, to ensure adherence to the eligibility criteria. Studies were included if the title and abstract included the term champion or its synonyms, along with technology in healthcare services, implementation, and health personnel’s experiences or attitudes. Any discrepancies were resolved through consensus among all team members. A total of 949 abstracts were excluded for not meeting this inclusion condition. During the initial search, 56 remaining studies underwent full-text screening, resulting in identification of 22 studies qualified for review.

In the updated search covering the period September 2022 to March 2023, 64 new studies were identified. Of these, 18 studies underwent full-text screening, and one study was included in our review. The total number of included studies is 23. The PRISMA flowchart (Fig.  1 ) illustrates the process.

Flow Chart illustrating the study selection and screening process

Data extraction

The research team developed an extraction form for the included studies utilizing an Excel spreadsheet. Following data extraction, the information included the Name of Author(s) Year of publication, Country/countries, Title of the article, Setting, Aim, Design, Participants, and Sample size of the studies, Technology utilized in healthcare services, name/title utilized to describe the Champion Role, how the studies were analyzed and details of Attitude/Experience with the role of champion. Data extraction was conducted by SP, and the results were deliberated in a workshop with the other researchers AB, and HE until a consensus was reached. Any discrepancies were resolved through discussions. The data extraction was categorized into three categories: qualitative, quantitative, and mixed methods, in preparation for quality appraisal.

Quality appraisal

The MMAT [ 26 ] was employed to assess the quality of the 23 included studies. Specifically designed for mixed studies reviews, the MMAT allows for the appraisal of the methodological quality of studies falling into five categories. The studies in our review encompassed qualitative, quantitative descriptive, and mixed methods studies. The MMAT begins with two screening questions to confirm the empirical nature of this study. Subsequently, all studies were categorized by type and evaluated utilizing specific criteria based on their research methods, with ratings of ‘Yes,’ ‘No’ or ‘Can’t tell.’ The MMAT discourages overall scores in favor of providing a detailed explanation for each criterion. Consequently, we did not rely on the MMAT’s overall methodical quality scores and continued to include all 23 studies for our review. Two researchers independently scored the studies, and any discrepancies were discussed among all team members until a consensus was reached. The results of the MMAT assessments are provided in Appendix  2 .

Data synthesis

Based on discussions of this material, additional tables were formulated to present a comprehensive overview of the study characteristics categorized by study design, study settings, technology included, and descriptions/characteristics of the champion role. To capture attitudes and experiences associated with the champion role, the findings from the included studies were translated into narrative texts [ 22 ]. Subsequently, the reviewers worked collaboratively to conduct a thematic analysis, drawing inspiration from Braun and Clarke [ 27 ]. Throughout the synthesis process, multiple meetings were conducted to discern and define the emerging themes and subthemes.

The adopting of new technology in healthcare services can be perceived as both an event and a process. According to Iqbal [ 28 ], experience is defined as the knowledge and understanding gained after an event or the process of living through or undergoing an event. This review synthesizes existing empirical knowledge regarding the experiences of occupying the champion role, and the perspectives of health personnel interacting with champions in technology implementation processes.

Study characteristics

The review encompassed a total of 23 studies, and an overview of these studies is presented in Table  1 . Of these, fourteen studies employed a qualitative design, four had quantitative design, and five utilized a mixed method design. The geographical distribution revealed that the majority of studies were conducted in the USA (8), followed by Australia (5), England (4), Canada (2), Norway (2), Ireland (1), and Malaysia (1). In terms of settings, 11 studies were conducted in hospitals, five in primary health care, three in home-based care settings, and four in a mixed settings where two or more settings collaborated. Various technologies were employed across these studies, with client health records (7) and telemedicine (5) being the most frequently utilized. All studies included experiences from champions or health personnel collaborating with champions in their respective healthcare services. Only three studies had the champion role as a main objective [ 29 , 30 , 31 ]. The remaining studies described champions as one of the strategies in technology implementation processes, including 10 evaluation studies (including feasibility studies [ 32 , 33 , 34 ] and one cost-benefit study [ 30 ]).

Several studies underscored the importance of champions for successful implementation [ 29 , 30 , 31 , 34 , 35 , 36 , 37 , 38 , 40 , 41 , 42 , 43 , 49 ]. Four studies specifically highlighted champions as a key factor for success [ 34 , 36 , 37 , 43 ], and one study went further to describe champions as the most important factor for successful implementation [ 39 ]. Additionally, one study associated champions with reduced labor cost [ 30 ].

Thin descriptions, yet clear expectations for technology champions’ role and -attributes

The analyses revealed that the concept of champions in studies pertaining to technology implementation in healthcare services varies, primarily as a result of the diversity of terms utilized to describe the role combined with short role descriptions. Nevertheless, the studies indicated clear expectations for the champion’s role and associated attributes.

The term champion

The term champion was expressed in 20 different forms across the 23 studies included in our review. Three studies utilized multiple terms within the same study [ 32 , 47 , 48 ] and 15 different authors [ 29 , 32 , 33 , 35 , 36 , 37 , 39 , 40 , 41 , 42 , 43 , 44 , 46 , 47 , 50 ] employed the term with different compositions (Table  1 ). Furthermore, four authors utilized the term Super user [ 30 , 31 , 49 , 51 ], while four authors employed the terms Facilitator [ 38 ], IT clinician [ 48 ], Leader [ 45 ], and Manager [ 34 ], each in combination with more specific terms (such as local opinion leaders, IT nurse, or practice manager).

Most studies associated champion roles with specific professions. In seven studies, the professional title was explicitly linked to the concept of champions, such as physician champions or clinical nurse champions, or through the strategic selection of specific professions [ 29 , 33 , 36 , 40 , 43 , 47 , 50 ]. Additionally, some studies did not specify professions, but utilized terms like clinicians [ 45 ] or health professionals [ 41 ].

All included articles portray the champion’s role as facilitating implementation and daily use of technology among staff. In four studies, the champion’s role was not elaborated beyond indicating that the individual holding the role is confident with an interest in technology [ 35 , 41 , 42 , 44 ]. The champion’s role was explicitly examined in six studies [ 29 , 30 , 31 , 33 , 46 , 50 ]. Furthermore, seven studies described the champion in both the methods and results [ 32 , 36 , 38 , 47 , 48 , 49 , 51 ]. In ten of the studies, champions were solely mentioned in the results [ 34 , 35 , 37 , 39 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 ].

Eight studies provided a specific description or definition of the champion [ 29 , 30 , 31 , 32 , 38 , 48 , 49 , 50 ]. The champion’s role was described as involving training in the specific technology, being an expert on the technology, providing support and assisting peers when needed. In some instance, the champion had a role in leading the implementation [ 50 ], while in other situations, the champion operated as a mediator [ 48 ].

The champions tasks

In the included studies, the champion role encompassed two interrelated facilitators tasks: promoting the technology and supporting others in adopting the technology in their daily practice. Promoting the technology involved encouraging staff adaptation [ 32 , 34 , 35 , 37 , 40 , 41 , 49 ], generally described as being enthusiastic about the technology [ 32 , 35 , 37 , 41 , 48 ], influencing the attitudes and beliefs of colleagues [ 42 , 45 ] and legitimizing the introduction of the technology [ 42 , 46 , 48 ]. Supporting others in technology adaption involved training and teaching [ 31 , 35 , 38 , 40 , 51 ], as well as providing technical support [ 30 , 31 , 39 , 43 , 49 ] and social support [ 49 ]. Only four studies reported that the champions received their own training to enable them able to support their colleagues [ 30 , 31 , 39 , 48 ]. Furthermore, eight studies [ 32 , 34 , 38 , 40 , 48 , 49 , 50 , 51 ], specified that the champion role included leadership and management responsibilities, mentioning tasks such as planning, organizing, coordinating, and mediating technology adaption without providing further details.

Desirable champion attributes

To effectively fulfill their role, champions should ideally possess clinical expertise and experience [ 29 , 35 , 38 , 40 , 48 ], stay professionally updated [ 37 , 48 ], and possess knowledge of the organization and workflows [ 29 , 34 , 46 ]. They should have the ability to understand and communicate effectively with healthcare personnel [ 31 , 32 , 46 , 49 ] and be proficient in IT language [ 51 ]. Moreover, champions should demonstrate a general technological interest and competence, and competence, along with specific knowledge of the technology to be implemented [ 32 , 37 , 49 ]. It is also emphasized that they should command formal and/or informal respect and authority in the organization [ 36 , 45 ], be accessible to others [ 39 , 43 ], possess leadership qualities [ 34 , 37 , 38 , 46 ], and understand and balance the needs of stakeholders [ 43 ]. Lastly, the champions should be enthusiastic promoters of the technology, engaging and supporting others [ 31 , 32 , 33 , 34 , 37 , 39 , 40 , 41 , 43 , 49 ], while also effectively coping with cultural resistance to change [ 31 , 46 ].

Anchoring and recruiting for the champion role

The champions were organized differently within services, holding various positions in the organizations, and being recruited for the role in different ways.

Anchoring the champion role

The champion’s role is primarily anchored at two levels: the management level and/or the clinical level, with two studies having champions at both levels [ 34 , 49 ]. Those working with the management actively participated in the planning of the technology implementation [ 29 , 36 , 40 , 41 , 45 ]. Serving as advisors to management, they leveraged their clinical knowledge to guide the implementation in alignment with the necessities and possibilities of daily work routines in the clinics. Champions in this capacity experienced having a clear formal position that enabled them to fulfil their role effectively [ 29 , 40 ]. Moreover, these champions served as bridge builders between the management and department levels [ 36 , 45 ], ensuring the necessary flow of information in both directions.

Champions anchored at the clinic level played a pivotal role in the practical implementation and facilitation of the daily use of technology [ 31 , 33 , 35 , 37 , 38 , 43 , 48 , 51 ]. Additionally, these champions actively participated in meetings with senior management to discuss the technology and its implementation in the clinic. This position conferred potential influence over health personnel [ 33 , 35 ]. Champions at the clinic level facilitated collaboration between employees, management, and suppliers [ 48 ]. Fontaine et al. [ 36 ] identified respected champions at the clinical level, possessing authority and formal support from all leadership levels, as the most important factor for success.

Only one study reported that the champions received additional compensation for their role [ 36 ], while another study mentioned champions having dedicated time to fulfil their role [ 46 ]. The remaining studies did not provide this information.

Recruiting for the role as champion

Several studies have reported different experiences regarding the management’s selection of champions. A study highlighted the distinctions between a volunteered role and an appointed champion’s role [ 31 ]. Some studies underscored that appointed champions were chosen based on technological expertise and skills [ 41 , 48 , 51 ]. Moreover, the selection criteria included champions’ interest in the specific technology [ 42 ] or experiential skills [ 40 ]. The remaining studies did not provide this information.

While the champion role was most frequently held by health personnel with clinical experience, one study deviated by hiring 150 newly qualified nurses as champions [ 30 ] for a large-scale implementation of an Electronic Health Record (EHR). Opting for clinical novices assisted in reducing implementation costs, as it avoided disrupting daily tasks and interfering with daily operations. According to Bullard [ 30 ], these super-user nurses became highly sought after post-implementation as a result of their technological confidence and competence.

Reported experiences of champions and health personnel

Drawing from the experiences of both champions and health personnel, it is essential for a champion to possess a combination of general knowledge and specific champion characteristics. Furthermore, champions are required to collaborate with individuals both within and outside the organization. The subsequent paragraphs delineate these experiences, categorizing them into four subsets: champions’ contextual knowledge and expertise, preferred performance of the champion role, recognizing that a champion alone is insufficient, and distinguishing between reactive and proactive champions.

Champions’ contextual knowledge and know-how

Health personnel with experience interacting with champions emphasized that a champion must be familiar with the department and its daily work routines [ 35 , 40 ]. Knowledge of the department’s daily routines made it easier for champions to facilitate the adaptation of technology. However, there was a divergence of opinions on whether champions were required to possess extensive clinical experience to fulfil their role. In most studies, having an experienced and competent clinician as a champion instilled a sense of confidence among health personnel. Conversely, Bullard’s study [ 30 ] exhibited that health personnel were satisfied with newly qualified nurses in the role of champion, despite their initial skepticism.

It is a generally expected that champions should possess technological knowledge beyond that of other health professionals [ 37 , 41 ]. Some health personnel perceived the champions as uncritical promoters of technology, with the impression that health personnel were being compelled to utilize technology [ 46 ]. Champions could also overestimate the readiness of health personnel to implement a technology, especially during the early phases of the implementation process [ 32 ]. Regardless of whether the champion is at the management level or the clinic level, champions themselves have acknowledged the importance of providing time and space for innovation. Moreover, the recruitment of champions should span all levels of the organization [ 34 , 46 ]. Furthermore, champions must be familiar with daily work routines, work tools, and work surfaces [ 38 , 40 , 43 ].

Preferable performance of the champion role

The studies identified several preferable characteristics of successful champions. Health personnel favored champions utilizing positive words when discussing technology and exhibiting positive attitudes while facilitating and adapting it [ 33 , 34 , 37 , 38 , 41 , 46 ]. Additionally, champions who were enthusiastic and engaging were considered good role models for the adoption of technology. Successful champions were perceived as knowledgeable and adept problem solvers who motivated and supported health personnel [ 41 , 43 , 44 , 48 ]. They were also valued for being available and responding promptly when contacted [ 42 ]. Health professionals noted that champions perceived as competent garnered respect in the organization [ 40 ]. Moreover, some health personnel felt that some certain champions wielded a greater influence based on how they encouraged the use of the system [ 48 ]. It was also emphasized that health personnel needed to feel it was safe to provide feedback to champions, especially when encountering difficulties or uncertainties [ 49 ].

A champion is not enough

The role of champions proved to be more demanding than expected [ 29 , 31 , 38 ], involving tasks such as handling an overwhelming number of questions or actively participating in the installation process to ensure the technology functions effectively in the department [ 29 ]. Regardless of the organizational characteristics or the champion’s profile, appointing the champion as a “solo implementation agent” is deemed unsuitable. If the organization begins with one champion, it is recommended that this individual promptly recruits others into the role [ 42 ].

Health personnel, reliant on champions’ expertise, found it beneficial to have champions in all departments, and these champions had to be actively engaged in day-to-day operations [ 31 , 33 , 34 , 37 ]. Champions themselves also noted that health personnel increased their technological expertise through their role as champions in the department [ 39 ].

Furthermore, the successful implementation of technology requires the collaboration of various professions and support functions, a task that cannot be solely addressed by a champion [ 29 , 43 , 48 ]. In Orchard et. al.‘s study [ 34 ], champions explicitly emphasized the necessity of support from other personnel in the organization, such as those responsible for the technical aspects and archiving routines, to provide essential assistance.

According to health personnel, the role of champions is vulnerable in case they become sick or leave their position [ 42 , 51 ]. In some of the included studies, only one or a few hold the position of champion [ 37 , 38 , 42 , 48 ]. Two studies observed that their implementations were not completed because champions left or reassigned for various reasons [ 32 , 51 ]. The health professionals in the study by Owens and Charles [ 32 ] expressed that champions must be replaced in such cases. Further, the study of Olsen et al., 2021 [ 42 ] highlights the need for quicky building a champion network within the organization.

Reactive and proactive champions

Health personnel and champions alike noted that champions played both a reactive and proactive role. The proactive role entailed facilitating measures such as training and coordination [ 31 , 32 , 33 , 34 , 37 , 39 , 40 , 41 , 43 , 48 , 49 ] as initiatives to generate enthusiasm for the technology [ 31 , 32 , 33 , 34 , 35 , 37 , 39 , 40 , 41 , 43 , 49 ]. On the other hand, the reactive role entailed hands-on support and troubleshooting [ 30 , 31 , 39 , 43 , 49 ].

In a study presenting experiences from both health personnel and champions, Yuan et al. [ 31 ] found that personnel observed differences in the assistance provided by appointed and self-chosen champions. Appointed champions demonstrated the technology, answered questions from health personnel, but quickly lost patience and track of employees who had received training [ 31 ]. Health personnel perceived that self-chosen champions were proactive and well-prepared to facilitate the utilization of technology, communicating with the staff as a group and being more competent in utilizing the technology in daily practice [ 31 ]. Health personnel also noted that volunteer champions were supportive, positive, and proactive in promoting the technology, whereas appointed champions acted on request and had a more reactive approach [ 31 ].

This review underscores the breadth of the concept of champion and the significant variation in the champion’s role in implementation of technology in healthcare services. This finding supports the results from previous reviews [ 10 , 18 , 19 , 20 ]. The majority of studies meeting our inclusion criteria did not specifically focus on the experiences of champions and health personnel regarding the champion role, with the exception of studies by Bullard [ 30 ], Gui et al. [ 29 ], Helmer-Smith et al. [ 33 ], Hogan-Murphy et al. [ 46 ], Rea et al. [ 50 ], and Yuan et al. [ 31 ].

The 23 studies encompassed in this review utilized 20 different terms for the champion role. In most studies, the champion’s role was briefly described in terms of the duties it entailed or should entail. This may be linked to the fact that the role of champions was not the primary focus of the study, but rather one of the strategies in the implementation process being investigated. This result reinforces the conclusions drawn by Miech et al. [ 10 ] and Shea et al. [ 12 ] regarding the lack of united understandings of the concept. Furthermore, in Santos et al.‘s [ 19 ] review, champions were only operationalized through presence or absence in 71.4% of the included studies. However, our review finds that there is a consistent and shared understanding that champions should promote and support technology implementation.

Several studies advocate for champions as an effective and recommended strategy for implementing technology [ 30 , 31 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 42 , 43 , 45 , 46 ]. However, we identified that few studies exclusively explore health personnel`s experiences within the champion role when implementing technology in healthcare services.

This suggests a general lack of information essential for understanding the pros, cons, and prerequisites for champions as a strategy within this field of knowledge. However, this review identifies, on a general basis, the types of support and structures required for champions to perform their role successfully from the perspectives of health personnel, contributing to Shea’s conceptual model [ 12 ].

Regarding the organization of the role, this review identified champions holding both formal appointed and informal roles, working in management or clinical settings, being recruited for their clinical and/or technological expertise, and either volunteering or being hired with specific benefits for the role. Regardless of these variations, anchoring the role is crucial for both the individuals holding the champion role and the health personnel interacting with them. Anchoring, in this context, is associated with the clarity of the role’s content and a match between role expectations and opportunities for fulfilment. Furthermore, the role should be valued by the management, preferably through dedicated time and/or salary support [ 34 , 36 , 46 ]. Additionally, our findings indicate that relying on a “solo champion” is vulnerable to issues such as illness, turnover, excessive workload, and individual champion performance [ 32 , 37 ]. Based on these insights, it appears preferable to appoint multiple champions, with roles at both management and clinical levels [ 33 ].

Some studies have explored the selection of champions and its impact on role performance, revealing diverse experiences [ 30 , 31 ]. Notably, Bullard [ 30 ], stands out for emphasizing long clinical experience, and hiring newly trained nurses as superusers to facilitate the use of electronic health records. Despite facing initial reluctance, these newly trained nurses gradually succeeded in their roles. This underscores the importance of considering contextual factors in the champion selection [ 30 , 52 ]. In Bullard’s study [ 30 ], the collaboration between newly trained nurses as digital natives and clinical experienced health personnel proved beneficial, highlighting the need to align champion selection with the organization’s needs based on personal characteristics. This finding aligns with Melkas et al.‘s [ 9 ] argument that implementing technology requires a deeper understanding of users, access to contextual know-how, and health personnel’s tacit knowledge.

To meet role expectations and effectively leverage their professional and technological expertise, champions should embody personal qualities such as the ability to engage others, take a leadership role, be accessible, supportive, and communicate clearly. These qualities align with the key attributes for change in healthcare champions described by Bonawitz et al. [ 15 ]. These attributes include influence, ownership, physical presence, persuasiveness, grit, and a participative leadership style (p.5). These findings suggest that the active performance of the role, beyond mere presence, is crucial for champions to be a successful strategy in technology implementation. Moreover, the recruitment process is not inconsequential. Identifying the right person for the role and providing them with adequate training, organizational support, and dedicated time to fulfill their responsibilities emerge as an important factor based on the insights from champions and health personnel.

Strengths and limitations

While this study benefits from identifying various terms associated with the role of champions, it acknowledges the possibility of missing some studies as a result of diverse descriptions of the role. Nonetheless, a notable strength of the study lies in its specific focus on the health personnel’s experiences in holding the champion role and the broader experiences of health personnel concerning champions in technology implementation within healthcare services. This approach contributes valuable insights into the characteristics of experiences and attitudes toward the role of champions in implementing technology. Lastly, the study emphasizes the relationship between the experiences with the champion role and the organizational setting’s characteristics.

The champion role was frequently inadequately defined [ 30 , 33 , 34 , 35 , 36 , 37 , 39 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 51 ], aligning with previous reviews [ 17 , 19 , 21 ]. As indicated by van Laere and Aggestam [ 52 ], this lack of clarity complicates the identification and comparison of champions across studies. Studies that lacking a distinct definition of the champion’s role were consequently excluded. Only studies written in English were included, introducing the possibility of overlooking relevant studies based on our chosen terms for identifying the champion’s role. Most of the included studies focused on technology implementation in a general context, with champions being just one of several measures. This approach resulted in scant descriptions, as champions were often discussed in the results, discussion, or implications sections rather than being the central focus of the research.

As highlighted by Hall et al. [ 18 ]., methodological issues and inadequate reporting in studies of the champion role create challenges for conducting high-quality reviews, introducing uncertainty around the findings. We have adopted a similar approach to Santos et al. [ 19 ], including all studies even when some issues were identified during the quality assessment. Our review shares the same limitations as previous review by Santos et al. [ 19 ] on the champion role.

Practical implications, policy, and future research

The findings emphasize the significance of the relationship between experiences with the champion role and characteristics of organizational settings as crucial factors for success in the champion role. Clear anchoring of the role within the organization is vital and may impact routines, workflows, staffing, and budgets. Despite limited evidence on the experience of the champion’s role, volunteering, hiring newly graduated health personnel, and appointing more than one champion are identified as facilitators of technology implementation. This study underscores the need for future empirical research including clear descriptions of the champion roles, details on study settings and the technologies to be adopted. This will enable the determination of outcomes and success factors in holding champions in technology implementation processes, transferability of knowledge between contexts and technologies as well as enhance the comparability of studies. Furthermore, there is a need for studies to explore experiences with the champion role, preferably from the perspective of multiple stakeholders, as well as focus on the champion role within various healthcare settings.

This study emphasizes that champions can hold significant positions when provided with a clear mandate, dedicated time, and training, contributing their professional, technological, and personal competencies to expedite technology adoption within services. It appears to be an advantage if the health personnel volunteer or apply for the role to facilitate engaged and proactive champions. The implementation of technology in healthcare services demands efforts from the entire service, and the experiences highlighted in this review exhibits that champions can play an important role. Consequently, empirical studies dedicated to the champion role, employing robust designs based current knowledge, are still needed to provide solid understanding of how champions can be a successful initiative when implementing technology in healthcare services.

Data availability

This review relies exclusively on previously published studies. The datasets supporting the conclusions of this article are included within the article and its supplementary files: Description and characteristics of included studies in Table  1 , Study characteristics. The search strategy is provided in Appendix  1 , and the Critical Appraisal Summary of included studies utilizing MMAT is presented in Appendix  2 .

Abbreviations

Electronic Health Record

Implementation Outcomes Framework

Preferred Reporting Items for Systematics and Meta-Analysis

Meskó B, Drobni Z, Bényei É, Gergely B, Győrffy Z. Digital health is a cultural transformation of traditional healthcare. mHealth. 2017;3:38. https://doi.org/10.21037/mhealth.2017.08.07 .

Article   PubMed   PubMed Central   Google Scholar  

Pérez Sust P, Solans O, Fajardo JC, Medina Peralta M, Rodenas P, Gabaldà J, et al. Turning the crisis into an opportunity: Digital health strategies deployed during the COVID-19 outbreak. JMIR Public Health Surveill. 2020;6:e19106. https://doi.org/10.2196/19106 .

Alotaibi YK, Federico F. The impact of health information technology on patient safety. Saudi MedJ. 2017;38:117380. https://doi.org/10.15537/smj.2017.12.20631 .

Article   Google Scholar  

Kuoppamäki S. The application and deployment of welfare technology in Swedish municipal care: a qualitative study of procurement practices among municipal actors. BMC Health Serv Res. 2021;21:918. https://doi.org/10.1186/s12913-021-06944-w .

Kraus S, Schiavone F, Pluzhnikova A, Invernizzi AC. Digital transformation in healthcare: analyzing the current state-of-research. J Bus Res. 2021;123:557–67. https://doi.org/10.1016/j.jbusres.2020.10.030 .

Frennert S. Approaches to welfare technology in municipal eldercare. JTechnolHum. 2020;38:22646. https://doi.org/10.1080/15228835.2020.1747043 .

Konttila J, Siira H, Kyngäs H, Lahtinen M, Elo S, Kääriäinen M, et al. Healthcare professionals’ competence in digitalisation: a systematic review. Clin Nurs. 2019;28:74561. https://doi.org/10.1111/jocn.14710 .

Jacob C, Sanchez-Vazquez A, Ivory C. Social, organizational, and technological factors impacting clinicians’ adoption of mobile health tools: systematic literature review. JMIR mHealth uHealth. 2020;8:e15935. https://doi.org/10.2196/15935 .

Melkas H, Hennala L, Pekkarinen S, Kyrki V. Impacts of robot implementation on care personnel and clients in elderly-care institutions. Int J Med Inf. 2020;134:104041. https://doi.org/10.1016/j.ijmedinf.2019.104041 .

Miech EJ, Rattray NA, Flanagan ME, Damschroder L, Schmid AA, Damush TM. Inside help: an integrative review of champions in healthcare-related implementation. SAGE Open Med. 2018;6. https://doi.org/10.1177/2050312118773261 .

Foong HF, Kyaw BM, Upton Z, Tudor Car L. Facilitators and barriers of using digital technology for the management of diabetic foot ulcers: a qualitative systematic review. Int Wound J. 2020;17:126681. https://doi.org/10.1111/iwj.13396 .

Shea CM. A conceptual model to guide research on the activities and effects of innovation champions. Implement Res Pract. 2021;2. https://doi.org/10.1177/2633489521990443 .

Hudson D. Physician engagement strategies in health information system implementations. Healthc Manage Forum. 2023;36:86–9. https://doi.org/10.1177/08404704221131921 .

Article   PubMed   Google Scholar  

Gullslett MK, Strand Bergmo T. Implementation of E-prescription for multidose dispensed drugs: qualitative study of general practitioners’’ experiences. JMIR HumFactors. 2022;9:e27431. https://doi.org/10.2196/27431 .

Bonawitz K, Wetmore M, Heisler M, Dalton VK, Damschroder LJ, Forman J, et al. Champions in context: which attributes matter for change efforts in healthcare? Implement Sci. 2020;15:62. https://doi.org/10.1186/s13012-020-01024-9 .

George ER, Sabin LL, Elliott PA, Wolff JA, Osani MC, McSwiggan Hong J, et al. Examining health care champions: a mixed-methods study exploring self and peer perspectives of champions. Implement Res Pract. 2022;3. https://doi.org/10.1177/26334895221077880 .

Shea CM, Belden CM. What is the extent of research on the characteristics, behaviors, and impacts of health information technology champions? A scoping review. BMC Med Inf Decis Mak. 2016;16:2. https://doi.org/10.1186/s12911-016-0240-4 .

Hall AM, Flodgren GM, Richmond HL, Welsh S, Thompson JY, Furlong BM, Sherriff A. Champions for improved adherence to guidelines in long-term care homes: a systematic review. Implement Sci Commun. 2021;2(1):85–85. https://doi.org/10.1186/s43058-021-00185-y .

Santos WJ, Graham ID, Lalonde M, Demery Varin M, Squires JE. The effectiveness of champions in implementing innovations in health care: a systematic review. Implement Sci Commun. 2022;3(1):1–80. https://doi.org/10.1186/s43058-022-00315-0 .

Wood K, Giannopoulos V, Louie E, Baillie A, Uribe G, Lee KS, Haber PS, Morley KC. The role of clinical champions in facilitating the use of evidence-based practice in drug and alcohol and mental health settings: a systematic review. Implement Res Pract. 2020;1:2633489520959072–2633489520959072. https://doi.org/10.1177/2633489520959072 .

Rigby K, Redley B, Hutchinson AM. Change agent’s role in facilitating use of technology in residential aged care: a systematic review. Int J Med Informatics. 2023;105216. https://doi.org/10.1016/j.ijmedinf.2023.105216 .

Pluye P, Hong QN. Combining the power of stories and the power of numbers: mixed methods research and mixed studies reviews. Annu Rev Public Health. 2014;35:29–45. https://doi.org/10.1146/annurev-publhealth-032013-182440 .

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. https://doi.org/10.1136/bmj.n71 .

Pettersen S, Berg A, Eide H. Experiences and attitudes to the role of champions in implementation of technology in health services. A systematic review. PROSPERO. 2022. https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42022335750 . Accessed [15 Feb 2023].

Covidence. Better Syst Rev Manag. https://www.covidence.org/ . 2023;26.

Hong QN, Fàbregues S, Bartlett G, Boardman F, Cargo M, Dagenais P, et al. The mixed methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers. Educ Inf. 2018;34:285–91. https://doi.org/10.3233/EFI-180221 .

Braun V, Clarke V. Thematic analysis: a practical guide. 1st ed. SAGE; 2022.

Iqbal MP, Manias E, Mimmo L, Mears S, Jack B, Hay L, Harrison R. Clinicians’ experience of providing care: a rapid review. BMC Health Serv Res. 2020;20:1–10. https://doi.org/10.1186/s12913-020-05812-3 .

Gui X, Chen Y, Zhou X, Reynolds TL, Zheng K, Hanauer DA. Physician champions’ perspectives and practices on electronic health records implementation: challenges and strategies. JAMIA open. 2020;3:5361. https://doi.org/10.1093/jamiaopen/ooz051 .

Bullard KL. Cost effective staffing for an EHR implementation. Nurs Econ. 2016;34:726.

Google Scholar  

Yuan CT, Bradley EH, Nembhard IM. A mixed methods study of how clinician ‘super users’ influence others during the implementation of electronic health records. BMC Med Inf Decis Mak. 2015;15:26. https://doi.org/10.1186/s12911-015-0154-6 .

Owens C, Charles N. Implementation of a text-messaging intervention for adolescents who self-harm (TeenTEXT): a feasibility study using normalisation process theory. Child Adolesc Psychiatry Ment Health. 2016;10:14. https://doi.org/10.1186/s13034-016-0101-z .

Helmer-Smith M, Fung C, Afkham A, Crowe L, Gazarin M, Keely E, et al. The feasibility of using electronic consultation in long-term care homes. JAm Med Dir Assoc. 2020;21:11661170e2. https://doi.org/10.1016/j.jamda.2020.03.003 .

Orchard J, Lowres N, Freedman SB, Ladak L, Lee W, Zwar N, et al. Screening for atrial fibrillation during influenza vaccinations by primary care nurses using a smartphone electrocardiograph (iECG): a feasibility study. Eur J Prev Cardiol. 2016;23:13–20. https://doi.org/10.1177/2047487316670255 .

Bee P, Lovell K, Airnes Z, Pruszynska A. Embedding telephone therapy in statutory mental health services: a qualitative, theory-driven analysis. BMC Psychiatry. 2016;16:56. https://doi.org/10.1186/s12888-016-0761-5 .

Fontaine P, Whitebird R, Solberg LI, Tillema J, Smithson A, Crabtree BF. Minnesota’s early experience with medical home implementation: viewpoints from the front lines. J Gen Intern Med. 2015;30(7):899–906. https://doi.org/10.1007/s11606-014-3136-y .

Kolltveit B-CH, Gjengedal E, Graue M, Iversen MM, Thorne S, Kirkevold M. Conditions for success in introducing telemedicine in diabetes foot care: a qualitative inquiry. BMC Nurs. 2017;16:2. https://doi.org/10.1186/s12912-017-0201-y .

Salbach NM, McDonald A, MacKay-Lyons M, Bulmer B, Howe JA, Bayley MT, et al. Experiences of physical therapists and professional leaders with implementing a toolkit to advance walking assessment poststroke: a realist evaluation. Phys Ther. 2021;101:1–11. https://doi.org/10.1093/ptj/pzab232 .

Schwarz M, Coccetti A, Draheim M, Gordon G. Perceptions of allied health staff of the implementation of an integrated electronic medical record across regional and metropolitan settings. Aust Health Rev. 2020;44:965–72. https://doi.org/10.1071/AH19024 .

Stewart J, McCorry N, Reid H, Hart N, Kee F. Implementation of remote asthma consulting in general practice in response to the COVID-19 pandemic: an evaluation using extended normalisation process theory. BJGP Open. 2022;6:1–10. https://doi.org/10.3399/BJGPO.2021.0189 .

Bennett-Levy J, Singer J, DuBois S, Hyde K. Translating mental health into practice: what are the barriers and enablers to e-mental health implementation by aboriginal and Torres Strait Islander health professionals? JMed. Internet Res. 2017;19:e1. https://doi.org/10.2196/jmir.6269 .

Olsen J, Peterson S, Stevens A. Implementing electronic health record-based National Diabetes Prevention Program referrals in a rural county. Public Health Nurs (Boston Mass). 2021;38(3):464–9. https://doi.org/10.1111/phn.12860 .

Yang L, Brown-Johnson CG, Miller-Kuhlmann R, Kling SMR, Saliba-Gustafsson EA, Shaw JG, et al. Accelerated launch of video visits in ambulatory neurology during COVID-19: key lessons from the Stanford experience. Neurology. 2020;95:305–11. https://doi.org/10.1212/WNL.0000000000010015 .

Article   CAS   PubMed   Google Scholar  

Buckingham SA, Sein K, Anil K, Demain S, Gunn H, Jones RB, et al. Telerehabilitation for physical disabilities and movement impairment: a service evaluation in South West England. JEval Clin Pract. 2022;28:108495. https://doi.org/10.1111/jep.13689 .

Chung OS, Robinson T, Johnson AM, Dowling NL, Ng CH, Yücel M, et al. Implementation of therapeutic virtual reality into psychiatric care: clinicians’ and service managers’’ perspectives. Front Psychiatry. 2022;12:791123. https://doi.org/10.3389/fpsyt.2021.791123 .

Hogan-Murphy D, Stewart D, Tonna A, Strath A, Cunningham S. Use of normalization process theory to explore key stakeholders’ perceptions of the facilitators and barriers to implementing electronic systems for medicines management in hospital settings. Res SocialAdm Pharm. 2021;17:398405. https://doi.org/10.1016/j.sapharm.2020.03.005 .

Moss SR, Martinez KA, Nathan C, Pfoh ER, Rothberg MB. Physicians’ views on utilization of an electronic health record-embedded calculator to assess risk for venous thromboembolism among medical inpatients: a qualitative study. TH Open. 2022;6:e33–9. https://doi.org/10.1055/s-0041-1742227 .

Yusof MM. A case study evaluation of a critical Care Information System adoption using the socio-technical and fit approach. Int J Med Inf. 2015;84:486–99. https://doi.org/10.1016/j.ijmedinf.2015.03.001 .

Dugstad J, Sundling V, Nilsen ER, Eide H. Nursing staff’s evaluation of facilitators and barriers during implementation of wireless nurse call systems in residential care facilities. A cross-sectional study. BMC Health Serv Res. 2020;20:163. https://doi.org/10.1186/s12913-020-4998-9 .

Rea K, Le-Jenkins U, Rutledge C. A technology intervention for nurses engaged in preventing catheter-associated urinary tract infections. Comput Inf Nurs. 2018;36:305–13. https://doi.org/10.1097/CIN.0000000000000429 .

Bail K, Davey R, Currie M, Gibson J, Merrick E, Redley B. Implementation pilot of a novel electronic bedside nursing chart: a mixed-methods case study. Aust Health Rev. 2020;44:672–6. https://doi.org/10.1071/AH18231 .

van Laere J, Aggestam L. Understanding champion behaviour in a health-care information system development project – how multiple champions and champion behaviours build a coherent whole. Eur J Inf Syst. 2016;25:47–63. https://doi.org/10.1057/ejis.2015.5 .

Download references

Acknowledgements

We would like to thank the librarian Malin E. Norman, at Nord university, for her assistance in the development of the search, as well as guidance regarding the scientific databases.

This study is a part of a PhD project undertaken by the first author, SP, and funded by Nord University, Norway. This research did not receive any specific grant from funding agencies in the public, commercial, as well as not-for-profit sectors.

Open access funding provided by Nord University

Author information

Authors and affiliations.

Faculty of Nursing and Health Sciences, Nord university, P.O. Box 474, N-7801, Namsos, Norway

Sissel Pettersen & Anita Berg

Centre for Health and Technology, Faculty of Health Sciences, University of South-Eastern Norway, PO Box 7053, N-3007, Drammen, Norway

You can also search for this author in PubMed   Google Scholar

Contributions

The first author/SP has been the project manager and was mainly responsible for all phases of the study. The second and third authors HE and AB have contributed to screening, quality assessment, analysis and discussion of findings. Drafting of the final manuscript has been a collaboration between the first/SP and third athor/AB. The final manuscript has been approved by all authors.

Corresponding author

Correspondence to Sissel Pettersen .

Ethics declarations

Ethics approval and consent to participate.

This review does not involve the processing of personal data, and given the nature of this study, formal consent is not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Pettersen, S., Eide, H. & Berg, A. The role of champions in the implementation of technology in healthcare services: a systematic mixed studies review. BMC Health Serv Res 24 , 456 (2024). https://doi.org/10.1186/s12913-024-10867-7

Download citation

Received : 19 June 2023

Accepted : 14 March 2024

Published : 11 April 2024

DOI : https://doi.org/10.1186/s12913-024-10867-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Technology implementation
• Healthcare personnel
• Healthcare services
• Mixed methods
• Organizational characteristics
• Technology adoption
• Role definitions
• Healthcare settings
• Systematic review

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

This paper is in the following e-collection/theme issue:

Published on 11.4.2024 in Vol 26 (2024)

Evaluating the Digital Health Experience for Patients in Primary Care: Mixed Methods Study

Authors of this article:

Original Paper

• Melinda Ada Choy 1, 2 , BMed, MMed, DCH, MD   ; 
• Kathleen O'Brien 1 , BSc, GDipStats, MBBS, DCH   ; 
• Katelyn Barnes 1, 2 , BAPSC, MND, PhD   ; 
• Elizabeth Ann Sturgiss 3 , BMed, MPH, MForensMed, PhD   ; 
• Elizabeth Rieger 1 , BA, MClinPsych, PhD   ; 
• Kirsty Douglas 1, 2 , MBBS, DipRACOG, Grad Cert HE, MD  

1 School of Medicine and Psychology, College of Health and Medicine, The Australian National University, Canberra, Australia

2 Academic Unit of General Practice, Office of Professional Leadership and Education, ACT Health Directorate, Canberra, Australia

3 School of Primary and Allied Health Care, Monash University, Melbourne, Australia

Corresponding Author:

Melinda Ada Choy, BMed, MMed, DCH, MD

School of Medicine and Psychology

College of Health and Medicine

The Australian National University

Phone: 61 51244947

Email: [email protected]

Background: The digital health divide for socioeconomic disadvantage describes a pattern in which patients considered socioeconomically disadvantaged, who are already marginalized through reduced access to face-to-face health care, are additionally hindered through less access to patient-initiated digital health. A comprehensive understanding of how patients with socioeconomic disadvantage access and experience digital health is essential for improving the digital health divide. Primary care patients, especially those with chronic disease, have experience of the stages of initial help seeking and self-management of their health, which renders them a key demographic for research on patient-initiated digital health access.

Objective: This study aims to provide comprehensive primary mixed methods data on the patient experience of barriers to digital health access, with a focus on the digital health divide.

Methods: We applied an exploratory mixed methods design to ensure that our survey was primarily shaped by the experiences of our interviewees. First, we qualitatively explored the experience of digital health for 19 patients with socioeconomic disadvantage and chronic disease and second, we quantitatively measured some of these findings by designing and administering a survey to 487 Australian general practice patients from 24 general practices.

Results: In our qualitative first phase, the key barriers found to accessing digital health included (1) strong patient preference for human-based health services; (2) low trust in digital health services; (3) high financial costs of necessary tools, maintenance, and repairs; (4) poor publicly available internet access options; (5) reduced capacity to engage due to increased life pressures; and (6) low self-efficacy and confidence in using digital health. In our quantitative second phase, 31% (151/487) of the survey participants were found to have never used a form of digital health, while 10.7% (52/487) were low- to medium-frequency users and 48.5% (236/487) were high-frequency users. High-frequency users were more likely to be interested in digital health and had higher self-efficacy. Low-frequency users were more likely to report difficulty affording the financial costs needed for digital access.

Conclusions: While general digital interest, financial cost, and digital health literacy and empowerment are clear factors in digital health access in a broad primary care population, the digital health divide is also facilitated in part by a stepped series of complex and cumulative barriers. Genuinely improving digital health access for 1 cohort or even 1 person requires a series of multiple different interventions tailored to specific sequential barriers. Within primary care, patient-centered care that continues to recognize the complex individual needs of, and barriers facing, each patient should be part of addressing the digital health divide.

Introduction

The promise of ehealth.

The rapid growth of digital health, sped up by the COVID-19 pandemic and associated lockdowns, brings the promise of improved health care efficiency, empowerment of consumers, and health care equity [ 1 ]. Digital health is the use of information and communication technology to improve health [ 2 ]. eHealth, which is a type of digital health, refers to the use of internet-based technology for health care and can be used by systems, providers, and patients [ 2 ]. At the time of this study (before COVID-19), examples of eHealth used by patients in Australia included searching for web-based health information, booking appointments on the web, participating in online peer-support health forums, using mobile phone health apps (mobile health), emailing health care providers, and patient portals for electronic health records.

Digital health is expected to improve chronic disease management and has already shown great potential in improving chronic disease health outcomes [ 3 , 4 ]. Just under half of the Australian population (47.3%) has at least 1 chronic disease [ 5 ]. Rates of chronic disease and complications from chronic disease are overrepresented among those with socioeconomic disadvantage [ 6 ]. Therefore, patients with chronic disease and socioeconomic disadvantage have a greater need for the potential benefits of digital health, such as an improvement in their health outcomes. However, there is a risk that those who could benefit most from digital health services are the least likely to receive them, exemplifying the inverse care law in the digital age by Hart [ 7 ].

Our Current Understanding of the Digital Health Divide

While the rapid growth of digital health brings the promise of health care equity, it may also intensify existing inequities [ 8 ]. The digital health divide for socioeconomic disadvantage describes a pattern in which patients considered socioeconomically disadvantaged who are already marginalized through poor access to traditional health care are additionally hindered through poor access to digital health [ 9 ]. In Australia, only 67.4% of households in the lowest household income quintile have home internet access, compared to 86% of the general population and 96.9% of households in the highest household income quintile [ 10 ]. Survey-based studies have also shown that even with internet access, effective eHealth use is lower in populations considered disadvantaged, which speaks to broader barriers to digital health access [ 11 ].

The ongoing COVID-19 global pandemic has sped up digital health transitions with the rapid uptake of telephone and video consultations, e-prescription, and the ongoing rollout of e-mental health in Australia. These have supported the continuation of health care delivery while limiting physical contact and the pandemic spread; however, the early evidence shows that the digital health divide remains problematic. A rapid review identified challenges with reduced digital access and digital literacy among the older adults and racial and ethnic minority groups, which are both groups at greater health risk from COVID-19 infections [ 12 ]. An Australian population study showed that the rapid uptake of telehealth during peak pandemic was not uniform, with the older adults, very young, and those with limited English language proficiency having a lower uptake of general practitioner (GP) telehealth services [ 13 ].

To ensure that digital health improves health care outcome gaps, it is essential to better understand the nature and nuance of the digital health divide for socioeconomic disadvantage. The nature of the digital health divide for socioeconomic disadvantage has been explored primarily through quantitative survey data, some qualitative papers, a few mixed methods papers, and systematic reviews [ 11 , 14 - 16 ]. Identified barriers include a lack of physical hardware and adequate internet bandwidth, a reduced inclination to seek out digital health, and a low ability and confidence to use digital health effectively [ 16 ]. The few mixed methods studies that exist on the digital health divide generally triangulate quantitative and qualitative data on a specific disease type or population subgroup to draw a combined conclusion [ 17 , 18 ]. These studies have found digital health access to be associated with education, ethnicity, and gender as well as trust, complementary face-to-face services, and the desire for alternative sources of information [ 17 , 19 ].

What This Work Adds

This project sought to extend previous research by using an exploratory mixed methods design to ensure that the first step and driver of our survey of a larger population was primarily shaped by the experiences of our interviewees within primary care. This differs from the triangulation method, which places the qualitative and quantitative data as equal first contributors to the findings and does not allow one type of data to determine the direction of the other [ 18 ]. We qualitatively explored the experience of digital health for patients with socioeconomic disadvantage and chronic disease and then quantitatively measured some of the qualitative findings via a survey of the Australian general practice patient population. Our key objective was to provide comprehensive primary mixed methods data, describing the experience and extent of barriers to accessing digital health and its benefits, with a focus on the digital health divide. We completed this research in a primary care context to investigate a diverse community-based population with conceivable reasons to seek digital help in managing their health. Findings from this mixed methods study were intended to provide health care providers and policy makers with a more detailed understanding of how specific barriers affect different aspects or steps of accessing digital health. Ultimately, understanding digital health access can influence the future design and implementation of digital health services by more effectively avoiding certain barriers or building in enablers to achieve improved digital health access not only for everyone but also especially for those in need.

Study Design

We conducted a sequential exploratory mixed methods study to explore a complex phenomenon in depth and then measure its prevalence. We qualitatively explored the experience of digital health for patients with chronic disease and socioeconomic disadvantage in the first phase. Data from the first phase informed a quantitative survey of the phenomenon across a wider population in the second phase [ 18 ]. Both stages of research were conducted before the COVID-19 pandemic in Australia.

Recruitment

Qualitative phase participants.

The eligibility criteria for the qualitative phase were as follows: English-speaking adults aged ≥18 years with at least 1 self-reported chronic disease and 1 marker of socioeconomic disadvantage (indicated by ownership of a Health Care Card or receiving a disability pension, unemployment, or a user of public housing). A chronic disease was defined to potential participants as a diagnosed long-term health condition that had lasted at least 6 months (or is expected to last for at least 6 months; examples are listed in Multimedia Appendix 1 ). The markers of socioeconomic disadvantage we used to identify potential participants were based on criteria typically used by local general practices to determine which patients can have lower or no out-of-pocket expenses. Apart from unemployment, the 3 other criteria to identify socioeconomic disadvantage are means-tested government-allocated public social services [ 20 ]. Qualitative phase participants were recruited from May to July 2019 through 3 general practices and 1 service organization that serve populations considered socioeconomically disadvantaged across urban, regional, and rural regions in the Australian Capital Territory and South Eastern New South Wales. A total of 2 recruitment methods were used in consultation with and as per the choice of the participating organizations. Potential participants were either provided with an opportunity to engage with researchers (KB and MAC) in the general practice waiting room or identified by the practice or organization as suitable for an interview. Interested participants were given a detailed verbal and written description of the project in a private space before providing written consent to be interviewed. All interview participants received an Aus $50 (US $32.68) grocery shopping voucher in acknowledgment of their time.

Quantitative Phase Participants

Eligibility for the quantitative phase was English-speaking adults aged ≥18 years. The eligibility criteria for the quantitative phase were deliberately broader than those for the qualitative phase to achieve a larger sample size within the limitations of recruitment and with the intention that the factors of socioeconomic disadvantage and having a chronic disease could be compared to the digital health access of a more general population. The quantitative phase participants were recruited from November 2019 to February 2020. Study information and paper-based surveys were distributed and collected through 24 general practices across the Australian Capital Territory and South Eastern New South Wales regions, with an option for web-based completion.

Ethical Considerations

Qualitative and quantitative phase research protocols, including the participant information sheet, were approved by the Australian Capital Territory Health Human Research Ethics Committee (2019/ETH/00013) and the Australian National University Human Research Ethics Committee (2019/ETH00003). Qualitative phase participants were given a verbal and written explanation of the study, including how and when they could opt out, before they provided written consent. All interview participants received an Aus $50 (US $32.68) grocery shopping voucher in acknowledgment of their time. Quantitative participants were given a written explanation and their informed consent was implied by return of a completed survey. Participants in both phases of the study were told that all their data was deidentified. Consent was implied through the return of a completed survey.

Qualitative Data Collection and Analysis

Participants were purposively sampled to represent a range in age, gender, degree of socioeconomic disadvantage, and experience of digital health. The sampling and sample size were reviewed regularly by the research team as the interviews were being completed to identify potential thematic saturation.

The interview guide was developed by the research team based on a review of the literature and the patient dimensions of the framework of access by Levesque et al [ 21 ]. The framework by Levesque et al [ 21 ] is a conceptualization of health care access comprising 5 service and patient dimensions of accessibility and ability. The patient dimensions are as follows: (1) ability to perceive, (2) ability to seek, (3) ability to reach, (4) ability to pay, and (5) ability to engage [ 21 ]. The key interview topics included (1) digital health use and access, including facilitators and barriers; (2) attitudes toward digital health; and (3) self-perception of digital health skills and potential training. The interview guide was reviewed for face and content validity by the whole research team, a patient advocate, a digital inclusion charity representative, and the general practices where recruitment occurred. The questions and guide were iteratively refined by the research team to ensure relevance and support reaching data saturation. The interview guide has been provided as Multimedia Appendix 1 . The interviews, which took 45 minutes on average, were taped and transcribed. An interview summary sheet and reflective journal were completed by the interviewer after each interview to also capture nonverbal cues and tone.

Interview transcriptions were coded and processed by inductive thematic analysis. Data collection and analysis were completed in parallel to support the identification of data saturation. Data saturation was defined as no significant new information arising from new interviews and was identified by discussion with the research team [ 22 ]. The 2 interviewers (MAC and KB) independently coded the first 5 transcripts and reflected on them with another researcher (EAS) to ensure intercoder validity and reliability. The rest of the interviews were coded independently by the 2 interviewers, who regularly met to reflect on emerging themes and thematic saturation. Data saturation was initially indicated after 15 interviews and subsequently confirmed with a total of 19 interviews. Coding disagreements and theme development were discussed with at least 1 other researcher (EAS, ER, and KD). Thematic saturation and the final themes were agreed upon by the entire research team.

Quantitative Survey Development

The final themes derived in the qualitative phase of the project guided the specific quantitative phase research questions. The final themes were a list of ordered cumulative barriers experienced by participants in accessing digital health and its benefits ( Figure 1 ). The quantitative survey was designed to test the association between barriers to access and the frequency of use of digital health as a proxy measure for digital health access.

In the survey, the participants were asked about their demographic details, health and chronic diseases, knowledge, use and experience of digital health tools, internet access, perception of digital resource affordability, trust in digital health and traditional health services, perceived capability, health care empowerment, eHealth literacy, and relationship with their GP.

Existing scales and questions from the literature and standardized Australian-based surveys were used whenever possible. We used selected questions and scales from the Australian Bureau of Statistics standards, the eHealth Literacy Scale (eHEALS), the eHealth Literacy Questionnaire, and the Southgate Institute for Health Society and Equity [ 17 , 23 - 26 ]. We adapted other scales from the ICEpop Capability Measure for Adults, the Health Care Empowerment Inventory (HCEI), the Patient-Doctor Relationship Questionnaire, and the Chao continuity questionnaire [ 23 , 27 - 29 ]. Where an existing scale to measure a barrier or theme did not exist, the research team designed the questions based on the literature. Our questions around the frequency of digital health use were informed by multiple existing Australian-based surveys on general technology use [ 30 , 31 ]. Most of the questions used a Likert scale. Every choice regarding the design, adaptation, or copy of questions for the survey was influenced by the qualitative findings and decided on by full agreement among the 2 researchers who completed and coded the interviews. A complete copy of the survey is provided in Multimedia Appendix 2 .

Pilot-testing of the survey was completed with 5 patients, 2 experts on digital inclusion, and 3 local GPs for both the paper surveys and web-based surveys via Qualtrics Core XM (Qualtrics LLC). The resulting feedback on face and content validity, functionality of the survey logic, and feasibility of questionnaire completion was incorporated into the final version of the survey.

The survey was offered on paper with a participant information sheet, which gave the patients the option to complete the web-based survey. The survey was handed out to every patient on paper to avoid sampling bias through the exclusion of participants who could not complete the web-based survey [ 32 ].

Quantitative Data Treatment and Analysis

Data were exported from Qualtrics Core XM to an SPSS (version 26; IBM Corp) data set. Data cleaning and screening were undertaken (KB and KO).

Descriptive statistics (number and percentage) were used to summarize participant characteristics, preference measures, and frequency of eHealth use. Significance testing was conducted using chi-square tests, with a threshold of P <.05; effect sizes were measured by the φ coefficient for 2×2 comparisons and Cramer V statistic for all others. Where the cells sizes were too small, the categories were collapsed for the purposes of significance testing. The interpretation of effect sizes was as per the study by Cohen [ 33 ]. The analysis was conducted in SPSS and SAS (version 9.4; SAS Institute).

Participant Characteristics

Participants’ self-reported characteristics included gender, indigenous status, income category, highest level of education, marital status, and language spoken at home.

Age was derived from participant-reported year of birth and year of survey completion as of 2019 and stratified into age groups. The state or territory of residence was derived from the participant-reported postcode. The remoteness area was derived using the postcode reported by the participants and mapped to a modified concordance from the Australian Bureau of Statistics. Occupation-free text responses were coded using the Australian Bureau of Statistics Census statistics level 1 and 2 descriptors. The country of birth was mapped to Australia, other Organisation for Economic Cooperation and Development countries, and non–Organisation for Economic Cooperation and Development countries.

Frequency of eHealth Use

A summary measure of the frequency of eHealth use was derived from the questions on the use of different types of eHealth.

Specifically, respondents were asked if they had ever used any form of web-based health (“eHealth“) and, if so, to rate how often (never, at least once, every now and then, and most days) against 6 types of “eHealth” (searching for health information online, booking appointments online, emailing health care providers, using health-related mobile phone apps, accessing My Health Record, and accessing online health forums). The frequency of eHealth use was then classified as follows:

• High user: answered “most days” to at least 1 question on eHealth use OR answered “every now and then” to at least 2 questions on eHealth use
• Never user: answered “no” to having ever used any form of eHealth OR “never” to all 6 questions on eHealth use
• Low or medium user: all other respondents.

The frequency of eHealth use was reported as unweighted descriptive statistics (counts and percentages) against demographic characteristics and for the elements of each of the themes identified in phase 1.

Overview of Key Themes

Data were reported against the 6 themes from the phase 1 results of preference, trust, cost, structural access, capacity to engage, and self-efficacy. Where the components of trust, cost, capacity to engage, and self-efficacy had missing data (for less than half of the components only), mean imputation was used to minimize data loss. For each theme, the analysis excluded those for whom the frequency of eHealth use was unknown.

Preference measures (survey section D1 parts 1 to 3) asked participants to report against measures with a 4-point Likert scale (strongly disagree, disagree, agree, and strongly agree). Chi-square tests were conducted after the categories were condensed into 2 by combining strongly disagree and as well as combining strongly agree and agree.

Summary measures for trust were created in 4 domains: trust from the eHealth Literacy Questionnaire (survey section D1 parts 4 to 8), trust from Southgate—GPs, specialists, or allied health (survey section D2 parts 1 to 5), trust from Southgate—digital health (survey section D2 parts 6, 7, 9, and 10), and trust from Southgate—books or pamphlets (survey section D2 part 8). The data were grouped as low, moderate, and high trust based on the assigned scores from the component data. Chi-square tests were conducted comparing low-to-moderate trust against high trust for GP, specialists, or allied health and comparing low trust against moderate-to-high trust for book or pamphlet.

Summary measures for cost were created from survey item C10. To measure cost, participants were asked about whether they considered certain items or services to be affordable. These included cost items mentioned in the qualitative phase interviews relating to mobile phones (1 that connects to the internet, 1 with enough memory space to download apps, downloads or apps requiring payment, repairs, and maintenance costs), having an iPad or tablet with internet connectivity, a home computer or laptop (owning, repairs, and maintenance), home fixed internet access, and an adequate monthly data allowance. These 9 items were scored as “yes definitely”=1 or 0 otherwise. Chi-square tests were conducted with never and low or medium eHealth users combined.

Structural Access

Structural access included asking where the internet is used by participants (survey section C8) and factors relating to internet access (survey section C8 parts 1-3) reporting against a 4-point Likert scale (strongly disagree, disagree, agree, and strongly agree). Chi-square tests were conducted with strongly disagree, disagree, agree, or strongly agree, and never, low, or medium eHealth use combined.

Capacity to Engage

Summary measures for capacity to engage were created from survey section E1. To measure the capacity to engage, participants were asked about feeling “settled and secure,” “being independent,” and “achievement and progress” as an adaptation of the ICEpop Capability Measure for Adults [ 27 ], reporting against a 4-point Likert-like scale. Responses were scored from 1 (“I am unable to feel settled and secure in any areas of my life”) to 4 (“I am able to feel settled and secure in all areas of my life”).

The summary capacity measure was derived by the summation of responses across the 3 questions, which were classified into 4 groups, A to D, based on these scores. Where fewer than half of the responses were missing, mean imputation was used; otherwise, the record was excluded. Groups A and B were combined for significance testing.

Self-Efficacy

Summary measures for self-efficacy were adapted from the eHEALS (E3) and the HCEI (E2) [ 23 , 24 ].

Survey section E3—eHEALS—comprised 8 questions, with participants reporting against a 5-point Likert scale for each (strongly disagree, disagree, neither, agree, and strongly agree). These responses were assigned 1 to 5 points, respectively. The summary eHEALS measure was derived by the summation of responses across the 8 questions, which were classified into 5 groups, A to E, based on these scores. Where fewer than half of the responses were missing, mean imputation was used; otherwise, the record was excluded. Groups A to C and D to E were combined for significance testing.

Survey section E2—HCEI—comprised 5 questions, with participants reporting against a 5-point Likert scale for each (strongly disagree, disagree, neither, agree, and strongly agree). Strongly disagree and disagree and neither were combined, and similarly agree and strongly agree were combined for significance testing.

Qualitative Results

The demographic characteristics of the patients that we interviewed are presented in Table 1 .

The key barriers found to accessing digital health included (1) strong patient preference for human-based health services; (2) low trust in digital health services; (3) high financial costs of necessary tools, maintenance, and repairs; (4) poor publicly available internet access options; (5) reduced capacity to engage due to increased life pressures; and (6) low self-efficacy and confidence in using digital health.

Rather than being an equal list of factors, our interviewees described these barriers as a stepped series of cumulative hurdles, which is illustrated in Figure 1 . Initial issues of preference and trust were foundational to a person even when considering the option of digital health, while digital health confidence and literacy were barriers to full engagement with and optimal use of digital health. Alternatively, interviewees who did use digital health had been enabled by the same factors that were barriers to others.

a GP: general practitioner.

b Multiple answers per respondent.

Strong Patient Preference for Human-Based Health Services

Some patients expressed a strong preference for human-based health services rather than digital health services. In answer to a question about how digital health services could be improved, a patient said the following:

Well, having an option where you can actually bypass actually having to go through the app and actually talk directly to someone. [Participant #10]

For some patients, this preference for human-based health services appeared to be related to a lack of exposure to eHealth. These patients were not at all interested in or had never thought about digital health options. A participant responded the following to the interviewer’s questions:

Interviewer: So when...something feels not right, how do you find out what’s going on?

Respondent: I talk to Doctor XX.

Interviewer: Do you ever Google your symptoms or look online for information?

Respondent: No, I have never even thought of doing that actually. [Participant #11]

For other patients, their preference for human-based health care stemmed from negative experiences with technology. These patients reported actively disliking computers and technology in general and were generally frustrated with what they saw as the pitfalls of technology. A patient stated the following:

If computers and internet weren’t so frigging slow because everything is on like the slowest speed network ever and there’s ads blocking everything. Ads, (expletive) ads. [Participant #9]

A patient felt that he was pushed out of the workforce due his inability to keep up with technology-based changes and thus made a decision to never own a computer:

But, you know, in those days when I was a lot younger those sorts of things weren’t about and they’re just going ahead in leaps and bounds and that’s one of the reasons why I retired early. I retired at 63 because it was just moving too fast and it’s all computers and all those sorts of things and I just couldn’t keep up. [Participant #17]

Low Trust in Digital Health Services

Several patients described low trust levels for digital and internet-based technology in general. Their low trust was generally based on stories they had heard of other people’s negative experiences. A patient said the following:

I don’t trust the internet to be quite honest. You hear all these stories about people getting ripped off and I’ve worked too hard to get what I’ve got rather than let some clown get it on the internet for me. [Participant #11]

Some of this distrust was specific to eHealth. For example, some patients were highly suspicious of the government’s motives with regard to digital health and were concerned about the privacy of their health information, which made them hesitant about the concept of a universal electronic health record. In response to the interviewer’s question, a participant said the following:

Interviewer: Are there any other ways you think that eHealth might help you?

Respondent: I’m sorry but it just keeps coming back to me, Big Brother. [Participant #7]

Another participant said the following:

I just would run a mile from it because I just wouldn’t trust it. It wouldn’t be used to, as I said, for insurance or job information. [Participant #16]

High Financial Costs of the Necessary Tools, Maintenance, and Repairs

A wide variety of patients described affordability issues across several different aspects of the costs involved in digital health. They expressed difficulty in paying for the following items: a mobile phone that could connect to the internet, a mobile phone with enough memory space to download apps, mobile phone apps requiring extra payment without advertisements, mobile phone repair costs such as a broken screen, a computer or laptop, home internet access, and adequate monthly data allowance and speeds to functionally use the internet. Current popular payment systems, such as plans, were not feasible for some patients. A participant stated the following:

I don’t have a computer...I’m not in the income bracket to own a computer really. Like I could, if I got one on a plan kind of thing or if I saved up for x-amount of time. But then like if I was going on the plan I’d be paying interest for having it on like lay-buy kind of thing, paying it off, and if it ever got lost or stolen I would still have to repay that off, which is always a hassle. And yeah. Yeah, I’m like financially not in the state where I’m able to...own a computer right now as I’m kind of paying off a number of debts. [Participant #9]

Poor Publicly Available Internet Access Options

Some patients described struggling without home internet access. While they noted some cost-free public internet access points, such as libraries, hotel bars, and restaurants, they often found these to be inconvenient, lacking in privacy, and constituting low-quality options for digital health. A patient stated the following:

...it’s incredibly slow at the library. And I know why...a friend I went to school with used to belong to the council and the way they set it up, they just got the raw end of the stick and it is really, really slow. It’s bizarre but you can go to the X Hotel and it’s heaps quicker. [Participant #15]

In response to the interviewer's question, a participant said the following:

Interviewer: And do you feel comfortable doing private stuff on computers at the library...?

Respondent: Not really, no, but I don’t have any other choice, so, yeah. [Participant #9]

Reduced Capacity to Engage Due to Increased Life Pressures

When discussing why they were not using digital health or why they had stopped using digital health, patients often described significant competing priorities and life pressures that affected their capacity to engage. An unemployed patient mentioned that his time and energy on the internet were focused primarily on finding work and that he barely had time to focus on his health in general, let alone engage in digital health.

Other patients reported that they often felt that their ability to learn about and spend time on digital health was taken up by caring for sick family members, paying basic bills, or learning English. Some patients said that the time they would have spent learning digital skills when they were growing up had been lost to adverse life circumstances such as being in jail:

So we didn’t have computers in the house when I was growing up. And I didn’t know I’ve never...I’ve been in and out of jail for 28 odd years so it sort of takes away from learning from this cause it’s a whole different… it’s a whole different way of using a telephone from a prison. [Participant #11]

Low Self-Efficacy and Confidence in Starting the Digital Health Process

Some patients had a pervasive self-perception of being slow learners and being unable to use technology. Their stories of being unconfident learners seemed to stem from the fact that they had been told throughout their lives that they were intellectually behind. A patient said the following:

The computer people...wouldn’t take my calls because I’ve always been dumb with that sort of stuff. Like I only found out this later on in life, but I’m actually severely numerically dyslexic. Like I have to triple-check everything with numbers. [Participant #7]

Another patient stated the following:

I like went to two English classes like a normal English class with all the kids and then another English class with about seven kids in there because I just couldn’t I don’t know maybe because I spoke another language at home and they sort of like know I was a bit backward. [Participant #6]

These patients and others had multiple missing pieces of information that they felt made it harder to engage in digital health compared to “easier” human-based services. A patient said the following:

Yeah I’ve heard of booking online but I just I don’t know I find it easier just to ring up. And I’ll answer an email from a health care provider but I wouldn’t know where to start to look for their email address. [Participant #11]

In contrast, the patients who did connect with digital health described themselves as independent question askers and proactive people. Even when they did not know how to use a specific digital health tool, they were confident in attempting to and asking for help when they needed it. A patient said the following:

I’m a “I will find my way through this, no matter how long it takes me” kind of person. So maybe it’s more my personality...If I have to ask for help from somewhere, wherever it is, I will definitely do that. [Participant #3]

Quantitative Results

A total of 487 valid survey responses were received from participants across 24 general practices. The participant characteristics are presented in detail in Table S1 in Multimedia Appendix 3 .

The mean age of the participants was approximately 50 years (females 48.9, SD 19.4 years; males 52.8, SD 20.0 years), and 68.2% (332/487) of the participants identified as female. Overall, 34.3% (151/439) of respondents reported never using eHealth, and 53.8% (236/439) reported high eHealth use.

There were statistically significant ( P <.05) differences in the frequency of eHealth use in terms of age group, gender, state, remoteness, highest level of education, employment status, occupation group, marital status, and language spoken at home, with effect sizes being small to medium. Specifically, high eHealth characteristics were associated with younger age, being female, living in an urban area, and being employed.

Table 2 presents the frequency of eHealth use against 3 internet preference questions.

Preference for using the internet and technology in general and for health needs in particular were significantly related to the frequency of eHealth use ( P <.05 for each), with the effect sizes being small to medium.

a Excludes those for whom frequency of eHealth use is unknown.

b Chi-square tests conducted with strongly disagree and disagree combined, and agree and strongly agree combined.

Table 3 presents the frequency of eHealth use against 4 measures of trust.

The degree of trust was not statistically significantly different for the frequency of eHealth use for any of the domains.

b eHLQ: eHealth Literacy Questionnaire.

c Derived from survey question D1, parts 4 to 8. Mean imputation used where ≤2 responses were missing. If >2 responses were missing, the records were excluded.

d Derived from survey question D2, parts 1 to 5. Mean imputation used where ≤2 responses were missing. If >2 responses were missing, the records were excluded.

e Chi-square test conducted comparing low-to-moderate trust against high trust.

f Derived from survey question D2, parts 6, 7, 9, and 10. Mean imputation used where ≤2 responses were missing. If >2 responses were missing, the records were excluded.

g Derived from survey question D2 part 8.

h Chi-square test conducted comparing low trust against moderate-to-high trust.

Affordability of items and services was reported as No cost difficulty or Cost difficulty. eHealth frequency of use responses were available for 273 participants; among those with no cost difficulty , 1% (2/204) were never users, 14.2% (29/204) were low or medium users, and 84.8% (173/204) were high users of eHealth; among those with cost difficulty , 1% (1/69) were never users, 26% (18/69) were low or medium users, and 73% (50/69) were high users. There was a statistically significant difference in the presence of cost as a barrier between never and low or medium eHealth users compared to high users ( χ 2 1 =5.25; P =.02), although the effect size was small.

Table 4 presents the frequency of eHealth use for elements of structural access.

Quality of internet access and feeling limited in access to the internet were significantly associated with frequency of eHealth use ( P <.05), although the effect sizes were small.

b N/A: not applicable (cell sizes insufficient for chi-square test).

c Chi-square tests conducted with strongly disagree and disagree combined, agree and strongly agree combined, and never and low or medium categories combined.

Table 5 presents the frequency of eHealth use against respondents’ capacity to engage.

Capacity to engage was not significantly different for the frequency of eHealth use ( P =.54). 

b Derived from survey item E1. Where 1 response was missing, the mean imputation was used. If >1 response was missing, the record was excluded.

c Chi-square tests conducted with groups A and B combined.

Table 6 presents the frequency of eHealth use for elements of self-efficacy.

Statistically significant results were observed for the relationship between self-efficacy by eHEALS (moderate effect size) and frequency of eHealth use as well as for some of the questions from the HCEI (reliance on health professionals or others to access and explain information; small effect size; P <.05).

b eHEALS: eHealth Literacy Scale.

c eHEALS derived from item E3 (8 parts). Where ≤ 4 responses were missing, mean imputation was used. If >4 responses were missing, the records were excluded. Groups A to C as well as groups D to E were combined for the chi-square test.

d Strongly disagree, disagree, neither, and agree or strongly agree combined for significance testing.

Principal Findings

This paper reports on the findings of a sequential exploratory mixed methods study on the barriers to digital health access for a group of patients in Australian family medicine, with a particular focus on chronic disease and socioeconomic disadvantage.

In the qualitative first phase, the patients with socioeconomic disadvantage and chronic disease described 6 cumulative barriers, as demonstrated in Figure 1 . Many nonusers of digital health preferred human-based services and were not interested in technology, while others were highly suspicious of the technology in general. Some digitally interested patients could not afford quality hardware and internet connectivity, a barrier that was doubled by low quality and privacy when accessing publicly available internet connections. Furthermore, although some digitally interested patients had internet access, their urgent life circumstances left scarce opportunity to access digital health and develop digital health skills and confidence.

In our quantitative second phase, 31% (151/487) of the survey participants from Australian general practices were found to have never used a form of digital health. Survey participants were more likely to use digital health tools frequently when they also had a general digital interest and a digital health interest. Those who did not frequently access digital health were more likely to report difficulty affording the financial costs needed for digital access. The survey participants who frequently accessed digital health were more likely to have high eHealth literacy and high levels of patient empowerment.

Comparison With Prior Work

In terms of general digital health access, the finding that 31% (151/487) of the survey participants had never used one of the described forms of eHealth is in keeping with an Australian-based general digital participation study that found that approximately 9% of the participants were nonusers and 17% rarely engaged with the internet at all [ 34 ]. With regard to the digital health divide, another Australian-based digital health divide study found that increased age, living in a lower socioeconomic area, being Aboriginal or Torres Strait Islander, being male, and having no tertiary education were factors negatively associated with access to digital health services [ 17 ]. Their findings correspond to our findings that higher-frequency users of eHealth were associated with younger age, being female, living in an urban area, and being employed. Both studies reinforce the evidence of the digital health divide based on gender, age, and socioeconomic disadvantage in Australia.

With regard to digital health barriers, our findings provide expanded details on the range of digital health items and services that present a cost barrier to consumers. Affordability is a known factor in digital access and digital health access, and it is measured often by general self-report or relative expenditure on internet access to income [ 30 ]. Our study revealed the comprehensive list of relevant costs for patients. Our study also demonstrated factors of cost affordability beyond the dollar value of an item, as interviewees described the struggle of using slow public internet access without privacy features and the risks involved in buying a computer in installments. When we reflected on the complexity and detail of the cost barrier in our survey, participants demonstrated a clear association between cost and the frequency of digital health use. This suggests that a way to improve digital health access for some people is to improve the quality, security, and accessibility of public internet access options as well as to provide free or subsidized hardware, internet connection, and maintenance options for those in need, work that is being done by at least 1 digital inclusion charity in the United Kingdom [ 35 ].

Many studies recognize the factors of eHealth literacy and digital confidence for beneficial digital health access [ 36 ]. Our interviews demonstrated that some patients with socioeconomic disadvantage have low digital confidence, but that this is often underlined by a socially reinforced lifelong low self-confidence in their intellectual ability. In contrast, active users, regardless of other demographic factors, described themselves as innately proactive question askers. This was reinforced by our finding of a relationship between health care empowerment and the frequency of eHealth use. This suggests that while digital health education and eHealth literacy programs can improve access for some patients, broader and deeper long-term solutions addressing socioeconomic drivers of digital exclusion are needed to improve digital health access for some patients with socioeconomic disadvantage [ 8 ]. The deep permeation of socially enforced low self-confidence and lifelong poverty experienced by some interviewees demonstrate that the provision of free hardware and a class on digital health skills can be, for some, a superficial offering when the key underlying factor is persistent general socioeconomic inequality.

The digital health divide literature tends to identify the digital health divide, the factors and barriers that contribute to it, and the potential for it to widen if not specifically addressed [ 16 ]. Our findings have also identified the divide and the barriers, but what this study adds through our qualitative phase in particular is a description of the complex interaction of those barriers and the stepped nature of some of those barriers as part of the individual’s experience in trying to access digital health.

Strengths and Limitations

A key strength of this study is the use of a sequential exploratory mixed methods design. The initial qualitative phase guided a phenomenological exploration of digital health access experiences for patients with chronic disease and socioeconomic disadvantage. Our results in both study phases stem from the patients’ real-life experiences of digital health access. While some of our results echo the findings of other survey-based studies on general digital and digital health participation, our method revealed a greater depth and detail of some of these barriers, as demonstrated in how our findings compare to prior work.

As mentioned previously, the emphasis of this study on the qualitative first phase is a strength that helped describe the interactions between different barriers. The interviewees described their experiences as cumulative unequal stepped barriers rather than as producing a nonordered list of equal barriers. These findings expand on the known complexity of the issue of digital exclusion and add weight to the understanding that improving digital health access needs diverse, complex solutions [ 17 ]. There is no panacea for every individual’s digital health access, and thus, patient-centered digital health services, often guided by health professionals within the continuity of primary care, are also required to address the digital health divide [ 37 ].

While the sequential exploratory design is a strength of the study, it also created some limitations for the second quantitative phase. Our commitment to using the qualitative interview findings to inform the survey questions meant that we were unable to use previously validated scales for every question and that our results were less likely to lead to a normal distribution. This likely affected our ability to demonstrate significant associations for some barriers. We expect that further modeling is required to control for baseline characteristics and determine barrier patterns for different types of users.

One strength of this study is that the survey was administered to a broad population of Australian family medicine patients with diverse patterns of health via both paper-based and digital options. Many other digital health studies use solely digital surveys, which can affect the sample. However, we cannot draw conclusions from our survey about patients with chronic disease due to the limitations of the sample size for these subgroups.

Another sample-based limitation of this study was that our qualitative population did not include anyone aged from 18 to 24 years, despite multiple efforts to recruit. Future research will hopefully address this demographic more specifically.

While not strictly a limitation, we recognize that because this research was before COVID-19, it did not include questions about telehealth, which has become much more mainstream in recent years. The patients may also have changed their frequency of eHealth use because of COVID-19 and an increased reliance on digital services in general. Future work in this area or future versions of this survey should include telehealth and acknowledge the impact of COVID-19. However, the larger concept of the digital health divide exists before and after COVID-19, and in fact, our widespread increased reliance on digital services makes the digital divide an even more pressing issue [ 12 ].

Conclusions

The experience of digital health access across Australian primary care is highly variable and more difficult to access for those with socioeconomic disadvantage. While general digital interest, financial cost, and digital health literacy and empowerment are clear factors in digital health access in a broad primary care population, the digital health divide is also facilitated in part by a stepped series of complex and cumulative barriers.

Genuinely improving digital health access for 1 cohort or even 1 person requires a series of multiple different interventions tailored to specific sequential barriers. Given the rapid expansion of digital health during the global COVID-19 pandemic, attention to these issues is necessary if we are to avoid entrenching inequities in access to health care. Within primary care, patient-centered care that continues to recognize the complex individual needs of, and barriers facing, each patient should be a part of addressing the digital health divide.

Acknowledgments

The authors are thankful to the patients who shared their experiences with them via interview and survey completion. The authors are also very grateful to the general practices in the Australian Capital Territory and New South Wales who kindly gave their time and effort to help organize interviews, administer, and post surveys in the midst of the stress of day-to-day practice life and the bushfires of 2018-2019. The authors thank and acknowledge the creators of the eHealth Literacy Scale, the eHealth Literacy Questionnaire, the ICEpop Capability Measure for Adults, the Health Care Empowerment Inventory, the Patient-Doctor Relationship Questionnaire, the Chao continuity questionnaire, and the Southgate Institute for Health Society and Equity for their generosity in sharing their work with the authors [ 17 , 19 - 25 ]. This study would not have been possible without the support of the administrative team of the Academic Unit of General Practice. This project was funded by the Royal Australian College of General Practitioners (RACGP) through the RACGP Foundation IPN Medical Centres Grant, and the authors gratefully acknowledge their support.

Data Availability

The data sets generated during this study are not publicly available due to the nature of our original ethics approval but are available from the corresponding author on reasonable request.

Authors' Contributions

MAC acquired the funding, conceptualized the project, and organized interview recruitment. MAC and KB conducted interviews and analyzed the qualitative data. EAS, ER, and KD contributed to project planning, supervision and qualitative data analysis. MAC, KB and KO wrote the survey and planned quantitative data analysis. MAC and KB recruited practices for survey administration. KO and KB conducted the quantitative data analysis. MAC and KO, with KB drafted the paper. EAS, ER, and KD helped with reviewing and editing the paper.

Conflicts of Interest

None declared.

Phase 1 interview guide.

Phase 2 survey: eHealth and digital divide.

Phase 2 participant characteristics by frequency of eHealth use.

• Eysenbach G. What is e-health? J Med Internet Res. 2001;3(2):E20. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Iyawa GE, Herselman M, Botha A. Digital health innovation ecosystems: from systematic literature review to conceptual framework. Procedia Comput Sci. 2016;100:244-252. [ FREE Full text ] [ CrossRef ]
• Berrouiguet S, Baca-García E, Brandt S, Walter M, Courtet P. Fundamentals for future mobile-health (mHealth): a systematic review of mobile phone and web-based text messaging in mental health. J Med Internet Res. Jun 10, 2016;18(6):e135. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Shen H, van der Kleij RM, van der Boog PJ, Chang X, Chavannes NH. Electronic health self-management interventions for patients with chronic kidney disease: systematic review of quantitative and qualitative evidence. J Med Internet Res. Nov 05, 2019;21(11):e12384. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Australia's health 2018. Australian Institute of Health and Welfare. 2018. URL: https://www.aihw.gov.au/reports/australias-health/australias-health-2018/contents/table-of-contents [accessed 2024-04-04]
• Australian Institute of Health and Welfare. Chronic Diseases and Associated Risk Factors in Australia, 2006. Canberra, Australia. Australian Institute of Health and Welfare; 2006.
• Hart JT. The inverse care law. The Lancet. Feb 27, 1971;297(7696):405-412. [ CrossRef ]
• Davies AR, Honeyman M, Gann B. Addressing the digital inverse care law in the time of COVID-19: potential for digital technology to exacerbate or mitigate health inequalities. J Med Internet Res. Apr 07, 2021;23(4):e21726. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Choi NG, Dinitto DM. The digital divide among low-income homebound older adults: internet use patterns, eHealth literacy, and attitudes toward computer/internet use. J Med Internet Res. May 02, 2013;15(5):e93. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Household use of information technology. Australian Bureau of Statistics. 2018. URL: https://tinyurl.com/4efm6u92 [accessed 2024-03-24]
• Kontos E, Blake KD, Chou WY, Prestin A. Predictors of eHealth usage: insights on the digital divide from the health information national trends survey 2012. J Med Internet Res. Jul 16, 2014;16(7):e172. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Litchfield I, Shukla D, Greenfield S. Impact of COVID-19 on the digital divide: a rapid review. BMJ Open. Oct 12, 2021;11(10):e053440. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Butler DC, Joshy G, Douglas KA, Sayeed MS, Welsh J, Douglas A, et al. Changes in general practice use and costs with COVID-19 and telehealth initiatives: analysis of Australian whole-population linked data. Br J Gen Pract. Apr 27, 2023;73(730):e364-e373. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Arsenijevic J, Tummers L, Bosma N. Adherence to electronic health tools among vulnerable groups: systematic literature review and meta-analysis. J Med Internet Res. Feb 06, 2020;22(2):e11613. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kontos EZ, Bennett GG, Viswanath K. Barriers and facilitators to home computer and internet use among urban novice computer users of low socioeconomic position. J Med Internet Res. Oct 22, 2007;9(4):e31. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Latulippe K, Hamel C, Giroux D. Social health inequalities and eHealth: a literature review with qualitative synthesis of theoretical and empirical studies. J Med Internet Res. Apr 27, 2017;19(4):e136. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Foley K, Freeman T, Ward P, Lawler A, Osborne R, Fisher M. Exploring access to, use of and benefits from population-oriented digital health services in Australia. Health Promot Int. Aug 30, 2021;36(4):1105-1115. [ CrossRef ] [ Medline ]
• Cresswell JW, Plano Clark VL. Designing and Conducting Mixed Methods Research. Thousand Oaks, CA. SAGE Publications; 2007.
• Tappen RM, Cooley ME, Luckmann R, Panday S. Digital health information disparities in older adults: a mixed methods study. J Racial Ethn Health Disparities. Feb 2022;9(1):82-92. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Who can get a card. Services Australia. URL: https://www.servicesaustralia.gov.au/who-can-get-health-care-card?context=21981 [accessed 2023-11-03]
• Levesque JF, Harris MF, Russell G. Patient-centred access to health care: conceptualising access at the interface of health systems and populations. Int J Equity Health. Mar 11, 2013;12:18. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bryant A, Charmaz K. The SAGE Handbook of Grounded Theory, Paperback Edition. Thousand Oaks, CA. SAGE Publications; 2010.
• Johnson MO, Rose CD, Dilworth SE, Neilands TB. Advances in the conceptualization and measurement of health care empowerment: development and validation of the health care empowerment inventory. PLoS One. 2012;7(9):e45692. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Norman CD, Skinner HA. eHEALS: the eHealth literacy scale. J Med Internet Res. Nov 14, 2006;8(4):e27. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kayser L, Karnoe A, Furstrand D, Batterham R, Christensen KB, Elsworth G, et al. A multidimensional tool based on the eHealth literacy framework: development and initial validity testing of the eHealth Literacy Questionnaire (eHLQ). J Med Internet Res. Feb 12, 2018;20(2):e36. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Standards. Australian Bureau of Statistics. URL: https://www.abs.gov.au/statistics/standards [accessed 2024-04-04]
• Al-Janabi H, Flynn TN, Coast J. Development of a self-report measure of capability wellbeing for adults: the ICECAP-A. Qual Life Res. Feb 2012;21(1):167-176. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Van der Feltz-Cornelis CM, Van Oppen P, Van Marwijk HW, De Beurs E, Van Dyck R. A patient-doctor relationship questionnaire (PDRQ-9) in primary care: development and psychometric evaluation. Gen Hosp Psychiatry. 2004;26(2):115-120. [ CrossRef ] [ Medline ]
• Chao J. Continuity of care: incorporating patient perceptions. Fam Med. 1988;20(5):333-337. [ Medline ]
• Wilson CK, Thomas J, Barraket J. Measuring digital inequality in Australia: the Australian digital inclusion index. JTDE. Jun 30, 2019;7(2):102-120. [ CrossRef ]
• Digital participation: a view of Australia's online behaviours. Australia Post. Jul 2017. URL: https://auspost.com.au/content/dam/auspost_corp/media/documents/white-paper-digital-inclusion.pdf [accessed 2024-04-04]
• Poli A, Kelfve S, Motel-Klingebiel A. A research tool for measuring non-participation of older people in research on digital health. BMC Public Health. Nov 08, 2019;19(1):1487. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Cohen J. Statistical Power Analysis for the Behavioral Sciences Second Edition. London, UK. Routledge; 1988.
• Borg K, Smith L. Digital inclusion and online behaviour: five typologies of Australian internet users. Behav Inf Technol. Feb 15, 2018;37(4):367-380. [ CrossRef ]
• Mathers A, Richardson J, Vincent S, Joseph C, Stone E. Good Things Foundation COVID-19 response report. Good Things Foundation. 2020. URL: https://tinyurl.com/2peu3kak [accessed 2024-04-04]
• Norman CD, Skinner HA. eHealth literacy: essential skills for consumer health in a networked world. J Med Internet Res. Jun 16, 2006;8(2):e9. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Neves AL, Burgers J. Digital technologies in primary care: implications for patient care and future research. Eur J Gen Pract. Dec 11, 2022;28(1):203-208. [ FREE Full text ] [ CrossRef ] [ Medline ]

Abbreviations

Edited by T Leung; submitted 03.07.23; peer-reviewed by T Freeman, H Shen; comments to author 16.08.23; revised version received 30.11.23; accepted 31.01.24; published 11.04.24.

©Melinda Ada Choy, Kathleen O'Brien, Katelyn Barnes, Elizabeth Ann Sturgiss, Elizabeth Rieger, Kirsty Douglas. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 11.04.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

• Open access
• Published: 03 April 2024

The environmental awareness of nurses as environmentally sustainable health care leaders: a mixed method analysis

• Olga María Luque-Alcaraz   ORCID: orcid.org/0000-0003-1598-1422 1 , 2 , 3 , 5 ,
• Pilar Aparicio-Martínez   ORCID: orcid.org/0000-0002-2940-8697 3 , 4 ,
• Antonio Gomera   ORCID: orcid.org/0000-0003-0603-3017 2 &
• Manuel Vaquero-Abellán   ORCID: orcid.org/0000-0002-0602-317X 2 , 3 , 4  

BMC Nursing volume  23 , Article number:  229 ( 2024 ) Cite this article

397 Accesses

6 Altmetric

Metrics details

People worldwide are concerned with the possibility of climate change, microplastics, air pollution, and extreme weather affecting human health. Countries are implementing measures to reduce environmental impacts. Nurses play a vital role, primarily through Green Teams, in the process of promoting sustainable practices and minimizing the environmental footprint of health care facilities. Despite existing knowledge on this topic, assessing nurses’ environmental awareness and behavior, including the barriers they face, is crucial with regard to improving sustainable health care practices.

To analyze the environmental awareness and behavior of nurses, especially nurse leaders, as members of the Green Team and to identify areas for improvement with regard to the creation of a sustainable environment.

A sequential mixed-method study was conducted to investigate Spanish nurses. The study utilized an online survey and interviews, including participant observation. An online survey was administered to collect quantitative data regarding environmental awareness and behavior. Qualitative interviews were conducted with environmental nurses in specific regions, with a focus on Andalusia, Spain.

Most of the surveyed nurses ( N  = 314) exhibited moderate environmental awareness (70.4%), but their environmental behavior and activities in the workplace were limited (52.23% of participants rarely performed relevant actions, and 35.03% indicated that doing so was difficult). Nurses who exhibited higher levels of environmental awareness were more likely to engage in sustainable behaviors such as waste reduction, energy conservation, and environmentally conscious purchasing decisions ( p  < 0.05). Additionally, the adjusted model indicated that nurses’ environmental behavior and activities in the workplace depend on the frequency of their environmental behaviors outside work as well as their sustainable knowledge ( p  < 0.01). The results of the qualitative study ( N  = 10) highlighted certain limitations in their daily practices related to environmental sustainability, including a lack of time, a lack of bins and the pandemic. Additionally, sustainable environmental behavior on the part of nursing leadership and the Green Team must be improved.

Conclusions

This study revealed that most nurses have adequate knowledge, attitudes, and behaviors related to environmental sustainability both inside and outside the workplace. Limitations were associated with their knowledge and behaviors outside of work. This study also highlighted the barriers and difficulties that nurses face in their attempts to engage in adequate environmental behaviors in the workplace. Based on these findings, interventions led by nurses and the Green Team should be developed to promote sustainable behaviors among nurses and address the barriers and limitations identified in this research.

Graphical Abstract

Peer Review reports

Introduction

The impact of climate change on human society is a global concern, especially with regard to microplastics, resource shortages, air pollution, droughts, and extreme weather. Such consequences affect human health both directly and indirectly, resulting in an increase in pathologies and a deterioration in medical attention [ 1 , 2 ]. In this context, diverse measures aimed at reducing the environmental impact of daily activities and minimizing the ecological footprint thereof [ 3 ] have been implemented by multiple countries [ 4 , 5 , 6 , 7 ]; these activities have been framed as environmental regulations in line with the Sustainable Development Goals (SDGs) [ 8 ].

The SDGs are being integrated into governments and a variety of other contexts, including the health care system. Spain is dedicated to such a goal, i.e., that of promoting a greener and more democratic health care transition. To achieve this goal, strategic plans have been developed to mitigate the effects of climate change [ 9 , 10 ]. One specific such program is the Strategic Health and Environment Plan (PESMA) [ 11 ], whose aim is to enhance the synergy between health and the environment innovatively by assessing the impact of the population in terms of 14 environmental indicators [ 12 ].

One such indicator focuses on the resources and support needed for sustainable practices, especially for nurses, due to the impact of the environment on their work [ 13 , 14 ]. The PESMA highlights the fact that health care providers should be included in strategies to reduce carbon footprints, build resilience to address the challenges associated with climate change and embrace a leadership role in the task of promoting sustainable health care practices [ 13 , 14 , 15 , 16 ]. Another critical aspect of PESMA focuses on education, training, and incentives that can promote sustainable behavior among health care workers, especially nurses [ 17 , 18 ]. As frontline health care workers, nurses have a unique opportunity to advocate for sustainable practices and reduce the environmental impact of the health care system. Nurses’ knowledge and behavior are limited despite the fact that nurses have positive attitudes toward environmental sustainability [ 19 ].

This situation stands in contrast to the role of nurses in the creation of more sustainable hospitals via the “Green Team” [ 20 ]. The Green Team, which originated in the United States of America a decade ago, is a committee that is responsible for finding and implementing sustainability projects to decrease the environmental impacts of daily operations. Members of various departments collaborate with sustainability staff to detect opportunities, spread awareness, and promote staff involvement in line with the Committee’s mission [ 21 ]. The team, which typically consists of and is led by nurses, aims to increase awareness of the health care industry’s effect on the environment and to develop tactics to mitigate the adverse environmental effects of hospitals.

In Spain, Green Teams, which span multiple disciplines and usually led by nursing professionals, are committed to sustainable change in health care [ 22 ]. Environmental nursing leaders on Green Teams control environmental sustainability in health care settings and provide education, resources, and support to other professionals with regard to the implementation of sustainable practices [ 23 ]. Accordingly, all nurses can contribute to the tasks of mitigating the impact of climate change on public health outcomes and promoting sustainable health for all [ 24 ]. These actions improve nurses’ knowledge, attitudes, and behavior in terms of sustainability and promote sustainable practices in health care settings, thus leading to a better understanding of the barriers faced by nurses in this context [ 24 , 25 , 26 ].

However, measuring and identifying nurses’ environmental awareness is essential for the promotion of sustainable hospitals [ 27 , 28 ]. Multidimensional indicators have been proposed for this purpose [ 16 ], the responsibility for which lies with nurse leaders on Green Teams. Nurses are responsible for promoting sustainability in health care organizations, as discussed by Kallio et al. (2018) [ 29 ], as well as for promoting nursing competencies related to environmental sustainability [ 30 ]. Several studies, including Harris et al. (2009) and Phiri et al. (2022), have examined nurses’ roles in environmental health and the effects of their leadership on the promotion of sustainability, especially during the COVID-19 pandemic, thereby emphasizing the importance of leadership [ 31 , 32 ].

As Ojemeni et al. (2019) discussed, leadership effectiveness in Green Teams, nursing teams and health care organizations must prioritize quality control and health care improvement to ensure sustainable development [ 33 ].

The topic of environmental management in health care organizations has been studied extensively, and an environmental or ecological model of care for promoting sustainability has been proposed [ 34 ]. As environmental creators and leaders on Green Teams, nurses are vital for minimizing hazardous waste in health care settings and improving awareness [ 35 ].

Although nurses have some degree of existing knowledge and awareness of sustainability, it is crucial to assess their proficiency in environmental matters and to gauge their environmental awareness. Such an evaluation can help identify areas for improvement within clinical management units [ 20 , 33 , 36 ]. Education and training programs can effectively promote sustainable behavior among nurses, but interventions should also address the barriers they face in their attempts to implement sustainable practices [ 37 ]. Therefore, it is imperative to examine the factors that foster sustainable behavior among nurses and to identify effective interventions that can promote sustainable health care practices and minimize the environmental footprint of health care facilities. Accordingly, this study aimed to analyze the environmental awareness and behavior of nurses, especially nurse leaders, as members of the Green Team and to identify areas for improvement with regard to creating a sustainable environment.

Study design

A sequential mixed-method study was conducted based on an online survey and interviews with a representative sample of Spanish nurses, including participant observation.

The study was divided into two phases. In the first phase, a cross-sectional, descriptive exploratory analysis was performed; this analysis relied on the results revealed using the Nurse’s Environmental Awareness Tool in Spanish (NEAT-es) [ 38 ], which was divided into three subscales: nursing awareness scale (NAS), environmental behaviors outside the workplace (PEB) and sustainable behaviors in the workplace (NPEB). In the second phase, qualitative interviews with environmental nurses (see Supplementary file 1 ) were conducted in regions featuring specific environmental units that were available in person (Andalusia).

Participants

The participants were recruited from public and private institutions associated with the National Health System, particularly from the nursing staff. The scope of the study focused on Spain, and the sample included all the nursing staff who completed the questionnaire and met the inclusion criteria.

The sampling process focused on the population of nurses in Spain in 2020, which was estimated to consist of 388,153 nurses. Therefore, a random sample of 314 participating individuals was sufficient to estimate the population with 95% confidence and an accuracy of +/- 2% units, which was expected to account for approximately 90% of the overall population. The inclusion and exclusion criteria used for the sample focused on nursing staff, nursing care auxiliary technicians, and students with relevant degrees, as this members of this group have the most significant presence in the health system and engage in direct and daily contact with environmental management in health centers (hospitals, primary care centers, sociosanitary centers and others). The remaining health and nonhealth personnel were excluded.

Additionally, the person from each unit who served as the environmental coordinator and other nurses from the ward who were members of the Green Team were asked to participate in the interviews and observations. The environmental coordinators, most of who were nursing supervisors, were determined based on the number of members of the Green Team and the sampling calculation used for the observational study. The interviews took place after various sessions, talks, or courses pertaining to environmental sustainability at the clinical management units.

Data collection

An intentional sampling process was implemented, and the data collection period spanned from November 2019 to March 2021. The observational data were collected in Spain via messages and posts on social media with the goal of quantifying nurses’ environmental awareness.

The initial sample of qualitative study included five environmental nursing leaders (NLs), 14 registered nurses (RNs), and ten nursing undergraduates. The final sample was reduced when the interviews reached data saturation ( N  = 10, five NLs, and five RNs). Before the interviews, a focal group composed of one nurse, one physician, two engineers and a psychologist was tested using the questions included in this research as part of a pilot study ( Supplementary file 1 ). These interviews were conducted at the beginning of the participant’s shift, usually in the morning, and they featured a median time of 30 min, a minimum of 20 min and a maximum of one hour per participant.

One researcher (O.A.L.) also observed nurses during their daily work after the interview from a position within the ward as an added team member or staff member. Nevertheless, the observer did not highlight mistakes or sustainability issues during the observation process. No other researcher was involved in this step of the ethnographic analysis to avoid bias with regard to observing a variety of tasks ranging from preparing medication to implementing treatments.

The data collected through the interviews were recorded on a Samsung Galaxy 31 A, and observations were collected in a field notebook based on the Google Keep and Evernote mobile applications from November 2019 to mid-March 2021. This study was conducted at a regional level 1 hospital in southern Spain, particularly in various clinical management units (neurosurgery, internal medicine, cardiology, traumatology, and COVID-19 units, among others), and it focused on nursing supervisors, who are the leaders who bear responsibility for environmental awareness (NLs), and registered nurses (RNs) who were members of the Green Team.

Data analysis

The quantitative data were analyzed by reference to descriptive statistics, including the mean, standard deviation (SD), and 95% confidence interval (CI); the relative frequencies of the variables were also analyzed. Normalization tests, Kolmogorov‒Smirnov tests with Lilliefors correction, and Q‒Q tests were used to compare the goodness-of-fit to an average data distribution with regard to continuous or discrete quantitative variables. The comparison of two or three independent means was performed using Student’s t test and analyses of variance for each variable. The Χ 2 test with Yates’ correction was used to compare percentages and Pearson’s correlation (r) coefficients across the quantitative variables. Finally, associations among the NPEB and the other variables were studied through multiple linear regression. Participant observation was used to support the qualitative study of the reflective ethnographic type [ 39 , 40 ], and this process ended when the data reached saturation. Two researchers developed transcripts for the interviews based on the recorded interviews and added descriptions based on the notes from the field notebook. The identification of themes and patrons was based on a process of triangulation among the researchers and by cross-checking the results. The interviews with nurses were analyzed to summarize the content analysis and identify keywords and concurrency among the terms. The themes thus identified included Green Teams, sustainable environmental behaviors, environment awareness, leadership barriers and limitations and areas for improvement.

EPIDAT (version 4.2) and SPSS (version 25) software were used to support the quantitative analysis. The computer program ATLAS.ti (version 22) and the Office Package with Microsoft Word Excel (version 2019) were used for the interviews and the visualization of the keywords based on the themes identified based on the records, observations and field notebooks.

Nurses’ awareness, knowledge, attitudes and skills.

The ages of the Spanish staff, mainly nurses, included in this study ( N  = 314) ranged from 19 to 68, with a mean age of 37.02 ± 12.7, CI = 95%, 35.6–38.4 years); in addition, 76.4% of these participants were women with more than 20 years of working experience (35.1%), and the majority were registered nurses (70.4%). Moreover, 113 (36%) participants worked at a local or regional hospital (30%) and were employees of a public institution (85.3%). Half of the nurses (157) worked only a morning shift (Table  1 ) in Andalusia, Madrid, or Catalonia (62.4%). The diverse autonomous regions on which this research focused were homogenously distributed and structured in line with the population. The analysis of these areas was also based on the specific inclusion of environmental units led by nurses (Andalusia, Madrid, and Catalonia), in contrast with regions featuring undetermined units or leaders related to this topic (such as Valencia) (37.5%).

Regarding nursing awareness, nurses scored higher on the PEB (31.83 ± 8.02 CI 95% 30.94–32.72 with regard to frequency vs. 32.36 ± 7.15 CI 95% 31.57–33.15 with respect to difficulty) than on the NAS (26.13 ± 9.91 CI 95% 25.03–27.23 with regard to knowledge vs. 47.39 ± 5.97 CI 95% 46.73–48.05 with respect to impact) and the NPEB (23.82 ± 6.45 CI 95% 23.10-24.53 with regard to frequency vs. 25.71 ± 6.31 CI 95% 25.01–26.41 with respect to difficulty). These results indicated that environmental knowledge among the Spanish population was limited (55.7%), although the nurses included in this research were aware of their potential impact on the environment (70.4%). The PEB subscale focused mostly on following environmental guidelines in their homes (57.3%) because these sustainable domestic tasks are easier for them (63.1%) than tasks in the professional field. The second subscale, NPEB, indicated that sustainable activities such as recycling were easy for the participants (57.6%), but sometimes they engaged in such activities less frequently than they would like (52.2%) (Fig.  1 and Fig.  2 ).

Representation of the frequency of nursing environmental behavior

Difficulty of engaging in adequate environmental behaviors

The sociodemographic variables indicated differences among the NEAT subscales (Table  2 ). Gender, working experience (with a median value of 10 years), and the position held in the institution and region were relevant with regard to environmental knowledge ( p  < 0.01), environmental behavior outside the workplace ( p  < 0.01), and environmental behavior in the workplace ( p  < 0.01).

The NPEB was associated with the worst scores, thereby reflecting the nurses’ environmental behavior and activities in the workplace (52.23% rarely performed relevant activities, and 35.03% indicated that doing so was difficult) (Fig.  1 and Fig.  2 ). The NPEB values pertaining to environmental behavior were positively linked to age ( r  = 0.412; p  < 0.001), NAS knowledge ( r  = 0.526; p  < 0.001), PEB frequency ( r  = 0. 57; p  < 0.001), PEB difficulty ( r  = 0.329; p  < 0.001), and finally, difficulty performing adequate environmental behaviors ( r  = 0.499; p  < 0.001). Additionally, the value of the NPEB with regard to the difficulty of performing adequate environmental behaviors was positively associated with age ( r  = 0.149; p  = 0.008), NAS knowledge ( r  = 0.249; p  < 0.001), PEB frequency ( r  = 0. 244; p  < 0.001) and PEB difficulty ( r  = 0.442; p  < 0.001).

Based on the relevance of certain sociodemographic variables, the nurses’ environmental awareness (NAS) and their behavior outside the workplace (PEB), linear multiple regression was performed to investigate nursing behavior in the workplace (NPEB). The initial model (square sum = 488.655; p  < 0.0001) indicated that age, the impact of nursing awareness (NAS), and the frequency of sustainable behaviors outside the workplace (PEB) were not relevant to nursing behavior in the workplace (NPEB) in terms of the frequency of performing adequate behavior or the difficulties experienced ( p  > 0.05). Based on these results, the adjusted model was calculated (Table  3 ), indicating that NPEB depends on PEB frequency and NAS knowledge ( p  < 0.01).

Nursing environmental behavior in the context of Green Teams: Barriers and areas for improvement.

The participants in the qualitative study ( N  = 10) included nine women and one man; their median age was 49 years; they exhibited an interval quartile range of 35–60; they had levels of working experience ranging between 20 and 30 years, and they worked only in the mornings (7/10). Furthermore, the group including nurses and nursing supervisors (5/10) exhibited higher levels of education (see Supplementary file 2 ). The themes identified via repetition and associations during the interviews and observations indicated links among nurses’ responsibilities on the Green Team since they conformed to the nature of such teams (i). This team and nursing leaders identified sustainable environmental behavior (ii) that could improve environmental awareness (iii), knowledge, aptitude, and skills. The nurses who are responsible for sustainable changes should be the leaders (iv), and the relevant barriers and limitations (v) and areas for improvement (vi) in diverse areas should be identified simultaneously.

Green teams were linked to nursing responsibilities in the context of environmental sustainability.

In the interviews, the Green Teams, led by environmental leader nurses and comprising various staff members, were identified as crucial committees dedicated to enhancing environmental awareness and knowledge among hospital staff. Participants indicated that these teams facilitated regular meetings to discuss sustainable practices and played a pivotal role in testing behaviors and knowledge related to environmental sustainability. The Green Teams were highlighted as platforms for fostering collaboration and discussion surrounding sustainable practices. Participants noted that these teams facilitated the main purpose of the team and its members to improve the hospital staff’s knowledge and attitudes via meetings (RN 2,3 and NL 1,3). Subsequently, the NL also indicated a key role of the team in the testing of behaviors and knowledge. The behavior of registered nurses should be tested using questions according to the NLs. Also, the NLs are included in disponibility of of proper disposal methods for medical waste:

“So, where is the rubbish bin for medicines, that white one that you showed in the session that is used for the remains of medicines that we do not give to patients?” [(NL5)]

By such comments, it can be inferred that the Green Team not only disseminates information, manages the training and measures knowledge but also ensures that staff members understand and adhere to best practices in waste management. These tasks of the NLs and other RNs in the Green Team contribute to the overall efficiency and effectiveness of environmental sustainability efforts within the hospital.

Sustainable environmental behaviors were emerged by Green Teams.

The results of the analysis indicated some degree of resistance among the nurses working at the clinical management units with regard to their lack of competencies, especially those pertaining to knowledge, skills and attitudes. The comments from the interviews highlighted potential factors contributing to this resistance, including age-related differences, varying levels of awareness, and challenges in applying the principles of reduce, reuse, and recycle (the three Rs). For instance, one repetitive comment expressed a sentiment of uncertainty, stating “It is what is, but we don’t know it or what to do with it” (RN 3,4,5, and NL 2,3).

“We know what the light packing is, and they (maintenance people) installed it to reduce the lights and reduce the expense and cost, but we don’t know what to do with the rubbish bins” [(NL 4)]

This comment highlights a disconnect between awareness of specific sustainable initiatives and the practical knowledge to implement them effectively. All comments reflect the importance of addressing knowledge gaps and providing practical guidance to support nurses in adopting sustainable environmental behaviours. By acknowledging and addressing these challenges, healthcare facilities can enhance their environmental stewardship efforts and promote a culture of sustainability among staff members.

Environmental awareness were drawn from the nursing responsibilities that led to the creation of the Green Team.

The comments indicated that environmental awareness among nurses was influenced by training sessions and courses on environmental sustainability. After receiving training featuring lectures and courses on environmental sustainability, the leaders also reflected on the ways in which nurses put the recommendations made during the environmental sustainability courses into practice. Moreover, the leaders indicated that education should be beyond formal training sessions. The environmental leaders were interested in supplementing these courses with environmental education practices for the general population, as noted, for example, in reports of discharge from patient care or cycling on the ward. These activities indicated the ideal of including a holistic approach to sustainability within the healthcare setting.

Relevant statements included, “We have to separate residues according to the material… light plastic goes to… it is important for the unit and all of us” (NL 2,5). One key point that the referees and registered nurses highlighted pertained to the climate, particularly the lack of water (NL 1–5 and RN 1,2).

“The drought is getting worse; I don’t know how we are going to keep up… we hope it rains soon” [(RN1)]

Overall, the interviews shed light on the efforts to foster environmental awareness among nurses through formal training and practical integration into everyday practices. These observations emphasize the importance of ongoing education and action in addressing environmental concerns within healthcare settings.

Leadership, which was linked by comments to the Green Teams.

The interviews revealed that leadership, particularly within the context of Green Teams, is crucial in promoting environmental awareness and fostering a culture of sustainability among nursing staff. All the participants ( n  = 10) indicated that the presence of adequate knowledge, meetings and awareness among nursing staff were the most important factors. These factors were identified as key drivers in promoting sustainable practices within the healthcare environment. NLs indicated the importance of creating a supportive working environment where nurses feel comfortable asking questions and seeking clarification without fear of negative feedback. Relevant statements included, “It is key to receive feedback from the nurses and provide a good working environment so that they can ask questions and reflect without negative comments” (NL 1,2,4, and RN 1,2). This working environment allowed the registered nurses to ask for help regarding the three Rs:

“Could you remind me (referring to the environmental coordinator) how the sustainable guidelines were included in the discharge report for the continuity of care; I remember some things from the course you gave us, but I want to convey it completely to my patient” [(RN2)]

Barriers and limitations, were drawn from nurses’ responsibilities.

Several nurses indicated that the difficulties they encountered with regard to performing environmental behaviors pertained to the lack of time, adequate bins, and space as well as the limited number of nurses per patient in the wards. Despite these challenges, participants noted a positive outcome in the form of increased awareness of sustainability issues among nurses, indicating a growing recognition of the importance of environmental stewardship within the healthcare setting. One factor that increased the barriers to environmental adequacy was the pandemic, which increased waste and rubbish. Despite these challenges, participants noted a positive outcome in the form of increased awareness of sustainability issues among nurses, indicating a growing recognition of the importance of environmental stewardship within the healthcare setting. Relevant statements included “There are not enough green rubbish bins for COVID waste” (EL 1,4,5 and RN1,2) and “How are we going to recycle if we don’t even have time to care for patients?” (RN 1,2 and NL 3).

All these comments indicated the barriers the nurses faced, but they also suggested possibilities for improvement. The pandemic, despite overloading nurses, also improved their awareness.

Areas subject to improvement emerged from nursing responsibilities, limitations and leadership.

Nurses indicated that despite their general levels of environmental awareness and the courses they had received, participants performed better regarding their recycling behaviors at home than at the hospital. Participants acknowledged performing better in recycling practices within their personal spaces, suggesting a potential gap in translating theoretical knowledge into practical action within the healthcare environment. Relevant statements included “It’s just that I recycle almost everything in my house, especially glass…, but here, there is no time…” (RN 1,4,5).

Moreover, time constraints emerged as a significant barrier impeding nurses’ ability to engage fully in environmental sustainability efforts. Participants cited the demanding nature of their work, particularly in the context of patient care responsibilities, as limiting their capacity to prioritize sustainability initiatives. This highlights the need for strategies to streamline environmental practices and integrate them seamlessly into nurses’ daily routines without adding undue burden.

Some statements also highlighted nurses’ willingness to improve paperwork and records. Nurses recognized the importance of incorporating environmental considerations into patient discharge reports and other documentation processes but sought further guidance on how to effectively implement these practices. Relevant statements included “Can you tell me how the patient’s continuity care report upon discharge was included in the recommendations for environmental sustainability… I want to do the report well with what you gave us in the clinical session the other day…” [(NL4)]

These comments indicated the opportunities for improvement in fostering a culture of environmental sustainability within the hospital setting. By addressing the identified challenges and providing targeted support and guidance, especially the lack of time, nurses can contribute to environmental stewardship efforts more effectively.

The current research highlights the relevance of nurses as promoters of environmentally sustainable behaviors in their roles as members of Green Teams and important leaders. The findings suggest that nurses exhibit acceptable knowledge, attitudes, and behaviors with regard to environmental sustainability both inside and outside the workplace. These results are complemented by a qualitative analysis indicating that such behaviors originate from nursing responsibility, Green Teams, leadership identification of barriers and areas of improvement. Both analyses highlight the fact that environmental nursing behavior in the workplace depends on sustainable behaviors outside the workplace. The qualitative analysis also identifies diverse barriers to the task of promoting sustainable behavior within the workplace, such as the COVID-19 pandemic and the need for more time to be allocated to this process. One key point identified by both analyses is that nurses have acceptable levels of knowledge; however, their attitudes, although as yet imperfect, are improving.

Several studies of nurses’ awareness of environmental sustainability have revealed that nurses exhibit moderate levels of awareness and a considerable degree of concern regarding the health impacts of climate change [ 37 , 42 , 43 ], as reflected in the NEAT-es results.

Interestingly, the participants exhibited a tendency to perform environmentally sustainable behaviors more consistently in their personal lives than in professional settings. These results are consistent with previous research on registered nurse and nursing students [ 36 , 41 , 42 ]. According to Swedish research, nurses generally recognize environmental issues but may lack awareness of the environmental impact of health care [ 43 ]. Polivka Barbara J. et al. (2012) highlighted the gap between nurses’ knowledge of sustainability and workplace behaviors, thereby emphasizing the need for education and training programs to promote sustainable practices [ 44 ]. These issues were also observed in a study conducted in Taiwan, which revealed that while nursing students exhibit positive attitudes toward sustainability, their knowledge and behaviors are inadequate [ 45 ].

By conducting qualitative analysis, this research also identified multiple barriers to the adoption of sustainable practices among nurses, including time constraints, disruptions caused by the COVID-19 pandemic, a lack of bins, and a lack of health care personnel. These findings are in line with those reported in other research, but certain barriers (in terms of resources, time, and support) to the implementation of sustainable practices in the workplace remain [ 29 ]. This study suggests that interventions should be designed to address these barriers and promote sustainable behavior among nurses, a suggestion which is consistent with the current research. These findings highlight the importance of comprehending nurses’ perspectives on environmental sustainability in health care contexts as well as the necessity for targeted interventions and support mechanisms [ 46 ]. The tasks assigned to nursing leaders and the Green Team involved addressing these barriers and promoting sustainable practices among nurses in the context of their professional roles. Environmental nursing leaders seem to be crucial with regard to establishing a more environmentally conscious health care environment, which is in line with recommendations to create a greener health care system [ 21 , 31 ]. Despite the results of the interviews, some global qualitative studies of nurses’ views on environmental issues have exhibited variations across countries [ 47 , 48 ]. In Sweden, nurses already exhibit pro-sustainability attitudes before the introduction of the 2030 SDGs [ 16 ]. However, the integration of environmental sustainability education into nursing programs can prepare future nurses more effectively to address the challenges associated with climate change and promote sustainable health outcomes [ 49 ].

Limitations

Although this investigation provides valuable insights, it is important to acknowledge its limitations. First, the study was conducted during the COVID-19 pandemic in Spain, which may have influenced the results due to the unique circumstances and stressors faced by health care workers during this period. Additionally, the assessment of nurses’ environmental awareness was performed on a larger scale, i.e., across multiple regions, and therefore may not accurately reflect individual attitudes and behaviors since the qualitative investigations focused on a specific region. However, this approach was adopted to minimize the risk of the ecological fallacy. Future studies could explore individual perspectives and experiences by reference to more diverse and representative samples.

Despite these limitations, this research is highly relevant because it sheds light on the role of nurses in the task of promoting environmental sustainability in health care settings. The research also emphasized the role of nursing leadership in the tasks of promoting environmental sustainability and providing nurses with the necessary resources and support to implement sustainable practices.

In conclusion, while nurses generally exhibit acceptable levels of knowledge, attitudes, and behaviors regarding environmental sustainability, a notable gap persists in terms of the frequency of sustainable actions within the professional settings in which they operate. This finding highlights the importance of closely aligning nurses’ personal and professional sustainability practices.

The qualitative analysis conducted as part of this study identified several barriers to the adoption of sustainable practices among nurses, including time constraints, disruptions resulting from the COVID-19 pandemic, issues with waste disposal, and challenges related to health care personnel. Despite the fact that these findings are in line with those reported in previous research, persistent barriers such as limited resources, time, and support hinder the implementation of sustainable practices in the workplace. Therefore, interventions aimed at addressing these barriers and promoting sustainable behavior among nurses are essential, as highlighted by both current research and the corresponding qualitative insights. Therefore, nursing leaders and Green Teams are pivotal with regard to overcoming these barriers and fostering sustainable practices within health care environments. Environmental nursing leaders in particular are instrumental to the cultivation of a more environmentally conscious health care system, thereby aligning with recommendations for greener health care practices.

Data availability

The datasets used and/or analyzed as part of the current study are available from the corresponding author upon reasonable request.

Romanello M, Di Napoli C, Drummond P, Green C, Kennard H, Lampard P, et al. The 2022 report of the Lancet countdown on health and climate change: health at the mercy of fossil fuels. Lancet. 2022;400(10363):1619–54. https://doi.org/10.1016/S0140-6736(22)01540-9

Article   PubMed   Google Scholar  

Watts N, Amann M, Ayeb-Karlsson S, Belesova K, Bouley T, Boykoff M, et al. The Lancet countdown on health and climate change: from 25 years of inaction to a global transformation for public health. Lancet. 2018;391(10120):581–630. https://doi.org/10.1016/S0140-6736(17)32464-9

Fadhullah W, Imran NIN, Ismail SNS, Jaafar MH, Abdullah H. Household solid waste management practices and perceptions among residents in the East Coast of Malaysia. BMC Public Health. 2022;22(1):1. https://doi.org/10.1186/s12889-021-12274-7

Article   PubMed   PubMed Central   Google Scholar  

Castañeda-Hidalgo H, Visovsky C, Hernández DE, González-Quirarte NH, Compeán-Ortiz L, Campiño SM. Nursing’s contributions to Sustainable Development Goals in Latin America through education, leadership, and partnerships. Int J Nurs Studi. 2021;121:104004. https://doi.org/10.1016/j.ijnurstu.2021.104004

Article   Google Scholar  

Dossey BM, Rosa WE, Beck DM. Nursing and the Sustainable Development Goals: from Nightingale to now. Am J Nurs. 2019;119(5):44–9. https://doi.org/10.1097/01.NAJ.0000557912.35398.8f

Rosa WE, Dossey BM, Koithan M, Kreitzer MJ, Manjrekar P, Meleis AI, et al. Nursing theory in the Quest for the Sustainable Development Goals. Nurs Sci Q. 2020;33(2):178–82. https://doi.org/10.1177/0894318420903495

United Nations Department of Economic and Social Affairs. Sustainable Development Knowledge Platform. In: Open Working Group proposal for Sustainable Development Goals. 2023. https://sustainabledevelopment.un.org/content/documents/1579SDGs Proposal.pdf.Accessed 17 Apr 2023.

Sanahuja JA. El Pacto Verde, NextGenerationEU y la nueva Europa geopolítica. In: Fundación Carolina.2022. https://www.fundacioncarolina.es/dt_fc_63/ . Accessed 17 Apr 2023.

Boto-Álvarez A, García-Fernández R. Implementation of the 2030 Agenda Sustainable Development Goals in Spain. Sustainability. 2020;12(6):2546. https://doi.org/10.3390/su12062546

Gomera Martínez A,Villamandos, de la Torre F, Vaquero Abellán. M,. Measurement and categorization of environmental awareness in university students: the contribution of the University to strengthen it. Ambientalización curricular y sostenibilidad. Nuevos retos de profesionalización docente. 2012;16(2):215– 28.

Ministerio de Sanidad. Plan Estratégico de Salud y Medioambiente 2022–2026. In: Ministerio para la Transición Ecológica y el Reto Demográfico.2021. https://www.sanidad.gob.es/ciudadanos/pesma/docs/241121_PESMA.pdf . Accessed 25 Apr 2023.

Ministerio de Sanidad. Plan Estratégico de Salud y Medioambiente. In: 1 a Programa de Actuación. Ministerio para la Transición Ecológica y el Reto Demográfico;2022. https://www.sanidad.gob.es/ciudadanos/pesma/docs/1er_PA_PESMA.pdf . Accessed 25 Apr 2023.

Hart J. Health Care without Harm: taking Environmental Action to improve lives and the planet. Integr Complement Ther. 2022;28(5):251–4. https://doi.org/10.1089/ict.2022.29036.pro

Health Care Without Harm. Hospitales que curan el planeta. In: Informe sobre el trabajo de los miembros de la Red Global de Hospitales Verdes y Saludables en América Latina. Red Global de Hospitales Verdes y Saludables en America Latina. 2022. https://saludsindanio.org/sites/default/files/documents-files/7287/Hospitales que curan el planeta 2022-FINAL_web_0 %281%29_0.pdf . Accessed 25 Apr 2023.

McDermott-Levy R. The nurse’s role on green teams: an environmental health opportunity. Pa Nurse. 2011;66(1):17–21.

PubMed   Google Scholar  

Schenk E, Johnson S. Nurse-sensitive environmental indicators: a qualitative study. J Nurs Manag. 2022;30(8):4378–86. https://doi.org/10.1111/jonm.13861 . Epub 2022 Oct 20.

Eckelman MJ, Sherman JD, MacNeill AJ. Life cycle environmental emissions and health damages from the Canadian healthcare system: an economic-environmental-epidemiological analysis. PLoS Med. 2018;15(7):e1002623. https://doi.org/10.1371/journal.pmed.1002623

Article   CAS   PubMed   PubMed Central   Google Scholar  

Mateen A, Nisar QA, Nasir N. Fostering pro-environmental behaviors in the healthcare organizations: an empirical analysis of psychological and strategic factors. Asia Pac Manage Rev. 2023;28(1):13–23. https://doi.org/10.1016/j.apmrv.2022.01.004

Michie S, West R. Sustained behavior change is key to preventing and tackling future pandemics. Nat Med. 2021;27(5):749–52. https://doi.org/10.1038/s41591-021-01345-2

Article   CAS   PubMed   Google Scholar  

Mejia EA, Sattler B. Starting a Health Care System Green Team. AORN J. 2009;90(1):33–40. https://doi.org/10.1016/j.aorn.2009.03.001

Practice Green Health. Comparison chart of hospital green teams and their structure. 2019. https://practicegreenhealth.org/sites/default/files/2019-02/Comparison chart of hospitals green team structure_2017.pdf . Accessed 1st Frebuary 2024.

Dickman E, Backler C, Berg CD, Komandt M, Schiller J. Climate change and oncology nursing: a call to action. Clin J Oncol Nurs. 2022;26(1):109–13. https://doi.org/10.1188/22.CJON.109-113

de la Gámez J, Padilla Fortes A. Análisis De La producción científica visible en internet de Los técnicos de salud ambiental del Servicio Andaluz De Salud. Rev GeI Inf Doc. 2010;20:409–25.

Google Scholar  

Filho W. Non-conventional learning on sustainable development: achieving the SDGs. Environ Sci Eur. 2021. https://doi.org/10.1186/s12302-021-00525-8 . 33;97.

Filho W, Tripathi S, Andrade Guerra JB, Gine R, Orlovic Lovren V, Willats J. Using the sustainable development goals towards a better understanding of sustainability challenges. Int J Sustainable Dev World Ecol. 2018;26:1–12. https://doi.org/10.1080/13504509.2018.1505674

Portela Dos Santos O, Melly P, Joost S, Verloo H. Climate Change, Environmental Health, and challenges for nursing Discipline. Int J Environ Res Public Health. 2023;20(9):5682. https://doi.org/10.3390/ijerph20095682

Sattler B, Randall KS, Choiniere D. Reducing Hazardous Chemical exposures in the neonatal intensive care unit: a New Role for nurses. Crit Care Nurs Q. 2012;35(1):102–11. https://doi.org/10.1097/CNQ.0b013e31823b2084

Trent L, Law J, Grimaldi D. Create intensive care green teams, there is no time to waste. Intensive Care Med. 2023;49(4):440–3. https://doi.org/10.1007/s00134-023-07015-w

Kallio H, Pietilä AM, Johnson M, Kangasniemi M. Environmental responsibility in hospital care: findings from a qualitative study. J Hosp Adm. 2018;7(5):56. https://doi.org/10.5430/jha.v7n5p56

Rosa WE, Catton H, Davidson PM, Hannaway CJ, Iro E, Klopper HC, et al. Nurses and midwives as Global Partners to achieve the Sustainable Development Goals in the Anthropocene. J Nurs Scholarsh. 2021;53(5):552–60. https://doi.org/10.1111/jnu.12672

Harris N, Pisa L, Talioaga S, Vezeau T. Hospitals going green: a holistic view of the issue and the critical role of the nurse leader. Holist Nurs Pract. 2009;23(2):101–11. https://doi.org/10.1097/HNP.0b013e3181a110fe

Phiri MM, MacPherson EE, Panulo M, Chidziwisano K, Kalua K, Chawanangwa M, Chirambo, et al. Preparedness for and impact of COVID-19 on primary health care delivery in urban and rural Malawi: a mixed methods study. BMJ Open. 2022;12(6):e051125. https://doi.org/10.1136/bmjopen-2021-051125

Ojemeni MT, Karanja V, Cadet G, Charles A, Dushimimana E, McMahon C, et al. Fostering nursing leadership: an important key to achieving sustainable development goals and universal health care. Int J Nurs Stud. 2019;100:103421. https://doi.org/10.1016/j.ijnurstu.2019.103421

German E. The environmental or ecological model of care. Nriagu JO. Encyclopedia of Environmental Health. Netherlands:Elsevier Science; 2019. pp. 447–50.

Vasset F, Fagerstrøm L, Frilund M. Sustainable Nursing Leadership in Nordic Health Care Organizations. Scand J Caring Sci. 2021;14(2):2–1527.

Kallio H, Pietilä AM, Kangasniemi M. Environmental responsibility in nursing in hospitals: a modified Delphi study of nurses’ views. J Clin Nurs. 2020;29(21–22):4045–56. https://doi.org/10.1111/jocn.15429

Richardson J, Grose J, Bradbury M, Kelsey J. Developing awareness of sustainability in nursing and midwifery using a scenario-based approach: evidence from a pre and post educational intervention study. Nurse Educ Today. 2017;54:51–5. https://doi.org/10.1016/j.nedt.2017.04.022

Luque-Alcaraz OM, Gomera A, Ruíz Á, Aparicio-Martinez P, Vaquero-Abellan M. Validation of the Spanish Version of the questionnaire on environmental awareness in nursing (NEAT). Healthcare. 2022;10(8):1420. https://doi.org/10.3390/healthcare10081420

Amezcua M. El Trabajo De Campo Etnográfico En Salud. Index Enferm (Gran). 2000;30:30–5.

Jamali HR. Does research using qualitative methods (grounded theory, ethnography, and phenomenology) have more impact? Libr Inform Sci Res. 2018;40(3):201–7. https://doi.org/10.1016/j.lisr.2018.09.002

Cowie J, Nicoll A, Dimova ED, Campbell P, Duncan EA. The barriers and facilitators influencing the sustainability of hospital-based interventions: a systematic review. BMC Health Serv Res. 2020;20(1):588. https://doi.org/10.1186/s12913-020-05434-9

Kircher M, Doheny BM, Raab K, Onello E, Gingerich S, Potter T. Understanding the knowledge, attitudes, and practices of Healthcare professionals toward Climate Change and Health in Minnesota. Challenges. 2022;13(2):57. https://doi.org/10.3390/challe13020057

Boström M, Andersson E, Berg M, Gustafsson K, Gustavsson E, Hysing E, et al. Conditions for transformative learning for Sustainable Development: A Theoretical Review and Approach. Sustainability. 2018;10(12):4479. https://doi.org/10.3390/su10124479

Polivka BJ, Chaudry RV, Crawford JM. Public Health nurses’ knowledge and attitudes regarding Climate Change. Environ Health Perspecti. 2012;120(3):321–5. https://doi.org/10.1289/ehp.1104025

Hsieh PL, Chen SH, Chang LC. School nurses’ perceptions, knowledge, and related factors Associated with evidence-based practice in Taiwan. Int J Environ Res Public Health. 2018;15(9):1845. https://doi.org/10.3390/ijerph15091845

Rojas-Perez HL, Díaz-Vásquez MA, Díaz-Manchay RJ, Zeña-Ñañez S, Failoc-Rojas VE, Smith D. Nurses’ environmental practices in Northern Peruvian hospitals. Workplace Health Saf. 2024;72(2):68–74. https://doi.org/10.1177/21650799231163130

Anåker A, Nilsson M, Holmner Å, Elf M. Nurses’ perceptions of climate and environmental issues: a qualitative study. J Adv Nurs. 2015;71(8):1883–91. https://doi.org/10.1111/jan.12655

Kallio H, Pietilä AM, Johnson M, Kangasniemi M. Systematic methodological review: developing a framework for a qualitative semi-structured interview guide. J Adv Nurs. 2016;72(12):2954–65. https://doi.org/10.1111/jan.13031

International Council of Nurses. International Nurses Day 2023 report. Value, protect, respect and invest in our nurses for a sustainable future for nursing and health care.2023. https://www.icn.ch/news/icn-marks-international-nurses-day-launch-its-our-nurses-our-future-campaign . Accessed 25 Apr 2023.

Download references

Acknowledgements

The authors would like to thank the Excellent Official Nursing School and all the professionals who participated in this research for their support.

This research received no external funding; however, the project did receive an award from the Excellent Official Nursing School in Cordoba, Spain, in 2020.

Author information

Authors and affiliations.

Neurosurgery Department, Hospital Universitario Reina Sofía, Andalusian Health Care System, 14071, Cordoba, Spain

Olga María Luque-Alcaraz

Environmental Protection Office (SEPA), University of Córdoba, Campus Rabanales, 14014, Córdoba, Spain

Olga María Luque-Alcaraz, Antonio Gomera & Manuel Vaquero-Abellán

GE 10 Research Groups of Clinical-Epidemiological Research in Primary Care, University Biomedical Program for Occupational Medicine, Occupational Epidemiology and Sustainability, Maimonides Institute of Biomedicine of Cordoba (IMIBIC), 14071, Cordoba, Spain

Olga María Luque-Alcaraz, Pilar Aparicio-Martínez & Manuel Vaquero-Abellán

Nursing, Pharmacology and Physiotherapy Department, Faculty of Medicine and Nursing, University of Cordoba, 14071, Cordoba, Spain

Pilar Aparicio-Martínez & Manuel Vaquero-Abellán

GA16 Lifestyles, Innovation and Health, Maimonides Institute of Biomedicine of Cordoba (IMIBIC), 14071, Cordoba, Spain

You can also search for this author in PubMed   Google Scholar

Contributions

A.G. and M. V-A. conceived and designed the study, and O.M. L. and P.A-M. acquired the data, analyzed and interpreted the data, and drafted the article. The publication and supervision of the article were the responsibility of A.G. and M. V-A. All authors contributed equally to the writing and preparation of the final manuscript.

Corresponding author

Correspondence to Pilar Aparicio-Martínez .

Ethics declarations

Ethical approval and consent to participate.

Ethical approval was obtained from the Ethics Committee of Reina Sofia Hospital of Cordoba, which is part of the Andalusian Health Care System in Spain (Act No. 267, ref.3605). This research was in line with the Organic Law 3/2018 of December 5 on the Protection of Personal Data and Guarantee of Digital Rights as well as the Nursing Ethics Code and the 1964 Declaration of Helsinki. The participants were informed of the study’s purpose before participation; their informed consent was obtained, and they were informed that they were able to withdraw from the study at any stage. All the data were obtained after informed consent was collected; in addition, the data were anonymized and saved securely in a database, thereby maintaining all stipulations of the Personal Data Law.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Luque-Alcaraz, O.M., Aparicio-Martínez, P., Gomera, A. et al. The environmental awareness of nurses as environmentally sustainable health care leaders: a mixed method analysis. BMC Nurs 23 , 229 (2024). https://doi.org/10.1186/s12912-024-01895-z

Download citation

Received : 26 June 2023

Accepted : 26 March 2024

Published : 03 April 2024

DOI : https://doi.org/10.1186/s12912-024-01895-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Environmental health
• Attitude of health personnel
• Sustainable development
• Health Knowledge, attitudes, and practices
• Organizational Culture

BMC Nursing

ISSN: 1472-6955

• General enquiries: [email protected]

• Open access
• Published: 06 April 2024

Delayed discharge in inpatient psychiatric care: a systematic review

• Ashley-Louise Teale   ORCID: orcid.org/0000-0002-1756-7711 1 ,
• Ceri Morgan   ORCID: orcid.org/0000-0002-2417-8677 1 ,
• Tom A. Jenkins   ORCID: orcid.org/0000-0001-7875-4417 1 &
• Pamela Jacobsen   ORCID: orcid.org/0000-0001-8847-7775 1  

International Journal of Mental Health Systems volume  18 , Article number:  14 ( 2024 ) Cite this article

240 Accesses

8 Altmetric

Metrics details

Delayed discharge is problematic. It is financially costly and can create barriers to delivering best patient care, by preventing return to usual functioning and delaying admissions of others in need. This systematic review aimed to collate existing evidence on delayed discharge in psychiatric inpatient settings and to develop understanding of factors and outcomes of delays in these services.

A search of relevant literature published between 2002 and 2022 was conducted on Pubmed, PsycInfo and Embase. Studies of any design, which published data on delayed discharge from psychiatric inpatient care in high income countries were included. Studies examining child and adolescent, general medical or forensic settings were excluded. A narrative synthesis method was utilised. Quality of research was appraised using the Mixed Methods Appraisal Tool (MMAT).

Eighteen studies from England, Canada, Australia, Ireland, and Norway met the inclusion criteria. Six main reasons for delayed discharge were identified: (1) accommodation needs, (2) challenges securing community or rehabilitation support, (3) funding difficulties, (4) family/carer factors, (5) forensic considerations and (6) person being out of area. Some demographic and clinical factors were also found to relate to delays, such as having a diagnosis of schizophrenia or other psychotic disorder, cognitive impairment, and increased service input prior to admission. Being unemployed and socially isolated were also linked to delays. Only one study commented on consequences of delays for patients, finding they experienced feelings of lack of choice and control. Four studies examined consequences on services, identifying high financial costs.

Overall, the findings suggest there are multiple interlinked factors relevant in delayed discharge that should be considered in practice and policy. Suggestions for future research are discussed, including investigating delayed discharge in other high-income countries, examining delayed discharge from child and forensic psychiatric settings, and exploring consequences of delays on patients and staff. We suggest that future research be consistent in terms used to define delayed discharge, to enhance the clarity of the evidence base.

Review registration number on PROSPERO

Date of registration.

9th December 2021.

Delayed discharge, also termed ‘bed blocking’ and ‘delayed transfer of care,’ refers to when patients remain in hospital beyond the time they are determined to be clinically fit to leave [ 1 , 2 ]. It is an international challenge, costly to individuals, health services and governments [ 3 , 4 ], impacting physical health settings, and also psychiatric inpatient services [ 5 ].

Psychiatric inpatient stays are one of the most expensive forms of treatment for mental health conditions, particularly when compared to care delivered in community settings [ 6 ]. Prolonged stays in mental health hospitals likely increase resource use and as such financial expenditure. This is particularly concerning in instances of delayed discharge when stays are determined to not be of clinical benefit. Delayed discharge also could prevent admission of new patients, contributing to bed crises, where there are not enough beds for all who require admission [ 7 ]. This can have consequences on the course of recovery for newly referred patients, either delaying admission, contributing to inappropriate placements, or leading to individuals being placed out of area [ 7 , 8 ]. Extended hospital stay could also detrimentally impact the delayed patient themselves, preventing their return to usual day-to-day functioning and make returning to the community increasingly difficult [ 9 , 10 ].

Existing reviews have examined predictors of longer stays in psychiatric inpatient settings, finding substance use and being employed are associated with shorter length of stay; while being female, having a diagnoses of mood or psychotic disorders and use of Electroconvulsive Therapy are associated with longer stay [ 11 ]. However, there is not to our knowledge a systematic review collating evidence examining delayed discharge in psychiatric settings. As delayed discharge is a unique experience, distinct from long stay driven by clinical need, it requires separate focus to further understand this specific experience.

Furthermore, a large body of evidence has examined delayed discharge in physical health settings with several systematic reviews, examining causes and outcomes. Such reviews have found that delayed discharges were linked to problems in discharge planning, transfer of care difficulties and patient age [ 12 , 13 ]. Outcomes for services included overcrowding and financial costs, whereas outcomes for patients included infections, depression, reduction in activities and mortality. There may be both overlapping and non-overlapping factors associated with delayed discharge between physical and psychiatric inpatient settings. For example, inpatient psychiatric services may differ in organisational structure, daily workings, and treatment focus from general medical services. The clinical population might also differ in psychiatric and physical health settings, for example in age, socio-economic status, and other demographic, plus clinical factors. As such, it is vital that separate attention be given to the area of psychiatric care.

This systematic review aims to fill the current research gap and synthesise existing literature on psychiatric delayed discharges. We aimed to synthesise the available international data from high-income countries, as the prevalence and underlying reasons for delayed discharge are likely to be highly sensitive to context and heterogeneous across countries. This is due to factors such as different models of healthcare funding, and the varying social role of the family in providing care, for example. Developing in-depth understanding of the causes and consequences of delays in a psychiatric inpatient context is important in informing practice and policies at a service, organisational, societal, and government level. This could help develop ways to reduce occurrence of delays and mitigate any negative impacts.

The aim of this review was to increase understanding of what is known about factors influencing delayed discharge in adult psychiatric inpatient settings. Secondary aims were to examine outcomes of delayed discharge for patients and compare findings across different psychiatric settings and age groups.

The systematic review protocol was pre-registered on PROSPERO before the review was started and the searches were run (PROSPERO: 292515). The review is reported in line with PRISMA guidelines [ 14 , 15 ]. The primary research question of this review is: What is known about factors associated with delayed discharge from inpatient psychiatric care settings?

Secondary research questions were:

What are the outcomes for those who have experienced delayed discharge from inpatient psychiatric settings, for example, in mental health outcomes, health outcomes, readmissions and quality of life?

What are the outcomes on services in terms of resources and costs from delayed psychiatric inpatient discharge?

What are the experiences of staff and patients of delayed discharge from inpatient mental health wards?

Are there differences between types of inpatient services, including acute, rehabilitation or specialist inpatient wards, in factors and costs, are there differences between working age adults and older adults, in experiences of delayed discharge, search strategy.

Initial searches were conducted on the 15th of January 2022, and updated on the 5th of August 2022. Pubmed, PsycInfo and Embase were searched.

Search terms (Appendix B in supplementary materials) were developed through examining key words of published studies on the topic, reviewing the terms used in comparative reviews based in physical health settings and thesaurus mapping. Terms included: “delayed discharge,” “bed blocking” and “long stays.” Search terms were piloted on each database prior to running the final search.

The search included studies published from 2002. A 20-year search timeframe was selected, as psychiatric inpatient care has adapted in response to changing need and updated knowledge over time. As such, studies published before 2002 are likely to be less relevant to current practice.

Following database searches, reference lists of included papers were examined, to identify any relevant studies missed in the search. A forward citation search was also conducted, to identify any relevant studies that were cited in the included papers.

Inclusion and exclusion criteria

Studies were included if they reported data related to delayed discharge or associated outcomes, in adult psychiatric inpatient wards. Specialist and rehabilitation psychiatric inpatient settings were included. Studies of any design were included, providing they were published in a peer-reviewed journal. Both quantitative and qualitative studies were included.

Studies exploring delayed discharge in child or adolescent units and/or forensic units were excluded. This was because the causes and outcomes of delays in such settings are likely unique, given the specialist context. For example, there is likely different systemic involvement from families and different governing legislation in these contexts. As such, it was determined that such settings were too disparate, and synthesising studies from these settings together with adult psychiatric settings could lead to inaccurate conclusions. Physical health settings were also excluded, given the different processes, procedures and treatment focus involved in such settings. In addition, reviews have already been conducted examining delayed discharge from such settings. Studies not conducted in high-income countries were also excluded. In this review, we included high-income countries as defined by World Bank criteria, accessed in January 2022 [ 16 ] (see Appendix C in supplementary material for the list of included countries). Globally, countries differ in the conceptualisation of mental health and provisions offered, therefore, limiting this review to only high-income countries would enable comparisons to be made.

Study selection and data extraction

Screening was conducted using Covidence Systematic Review Software [ 17 ]. All records were independently double-screened by two reviewers at both title/abstract and full-text stage. Conflicts were resolved by discussion to reach consensus, with referral to the senior author (PJ) when needed.

A standardised template was used for data extraction, with all included studies being independently double extracted by two reviewers, with consensus achieved by discussion where needed.

A narrative synthesis method was used. For data examining reasons for delayed discharge, a deductive approach was taken initially. Authors identified possible reasons for delays based on existing literature and organised data under these categories/themes. Any data that did not fit into the pre-defined categories was pooled as ‘other’. All categories were then reviewed, with particular attention placed on the ‘other’ categories, to determine if additional categories need to be added or existing categories adapted. Sub-categories were identified when appropriate through coding. Once categories were established, the number of papers which reported each reason/factor were tabulated and data was reviewed to examine relationships, exploring both links and disparities within and between studies. The final synthesis was checked by three authors (AT, TJ, and CM), to achieve final agreement.

Data relating to outcomes/consequences of delayed discharge was synthesised in a similar way, with data initially organised into three categories: (1) consequences for patients, (2) consequences for service, (3) consequences for staff. Categories were reviewed by the authors following synthesis. Financial costs were converted to US dollars by the authors to support comparison.

Quality assessment of the included studies formed part of the synthesis with the appraisal of quality considered in the interpretation of results.

Quality Assessment

Quality assessment of studies was completed during the synthesis stage. In the protocol, we initially outlined that the Quality Assessment Tool for Studies of Diverse Designs (QATSDD) would be utilised [ 18 ]. However, following a trial of this tool with the included papers, we noted disparities in interpretations between authors. Therefore, the Mixed Methods Appraisal Tool (MMAT) was established to be a more suitable appraisal of quality for the included studies. The MMAT was developed for assessing and comparing the quality of studies using quantitative, qualitative and mixed-methods design, in one tool [ 19 ]. This tool was selected as studies of different designs were included in the review and this tool allows for quality appraisal across five different study types, distinguishing between methodology.

Two initial screening questions were answered to determine appropriateness of using the MMAT to assess quality of the study (are there clear research questions and do the collected data address the research questions). If screening questions are not passed, this tool is deemed inappropriate. Providing the screening questions were passed, quality was assessed on five questions within one of five categories. The category in which questions were answered was determined by study design. The MMAT discourages from scoring and assigning qualitative labels to describe quality, instead advises a more detailed evaluation of quality [ 19 ]. This approach has therefore been taken in this paper.

To achieve reliable and accurate quality ratings, every study was quality rated by two members of the research team and conflicts were discussed to reach consensus.

Identification of studies

Figure  1 (PRISMA flowchart) shows the study selection process. After removing duplicates, a total of 4891 papers were identified for screening. 4397 papers were excluded at title and abstract stage. Full texts were then obtained for 492 papers. Two full texts could not be obtained via the library service and the authors did not respond to a request for the paper. There were four papers obtained that were erratum’s, all of which related to excluded studies that were not examining delayed discharge and as such, were not linked to the included studies. Following full text screening 18 papers were eligible for inclusion. Each paper represented a different study.

Preferred reporting items for systematic reviews and meta-analyses (PRISMA) flowchart

Study characteristics

Table  1 shows the characteristics of the 18 included studies. Twelve of these studies examined delayed discharge as a primary outcome, with three of these studies specifically examining Housing Related Delayed Discharge (HRDD). HRDD is defined as instances where delayed discharge is attributed to housing issues. The remaining studies ( n  = 6) reported delayed discharge as secondary outcomes. Fifteen studies were of quantitative observational design, two studies used mixed methodologies and one was qualitative.

In the included studies, there was a range of psychiatric inpatient settings: psychiatric/general mental health units ( n  = 11), Psychiatric Intensive Care Units (PICUs) ( n  = 2), older adult psychiatric units ( n  = 3) and Mental Health Trusts ( n  = 1). One study looked across three inpatient settings: acute psychiatric, PICU and older adult. Studies were conducted in five high income countries (England = 10, Ireland = 1, Australia = 3, Canada = 3, and Norway = 1). There were no studies from any other high-income countries, identified in the search.

The MMAT quality scores are shown (Table  2 ). One included study [ 20 ] did not meet initial criteria to be assessed using this tool, as the research questions were unclear.

All studies were of fairly good quality, with all studies meeting at least three out of five of the quality assessment criteria. Quality was highest in Australian and Canadian studies, with included papers in these countries meeting all five quality assessment criteria [ 21 , 22 , 23 , 24 , 25 , 26 ]. Quality assessment ratings indicate that three quantitative descriptive studies included, did not clearly report use of a representative sample or appropriate measures. Ratings per question are shown in Table two.

Research Q1

What is known about factors associated with delayed discharge.

Thirteen studies identified reasons for delayed discharge (Table  1 ). The results showed that there are many complex reasons for delays with often overlapping contributing factors. We categorised reasons for delay into six categories: (1) accommodation needs, (2) difficulty securing rehabilitation or community support, (3) finance/funding challenges, (4) family/carer factors, (5) forensic factors, (6) patient being out of area.

The most common reason for delays was due to accommodation and placement factors. This was identified as a contributing reason for delay in twelve studies and a further two studies assessed Housing-Related Delayed Discharge (HRDD), suggesting accommodation factors contributing to delay in these cases. Accommodation/placement factors included limited availability of placements ( n  = 7), difficulty finding appropriate placements ( n  = 5), awaiting or undergoing placement assessment ( n  = 3), challenges in person returning to accommodation ( n  = 3), e.g., awaiting repairs or adaptations to their home, individuals being rejected from placement ( n  = 2), patients/family rejecting placement ( n  = 2) and awaiting transfer ( n  = 1). It should be noted that one of the studies which examined specific accommodation factors was unable to be quality assessed due to not having clear research questions and therefore did not meet the screening criteria for assessment with the MMAT [ 20 ], and two studies only met three of the five quality assessment criteria, with queries regarding the quality of measures used and analysis technique for one study [ 27 ], and some difficulties integrating and meeting the full quality criteria for the mixed methods approaches used in the second [ 28 ]. The second reason identified for delays was difficulty sourcing support for the person to enable discharge, such as community, rehabilitation, and homecare support. This contributed to delays in twelve studies. Eight of these studies met four to five of the quality assessment criteria, one was not able to be assessed [ 20 ], and three only met three of the five quality assessment criteria [ 27 , 28 , 29 ]. A third reason for delay was finance/funding challenges identified in nine studies. These included challenges obtaining funding, patients/families’ refusal to pay for placements and funding applications being rejected. Six studies identified family/carers factors in creating delays, such as family conflict, family not wanting the person to live with them and ongoing family discussion. The quality of two of the studies identifying family and finance factors should be considered, as one of these studies was unable to be quality assessed due to a lack of clear research questions [ 20 ] and a second met only three of the five quality assessment criteria [ 28 ]. The fifth reason identified in this review as contributing to delay was forensic factors, which accounted for delays in three studies, all of good methodological quality. Forensic delays incorporated delay by Ministry of Justice and awaiting forensic assessment. Person being out of area was highlighted as a reason for delay in only one study and it was not possible to quality assess this study due to no specific research questions identified [ 20 ], suggesting limited exploration or evidence for out of areas contributing to delays.

Fourteen studies included in this review examined the demographic and clinical factors relevant in delays, with eight conducting significance testing to establish associations. Significant associations with delay were having a diagnosis of schizophrenia or other psychotic disorder ( n  = 4), cognitive impairment ( n  = 3) and type/amount of service input prior to admission ( n  = 3). All studies reporting these significant results were of a good methodological quality, achieving at least four of the five MMAT quality criteria. Results were mainly consistent across those studies which examined significance, however, there was one study of good quality that did not find significant association with schizophrenia diagnosis [ 22 ]. The impact of physical health differed between Australia and England, where in one English study having fair-excellent health was more associated with delays [ 30 ], though two Australian studies found poorer physical health linked to delays [ 24 , 25 ]. Findings related to demographic characteristics, including gender, age, ethnicity, socio-economic status, were inconsistent across studies. The only consistent finding was that a smaller proportion of the delayed group were employed ( n  = 3). One of these studies found significant association between being unemployed and delayed discharge. The two other studies found only one member of the delayed group was employed, less than non-delayed groups, though this was not significance tested. There was some indication that being not being married and lacking a support network, was higher in delayed groups. One study found significant relationships to being unmarried and another finding that the delayed group were visited significantly less often by relatives. The other studies did not conduct significance testing. However, there was no significant relationship related to marriage between delayed and non-delayed groups in two studies [ 22 , 31 ]. One of these studies only clearly met three of the quality assessment criteria [ 31 ], though the other met all five quality assessment criteria. Being male was significantly associated with delays in two Canadian studies [ 21 , 22 ]. No significant association with gender was found in other studies.

The supplementary materials provide additional analysis of results for research question one, further describing each study’s findings. Additional materials also include tables showing tabulation of which study examined each variable.

Research Q2

What are the outcomes for those who have experienced delayed discharge from inpatient psychiatric settings for example, in mental health outcomes, health outcomes, readmissions and quality of life.

Only one study examined individual outcomes of delayed discharge for patients [ 26 ]. As such, there is limited data to draw conclusions to answer this research question. The study that evaluated patient outcomes was of qualitative design and good quality. The study explored Housing-Related Delayed Discharge (HRDD) in Australia for 10 patients using semi-structured interviews. They found consequences of lack of choice and control for patients, which impacted mental wellbeing, physical health and created a sense of anticipation for transition to community. Some participants highlighted a positive outcome of delayed discharge in preventing homelessness.

Research Q3

What is the outcome on services in terms of resources and costs from delayed psychiatric inpatient discharge.

Four studies assessed financial costs of delayed discharge for services, providing limited evidence in terms of financial outcomes. Each study focused on a different country. At an old age psychiatry unit in England, delayed discharges were estimated to cost over $855,820 for the year [ 20 ]. Notably, this study was not quality assessed due to the omission of research questions. In a high-quality paper from Australia, HRDD cost the health district $2,828,174 over one year [ 25 ]. While both papers present yearly costs, there is disparity in area covered, contributing to difficulty making comparisons regarding financial expenditure. Two studies calculated financial expenditure and did not present the cost per year. In a Canadian study, using the median number of delayed days ( M  = 17), it was calculated that the average cost incurred by one episode of delayed days was approximately $5,746 [ 21 ]. Furthermore, in Norway, $491,406 was allocated to delays on the acute ward included in the study, though methodological quality might be queried, due to lack of clarity on whether the sample was representative and the appropriateness of measures utilised [ 29 ]. The information necessary to calculate costs per year or costs per delayed day, to enable comparisons to be made across studies, has not included in the studies.

Aside from financial costs, no other type of outcome for services were assessed.

Research Q4

None of the included studies explored specific experiences of delayed discharge for staff. Some information on experiences for patients is detailed in question two.

Research Q5

This systematic review identified studies in acute psychiatric, older adult and Psychiatric Intensive Care Unit (PICU) settings. Only one study included Learning Disability inpatient care settings [ 28 ]. This study was of mixed-method design and met three quality assessment criteria. No studies reported data from rehabilitation units. There were few differences identified between types of setting. Prevalence of delayed discharge was highest in older adult settings (56.9%) [ 30 ] and PICU settings (51.1%) [ 32 ], compared to working age adult settings (18–32%) [ 31 , 33 ]. However, the highest proportion of delayed days was found in acute psychiatric settings in Norway acute psychiatric units (54.8%) [ 29 ]. More information on prevalence is provided in supplementary materials.

Reasons for delay did not vary much across type of setting. There is a potential service difference in the impact of physical health in delays, as having fair-excellent health was more associated with delays in an English older adult study [ 30 ], while in a working age adult sample in Australian studies [ 24 , 25 ], having poor health was more associated with delays. However, this could represent a disparity in country. There were some other differences across countries found. Forensic reasons for delay were only found in the UK ( n  = 2), as was due to patient being out of area ( n  = 1). In UK settings, there was no significant difference found in gender between those delayed and those not [ 30 , 34 ], though there was in Canada [ 21 ]. England and Australia were the only countries identifying funding issues as contributing to delay. Each country will have its own respective funding system, which could impact delays. For example, two Australian studies identified difficulties with their own National Disability Insurance Scheme [ 24 , 25 ].

Research Q6

Only five of the included studies looked specifically at older adult settings, all of which were in the UK. A further five studies, from the UK and Canada, included older adults within their sample, despite not examining a specific older adult setting.

The highest proportion of inpatients experiencing delayed discharge were from older adult settings, with one study identifying 56.9% [ 30 ] of inpatients experiencing delays. There were lower rates of delayed patients in working age adult psychiatric inpatient settings in comparison, with 3.5% [ 21 , 25 ] to 39.1% [ 29 ] of patients experiencing delay. Similarly, two studies in Canada identified that a higher proportion of older adults made up the delayed group compared to the non-delayed group, suggesting that older adult inpatients are more likely to experience delay [ 21 , 22 ]. However, two English studies found delayed discharge was not associated with age [ 31 , 35 ]. One of these studies met only three quality assessment criteria, with lack of clarity regarding the quality of sampling and representativeness of the sample [ 31 ].

In terms of reasons for delay, no clear differences were found across age groups. Although when limiting comparisons to studies conducted in the UK, family/carer factors was identified as a reason for delay more frequently in older adult samples ( n  = 3) compared to studies looking at working age adults ( n  = 1). To support this finding, one study in England found that eight older adult trusts identified patient/carer exercising choice as a reason for delay, whereas the same was true for only four working age adult trusts [ 28 ]. However, this finding cannot be generalised across all countries. There is also some indication that cognitive impairment/dementia might increase likelihood of delay in older adult samples, as two studies identified the role of dementia and greater cognitive impairment in the delayed older adult groups [ 20 , 30 ]. A further two studies examined the impact of cognitive impairment, finding association with delay [ 21 , 22 ]. However, these studies included working age samples, so it is unclear who in the sample this impacted. In addition, physical health status could cause delays differently in older adult populations. In an older adult UK sample having fair-excellent health was more associated with delays [ 30 ], whereas two Australian studies in working age adult inpatient settings found poorer physical health increased delays [ 24 , 25 ]. This difference could however be attributed to country or setting. Funding was identified as a reason for delay in all studies in older adult settings ( n  = 5), but the same was not true for the other setting types. Forensic factors were not found to be a reason for delay in any of the studies with older adult inpatients, conversely patient being out of area was only identified as a reason for delay in an older adult sample [ 20 ].

This systematic review aimed to fill a research gap and examine factors contributing to delayed discharge in adult psychiatric inpatient settings and explore associated consequences. This adds a unique contribution to the evidence base, which predominantly has focused on delayed discharge from physical health settings. Eighteen studies were included for synthesis.

The findings suggest that there are varying inter-related reasons for delay, including accommodation or placement needs, difficulties securing the required support services, funding and finance challenges, family/carer factors, forensic factors and the person being out of area. There were mixed findings regarding demographic and clinical characteristics associated with delays. However, this review showed that delays could be associated with the person having diagnosis of schizophrenia or other psychotic disorder, cognitive impairment, being unemployed and receiving increased service input prior to admission.

There were only a few studies that commented on outcomes of delays. Only one study examined outcomes for patients, identifying feelings of lack of choice and control, while four studies looked at financial outcomes for services, finding large costs associated with delays. This points to a lack of evidence examining the outcomes and experiences of psychiatric delayed discharge, and therefore requires further attention in research.

This review adds to and expands on existing findings, identifying similarities and differences between longer stay generally. For example, one review [ 11 ] found that long stay was associated with mood and psychotic disorders, use of Electroconvulsive Therapy, and being female. Being married, employed, and using substances were associated with a shorter stay [ 11 ]. Our review found that psychiatric delayed discharge was also associated with diagnosis of schizophrenia or other psychotic disorder and being unemployed. However, we found delayed discharge to be associated with cognitive impairment and increased service input prior to admission, but not gender or treatment. This could suggest some important differences in those at risk of delays or those requiring longer inpatient treatment. It is important to note however, the review by Gopalakrishina and colleagues did not distinguish between those patients with long stay clinically warranted and delayed discharge patients [ 11 ]. It would be of benefit for future research on long stay patients to better define their sample based on those who clinically needed treatment or longer stay patients in the context of delayed discharge, allowing similarities and differences to be better explored. This will support policy makers and service managers to better identify those at risk of delays that are not clinically necessary, and those who might need additional clinical input. The findings in this review provide some suggestion that there could be benefit in considering a person’s social context when they are admitted to psychiatric inpatient care, including their living situation at admission, employment status and cognitive functioning. Identifying patients at higher risk of delays earlier in admission might be useful, to ensure more time be given to organise and find appropriate accommodations, placements and service support and facilitate discharge. Wider policy and structural changes are needed, such as improving the availability of appropriate accommodation placements.

It is important to highlight that there were discrepancies across studies in language used to term delayed discharge, e.g., ‘alternate level of care,’ ‘waiting days’ and ‘prolonged stay.’ Due to such discrepancies in definitions and terminology, during the screening process it was at times difficult to determine if studies were focused on delayed discharge or longer lengths of stay clinically required. In this review, studies were excluded if the focus was unclear to prevent incorrect conclusions being drawn related to the unique experience of delayed discharge. However, this means other relevant findings might have been missed. It would therefore be useful for future research on psychiatric inpatient care to ensure clarity in the terminology and definitions used in reports. There were also discrepancies in the way financial costs related to delays were reported, i.e., whether reported as cost per day, cost per year. This made comparing the costs across countries challenging and prevented clear conclusions being drawn. Future research should therefore aim to ensure clarity when reporting financial expenditure, for example, by calculating the daily cost of delays. It is important to highlight that only eighteen studies were identified over the 20-year search period, suggesting this area has not yet been subject to much research focus. All high-income countries met inclusion, but the final sample included studies from only five countries. It might have been expected that studies in other high-income countries be identified, particularly given the expensive nature of inpatient stays and as such delayed discharge. It might be beneficial for future research to further examine delayed discharge in psychiatric settings across other countries, particularly in the USA and EU. For the purposes of this review, studies not conducted in high-income countries were excluded. This was because lower-income countries might experience different factors contributing to delays due to differences in healthcare funding and social factors. As such, separate attention should be given to these settings, to understand similarities or differences in reasons for delays across low- and mid- income countries. Studies on forensic psychiatric settings and child and adolescent settings were also excluded in this instance, so again, there might be benefit in future research examining these areas.

Furthermore, future research could look not only at factors creating delays, but those causing longer delays. Some of the studies in this review began examining this, but more research in this area could be of interest. Finally, while the quality of included studies was relatively high, the studies were primarily of quantitative audit design and infrequently conducted significance testing. As such, further exploration of associations using significance testing would strengthen the evidence base.

In conclusion, 18 studies identified reasons for delayed discharge, including accommodation and placement related factors, challenges securing appropriate support, funding difficulties, family/carer factors, forensic factors and person being out of area. Delay was associated with having a diagnosis of schizophrenia or other psychotic disorder, cognitive impairment, increased service involvement prior to admission, and being unemployed. Service, societal and policy changes might be indicated, to improve accommodation and care provisions following discharge. Future research should continue to examine prolonged inpatient psychiatric stays, ensuring to distinguish between long stays and delayed discharge and improve clarity in terminology used.

Data availability

The data on which this review is based will be made publicly available on publication. A link to data for anonymous peer-review is here: https://osf.io/j4kng/?view_only=1fbf2558d9d044bbb1778fccd5fd6f51 .

Bryan K. Policies for reducing delayed discharge from hospital. Br Med Bull. 2010;95(1):33–46.

Article   PubMed   Google Scholar  

NHS England. Monthly delayed transfers of care situation reports: definitions and guidance. London: NHS England; 2015.

Google Scholar  

House of Commons Health Committee. Delayed Discharges (third report). In: Committee H, editor. 2002.

Rojas-García A, Turner S, Pizzo E, Hudson E, Thomas J, Raine R. Impact and experiences of delayed discharge: a mixed‐studies systematic review. Health Expect. 2018;21(1):41–56.

Christ W. Factors delaying discharge of psychiatric patients. Health Soc Work. 1984;9(3):178–87.

Article   CAS   Google Scholar  

Department of Health. NHS reference costs 2013 to 2014. Department of Health London; 2013.

Glasby J, Lester H. Delayed hospital discharge and mental health: the policy implications of recent research. Social Policy Adm. 2004;38(7):744–57.

Article   Google Scholar  

National Institute of Health and Care Excellence [NICE]. Transition between inpatient mental health settings and community or care home settings. National Institue for Health and Care Excellence; 2016.

Katsakou C, Rose D, Amos T, Bowers L, McCabe R, Oliver D, et al. Psychiatric patients’ views on why their involuntary hospitalisation was right or wrong: a qualitative study. Soc Psychiatry Psychiatr Epidemiol. 2012;47:1169–79.

Csipke E, Williams P, Rose D, Koeser L, McCrone P, Wykes T, et al. Following the Francis report: investigating patient experience of mental health in-patient care. Br J Psychiatry. 2016;209(1):35–9.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Gopalakrishna G, Ithman M, Malwitz K. Predictors of length of stay in a psychiatric hospital. Int J Psychiatry Clin Pract. 2015;19(4):238–44.

Micallef A, Buttigieg SC, Tomaselli G, Garg L. Defining delayed discharges of inpatients and their impact in acute hospital care: a scoping review. Int J Health Policy Manage. 2022;11(2):103.

Cadel L, Guilcher SJ, Kokorelias KM, Sutherland J, Glasby J, Kiran T, et al. Initiatives for improving delayed discharge from a hospital setting: a scoping review. BMJ open. 2021;11(2):e044291.

Article   PubMed   PubMed Central   Google Scholar  

Moher D, Liberati A, Tetzlaff J, Altman DG. PRISMA Group* t. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151(4):264–9.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. Updating guidance for reporting systematic reviews: development of the PRISMA 2020 statement. J Clin Epidemiol. 2021;134:103–12.

World Bank. World Bank Country and Lending Groups 2022 [ https://datahelpdesk.worldbank.org/knowledgebase/articles/906519.]

Covidence systematic review software. Veritas health innovation Melbourne, Australia. 2020. Available at: www.covidence.org.

Sirriyeh R, Lawton R, Gardner P, Armitage G. Reviewing studies with diverse designs: the development and evaluation of a new tool. J Eval Clin Pract. 2012;18(4):746–52.

Hong QN, Fàbregues S, Bartlett G, Boardman F, Cargo M, Dagenais P, et al. The mixed methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers. Educ Inform. 2018;34(4):285–91.

Hanif I, Rathod B. Delays in discharging elderly psychiatric in-patients. Psychiatr Bull. 2008;32(6):211–3.

Little J, Hirdes JP, Daniel I, editors. ALC status in in-patient mental health settings: evidence based on the Ontario Mental Health Reporting System. Healthcare Management Forum. Los Angeles, CA: SAGE Publications Sage CA; 2015.

Little J, Hirdes JP, Perlman CM, Meyer SB. Clinical predictors of delayed discharges in inpatient mental health settings across Ontario. Adm Policy Mental Health Mental Health Serv Res. 2019;46:105–14.

Aflalo M, Soucy N, Xue X, Colacone A, Jourdenais E, Boivin J-F. Characteristics and needs of psychiatric patients with prolonged hospital stay. Can J Psychiatry. 2015;60(4):181–8.

Honey A, Arblaster K, Nguyen J, Heard R. Predicting Housing Related delayed discharge from Mental Health Inpatient units: a Case Control Study. Adm Policy Mental Health Mental Health Serv Res. 2022;49(6):962–72.

Nguyen J, Honey A, Arblaster K, Heard R. Housing-related delayed discharge from inpatient mental health units: Magnitude and contributors in a metropolitan mental health service. Australian J Social Issues. 2022;57(1):144–63.

Chuah CPT, Honey A, Arblaster K. I’m institutionalised… there’s not much I can do’: lived experience of housing related delayed discharge. Aust Occup Ther J. 2022;69(5):574–84.

Cowman J, Whitty P. Prevalence of housing needs among inpatients: a 1 year audit of housing needs in the acute mental health unit in Tallaght Hospital. Ir J Psychol Med. 2016;33(3):159–64.

Article   CAS   PubMed   Google Scholar  

Lewis R, Glasby J. Delayed discharge from mental health hospitals: results of an English postal survey. Health Soc Care Commun. 2006;14(3):225–30.

Berg JE, Restan A. Duration of bed occupancy as calculated at a random chosen day in an acute care ward. Implications for the use of scarce resources in psychiatric care. Ann Gen Psychiatry. 2005;4(1):1–6.

Tucker S, Hargreaves C, Wilberforce M, Brand C, Challis D. What becomes of people admitted to acute old age psychiatry wards? An exploration of factors affecting length of stay, delayed discharge and discharge destination. Int J Geriatr Psychiatry. 2017;32(9):1027–36.

Tyrer P, Suryanarayan G, Rao B, Cicchetti D, Fulop N, Roberts F, et al. The bed requirement inventory: a simple measure to estimate the need for a psychiatric bed. Int J Soc Psychiatry. 2006;52(3):267–77.

Onyon R, Khan S, George M. Delayed discharges from a psychiatric intensive care unit–are we detaining patients unlawfully? J Psychiatric Intensive Care. 2006;2(2):59–64.

Impey M, Milner E. Delayed discharge from mental health inpatient care in the UK. Mental Health Pract. 2013;16(9).

Haw C, Otuwehinmi O, Kotterbova E. Out of area admissions to two independent sector PICUs: patient characteristics, length of stay and delayed discharges. J Psychiatric Intensive Care. 2017;13(1):27–36.

Poole R, Pearsall A, Ryan T. Delayed discharges in an urban in-patient mental health service in England. Psychiatric Bull. 2014;38(2):66–70.

Commander M, Rooprai D. Survey of long-stay patients on acute psychiatric wards. Psychiatr Bull. 2008;32(10):380–3.

Paton JM, Fahy MA, Livingston GA. Delayed discharge—a solvable problem? The place of intermediate care in mental health care of older people. Aging Ment Health. 2004;8(1):34–9.

Download references

No funding to declare.

Author information

Authors and affiliations.

Department of Psychology, University of Bath, Bath, BA2 7AY, UK

Ashley-Louise Teale, Ceri Morgan, Tom A. Jenkins & Pamela Jacobsen

You can also search for this author in PubMed   Google Scholar

Contributions

AT and PJ formulated the initial research questions and developed the systematic review protocol. AT ran the searches on databases. AT, CM and TJ conducted the screening, data extraction and quality assessment. PJ acted as senior reviewer to resolve any conflicts. AT synthesised the results. All authors contributed to data synthesis and interpretation. AT wrote the paper. All authors read and approved the final version of manuscript for submission.

Corresponding author

Correspondence to Ashley-Louise Teale .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Teale, AL., Morgan, C., Jenkins, T.A. et al. Delayed discharge in inpatient psychiatric care: a systematic review. Int J Ment Health Syst 18 , 14 (2024). https://doi.org/10.1186/s13033-024-00635-9

Download citation

Received : 28 July 2023

Accepted : 25 March 2024

Published : 06 April 2024

DOI : https://doi.org/10.1186/s13033-024-00635-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Delayed discharge
• Bed blocking
• Delayed transfer
• Psychiatric inpatient
• Inpatient treatment
• Prolonged stays
• Length of stay

International Journal of Mental Health Systems

ISSN: 1752-4458

• Submission enquiries: [email protected]

Retirement planning – a systematic review of literature and future research directions

• Published: 28 October 2023

Cite this article

• Kavita Karan Ingale   ORCID: orcid.org/0000-0003-3570-4211 1 &
• Ratna Achuta Paluri 2  

513 Accesses

Explore all metrics

Rising life expectancy and an aging population across nations are leading to an increased need for long-term financial savings and a focus on the financial well-being of retired individuals amidst changing policy framework. This study is a systematic review based on a scientific way of producing high-quality evidence based on 191 articles from the Scopus and Web of Science databases. It adopts the Theory, Context, Characteristics, and Method (TCCM) framework to analyze literature. This study provides collective insights into financial decision-making for retirement savings and identifies constructs for operationalizing and measuring financial behavior for retirement planning. Further, it indicates the need for an interdisciplinary approach. Though cognitive areas were studied extensively, the non-cognitive areas received little attention. Qualitative research design is gaining prominence in research over other methods, with the sparse application of mixed methods design. The study’s TCCM framework explicates several areas for further research. Furthermore, it guides the practice and policy by integrating empirical evidence and concomitant findings. Coherent synthesis of the extant literature reconciles the highly fragmented field of retirement planning. No research reports prospective areas for further analysis based on the TCCM framework on retirement planning, which highlights the uniqueness of the study.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA) Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

A Research Proposal to Examine Psychological Factors Influence on Financial Planning for Retirement in China

Domains and determinants of retirement timing: A systematic review of longitudinal studies

Micky Scharn, Ranu Sewdas, … Allard J. van der Beek

Reinventing Retirement

Deanna L. Sharpe

Data Availability

The research data will be made available on request.

Acknowledgment.

Elderly population is defined as a population aged 65 years and over.

Defined benefit plan guarantees benefits to the employee, while defined contribution plan requires employees to decide on their own investment and bear the financial risks identified with it.

“The old-age dependency ratio is defined as the number of individuals aged 65 and over per 100 people of working age defined as those at ages 20 to 64”(OECD 2023 ).

Adams GA, Rau BL (2011) Putting off tomorrow to do what you want today: planning for Retirement. Am Psychol 66(3):180–192. https://doi.org/10.1037/a0022131

Article   Google Scholar  

Aegon Cfor, Longevity, Retirement ICR (2016) The Aegon Retirement Readiness Survey 2016. In The Aegon Retirement Readiness Survey 2016 . https://www.aegon.com/contentassets/c6a4b1cdded34f1b85a4f21d4c66e5d3/2016-aegon-retirement-readiness-report-india.pdf

Agarwalla SK, Barua SK, Jacob J, Varma JR (2015) Financial Literacy among Working Young in Urban India. World Development , 67 (2013), 101–109. https://doi.org/10.1016/j.worlddev.2014.10.004

Ajzen I (1991) The theory of Planned Behavior. Organ Behav Hum Decis Process 50:179–211. https://doi.org/10.47985/dcidj.475

Anderson A, Baker F, Robinson DT (2017) Precautionary savings, retirement planning, and misperceptions of financial literacy. J Financ Econ 126(2):383–398. https://doi.org/10.1016/j.jfineco.2017.07.008

Atkinson A, Messy FA (2011) Assessing financial literacy in 12 countries: an OECD/INFE international pilot exercise. J Pension Econ Finance 10(4):657–665. https://doi.org/10.1017/S1474747211000539`

Aydin AE, Akben Selcuk E (2019) An investigation of financial literacy, money ethics, and time preferences among college students: a structural equation model. Int J Bank Mark 37(3):880–900. https://doi.org/10.1108/IJBM-05-2018-0120

Bapat D (2020) Antecedents to responsible financial management behavior among young adults: the moderating role of financial risk tolerance. Int J Bank Mark 38(5):1177–1194. https://doi.org/10.1108/IJBM-10-2019-0356

Beckett A, Hewer P, Howcroft B (2000) An exposition of consumer behaviour in the financial services industry. Int J Bank Mark 18(1):15–26. https://doi.org/10.1108/02652320010315325

Białowolski P (2019) Economic sentiment as a driver for household financial behavior. J Behav Experimental Econ 80(August 2017):59–66. https://doi.org/10.1016/j.socec.2019.03.006

Binswanger J, Carman KG (2012) How real people make long-term decisions: the case of retirement preparation. J Economic Behav Organ 81(1):39–60. https://doi.org/10.1016/j.jebo.2011.08.010

Brounen D, Koedijk KG, Pownall RAJ (2016) Household financial planning and savings behavior. J Int Money Finance 69:95–107. https://doi.org/10.1016/j.jimonfin.2016.06.011

Brown R, Jones M (2015) Mapping and exploring the topography of contemporary financial accounting research. Br Acc Rev 47(3):237–261. https://doi.org/10.1016/j.bar.2014.08.006

Brown S, Gray D (2016) Household finances and well-being in Australia: an empirical analysis of comparison effects. J Econ Psychol 53:17–36. https://doi.org/10.1016/j.joep.2015.12.006

Brown S, Taylor K (2014) Household finances and the big five personality traits. J Econ Psychol 45:197–212. https://doi.org/10.1016/j.joep.2014.10.006

Brown S, Taylor K (2016) Early influences on saving behaviour: analysis of British panel data. J Bank Finance 62:1–14. https://doi.org/10.1016/j.jbankfin.2015.09.011

Brüggen EC, Post T, Schmitz K (2019) Interactivity in online pension planners enhances engagement with retirement planning – but not for everyone. J Serv Mark 33(4):488–501. https://doi.org/10.1108/JSM-02-2018-0082

Bruggen E, Post T, Katharina S (2019) Interactivity in online pension planners enhances engagement with retirement planning but not for everyone. J Serv Mark 33(4):488–501

Calcagno R, Monticone C (2015) Financial literacy and the demand for financial advice. J Bank Finance 50:363–380. https://doi.org/10.1016/j.jbankfin.2014.03.013

Campbell JY (2006) Household finance. J Finance 61(4):1553–1604. https://doi.org/10.1111/j.1540-6261.2006.00883.x

Choudhury K (2015) Service quality and customers’ behavioural intentions: class and mass banking and implications for the consumer and society. Asia Pac J Mark Logistics 27(5):735–757

Chowdhry N, Jung J, Dholakia U (2018) Association for consumer research. Adv Consum Res 42:42–46

Google Scholar  

Clark GL, Knox-Hayes J, Strauss K (2009) Financial sophistication, salience, and the scale of deliberation in UK retirement planning. Environ Plann A 41(10):2496–2515. https://doi.org/10.1068/a41265

Clark R, Lusardi A, Mitchell OS (2017) Employee Financial Literacy and Retirement Plan Behavior: a case study. Econ Inq 55(1):248–259. https://doi.org/10.1111/ecin.12389

Collins JM, Urban C (2016) The role of information on Retirement Planning: evidence from a field study. Econ Inq 54(4):1860–1872. https://doi.org/10.1111/ecin.12349

Creswell J (2009) Research Design Qualitative Quantitative and Mixed Methods Approaches. In Sage Publishing: Vol. Third edit . https://doi.org/10.1002/tl.20234

Csorba L (2020) The determining factors of financial culture, financial literacy, and financial behavior. Public Finance Q 65:67–83. https://doi.org/10.35551/PFQ_2020_1_6

Davidoff T, Gerhard P, Post T (2017) Reverse mortgages: what homeowners (don’t) know and how it matters. J Economic Behav Organ 133:151–171. https://doi.org/10.1016/j.jebo.2016.11.007

Davis FD (1989) Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly: Management Information Systems 13(3):319–339. https://doi.org/10.2307/249008

Devlin J (2001) Consumer evaluation and competitive advantage in retail financial services - a research agenda. Eur J Mark 35(5/6):639–660

Dholakia U, Tam L, Yoon S, Wong N (2016) The ant and the grasshopper: understanding personal saving orientation of consumers. J Consum Res 43(1):134–155. https://doi.org/10.1093/jcr/ucw004

Dolls M, Doerrenberg P, Peichl A, Stichnoth H (2018) Do retirement savings increase in response to information about retirement and expected pensions? J Public Econ 158(July 2017):168–179. https://doi.org/10.1016/j.jpubeco.2017.12.014

Dragos SL, Dragos CM, Muresan GM (2020) From intention to the decision in purchasing life insurance and private pensions: different effects of knowledge and behavioural factors. J Behav Experimental Econ 87(March):101555. https://doi.org/10.1016/j.socec.2020.101555

Drever AI, Odders-white E, Kalish CW, Hoagland EM, Nelms EN, Drever AI, Odders-white E, Charles W, Else-quest NM, Hoagland EM, Nelms EN (2015) Foundations of Financial Weil-Being: Insights into the Role of Executive Function, Financial Socialization, and Experience-Based Learning in Childhood and Youth Source : The Journal of Consumer Affairs, Vol. 49, No. 1, Special Issue on Starting Ea. The Journal of Consumer Affairs , 49 (1)

Duflo E, Saez E (2002) Participation and investment decisions in a retirement plan: the influence of colleagues’ choices. J Public Econ 85(1):121–148. https://doi.org/10.1016/S0047-2727(01)00098-6

Duxbury D, Summers B, Hudson R, Keasey K (2013) How people evaluate defined contribution, annuity-based pension arrangements: a behavioral exploration. J Econ Psychol 34:256–269. https://doi.org/10.1016/j.joep.2012.10.008

Earl J, Bednall T, Muratore A (2015) A matter of time: why some people plan for retirement and others do not. Work Aging and Retirement 1(2):181–189. https://doi.org/10.1093/workar/wau005

Employees Benefits Research Institute (2020) EBRI Retirement Confidence Survey Report (Issue 202)

Engel JF, Kollat DT, Blackwell RD (1968) A model of consumer motivation and behavior. In: Research in consumer behavior. Holt, Rinehart and Winston, Inc., New York, pp 3–20

Erasmus A, Boshoff E, Rousseau G (2001) Consumer decision-making models within the discipline of consumer science: a critical approach. J Family Ecol Consumer Sci /Tydskrif Vir Gesinsekologie En Verbruikerswetenskappe 29(1):82–90. https://doi.org/10.4314/jfecs.v29i1.52799

Farrell L, Fry TRL, Risse L (2016) The significance of financial self-efficacy in explaining women’s personal finance behaviour. J Econ Psychol 54:85–99

Fernandes D, Lynch JG, Netemeyer RG (2014) Financial literacy, financial education, and downstream financial behaviors. Manage Sci 60(8):1861–1883. https://doi.org/10.1287/mnsc.2013.1849

Filbec G, Ricciardi V, Evensky H, Fan S, Holzhauer H, Spieler A (2017) Behavioral finance: a panel discussion. J Behav Experimental Finance 15:52–58. https://doi.org/10.1016/j.jbef.2015.07.003

Fishbein M (1979) A theory of reasoned action: some applications and implications. Nebraska Symposium on Motivation 27:65–116

Fisher PJ, Montalto CP (2010) Effect of saving motives and horizon on saving behaviors. J Econ Psychol 31(1):92–105. https://doi.org/10.1016/j.joep.2009.11.002

Flores SAM, Vieira KM (2014) Propensity toward indebtedness: an analysis using behavioral factors. J Behav Exp Finance 3:1–10

Foxall GR, Pallister JG (1998) Measuring purchase decision involvement for financial services: comparison of the Zaichkowsky and Mittal scales. Int J Bank Mark 16(5):180–194. https://doi.org/10.1108/02652329810228181

Friedman M (1957) Introduction to “A theory of the consumption function”. In: A theory of the consumption function. Princeton University Press, pp 1–6

Frydman C, Camerer CF (2016) The psychology and neuroscience of financial decision making. Trends Cogn Sci 20(9):661–675. https://doi.org/10.1016/j.tics.2016.07.003

Gardarsdóttir RB, Dittmar H (2012) The relationship of materialism to debt and financial well-being: the case of Iceland’s perceived prosperity. J Econ Psychol 33(3):471–481. https://doi.org/10.1016/j.joep.2011.12.008

Gathergood J (2012) Self-control, financial literacy and consumer over-indebtedness. J Econ Psychol 33(3):590–602

Gerhard P, Gladstone JJ, Hoffmann AOI (2018) Psychological characteristics and household savings behavior: the importance of accounting for latent heterogeneity. J Economic Behav Organ 148:66–82. https://doi.org/10.1016/j.jebo.2018.02.013

Gibbs PT (2009) Time, temporality, and

Goedde-Menke M, Lehmensiek-Starke M, Nolte S (2014) An empirical test of competing hypotheses for the annuity puzzle. J Econ Psychol 43:75–91

Gough O, Nurullah M (2009) Understanding what drives the purchase decision in pension and investment products. J Financial Serv Mark 14(2):152–172. https://doi.org/10.1057/fsm.2009.14

Griffin B, Loe D, Hesketh B (2012) Using Proactivity, Time Discounting, and the theory of Planned Behavior to identify predictors of Retirement Planning. Educ Gerontol 38(12):877–889. https://doi.org/10.1080/03601277.2012.660857

Gritten A (2011) New insights into consumer confidence in financial services. Int J Bank Mark 29(2):90–106. https://doi.org/10.1108/02652321111107602

Grohmann A (2018) Financial literacy and financial behavior: Evidence from the emerging Asian middle class. Pacific Basin Finance Journal , 48 (November 2017), 129–143. https://doi.org/10.1016/j.pacfin.2018.01.007

Grohmann A, Kouwenberg R, Menkhoff L (2015) Childhood roots of financial literacy. J Econ Psychol 51:114–133. https://doi.org/10.1016/j.joep.2015.09.002

Hair JF, Sarstedt M, Ringle CM, Mena JA (2012) An assessment of the use of partial least squares structural equation modeling in marketing research . 414–433. https://doi.org/10.1007/s11747-011-0261-6

Hanna SD, Kim KT, Chen SCC (2016) Retirement savings. In: Handbook of consumer finance research, pp 33–43

Harrison T, Waite K, White P (2006) Analysis by paralysis: the pension purchase decision process. Int J Bank Mark 24(1):5–23. https://doi.org/10.1108/02652320610642317

Hastings J, Mitchell O (2011) How financial literact and impatience shape retirement wealth and investment behaviors. Pengaruh Harga Diskon Dan Persepsi Produk Terhadap Nilai Belanja Serta Perilaku Pembelian Konsumen, NBER Working paper, 1–28

Hauff J, Carlander A, Amelie G, Tommy G, Holmen M (2016) Breaking the ice of low financial involvement: does narrative information format from a trusted sender increase savings in mutual funds? Int J Bank Mark 34(2):151–170

Hentzen JK, Hoffmann A, Dolan R, Pala E (2021) Artificial intelligence in customer-facing financial services: a systematic literature review and agenda for future research. Int J Bank Mark. https://doi.org/10.1108/IJBM-09-2021-0417

Hershey DA, Mowen JC (2000) Psychological determinants of financial preparedness for retirement. Gerontologist 40(6):687–697. https://doi.org/10.1093/geront/40.6.687

Hershey DA, Henkens K, Van Dalen HP (2007) Mapping the minds of retirement planners: a cross-cultural perspective. J Cross-Cult Psychol 38(3):361–382. https://doi.org/10.1177/0022022107300280

Hershey DA, Jacobs-Lawson JM, McArdle JJ, Hamagami F (2007b) Psychological foundations of financial planning for retirement. J Adult Dev 14(1–2):26–36. https://doi.org/10.1007/s10804-007-9028-1

Hershey DA, Jacobs-Lawson JM, McArdle JJ, Hamagami F (2008) Psychological foundations of financial planning for retirement. J Adult Dev 14(1–2):26–36. https://doi.org/10.1007/s10804-007-9028-1

Hershfield H, Goldstein D, Sharpe W, Fox J, Yeykelis L, Carstensen L, Bailenson J (2011) Increasing saving behavior through age-progressed renderings of the future self. J Mark Res 48:23–37

Hoffmann AOI, Broekhuizen TLJ (2009) Susceptibility to and impact of interpersonal influence in an investment context. J Acad Mark Sci 37:488–503

Hoffmann AOI, Broekhuizen TLJ (2010) Understanding investors’ decisions to purchase innovative products: drivers of adoption timing and range. Int J Res Mark 27(4):342–355. https://doi.org/10.1016/j.ijresmar.2010.08.002

Hoffmann AOI, Plotkina D (2020a) Positive framing when assessing the personal resources to manage one’s finances increases consumers’ retirement self-efficacy and improves retirement goal clarity. Psychol Mark 38(12):2286–2304. https://doi.org/10.1002/mar.21563

Hoffmann AOI, Plotkina D (2020b) Why and when does financial information affect retirement planning intentions and which consumers are more likely to act on them? Journal of Business Research , 117 (September 2019), 411–431. https://doi.org/10.1016/j.jbusres.2020.06.023

Hoffmann AOI, Plotkina D (2021) Let your past define your future. How recalling successful financial experiences can increase beliefs of self-efficacy in financial planning. J Consum Aff 55(3):847–871. https://doi.org/10.1111/joca.12378

Hoffmann AOI, Risse L (2020) Do good things come in pairs? How personality traits help explain individuals’ simultaneous pursuit of a healthy lifestyle and financially responsible behavior. J Consum Aff 54(3):1082–1120. https://doi.org/10.1111/joca.12317

Hsiao YJ, Tsai WC (2018) Financial literacy and participation in the derivatives markets. J Bank Finance 88:15–29

Huhmann BA, McQuitty S (2009) A model of consumer financial numeracy. Int J Bank Mark 27(4):270–293. https://doi.org/10.1108/02652320910968359

Huston SJ (2010) Measuring financial literacy. J Consum Aff 44(2):296–316. https://doi.org/10.1111/j.1745-6606.2010.01170.x

Ijevleva K, Arefjevs I (2014) Analysis of the Aggregate Financial Behaviour of customers using the Transtheoretical Model of Change. Procedia - Social and Behavioral Sciences 156(April):435–438. https://doi.org/10.1016/j.sbspro.2014.11.217

Ingale KK, Paluri RA (2020) Financial literacy and financial behavior: a bibliometric analysis. Rev Behav Finance. https://doi.org/10.1108/RBF-06-2020-0141

Jacobs-Lawson J, Hershey D (2005) Influence of future time perspective, financial knowledge, and financial risk tolerance on retirement savings behavior. Financial Serv Rev 14:331–344. https://doi.org/10.1088/1751-8113/44/8/085201

Jappelli T, Padula M (2013) Investment in financial literacy and saving decisions. J Bank Finance 37(8):2779–2792. https://doi.org/10.1016/j.jbankfin.2013.03.019

Kadoya Y, Rahim Khan MS (2020) Financial literacy in Japan: new evidence using financial knowledge, behavior, and attitude. Sustain (Switzerland) 12(9). https://doi.org/10.3390/su12093683

Kamil NSSN, Musa R, Sahak SZ (2014) Examining the Role of Financial Intelligence Quotient (FiQ) in explaining credit card usage behavior: a conceptual Framework. Procedia - Social and Behavioral Sciences 130:568–576. https://doi.org/10.1016/j.sbspro.2014.04.066

Kerry MJ (2018) Psychological antecedents of retirement planning: a systematic review. Front Psychol 9(OCT). https://doi.org/10.3389/fpsyg.2018.01870

Kerry MJ, Embretson SE (2018) An experimental evaluation of competing age predictions of future time perspective between workplace and retirement domains. Front Psychol 8(JAN):1–9. https://doi.org/10.3389/fpsyg.2017.02316

Kiliyanni AL, Sivaraman S (2016) The perception-reality gap in financial literacy: evidence from the most literate state in India. Int Rev Econ Educ 23:47–64. https://doi.org/10.1016/j.iree.2016.07.001

Kimiyaghalam F, Mansori S, Safari M, Yap S (2017) Parents’ influence on retirement planning in Malaysia. Family Consumer Sci Res J 45(3):315–325

Klapper L, Lusardi A, Panos GA (2013) Financial literacy and its consequences: evidence from Russia during the financial crisis. J Bank Finance 37(10):3904–3923

Koehler DJ, Langstaff J, Liu WQ (2015) A simulated financial savings task for studying consumption and retirement decision-making. J Econ Psychol 46:89–97. https://doi.org/10.1016/j.joep.2014.12.004

Kramer MM (2016) Financial literacy, confidence, and financial advice seeking. Journal of Economic Behavior and Organization , 131 (June 2015), 198–217. https://doi.org/10.1016/j.jebo.2016.08.016

Kumar S, Tomar S, Verma D (2019) Women’s financial planning for retirement: systematic literature review and future research agenda. Int J Bank Mark 37(1):120–141. https://doi.org/10.1108/IJBM-08-2017-0165

Kwon KN, Lee J (2009) The effects of reference point, knowledge, and risk propensity on the evaluation of financial products. J Bus Res 62(7):719–725. https://doi.org/10.1016/j.jbusres.2008.07.002

Landerretche OM, Martínez C (2013) Voluntary savings, financial behavior, and pension finance literacy: evidence from Chile. J Pension Econ Finance 12(3):251–297. https://doi.org/10.1017/S1474747212000340

Lee T (2017) (David). Clear, conspicuous, and improving: US corporate websites for critical financial literacy in retirement. International Journal of Bank Marketing , 35 (5), 761–780. https://doi.org/10.1108/IJBM-01-2016-0010

Liang C-J, Wang Wen‐Hung, Farquhar JD (2009) (2009). The influence of customer perceptions on financial performance in financial services. International Journal of Bank Marketing , 27 (2), 129–149

Liberman N, Trope Y (2003) Construal level theory of intertemporal judgment and decision. In: Loewenstein G, Read D, Baumeister R (eds) Time and decision: economic and psychological perspectives on intertemporal choice, pp 245–276

Lim KL, Soutar GN, Lee JA (2013) Factors affecting investment intentions: a consumer behaviour perspective. J Financ Serv Mark 18:301–315

Lin C, Hsiao YJ, Yeh CY (2017) Financial literacy, financial advisors, and information sources on demand for life insurance. Pac Basin Finance J 43(March):218–237. https://doi.org/10.1016/j.pacfin.2017.04.002

Lown JM (2011) Development and validation of a Financial Self-Efficacy Scale. J Financial Couns Plann 22(2):54–63

Lusardi A, Mitchell OS (2007) Baby Boomer retirement security: the roles of planning, financial literacy, and housing wealth. J Monet Econ 54(1):205–224. https://doi.org/10.1016/j.jmoneco.2006.12.001

Maloney M, McCarthy A (2017) Understanding pension communications at the organizational level: insights from bounded rationality theory & implications for HRM. Hum Resource Manage Rev 27(2):338–352. https://doi.org/10.1016/j.hrmr.2016.08.001

Marjanovic Z, Fiksenbaum L, Greenglass E (2018) Financial threat correlates with acute economic hardship and behavioral intentions that can improve one’s personal finances and health. J Behav Experimental Econ 77(April):151–157. https://doi.org/10.1016/j.socec.2018.09.012

Marques S, Mariano J, Lima ML, Abrams D (2018) Are you talking to the future me? The moderator role of future self-relevance on the effects of aging salience in retirement savings. J Appl Soc Psychol 48(7):360–368. https://doi.org/10.1111/jasp.12516

McKechnie S (1992) Consumer buying behaviour in financial services: an overview. Int J Bank Mark 10(5):5–39. https://doi.org/10.1108/02652329210016803

Milner T, Rosenstreich D (2013a) A review of consumer decision-making models and development of a new model for financial services. J Financial Serv Mark 18(2):106–120. https://doi.org/10.1057/fsm.2013.7

Milner T, Rosenstreich D (2013b) Insights into mature consumers of financial services. J Consumer Mark 30(3):248–257. https://doi.org/10.1108/07363761311328919

Mitchell OS, Mukherjee A (2017) Assessing the demand for micro pensions among India’s poor. J Econ Ageing 9:30–40. https://doi.org/10.1016/j.jeoa.2016.05.004

Mitchell O, Utkus S (2003) Lessons from Behavioral Finance for Retirement Plan Design (PRC WP 2003-6). http://prc.wharton.upenn.edu/prc/prc.html

Modigliani F, Brumberg RH (1954) Utility analysis and the consumption function: an interpretation of cross-section data. In: Kurihara KK (ed) Post-Keynesian economics. Rutgers University Press, New Brunswick, pp 388–436

Moher D, Liberati A, Tetzlaff J, Altman DG, Altman D, Antes G, Atkins D, Barbour V, Barrowman N, Berlin JA, Clark J, Clarke M, Cook D, D’Amico R, Deeks JJ, Devereaux PJ, Dickersin K, Egger M, Ernst E, …, Tugwell P (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 6(7). https://doi.org/10.1371/journal.pmed.1000097

Monti M, Pelligra V, Martignon L, Berg N (2014) Retail investors and financial advisors: new evidence on trust and advice taking heuristics. J Bus Res 67(8):1749–1757. https://doi.org/10.1016/j.jbusres.2014.02.022

Mouna A, Anis J (2017) Financial literacy in Tunisia: its determinants and its implications on investment behavior. Res Int Bus Finance 39:568–577

Mullainathan S, Thaler R (2000) Massachusetts Institute of Technology Department of Economics Working Paper Series . September

Nga KH, Yeoh KK (2018) An exploratory model on retirement savings behaviour: a Malaysian study. Int J Bus Soc 19(3):637–659

OECD (2023) Old-age dependency ratio (indicator). https://doi.org/10.1787/e0255c98-en . Accessed 13 Oct 2023

Onwuegbuzie AJ, Collins KM (2007) A typology of mixed methods sampling designs in social science research. Qualitative Rep 12(2):474–498

Pallister JG, Wang HC, Foxall GR (2007) An application of the style/involvement model to financial services. Technovation 27(1–2):78–88. https://doi.org/10.1016/j.technovation.2005.10.001

Pan L, Pezzuti T, Lu W, Pechmann C (2019) Hyperopia and frugality: different motivational drivers and yet similar effects on consumer spending. J Bus Res 95(August 2018):347–356

Parise G, Peijnenburg K (2017) Understanding the Determinants of Financial Outcomes and Choices: The Role of Noncognitive Abilities. BIS Working Papers

Paul J, Rosado-Serrano A (2019) Gradual internationalization vs Born-Global/International new venture models: a review and research agenda. Int Mark Rev 36(6):830–858. https://doi.org/10.1108/IMR-10-2018-0280

Paul J, Criado AR (2020) The art of writing literature review: what do we know and what do we need to know? Int Bus Rev 29(4):101717. https://doi.org/10.1016/j.ibusrev.2020.101717

Paul J, Khatri P, Kaur Duggal H (2023) Frameworks for developing impactful systematic literature reviews and theory building: what, why and how? J Decis Syst 00(00):1–14. https://doi.org/10.1080/12460125.2023.2197700

Petkoska J, Earl JK (2009) Understanding the influence of demographic and psychological variables on Retirement Planning. Psychol Aging 24(1):245–251. https://doi.org/10.1037/a0014096

Piotrowska M (2019) The importance of personality characteristics and behavioral constraints for retirement saving. Econ Anal Policy 64:194–220

Plath DA, Stevenson TH (2005) Financial services consumption behavior across Hispanic American consumers. J Bus Res 58(8):1089–1099. https://doi.org/10.1016/j.jbusres.2004.03.003

Poterba JM (2015) Saver heterogeneity and the challenge of assessing retirement saving adequacy. Natl Tax J 68(2):377–388. https://doi.org/10.17310/ntj.2015.2.06

Potrich ACG, Vieira KM, Kirch G (2018) How well do women do when it comes to financial literacy? Proposition of an indicator and analysis of gender differences. J Behav Experimental Finance 17:28–41. https://doi.org/10.1016/j.jbef.2017.12.005

Rai D, Lin CW (2019) (Wilson). The influence of implicit self-theories on consumer financial decision making. Journal of Business Research , 95 (August 2018), 316–325. https://doi.org/10.1016/j.jbusres.2018.08.016

Ramalho TB, Forte D (2019) Financial literacy in Brazil – do knowledge and self-confidence relate with behavior? RAUSP Manage J 54(1):77–95. https://doi.org/10.1108/RAUSP-04-2018-0008

Rana J, Paul J (2017) Consumer behavior and purchase intention for organic food: a review and research agenda. J Retailing Consumer Serv 38(June):157–165. https://doi.org/10.1016/j.jretconser.2017.06.004

Ranyard R, McNair S, Nicolini G, Duxbury D (2020) An item response theory approach to constructing and evaluating brief and in-depth financial literacy scales. J Consum Aff 54(3):1121–1156. https://doi.org/10.1111/joca.12322

RBI Household Finance Committee (2017) Indian household finance. Reserve Bank of India, Mumbai

Ruefenacht M, Schlager T, Maas P, Puustinen P (2015) Drivers of long-term savings behavior from consumer’s perspective. Electron Libr 34(1):1–5

Scholz JK, Seshadri A, Khitatrakun S (2006) Are Americans saving “optimally” for retirement? J Polit Econ 114(4):607–643

Schuabb T, França LH, Amorim SM (2019) Retirement savings model tested with Brazilian private health care workers. Front Psychol 10(JULY):1–11. https://doi.org/10.3389/fpsyg.2019.01701

Schuhen M, Schurkmann S (2014) International Review of Economics Education. Int Rev Econ Educ 16:1–11

Segel-Karpas D, Werner P (2014) Perceived financial retirement preparedness and its correlates: a national study in Israel. Int J Aging Hum Dev 79(4):279–301. https://doi.org/10.1177/0091415015574177

Seth H, Talwar S, Bhatia A, Saxena A, Dhir A (2020) Consumer resistance and inertia of retail investors: Development of the resistance adoption inertia continuance (RAIC) framework. Journal of Retailing and Consumer Services , 55 (August 2019), 102071. https://doi.org/10.1016/j.jretconser.2020.102071

Sewell M (2008) Behavioural finance. Economist 389(8604):1–13. https://doi.org/10.1057/9780230280786_5

Shefrin HM, Thaler RH (1988) The behavioral life‐cycle hypothesis. Econ Inq 26(4):609–643

Shim S, Serido J, Tang C (2012) The ant and the grasshopper revisited: the present psychological benefits of saving and future oriented financial behavior. J Econ Psychol 33(1):155–165

Simon HA (1978) Information-processing theory of human problem solving. In: Handbook of learning and cognitive processes, vol 5, pp 271–295

Sivaramakrishnan S, Srivastava M, Rastogi A (2017) Attitudinal factors, financial literacy, and stock market participation. Int J Bank Mark 34(1):1–5

Snyder H (2019) Literature review as a research methodology: an overview and guidelines. J Bus Res 104(August):333–339. https://doi.org/10.1016/j.jbusres.2019.07.039

Stawski RS, Hershey DA, Jacobs-Lawson JM (2007) Goal clarity and financial planning activities as determinants of retirement savings contributions. Int J Aging Hum Dev 64(1):13–32. https://doi.org/10.2190/13GK-5H72-H324-16P2

Steinert JI, Zenker J, Filipiak U, Movsisyan A, Cluver LD, Shenderovich Y (2018) Do saving promotion interventions increase household savings, consumption, and investments in Sub-saharan Africa? A systematic review and meta-analysis. World Dev 104:238–256. https://doi.org/10.1016/j.worlddev.2017.11.018

Steinhart Y, Mazursky D (2010) Purchase availability and involvement antecedents among financial products. Int J Bank Mark 28(2):113–135. https://doi.org/10.1108/02652321011018314

Strömbäck C, Lind T, Skagerlund K, Västfjäll D, Tinghög G (2017) Does self-control predict financial behavior and financial well-being? J Behav Experimental Finance 14:30–38. https://doi.org/10.1016/j.jbef.2017.04.002

Strömbäck C, Skagerlund K, Västfjäll D, Tinghög G (2020) Subjective self-control but not objective measures of executive functions predict financial behavior and well-being. Journal of Behavioral and Experimental Finance , 27 . https://doi.org/10.1016/j.jbef.2020.100339

Tam L, Dholakia U (2014) Saving in cycles: how to get people to save more money. Psychol Sci 25(2):531–537. https://doi.org/10.1177/0956797613512129

Tang N, Baker A (2016) Self-esteem, financial knowledge and financial behavior. J Econ Psychol 54:164–176

Tate M, Evermann J, Gable G (2015) An integrated framework for theories of individual attitudes toward technology. Inform Manage 52(6):710–727. https://doi.org/10.1016/j.im.2015.06.005

Taylor MP, Jenkins SP, Sacker A (2011) Financial capability and psychological health. J Econ Psychol 32(5):710–723. https://doi.org/10.1016/j.joep.2011.05.006

Tennyson S, Yang HK (2014) The role of life experience in long-term care insurance decisions. Journal of Economic Psychology , 42 (2014), 175–188. https://doi.org/10.1016/j.joep.2014.04.002

Thaler BRH (1994) Psychology and savings policies. Am Econ Rev 84(2):175–179. http://www.jstor.org/stable/3132220

Thaler R (1980) Toward a positive theory of consumer choice. J Econ Behav Organ 1:39–60

Thaler RH (2005) Advances in behavioral finance. Adv Behav Finance 2:1–694. https://doi.org/10.2307/2329257

Thaler R, Shefrin H (1981) An economic theory of self-control. J Polit Econ 89(2):392–406

Tomar S, Kent Baker H, Kumar S, Hoffmann AOI (2021) Psychological determinants of retirement financial planning behavior. Journal of Business Research , 133 (November 2020), 432–449. https://doi.org/10.1016/j.jbusres.2021.05.007

Topa G, Moriano JA, Depolo M, Alcover CM, Morales JF (2009) Antecedents and consequences of retirement planning and decision-making: a meta-analysis and model. J Vocat Behav 75(1):38–55. https://doi.org/10.1016/j.jvb.2009.03.002

Topa G, Moriano JA, Depolo M, Alcover CM, Moreno A (2011) Retirement and wealth relationships: Meta-analysis and SEM. Res Aging 33(5):501–528. https://doi.org/10.1177/0164027511410549

Tranfield D, Denyer D, Smart P (2003) Towards a methodology for developing evidence-informed management knowledge by means of systematic review. Br J Manag 14(3):207–222. https://doi.org/10.1111/1467-8551.00375

Ülkümen G, Cheema A (2011) Framing goals to influence personal savings: the role of specificity and construal level. J Mark Res 48(6):958–969. https://doi.org/10.1509/jmr.09.0516

United Nations, Department of Economic and Social, Affairs PD (2020) (2019). World Population Ageing 2019. In United Nations . http://link.springer.com/chapter/10.1007/ 978-94-007-5204-7_6

Utkarsh, Pandey A, Ashta A, Spiegelman E, Sutan A (2020) Catch them young: impact of financial Socialization, financial literacy and attitude towards money on the financial well-being of young adults. Int J Consumer Stud 44(6):531–541. https://doi.org/10.1111/ijcs.12583

Valente TW, Paredes P, Poppe P (1998) Matching the message to the process: the relative ordering of knowledge, attitudes, and practices in behavior change research. Hum Commun Res 24(3):366–385

Van Rooij M, Teppa F (2014) Personal traits and individual choices: taking action in economic and non-economic decisions. J Econ Behav Organ 100:33–43

van Rooij M, Lusardi A, Alessie R (2011) Financial literacy and stock market participation. J Financ Econ 101(2):449–472. https://doi.org/10.1016/j.jfineco.2011.03.006

Van Rooij MCJ, Lusardi A, Alessie RJM (2011a) Financial literacy and retirement planning in the Netherlands. J Econ Psychol 32(4):593–608. https://doi.org/10.1016/j.joep.2011.02.004

van Schie RJG, Dellaert BGC, Donkers B (2015) Promoting later planned retirement: construal level intervention impact reverses with age. J Econ Psychol 50:124–131. https://doi.org/10.1016/j.joep.2015.06.010

Venkatesh V, Morris M, Davis G, Davis F (2003) Factors influencing the Use of M-Banking by academics: Case Study sms-based M-Banking. MIS Q 27(3):425–478

Vitt LA (2004) Consumers’ financial decisions and the psychology of values. J Financial Service Professionals 58(November):68–77. http://search.ebscohost.com/login.aspx?direct=true &db=bth&AN=14888952&site=ehost-live

Wang L, Lu W, Malhotra NK (2011) Demographics, attitude, personality, and credit card features correlate with credit card debt: a view from China. J Econ Psychol 32(1):179–193. https://doi.org/10.1016/j.joep.2010.11.006

World Economic Forum (2019) Investing in (and for) our future. Issue June. www.weforum.org

Xia T, Wang Z, Li K (2014) Financial literacy overconfidence and stock market participation. Soc Indic Res 119(3):1233–1245. https://doi.org/10.1007/s11205-013-0555-9

Xiao JJ, Chen C, Chen F (2014) Consumer financial capability and financial satisfaction. Soc Indic Res 118(1):415–432. https://doi.org/10.1007/s11205-013-0414-8

Yeung DY, Zhou X (2017) Planning for retirement: longitudinal effect on retirement resources and post-retirement well-being. Front Psychol 8:1300

Zhou R, Pham MT (2004) Promotion and prevention across mental accounts: when financial products dictate consumers’ investment goals. J Consum Res 31(1):125–135. https://doi.org/10.1086/383429

Download references

Acknowledgements

Authors would like to acknowledge the academicians and researchers who guided the search of the article and would like to thank the experts for the valuable inputs to refine the work.

There is no funding received for this research.

Author information

Authors and affiliations.

Symbiosis International (Deemed University), Pune, India

Kavita Karan Ingale

Symbiosis Institute of Operations Management, Symbiosis International (Deemed) University, Pune, India

Ratna Achuta Paluri

You can also search for this author in PubMed   Google Scholar

Contributions

Both authors contributed to the conceptualization, research design, methodology, analysis of the data,writing of the manuscript and its revision.

Corresponding author

Correspondence to Kavita Karan Ingale .

Ethics declarations

Conflict of interest.

The authors declare that there is no conflict of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Ingale, K.K., Paluri, R.A. Retirement planning – a systematic review of literature and future research directions. Manag Rev Q (2023). https://doi.org/10.1007/s11301-023-00377-x

Download citation

Received : 14 December 2022

Accepted : 04 October 2023

Published : 28 October 2023

DOI : https://doi.org/10.1007/s11301-023-00377-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Retirement planning
• Systematic literature review
• Financial behavior
• Household finance
• Long-term savings
• Pension plan
• Financial literacy
• TCCM framework
• Find a journal
• Publish with us
• Track your research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• JRSM Short Rep
• v.4(6); 2013 Jun

Using mixed methods in health research

Shema tariq.

1 School of Health Sciences, City University London, EC1A 7QN, London, UK

Jenny Woodman

2 MRC Centre of Epidemiology for Child Health, UCL Institute of Child Health, WC1N 1EH, London, UK

Mixed methods research is the use of quantitative and qualitative methods in a single study or series of studies. It is an emergent methodology which is increasingly used by health researchers, especially within health services research. There is a growing literature on the theory, design and critical appraisal of mixed methods research. However, there are few papers that summarize this methodological approach for health practitioners who wish to conduct or critically engage with mixed methods studies. The objective of this paper is to provide an accessible introduction to mixed methods for clinicians and researchers unfamiliar with this approach. We present a synthesis of key methodological literature on mixed methods research, with examples from our own work and that of others, to illustrate the practical applications of this approach within health research. We summarize definitions of mixed methods research, the value of this approach, key aspects of study design and analysis, and discuss the potential challenges of combining quantitative and qualitative methods and data. One of the key challenges within mixed methods research is the successful integration of quantitative and qualitative data during analysis and interpretation. However, the integration of different types of data can generate insights into a research question, resulting in enriched understanding of complex health research problems.

Introduction

Mixed methods research is the use of quantitative and qualitative methods in one study. Research is often dichotomized as quantitative or qualitative. Quantitative research, such as clinical trials or observational studies, generates numerical data. On the other hand qualitative approaches tend to generate non-numerical data, using methods such as semi-structured interviews, focus group discussions and participant observation. Historically, quantitative methods have dominated health research. However, qualitative methods have been increasingly accepted by the health research community in the past two decades, with a rise in publication of qualitative studies. 1 As the value of qualitative approaches has been recognized, there has been a growing interest in combining qualitative and quantitative methods. A recent review of health services research within England has shown an increase in the proportion of studies classified as mixed methods from 17% in the mid-1990s to 30% in the early 2000s. 2 In this paper, we present a synthesis of key literature on mixed methods research, with examples from our own work and that of others to illustrate the practical applications of this approach. This paper is aimed at health researchers and practitioners who are new to the field of mixed methods research and may only have experience of either quantitative or qualitative approaches and methodologies. We wish to provide these readers with an accessible introduction to the increasingly popular methodology of mixed methods research. We hope this will help readers to consider whether their research questions might best be answered by a mixed methods study design, and to engage critically with health research that uses this approach.

The authors each independently carried out a narrative literature review and met to discuss findings. Literature was identified via searches of PubMed, Google and Google Scholar, and hand-searches of the Journal of Mixed Methods Research, with relevant publications selected after discussion. An important consideration was that papers either had a methodological focus or contained a detailed description of their mixed methods design. For PubMed and Google searches, similar terms were used. For example, the PubMed strategy consisted of title and abstract searches for: ((mixed methods) OR ((mixed OR (qualitative AND quantitative)) AND methods)). We also drew upon recommendations from mixed methods conferences and seminars, and reference lists from key publications.

What is mixed methods research?

The most widely accepted definition of mixed methods research is research that ‘focuses on collecting, analysing, and mixing both quantitative and qualitative data in a single study or a series of studies’. 3 Central to the definition is the use of both quantitative and qualitative methods in one study (or a series of connected studies). Separate quantitative and qualitative studies addressing the same research question independently would not be considered ‘mixed methods’ as there would be no integration of approaches at the design, analysis or presentation stage. A recent innovation in mixed methods research is the mixed methods systematic review, which sets out to systematically appraise both quantitative and qualitative literature on a subject area and then synthesize the findings.

Why are mixed methods approaches used?

The underlying assumption of mixed methods research is that it can address some research questions more comprehensively than by using either quantitative or qualitative methods alone. 3 Questions that profit most from a mixed methods design tend to be broad and complex, with multiple facets that may each be best explored by quantitative or qualitative methods. See Boxes 1 and ​ and2 2 for examples from our own work.

Examples of authors’ mixed methods research – JW.

Examples of authors’ mixed methods research – ST.

Usually, quantitative research is associated with a positivist stance and a belief that reality that can be measured and observed objectively. Most commonly, it sets out to test an a priori hypothesis and is therefore conventionally described as ‘deductive’. Strengths of quantitative research include its procedures to minimize confounding and its potential to generate generalizable findings if based on samples that are both large enough and representative. It remains the dominant paradigm in health research. However, this deductive approach is less suited to generating hypotheses about how or why things are happening, or explaining complex social or cultural phenomena.

Qualitative research most often comes from an interpretive framework and is usually informed by the belief that there are multiple realities shaped by personal viewpoints, context and meaning. In-depth qualitative research aims to provide a rich description of views, beliefs and meaning. It also tends to acknowledge the role of researcher and context in shaping and producing the data. Qualitative approaches are described as ‘inductive’ as questions are often open-ended with the analysis allowing hypotheses to emerge from data. High-quality qualitative research can generate robust theory that is applicable to contexts outside of the study area in question, helping to guide practitioners and policy-makers. 8 However, for research that aims to directly impact on policy and practice, the findings of qualitative research can be limited by the small sample sizes that are necessary for in-depth exploratory work and the consequent lack of generalizabilty.

Mixed methods research therefore has the potential to harness the strengths and counterbalance the weaknesses of both approaches and can be especially powerful when addressing complex, multifaceted issues such as health services interventions 9 and living with chronic illness. 10

There are many reasons why researchers choose to combine quantitative and qualitative methods in a study. 11 , 12 We list some common reasons below, using a hypothetical research question about adolescents’ adherence to anticonvulsant medication to illustrate real world applications.

• Complementarity: Using data obtained by one method to illustrate results from another. An example of this would be a survey of adolescents with epilepsy demonstrating poor levels of adherence. Semi-structured interviews with a sub-group of those surveyed may allow us to explore barriers to adherence.
• Development: Using results from one method to develop or inform the use of the other method. A focus group conducted with a group of adolescents with epilepsy may identify mobile phone technology as a potentially important tool in adherence support. We could then develop a mobile phone ‘app’ that reminds patients to take their medication and conduct an intervention study to assess its impact on adherence levels.
• Initiation: Using results from different methods specifically to look for areas of incongruence in order to generate new insights. An illustration of this would be a study exploring the discrepancy between reported adherence in clinic consultations and actual medication adherence. A review of case notes may find adherence levels of over 90% in a clinic population; however, semi-structured interviews with peer researchers may reveal lower levels of adherence and barriers to open discussion with clinicians.
• Expansion: Setting out to examine different aspects of a research question, where each aspect warrants different methods. We may wish to conduct a study that explores adherence more broadly. A large-scale survey of adolescents with epilepsy would provide information on adherence levels and associations whilst interviews and focus groups may allow us to engage with individual experiences of chronic illness and medication in adolescence.
• Triangulation: Using data obtained by both methods to corroborate findings. For example, we could conduct a clinical study measuring drug levels in individuals and documenting self-reported adherence. Qualitative methods such as video diaries may confirm adherence levels.

To this list we would also add political commitment. That is to say, researchers may recognize, and wish to deploy, the strengths of quantitative research in producing generalizable results but may also be committed to representing the voice of participants in their work.

Whatever the reasons for mixing methods, it is important that authors present these explicitly as it allows us to assess if a mixed methods study design is appropriate for answering the research question. 3 , 13

How is mixed methods research conducted?

When embarking on a mixed methods research project it is important to consider:

• the methods that will be used;
• the priority of the methods;
• the sequence in which the methods are to be used.

A wide variety of methods exists by which to collect both quantitative and qualitative data. Both the research question and the data required will be the main determinants of the methods used. To a lesser extent, the choice of methods may be influenced by feasibility, the research team’s skills and experience and time constraints.

Priority of methods relates to the emphasis placed on each method in the study. For instance, the study may be predominantly a quantitative study with a small qualitative component, or vice versa. Alternatively, both quantitative and qualitative methods and data may have equal weighting. The emphasis given to each component of the study will be driven mainly by the research question, the skills of the research team and feasibility.

Finally, researchers must decide when each method is to be used in the study. For instance a team may choose to start with a quantitative phase followed by a qualitative phase, or vice versa. Some studies use both quantitative and qualitative methods concurrently. Again the choice of when to use each method is largely dependent on the research question.

The priority and sequence of mixing methods have been elaborated in a typology of mixed methods research models. See Table 1 for typology and specific examples.

Examples of studies using mixed methods.

How is data analysed in a mixed methods project?

The most important, and perhaps most difficult, aspect of mixed methods research is integrating the qualitative and quantitative data. One approach is to analyse the two data types separately and to then undertake a second stage of analysis where the data and findings from both studies are compared, contrasted and combined. 19 The quantitative and qualitative data are kept analytically distinct and are analysed using techniques usually associated with that type of data; for example, statistical techniques could be used to analyse survey data whilst thematic analysis may be used to analyse interview data. In this approach, the integrity of each data is preserved whilst also capitalizing on the potential for enhanced understanding from combining the two data and sets of findings.

Another approach to mixed methods data analysis is the integrative strategy. 20 Rather than keeping the datasets separate, one type of data may be transformed into another type. That is to say that qualitative data may be turned into quantitative data (‘quantitizing’) or quantitative data may be converted into qualitative data (‘qualitizing’). 21 The former is probably the most common method of this type of integrated analysis. Quantitative transformation is achieved by the numerical coding of qualitative data to create variables that may relate to themes or constructs, allowing statements such as ‘six of 10 participants spoke of the financial barriers to accessing health care’. These data can then be combined with the quantitative dataset and analysed together. Transforming quantitative data into qualitative data is less common. An example of this is the development of narrative psychological ‘types’ from numerical data obtained by questionnaires. 22

Potential challenges in conducting mixed methods research

Despite its considerable strengths as an approach, mixed methods research can present researchers with challenges. 23 , 24

Firstly, combining methodologies has sometimes been seen as problematic because of the view that quantitative and qualitative belong to separate and incompatible paradigms. In this context, paradigms are the set of practices and beliefs held by an academic community at a given point in time. 25 Researchers subscribing to this view argue that it is neither possible nor desirable to combine quantitative and qualitative methods in a study as they represent essentially different and conflicting ways of viewing the world and how we collect information about it. 8 Other researchers take a more pragmatic view, believing that concerns about the incommensurability of worldviews can be set aside if the combination of quantitative and qualitative methods addresses the research question effectively. This pragmatic view informs much applied mixed methods research in health services or policy. 8

Secondly, combining two methods in one study can be time consuming and requires experience and skills in both quantitative and qualitative methods. This can mean, in reality, that a mixed methods project requires a team rather than a lone researcher in order to conduct the study rigorously and within the specified time frame. However, it is important that a team comprising members from different disciplines work well together, rather than becoming compartmentalized. 26 We believe that a project leader with experience in both quantitative and qualitative methods can act as an important bridge in a mixed methods team.

Thirdly, achieving true integration of the different types of data can be difficult. We have suggested various analytic strategies above but this can be hard to achieve as it requires innovative thinking to move between different types of data and make meaningful links between them. It is therefore important to reflect on the results of a study and ask if your understanding has been enriched by the combination of different types of data. If this is not the case then integration may not have occurred sufficiently. 23

Finally, many researchers cite the difficulty in presenting the results of mixed methods study as a barrier to conducting this type of research. 23 Researchers may decide to present their quantitative and qualitative data separately for different audiences. This strategy may involve a decision to publish additional work focusing on the interpretations and conclusions which come from comparing and contrasting findings from the different data types. See Box 1 for an example of this type of publication strategy. Many journals in the medical sciences have a distinct methodological base and relatively restrictive word limits which may preclude the publication of complex, mixed methods studies. However, as the number of mixed methods studies increases in the health research literature we would expect researchers to feel more confident in the presentation of this type of work.

Many of the areas we explore in health are complex and multifaceted. Mixed methods research (combining quantitative and qualitative methods in one study) is an innovative and increasingly popular way of addressing these complexities. Although mixed methods research presents some challenges, in much the same way as every methodology does, this approach provides the research team with a wider range of tools at their disposal in order to answer a question. We believe that the production and integration of different types of data and the combination of skill sets in a team can generate insights into a research question, resulting in enriched understanding.

DECLARATIONS

Competing interests.

None declared

This work was funded by the Medical Research Council (MRC) [grant number: G0701648 to ST], and the MRC with the Economic and Social Research Council (ESRC) [grant number: G0800112 to JW]

Ethical approval

No ethical approval was required for this work

Contributorship

This work was conceived by both ST and JW who each carried out an independent literature review and collaborated on the structure and content of this report. ST wrote the manuscript with revisions and editing done by JW

Acknowledgements

We thank Professors Jonathan Elford and Ruth Gilbert for their comments on draft manuscripts

This article was submitted by the authors and peer reviewed by Geoffrey Harding

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• Write for Us
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Volume 24, Issue 2
• Five tips for developing useful literature summary tables for writing review articles
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0003-0157-5319 Ahtisham Younas 1 , 2 ,
• http://orcid.org/0000-0002-7839-8130 Parveen Ali 3 , 4
• 1 Memorial University of Newfoundland , St John's , Newfoundland , Canada
• 2 Swat College of Nursing , Pakistan
• 3 School of Nursing and Midwifery , University of Sheffield , Sheffield , South Yorkshire , UK
• 4 Sheffield University Interpersonal Violence Research Group , Sheffield University , Sheffield , UK
• Correspondence to Ahtisham Younas, Memorial University of Newfoundland, St John's, NL A1C 5C4, Canada; ay6133{at}mun.ca

https://doi.org/10.1136/ebnurs-2021-103417

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

Literature reviews offer a critical synthesis of empirical and theoretical literature to assess the strength of evidence, develop guidelines for practice and policymaking, and identify areas for future research. 1 It is often essential and usually the first task in any research endeavour, particularly in masters or doctoral level education. For effective data extraction and rigorous synthesis in reviews, the use of literature summary tables is of utmost importance. A literature summary table provides a synopsis of an included article. It succinctly presents its purpose, methods, findings and other relevant information pertinent to the review. The aim of developing these literature summary tables is to provide the reader with the information at one glance. Since there are multiple types of reviews (eg, systematic, integrative, scoping, critical and mixed methods) with distinct purposes and techniques, 2 there could be various approaches for developing literature summary tables making it a complex task specialty for the novice researchers or reviewers. Here, we offer five tips for authors of the review articles, relevant to all types of reviews, for creating useful and relevant literature summary tables. We also provide examples from our published reviews to illustrate how useful literature summary tables can be developed and what sort of information should be provided.

Tip 1: provide detailed information about frameworks and methods

• Download figure
• Open in new tab
• Download powerpoint

Tabular literature summaries from a scoping review. Source: Rasheed et al . 3

The provision of information about conceptual and theoretical frameworks and methods is useful for several reasons. First, in quantitative (reviews synthesising the results of quantitative studies) and mixed reviews (reviews synthesising the results of both qualitative and quantitative studies to address a mixed review question), it allows the readers to assess the congruence of the core findings and methods with the adapted framework and tested assumptions. In qualitative reviews (reviews synthesising results of qualitative studies), this information is beneficial for readers to recognise the underlying philosophical and paradigmatic stance of the authors of the included articles. For example, imagine the authors of an article, included in a review, used phenomenological inquiry for their research. In that case, the review authors and the readers of the review need to know what kind of (transcendental or hermeneutic) philosophical stance guided the inquiry. Review authors should, therefore, include the philosophical stance in their literature summary for the particular article. Second, information about frameworks and methods enables review authors and readers to judge the quality of the research, which allows for discerning the strengths and limitations of the article. For example, if authors of an included article intended to develop a new scale and test its psychometric properties. To achieve this aim, they used a convenience sample of 150 participants and performed exploratory (EFA) and confirmatory factor analysis (CFA) on the same sample. Such an approach would indicate a flawed methodology because EFA and CFA should not be conducted on the same sample. The review authors must include this information in their summary table. Omitting this information from a summary could lead to the inclusion of a flawed article in the review, thereby jeopardising the review’s rigour.

Tip 2: include strengths and limitations for each article

Critical appraisal of individual articles included in a review is crucial for increasing the rigour of the review. Despite using various templates for critical appraisal, authors often do not provide detailed information about each reviewed article’s strengths and limitations. Merely noting the quality score based on standardised critical appraisal templates is not adequate because the readers should be able to identify the reasons for assigning a weak or moderate rating. Many recent critical appraisal checklists (eg, Mixed Methods Appraisal Tool) discourage review authors from assigning a quality score and recommend noting the main strengths and limitations of included studies. It is also vital that methodological and conceptual limitations and strengths of the articles included in the review are provided because not all review articles include empirical research papers. Rather some review synthesises the theoretical aspects of articles. Providing information about conceptual limitations is also important for readers to judge the quality of foundations of the research. For example, if you included a mixed-methods study in the review, reporting the methodological and conceptual limitations about ‘integration’ is critical for evaluating the study’s strength. Suppose the authors only collected qualitative and quantitative data and did not state the intent and timing of integration. In that case, the strength of the study is weak. Integration only occurred at the levels of data collection. However, integration may not have occurred at the analysis, interpretation and reporting levels.

Tip 3: write conceptual contribution of each reviewed article

While reading and evaluating review papers, we have observed that many review authors only provide core results of the article included in a review and do not explain the conceptual contribution offered by the included article. We refer to conceptual contribution as a description of how the article’s key results contribute towards the development of potential codes, themes or subthemes, or emerging patterns that are reported as the review findings. For example, the authors of a review article noted that one of the research articles included in their review demonstrated the usefulness of case studies and reflective logs as strategies for fostering compassion in nursing students. The conceptual contribution of this research article could be that experiential learning is one way to teach compassion to nursing students, as supported by case studies and reflective logs. This conceptual contribution of the article should be mentioned in the literature summary table. Delineating each reviewed article’s conceptual contribution is particularly beneficial in qualitative reviews, mixed-methods reviews, and critical reviews that often focus on developing models and describing or explaining various phenomena. Figure 2 offers an example of a literature summary table. 4

Tabular literature summaries from a critical review. Source: Younas and Maddigan. 4

Tip 4: compose potential themes from each article during summary writing

While developing literature summary tables, many authors use themes or subthemes reported in the given articles as the key results of their own review. Such an approach prevents the review authors from understanding the article’s conceptual contribution, developing rigorous synthesis and drawing reasonable interpretations of results from an individual article. Ultimately, it affects the generation of novel review findings. For example, one of the articles about women’s healthcare-seeking behaviours in developing countries reported a theme ‘social-cultural determinants of health as precursors of delays’. Instead of using this theme as one of the review findings, the reviewers should read and interpret beyond the given description in an article, compare and contrast themes, findings from one article with findings and themes from another article to find similarities and differences and to understand and explain bigger picture for their readers. Therefore, while developing literature summary tables, think twice before using the predeveloped themes. Including your themes in the summary tables (see figure 1 ) demonstrates to the readers that a robust method of data extraction and synthesis has been followed.

Tip 5: create your personalised template for literature summaries

Often templates are available for data extraction and development of literature summary tables. The available templates may be in the form of a table, chart or a structured framework that extracts some essential information about every article. The commonly used information may include authors, purpose, methods, key results and quality scores. While extracting all relevant information is important, such templates should be tailored to meet the needs of the individuals’ review. For example, for a review about the effectiveness of healthcare interventions, a literature summary table must include information about the intervention, its type, content timing, duration, setting, effectiveness, negative consequences, and receivers and implementers’ experiences of its usage. Similarly, literature summary tables for articles included in a meta-synthesis must include information about the participants’ characteristics, research context and conceptual contribution of each reviewed article so as to help the reader make an informed decision about the usefulness or lack of usefulness of the individual article in the review and the whole review.

In conclusion, narrative or systematic reviews are almost always conducted as a part of any educational project (thesis or dissertation) or academic or clinical research. Literature reviews are the foundation of research on a given topic. Robust and high-quality reviews play an instrumental role in guiding research, practice and policymaking. However, the quality of reviews is also contingent on rigorous data extraction and synthesis, which require developing literature summaries. We have outlined five tips that could enhance the quality of the data extraction and synthesis process by developing useful literature summaries.

• Aromataris E ,
• Rasheed SP ,

Twitter @Ahtisham04, @parveenazamali

Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

Competing interests None declared.

Patient consent for publication Not required.

Provenance and peer review Not commissioned; externally peer reviewed.

Read the full text or download the PDF: